2024-09-21T03:43:09.840 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T03:43:09.889 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T03:43:09.986 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913487 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/blogbench}} email: pdonnell@redhat.com first_in_suite: false job_id: '7913487' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mgr/snap_schedule/allow_m_granularity: true mgr/snap_schedule/dump_on_update: true mgr/snap_schedule/log_level: debug mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mds: mds_export_ephemeral_random_max: 0.1 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request mgr-modules: - snap_schedule sha1: 1843221895adccfb3b692060251dd6a0183bc410 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi064.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPrmEEz5oZmSaxxWHXg0m7kPmbbUZwKYt8coKlhsqkawRjeAFEYagktCbR1Dt0aceFc8rSaRPZH9WGQavDRFcdg= smithi119.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI6F8dPVBg6+UOlxuNcq+XT2u0T/5uX7GyKzeo+T2ytWvct54cF33FafYy861nJinbdpqx5v3TWSGzROZNCQSD4= smithi131.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGdNec9bQoefCWWz7NVPLR8wb/Dv+Xx8DT2/VSokykw2E6WIy5BFu8qNU2I3mjC5hMRNKOOvcn7iXGiJVbEZJ3A= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T03:43:09.986 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T03:43:09.987 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T03:43:09.987 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T03:43:09.989 INFO:teuthology.task.internal:Checking packages... 2024-09-21T03:43:10.009 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T03:43:10.009 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T03:43:10.009 INFO:teuthology.packaging:ref: None 2024-09-21T03:43:10.009 INFO:teuthology.packaging:tag: None 2024-09-21T03:43:10.009 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T03:43:10.009 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:43:10.009 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T03:43:10.161 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120.g18432218 2024-09-21T03:43:10.163 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T03:43:10.170 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T03:43:10.170 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T03:43:10.176 INFO:teuthology.task.internal:Saving configuration 2024-09-21T03:43:10.197 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T03:43:10.204 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T03:43:10.230 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913487', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 03:38:10.050719', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPrmEEz5oZmSaxxWHXg0m7kPmbbUZwKYt8coKlhsqkawRjeAFEYagktCbR1Dt0aceFc8rSaRPZH9WGQavDRFcdg='} 2024-09-21T03:43:10.250 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913487', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 03:38:10.048944', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI6F8dPVBg6+UOlxuNcq+XT2u0T/5uX7GyKzeo+T2ytWvct54cF33FafYy861nJinbdpqx5v3TWSGzROZNCQSD4='} 2024-09-21T03:43:10.272 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi131.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913487', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 03:38:10.052181', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGdNec9bQoefCWWz7NVPLR8wb/Dv+Xx8DT2/VSokykw2E6WIy5BFu8qNU2I3mjC5hMRNKOOvcn7iXGiJVbEZJ3A='} 2024-09-21T03:43:10.272 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T03:43:10.278 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-21T03:43:10.278 INFO:teuthology.task.internal:roles: ubuntu@smithi119.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-21T03:43:10.278 INFO:teuthology.task.internal:roles: ubuntu@smithi131.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-21T03:43:10.278 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T03:43:10.358 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7c47ab4160>, signals=[15]) 2024-09-21T03:43:10.359 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T03:43:10.365 INFO:teuthology.task.internal:Opening connections... 2024-09-21T03:43:10.365 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi064.front.sepia.ceph.com 2024-09-21T03:43:10.366 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:43:10.439 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi119.front.sepia.ceph.com 2024-09-21T03:43:10.440 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:43:10.513 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi131.front.sepia.ceph.com 2024-09-21T03:43:10.514 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:43:10.588 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T03:43:10.600 DEBUG:teuthology.orchestra.run.smithi064:> uname -m 2024-09-21T03:43:10.618 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2024-09-21T03:43:10.618 DEBUG:teuthology.orchestra.run.smithi064:> cat /etc/os-release 2024-09-21T03:43:10.676 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="CentOS Stream" 2024-09-21T03:43:10.676 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="9" 2024-09-21T03:43:10.677 INFO:teuthology.orchestra.run.smithi064.stdout:ID="centos" 2024-09-21T03:43:10.677 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="rhel fedora" 2024-09-21T03:43:10.677 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="9" 2024-09-21T03:43:10.677 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el9" 2024-09-21T03:43:10.677 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T03:43:10.677 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31" 2024-09-21T03:43:10.677 INFO:teuthology.orchestra.run.smithi064.stdout:LOGO="fedora-logo-icon" 2024-09-21T03:43:10.677 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T03:43:10.677 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://centos.org/" 2024-09-21T03:43:10.677 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T03:43:10.677 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T03:43:10.677 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T03:43:10.678 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2024-09-21T03:43:10.702 DEBUG:teuthology.orchestra.run.smithi119:> uname -m 2024-09-21T03:43:10.720 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64 2024-09-21T03:43:10.720 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2024-09-21T03:43:10.778 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="CentOS Stream" 2024-09-21T03:43:10.779 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="9" 2024-09-21T03:43:10.779 INFO:teuthology.orchestra.run.smithi119.stdout:ID="centos" 2024-09-21T03:43:10.779 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE="rhel fedora" 2024-09-21T03:43:10.779 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="9" 2024-09-21T03:43:10.779 INFO:teuthology.orchestra.run.smithi119.stdout:PLATFORM_ID="platform:el9" 2024-09-21T03:43:10.779 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T03:43:10.779 INFO:teuthology.orchestra.run.smithi119.stdout:ANSI_COLOR="0;31" 2024-09-21T03:43:10.779 INFO:teuthology.orchestra.run.smithi119.stdout:LOGO="fedora-logo-icon" 2024-09-21T03:43:10.779 INFO:teuthology.orchestra.run.smithi119.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T03:43:10.779 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://centos.org/" 2024-09-21T03:43:10.779 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T03:43:10.779 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T03:43:10.779 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T03:43:10.780 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2024-09-21T03:43:10.802 DEBUG:teuthology.orchestra.run.smithi131:> uname -m 2024-09-21T03:43:10.820 INFO:teuthology.orchestra.run.smithi131.stdout:x86_64 2024-09-21T03:43:10.820 DEBUG:teuthology.orchestra.run.smithi131:> cat /etc/os-release 2024-09-21T03:43:10.878 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="CentOS Stream" 2024-09-21T03:43:10.879 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="9" 2024-09-21T03:43:10.879 INFO:teuthology.orchestra.run.smithi131.stdout:ID="centos" 2024-09-21T03:43:10.879 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="rhel fedora" 2024-09-21T03:43:10.879 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="9" 2024-09-21T03:43:10.879 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el9" 2024-09-21T03:43:10.879 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T03:43:10.879 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31" 2024-09-21T03:43:10.879 INFO:teuthology.orchestra.run.smithi131.stdout:LOGO="fedora-logo-icon" 2024-09-21T03:43:10.879 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T03:43:10.879 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://centos.org/" 2024-09-21T03:43:10.879 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T03:43:10.879 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T03:43:10.879 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T03:43:10.880 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2024-09-21T03:43:10.902 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T03:43:10.912 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T03:43:10.918 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T03:43:10.918 DEBUG:teuthology.orchestra.run.smithi064:> test '!' -e /home/ubuntu/cephtest 2024-09-21T03:43:10.920 DEBUG:teuthology.orchestra.run.smithi119:> test '!' -e /home/ubuntu/cephtest 2024-09-21T03:43:10.922 DEBUG:teuthology.orchestra.run.smithi131:> test '!' -e /home/ubuntu/cephtest 2024-09-21T03:43:10.939 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T03:43:10.945 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T03:43:10.945 DEBUG:teuthology.orchestra.run.smithi064:> test -z $(ls -A /var/lib/ceph) 2024-09-21T03:43:10.978 DEBUG:teuthology.orchestra.run.smithi119:> test -z $(ls -A /var/lib/ceph) 2024-09-21T03:43:10.980 DEBUG:teuthology.orchestra.run.smithi131:> test -z $(ls -A /var/lib/ceph) 2024-09-21T03:43:11.087 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T03:43:11.153 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T03:43:11.167 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-21T03:43:11.167 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-21T03:43:11.167 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-21T03:43:11.167 DEBUG:teuthology.orchestra.run.smithi064:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T03:43:11.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:43:11.185 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2024-09-21T03:43:11.242 INFO:teuthology.orchestra.run.smithi064.stdout:5.14.0-503.el9.x86_64 2024-09-21T03:43:11.242 INFO:teuthology.task.kernel:Running kernel on smithi064: 5.14.0-503.el9.x86_64 2024-09-21T03:43:11.242 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum install -y kernel 2024-09-21T03:43:14.706 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.3 MB 00:01 2024-09-21T03:43:16.456 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - AppStream 33 MB/s | 20 MB 00:00 2024-09-21T03:43:21.515 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - CRB 5.8 MB/s | 6.5 MB 00:01 2024-09-21T03:43:22.958 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - Extras packages 112 kB/s | 19 kB 00:00 2024-09-21T03:43:24.228 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-21T03:43:29.826 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-21T03:43:30.006 INFO:teuthology.orchestra.run.smithi064.stdout:Last metadata expiration check: 0:00:01 ago on Sat 21 Sep 2024 03:43:29 AM UTC. 2024-09-21T03:43:31.799 INFO:teuthology.orchestra.run.smithi064.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T03:43:31.799 INFO:teuthology.orchestra.run.smithi064.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T03:43:31.894 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-21T03:43:31.903 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-21T03:43:31.903 INFO:teuthology.orchestra.run.smithi064.stdout: Package Architecture Version Repository Size 2024-09-21T03:43:31.903 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-21T03:43:31.903 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-09-21T03:43:31.903 INFO:teuthology.orchestra.run.smithi064.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T03:43:31.903 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-09-21T03:43:31.903 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T03:43:31.903 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T03:43:31.903 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T03:43:31.903 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:43:31.903 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-09-21T03:43:31.904 INFO:teuthology.orchestra.run.smithi064.stdout:================================================================================ 2024-09-21T03:43:31.904 INFO:teuthology.orchestra.run.smithi064.stdout:Install 4 Packages 2024-09-21T03:43:31.904 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:43:31.904 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 79 M 2024-09-21T03:43:31.904 INFO:teuthology.orchestra.run.smithi064.stdout:Installed size: 126 M 2024-09-21T03:43:31.904 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-09-21T03:43:32.111 INFO:teuthology.orchestra.run.smithi064.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 445 kB/s | 36 kB 00:00 2024-09-21T03:43:32.662 INFO:teuthology.orchestra.run.smithi064.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-09-21T03:43:33.286 INFO:teuthology.orchestra.run.smithi064.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 27 MB/s | 35 MB 00:01 2024-09-21T03:43:33.578 INFO:teuthology.orchestra.run.smithi064.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 19 MB/s | 29 MB 00:01 2024-09-21T03:43:33.578 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:43:33.578 INFO:teuthology.orchestra.run.smithi064.stdout:Total 47 MB/s | 79 MB 00:01 2024-09-21T03:43:34.286 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-09-21T03:43:34.661 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-09-21T03:43:34.661 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-09-21T03:43:35.664 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-09-21T03:43:35.665 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-09-21T03:43:38.221 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-09-21T03:43:38.881 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T03:43:38.961 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:43:41.706 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:43:41.837 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:43:46.965 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:43:47.021 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:43:52.157 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:44:31.352 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:44:31.394 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:44:31.797 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:44:31.797 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T03:44:31.797 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T03:44:31.797 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T03:44:32.635 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T03:44:32.636 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:44:32.636 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-09-21T03:44:32.636 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T03:44:32.636 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T03:44:32.636 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T03:44:32.636 INFO:teuthology.orchestra.run.smithi064.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T03:44:32.636 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:44:32.636 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-21T03:44:32.985 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T03:44:33.117 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T03:44:33.118 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T03:44:33.118 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T03:44:33.118 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T03:44:33.118 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T03:44:33.118 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-21T03:44:33.118 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi064.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T03:44:33.118 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T03:44:33.177 INFO:teuthology.orchestra.run.smithi064.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T03:44:33.177 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T03:44:33.177 DEBUG:teuthology.orchestra.run.smithi064:> sudo rpm -qi grub2-tools 2024-09-21T03:44:33.208 INFO:teuthology.orchestra.run.smithi064.stdout:Name : grub2-tools 2024-09-21T03:44:33.208 INFO:teuthology.orchestra.run.smithi064.stdout:Epoch : 1 2024-09-21T03:44:33.208 INFO:teuthology.orchestra.run.smithi064.stdout:Version : 2.06 2024-09-21T03:44:33.209 INFO:teuthology.orchestra.run.smithi064.stdout:Release : 82.el9 2024-09-21T03:44:33.209 INFO:teuthology.orchestra.run.smithi064.stdout:Architecture: x86_64 2024-09-21T03:44:33.209 INFO:teuthology.orchestra.run.smithi064.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T03:44:33.209 INFO:teuthology.orchestra.run.smithi064.stdout:Group : Unspecified 2024-09-21T03:44:33.209 INFO:teuthology.orchestra.run.smithi064.stdout:Size : 8274670 2024-09-21T03:44:33.209 INFO:teuthology.orchestra.run.smithi064.stdout:License : GPLv3+ 2024-09-21T03:44:33.209 INFO:teuthology.orchestra.run.smithi064.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T03:44:33.209 INFO:teuthology.orchestra.run.smithi064.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T03:44:33.209 INFO:teuthology.orchestra.run.smithi064.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T03:44:33.209 INFO:teuthology.orchestra.run.smithi064.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T03:44:33.210 INFO:teuthology.orchestra.run.smithi064.stdout:Packager : builder@centos.org 2024-09-21T03:44:33.210 INFO:teuthology.orchestra.run.smithi064.stdout:Vendor : CentOS 2024-09-21T03:44:33.210 INFO:teuthology.orchestra.run.smithi064.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T03:44:33.210 INFO:teuthology.orchestra.run.smithi064.stdout:Summary : Support tools for GRUB. 2024-09-21T03:44:33.210 INFO:teuthology.orchestra.run.smithi064.stdout:Description : 2024-09-21T03:44:33.210 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:44:33.210 INFO:teuthology.orchestra.run.smithi064.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T03:44:33.210 INFO:teuthology.orchestra.run.smithi064.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T03:44:33.210 INFO:teuthology.orchestra.run.smithi064.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T03:44:33.210 INFO:teuthology.orchestra.run.smithi064.stdout:hardware devices. 2024-09-21T03:44:33.210 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:44:33.210 INFO:teuthology.orchestra.run.smithi064.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T03:44:33.211 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T03:44:33.211 INFO:teuthology.task.kernel:Updating grub on smithi064 to boot 5.14.0-508.el9.x86_64 2024-09-21T03:44:33.212 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T03:44:34.000 INFO:teuthology.orchestra.run.smithi064.stderr:Generating grub configuration file ... 2024-09-21T03:44:35.283 INFO:teuthology.orchestra.run.smithi064.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T03:44:35.322 INFO:teuthology.orchestra.run.smithi064.stderr:done 2024-09-21T03:44:35.329 DEBUG:teuthology.orchestra.run.smithi064:> mktemp 2024-09-21T03:44:35.344 INFO:teuthology.orchestra.run.smithi064.stdout:/tmp/tmp.S4TxqWSecX 2024-09-21T03:44:35.344 DEBUG:teuthology.orchestra.run.smithi064:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.S4TxqWSecX 2024-09-21T03:44:35.411 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0666 /tmp/tmp.S4TxqWSecX 2024-09-21T03:44:35.553 DEBUG:teuthology.orchestra.remote:smithi064:/tmp/tmp.S4TxqWSecX is 6KB 2024-09-21T03:44:35.601 DEBUG:teuthology.orchestra.run.smithi064:> rm -fr /tmp/tmp.S4TxqWSecX 2024-09-21T03:44:35.615 DEBUG:teuthology.orchestra.run.smithi064:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T03:44:35.679 INFO:teuthology.orchestra.run.smithi064.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T03:44:35.680 INFO:teuthology.orchestra.run.smithi064.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T03:44:35.680 INFO:teuthology.orchestra.run.smithi064.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T03:44:35.680 INFO:teuthology.orchestra.run.smithi064.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T03:44:35.681 DEBUG:teuthology.orchestra.run.smithi064:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T03:44:35.839 DEBUG:teuthology.orchestra.run.smithi064:> sudo shutdown -r now 2024-09-21T03:45:05.842 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-21T03:45:05.843 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-09-21T03:45:05.843 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:45:40.335 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2024-09-21T03:45:49.336 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-09-21T03:45:49.336 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:46:07.731 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.64 2024-09-21T03:46:19.731 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-09-21T03:46:19.732 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:46:20.030 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-09-21T03:46:20.405 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-09-21T03:46:20.405 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-21T03:46:20.405 DEBUG:teuthology.orchestra.run.smithi064:> uname -r 2024-09-21T03:46:20.420 INFO:teuthology.orchestra.run.smithi064.stdout:5.14.0-508.el9.x86_64 2024-09-21T03:46:20.420 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T03:46:20.420 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T03:46:20.420 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T03:46:21.421 DEBUG:teuthology.parallel:result is None 2024-09-21T03:46:21.421 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T03:46:21.433 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T03:46:21.434 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T03:46:21.436 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T03:46:21.438 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T03:46:21.456 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T03:46:21.463 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T03:46:21.478 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T03:46:21.478 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T03:46:21.491 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T03:46:21.497 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T03:46:21.602 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T03:46:21.608 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T03:46:21.608 DEBUG:teuthology.orchestra.run.smithi119:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T03:46:21.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:46:21.625 DEBUG:teuthology.orchestra.run.smithi131:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T03:46:21.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:46:21.643 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T03:46:21.645 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T03:46:21.669 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T03:46:21.702 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:46:21.719 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:46:21.744 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:46:21.745 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:46:21.762 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:46:22.106 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T03:46:22.108 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T03:46:22.115 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T03:46:22.115 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T03:46:22.117 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T03:46:22.119 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T03:46:22.148 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T03:46:22.157 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T03:46:22.157 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T03:46:22.184 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T03:46:22.190 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T03:46:22.207 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:46:22.280 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:46:22.327 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:46:22.411 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:46:22.450 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-21T03:46:22.451 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T03:46:22.516 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:46:22.559 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:46:22.619 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:46:22.702 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:46:22.746 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-21T03:46:22.746 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T03:46:22.813 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:46:22.857 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T03:46:22.921 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:46:22.966 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T03:46:23.008 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-21T03:46:23.008 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T03:46:23.075 DEBUG:teuthology.orchestra.run.smithi064:> sudo service rsyslog restart 2024-09-21T03:46:23.078 DEBUG:teuthology.orchestra.run.smithi119:> sudo service rsyslog restart 2024-09-21T03:46:23.080 DEBUG:teuthology.orchestra.run.smithi131:> sudo service rsyslog restart 2024-09-21T03:46:23.122 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:46:23.140 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:46:23.151 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:46:23.544 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T03:46:23.551 INFO:teuthology.task.internal:Starting timer... 2024-09-21T03:46:23.552 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T03:46:23.582 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T03:46:23.598 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-21T03:46:23.599 DEBUG:teuthology.orchestra.run.smithi064:> sudo service auditd rotate 2024-09-21T03:46:23.677 INFO:teuthology.orchestra.run.smithi064.stdout:Rotating logs: 2024-09-21T03:46:23.678 DEBUG:teuthology.orchestra.run.smithi119:> sudo service auditd rotate 2024-09-21T03:46:23.725 INFO:teuthology.orchestra.run.smithi119.stdout:Rotating logs: 2024-09-21T03:46:23.727 DEBUG:teuthology.orchestra.run.smithi131:> sudo service auditd rotate 2024-09-21T03:46:23.780 INFO:teuthology.orchestra.run.smithi131.stdout:Rotating logs: 2024-09-21T03:46:23.782 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T03:46:23.782 DEBUG:teuthology.orchestra.run.smithi064:> /usr/sbin/getenforce 2024-09-21T03:46:23.843 INFO:teuthology.orchestra.run.smithi064.stdout:Permissive 2024-09-21T03:46:23.843 DEBUG:teuthology.orchestra.run.smithi119:> /usr/sbin/getenforce 2024-09-21T03:46:23.867 INFO:teuthology.orchestra.run.smithi119.stdout:Permissive 2024-09-21T03:46:23.867 DEBUG:teuthology.orchestra.run.smithi131:> /usr/sbin/getenforce 2024-09-21T03:46:23.902 INFO:teuthology.orchestra.run.smithi131.stdout:Permissive 2024-09-21T03:46:23.902 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi064.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi119.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi131.front.sepia.ceph.com': 'permissive'} 2024-09-21T03:46:23.902 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:46:23.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:46:23.929 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:46:23.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:46:23.958 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:46:23.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:46:23.984 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T03:46:23.984 DEBUG:teuthology.orchestra.run.smithi064:> sudo /usr/sbin/setenforce permissive 2024-09-21T03:46:24.050 DEBUG:teuthology.orchestra.run.smithi119:> sudo /usr/sbin/setenforce permissive 2024-09-21T03:46:24.089 DEBUG:teuthology.orchestra.run.smithi131:> sudo /usr/sbin/setenforce permissive 2024-09-21T03:46:24.121 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T03:46:24.130 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-21T03:46:24.132 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-21T03:46:24.267 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T03:46:24.277 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T03:46:24.277 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi064.front.sepia.ceph.com,smithi119.front.sepia.ceph.com,smithi131.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T03:50:30.070 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi064.front.sepia.ceph.com'), Remote(name='ubuntu@smithi119.front.sepia.ceph.com'), Remote(name='ubuntu@smithi131.front.sepia.ceph.com')] 2024-09-21T03:50:30.071 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-09-21T03:50:30.072 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:50:30.143 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-09-21T03:50:30.225 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-09-21T03:50:30.225 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-09-21T03:50:30.226 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:50:30.298 DEBUG:teuthology.orchestra.run.smithi119:> true 2024-09-21T03:50:30.387 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-09-21T03:50:30.387 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-09-21T03:50:30.387 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T03:50:30.462 DEBUG:teuthology.orchestra.run.smithi131:> true 2024-09-21T03:50:30.543 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-09-21T03:50:30.543 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T03:50:30.553 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T03:50:30.553 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T03:50:30.553 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:50:30.556 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T03:50:30.556 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:50:30.558 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T03:50:30.558 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:50:30.584 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T03:50:30.586 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T03:50:30.600 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T03:50:30.601 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T03:50:30.614 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T03:50:30.626 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: ntpd: command not found 2024-09-21T03:50:30.627 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: ntpd: command not found 2024-09-21T03:50:30.629 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T03:50:30.638 INFO:teuthology.orchestra.run.smithi119.stdout:506 Cannot talk to daemon 2024-09-21T03:50:30.638 INFO:teuthology.orchestra.run.smithi064.stdout:506 Cannot talk to daemon 2024-09-21T03:50:30.652 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T03:50:30.653 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T03:50:30.655 INFO:teuthology.orchestra.run.smithi131.stderr:sudo: ntpd: command not found 2024-09-21T03:50:30.666 INFO:teuthology.orchestra.run.smithi131.stdout:506 Cannot talk to daemon 2024-09-21T03:50:30.666 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T03:50:30.668 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T03:50:30.680 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T03:50:30.695 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T03:50:30.725 INFO:teuthology.orchestra.run.smithi119.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:50:30.728 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:50:30.728 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2024-09-21T03:50:30.728 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:50:30.728 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:50:30.728 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:50:30.728 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:50:30.729 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:50:30.732 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:50:30.732 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-09-21T03:50:30.732 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:50:30.732 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:50:30.732 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:50:30.733 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:50:30.753 INFO:teuthology.orchestra.run.smithi131.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:50:30.756 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:50:30.756 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2024-09-21T03:50:30.756 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:50:30.756 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:50:30.757 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:50:30.757 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:50:30.757 INFO:teuthology.run_tasks:Running task install... 2024-09-21T03:50:30.765 DEBUG:teuthology.task.install:project ceph 2024-09-21T03:50:30.765 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T03:50:30.765 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T03:50:30.765 INFO:teuthology.task.install:Using flavor: default 2024-09-21T03:50:30.771 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T03:50:30.771 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T03:50:30.771 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T03:50:30.771 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:50:30.773 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T03:50:30.773 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:50:30.774 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T03:50:30.774 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:50:30.913 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T03:50:30.913 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T03:50:30.936 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T03:50:30.936 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T03:50:30.946 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T03:50:30.946 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T03:50:31.024 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T03:50:31.024 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-21T03:50:31.024 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T03:50:31.041 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T03:50:31.041 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-21T03:50:31.041 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T03:50:31.051 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T03:50:31.052 DEBUG:teuthology.orchestra.run.smithi064:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T03:50:31.053 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T03:50:31.054 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-21T03:50:31.054 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T03:50:31.067 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T03:50:31.067 DEBUG:teuthology.orchestra.run.smithi131:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T03:50:31.078 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T03:50:31.079 DEBUG:teuthology.orchestra.run.smithi119:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T03:50:31.118 DEBUG:teuthology.orchestra.run.smithi064:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T03:50:31.133 DEBUG:teuthology.orchestra.run.smithi131:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T03:50:31.144 DEBUG:teuthology.orchestra.run.smithi119:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T03:50:31.195 DEBUG:teuthology.orchestra.run.smithi064:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T03:50:31.209 DEBUG:teuthology.orchestra.run.smithi131:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T03:50:31.218 DEBUG:teuthology.orchestra.run.smithi119:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T03:50:31.261 INFO:teuthology.orchestra.run.smithi064.stdout:check_obsoletes = 1 2024-09-21T03:50:31.263 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum clean all 2024-09-21T03:50:31.272 INFO:teuthology.orchestra.run.smithi131.stdout:check_obsoletes = 1 2024-09-21T03:50:31.274 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum clean all 2024-09-21T03:50:31.283 INFO:teuthology.orchestra.run.smithi119.stdout:check_obsoletes = 1 2024-09-21T03:50:31.284 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum clean all 2024-09-21T03:50:31.498 INFO:teuthology.orchestra.run.smithi131.stdout:45 files removed 2024-09-21T03:50:31.506 INFO:teuthology.orchestra.run.smithi064.stdout:45 files removed 2024-09-21T03:50:31.510 INFO:teuthology.orchestra.run.smithi119.stdout:45 files removed 2024-09-21T03:50:31.525 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T03:50:31.533 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T03:50:31.536 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T03:50:32.062 INFO:teuthology.orchestra.run.smithi131.stdout:ceph packages for x86_64 325 kB/s | 84 kB 00:00 2024-09-21T03:50:32.083 INFO:teuthology.orchestra.run.smithi119.stdout:ceph packages for x86_64 308 kB/s | 84 kB 00:00 2024-09-21T03:50:32.114 INFO:teuthology.orchestra.run.smithi064.stdout:ceph packages for x86_64 300 kB/s | 84 kB 00:00 2024-09-21T03:50:32.362 INFO:teuthology.orchestra.run.smithi131.stdout:ceph noarch packages 94 kB/s | 24 kB 00:00 2024-09-21T03:50:32.375 INFO:teuthology.orchestra.run.smithi119.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-21T03:50:32.406 INFO:teuthology.orchestra.run.smithi064.stdout:ceph noarch packages 98 kB/s | 24 kB 00:00 2024-09-21T03:50:32.638 INFO:teuthology.orchestra.run.smithi131.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-21T03:50:32.651 INFO:teuthology.orchestra.run.smithi119.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-21T03:50:32.690 INFO:teuthology.orchestra.run.smithi064.stdout:ceph source packages 7.8 kB/s | 1.9 kB 00:00 2024-09-21T03:50:33.388 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.3 MB 00:00 2024-09-21T03:50:34.334 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.3 MB 00:01 2024-09-21T03:50:34.382 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.3 MB 00:01 2024-09-21T03:50:35.298 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-21T03:50:36.086 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-09-21T03:50:36.175 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-21T03:50:40.965 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.5 MB 00:01 2024-09-21T03:50:41.411 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.5 MB 00:01 2024-09-21T03:50:41.667 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.5 MB 00:01 2024-09-21T03:50:42.633 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-21T03:50:43.153 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 19 kB 00:00 2024-09-21T03:50:43.418 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 19 kB 00:00 2024-09-21T03:50:43.512 INFO:teuthology.orchestra.run.smithi131.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-21T03:50:43.907 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-21T03:50:44.505 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-21T03:50:48.893 INFO:teuthology.orchestra.run.smithi131.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-21T03:50:49.347 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-21T03:50:50.161 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-21T03:50:51.012 INFO:teuthology.orchestra.run.smithi131.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:50:51.013 INFO:teuthology.orchestra.run.smithi131.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:50:51.025 INFO:teuthology.orchestra.run.smithi131.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T03:50:51.026 INFO:teuthology.orchestra.run.smithi131.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T03:50:51.027 INFO:teuthology.orchestra.run.smithi131.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T03:50:51.027 INFO:teuthology.orchestra.run.smithi131.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T03:50:51.029 INFO:teuthology.orchestra.run.smithi131.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T03:50:51.031 INFO:teuthology.orchestra.run.smithi131.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T03:50:51.032 INFO:teuthology.orchestra.run.smithi131.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T03:50:51.096 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-21T03:50:51.121 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================= 2024-09-21T03:50:51.121 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-21T03:50:51.121 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================= 2024-09-21T03:50:51.121 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-21T03:50:51.121 INFO:teuthology.orchestra.run.smithi131.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T03:50:51.122 INFO:teuthology.orchestra.run.smithi131.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T03:50:51.122 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T03:50:51.122 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T03:50:51.122 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:50:51.122 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T03:50:51.122 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T03:50:51.122 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T03:50:51.122 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T03:50:51.122 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T03:50:51.122 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T03:50:51.122 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T03:50:51.122 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T03:50:51.123 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T03:50:51.123 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T03:50:51.123 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T03:50:51.123 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T03:50:51.123 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T03:50:51.123 INFO:teuthology.orchestra.run.smithi131.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T03:50:51.123 INFO:teuthology.orchestra.run.smithi131.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T03:50:51.123 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T03:50:51.123 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T03:50:51.123 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T03:50:51.123 INFO:teuthology.orchestra.run.smithi131.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T03:50:51.123 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T03:50:51.123 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T03:50:51.124 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T03:50:51.124 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T03:50:51.124 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T03:50:51.124 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T03:50:51.124 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T03:50:51.124 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T03:50:51.124 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T03:50:51.124 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T03:50:51.124 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T03:50:51.124 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T03:50:51.124 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T03:50:51.124 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrading: 2024-09-21T03:50:51.124 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T03:50:51.125 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T03:50:51.125 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T03:50:51.125 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T03:50:51.125 INFO:teuthology.orchestra.run.smithi131.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T03:50:51.125 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T03:50:51.125 INFO:teuthology.orchestra.run.smithi131.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T03:50:51.125 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T03:50:51.125 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-21T03:50:51.125 INFO:teuthology.orchestra.run.smithi131.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T03:50:51.125 INFO:teuthology.orchestra.run.smithi131.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T03:50:51.125 INFO:teuthology.orchestra.run.smithi131.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T03:50:51.125 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T03:50:51.125 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T03:50:51.125 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T03:50:51.125 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T03:50:51.126 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T03:50:51.126 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T03:50:51.126 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T03:50:51.126 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T03:50:51.126 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T03:50:51.126 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T03:50:51.126 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T03:50:51.126 INFO:teuthology.orchestra.run.smithi131.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T03:50:51.126 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T03:50:51.126 INFO:teuthology.orchestra.run.smithi131.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T03:50:51.126 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T03:50:51.126 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T03:50:51.126 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T03:50:51.127 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T03:50:51.127 INFO:teuthology.orchestra.run.smithi131.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T03:50:51.127 INFO:teuthology.orchestra.run.smithi131.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T03:50:51.127 INFO:teuthology.orchestra.run.smithi131.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T03:50:51.127 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T03:50:51.127 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T03:50:51.127 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T03:50:51.127 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T03:50:51.127 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T03:50:51.127 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T03:50:51.127 INFO:teuthology.orchestra.run.smithi131.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T03:50:51.127 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T03:50:51.127 INFO:teuthology.orchestra.run.smithi131.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T03:50:51.127 INFO:teuthology.orchestra.run.smithi131.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T03:50:51.128 INFO:teuthology.orchestra.run.smithi131.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T03:50:51.128 INFO:teuthology.orchestra.run.smithi131.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T03:50:51.128 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T03:50:51.128 INFO:teuthology.orchestra.run.smithi131.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T03:50:51.128 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T03:50:51.128 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T03:50:51.128 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T03:50:51.128 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T03:50:51.128 INFO:teuthology.orchestra.run.smithi131.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T03:50:51.128 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T03:50:51.128 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T03:50:51.129 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T03:50:51.129 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T03:50:51.129 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:50:51.129 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T03:50:51.129 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T03:50:51.129 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T03:50:51.129 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T03:50:51.129 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T03:50:51.129 INFO:teuthology.orchestra.run.smithi131.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T03:50:51.129 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T03:50:51.129 INFO:teuthology.orchestra.run.smithi131.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T03:50:51.129 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T03:50:51.129 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T03:50:51.130 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T03:50:51.130 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T03:50:51.130 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T03:50:51.130 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T03:50:51.130 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T03:50:51.130 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T03:50:51.130 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T03:50:51.130 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T03:50:51.130 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T03:50:51.130 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T03:50:51.130 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T03:50:51.130 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T03:50:51.130 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T03:50:51.131 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T03:50:51.131 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T03:50:51.131 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T03:50:51.131 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T03:50:51.131 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T03:50:51.131 INFO:teuthology.orchestra.run.smithi131.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T03:50:51.131 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T03:50:51.131 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T03:50:51.131 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T03:50:51.131 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T03:50:51.131 INFO:teuthology.orchestra.run.smithi131.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T03:50:51.131 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T03:50:51.131 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T03:50:51.131 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T03:50:51.132 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T03:50:51.132 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T03:50:51.132 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T03:50:51.132 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T03:50:51.132 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T03:50:51.132 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T03:50:51.132 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T03:50:51.132 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T03:50:51.132 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T03:50:51.132 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T03:50:51.132 INFO:teuthology.orchestra.run.smithi131.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T03:50:51.132 INFO:teuthology.orchestra.run.smithi131.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T03:50:51.132 INFO:teuthology.orchestra.run.smithi131.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T03:50:51.132 INFO:teuthology.orchestra.run.smithi131.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T03:50:51.132 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T03:50:51.133 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T03:50:51.133 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T03:50:51.133 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T03:50:51.133 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T03:50:51.133 INFO:teuthology.orchestra.run.smithi131.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T03:50:51.133 INFO:teuthology.orchestra.run.smithi131.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T03:50:51.133 INFO:teuthology.orchestra.run.smithi131.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T03:50:51.133 INFO:teuthology.orchestra.run.smithi131.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T03:50:51.133 INFO:teuthology.orchestra.run.smithi131.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T03:50:51.133 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T03:50:51.133 INFO:teuthology.orchestra.run.smithi131.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T03:50:51.133 INFO:teuthology.orchestra.run.smithi131.stdout:Installing weak dependencies: 2024-09-21T03:50:51.133 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T03:50:51.134 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-21T03:50:51.134 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-21T03:50:51.134 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================= 2024-09-21T03:50:51.134 INFO:teuthology.orchestra.run.smithi131.stdout:Install 149 Packages 2024-09-21T03:50:51.134 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrade 8 Packages 2024-09-21T03:50:51.134 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-21T03:50:51.134 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 228 M 2024-09-21T03:50:51.134 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-21T03:50:51.492 INFO:teuthology.orchestra.run.smithi119.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:50:51.493 INFO:teuthology.orchestra.run.smithi119.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:50:51.506 INFO:teuthology.orchestra.run.smithi119.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T03:50:51.507 INFO:teuthology.orchestra.run.smithi119.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T03:50:51.508 INFO:teuthology.orchestra.run.smithi119.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T03:50:51.509 INFO:teuthology.orchestra.run.smithi119.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T03:50:51.511 INFO:teuthology.orchestra.run.smithi119.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T03:50:51.514 INFO:teuthology.orchestra.run.smithi119.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T03:50:51.515 INFO:teuthology.orchestra.run.smithi119.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T03:50:51.577 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-09-21T03:50:51.593 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================= 2024-09-21T03:50:51.594 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-09-21T03:50:51.594 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================= 2024-09-21T03:50:51.594 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-09-21T03:50:51.594 INFO:teuthology.orchestra.run.smithi119.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T03:50:51.594 INFO:teuthology.orchestra.run.smithi119.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T03:50:51.594 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T03:50:51.594 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T03:50:51.594 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:50:51.594 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T03:50:51.594 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T03:50:51.594 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T03:50:51.595 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T03:50:51.595 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T03:50:51.595 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T03:50:51.595 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T03:50:51.595 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T03:50:51.595 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T03:50:51.595 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T03:50:51.595 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T03:50:51.595 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T03:50:51.595 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T03:50:51.595 INFO:teuthology.orchestra.run.smithi119.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T03:50:51.596 INFO:teuthology.orchestra.run.smithi119.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T03:50:51.596 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T03:50:51.596 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T03:50:51.596 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T03:50:51.596 INFO:teuthology.orchestra.run.smithi119.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T03:50:51.596 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T03:50:51.596 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T03:50:51.596 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T03:50:51.596 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T03:50:51.597 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T03:50:51.597 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T03:50:51.597 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T03:50:51.597 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T03:50:51.597 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T03:50:51.597 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T03:50:51.597 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T03:50:51.597 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T03:50:51.597 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T03:50:51.597 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrading: 2024-09-21T03:50:51.597 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T03:50:51.597 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T03:50:51.598 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T03:50:51.598 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T03:50:51.598 INFO:teuthology.orchestra.run.smithi119.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T03:50:51.598 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T03:50:51.598 INFO:teuthology.orchestra.run.smithi119.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T03:50:51.598 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T03:50:51.598 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-09-21T03:50:51.598 INFO:teuthology.orchestra.run.smithi119.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T03:50:51.598 INFO:teuthology.orchestra.run.smithi119.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T03:50:51.599 INFO:teuthology.orchestra.run.smithi119.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T03:50:51.599 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T03:50:51.599 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T03:50:51.599 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T03:50:51.599 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T03:50:51.599 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T03:50:51.599 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T03:50:51.599 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T03:50:51.600 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T03:50:51.600 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T03:50:51.600 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T03:50:51.600 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T03:50:51.600 INFO:teuthology.orchestra.run.smithi119.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T03:50:51.600 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T03:50:51.600 INFO:teuthology.orchestra.run.smithi119.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T03:50:51.600 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T03:50:51.600 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T03:50:51.600 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T03:50:51.600 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T03:50:51.600 INFO:teuthology.orchestra.run.smithi119.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T03:50:51.600 INFO:teuthology.orchestra.run.smithi119.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T03:50:51.601 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T03:50:51.601 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T03:50:51.601 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T03:50:51.601 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T03:50:51.601 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T03:50:51.601 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T03:50:51.601 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T03:50:51.601 INFO:teuthology.orchestra.run.smithi119.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T03:50:51.602 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T03:50:51.602 INFO:teuthology.orchestra.run.smithi119.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T03:50:51.602 INFO:teuthology.orchestra.run.smithi119.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T03:50:51.602 INFO:teuthology.orchestra.run.smithi119.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T03:50:51.602 INFO:teuthology.orchestra.run.smithi119.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T03:50:51.602 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T03:50:51.602 INFO:teuthology.orchestra.run.smithi119.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T03:50:51.602 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T03:50:51.602 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T03:50:51.602 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T03:50:51.603 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T03:50:51.603 INFO:teuthology.orchestra.run.smithi119.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T03:50:51.603 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T03:50:51.603 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T03:50:51.603 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T03:50:51.603 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T03:50:51.603 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:50:51.603 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T03:50:51.603 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T03:50:51.603 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T03:50:51.603 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T03:50:51.603 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T03:50:51.604 INFO:teuthology.orchestra.run.smithi119.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T03:50:51.604 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T03:50:51.604 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T03:50:51.604 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T03:50:51.604 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T03:50:51.604 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T03:50:51.604 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T03:50:51.604 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T03:50:51.604 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T03:50:51.605 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T03:50:51.605 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T03:50:51.605 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T03:50:51.605 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T03:50:51.605 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T03:50:51.605 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T03:50:51.605 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T03:50:51.605 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T03:50:51.605 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T03:50:51.605 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T03:50:51.605 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T03:50:51.606 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T03:50:51.606 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T03:50:51.606 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T03:50:51.606 INFO:teuthology.orchestra.run.smithi119.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T03:50:51.606 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T03:50:51.606 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T03:50:51.606 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T03:50:51.606 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T03:50:51.606 INFO:teuthology.orchestra.run.smithi119.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T03:50:51.606 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T03:50:51.606 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T03:50:51.607 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T03:50:51.607 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T03:50:51.607 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T03:50:51.607 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T03:50:51.607 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T03:50:51.607 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T03:50:51.607 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T03:50:51.607 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T03:50:51.608 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T03:50:51.608 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T03:50:51.608 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T03:50:51.608 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T03:50:51.608 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T03:50:51.608 INFO:teuthology.orchestra.run.smithi119.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T03:50:51.608 INFO:teuthology.orchestra.run.smithi119.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T03:50:51.608 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T03:50:51.608 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T03:50:51.608 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T03:50:51.608 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T03:50:51.609 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T03:50:51.609 INFO:teuthology.orchestra.run.smithi119.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T03:50:51.609 INFO:teuthology.orchestra.run.smithi119.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T03:50:51.609 INFO:teuthology.orchestra.run.smithi119.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T03:50:51.609 INFO:teuthology.orchestra.run.smithi119.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T03:50:51.609 INFO:teuthology.orchestra.run.smithi119.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T03:50:51.609 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T03:50:51.609 INFO:teuthology.orchestra.run.smithi119.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T03:50:51.609 INFO:teuthology.orchestra.run.smithi119.stdout:Installing weak dependencies: 2024-09-21T03:50:51.609 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T03:50:51.609 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T03:50:51.609 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-09-21T03:50:51.610 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================= 2024-09-21T03:50:51.610 INFO:teuthology.orchestra.run.smithi119.stdout:Install 149 Packages 2024-09-21T03:50:51.610 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrade 8 Packages 2024-09-21T03:50:51.610 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T03:50:51.611 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 228 M 2024-09-21T03:50:51.611 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-09-21T03:50:51.836 INFO:teuthology.orchestra.run.smithi131.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 54 kB/s | 6.6 kB 00:00 2024-09-21T03:50:52.061 INFO:teuthology.orchestra.run.smithi131.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 3.8 MB/s | 873 kB 00:00 2024-09-21T03:50:52.112 INFO:teuthology.orchestra.run.smithi131.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 2.9 MB/s | 148 kB 00:00 2024-09-21T03:50:52.404 INFO:teuthology.orchestra.run.smithi131.stdout:(4/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 7.8 MB/s | 2.3 MB 00:00 2024-09-21T03:50:52.440 INFO:teuthology.orchestra.run.smithi064.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:50:52.442 INFO:teuthology.orchestra.run.smithi064.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T03:50:52.448 INFO:teuthology.orchestra.run.smithi119.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 52 kB/s | 6.6 kB 00:00 2024-09-21T03:50:52.455 INFO:teuthology.orchestra.run.smithi064.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T03:50:52.457 INFO:teuthology.orchestra.run.smithi064.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T03:50:52.458 INFO:teuthology.orchestra.run.smithi064.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T03:50:52.459 INFO:teuthology.orchestra.run.smithi064.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T03:50:52.461 INFO:teuthology.orchestra.run.smithi064.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T03:50:52.463 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T03:50:52.464 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T03:50:52.504 INFO:teuthology.orchestra.run.smithi131.stdout:(5/157): ceph-base-19.3.0-5120.g18432218.el9.x8 7.0 MB/s | 5.5 MB 00:00 2024-09-21T03:50:52.534 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-09-21T03:50:52.554 INFO:teuthology.orchestra.run.smithi064.stdout:======================================================================================= 2024-09-21T03:50:52.554 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-09-21T03:50:52.554 INFO:teuthology.orchestra.run.smithi064.stdout:======================================================================================= 2024-09-21T03:50:52.555 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-09-21T03:50:52.555 INFO:teuthology.orchestra.run.smithi064.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T03:50:52.555 INFO:teuthology.orchestra.run.smithi064.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T03:50:52.555 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T03:50:52.555 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T03:50:52.555 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:50:52.555 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T03:50:52.555 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T03:50:52.555 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T03:50:52.555 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T03:50:52.555 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T03:50:52.555 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T03:50:52.555 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T03:50:52.556 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T03:50:52.556 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T03:50:52.556 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T03:50:52.556 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T03:50:52.556 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T03:50:52.556 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T03:50:52.556 INFO:teuthology.orchestra.run.smithi064.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T03:50:52.556 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T03:50:52.556 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T03:50:52.556 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T03:50:52.556 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T03:50:52.556 INFO:teuthology.orchestra.run.smithi064.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T03:50:52.557 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T03:50:52.557 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T03:50:52.557 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T03:50:52.557 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T03:50:52.557 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T03:50:52.557 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T03:50:52.557 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T03:50:52.557 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T03:50:52.557 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T03:50:52.557 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T03:50:52.557 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T03:50:52.557 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T03:50:52.557 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T03:50:52.557 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrading: 2024-09-21T03:50:52.558 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T03:50:52.558 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T03:50:52.558 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T03:50:52.558 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T03:50:52.558 INFO:teuthology.orchestra.run.smithi064.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T03:50:52.558 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T03:50:52.558 INFO:teuthology.orchestra.run.smithi064.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T03:50:52.558 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T03:50:52.558 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-09-21T03:50:52.558 INFO:teuthology.orchestra.run.smithi064.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T03:50:52.558 INFO:teuthology.orchestra.run.smithi064.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T03:50:52.558 INFO:teuthology.orchestra.run.smithi064.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T03:50:52.558 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T03:50:52.559 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T03:50:52.559 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T03:50:52.559 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T03:50:52.559 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T03:50:52.559 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T03:50:52.559 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T03:50:52.559 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T03:50:52.559 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T03:50:52.559 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T03:50:52.559 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T03:50:52.559 INFO:teuthology.orchestra.run.smithi064.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T03:50:52.559 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T03:50:52.560 INFO:teuthology.orchestra.run.smithi064.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T03:50:52.560 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T03:50:52.560 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T03:50:52.560 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T03:50:52.560 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T03:50:52.560 INFO:teuthology.orchestra.run.smithi064.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T03:50:52.560 INFO:teuthology.orchestra.run.smithi064.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T03:50:52.560 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T03:50:52.560 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T03:50:52.560 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T03:50:52.560 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T03:50:52.560 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T03:50:52.560 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T03:50:52.560 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T03:50:52.561 INFO:teuthology.orchestra.run.smithi064.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T03:50:52.561 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T03:50:52.561 INFO:teuthology.orchestra.run.smithi064.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T03:50:52.561 INFO:teuthology.orchestra.run.smithi064.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T03:50:52.561 INFO:teuthology.orchestra.run.smithi064.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T03:50:52.561 INFO:teuthology.orchestra.run.smithi064.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T03:50:52.561 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T03:50:52.561 INFO:teuthology.orchestra.run.smithi064.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T03:50:52.561 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T03:50:52.561 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T03:50:52.561 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T03:50:52.561 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T03:50:52.562 INFO:teuthology.orchestra.run.smithi064.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T03:50:52.562 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T03:50:52.562 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T03:50:52.562 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T03:50:52.562 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T03:50:52.562 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T03:50:52.562 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T03:50:52.562 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T03:50:52.562 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T03:50:52.562 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T03:50:52.562 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T03:50:52.562 INFO:teuthology.orchestra.run.smithi064.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T03:50:52.562 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T03:50:52.562 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T03:50:52.563 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T03:50:52.563 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T03:50:52.563 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T03:50:52.563 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T03:50:52.563 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T03:50:52.563 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T03:50:52.563 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T03:50:52.563 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T03:50:52.563 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T03:50:52.563 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T03:50:52.563 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T03:50:52.563 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T03:50:52.563 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T03:50:52.563 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T03:50:52.564 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T03:50:52.564 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T03:50:52.564 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T03:50:52.564 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T03:50:52.564 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T03:50:52.564 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T03:50:52.564 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T03:50:52.564 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T03:50:52.564 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T03:50:52.564 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T03:50:52.564 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T03:50:52.564 INFO:teuthology.orchestra.run.smithi064.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T03:50:52.564 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T03:50:52.564 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T03:50:52.565 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T03:50:52.565 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T03:50:52.565 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T03:50:52.565 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T03:50:52.565 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T03:50:52.565 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T03:50:52.565 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T03:50:52.565 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T03:50:52.565 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T03:50:52.565 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T03:50:52.565 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T03:50:52.565 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T03:50:52.565 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T03:50:52.565 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T03:50:52.566 INFO:teuthology.orchestra.run.smithi064.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T03:50:52.566 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T03:50:52.566 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T03:50:52.566 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T03:50:52.566 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T03:50:52.566 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T03:50:52.566 INFO:teuthology.orchestra.run.smithi064.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T03:50:52.566 INFO:teuthology.orchestra.run.smithi064.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T03:50:52.566 INFO:teuthology.orchestra.run.smithi064.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T03:50:52.566 INFO:teuthology.orchestra.run.smithi064.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T03:50:52.566 INFO:teuthology.orchestra.run.smithi064.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T03:50:52.566 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T03:50:52.566 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T03:50:52.567 INFO:teuthology.orchestra.run.smithi064.stdout:Installing weak dependencies: 2024-09-21T03:50:52.567 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T03:50:52.567 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:50:52.567 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-09-21T03:50:52.567 INFO:teuthology.orchestra.run.smithi064.stdout:======================================================================================= 2024-09-21T03:50:52.567 INFO:teuthology.orchestra.run.smithi064.stdout:Install 149 Packages 2024-09-21T03:50:52.567 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrade 8 Packages 2024-09-21T03:50:52.567 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:50:52.567 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 228 M 2024-09-21T03:50:52.568 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-09-21T03:50:52.604 INFO:teuthology.orchestra.run.smithi131.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 7.6 MB/s | 1.5 MB 00:00 2024-09-21T03:50:52.748 INFO:teuthology.orchestra.run.smithi119.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 2.8 MB/s | 873 kB 00:00 2024-09-21T03:50:52.815 INFO:teuthology.orchestra.run.smithi119.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 2.2 MB/s | 148 kB 00:00 2024-09-21T03:50:52.905 INFO:teuthology.orchestra.run.smithi131.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 12 MB/s | 5.0 MB 00:00 2024-09-21T03:50:52.921 INFO:teuthology.orchestra.run.smithi064.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 43 kB/s | 6.6 kB 00:00 2024-09-21T03:50:53.457 INFO:teuthology.orchestra.run.smithi119.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 4.9 MB/s | 5.5 MB 00:01 2024-09-21T03:50:53.499 INFO:teuthology.orchestra.run.smithi119.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 3.3 MB/s | 2.3 MB 00:00 2024-09-21T03:50:53.529 INFO:teuthology.orchestra.run.smithi064.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 1.4 MB/s | 873 kB 00:00 2024-09-21T03:50:53.572 INFO:teuthology.orchestra.run.smithi131.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 11 MB/s | 20 MB 00:01 2024-09-21T03:50:53.647 INFO:teuthology.orchestra.run.smithi131.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 337 kB/s | 25 kB 00:00 2024-09-21T03:50:53.655 INFO:teuthology.orchestra.run.smithi064.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.2 MB/s | 148 kB 00:00 2024-09-21T03:50:53.674 INFO:teuthology.orchestra.run.smithi119.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 7.0 MB/s | 1.5 MB 00:00 2024-09-21T03:50:53.797 INFO:teuthology.orchestra.run.smithi131.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 12 MB/s | 10 MB 00:00 2024-09-21T03:50:53.897 INFO:teuthology.orchestra.run.smithi131.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 2.2 MB/s | 226 kB 00:00 2024-09-21T03:50:53.989 INFO:teuthology.orchestra.run.smithi131.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 295 kB/s | 27 kB 00:00 2024-09-21T03:50:54.190 INFO:teuthology.orchestra.run.smithi131.stdout:(13/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 11 MB/s | 18 MB 00:01 2024-09-21T03:50:54.207 INFO:teuthology.orchestra.run.smithi131.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 146 kB/s | 32 kB 00:00 2024-09-21T03:50:54.282 INFO:teuthology.orchestra.run.smithi131.stdout:(15/157): libcephfs2-19.3.0-5120.g18432218.el9. 8.2 MB/s | 766 kB 00:00 2024-09-21T03:50:54.307 INFO:teuthology.orchestra.run.smithi131.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 1.5 MB/s | 158 kB 00:00 2024-09-21T03:50:54.374 INFO:teuthology.orchestra.run.smithi131.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 1.3 MB/s | 127 kB 00:00 2024-09-21T03:50:54.416 INFO:teuthology.orchestra.run.smithi131.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 4.6 MB/s | 507 kB 00:00 2024-09-21T03:50:54.525 INFO:teuthology.orchestra.run.smithi131.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 416 kB/s | 45 kB 00:00 2024-09-21T03:50:54.625 INFO:teuthology.orchestra.run.smithi131.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.4 MB/s | 148 kB 00:00 2024-09-21T03:50:54.638 INFO:teuthology.orchestra.run.smithi064.stdout:(4/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 2.3 MB/s | 2.3 MB 00:00 2024-09-21T03:50:54.709 INFO:teuthology.orchestra.run.smithi131.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 1.9 MB/s | 161 kB 00:00 2024-09-21T03:50:54.809 INFO:teuthology.orchestra.run.smithi131.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 3.1 MB/s | 321 kB 00:00 2024-09-21T03:50:54.821 INFO:teuthology.orchestra.run.smithi064.stdout:(5/157): ceph-base-19.3.0-5120.g18432218.el9.x8 2.7 MB/s | 5.5 MB 00:02 2024-09-21T03:50:54.874 INFO:teuthology.orchestra.run.smithi119.stdout:(7/157): ceph-common-19.3.0-5120.g18432218.el9. 7.8 MB/s | 20 MB 00:02 2024-09-21T03:50:54.884 INFO:teuthology.orchestra.run.smithi131.stdout:(23/157): python3-rbd-19.3.0-5120.g18432218.el9 4.0 MB/s | 307 kB 00:00 2024-09-21T03:50:54.941 INFO:teuthology.orchestra.run.smithi119.stdout:(8/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 3.4 MB/s | 5.0 MB 00:01 2024-09-21T03:50:55.002 INFO:teuthology.orchestra.run.smithi131.stdout:(24/157): librgw2-19.3.0-5120.g18432218.el9.x86 8.4 MB/s | 5.3 MB 00:00 2024-09-21T03:50:55.008 INFO:teuthology.orchestra.run.smithi119.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 378 kB/s | 25 kB 00:00 2024-09-21T03:50:55.022 INFO:teuthology.orchestra.run.smithi064.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 4.0 MB/s | 1.5 MB 00:00 2024-09-21T03:50:55.027 INFO:teuthology.orchestra.run.smithi131.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 700 kB/s | 99 kB 00:00 2024-09-21T03:50:55.060 INFO:teuthology.orchestra.run.smithi131.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.4 MB/s | 86 kB 00:00 2024-09-21T03:50:55.169 INFO:teuthology.orchestra.run.smithi131.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 1.5 MB/s | 170 kB 00:00 2024-09-21T03:50:55.244 INFO:teuthology.orchestra.run.smithi131.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 457 kB/s | 34 kB 00:00 2024-09-21T03:50:55.316 INFO:teuthology.orchestra.run.smithi119.stdout:(10/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 11 MB/s | 18 MB 00:01 2024-09-21T03:50:55.337 INFO:teuthology.orchestra.run.smithi131.stdout:(29/157): rbd-mirror-19.3.0-5120.g18432218.el9. 10 MB/s | 3.1 MB 00:00 2024-09-21T03:50:55.362 INFO:teuthology.orchestra.run.smithi131.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 1.3 MB/s | 154 kB 00:00 2024-09-21T03:50:55.383 INFO:teuthology.orchestra.run.smithi119.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 3.3 MB/s | 226 kB 00:00 2024-09-21T03:50:55.467 INFO:teuthology.orchestra.run.smithi119.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 325 kB/s | 27 kB 00:00 2024-09-21T03:50:55.497 INFO:teuthology.orchestra.run.smithi064.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 7.4 MB/s | 5.0 MB 00:00 2024-09-21T03:50:55.542 INFO:teuthology.orchestra.run.smithi119.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 424 kB/s | 32 kB 00:00 2024-09-21T03:50:56.104 INFO:teuthology.orchestra.run.smithi131.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 9.9 MB/s | 7.4 MB 00:00 2024-09-21T03:50:56.212 INFO:teuthology.orchestra.run.smithi131.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 2.6 MB/s | 292 kB 00:00 2024-09-21T03:50:56.282 INFO:teuthology.orchestra.run.smithi131.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 707 kB/s | 49 kB 00:00 2024-09-21T03:50:56.342 INFO:teuthology.orchestra.run.smithi119.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 958 kB/s | 766 kB 00:00 2024-09-21T03:50:56.349 INFO:teuthology.orchestra.run.smithi131.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 246 kB/s | 16 kB 00:00 2024-09-21T03:50:56.425 INFO:teuthology.orchestra.run.smithi131.stdout:(35/157): ceph-volume-19.3.0-5120.g18432218.el9 3.7 MB/s | 285 kB 00:00 2024-09-21T03:50:56.517 INFO:teuthology.orchestra.run.smithi131.stdout:(36/157): cephadm-19.3.0-5120.g18432218.el9.noa 8.4 MB/s | 784 kB 00:00 2024-09-21T03:50:56.568 INFO:teuthology.orchestra.run.smithi119.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 705 kB/s | 158 kB 00:00 2024-09-21T03:50:56.651 INFO:teuthology.orchestra.run.smithi119.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 1.5 MB/s | 127 kB 00:00 2024-09-21T03:50:56.706 INFO:teuthology.orchestra.run.smithi064.stdout:(8/157): ceph-radosgw-19.3.0-5120.g18432218.el9 8.6 MB/s | 10 MB 00:01 2024-09-21T03:50:56.709 INFO:teuthology.orchestra.run.smithi131.stdout:(37/157): ceph-mgr-dashboard-19.3.0-5120.g18432 10 MB/s | 14 MB 00:01 2024-09-21T03:50:56.743 INFO:teuthology.orchestra.run.smithi131.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 81 kB/s | 18 kB 00:00 2024-09-21T03:50:56.756 INFO:teuthology.orchestra.run.smithi064.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 505 kB/s | 25 kB 00:00 2024-09-21T03:50:56.827 INFO:teuthology.orchestra.run.smithi119.stdout:(17/157): libradosstriper1-19.3.0-5120.g1843221 2.8 MB/s | 507 kB 00:00 2024-09-21T03:50:57.176 INFO:teuthology.orchestra.run.smithi131.stdout:(39/157): ceph-test-19.3.0-5120.g18432218.el9.x 11 MB/s | 40 MB 00:03 2024-09-21T03:50:57.202 INFO:teuthology.orchestra.run.smithi131.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 162 kB/s | 80 kB 00:00 2024-09-21T03:50:57.265 INFO:teuthology.orchestra.run.smithi064.stdout:(10/157): ceph-common-19.3.0-5120.g18432218.el9 4.4 MB/s | 20 MB 00:04 2024-09-21T03:50:57.293 INFO:teuthology.orchestra.run.smithi131.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 801 kB 00:00 2024-09-21T03:50:57.319 INFO:teuthology.orchestra.run.smithi131.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-09-21T03:50:57.344 INFO:teuthology.orchestra.run.smithi131.stdout:(43/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-21T03:50:57.369 INFO:teuthology.orchestra.run.smithi131.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-21T03:50:57.395 INFO:teuthology.orchestra.run.smithi131.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-21T03:50:57.420 INFO:teuthology.orchestra.run.smithi131.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-21T03:50:57.445 INFO:teuthology.orchestra.run.smithi131.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-21T03:50:57.448 INFO:teuthology.orchestra.run.smithi064.stdout:(11/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 7.3 MB/s | 18 MB 00:02 2024-09-21T03:50:57.473 INFO:teuthology.orchestra.run.smithi064.stdout:(12/157): cephfs-mirror-19.3.0-5120.g18432218.e 1.1 MB/s | 226 kB 00:00 2024-09-21T03:50:57.479 INFO:teuthology.orchestra.run.smithi131.stdout:(48/157): python3-cryptography-36.0.1-4.el9.x86 6.6 MB/s | 1.2 MB 00:00 2024-09-21T03:50:57.498 INFO:teuthology.orchestra.run.smithi064.stdout:(13/157): cephfs-shell-19.3.0-5120.g18432218.el 540 kB/s | 27 kB 00:00 2024-09-21T03:50:57.524 INFO:teuthology.orchestra.run.smithi064.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 635 kB/s | 32 kB 00:00 2024-09-21T03:50:57.552 INFO:teuthology.orchestra.run.smithi119.stdout:(18/157): ceph-radosgw-19.3.0-5120.g18432218.el 3.9 MB/s | 10 MB 00:02 2024-09-21T03:50:57.574 INFO:teuthology.orchestra.run.smithi064.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 3.1 MB/s | 158 kB 00:00 2024-09-21T03:50:57.608 INFO:teuthology.orchestra.run.smithi064.stdout:(16/157): libcephfs2-19.3.0-5120.g18432218.el9. 6.9 MB/s | 766 kB 00:00 2024-09-21T03:50:57.627 INFO:teuthology.orchestra.run.smithi119.stdout:(19/157): librgw2-19.3.0-5120.g18432218.el9.x86 6.6 MB/s | 5.3 MB 00:00 2024-09-21T03:50:57.633 INFO:teuthology.orchestra.run.smithi064.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 2.1 MB/s | 127 kB 00:00 2024-09-21T03:50:57.652 INFO:teuthology.orchestra.run.smithi119.stdout:(20/157): python3-ceph-argparse-19.3.0-5120.g18 452 kB/s | 45 kB 00:00 2024-09-21T03:50:57.677 INFO:teuthology.orchestra.run.smithi119.stdout:(21/157): python3-ceph-common-19.3.0-5120.g1843 2.9 MB/s | 148 kB 00:00 2024-09-21T03:50:57.683 INFO:teuthology.orchestra.run.smithi064.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 6.6 MB/s | 507 kB 00:00 2024-09-21T03:50:57.703 INFO:teuthology.orchestra.run.smithi119.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 3.1 MB/s | 161 kB 00:00 2024-09-21T03:50:57.733 INFO:teuthology.orchestra.run.smithi064.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 908 kB/s | 45 kB 00:00 2024-09-21T03:50:57.761 INFO:teuthology.orchestra.run.smithi119.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 3.8 MB/s | 321 kB 00:00 2024-09-21T03:50:57.784 INFO:teuthology.orchestra.run.smithi064.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 2.9 MB/s | 148 kB 00:00 2024-09-21T03:50:57.812 INFO:teuthology.orchestra.run.smithi119.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 2.8 MB/s | 307 kB 00:00 2024-09-21T03:50:57.834 INFO:teuthology.orchestra.run.smithi064.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 3.2 MB/s | 161 kB 00:00 2024-09-21T03:50:57.837 INFO:teuthology.orchestra.run.smithi119.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.3 MB/s | 99 kB 00:00 2024-09-21T03:50:57.862 INFO:teuthology.orchestra.run.smithi119.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T03:50:57.895 INFO:teuthology.orchestra.run.smithi064.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 5.2 MB/s | 321 kB 00:00 2024-09-21T03:50:57.913 INFO:teuthology.orchestra.run.smithi119.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 3.3 MB/s | 170 kB 00:00 2024-09-21T03:50:57.954 INFO:teuthology.orchestra.run.smithi064.stdout:(23/157): python3-rbd-19.3.0-5120.g18432218.el9 5.1 MB/s | 307 kB 00:00 2024-09-21T03:50:57.963 INFO:teuthology.orchestra.run.smithi119.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 687 kB/s | 34 kB 00:00 2024-09-21T03:50:58.004 INFO:teuthology.orchestra.run.smithi064.stdout:(24/157): python3-rgw-19.3.0-5120.g18432218.el9 1.9 MB/s | 99 kB 00:00 2024-09-21T03:50:58.014 INFO:teuthology.orchestra.run.smithi119.stdout:(29/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T03:50:58.054 INFO:teuthology.orchestra.run.smithi064.stdout:(25/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T03:50:58.146 INFO:teuthology.orchestra.run.smithi131.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 160 kB/s | 107 kB 00:00 2024-09-21T03:50:58.214 INFO:teuthology.orchestra.run.smithi119.stdout:(30/157): rbd-mirror-19.3.0-5120.g18432218.el9. 8.3 MB/s | 3.1 MB 00:00 2024-09-21T03:50:58.221 INFO:teuthology.orchestra.run.smithi064.stdout:(26/157): librgw2-19.3.0-5120.g18432218.el9.x86 9.0 MB/s | 5.3 MB 00:00 2024-09-21T03:50:58.263 INFO:teuthology.orchestra.run.smithi064.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 4.0 MB/s | 170 kB 00:00 2024-09-21T03:50:58.321 INFO:teuthology.orchestra.run.smithi064.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 588 kB/s | 34 kB 00:00 2024-09-21T03:50:58.380 INFO:teuthology.orchestra.run.smithi131.stdout:(50/157): automake-1.16.2-8.el9.noarch.rpm 718 kB/s | 689 kB 00:00 2024-09-21T03:50:58.405 INFO:teuthology.orchestra.run.smithi064.stdout:(29/157): rbd-mirror-19.3.0-5120.g18432218.el9. 8.8 MB/s | 3.1 MB 00:00 2024-09-21T03:50:58.430 INFO:teuthology.orchestra.run.smithi064.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 1.4 MB/s | 154 kB 00:00 2024-09-21T03:50:58.438 INFO:teuthology.orchestra.run.smithi131.stdout:(51/157): bison-3.7.4-5.el9.x86_64.rpm 950 kB/s | 943 kB 00:00 2024-09-21T03:50:58.688 INFO:teuthology.orchestra.run.smithi131.stdout:(52/157): elfutils-libelf-devel-0.191-4.el9.x86 40 kB/s | 22 kB 00:00 2024-09-21T03:50:58.905 INFO:teuthology.orchestra.run.smithi119.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 11 MB/s | 7.4 MB 00:00 2024-09-21T03:50:58.964 INFO:teuthology.orchestra.run.smithi119.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 4.9 MB/s | 292 kB 00:00 2024-09-21T03:50:58.980 INFO:teuthology.orchestra.run.smithi131.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 60 kB/s | 33 kB 00:00 2024-09-21T03:50:59.022 INFO:teuthology.orchestra.run.smithi119.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 846 kB/s | 49 kB 00:00 2024-09-21T03:50:59.081 INFO:teuthology.orchestra.run.smithi119.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 283 kB/s | 16 kB 00:00 2024-09-21T03:50:59.140 INFO:teuthology.orchestra.run.smithi119.stdout:(35/157): ceph-volume-19.3.0-5120.g18432218.el9 4.8 MB/s | 285 kB 00:00 2024-09-21T03:50:59.214 INFO:teuthology.orchestra.run.smithi131.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 380 kB/s | 313 kB 00:00 2024-09-21T03:50:59.215 INFO:teuthology.orchestra.run.smithi119.stdout:(36/157): cephadm-19.3.0-5120.g18432218.el9.noa 10 MB/s | 784 kB 00:00 2024-09-21T03:50:59.257 INFO:teuthology.orchestra.run.smithi119.stdout:(37/157): cephfs-top-19.3.0-5120.g18432218.el9. 443 kB/s | 18 kB 00:00 2024-09-21T03:50:59.357 INFO:teuthology.orchestra.run.smithi119.stdout:(38/157): fuse-2.9.9-16.el9.x86_64.rpm 798 kB/s | 80 kB 00:00 2024-09-21T03:50:59.457 INFO:teuthology.orchestra.run.smithi119.stdout:(39/157): libgfortran-11.5.0-2.el9.x86_64.rpm 7.8 MB/s | 801 kB 00:00 2024-09-21T03:50:59.489 INFO:teuthology.orchestra.run.smithi064.stdout:(31/157): ceph-test-19.3.0-5120.g18432218.el9.x 15 MB/s | 40 MB 00:02 2024-09-21T03:50:59.490 INFO:teuthology.orchestra.run.smithi131.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 36 kB/s | 18 kB 00:00 2024-09-21T03:50:59.499 INFO:teuthology.orchestra.run.smithi119.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-21T03:50:59.541 INFO:teuthology.orchestra.run.smithi119.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 6.0 MB/s | 253 kB 00:00 2024-09-21T03:50:59.581 INFO:teuthology.orchestra.run.smithi064.stdout:(32/157): ceph-mgr-diskprediction-local-19.3.0- 6.4 MB/s | 7.4 MB 00:01 2024-09-21T03:50:59.606 INFO:teuthology.orchestra.run.smithi064.stdout:(33/157): ceph-mgr-modules-core-19.3.0-5120.g18 2.5 MB/s | 292 kB 00:00 2024-09-21T03:50:59.617 INFO:teuthology.orchestra.run.smithi119.stdout:(42/157): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-09-21T03:50:59.631 INFO:teuthology.orchestra.run.smithi064.stdout:(34/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 989 kB/s | 49 kB 00:00 2024-09-21T03:50:59.657 INFO:teuthology.orchestra.run.smithi064.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 327 kB/s | 16 kB 00:00 2024-09-21T03:50:59.659 INFO:teuthology.orchestra.run.smithi119.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-21T03:50:59.681 INFO:teuthology.orchestra.run.smithi131.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 24 kB/s | 11 kB 00:00 2024-09-21T03:50:59.701 INFO:teuthology.orchestra.run.smithi119.stdout:(44/157): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-09-21T03:50:59.742 INFO:teuthology.orchestra.run.smithi119.stdout:(45/157): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-09-21T03:50:59.756 INFO:teuthology.orchestra.run.smithi131.stdout:(57/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.8 MB/s | 3.0 MB 00:01 2024-09-21T03:50:59.784 INFO:teuthology.orchestra.run.smithi119.stdout:(46/157): python3-urllib3-1.26.5-6.el9.noarch.r 5.0 MB/s | 215 kB 00:00 2024-09-21T03:50:59.815 INFO:teuthology.orchestra.run.smithi064.stdout:(36/157): ceph-mgr-dashboard-19.3.0-5120.g18432 10 MB/s | 14 MB 00:01 2024-09-21T03:50:59.841 INFO:teuthology.orchestra.run.smithi064.stdout:(37/157): ceph-volume-19.3.0-5120.g18432218.el9 1.3 MB/s | 285 kB 00:00 2024-09-21T03:50:59.843 INFO:teuthology.orchestra.run.smithi119.stdout:(47/157): automake-1.16.2-8.el9.noarch.rpm 12 MB/s | 689 kB 00:00 2024-09-21T03:50:59.866 INFO:teuthology.orchestra.run.smithi064.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 370 kB/s | 18 kB 00:00 2024-09-21T03:50:59.894 INFO:teuthology.orchestra.run.smithi119.stdout:(48/157): bison-3.7.4-5.el9.x86_64.rpm 18 MB/s | 943 kB 00:00 2024-09-21T03:50:59.908 INFO:teuthology.orchestra.run.smithi064.stdout:(39/157): cephadm-19.3.0-5120.g18432218.el9.noa 3.1 MB/s | 784 kB 00:00 2024-09-21T03:50:59.935 INFO:teuthology.orchestra.run.smithi119.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-09-21T03:50:59.977 INFO:teuthology.orchestra.run.smithi119.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 523 kB/s | 22 kB 00:00 2024-09-21T03:51:00.019 INFO:teuthology.orchestra.run.smithi119.stdout:(51/157): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 313 kB 00:00 2024-09-21T03:51:00.053 INFO:teuthology.orchestra.run.smithi119.stdout:(52/157): flexiblas-3.0.4-8.el9.x86_64.rpm 973 kB/s | 33 kB 00:00 2024-09-21T03:51:00.145 INFO:teuthology.orchestra.run.smithi119.stdout:(53/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 33 MB/s | 3.0 MB 00:00 2024-09-21T03:51:00.179 INFO:teuthology.orchestra.run.smithi119.stdout:(54/157): flexiblas-openblas-openmp-3.0.4-8.el9 547 kB/s | 18 kB 00:00 2024-09-21T03:51:00.207 INFO:teuthology.orchestra.run.smithi131.stdout:(58/157): libnbd-1.20.2-2.el9.x86_64.rpm 238 kB/s | 171 kB 00:00 2024-09-21T03:51:00.220 INFO:teuthology.orchestra.run.smithi119.stdout:(55/157): libaio-devel-0.3.111-13.el9.x86_64.rp 269 kB/s | 11 kB 00:00 2024-09-21T03:51:00.263 INFO:teuthology.orchestra.run.smithi119.stdout:(56/157): libnbd-1.20.2-2.el9.x86_64.rpm 4.0 MB/s | 171 kB 00:00 2024-09-21T03:51:00.325 INFO:teuthology.orchestra.run.smithi064.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 165 kB/s | 80 kB 00:00 2024-09-21T03:51:00.373 INFO:teuthology.orchestra.run.smithi131.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 73 kB/s | 45 kB 00:00 2024-09-21T03:51:00.397 INFO:teuthology.orchestra.run.smithi119.stdout:(57/157): ceph-mgr-dashboard-19.3.0-5120.g18432 5.9 MB/s | 14 MB 00:02 2024-09-21T03:51:00.407 INFO:teuthology.orchestra.run.smithi131.stdout:(60/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 221 kB/s | 160 kB 00:00 2024-09-21T03:51:00.422 INFO:teuthology.orchestra.run.smithi119.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-09-21T03:51:00.458 INFO:teuthology.orchestra.run.smithi064.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 347 kB/s | 191 kB 00:00 2024-09-21T03:51:00.472 INFO:teuthology.orchestra.run.smithi119.stdout:(59/157): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-09-21T03:51:00.506 INFO:teuthology.orchestra.run.smithi119.stdout:(60/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 416 kB/s | 45 kB 00:00 2024-09-21T03:51:00.539 INFO:teuthology.orchestra.run.smithi119.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 8.5 MB/s | 581 kB 00:00 2024-09-21T03:51:00.565 INFO:teuthology.orchestra.run.smithi119.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-09-21T03:51:00.590 INFO:teuthology.orchestra.run.smithi119.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-21T03:51:00.632 INFO:teuthology.orchestra.run.smithi119.stdout:(64/157): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 188 kB 00:00 2024-09-21T03:51:00.657 INFO:teuthology.orchestra.run.smithi119.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-21T03:51:00.683 INFO:teuthology.orchestra.run.smithi119.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 766 kB/s | 38 kB 00:00 2024-09-21T03:51:00.683 INFO:teuthology.orchestra.run.smithi064.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 980 kB/s | 801 kB 00:00 2024-09-21T03:51:00.709 INFO:teuthology.orchestra.run.smithi064.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 660 kB/s | 253 kB 00:00 2024-09-21T03:51:00.801 INFO:teuthology.orchestra.run.smithi064.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 912 kB/s | 106 kB 00:00 2024-09-21T03:51:00.816 INFO:teuthology.orchestra.run.smithi119.stdout:(67/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 33 MB/s | 4.4 MB 00:00 2024-09-21T03:51:00.826 INFO:teuthology.orchestra.run.smithi064.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-09-21T03:51:00.891 INFO:teuthology.orchestra.run.smithi119.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 21 MB/s | 4.9 MB 00:00 2024-09-21T03:51:00.918 INFO:teuthology.orchestra.run.smithi064.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-09-21T03:51:00.933 INFO:teuthology.orchestra.run.smithi119.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-21T03:51:00.941 INFO:teuthology.orchestra.run.smithi131.stdout:(61/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 46 kB/s | 24 kB 00:00 2024-09-21T03:51:00.943 INFO:teuthology.orchestra.run.smithi064.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-09-21T03:51:00.967 INFO:teuthology.orchestra.run.smithi119.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-21T03:51:00.992 INFO:teuthology.orchestra.run.smithi119.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-21T03:51:01.043 INFO:teuthology.orchestra.run.smithi064.stdout:(48/157): python3-cryptography-36.0.1-4.el9.x86 2.1 MB/s | 1.2 MB 00:00 2024-09-21T03:51:01.068 INFO:teuthology.orchestra.run.smithi119.stdout:(72/157): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-21T03:51:01.109 INFO:teuthology.orchestra.run.smithi119.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-09-21T03:51:01.157 INFO:teuthology.orchestra.run.smithi131.stdout:(62/157): librdkafka-1.6.1-102.el9.x86_64.rpm 697 kB/s | 662 kB 00:00 2024-09-21T03:51:01.177 INFO:teuthology.orchestra.run.smithi064.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 2.6 MB/s | 689 kB 00:00 2024-09-21T03:51:01.185 INFO:teuthology.orchestra.run.smithi119.stdout:(74/157): postgresql-server-13.16-1.el9.x86_64. 27 MB/s | 5.8 MB 00:00 2024-09-21T03:51:01.210 INFO:teuthology.orchestra.run.smithi064.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 641 kB/s | 107 kB 00:00 2024-09-21T03:51:01.224 INFO:teuthology.orchestra.run.smithi131.stdout:(63/157): libtool-2.4.6-46.el9.x86_64.rpm 683 kB/s | 581 kB 00:00 2024-09-21T03:51:01.227 INFO:teuthology.orchestra.run.smithi119.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-21T03:51:01.236 INFO:teuthology.orchestra.run.smithi064.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 372 kB/s | 22 kB 00:00 2024-09-21T03:51:01.295 INFO:teuthology.orchestra.run.smithi064.stdout:(52/157): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-09-21T03:51:01.302 INFO:teuthology.orchestra.run.smithi119.stdout:(76/157): python3-babel-2.9.1-2.el9.noarch.rpm 31 MB/s | 6.0 MB 00:00 2024-09-21T03:51:01.320 INFO:teuthology.orchestra.run.smithi064.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-21T03:51:01.327 INFO:teuthology.orchestra.run.smithi119.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-21T03:51:01.352 INFO:teuthology.orchestra.run.smithi119.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-21T03:51:01.353 INFO:teuthology.orchestra.run.smithi064.stdout:(54/157): bison-3.7.4-5.el9.x86_64.rpm 2.2 MB/s | 943 kB 00:00 2024-09-21T03:51:01.378 INFO:teuthology.orchestra.run.smithi119.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-09-21T03:51:01.379 INFO:teuthology.orchestra.run.smithi064.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 310 kB/s | 18 kB 00:00 2024-09-21T03:51:01.403 INFO:teuthology.orchestra.run.smithi119.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-21T03:51:01.404 INFO:teuthology.orchestra.run.smithi064.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-21T03:51:01.428 INFO:teuthology.orchestra.run.smithi119.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-21T03:51:01.453 INFO:teuthology.orchestra.run.smithi119.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-09-21T03:51:01.533 INFO:teuthology.orchestra.run.smithi131.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 80 kB/s | 47 kB 00:00 2024-09-21T03:51:01.554 INFO:teuthology.orchestra.run.smithi064.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 974 kB/s | 171 kB 00:00 2024-09-21T03:51:01.580 INFO:teuthology.orchestra.run.smithi064.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 914 kB/s | 160 kB 00:00 2024-09-21T03:51:01.722 INFO:teuthology.orchestra.run.smithi064.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 271 kB/s | 45 kB 00:00 2024-09-21T03:51:01.781 INFO:teuthology.orchestra.run.smithi064.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-09-21T03:51:01.821 INFO:teuthology.orchestra.run.smithi119.stdout:(83/157): ceph-test-19.3.0-5120.g18432218.el9.x 5.9 MB/s | 40 MB 00:06 2024-09-21T03:51:01.831 INFO:teuthology.orchestra.run.smithi064.stdout:(61/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 491 kB/s | 24 kB 00:00 2024-09-21T03:51:01.846 INFO:teuthology.orchestra.run.smithi119.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 639 kB/s | 250 kB 00:00 2024-09-21T03:51:01.881 INFO:teuthology.orchestra.run.smithi064.stdout:(62/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 958 kB/s | 47 kB 00:00 2024-09-21T03:51:01.914 INFO:teuthology.orchestra.run.smithi064.stdout:(63/157): libtool-2.4.6-46.el9.x86_64.rpm 2.9 MB/s | 581 kB 00:00 2024-09-21T03:51:01.921 INFO:teuthology.orchestra.run.smithi119.stdout:(85/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 10 MB/s | 5.1 MB 00:00 2024-09-21T03:51:01.942 INFO:teuthology.orchestra.run.smithi131.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 377 kB/s | 292 kB 00:00 2024-09-21T03:51:01.947 INFO:teuthology.orchestra.run.smithi119.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 771 kB/s | 77 kB 00:00 2024-09-21T03:51:01.972 INFO:teuthology.orchestra.run.smithi119.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 846 kB/s | 42 kB 00:00 2024-09-21T03:51:01.984 INFO:teuthology.orchestra.run.smithi131.stdout:(66/157): lua-5.4.4-4.el9.x86_64.rpm 248 kB/s | 188 kB 00:00 2024-09-21T03:51:01.997 INFO:teuthology.orchestra.run.smithi119.stdout:(88/157): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 222 kB 00:00 2024-09-21T03:51:01.998 INFO:teuthology.orchestra.run.smithi064.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-21T03:51:02.023 INFO:teuthology.orchestra.run.smithi119.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-09-21T03:51:02.023 INFO:teuthology.orchestra.run.smithi064.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-21T03:51:02.040 INFO:teuthology.orchestra.run.smithi064.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 917 kB/s | 38 kB 00:00 2024-09-21T03:51:02.048 INFO:teuthology.orchestra.run.smithi119.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-09-21T03:51:02.073 INFO:teuthology.orchestra.run.smithi119.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-21T03:51:02.090 INFO:teuthology.orchestra.run.smithi119.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-09-21T03:51:02.092 INFO:teuthology.orchestra.run.smithi131.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 69 kB/s | 38 kB 00:00 2024-09-21T03:51:02.115 INFO:teuthology.orchestra.run.smithi119.stdout:(93/157): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-09-21T03:51:02.132 INFO:teuthology.orchestra.run.smithi119.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 997 kB/s | 42 kB 00:00 2024-09-21T03:51:02.149 INFO:teuthology.orchestra.run.smithi119.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-09-21T03:51:02.182 INFO:teuthology.orchestra.run.smithi119.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-21T03:51:02.199 INFO:teuthology.orchestra.run.smithi064.stdout:(67/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.3 MB/s | 3.0 MB 00:00 2024-09-21T03:51:02.208 INFO:teuthology.orchestra.run.smithi119.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 615 kB/s | 36 kB 00:00 2024-09-21T03:51:02.233 INFO:teuthology.orchestra.run.smithi119.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-21T03:51:02.267 INFO:teuthology.orchestra.run.smithi119.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-21T03:51:02.292 INFO:teuthology.orchestra.run.smithi119.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 986 kB/s | 58 kB 00:00 2024-09-21T03:51:02.317 INFO:teuthology.orchestra.run.smithi119.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-21T03:51:02.342 INFO:teuthology.orchestra.run.smithi119.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T03:51:02.368 INFO:teuthology.orchestra.run.smithi119.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-09-21T03:51:02.651 INFO:teuthology.orchestra.run.smithi119.stdout:(104/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-09-21T03:51:02.677 INFO:teuthology.orchestra.run.smithi119.stdout:(105/157): lua-devel-5.4.4-4.el9.x86_64.rpm 67 kB/s | 22 kB 00:00 2024-09-21T03:51:02.707 INFO:teuthology.orchestra.run.smithi064.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 3.1 MB/s | 1.6 MB 00:00 2024-09-21T03:51:02.735 INFO:teuthology.orchestra.run.smithi119.stdout:(106/157): abseil-cpp-20211102.0-3.el9.x86_64.r 6.4 MB/s | 548 kB 00:00 2024-09-21T03:51:02.761 INFO:teuthology.orchestra.run.smithi119.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 778 kB/s | 19 kB 00:00 2024-09-21T03:51:02.802 INFO:teuthology.orchestra.run.smithi119.stdout:(108/157): gperftools-libs-2.9.1-3.el9.x86_64.r 2.4 MB/s | 308 kB 00:00 2024-09-21T03:51:02.828 INFO:teuthology.orchestra.run.smithi119.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-21T03:51:02.853 INFO:teuthology.orchestra.run.smithi119.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T03:51:02.886 INFO:teuthology.orchestra.run.smithi119.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-21T03:51:02.921 INFO:teuthology.orchestra.run.smithi119.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-21T03:51:02.979 INFO:teuthology.orchestra.run.smithi119.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-21T03:51:03.007 INFO:teuthology.orchestra.run.smithi064.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 3.0 MB/s | 910 kB 00:00 2024-09-21T03:51:03.063 INFO:teuthology.orchestra.run.smithi119.stdout:(114/157): libarrow-9.0.0-11.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-09-21T03:51:03.068 INFO:teuthology.orchestra.run.smithi131.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 4.4 MB/s | 4.9 MB 00:01 2024-09-21T03:51:03.087 INFO:teuthology.orchestra.run.smithi119.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T03:51:03.091 INFO:teuthology.orchestra.run.smithi064.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 1.6 MB/s | 137 kB 00:00 2024-09-21T03:51:03.121 INFO:teuthology.orchestra.run.smithi119.stdout:(116/157): python3-asyncssh-2.13.2-5.el9.noarch 3.8 MB/s | 548 kB 00:00 2024-09-21T03:51:03.135 INFO:teuthology.orchestra.run.smithi131.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.8 MB/s | 4.4 MB 00:01 2024-09-21T03:51:03.147 INFO:teuthology.orchestra.run.smithi119.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 546 kB/s | 32 kB 00:00 2024-09-21T03:51:03.168 INFO:teuthology.orchestra.run.smithi131.stdout:(70/157): postgresql-13.16-1.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-09-21T03:51:03.172 INFO:teuthology.orchestra.run.smithi119.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-21T03:51:03.183 INFO:teuthology.orchestra.run.smithi064.stdout:(71/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2024-09-21T03:51:03.197 INFO:teuthology.orchestra.run.smithi119.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-09-21T03:51:03.222 INFO:teuthology.orchestra.run.smithi119.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-21T03:51:03.248 INFO:teuthology.orchestra.run.smithi119.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-21T03:51:03.273 INFO:teuthology.orchestra.run.smithi119.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-21T03:51:03.307 INFO:teuthology.orchestra.run.smithi119.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-21T03:51:03.332 INFO:teuthology.orchestra.run.smithi119.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-21T03:51:03.357 INFO:teuthology.orchestra.run.smithi119.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 707 kB/s | 18 kB 00:00 2024-09-21T03:51:03.391 INFO:teuthology.orchestra.run.smithi119.stdout:(126/157): protobuf-compiler-3.14.0-13.el9.x86_ 844 kB/s | 863 kB 00:01 2024-09-21T03:51:03.416 INFO:teuthology.orchestra.run.smithi119.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 395 kB/s | 23 kB 00:00 2024-09-21T03:51:03.441 INFO:teuthology.orchestra.run.smithi119.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 780 kB/s | 19 kB 00:00 2024-09-21T03:51:03.491 INFO:teuthology.orchestra.run.smithi119.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-09-21T03:51:03.492 INFO:teuthology.orchestra.run.smithi064.stdout:(72/157): openblas-openmp-0.3.26-2.el9.x86_64.r 3.4 MB/s | 4.9 MB 00:01 2024-09-21T03:51:03.516 INFO:teuthology.orchestra.run.smithi119.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 2.0 MB/s | 46 kB 00:00 2024-09-21T03:51:03.525 INFO:teuthology.orchestra.run.smithi064.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 3.0 MB/s | 1.0 MB 00:00 2024-09-21T03:51:03.533 INFO:teuthology.orchestra.run.smithi119.stdout:(131/157): python3-jaraco-text-3.2.0-6.el9.noar 167 kB/s | 20 kB 00:00 2024-09-21T03:51:03.550 INFO:teuthology.orchestra.run.smithi064.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-21T03:51:03.559 INFO:teuthology.orchestra.run.smithi119.stdout:(132/157): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-09-21T03:51:03.584 INFO:teuthology.orchestra.run.smithi119.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-21T03:51:03.609 INFO:teuthology.orchestra.run.smithi119.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-21T03:51:03.617 INFO:teuthology.orchestra.run.smithi064.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 919 kB/s | 61 kB 00:00 2024-09-21T03:51:03.635 INFO:teuthology.orchestra.run.smithi119.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-21T03:51:03.660 INFO:teuthology.orchestra.run.smithi119.stdout:(136/157): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-21T03:51:03.677 INFO:teuthology.orchestra.run.smithi119.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 677 kB/s | 28 kB 00:00 2024-09-21T03:51:03.694 INFO:teuthology.orchestra.run.smithi119.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 914 kB/s | 31 kB 00:00 2024-09-21T03:51:03.709 INFO:teuthology.orchestra.run.smithi064.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-09-21T03:51:03.711 INFO:teuthology.orchestra.run.smithi119.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 3.5 MB/s | 59 kB 00:00 2024-09-21T03:51:03.744 INFO:teuthology.orchestra.run.smithi119.stdout:(140/157): python3-kubernetes-26.1.0-2.el9.noar 3.5 MB/s | 1.0 MB 00:00 2024-09-21T03:51:03.770 INFO:teuthology.orchestra.run.smithi119.stdout:(141/157): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-09-21T03:51:03.787 INFO:teuthology.orchestra.run.smithi119.stdout:(142/157): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-09-21T03:51:03.801 INFO:teuthology.orchestra.run.smithi064.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 2.7 MB/s | 248 kB 00:00 2024-09-21T03:51:03.812 INFO:teuthology.orchestra.run.smithi119.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-09-21T03:51:03.835 INFO:teuthology.orchestra.run.smithi131.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 196 kB/s | 137 kB 00:00 2024-09-21T03:51:03.837 INFO:teuthology.orchestra.run.smithi119.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-21T03:51:03.851 INFO:teuthology.orchestra.run.smithi064.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-09-21T03:51:03.862 INFO:teuthology.orchestra.run.smithi119.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-09-21T03:51:03.887 INFO:teuthology.orchestra.run.smithi119.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 396 kB/s | 20 kB 00:00 2024-09-21T03:51:03.910 INFO:teuthology.orchestra.run.smithi064.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 329 kB/s | 19 kB 00:00 2024-09-21T03:51:03.921 INFO:teuthology.orchestra.run.smithi119.stdout:(147/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-09-21T03:51:03.955 INFO:teuthology.orchestra.run.smithi119.stdout:(148/157): re2-20211101-20.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-09-21T03:51:03.997 INFO:teuthology.orchestra.run.smithi119.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-21T03:51:04.002 INFO:teuthology.orchestra.run.smithi064.stdout:(80/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-09-21T03:51:04.069 INFO:teuthology.orchestra.run.smithi064.stdout:(81/157): python3-markupsafe-1.1.1-12.el9.x86_6 521 kB/s | 35 kB 00:00 2024-09-21T03:51:04.102 INFO:teuthology.orchestra.run.smithi131.stdout:(72/157): postgresql-contrib-13.16-1.el9.x86_64 880 kB/s | 910 kB 00:01 2024-09-21T03:51:04.205 INFO:teuthology.orchestra.run.smithi119.stdout:(150/157): NetworkManager-1.51.0-1.el9.x86_64.r 11 MB/s | 2.3 MB 00:00 2024-09-21T03:51:04.272 INFO:teuthology.orchestra.run.smithi119.stdout:(151/157): librbd1-19.3.0-5120.g18432218.el9.x8 10 MB/s | 3.2 MB 00:00 2024-09-21T03:51:04.314 INFO:teuthology.orchestra.run.smithi119.stdout:(152/157): NetworkManager-libnm-1.51.0-1.el9.x8 17 MB/s | 1.8 MB 00:00 2024-09-21T03:51:04.327 INFO:teuthology.orchestra.run.smithi131.stdout:(73/157): postgresql-server-13.16-1.el9.x86_64. 5.0 MB/s | 5.8 MB 00:01 2024-09-21T03:51:04.356 INFO:teuthology.orchestra.run.smithi119.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 5.8 MB/s | 249 kB 00:00 2024-09-21T03:51:04.373 INFO:teuthology.orchestra.run.smithi119.stdout:(154/157): NetworkManager-team-1.51.0-1.el9.x86 408 kB/s | 41 kB 00:00 2024-09-21T03:51:04.423 INFO:teuthology.orchestra.run.smithi119.stdout:(155/157): librados2-19.3.0-5120.g18432218.el9. 6.7 MB/s | 3.4 MB 00:00 2024-09-21T03:51:04.465 INFO:teuthology.orchestra.run.smithi119.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 12 MB/s | 1.4 MB 00:00 2024-09-21T03:51:04.574 INFO:teuthology.orchestra.run.smithi119.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 12 MB/s | 2.4 MB 00:00 2024-09-21T03:51:04.580 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:51:04.580 INFO:teuthology.orchestra.run.smithi119.stdout:Total 18 MB/s | 228 MB 00:12 2024-09-21T03:51:04.761 INFO:teuthology.orchestra.run.smithi131.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 133 kB/s | 87 kB 00:00 2024-09-21T03:51:04.794 INFO:teuthology.orchestra.run.smithi131.stdout:(75/157): protobuf-3.14.0-13.el9.x86_64.rpm 1.1 MB/s | 1.0 MB 00:00 2024-09-21T03:51:05.053 INFO:teuthology.orchestra.run.smithi064.stdout:(82/157): postgresql-server-13.16-1.el9.x86_64. 2.9 MB/s | 5.8 MB 00:01 2024-09-21T03:51:05.128 INFO:teuthology.orchestra.run.smithi064.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 3.7 MB/s | 6.0 MB 00:01 2024-09-21T03:51:05.162 INFO:teuthology.orchestra.run.smithi064.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-09-21T03:51:05.229 INFO:teuthology.orchestra.run.smithi064.stdout:(85/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.4 MB/s | 5.1 MB 00:01 2024-09-21T03:51:05.270 INFO:teuthology.orchestra.run.smithi064.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 713 kB/s | 77 kB 00:00 2024-09-21T03:51:05.296 INFO:teuthology.orchestra.run.smithi064.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 633 kB/s | 42 kB 00:00 2024-09-21T03:51:05.321 INFO:teuthology.orchestra.run.smithi064.stdout:(88/157): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 222 kB 00:00 2024-09-21T03:51:05.363 INFO:teuthology.orchestra.run.smithi064.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-09-21T03:51:05.388 INFO:teuthology.orchestra.run.smithi064.stdout:(90/157): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-09-21T03:51:05.411 INFO:teuthology.orchestra.run.smithi131.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 94 kB/s | 61 kB 00:00 2024-09-21T03:51:05.413 INFO:teuthology.orchestra.run.smithi064.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-09-21T03:51:05.438 INFO:teuthology.orchestra.run.smithi064.stdout:(92/157): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-09-21T03:51:05.464 INFO:teuthology.orchestra.run.smithi064.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 711 kB/s | 54 kB 00:00 2024-09-21T03:51:05.489 INFO:teuthology.orchestra.run.smithi064.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 830 kB/s | 42 kB 00:00 2024-09-21T03:51:05.514 INFO:teuthology.orchestra.run.smithi064.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 880 kB/s | 44 kB 00:00 2024-09-21T03:51:05.565 INFO:teuthology.orchestra.run.smithi064.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-09-21T03:51:05.590 INFO:teuthology.orchestra.run.smithi064.stdout:(97/157): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-09-21T03:51:05.615 INFO:teuthology.orchestra.run.smithi064.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-21T03:51:05.665 INFO:teuthology.orchestra.run.smithi064.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T03:51:05.678 INFO:teuthology.orchestra.run.smithi131.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 277 kB/s | 245 kB 00:00 2024-09-21T03:51:05.716 INFO:teuthology.orchestra.run.smithi064.stdout:(100/157): socat-1.7.4.1-6.el9.x86_64.rpm 2.4 MB/s | 304 kB 00:00 2024-09-21T03:51:05.741 INFO:teuthology.orchestra.run.smithi064.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.2 MB/s | 96 kB 00:00 2024-09-21T03:51:05.766 INFO:teuthology.orchestra.run.smithi064.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T03:51:05.792 INFO:teuthology.orchestra.run.smithi064.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-09-21T03:51:05.825 INFO:teuthology.orchestra.run.smithi064.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 381 kB/s | 22 kB 00:00 2024-09-21T03:51:05.831 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-09-21T03:51:05.959 INFO:teuthology.orchestra.run.smithi064.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 4.0 MB/s | 548 kB 00:00 2024-09-21T03:51:05.982 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-09-21T03:51:05.982 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-09-21T03:51:05.993 INFO:teuthology.orchestra.run.smithi064.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-21T03:51:06.018 INFO:teuthology.orchestra.run.smithi064.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 784 kB/s | 19 kB 00:00 2024-09-21T03:51:06.051 INFO:teuthology.orchestra.run.smithi064.stdout:(108/157): protobuf-compiler-3.14.0-13.el9.x86_ 3.2 MB/s | 863 kB 00:00 2024-09-21T03:51:06.102 INFO:teuthology.orchestra.run.smithi064.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-09-21T03:51:06.127 INFO:teuthology.orchestra.run.smithi064.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T03:51:06.160 INFO:teuthology.orchestra.run.smithi064.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-21T03:51:06.195 INFO:teuthology.orchestra.run.smithi064.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-21T03:51:06.212 INFO:teuthology.orchestra.run.smithi131.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 48 kB/s | 26 kB 00:00 2024-09-21T03:51:06.254 INFO:teuthology.orchestra.run.smithi131.stdout:(79/157): python3-jinja2-2.11.3-6.el9.noarch.rp 295 kB/s | 248 kB 00:00 2024-09-21T03:51:06.261 INFO:teuthology.orchestra.run.smithi064.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-21T03:51:06.312 INFO:teuthology.orchestra.run.smithi064.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-21T03:51:06.337 INFO:teuthology.orchestra.run.smithi064.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T03:51:06.362 INFO:teuthology.orchestra.run.smithi064.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T03:51:06.388 INFO:teuthology.orchestra.run.smithi064.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 563 kB/s | 14 kB 00:00 2024-09-21T03:51:06.421 INFO:teuthology.orchestra.run.smithi064.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 5.0 MB/s | 172 kB 00:00 2024-09-21T03:51:06.455 INFO:teuthology.orchestra.run.smithi064.stdout:(119/157): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2024-09-21T03:51:06.522 INFO:teuthology.orchestra.run.smithi064.stdout:(120/157): libarrow-9.0.0-11.el9.x86_64.rpm 8.8 MB/s | 4.4 MB 00:00 2024-09-21T03:51:06.547 INFO:teuthology.orchestra.run.smithi064.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-09-21T03:51:06.562 INFO:teuthology.orchestra.run.smithi131.stdout:(80/157): python3-babel-2.9.1-2.el9.noarch.rpm 2.7 MB/s | 6.0 MB 00:02 2024-09-21T03:51:06.589 INFO:teuthology.orchestra.run.smithi064.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 5.2 MB/s | 220 kB 00:00 2024-09-21T03:51:06.623 INFO:teuthology.orchestra.run.smithi064.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-21T03:51:06.648 INFO:teuthology.orchestra.run.smithi064.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-21T03:51:06.665 INFO:teuthology.orchestra.run.smithi064.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-21T03:51:06.690 INFO:teuthology.orchestra.run.smithi064.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 930 kB/s | 23 kB 00:00 2024-09-21T03:51:06.715 INFO:teuthology.orchestra.run.smithi064.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 784 kB/s | 19 kB 00:00 2024-09-21T03:51:06.755 INFO:teuthology.orchestra.run.smithi131.stdout:(81/157): python3-jsonpointer-2.0-4.el9.noarch. 35 kB/s | 19 kB 00:00 2024-09-21T03:51:06.835 INFO:teuthology.orchestra.run.smithi064.stdout:(128/157): python3-grpcio-1.46.7-10.el9.x86_64. 9.3 MB/s | 2.0 MB 00:00 2024-09-21T03:51:06.835 INFO:teuthology.orchestra.run.smithi064.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-09-21T03:51:06.835 INFO:teuthology.orchestra.run.smithi064.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T03:51:06.841 INFO:teuthology.orchestra.run.smithi064.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T03:51:06.867 INFO:teuthology.orchestra.run.smithi064.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T03:51:06.900 INFO:teuthology.orchestra.run.smithi064.stdout:(133/157): python3-pecan-1.4.2-3.el9.noarch.rpm 8.1 MB/s | 272 kB 00:00 2024-09-21T03:51:06.934 INFO:teuthology.orchestra.run.smithi064.stdout:(134/157): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-21T03:51:06.959 INFO:teuthology.orchestra.run.smithi064.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-09-21T03:51:06.985 INFO:teuthology.orchestra.run.smithi064.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T03:51:07.010 INFO:teuthology.orchestra.run.smithi064.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-21T03:51:07.035 INFO:teuthology.orchestra.run.smithi064.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-21T03:51:07.038 INFO:teuthology.orchestra.run.smithi131.stdout:(82/157): python3-mako-1.1.4-6.el9.noarch.rpm 220 kB/s | 172 kB 00:00 2024-09-21T03:51:07.060 INFO:teuthology.orchestra.run.smithi064.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T03:51:07.086 INFO:teuthology.orchestra.run.smithi064.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-21T03:51:07.111 INFO:teuthology.orchestra.run.smithi064.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 709 kB/s | 36 kB 00:00 2024-09-21T03:51:07.113 INFO:teuthology.orchestra.run.smithi131.stdout:(83/157): python3-markupsafe-1.1.1-12.el9.x86_6 63 kB/s | 35 kB 00:00 2024-09-21T03:51:07.137 INFO:teuthology.orchestra.run.smithi064.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 3.0 MB/s | 74 kB 00:00 2024-09-21T03:51:07.162 INFO:teuthology.orchestra.run.smithi064.stdout:(143/157): python3-webob-1.8.8-2.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-09-21T03:51:07.187 INFO:teuthology.orchestra.run.smithi064.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-21T03:51:07.212 INFO:teuthology.orchestra.run.smithi064.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 807 kB/s | 20 kB 00:00 2024-09-21T03:51:07.237 INFO:teuthology.orchestra.run.smithi064.stdout:(146/157): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-21T03:51:07.263 INFO:teuthology.orchestra.run.smithi064.stdout:(147/157): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-09-21T03:51:07.313 INFO:teuthology.orchestra.run.smithi064.stdout:(148/157): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-21T03:51:07.563 INFO:teuthology.orchestra.run.smithi064.stdout:(149/157): librados2-19.3.0-5120.g18432218.el9. 11 MB/s | 3.4 MB 00:00 2024-09-21T03:51:07.622 INFO:teuthology.orchestra.run.smithi064.stdout:(150/157): librbd1-19.3.0-5120.g18432218.el9.x8 10 MB/s | 3.2 MB 00:00 2024-09-21T03:51:07.785 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-09-21T03:51:07.785 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-09-21T03:51:07.922 INFO:teuthology.orchestra.run.smithi131.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 284 kB/s | 250 kB 00:00 2024-09-21T03:51:07.947 INFO:teuthology.orchestra.run.smithi131.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 267 kB/s | 222 kB 00:00 2024-09-21T03:51:08.522 INFO:teuthology.orchestra.run.smithi131.stdout:(86/157): python3-prettytable-0.7.2-27.el9.noar 73 kB/s | 42 kB 00:00 2024-09-21T03:51:08.598 INFO:teuthology.orchestra.run.smithi131.stdout:(87/157): python3-packaging-20.9-5.el9.noarch.r 114 kB/s | 77 kB 00:00 2024-09-21T03:51:08.739 INFO:teuthology.orchestra.run.smithi064.stdout:(151/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 4.9 MB/s | 16 MB 00:03 2024-09-21T03:51:08.781 INFO:teuthology.orchestra.run.smithi064.stdout:(152/157): NetworkManager-1.51.0-1.el9.x86_64.r 1.9 MB/s | 2.3 MB 00:01 2024-09-21T03:51:08.898 INFO:teuthology.orchestra.run.smithi064.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 2.1 MB/s | 249 kB 00:00 2024-09-21T03:51:08.998 INFO:teuthology.orchestra.run.smithi064.stdout:(154/157): NetworkManager-libnm-1.51.0-1.el9.x8 1.3 MB/s | 1.8 MB 00:01 2024-09-21T03:51:09.131 INFO:teuthology.orchestra.run.smithi064.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 5.8 MB/s | 1.4 MB 00:00 2024-09-21T03:51:09.157 INFO:teuthology.orchestra.run.smithi064.stdout:(156/157): NetworkManager-team-1.51.0-1.el9.x86 98 kB/s | 41 kB 00:00 2024-09-21T03:51:09.425 INFO:teuthology.orchestra.run.smithi131.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 298 kB/s | 269 kB 00:00 2024-09-21T03:51:09.456 INFO:teuthology.orchestra.run.smithi131.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 185 kB/s | 159 kB 00:00 2024-09-21T03:51:10.049 INFO:teuthology.orchestra.run.smithi131.stdout:(90/157): python3-pytz-2021.1-5.el9.noarch.rpm 86 kB/s | 51 kB 00:00 2024-09-21T03:51:10.165 INFO:teuthology.orchestra.run.smithi064.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.0 MB/s | 2.4 MB 00:01 2024-09-21T03:51:10.170 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:51:10.170 INFO:teuthology.orchestra.run.smithi064.stdout:Total 13 MB/s | 228 MB 00:17 2024-09-21T03:51:10.407 INFO:teuthology.orchestra.run.smithi131.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 284 kB/s | 279 kB 00:00 2024-09-21T03:51:10.649 INFO:teuthology.orchestra.run.smithi131.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 89 kB/s | 54 kB 00:00 2024-09-21T03:51:11.241 INFO:teuthology.orchestra.run.smithi131.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 71 kB/s | 42 kB 00:00 2024-09-21T03:51:11.449 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-09-21T03:51:11.618 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-09-21T03:51:11.618 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-09-21T03:51:11.816 INFO:teuthology.orchestra.run.smithi131.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 77 kB/s | 44 kB 00:00 2024-09-21T03:51:12.616 INFO:teuthology.orchestra.run.smithi131.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 276 kB/s | 221 kB 00:00 2024-09-21T03:51:13.208 INFO:teuthology.orchestra.run.smithi131.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 61 kB/s | 36 kB 00:00 2024-09-21T03:51:13.442 INFO:teuthology.orchestra.run.smithi131.stdout:(97/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 781 kB/s | 5.1 MB 00:06 2024-09-21T03:51:13.817 INFO:teuthology.orchestra.run.smithi131.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 78 kB/s | 47 kB 00:00 2024-09-21T03:51:13.975 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-09-21T03:51:13.975 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-09-21T03:51:14.011 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-09-21T03:51:14.127 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T03:51:14.165 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T03:51:14.219 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T03:51:14.261 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T03:51:14.286 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T03:51:14.328 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T03:51:14.417 INFO:teuthology.orchestra.run.smithi131.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 314 kB/s | 304 kB 00:00 2024-09-21T03:51:14.468 INFO:teuthology.orchestra.run.smithi131.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 89 kB/s | 58 kB 00:00 2024-09-21T03:51:14.802 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T03:51:14.879 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:51:14.988 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:51:15.022 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:51:15.097 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:51:15.126 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:51:15.143 INFO:teuthology.orchestra.run.smithi131.stdout:(101/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 94 kB/s | 64 kB 00:00 2024-09-21T03:51:15.147 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T03:51:15.168 INFO:teuthology.orchestra.run.smithi131.stdout:(102/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 128 kB/s | 96 kB 00:00 2024-09-21T03:51:15.252 INFO:teuthology.orchestra.run.smithi131.stdout:(103/157): lua-devel-5.4.4-4.el9.x86_64.rpm 268 kB/s | 22 kB 00:00 2024-09-21T03:51:15.299 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T03:51:15.357 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T03:51:15.385 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:51:15.491 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:51:15.519 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:51:15.535 INFO:teuthology.orchestra.run.smithi131.stdout:(104/157): protobuf-compiler-3.14.0-13.el9.x86_ 3.0 MB/s | 863 kB 00:00 2024-09-21T03:51:15.591 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:51:15.611 INFO:teuthology.orchestra.run.smithi131.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 7.2 MB/s | 548 kB 00:00 2024-09-21T03:51:15.612 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:51:15.634 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:51:15.644 INFO:teuthology.orchestra.run.smithi131.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-21T03:51:15.669 INFO:teuthology.orchestra.run.smithi131.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 786 kB/s | 19 kB 00:00 2024-09-21T03:51:15.753 INFO:teuthology.orchestra.run.smithi131.stdout:(108/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 74 kB/s | 45 kB 00:00 2024-09-21T03:51:15.822 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:51:15.837 INFO:teuthology.orchestra.run.smithi131.stdout:(109/157): libarrow-9.0.0-11.el9.x86_64.rpm 26 MB/s | 4.4 MB 00:00 2024-09-21T03:51:15.871 INFO:teuthology.orchestra.run.smithi131.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.5 MB/s | 49 kB 00:00 2024-09-21T03:51:15.896 INFO:teuthology.orchestra.run.smithi131.stdout:(111/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 183 kB/s | 26 kB 00:00 2024-09-21T03:51:15.921 INFO:teuthology.orchestra.run.smithi131.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-21T03:51:15.946 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:51:15.955 INFO:teuthology.orchestra.run.smithi131.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-21T03:51:16.005 INFO:teuthology.orchestra.run.smithi131.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 839 kB 00:00 2024-09-21T03:51:16.031 INFO:teuthology.orchestra.run.smithi131.stdout:(115/157): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-09-21T03:51:16.056 INFO:teuthology.orchestra.run.smithi131.stdout:(116/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 864 kB/s | 43 kB 00:00 2024-09-21T03:51:16.077 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T03:51:16.081 INFO:teuthology.orchestra.run.smithi131.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 643 kB/s | 32 kB 00:00 2024-09-21T03:51:16.098 INFO:teuthology.orchestra.run.smithi131.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 337 kB/s | 14 kB 00:00 2024-09-21T03:51:16.123 INFO:teuthology.orchestra.run.smithi131.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-09-21T03:51:16.157 INFO:teuthology.orchestra.run.smithi131.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 5.9 MB/s | 358 kB 00:00 2024-09-21T03:51:16.182 INFO:teuthology.orchestra.run.smithi131.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-21T03:51:16.187 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:51:16.208 INFO:teuthology.orchestra.run.smithi131.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-21T03:51:16.241 INFO:teuthology.orchestra.run.smithi131.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-21T03:51:16.245 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:51:16.267 INFO:teuthology.orchestra.run.smithi131.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-21T03:51:16.292 INFO:teuthology.orchestra.run.smithi131.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 705 kB/s | 18 kB 00:00 2024-09-21T03:51:16.317 INFO:teuthology.orchestra.run.smithi131.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 930 kB/s | 23 kB 00:00 2024-09-21T03:51:16.342 INFO:teuthology.orchestra.run.smithi131.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 780 kB/s | 19 kB 00:00 2024-09-21T03:51:16.368 INFO:teuthology.orchestra.run.smithi131.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 789 kB/s | 20 kB 00:00 2024-09-21T03:51:16.410 INFO:teuthology.orchestra.run.smithi131.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 9.0 MB/s | 2.0 MB 00:00 2024-09-21T03:51:16.435 INFO:teuthology.orchestra.run.smithi131.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T03:51:16.460 INFO:teuthology.orchestra.run.smithi131.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T03:51:16.461 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:51:16.486 INFO:teuthology.orchestra.run.smithi131.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T03:51:16.519 INFO:teuthology.orchestra.run.smithi131.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-09-21T03:51:16.545 INFO:teuthology.orchestra.run.smithi131.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 662 kB/s | 16 kB 00:00 2024-09-21T03:51:16.566 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T03:51:16.570 INFO:teuthology.orchestra.run.smithi131.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-21T03:51:16.591 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T03:51:16.612 INFO:teuthology.orchestra.run.smithi131.stdout:(136/157): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-09-21T03:51:16.625 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T03:51:16.637 INFO:teuthology.orchestra.run.smithi131.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 422 kB/s | 28 kB 00:00 2024-09-21T03:51:16.663 INFO:teuthology.orchestra.run.smithi131.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 610 kB/s | 31 kB 00:00 2024-09-21T03:51:16.688 INFO:teuthology.orchestra.run.smithi131.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T03:51:16.698 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T03:51:16.713 INFO:teuthology.orchestra.run.smithi131.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-21T03:51:16.738 INFO:teuthology.orchestra.run.smithi131.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 710 kB/s | 36 kB 00:00 2024-09-21T03:51:16.743 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T03:51:16.764 INFO:teuthology.orchestra.run.smithi131.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-21T03:51:16.775 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T03:51:16.789 INFO:teuthology.orchestra.run.smithi131.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-21T03:51:16.814 INFO:teuthology.orchestra.run.smithi131.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-21T03:51:16.818 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T03:51:16.840 INFO:teuthology.orchestra.run.smithi131.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 796 kB/s | 20 kB 00:00 2024-09-21T03:51:16.851 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T03:51:16.865 INFO:teuthology.orchestra.run.smithi131.stdout:(146/157): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-21T03:51:16.879 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T03:51:16.891 INFO:teuthology.orchestra.run.smithi131.stdout:(147/157): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-09-21T03:51:16.947 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T03:51:16.949 INFO:teuthology.orchestra.run.smithi131.stdout:(148/157): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-21T03:51:17.026 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T03:51:17.056 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T03:51:17.092 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T03:51:17.117 INFO:teuthology.orchestra.run.smithi131.stdout:(149/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.4 MB/s | 16 MB 00:06 2024-09-21T03:51:17.172 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T03:51:17.211 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T03:51:17.254 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T03:51:17.267 INFO:teuthology.orchestra.run.smithi131.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 9.0 MB/s | 3.4 MB 00:00 2024-09-21T03:51:17.283 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T03:51:17.325 INFO:teuthology.orchestra.run.smithi131.stdout:(151/157): librbd1-19.3.0-5120.g18432218.el9.x8 8.4 MB/s | 3.2 MB 00:00 2024-09-21T03:51:17.344 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T03:51:17.384 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T03:51:17.400 INFO:teuthology.orchestra.run.smithi131.stdout:(152/157): NetworkManager-1.51.0-1.el9.x86_64.r 8.2 MB/s | 2.3 MB 00:00 2024-09-21T03:51:17.417 INFO:teuthology.orchestra.run.smithi131.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 446 kB/s | 41 kB 00:00 2024-09-21T03:51:17.453 INFO:teuthology.orchestra.run.smithi131.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 4.8 MB/s | 249 kB 00:00 2024-09-21T03:51:17.458 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T03:51:17.502 INFO:teuthology.orchestra.run.smithi131.stdout:(155/157): NetworkManager-libnm-1.51.0-1.el9.x8 7.7 MB/s | 1.8 MB 00:00 2024-09-21T03:51:17.560 INFO:teuthology.orchestra.run.smithi131.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-09-21T03:51:17.593 INFO:teuthology.orchestra.run.smithi131.stdout:(157/157): openssl-3.2.2-6.el9.x86_64.rpm 7.7 MB/s | 1.4 MB 00:00 2024-09-21T03:51:17.599 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T03:51:17.599 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:51:17.599 INFO:teuthology.orchestra.run.smithi131.stdout:Total 8.6 MB/s | 228 MB 00:26 2024-09-21T03:51:17.653 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T03:51:18.176 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T03:51:18.265 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T03:51:18.294 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T03:51:18.337 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T03:51:18.384 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T03:51:18.425 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T03:51:18.593 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T03:51:18.653 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T03:51:18.744 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T03:51:18.817 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T03:51:18.831 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-21T03:51:18.890 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T03:51:18.922 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T03:51:18.958 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T03:51:18.983 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-21T03:51:18.983 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-21T03:51:19.004 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T03:51:19.041 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T03:51:19.084 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T03:51:19.126 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T03:51:19.269 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T03:51:19.313 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T03:51:19.354 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T03:51:19.457 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T03:51:19.496 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T03:51:19.536 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T03:51:19.562 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T03:51:19.586 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T03:51:19.661 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T03:51:20.011 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T03:51:20.060 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T03:51:20.089 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T03:51:20.123 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T03:51:20.162 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T03:51:20.194 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T03:51:20.223 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T03:51:20.258 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T03:51:20.291 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T03:51:20.333 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T03:51:20.367 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T03:51:20.453 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T03:51:20.505 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T03:51:20.536 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T03:51:20.655 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T03:51:20.704 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T03:51:20.716 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-09-21T03:51:20.811 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-21T03:51:20.811 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-21T03:51:20.828 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T03:51:20.866 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T03:51:20.920 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T03:51:20.963 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T03:51:20.997 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T03:51:21.051 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T03:51:21.415 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T03:51:21.421 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T03:51:21.468 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T03:51:21.501 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T03:51:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:51:21.601 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T03:51:21.657 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T03:51:21.669 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:51:21.698 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:51:21.701 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T03:51:21.788 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T03:51:21.791 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:51:21.820 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:51:21.829 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T03:51:21.850 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T03:51:22.023 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T03:51:22.023 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T03:51:22.052 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T03:51:22.084 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T03:51:22.111 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:51:22.129 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T03:51:22.211 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:51:22.237 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:51:22.291 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:51:22.322 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:51:22.342 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:51:22.536 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:51:22.717 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:51:22.767 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T03:51:22.845 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T03:51:22.891 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T03:51:22.960 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:51:23.030 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:51:23.249 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:51:23.374 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T03:51:23.431 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T03:51:23.469 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T03:51:23.545 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T03:51:23.595 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T03:51:23.636 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T03:51:23.679 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T03:51:23.712 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T03:51:23.739 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T03:51:23.811 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T03:51:23.871 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T03:51:23.901 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T03:51:23.944 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T03:51:24.028 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T03:51:24.063 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T03:51:24.105 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T03:51:24.107 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T03:51:24.143 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T03:51:24.214 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T03:51:24.218 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T03:51:24.241 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T03:51:24.267 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T03:51:24.274 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T03:51:24.346 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T03:51:24.489 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T03:51:24.554 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T03:51:24.600 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T03:51:24.646 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:51:24.737 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:51:24.767 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T03:51:24.803 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T03:51:25.122 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T03:51:25.218 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T03:51:25.251 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T03:51:25.295 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T03:51:25.301 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T03:51:25.332 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:51:25.332 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T03:51:25.376 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T03:51:25.402 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:51:25.434 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T03:51:25.562 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T03:51:25.619 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T03:51:25.716 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T03:51:25.785 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T03:51:25.859 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T03:51:25.896 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T03:51:25.932 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T03:51:25.980 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T03:51:26.014 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T03:51:26.058 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T03:51:26.100 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T03:51:26.246 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T03:51:26.284 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T03:51:26.326 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T03:51:26.613 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T03:51:26.652 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T03:51:26.693 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T03:51:26.741 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T03:51:26.766 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T03:51:26.871 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T03:51:27.006 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T03:51:27.040 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T03:51:27.068 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T03:51:27.102 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T03:51:27.142 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T03:51:27.178 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-21T03:51:27.182 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T03:51:27.211 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T03:51:27.241 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T03:51:27.273 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T03:51:27.275 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T03:51:27.316 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T03:51:27.327 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T03:51:27.350 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T03:51:27.380 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T03:51:27.431 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T03:51:27.440 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T03:51:27.465 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T03:51:27.498 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T03:51:27.508 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T03:51:27.529 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T03:51:27.637 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T03:51:27.682 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:51:27.689 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T03:51:27.742 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:51:27.847 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T03:51:27.967 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:51:28.091 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:51:28.118 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:51:28.200 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:51:28.230 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:51:28.259 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T03:51:28.312 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:51:28.352 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:51:28.424 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T03:51:28.446 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:51:28.447 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T03:51:28.447 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T03:51:28.447 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T03:51:28.453 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T03:51:28.476 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:51:28.486 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T03:51:28.501 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T03:51:28.515 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:51:28.535 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T03:51:28.628 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:51:28.639 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T03:51:28.657 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:51:28.692 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T03:51:28.720 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T03:51:28.737 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T03:51:28.742 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:51:28.765 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T03:51:28.827 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T03:51:28.863 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T03:51:28.960 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:51:29.063 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T03:51:29.093 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T03:51:29.101 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T03:51:29.174 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T03:51:29.224 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T03:51:29.327 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:51:29.392 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:51:29.618 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T03:51:29.728 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T03:51:29.775 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T03:51:29.809 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T03:51:29.864 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T03:51:29.891 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T03:51:29.936 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T03:51:29.976 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T03:51:29.981 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T03:51:30.028 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T03:51:30.061 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T03:51:30.106 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T03:51:30.175 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T03:51:30.228 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T03:51:30.258 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T03:51:30.293 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T03:51:30.383 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T03:51:30.421 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T03:51:30.465 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T03:51:30.493 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T03:51:30.564 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T03:51:30.609 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T03:51:30.683 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T03:51:30.815 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T03:51:30.878 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T03:51:31.317 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T03:51:31.399 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T03:51:31.414 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T03:51:31.427 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T03:51:31.460 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T03:51:31.507 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T03:51:31.536 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T03:51:31.587 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T03:51:31.633 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T03:51:31.672 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T03:51:31.812 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T03:51:31.831 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T03:51:31.856 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:51:31.891 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T03:51:31.957 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:51:31.986 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T03:51:31.991 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T03:51:32.032 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T03:51:32.065 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T03:51:32.138 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T03:51:32.169 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T03:51:32.205 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T03:51:32.252 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T03:51:32.288 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T03:51:32.331 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T03:51:32.382 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T03:51:32.518 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T03:51:32.557 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T03:51:32.558 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T03:51:32.584 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:51:32.600 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T03:51:32.655 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:51:32.686 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T03:51:32.702 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T03:51:32.733 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T03:51:32.773 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T03:51:32.807 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T03:51:32.840 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T03:51:32.915 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T03:51:33.059 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T03:51:33.098 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T03:51:33.126 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T03:51:33.168 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T03:51:33.208 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T03:51:33.240 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T03:51:33.269 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T03:51:33.304 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T03:51:33.345 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T03:51:33.387 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T03:51:33.421 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T03:51:33.508 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T03:51:33.552 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T03:51:33.583 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T03:51:33.685 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T03:51:33.743 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T03:51:34.459 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T03:51:34.507 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T03:51:34.540 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T03:51:34.666 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T03:51:34.729 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T03:51:34.782 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T03:51:34.871 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T03:51:34.919 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T03:51:35.101 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T03:51:35.133 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T03:51:35.210 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T03:51:35.425 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:51:35.457 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:51:35.864 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T03:51:35.981 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T03:51:36.060 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:51:36.102 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:51:36.178 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:51:36.178 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T03:51:36.178 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T03:51:36.178 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:51:36.211 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:51:37.195 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T03:51:37.271 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T03:51:37.299 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T03:51:37.332 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T03:51:37.655 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T03:51:37.704 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:51:37.795 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T03:51:37.825 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T03:51:37.860 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T03:51:38.357 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T03:51:38.398 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:51:38.469 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T03:51:38.500 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T03:51:40.296 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:51:40.296 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /sys 2024-09-21T03:51:40.296 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /proc 2024-09-21T03:51:40.296 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /mnt 2024-09-21T03:51:40.296 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /var/tmp 2024-09-21T03:51:40.296 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /home 2024-09-21T03:51:40.296 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /root 2024-09-21T03:51:40.296 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /tmp 2024-09-21T03:51:40.296 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T03:51:40.470 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:51:40.487 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:51:40.487 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:51:40.487 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T03:51:40.487 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T03:51:40.772 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:51:40.813 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:51:41.377 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T03:51:41.425 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:51:41.515 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T03:51:41.515 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T03:51:41.515 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T03:51:41.515 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-21T03:51:41.551 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:51:42.090 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:51:42.118 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:51:42.118 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:51:42.118 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T03:51:42.118 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:51:42.118 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:51:42.118 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T03:51:42.349 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:51:42.372 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:51:42.373 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:51:42.373 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T03:51:42.373 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:51:42.373 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:51:42.373 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T03:51:42.841 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:51:42.865 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:51:42.865 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:51:42.865 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T03:51:42.865 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:51:42.865 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:51:42.865 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T03:51:42.972 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T03:51:43.028 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T03:51:43.083 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T03:51:43.276 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T03:51:43.325 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T03:51:43.403 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T03:51:43.492 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T03:51:43.631 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T03:51:43.675 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T03:51:43.707 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T03:51:43.734 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T03:51:44.136 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:51:44.166 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:51:44.198 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T03:51:44.265 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T03:51:44.289 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:51:47.460 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:51:47.528 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:51:48.424 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:51:48.424 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /sys 2024-09-21T03:51:48.424 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /proc 2024-09-21T03:51:48.424 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /mnt 2024-09-21T03:51:48.424 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /var/tmp 2024-09-21T03:51:48.424 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /home 2024-09-21T03:51:48.424 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /root 2024-09-21T03:51:48.424 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /tmp 2024-09-21T03:51:48.424 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:51:48.490 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:51:48.520 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:51:48.608 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:51:48.626 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:51:48.626 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:51:48.626 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T03:51:48.626 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:51:48.635 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:51:48.803 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T03:51:48.831 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:51:48.854 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:51:48.854 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:51:48.854 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T03:51:48.855 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:51:48.855 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:51:48.855 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T03:51:48.898 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:51:48.910 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:51:49.009 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T03:51:49.063 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T03:51:50.027 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T03:51:50.059 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:51:50.082 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:51:50.083 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:51:50.083 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T03:51:50.083 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:51:50.083 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:51:50.083 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T03:51:50.126 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:51:50.149 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:51:50.149 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:51:50.149 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T03:51:50.149 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T03:51:50.201 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:51:50.225 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:51:50.225 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:51:50.225 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T03:51:50.225 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:51:50.225 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:51:50.225 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T03:51:50.286 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:51:50.315 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:51:50.315 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:51:50.315 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T03:51:50.315 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:51:50.315 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:51:50.315 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:51:50.544 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:51:50.554 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:51:50.568 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:51:50.568 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:51:50.568 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T03:51:50.568 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:51:50.568 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:51:50.568 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T03:51:50.579 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:51:50.579 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:51:50.579 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T03:51:50.579 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:51:50.579 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:51:50.579 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:51:51.071 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:51:51.096 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:51:51.096 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:51:51.096 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T03:51:51.096 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:51:51.096 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:51:51.096 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:51:51.202 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T03:51:51.260 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T03:51:51.324 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T03:51:51.519 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T03:51:51.573 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T03:51:51.652 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T03:51:51.750 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T03:51:51.894 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T03:51:51.939 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T03:51:51.971 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T03:51:51.998 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T03:51:52.422 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:51:52.447 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:51:52.480 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T03:51:52.549 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T03:51:52.579 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:51:53.399 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T03:51:53.399 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /sys 2024-09-21T03:51:53.399 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /proc 2024-09-21T03:51:53.399 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /mnt 2024-09-21T03:51:53.399 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /var/tmp 2024-09-21T03:51:53.399 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /home 2024-09-21T03:51:53.400 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /root 2024-09-21T03:51:53.400 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /tmp 2024-09-21T03:51:53.400 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-21T03:51:53.573 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:51:53.590 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T03:51:53.590 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:51:53.590 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T03:51:53.590 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-21T03:51:54.582 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T03:51:54.622 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T03:51:54.663 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T03:51:54.687 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T03:51:54.714 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T03:51:54.895 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T03:51:54.944 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T03:51:54.974 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T03:51:55.004 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T03:51:55.042 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T03:51:55.210 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:51:55.237 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T03:51:55.237 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:51:55.237 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T03:51:55.237 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:51:55.237 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T03:51:55.237 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-21T03:51:55.469 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:51:55.492 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T03:51:55.492 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:51:55.493 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T03:51:55.493 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:51:55.493 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T03:51:55.493 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-21T03:51:55.628 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:51:55.664 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:51:55.686 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:51:55.946 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:51:55.961 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:51:55.985 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T03:51:55.985 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:51:55.985 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T03:51:55.985 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:51:55.985 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T03:51:55.985 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-21T03:51:56.018 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:51:56.100 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T03:51:56.157 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T03:51:56.221 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T03:51:56.407 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T03:51:56.462 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T03:51:56.514 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T03:51:56.549 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T03:51:56.602 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T03:51:56.638 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T03:51:56.644 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T03:51:56.731 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T03:51:56.764 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T03:51:56.786 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T03:51:56.789 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T03:51:56.815 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T03:51:56.815 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:51:56.821 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T03:51:56.853 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T03:51:56.880 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T03:51:56.907 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:51:56.907 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:51:57.000 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:51:57.025 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T03:51:57.030 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:51:57.031 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:51:57.031 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:51:57.068 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:51:57.189 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:51:57.218 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:51:57.218 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:51:57.302 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:51:57.332 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:51:57.359 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T03:51:57.362 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T03:51:57.365 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:51:57.394 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T03:51:57.395 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:51:57.421 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:51:57.421 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:51:57.422 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T03:51:57.422 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:51:57.422 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:51:57.422 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:51:57.461 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T03:51:57.471 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:51:57.483 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:51:57.494 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:51:57.586 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T03:51:57.644 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T03:51:58.664 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T03:51:58.698 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:51:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:51:58.724 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:51:58.724 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T03:51:58.724 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:51:58.724 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:51:58.724 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:51:58.766 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:51:58.790 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:51:58.790 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:51:58.790 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T03:51:58.790 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:51:58.842 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:51:58.867 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:51:58.867 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:51:58.867 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T03:51:58.867 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:51:58.867 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:51:58.867 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:51:59.200 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:51:59.225 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:51:59.225 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:51:59.226 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T03:51:59.226 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:51:59.226 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:51:59.226 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:52:00.695 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T03:52:00.750 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:52:01.711 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T03:52:01.734 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:52:01.850 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T03:52:02.018 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T03:52:02.044 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:52:02.069 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T03:52:02.069 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:52:02.069 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T03:52:02.069 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:52:02.070 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T03:52:02.070 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-21T03:52:02.112 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:52:02.124 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T03:52:02.213 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T03:52:02.260 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T03:52:03.225 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T03:52:03.256 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:52:03.280 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T03:52:03.281 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:52:03.281 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T03:52:03.281 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:52:03.281 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T03:52:03.281 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-21T03:52:03.324 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:52:03.346 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T03:52:03.347 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:52:03.347 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T03:52:03.347 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-21T03:52:03.391 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:52:03.414 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T03:52:03.415 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:52:03.415 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T03:52:03.415 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:52:03.415 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T03:52:03.415 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-21T03:52:03.473 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T03:52:03.522 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T03:52:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T03:52:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T03:52:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T03:52:03.733 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:52:03.756 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T03:52:03.756 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T03:52:03.756 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T03:52:03.757 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:52:03.757 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T03:52:03.757 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-21T03:52:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T03:52:03.873 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:52:03.873 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T03:52:03.873 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T03:52:03.873 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T03:52:03.873 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T03:52:03.873 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T03:52:03.873 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T03:52:03.873 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T03:52:03.873 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:52:03.874 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:52:03.874 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:52:03.874 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T03:52:03.874 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T03:52:03.874 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T03:52:03.874 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:52:03.874 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T03:52:03.874 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T03:52:03.874 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T03:52:03.874 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T03:52:03.874 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T03:52:03.874 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T03:52:03.874 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T03:52:03.874 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T03:52:03.875 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T03:52:03.875 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T03:52:03.875 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T03:52:03.875 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T03:52:03.875 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T03:52:03.875 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T03:52:03.875 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T03:52:03.875 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T03:52:03.875 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T03:52:03.875 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T03:52:03.875 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T03:52:03.875 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T03:52:03.875 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T03:52:03.877 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T03:52:03.877 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T03:52:03.877 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T03:52:03.877 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T03:52:03.877 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T03:52:03.877 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T03:52:03.877 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T03:52:03.877 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T03:52:03.877 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T03:52:03.877 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T03:52:03.877 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T03:52:03.877 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T03:52:03.878 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T03:52:03.878 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T03:52:03.878 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T03:52:03.878 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T03:52:03.878 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T03:52:03.878 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T03:52:03.878 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T03:52:03.878 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T03:52:03.878 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T03:52:03.878 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T03:52:03.878 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T03:52:03.878 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T03:52:03.878 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T03:52:03.878 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T03:52:03.878 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T03:52:03.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T03:52:03.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T03:52:03.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T03:52:03.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T03:52:03.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T03:52:03.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T03:52:03.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T03:52:03.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T03:52:03.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T03:52:03.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T03:52:03.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T03:52:03.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T03:52:03.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T03:52:03.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T03:52:03.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T03:52:03.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T03:52:03.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T03:52:03.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T03:52:03.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T03:52:03.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T03:52:03.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T03:52:03.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T03:52:03.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T03:52:03.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T03:52:03.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T03:52:03.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T03:52:03.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T03:52:03.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T03:52:03.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T03:52:03.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T03:52:03.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T03:52:03.881 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T03:52:03.881 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T03:52:03.881 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T03:52:03.881 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T03:52:03.881 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T03:52:03.881 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T03:52:03.881 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T03:52:03.881 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T03:52:03.881 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T03:52:03.881 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T03:52:03.881 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T03:52:03.882 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T03:52:03.882 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T03:52:03.882 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T03:52:03.882 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T03:52:03.882 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T03:52:03.882 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T03:52:03.882 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T03:52:03.882 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T03:52:03.882 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T03:52:03.882 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T03:52:03.882 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T03:52:03.882 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T03:52:03.882 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T03:52:03.883 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T03:52:03.883 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T03:52:03.883 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T03:52:03.883 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T03:52:03.883 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T03:52:03.883 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T03:52:03.883 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T03:52:03.883 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T03:52:03.883 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T03:52:03.883 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T03:52:03.883 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T03:52:03.883 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T03:52:03.883 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T03:52:03.883 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T03:52:03.884 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T03:52:03.884 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T03:52:03.884 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T03:52:03.884 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T03:52:03.884 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T03:52:03.884 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T03:52:03.884 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T03:52:03.884 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T03:52:03.884 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T03:52:03.884 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T03:52:03.884 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T03:52:03.884 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T03:52:03.884 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T03:52:03.885 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T03:52:03.885 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T03:52:03.885 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T03:52:03.885 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T03:52:03.885 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T03:52:03.885 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T03:52:03.885 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T03:52:03.885 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T03:52:03.885 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T03:52:03.885 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T03:52:03.885 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T03:52:03.885 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T03:52:03.886 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T03:52:03.886 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T03:52:03.886 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T03:52:03.886 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T03:52:03.886 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T03:52:03.886 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T03:52:03.886 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T03:52:03.886 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T03:52:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T03:52:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T03:52:03.945 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T03:52:03.991 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T03:52:04.566 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:52:04.605 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:52:04.627 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:52:05.528 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T03:52:05.627 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T03:52:05.676 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T03:52:05.771 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T03:52:05.805 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T03:52:05.830 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T03:52:05.865 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T03:52:05.865 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:52:05.957 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:52:05.957 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:52:06.041 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:52:06.066 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T03:52:06.071 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:52:06.071 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:52:06.260 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:52:06.260 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:52:06.343 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:52:06.411 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:52:06.628 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:52:06.628 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T03:52:06.628 INFO:teuthology.orchestra.run.smithi119.stdout:Upgraded: 2024-09-21T03:52:06.628 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T03:52:06.628 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T03:52:06.628 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T03:52:06.629 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T03:52:06.629 INFO:teuthology.orchestra.run.smithi119.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.629 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.629 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T03:52:06.629 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T03:52:06.629 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-09-21T03:52:06.629 INFO:teuthology.orchestra.run.smithi119.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T03:52:06.629 INFO:teuthology.orchestra.run.smithi119.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T03:52:06.629 INFO:teuthology.orchestra.run.smithi119.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T03:52:06.629 INFO:teuthology.orchestra.run.smithi119.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T03:52:06.629 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.629 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.629 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.630 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.630 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:06.630 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.630 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.630 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.630 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:06.630 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:06.630 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:06.630 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:06.630 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:06.630 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.630 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.631 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:06.631 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.631 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.631 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.631 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:06.631 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:06.631 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.631 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.631 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:06.631 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T03:52:06.631 INFO:teuthology.orchestra.run.smithi119.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T03:52:06.631 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T03:52:06.631 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T03:52:06.631 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T03:52:06.632 INFO:teuthology.orchestra.run.smithi119.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T03:52:06.632 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T03:52:06.632 INFO:teuthology.orchestra.run.smithi119.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T03:52:06.632 INFO:teuthology.orchestra.run.smithi119.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T03:52:06.632 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T03:52:06.632 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T03:52:06.632 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.632 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.632 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.632 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T03:52:06.632 INFO:teuthology.orchestra.run.smithi119.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T03:52:06.632 INFO:teuthology.orchestra.run.smithi119.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T03:52:06.632 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T03:52:06.632 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T03:52:06.633 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T03:52:06.633 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.633 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.633 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T03:52:06.633 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.633 INFO:teuthology.orchestra.run.smithi119.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T03:52:06.633 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T03:52:06.633 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T03:52:06.633 INFO:teuthology.orchestra.run.smithi119.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T03:52:06.633 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T03:52:06.633 INFO:teuthology.orchestra.run.smithi119.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T03:52:06.633 INFO:teuthology.orchestra.run.smithi119.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T03:52:06.634 INFO:teuthology.orchestra.run.smithi119.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T03:52:06.634 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T03:52:06.634 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T03:52:06.634 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T03:52:06.634 INFO:teuthology.orchestra.run.smithi119.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T03:52:06.634 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T03:52:06.634 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T03:52:06.634 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T03:52:06.634 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T03:52:06.634 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T03:52:06.634 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T03:52:06.634 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T03:52:06.635 INFO:teuthology.orchestra.run.smithi119.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T03:52:06.635 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T03:52:06.635 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T03:52:06.635 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T03:52:06.635 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.635 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.635 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.635 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T03:52:06.635 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T03:52:06.635 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T03:52:06.635 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T03:52:06.635 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T03:52:06.635 INFO:teuthology.orchestra.run.smithi119.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T03:52:06.636 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T03:52:06.636 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T03:52:06.636 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T03:52:06.636 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T03:52:06.636 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T03:52:06.636 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T03:52:06.636 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T03:52:06.636 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T03:52:06.636 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T03:52:06.636 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T03:52:06.636 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T03:52:06.636 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T03:52:06.636 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T03:52:06.636 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T03:52:06.637 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T03:52:06.637 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T03:52:06.637 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T03:52:06.637 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T03:52:06.637 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T03:52:06.637 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T03:52:06.637 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T03:52:06.637 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T03:52:06.637 INFO:teuthology.orchestra.run.smithi119.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T03:52:06.637 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T03:52:06.637 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T03:52:06.637 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T03:52:06.637 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T03:52:06.637 INFO:teuthology.orchestra.run.smithi119.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T03:52:06.638 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T03:52:06.638 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T03:52:06.638 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T03:52:06.638 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T03:52:06.638 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T03:52:06.638 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T03:52:06.638 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.638 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.638 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T03:52:06.638 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T03:52:06.638 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T03:52:06.638 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.638 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T03:52:06.638 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T03:52:06.639 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T03:52:06.639 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T03:52:06.639 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T03:52:06.639 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T03:52:06.639 INFO:teuthology.orchestra.run.smithi119.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T03:52:06.639 INFO:teuthology.orchestra.run.smithi119.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T03:52:06.639 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T03:52:06.639 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T03:52:06.639 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T03:52:06.639 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T03:52:06.639 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T03:52:06.639 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T03:52:06.639 INFO:teuthology.orchestra.run.smithi119.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T03:52:06.639 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.640 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.640 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:06.640 INFO:teuthology.orchestra.run.smithi119.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T03:52:06.640 INFO:teuthology.orchestra.run.smithi119.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T03:52:06.640 INFO:teuthology.orchestra.run.smithi119.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T03:52:06.640 INFO:teuthology.orchestra.run.smithi119.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T03:52:06.640 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T03:52:06.640 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T03:52:06.640 INFO:teuthology.orchestra.run.smithi119.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T03:52:06.641 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T03:52:06.641 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-09-21T03:52:07.038 DEBUG:teuthology.parallel:result is None 2024-09-21T03:52:07.729 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T03:52:07.771 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T03:52:07.811 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T03:52:07.835 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T03:52:07.862 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T03:52:08.046 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T03:52:08.093 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T03:52:08.123 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T03:52:08.153 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T03:52:08.190 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T03:52:08.706 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:52:08.747 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:52:08.768 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T03:52:09.605 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T03:52:09.702 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T03:52:09.744 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T03:52:09.825 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T03:52:09.864 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T03:52:09.889 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T03:52:09.915 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T03:52:09.915 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:52:09.999 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T03:52:09.999 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:52:10.100 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T03:52:10.125 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T03:52:10.130 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:52:10.130 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:52:10.301 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T03:52:10.302 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:52:10.377 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T03:52:10.441 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:52:12.668 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:52:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T03:52:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T03:52:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T03:52:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T03:52:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T03:52:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T03:52:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T03:52:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:52:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:52:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:52:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T03:52:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T03:52:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T03:52:12.669 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:52:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T03:52:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T03:52:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T03:52:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T03:52:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T03:52:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T03:52:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T03:52:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T03:52:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T03:52:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T03:52:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T03:52:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T03:52:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T03:52:12.670 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T03:52:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T03:52:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T03:52:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T03:52:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T03:52:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T03:52:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T03:52:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T03:52:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T03:52:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T03:52:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T03:52:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T03:52:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T03:52:12.671 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T03:52:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T03:52:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T03:52:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T03:52:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T03:52:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T03:52:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T03:52:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T03:52:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T03:52:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T03:52:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T03:52:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T03:52:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T03:52:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T03:52:12.672 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T03:52:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T03:52:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T03:52:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T03:52:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T03:52:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T03:52:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T03:52:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T03:52:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T03:52:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T03:52:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T03:52:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T03:52:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T03:52:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T03:52:12.673 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T03:52:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T03:52:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T03:52:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T03:52:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T03:52:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T03:52:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T03:52:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T03:52:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T03:52:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T03:52:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T03:52:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T03:52:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T03:52:12.674 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T03:52:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T03:52:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T03:52:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T03:52:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T03:52:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T03:52:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T03:52:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T03:52:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T03:52:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T03:52:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T03:52:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T03:52:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T03:52:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T03:52:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T03:52:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T03:52:12.675 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T03:52:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T03:52:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T03:52:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T03:52:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T03:52:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T03:52:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T03:52:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T03:52:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T03:52:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T03:52:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T03:52:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T03:52:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T03:52:12.676 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T03:52:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T03:52:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T03:52:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T03:52:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T03:52:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T03:52:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T03:52:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T03:52:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T03:52:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T03:52:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T03:52:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T03:52:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T03:52:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T03:52:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T03:52:12.677 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T03:52:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T03:52:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T03:52:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T03:52:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T03:52:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T03:52:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T03:52:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T03:52:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T03:52:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T03:52:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T03:52:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T03:52:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T03:52:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T03:52:12.678 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T03:52:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T03:52:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T03:52:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T03:52:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T03:52:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T03:52:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T03:52:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T03:52:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T03:52:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T03:52:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T03:52:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T03:52:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T03:52:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T03:52:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T03:52:12.679 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T03:52:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T03:52:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T03:52:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T03:52:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T03:52:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T03:52:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T03:52:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T03:52:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T03:52:12.680 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T03:52:15.440 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:52:15.440 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:52:15.440 INFO:teuthology.orchestra.run.smithi064.stdout:Upgraded: 2024-09-21T03:52:15.440 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T03:52:15.440 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T03:52:15.440 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T03:52:15.440 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T03:52:15.440 INFO:teuthology.orchestra.run.smithi064.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.441 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.441 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T03:52:15.441 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T03:52:15.441 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-09-21T03:52:15.441 INFO:teuthology.orchestra.run.smithi064.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T03:52:15.441 INFO:teuthology.orchestra.run.smithi064.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T03:52:15.441 INFO:teuthology.orchestra.run.smithi064.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T03:52:15.441 INFO:teuthology.orchestra.run.smithi064.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T03:52:15.441 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.441 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.441 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.441 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.441 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:15.441 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.442 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.442 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.442 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:15.442 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:15.442 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:15.442 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:15.442 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:15.442 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.442 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.442 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:15.442 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.442 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.442 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.443 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:15.443 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:15.443 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.443 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.443 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:15.443 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T03:52:15.443 INFO:teuthology.orchestra.run.smithi064.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T03:52:15.443 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T03:52:15.443 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T03:52:15.443 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T03:52:15.443 INFO:teuthology.orchestra.run.smithi064.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T03:52:15.443 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T03:52:15.443 INFO:teuthology.orchestra.run.smithi064.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T03:52:15.443 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T03:52:15.444 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T03:52:15.444 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T03:52:15.444 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.444 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.444 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.444 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T03:52:15.444 INFO:teuthology.orchestra.run.smithi064.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T03:52:15.444 INFO:teuthology.orchestra.run.smithi064.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T03:52:15.444 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T03:52:15.444 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T03:52:15.444 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T03:52:15.444 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.445 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.445 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T03:52:15.445 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.445 INFO:teuthology.orchestra.run.smithi064.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T03:52:15.445 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T03:52:15.445 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T03:52:15.445 INFO:teuthology.orchestra.run.smithi064.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T03:52:15.445 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T03:52:15.445 INFO:teuthology.orchestra.run.smithi064.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T03:52:15.445 INFO:teuthology.orchestra.run.smithi064.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T03:52:15.445 INFO:teuthology.orchestra.run.smithi064.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T03:52:15.445 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T03:52:15.445 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T03:52:15.445 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T03:52:15.446 INFO:teuthology.orchestra.run.smithi064.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T03:52:15.446 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T03:52:15.446 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T03:52:15.446 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T03:52:15.446 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T03:52:15.446 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T03:52:15.446 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T03:52:15.446 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T03:52:15.446 INFO:teuthology.orchestra.run.smithi064.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T03:52:15.446 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T03:52:15.446 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T03:52:15.446 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T03:52:15.446 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.447 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.447 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.447 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T03:52:15.447 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T03:52:15.447 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T03:52:15.447 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T03:52:15.447 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T03:52:15.447 INFO:teuthology.orchestra.run.smithi064.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T03:52:15.447 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T03:52:15.447 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T03:52:15.447 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T03:52:15.447 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T03:52:15.447 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T03:52:15.447 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T03:52:15.448 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T03:52:15.448 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T03:52:15.448 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T03:52:15.448 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T03:52:15.448 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T03:52:15.448 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T03:52:15.448 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T03:52:15.448 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T03:52:15.448 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T03:52:15.448 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T03:52:15.448 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T03:52:15.448 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T03:52:15.448 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T03:52:15.448 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T03:52:15.449 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T03:52:15.449 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T03:52:15.449 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T03:52:15.449 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T03:52:15.449 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T03:52:15.449 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T03:52:15.449 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T03:52:15.449 INFO:teuthology.orchestra.run.smithi064.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T03:52:15.449 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T03:52:15.449 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T03:52:15.449 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T03:52:15.449 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T03:52:15.449 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T03:52:15.449 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T03:52:15.450 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.450 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.450 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T03:52:15.450 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T03:52:15.450 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T03:52:15.450 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.450 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T03:52:15.450 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T03:52:15.450 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T03:52:15.450 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T03:52:15.450 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T03:52:15.450 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T03:52:15.450 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T03:52:15.450 INFO:teuthology.orchestra.run.smithi064.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T03:52:15.451 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T03:52:15.451 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T03:52:15.451 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T03:52:15.451 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T03:52:15.451 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T03:52:15.451 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T03:52:15.451 INFO:teuthology.orchestra.run.smithi064.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T03:52:15.451 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.451 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.451 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:15.451 INFO:teuthology.orchestra.run.smithi064.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T03:52:15.451 INFO:teuthology.orchestra.run.smithi064.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T03:52:15.451 INFO:teuthology.orchestra.run.smithi064.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T03:52:15.451 INFO:teuthology.orchestra.run.smithi064.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T03:52:15.452 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T03:52:15.452 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T03:52:15.452 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T03:52:15.452 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:52:15.452 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-09-21T03:52:15.883 DEBUG:teuthology.parallel:result is None 2024-09-21T03:52:16.965 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:52:16.965 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T03:52:16.966 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T03:52:16.966 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T03:52:16.966 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T03:52:16.966 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T03:52:16.966 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T03:52:16.966 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T03:52:16.966 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T03:52:16.966 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T03:52:16.966 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T03:52:16.966 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T03:52:16.966 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T03:52:16.966 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T03:52:16.967 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T03:52:16.967 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T03:52:16.967 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T03:52:16.967 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T03:52:16.967 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T03:52:16.967 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T03:52:16.967 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T03:52:16.967 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T03:52:16.967 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T03:52:16.967 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T03:52:16.967 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T03:52:16.967 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T03:52:16.967 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T03:52:16.967 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T03:52:16.967 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T03:52:16.967 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T03:52:16.968 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T03:52:16.979 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T03:52:16.979 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T03:52:16.979 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T03:52:16.979 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T03:52:16.979 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T03:52:16.979 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T03:52:16.979 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T03:52:16.979 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T03:52:16.979 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T03:52:16.979 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T03:52:16.979 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T03:52:16.979 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T03:52:16.980 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T03:52:16.980 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T03:52:16.980 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T03:52:16.980 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T03:52:16.980 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T03:52:16.980 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T03:52:16.980 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T03:52:16.980 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T03:52:16.980 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T03:52:16.980 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T03:52:16.980 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T03:52:16.980 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T03:52:16.980 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T03:52:16.980 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T03:52:16.980 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T03:52:16.981 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T03:52:16.981 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T03:52:16.981 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T03:52:16.981 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T03:52:16.981 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T03:52:16.981 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T03:52:16.981 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T03:52:16.981 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T03:52:16.981 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T03:52:16.981 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T03:52:16.981 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T03:52:16.981 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T03:52:16.981 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T03:52:16.981 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T03:52:16.981 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T03:52:16.982 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T03:52:16.982 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T03:52:16.982 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T03:52:16.982 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T03:52:16.982 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T03:52:16.982 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T03:52:16.982 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T03:52:16.982 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T03:52:16.982 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T03:52:16.982 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T03:52:16.982 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T03:52:16.982 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T03:52:16.982 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T03:52:16.983 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T03:52:16.983 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T03:52:16.983 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T03:52:16.983 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T03:52:16.983 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T03:52:16.983 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T03:52:16.983 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T03:52:16.983 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T03:52:16.983 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T03:52:16.983 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T03:52:16.983 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T03:52:16.983 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T03:52:16.983 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T03:52:16.983 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T03:52:16.983 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T03:52:16.984 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T03:52:16.984 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T03:52:16.984 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T03:52:16.984 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T03:52:16.984 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T03:52:16.984 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T03:52:16.984 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T03:52:16.984 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T03:52:16.984 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T03:52:16.984 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T03:52:16.984 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T03:52:16.984 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T03:52:16.984 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T03:52:16.984 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T03:52:16.984 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T03:52:16.984 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T03:52:16.985 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T03:52:16.985 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T03:52:16.985 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T03:52:16.985 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T03:52:16.985 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T03:52:16.985 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T03:52:16.985 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T03:52:16.985 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T03:52:16.985 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T03:52:16.985 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T03:52:16.985 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T03:52:16.985 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T03:52:16.985 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T03:52:16.985 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T03:52:16.986 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T03:52:16.986 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T03:52:16.986 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T03:52:16.986 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T03:52:16.986 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T03:52:16.986 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T03:52:16.986 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T03:52:16.986 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T03:52:16.986 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T03:52:16.986 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T03:52:16.986 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T03:52:16.986 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T03:52:16.986 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T03:52:16.987 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T03:52:16.987 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T03:52:16.987 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T03:52:16.987 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T03:52:16.987 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T03:52:16.987 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T03:52:16.987 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T03:52:16.987 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T03:52:16.988 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T03:52:16.988 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T03:52:16.988 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T03:52:16.988 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T03:52:16.988 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T03:52:16.988 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T03:52:16.989 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T03:52:16.989 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T03:52:16.989 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T03:52:16.989 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T03:52:16.989 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T03:52:16.989 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T03:52:16.990 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T03:52:19.755 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T03:52:19.755 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-21T03:52:19.756 INFO:teuthology.orchestra.run.smithi131.stdout:Upgraded: 2024-09-21T03:52:19.756 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T03:52:19.756 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T03:52:19.756 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T03:52:19.756 INFO:teuthology.orchestra.run.smithi131.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T03:52:19.756 INFO:teuthology.orchestra.run.smithi131.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.756 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.756 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T03:52:19.756 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T03:52:19.756 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-21T03:52:19.756 INFO:teuthology.orchestra.run.smithi131.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T03:52:19.756 INFO:teuthology.orchestra.run.smithi131.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T03:52:19.756 INFO:teuthology.orchestra.run.smithi131.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T03:52:19.757 INFO:teuthology.orchestra.run.smithi131.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T03:52:19.757 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.757 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.757 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.757 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.757 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:19.757 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.757 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.757 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.757 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:19.757 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:19.757 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:19.757 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:19.757 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:19.758 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.758 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.758 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:19.758 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.758 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.758 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.758 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:19.758 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:19.758 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.758 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.758 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T03:52:19.758 INFO:teuthology.orchestra.run.smithi131.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T03:52:19.758 INFO:teuthology.orchestra.run.smithi131.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T03:52:19.759 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T03:52:19.759 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T03:52:19.759 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T03:52:19.759 INFO:teuthology.orchestra.run.smithi131.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T03:52:19.759 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T03:52:19.759 INFO:teuthology.orchestra.run.smithi131.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T03:52:19.759 INFO:teuthology.orchestra.run.smithi131.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T03:52:19.759 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T03:52:19.759 INFO:teuthology.orchestra.run.smithi131.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T03:52:19.759 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.759 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.759 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.759 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T03:52:19.760 INFO:teuthology.orchestra.run.smithi131.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T03:52:19.760 INFO:teuthology.orchestra.run.smithi131.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T03:52:19.760 INFO:teuthology.orchestra.run.smithi131.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T03:52:19.760 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T03:52:19.760 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T03:52:19.760 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.760 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.760 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T03:52:19.760 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.760 INFO:teuthology.orchestra.run.smithi131.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T03:52:19.760 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T03:52:19.760 INFO:teuthology.orchestra.run.smithi131.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T03:52:19.760 INFO:teuthology.orchestra.run.smithi131.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T03:52:19.760 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T03:52:19.760 INFO:teuthology.orchestra.run.smithi131.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T03:52:19.761 INFO:teuthology.orchestra.run.smithi131.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T03:52:19.761 INFO:teuthology.orchestra.run.smithi131.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T03:52:19.761 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T03:52:19.761 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T03:52:19.761 INFO:teuthology.orchestra.run.smithi131.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T03:52:19.761 INFO:teuthology.orchestra.run.smithi131.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T03:52:19.761 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T03:52:19.761 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T03:52:19.761 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T03:52:19.761 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T03:52:19.761 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T03:52:19.761 INFO:teuthology.orchestra.run.smithi131.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T03:52:19.761 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T03:52:19.761 INFO:teuthology.orchestra.run.smithi131.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T03:52:19.762 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T03:52:19.762 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T03:52:19.762 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T03:52:19.762 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.762 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.762 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.762 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T03:52:19.762 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T03:52:19.762 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T03:52:19.762 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T03:52:19.762 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T03:52:19.762 INFO:teuthology.orchestra.run.smithi131.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T03:52:19.762 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T03:52:19.762 INFO:teuthology.orchestra.run.smithi131.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T03:52:19.762 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T03:52:19.763 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T03:52:19.763 INFO:teuthology.orchestra.run.smithi131.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T03:52:19.763 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T03:52:19.763 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T03:52:19.763 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T03:52:19.763 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T03:52:19.763 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T03:52:19.763 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T03:52:19.763 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T03:52:19.763 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T03:52:19.763 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T03:52:19.763 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T03:52:19.763 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T03:52:19.763 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T03:52:19.764 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T03:52:19.764 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T03:52:19.764 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T03:52:19.764 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T03:52:19.764 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T03:52:19.764 INFO:teuthology.orchestra.run.smithi131.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T03:52:19.764 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T03:52:19.764 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T03:52:19.764 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T03:52:19.764 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T03:52:19.764 INFO:teuthology.orchestra.run.smithi131.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T03:52:19.764 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T03:52:19.764 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T03:52:19.764 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T03:52:19.764 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T03:52:19.764 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T03:52:19.765 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T03:52:19.765 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.765 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.765 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T03:52:19.765 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T03:52:19.765 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T03:52:19.765 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.765 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T03:52:19.765 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T03:52:19.765 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T03:52:19.765 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T03:52:19.765 INFO:teuthology.orchestra.run.smithi131.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T03:52:19.765 INFO:teuthology.orchestra.run.smithi131.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T03:52:19.765 INFO:teuthology.orchestra.run.smithi131.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T03:52:19.765 INFO:teuthology.orchestra.run.smithi131.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T03:52:19.765 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T03:52:19.766 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T03:52:19.766 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T03:52:19.766 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T03:52:19.766 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T03:52:19.766 INFO:teuthology.orchestra.run.smithi131.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T03:52:19.766 INFO:teuthology.orchestra.run.smithi131.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T03:52:19.766 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.766 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.766 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T03:52:19.767 INFO:teuthology.orchestra.run.smithi131.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T03:52:19.767 INFO:teuthology.orchestra.run.smithi131.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T03:52:19.767 INFO:teuthology.orchestra.run.smithi131.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T03:52:19.767 INFO:teuthology.orchestra.run.smithi131.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T03:52:19.767 INFO:teuthology.orchestra.run.smithi131.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T03:52:19.767 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T03:52:19.768 INFO:teuthology.orchestra.run.smithi131.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T03:52:19.768 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-21T03:52:19.768 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-21T03:52:20.173 DEBUG:teuthology.parallel:result is None 2024-09-21T03:52:20.174 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:52:20.326 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T03:52:20.353 INFO:teuthology.orchestra.run.smithi064.stdout:19.3.0-5120.g18432218.el9 2024-09-21T03:52:20.353 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T03:52:20.354 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T03:52:20.356 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:52:20.500 DEBUG:teuthology.orchestra.run.smithi119:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T03:52:20.528 INFO:teuthology.orchestra.run.smithi119.stdout:19.3.0-5120.g18432218.el9 2024-09-21T03:52:20.528 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T03:52:20.528 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T03:52:20.530 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:52:20.676 DEBUG:teuthology.orchestra.run.smithi131:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T03:52:20.700 INFO:teuthology.orchestra.run.smithi131.stdout:19.3.0-5120.g18432218.el9 2024-09-21T03:52:20.700 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T03:52:20.700 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T03:52:20.702 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T03:52:20.702 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-21T03:52:20.702 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T03:52:20.735 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-21T03:52:20.735 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T03:52:20.767 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-21T03:52:20.768 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T03:52:20.796 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T03:52:20.797 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-21T03:52:20.797 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T03:52:20.826 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T03:52:20.895 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-21T03:52:20.895 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T03:52:20.925 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T03:52:20.996 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-21T03:52:20.996 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T03:52:21.022 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T03:52:21.089 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T03:52:21.089 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-21T03:52:21.089 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T03:52:21.119 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T03:52:21.188 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-21T03:52:21.188 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T03:52:21.213 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T03:52:21.279 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-21T03:52:21.279 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T03:52:21.305 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T03:52:21.370 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T03:52:21.371 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-21T03:52:21.371 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T03:52:21.402 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T03:52:21.473 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-21T03:52:21.473 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T03:52:21.504 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T03:52:21.568 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-21T03:52:21.569 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T03:52:21.595 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T03:52:21.660 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-21T03:52:21.752 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mgr/snap_schedule/allow_m_granularity': True, 'mgr/snap_schedule/dump_on_update': True, 'mgr/snap_schedule/log_level': 'debug', 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mds': {'mds_export_ephemeral_random_max': 0.1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'mgr-modules': ['snap_schedule'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-21T03:52:21.752 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:52:21.752 INFO:tasks.cephadm:Cluster fsid is e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:52:21.753 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-21T03:52:21.753 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.64', 'mon.b': '172.21.15.119', 'mon.c': '172.21.15.131'} 2024-09-21T03:52:21.753 INFO:tasks.cephadm:First mon is mon.a on smithi064 2024-09-21T03:52:21.753 INFO:tasks.cephadm:First mgr is x 2024-09-21T03:52:21.753 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-21T03:52:21.753 DEBUG:teuthology.orchestra.run.smithi064:> sudo hostname $(hostname -s) 2024-09-21T03:52:21.785 DEBUG:teuthology.orchestra.run.smithi119:> sudo hostname $(hostname -s) 2024-09-21T03:52:21.810 DEBUG:teuthology.orchestra.run.smithi131:> sudo hostname $(hostname -s) 2024-09-21T03:52:21.835 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-21T03:52:21.835 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:52:21.981 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.4.75+confusa13', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-21 00:04:17.484514', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['x86_64', 'source', 'arm64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.2.22+braggi22', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-20 22:54:16.499090', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-21T03:52:22.125 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-pdonnell-testing-20240920.212106-debug, sha1 1843221895adccfb3b692060251dd6a0183bc410 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:52:22.126 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T03:52:22.126 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T03:52:22.126 DEBUG:teuthology.orchestra.run.smithi064:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T03:52:22.686 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 03:52 /home/ubuntu/cephtest/cephadm 2024-09-21T03:52:22.686 DEBUG:teuthology.orchestra.run.smithi119:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T03:52:23.282 INFO:teuthology.orchestra.run.smithi119.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 03:52 /home/ubuntu/cephtest/cephadm 2024-09-21T03:52:23.283 DEBUG:teuthology.orchestra.run.smithi131:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T03:52:23.906 INFO:teuthology.orchestra.run.smithi131.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 03:52 /home/ubuntu/cephtest/cephadm 2024-09-21T03:52:23.907 DEBUG:teuthology.orchestra.run.smithi064:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T03:52:23.923 DEBUG:teuthology.orchestra.run.smithi119:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T03:52:23.943 DEBUG:teuthology.orchestra.run.smithi131:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T03:52:23.984 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 on all hosts... 2024-09-21T03:52:23.984 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T03:52:23.986 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T03:52:23.989 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T03:52:25.534 INFO:teuthology.orchestra.run.smithi064.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:52:25.696 INFO:teuthology.orchestra.run.smithi131.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:52:25.706 INFO:teuthology.orchestra.run.smithi119.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:52:58.510 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T03:52:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T03:52:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T03:52:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: "repo_digests": [ 2024-09-21T03:52:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T03:52:58.511 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-21T03:52:58.511 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T03:53:08.509 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-21T03:53:08.509 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T03:53:08.510 INFO:teuthology.orchestra.run.smithi131.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T03:53:08.510 INFO:teuthology.orchestra.run.smithi131.stdout: "repo_digests": [ 2024-09-21T03:53:08.510 INFO:teuthology.orchestra.run.smithi131.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T03:53:08.510 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-09-21T03:53:08.510 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-21T03:53:22.972 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-21T03:53:22.972 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T03:53:22.972 INFO:teuthology.orchestra.run.smithi119.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T03:53:22.972 INFO:teuthology.orchestra.run.smithi119.stdout: "repo_digests": [ 2024-09-21T03:53:22.972 INFO:teuthology.orchestra.run.smithi119.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T03:53:22.972 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-09-21T03:53:22.973 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-21T03:53:22.998 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph 2024-09-21T03:53:23.033 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph 2024-09-21T03:53:23.060 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /etc/ceph 2024-09-21T03:53:23.094 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 777 /etc/ceph 2024-09-21T03:53:23.124 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 777 /etc/ceph 2024-09-21T03:53:23.149 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 777 /etc/ceph 2024-09-21T03:53:23.178 INFO:tasks.cephadm:Writing seed config... 2024-09-21T03:53:23.179 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-21T03:53:23.179 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-21T03:53:23.179 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-21T03:53:23.179 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-21T03:53:23.179 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-21T03:53:23.179 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-21T03:53:23.179 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-21T03:53:23.179 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-21T03:53:23.179 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-21T03:53:23.179 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-21T03:53:23.179 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-21T03:53:23.179 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-21T03:53:23.179 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-21T03:53:23.180 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-21T03:53:23.180 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-21T03:53:23.180 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-21T03:53:23.180 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-21T03:53:23.180 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-21T03:53:23.180 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-21T03:53:23.180 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-21T03:53:23.180 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-21T03:53:23.180 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-21T03:53:23.180 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-21T03:53:23.180 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-21T03:53:23.181 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-21T03:53:23.181 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-21T03:53:23.200 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = e7c927ba-77cc-11ef-baf5-efdc52797490 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_export_ephemeral_random_max = 0.1 2024-09-21T03:53:23.200 DEBUG:teuthology.orchestra.run.smithi064:mon.a> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mon.a.service 2024-09-21T03:53:23.244 DEBUG:teuthology.orchestra.run.smithi064:mgr.x> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mgr.x.service 2024-09-21T03:53:23.287 INFO:tasks.cephadm:Bootstrapping... 2024-09-21T03:53:23.287 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 -v bootstrap --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.64 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:53:23.505 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-09-21T03:53:23.505 INFO:teuthology.orchestra.run.smithi064.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', '-v', 'bootstrap', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.64', '--skip-admin-label'] 2024-09-21T03:53:23.506 INFO:teuthology.orchestra.run.smithi064.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-21T03:53:23.506 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying podman|docker is present... 2024-09-21T03:53:23.530 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stdout 5.2.2 2024-09-21T03:53:23.530 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying lvm2 is present... 2024-09-21T03:53:23.531 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying time synchronization is in place... 2024-09-21T03:53:23.538 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T03:53:23.539 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T03:53:23.545 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T03:53:23.545 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-09-21T03:53:23.552 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout enabled 2024-09-21T03:53:23.558 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout active 2024-09-21T03:53:23.559 INFO:teuthology.orchestra.run.smithi064.stdout:Unit chronyd.service is enabled and running 2024-09-21T03:53:23.559 INFO:teuthology.orchestra.run.smithi064.stdout:Repeating the final host check... 2024-09-21T03:53:23.581 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stdout 5.2.2 2024-09-21T03:53:23.582 INFO:teuthology.orchestra.run.smithi064.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-21T03:53:23.582 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl is present 2024-09-21T03:53:23.582 INFO:teuthology.orchestra.run.smithi064.stdout:lvcreate is present 2024-09-21T03:53:23.589 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T03:53:23.589 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T03:53:23.595 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T03:53:23.595 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-09-21T03:53:23.602 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout enabled 2024-09-21T03:53:23.608 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout active 2024-09-21T03:53:23.609 INFO:teuthology.orchestra.run.smithi064.stdout:Unit chronyd.service is enabled and running 2024-09-21T03:53:23.609 INFO:teuthology.orchestra.run.smithi064.stdout:Host looks OK 2024-09-21T03:53:23.609 INFO:teuthology.orchestra.run.smithi064.stdout:Cluster fsid: e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:53:23.609 INFO:teuthology.orchestra.run.smithi064.stdout:Acquiring lock 140462923140736 on /run/cephadm/e7c927ba-77cc-11ef-baf5-efdc52797490.lock 2024-09-21T03:53:23.609 INFO:teuthology.orchestra.run.smithi064.stdout:Lock 140462923140736 acquired on /run/cephadm/e7c927ba-77cc-11ef-baf5-efdc52797490.lock 2024-09-21T03:53:23.609 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying IP 172.21.15.64 port 3300 ... 2024-09-21T03:53:23.609 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying IP 172.21.15.64 port 6789 ... 2024-09-21T03:53:23.610 INFO:teuthology.orchestra.run.smithi064.stdout:Base mon IP(s) is [172.21.15.64:3300, 172.21.15.64:6789], mon addrv is [v2:172.21.15.64:3300,v1:172.21.15.64:6789] 2024-09-21T03:53:23.613 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.64 metric 100 2024-09-21T03:53:23.613 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.64 metric 100 2024-09-21T03:53:23.615 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-21T03:53:23.616 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-21T03:53:23.616 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-21T03:53:23.618 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-21T03:53:23.618 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-21T03:53:23.618 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T03:53:23.618 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-21T03:53:23.618 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:feb6:f8df/64 scope link noprefixroute 2024-09-21T03:53:23.618 INFO:teuthology.orchestra.run.smithi064.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T03:53:23.619 INFO:teuthology.orchestra.run.smithi064.stdout:Mon IP `172.21.15.64` is in CIDR network `172.21.0.0/20` 2024-09-21T03:53:23.619 INFO:teuthology.orchestra.run.smithi064.stdout:Mon IP `172.21.15.64` is in CIDR network `172.21.0.0/20` 2024-09-21T03:53:23.619 INFO:teuthology.orchestra.run.smithi064.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-21T03:53:23.619 INFO:teuthology.orchestra.run.smithi064.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-21T03:53:23.620 INFO:teuthology.orchestra.run.smithi064.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:53:26.657 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stdout ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T03:53:26.657 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T03:53:26.657 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Getting image source signatures 2024-09-21T03:53:26.657 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Copying blob sha256:8425bc7408d91b180e7fdc62f23e7ae6603757f4f36e9969a4eaf9fd9416b764 2024-09-21T03:53:26.657 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-21T03:53:26.657 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Copying config sha256:ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T03:53:26.657 INFO:teuthology.orchestra.run.smithi064.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-21T03:53:33.552 INFO:teuthology.orchestra.run.smithi064.stdout:ceph: stdout ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:53:33.552 INFO:teuthology.orchestra.run.smithi064.stdout:Ceph version: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T03:53:33.553 INFO:teuthology.orchestra.run.smithi064.stdout:Extracting ceph user uid/gid from container image... 2024-09-21T03:53:34.783 INFO:teuthology.orchestra.run.smithi064.stdout:stat: stdout 167 167 2024-09-21T03:53:34.783 INFO:teuthology.orchestra.run.smithi064.stdout:Creating initial keys... 2024-09-21T03:53:35.731 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-authtool: stdout AQA/Q+5mGE6ICBAAkCBi2QLk8u8PU0eJ6X1Mlg== 2024-09-21T03:53:36.536 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-authtool: stdout AQBAQ+5m+5gnAxAACtq/2kTovmwB1fNgyqguGg== 2024-09-21T03:53:37.392 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph-authtool: stdout AQBAQ+5maNirMhAA1h0ZB8OUXfCH5s6juR6Msw== 2024-09-21T03:53:37.393 INFO:teuthology.orchestra.run.smithi064.stdout:Creating initial monmap... 2024-09-21T03:53:38.240 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T03:53:38.240 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-21T03:53:38.240 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:53:38.241 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T03:53:38.241 INFO:teuthology.orchestra.run.smithi064.stdout:monmaptool for a [v2:172.21.15.64:3300,v1:172.21.15.64:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T03:53:38.241 INFO:teuthology.orchestra.run.smithi064.stdout:setting min_mon_release = quincy 2024-09-21T03:53:38.241 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: set fsid to e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:53:38.241 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T03:53:38.241 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:53:38.241 INFO:teuthology.orchestra.run.smithi064.stdout:Creating mon... 2024-09-21T03:53:39.705 INFO:teuthology.orchestra.run.smithi064.stdout:create mon.a on 2024-09-21T03:53:39.923 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-21T03:53:40.069 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-21T03:53:40.323 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-e7c927ba-77cc-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-e7c927ba-77cc-11ef-baf5-efdc52797490.target. 2024-09-21T03:53:40.323 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-e7c927ba-77cc-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-e7c927ba-77cc-11ef-baf5-efdc52797490.target. 2024-09-21T03:53:40.546 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mon.a 2024-09-21T03:53:40.546 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mon.a.service: Unit ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mon.a.service not loaded. 2024-09-21T03:53:40.718 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e7c927ba-77cc-11ef-baf5-efdc52797490.target.wants/ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mon.a.service → /etc/systemd/system/ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@.service. 2024-09-21T03:53:41.219 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:41 smithi064 podman[30795]: 2024-09-21 03:53:41.166419374 +0000 UTC m=+0.326035588 container start f7ddcc02c88a4cfddfd2385fed0f7528eb9a492a9a4f2167cb39c2c788806947 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:53:41.271 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:53:41.271 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2024-09-21T03:53:41.279 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:53:41.279 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-09-21T03:53:41.279 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2024-09-21T03:53:41.279 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T03:53:41.280 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mon to start... 2024-09-21T03:53:41.280 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mon... 2024-09-21T03:53:41.485 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:41 smithi064 ceph-mon[30814]: mkfs e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:53:41.485 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:41 smithi064 ceph-mon[30814]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T03:53:41.485 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:41 smithi064 bash[30795]: f7ddcc02c88a4cfddfd2385fed0f7528eb9a492a9a4f2167cb39c2c788806947 2024-09-21T03:53:41.486 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:41 smithi064 systemd[1]: Started Ceph mon.a for e7c927ba-77cc-11ef-baf5-efdc52797490. 2024-09-21T03:53:42.338 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:42 smithi064 ceph-mon[30814]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T03:53:42.338 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:42 smithi064 ceph-mon[30814]: monmap epoch 1 2024-09-21T03:53:42.338 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:42 smithi064 ceph-mon[30814]: fsid e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:53:42.338 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:42 smithi064 ceph-mon[30814]: last_changed 2024-09-21T03:53:37.752699+0000 2024-09-21T03:53:42.338 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:42 smithi064 ceph-mon[30814]: created 2024-09-21T03:53:37.752699+0000 2024-09-21T03:53:42.338 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:42 smithi064 ceph-mon[30814]: min_mon_release 19 (squid) 2024-09-21T03:53:42.338 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:42 smithi064 ceph-mon[30814]: election_strategy: 1 2024-09-21T03:53:42.338 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:42 smithi064 ceph-mon[30814]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-21T03:53:42.338 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:42 smithi064 ceph-mon[30814]: fsmap 2024-09-21T03:53:42.339 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:42 smithi064 ceph-mon[30814]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T03:53:42.339 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:42 smithi064 ceph-mon[30814]: mgrmap e1: no daemons active 2024-09-21T03:53:42.339 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:42 smithi064 ceph-mon[30814]: from='client.? 172.21.15.64:0/3804766905' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-21T03:53:42.340 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout cluster: 2024-09-21T03:53:42.342 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout id: e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:53:42.342 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-21T03:53:42.342 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-21T03:53:42.342 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout services: 2024-09-21T03:53:42.343 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.539041s) 2024-09-21T03:53:42.343 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-21T03:53:42.343 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-21T03:53:42.343 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-21T03:53:42.343 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout data: 2024-09-21T03:53:42.343 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-21T03:53:42.343 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-21T03:53:42.343 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-21T03:53:42.343 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout pgs: 2024-09-21T03:53:42.343 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-21T03:53:42.343 INFO:teuthology.orchestra.run.smithi064.stdout:mon is available 2024-09-21T03:53:42.343 INFO:teuthology.orchestra.run.smithi064.stdout:Assimilating anything we can from ceph.conf... 2024-09-21T03:53:43.306 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-21T03:53:43.307 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T03:53:43.307 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout fsid = e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:53:43.307 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T03:53:43.307 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.64:3300,v1:172.21.15.64:6789] 2024-09-21T03:53:43.307 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T03:53:43.307 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T03:53:43.307 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T03:53:43.307 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T03:53:43.307 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-21T03:53:43.307 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T03:53:43.307 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T03:53:43.307 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-21T03:53:43.307 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T03:53:43.308 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T03:53:43.308 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T03:53:43.308 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T03:53:43.308 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T03:53:43.308 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T03:53:43.308 INFO:teuthology.orchestra.run.smithi064.stdout:Generating new minimal ceph.conf... 2024-09-21T03:53:43.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:43 smithi064 ceph-mon[30814]: from='client.? 172.21.15.64:0/597637750' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T03:53:43.420 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:43 smithi064 ceph-mon[30814]: from='client.? 172.21.15.64:0/597637750' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-21T03:53:44.304 INFO:teuthology.orchestra.run.smithi064.stdout:Restarting the monitor... 2024-09-21T03:53:44.513 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:44 smithi064 ceph-mon[30814]: from='client.? 172.21.15.64:0/3329194697' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:53:44.513 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:44 smithi064 systemd[1]: Stopping Ceph mon.a for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T03:53:44.988 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:44 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-a[30810]: 2024-09-21T03:53:44.585+0000 7f0489257640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:53:44.988 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:44 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-a[30810]: 2024-09-21T03:53:44.585+0000 7f0489257640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-21T03:53:44.988 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:44 smithi064 podman[31056]: 2024-09-21 03:53:44.739513748 +0000 UTC m=+0.272525131 container died f7ddcc02c88a4cfddfd2385fed0f7528eb9a492a9a4f2167cb39c2c788806947 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-a, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-21T03:53:45.246 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 podman[31056]: 2024-09-21 03:53:45.077352724 +0000 UTC m=+0.610364110 container cleanup f7ddcc02c88a4cfddfd2385fed0f7528eb9a492a9a4f2167cb39c2c788806947 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-a, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-21T03:53:45.246 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 bash[31056]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-a 2024-09-21T03:53:45.497 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 podman[31068]: 2024-09-21 03:53:45.244833586 +0000 UTC m=+0.501740083 container remove f7ddcc02c88a4cfddfd2385fed0f7528eb9a492a9a4f2167cb39c2c788806947 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-a, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:53:45.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 systemd[1]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-21T03:53:45.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 systemd[1]: Stopped Ceph mon.a for e7c927ba-77cc-11ef-baf5-efdc52797490. 2024-09-21T03:53:45.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 systemd[1]: Starting Ceph mon.a for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T03:53:45.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 podman[31161]: 2024-09-21 03:53:45.695518869 +0000 UTC m=+0.099866367 container create 4084045b6479125719712086b6b208240aed1f65011099f1ae40e1612fe2e7ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-a, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-21T03:53:45.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 podman[31161]: 2024-09-21 03:53:45.611514 +0000 UTC m=+0.015861464 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:53:45.978 INFO:teuthology.orchestra.run.smithi064.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-21T03:53:46.102 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 podman[31161]: 2024-09-21 03:53:45.88978792 +0000 UTC m=+0.294135385 container init 4084045b6479125719712086b6b208240aed1f65011099f1ae40e1612fe2e7ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-21T03:53:46.102 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 podman[31161]: 2024-09-21 03:53:45.893200328 +0000 UTC m=+0.297547785 container start 4084045b6479125719712086b6b208240aed1f65011099f1ae40e1612fe2e7ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-21T03:53:46.102 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T03:53:46.102 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T03:53:46.102 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: pidfile_write: ignore empty --pid-file 2024-09-21T03:53:46.102 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: load: jerasure load: lrc 2024-09-21T03:53:46.102 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: RocksDB version: 7.9.2 2024-09-21T03:53:46.102 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Git sha 0 2024-09-21T03:53:46.102 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T03:53:46.103 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: DB SUMMARY 2024-09-21T03:53:46.103 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: DB Session ID: EXHY9E41KT0BBNYG59D2 2024-09-21T03:53:46.103 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: CURRENT file: CURRENT 2024-09-21T03:53:46.103 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T03:53:46.103 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-21T03:53:46.103 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-21T03:53:46.103 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 94834 ; 2024-09-21T03:53:46.103 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.error_if_exists: 0 2024-09-21T03:53:46.103 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.create_if_missing: 0 2024-09-21T03:53:46.103 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.paranoid_checks: 1 2024-09-21T03:53:46.103 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T03:53:46.103 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T03:53:46.103 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T03:53:46.104 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.env: 0x559dd4adad60 2024-09-21T03:53:46.104 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T03:53:46.104 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.info_log: 0x559dda6bf360 2024-09-21T03:53:46.104 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T03:53:46.105 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.statistics: (nil) 2024-09-21T03:53:46.105 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.use_fsync: 0 2024-09-21T03:53:46.105 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.max_log_file_size: 0 2024-09-21T03:53:46.105 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T03:53:46.105 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T03:53:46.105 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T03:53:46.105 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T03:53:46.106 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.allow_fallocate: 1 2024-09-21T03:53:46.106 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T03:53:46.106 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T03:53:46.106 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.use_direct_reads: 0 2024-09-21T03:53:46.106 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T03:53:46.106 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T03:53:46.106 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.db_log_dir: 2024-09-21T03:53:46.106 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.wal_dir: 2024-09-21T03:53:46.106 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T03:53:46.106 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T03:53:46.106 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T03:53:46.106 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T03:53:46.106 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T03:53:46.106 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T03:53:46.107 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T03:53:46.107 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T03:53:46.107 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.write_buffer_manager: 0x559dda6c3d60 2024-09-21T03:53:46.107 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T03:53:46.107 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T03:53:46.107 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T03:53:46.107 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T03:53:46.107 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T03:53:46.107 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T03:53:46.107 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T03:53:46.107 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T03:53:46.107 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.unordered_write: 0 2024-09-21T03:53:46.107 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T03:53:46.108 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T03:53:46.108 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T03:53:46.108 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T03:53:46.108 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.row_cache: None 2024-09-21T03:53:46.108 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.wal_filter: None 2024-09-21T03:53:46.108 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T03:53:46.108 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T03:53:46.108 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.two_write_queues: 0 2024-09-21T03:53:46.108 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T03:53:46.108 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.wal_compression: 0 2024-09-21T03:53:46.108 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.atomic_flush: 0 2024-09-21T03:53:46.108 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T03:53:46.108 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T03:53:46.108 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T03:53:46.108 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.log_readahead_size: 0 2024-09-21T03:53:46.109 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T03:53:46.109 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T03:53:46.109 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T03:53:46.109 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T03:53:46.109 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T03:53:46.109 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T03:53:46.109 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T03:53:46.109 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.max_background_jobs: 2 2024-09-21T03:53:46.109 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.max_background_compactions: -1 2024-09-21T03:53:46.109 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.max_subcompactions: 1 2024-09-21T03:53:46.109 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T03:53:46.109 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T03:53:46.109 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T03:53:46.109 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T03:53:46.109 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T03:53:46.110 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T03:53:46.110 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T03:53:46.110 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T03:53:46.110 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.max_open_files: -1 2024-09-21T03:53:46.110 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T03:53:46.110 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T03:53:46.110 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T03:53:46.110 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T03:53:46.110 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.max_background_flushes: -1 2024-09-21T03:53:46.110 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Compression algorithms supported: 2024-09-21T03:53:46.110 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: kZSTD supported: 0 2024-09-21T03:53:46.110 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: kXpressCompression supported: 0 2024-09-21T03:53:46.110 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: kBZip2Compression supported: 0 2024-09-21T03:53:46.110 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T03:53:46.110 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: kLZ4Compression supported: 1 2024-09-21T03:53:46.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: kZlibCompression supported: 1 2024-09-21T03:53:46.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T03:53:46.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: kSnappyCompression supported: 1 2024-09-21T03:53:46.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T03:53:46.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T03:53:46.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-21T03:53:46.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T03:53:46.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T03:53:46.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.merge_operator: 2024-09-21T03:53:46.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.compaction_filter: None 2024-09-21T03:53:46.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T03:53:46.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T03:53:46.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T03:53:46.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T03:53:46.112 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559dda6befa0) 2024-09-21T03:53:46.112 INFO:journalctl@ceph.mon.a.smithi064.stdout: cache_index_and_filter_blocks: 1 2024-09-21T03:53:46.112 INFO:journalctl@ceph.mon.a.smithi064.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T03:53:46.112 INFO:journalctl@ceph.mon.a.smithi064.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T03:53:46.112 INFO:journalctl@ceph.mon.a.smithi064.stdout: pin_top_level_index_and_filter: 1 2024-09-21T03:53:46.112 INFO:journalctl@ceph.mon.a.smithi064.stdout: index_type: 0 2024-09-21T03:53:46.112 INFO:journalctl@ceph.mon.a.smithi064.stdout: data_block_index_type: 0 2024-09-21T03:53:46.112 INFO:journalctl@ceph.mon.a.smithi064.stdout: index_shortening: 1 2024-09-21T03:53:46.112 INFO:journalctl@ceph.mon.a.smithi064.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T03:53:46.112 INFO:journalctl@ceph.mon.a.smithi064.stdout: checksum: 4 2024-09-21T03:53:46.112 INFO:journalctl@ceph.mon.a.smithi064.stdout: no_block_cache: 0 2024-09-21T03:53:46.112 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_cache: 0x559dd6683610 2024-09-21T03:53:46.112 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_cache_name: BinnedLRUCache 2024-09-21T03:53:46.112 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_cache_options: 2024-09-21T03:53:46.112 INFO:journalctl@ceph.mon.a.smithi064.stdout: capacity : 536870912 2024-09-21T03:53:46.112 INFO:journalctl@ceph.mon.a.smithi064.stdout: num_shard_bits : 4 2024-09-21T03:53:46.113 INFO:journalctl@ceph.mon.a.smithi064.stdout: strict_capacity_limit : 0 2024-09-21T03:53:46.113 INFO:journalctl@ceph.mon.a.smithi064.stdout: high_pri_pool_ratio: 0.000 2024-09-21T03:53:46.113 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_cache_compressed: (nil) 2024-09-21T03:53:46.113 INFO:journalctl@ceph.mon.a.smithi064.stdout: persistent_cache: (nil) 2024-09-21T03:53:46.113 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_size: 4096 2024-09-21T03:53:46.113 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_size_deviation: 10 2024-09-21T03:53:46.113 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_restart_interval: 16 2024-09-21T03:53:46.113 INFO:journalctl@ceph.mon.a.smithi064.stdout: index_block_restart_interval: 1 2024-09-21T03:53:46.113 INFO:journalctl@ceph.mon.a.smithi064.stdout: metadata_block_size: 4096 2024-09-21T03:53:46.113 INFO:journalctl@ceph.mon.a.smithi064.stdout: partition_filters: 0 2024-09-21T03:53:46.113 INFO:journalctl@ceph.mon.a.smithi064.stdout: use_delta_encoding: 1 2024-09-21T03:53:46.113 INFO:journalctl@ceph.mon.a.smithi064.stdout: filter_policy: bloomfilter 2024-09-21T03:53:46.113 INFO:journalctl@ceph.mon.a.smithi064.stdout: whole_key_filtering: 1 2024-09-21T03:53:46.113 INFO:journalctl@ceph.mon.a.smithi064.stdout: verify_compression: 0 2024-09-21T03:53:46.113 INFO:journalctl@ceph.mon.a.smithi064.stdout: read_amp_bytes_per_bit: 0 2024-09-21T03:53:46.113 INFO:journalctl@ceph.mon.a.smithi064.stdout: format_version: 5 2024-09-21T03:53:46.114 INFO:journalctl@ceph.mon.a.smithi064.stdout: enable_index_compression: 1 2024-09-21T03:53:46.114 INFO:journalctl@ceph.mon.a.smithi064.stdout: block_align: 0 2024-09-21T03:53:46.114 INFO:journalctl@ceph.mon.a.smithi064.stdout: max_auto_readahead_size: 262144 2024-09-21T03:53:46.114 INFO:journalctl@ceph.mon.a.smithi064.stdout: prepopulate_block_cache: 0 2024-09-21T03:53:46.114 INFO:journalctl@ceph.mon.a.smithi064.stdout: initial_auto_readahead_size: 8192 2024-09-21T03:53:46.114 INFO:journalctl@ceph.mon.a.smithi064.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T03:53:46.114 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T03:53:46.114 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T03:53:46.114 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.compression: NoCompression 2024-09-21T03:53:46.114 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T03:53:46.114 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T03:53:46.114 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T03:53:46.114 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.num_levels: 7 2024-09-21T03:53:46.114 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T03:53:46.114 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T03:53:46.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T03:53:46.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T03:53:46.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T03:53:46.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T03:53:46.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T03:53:46.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:53:46.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T03:53:46.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T03:53:46.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:53:46.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:53:46.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T03:53:46.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T03:53:46.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T03:53:46.115 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T03:53:46.116 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:53:46.116 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:53:46.116 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T03:53:46.116 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T03:53:46.116 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:53:46.116 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T03:53:46.116 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T03:53:46.116 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T03:53:46.116 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T03:53:46.116 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T03:53:46.116 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T03:53:46.116 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T03:53:46.116 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T03:53:46.116 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T03:53:46.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T03:53:46.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T03:53:46.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T03:53:46.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T03:53:46.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T03:53:46.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T03:53:46.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T03:53:46.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T03:53:46.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T03:53:46.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T03:53:46.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T03:53:46.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T03:53:46.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T03:53:46.117 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T03:53:46.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T03:53:46.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T03:53:46.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T03:53:46.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T03:53:46.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T03:53:46.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T03:53:46.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T03:53:46.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T03:53:46.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T03:53:46.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T03:53:46.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.inplace_update_support: 0 2024-09-21T03:53:46.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T03:53:46.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T03:53:46.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T03:53:46.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T03:53:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.bloom_locality: 0 2024-09-21T03:53:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.max_successive_merges: 0 2024-09-21T03:53:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T03:53:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T03:53:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T03:53:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T03:53:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.ttl: 2592000 2024-09-21T03:53:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T03:53:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T03:53:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T03:53:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.enable_blob_files: false 2024-09-21T03:53:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.min_blob_size: 0 2024-09-21T03:53:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T03:53:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T03:53:46.119 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T03:53:46.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T03:53:46.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T03:53:46.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T03:53:46.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T03:53:46.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T03:53:46.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-21T03:53:46.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-21T03:53:46.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 868eca93-a380-4068-8492-9027d6c02858 2024-09-21T03:53:46.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726890825929714, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-21T03:53:46.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-21T03:53:46.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726890825930913, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 91047, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 283, "table_properties": {"data_size": 89095, "index_size": 272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11605, "raw_average_key_size": 47, "raw_value_size": 82526, "raw_average_value_size": 341, "num_data_blocks": 12, "num_entries": 242, "num_filter_entries": 242, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726890825, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "868eca93-a380-4068-8492-9027d6c02858", "db_session_id": "EXHY9E41KT0BBNYG59D2", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-21T03:53:46.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726890825931012, "job": 1, "event": "recovery_finished"} 2024-09-21T03:53:46.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-21T03:53:46.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T03:53:46.120 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559dda6e2a80 2024-09-21T03:53:46.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: DB pointer 0x559ddb488000 2024-09-21T03:53:46.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T03:53:46.121 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T03:53:46.121 INFO:journalctl@ceph.mon.a.smithi064.stdout: ** DB Stats ** 2024-09-21T03:53:46.121 INFO:journalctl@ceph.mon.a.smithi064.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:53:46.121 INFO:journalctl@ceph.mon.a.smithi064.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T03:53:46.121 INFO:journalctl@ceph.mon.a.smithi064.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:53:46.121 INFO:journalctl@ceph.mon.a.smithi064.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:53:46.121 INFO:journalctl@ceph.mon.a.smithi064.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T03:53:46.121 INFO:journalctl@ceph.mon.a.smithi064.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:53:46.121 INFO:journalctl@ceph.mon.a.smithi064.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:53:46.121 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-09-21T03:53:46.121 INFO:journalctl@ceph.mon.a.smithi064.stdout: ** Compaction Stats [default] ** 2024-09-21T03:53:46.121 INFO:journalctl@ceph.mon.a.smithi064.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:53:46.122 INFO:journalctl@ceph.mon.a.smithi064.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T03:53:46.122 INFO:journalctl@ceph.mon.a.smithi064.stdout: L0 2/0 90.77 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 120.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:53:46.122 INFO:journalctl@ceph.mon.a.smithi064.stdout: Sum 2/0 90.77 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 120.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:53:46.122 INFO:journalctl@ceph.mon.a.smithi064.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 120.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:53:46.122 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-09-21T03:53:46.122 INFO:journalctl@ceph.mon.a.smithi064.stdout: ** Compaction Stats [default] ** 2024-09-21T03:53:46.122 INFO:journalctl@ceph.mon.a.smithi064.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:53:46.122 INFO:journalctl@ceph.mon.a.smithi064.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T03:53:46.122 INFO:journalctl@ceph.mon.a.smithi064.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 120.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T03:53:46.122 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-09-21T03:53:46.122 INFO:journalctl@ceph.mon.a.smithi064.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T03:53:46.122 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-09-21T03:53:46.122 INFO:journalctl@ceph.mon.a.smithi064.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:53:46.122 INFO:journalctl@ceph.mon.a.smithi064.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T03:53:46.122 INFO:journalctl@ceph.mon.a.smithi064.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T03:53:46.123 INFO:journalctl@ceph.mon.a.smithi064.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T03:53:46.123 INFO:journalctl@ceph.mon.a.smithi064.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T03:53:46.123 INFO:journalctl@ceph.mon.a.smithi064.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T03:53:46.123 INFO:journalctl@ceph.mon.a.smithi064.stdout: Cumulative compaction: 0.00 GB write, 24.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:53:46.123 INFO:journalctl@ceph.mon.a.smithi064.stdout: Interval compaction: 0.00 GB write, 24.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:53:46.123 INFO:journalctl@ceph.mon.a.smithi064.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T03:53:46.123 INFO:journalctl@ceph.mon.a.smithi064.stdout: Block cache BinnedLRUCache@0x559dd6683610#2 capacity: 512.00 MB usage: 1.31 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-21T03:53:46.123 INFO:journalctl@ceph.mon.a.smithi064.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.84 KB,0.000160933%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-21T03:53:46.123 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-09-21T03:53:46.123 INFO:journalctl@ceph.mon.a.smithi064.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T03:53:46.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: starting mon.a rank 0 at public addrs [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] at bind addrs [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:53:46.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: mon.a@-1(???) e1 preinit fsid e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:53:46.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: mon.a@-1(???).mds e1 new map 2024-09-21T03:53:46.124 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: mon.a@-1(???).mds e1 print_map 2024-09-21T03:53:46.124 INFO:journalctl@ceph.mon.a.smithi064.stdout: e1 2024-09-21T03:53:46.124 INFO:journalctl@ceph.mon.a.smithi064.stdout: btime 2024-09-21T03:53:41:214835+0000 2024-09-21T03:53:46.124 INFO:journalctl@ceph.mon.a.smithi064.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T03:53:46.124 INFO:journalctl@ceph.mon.a.smithi064.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:53:46.124 INFO:journalctl@ceph.mon.a.smithi064.stdout: legacy client fscid: -1 2024-09-21T03:53:46.124 INFO:journalctl@ceph.mon.a.smithi064.stdout: 2024-09-21T03:53:46.124 INFO:journalctl@ceph.mon.a.smithi064.stdout: No filesystems configured 2024-09-21T03:53:46.124 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T03:53:46.124 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:53:46.124 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:53:46.124 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:53:46.124 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-21T03:53:46.124 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 bash[31161]: 4084045b6479125719712086b6b208240aed1f65011099f1ae40e1612fe2e7ce 2024-09-21T03:53:46.124 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 systemd[1]: Started Ceph mon.a for e7c927ba-77cc-11ef-baf5-efdc52797490. 2024-09-21T03:53:46.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T03:53:46.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: monmap epoch 1 2024-09-21T03:53:46.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: fsid e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:53:46.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: last_changed 2024-09-21T03:53:37.752699+0000 2024-09-21T03:53:46.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: created 2024-09-21T03:53:37.752699+0000 2024-09-21T03:53:46.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: min_mon_release 19 (squid) 2024-09-21T03:53:46.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: election_strategy: 1 2024-09-21T03:53:46.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-21T03:53:46.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: fsmap 2024-09-21T03:53:46.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T03:53:46.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:45 smithi064 ceph-mon[31180]: mgrmap e1: no daemons active 2024-09-21T03:53:47.110 INFO:teuthology.orchestra.run.smithi064.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-21T03:53:47.131 INFO:teuthology.orchestra.run.smithi064.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:53:47.131 INFO:teuthology.orchestra.run.smithi064.stdout:Creating mgr... 2024-09-21T03:53:47.131 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-21T03:53:47.132 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-21T03:53:47.390 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mgr.x 2024-09-21T03:53:47.390 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mgr.x.service: Unit ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mgr.x.service not loaded. 2024-09-21T03:53:47.551 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e7c927ba-77cc-11ef-baf5-efdc52797490.target.wants/ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mgr.x.service → /etc/systemd/system/ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@.service. 2024-09-21T03:53:47.865 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:47 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1617893512' entity='client.admin' 2024-09-21T03:53:48.050 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:53:48.050 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2024-09-21T03:53:48.057 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:53:48.057 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-09-21T03:53:48.058 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2024-09-21T03:53:48.058 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T03:53:48.067 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:53:48.067 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2024-09-21T03:53:48.074 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:53:48.075 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-09-21T03:53:48.075 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2024-09-21T03:53:48.075 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-21T03:53:48.075 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr to start... 2024-09-21T03:53:48.075 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr... 2024-09-21T03:53:48.203 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:53:48 smithi064 systemd[1]: Started Ceph mgr.x for e7c927ba-77cc-11ef-baf5-efdc52797490. 2024-09-21T03:53:48.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:48 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1255349536' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T03:53:49.155 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-21T03:53:49.155 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:53:49.155 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "e7c927ba-77cc-11ef-baf5-efdc52797490", 2024-09-21T03:53:49.156 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T03:53:49.156 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T03:53:49.156 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T03:53:49.156 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T03:53:49.156 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:53:49.156 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T03:53:49.156 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T03:53:49.156 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2024-09-21T03:53:49.156 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:53:49.156 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T03:53:49.156 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T03:53:49.156 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:53:49.156 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-21T03:53:49.157 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T03:53:49.157 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:53:49.157 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T03:53:49.157 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T03:53:49.157 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:53:49.157 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T03:53:49.157 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:53:49.157 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T03:53:49.157 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T03:53:49.157 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T03:53:49.157 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T03:53:49.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T03:53:49.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T03:53:49.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:53:49.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T03:53:49.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T03:53:49.161 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T03:53:49.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T03:53:49.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T03:53:49.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T03:53:49.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T03:53:49.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T03:53:49.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T03:53:49.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:53:49.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T03:53:49.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:53:49.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T03:53:41:214835+0000", 2024-09-21T03:53:49.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T03:53:49.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T03:53:49.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:53:49.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T03:53:49.162 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T03:53:49.163 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T03:53:49.163 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T03:53:49.163 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T03:53:49.163 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T03:53:49.163 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T03:53:49.163 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:53:49.163 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:53:49.163 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:53:49.163 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T03:53:49.163 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:53:49.164 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T03:53:41.215665+0000", 2024-09-21T03:53:49.164 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:53:49.164 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:53:49.164 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T03:53:49.164 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:53:49.164 INFO:teuthology.orchestra.run.smithi064.stdout:mgr not available, waiting (1/15)... 2024-09-21T03:53:52.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:51 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2649857532' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T03:53:52.204 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-21T03:53:52.204 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:53:52.204 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "e7c927ba-77cc-11ef-baf5-efdc52797490", 2024-09-21T03:53:52.204 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T03:53:52.204 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T03:53:52.205 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T03:53:52.205 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T03:53:52.205 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:53:52.205 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T03:53:52.205 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T03:53:52.205 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2024-09-21T03:53:52.205 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:53:52.205 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T03:53:52.205 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T03:53:52.205 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:53:52.205 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-21T03:53:52.205 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T03:53:52.205 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:53:52.206 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T03:53:52.206 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T03:53:52.206 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:53:52.206 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T03:53:52.206 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:53:52.206 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T03:53:52.206 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T03:53:52.206 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T03:53:52.210 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T03:53:52.210 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T03:53:52.210 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T03:53:52.210 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:53:52.210 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T03:53:52.210 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T03:53:52.210 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T03:53:52.211 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T03:53:52.211 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T03:53:52.211 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T03:53:52.211 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T03:53:52.211 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T03:53:52.211 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T03:53:52.211 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:53:52.211 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T03:53:52.211 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:53:52.211 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T03:53:41:214835+0000", 2024-09-21T03:53:52.211 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T03:53:52.211 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T03:53:52.211 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:53:52.212 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T03:53:52.212 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T03:53:52.212 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T03:53:52.212 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T03:53:52.212 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T03:53:52.212 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T03:53:52.212 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T03:53:52.212 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:53:52.212 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:53:52.212 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:53:52.212 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T03:53:52.212 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:53:52.212 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T03:53:41.215665+0000", 2024-09-21T03:53:52.213 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:53:52.213 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:53:52.213 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T03:53:52.213 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:53:52.213 INFO:teuthology.orchestra.run.smithi064.stdout:mgr not available, waiting (2/15)... 2024-09-21T03:53:53.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:53 smithi064 ceph-mon[31180]: Activating manager daemon x 2024-09-21T03:53:53.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:53 smithi064 ceph-mon[31180]: mgrmap e2: x(active, starting, since 0.00393773s) 2024-09-21T03:53:53.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:53 smithi064 ceph-mon[31180]: from='mgr.14100 172.21.15.64:0/284288875' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:53:53.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:53 smithi064 ceph-mon[31180]: from='mgr.14100 172.21.15.64:0/284288875' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:53:53.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:53 smithi064 ceph-mon[31180]: from='mgr.14100 172.21.15.64:0/284288875' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:53:53.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:53 smithi064 ceph-mon[31180]: from='mgr.14100 172.21.15.64:0/284288875' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:53:53.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:53 smithi064 ceph-mon[31180]: from='mgr.14100 172.21.15.64:0/284288875' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:53:53.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:53 smithi064 ceph-mon[31180]: Manager daemon x is now available 2024-09-21T03:53:53.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:53 smithi064 ceph-mon[31180]: from='mgr.14100 172.21.15.64:0/284288875' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:53:53.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:53 smithi064 ceph-mon[31180]: from='mgr.14100 172.21.15.64:0/284288875' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:53:53.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:53 smithi064 ceph-mon[31180]: from='mgr.14100 172.21.15.64:0/284288875' entity='mgr.x' 2024-09-21T03:53:53.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:53 smithi064 ceph-mon[31180]: from='mgr.14100 172.21.15.64:0/284288875' entity='mgr.x' 2024-09-21T03:53:53.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:53 smithi064 ceph-mon[31180]: from='mgr.14100 172.21.15.64:0/284288875' entity='mgr.x' 2024-09-21T03:53:55.428 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-21T03:53:55.429 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:53:55.429 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsid": "e7c927ba-77cc-11ef-baf5-efdc52797490", 2024-09-21T03:53:55.429 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T03:53:55.429 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T03:53:55.429 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T03:53:55.429 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T03:53:55.429 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:53:55.429 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T03:53:55.429 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T03:53:55.429 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 0 2024-09-21T03:53:55.429 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:53:55.429 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T03:53:55.429 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T03:53:55.429 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:53:55.430 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-21T03:53:55.430 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T03:53:55.430 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:53:55.430 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T03:53:55.430 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T03:53:55.430 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:53:55.430 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T03:53:55.430 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:53:55.430 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T03:53:55.430 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T03:53:55.430 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T03:53:55.430 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T03:53:55.430 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T03:53:55.430 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T03:53:55.431 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:53:55.431 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T03:53:55.431 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T03:53:55.431 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T03:53:55.431 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T03:53:55.431 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T03:53:55.431 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T03:53:55.431 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T03:53:55.431 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T03:53:55.431 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T03:53:55.431 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:53:55.431 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T03:53:55.431 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:53:55.431 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T03:53:41:214835+0000", 2024-09-21T03:53:55.431 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T03:53:55.431 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T03:53:55.431 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:53:55.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T03:53:55.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T03:53:55.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T03:53:55.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T03:53:55.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T03:53:55.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T03:53:55.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T03:53:55.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ], 2024-09-21T03:53:55.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:53:55.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:53:55.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T03:53:55.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T03:53:55.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T03:53:41.215665+0000", 2024-09-21T03:53:55.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T03:53:55.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout }, 2024-09-21T03:53:55.432 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T03:53:55.433 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:53:55.433 INFO:teuthology.orchestra.run.smithi064.stdout:mgr is available 2024-09-21T03:53:55.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:55 smithi064 ceph-mon[31180]: mgrmap e3: x(active, since 1.00904s) 2024-09-21T03:53:55.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:55 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/181217352' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T03:53:56.477 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:56 smithi064 ceph-mon[31180]: mgrmap e4: x(active, since 2s) 2024-09-21T03:53:56.477 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:56 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1700539903' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T03:53:56.482 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-21T03:53:56.482 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T03:53:56.482 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout fsid = e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:53:56.482 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T03:53:56.482 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.64:3300,v1:172.21.15.64:6789] 2024-09-21T03:53:56.482 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T03:53:56.482 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T03:53:56.482 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T03:53:56.482 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T03:53:56.482 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-21T03:53:56.482 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T03:53:56.482 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T03:53:56.483 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 2024-09-21T03:53:56.483 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T03:53:56.483 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T03:53:56.483 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T03:53:56.483 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T03:53:56.483 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T03:53:56.483 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T03:53:56.483 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling cephadm module... 2024-09-21T03:53:57.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:57 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2334075485' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-21T03:53:57.511 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:53:57 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: ignoring --setuser ceph since I am not root 2024-09-21T03:53:57.511 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:53:57 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: ignoring --setgroup ceph since I am not root 2024-09-21T03:53:58.473 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:58 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2334075485' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-21T03:53:58.473 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:58 smithi064 ceph-mon[31180]: mgrmap e5: x(active, since 4s) 2024-09-21T03:53:59.035 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:53:59.036 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-21T03:53:59.036 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T03:53:59.036 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T03:53:59.036 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T03:53:59.036 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:53:59.036 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for the mgr to restart... 2024-09-21T03:53:59.036 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr epoch 5... 2024-09-21T03:53:59.337 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:53:59 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/25131090' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T03:54:03.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:02 smithi064 ceph-mon[31180]: Active manager daemon x restarted 2024-09-21T03:54:03.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:02 smithi064 ceph-mon[31180]: Activating manager daemon x 2024-09-21T03:54:03.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:02 smithi064 ceph-mon[31180]: osdmap e2: 0 total, 0 up, 0 in 2024-09-21T03:54:03.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:02 smithi064 ceph-mon[31180]: mgrmap e6: x(active, starting, since 0.00406134s) 2024-09-21T03:54:03.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:02 smithi064 ceph-mon[31180]: from='mgr.14118 172.21.15.64:0/2225958686' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:54:03.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:02 smithi064 ceph-mon[31180]: from='mgr.14118 172.21.15.64:0/2225958686' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:54:03.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:02 smithi064 ceph-mon[31180]: from='mgr.14118 172.21.15.64:0/2225958686' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:54:03.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:02 smithi064 ceph-mon[31180]: from='mgr.14118 172.21.15.64:0/2225958686' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:54:03.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:02 smithi064 ceph-mon[31180]: from='mgr.14118 172.21.15.64:0/2225958686' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:54:03.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:02 smithi064 ceph-mon[31180]: Manager daemon x is now available 2024-09-21T03:54:04.116 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:54:04.116 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-21T03:54:04.116 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T03:54:04.116 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:54:04.116 INFO:teuthology.orchestra.run.smithi064.stdout:mgr epoch 5 is available 2024-09-21T03:54:04.117 INFO:teuthology.orchestra.run.smithi064.stdout:Verifying orchestrator module is enabled... 2024-09-21T03:54:04.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:04 smithi064 ceph-mon[31180]: mgrmap e7: x(active, since 1.0058s) 2024-09-21T03:54:04.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:04 smithi064 ceph-mon[31180]: from='mgr.14118 172.21.15.64:0/2225958686' entity='mgr.x' 2024-09-21T03:54:04.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:04 smithi064 ceph-mon[31180]: from='mgr.14118 172.21.15.64:0/2225958686' entity='mgr.x' 2024-09-21T03:54:04.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:04 smithi064 ceph-mon[31180]: Found migration_current of "None". Setting to last migration. 2024-09-21T03:54:04.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:04 smithi064 ceph-mon[31180]: from='mgr.14118 172.21.15.64:0/2225958686' entity='mgr.x' 2024-09-21T03:54:04.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:04 smithi064 ceph-mon[31180]: from='mgr.14118 172.21.15.64:0/2225958686' entity='mgr.x' 2024-09-21T03:54:04.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:04 smithi064 ceph-mon[31180]: from='mgr.14118 172.21.15.64:0/2225958686' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:54:04.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:04 smithi064 ceph-mon[31180]: from='mgr.14118 172.21.15.64:0/2225958686' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:54:04.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:04 smithi064 ceph-mon[31180]: from='mgr.14118 172.21.15.64:0/2225958686' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:54:04.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:04 smithi064 ceph-mon[31180]: from='mgr.14118 172.21.15.64:0/2225958686' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:54:05.470 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: 2024-09-21T03:54:05.379+0000 7fb2b68d6640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:03:54:05] ENGINE Error in HTTPServer.serve 2024-09-21T03:54:05.470 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: Traceback (most recent call last): 2024-09-21T03:54:05.470 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:54:05.470 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: self._connections.run(self.expiration_interval) 2024-09-21T03:54:05.470 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:54:05.471 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: self._run(expiration_interval) 2024-09-21T03:54:05.471 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:54:05.471 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:54:05.471 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:54:05.471 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:54:05.471 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:54:05.471 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: s = self.context.wrap_socket( 2024-09-21T03:54:05.471 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:54:05.471 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: return self.sslsocket_class._create( 2024-09-21T03:54:05.471 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:54:05.471 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: self.do_handshake() 2024-09-21T03:54:05.472 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:54:05.472 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: self._sslobj.do_handshake() 2024-09-21T03:54:05.472 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:54:05.472 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: 2024-09-21T03:54:05.472 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2024-09-21T03:54:05.472 INFO:teuthology.orchestra.run.smithi064.stdout:Setting orchestrator backend to cephadm... 2024-09-21T03:54:05.756 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:05 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/4051796340' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2024-09-21T03:54:05.756 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:05 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/4051796340' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2024-09-21T03:54:05.756 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:05 smithi064 ceph-mon[31180]: mgrmap e8: x(active, since 2s) 2024-09-21T03:54:05.756 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:05 smithi064 ceph-mon[31180]: [21/Sep/2024:03:54:05] ENGINE Bus STARTING 2024-09-21T03:54:05.756 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:05 smithi064 ceph-mon[31180]: from='mgr.14118 172.21.15.64:0/2225958686' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:54:07.230 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:07 smithi064 ceph-mon[31180]: [21/Sep/2024:03:54:05] ENGINE Serving on https://172.21.15.64:7150 2024-09-21T03:54:07.230 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:07 smithi064 ceph-mon[31180]: [21/Sep/2024:03:54:05] ENGINE Error in HTTPServer.serve 2024-09-21T03:54:07.231 INFO:journalctl@ceph.mon.a.smithi064.stdout: Traceback (most recent call last): 2024-09-21T03:54:07.231 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:54:07.231 INFO:journalctl@ceph.mon.a.smithi064.stdout: self._connections.run(self.expiration_interval) 2024-09-21T03:54:07.231 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:54:07.231 INFO:journalctl@ceph.mon.a.smithi064.stdout: self._run(expiration_interval) 2024-09-21T03:54:07.231 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:54:07.231 INFO:journalctl@ceph.mon.a.smithi064.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:54:07.231 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:54:07.231 INFO:journalctl@ceph.mon.a.smithi064.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:54:07.231 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:54:07.231 INFO:journalctl@ceph.mon.a.smithi064.stdout: s = self.context.wrap_socket( 2024-09-21T03:54:07.231 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:54:07.231 INFO:journalctl@ceph.mon.a.smithi064.stdout: return self.sslsocket_class._create( 2024-09-21T03:54:07.231 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:54:07.231 INFO:journalctl@ceph.mon.a.smithi064.stdout: self.do_handshake() 2024-09-21T03:54:07.232 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:54:07.232 INFO:journalctl@ceph.mon.a.smithi064.stdout: self._sslobj.do_handshake() 2024-09-21T03:54:07.232 INFO:journalctl@ceph.mon.a.smithi064.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:54:07.232 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:07 smithi064 ceph-mon[31180]: [21/Sep/2024:03:54:05] ENGINE Serving on http://172.21.15.64:8765 2024-09-21T03:54:07.232 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:07 smithi064 ceph-mon[31180]: [21/Sep/2024:03:54:05] ENGINE Bus STARTED 2024-09-21T03:54:07.232 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:07 smithi064 ceph-mon[31180]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:54:07.232 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:07 smithi064 ceph-mon[31180]: from='mgr.14118 172.21.15.64:0/2225958686' entity='mgr.x' 2024-09-21T03:54:07.232 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:07 smithi064 ceph-mon[31180]: from='mgr.14118 172.21.15.64:0/2225958686' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:54:07.751 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-21T03:54:07.751 INFO:teuthology.orchestra.run.smithi064.stdout:Generating ssh key... 2024-09-21T03:54:08.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:08 smithi064 ceph-mon[31180]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:54:09.011 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:08 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: Generating public/private rsa key pair. 2024-09-21T03:54:09.012 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:08 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: Your identification has been saved in /tmp/tmpgner2ntn/key 2024-09-21T03:54:09.012 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:08 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: Your public key has been saved in /tmp/tmpgner2ntn/key.pub 2024-09-21T03:54:09.012 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:08 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: The key fingerprint is: 2024-09-21T03:54:09.012 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:08 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: SHA256:OtSwkbYblOFTVM8v/JWZyR6/F49CN8tkQN7gYlwkfD0 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:54:09.012 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:08 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: The key's randomart image is: 2024-09-21T03:54:09.012 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:08 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: +---[RSA 3072]----+ 2024-09-21T03:54:09.012 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:08 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: | ..ooo... | 2024-09-21T03:54:09.012 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:08 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: | . = .+= E | 2024-09-21T03:54:09.012 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:08 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: | X . *oo . | 2024-09-21T03:54:09.012 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:08 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: | o B +.+.o =| 2024-09-21T03:54:09.012 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:08 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: | = S. .o..B.| 2024-09-21T03:54:09.012 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:08 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: | . + .o*oo| 2024-09-21T03:54:09.013 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:08 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: | + . =.=+| 2024-09-21T03:54:09.013 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:08 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: | . . + +| 2024-09-21T03:54:09.013 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:08 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: | . ..| 2024-09-21T03:54:09.013 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:08 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: +----[SHA256]-----+ 2024-09-21T03:54:09.941 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:09 smithi064 ceph-mon[31180]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:54:09.941 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:09 smithi064 ceph-mon[31180]: Generating ssh key... 2024-09-21T03:54:09.941 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:09 smithi064 ceph-mon[31180]: from='mgr.14118 172.21.15.64:0/2225958686' entity='mgr.x' 2024-09-21T03:54:09.942 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:09 smithi064 ceph-mon[31180]: from='mgr.14118 172.21.15.64:0/2225958686' entity='mgr.x' 2024-09-21T03:54:10.472 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzyWcsWId3jSZQ5ZWpNq6P4TrliwZCol7g057T4W/tzr8bEk6n83DbTvG693+/+H4++4qylZxCU3qLi/0jw5bDIu/CGB964UicJx/cytRGBhvUAL65WEz+0p0adO3Vb/oT/JOXn9lAEvMhqUuu9k3YWseiuML1mBSUU7krIhANW1kfdthmeNjaRFDxZBExgfmP+G+FP2LwuWu5m3CSp2CuEgvJVJ+QVsVBTjdzqpGeydFmMWQ2dXbMT7tSilfXao+cNWa6DxaKGe7otlwymt0QQwTVY1FlzR4GL95MR53pxDsGOzYNZECnSps5S6Ha5OuCxvW09PEuFESj0mUMQj88gu2x3IxHz/hSYaD366puz/pCy3TeiBWevDq9C/7UsGXLCC6HU/E1o6KcRCDa+sWEvqpZle83fMa0bAyYkiVbzzBousOrBnQD9XkQ/C/Z0OKYovRfKgQjERHEy+BNb9RLrArO5SaOovTiULCWFAKRnPLguPZKduyAAMXzCWYcJ7s= ceph-e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:54:10.473 INFO:teuthology.orchestra.run.smithi064.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:54:10.473 INFO:teuthology.orchestra.run.smithi064.stdout:Adding key to root@localhost authorized_keys... 2024-09-21T03:54:10.480 INFO:teuthology.orchestra.run.smithi064.stdout:Adding host smithi064... 2024-09-21T03:54:10.864 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:10 smithi064 ceph-mon[31180]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:54:11.773 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:11 smithi064 ceph-mon[31180]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi064", "addr": "172.21.15.64", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:54:12.940 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:12 smithi064 ceph-mon[31180]: Deploying cephadm binary to smithi064 2024-09-21T03:54:15.371 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Added host 'smithi064' with addr '172.21.15.64' 2024-09-21T03:54:15.372 INFO:teuthology.orchestra.run.smithi064.stdout:Deploying unmanaged mon service... 2024-09-21T03:54:16.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:15 smithi064 ceph-mon[31180]: from='mgr.14118 172.21.15.64:0/2225958686' entity='mgr.x' 2024-09-21T03:54:16.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:15 smithi064 ceph-mon[31180]: Added host smithi064 2024-09-21T03:54:16.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:15 smithi064 ceph-mon[31180]: from='mgr.14118 172.21.15.64:0/2225958686' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:54:16.738 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-21T03:54:16.739 INFO:teuthology.orchestra.run.smithi064.stdout:Deploying unmanaged mgr service... 2024-09-21T03:54:17.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:17 smithi064 ceph-mon[31180]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:54:17.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:17 smithi064 ceph-mon[31180]: Saving service mon spec with placement count:5 2024-09-21T03:54:17.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:17 smithi064 ceph-mon[31180]: from='mgr.14118 172.21.15.64:0/2225958686' entity='mgr.x' 2024-09-21T03:54:17.948 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-21T03:54:18.492 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:18 smithi064 ceph-mon[31180]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:54:18.492 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:18 smithi064 ceph-mon[31180]: Saving service mgr spec with placement count:2 2024-09-21T03:54:18.492 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:18 smithi064 ceph-mon[31180]: from='mgr.14118 172.21.15.64:0/2225958686' entity='mgr.x' 2024-09-21T03:54:19.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:19 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2780071472' entity='client.admin' 2024-09-21T03:54:20.707 INFO:teuthology.orchestra.run.smithi064.stdout:Enabling the dashboard module... 2024-09-21T03:54:21.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:20 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3796863582' entity='client.admin' 2024-09-21T03:54:21.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:20 smithi064 ceph-mon[31180]: from='mgr.14118 172.21.15.64:0/2225958686' entity='mgr.x' 2024-09-21T03:54:21.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:20 smithi064 ceph-mon[31180]: from='mgr.14118 172.21.15.64:0/2225958686' entity='mgr.x' 2024-09-21T03:54:22.197 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:21 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/4098334040' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-21T03:54:22.197 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:21 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: ignoring --setuser ceph since I am not root 2024-09-21T03:54:22.197 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:21 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: ignoring --setgroup ceph since I am not root 2024-09-21T03:54:23.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:22 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/4098334040' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-21T03:54:23.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:22 smithi064 ceph-mon[31180]: mgrmap e9: x(active, since 19s) 2024-09-21T03:54:23.732 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:54:23.732 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-21T03:54:23.733 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T03:54:23.733 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T03:54:23.733 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T03:54:23.733 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:54:23.733 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for the mgr to restart... 2024-09-21T03:54:23.733 INFO:teuthology.orchestra.run.smithi064.stdout:Waiting for mgr epoch 9... 2024-09-21T03:54:24.199 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:23 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1458431756' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T03:54:27.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:27 smithi064 ceph-mon[31180]: Active manager daemon x restarted 2024-09-21T03:54:27.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:27 smithi064 ceph-mon[31180]: Activating manager daemon x 2024-09-21T03:54:27.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:27 smithi064 ceph-mon[31180]: osdmap e3: 0 total, 0 up, 0 in 2024-09-21T03:54:27.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:27 smithi064 ceph-mon[31180]: mgrmap e10: x(active, starting, since 0.0063955s) 2024-09-21T03:54:27.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:27 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:54:27.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:27 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:54:27.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:27 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:54:27.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:27 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:54:27.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:27 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:54:27.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:27 smithi064 ceph-mon[31180]: Manager daemon x is now available 2024-09-21T03:54:28.502 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:28 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:54:28.502 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:28 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:54:28.502 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:28 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:54:28.502 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:28 smithi064 ceph-mon[31180]: mgrmap e11: x(active, since 1.00815s) 2024-09-21T03:54:28.747 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout { 2024-09-21T03:54:28.747 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-21T03:54:28.748 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T03:54:28.748 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout } 2024-09-21T03:54:28.748 INFO:teuthology.orchestra.run.smithi064.stdout:mgr epoch 9 is available 2024-09-21T03:54:28.748 INFO:teuthology.orchestra.run.smithi064.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-21T03:54:29.500 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:29 smithi064 ceph-mon[31180]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T03:54:29.500 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:29 smithi064 ceph-mon[31180]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T03:54:29.501 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:29 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: 2024-09-21T03:54:29.192+0000 7f1c3e59d640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:03:54:29] ENGINE Error in HTTPServer.serve 2024-09-21T03:54:29.501 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:29 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: Traceback (most recent call last): 2024-09-21T03:54:29.501 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:29 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:54:29.501 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:29 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: self._connections.run(self.expiration_interval) 2024-09-21T03:54:29.501 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:29 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:54:29.501 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:29 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: self._run(expiration_interval) 2024-09-21T03:54:29.501 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:29 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:54:29.502 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:29 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:54:29.502 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:29 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:54:29.502 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:29 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:54:29.502 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:29 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:54:29.502 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:29 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: s = self.context.wrap_socket( 2024-09-21T03:54:29.502 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:29 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:54:29.502 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:29 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: return self.sslsocket_class._create( 2024-09-21T03:54:29.502 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:29 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:54:29.502 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:29 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: self.do_handshake() 2024-09-21T03:54:29.502 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:29 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:54:29.502 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:29 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: self._sslobj.do_handshake() 2024-09-21T03:54:29.503 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:29 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:54:29.503 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:54:29 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: 2024-09-21T03:54:30.495 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:30 smithi064 ceph-mon[31180]: [21/Sep/2024:03:54:28] ENGINE Bus STARTING 2024-09-21T03:54:30.496 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:30 smithi064 ceph-mon[31180]: [21/Sep/2024:03:54:29] ENGINE Serving on http://172.21.15.64:8765 2024-09-21T03:54:30.496 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:30 smithi064 ceph-mon[31180]: [21/Sep/2024:03:54:29] ENGINE Serving on https://172.21.15.64:7150 2024-09-21T03:54:30.496 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:30 smithi064 ceph-mon[31180]: [21/Sep/2024:03:54:29] ENGINE Bus STARTED 2024-09-21T03:54:30.496 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:30 smithi064 ceph-mon[31180]: [21/Sep/2024:03:54:29] ENGINE Error in HTTPServer.serve 2024-09-21T03:54:30.496 INFO:journalctl@ceph.mon.a.smithi064.stdout: Traceback (most recent call last): 2024-09-21T03:54:30.496 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:54:30.496 INFO:journalctl@ceph.mon.a.smithi064.stdout: self._connections.run(self.expiration_interval) 2024-09-21T03:54:30.496 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:54:30.496 INFO:journalctl@ceph.mon.a.smithi064.stdout: self._run(expiration_interval) 2024-09-21T03:54:30.496 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:54:30.497 INFO:journalctl@ceph.mon.a.smithi064.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:54:30.497 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:54:30.497 INFO:journalctl@ceph.mon.a.smithi064.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:54:30.497 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:54:30.497 INFO:journalctl@ceph.mon.a.smithi064.stdout: s = self.context.wrap_socket( 2024-09-21T03:54:30.497 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:54:30.497 INFO:journalctl@ceph.mon.a.smithi064.stdout: return self.sslsocket_class._create( 2024-09-21T03:54:30.497 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:54:30.497 INFO:journalctl@ceph.mon.a.smithi064.stdout: self.do_handshake() 2024-09-21T03:54:30.497 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:54:30.497 INFO:journalctl@ceph.mon.a.smithi064.stdout: self._sslobj.do_handshake() 2024-09-21T03:54:30.497 INFO:journalctl@ceph.mon.a.smithi064.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:54:30.498 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:30 smithi064 ceph-mon[31180]: mgrmap e12: x(active, since 2s) 2024-09-21T03:54:30.498 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:30 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:30.498 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:30 smithi064 ceph-mon[31180]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:54:31.056 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+zCCAuOgAwIBAgIUFdvWJhke0HWN1ov+5Y4yR2HTwS8wDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkyMTAzNTQzMFoXDTM0\nMDkyMjAzNTQzMFowFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjY0MIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEAosbalosnTRsVTIxAuRNxuIOlzgkTFWUKQWEE\n+YD/duYZYnJt+WDFjtQ8Z5XdvSI9yQJyQxm2ukasID+PUMOUs4fVPGFlUYGNXJj2\ngj/WqE91pUWyXQLbKnI7D4J4EEMJm9uzCPWZIibipTo0IMZm86rCF85/CtXAcNHE\nlInwfFp1E9VkIBhmxNg+S/d6dDfKMxTwIpbgpxq1ep+LHHazqsIrAB8S5/HzWLOQ\nJgL3hWvbQNSlvIFquxddUqIbQ/b4PRNUsf8cQ1s9DPOfTP8/RtpY0WfIQLTUmjek\nWZqKMbneVSmqh1VM3qA8i8lhVZbttMQBK8ShmPgufUev+pj7CxFYI/dHCns077gG\nPmeZV1x8LqM9Ds7w3ZIJZ7fUlfKYhIebPY2pn2VgVtrrr2il38fFWOu0jm8w41M7\nGMcOM8ZJbojRKeOBlXvTuEYd0XYQEwDkiRR/TIsNP2JzrNwlLZyWE7Tl50HWhurq\nTabYgj/Z9VFhVZ7a2VH0p900U/QDwXvVccEXELlpnNKgui9Pw0sIhIrDz+SpGfNT\n4UWDH/sigwRJ8ks+dFucsVKMVN0eSPf9fvH6z0jZBUGBRiSD+bSP+SWmEjJoS/RC\nJY5E9sgSwy+epjoiapV1nltQLwfigIcMvBIEgLnAlFrQPrybiFPSD8joZ01MAtUh\nONLjIT8CAwEAAaM/MD0wLQYDVR0RBCYwJIIJc21pdGhpMDY0ghFkYXNoYm9hcmRf\nc2VydmVyc4cErBUPQDAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQB3\nnYnw1BRZTgK3MyH/pVnK92YMRn/EnZew1p6Ge7dE5DixQWsIFAFkvUPBYFLYqRfe\njXZdeCKo/e7rj1377gmoJO2Cqetw8rFOf7mHKy/uYfpTyoWoTDPt+TgnmY4FVkzR\ne4RwykMLf/eHb7PHnX3qdd6MZ0fmgXB/Lmb4A0yT4pVlee2SfCMkLJwTN6GB+Mek\nMt32c+YlYwETxUqWBgmRX9e/vSk4z6yPRYpxbGctq5wtBy8U4ylE5/QO5a9J1V0I\nhgu5l/OFsffF3CYtt2/1356HTOJUNcVQG2aUHPdox2qP1EaFJu0QSjlw0tnzONW4\nKKKYHFYjsHMwZLq3NM1rZhA98+Mp/vw+wE+gy1CpABvYY8/Y3pGQU3In8wgeGuBl\nno9duaZRgJfpD7LxSf1Ue8csDfvBaBGtsURTE+qLu+s0gKMFtNY6N25F7NwPYv/t\nAaObqXzxIHK5S++tcbmQdaQbK209FVnBeWIdzBW1g+pXJUr3bjtytGtx+BrltHBI\nRqapDSDpJN0Ld5b4VOf1vE5QXD16HkyGQ7he8CbzYQE1Y2f0p01m0aAE+SE+ZbFB\nAASNnje2gvAocVaW5bsDWA7bhDkGimOdKHKi6BmHSclqVSkZYd2izxuLSM6uk+UX\n2dlBARGYlAYE0MJIXCoKNXxi4Jr82nBR2aKfaPCpXQ==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKQIBAAKCAgEAosbalosnTRsVTIxAuRNxuIOlzgkTFWUKQWEE+YD/duYZYnJt\n+WDFjtQ8Z5XdvSI9yQJyQxm2ukasID+PUMOUs4fVPGFlUYGNXJj2gj/WqE91pUWy\nXQLbKnI7D4J4EEMJm9uzCPWZIibipTo0IMZm86rCF85/CtXAcNHElInwfFp1E9Vk\nIBhmxNg+S/d6dDfKMxTwIpbgpxq1ep+LHHazqsIrAB8S5/HzWLOQJgL3hWvbQNSl\nvIFquxddUqIbQ/b4PRNUsf8cQ1s9DPOfTP8/RtpY0WfIQLTUmjekWZqKMbneVSmq\nh1VM3qA8i8lhVZbttMQBK8ShmPgufUev+pj7CxFYI/dHCns077gGPmeZV1x8LqM9\nDs7w3ZIJZ7fUlfKYhIebPY2pn2VgVtrrr2il38fFWOu0jm8w41M7GMcOM8ZJbojR\nKeOBlXvTuEYd0XYQEwDkiRR/TIsNP2JzrNwlLZyWE7Tl50HWhurqTabYgj/Z9VFh\nVZ7a2VH0p900U/QDwXvVccEXELlpnNKgui9Pw0sIhIrDz+SpGfNT4UWDH/sigwRJ\n8ks+dFucsVKMVN0eSPf9fvH6z0jZBUGBRiSD+bSP+SWmEjJoS/RCJY5E9sgSwy+e\npjoiapV1nltQLwfigIcMvBIEgLnAlFrQPrybiFPSD8joZ01MAtUhONLjIT8CAwEA\nAQKCAgAHOuv0qeOqAhz1FHm/fsSbXIWd3DtTxPQqqUOOTZOLVXclL04BOLItwPGZ\n7Ec4InA6IHiw0FCUf5UfrZMRFE7jlpH7J5xXfXio90ZjzLAdM8JW//yHMxxHC2Hi\nM3pfxoqDuMOAD2Co6q9AhZBRMF7cTeZnJk3wTkRI/phLvMDI1T/EDMNwxYSzE6PY\nvfpbdSrvD+wGrUr3QTF0a+fC8ObUG+HLFJZDZTTY6MjrV1BlRcAsZgWilOEHxGbp\nx7eMUkj+dYBak6lOuJvCmoOl6OSHoiU1z/VOInobC4OqUzn2MXpVsA8aCcjrTpTB\n5MUFBzoxwE8JAafn4oY2HFwNtoPYDNNUkGX1RCDYk1JCuk0W6HZ1ItIrHZCmfVcN\nQxkMMoL520185uWorVn0AIjkNxD1MzW1xyqG1OGLndC2ysKNaVJe4BPHx/1m25mp\nMuVkGxN9oJXinyTR2N0Y8HRMI7yFl924+25D9o/t+4xayqmoqAEdjKC0tQfBlw5K\nrvajoIlZjXA0A4ETxweubK7tHTPSbkyd5+byTZioaWtTCbWVTI8FbJKdnNqFJgMZ\nlSfJMEppd4iqTnJVbvv0MHlJR1lwE0n/lvkoIgBTNhDGjZv7dggTwKplmjed9KtJ\nglJ9A0HJ8BOSBO3CiA+M7MrwP8Ogw907ekWvMjFXh+kGrnkvaQKCAQEA25jNA7+0\nQ8aY0tdilWTNasgDYIko+b8fN44A7Hic7cqNdXlMn5XtqFJGlKScc8lvw68Z41rA\nh7oYtffnUX+1aKiUPrUI9RPcjBByteEtOr0yqOixe2E8yqnZWQ6vASo2UR0SUTXQ\nl0bN9Bmvile70QUfDKcZEYUusingnINi0oMket0m5vQReb2EjZTKPot/wE4VdgNR\nC3Hi+QrFzuS9ougj0n2MrdGlcjvvglvQRSfj8jgMcdSaRBsBeTJA2SmpvrpLRWwm\nwXVLUx9KJ88LqwXur42xYkjsIIxqe0UxS8R/fSaTkq/36kdH2z3n59W6pesd0bdz\n1MqCG2Q+SyyS3QKCAQEAvcK8MlyJ1L/51mxTdg7LD5JvNyfPsdyy1Xc3HdJnsF1d\n2dG4QuRUsdSxsZObGElf6GSal2f2leiLhyju2F+a9UKPrM+4H9YJDUQJcyn/YwVs\n3mcGjoaJqtjoJIw/5aawNA1gng9Ln2b1ExpIir43/v0nX7QNFiHQXjKSuo0CcKcJ\nR4VmZ10wqbC3PL/MDHleh47iamuzemrWDvmnunScX8NMrKZ6PXfjiAEGCLQ5qPhw\nSK8JOUyl/xRyYFGuQ3aBJOCYPpxRWjhIRJ89r5TeL0t2xsI5eNpFLlwgpnDjBdSO\nHCV33Ykr3whLUyw8CO7p9BSOomaN/wKgXNJ48nycywKCAQEAxsCBrgbwbPdukkYH\nIBfgI+lrxQ/fWku1QV7xTxXLUx2k8tIam8XjGBl/cEGcqN+EnKP61l/pKe8xNmFt\np8Hiuxr5MdhAhzC2yL6KVDu96BAdresN2BGmJ/ealJRJqNA/4SJi1cXTSVf4SBqH\nWrsVbpUoCRekBMt5/B8TABWad9XiSqGm+BmnR3xUYKBufDVeJvuFSQ6vZ6BoQeeS\nKTO2v59D7NtdXGUHTWEZ4e6fjjgQg8Ze3RgDl+ipaLwnHWYsEEgkoGzGlwpGZrFH\nOMiYUu5HnfSsu7wsmzj7lx6FA4KRsQAdSQKjbBOf4bsiDh7PlCQ9FPbpi4rOP2R3\nTSrGsQKCAQAJq78Dogu0OAR3lRW70o0uwfvBB9ph6imDD4xTjMGUD24VB8coQZK/\nbWOQ9UDWQAcPRPFL/hRnF27k+IUTqeuvzDCeUgfEN5hGgM0qfV1yongl7YI8CUdC\n2cMa+brX7WLVhEkERStJeGnNC23fBEq+SBCVpAt2sUp5QoKP11fkfd7PMlFBu1Ib\nk9IC2pN/nz/eEL4hRlPjTUaThM2zhvXhO4bQ1bstHssNx1+TWfwGTIMMcT87GFjt\nYxFhL0h+T6/NPVUE+9p5ipzEQolWRXP9XQ80mflr21PjRPa1BNPNHEQR0ycEt1Tx\nDO8p0ftNST/LHLEYPbnG4U9i66cWVJt5AoIBAQCOST2S+ShqcEDFFkn9NhLP98eU\nXZnXxxixKDMJweAEPpe+iLc5UNnjibfhtqrZxkTz/WPi22GG+q8SwE+i7aIioNJn\nL+vVDxGzCwhSnY++BPgzG6g6c65jWolFeMHPXSOdVjIQFNTzbHYSUV82fSA6uu3U\nmrvhsP8BDiT/kZwTcgx8NOIf+J5ryNsYAvetd100YbeAulDbpmwMBxfRTL+pdNEH\ncZHoA2ttFwlalfp+TwBHsIcJjRWSs7MckrVPHwW+vxV2lC20H6Z+DiRot4DfUnD9\nP0DpqyE+/UdMlTBwWlS7TeaiAsXkkP40XUGM+asZ0txQm9biTmnuP3zty2PT\n-----END RSA PRIVATE KEY-----\n"} 2024-09-21T03:54:31.365 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:31 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:32.338 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-21T03:54:32.931 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:32 smithi064 ceph-mon[31180]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:54:32.931 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:32 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:34.117 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-21T03:54:34.117 INFO:teuthology.orchestra.run.smithi064.stdout:Creating initial admin user... 2024-09-21T03:54:34.417 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:34 smithi064 ceph-mon[31180]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:54:34.417 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:34 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:35.528 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:35 smithi064 ceph-mon[31180]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:54:35.528 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:35 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:35.531 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$X7.doD/TooIV6XgCFUarqOPqsv2V9N6V/GhpwJm6QAaA2h2wTwRrG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726890874, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-21T03:54:35.531 INFO:teuthology.orchestra.run.smithi064.stdout:Fetching dashboard port number... 2024-09-21T03:54:36.443 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:36 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1631417256' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-21T03:54:36.687 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stdout 8443 2024-09-21T03:54:36.699 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T03:54:36.699 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout disabled 2024-09-21T03:54:36.707 INFO:teuthology.orchestra.run.smithi064.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T03:54:36.707 INFO:teuthology.orchestra.run.smithi064.stdout:systemctl: stdout inactive 2024-09-21T03:54:36.707 INFO:teuthology.orchestra.run.smithi064.stdout:firewalld.service is not enabled 2024-09-21T03:54:36.707 INFO:teuthology.orchestra.run.smithi064.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-21T03:54:36.709 INFO:teuthology.orchestra.run.smithi064.stdout:Ceph Dashboard is now available at: 2024-09-21T03:54:36.709 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:54:36.709 INFO:teuthology.orchestra.run.smithi064.stdout: URL: https://smithi064.front.sepia.ceph.com:8443/ 2024-09-21T03:54:36.710 INFO:teuthology.orchestra.run.smithi064.stdout: User: admin 2024-09-21T03:54:36.710 INFO:teuthology.orchestra.run.smithi064.stdout: Password: j11fdhslg0 2024-09-21T03:54:36.710 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:54:36.710 INFO:teuthology.orchestra.run.smithi064.stdout:Saving cluster configuration to /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config directory 2024-09-21T03:54:37.969 INFO:teuthology.orchestra.run.smithi064.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-21T03:54:37.970 INFO:teuthology.orchestra.run.smithi064.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-21T03:54:37.970 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:54:37.970 INFO:teuthology.orchestra.run.smithi064.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:54:37.970 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:54:37.971 INFO:teuthology.orchestra.run.smithi064.stdout:Or, if you are only running a single cluster on this host: 2024-09-21T03:54:37.971 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:54:37.971 INFO:teuthology.orchestra.run.smithi064.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-21T03:54:37.971 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:54:37.971 INFO:teuthology.orchestra.run.smithi064.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-21T03:54:37.971 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:54:37.971 INFO:teuthology.orchestra.run.smithi064.stdout: ceph telemetry on 2024-09-21T03:54:37.972 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:54:37.972 INFO:teuthology.orchestra.run.smithi064.stdout:For more information see: 2024-09-21T03:54:37.972 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:54:37.972 INFO:teuthology.orchestra.run.smithi064.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-21T03:54:37.972 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:54:37.972 INFO:teuthology.orchestra.run.smithi064.stdout:Bootstrap complete. 2024-09-21T03:54:38.009 INFO:tasks.cephadm:Fetching config... 2024-09-21T03:54:38.009 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-21T03:54:38.009 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-21T03:54:38.025 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-21T03:54:38.026 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-21T03:54:38.026 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-21T03:54:38.082 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-21T03:54:38.082 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-21T03:54:38.082 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/keyring of=/dev/stdout 2024-09-21T03:54:38.148 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-21T03:54:38.149 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-21T03:54:38.149 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-21T03:54:38.204 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-21T03:54:38.205 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzyWcsWId3jSZQ5ZWpNq6P4TrliwZCol7g057T4W/tzr8bEk6n83DbTvG693+/+H4++4qylZxCU3qLi/0jw5bDIu/CGB964UicJx/cytRGBhvUAL65WEz+0p0adO3Vb/oT/JOXn9lAEvMhqUuu9k3YWseiuML1mBSUU7krIhANW1kfdthmeNjaRFDxZBExgfmP+G+FP2LwuWu5m3CSp2CuEgvJVJ+QVsVBTjdzqpGeydFmMWQ2dXbMT7tSilfXao+cNWa6DxaKGe7otlwymt0QQwTVY1FlzR4GL95MR53pxDsGOzYNZECnSps5S6Ha5OuCxvW09PEuFESj0mUMQj88gu2x3IxHz/hSYaD366puz/pCy3TeiBWevDq9C/7UsGXLCC6HU/E1o6KcRCDa+sWEvqpZle83fMa0bAyYkiVbzzBousOrBnQD9XkQ/C/Z0OKYovRfKgQjERHEy+BNb9RLrArO5SaOovTiULCWFAKRnPLguPZKduyAAMXzCWYcJ7s= ceph-e7c927ba-77cc-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T03:54:38.280 INFO:teuthology.orchestra.run.smithi064.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzyWcsWId3jSZQ5ZWpNq6P4TrliwZCol7g057T4W/tzr8bEk6n83DbTvG693+/+H4++4qylZxCU3qLi/0jw5bDIu/CGB964UicJx/cytRGBhvUAL65WEz+0p0adO3Vb/oT/JOXn9lAEvMhqUuu9k3YWseiuML1mBSUU7krIhANW1kfdthmeNjaRFDxZBExgfmP+G+FP2LwuWu5m3CSp2CuEgvJVJ+QVsVBTjdzqpGeydFmMWQ2dXbMT7tSilfXao+cNWa6DxaKGe7otlwymt0QQwTVY1FlzR4GL95MR53pxDsGOzYNZECnSps5S6Ha5OuCxvW09PEuFESj0mUMQj88gu2x3IxHz/hSYaD366puz/pCy3TeiBWevDq9C/7UsGXLCC6HU/E1o6KcRCDa+sWEvqpZle83fMa0bAyYkiVbzzBousOrBnQD9XkQ/C/Z0OKYovRfKgQjERHEy+BNb9RLrArO5SaOovTiULCWFAKRnPLguPZKduyAAMXzCWYcJ7s= ceph-e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:54:38.292 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzyWcsWId3jSZQ5ZWpNq6P4TrliwZCol7g057T4W/tzr8bEk6n83DbTvG693+/+H4++4qylZxCU3qLi/0jw5bDIu/CGB964UicJx/cytRGBhvUAL65WEz+0p0adO3Vb/oT/JOXn9lAEvMhqUuu9k3YWseiuML1mBSUU7krIhANW1kfdthmeNjaRFDxZBExgfmP+G+FP2LwuWu5m3CSp2CuEgvJVJ+QVsVBTjdzqpGeydFmMWQ2dXbMT7tSilfXao+cNWa6DxaKGe7otlwymt0QQwTVY1FlzR4GL95MR53pxDsGOzYNZECnSps5S6Ha5OuCxvW09PEuFESj0mUMQj88gu2x3IxHz/hSYaD366puz/pCy3TeiBWevDq9C/7UsGXLCC6HU/E1o6KcRCDa+sWEvqpZle83fMa0bAyYkiVbzzBousOrBnQD9XkQ/C/Z0OKYovRfKgQjERHEy+BNb9RLrArO5SaOovTiULCWFAKRnPLguPZKduyAAMXzCWYcJ7s= ceph-e7c927ba-77cc-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T03:54:38.333 INFO:teuthology.orchestra.run.smithi119.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzyWcsWId3jSZQ5ZWpNq6P4TrliwZCol7g057T4W/tzr8bEk6n83DbTvG693+/+H4++4qylZxCU3qLi/0jw5bDIu/CGB964UicJx/cytRGBhvUAL65WEz+0p0adO3Vb/oT/JOXn9lAEvMhqUuu9k3YWseiuML1mBSUU7krIhANW1kfdthmeNjaRFDxZBExgfmP+G+FP2LwuWu5m3CSp2CuEgvJVJ+QVsVBTjdzqpGeydFmMWQ2dXbMT7tSilfXao+cNWa6DxaKGe7otlwymt0QQwTVY1FlzR4GL95MR53pxDsGOzYNZECnSps5S6Ha5OuCxvW09PEuFESj0mUMQj88gu2x3IxHz/hSYaD366puz/pCy3TeiBWevDq9C/7UsGXLCC6HU/E1o6KcRCDa+sWEvqpZle83fMa0bAyYkiVbzzBousOrBnQD9XkQ/C/Z0OKYovRfKgQjERHEy+BNb9RLrArO5SaOovTiULCWFAKRnPLguPZKduyAAMXzCWYcJ7s= ceph-e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:54:38.345 DEBUG:teuthology.orchestra.run.smithi131:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzyWcsWId3jSZQ5ZWpNq6P4TrliwZCol7g057T4W/tzr8bEk6n83DbTvG693+/+H4++4qylZxCU3qLi/0jw5bDIu/CGB964UicJx/cytRGBhvUAL65WEz+0p0adO3Vb/oT/JOXn9lAEvMhqUuu9k3YWseiuML1mBSUU7krIhANW1kfdthmeNjaRFDxZBExgfmP+G+FP2LwuWu5m3CSp2CuEgvJVJ+QVsVBTjdzqpGeydFmMWQ2dXbMT7tSilfXao+cNWa6DxaKGe7otlwymt0QQwTVY1FlzR4GL95MR53pxDsGOzYNZECnSps5S6Ha5OuCxvW09PEuFESj0mUMQj88gu2x3IxHz/hSYaD366puz/pCy3TeiBWevDq9C/7UsGXLCC6HU/E1o6KcRCDa+sWEvqpZle83fMa0bAyYkiVbzzBousOrBnQD9XkQ/C/Z0OKYovRfKgQjERHEy+BNb9RLrArO5SaOovTiULCWFAKRnPLguPZKduyAAMXzCWYcJ7s= ceph-e7c927ba-77cc-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T03:54:38.387 INFO:teuthology.orchestra.run.smithi131.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzyWcsWId3jSZQ5ZWpNq6P4TrliwZCol7g057T4W/tzr8bEk6n83DbTvG693+/+H4++4qylZxCU3qLi/0jw5bDIu/CGB964UicJx/cytRGBhvUAL65WEz+0p0adO3Vb/oT/JOXn9lAEvMhqUuu9k3YWseiuML1mBSUU7krIhANW1kfdthmeNjaRFDxZBExgfmP+G+FP2LwuWu5m3CSp2CuEgvJVJ+QVsVBTjdzqpGeydFmMWQ2dXbMT7tSilfXao+cNWa6DxaKGe7otlwymt0QQwTVY1FlzR4GL95MR53pxDsGOzYNZECnSps5S6Ha5OuCxvW09PEuFESj0mUMQj88gu2x3IxHz/hSYaD366puz/pCy3TeiBWevDq9C/7UsGXLCC6HU/E1o6KcRCDa+sWEvqpZle83fMa0bAyYkiVbzzBousOrBnQD9XkQ/C/Z0OKYovRfKgQjERHEy+BNb9RLrArO5SaOovTiULCWFAKRnPLguPZKduyAAMXzCWYcJ7s= ceph-e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:54:38.399 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-21T03:54:38.655 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:38.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:38 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3585671136' entity='client.admin' 2024-09-21T03:54:40.324 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-21T03:54:40.325 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-21T03:54:40.576 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:40.697 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:40 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/961645773' entity='client.admin' 2024-09-21T03:54:40.698 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:40 smithi064 ceph-mon[31180]: mgrmap e13: x(active, since 12s) 2024-09-21T03:54:40.698 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:40 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:40.698 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:40 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:40.698 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:40 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi064", "name": "osd_memory_target"} : dispatch 2024-09-21T03:54:40.698 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:40 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:40.698 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:40 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:54:40.698 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:40 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:42.093 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi119 2024-09-21T03:54:42.094 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-21T03:54:42.094 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/etc/ceph/ceph.conf 2024-09-21T03:54:42.113 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-21T03:54:42.113 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:54:42.172 INFO:tasks.cephadm:Adding host smithi119 to orchestrator... 2024-09-21T03:54:42.172 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph orch host add smithi119 2024-09-21T03:54:42.424 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:42.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:42 smithi064 ceph-mon[31180]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:54:42.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:42 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:42.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:42 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:54:42.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:42 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:54:42.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:42 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:54:42.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:42 smithi064 ceph-mon[31180]: Updating smithi064:/etc/ceph/ceph.conf 2024-09-21T03:54:42.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:42 smithi064 ceph-mon[31180]: Updating smithi064:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:54:42.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:42 smithi064 ceph-mon[31180]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:54:42.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:42 smithi064 ceph-mon[31180]: Updating smithi064:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:54:42.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:42 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:42.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:42 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:42.513 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:42 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:44.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:44 smithi064 ceph-mon[31180]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi119", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:54:45.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:45 smithi064 ceph-mon[31180]: Deploying cephadm binary to smithi119 2024-09-21T03:54:46.794 INFO:teuthology.orchestra.run.smithi064.stdout:Added host 'smithi119' with addr '172.21.15.119' 2024-09-21T03:54:47.392 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T03:54:47.643 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:48.066 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:47 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:48.066 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:47 smithi064 ceph-mon[31180]: Added host smithi119 2024-09-21T03:54:48.066 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:47 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:54:48.066 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:47 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:48.067 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:47 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:48.293 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:54:48.293 INFO:teuthology.orchestra.run.smithi064.stdout:[{"addr": "172.21.15.64", "hostname": "smithi064", "labels": [], "status": ""}, {"addr": "172.21.15.119", "hostname": "smithi119", "labels": [], "status": ""}] 2024-09-21T03:54:48.847 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:48 smithi064 ceph-mon[31180]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:54:48.847 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:48 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:48.848 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi131 2024-09-21T03:54:48.848 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-21T03:54:48.848 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/etc/ceph/ceph.conf 2024-09-21T03:54:48.867 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-21T03:54:48.867 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:54:48.925 INFO:tasks.cephadm:Adding host smithi131 to orchestrator... 2024-09-21T03:54:48.925 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph orch host add smithi131 2024-09-21T03:54:49.174 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:49.889 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:49 smithi064 ceph-mon[31180]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:54:51.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:50 smithi064 ceph-mon[31180]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:54:51.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:50 smithi064 ceph-mon[31180]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi131", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:54:52.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:51 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:52.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:51 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:52.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:51 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:52.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:51 smithi064 ceph-mon[31180]: Deploying cephadm binary to smithi131 2024-09-21T03:54:52.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:51 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:52.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:51 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi119", "name": "osd_memory_target"} : dispatch 2024-09-21T03:54:52.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:51 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:54:52.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:51 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:54:52.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:51 smithi064 ceph-mon[31180]: Updating smithi119:/etc/ceph/ceph.conf 2024-09-21T03:54:52.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:51 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:52.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:51 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:52.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:51 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:53.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:52 smithi064 ceph-mon[31180]: Updating smithi119:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:54:53.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:52 smithi064 ceph-mon[31180]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:54:53.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:52 smithi064 ceph-mon[31180]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:54:53.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:52 smithi064 ceph-mon[31180]: Updating smithi119:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:54:53.552 INFO:teuthology.orchestra.run.smithi064.stdout:Added host 'smithi131' with addr '172.21.15.131' 2024-09-21T03:54:54.272 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T03:54:54.524 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:54.848 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:54 smithi064 ceph-mon[31180]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:54:54.848 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:54 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:54.848 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:54 smithi064 ceph-mon[31180]: Added host smithi131 2024-09-21T03:54:54.848 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:54 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:54:54.848 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:54 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:54.849 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:54 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:55.272 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:54:55.272 INFO:teuthology.orchestra.run.smithi064.stdout:[{"addr": "172.21.15.64", "hostname": "smithi064", "labels": [], "status": ""}, {"addr": "172.21.15.119", "hostname": "smithi119", "labels": [], "status": ""}, {"addr": "172.21.15.131", "hostname": "smithi131", "labels": [], "status": ""}] 2024-09-21T03:54:55.862 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-21T03:54:55.862 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd crush tunables default 2024-09-21T03:54:56.113 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:54:56.224 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:55 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:56.866 INFO:teuthology.orchestra.run.smithi064.stderr:adjusted tunables profile to default 2024-09-21T03:54:57.227 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:56 smithi064 ceph-mon[31180]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:54:57.227 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:56 smithi064 ceph-mon[31180]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:54:57.227 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:56 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2434181406' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T03:54:57.432 INFO:tasks.cephadm:Adding mon.a on smithi064 2024-09-21T03:54:57.432 INFO:tasks.cephadm:Adding mon.b on smithi119 2024-09-21T03:54:57.432 INFO:tasks.cephadm:Adding mon.c on smithi131 2024-09-21T03:54:57.432 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph orch apply mon '3;smithi064:172.21.15.64=a;smithi119:172.21.15.119=b;smithi131:172.21.15.131=c' 2024-09-21T03:54:57.674 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T03:54:57.756 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T03:54:58.124 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:57 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2434181406' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T03:54:58.124 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:57 smithi064 ceph-mon[31180]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:54:58.124 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:57 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:58.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:57 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:58.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:57 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:58.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:57 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:58.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:57 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi131", "name": "osd_memory_target"} : dispatch 2024-09-21T03:54:58.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:57 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:54:58.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:57 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:54:58.458 INFO:teuthology.orchestra.run.smithi131.stdout:Scheduled mon update... 2024-09-21T03:54:59.010 DEBUG:teuthology.orchestra.run.smithi119:mon.b> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mon.b.service 2024-09-21T03:54:59.013 DEBUG:teuthology.orchestra.run.smithi131:mon.c> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mon.c.service 2024-09-21T03:54:59.015 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:54:59.015 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:54:59.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:58 smithi064 ceph-mon[31180]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:54:59.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:58 smithi064 ceph-mon[31180]: Updating smithi131:/etc/ceph/ceph.conf 2024-09-21T03:54:59.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:58 smithi064 ceph-mon[31180]: Updating smithi131:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:54:59.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:58 smithi064 ceph-mon[31180]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:54:59.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:58 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:59.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:58 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:59.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:58 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:59.263 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:58 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:59.263 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:58 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:54:59.263 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:58 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:54:59.263 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:58 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:54:59.263 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:58 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:54:59.263 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:58 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:54:59.263 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:58 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:54:59.737 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:55:00.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:59 smithi064 ceph-mon[31180]: Updating smithi131:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:55:00.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:59 smithi064 ceph-mon[31180]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi064:172.21.15.64=a;smithi119:172.21.15.119=b;smithi131:172.21.15.131=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:55:00.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:59 smithi064 ceph-mon[31180]: Saving service mon spec with placement smithi064:172.21.15.64=a;smithi119:172.21.15.119=b;smithi131:172.21.15.131=c;count:3 2024-09-21T03:55:00.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:54:59 smithi064 ceph-mon[31180]: Deploying daemon mon.c on smithi131 2024-09-21T03:55:00.478 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-21T03:55:00.478 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":1,"fsid":"e7c927ba-77cc-11ef-baf5-efdc52797490","modified":"2024-09-21T03:53:37.752699Z","created":"2024-09-21T03:53:37.752699Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T03:55:00.478 INFO:teuthology.orchestra.run.smithi131.stderr:dumped monmap epoch 1 2024-09-21T03:55:01.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:00 smithi064 ceph-mon[31180]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:01.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:00 smithi064 ceph-mon[31180]: from='client.? 172.21.15.131:0/1293599846' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:55:02.231 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:55:02.232 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:55:02.560 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:55:03.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:02 smithi064 ceph-mon[31180]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:03.419 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-21T03:55:03.419 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":1,"fsid":"e7c927ba-77cc-11ef-baf5-efdc52797490","modified":"2024-09-21T03:53:37.752699Z","created":"2024-09-21T03:53:37.752699Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T03:55:03.419 INFO:teuthology.orchestra.run.smithi131.stderr:dumped monmap epoch 1 2024-09-21T03:55:03.420 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:03 smithi131 systemd[1]: Starting Ceph mon.c for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T03:55:04.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:03 smithi064 ceph-mon[31180]: from='client.? 172.21.15.131:0/2877678002' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:55:04.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 podman[35170]: 2024-09-21 03:55:04.156029382 +0000 UTC m=+0.517265546 container create 0697ec341458496080f3751f3b8ea733520057582b88af2a59b8919896d82e7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-c, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:55:04.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 podman[35170]: 2024-09-21 03:55:04.086686521 +0000 UTC m=+0.447922696 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:55:04.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 podman[35170]: 2024-09-21 03:55:04.34170457 +0000 UTC m=+0.702940733 container init 0697ec341458496080f3751f3b8ea733520057582b88af2a59b8919896d82e7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-c, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:55:04.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 podman[35170]: 2024-09-21 03:55:04.344847481 +0000 UTC m=+0.706083646 container start 0697ec341458496080f3751f3b8ea733520057582b88af2a59b8919896d82e7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-09-21T03:55:04.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T03:55:04.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T03:55:04.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: pidfile_write: ignore empty --pid-file 2024-09-21T03:55:04.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: load: jerasure load: lrc 2024-09-21T03:55:04.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: RocksDB version: 7.9.2 2024-09-21T03:55:04.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Git sha 0 2024-09-21T03:55:04.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T03:55:04.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: DB SUMMARY 2024-09-21T03:55:04.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: DB Session ID: 3HFU9SNX761LWDLB56ZF 2024-09-21T03:55:04.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: CURRENT file: CURRENT 2024-09-21T03:55:04.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T03:55:04.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-21T03:55:04.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-21T03:55:04.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-21T03:55:04.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.error_if_exists: 0 2024-09-21T03:55:04.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.create_if_missing: 0 2024-09-21T03:55:04.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.paranoid_checks: 1 2024-09-21T03:55:04.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T03:55:04.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T03:55:04.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T03:55:04.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.env: 0x555efbaced60 2024-09-21T03:55:04.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T03:55:04.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.info_log: 0x555f01789360 2024-09-21T03:55:04.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T03:55:04.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.statistics: (nil) 2024-09-21T03:55:04.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.use_fsync: 0 2024-09-21T03:55:04.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.max_log_file_size: 0 2024-09-21T03:55:04.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T03:55:04.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T03:55:04.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T03:55:04.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T03:55:04.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.allow_fallocate: 1 2024-09-21T03:55:04.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T03:55:04.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T03:55:04.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.use_direct_reads: 0 2024-09-21T03:55:04.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T03:55:04.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T03:55:04.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.db_log_dir: 2024-09-21T03:55:04.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.wal_dir: 2024-09-21T03:55:04.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T03:55:04.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T03:55:04.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T03:55:04.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T03:55:04.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T03:55:04.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T03:55:04.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T03:55:04.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T03:55:04.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.write_buffer_manager: 0x555f0178dd60 2024-09-21T03:55:04.809 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T03:55:04.809 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T03:55:04.809 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T03:55:04.809 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T03:55:04.809 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T03:55:04.809 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T03:55:04.809 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T03:55:04.809 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T03:55:04.809 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.unordered_write: 0 2024-09-21T03:55:04.809 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T03:55:04.809 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T03:55:04.809 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T03:55:04.809 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T03:55:04.809 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.row_cache: None 2024-09-21T03:55:04.810 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.wal_filter: None 2024-09-21T03:55:04.810 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T03:55:04.810 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T03:55:04.810 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.two_write_queues: 0 2024-09-21T03:55:04.810 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T03:55:04.810 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.wal_compression: 0 2024-09-21T03:55:04.810 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.atomic_flush: 0 2024-09-21T03:55:04.810 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T03:55:04.810 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T03:55:04.810 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T03:55:04.810 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.log_readahead_size: 0 2024-09-21T03:55:04.810 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T03:55:04.810 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T03:55:04.811 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T03:55:04.811 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T03:55:04.811 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T03:55:04.811 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T03:55:04.811 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T03:55:04.811 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.max_background_jobs: 2 2024-09-21T03:55:04.811 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.max_background_compactions: -1 2024-09-21T03:55:04.811 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.max_subcompactions: 1 2024-09-21T03:55:04.811 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T03:55:04.811 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T03:55:04.811 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T03:55:04.811 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T03:55:04.811 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T03:55:04.811 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T03:55:04.812 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T03:55:04.812 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T03:55:04.812 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.max_open_files: -1 2024-09-21T03:55:04.812 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T03:55:04.812 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T03:55:04.812 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T03:55:04.812 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T03:55:04.812 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.max_background_flushes: -1 2024-09-21T03:55:04.812 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Compression algorithms supported: 2024-09-21T03:55:04.812 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: kZSTD supported: 0 2024-09-21T03:55:04.812 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: kXpressCompression supported: 0 2024-09-21T03:55:04.812 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: kBZip2Compression supported: 0 2024-09-21T03:55:04.812 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T03:55:04.812 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: kLZ4Compression supported: 1 2024-09-21T03:55:04.812 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: kZlibCompression supported: 1 2024-09-21T03:55:04.813 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T03:55:04.813 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: kSnappyCompression supported: 1 2024-09-21T03:55:04.813 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T03:55:04.813 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T03:55:04.813 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-21T03:55:04.813 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T03:55:04.813 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T03:55:04.813 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.merge_operator: 2024-09-21T03:55:04.813 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.compaction_filter: None 2024-09-21T03:55:04.813 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T03:55:04.813 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T03:55:04.813 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T03:55:04.813 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T03:55:04.813 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555f01788fa0) 2024-09-21T03:55:04.814 INFO:journalctl@ceph.mon.c.smithi131.stdout: cache_index_and_filter_blocks: 1 2024-09-21T03:55:04.814 INFO:journalctl@ceph.mon.c.smithi131.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T03:55:04.814 INFO:journalctl@ceph.mon.c.smithi131.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T03:55:04.814 INFO:journalctl@ceph.mon.c.smithi131.stdout: pin_top_level_index_and_filter: 1 2024-09-21T03:55:04.814 INFO:journalctl@ceph.mon.c.smithi131.stdout: index_type: 0 2024-09-21T03:55:04.814 INFO:journalctl@ceph.mon.c.smithi131.stdout: data_block_index_type: 0 2024-09-21T03:55:04.814 INFO:journalctl@ceph.mon.c.smithi131.stdout: index_shortening: 1 2024-09-21T03:55:04.814 INFO:journalctl@ceph.mon.c.smithi131.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T03:55:04.814 INFO:journalctl@ceph.mon.c.smithi131.stdout: checksum: 4 2024-09-21T03:55:04.814 INFO:journalctl@ceph.mon.c.smithi131.stdout: no_block_cache: 0 2024-09-21T03:55:04.814 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_cache: 0x555efd74d610 2024-09-21T03:55:04.814 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_cache_name: BinnedLRUCache 2024-09-21T03:55:04.814 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_cache_options: 2024-09-21T03:55:04.815 INFO:journalctl@ceph.mon.c.smithi131.stdout: capacity : 536870912 2024-09-21T03:55:04.815 INFO:journalctl@ceph.mon.c.smithi131.stdout: num_shard_bits : 4 2024-09-21T03:55:04.815 INFO:journalctl@ceph.mon.c.smithi131.stdout: strict_capacity_limit : 0 2024-09-21T03:55:04.815 INFO:journalctl@ceph.mon.c.smithi131.stdout: high_pri_pool_ratio: 0.000 2024-09-21T03:55:04.815 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_cache_compressed: (nil) 2024-09-21T03:55:04.815 INFO:journalctl@ceph.mon.c.smithi131.stdout: persistent_cache: (nil) 2024-09-21T03:55:04.815 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_size: 4096 2024-09-21T03:55:04.815 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_size_deviation: 10 2024-09-21T03:55:04.815 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_restart_interval: 16 2024-09-21T03:55:04.815 INFO:journalctl@ceph.mon.c.smithi131.stdout: index_block_restart_interval: 1 2024-09-21T03:55:04.815 INFO:journalctl@ceph.mon.c.smithi131.stdout: metadata_block_size: 4096 2024-09-21T03:55:04.815 INFO:journalctl@ceph.mon.c.smithi131.stdout: partition_filters: 0 2024-09-21T03:55:04.815 INFO:journalctl@ceph.mon.c.smithi131.stdout: use_delta_encoding: 1 2024-09-21T03:55:04.815 INFO:journalctl@ceph.mon.c.smithi131.stdout: filter_policy: bloomfilter 2024-09-21T03:55:04.816 INFO:journalctl@ceph.mon.c.smithi131.stdout: whole_key_filtering: 1 2024-09-21T03:55:04.816 INFO:journalctl@ceph.mon.c.smithi131.stdout: verify_compression: 0 2024-09-21T03:55:04.816 INFO:journalctl@ceph.mon.c.smithi131.stdout: read_amp_bytes_per_bit: 0 2024-09-21T03:55:04.816 INFO:journalctl@ceph.mon.c.smithi131.stdout: format_version: 5 2024-09-21T03:55:04.816 INFO:journalctl@ceph.mon.c.smithi131.stdout: enable_index_compression: 1 2024-09-21T03:55:04.816 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_align: 0 2024-09-21T03:55:04.816 INFO:journalctl@ceph.mon.c.smithi131.stdout: max_auto_readahead_size: 262144 2024-09-21T03:55:04.816 INFO:journalctl@ceph.mon.c.smithi131.stdout: prepopulate_block_cache: 0 2024-09-21T03:55:04.816 INFO:journalctl@ceph.mon.c.smithi131.stdout: initial_auto_readahead_size: 8192 2024-09-21T03:55:04.816 INFO:journalctl@ceph.mon.c.smithi131.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T03:55:04.816 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T03:55:04.816 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T03:55:04.816 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.compression: NoCompression 2024-09-21T03:55:04.816 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T03:55:04.816 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T03:55:04.816 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T03:55:04.817 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.num_levels: 7 2024-09-21T03:55:04.817 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T03:55:04.817 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T03:55:04.817 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T03:55:04.817 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T03:55:04.817 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T03:55:04.817 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T03:55:04.817 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T03:55:04.817 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:55:04.817 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T03:55:04.817 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T03:55:04.817 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:55:04.817 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:55:04.817 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T03:55:04.818 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T03:55:04.818 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T03:55:04.818 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T03:55:04.818 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:55:04.818 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:55:04.818 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T03:55:04.818 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T03:55:04.818 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:55:04.818 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T03:55:04.818 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T03:55:04.818 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T03:55:04.818 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T03:55:04.818 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T03:55:04.818 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T03:55:04.818 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T03:55:04.819 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T03:55:04.819 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T03:55:04.819 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T03:55:04.819 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T03:55:04.819 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T03:55:04.819 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T03:55:04.819 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T03:55:04.819 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T03:55:04.819 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T03:55:04.819 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T03:55:04.819 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T03:55:04.819 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T03:55:04.819 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T03:55:04.820 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T03:55:04.820 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T03:55:04.820 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T03:55:04.820 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T03:55:04.820 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T03:55:04.820 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T03:55:04.820 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T03:55:04.820 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T03:55:04.820 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T03:55:04.820 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T03:55:04.820 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T03:55:04.820 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T03:55:04.820 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T03:55:04.820 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.inplace_update_support: 0 2024-09-21T03:55:04.821 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T03:55:04.821 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T03:55:04.821 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T03:55:04.821 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T03:55:04.821 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.bloom_locality: 0 2024-09-21T03:55:04.821 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.max_successive_merges: 0 2024-09-21T03:55:04.821 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T03:55:04.821 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T03:55:04.821 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T03:55:04.821 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T03:55:04.821 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.ttl: 2592000 2024-09-21T03:55:04.821 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T03:55:04.821 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T03:55:04.822 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T03:55:04.822 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.enable_blob_files: false 2024-09-21T03:55:04.822 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.min_blob_size: 0 2024-09-21T03:55:04.822 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T03:55:04.822 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T03:55:04.823 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T03:55:04.823 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T03:55:04.823 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T03:55:04.823 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T03:55:04.823 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T03:55:04.823 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T03:55:04.823 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-21T03:55:04.823 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-21T03:55:04.823 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 52ee8e4d-250b-426c-a08e-ba6fb4b06216 2024-09-21T03:55:04.823 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726890904381694, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-21T03:55:04.823 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-21T03:55:04.823 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726890904382163, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726890904, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "52ee8e4d-250b-426c-a08e-ba6fb4b06216", "db_session_id": "3HFU9SNX761LWDLB56ZF", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-21T03:55:04.823 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726890904382256, "job": 1, "event": "recovery_finished"} 2024-09-21T03:55:04.823 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-21T03:55:04.824 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T03:55:04.824 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555f017aca80 2024-09-21T03:55:04.824 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: DB pointer 0x555f02552000 2024-09-21T03:55:04.824 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T03:55:04.824 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T03:55:04.824 INFO:journalctl@ceph.mon.c.smithi131.stdout: ** DB Stats ** 2024-09-21T03:55:04.824 INFO:journalctl@ceph.mon.c.smithi131.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:55:04.824 INFO:journalctl@ceph.mon.c.smithi131.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T03:55:04.824 INFO:journalctl@ceph.mon.c.smithi131.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:55:04.824 INFO:journalctl@ceph.mon.c.smithi131.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:55:04.824 INFO:journalctl@ceph.mon.c.smithi131.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T03:55:04.824 INFO:journalctl@ceph.mon.c.smithi131.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:55:04.824 INFO:journalctl@ceph.mon.c.smithi131.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:55:04.824 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-09-21T03:55:04.825 INFO:journalctl@ceph.mon.c.smithi131.stdout: ** Compaction Stats [default] ** 2024-09-21T03:55:04.825 INFO:journalctl@ceph.mon.c.smithi131.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:55:04.825 INFO:journalctl@ceph.mon.c.smithi131.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T03:55:04.825 INFO:journalctl@ceph.mon.c.smithi131.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T03:55:04.825 INFO:journalctl@ceph.mon.c.smithi131.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T03:55:04.825 INFO:journalctl@ceph.mon.c.smithi131.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T03:55:04.825 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-09-21T03:55:04.825 INFO:journalctl@ceph.mon.c.smithi131.stdout: ** Compaction Stats [default] ** 2024-09-21T03:55:04.825 INFO:journalctl@ceph.mon.c.smithi131.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:55:04.825 INFO:journalctl@ceph.mon.c.smithi131.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T03:55:04.825 INFO:journalctl@ceph.mon.c.smithi131.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T03:55:04.825 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-09-21T03:55:04.826 INFO:journalctl@ceph.mon.c.smithi131.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T03:55:04.826 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-09-21T03:55:04.826 INFO:journalctl@ceph.mon.c.smithi131.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:55:04.826 INFO:journalctl@ceph.mon.c.smithi131.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T03:55:04.826 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T03:55:04.826 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T03:55:04.826 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T03:55:04.826 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T03:55:04.826 INFO:journalctl@ceph.mon.c.smithi131.stdout: Cumulative compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:55:04.826 INFO:journalctl@ceph.mon.c.smithi131.stdout: Interval compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:55:04.826 INFO:journalctl@ceph.mon.c.smithi131.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T03:55:04.826 INFO:journalctl@ceph.mon.c.smithi131.stdout: Block cache BinnedLRUCache@0x555efd74d610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-21T03:55:04.826 INFO:journalctl@ceph.mon.c.smithi131.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-21T03:55:04.826 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-09-21T03:55:04.827 INFO:journalctl@ceph.mon.c.smithi131.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T03:55:04.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-21T03:55:04.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: using public_addr v2:172.21.15.131:0/0 -> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] 2024-09-21T03:55:04.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: starting mon.c rank -1 at public addrs [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] at bind addrs [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:55:04.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: mon.c@-1(???) e0 preinit fsid e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:55:04.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: mon.c@-1(synchronizing).mds e1 new map 2024-09-21T03:55:04.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-21T03:55:04.827 INFO:journalctl@ceph.mon.c.smithi131.stdout: e1 2024-09-21T03:55:04.827 INFO:journalctl@ceph.mon.c.smithi131.stdout: btime 2024-09-21T03:53:41:214835+0000 2024-09-21T03:55:04.827 INFO:journalctl@ceph.mon.c.smithi131.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T03:55:04.827 INFO:journalctl@ceph.mon.c.smithi131.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:55:04.827 INFO:journalctl@ceph.mon.c.smithi131.stdout: legacy client fscid: -1 2024-09-21T03:55:04.827 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-09-21T03:55:04.827 INFO:journalctl@ceph.mon.c.smithi131.stdout: No filesystems configured 2024-09-21T03:55:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T03:55:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T03:55:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T03:55:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T03:55:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T03:55:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T03:55:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T03:55:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:55:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:55:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:55:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: Updating smithi119:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:55:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:55:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: Updating smithi119:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:55:04.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:04.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:04.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: Added host smithi131 2024-09-21T03:55:04.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:55:04.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:04.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:04.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:04.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:04.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:55:04.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2434181406' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T03:55:04.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2434181406' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T03:55:04.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:55:04.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:04.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:04.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:04.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:04.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi131", "name": "osd_memory_target"} : dispatch 2024-09-21T03:55:04.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:04.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:55:04.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:04.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: Updating smithi131:/etc/ceph/ceph.conf 2024-09-21T03:55:04.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: Updating smithi131:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:55:04.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:55:04.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:04.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:04.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:04.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:04.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:55:04.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:04.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:55:04.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:04.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:55:04.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:04.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: Updating smithi131:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:55:04.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi064:172.21.15.64=a;smithi119:172.21.15.119=b;smithi131:172.21.15.131=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:55:04.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: Saving service mon spec with placement smithi064:172.21.15.64=a;smithi119:172.21.15.119=b;smithi131:172.21.15.131=c;count:3 2024-09-21T03:55:04.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: Deploying daemon mon.c on smithi131 2024-09-21T03:55:04.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:04.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='client.? 172.21.15.131:0/1293599846' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:55:04.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:04.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: from='client.? 172.21.15.131:0/2877678002' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:55:04.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 ceph-mon[35193]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T03:55:04.832 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 bash[35170]: 0697ec341458496080f3751f3b8ea733520057582b88af2a59b8919896d82e7a 2024-09-21T03:55:04.832 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:04 smithi131 systemd[1]: Started Ceph mon.c for e7c927ba-77cc-11ef-baf5-efdc52797490. 2024-09-21T03:55:05.045 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:55:05.046 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:55:05.299 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:55:09.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:55:09.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: mon.a calling monitor election 2024-09-21T03:55:09.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:55:09.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:09.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:55:09.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:55:09.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: mon.c calling monitor election 2024-09-21T03:55:09.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:09.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:55:09.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:55:09.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:55:09.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:55:09.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: monmap epoch 2 2024-09-21T03:55:09.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: fsid e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:55:09.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: last_changed 2024-09-21T03:55:04.409045+0000 2024-09-21T03:55:09.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: created 2024-09-21T03:53:37.752699+0000 2024-09-21T03:55:09.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: min_mon_release 19 (squid) 2024-09-21T03:55:09.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: election_strategy: 1 2024-09-21T03:55:09.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-21T03:55:09.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: 1: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.c 2024-09-21T03:55:09.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: fsmap 2024-09-21T03:55:09.764 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:55:09.764 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: mgrmap e13: x(active, since 42s) 2024-09-21T03:55:09.764 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: overall HEALTH_OK 2024-09-21T03:55:09.764 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:09.764 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:09.764 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:09.764 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:55:09.764 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:09 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:09.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:55:09.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: mon.a calling monitor election 2024-09-21T03:55:09.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:55:09.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:09.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:55:09.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:55:09.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: mon.c calling monitor election 2024-09-21T03:55:09.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:09.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:55:09.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:55:09.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:55:09.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:55:09.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: monmap epoch 2 2024-09-21T03:55:09.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: fsid e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:55:09.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: last_changed 2024-09-21T03:55:04.409045+0000 2024-09-21T03:55:09.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: created 2024-09-21T03:53:37.752699+0000 2024-09-21T03:55:09.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: min_mon_release 19 (squid) 2024-09-21T03:55:09.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: election_strategy: 1 2024-09-21T03:55:09.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-21T03:55:09.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: 1: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.c 2024-09-21T03:55:09.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: fsmap 2024-09-21T03:55:09.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:55:09.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: mgrmap e13: x(active, since 42s) 2024-09-21T03:55:09.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: overall HEALTH_OK 2024-09-21T03:55:09.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:09.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:09.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:09.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:55:09.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:09 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:10.405 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-21T03:55:10.405 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":2,"fsid":"e7c927ba-77cc-11ef-baf5-efdc52797490","modified":"2024-09-21T03:55:04.409045Z","created":"2024-09-21T03:53:37.752699Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:3300","nonce":0},{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T03:55:10.405 INFO:teuthology.orchestra.run.smithi131.stderr:dumped monmap epoch 2 2024-09-21T03:55:10.681 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:10 smithi131 ceph-mon[35193]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:10.682 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:10 smithi131 ceph-mon[35193]: Deploying daemon mon.b on smithi119 2024-09-21T03:55:10.682 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:10 smithi131 ceph-mon[35193]: from='client.? 172.21.15.131:0/2593332261' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:55:10.682 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:10 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:55:10.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:10 smithi064 ceph-mon[31180]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:10.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:10 smithi064 ceph-mon[31180]: Deploying daemon mon.b on smithi119 2024-09-21T03:55:10.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:10 smithi064 ceph-mon[31180]: from='client.? 172.21.15.131:0/2593332261' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:55:10.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:10 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:55:11.761 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:55:11 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: 2024-09-21T03:55:11.407+0000 7f1c777dd640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-21T03:55:11.936 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:55:11.936 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:55:12.179 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:55:12.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:12 smithi131 ceph-mon[35193]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:12.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:12 smithi064 ceph-mon[31180]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:12.865 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-21T03:55:12.865 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":2,"fsid":"e7c927ba-77cc-11ef-baf5-efdc52797490","modified":"2024-09-21T03:55:04.409045Z","created":"2024-09-21T03:53:37.752699Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:3300","nonce":0},{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T03:55:12.866 INFO:teuthology.orchestra.run.smithi131.stderr:dumped monmap epoch 2 2024-09-21T03:55:13.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 podman[34792]: 2024-09-21 03:55:13.057367383 +0000 UTC m=+0.340468763 container start 4dfbc9648e428f4c05d41cda85e0bbc8b3541ba5003ad1a4923d619e1efc40fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:55:13.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T03:55:13.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T03:55:13.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: pidfile_write: ignore empty --pid-file 2024-09-21T03:55:13.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: load: jerasure load: lrc 2024-09-21T03:55:13.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: RocksDB version: 7.9.2 2024-09-21T03:55:13.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Git sha 0 2024-09-21T03:55:13.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T03:55:13.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: DB SUMMARY 2024-09-21T03:55:13.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: DB Session ID: 3ZKH4PN7NX247IC71PT5 2024-09-21T03:55:13.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: CURRENT file: CURRENT 2024-09-21T03:55:13.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T03:55:13.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-21T03:55:13.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-21T03:55:13.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-21T03:55:13.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.error_if_exists: 0 2024-09-21T03:55:13.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.create_if_missing: 0 2024-09-21T03:55:13.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.paranoid_checks: 1 2024-09-21T03:55:13.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T03:55:13.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T03:55:13.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T03:55:13.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.env: 0x55a370b0ed60 2024-09-21T03:55:13.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T03:55:13.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.info_log: 0x55a376ca1360 2024-09-21T03:55:13.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T03:55:13.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.statistics: (nil) 2024-09-21T03:55:13.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.use_fsync: 0 2024-09-21T03:55:13.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.max_log_file_size: 0 2024-09-21T03:55:13.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T03:55:13.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T03:55:13.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T03:55:13.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T03:55:13.509 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.allow_fallocate: 1 2024-09-21T03:55:13.509 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T03:55:13.509 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T03:55:13.509 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.use_direct_reads: 0 2024-09-21T03:55:13.509 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T03:55:13.509 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T03:55:13.509 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.db_log_dir: 2024-09-21T03:55:13.509 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.wal_dir: 2024-09-21T03:55:13.509 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T03:55:13.509 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T03:55:13.509 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T03:55:13.509 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T03:55:13.511 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T03:55:13.511 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T03:55:13.511 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T03:55:13.511 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T03:55:13.511 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.write_buffer_manager: 0x55a376ca5d60 2024-09-21T03:55:13.511 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T03:55:13.511 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T03:55:13.511 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T03:55:13.511 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T03:55:13.511 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T03:55:13.511 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T03:55:13.512 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T03:55:13.512 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T03:55:13.512 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.unordered_write: 0 2024-09-21T03:55:13.512 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T03:55:13.512 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T03:55:13.512 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T03:55:13.512 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T03:55:13.512 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.row_cache: None 2024-09-21T03:55:13.512 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.wal_filter: None 2024-09-21T03:55:13.512 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T03:55:13.512 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T03:55:13.512 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.two_write_queues: 0 2024-09-21T03:55:13.512 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T03:55:13.512 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.wal_compression: 0 2024-09-21T03:55:13.513 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.atomic_flush: 0 2024-09-21T03:55:13.513 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T03:55:13.513 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T03:55:13.513 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T03:55:13.513 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.log_readahead_size: 0 2024-09-21T03:55:13.513 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T03:55:13.513 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T03:55:13.513 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T03:55:13.513 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T03:55:13.513 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T03:55:13.513 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T03:55:13.513 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T03:55:13.513 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.max_background_jobs: 2 2024-09-21T03:55:13.513 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.max_background_compactions: -1 2024-09-21T03:55:13.513 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.max_subcompactions: 1 2024-09-21T03:55:13.514 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T03:55:13.514 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T03:55:13.514 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T03:55:13.514 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T03:55:13.514 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T03:55:13.514 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T03:55:13.514 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T03:55:13.514 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T03:55:13.514 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.max_open_files: -1 2024-09-21T03:55:13.514 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T03:55:13.514 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T03:55:13.514 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T03:55:13.514 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T03:55:13.514 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.max_background_flushes: -1 2024-09-21T03:55:13.514 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Compression algorithms supported: 2024-09-21T03:55:13.515 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: kZSTD supported: 0 2024-09-21T03:55:13.515 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: kXpressCompression supported: 0 2024-09-21T03:55:13.515 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: kBZip2Compression supported: 0 2024-09-21T03:55:13.515 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T03:55:13.515 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: kLZ4Compression supported: 1 2024-09-21T03:55:13.515 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: kZlibCompression supported: 1 2024-09-21T03:55:13.515 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T03:55:13.515 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: kSnappyCompression supported: 1 2024-09-21T03:55:13.515 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T03:55:13.515 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T03:55:13.515 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-21T03:55:13.515 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T03:55:13.515 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T03:55:13.515 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.merge_operator: 2024-09-21T03:55:13.515 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.compaction_filter: None 2024-09-21T03:55:13.516 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T03:55:13.516 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T03:55:13.516 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T03:55:13.516 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T03:55:13.516 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a376ca0fa0) 2024-09-21T03:55:13.516 INFO:journalctl@ceph.mon.b.smithi119.stdout: cache_index_and_filter_blocks: 1 2024-09-21T03:55:13.516 INFO:journalctl@ceph.mon.b.smithi119.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T03:55:13.516 INFO:journalctl@ceph.mon.b.smithi119.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T03:55:13.516 INFO:journalctl@ceph.mon.b.smithi119.stdout: pin_top_level_index_and_filter: 1 2024-09-21T03:55:13.516 INFO:journalctl@ceph.mon.b.smithi119.stdout: index_type: 0 2024-09-21T03:55:13.516 INFO:journalctl@ceph.mon.b.smithi119.stdout: data_block_index_type: 0 2024-09-21T03:55:13.516 INFO:journalctl@ceph.mon.b.smithi119.stdout: index_shortening: 1 2024-09-21T03:55:13.516 INFO:journalctl@ceph.mon.b.smithi119.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T03:55:13.516 INFO:journalctl@ceph.mon.b.smithi119.stdout: checksum: 4 2024-09-21T03:55:13.517 INFO:journalctl@ceph.mon.b.smithi119.stdout: no_block_cache: 0 2024-09-21T03:55:13.517 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_cache: 0x55a372c65610 2024-09-21T03:55:13.517 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_cache_name: BinnedLRUCache 2024-09-21T03:55:13.517 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_cache_options: 2024-09-21T03:55:13.517 INFO:journalctl@ceph.mon.b.smithi119.stdout: capacity : 536870912 2024-09-21T03:55:13.517 INFO:journalctl@ceph.mon.b.smithi119.stdout: num_shard_bits : 4 2024-09-21T03:55:13.517 INFO:journalctl@ceph.mon.b.smithi119.stdout: strict_capacity_limit : 0 2024-09-21T03:55:13.517 INFO:journalctl@ceph.mon.b.smithi119.stdout: high_pri_pool_ratio: 0.000 2024-09-21T03:55:13.517 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_cache_compressed: (nil) 2024-09-21T03:55:13.517 INFO:journalctl@ceph.mon.b.smithi119.stdout: persistent_cache: (nil) 2024-09-21T03:55:13.517 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_size: 4096 2024-09-21T03:55:13.517 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_size_deviation: 10 2024-09-21T03:55:13.517 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_restart_interval: 16 2024-09-21T03:55:13.517 INFO:journalctl@ceph.mon.b.smithi119.stdout: index_block_restart_interval: 1 2024-09-21T03:55:13.517 INFO:journalctl@ceph.mon.b.smithi119.stdout: metadata_block_size: 4096 2024-09-21T03:55:13.517 INFO:journalctl@ceph.mon.b.smithi119.stdout: partition_filters: 0 2024-09-21T03:55:13.518 INFO:journalctl@ceph.mon.b.smithi119.stdout: use_delta_encoding: 1 2024-09-21T03:55:13.518 INFO:journalctl@ceph.mon.b.smithi119.stdout: filter_policy: bloomfilter 2024-09-21T03:55:13.518 INFO:journalctl@ceph.mon.b.smithi119.stdout: whole_key_filtering: 1 2024-09-21T03:55:13.518 INFO:journalctl@ceph.mon.b.smithi119.stdout: verify_compression: 0 2024-09-21T03:55:13.518 INFO:journalctl@ceph.mon.b.smithi119.stdout: read_amp_bytes_per_bit: 0 2024-09-21T03:55:13.518 INFO:journalctl@ceph.mon.b.smithi119.stdout: format_version: 5 2024-09-21T03:55:13.518 INFO:journalctl@ceph.mon.b.smithi119.stdout: enable_index_compression: 1 2024-09-21T03:55:13.518 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_align: 0 2024-09-21T03:55:13.518 INFO:journalctl@ceph.mon.b.smithi119.stdout: max_auto_readahead_size: 262144 2024-09-21T03:55:13.518 INFO:journalctl@ceph.mon.b.smithi119.stdout: prepopulate_block_cache: 0 2024-09-21T03:55:13.518 INFO:journalctl@ceph.mon.b.smithi119.stdout: initial_auto_readahead_size: 8192 2024-09-21T03:55:13.518 INFO:journalctl@ceph.mon.b.smithi119.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T03:55:13.518 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T03:55:13.518 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T03:55:13.518 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.compression: NoCompression 2024-09-21T03:55:13.519 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T03:55:13.519 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T03:55:13.519 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T03:55:13.519 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.num_levels: 7 2024-09-21T03:55:13.519 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T03:55:13.519 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T03:55:13.519 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T03:55:13.519 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T03:55:13.519 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T03:55:13.519 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T03:55:13.519 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T03:55:13.519 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:55:13.519 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T03:55:13.519 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T03:55:13.520 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:55:13.520 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:55:13.520 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T03:55:13.520 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T03:55:13.520 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T03:55:13.520 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T03:55:13.520 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T03:55:13.520 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T03:55:13.520 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T03:55:13.520 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T03:55:13.520 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T03:55:13.520 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T03:55:13.520 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T03:55:13.520 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T03:55:13.520 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T03:55:13.521 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T03:55:13.521 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T03:55:13.521 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T03:55:13.521 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T03:55:13.521 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T03:55:13.521 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T03:55:13.521 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T03:55:13.521 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T03:55:13.521 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T03:55:13.521 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T03:55:13.521 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T03:55:13.521 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T03:55:13.521 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T03:55:13.521 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T03:55:13.521 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T03:55:13.522 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T03:55:13.522 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T03:55:13.522 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T03:55:13.522 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T03:55:13.522 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T03:55:13.522 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T03:55:13.522 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T03:55:13.522 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T03:55:13.522 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T03:55:13.522 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T03:55:13.522 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T03:55:13.522 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T03:55:13.522 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T03:55:13.522 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T03:55:13.522 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.inplace_update_support: 0 2024-09-21T03:55:13.523 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T03:55:13.523 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T03:55:13.523 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T03:55:13.523 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T03:55:13.523 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.bloom_locality: 0 2024-09-21T03:55:13.523 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.max_successive_merges: 0 2024-09-21T03:55:13.523 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T03:55:13.523 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T03:55:13.523 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T03:55:13.523 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T03:55:13.523 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.ttl: 2592000 2024-09-21T03:55:13.523 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T03:55:13.523 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T03:55:13.523 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T03:55:13.523 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.enable_blob_files: false 2024-09-21T03:55:13.524 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.min_blob_size: 0 2024-09-21T03:55:13.524 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T03:55:13.524 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T03:55:13.524 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T03:55:13.524 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T03:55:13.524 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T03:55:13.524 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T03:55:13.524 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T03:55:13.524 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T03:55:13.524 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-21T03:55:13.524 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-21T03:55:13.524 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 040c97ca-a93f-454b-a687-b2a2b0df607f 2024-09-21T03:55:13.524 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726890913094438, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-21T03:55:13.524 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-21T03:55:13.525 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726890913094858, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726890913, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "040c97ca-a93f-454b-a687-b2a2b0df607f", "db_session_id": "3ZKH4PN7NX247IC71PT5", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-21T03:55:13.525 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726890913094950, "job": 1, "event": "recovery_finished"} 2024-09-21T03:55:13.525 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-21T03:55:13.525 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T03:55:13.525 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a376cc4a80 2024-09-21T03:55:13.525 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: DB pointer 0x55a377a6a000 2024-09-21T03:55:13.525 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T03:55:13.525 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T03:55:13.525 INFO:journalctl@ceph.mon.b.smithi119.stdout: ** DB Stats ** 2024-09-21T03:55:13.525 INFO:journalctl@ceph.mon.b.smithi119.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:55:13.525 INFO:journalctl@ceph.mon.b.smithi119.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T03:55:13.525 INFO:journalctl@ceph.mon.b.smithi119.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:55:13.525 INFO:journalctl@ceph.mon.b.smithi119.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:55:13.525 INFO:journalctl@ceph.mon.b.smithi119.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T03:55:13.525 INFO:journalctl@ceph.mon.b.smithi119.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T03:55:13.525 INFO:journalctl@ceph.mon.b.smithi119.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T03:55:13.526 INFO:journalctl@ceph.mon.b.smithi119.stdout: 2024-09-21T03:55:13.526 INFO:journalctl@ceph.mon.b.smithi119.stdout: ** Compaction Stats [default] ** 2024-09-21T03:55:13.526 INFO:journalctl@ceph.mon.b.smithi119.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:55:13.526 INFO:journalctl@ceph.mon.b.smithi119.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T03:55:13.526 INFO:journalctl@ceph.mon.b.smithi119.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T03:55:13.526 INFO:journalctl@ceph.mon.b.smithi119.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T03:55:13.526 INFO:journalctl@ceph.mon.b.smithi119.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T03:55:13.526 INFO:journalctl@ceph.mon.b.smithi119.stdout: 2024-09-21T03:55:13.526 INFO:journalctl@ceph.mon.b.smithi119.stdout: ** Compaction Stats [default] ** 2024-09-21T03:55:13.526 INFO:journalctl@ceph.mon.b.smithi119.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T03:55:13.526 INFO:journalctl@ceph.mon.b.smithi119.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T03:55:13.526 INFO:journalctl@ceph.mon.b.smithi119.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T03:55:13.526 INFO:journalctl@ceph.mon.b.smithi119.stdout: 2024-09-21T03:55:13.527 INFO:journalctl@ceph.mon.b.smithi119.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T03:55:13.527 INFO:journalctl@ceph.mon.b.smithi119.stdout: 2024-09-21T03:55:13.527 INFO:journalctl@ceph.mon.b.smithi119.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T03:55:13.527 INFO:journalctl@ceph.mon.b.smithi119.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T03:55:13.527 INFO:journalctl@ceph.mon.b.smithi119.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T03:55:13.527 INFO:journalctl@ceph.mon.b.smithi119.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T03:55:13.527 INFO:journalctl@ceph.mon.b.smithi119.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T03:55:13.527 INFO:journalctl@ceph.mon.b.smithi119.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T03:55:13.527 INFO:journalctl@ceph.mon.b.smithi119.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:55:13.527 INFO:journalctl@ceph.mon.b.smithi119.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T03:55:13.527 INFO:journalctl@ceph.mon.b.smithi119.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T03:55:13.527 INFO:journalctl@ceph.mon.b.smithi119.stdout: Block cache BinnedLRUCache@0x55a372c65610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-21T03:55:13.527 INFO:journalctl@ceph.mon.b.smithi119.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-21T03:55:13.527 INFO:journalctl@ceph.mon.b.smithi119.stdout: 2024-09-21T03:55:13.527 INFO:journalctl@ceph.mon.b.smithi119.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T03:55:13.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-21T03:55:13.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: using public_addr v2:172.21.15.119:0/0 -> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-09-21T03:55:13.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: starting mon.b rank -1 at public addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] at bind addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:55:13.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: mon.b@-1(???) e0 preinit fsid e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:55:13.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: mon.b@-1(synchronizing).mds e1 new map 2024-09-21T03:55:13.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-21T03:55:13.528 INFO:journalctl@ceph.mon.b.smithi119.stdout: e1 2024-09-21T03:55:13.528 INFO:journalctl@ceph.mon.b.smithi119.stdout: btime 2024-09-21T03:53:41:214835+0000 2024-09-21T03:55:13.528 INFO:journalctl@ceph.mon.b.smithi119.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T03:55:13.528 INFO:journalctl@ceph.mon.b.smithi119.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T03:55:13.528 INFO:journalctl@ceph.mon.b.smithi119.stdout: legacy client fscid: -1 2024-09-21T03:55:13.528 INFO:journalctl@ceph.mon.b.smithi119.stdout: 2024-09-21T03:55:13.528 INFO:journalctl@ceph.mon.b.smithi119.stdout: No filesystems configured 2024-09-21T03:55:13.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T03:55:13.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T03:55:13.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T03:55:13.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T03:55:13.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T03:55:13.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T03:55:13.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T03:55:13.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:55:13.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:55:13.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T03:55:13.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: Updating smithi119:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:55:13.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:55:13.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:13.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: Updating smithi119:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:55:13.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:13.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:13.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: Added host smithi131 2024-09-21T03:55:13.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:55:13.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:13.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:13.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:13.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:13.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:55:13.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2434181406' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T03:55:13.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2434181406' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T03:55:13.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:55:13.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:13.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:13.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:13.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:13.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi131", "name": "osd_memory_target"} : dispatch 2024-09-21T03:55:13.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:13.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:55:13.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:13.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: Updating smithi131:/etc/ceph/ceph.conf 2024-09-21T03:55:13.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: Updating smithi131:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:55:13.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:55:13.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:13.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:13.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:13.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:13.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:55:13.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:13.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:55:13.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:13.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:55:13.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:13.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: Updating smithi131:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:55:13.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi064:172.21.15.64=a;smithi119:172.21.15.119=b;smithi131:172.21.15.131=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:55:13.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: Saving service mon spec with placement smithi064:172.21.15.64=a;smithi119:172.21.15.119=b;smithi131:172.21.15.131=c;count:3 2024-09-21T03:55:13.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: Deploying daemon mon.c on smithi131 2024-09-21T03:55:13.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:13.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='client.? 172.21.15.131:0/1293599846' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:55:13.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:13.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='client.? 172.21.15.131:0/2877678002' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:55:13.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:55:13.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: mon.a calling monitor election 2024-09-21T03:55:13.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:55:13.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:13.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:55:13.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:55:13.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: mon.c calling monitor election 2024-09-21T03:55:13.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:13.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:55:13.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:55:13.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:55:13.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:55:13.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: monmap epoch 2 2024-09-21T03:55:13.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: fsid e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:55:13.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: last_changed 2024-09-21T03:55:04.409045+0000 2024-09-21T03:55:13.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: created 2024-09-21T03:53:37.752699+0000 2024-09-21T03:55:13.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: min_mon_release 19 (squid) 2024-09-21T03:55:13.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: election_strategy: 1 2024-09-21T03:55:13.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-21T03:55:13.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: 1: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.c 2024-09-21T03:55:13.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: fsmap 2024-09-21T03:55:13.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:55:13.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: mgrmap e13: x(active, since 42s) 2024-09-21T03:55:13.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: overall HEALTH_OK 2024-09-21T03:55:13.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:13.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:13.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:13.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:55:13.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:13.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:13.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: Deploying daemon mon.b on smithi119 2024-09-21T03:55:13.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='client.? 172.21.15.131:0/2593332261' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:55:13.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:55:13.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:13.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 ceph-mon[34811]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T03:55:13.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 bash[34792]: 4dfbc9648e428f4c05d41cda85e0bbc8b3541ba5003ad1a4923d619e1efc40fd 2024-09-21T03:55:13.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:13 smithi119 systemd[1]: Started Ceph mon.b for e7c927ba-77cc-11ef-baf5-efdc52797490. 2024-09-21T03:55:14.466 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T03:55:14.466 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T03:55:14.718 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:55:18.441 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-21T03:55:18.441 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":3,"fsid":"e7c927ba-77cc-11ef-baf5-efdc52797490","modified":"2024-09-21T03:55:13.123024Z","created":"2024-09-21T03:53:37.752699Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:3300","nonce":0},{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T03:55:18.441 INFO:teuthology.orchestra.run.smithi131.stderr:dumped monmap epoch 3 2024-09-21T03:55:18.441 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:55:18.441 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:55:18.441 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: mon.a calling monitor election 2024-09-21T03:55:18.442 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: mon.c calling monitor election 2024-09-21T03:55:18.442 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:55:18.442 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:18.442 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:55:18.442 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:55:18.442 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:18.442 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:55:18.442 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:55:18.442 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:18.442 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:55:18.442 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:55:18.442 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: monmap epoch 3 2024-09-21T03:55:18.443 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: fsid e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:55:18.443 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: last_changed 2024-09-21T03:55:13.123024+0000 2024-09-21T03:55:18.443 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: created 2024-09-21T03:53:37.752699+0000 2024-09-21T03:55:18.443 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: min_mon_release 19 (squid) 2024-09-21T03:55:18.443 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: election_strategy: 1 2024-09-21T03:55:18.443 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-21T03:55:18.443 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: 1: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.c 2024-09-21T03:55:18.443 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: 2: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-21T03:55:18.443 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: fsmap 2024-09-21T03:55:18.443 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:55:18.443 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: mgrmap e13: x(active, since 50s) 2024-09-21T03:55:18.443 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: overall HEALTH_OK 2024-09-21T03:55:18.443 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:18.444 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:18.444 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:18.444 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:18 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:18.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:55:18.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:55:18.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: mon.a calling monitor election 2024-09-21T03:55:18.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: mon.c calling monitor election 2024-09-21T03:55:18.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:55:18.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:18.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:55:18.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:55:18.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:18.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:55:18.513 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:55:18.513 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:18.513 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:55:18.513 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:55:18.513 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: monmap epoch 3 2024-09-21T03:55:18.513 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: fsid e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:55:18.513 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: last_changed 2024-09-21T03:55:13.123024+0000 2024-09-21T03:55:18.513 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: created 2024-09-21T03:53:37.752699+0000 2024-09-21T03:55:18.513 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: min_mon_release 19 (squid) 2024-09-21T03:55:18.513 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: election_strategy: 1 2024-09-21T03:55:18.513 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-21T03:55:18.513 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: 1: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.c 2024-09-21T03:55:18.513 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: 2: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-21T03:55:18.513 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: fsmap 2024-09-21T03:55:18.513 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:55:18.514 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: mgrmap e13: x(active, since 50s) 2024-09-21T03:55:18.514 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: overall HEALTH_OK 2024-09-21T03:55:18.514 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:18.514 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:18.514 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:18.514 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:18 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:19.057 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-21T03:55:19.057 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config generate-minimal-conf 2024-09-21T03:55:19.313 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:19 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:55:19.313 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:19 smithi131 ceph-mon[35193]: from='client.? 172.21.15.131:0/2211984894' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:55:19.314 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:19 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:19.314 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:19 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:19.314 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:19 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:55:19.326 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:19.447 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:19 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:55:19.447 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:19 smithi064 ceph-mon[31180]: from='client.? 172.21.15.131:0/2211984894' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:55:19.447 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:19 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:19.447 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:19 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:19.447 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:19 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:55:20.009 INFO:teuthology.orchestra.run.smithi064.stdout:# minimal ceph.conf for e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:55:20.010 INFO:teuthology.orchestra.run.smithi064.stdout:[global] 2024-09-21T03:55:20.010 INFO:teuthology.orchestra.run.smithi064.stdout: fsid = e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:55:20.010 INFO:teuthology.orchestra.run.smithi064.stdout: mon_host = [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] 2024-09-21T03:55:20.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:55:20.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:55:20.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: mon.a calling monitor election 2024-09-21T03:55:20.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: mon.c calling monitor election 2024-09-21T03:55:20.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:55:20.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:20.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:55:20.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:55:20.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:20.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:55:20.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:55:20.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:20.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:55:20.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T03:55:20.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: monmap epoch 3 2024-09-21T03:55:20.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: fsid e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:55:20.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: last_changed 2024-09-21T03:55:13.123024+0000 2024-09-21T03:55:20.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: created 2024-09-21T03:53:37.752699+0000 2024-09-21T03:55:20.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: min_mon_release 19 (squid) 2024-09-21T03:55:20.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: election_strategy: 1 2024-09-21T03:55:20.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-21T03:55:20.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: 1: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.c 2024-09-21T03:55:20.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: 2: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-21T03:55:20.509 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: fsmap 2024-09-21T03:55:20.509 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:55:20.509 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: mgrmap e13: x(active, since 50s) 2024-09-21T03:55:20.509 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: overall HEALTH_OK 2024-09-21T03:55:20.509 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:20.509 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:20.509 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:20.509 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:20.509 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:55:20.509 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: from='client.? 172.21.15.131:0/2211984894' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T03:55:20.509 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:20.509 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:20.509 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:20 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:55:20.576 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-21T03:55:20.577 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-21T03:55:20.577 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T03:55:20.605 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-21T03:55:20.605 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:55:20.671 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-21T03:55:20.671 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T03:55:20.701 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-21T03:55:20.701 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:55:20.767 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-21T03:55:20.767 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T03:55:20.796 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-21T03:55:20.796 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:55:20.861 INFO:tasks.cephadm:enabling module snap_schedule 2024-09-21T03:55:20.862 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- sudo ceph --cluster ceph mgr module enable snap_schedule 2024-09-21T03:55:21.466 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:21 smithi064 ceph-mon[31180]: mon.b calling monitor election 2024-09-21T03:55:21.466 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:21 smithi064 ceph-mon[31180]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T03:55:21.466 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:21 smithi064 ceph-mon[31180]: Reconfiguring daemon mon.a on smithi064 2024-09-21T03:55:21.467 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:21 smithi064 ceph-mon[31180]: mon.b calling monitor election 2024-09-21T03:55:21.467 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:21 smithi064 ceph-mon[31180]: mon.c calling monitor election 2024-09-21T03:55:21.467 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:21 smithi064 ceph-mon[31180]: mon.a calling monitor election 2024-09-21T03:55:21.467 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:21 smithi064 ceph-mon[31180]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T03:55:21.467 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:21 smithi064 ceph-mon[31180]: monmap epoch 3 2024-09-21T03:55:21.467 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:21 smithi064 ceph-mon[31180]: fsid e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:55:21.467 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:21 smithi064 ceph-mon[31180]: last_changed 2024-09-21T03:55:13.123024+0000 2024-09-21T03:55:21.467 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:21 smithi064 ceph-mon[31180]: created 2024-09-21T03:53:37.752699+0000 2024-09-21T03:55:21.467 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:21 smithi064 ceph-mon[31180]: min_mon_release 19 (squid) 2024-09-21T03:55:21.467 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:21 smithi064 ceph-mon[31180]: election_strategy: 1 2024-09-21T03:55:21.467 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:21 smithi064 ceph-mon[31180]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-21T03:55:21.467 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:21 smithi064 ceph-mon[31180]: 1: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.c 2024-09-21T03:55:21.467 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:21 smithi064 ceph-mon[31180]: 2: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-21T03:55:21.467 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:21 smithi064 ceph-mon[31180]: fsmap 2024-09-21T03:55:21.468 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:21 smithi064 ceph-mon[31180]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:55:21.468 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:21 smithi064 ceph-mon[31180]: mgrmap e13: x(active, since 52s) 2024-09-21T03:55:21.468 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:21 smithi064 ceph-mon[31180]: overall HEALTH_OK 2024-09-21T03:55:21.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:21 smithi119 ceph-mon[34811]: mon.b calling monitor election 2024-09-21T03:55:21.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:21 smithi119 ceph-mon[34811]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T03:55:21.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:21 smithi119 ceph-mon[34811]: Reconfiguring daemon mon.a on smithi064 2024-09-21T03:55:21.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:21 smithi119 ceph-mon[34811]: mon.b calling monitor election 2024-09-21T03:55:21.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:21 smithi119 ceph-mon[34811]: mon.c calling monitor election 2024-09-21T03:55:21.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:21 smithi119 ceph-mon[34811]: mon.a calling monitor election 2024-09-21T03:55:21.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:21 smithi119 ceph-mon[34811]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T03:55:21.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:21 smithi119 ceph-mon[34811]: monmap epoch 3 2024-09-21T03:55:21.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:21 smithi119 ceph-mon[34811]: fsid e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:55:21.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:21 smithi119 ceph-mon[34811]: last_changed 2024-09-21T03:55:13.123024+0000 2024-09-21T03:55:21.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:21 smithi119 ceph-mon[34811]: created 2024-09-21T03:53:37.752699+0000 2024-09-21T03:55:21.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:21 smithi119 ceph-mon[34811]: min_mon_release 19 (squid) 2024-09-21T03:55:21.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:21 smithi119 ceph-mon[34811]: election_strategy: 1 2024-09-21T03:55:21.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:21 smithi119 ceph-mon[34811]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-21T03:55:21.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:21 smithi119 ceph-mon[34811]: 1: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.c 2024-09-21T03:55:21.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:21 smithi119 ceph-mon[34811]: 2: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-21T03:55:21.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:21 smithi119 ceph-mon[34811]: fsmap 2024-09-21T03:55:21.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:21 smithi119 ceph-mon[34811]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:55:21.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:21 smithi119 ceph-mon[34811]: mgrmap e13: x(active, since 52s) 2024-09-21T03:55:21.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:21 smithi119 ceph-mon[34811]: overall HEALTH_OK 2024-09-21T03:55:21.553 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:21.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:21 smithi131 ceph-mon[35193]: mon.b calling monitor election 2024-09-21T03:55:21.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:21 smithi131 ceph-mon[35193]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T03:55:21.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:21 smithi131 ceph-mon[35193]: Reconfiguring daemon mon.a on smithi064 2024-09-21T03:55:21.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:21 smithi131 ceph-mon[35193]: mon.b calling monitor election 2024-09-21T03:55:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:21 smithi131 ceph-mon[35193]: mon.c calling monitor election 2024-09-21T03:55:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:21 smithi131 ceph-mon[35193]: mon.a calling monitor election 2024-09-21T03:55:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:21 smithi131 ceph-mon[35193]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T03:55:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:21 smithi131 ceph-mon[35193]: monmap epoch 3 2024-09-21T03:55:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:21 smithi131 ceph-mon[35193]: fsid e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T03:55:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:21 smithi131 ceph-mon[35193]: last_changed 2024-09-21T03:55:13.123024+0000 2024-09-21T03:55:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:21 smithi131 ceph-mon[35193]: created 2024-09-21T03:53:37.752699+0000 2024-09-21T03:55:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:21 smithi131 ceph-mon[35193]: min_mon_release 19 (squid) 2024-09-21T03:55:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:21 smithi131 ceph-mon[35193]: election_strategy: 1 2024-09-21T03:55:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:21 smithi131 ceph-mon[35193]: 0: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.a 2024-09-21T03:55:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:21 smithi131 ceph-mon[35193]: 1: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.c 2024-09-21T03:55:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:21 smithi131 ceph-mon[35193]: 2: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-09-21T03:55:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:21 smithi131 ceph-mon[35193]: fsmap 2024-09-21T03:55:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:21 smithi131 ceph-mon[35193]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T03:55:21.555 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:21 smithi131 ceph-mon[35193]: mgrmap e13: x(active, since 52s) 2024-09-21T03:55:21.555 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:21 smithi131 ceph-mon[35193]: overall HEALTH_OK 2024-09-21T03:55:22.207 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:22 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:55:22.207 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:22 smithi119 ceph-mon[34811]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:22.207 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:22 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:22.207 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:22 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:22.207 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:22 smithi119 ceph-mon[34811]: Reconfiguring mon.b (monmap changed)... 2024-09-21T03:55:22.207 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:22 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:55:22.207 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:22 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:55:22.208 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:22 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:22.208 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:22 smithi119 ceph-mon[34811]: Reconfiguring daemon mon.b on smithi119 2024-09-21T03:55:22.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:22 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:55:22.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:22 smithi064 ceph-mon[31180]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:22.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:22 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:22.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:22 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:22.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:22 smithi064 ceph-mon[31180]: Reconfiguring mon.b (monmap changed)... 2024-09-21T03:55:22.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:22 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:55:22.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:22 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:55:22.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:22 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:22.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:22 smithi064 ceph-mon[31180]: Reconfiguring daemon mon.b on smithi119 2024-09-21T03:55:22.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:22 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:55:22.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:22 smithi131 ceph-mon[35193]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:22.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:22 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:22.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:22 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:22.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:22 smithi131 ceph-mon[35193]: Reconfiguring mon.b (monmap changed)... 2024-09-21T03:55:22.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:22 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:55:22.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:22 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:55:22.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:22 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:22.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:22 smithi131 ceph-mon[35193]: Reconfiguring daemon mon.b on smithi119 2024-09-21T03:55:23.342 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:23 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1006484810' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T03:55:23.342 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:23 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:23.343 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:23 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:23.343 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:23 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:23.343 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:23 smithi131 ceph-mon[35193]: Reconfiguring mon.c (monmap changed)... 2024-09-21T03:55:23.343 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:23 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:55:23.343 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:23 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:55:23.343 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:23 smithi131 ceph-mon[35193]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:23.343 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:23 smithi131 ceph-mon[35193]: Reconfiguring daemon mon.c on smithi131 2024-09-21T03:55:23.436 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:55:23 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: ignoring --setuser ceph since I am not root 2024-09-21T03:55:23.436 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:55:23 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: ignoring --setgroup ceph since I am not root 2024-09-21T03:55:23.436 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:23 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1006484810' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T03:55:23.436 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:23 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:23.436 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:23 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:23.436 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:23 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:23.436 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:23 smithi064 ceph-mon[31180]: Reconfiguring mon.c (monmap changed)... 2024-09-21T03:55:23.436 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:23 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:55:23.436 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:23 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:55:23.436 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:23 smithi064 ceph-mon[31180]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:23.437 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:23 smithi064 ceph-mon[31180]: Reconfiguring daemon mon.c on smithi131 2024-09-21T03:55:23.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:23 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1006484810' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T03:55:23.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:23 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:23.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:23 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:23.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:23 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' 2024-09-21T03:55:23.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:23 smithi119 ceph-mon[34811]: Reconfiguring mon.c (monmap changed)... 2024-09-21T03:55:23.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:23 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:55:23.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:23 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:55:23.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:23 smithi119 ceph-mon[34811]: from='mgr.14152 172.21.15.64:0/2686660536' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:23.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:23 smithi119 ceph-mon[34811]: Reconfiguring daemon mon.c on smithi131 2024-09-21T03:55:23.744 INFO:tasks.cephadm:Adding mgr.x on smithi064 2024-09-21T03:55:23.744 INFO:tasks.cephadm:Adding mgr.y on smithi119 2024-09-21T03:55:23.745 INFO:tasks.cephadm:Adding mgr.z on smithi131 2024-09-21T03:55:23.745 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph orch apply mgr '3;smithi064=x;smithi119=y;smithi131=z' 2024-09-21T03:55:24.221 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:55:24.345 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:24 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1006484810' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T03:55:24.345 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:24 smithi131 ceph-mon[35193]: mgrmap e14: x(active, since 55s) 2024-09-21T03:55:24.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:24 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1006484810' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T03:55:24.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:24 smithi119 ceph-mon[34811]: mgrmap e14: x(active, since 55s) 2024-09-21T03:55:24.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:24 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1006484810' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T03:55:24.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:24 smithi064 ceph-mon[31180]: mgrmap e14: x(active, since 55s) 2024-09-21T03:55:28.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:28 smithi119 ceph-mon[34811]: Active manager daemon x restarted 2024-09-21T03:55:28.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:28 smithi119 ceph-mon[34811]: Activating manager daemon x 2024-09-21T03:55:28.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:28 smithi119 ceph-mon[34811]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T03:55:28.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:28 smithi119 ceph-mon[34811]: mgrmap e15: x(active, starting, since 0.00725842s) 2024-09-21T03:55:28.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:28 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:55:28.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:28 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:55:28.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:28 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:55:28.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:28 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:55:28.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:28 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:55:28.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:28 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:55:28.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:28 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:55:28.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:28 smithi119 ceph-mon[34811]: Manager daemon x is now available 2024-09-21T03:55:28.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:28 smithi064 ceph-mon[31180]: Active manager daemon x restarted 2024-09-21T03:55:28.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:28 smithi064 ceph-mon[31180]: Activating manager daemon x 2024-09-21T03:55:28.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:28 smithi064 ceph-mon[31180]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T03:55:28.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:28 smithi064 ceph-mon[31180]: mgrmap e15: x(active, starting, since 0.00725842s) 2024-09-21T03:55:28.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:28 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:55:28.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:28 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:55:28.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:28 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:55:28.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:28 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:55:28.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:28 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:55:28.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:28 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:55:28.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:28 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:55:28.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:28 smithi064 ceph-mon[31180]: Manager daemon x is now available 2024-09-21T03:55:28.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:28 smithi131 ceph-mon[35193]: Active manager daemon x restarted 2024-09-21T03:55:28.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:28 smithi131 ceph-mon[35193]: Activating manager daemon x 2024-09-21T03:55:28.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:28 smithi131 ceph-mon[35193]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T03:55:28.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:28 smithi131 ceph-mon[35193]: mgrmap e15: x(active, starting, since 0.00725842s) 2024-09-21T03:55:28.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:28 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:55:28.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:28 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:55:28.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:28 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:55:28.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:28 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T03:55:28.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:28 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T03:55:28.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:28 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T03:55:28.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:28 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T03:55:28.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:28 smithi131 ceph-mon[35193]: Manager daemon x is now available 2024-09-21T03:55:29.416 INFO:teuthology.orchestra.run.smithi131.stdout:Scheduled mgr update... 2024-09-21T03:55:29.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:55:29.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:55:29.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:55:29.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:29 smithi119 ceph-mon[34811]: mgrmap e16: x(active, since 1.0111s) 2024-09-21T03:55:29.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:29.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:55:29.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:55:29.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:55:29.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:29 smithi064 ceph-mon[31180]: mgrmap e16: x(active, since 1.0111s) 2024-09-21T03:55:29.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:29.761 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:55:29.762 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T03:55:29.762 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T03:55:29.762 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:29 smithi131 ceph-mon[35193]: mgrmap e16: x(active, since 1.0111s) 2024-09-21T03:55:29.762 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:30.056 DEBUG:teuthology.orchestra.run.smithi119:mgr.y> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mgr.y.service 2024-09-21T03:55:30.059 DEBUG:teuthology.orchestra.run.smithi131:mgr.z> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mgr.z.service 2024-09-21T03:55:30.062 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-21T03:55:30.062 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set client client_mount_timeout 600 2024-09-21T03:55:30.065 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-21T03:55:30.065 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set client debug_client 20 2024-09-21T03:55:30.109 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-21T03:55:30.109 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set client debug_ms 1 2024-09-21T03:55:30.152 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-21T03:55:30.152 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set client rados_mon_op_timeout 900 2024-09-21T03:55:30.195 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-21T03:55:30.195 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set client rados_osd_op_timeout 900 2024-09-21T03:55:30.238 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-21T03:55:30.238 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds 20 2024-09-21T03:55:30.281 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-21T03:55:30.281 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds_balancer 20 2024-09-21T03:55:30.336 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-21T03:55:30.336 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set mds debug_ms 2 2024-09-21T03:55:30.388 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-21T03:55:30.388 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-09-21T03:55:30.432 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-09-21T03:55:30.432 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_merge_size 5 2024-09-21T03:55:30.485 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-09-21T03:55:30.486 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_bits 3 2024-09-21T03:55:30.539 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-09-21T03:55:30.540 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_size 100 2024-09-21T03:55:30.550 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:30.559 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-21T03:55:30.559 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_frag True 2024-09-21T03:55:30.610 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-21T03:55:30.610 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_scatterstat True 2024-09-21T03:55:30.651 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:30.664 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-21T03:55:30.664 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set mds mds_op_complaint_time 180 2024-09-21T03:55:30.710 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-21T03:55:30.711 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set mds mds_verify_scatter True 2024-09-21T03:55:30.768 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-21T03:55:30.768 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set mds osd_op_complaint_time 180 2024-09-21T03:55:30.822 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T03:55:30.823 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set mds rados_mon_op_timeout 900 2024-09-21T03:55:30.879 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T03:55:30.880 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set mds rados_osd_op_timeout 900 2024-09-21T03:55:30.941 INFO:tasks.cephadm:setting config [mgr] client_mount_timeout = 30 2024-09-21T03:55:30.942 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set mgr client_mount_timeout 30 2024-09-21T03:55:30.994 INFO:tasks.cephadm:setting config [mgr] debug_cephsqlite = 20 2024-09-21T03:55:30.995 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set mgr debug_cephsqlite 20 2024-09-21T03:55:31.046 INFO:tasks.cephadm:setting config [mgr] debug_client = 20 2024-09-21T03:55:31.047 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set mgr debug_client 20 2024-09-21T03:55:31.097 INFO:tasks.cephadm:setting config [mgr] debug_mgr = 20 2024-09-21T03:55:31.097 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set mgr debug_mgr 20 2024-09-21T03:55:31.144 INFO:tasks.cephadm:setting config [mgr] debug_ms = 1 2024-09-21T03:55:31.144 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set mgr debug_ms 1 2024-09-21T03:55:31.190 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/allow_m_granularity = True 2024-09-21T03:55:31.190 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-09-21T03:55:31.287 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/dump_on_update = True 2024-09-21T03:55:31.287 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-09-21T03:55:31.288 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-mon[31180]: Saving service mgr spec with placement smithi064=x;smithi119=y;smithi131=z;count:3 2024-09-21T03:55:31.288 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:31.288 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:31.288 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:31.288 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:31.288 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:31.288 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:31.288 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi119", "name": "osd_memory_target"} : dispatch 2024-09-21T03:55:31.288 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:31.289 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:31.338 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/log_level = debug 2024-09-21T03:55:31.338 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/log_level debug 2024-09-21T03:55:31.400 INFO:tasks.cephadm:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T03:55:31.400 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T03:55:31.452 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-21T03:55:31.452 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set mon mon_op_complaint_time 120 2024-09-21T03:55:31.505 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-21T03:55:31.505 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config set osd osd_op_complaint_time 180 2024-09-21T03:55:31.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:31 smithi119 ceph-mon[34811]: Saving service mgr spec with placement smithi064=x;smithi119=y;smithi131=z;count:3 2024-09-21T03:55:31.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:31 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:31.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:31 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:31.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:31 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:31.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:31 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:31.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:31 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:31.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:31 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:31.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:31 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi119", "name": "osd_memory_target"} : dispatch 2024-09-21T03:55:31.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:31 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:31.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:31 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:31.511 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: 2024-09-21T03:55:31.433+0000 7fa70add0640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:03:55:31] ENGINE Error in HTTPServer.serve 2024-09-21T03:55:31.511 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: Traceback (most recent call last): 2024-09-21T03:55:31.511 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:55:31.511 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: self._connections.run(self.expiration_interval) 2024-09-21T03:55:31.511 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:55:31.511 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: self._run(expiration_interval) 2024-09-21T03:55:31.511 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:55:31.511 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:55:31.511 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:55:31.511 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:55:31.511 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:55:31.511 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: s = self.context.wrap_socket( 2024-09-21T03:55:31.511 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:55:31.512 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: return self.sslsocket_class._create( 2024-09-21T03:55:31.512 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:55:31.512 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: self.do_handshake() 2024-09-21T03:55:31.512 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:55:31.512 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: self._sslobj.do_handshake() 2024-09-21T03:55:31.512 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:55:31.512 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 03:55:31 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: 2024-09-21T03:55:31.524 DEBUG:tasks.cephadm:set 30 configs 2024-09-21T03:55:31.524 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi064') 2024-09-21T03:55:31.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:31 smithi131 ceph-mon[35193]: Saving service mgr spec with placement smithi064=x;smithi119=y;smithi131=z;count:3 2024-09-21T03:55:31.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:31 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:31.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:31 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:31.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:31 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:31.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:31 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:31.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:31 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:31.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:31 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:31.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:31 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi119", "name": "osd_memory_target"} : dispatch 2024-09-21T03:55:31.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:31 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:31.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:31 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:31.841 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:31.900 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:32.313 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:32 smithi064 ceph-mon[31180]: [21/Sep/2024:03:55:31] ENGINE Bus STARTING 2024-09-21T03:55:32.313 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:32 smithi064 ceph-mon[31180]: mgrmap e17: x(active, since 2s) 2024-09-21T03:55:32.313 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:32 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:32.313 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:32 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:32.313 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:32 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi131", "name": "osd_memory_target"} : dispatch 2024-09-21T03:55:32.313 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:32 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1558139207' entity='client.admin' 2024-09-21T03:55:32.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:32 smithi119 ceph-mon[34811]: [21/Sep/2024:03:55:31] ENGINE Bus STARTING 2024-09-21T03:55:32.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:32 smithi119 ceph-mon[34811]: mgrmap e17: x(active, since 2s) 2024-09-21T03:55:32.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:32 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:32.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:32 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:32.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:32 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi131", "name": "osd_memory_target"} : dispatch 2024-09-21T03:55:32.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:32 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1558139207' entity='client.admin' 2024-09-21T03:55:32.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:32 smithi131 ceph-mon[35193]: [21/Sep/2024:03:55:31] ENGINE Bus STARTING 2024-09-21T03:55:32.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:32 smithi131 ceph-mon[35193]: mgrmap e17: x(active, since 2s) 2024-09-21T03:55:32.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:32 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:32.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:32 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:32.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:32 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi131", "name": "osd_memory_target"} : dispatch 2024-09-21T03:55:32.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:32 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1558139207' entity='client.admin' 2024-09-21T03:55:33.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:33 smithi064 ceph-mon[31180]: [21/Sep/2024:03:55:31] ENGINE Serving on http://172.21.15.64:8765 2024-09-21T03:55:33.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:33 smithi064 ceph-mon[31180]: [21/Sep/2024:03:55:31] ENGINE Serving on https://172.21.15.64:7150 2024-09-21T03:55:33.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:33 smithi064 ceph-mon[31180]: [21/Sep/2024:03:55:31] ENGINE Bus STARTED 2024-09-21T03:55:33.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:33 smithi064 ceph-mon[31180]: [21/Sep/2024:03:55:31] ENGINE Error in HTTPServer.serve 2024-09-21T03:55:33.262 INFO:journalctl@ceph.mon.a.smithi064.stdout: Traceback (most recent call last): 2024-09-21T03:55:33.262 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:55:33.262 INFO:journalctl@ceph.mon.a.smithi064.stdout: self._connections.run(self.expiration_interval) 2024-09-21T03:55:33.262 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:55:33.262 INFO:journalctl@ceph.mon.a.smithi064.stdout: self._run(expiration_interval) 2024-09-21T03:55:33.263 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:55:33.263 INFO:journalctl@ceph.mon.a.smithi064.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:55:33.263 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:55:33.263 INFO:journalctl@ceph.mon.a.smithi064.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:55:33.263 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:55:33.263 INFO:journalctl@ceph.mon.a.smithi064.stdout: s = self.context.wrap_socket( 2024-09-21T03:55:33.263 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:55:33.263 INFO:journalctl@ceph.mon.a.smithi064.stdout: return self.sslsocket_class._create( 2024-09-21T03:55:33.263 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:55:33.263 INFO:journalctl@ceph.mon.a.smithi064.stdout: self.do_handshake() 2024-09-21T03:55:33.263 INFO:journalctl@ceph.mon.a.smithi064.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:55:33.263 INFO:journalctl@ceph.mon.a.smithi064.stdout: self._sslobj.do_handshake() 2024-09-21T03:55:33.263 INFO:journalctl@ceph.mon.a.smithi064.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:55:33.263 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:33 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1319973145' entity='client.admin' 2024-09-21T03:55:33.264 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:33 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:33.264 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:33 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:33.264 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:33 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi064", "name": "osd_memory_target"} : dispatch 2024-09-21T03:55:33.264 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:33 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:33.264 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:33 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:55:33.318 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:33.352 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi064') 2024-09-21T03:55:33.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:33 smithi119 ceph-mon[34811]: [21/Sep/2024:03:55:31] ENGINE Serving on http://172.21.15.64:8765 2024-09-21T03:55:33.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:33 smithi119 ceph-mon[34811]: [21/Sep/2024:03:55:31] ENGINE Serving on https://172.21.15.64:7150 2024-09-21T03:55:33.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:33 smithi119 ceph-mon[34811]: [21/Sep/2024:03:55:31] ENGINE Bus STARTED 2024-09-21T03:55:33.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:33 smithi119 ceph-mon[34811]: [21/Sep/2024:03:55:31] ENGINE Error in HTTPServer.serve 2024-09-21T03:55:33.507 INFO:journalctl@ceph.mon.b.smithi119.stdout: Traceback (most recent call last): 2024-09-21T03:55:33.507 INFO:journalctl@ceph.mon.b.smithi119.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:55:33.507 INFO:journalctl@ceph.mon.b.smithi119.stdout: self._connections.run(self.expiration_interval) 2024-09-21T03:55:33.507 INFO:journalctl@ceph.mon.b.smithi119.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:55:33.507 INFO:journalctl@ceph.mon.b.smithi119.stdout: self._run(expiration_interval) 2024-09-21T03:55:33.507 INFO:journalctl@ceph.mon.b.smithi119.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:55:33.507 INFO:journalctl@ceph.mon.b.smithi119.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:55:33.507 INFO:journalctl@ceph.mon.b.smithi119.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:55:33.507 INFO:journalctl@ceph.mon.b.smithi119.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:55:33.507 INFO:journalctl@ceph.mon.b.smithi119.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:55:33.507 INFO:journalctl@ceph.mon.b.smithi119.stdout: s = self.context.wrap_socket( 2024-09-21T03:55:33.507 INFO:journalctl@ceph.mon.b.smithi119.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:55:33.507 INFO:journalctl@ceph.mon.b.smithi119.stdout: return self.sslsocket_class._create( 2024-09-21T03:55:33.508 INFO:journalctl@ceph.mon.b.smithi119.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:55:33.508 INFO:journalctl@ceph.mon.b.smithi119.stdout: self.do_handshake() 2024-09-21T03:55:33.508 INFO:journalctl@ceph.mon.b.smithi119.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:55:33.508 INFO:journalctl@ceph.mon.b.smithi119.stdout: self._sslobj.do_handshake() 2024-09-21T03:55:33.508 INFO:journalctl@ceph.mon.b.smithi119.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:55:33.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:33 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1319973145' entity='client.admin' 2024-09-21T03:55:33.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:33 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:33.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:33 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:33.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:33 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi064", "name": "osd_memory_target"} : dispatch 2024-09-21T03:55:33.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:33 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:33.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:33 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:55:33.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:33 smithi131 ceph-mon[35193]: [21/Sep/2024:03:55:31] ENGINE Serving on http://172.21.15.64:8765 2024-09-21T03:55:33.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:33 smithi131 ceph-mon[35193]: [21/Sep/2024:03:55:31] ENGINE Serving on https://172.21.15.64:7150 2024-09-21T03:55:33.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:33 smithi131 ceph-mon[35193]: [21/Sep/2024:03:55:31] ENGINE Bus STARTED 2024-09-21T03:55:33.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:33 smithi131 ceph-mon[35193]: [21/Sep/2024:03:55:31] ENGINE Error in HTTPServer.serve 2024-09-21T03:55:33.553 INFO:journalctl@ceph.mon.c.smithi131.stdout: Traceback (most recent call last): 2024-09-21T03:55:33.553 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T03:55:33.553 INFO:journalctl@ceph.mon.c.smithi131.stdout: self._connections.run(self.expiration_interval) 2024-09-21T03:55:33.554 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T03:55:33.554 INFO:journalctl@ceph.mon.c.smithi131.stdout: self._run(expiration_interval) 2024-09-21T03:55:33.554 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T03:55:33.554 INFO:journalctl@ceph.mon.c.smithi131.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T03:55:33.554 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T03:55:33.554 INFO:journalctl@ceph.mon.c.smithi131.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T03:55:33.554 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T03:55:33.554 INFO:journalctl@ceph.mon.c.smithi131.stdout: s = self.context.wrap_socket( 2024-09-21T03:55:33.554 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T03:55:33.554 INFO:journalctl@ceph.mon.c.smithi131.stdout: return self.sslsocket_class._create( 2024-09-21T03:55:33.554 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T03:55:33.554 INFO:journalctl@ceph.mon.c.smithi131.stdout: self.do_handshake() 2024-09-21T03:55:33.554 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T03:55:33.554 INFO:journalctl@ceph.mon.c.smithi131.stdout: self._sslobj.do_handshake() 2024-09-21T03:55:33.555 INFO:journalctl@ceph.mon.c.smithi131.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T03:55:33.555 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:33 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1319973145' entity='client.admin' 2024-09-21T03:55:33.555 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:33 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:33.555 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:33 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:33.555 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:33 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi064", "name": "osd_memory_target"} : dispatch 2024-09-21T03:55:33.555 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:33 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:33.555 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:33 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:55:33.647 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi064') 2024-09-21T03:55:33.717 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:33.718 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:34.196 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:34 smithi131 ceph-mon[35193]: Updating smithi064:/etc/ceph/ceph.conf 2024-09-21T03:55:34.197 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:34 smithi131 ceph-mon[35193]: Updating smithi119:/etc/ceph/ceph.conf 2024-09-21T03:55:34.197 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:34 smithi131 ceph-mon[35193]: Updating smithi131:/etc/ceph/ceph.conf 2024-09-21T03:55:34.197 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:34 smithi131 ceph-mon[35193]: Updating smithi064:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:55:34.197 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:34 smithi131 ceph-mon[35193]: Updating smithi119:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:55:34.197 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:34 smithi131 ceph-mon[35193]: Updating smithi131:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:55:34.197 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:34 smithi131 ceph-mon[35193]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:55:34.197 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:34 smithi131 ceph-mon[35193]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:55:34.197 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:34 smithi131 ceph-mon[35193]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:55:34.197 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:34 smithi131 ceph-mon[35193]: Updating smithi064:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:55:34.472 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:34 smithi064 ceph-mon[31180]: Updating smithi064:/etc/ceph/ceph.conf 2024-09-21T03:55:34.472 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:34 smithi064 ceph-mon[31180]: Updating smithi119:/etc/ceph/ceph.conf 2024-09-21T03:55:34.473 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:34 smithi064 ceph-mon[31180]: Updating smithi131:/etc/ceph/ceph.conf 2024-09-21T03:55:34.473 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:34 smithi064 ceph-mon[31180]: Updating smithi064:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:55:34.473 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:34 smithi064 ceph-mon[31180]: Updating smithi119:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:55:34.473 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:34 smithi064 ceph-mon[31180]: Updating smithi131:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:55:34.473 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:34 smithi064 ceph-mon[31180]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:55:34.473 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:34 smithi064 ceph-mon[31180]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:55:34.473 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:34 smithi064 ceph-mon[31180]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:55:34.473 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:34 smithi064 ceph-mon[31180]: Updating smithi064:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:55:34.473 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:34 smithi064 ceph-mon[31180]: Updating smithi119:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:55:34.473 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:34 smithi064 ceph-mon[31180]: Updating smithi131:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:55:34.473 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:34 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:34.473 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:34 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:34.473 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:34 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:34.473 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:34 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:34.474 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:34 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:34.474 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:34 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:34.474 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:34 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:34.474 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:34 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:55:34.474 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:34 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:55:34.474 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:34 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:55:34.474 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:34 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:34.474 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:34 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1318069462' entity='client.admin' 2024-09-21T03:55:34.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:34 smithi119 ceph-mon[34811]: Updating smithi064:/etc/ceph/ceph.conf 2024-09-21T03:55:34.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:34 smithi119 ceph-mon[34811]: Updating smithi119:/etc/ceph/ceph.conf 2024-09-21T03:55:34.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:34 smithi119 ceph-mon[34811]: Updating smithi131:/etc/ceph/ceph.conf 2024-09-21T03:55:34.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:34 smithi119 ceph-mon[34811]: Updating smithi064:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:55:34.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:34 smithi119 ceph-mon[34811]: Updating smithi119:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:55:34.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:34 smithi119 ceph-mon[34811]: Updating smithi131:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T03:55:34.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:34 smithi119 ceph-mon[34811]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:55:34.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:34 smithi119 ceph-mon[34811]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:55:34.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:34 smithi119 ceph-mon[34811]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-09-21T03:55:34.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:34 smithi119 ceph-mon[34811]: Updating smithi064:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:55:34.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:34 smithi119 ceph-mon[34811]: Updating smithi119:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:55:34.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:34 smithi119 ceph-mon[34811]: Updating smithi131:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:55:34.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:34 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:34.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:34 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:34.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:34 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:34.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:34 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:34.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:34 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:34.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:34 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:34.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:34 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:34.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:34 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:55:34.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:34 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:55:34.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:34 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:55:34.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:34 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:34.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:34 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1318069462' entity='client.admin' 2024-09-21T03:55:34.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:34 smithi131 ceph-mon[35193]: Updating smithi119:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:55:34.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:34 smithi131 ceph-mon[35193]: Updating smithi131:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T03:55:34.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:34 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:34.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:34 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:34.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:34 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:34.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:34 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:34.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:34 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:34.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:34 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:34.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:34 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:34.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:34 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:55:34.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:34 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:55:34.522 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:34 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:55:34.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:34 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:34.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:34 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1318069462' entity='client.admin' 2024-09-21T03:55:35.012 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:35.012 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:35.031 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:35.031 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:35.039 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:35.051 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:35.289 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:35.290 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:35.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:35 smithi119 ceph-mon[34811]: Deploying daemon mgr.z on smithi131 2024-09-21T03:55:35.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:35 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1537337801' entity='client.admin' 2024-09-21T03:55:35.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:35 smithi064 ceph-mon[31180]: Deploying daemon mgr.z on smithi131 2024-09-21T03:55:35.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:35 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1537337801' entity='client.admin' 2024-09-21T03:55:35.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:35 smithi131 ceph-mon[35193]: Deploying daemon mgr.z on smithi131 2024-09-21T03:55:35.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:35 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1537337801' entity='client.admin' 2024-09-21T03:55:35.553 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 21 03:55:35 smithi131 systemd[1]: Started Ceph mgr.z for e7c927ba-77cc-11ef-baf5-efdc52797490. 2024-09-21T03:55:35.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:35.754+0000 7f911ef2e640 1 -- 172.21.15.64:0/3748995836 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f910803fb70 msgr2=0x7f9108042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:35.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:35.754+0000 7f911ef2e640 1 --2- 172.21.15.64:0/3748995836 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f910803fb70 0x7f9108042080 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9118109260 tx=0x7f9104005f90 comp rx=0 tx=0).stop 2024-09-21T03:55:35.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:35.754+0000 7f911ef2e640 1 -- 172.21.15.64:0/3748995836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9118103160 msgr2=0x7f911819a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:35.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:35.754+0000 7f911ef2e640 1 --2- 172.21.15.64:0/3748995836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9118103160 0x7f911819a3c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f910c00e9f0 tx=0x7f910c00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:55:35.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:35.754+0000 7f911ef2e640 1 -- 172.21.15.64:0/3748995836 shutdown_connections 2024-09-21T03:55:35.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:35.754+0000 7f911ef2e640 1 --2- 172.21.15.64:0/3748995836 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f910803fb70 0x7f9108042080 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:35.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:35.754+0000 7f911ef2e640 1 --2- 172.21.15.64:0/3748995836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9118105ac0 0x7f911819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:35.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:35.754+0000 7f911ef2e640 1 --2- 172.21.15.64:0/3748995836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9118105170 0x7f911819a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:35.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:35.754+0000 7f911ef2e640 1 --2- 172.21.15.64:0/3748995836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9118103160 0x7f911819a3c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:35.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:35.754+0000 7f911ef2e640 1 -- 172.21.15.64:0/3748995836 >> 172.21.15.64:0/3748995836 conn(0x7f9118077c60 msgr2=0x7f91181010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:35.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:35.754+0000 7f911ef2e640 1 -- 172.21.15.64:0/3748995836 shutdown_connections 2024-09-21T03:55:35.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:35.754+0000 7f911ef2e640 1 -- 172.21.15.64:0/3748995836 wait complete. 2024-09-21T03:55:35.975 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:36.349 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:36.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:36 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:36.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:36 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:36.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:36 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:36.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:36 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:55:36.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:36 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:55:36.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:36 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:55:36.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:36 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:36.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:36 smithi064 ceph-mon[31180]: Deploying daemon mgr.y on smithi119 2024-09-21T03:55:36.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:36 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2030204370' entity='client.admin' 2024-09-21T03:55:36.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:36 smithi064 ceph-mon[31180]: from='client.? ' entity='client.admin' 2024-09-21T03:55:36.682 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:36 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:36.682 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:36 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:36.682 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:36 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:36.682 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:36 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:55:36.683 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:36 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:55:36.683 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:36 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:55:36.683 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:36 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:36.683 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:36 smithi131 ceph-mon[35193]: Deploying daemon mgr.y on smithi119 2024-09-21T03:55:36.683 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:36 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2030204370' entity='client.admin' 2024-09-21T03:55:36.683 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:36 smithi131 ceph-mon[35193]: from='client.? ' entity='client.admin' 2024-09-21T03:55:36.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:36 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:36.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:36 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:36.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:36 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:36.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:36 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:55:36.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:36 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T03:55:36.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:36 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:55:36.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:36 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:36.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:36 smithi119 ceph-mon[34811]: Deploying daemon mgr.y on smithi119 2024-09-21T03:55:36.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:36 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2030204370' entity='client.admin' 2024-09-21T03:55:36.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:36 smithi119 ceph-mon[34811]: from='client.? ' entity='client.admin' 2024-09-21T03:55:36.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.750+0000 7f7468086640 1 -- 172.21.15.64:0/3049407632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7460103bb0 msgr2=0x7f7460103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:36.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.750+0000 7f7468086640 1 --2- 172.21.15.64:0/3049407632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7460103bb0 0x7f7460103fc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f745c009900 tx=0x7f745c02a410 comp rx=0 tx=0).stop 2024-09-21T03:55:36.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.750+0000 7f7468086640 1 -- 172.21.15.64:0/3049407632 shutdown_connections 2024-09-21T03:55:36.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.750+0000 7f7468086640 1 --2- 172.21.15.64:0/3049407632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7460109e40 0x7f7460110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:36.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.750+0000 7f7468086640 1 --2- 172.21.15.64:0/3049407632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74601094f0 0x7f7460109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:36.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.750+0000 7f7468086640 1 --2- 172.21.15.64:0/3049407632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7460103bb0 0x7f7460103fc0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:36.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.750+0000 7f7468086640 1 -- 172.21.15.64:0/3049407632 >> 172.21.15.64:0/3049407632 conn(0x7f74600ff600 msgr2=0x7f7460101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:36.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.750+0000 7f7468086640 1 -- 172.21.15.64:0/3049407632 shutdown_connections 2024-09-21T03:55:36.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.750+0000 7f7468086640 1 -- 172.21.15.64:0/3049407632 wait complete. 2024-09-21T03:55:36.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.751+0000 7f7468086640 1 Processor -- start 2024-09-21T03:55:36.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.751+0000 7f7468086640 1 -- start start 2024-09-21T03:55:36.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.751+0000 7f7468086640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7460103bb0 0x7f7460075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:36.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.751+0000 7f7468086640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74601094f0 0x7f7460075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:36.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.751+0000 7f7468086640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7460109e40 0x7f7460076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:36.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.751+0000 7f7468086640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7460113b00 con 0x7f7460103bb0 2024-09-21T03:55:36.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.751+0000 7f7468086640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7460113980 con 0x7f74601094f0 2024-09-21T03:55:36.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.751+0000 7f7468086640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7460113c80 con 0x7f7460109e40 2024-09-21T03:55:36.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.752+0000 7f7465e22640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7460103bb0 0x7f7460075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:36.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.752+0000 7f7465e22640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7460103bb0 0x7f7460075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42982/0 (socket says 172.21.15.64:42982) 2024-09-21T03:55:36.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.752+0000 7f7465e22640 1 -- 172.21.15.64:0/1653868793 learned_addr learned my addr 172.21.15.64:0/1653868793 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:55:36.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.752+0000 7f7466623640 1 --2- 172.21.15.64:0/1653868793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7460109e40 0x7f7460076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:36.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.752+0000 7f7465621640 1 --2- 172.21.15.64:0/1653868793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74601094f0 0x7f7460075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:36.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.752+0000 7f7465e22640 1 -- 172.21.15.64:0/1653868793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7460109e40 msgr2=0x7f7460076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:36.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.752+0000 7f7465e22640 1 --2- 172.21.15.64:0/1653868793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7460109e40 0x7f7460076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:36.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.752+0000 7f7465e22640 1 -- 172.21.15.64:0/1653868793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74601094f0 msgr2=0x7f7460075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:36.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.752+0000 7f7465e22640 1 --2- 172.21.15.64:0/1653868793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74601094f0 0x7f7460075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:36.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.752+0000 7f7465e22640 1 -- 172.21.15.64:0/1653868793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74601a8810 con 0x7f7460103bb0 2024-09-21T03:55:36.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.752+0000 7f7465e22640 1 --2- 172.21.15.64:0/1653868793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7460103bb0 0x7f7460075710 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f745c005940 tx=0x7f745c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:36.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.752+0000 7f7456ffd640 1 -- 172.21.15.64:0/1653868793 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c03f030 con 0x7f7460103bb0 2024-09-21T03:55:36.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.752+0000 7f7456ffd640 1 -- 172.21.15.64:0/1653868793 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f745c003aa0 con 0x7f7460103bb0 2024-09-21T03:55:36.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.752+0000 7f7468086640 1 -- 172.21.15.64:0/1653868793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74601a8aa0 con 0x7f7460103bb0 2024-09-21T03:55:36.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.752+0000 7f7456ffd640 1 -- 172.21.15.64:0/1653868793 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745c035070 con 0x7f7460103bb0 2024-09-21T03:55:36.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.752+0000 7f7468086640 1 -- 172.21.15.64:0/1653868793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74601a8fd0 con 0x7f7460103bb0 2024-09-21T03:55:36.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.753+0000 7f7456ffd640 1 -- 172.21.15.64:0/1653868793 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7f745c042030 con 0x7f7460103bb0 2024-09-21T03:55:36.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.754+0000 7f7468086640 1 -- 172.21.15.64:0/1653868793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7460103fc0 con 0x7f7460103bb0 2024-09-21T03:55:36.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.754+0000 7f7456ffd640 1 --2- 172.21.15.64:0/1653868793 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f744c03fb70 0x7f744c042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:36.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.754+0000 7f7456ffd640 1 -- 172.21.15.64:0/1653868793 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f745c074d20 con 0x7f7460103bb0 2024-09-21T03:55:36.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.754+0000 7f7465621640 1 --2- 172.21.15.64:0/1653868793 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f744c03fb70 0x7f744c042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:36.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.754+0000 7f7465621640 1 --2- 172.21.15.64:0/1653868793 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f744c03fb70 0x7f744c042080 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7460109080 tx=0x7f7450005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:36.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.759+0000 7f7456ffd640 1 -- 172.21.15.64:0/1653868793 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f745c036df0 con 0x7f7460103bb0 2024-09-21T03:55:36.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.948+0000 7f7468086640 1 -- 172.21.15.64:0/1653868793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f746002e640 con 0x7f7460103bb0 2024-09-21T03:55:36.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.949+0000 7f05f3a8b640 1 -- 172.21.15.64:0/4116929191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05ec1025d0 msgr2=0x7f05ec1029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:36.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.949+0000 7f05f3a8b640 1 --2- 172.21.15.64:0/4116929191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05ec1025d0 0x7f05ec1029e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009860 tx=0x7f05e802a3d0 comp rx=0 tx=0).stop 2024-09-21T03:55:36.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.949+0000 7f05f3a8b640 1 -- 172.21.15.64:0/4116929191 shutdown_connections 2024-09-21T03:55:36.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.949+0000 7f05f3a8b640 1 --2- 172.21.15.64:0/4116929191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05ec110180 0x7f05ec1125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:36.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.949+0000 7f05f3a8b640 1 --2- 172.21.15.64:0/4116929191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05ec102f20 0x7f05ec10fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:36.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.949+0000 7f05f3a8b640 1 --2- 172.21.15.64:0/4116929191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05ec1025d0 0x7f05ec1029e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:36.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.949+0000 7f05f3a8b640 1 -- 172.21.15.64:0/4116929191 >> 172.21.15.64:0/4116929191 conn(0x7f05ec079730 msgr2=0x7f05ec079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:36.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.949+0000 7f05f3a8b640 1 -- 172.21.15.64:0/4116929191 shutdown_connections 2024-09-21T03:55:36.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.949+0000 7f05f3a8b640 1 -- 172.21.15.64:0/4116929191 wait complete. 2024-09-21T03:55:36.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.950+0000 7f05f3a8b640 1 Processor -- start 2024-09-21T03:55:36.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.950+0000 7f05f3a8b640 1 -- start start 2024-09-21T03:55:36.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.950+0000 7f05f3a8b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05ec1025d0 0x7f05ec19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:36.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.950+0000 7f05f3a8b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05ec102f20 0x7f05ec19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:36.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.950+0000 7f05f3a8b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05ec110180 0x7f05ec1a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:36.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.950+0000 7f05f3a8b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec115c80 con 0x7f05ec102f20 2024-09-21T03:55:36.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.950+0000 7f05f3a8b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec115b00 con 0x7f05ec110180 2024-09-21T03:55:36.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.950+0000 7f05f3a8b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec115e00 con 0x7f05ec1025d0 2024-09-21T03:55:36.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.950+0000 7f05f1026640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05ec102f20 0x7f05ec19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:36.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.950+0000 7f05f1026640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05ec102f20 0x7f05ec19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42998/0 (socket says 172.21.15.64:42998) 2024-09-21T03:55:36.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.950+0000 7f05f1026640 1 -- 172.21.15.64:0/731616234 learned_addr learned my addr 172.21.15.64:0/731616234 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:55:36.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.950+0000 7f05f1827640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05ec1025d0 0x7f05ec19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:36.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.951+0000 7f05f2028640 1 --2- 172.21.15.64:0/731616234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05ec110180 0x7f05ec1a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:36.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.951+0000 7f05f1026640 1 -- 172.21.15.64:0/731616234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05ec1025d0 msgr2=0x7f05ec19c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:36.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.951+0000 7f05f1026640 1 --2- 172.21.15.64:0/731616234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05ec1025d0 0x7f05ec19c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:36.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.951+0000 7f05f1026640 1 -- 172.21.15.64:0/731616234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05ec110180 msgr2=0x7f05ec1a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:36.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.951+0000 7f05f1026640 1 --2- 172.21.15.64:0/731616234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05ec110180 0x7f05ec1a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:36.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.951+0000 7f05f1026640 1 -- 172.21.15.64:0/731616234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ec1a2640 con 0x7f05ec102f20 2024-09-21T03:55:36.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.951+0000 7f05f1827640 1 --2- 172.21.15.64:0/731616234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05ec1025d0 0x7f05ec19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:55:36.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.951+0000 7f05f1026640 1 --2- 172.21.15.64:0/731616234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05ec102f20 0x7f05ec19cc70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00e9c0 tx=0x7f05dc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:36.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.952+0000 7f7456ffd640 1 -- 172.21.15.64:0/1653868793 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v16)=0 v16) ==== 115+0+0 (secure 0 0 0) 0x7f745c003c50 con 0x7f7460103bb0 2024-09-21T03:55:36.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.953+0000 7f05e2ffd640 1 -- 172.21.15.64:0/731616234 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc00cde0 con 0x7f05ec102f20 2024-09-21T03:55:36.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.954+0000 7f05f3a8b640 1 -- 172.21.15.64:0/731616234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05ec1a2930 con 0x7f05ec102f20 2024-09-21T03:55:36.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.954+0000 7f05e2ffd640 1 -- 172.21.15.64:0/731616234 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05dc0035f0 con 0x7f05ec102f20 2024-09-21T03:55:36.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.954+0000 7f05e2ffd640 1 -- 172.21.15.64:0/731616234 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc010690 con 0x7f05ec102f20 2024-09-21T03:55:36.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.954+0000 7f05f3a8b640 1 -- 172.21.15.64:0/731616234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05ec1a9090 con 0x7f05ec102f20 2024-09-21T03:55:36.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.955+0000 7f05e2ffd640 1 -- 172.21.15.64:0/731616234 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7f05dc010830 con 0x7f05ec102f20 2024-09-21T03:55:36.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.955+0000 7f05f3a8b640 1 -- 172.21.15.64:0/731616234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05ec1029e0 con 0x7f05ec102f20 2024-09-21T03:55:36.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.955+0000 7f7468086640 1 -- 172.21.15.64:0/1653868793 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f744c03fb70 msgr2=0x7f744c042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:36.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.955+0000 7f7468086640 1 --2- 172.21.15.64:0/1653868793 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f744c03fb70 0x7f744c042080 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7460109080 tx=0x7f7450005f90 comp rx=0 tx=0).stop 2024-09-21T03:55:36.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.955+0000 7f7468086640 1 -- 172.21.15.64:0/1653868793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7460103bb0 msgr2=0x7f7460075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:36.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.955+0000 7f7468086640 1 --2- 172.21.15.64:0/1653868793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7460103bb0 0x7f7460075710 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f745c005940 tx=0x7f745c004270 comp rx=0 tx=0).stop 2024-09-21T03:55:36.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.955+0000 7f7468086640 1 -- 172.21.15.64:0/1653868793 shutdown_connections 2024-09-21T03:55:36.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.955+0000 7f7468086640 1 --2- 172.21.15.64:0/1653868793 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f744c03fb70 0x7f744c042080 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:36.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.955+0000 7f7468086640 1 --2- 172.21.15.64:0/1653868793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7460109e40 0x7f7460076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:36.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.955+0000 7f7468086640 1 --2- 172.21.15.64:0/1653868793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74601094f0 0x7f7460075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:36.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.955+0000 7f7468086640 1 --2- 172.21.15.64:0/1653868793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7460103bb0 0x7f7460075710 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:36.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.955+0000 7f7468086640 1 -- 172.21.15.64:0/1653868793 >> 172.21.15.64:0/1653868793 conn(0x7f74600ff600 msgr2=0x7f7460101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:36.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.955+0000 7f05e2ffd640 1 --2- 172.21.15.64:0/731616234 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f05d803fb70 0x7f05d8042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:36.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.955+0000 7f05e2ffd640 1 -- 172.21.15.64:0/731616234 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f05dc0548a0 con 0x7f05ec102f20 2024-09-21T03:55:36.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.955+0000 7f05f1827640 1 --2- 172.21.15.64:0/731616234 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f05d803fb70 0x7f05d8042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:36.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.956+0000 7f7468086640 1 -- 172.21.15.64:0/1653868793 shutdown_connections 2024-09-21T03:55:36.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.956+0000 7f7468086640 1 -- 172.21.15.64:0/1653868793 wait complete. 2024-09-21T03:55:36.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.956+0000 7f05f1827640 1 --2- 172.21.15.64:0/731616234 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f05d803fb70 0x7f05d8042080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f05e8032ff0 tx=0x7f05e8005c90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:36.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:36.961+0000 7f05e2ffd640 1 -- 172.21.15.64:0/731616234 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f05dc016330 con 0x7f05ec102f20 2024-09-21T03:55:37.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:37.153+0000 7f05f3a8b640 1 -- 172.21.15.64:0/731616234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f05ec1a93d0 con 0x7f05ec102f20 2024-09-21T03:55:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:37.156+0000 7f05e2ffd640 1 -- 172.21.15.64:0/731616234 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v17)=0 v17) ==== 116+0+0 (secure 0 0 0) 0x7f05dc0094d0 con 0x7f05ec102f20 2024-09-21T03:55:37.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:37.159+0000 7f05f3a8b640 1 -- 172.21.15.64:0/731616234 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f05d803fb70 msgr2=0x7f05d8042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:37.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:37.159+0000 7f05f3a8b640 1 --2- 172.21.15.64:0/731616234 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f05d803fb70 0x7f05d8042080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f05e8032ff0 tx=0x7f05e8005c90 comp rx=0 tx=0).stop 2024-09-21T03:55:37.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:37.159+0000 7f05f3a8b640 1 -- 172.21.15.64:0/731616234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05ec102f20 msgr2=0x7f05ec19cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:37.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:37.159+0000 7f05f3a8b640 1 --2- 172.21.15.64:0/731616234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05ec102f20 0x7f05ec19cc70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00e9c0 tx=0x7f05dc00ee80 comp rx=0 tx=0).stop 2024-09-21T03:55:37.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:37.159+0000 7f05f3a8b640 1 -- 172.21.15.64:0/731616234 shutdown_connections 2024-09-21T03:55:37.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:37.159+0000 7f05f3a8b640 1 --2- 172.21.15.64:0/731616234 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f05d803fb70 0x7f05d8042080 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:37.159+0000 7f05f3a8b640 1 --2- 172.21.15.64:0/731616234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05ec110180 0x7f05ec1a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:37.159+0000 7f05f3a8b640 1 --2- 172.21.15.64:0/731616234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05ec102f20 0x7f05ec19cc70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:37.159+0000 7f05f3a8b640 1 --2- 172.21.15.64:0/731616234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05ec1025d0 0x7f05ec19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:37.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:37.159+0000 7f05f3a8b640 1 -- 172.21.15.64:0/731616234 >> 172.21.15.64:0/731616234 conn(0x7f05ec079730 msgr2=0x7f05ec109d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:37.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:37.159+0000 7f05f3a8b640 1 -- 172.21.15.64:0/731616234 shutdown_connections 2024-09-21T03:55:37.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:37.159+0000 7f05f3a8b640 1 -- 172.21.15.64:0/731616234 wait complete. 2024-09-21T03:55:37.215 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Sep 21 03:55:36 smithi119 systemd[1]: Starting Ceph mgr.y for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T03:55:37.496 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Sep 21 03:55:37 smithi119 podman[36318]: 2024-09-21 03:55:37.116301017 +0000 UTC m=+0.015733918 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:55:37.496 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Sep 21 03:55:37 smithi119 podman[36318]: 2024-09-21 03:55:37.233314842 +0000 UTC m=+0.132747741 container create 5018b50d1d117dd77e0332d1ab63893d881ae01b81b1aed69095e333b6154a07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-09-21T03:55:37.496 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Sep 21 03:55:37 smithi119 podman[36318]: 2024-09-21 03:55:37.418942361 +0000 UTC m=+0.318375261 container init 5018b50d1d117dd77e0332d1ab63893d881ae01b81b1aed69095e333b6154a07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-y, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-21T03:55:37.496 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Sep 21 03:55:37 smithi119 podman[36318]: 2024-09-21 03:55:37.421588222 +0000 UTC m=+0.321021122 container start 5018b50d1d117dd77e0332d1ab63893d881ae01b81b1aed69095e333b6154a07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-y, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-21T03:55:37.496 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Sep 21 03:55:37 smithi119 ceph-mgr[36337]: -- 172.21.15.119:0/4109950105 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562fcf0d50e0 con 0x562fd3e78480 2024-09-21T03:55:37.496 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Sep 21 03:55:37 smithi119 bash[36318]: 5018b50d1d117dd77e0332d1ab63893d881ae01b81b1aed69095e333b6154a07 2024-09-21T03:55:37.547 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:37.756 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Sep 21 03:55:37 smithi119 systemd[1]: Started Ceph mgr.y for e7c927ba-77cc-11ef-baf5-efdc52797490. 2024-09-21T03:55:37.954 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi064') 2024-09-21T03:55:38.225 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:37 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1653868793' entity='client.admin' 2024-09-21T03:55:38.225 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:37 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/731616234' entity='client.admin' 2024-09-21T03:55:38.225 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:37 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:38.225 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:37 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:38.226 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:37 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:38.226 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:37 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:38.226 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:37 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:55:38.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:37 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1653868793' entity='client.admin' 2024-09-21T03:55:38.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:37 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/731616234' entity='client.admin' 2024-09-21T03:55:38.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:37 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:38.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:37 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:38.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:37 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:38.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:37 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:38.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:37 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:55:38.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:37 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1653868793' entity='client.admin' 2024-09-21T03:55:38.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:37 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/731616234' entity='client.admin' 2024-09-21T03:55:38.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:37 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:38.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:37 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:38.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:37 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:38.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:37 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:38.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:37 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:55:38.327 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi064') 2024-09-21T03:55:38.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.492+0000 7f9cdbdb5640 1 -- 172.21.15.64:0/874597131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cd4105300 msgr2=0x7f9cd410d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:38.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.492+0000 7f9cdbdb5640 1 --2- 172.21.15.64:0/874597131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cd4105300 0x7f9cd410d850 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8009900 tx=0x7f9cc802a410 comp rx=0 tx=0).stop 2024-09-21T03:55:38.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.492+0000 7f9cdbdb5640 1 -- 172.21.15.64:0/874597131 shutdown_connections 2024-09-21T03:55:38.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.492+0000 7f9cdbdb5640 1 --2- 172.21.15.64:0/874597131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cd410de10 0x7f9cd4110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:38.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.492+0000 7f9cdbdb5640 1 --2- 172.21.15.64:0/874597131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cd4105300 0x7f9cd410d850 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:38.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.492+0000 7f9cdbdb5640 1 --2- 172.21.15.64:0/874597131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cd41049b0 0x7f9cd4104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:38.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.492+0000 7f9cdbdb5640 1 -- 172.21.15.64:0/874597131 >> 172.21.15.64:0/874597131 conn(0x7f9cd4079730 msgr2=0x7f9cd4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:38.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.492+0000 7f9cdbdb5640 1 -- 172.21.15.64:0/874597131 shutdown_connections 2024-09-21T03:55:38.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.492+0000 7f9cdbdb5640 1 -- 172.21.15.64:0/874597131 wait complete. 2024-09-21T03:55:38.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.493+0000 7f9cdbdb5640 1 Processor -- start 2024-09-21T03:55:38.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.493+0000 7f9cdbdb5640 1 -- start start 2024-09-21T03:55:38.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.493+0000 7f9cdbdb5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cd41049b0 0x7f9cd419c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:38.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.493+0000 7f9cdbdb5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cd4105300 0x7f9cd419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:38.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.493+0000 7f9cdbdb5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cd410de10 0x7f9cd41a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:38.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.493+0000 7f9cdbdb5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd4113b00 con 0x7f9cd41049b0 2024-09-21T03:55:38.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.493+0000 7f9cdbdb5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd4113980 con 0x7f9cd410de10 2024-09-21T03:55:38.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.493+0000 7f9cdbdb5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd4113c80 con 0x7f9cd4105300 2024-09-21T03:55:38.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.493+0000 7f9cd9b51640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cd41049b0 0x7f9cd419c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:38.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.493+0000 7f9cd9b51640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cd41049b0 0x7f9cd419c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43022/0 (socket says 172.21.15.64:43022) 2024-09-21T03:55:38.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.493+0000 7f9cd9b51640 1 -- 172.21.15.64:0/3015181849 learned_addr learned my addr 172.21.15.64:0/3015181849 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:55:38.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.493+0000 7f9cd9350640 1 --2- 172.21.15.64:0/3015181849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cd4105300 0x7f9cd419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:38.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.493+0000 7f9cda352640 1 --2- 172.21.15.64:0/3015181849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cd410de10 0x7f9cd41a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:38.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.493+0000 7f9cd9b51640 1 -- 172.21.15.64:0/3015181849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cd4105300 msgr2=0x7f9cd419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:38.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.493+0000 7f9cd9b51640 1 --2- 172.21.15.64:0/3015181849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cd4105300 0x7f9cd419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:38.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.493+0000 7f9cd9b51640 1 -- 172.21.15.64:0/3015181849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cd410de10 msgr2=0x7f9cd41a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:38.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.493+0000 7f9cd9b51640 1 --2- 172.21.15.64:0/3015181849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cd410de10 0x7f9cd41a1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:38.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.493+0000 7f9cd9b51640 1 -- 172.21.15.64:0/3015181849 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd41a2680 con 0x7f9cd41049b0 2024-09-21T03:55:38.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.493+0000 7f9cda352640 1 --2- 172.21.15.64:0/3015181849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cd410de10 0x7f9cd41a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:38.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.494+0000 7f9cd9b51640 1 --2- 172.21.15.64:0/3015181849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cd41049b0 0x7f9cd419c710 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00d770 tx=0x7f9ccc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:38.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.494+0000 7f9cd2ffd640 1 -- 172.21.15.64:0/3015181849 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc017070 con 0x7f9cd41049b0 2024-09-21T03:55:38.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.494+0000 7f9cdbdb5640 1 -- 172.21.15.64:0/3015181849 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd41a2970 con 0x7f9cd41049b0 2024-09-21T03:55:38.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.494+0000 7f9cd2ffd640 1 -- 172.21.15.64:0/3015181849 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ccc009c30 con 0x7f9cd41049b0 2024-09-21T03:55:38.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.494+0000 7f9cd2ffd640 1 -- 172.21.15.64:0/3015181849 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc003cc0 con 0x7f9cd41049b0 2024-09-21T03:55:38.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.494+0000 7f9cdbdb5640 1 -- 172.21.15.64:0/3015181849 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd41a90d0 con 0x7f9cd41049b0 2024-09-21T03:55:38.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.495+0000 7f9cd2ffd640 1 -- 172.21.15.64:0/3015181849 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7f9ccc003e60 con 0x7f9cd41049b0 2024-09-21T03:55:38.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.495+0000 7f9cdbdb5640 1 -- 172.21.15.64:0/3015181849 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cd4104dc0 con 0x7f9cd41049b0 2024-09-21T03:55:38.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.495+0000 7f9cd2ffd640 1 --2- 172.21.15.64:0/3015181849 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f9cbc03fb70 0x7f9cbc042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:38.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.495+0000 7f9cd2ffd640 1 -- 172.21.15.64:0/3015181849 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f9ccc053f40 con 0x7f9cd41049b0 2024-09-21T03:55:38.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.495+0000 7f9cd9350640 1 --2- 172.21.15.64:0/3015181849 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f9cbc03fb70 0x7f9cbc042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:38.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.496+0000 7f9cd9350640 1 --2- 172.21.15.64:0/3015181849 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f9cbc03fb70 0x7f9cbc042080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9cc802aa00 tx=0x7f9cc8005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:38.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.500+0000 7f9cd2ffd640 1 -- 172.21.15.64:0/3015181849 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9ccc009de0 con 0x7f9cd41049b0 2024-09-21T03:55:38.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.689+0000 7f9cdbdb5640 1 -- 172.21.15.64:0/3015181849 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f9cd41a93b0 con 0x7f9cd41049b0 2024-09-21T03:55:38.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.693+0000 7f9cd2ffd640 1 -- 172.21.15.64:0/3015181849 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v18)=0 v18) ==== 106+0+0 (secure 0 0 0) 0x7f9ccc00f3c0 con 0x7f9cd41049b0 2024-09-21T03:55:38.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.696+0000 7f9cdbdb5640 1 -- 172.21.15.64:0/3015181849 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f9cbc03fb70 msgr2=0x7f9cbc042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:38.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.696+0000 7f9cdbdb5640 1 --2- 172.21.15.64:0/3015181849 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f9cbc03fb70 0x7f9cbc042080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9cc802aa00 tx=0x7f9cc8005a20 comp rx=0 tx=0).stop 2024-09-21T03:55:38.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.696+0000 7f9cdbdb5640 1 -- 172.21.15.64:0/3015181849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cd41049b0 msgr2=0x7f9cd419c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:38.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.696+0000 7f9cdbdb5640 1 --2- 172.21.15.64:0/3015181849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cd41049b0 0x7f9cd419c710 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00d770 tx=0x7f9ccc00dc30 comp rx=0 tx=0).stop 2024-09-21T03:55:38.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.696+0000 7f9cdbdb5640 1 -- 172.21.15.64:0/3015181849 shutdown_connections 2024-09-21T03:55:38.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.696+0000 7f9cdbdb5640 1 --2- 172.21.15.64:0/3015181849 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f9cbc03fb70 0x7f9cbc042080 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:38.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.696+0000 7f9cdbdb5640 1 --2- 172.21.15.64:0/3015181849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cd410de10 0x7f9cd41a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:38.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.696+0000 7f9cdbdb5640 1 --2- 172.21.15.64:0/3015181849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cd4105300 0x7f9cd419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:38.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.696+0000 7f9cdbdb5640 1 --2- 172.21.15.64:0/3015181849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9cd41049b0 0x7f9cd419c710 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:38.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.696+0000 7f9cdbdb5640 1 -- 172.21.15.64:0/3015181849 >> 172.21.15.64:0/3015181849 conn(0x7f9cd4079730 msgr2=0x7f9cd41036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:38.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.696+0000 7f9cdbdb5640 1 -- 172.21.15.64:0/3015181849 shutdown_connections 2024-09-21T03:55:38.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:38.696+0000 7f9cdbdb5640 1 -- 172.21.15.64:0/3015181849 wait complete. 2024-09-21T03:55:39.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:39 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3015181849' entity='client.admin' 2024-09-21T03:55:39.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:39 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:39.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:39 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:39.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:39 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:39.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:39 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:39.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:39 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:39.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:39 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:55:39.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:39 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:39.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:39 smithi064 ceph-mon[31180]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T03:55:39.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:39 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:55:39.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:39 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:55:39.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:39 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:39.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:39 smithi064 ceph-mon[31180]: Reconfiguring daemon mgr.x on smithi064 2024-09-21T03:55:39.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:39 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:40.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:39 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3015181849' entity='client.admin' 2024-09-21T03:55:40.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:39 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:40.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:39 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:40.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:39 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:40.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:39 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:40.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:39 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:40.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:39 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:55:40.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:39 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:40.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:39 smithi119 ceph-mon[34811]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T03:55:40.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:39 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:55:40.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:39 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:55:40.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:39 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:40.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:39 smithi119 ceph-mon[34811]: Reconfiguring daemon mgr.x on smithi064 2024-09-21T03:55:40.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:39 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:40.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:39 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3015181849' entity='client.admin' 2024-09-21T03:55:40.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:39 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:40.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:39 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:40.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:39 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:40.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:39 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:40.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:39 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:40.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:39 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:55:40.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:39 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:40.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:39 smithi131 ceph-mon[35193]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T03:55:40.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:39 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T03:55:40.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:39 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T03:55:40.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:39 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:40.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:39 smithi131 ceph-mon[35193]: Reconfiguring daemon mgr.x on smithi064 2024-09-21T03:55:40.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:39 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:40.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.445+0000 7f4f3ccb5640 1 -- 172.21.15.64:0/2793683942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f38108d20 msgr2=0x7f4f3810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:40.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.445+0000 7f4f3ccb5640 1 --2- 172.21.15.64:0/2793683942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f38108d20 0x7f4f3810f550 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009ca0 tx=0x7f4f3002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:55:40.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.445+0000 7f4f3ccb5640 1 -- 172.21.15.64:0/2793683942 shutdown_connections 2024-09-21T03:55:40.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.445+0000 7f4f3ccb5640 1 --2- 172.21.15.64:0/2793683942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f38108d20 0x7f4f3810f550 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:40.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.445+0000 7f4f3ccb5640 1 --2- 172.21.15.64:0/2793683942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f381083d0 0x7f4f381087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:40.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.445+0000 7f4f3ccb5640 1 --2- 172.21.15.64:0/2793683942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f38102a90 0x7f4f38102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:40.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.445+0000 7f4f3ccb5640 1 -- 172.21.15.64:0/2793683942 >> 172.21.15.64:0/2793683942 conn(0x7f4f38079730 msgr2=0x7f4f38079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:40.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.445+0000 7f4f3ccb5640 1 -- 172.21.15.64:0/2793683942 shutdown_connections 2024-09-21T03:55:40.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.445+0000 7f4f3ccb5640 1 -- 172.21.15.64:0/2793683942 wait complete. 2024-09-21T03:55:40.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.446+0000 7f4f3ccb5640 1 Processor -- start 2024-09-21T03:55:40.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.447+0000 7f4f3ccb5640 1 -- start start 2024-09-21T03:55:40.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.447+0000 7f4f3ccb5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f38102a90 0x7f4f3819b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:40.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.447+0000 7f4f3ccb5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f381083d0 0x7f4f3819b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:40.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.447+0000 7f4f3ccb5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f38108d20 0x7f4f3819bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:40.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.447+0000 7f4f3ccb5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4f381129e0 con 0x7f4f381083d0 2024-09-21T03:55:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.447+0000 7f4f3ccb5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4f38112860 con 0x7f4f38108d20 2024-09-21T03:55:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.447+0000 7f4f3ccb5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4f38112b60 con 0x7f4f38102a90 2024-09-21T03:55:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.447+0000 7f4f2dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f381083d0 0x7f4f3819b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.447+0000 7f4f2dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f381083d0 0x7f4f3819b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43034/0 (socket says 172.21.15.64:43034) 2024-09-21T03:55:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.447+0000 7f4f2dd9b640 1 -- 172.21.15.64:0/3376635841 learned_addr learned my addr 172.21.15.64:0/3376635841 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:55:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.447+0000 7f4f3659c640 1 --2- 172.21.15.64:0/3376635841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f38102a90 0x7f4f3819b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.447+0000 7f4f36d9d640 1 --2- 172.21.15.64:0/3376635841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f38108d20 0x7f4f3819bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.447+0000 7f4f2dd9b640 1 -- 172.21.15.64:0/3376635841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f38102a90 msgr2=0x7f4f3819b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.447+0000 7f4f2dd9b640 1 --2- 172.21.15.64:0/3376635841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f38102a90 0x7f4f3819b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.447+0000 7f4f2dd9b640 1 -- 172.21.15.64:0/3376635841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f38108d20 msgr2=0x7f4f3819bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.447+0000 7f4f2dd9b640 1 --2- 172.21.15.64:0/3376635841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f38108d20 0x7f4f3819bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.447+0000 7f4f2dd9b640 1 -- 172.21.15.64:0/3376635841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f381a6520 con 0x7f4f381083d0 2024-09-21T03:55:40.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.447+0000 7f4f3659c640 1 --2- 172.21.15.64:0/3376635841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f38102a90 0x7f4f3819b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:55:40.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.448+0000 7f4f2dd9b640 1 --2- 172.21.15.64:0/3376635841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f381083d0 0x7f4f3819b750 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4f2000e9c0 tx=0x7f4f2000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:40.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.448+0000 7f4f2ffff640 1 -- 172.21.15.64:0/3376635841 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2000ccc0 con 0x7f4f381083d0 2024-09-21T03:55:40.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.448+0000 7f4f3ccb5640 1 -- 172.21.15.64:0/3376635841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f381a6810 con 0x7f4f381083d0 2024-09-21T03:55:40.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.448+0000 7f4f2ffff640 1 -- 172.21.15.64:0/3376635841 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f200054d0 con 0x7f4f381083d0 2024-09-21T03:55:40.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.448+0000 7f4f2ffff640 1 -- 172.21.15.64:0/3376635841 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f20010430 con 0x7f4f381083d0 2024-09-21T03:55:40.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.448+0000 7f4f3ccb5640 1 -- 172.21.15.64:0/3376635841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f381a6d50 con 0x7f4f381083d0 2024-09-21T03:55:40.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.449+0000 7f4f2ffff640 1 -- 172.21.15.64:0/3376635841 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7f4f200105d0 con 0x7f4f381083d0 2024-09-21T03:55:40.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.449+0000 7f4f3ccb5640 1 -- 172.21.15.64:0/3376635841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f38103d10 con 0x7f4f381083d0 2024-09-21T03:55:40.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.449+0000 7f4f2ffff640 1 --2- 172.21.15.64:0/3376635841 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4f2403fb70 0x7f4f24042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:40.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.449+0000 7f4f2ffff640 1 -- 172.21.15.64:0/3376635841 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f4f200046c0 con 0x7f4f381083d0 2024-09-21T03:55:40.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.449+0000 7f4f3659c640 1 --2- 172.21.15.64:0/3376635841 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4f2403fb70 0x7f4f24042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:40.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.450+0000 7f4f3659c640 1 --2- 172.21.15.64:0/3376635841 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4f2403fb70 0x7f4f24042080 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4f3806c740 tx=0x7f4f2800b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:40.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.454+0000 7f4f2ffff640 1 -- 172.21.15.64:0/3376635841 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4f20016330 con 0x7f4f381083d0 2024-09-21T03:55:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.647+0000 7f4f3ccb5640 1 -- 172.21.15.64:0/3376635841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f4f3802e640 con 0x7f4f381083d0 2024-09-21T03:55:40.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.651+0000 7f4f2ffff640 1 -- 172.21.15.64:0/3376635841 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v19)=0 v19) ==== 108+0+0 (secure 0 0 0) 0x7f4f200580b0 con 0x7f4f381083d0 2024-09-21T03:55:40.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.653+0000 7f4f3ccb5640 1 -- 172.21.15.64:0/3376635841 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4f2403fb70 msgr2=0x7f4f24042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:40.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.653+0000 7f4f3ccb5640 1 --2- 172.21.15.64:0/3376635841 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4f2403fb70 0x7f4f24042080 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4f3806c740 tx=0x7f4f2800b410 comp rx=0 tx=0).stop 2024-09-21T03:55:40.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.653+0000 7f4f3ccb5640 1 -- 172.21.15.64:0/3376635841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f381083d0 msgr2=0x7f4f3819b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:40.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.653+0000 7f4f3ccb5640 1 --2- 172.21.15.64:0/3376635841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f381083d0 0x7f4f3819b750 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4f2000e9c0 tx=0x7f4f2000ee80 comp rx=0 tx=0).stop 2024-09-21T03:55:40.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.653+0000 7f4f3ccb5640 1 -- 172.21.15.64:0/3376635841 shutdown_connections 2024-09-21T03:55:40.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.653+0000 7f4f3ccb5640 1 --2- 172.21.15.64:0/3376635841 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4f2403fb70 0x7f4f24042080 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:40.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.653+0000 7f4f3ccb5640 1 --2- 172.21.15.64:0/3376635841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f38108d20 0x7f4f3819bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:40.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.653+0000 7f4f3ccb5640 1 --2- 172.21.15.64:0/3376635841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f381083d0 0x7f4f3819b750 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:40.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.653+0000 7f4f3ccb5640 1 --2- 172.21.15.64:0/3376635841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f38102a90 0x7f4f3819b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:40.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.653+0000 7f4f3ccb5640 1 -- 172.21.15.64:0/3376635841 >> 172.21.15.64:0/3376635841 conn(0x7f4f38079730 msgr2=0x7f4f38106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:40.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.653+0000 7f4f3ccb5640 1 -- 172.21.15.64:0/3376635841 shutdown_connections 2024-09-21T03:55:40.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.653+0000 7f4f3ccb5640 1 -- 172.21.15.64:0/3376635841 wait complete. 2024-09-21T03:55:40.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.904+0000 7f9057741640 1 -- 172.21.15.64:0/1257358871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9050102ae0 msgr2=0x7f9050102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:40.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.904+0000 7f9057741640 1 --2- 172.21.15.64:0/1257358871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9050102ae0 0x7f9050102ef0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f904c009900 tx=0x7f904c02a410 comp rx=0 tx=0).stop 2024-09-21T03:55:40.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.904+0000 7f9057741640 1 -- 172.21.15.64:0/1257358871 shutdown_connections 2024-09-21T03:55:40.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.904+0000 7f9057741640 1 --2- 172.21.15.64:0/1257358871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9050108d20 0x7f905010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:40.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.904+0000 7f9057741640 1 --2- 172.21.15.64:0/1257358871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90501083d0 0x7f90501087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:40.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.904+0000 7f9057741640 1 --2- 172.21.15.64:0/1257358871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9050102ae0 0x7f9050102ef0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:40.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.904+0000 7f9057741640 1 -- 172.21.15.64:0/1257358871 >> 172.21.15.64:0/1257358871 conn(0x7f9050079760 msgr2=0x7f9050079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:40.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.904+0000 7f9057741640 1 -- 172.21.15.64:0/1257358871 shutdown_connections 2024-09-21T03:55:40.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.904+0000 7f9057741640 1 -- 172.21.15.64:0/1257358871 wait complete. 2024-09-21T03:55:40.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.905+0000 7f9057741640 1 Processor -- start 2024-09-21T03:55:40.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.905+0000 7f9057741640 1 -- start start 2024-09-21T03:55:40.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.905+0000 7f9057741640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9050102ae0 0x7f905019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:40.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.905+0000 7f9057741640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90501083d0 0x7f905019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:40.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.905+0000 7f9057741640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9050108d20 0x7f905019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:40.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.905+0000 7f9057741640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9050112990 con 0x7f9050102ae0 2024-09-21T03:55:40.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.905+0000 7f9057741640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9050112810 con 0x7f90501083d0 2024-09-21T03:55:40.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.905+0000 7f9057741640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9050112b10 con 0x7f9050108d20 2024-09-21T03:55:40.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.905+0000 7f90554dd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9050102ae0 0x7f905019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:40.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.905+0000 7f90554dd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9050102ae0 0x7f905019a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43042/0 (socket says 172.21.15.64:43042) 2024-09-21T03:55:40.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.905+0000 7f90554dd640 1 -- 172.21.15.64:0/2702100742 learned_addr learned my addr 172.21.15.64:0/2702100742 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:55:40.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.905+0000 7f9055cde640 1 --2- 172.21.15.64:0/2702100742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9050108d20 0x7f905019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:40.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.905+0000 7f9054cdc640 1 --2- 172.21.15.64:0/2702100742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90501083d0 0x7f905019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:40.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.906+0000 7f90554dd640 1 -- 172.21.15.64:0/2702100742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9050108d20 msgr2=0x7f905019fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:40.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.906+0000 7f90554dd640 1 --2- 172.21.15.64:0/2702100742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9050108d20 0x7f905019fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:40.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.906+0000 7f90554dd640 1 -- 172.21.15.64:0/2702100742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90501083d0 msgr2=0x7f905019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:40.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.906+0000 7f90554dd640 1 --2- 172.21.15.64:0/2702100742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90501083d0 0x7f905019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:40.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.906+0000 7f90554dd640 1 -- 172.21.15.64:0/2702100742 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90501a03c0 con 0x7f9050102ae0 2024-09-21T03:55:40.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.906+0000 7f90554dd640 1 --2- 172.21.15.64:0/2702100742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9050102ae0 0x7f905019a450 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f904c005940 tx=0x7f904c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:40.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.906+0000 7f90427fc640 1 -- 172.21.15.64:0/2702100742 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c03f030 con 0x7f9050102ae0 2024-09-21T03:55:40.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.906+0000 7f90427fc640 1 -- 172.21.15.64:0/2702100742 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f904c003cb0 con 0x7f9050102ae0 2024-09-21T03:55:40.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.906+0000 7f9057741640 1 -- 172.21.15.64:0/2702100742 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90501a0650 con 0x7f9050102ae0 2024-09-21T03:55:40.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.906+0000 7f90427fc640 1 -- 172.21.15.64:0/2702100742 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c035070 con 0x7f9050102ae0 2024-09-21T03:55:40.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.906+0000 7f9057741640 1 -- 172.21.15.64:0/2702100742 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90501a6e10 con 0x7f9050102ae0 2024-09-21T03:55:40.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.907+0000 7f9057741640 1 -- 172.21.15.64:0/2702100742 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9050103d10 con 0x7f9050102ae0 2024-09-21T03:55:40.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.908+0000 7f90427fc640 1 -- 172.21.15.64:0/2702100742 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 103194+0+0 (secure 0 0 0) 0x7f904c042030 con 0x7f9050102ae0 2024-09-21T03:55:40.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.908+0000 7f90427fc640 1 --2- 172.21.15.64:0/2702100742 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f903c07b020 0x7f903c07d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:40.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.909+0000 7f9054cdc640 1 --2- 172.21.15.64:0/2702100742 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f903c07b020 0x7f903c07d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:40.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.909+0000 7f90427fc640 1 -- 172.21.15.64:0/2702100742 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f904c0bd670 con 0x7f9050102ae0 2024-09-21T03:55:40.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.909+0000 7f9054cdc640 1 --2- 172.21.15.64:0/2702100742 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f903c07b020 0x7f903c07d530 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9050100d20 tx=0x7f904400b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:40.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:40.913+0000 7f90427fc640 1 -- 172.21.15.64:0/2702100742 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f904c08a080 con 0x7f9050102ae0 2024-09-21T03:55:41.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:40 smithi119 ceph-mon[34811]: Standby manager daemon z started 2024-09-21T03:55:41.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:40 smithi119 ceph-mon[34811]: from='mgr.? 172.21.15.131:0/3087402320' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T03:55:41.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:40 smithi119 ceph-mon[34811]: from='mgr.? 172.21.15.131:0/3087402320' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:55:41.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:40 smithi119 ceph-mon[34811]: from='mgr.? 172.21.15.131:0/3087402320' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T03:55:41.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:40 smithi119 ceph-mon[34811]: from='mgr.? 172.21.15.131:0/3087402320' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:55:41.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:40 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3376635841' entity='client.admin' 2024-09-21T03:55:41.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:40 smithi064 ceph-mon[31180]: Standby manager daemon z started 2024-09-21T03:55:41.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:40 smithi064 ceph-mon[31180]: from='mgr.? 172.21.15.131:0/3087402320' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T03:55:41.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:40 smithi064 ceph-mon[31180]: from='mgr.? 172.21.15.131:0/3087402320' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:55:41.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:40 smithi064 ceph-mon[31180]: from='mgr.? 172.21.15.131:0/3087402320' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T03:55:41.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:40 smithi064 ceph-mon[31180]: from='mgr.? 172.21.15.131:0/3087402320' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:55:41.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:40 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3376635841' entity='client.admin' 2024-09-21T03:55:41.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:40 smithi131 ceph-mon[35193]: Standby manager daemon z started 2024-09-21T03:55:41.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:40 smithi131 ceph-mon[35193]: from='mgr.? 172.21.15.131:0/3087402320' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T03:55:41.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:40 smithi131 ceph-mon[35193]: from='mgr.? 172.21.15.131:0/3087402320' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:55:41.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:40 smithi131 ceph-mon[35193]: from='mgr.? 172.21.15.131:0/3087402320' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T03:55:41.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:40 smithi131 ceph-mon[35193]: from='mgr.? 172.21.15.131:0/3087402320' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:55:41.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:40 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3376635841' entity='client.admin' 2024-09-21T03:55:41.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.103+0000 7f9057741640 1 -- 172.21.15.64:0/2702100742 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f90501a7070 con 0x7f9050102ae0 2024-09-21T03:55:41.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.106+0000 7f90427fc640 1 -- 172.21.15.64:0/2702100742 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v20)=0 v20) ==== 119+0+0 (secure 0 0 0) 0x7f904c08ef30 con 0x7f9050102ae0 2024-09-21T03:55:41.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.109+0000 7f9057741640 1 -- 172.21.15.64:0/2702100742 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f903c07b020 msgr2=0x7f903c07d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:41.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.109+0000 7f9057741640 1 --2- 172.21.15.64:0/2702100742 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f903c07b020 0x7f903c07d530 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9050100d20 tx=0x7f904400b390 comp rx=0 tx=0).stop 2024-09-21T03:55:41.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.109+0000 7f9057741640 1 -- 172.21.15.64:0/2702100742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9050102ae0 msgr2=0x7f905019a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:41.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.109+0000 7f9057741640 1 --2- 172.21.15.64:0/2702100742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9050102ae0 0x7f905019a450 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f904c005940 tx=0x7f904c0042f0 comp rx=0 tx=0).stop 2024-09-21T03:55:41.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.109+0000 7f9057741640 1 -- 172.21.15.64:0/2702100742 shutdown_connections 2024-09-21T03:55:41.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.109+0000 7f9057741640 1 --2- 172.21.15.64:0/2702100742 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f903c07b020 0x7f903c07d530 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:41.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.109+0000 7f9057741640 1 --2- 172.21.15.64:0/2702100742 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9050108d20 0x7f905019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:41.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.109+0000 7f9057741640 1 --2- 172.21.15.64:0/2702100742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90501083d0 0x7f905019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:41.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.109+0000 7f9057741640 1 --2- 172.21.15.64:0/2702100742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9050102ae0 0x7f905019a450 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:41.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.109+0000 7f9057741640 1 -- 172.21.15.64:0/2702100742 >> 172.21.15.64:0/2702100742 conn(0x7f9050079760 msgr2=0x7f90501066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:41.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.109+0000 7f9057741640 1 -- 172.21.15.64:0/2702100742 shutdown_connections 2024-09-21T03:55:41.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.109+0000 7f9057741640 1 -- 172.21.15.64:0/2702100742 wait complete. 2024-09-21T03:55:41.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.619+0000 7f840f2d7640 1 -- 172.21.15.64:0/661806891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8408108d20 msgr2=0x7f840810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:41.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.619+0000 7f840f2d7640 1 --2- 172.21.15.64:0/661806891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8408108d20 0x7f840810f550 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8400009ca0 tx=0x7f840002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:55:41.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.619+0000 7f840f2d7640 1 -- 172.21.15.64:0/661806891 shutdown_connections 2024-09-21T03:55:41.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.619+0000 7f840f2d7640 1 --2- 172.21.15.64:0/661806891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8408108d20 0x7f840810f550 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:41.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.619+0000 7f840f2d7640 1 --2- 172.21.15.64:0/661806891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84081083d0 0x7f84081087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:41.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.619+0000 7f840f2d7640 1 --2- 172.21.15.64:0/661806891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8408102ae0 0x7f8408102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:41.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.619+0000 7f840f2d7640 1 -- 172.21.15.64:0/661806891 >> 172.21.15.64:0/661806891 conn(0x7f84080797b0 msgr2=0x7f8408079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:41.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.619+0000 7f840f2d7640 1 -- 172.21.15.64:0/661806891 shutdown_connections 2024-09-21T03:55:41.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.619+0000 7f840f2d7640 1 -- 172.21.15.64:0/661806891 wait complete. 2024-09-21T03:55:41.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.620+0000 7f840f2d7640 1 Processor -- start 2024-09-21T03:55:41.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.620+0000 7f840f2d7640 1 -- start start 2024-09-21T03:55:41.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.620+0000 7f840f2d7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8408102ae0 0x7f840819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:41.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.620+0000 7f840f2d7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84081083d0 0x7f840819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:41.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.620+0000 7f840f2d7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8408108d20 0x7f840819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:41.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.620+0000 7f840f2d7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f84081129e0 con 0x7f84081083d0 2024-09-21T03:55:41.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.620+0000 7f840f2d7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8408112860 con 0x7f8408102ae0 2024-09-21T03:55:41.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.620+0000 7f840f2d7640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8408112b60 con 0x7f8408108d20 2024-09-21T03:55:41.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.620+0000 7f840c872640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84081083d0 0x7f840819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:41.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.620+0000 7f840c872640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84081083d0 0x7f840819a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56408/0 (socket says 172.21.15.64:56408) 2024-09-21T03:55:41.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.620+0000 7f840c872640 1 -- 172.21.15.64:0/702058846 learned_addr learned my addr 172.21.15.64:0/702058846 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:55:41.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.620+0000 7f840d073640 1 --2- 172.21.15.64:0/702058846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8408102ae0 0x7f840819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:41.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.620+0000 7f840d874640 1 --2- 172.21.15.64:0/702058846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8408108d20 0x7f840819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:41.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.621+0000 7f840c872640 1 -- 172.21.15.64:0/702058846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8408108d20 msgr2=0x7f840819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:41.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.621+0000 7f840c872640 1 --2- 172.21.15.64:0/702058846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8408108d20 0x7f840819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:41.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.621+0000 7f840c872640 1 -- 172.21.15.64:0/702058846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8408102ae0 msgr2=0x7f840819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:41.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.621+0000 7f840c872640 1 --2- 172.21.15.64:0/702058846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8408102ae0 0x7f840819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:41.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.621+0000 7f840c872640 1 -- 172.21.15.64:0/702058846 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84081a03b0 con 0x7f84081083d0 2024-09-21T03:55:41.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.621+0000 7f840c872640 1 --2- 172.21.15.64:0/702058846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84081083d0 0x7f840819a9e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00e9c0 tx=0x7f83fc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:41.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.621+0000 7f83fa7fc640 1 -- 172.21.15.64:0/702058846 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc00ccc0 con 0x7f84081083d0 2024-09-21T03:55:41.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.621+0000 7f83fa7fc640 1 -- 172.21.15.64:0/702058846 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83fc0054d0 con 0x7f84081083d0 2024-09-21T03:55:41.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.621+0000 7f840f2d7640 1 -- 172.21.15.64:0/702058846 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84081a06a0 con 0x7f84081083d0 2024-09-21T03:55:41.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.621+0000 7f83fa7fc640 1 -- 172.21.15.64:0/702058846 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc010430 con 0x7f84081083d0 2024-09-21T03:55:41.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.621+0000 7f840f2d7640 1 -- 172.21.15.64:0/702058846 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84081a6e00 con 0x7f84081083d0 2024-09-21T03:55:41.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.622+0000 7f840f2d7640 1 -- 172.21.15.64:0/702058846 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8408102ef0 con 0x7f84081083d0 2024-09-21T03:55:41.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.623+0000 7f83fa7fc640 1 -- 172.21.15.64:0/702058846 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 103194+0+0 (secure 0 0 0) 0x7f83fc005020 con 0x7f84081083d0 2024-09-21T03:55:41.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.623+0000 7f83fa7fc640 1 --2- 172.21.15.64:0/702058846 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f83f407b020 0x7f83f407d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:41.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.623+0000 7f83fa7fc640 1 -- 172.21.15.64:0/702058846 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f83fc09d240 con 0x7f84081083d0 2024-09-21T03:55:41.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.623+0000 7f840d073640 1 --2- 172.21.15.64:0/702058846 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f83f407b020 0x7f83f407d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:41.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.624+0000 7f840d073640 1 --2- 172.21.15.64:0/702058846 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f83f407b020 0x7f83f407d530 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8404009bd0 tx=0x7f8404005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:41.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.628+0000 7f83fa7fc640 1 -- 172.21.15.64:0/702058846 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f83fc0657d0 con 0x7f84081083d0 2024-09-21T03:55:41.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.824+0000 7f840f2d7640 1 -- 172.21.15.64:0/702058846 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7f84081a70a0 con 0x7f84081083d0 2024-09-21T03:55:41.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.828+0000 7f83fa7fc640 1 -- 172.21.15.64:0/702058846 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v22)=0 v22) ==== 127+0+0 (secure 0 0 0) 0x7f83fc069bb0 con 0x7f84081083d0 2024-09-21T03:55:41.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.830+0000 7f840f2d7640 1 -- 172.21.15.64:0/702058846 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f83f407b020 msgr2=0x7f83f407d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:41.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.830+0000 7f840f2d7640 1 --2- 172.21.15.64:0/702058846 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f83f407b020 0x7f83f407d530 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8404009bd0 tx=0x7f8404005f90 comp rx=0 tx=0).stop 2024-09-21T03:55:41.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.830+0000 7f840f2d7640 1 -- 172.21.15.64:0/702058846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84081083d0 msgr2=0x7f840819a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:41.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.830+0000 7f840f2d7640 1 --2- 172.21.15.64:0/702058846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84081083d0 0x7f840819a9e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00e9c0 tx=0x7f83fc00ee80 comp rx=0 tx=0).stop 2024-09-21T03:55:41.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.831+0000 7f840f2d7640 1 -- 172.21.15.64:0/702058846 shutdown_connections 2024-09-21T03:55:41.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.831+0000 7f840f2d7640 1 --2- 172.21.15.64:0/702058846 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f83f407b020 0x7f83f407d530 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:41.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.831+0000 7f840f2d7640 1 --2- 172.21.15.64:0/702058846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8408108d20 0x7f840819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:41.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.831+0000 7f840f2d7640 1 --2- 172.21.15.64:0/702058846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84081083d0 0x7f840819a9e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:41.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.831+0000 7f840f2d7640 1 --2- 172.21.15.64:0/702058846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8408102ae0 0x7f840819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:41.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.831+0000 7f840f2d7640 1 -- 172.21.15.64:0/702058846 >> 172.21.15.64:0/702058846 conn(0x7f84080797b0 msgr2=0x7f84081018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:41.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.831+0000 7f840f2d7640 1 -- 172.21.15.64:0/702058846 shutdown_connections 2024-09-21T03:55:41.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.831+0000 7f840f2d7640 1 -- 172.21.15.64:0/702058846 wait complete. 2024-09-21T03:55:41.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.396+0000 7f1284695640 1 -- 172.21.15.64:0/4240451824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f127c104880 msgr2=0x7f127c104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:41.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.396+0000 7f1284695640 1 --2- 172.21.15.64:0/4240451824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f127c104880 0x7f127c104c90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1278009900 tx=0x7f127802a410 comp rx=0 tx=0).stop 2024-09-21T03:55:41.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.396+0000 7f1284695640 1 -- 172.21.15.64:0/4240451824 shutdown_connections 2024-09-21T03:55:41.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.396+0000 7f1284695640 1 --2- 172.21.15.64:0/4240451824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f127c10de30 0x7f127c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:41.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.396+0000 7f1284695640 1 --2- 172.21.15.64:0/4240451824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f127c1052b0 0x7f127c10d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:41.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.396+0000 7f1284695640 1 --2- 172.21.15.64:0/4240451824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f127c104880 0x7f127c104c90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:41.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.397+0000 7f1284695640 1 -- 172.21.15.64:0/4240451824 >> 172.21.15.64:0/4240451824 conn(0x7f127c079730 msgr2=0x7f127c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:41.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.397+0000 7f1284695640 1 -- 172.21.15.64:0/4240451824 shutdown_connections 2024-09-21T03:55:41.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.397+0000 7f1284695640 1 -- 172.21.15.64:0/4240451824 wait complete. 2024-09-21T03:55:41.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.397+0000 7f1284695640 1 Processor -- start 2024-09-21T03:55:41.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.397+0000 7f1284695640 1 -- start start 2024-09-21T03:55:41.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.398+0000 7f1284695640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f127c104880 0x7f127c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:41.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.398+0000 7f1284695640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f127c1052b0 0x7f127c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:41.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.398+0000 7f1284695640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f127c10de30 0x7f127c19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:41.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.398+0000 7f1284695640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f127c113a70 con 0x7f127c10de30 2024-09-21T03:55:41.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.398+0000 7f1284695640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f127c1138f0 con 0x7f127c1052b0 2024-09-21T03:55:41.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.398+0000 7f1284695640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f127c113bf0 con 0x7f127c104880 2024-09-21T03:55:41.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.398+0000 7f1282c32640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f127c10de30 0x7f127c19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:41.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.398+0000 7f1281c30640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f127c1052b0 0x7f127c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:41.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.398+0000 7f1282c32640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f127c10de30 0x7f127c19fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56394/0 (socket says 172.21.15.64:56394) 2024-09-21T03:55:41.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.398+0000 7f1282c32640 1 -- 172.21.15.64:0/1366553749 learned_addr learned my addr 172.21.15.64:0/1366553749 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:55:41.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.398+0000 7f1282431640 1 --2- 172.21.15.64:0/1366553749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f127c104880 0x7f127c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:41.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.398+0000 7f1282c32640 1 -- 172.21.15.64:0/1366553749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f127c104880 msgr2=0x7f127c19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:41.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.398+0000 7f1282c32640 1 --2- 172.21.15.64:0/1366553749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f127c104880 0x7f127c19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:41.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.398+0000 7f1282c32640 1 -- 172.21.15.64:0/1366553749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f127c1052b0 msgr2=0x7f127c19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:41.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.398+0000 7f1282c32640 1 --2- 172.21.15.64:0/1366553749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f127c1052b0 0x7f127c19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:41.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.398+0000 7f1282c32640 1 -- 172.21.15.64:0/1366553749 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f127c1a0450 con 0x7f127c10de30 2024-09-21T03:55:41.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.398+0000 7f1282431640 1 --2- 172.21.15.64:0/1366553749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f127c104880 0x7f127c19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:41.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.398+0000 7f1281c30640 1 --2- 172.21.15.64:0/1366553749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f127c1052b0 0x7f127c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:55:41.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.398+0000 7f1282c32640 1 --2- 172.21.15.64:0/1366553749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f127c10de30 0x7f127c19fd20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f127400eb10 tx=0x7f127400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:41.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.399+0000 7f12737fe640 1 -- 172.21.15.64:0/1366553749 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1274004b20 con 0x7f127c10de30 2024-09-21T03:55:41.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.399+0000 7f1284695640 1 -- 172.21.15.64:0/1366553749 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f127c1a0740 con 0x7f127c10de30 2024-09-21T03:55:41.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.399+0000 7f12737fe640 1 -- 172.21.15.64:0/1366553749 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1274004cc0 con 0x7f127c10de30 2024-09-21T03:55:41.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.399+0000 7f12737fe640 1 -- 172.21.15.64:0/1366553749 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1274010670 con 0x7f127c10de30 2024-09-21T03:55:41.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.399+0000 7f1284695640 1 -- 172.21.15.64:0/1366553749 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f127c1a6ea0 con 0x7f127c10de30 2024-09-21T03:55:41.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.400+0000 7f1284695640 1 -- 172.21.15.64:0/1366553749 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f127c104c90 con 0x7f127c10de30 2024-09-21T03:55:41.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.401+0000 7f12737fe640 1 -- 172.21.15.64:0/1366553749 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 103194+0+0 (secure 0 0 0) 0x7f1274005020 con 0x7f127c10de30 2024-09-21T03:55:41.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.401+0000 7f12737fe640 1 --2- 172.21.15.64:0/1366553749 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f126807b020 0x7f126807d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:41.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.401+0000 7f1282431640 1 --2- 172.21.15.64:0/1366553749 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f126807b020 0x7f126807d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:41.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.402+0000 7f12737fe640 1 -- 172.21.15.64:0/1366553749 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f127409d430 con 0x7f127c10de30 2024-09-21T03:55:41.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.402+0000 7f1282431640 1 --2- 172.21.15.64:0/1366553749 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f126807b020 0x7f126807d530 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1278000c00 tx=0x7f1278032cc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:41.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.406+0000 7f12737fe640 1 -- 172.21.15.64:0/1366553749 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f12740659e0 con 0x7f127c10de30 2024-09-21T03:55:41.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.623+0000 7f1284695640 1 -- 172.21.15.64:0/1366553749 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f127c108fc0 con 0x7f127c10de30 2024-09-21T03:55:41.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.631+0000 7f12737fe640 1 -- 172.21.15.64:0/1366553749 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v21)=0 v21) ==== 120+0+0 (secure 0 0 0) 0x7f1274069dc0 con 0x7f127c10de30 2024-09-21T03:55:41.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.635+0000 7f1284695640 1 -- 172.21.15.64:0/1366553749 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f126807b020 msgr2=0x7f126807d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:41.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.635+0000 7f1284695640 1 --2- 172.21.15.64:0/1366553749 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f126807b020 0x7f126807d530 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1278000c00 tx=0x7f1278032cc0 comp rx=0 tx=0).stop 2024-09-21T03:55:41.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.635+0000 7f1284695640 1 -- 172.21.15.64:0/1366553749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f127c10de30 msgr2=0x7f127c19fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:41.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.635+0000 7f1284695640 1 --2- 172.21.15.64:0/1366553749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f127c10de30 0x7f127c19fd20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f127400eb10 tx=0x7f127400c2d0 comp rx=0 tx=0).stop 2024-09-21T03:55:41.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.635+0000 7f1284695640 1 -- 172.21.15.64:0/1366553749 shutdown_connections 2024-09-21T03:55:41.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.635+0000 7f1284695640 1 --2- 172.21.15.64:0/1366553749 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f126807b020 0x7f126807d530 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:41.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.635+0000 7f1284695640 1 --2- 172.21.15.64:0/1366553749 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f127c10de30 0x7f127c19fd20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:41.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.635+0000 7f1284695640 1 --2- 172.21.15.64:0/1366553749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f127c1052b0 0x7f127c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:41.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.635+0000 7f1284695640 1 --2- 172.21.15.64:0/1366553749 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f127c104880 0x7f127c19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:41.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.635+0000 7f1284695640 1 -- 172.21.15.64:0/1366553749 >> 172.21.15.64:0/1366553749 conn(0x7f127c079730 msgr2=0x7f127c1008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:41.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.635+0000 7f1284695640 1 -- 172.21.15.64:0/1366553749 shutdown_connections 2024-09-21T03:55:41.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:41.635+0000 7f1284695640 1 -- 172.21.15.64:0/1366553749 wait complete. 2024-09-21T03:55:42.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:41 smithi119 ceph-mon[34811]: mgrmap e18: x(active, since 12s), standbys: z 2024-09-21T03:55:42.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:41 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T03:55:42.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:41 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2702100742' entity='client.admin' 2024-09-21T03:55:42.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:41 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1366553749' entity='client.admin' 2024-09-21T03:55:42.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:41 smithi064 ceph-mon[31180]: mgrmap e18: x(active, since 12s), standbys: z 2024-09-21T03:55:42.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:41 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T03:55:42.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:41 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2702100742' entity='client.admin' 2024-09-21T03:55:42.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:41 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1366553749' entity='client.admin' 2024-09-21T03:55:42.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:41 smithi131 ceph-mon[35193]: mgrmap e18: x(active, since 12s), standbys: z 2024-09-21T03:55:42.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:41 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T03:55:42.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:41 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2702100742' entity='client.admin' 2024-09-21T03:55:42.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:41 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1366553749' entity='client.admin' 2024-09-21T03:55:42.507 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi064') 2024-09-21T03:55:42.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.775+0000 7fd2d2383640 1 -- 172.21.15.64:0/2916601623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2cc1094a0 msgr2=0x7fd2cc1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:42.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.775+0000 7fd2d2383640 1 --2- 172.21.15.64:0/2916601623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2cc1094a0 0x7fd2cc1098b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4009900 tx=0x7fd2b402a410 comp rx=0 tx=0).stop 2024-09-21T03:55:42.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.775+0000 7fd2d2383640 1 -- 172.21.15.64:0/2916601623 shutdown_connections 2024-09-21T03:55:42.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.775+0000 7fd2d2383640 1 --2- 172.21.15.64:0/2916601623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2cc109df0 0x7fd2cc110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:42.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.775+0000 7fd2d2383640 1 --2- 172.21.15.64:0/2916601623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2cc1094a0 0x7fd2cc1098b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:42.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.775+0000 7fd2d2383640 1 --2- 172.21.15.64:0/2916601623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2cc103b60 0x7fd2cc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:42.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.775+0000 7fd2d2383640 1 -- 172.21.15.64:0/2916601623 >> 172.21.15.64:0/2916601623 conn(0x7fd2cc07b850 msgr2=0x7fd2cc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:42.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.775+0000 7fd2d2383640 1 -- 172.21.15.64:0/2916601623 shutdown_connections 2024-09-21T03:55:42.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.775+0000 7fd2d2383640 1 -- 172.21.15.64:0/2916601623 wait complete. 2024-09-21T03:55:42.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.775+0000 7fd2d2383640 1 Processor -- start 2024-09-21T03:55:42.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.776+0000 7fd2d2383640 1 -- start start 2024-09-21T03:55:42.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.776+0000 7fd2d2383640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2cc103b60 0x7fd2cc19c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:42.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.776+0000 7fd2d2383640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2cc1094a0 0x7fd2cc19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:42.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.776+0000 7fd2d2383640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2cc109df0 0x7fd2cc1a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:42.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.776+0000 7fd2d2383640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc113ab0 con 0x7fd2cc109df0 2024-09-21T03:55:42.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.776+0000 7fd2d2383640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc113930 con 0x7fd2cc103b60 2024-09-21T03:55:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.776+0000 7fd2d2383640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc113c30 con 0x7fd2cc1094a0 2024-09-21T03:55:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.776+0000 7fd2d0920640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2cc109df0 0x7fd2cc1a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.776+0000 7fd2cbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2cc103b60 0x7fd2cc19c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.776+0000 7fd2cbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2cc103b60 0x7fd2cc19c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:57550/0 (socket says 172.21.15.64:57550) 2024-09-21T03:55:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.776+0000 7fd2cbfff640 1 -- 172.21.15.64:0/1544593417 learned_addr learned my addr 172.21.15.64:0/1544593417 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:55:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.776+0000 7fd2d0920640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2cc109df0 0x7fd2cc1a1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56414/0 (socket says 172.21.15.64:56414) 2024-09-21T03:55:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.776+0000 7fd2cb7fe640 1 --2- 172.21.15.64:0/1544593417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2cc1094a0 0x7fd2cc19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.777+0000 7fd2cbfff640 1 -- 172.21.15.64:0/1544593417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2cc1094a0 msgr2=0x7fd2cc19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.777+0000 7fd2cbfff640 1 --2- 172.21.15.64:0/1544593417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2cc1094a0 0x7fd2cc19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.777+0000 7fd2cbfff640 1 -- 172.21.15.64:0/1544593417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2cc109df0 msgr2=0x7fd2cc1a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.777+0000 7fd2cbfff640 1 --2- 172.21.15.64:0/1544593417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2cc109df0 0x7fd2cc1a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.777+0000 7fd2cbfff640 1 -- 172.21.15.64:0/1544593417 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2cc1a2590 con 0x7fd2cc103b60 2024-09-21T03:55:42.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.777+0000 7fd2cb7fe640 1 --2- 172.21.15.64:0/1544593417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2cc1094a0 0x7fd2cc19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:42.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.777+0000 7fd2cbfff640 1 --2- 172.21.15.64:0/1544593417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2cc103b60 0x7fd2cc19c6e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00e9f0 tx=0x7fd2bc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:42.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.777+0000 7fd2c97fa640 1 -- 172.21.15.64:0/1544593417 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc00cd80 con 0x7fd2cc103b60 2024-09-21T03:55:42.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.777+0000 7fd2c97fa640 1 -- 172.21.15.64:0/1544593417 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2bc0108e0 con 0x7fd2cc103b60 2024-09-21T03:55:42.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.777+0000 7fd2d2383640 1 -- 172.21.15.64:0/1544593417 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2cc1a2880 con 0x7fd2cc103b60 2024-09-21T03:55:42.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.777+0000 7fd2d2383640 1 -- 172.21.15.64:0/1544593417 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2cc1a90a0 con 0x7fd2cc103b60 2024-09-21T03:55:42.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.777+0000 7fd2c97fa640 1 -- 172.21.15.64:0/1544593417 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc018980 con 0x7fd2cc103b60 2024-09-21T03:55:42.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.778+0000 7fd2d2383640 1 -- 172.21.15.64:0/1544593417 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2cc103f70 con 0x7fd2cc103b60 2024-09-21T03:55:42.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.779+0000 7fd2c97fa640 1 -- 172.21.15.64:0/1544593417 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) ==== 103194+0+0 (secure 0 0 0) 0x7fd2bc010430 con 0x7fd2cc103b60 2024-09-21T03:55:42.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.779+0000 7fd2c97fa640 1 --2- 172.21.15.64:0/1544593417 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd2b807b050 0x7fd2b807d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:42.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.780+0000 7fd2cb7fe640 1 --2- 172.21.15.64:0/1544593417 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd2b807b050 0x7fd2b807d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:42.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.780+0000 7fd2c97fa640 1 -- 172.21.15.64:0/1544593417 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd2bc09d170 con 0x7fd2cc103b60 2024-09-21T03:55:42.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.780+0000 7fd2cb7fe640 1 --2- 172.21.15.64:0/1544593417 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd2b807b050 0x7fd2b807d560 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4009230 tx=0x7fd2b4000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:42.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.785+0000 7fd2c97fa640 1 -- 172.21.15.64:0/1544593417 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd2bc069bc0 con 0x7fd2cc103b60 2024-09-21T03:55:42.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.837+0000 7fd2c97fa640 1 -- 172.21.15.64:0/1544593417 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fd2bc018b20 con 0x7fd2cc103b60 2024-09-21T03:55:42.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.976+0000 7fd2d2383640 1 -- 172.21.15.64:0/1544593417 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7fd2cc1a9270 con 0x7fd2cc103b60 2024-09-21T03:55:42.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.982+0000 7fd2c97fa640 1 -- 172.21.15.64:0/1544593417 <== mon.2 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v23)=0 v23) ==== 118+0+0 (secure 0 0 0) 0x7fd2bc0b6bd0 con 0x7fd2cc103b60 2024-09-21T03:55:42.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.984+0000 7fd2d2383640 1 -- 172.21.15.64:0/1544593417 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd2b807b050 msgr2=0x7fd2b807d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:42.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.984+0000 7fd2d2383640 1 --2- 172.21.15.64:0/1544593417 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd2b807b050 0x7fd2b807d560 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4009230 tx=0x7fd2b4000bc0 comp rx=0 tx=0).stop 2024-09-21T03:55:42.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.984+0000 7fd2d2383640 1 -- 172.21.15.64:0/1544593417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2cc103b60 msgr2=0x7fd2cc19c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:42.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.984+0000 7fd2d2383640 1 --2- 172.21.15.64:0/1544593417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2cc103b60 0x7fd2cc19c6e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00e9f0 tx=0x7fd2bc00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:55:42.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.984+0000 7fd2d2383640 1 -- 172.21.15.64:0/1544593417 shutdown_connections 2024-09-21T03:55:42.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.984+0000 7fd2d2383640 1 --2- 172.21.15.64:0/1544593417 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd2b807b050 0x7fd2b807d560 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:42.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.984+0000 7fd2d2383640 1 --2- 172.21.15.64:0/1544593417 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2cc109df0 0x7fd2cc1a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:42.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.984+0000 7fd2d2383640 1 --2- 172.21.15.64:0/1544593417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2cc1094a0 0x7fd2cc19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:42.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.984+0000 7fd2d2383640 1 --2- 172.21.15.64:0/1544593417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2cc103b60 0x7fd2cc19c6e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:42.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.984+0000 7fd2d2383640 1 -- 172.21.15.64:0/1544593417 >> 172.21.15.64:0/1544593417 conn(0x7fd2cc07b850 msgr2=0x7fd2cc105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:42.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.984+0000 7fd2d2383640 1 -- 172.21.15.64:0/1544593417 shutdown_connections 2024-09-21T03:55:42.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:42.984+0000 7fd2d2383640 1 -- 172.21.15.64:0/1544593417 wait complete. 2024-09-21T03:55:43.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:42 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/702058846' entity='client.admin' 2024-09-21T03:55:43.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:42 smithi064 ceph-mon[31180]: Standby manager daemon y started 2024-09-21T03:55:43.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:42 smithi064 ceph-mon[31180]: from='mgr.? 172.21.15.119:0/438839331' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T03:55:43.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:42 smithi064 ceph-mon[31180]: from='mgr.? 172.21.15.119:0/438839331' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:55:43.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:42 smithi064 ceph-mon[31180]: from='mgr.? 172.21.15.119:0/438839331' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T03:55:43.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:42 smithi064 ceph-mon[31180]: from='mgr.? 172.21.15.119:0/438839331' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:55:43.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:42 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/702058846' entity='client.admin' 2024-09-21T03:55:43.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:42 smithi119 ceph-mon[34811]: Standby manager daemon y started 2024-09-21T03:55:43.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:42 smithi119 ceph-mon[34811]: from='mgr.? 172.21.15.119:0/438839331' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T03:55:43.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:42 smithi119 ceph-mon[34811]: from='mgr.? 172.21.15.119:0/438839331' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:55:43.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:42 smithi119 ceph-mon[34811]: from='mgr.? 172.21.15.119:0/438839331' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T03:55:43.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:42 smithi119 ceph-mon[34811]: from='mgr.? 172.21.15.119:0/438839331' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:55:43.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:42 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/702058846' entity='client.admin' 2024-09-21T03:55:43.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:42 smithi131 ceph-mon[35193]: Standby manager daemon y started 2024-09-21T03:55:43.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:42 smithi131 ceph-mon[35193]: from='mgr.? 172.21.15.119:0/438839331' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T03:55:43.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:42 smithi131 ceph-mon[35193]: from='mgr.? 172.21.15.119:0/438839331' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T03:55:43.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:42 smithi131 ceph-mon[35193]: from='mgr.? 172.21.15.119:0/438839331' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T03:55:43.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:42 smithi131 ceph-mon[35193]: from='mgr.? 172.21.15.119:0/438839331' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T03:55:44.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:43 smithi064 ceph-mon[31180]: mgrmap e19: x(active, since 14s), standbys: z, y 2024-09-21T03:55:44.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:43 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T03:55:44.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:43 smithi064 ceph-mon[31180]: from='client.? ' entity='client.admin' 2024-09-21T03:55:44.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:43 smithi119 ceph-mon[34811]: mgrmap e19: x(active, since 14s), standbys: z, y 2024-09-21T03:55:44.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:43 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T03:55:44.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:43 smithi119 ceph-mon[34811]: from='client.? ' entity='client.admin' 2024-09-21T03:55:44.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:43 smithi131 ceph-mon[35193]: mgrmap e19: x(active, since 14s), standbys: z, y 2024-09-21T03:55:44.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:43 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T03:55:44.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:43 smithi131 ceph-mon[35193]: from='client.? ' entity='client.admin' 2024-09-21T03:55:44.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.562+0000 7f67a2f51640 1 -- 172.21.15.64:0/138469058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f679c108d20 msgr2=0x7f679c10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:44.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.562+0000 7f67a2f51640 1 --2- 172.21.15.64:0/138469058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f679c108d20 0x7f679c10f550 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6794009ca0 tx=0x7f679402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:55:44.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.562+0000 7f67a2f51640 1 -- 172.21.15.64:0/138469058 shutdown_connections 2024-09-21T03:55:44.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.562+0000 7f67a2f51640 1 --2- 172.21.15.64:0/138469058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f679c108d20 0x7f679c10f550 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:44.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.562+0000 7f67a2f51640 1 --2- 172.21.15.64:0/138469058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f679c1083d0 0x7f679c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:44.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.562+0000 7f67a2f51640 1 --2- 172.21.15.64:0/138469058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f679c102ae0 0x7f679c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:44.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.562+0000 7f67a2f51640 1 -- 172.21.15.64:0/138469058 >> 172.21.15.64:0/138469058 conn(0x7f679c079760 msgr2=0x7f679c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:44.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.562+0000 7f67a2f51640 1 -- 172.21.15.64:0/138469058 shutdown_connections 2024-09-21T03:55:44.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.562+0000 7f67a2f51640 1 -- 172.21.15.64:0/138469058 wait complete. 2024-09-21T03:55:44.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.563+0000 7f67a2f51640 1 Processor -- start 2024-09-21T03:55:44.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.563+0000 7f67a2f51640 1 -- start start 2024-09-21T03:55:44.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.563+0000 7f67a2f51640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f679c102ae0 0x7f679c19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:44.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.563+0000 7f67a2f51640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f679c1083d0 0x7f679c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:44.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.563+0000 7f67a2f51640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f679c108d20 0x7f679c19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:44.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.563+0000 7f67a2f51640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f679c112990 con 0x7f679c108d20 2024-09-21T03:55:44.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.563+0000 7f67a2f51640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f679c112810 con 0x7f679c102ae0 2024-09-21T03:55:44.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.563+0000 7f67a2f51640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f679c112b10 con 0x7f679c1083d0 2024-09-21T03:55:44.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.563+0000 7f67a14ee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f679c108d20 0x7f679c19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:44.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.563+0000 7f67a14ee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f679c108d20 0x7f679c19fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56424/0 (socket says 172.21.15.64:56424) 2024-09-21T03:55:44.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.563+0000 7f67a14ee640 1 -- 172.21.15.64:0/3496274156 learned_addr learned my addr 172.21.15.64:0/3496274156 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:55:44.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.563+0000 7f67a0ced640 1 --2- 172.21.15.64:0/3496274156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f679c102ae0 0x7f679c19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:44.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.563+0000 7f679bfff640 1 --2- 172.21.15.64:0/3496274156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f679c1083d0 0x7f679c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:44.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.563+0000 7f67a14ee640 1 -- 172.21.15.64:0/3496274156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f679c1083d0 msgr2=0x7f679c19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:44.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.563+0000 7f67a14ee640 1 --2- 172.21.15.64:0/3496274156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f679c1083d0 0x7f679c19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:44.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.563+0000 7f67a14ee640 1 -- 172.21.15.64:0/3496274156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f679c102ae0 msgr2=0x7f679c19a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:44.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.563+0000 7f67a14ee640 1 --2- 172.21.15.64:0/3496274156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f679c102ae0 0x7f679c19a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:44.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.564+0000 7f67a14ee640 1 -- 172.21.15.64:0/3496274156 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f679c1a03a0 con 0x7f679c108d20 2024-09-21T03:55:44.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.564+0000 7f67a14ee640 1 --2- 172.21.15.64:0/3496274156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f679c108d20 0x7f679c19fc70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6794005cc0 tx=0x7f6794031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:44.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.564+0000 7f6799ffb640 1 -- 172.21.15.64:0/3496274156 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6794003b70 con 0x7f679c108d20 2024-09-21T03:55:44.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.564+0000 7f6799ffb640 1 -- 172.21.15.64:0/3496274156 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67940318f0 con 0x7f679c108d20 2024-09-21T03:55:44.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.564+0000 7f67a2f51640 1 -- 172.21.15.64:0/3496274156 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f679c1a0630 con 0x7f679c108d20 2024-09-21T03:55:44.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.564+0000 7f6799ffb640 1 -- 172.21.15.64:0/3496274156 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679402f950 con 0x7f679c108d20 2024-09-21T03:55:44.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.564+0000 7f67a2f51640 1 -- 172.21.15.64:0/3496274156 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f679c1a6df0 con 0x7f679c108d20 2024-09-21T03:55:44.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.565+0000 7f67a2f51640 1 -- 172.21.15.64:0/3496274156 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f679c103d10 con 0x7f679c108d20 2024-09-21T03:55:44.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.568+0000 7f6799ffb640 1 -- 172.21.15.64:0/3496274156 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f6794004b40 con 0x7f679c108d20 2024-09-21T03:55:44.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.569+0000 7f6799ffb640 1 --2- 172.21.15.64:0/3496274156 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f678c0b65d0 0x7f678c0b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:44.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.569+0000 7f67a0ced640 1 --2- 172.21.15.64:0/3496274156 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f678c0b65d0 0x7f678c0b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:44.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.569+0000 7f6799ffb640 1 -- 172.21.15.64:0/3496274156 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f67941069e0 con 0x7f679c108d20 2024-09-21T03:55:44.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.569+0000 7f67a0ced640 1 --2- 172.21.15.64:0/3496274156 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f678c0b65d0 0x7f678c0b8ae0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6790009bd0 tx=0x7f6790005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:44.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.572+0000 7f6799ffb640 1 -- 172.21.15.64:0/3496274156 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f67940cedd0 con 0x7f679c108d20 2024-09-21T03:55:44.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.762+0000 7f67a2f51640 1 -- 172.21.15.64:0/3496274156 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f679c102ef0 con 0x7f679c108d20 2024-09-21T03:55:44.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.766+0000 7f6799ffb640 1 -- 172.21.15.64:0/3496274156 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v24)=0 v24) ==== 120+0+0 (secure 0 0 0) 0x7f679c102ef0 con 0x7f679c108d20 2024-09-21T03:55:44.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.769+0000 7f67a2f51640 1 -- 172.21.15.64:0/3496274156 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f678c0b65d0 msgr2=0x7f678c0b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:44.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.769+0000 7f67a2f51640 1 --2- 172.21.15.64:0/3496274156 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f678c0b65d0 0x7f678c0b8ae0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6790009bd0 tx=0x7f6790005f90 comp rx=0 tx=0).stop 2024-09-21T03:55:44.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.769+0000 7f67a2f51640 1 -- 172.21.15.64:0/3496274156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f679c108d20 msgr2=0x7f679c19fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:44.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.769+0000 7f67a2f51640 1 --2- 172.21.15.64:0/3496274156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f679c108d20 0x7f679c19fc70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6794005cc0 tx=0x7f6794031700 comp rx=0 tx=0).stop 2024-09-21T03:55:44.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.769+0000 7f67a2f51640 1 -- 172.21.15.64:0/3496274156 shutdown_connections 2024-09-21T03:55:44.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.769+0000 7f67a2f51640 1 --2- 172.21.15.64:0/3496274156 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f678c0b65d0 0x7f678c0b8ae0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:44.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.769+0000 7f67a2f51640 1 --2- 172.21.15.64:0/3496274156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f679c108d20 0x7f679c19fc70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:44.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.769+0000 7f67a2f51640 1 --2- 172.21.15.64:0/3496274156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f679c1083d0 0x7f679c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:44.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.769+0000 7f67a2f51640 1 --2- 172.21.15.64:0/3496274156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f679c102ae0 0x7f679c19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:44.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.769+0000 7f67a2f51640 1 -- 172.21.15.64:0/3496274156 >> 172.21.15.64:0/3496274156 conn(0x7f679c079760 msgr2=0x7f679c06be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:44.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.769+0000 7f67a2f51640 1 -- 172.21.15.64:0/3496274156 shutdown_connections 2024-09-21T03:55:44.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:44.769+0000 7f67a2f51640 1 -- 172.21.15.64:0/3496274156 wait complete. 2024-09-21T03:55:45.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.277+0000 7f9587096640 1 -- 172.21.15.64:0/618063809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95801083a0 msgr2=0x7f95801087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:45.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.277+0000 7f9587096640 1 --2- 172.21.15.64:0/618063809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95801083a0 0x7f95801087b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9570009900 tx=0x7f957002a410 comp rx=0 tx=0).stop 2024-09-21T03:55:45.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.277+0000 7f9587096640 1 -- 172.21.15.64:0/618063809 shutdown_connections 2024-09-21T03:55:45.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.277+0000 7f9587096640 1 --2- 172.21.15.64:0/618063809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9580108cf0 0x7f958010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:45.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.277+0000 7f9587096640 1 --2- 172.21.15.64:0/618063809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95801083a0 0x7f95801087b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:45.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.277+0000 7f9587096640 1 --2- 172.21.15.64:0/618063809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9580102a60 0x7f9580102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:45.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.277+0000 7f9587096640 1 -- 172.21.15.64:0/618063809 >> 172.21.15.64:0/618063809 conn(0x7f9580079730 msgr2=0x7f9580079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:45.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.277+0000 7f9587096640 1 -- 172.21.15.64:0/618063809 shutdown_connections 2024-09-21T03:55:45.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.277+0000 7f9587096640 1 -- 172.21.15.64:0/618063809 wait complete. 2024-09-21T03:55:45.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.278+0000 7f9587096640 1 Processor -- start 2024-09-21T03:55:45.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.278+0000 7f9587096640 1 -- start start 2024-09-21T03:55:45.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.278+0000 7f9587096640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9580102a60 0x7f958019b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:45.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.278+0000 7f9587096640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95801083a0 0x7f958019b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:45.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.278+0000 7f9587096640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9580108cf0 0x7f958019bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:45.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.278+0000 7f9587096640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f95801129b0 con 0x7f9580108cf0 2024-09-21T03:55:45.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.278+0000 7f9587096640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9580112830 con 0x7f9580102a60 2024-09-21T03:55:45.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.278+0000 7f9587096640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9580112b30 con 0x7f95801083a0 2024-09-21T03:55:45.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.278+0000 7f9585633640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9580108cf0 0x7f958019bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:45.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.278+0000 7f9585633640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9580108cf0 0x7f958019bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56432/0 (socket says 172.21.15.64:56432) 2024-09-21T03:55:45.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.278+0000 7f9585633640 1 -- 172.21.15.64:0/2201866174 learned_addr learned my addr 172.21.15.64:0/2201866174 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:55:45.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.278+0000 7f9577fff640 1 --2- 172.21.15.64:0/2201866174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95801083a0 0x7f958019b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:45.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.278+0000 7f9584e32640 1 --2- 172.21.15.64:0/2201866174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9580102a60 0x7f958019b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:45.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.279+0000 7f9585633640 1 -- 172.21.15.64:0/2201866174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95801083a0 msgr2=0x7f958019b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:45.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.279+0000 7f9585633640 1 --2- 172.21.15.64:0/2201866174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95801083a0 0x7f958019b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:45.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.279+0000 7f9585633640 1 -- 172.21.15.64:0/2201866174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9580102a60 msgr2=0x7f958019b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:45.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.279+0000 7f9585633640 1 --2- 172.21.15.64:0/2201866174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9580102a60 0x7f958019b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:45.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.279+0000 7f9585633640 1 -- 172.21.15.64:0/2201866174 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95801a6500 con 0x7f9580108cf0 2024-09-21T03:55:45.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.279+0000 7f9585633640 1 --2- 172.21.15.64:0/2201866174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9580108cf0 0x7f958019bcc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f957800eaa0 tx=0x7f957800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:45.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.279+0000 7f9575ffb640 1 -- 172.21.15.64:0/2201866174 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957800cdb0 con 0x7f9580108cf0 2024-09-21T03:55:45.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.279+0000 7f9575ffb640 1 -- 172.21.15.64:0/2201866174 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9578005020 con 0x7f9580108cf0 2024-09-21T03:55:45.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.279+0000 7f9587096640 1 -- 172.21.15.64:0/2201866174 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95801a67f0 con 0x7f9580108cf0 2024-09-21T03:55:45.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.279+0000 7f9575ffb640 1 -- 172.21.15.64:0/2201866174 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95780116d0 con 0x7f9580108cf0 2024-09-21T03:55:45.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.279+0000 7f9587096640 1 -- 172.21.15.64:0/2201866174 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95801a6cb0 con 0x7f9580108cf0 2024-09-21T03:55:45.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.280+0000 7f9587096640 1 -- 172.21.15.64:0/2201866174 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9580102e70 con 0x7f9580108cf0 2024-09-21T03:55:45.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.282+0000 7f9575ffb640 1 -- 172.21.15.64:0/2201866174 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f95780051d0 con 0x7f9580108cf0 2024-09-21T03:55:45.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.283+0000 7f9575ffb640 1 --2- 172.21.15.64:0/2201866174 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f95680b64d0 0x7f95680b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:45.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.283+0000 7f9584e32640 1 --2- 172.21.15.64:0/2201866174 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f95680b64d0 0x7f95680b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:45.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.283+0000 7f9575ffb640 1 -- 172.21.15.64:0/2201866174 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f95780e59a0 con 0x7f9580108cf0 2024-09-21T03:55:45.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.283+0000 7f9584e32640 1 --2- 172.21.15.64:0/2201866174 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f95680b64d0 0x7f95680b89e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f957c005fd0 tx=0x7f957c005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:45.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.287+0000 7f9575ffb640 1 -- 172.21.15.64:0/2201866174 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9578016030 con 0x7f9580108cf0 2024-09-21T03:55:45.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.475+0000 7f9587096640 1 -- 172.21.15.64:0/2201866174 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f958002e640 con 0x7f9580108cf0 2024-09-21T03:55:45.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.482+0000 7f9575ffb640 1 -- 172.21.15.64:0/2201866174 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v25)=0 v25) ==== 122+0+0 (secure 0 0 0) 0x7f95780ade30 con 0x7f9580108cf0 2024-09-21T03:55:45.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.484+0000 7f9587096640 1 -- 172.21.15.64:0/2201866174 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f95680b64d0 msgr2=0x7f95680b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:45.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.484+0000 7f9587096640 1 --2- 172.21.15.64:0/2201866174 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f95680b64d0 0x7f95680b89e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f957c005fd0 tx=0x7f957c005850 comp rx=0 tx=0).stop 2024-09-21T03:55:45.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.484+0000 7f9587096640 1 -- 172.21.15.64:0/2201866174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9580108cf0 msgr2=0x7f958019bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:45.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.484+0000 7f9587096640 1 --2- 172.21.15.64:0/2201866174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9580108cf0 0x7f958019bcc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f957800eaa0 tx=0x7f957800ef60 comp rx=0 tx=0).stop 2024-09-21T03:55:45.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.484+0000 7f9587096640 1 -- 172.21.15.64:0/2201866174 shutdown_connections 2024-09-21T03:55:45.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.484+0000 7f9587096640 1 --2- 172.21.15.64:0/2201866174 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f95680b64d0 0x7f95680b89e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:45.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.484+0000 7f9587096640 1 --2- 172.21.15.64:0/2201866174 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9580108cf0 0x7f958019bcc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:45.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.484+0000 7f9587096640 1 --2- 172.21.15.64:0/2201866174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95801083a0 0x7f958019b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:45.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.484+0000 7f9587096640 1 --2- 172.21.15.64:0/2201866174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9580102a60 0x7f958019b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:45.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.484+0000 7f9587096640 1 -- 172.21.15.64:0/2201866174 >> 172.21.15.64:0/2201866174 conn(0x7f9580079730 msgr2=0x7f95801043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:45.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.484+0000 7f9587096640 1 -- 172.21.15.64:0/2201866174 shutdown_connections 2024-09-21T03:55:45.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:45.484+0000 7f9587096640 1 -- 172.21.15.64:0/2201866174 wait complete. 2024-09-21T03:55:46.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:45 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3496274156' entity='client.admin' 2024-09-21T03:55:46.023 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:45 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2201866174' entity='client.admin' 2024-09-21T03:55:46.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:45 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3496274156' entity='client.admin' 2024-09-21T03:55:46.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:45 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2201866174' entity='client.admin' 2024-09-21T03:55:46.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:45 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3496274156' entity='client.admin' 2024-09-21T03:55:46.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:45 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2201866174' entity='client.admin' 2024-09-21T03:55:46.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.538+0000 7f2fbd895640 1 -- 172.21.15.64:0/2645741863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fb8102a60 msgr2=0x7f2fb8102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:46.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.538+0000 7f2fbd895640 1 --2- 172.21.15.64:0/2645741863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fb8102a60 0x7f2fb8102e70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009900 tx=0x7f2fa802a410 comp rx=0 tx=0).stop 2024-09-21T03:55:46.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.539+0000 7f2fbd895640 1 -- 172.21.15.64:0/2645741863 shutdown_connections 2024-09-21T03:55:46.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.539+0000 7f2fbd895640 1 --2- 172.21.15.64:0/2645741863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb8108cf0 0x7f2fb810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:46.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.539+0000 7f2fbd895640 1 --2- 172.21.15.64:0/2645741863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb81083a0 0x7f2fb81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:46.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.539+0000 7f2fbd895640 1 --2- 172.21.15.64:0/2645741863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fb8102a60 0x7f2fb8102e70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:46.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.539+0000 7f2fbd895640 1 -- 172.21.15.64:0/2645741863 >> 172.21.15.64:0/2645741863 conn(0x7f2fb8079730 msgr2=0x7f2fb8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:46.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.539+0000 7f2fbd895640 1 -- 172.21.15.64:0/2645741863 shutdown_connections 2024-09-21T03:55:46.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.539+0000 7f2fbd895640 1 -- 172.21.15.64:0/2645741863 wait complete. 2024-09-21T03:55:46.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.539+0000 7f2fbd895640 1 Processor -- start 2024-09-21T03:55:46.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.540+0000 7f2fbd895640 1 -- start start 2024-09-21T03:55:46.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.540+0000 7f2fbd895640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb8102a60 0x7f2fb819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:46.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.540+0000 7f2fbd895640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb81083a0 0x7f2fb819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:46.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.540+0000 7f2fbd895640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fb8108cf0 0x7f2fb819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:46.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.540+0000 7f2fbd895640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb81129b0 con 0x7f2fb8108cf0 2024-09-21T03:55:46.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.540+0000 7f2fbd895640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb8112830 con 0x7f2fb81083a0 2024-09-21T03:55:46.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.540+0000 7f2fbd895640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb8112b30 con 0x7f2fb8102a60 2024-09-21T03:55:46.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.540+0000 7f2fb77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fb8108cf0 0x7f2fb819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:46.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.540+0000 7f2fb77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fb8108cf0 0x7f2fb819fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56440/0 (socket says 172.21.15.64:56440) 2024-09-21T03:55:46.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.540+0000 7f2fb77fe640 1 -- 172.21.15.64:0/1224466179 learned_addr learned my addr 172.21.15.64:0/1224466179 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:55:46.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.540+0000 7f2fb67fc640 1 --2- 172.21.15.64:0/1224466179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb81083a0 0x7f2fb819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:46.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.540+0000 7f2fb6ffd640 1 --2- 172.21.15.64:0/1224466179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb8102a60 0x7f2fb819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:46.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.540+0000 7f2fb77fe640 1 -- 172.21.15.64:0/1224466179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb8102a60 msgr2=0x7f2fb819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:46.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.540+0000 7f2fb77fe640 1 --2- 172.21.15.64:0/1224466179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb8102a60 0x7f2fb819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:46.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.540+0000 7f2fb77fe640 1 -- 172.21.15.64:0/1224466179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb81083a0 msgr2=0x7f2fb819aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:46.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.540+0000 7f2fb77fe640 1 --2- 172.21.15.64:0/1224466179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb81083a0 0x7f2fb819aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:46.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.540+0000 7f2fb77fe640 1 -- 172.21.15.64:0/1224466179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb81a0400 con 0x7f2fb8108cf0 2024-09-21T03:55:46.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.541+0000 7f2fb77fe640 1 --2- 172.21.15.64:0/1224466179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fb8108cf0 0x7f2fb819fcd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2fb000cad0 tx=0x7f2fb000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:46.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.541+0000 7f2fbc893640 1 -- 172.21.15.64:0/1224466179 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb000ae20 con 0x7f2fb8108cf0 2024-09-21T03:55:46.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.541+0000 7f2fbc893640 1 -- 172.21.15.64:0/1224466179 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fb0005020 con 0x7f2fb8108cf0 2024-09-21T03:55:46.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.541+0000 7f2fbd895640 1 -- 172.21.15.64:0/1224466179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb81a06f0 con 0x7f2fb8108cf0 2024-09-21T03:55:46.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.541+0000 7f2fbc893640 1 -- 172.21.15.64:0/1224466179 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb000f6b0 con 0x7f2fb8108cf0 2024-09-21T03:55:46.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.541+0000 7f2fbd895640 1 -- 172.21.15.64:0/1224466179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb81a6e50 con 0x7f2fb8108cf0 2024-09-21T03:55:46.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.542+0000 7f2fbd895640 1 -- 172.21.15.64:0/1224466179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb8102e70 con 0x7f2fb8108cf0 2024-09-21T03:55:46.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.544+0000 7f2fbc893640 1 -- 172.21.15.64:0/1224466179 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f2fb00051d0 con 0x7f2fb8108cf0 2024-09-21T03:55:46.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.544+0000 7f2fbc893640 1 --2- 172.21.15.64:0/1224466179 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2fa00b6550 0x7f2fa00b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:46.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.545+0000 7f2fb6ffd640 1 --2- 172.21.15.64:0/1224466179 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2fa00b6550 0x7f2fa00b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:46.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.545+0000 7f2fbc893640 1 -- 172.21.15.64:0/1224466179 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f2fb00e5ae0 con 0x7f2fb8108cf0 2024-09-21T03:55:46.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.545+0000 7f2fb6ffd640 1 --2- 172.21.15.64:0/1224466179 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2fa00b6550 0x7f2fa00b8a60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8000c00 tx=0x7f2fa8031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:46.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.548+0000 7f2fbc893640 1 -- 172.21.15.64:0/1224466179 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2fb0014030 con 0x7f2fb8108cf0 2024-09-21T03:55:46.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.740+0000 7f2fbd895640 1 -- 172.21.15.64:0/1224466179 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7f2fb81a71c0 con 0x7f2fb8108cf0 2024-09-21T03:55:46.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.744+0000 7f2fbc893640 1 -- 172.21.15.64:0/1224466179 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v26)=0 v26) ==== 116+0+0 (secure 0 0 0) 0x7f2fb00adff0 con 0x7f2fb8108cf0 2024-09-21T03:55:46.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.746+0000 7f2fbd895640 1 -- 172.21.15.64:0/1224466179 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2fa00b6550 msgr2=0x7f2fa00b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:46.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.746+0000 7f2fbd895640 1 --2- 172.21.15.64:0/1224466179 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2fa00b6550 0x7f2fa00b8a60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8000c00 tx=0x7f2fa8031400 comp rx=0 tx=0).stop 2024-09-21T03:55:46.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.746+0000 7f2fbd895640 1 -- 172.21.15.64:0/1224466179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fb8108cf0 msgr2=0x7f2fb819fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:46.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.746+0000 7f2fbd895640 1 --2- 172.21.15.64:0/1224466179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fb8108cf0 0x7f2fb819fcd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2fb000cad0 tx=0x7f2fb000cf90 comp rx=0 tx=0).stop 2024-09-21T03:55:46.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.747+0000 7f2fbd895640 1 -- 172.21.15.64:0/1224466179 shutdown_connections 2024-09-21T03:55:46.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.747+0000 7f2fbd895640 1 --2- 172.21.15.64:0/1224466179 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2fa00b6550 0x7f2fa00b8a60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:46.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.747+0000 7f2fbd895640 1 --2- 172.21.15.64:0/1224466179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2fb8108cf0 0x7f2fb819fcd0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:46.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.747+0000 7f2fbd895640 1 --2- 172.21.15.64:0/1224466179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb81083a0 0x7f2fb819aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:46.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.747+0000 7f2fbd895640 1 --2- 172.21.15.64:0/1224466179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fb8102a60 0x7f2fb819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:46.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.747+0000 7f2fbd895640 1 -- 172.21.15.64:0/1224466179 >> 172.21.15.64:0/1224466179 conn(0x7f2fb8079730 msgr2=0x7f2fb8101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:46.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.747+0000 7f2fbd895640 1 -- 172.21.15.64:0/1224466179 shutdown_connections 2024-09-21T03:55:46.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:46.747+0000 7f2fbd895640 1 -- 172.21.15.64:0/1224466179 wait complete. 2024-09-21T03:55:47.273 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi064') 2024-09-21T03:55:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.710+0000 7f2371e3b640 1 -- 172.21.15.64:0/1508161700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f236c1083b0 msgr2=0x7f236c1087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:47.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.710+0000 7f2371e3b640 1 --2- 172.21.15.64:0/1508161700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f236c1083b0 0x7f236c1087c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2354009900 tx=0x7f235402e800 comp rx=0 tx=0).stop 2024-09-21T03:55:47.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.710+0000 7f2371e3b640 1 -- 172.21.15.64:0/1508161700 shutdown_connections 2024-09-21T03:55:47.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.710+0000 7f2371e3b640 1 --2- 172.21.15.64:0/1508161700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f236c108d00 0x7f236c10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:47.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.710+0000 7f2371e3b640 1 --2- 172.21.15.64:0/1508161700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f236c1083b0 0x7f236c1087c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:47.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.710+0000 7f2371e3b640 1 --2- 172.21.15.64:0/1508161700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f236c102ac0 0x7f236c102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:47.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.710+0000 7f2371e3b640 1 -- 172.21.15.64:0/1508161700 >> 172.21.15.64:0/1508161700 conn(0x7f236c079790 msgr2=0x7f236c079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:47.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.710+0000 7f2371e3b640 1 -- 172.21.15.64:0/1508161700 shutdown_connections 2024-09-21T03:55:47.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.710+0000 7f2371e3b640 1 -- 172.21.15.64:0/1508161700 wait complete. 2024-09-21T03:55:47.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.711+0000 7f2371e3b640 1 Processor -- start 2024-09-21T03:55:47.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.711+0000 7f2371e3b640 1 -- start start 2024-09-21T03:55:47.714 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:47.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.712+0000 7f2371e3b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f236c102ac0 0x7f236c1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:47.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.712+0000 7f2371e3b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f236c1083b0 0x7f236c198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:47.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.712+0000 7f2371e3b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f236c108d00 0x7f236c19db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:47.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.712+0000 7f2371e3b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f236c111900 con 0x7f236c108d00 2024-09-21T03:55:47.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.712+0000 7f2371e3b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f236c111780 con 0x7f236c102ac0 2024-09-21T03:55:47.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.712+0000 7f2371e3b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f236c111a80 con 0x7f236c1083b0 2024-09-21T03:55:47.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.712+0000 7f236b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f236c102ac0 0x7f236c1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:47.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.712+0000 7f2363fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f236c1083b0 0x7f236c198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:47.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.712+0000 7f236b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f236c102ac0 0x7f236c1982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:57622/0 (socket says 172.21.15.64:57622) 2024-09-21T03:55:47.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.712+0000 7f236b7fe640 1 -- 172.21.15.64:0/4164079250 learned_addr learned my addr 172.21.15.64:0/4164079250 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:55:47.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.713+0000 7f236bfff640 1 --2- 172.21.15.64:0/4164079250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f236c108d00 0x7f236c19db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:47.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.713+0000 7f236b7fe640 1 -- 172.21.15.64:0/4164079250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f236c1083b0 msgr2=0x7f236c198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:47.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.713+0000 7f236b7fe640 1 --2- 172.21.15.64:0/4164079250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f236c1083b0 0x7f236c198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:47.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.713+0000 7f236b7fe640 1 -- 172.21.15.64:0/4164079250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f236c108d00 msgr2=0x7f236c19db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:47.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.713+0000 7f236b7fe640 1 --2- 172.21.15.64:0/4164079250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f236c108d00 0x7f236c19db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:47.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.713+0000 7f236b7fe640 1 -- 172.21.15.64:0/4164079250 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236c19e240 con 0x7f236c102ac0 2024-09-21T03:55:47.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.713+0000 7f236bfff640 1 --2- 172.21.15.64:0/4164079250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f236c108d00 0x7f236c19db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:55:47.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.713+0000 7f236b7fe640 1 --2- 172.21.15.64:0/4164079250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f236c102ac0 0x7f236c1982d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f235c00e9f0 tx=0x7f235c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:47.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.714+0000 7f23697fa640 1 -- 172.21.15.64:0/4164079250 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c00cd80 con 0x7f236c102ac0 2024-09-21T03:55:47.718 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:47.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.714+0000 7f23697fa640 1 -- 172.21.15.64:0/4164079250 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f235c0108e0 con 0x7f236c102ac0 2024-09-21T03:55:47.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.714+0000 7f2371e3b640 1 -- 172.21.15.64:0/4164079250 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f236c19e530 con 0x7f236c102ac0 2024-09-21T03:55:47.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.714+0000 7f23697fa640 1 -- 172.21.15.64:0/4164079250 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c018980 con 0x7f236c102ac0 2024-09-21T03:55:47.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.714+0000 7f2371e3b640 1 -- 172.21.15.64:0/4164079250 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f236c1a4c90 con 0x7f236c102ac0 2024-09-21T03:55:47.719 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:47.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.716+0000 7f2371e3b640 1 -- 172.21.15.64:0/4164079250 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f236c102ed0 con 0x7f236c102ac0 2024-09-21T03:55:47.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.717+0000 7f23697fa640 1 -- 172.21.15.64:0/4164079250 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f235c018b20 con 0x7f236c102ac0 2024-09-21T03:55:47.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.718+0000 7f23697fa640 1 --2- 172.21.15.64:0/4164079250 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f23580b6550 0x7f23580b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:47.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.719+0000 7f2363fff640 1 --2- 172.21.15.64:0/4164079250 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f23580b6550 0x7f23580b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:47.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.719+0000 7f23697fa640 1 -- 172.21.15.64:0/4164079250 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f235c0e5a60 con 0x7f236c102ac0 2024-09-21T03:55:47.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.719+0000 7f2363fff640 1 --2- 172.21.15.64:0/4164079250 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f23580b6550 0x7f23580b8a60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f235402ed70 tx=0x7f23540363f0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:47.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.722+0000 7f23697fa640 1 -- 172.21.15.64:0/4164079250 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f235c0adef0 con 0x7f236c102ac0 2024-09-21T03:55:47.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.922+0000 7f2371e3b640 1 -- 172.21.15.64:0/4164079250 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f236c199400 con 0x7f236c102ac0 2024-09-21T03:55:47.924 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:47 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1224466179' entity='client.admin' 2024-09-21T03:55:47.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.929+0000 7f23697fa640 1 -- 172.21.15.64:0/4164079250 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v27)=0 v27) ==== 121+0+0 (secure 0 0 0) 0x7f235c0b22d0 con 0x7f236c102ac0 2024-09-21T03:55:47.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.932+0000 7f2371e3b640 1 -- 172.21.15.64:0/4164079250 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f23580b6550 msgr2=0x7f23580b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:47.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.932+0000 7f2371e3b640 1 --2- 172.21.15.64:0/4164079250 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f23580b6550 0x7f23580b8a60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f235402ed70 tx=0x7f23540363f0 comp rx=0 tx=0).stop 2024-09-21T03:55:47.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.932+0000 7f2371e3b640 1 -- 172.21.15.64:0/4164079250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f236c102ac0 msgr2=0x7f236c1982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:47.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.932+0000 7f2371e3b640 1 --2- 172.21.15.64:0/4164079250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f236c102ac0 0x7f236c1982d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f235c00e9f0 tx=0x7f235c00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:55:47.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.932+0000 7f2371e3b640 1 -- 172.21.15.64:0/4164079250 shutdown_connections 2024-09-21T03:55:47.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.932+0000 7f2371e3b640 1 --2- 172.21.15.64:0/4164079250 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f23580b6550 0x7f23580b8a60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:47.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.932+0000 7f2371e3b640 1 --2- 172.21.15.64:0/4164079250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f236c108d00 0x7f236c19db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:47.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.932+0000 7f2371e3b640 1 --2- 172.21.15.64:0/4164079250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f236c1083b0 0x7f236c198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:47.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.932+0000 7f2371e3b640 1 --2- 172.21.15.64:0/4164079250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f236c102ac0 0x7f236c1982d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:47.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.932+0000 7f2371e3b640 1 -- 172.21.15.64:0/4164079250 >> 172.21.15.64:0/4164079250 conn(0x7f236c079790 msgr2=0x7f236c1064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:47.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.932+0000 7f2371e3b640 1 -- 172.21.15.64:0/4164079250 shutdown_connections 2024-09-21T03:55:47.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:47.932+0000 7f2371e3b640 1 -- 172.21.15.64:0/4164079250 wait complete. 2024-09-21T03:55:48.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:47 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1224466179' entity='client.admin' 2024-09-21T03:55:48.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:47 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1224466179' entity='client.admin' 2024-09-21T03:55:48.062 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:49.202 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:48 smithi064 ceph-mon[31180]: from='client.? ' entity='client.admin' 2024-09-21T03:55:49.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:48 smithi119 ceph-mon[34811]: from='client.? ' entity='client.admin' 2024-09-21T03:55:49.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:48 smithi131 ceph-mon[35193]: from='client.? ' entity='client.admin' 2024-09-21T03:55:49.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.420+0000 7f292c779640 1 -- 172.21.15.64:0/2507741531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2924108d20 msgr2=0x7f292410f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:49.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.420+0000 7f292c779640 1 --2- 172.21.15.64:0/2507741531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2924108d20 0x7f292410f550 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f291c009ca0 tx=0x7f291c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:55:49.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.421+0000 7f292c779640 1 -- 172.21.15.64:0/2507741531 shutdown_connections 2024-09-21T03:55:49.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.421+0000 7f292c779640 1 --2- 172.21.15.64:0/2507741531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2924108d20 0x7f292410f550 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:49.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.421+0000 7f292c779640 1 --2- 172.21.15.64:0/2507741531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29241083d0 0x7f29241087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:49.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.421+0000 7f292c779640 1 --2- 172.21.15.64:0/2507741531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2924102ae0 0x7f2924102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:49.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.421+0000 7f292c779640 1 -- 172.21.15.64:0/2507741531 >> 172.21.15.64:0/2507741531 conn(0x7f29240797b0 msgr2=0x7f2924079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:49.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.421+0000 7f292c779640 1 -- 172.21.15.64:0/2507741531 shutdown_connections 2024-09-21T03:55:49.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.421+0000 7f292c779640 1 -- 172.21.15.64:0/2507741531 wait complete. 2024-09-21T03:55:49.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.421+0000 7f292c779640 1 Processor -- start 2024-09-21T03:55:49.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.421+0000 7f292c779640 1 -- start start 2024-09-21T03:55:49.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.422+0000 7f292c779640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2924102ae0 0x7f292419b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:49.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.422+0000 7f292c779640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29241083d0 0x7f292419b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:49.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.422+0000 7f292c779640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2924108d20 0x7f292419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:49.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.422+0000 7f292c779640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f29241129e0 con 0x7f29241083d0 2024-09-21T03:55:49.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.422+0000 7f292c779640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2924112860 con 0x7f2924102ae0 2024-09-21T03:55:49.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.422+0000 7f292c779640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2924112b60 con 0x7f2924108d20 2024-09-21T03:55:49.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.422+0000 7f2929d14640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29241083d0 0x7f292419b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:49.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.422+0000 7f2929d14640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29241083d0 0x7f292419b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56488/0 (socket says 172.21.15.64:56488) 2024-09-21T03:55:49.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.422+0000 7f2929d14640 1 -- 172.21.15.64:0/3201253064 learned_addr learned my addr 172.21.15.64:0/3201253064 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:55:49.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.422+0000 7f292a515640 1 --2- 172.21.15.64:0/3201253064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2924102ae0 0x7f292419b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:49.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.422+0000 7f292ad16640 1 --2- 172.21.15.64:0/3201253064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2924108d20 0x7f292419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:49.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.422+0000 7f2929d14640 1 -- 172.21.15.64:0/3201253064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2924108d20 msgr2=0x7f292419bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:49.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.422+0000 7f2929d14640 1 --2- 172.21.15.64:0/3201253064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2924108d20 0x7f292419bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:49.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.422+0000 7f2929d14640 1 -- 172.21.15.64:0/3201253064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2924102ae0 msgr2=0x7f292419b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:49.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.422+0000 7f2929d14640 1 --2- 172.21.15.64:0/3201253064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2924102ae0 0x7f292419b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:49.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.422+0000 7f2929d14640 1 -- 172.21.15.64:0/3201253064 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2924199060 con 0x7f29241083d0 2024-09-21T03:55:49.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.422+0000 7f292ad16640 1 --2- 172.21.15.64:0/3201253064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2924108d20 0x7f292419bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:49.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.423+0000 7f2929d14640 1 --2- 172.21.15.64:0/3201253064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29241083d0 0x7f292419b730 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f291400c970 tx=0x7f291400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:49.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.423+0000 7f291b7fe640 1 -- 172.21.15.64:0/3201253064 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914007bb0 con 0x7f29241083d0 2024-09-21T03:55:49.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.423+0000 7f291b7fe640 1 -- 172.21.15.64:0/3201253064 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2914007d50 con 0x7f29241083d0 2024-09-21T03:55:49.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.423+0000 7f292c779640 1 -- 172.21.15.64:0/3201253064 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2924199350 con 0x7f29241083d0 2024-09-21T03:55:49.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.423+0000 7f291b7fe640 1 -- 172.21.15.64:0/3201253064 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914004d90 con 0x7f29241083d0 2024-09-21T03:55:49.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.423+0000 7f292c779640 1 -- 172.21.15.64:0/3201253064 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2924199890 con 0x7f29241083d0 2024-09-21T03:55:49.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.424+0000 7f292c779640 1 -- 172.21.15.64:0/3201253064 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2924102ef0 con 0x7f29241083d0 2024-09-21T03:55:49.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.426+0000 7f291b7fe640 1 -- 172.21.15.64:0/3201253064 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f2914004f50 con 0x7f29241083d0 2024-09-21T03:55:49.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.427+0000 7f291b7fe640 1 --2- 172.21.15.64:0/3201253064 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f29100b64d0 0x7f29100b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:49.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.427+0000 7f292a515640 1 --2- 172.21.15.64:0/3201253064 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f29100b64d0 0x7f29100b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:49.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.427+0000 7f291b7fe640 1 -- 172.21.15.64:0/3201253064 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f29140e5820 con 0x7f29241083d0 2024-09-21T03:55:49.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.427+0000 7f292a515640 1 --2- 172.21.15.64:0/3201253064 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f29100b64d0 0x7f29100b89e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f292406c3e0 tx=0x7f292000a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:49.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.430+0000 7f291b7fe640 1 -- 172.21.15.64:0/3201253064 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2914014030 con 0x7f29241083d0 2024-09-21T03:55:49.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.625+0000 7f292c779640 1 -- 172.21.15.64:0/3201253064 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f292406bc90 con 0x7f29241083d0 2024-09-21T03:55:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.630+0000 7f291b7fe640 1 -- 172.21.15.64:0/3201253064 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v28)=0 v28) ==== 133+0+0 (secure 0 0 0) 0x7f29140adcb0 con 0x7f29241083d0 2024-09-21T03:55:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.632+0000 7f292c779640 1 -- 172.21.15.64:0/3201253064 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f29100b64d0 msgr2=0x7f29100b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.632+0000 7f292c779640 1 --2- 172.21.15.64:0/3201253064 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f29100b64d0 0x7f29100b89e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f292406c3e0 tx=0x7f292000a2d0 comp rx=0 tx=0).stop 2024-09-21T03:55:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.632+0000 7f292c779640 1 -- 172.21.15.64:0/3201253064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29241083d0 msgr2=0x7f292419b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.632+0000 7f292c779640 1 --2- 172.21.15.64:0/3201253064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29241083d0 0x7f292419b730 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f291400c970 tx=0x7f291400cd70 comp rx=0 tx=0).stop 2024-09-21T03:55:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.632+0000 7f292c779640 1 -- 172.21.15.64:0/3201253064 shutdown_connections 2024-09-21T03:55:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.632+0000 7f292c779640 1 --2- 172.21.15.64:0/3201253064 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f29100b64d0 0x7f29100b89e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.632+0000 7f292c779640 1 --2- 172.21.15.64:0/3201253064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2924108d20 0x7f292419bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.632+0000 7f292c779640 1 --2- 172.21.15.64:0/3201253064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29241083d0 0x7f292419b730 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.632+0000 7f292c779640 1 --2- 172.21.15.64:0/3201253064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2924102ae0 0x7f292419b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.632+0000 7f292c779640 1 -- 172.21.15.64:0/3201253064 >> 172.21.15.64:0/3201253064 conn(0x7f29240797b0 msgr2=0x7f29241063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.632+0000 7f292c779640 1 -- 172.21.15.64:0/3201253064 shutdown_connections 2024-09-21T03:55:49.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:49.632+0000 7f292c779640 1 -- 172.21.15.64:0/3201253064 wait complete. 2024-09-21T03:55:50.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:49 smithi131 ceph-mon[35193]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:50.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:49 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3201253064' entity='client.admin' 2024-09-21T03:55:50.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:49 smithi119 ceph-mon[34811]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:50.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:49 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3201253064' entity='client.admin' 2024-09-21T03:55:50.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:49 smithi064 ceph-mon[31180]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:50.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:49 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3201253064' entity='client.admin' 2024-09-21T03:55:50.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.705+0000 7fa648343640 1 -- 172.21.15.64:0/2693901935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa640109e10 msgr2=0x7fa640110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:50.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.705+0000 7fa648343640 1 --2- 172.21.15.64:0/2693901935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa640109e10 0x7fa640110640 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa630009ca0 tx=0x7fa63002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.705+0000 7fa648343640 1 -- 172.21.15.64:0/2693901935 shutdown_connections 2024-09-21T03:55:50.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.705+0000 7fa648343640 1 --2- 172.21.15.64:0/2693901935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa640109e10 0x7fa640110640 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.705+0000 7fa648343640 1 --2- 172.21.15.64:0/2693901935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6401094c0 0x7fa6401098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.705+0000 7fa648343640 1 --2- 172.21.15.64:0/2693901935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa640103b80 0x7fa640103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.705+0000 7fa648343640 1 -- 172.21.15.64:0/2693901935 >> 172.21.15.64:0/2693901935 conn(0x7fa6400ff610 msgr2=0x7fa640101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:50.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.705+0000 7fa648343640 1 -- 172.21.15.64:0/2693901935 shutdown_connections 2024-09-21T03:55:50.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.705+0000 7fa648343640 1 -- 172.21.15.64:0/2693901935 wait complete. 2024-09-21T03:55:50.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.706+0000 7fa648343640 1 Processor -- start 2024-09-21T03:55:50.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.706+0000 7fa648343640 1 -- start start 2024-09-21T03:55:50.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.706+0000 7fa648343640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa640103b80 0x7fa64019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:50.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.706+0000 7fa648343640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6401094c0 0x7fa64019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:50.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.706+0000 7fa648343640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa640109e10 0x7fa64019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:50.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.706+0000 7fa648343640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa640113ad0 con 0x7fa640109e10 2024-09-21T03:55:50.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.706+0000 7fa648343640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa640113950 con 0x7fa6401094c0 2024-09-21T03:55:50.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.706+0000 7fa648343640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa640113c50 con 0x7fa640103b80 2024-09-21T03:55:50.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.706+0000 7fa6468e0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa640109e10 0x7fa64019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:50.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.706+0000 7fa6468e0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa640109e10 0x7fa64019fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56504/0 (socket says 172.21.15.64:56504) 2024-09-21T03:55:50.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.707+0000 7fa6468e0640 1 -- 172.21.15.64:0/948274415 learned_addr learned my addr 172.21.15.64:0/948274415 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:55:50.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.707+0000 7fa6460df640 1 --2- 172.21.15.64:0/948274415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa640103b80 0x7fa64019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:50.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.707+0000 7fa6458de640 1 --2- 172.21.15.64:0/948274415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6401094c0 0x7fa64019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:50.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.707+0000 7fa6468e0640 1 -- 172.21.15.64:0/948274415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa640103b80 msgr2=0x7fa64019a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:50.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.707+0000 7fa6468e0640 1 --2- 172.21.15.64:0/948274415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa640103b80 0x7fa64019a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.707+0000 7fa6468e0640 1 -- 172.21.15.64:0/948274415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6401094c0 msgr2=0x7fa64019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:50.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.707+0000 7fa6468e0640 1 --2- 172.21.15.64:0/948274415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6401094c0 0x7fa64019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.707+0000 7fa6468e0640 1 -- 172.21.15.64:0/948274415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6401a02f0 con 0x7fa640109e10 2024-09-21T03:55:50.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.707+0000 7fa6458de640 1 --2- 172.21.15.64:0/948274415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6401094c0 0x7fa64019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:50.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.707+0000 7fa6468e0640 1 --2- 172.21.15.64:0/948274415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa640109e10 0x7fa64019fc80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa630005cc0 tx=0x7fa63002f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:50.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.707+0000 7fa63f7fe640 1 -- 172.21.15.64:0/948274415 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa630003cf0 con 0x7fa640109e10 2024-09-21T03:55:50.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.707+0000 7fa63f7fe640 1 -- 172.21.15.64:0/948274415 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6300044f0 con 0x7fa640109e10 2024-09-21T03:55:50.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.707+0000 7fa648343640 1 -- 172.21.15.64:0/948274415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6401a0580 con 0x7fa640109e10 2024-09-21T03:55:50.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.707+0000 7fa63f7fe640 1 -- 172.21.15.64:0/948274415 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63002fc30 con 0x7fa640109e10 2024-09-21T03:55:50.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.707+0000 7fa648343640 1 -- 172.21.15.64:0/948274415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6401a6e00 con 0x7fa640109e10 2024-09-21T03:55:50.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.709+0000 7fa648343640 1 -- 172.21.15.64:0/948274415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa640103f90 con 0x7fa640109e10 2024-09-21T03:55:50.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.710+0000 7fa63f7fe640 1 -- 172.21.15.64:0/948274415 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fa6300046a0 con 0x7fa640109e10 2024-09-21T03:55:50.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.711+0000 7fa63f7fe640 1 --2- 172.21.15.64:0/948274415 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa6280b6580 0x7fa6280b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:50.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.711+0000 7fa6460df640 1 --2- 172.21.15.64:0/948274415 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa6280b6580 0x7fa6280b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:50.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.711+0000 7fa63f7fe640 1 -- 172.21.15.64:0/948274415 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fa630106d30 con 0x7fa640109e10 2024-09-21T03:55:50.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.711+0000 7fa6460df640 1 --2- 172.21.15.64:0/948274415 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa6280b6580 0x7fa6280b8a90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa63800abe0 tx=0x7fa63800a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:50.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.715+0000 7fa63f7fe640 1 -- 172.21.15.64:0/948274415 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa6300cf1c0 con 0x7fa640109e10 2024-09-21T03:55:50.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.831+0000 7f039461f640 1 -- 172.21.15.64:0/5586316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f038c1083a0 msgr2=0x7f038c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:50.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.831+0000 7f039461f640 1 --2- 172.21.15.64:0/5586316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f038c1083a0 0x7f038c1087b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f037c009900 tx=0x7f037c02a410 comp rx=0 tx=0).stop 2024-09-21T03:55:50.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.832+0000 7f039461f640 1 -- 172.21.15.64:0/5586316 shutdown_connections 2024-09-21T03:55:50.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.832+0000 7f039461f640 1 --2- 172.21.15.64:0/5586316 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f038c108cf0 0x7f038c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.832+0000 7f039461f640 1 --2- 172.21.15.64:0/5586316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f038c1083a0 0x7f038c1087b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.832+0000 7f039461f640 1 --2- 172.21.15.64:0/5586316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f038c102a60 0x7f038c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.832+0000 7f039461f640 1 -- 172.21.15.64:0/5586316 >> 172.21.15.64:0/5586316 conn(0x7f038c079730 msgr2=0x7f038c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:50.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.832+0000 7f039461f640 1 -- 172.21.15.64:0/5586316 shutdown_connections 2024-09-21T03:55:50.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.832+0000 7f039461f640 1 -- 172.21.15.64:0/5586316 wait complete. 2024-09-21T03:55:50.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.832+0000 7f039461f640 1 Processor -- start 2024-09-21T03:55:50.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.832+0000 7f039461f640 1 -- start start 2024-09-21T03:55:50.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.833+0000 7f039461f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f038c102a60 0x7f038c0ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:50.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.833+0000 7f039461f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f038c1083a0 0x7f038c0ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:50.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.833+0000 7f039461f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f038c108cf0 0x7f038c1000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:50.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.833+0000 7f039461f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f038c112960 con 0x7f038c102a60 2024-09-21T03:55:50.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.833+0000 7f039461f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f038c1127e0 con 0x7f038c108cf0 2024-09-21T03:55:50.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.833+0000 7f039461f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f038c112ae0 con 0x7f038c1083a0 2024-09-21T03:55:50.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.833+0000 7f03923bb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f038c102a60 0x7f038c0ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:50.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.833+0000 7f03923bb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f038c102a60 0x7f038c0ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56534/0 (socket says 172.21.15.64:56534) 2024-09-21T03:55:50.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.833+0000 7f03923bb640 1 -- 172.21.15.64:0/2841199590 learned_addr learned my addr 172.21.15.64:0/2841199590 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:55:50.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.833+0000 7f0392bbc640 1 --2- 172.21.15.64:0/2841199590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f038c108cf0 0x7f038c1000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:50.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.833+0000 7f0391bba640 1 --2- 172.21.15.64:0/2841199590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f038c1083a0 0x7f038c0ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:50.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.833+0000 7f03923bb640 1 -- 172.21.15.64:0/2841199590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f038c1083a0 msgr2=0x7f038c0ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:50.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.833+0000 7f03923bb640 1 --2- 172.21.15.64:0/2841199590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f038c1083a0 0x7f038c0ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.833+0000 7f03923bb640 1 -- 172.21.15.64:0/2841199590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f038c108cf0 msgr2=0x7f038c1000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:50.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.833+0000 7f03923bb640 1 --2- 172.21.15.64:0/2841199590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f038c108cf0 0x7f038c1000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.833+0000 7f03923bb640 1 -- 172.21.15.64:0/2841199590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f038c1a8720 con 0x7f038c102a60 2024-09-21T03:55:50.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.833+0000 7f03923bb640 1 --2- 172.21.15.64:0/2841199590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f038c102a60 0x7f038c0ff670 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f038800d950 tx=0x7f038800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:50.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.834+0000 7f03837fe640 1 -- 172.21.15.64:0/2841199590 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0388017030 con 0x7f038c102a60 2024-09-21T03:55:50.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.834+0000 7f03837fe640 1 -- 172.21.15.64:0/2841199590 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0388004020 con 0x7f038c102a60 2024-09-21T03:55:50.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.834+0000 7f039461f640 1 -- 172.21.15.64:0/2841199590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f038c1a8a10 con 0x7f038c102a60 2024-09-21T03:55:50.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.834+0000 7f03837fe640 1 -- 172.21.15.64:0/2841199590 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03880047a0 con 0x7f038c102a60 2024-09-21T03:55:50.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.834+0000 7f039461f640 1 -- 172.21.15.64:0/2841199590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f038c1a8f70 con 0x7f038c102a60 2024-09-21T03:55:50.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.835+0000 7f039461f640 1 -- 172.21.15.64:0/2841199590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f038c102e70 con 0x7f038c102a60 2024-09-21T03:55:50.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.836+0000 7f03837fe640 1 -- 172.21.15.64:0/2841199590 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f0388009a30 con 0x7f038c102a60 2024-09-21T03:55:50.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.837+0000 7f03837fe640 1 --2- 172.21.15.64:0/2841199590 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f03780b6550 0x7f03780b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:50.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.837+0000 7f0391bba640 1 --2- 172.21.15.64:0/2841199590 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f03780b6550 0x7f03780b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:50.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.837+0000 7f03837fe640 1 -- 172.21.15.64:0/2841199590 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f03880e5110 con 0x7f038c102a60 2024-09-21T03:55:50.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.838+0000 7f0391bba640 1 --2- 172.21.15.64:0/2841199590 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f03780b6550 0x7f03780b8a60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f037c000c00 tx=0x7f037c031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:50.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.842+0000 7f03837fe640 1 -- 172.21.15.64:0/2841199590 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f03880ad5a0 con 0x7f038c102a60 2024-09-21T03:55:50.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.938+0000 7fa648343640 1 -- 172.21.15.64:0/948274415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7fa64010bc60 con 0x7fa640109e10 2024-09-21T03:55:50.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.944+0000 7fa63f7fe640 1 -- 172.21.15.64:0/948274415 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v29)=0 v29) ==== 117+0+0 (secure 0 0 0) 0x7fa6300d35a0 con 0x7fa640109e10 2024-09-21T03:55:50.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.946+0000 7fa648343640 1 -- 172.21.15.64:0/948274415 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa6280b6580 msgr2=0x7fa6280b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:50.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.946+0000 7fa648343640 1 --2- 172.21.15.64:0/948274415 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa6280b6580 0x7fa6280b8a90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa63800abe0 tx=0x7fa63800a2d0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.946+0000 7fa648343640 1 -- 172.21.15.64:0/948274415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa640109e10 msgr2=0x7fa64019fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:50.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.946+0000 7fa648343640 1 --2- 172.21.15.64:0/948274415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa640109e10 0x7fa64019fc80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa630005cc0 tx=0x7fa63002f470 comp rx=0 tx=0).stop 2024-09-21T03:55:50.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.946+0000 7fa648343640 1 -- 172.21.15.64:0/948274415 shutdown_connections 2024-09-21T03:55:50.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.946+0000 7fa648343640 1 --2- 172.21.15.64:0/948274415 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa6280b6580 0x7fa6280b8a90 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.946+0000 7fa648343640 1 --2- 172.21.15.64:0/948274415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa640109e10 0x7fa64019fc80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.946+0000 7fa648343640 1 --2- 172.21.15.64:0/948274415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6401094c0 0x7fa64019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.946+0000 7fa648343640 1 --2- 172.21.15.64:0/948274415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa640103b80 0x7fa64019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:50.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.946+0000 7fa648343640 1 -- 172.21.15.64:0/948274415 >> 172.21.15.64:0/948274415 conn(0x7fa6400ff610 msgr2=0x7fa640101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:50.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.947+0000 7fa648343640 1 -- 172.21.15.64:0/948274415 shutdown_connections 2024-09-21T03:55:50.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:50.947+0000 7fa648343640 1 -- 172.21.15.64:0/948274415 wait complete. 2024-09-21T03:55:51.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:51.047+0000 7f039461f640 1 -- 172.21.15.64:0/2841199590 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f038c02e640 con 0x7f038c102a60 2024-09-21T03:55:51.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:51.051+0000 7f03837fe640 1 -- 172.21.15.64:0/2841199590 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v30)=0 v30) ==== 125+0+0 (secure 0 0 0) 0x7f03880b1980 con 0x7f038c102a60 2024-09-21T03:55:51.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:51.054+0000 7f039461f640 1 -- 172.21.15.64:0/2841199590 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f03780b6550 msgr2=0x7f03780b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:51.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:51.054+0000 7f039461f640 1 --2- 172.21.15.64:0/2841199590 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f03780b6550 0x7f03780b8a60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f037c000c00 tx=0x7f037c031400 comp rx=0 tx=0).stop 2024-09-21T03:55:51.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:51.054+0000 7f039461f640 1 -- 172.21.15.64:0/2841199590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f038c102a60 msgr2=0x7f038c0ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:51.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:51.054+0000 7f039461f640 1 --2- 172.21.15.64:0/2841199590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f038c102a60 0x7f038c0ff670 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f038800d950 tx=0x7f038800de10 comp rx=0 tx=0).stop 2024-09-21T03:55:51.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:51.054+0000 7f039461f640 1 -- 172.21.15.64:0/2841199590 shutdown_connections 2024-09-21T03:55:51.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:51.054+0000 7f039461f640 1 --2- 172.21.15.64:0/2841199590 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f03780b6550 0x7f03780b8a60 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:51.054+0000 7f039461f640 1 --2- 172.21.15.64:0/2841199590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f038c108cf0 0x7f038c1000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:51.054+0000 7f039461f640 1 --2- 172.21.15.64:0/2841199590 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f038c1083a0 0x7f038c0ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:51.054+0000 7f039461f640 1 --2- 172.21.15.64:0/2841199590 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f038c102a60 0x7f038c0ff670 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:51.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:51.054+0000 7f039461f640 1 -- 172.21.15.64:0/2841199590 >> 172.21.15.64:0/2841199590 conn(0x7f038c079730 msgr2=0x7f038c06b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:51.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:51.054+0000 7f039461f640 1 -- 172.21.15.64:0/2841199590 shutdown_connections 2024-09-21T03:55:51.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:51.054+0000 7f039461f640 1 -- 172.21.15.64:0/2841199590 wait complete. 2024-09-21T03:55:51.635 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:51.635 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:51.637 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:51.637 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:51.638 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:51.639 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:52.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.131+0000 7f86407f5640 1 -- 172.21.15.64:0/3631689715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8638104a60 msgr2=0x7f8638104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:52.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.131+0000 7f86407f5640 1 --2- 172.21.15.64:0/3631689715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8638104a60 0x7f8638104e70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8634009900 tx=0x7f863402a410 comp rx=0 tx=0).stop 2024-09-21T03:55:52.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.131+0000 7f86407f5640 1 -- 172.21.15.64:0/3631689715 shutdown_connections 2024-09-21T03:55:52.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.131+0000 7f86407f5640 1 --2- 172.21.15.64:0/3631689715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f863810dec0 0x7f86381103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:52.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.131+0000 7f86407f5640 1 --2- 172.21.15.64:0/3631689715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86381053b0 0x7f863810d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:52.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.131+0000 7f86407f5640 1 --2- 172.21.15.64:0/3631689715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8638104a60 0x7f8638104e70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:52.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.131+0000 7f86407f5640 1 -- 172.21.15.64:0/3631689715 >> 172.21.15.64:0/3631689715 conn(0x7f86380796d0 msgr2=0x7f8638079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:52.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.131+0000 7f86407f5640 1 -- 172.21.15.64:0/3631689715 shutdown_connections 2024-09-21T03:55:52.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.131+0000 7f86407f5640 1 -- 172.21.15.64:0/3631689715 wait complete. 2024-09-21T03:55:52.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.132+0000 7f86407f5640 1 Processor -- start 2024-09-21T03:55:52.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.132+0000 7f86407f5640 1 -- start start 2024-09-21T03:55:52.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.132+0000 7f86407f5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8638104a60 0x7f863819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:52.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.132+0000 7f86407f5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86381053b0 0x7f863819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:52.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.132+0000 7f86407f5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f863810dec0 0x7f863819fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:52.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.132+0000 7f86407f5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8638113b90 con 0x7f863810dec0 2024-09-21T03:55:52.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.132+0000 7f86407f5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8638113a10 con 0x7f8638104a60 2024-09-21T03:55:52.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.132+0000 7f86407f5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8638113d10 con 0x7f86381053b0 2024-09-21T03:55:52.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.132+0000 7f863ed92640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f863810dec0 0x7f863819fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:52.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.132+0000 7f863ed92640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f863810dec0 0x7f863819fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41996/0 (socket says 172.21.15.64:41996) 2024-09-21T03:55:52.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.132+0000 7f863ed92640 1 -- 172.21.15.64:0/4183614757 learned_addr learned my addr 172.21.15.64:0/4183614757 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:55:52.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.132+0000 7f863dd90640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86381053b0 0x7f863819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:52.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.132+0000 7f863e591640 1 --2- 172.21.15.64:0/4183614757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8638104a60 0x7f863819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:52.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.132+0000 7f863ed92640 1 -- 172.21.15.64:0/4183614757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86381053b0 msgr2=0x7f863819aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:52.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.132+0000 7f863ed92640 1 --2- 172.21.15.64:0/4183614757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86381053b0 0x7f863819aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:52.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.132+0000 7f863ed92640 1 -- 172.21.15.64:0/4183614757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8638104a60 msgr2=0x7f863819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:52.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.132+0000 7f863ed92640 1 --2- 172.21.15.64:0/4183614757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8638104a60 0x7f863819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:52.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.132+0000 7f863ed92640 1 -- 172.21.15.64:0/4183614757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86381a03f0 con 0x7f863810dec0 2024-09-21T03:55:52.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.132+0000 7f863dd90640 1 --2- 172.21.15.64:0/4183614757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86381053b0 0x7f863819aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:55:52.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.133+0000 7f863ed92640 1 --2- 172.21.15.64:0/4183614757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f863810dec0 0x7f863819fd80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f863000eb10 tx=0x7f863000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:52.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.133+0000 7f862f7fe640 1 -- 172.21.15.64:0/4183614757 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863000ce20 con 0x7f863810dec0 2024-09-21T03:55:52.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.133+0000 7f862f7fe640 1 -- 172.21.15.64:0/4183614757 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8630005020 con 0x7f863810dec0 2024-09-21T03:55:52.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.133+0000 7f86407f5640 1 -- 172.21.15.64:0/4183614757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86381a06e0 con 0x7f863810dec0 2024-09-21T03:55:52.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.133+0000 7f862f7fe640 1 -- 172.21.15.64:0/4183614757 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630010690 con 0x7f863810dec0 2024-09-21T03:55:52.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.133+0000 7f86407f5640 1 -- 172.21.15.64:0/4183614757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86381a6f00 con 0x7f863810dec0 2024-09-21T03:55:52.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.134+0000 7f86407f5640 1 -- 172.21.15.64:0/4183614757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8638104e70 con 0x7f863810dec0 2024-09-21T03:55:52.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.135+0000 7f862f7fe640 1 -- 172.21.15.64:0/4183614757 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f8630005510 con 0x7f863810dec0 2024-09-21T03:55:52.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.136+0000 7f862f7fe640 1 --2- 172.21.15.64:0/4183614757 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f86240b6500 0x7f86240b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:52.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.136+0000 7f862f7fe640 1 -- 172.21.15.64:0/4183614757 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f86300e5f20 con 0x7f863810dec0 2024-09-21T03:55:52.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.136+0000 7f863e591640 1 --2- 172.21.15.64:0/4183614757 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f86240b6500 0x7f86240b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:52.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.137+0000 7f863e591640 1 --2- 172.21.15.64:0/4183614757 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f86240b6500 0x7f86240b8a10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8634000c00 tx=0x7f8634031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:52.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.139+0000 7f862f7fe640 1 -- 172.21.15.64:0/4183614757 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8630056710 con 0x7f863810dec0 2024-09-21T03:55:52.250 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:51 smithi064 ceph-mon[31180]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:52.250 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:51 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/948274415' entity='client.admin' 2024-09-21T03:55:52.250 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:51 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2841199590' entity='client.admin' 2024-09-21T03:55:52.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:51 smithi119 ceph-mon[34811]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:52.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:51 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/948274415' entity='client.admin' 2024-09-21T03:55:52.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:51 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2841199590' entity='client.admin' 2024-09-21T03:55:52.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:51 smithi131 ceph-mon[35193]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:52.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:51 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/948274415' entity='client.admin' 2024-09-21T03:55:52.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:51 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2841199590' entity='client.admin' 2024-09-21T03:55:52.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.333+0000 7f86407f5640 1 -- 172.21.15.64:0/4183614757 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f863819b670 con 0x7f863810dec0 2024-09-21T03:55:52.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.337+0000 7f862f7fe640 1 -- 172.21.15.64:0/4183614757 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v31)=0 v31) ==== 129+0+0 (secure 0 0 0) 0x7f86300ae3b0 con 0x7f863810dec0 2024-09-21T03:55:52.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.339+0000 7f86407f5640 1 -- 172.21.15.64:0/4183614757 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f86240b6500 msgr2=0x7f86240b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:52.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.339+0000 7f86407f5640 1 --2- 172.21.15.64:0/4183614757 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f86240b6500 0x7f86240b8a10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8634000c00 tx=0x7f8634031400 comp rx=0 tx=0).stop 2024-09-21T03:55:52.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.339+0000 7f86407f5640 1 -- 172.21.15.64:0/4183614757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f863810dec0 msgr2=0x7f863819fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:52.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.339+0000 7f86407f5640 1 --2- 172.21.15.64:0/4183614757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f863810dec0 0x7f863819fd80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f863000eb10 tx=0x7f863000ef10 comp rx=0 tx=0).stop 2024-09-21T03:55:52.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.339+0000 7f86407f5640 1 -- 172.21.15.64:0/4183614757 shutdown_connections 2024-09-21T03:55:52.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.339+0000 7f86407f5640 1 --2- 172.21.15.64:0/4183614757 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f86240b6500 0x7f86240b8a10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:52.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.340+0000 7f86407f5640 1 --2- 172.21.15.64:0/4183614757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f863810dec0 0x7f863819fd80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:52.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.340+0000 7f86407f5640 1 --2- 172.21.15.64:0/4183614757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86381053b0 0x7f863819aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:52.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.340+0000 7f86407f5640 1 --2- 172.21.15.64:0/4183614757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8638104a60 0x7f863819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:52.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.340+0000 7f86407f5640 1 -- 172.21.15.64:0/4183614757 >> 172.21.15.64:0/4183614757 conn(0x7f86380796d0 msgr2=0x7f86380ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:52.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.340+0000 7f86407f5640 1 -- 172.21.15.64:0/4183614757 shutdown_connections 2024-09-21T03:55:52.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:52.340+0000 7f86407f5640 1 -- 172.21.15.64:0/4183614757 wait complete. 2024-09-21T03:55:53.222 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:53 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:53.222 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:53 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:53.222 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:53 smithi064 ceph-mon[31180]: Reconfiguring mon.c (monmap changed)... 2024-09-21T03:55:53.222 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:53 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:55:53.222 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:53 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:55:53.222 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:53 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:53.222 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:53 smithi064 ceph-mon[31180]: Reconfiguring daemon mon.c on smithi131 2024-09-21T03:55:53.222 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:53 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/4183614757' entity='client.admin' 2024-09-21T03:55:53.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.225+0000 7fde8301e640 1 -- 172.21.15.64:0/3734365474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde7c1083d0 msgr2=0x7fde7c1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:53.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.225+0000 7fde8301e640 1 --2- 172.21.15.64:0/3734365474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde7c1083d0 0x7fde7c1087e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fde70009900 tx=0x7fde7002a410 comp rx=0 tx=0).stop 2024-09-21T03:55:53.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.225+0000 7fde8301e640 1 -- 172.21.15.64:0/3734365474 shutdown_connections 2024-09-21T03:55:53.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.225+0000 7fde8301e640 1 --2- 172.21.15.64:0/3734365474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde7c108d20 0x7fde7c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:53.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.225+0000 7fde8301e640 1 --2- 172.21.15.64:0/3734365474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde7c1083d0 0x7fde7c1087e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:53.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.225+0000 7fde8301e640 1 --2- 172.21.15.64:0/3734365474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde7c102ae0 0x7fde7c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:53.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.225+0000 7fde8301e640 1 -- 172.21.15.64:0/3734365474 >> 172.21.15.64:0/3734365474 conn(0x7fde7c079760 msgr2=0x7fde7c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:53.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.225+0000 7fde8301e640 1 -- 172.21.15.64:0/3734365474 shutdown_connections 2024-09-21T03:55:53.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.225+0000 7fde8301e640 1 -- 172.21.15.64:0/3734365474 wait complete. 2024-09-21T03:55:53.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.225+0000 7fde8301e640 1 Processor -- start 2024-09-21T03:55:53.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.225+0000 7fde8301e640 1 -- start start 2024-09-21T03:55:53.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.226+0000 7fde8301e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde7c102ae0 0x7fde7c19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:53.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.226+0000 7fde8301e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde7c1083d0 0x7fde7c19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:53.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.226+0000 7fde8301e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde7c108d20 0x7fde7c19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:53.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.226+0000 7fde8301e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c1129e0 con 0x7fde7c108d20 2024-09-21T03:55:53.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.226+0000 7fde8301e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c112860 con 0x7fde7c102ae0 2024-09-21T03:55:53.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.226+0000 7fde8301e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c112b60 con 0x7fde7c1083d0 2024-09-21T03:55:53.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.226+0000 7fde815bb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde7c108d20 0x7fde7c19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:53.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.226+0000 7fde815bb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde7c108d20 0x7fde7c19fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42016/0 (socket says 172.21.15.64:42016) 2024-09-21T03:55:53.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.226+0000 7fde815bb640 1 -- 172.21.15.64:0/4003166011 learned_addr learned my addr 172.21.15.64:0/4003166011 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:55:53.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.226+0000 7fde7bfff640 1 --2- 172.21.15.64:0/4003166011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde7c1083d0 0x7fde7c19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:53.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.226+0000 7fde80dba640 1 --2- 172.21.15.64:0/4003166011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde7c102ae0 0x7fde7c19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:53.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.226+0000 7fde815bb640 1 -- 172.21.15.64:0/4003166011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde7c1083d0 msgr2=0x7fde7c19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:53.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.226+0000 7fde815bb640 1 --2- 172.21.15.64:0/4003166011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde7c1083d0 0x7fde7c19aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:53.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.226+0000 7fde815bb640 1 -- 172.21.15.64:0/4003166011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde7c102ae0 msgr2=0x7fde7c19a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:53.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.226+0000 7fde815bb640 1 --2- 172.21.15.64:0/4003166011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde7c102ae0 0x7fde7c19a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:53.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.226+0000 7fde815bb640 1 -- 172.21.15.64:0/4003166011 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c1a0490 con 0x7fde7c108d20 2024-09-21T03:55:53.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.227+0000 7fde815bb640 1 --2- 172.21.15.64:0/4003166011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde7c108d20 0x7fde7c19fd60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fde7400eaa0 tx=0x7fde7400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:53.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.227+0000 7fde79ffb640 1 -- 172.21.15.64:0/4003166011 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7400cdb0 con 0x7fde7c108d20 2024-09-21T03:55:53.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.227+0000 7fde79ffb640 1 -- 172.21.15.64:0/4003166011 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde74005020 con 0x7fde7c108d20 2024-09-21T03:55:53.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.227+0000 7fde8301e640 1 -- 172.21.15.64:0/4003166011 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde7c1a0780 con 0x7fde7c108d20 2024-09-21T03:55:53.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.227+0000 7fde79ffb640 1 -- 172.21.15.64:0/4003166011 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde740116d0 con 0x7fde7c108d20 2024-09-21T03:55:53.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.227+0000 7fde8301e640 1 -- 172.21.15.64:0/4003166011 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde7c0ff550 con 0x7fde7c108d20 2024-09-21T03:55:53.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.228+0000 7fde8301e640 1 -- 172.21.15.64:0/4003166011 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde7c103d10 con 0x7fde7c108d20 2024-09-21T03:55:53.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.230+0000 7fde79ffb640 1 -- 172.21.15.64:0/4003166011 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fde740051d0 con 0x7fde7c108d20 2024-09-21T03:55:53.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.230+0000 7fde79ffb640 1 --2- 172.21.15.64:0/4003166011 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fde640b64d0 0x7fde640b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:53.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.230+0000 7fde80dba640 1 --2- 172.21.15.64:0/4003166011 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fde640b64d0 0x7fde640b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:53.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.230+0000 7fde79ffb640 1 -- 172.21.15.64:0/4003166011 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fde740e57d0 con 0x7fde7c108d20 2024-09-21T03:55:53.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.231+0000 7fde80dba640 1 --2- 172.21.15.64:0/4003166011 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fde640b64d0 0x7fde640b89e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0058c0 tx=0x7fde6c010040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:53.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.235+0000 7fde79ffb640 1 -- 172.21.15.64:0/4003166011 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fde74016030 con 0x7fde7c108d20 2024-09-21T03:55:53.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:53 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:53.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:53 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:53.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:53 smithi119 ceph-mon[34811]: Reconfiguring mon.c (monmap changed)... 2024-09-21T03:55:53.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:53 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:55:53.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:53 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:55:53.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:53 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:53.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:53 smithi119 ceph-mon[34811]: Reconfiguring daemon mon.c on smithi131 2024-09-21T03:55:53.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:53 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/4183614757' entity='client.admin' 2024-09-21T03:55:53.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:53 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:53.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:53 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:53.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:53 smithi131 ceph-mon[35193]: Reconfiguring mon.c (monmap changed)... 2024-09-21T03:55:53.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:53 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T03:55:53.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:53 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T03:55:53.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:53 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:53.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:53 smithi131 ceph-mon[35193]: Reconfiguring daemon mon.c on smithi131 2024-09-21T03:55:53.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:53 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/4183614757' entity='client.admin' 2024-09-21T03:55:53.424 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.422+0000 7fde8301e640 1 -- 172.21.15.64:0/4003166011 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7fde7c10bb20 con 0x7fde7c108d20 2024-09-21T03:55:53.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.428+0000 7fde79ffb640 1 -- 172.21.15.64:0/4003166011 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v32)=0 v32) ==== 111+0+0 (secure 0 0 0) 0x7fde740b2040 con 0x7fde7c108d20 2024-09-21T03:55:53.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.430+0000 7fde8301e640 1 -- 172.21.15.64:0/4003166011 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fde640b64d0 msgr2=0x7fde640b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:53.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.430+0000 7fde8301e640 1 --2- 172.21.15.64:0/4003166011 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fde640b64d0 0x7fde640b89e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0058c0 tx=0x7fde6c010040 comp rx=0 tx=0).stop 2024-09-21T03:55:53.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.430+0000 7fde8301e640 1 -- 172.21.15.64:0/4003166011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde7c108d20 msgr2=0x7fde7c19fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:53.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.430+0000 7fde8301e640 1 --2- 172.21.15.64:0/4003166011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde7c108d20 0x7fde7c19fd60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fde7400eaa0 tx=0x7fde7400ef60 comp rx=0 tx=0).stop 2024-09-21T03:55:53.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.431+0000 7fde8301e640 1 -- 172.21.15.64:0/4003166011 shutdown_connections 2024-09-21T03:55:53.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.431+0000 7fde8301e640 1 --2- 172.21.15.64:0/4003166011 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fde640b64d0 0x7fde640b89e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:53.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.431+0000 7fde8301e640 1 --2- 172.21.15.64:0/4003166011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fde7c108d20 0x7fde7c19fd60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:53.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.431+0000 7fde8301e640 1 --2- 172.21.15.64:0/4003166011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde7c1083d0 0x7fde7c19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:53.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.431+0000 7fde8301e640 1 --2- 172.21.15.64:0/4003166011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde7c102ae0 0x7fde7c19a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:53.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.431+0000 7fde8301e640 1 -- 172.21.15.64:0/4003166011 >> 172.21.15.64:0/4003166011 conn(0x7fde7c079760 msgr2=0x7fde7c06bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:53.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.431+0000 7fde8301e640 1 -- 172.21.15.64:0/4003166011 shutdown_connections 2024-09-21T03:55:53.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.431+0000 7fde8301e640 1 -- 172.21.15.64:0/4003166011 wait complete. 2024-09-21T03:55:53.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.444+0000 7f7e038ff640 1 -- 172.21.15.64:0/3866992064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dfc105270 msgr2=0x7f7dfc10d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:53.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.444+0000 7f7e038ff640 1 --2- 172.21.15.64:0/3866992064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dfc105270 0x7f7dfc10d8b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7dec009900 tx=0x7f7dec02a410 comp rx=0 tx=0).stop 2024-09-21T03:55:53.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.444+0000 7f7e038ff640 1 -- 172.21.15.64:0/3866992064 shutdown_connections 2024-09-21T03:55:53.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.444+0000 7f7e038ff640 1 --2- 172.21.15.64:0/3866992064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dfc10ddf0 0x7f7dfc110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:53.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.444+0000 7f7e038ff640 1 --2- 172.21.15.64:0/3866992064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dfc105270 0x7f7dfc10d8b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:53.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.444+0000 7f7e038ff640 1 --2- 172.21.15.64:0/3866992064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dfc104840 0x7f7dfc104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:53.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.444+0000 7f7e038ff640 1 -- 172.21.15.64:0/3866992064 >> 172.21.15.64:0/3866992064 conn(0x7f7dfc079730 msgr2=0x7f7dfc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:53.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.444+0000 7f7e038ff640 1 -- 172.21.15.64:0/3866992064 shutdown_connections 2024-09-21T03:55:53.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.444+0000 7f7e038ff640 1 -- 172.21.15.64:0/3866992064 wait complete. 2024-09-21T03:55:53.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.445+0000 7f7e038ff640 1 Processor -- start 2024-09-21T03:55:53.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.445+0000 7f7e038ff640 1 -- start start 2024-09-21T03:55:53.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.445+0000 7f7e038ff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dfc104840 0x7f7dfc19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:53.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.445+0000 7f7e038ff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dfc105270 0x7f7dfc19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:53.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.445+0000 7f7e038ff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dfc10ddf0 0x7f7dfc19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:53.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.445+0000 7f7e038ff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc113a70 con 0x7f7dfc105270 2024-09-21T03:55:53.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.445+0000 7f7e038ff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc1138f0 con 0x7f7dfc10ddf0 2024-09-21T03:55:53.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.445+0000 7f7e038ff640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc113bf0 con 0x7f7dfc104840 2024-09-21T03:55:53.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.445+0000 7f7e00e9a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dfc105270 0x7f7dfc19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:53.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.445+0000 7f7e00e9a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dfc105270 0x7f7dfc19aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42040/0 (socket says 172.21.15.64:42040) 2024-09-21T03:55:53.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.445+0000 7f7e00e9a640 1 -- 172.21.15.64:0/761651031 learned_addr learned my addr 172.21.15.64:0/761651031 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:55:53.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.445+0000 7f7e0169b640 1 --2- 172.21.15.64:0/761651031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dfc104840 0x7f7dfc19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:53.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.446+0000 7f7e01e9c640 1 --2- 172.21.15.64:0/761651031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dfc10ddf0 0x7f7dfc19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:53.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.446+0000 7f7e00e9a640 1 -- 172.21.15.64:0/761651031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dfc104840 msgr2=0x7f7dfc19a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:53.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.446+0000 7f7e00e9a640 1 --2- 172.21.15.64:0/761651031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dfc104840 0x7f7dfc19a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:53.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.446+0000 7f7e00e9a640 1 -- 172.21.15.64:0/761651031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dfc10ddf0 msgr2=0x7f7dfc19fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:53.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.446+0000 7f7e00e9a640 1 --2- 172.21.15.64:0/761651031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dfc10ddf0 0x7f7dfc19fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:53.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.446+0000 7f7e00e9a640 1 -- 172.21.15.64:0/761651031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dfc1a0490 con 0x7f7dfc105270 2024-09-21T03:55:53.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.446+0000 7f7e01e9c640 1 --2- 172.21.15.64:0/761651031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dfc10ddf0 0x7f7dfc19fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:55:53.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.446+0000 7f7e00e9a640 1 --2- 172.21.15.64:0/761651031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dfc105270 0x7f7dfc19aac0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7dec02a930 tx=0x7f7dec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:53.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.446+0000 7f7df27fc640 1 -- 172.21.15.64:0/761651031 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec03f030 con 0x7f7dfc105270 2024-09-21T03:55:53.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.446+0000 7f7df27fc640 1 -- 172.21.15.64:0/761651031 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dec003aa0 con 0x7f7dfc105270 2024-09-21T03:55:53.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.446+0000 7f7e038ff640 1 -- 172.21.15.64:0/761651031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dfc1a0720 con 0x7f7dfc105270 2024-09-21T03:55:53.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.446+0000 7f7df27fc640 1 -- 172.21.15.64:0/761651031 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec035070 con 0x7f7dfc105270 2024-09-21T03:55:53.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.446+0000 7f7e038ff640 1 -- 172.21.15.64:0/761651031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dfc1a6ee0 con 0x7f7dfc105270 2024-09-21T03:55:53.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.447+0000 7f7e038ff640 1 -- 172.21.15.64:0/761651031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dfc104c50 con 0x7f7dfc105270 2024-09-21T03:55:53.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.449+0000 7f7df27fc640 1 -- 172.21.15.64:0/761651031 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f7dec042030 con 0x7f7dfc105270 2024-09-21T03:55:53.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.450+0000 7f7df27fc640 1 --2- 172.21.15.64:0/761651031 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7de80b64d0 0x7f7de80b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:53.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.450+0000 7f7e0169b640 1 --2- 172.21.15.64:0/761651031 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7de80b64d0 0x7f7de80b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:53.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.451+0000 7f7df27fc640 1 -- 172.21.15.64:0/761651031 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f7dec106230 con 0x7f7dfc105270 2024-09-21T03:55:53.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.451+0000 7f7e0169b640 1 --2- 172.21.15.64:0/761651031 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7de80b64d0 0x7f7de80b89e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc06bc00 tx=0x7f7df800b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:53.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.454+0000 7f7df27fc640 1 -- 172.21.15.64:0/761651031 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7dec0ce6c0 con 0x7f7dfc105270 2024-09-21T03:55:53.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.652+0000 7f7e038ff640 1 -- 172.21.15.64:0/761651031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f7dfc19b650 con 0x7f7dfc105270 2024-09-21T03:55:53.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.652+0000 7f7df27fc640 1 -- 172.21.15.64:0/761651031 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v32)=0 v32) ==== 106+0+0 (secure 0 0 0) 0x7f7dfc19b650 con 0x7f7dfc105270 2024-09-21T03:55:53.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.655+0000 7f7e038ff640 1 -- 172.21.15.64:0/761651031 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7de80b64d0 msgr2=0x7f7de80b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:53.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.655+0000 7f7e038ff640 1 --2- 172.21.15.64:0/761651031 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7de80b64d0 0x7f7de80b89e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc06bc00 tx=0x7f7df800b390 comp rx=0 tx=0).stop 2024-09-21T03:55:53.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.655+0000 7f7e038ff640 1 -- 172.21.15.64:0/761651031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dfc105270 msgr2=0x7f7dfc19aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:53.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.655+0000 7f7e038ff640 1 --2- 172.21.15.64:0/761651031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dfc105270 0x7f7dfc19aac0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7dec02a930 tx=0x7f7dec004270 comp rx=0 tx=0).stop 2024-09-21T03:55:53.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.655+0000 7f7e038ff640 1 -- 172.21.15.64:0/761651031 shutdown_connections 2024-09-21T03:55:53.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.655+0000 7f7e038ff640 1 --2- 172.21.15.64:0/761651031 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7de80b64d0 0x7f7de80b89e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:53.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.655+0000 7f7e038ff640 1 --2- 172.21.15.64:0/761651031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dfc10ddf0 0x7f7dfc19fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:53.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.655+0000 7f7e038ff640 1 --2- 172.21.15.64:0/761651031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dfc105270 0x7f7dfc19aac0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:53.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.655+0000 7f7e038ff640 1 --2- 172.21.15.64:0/761651031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7dfc104840 0x7f7dfc19a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:53.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.655+0000 7f7e038ff640 1 -- 172.21.15.64:0/761651031 >> 172.21.15.64:0/761651031 conn(0x7f7dfc079730 msgr2=0x7f7dfc100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:53.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.655+0000 7f7e038ff640 1 -- 172.21.15.64:0/761651031 shutdown_connections 2024-09-21T03:55:53.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:53.655+0000 7f7e038ff640 1 -- 172.21.15.64:0/761651031 wait complete. 2024-09-21T03:55:54.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:54 smithi064 ceph-mon[31180]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:54.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:54 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/4003166011' entity='client.admin' 2024-09-21T03:55:54.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:54 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:54.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:54 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:54.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:54 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:55:54.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:54 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:54.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:54 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:55:54.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:54 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:54.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:54 smithi131 ceph-mon[35193]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:54.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:54 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/4003166011' entity='client.admin' 2024-09-21T03:55:54.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:54 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:54.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:54 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:54.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:54 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:55:54.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:54 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:54.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:54 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:55:54.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:54 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:54.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:54 smithi119 ceph-mon[34811]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:54.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:54 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/4003166011' entity='client.admin' 2024-09-21T03:55:54.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:54 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:54.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:54 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:54.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:54 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:55:54.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:54 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:55:54.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:54 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:55:54.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:54 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:55:55.103 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:55.104 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:55:55.739 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi064') 2024-09-21T03:55:55.739 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi064') 2024-09-21T03:55:55.740 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi064') 2024-09-21T03:55:55.740 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi064') 2024-09-21T03:55:55.740 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi064') 2024-09-21T03:55:55.740 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi064') 2024-09-21T03:55:55.740 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi064') 2024-09-21T03:55:55.740 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi064') 2024-09-21T03:55:55.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.868+0000 7fdd2b38d640 1 -- 172.21.15.64:0/3031727111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd24105380 msgr2=0x7fdd2410d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:55.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.868+0000 7fdd2b38d640 1 --2- 172.21.15.64:0/3031727111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd24105380 0x7fdd2410d8d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009900 tx=0x7fdd1402a410 comp rx=0 tx=0).stop 2024-09-21T03:55:55.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.868+0000 7fdd2b38d640 1 -- 172.21.15.64:0/3031727111 shutdown_connections 2024-09-21T03:55:55.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.868+0000 7fdd2b38d640 1 --2- 172.21.15.64:0/3031727111 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd2410de60 0x7fdd24110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:55.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.868+0000 7fdd2b38d640 1 --2- 172.21.15.64:0/3031727111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd24105380 0x7fdd2410d8d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:55.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.868+0000 7fdd2b38d640 1 --2- 172.21.15.64:0/3031727111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd24104a30 0x7fdd24104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:55.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.868+0000 7fdd2b38d640 1 -- 172.21.15.64:0/3031727111 >> 172.21.15.64:0/3031727111 conn(0x7fdd24079760 msgr2=0x7fdd24079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:55.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.869+0000 7fdd2b38d640 1 -- 172.21.15.64:0/3031727111 shutdown_connections 2024-09-21T03:55:55.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.869+0000 7fdd2b38d640 1 -- 172.21.15.64:0/3031727111 wait complete. 2024-09-21T03:55:55.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.869+0000 7fdd2b38d640 1 Processor -- start 2024-09-21T03:55:55.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.869+0000 7fdd2b38d640 1 -- start start 2024-09-21T03:55:55.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.870+0000 7fdd2b38d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd24104a30 0x7fdd2419c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:55.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.870+0000 7fdd2b38d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd24105380 0x7fdd2419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:55.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.870+0000 7fdd2b38d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd2410de60 0x7fdd241a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:55.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.870+0000 7fdd2b38d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdd24113aa0 con 0x7fdd24104a30 2024-09-21T03:55:55.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.870+0000 7fdd2b38d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdd24113920 con 0x7fdd2410de60 2024-09-21T03:55:55.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.870+0000 7fdd2b38d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdd24113c20 con 0x7fdd24105380 2024-09-21T03:55:55.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.870+0000 7fdd29129640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd24104a30 0x7fdd2419c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:55.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.870+0000 7fdd29129640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd24104a30 0x7fdd2419c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42052/0 (socket says 172.21.15.64:42052) 2024-09-21T03:55:55.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.870+0000 7fdd29129640 1 -- 172.21.15.64:0/483903257 learned_addr learned my addr 172.21.15.64:0/483903257 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:55:55.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.870+0000 7fdd2992a640 1 --2- 172.21.15.64:0/483903257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd2410de60 0x7fdd241a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:55.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.870+0000 7fdd28928640 1 --2- 172.21.15.64:0/483903257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd24105380 0x7fdd2419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:55.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.870+0000 7fdd29129640 1 -- 172.21.15.64:0/483903257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd24105380 msgr2=0x7fdd2419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:55.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.870+0000 7fdd29129640 1 --2- 172.21.15.64:0/483903257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd24105380 0x7fdd2419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:55.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.870+0000 7fdd29129640 1 -- 172.21.15.64:0/483903257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd2410de60 msgr2=0x7fdd241a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:55.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.870+0000 7fdd29129640 1 --2- 172.21.15.64:0/483903257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd2410de60 0x7fdd241a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:55.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.870+0000 7fdd29129640 1 -- 172.21.15.64:0/483903257 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd241a26c0 con 0x7fdd24104a30 2024-09-21T03:55:55.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.870+0000 7fdd2992a640 1 --2- 172.21.15.64:0/483903257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd2410de60 0x7fdd241a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:55.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.870+0000 7fdd28928640 1 --2- 172.21.15.64:0/483903257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd24105380 0x7fdd2419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:55.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.870+0000 7fdd29129640 1 --2- 172.21.15.64:0/483903257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd24104a30 0x7fdd2419c750 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000e9c0 tx=0x7fdd2000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:55.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.871+0000 7fdd1a7fc640 1 -- 172.21.15.64:0/483903257 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2000cd80 con 0x7fdd24104a30 2024-09-21T03:55:55.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.871+0000 7fdd2b38d640 1 -- 172.21.15.64:0/483903257 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd241a29b0 con 0x7fdd24104a30 2024-09-21T03:55:55.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.871+0000 7fdd1a7fc640 1 -- 172.21.15.64:0/483903257 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd200035f0 con 0x7fdd24104a30 2024-09-21T03:55:55.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.871+0000 7fdd1a7fc640 1 -- 172.21.15.64:0/483903257 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20010690 con 0x7fdd24104a30 2024-09-21T03:55:55.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.871+0000 7fdd2b38d640 1 -- 172.21.15.64:0/483903257 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd241a9110 con 0x7fdd24104a30 2024-09-21T03:55:55.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.872+0000 7fdd2b38d640 1 -- 172.21.15.64:0/483903257 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd2406c1f0 con 0x7fdd24104a30 2024-09-21T03:55:55.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.873+0000 7fdd1a7fc640 1 -- 172.21.15.64:0/483903257 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fdd200037a0 con 0x7fdd24104a30 2024-09-21T03:55:55.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.874+0000 7fdd1a7fc640 1 --2- 172.21.15.64:0/483903257 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdd100b64d0 0x7fdd100b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:55.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.874+0000 7fdd28928640 1 --2- 172.21.15.64:0/483903257 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdd100b64d0 0x7fdd100b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:55.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.874+0000 7fdd1a7fc640 1 -- 172.21.15.64:0/483903257 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fdd200e5d10 con 0x7fdd24104a30 2024-09-21T03:55:55.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.875+0000 7fdd28928640 1 --2- 172.21.15.64:0/483903257 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdd100b64d0 0x7fdd100b89e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009230 tx=0x7fdd14000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:55.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:55.877+0000 7fdd1a7fc640 1 -- 172.21.15.64:0/483903257 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdd200ae1a0 con 0x7fdd24104a30 2024-09-21T03:55:56.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.071+0000 7fdd2b38d640 1 -- 172.21.15.64:0/483903257 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7fdd241a9400 con 0x7fdd24104a30 2024-09-21T03:55:56.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.077+0000 7fdd1a7fc640 1 -- 172.21.15.64:0/483903257 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v33)=0 v33) ==== 121+0+0 (secure 0 0 0) 0x7fdd200b2580 con 0x7fdd24104a30 2024-09-21T03:55:56.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.079+0000 7fdd2b38d640 1 -- 172.21.15.64:0/483903257 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdd100b64d0 msgr2=0x7fdd100b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:56.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.079+0000 7fdd2b38d640 1 --2- 172.21.15.64:0/483903257 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdd100b64d0 0x7fdd100b89e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009230 tx=0x7fdd14000bc0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.079+0000 7fdd2b38d640 1 -- 172.21.15.64:0/483903257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd24104a30 msgr2=0x7fdd2419c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:56.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.079+0000 7fdd2b38d640 1 --2- 172.21.15.64:0/483903257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd24104a30 0x7fdd2419c750 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000e9c0 tx=0x7fdd2000ee80 comp rx=0 tx=0).stop 2024-09-21T03:55:56.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.079+0000 7fdd2b38d640 1 -- 172.21.15.64:0/483903257 shutdown_connections 2024-09-21T03:55:56.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.079+0000 7fdd2b38d640 1 --2- 172.21.15.64:0/483903257 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdd100b64d0 0x7fdd100b89e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.079+0000 7fdd2b38d640 1 --2- 172.21.15.64:0/483903257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd2410de60 0x7fdd241a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.079+0000 7fdd2b38d640 1 --2- 172.21.15.64:0/483903257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd24105380 0x7fdd2419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.079+0000 7fdd2b38d640 1 --2- 172.21.15.64:0/483903257 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd24104a30 0x7fdd2419c750 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.079+0000 7fdd2b38d640 1 -- 172.21.15.64:0/483903257 >> 172.21.15.64:0/483903257 conn(0x7fdd24079760 msgr2=0x7fdd24100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:56.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.079+0000 7fdd2b38d640 1 -- 172.21.15.64:0/483903257 shutdown_connections 2024-09-21T03:55:56.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.079+0000 7fdd2b38d640 1 -- 172.21.15.64:0/483903257 wait complete. 2024-09-21T03:55:56.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:56 smithi064 ceph-mon[31180]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:56.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:56 smithi131 ceph-mon[35193]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:56.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:56 smithi119 ceph-mon[34811]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.519+0000 7fdd055bf640 1 -- 172.21.15.64:0/1344006291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd00108d30 msgr2=0x7fdd0010f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.519+0000 7fdd055bf640 1 --2- 172.21.15.64:0/1344006291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd00108d30 0x7fdd0010f560 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8009ca0 tx=0x7fdcf802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.519+0000 7fdd055bf640 1 -- 172.21.15.64:0/1344006291 shutdown_connections 2024-09-21T03:55:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.519+0000 7fdd055bf640 1 --2- 172.21.15.64:0/1344006291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd00108d30 0x7fdd0010f560 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.519+0000 7fdd055bf640 1 --2- 172.21.15.64:0/1344006291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd001083e0 0x7fdd001087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.519+0000 7fdd055bf640 1 --2- 172.21.15.64:0/1344006291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd00102aa0 0x7fdd00102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.519+0000 7fdd055bf640 1 -- 172.21.15.64:0/1344006291 >> 172.21.15.64:0/1344006291 conn(0x7fdd00079750 msgr2=0x7fdd00079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.519+0000 7fdd055bf640 1 -- 172.21.15.64:0/1344006291 shutdown_connections 2024-09-21T03:55:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.519+0000 7fdd055bf640 1 -- 172.21.15.64:0/1344006291 wait complete. 2024-09-21T03:55:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.519+0000 7fdd055bf640 1 Processor -- start 2024-09-21T03:55:56.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.520+0000 7fdd055bf640 1 -- start start 2024-09-21T03:55:56.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.520+0000 7fdd055bf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd00102aa0 0x7fdd000ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:56.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.520+0000 7fdd055bf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd001083e0 0x7fdd000ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:56.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.520+0000 7fdd055bf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd00108d30 0x7fdd001000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:56.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.520+0000 7fdd055bf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdd001129f0 con 0x7fdd00102aa0 2024-09-21T03:55:56.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.520+0000 7fdd055bf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdd00112870 con 0x7fdd00108d30 2024-09-21T03:55:56.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.520+0000 7fdd055bf640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdd00112b70 con 0x7fdd001083e0 2024-09-21T03:55:56.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.520+0000 7fdcfeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd00102aa0 0x7fdd000ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:56.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.520+0000 7fdcfeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd00102aa0 0x7fdd000ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42082/0 (socket says 172.21.15.64:42082) 2024-09-21T03:55:56.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.520+0000 7fdcfeffd640 1 -- 172.21.15.64:0/2216194412 learned_addr learned my addr 172.21.15.64:0/2216194412 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:55:56.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.520+0000 7fdcff7fe640 1 --2- 172.21.15.64:0/2216194412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd00108d30 0x7fdd001000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:56.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.520+0000 7fdcfe7fc640 1 --2- 172.21.15.64:0/2216194412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd001083e0 0x7fdd000ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:56.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.520+0000 7fdcfeffd640 1 -- 172.21.15.64:0/2216194412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd001083e0 msgr2=0x7fdd000ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:56.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.520+0000 7fdcfeffd640 1 --2- 172.21.15.64:0/2216194412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd001083e0 0x7fdd000ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.520+0000 7fdcfeffd640 1 -- 172.21.15.64:0/2216194412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd00108d30 msgr2=0x7fdd001000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:56.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.520+0000 7fdcfeffd640 1 --2- 172.21.15.64:0/2216194412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd00108d30 0x7fdd001000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.520+0000 7fdcfeffd640 1 -- 172.21.15.64:0/2216194412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd001a87b0 con 0x7fdd00102aa0 2024-09-21T03:55:56.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.520+0000 7fdcfe7fc640 1 --2- 172.21.15.64:0/2216194412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd001083e0 0x7fdd000ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:55:56.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.520+0000 7fdcff7fe640 1 --2- 172.21.15.64:0/2216194412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd00108d30 0x7fdd001000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:56.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.520+0000 7fdcfeffd640 1 --2- 172.21.15.64:0/2216194412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd00102aa0 0x7fdd000ff670 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000c970 tx=0x7fdcf000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:56.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.521+0000 7fdce7fff640 1 -- 172.21.15.64:0/2216194412 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0007bb0 con 0x7fdd00102aa0 2024-09-21T03:55:56.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.521+0000 7fdd055bf640 1 -- 172.21.15.64:0/2216194412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd001a8aa0 con 0x7fdd00102aa0 2024-09-21T03:55:56.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.521+0000 7fdce7fff640 1 -- 172.21.15.64:0/2216194412 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcf0007d50 con 0x7fdd00102aa0 2024-09-21T03:55:56.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.521+0000 7fdce7fff640 1 -- 172.21.15.64:0/2216194412 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf000f450 con 0x7fdd00102aa0 2024-09-21T03:55:56.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.521+0000 7fdd055bf640 1 -- 172.21.15.64:0/2216194412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd001a8f90 con 0x7fdd00102aa0 2024-09-21T03:55:56.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.522+0000 7fdd055bf640 1 -- 172.21.15.64:0/2216194412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd00102eb0 con 0x7fdd00102aa0 2024-09-21T03:55:56.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.523+0000 7fdce7fff640 1 -- 172.21.15.64:0/2216194412 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fdcf0004f00 con 0x7fdd00102aa0 2024-09-21T03:55:56.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.524+0000 7fdce7fff640 1 --2- 172.21.15.64:0/2216194412 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdce80b64d0 0x7fdce80b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:56.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.524+0000 7fdcfe7fc640 1 --2- 172.21.15.64:0/2216194412 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdce80b64d0 0x7fdce80b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:56.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.524+0000 7fdce7fff640 1 -- 172.21.15.64:0/2216194412 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fdcf00e58d0 con 0x7fdd00102aa0 2024-09-21T03:55:56.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.525+0000 7fdcfe7fc640 1 --2- 172.21.15.64:0/2216194412 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdce80b64d0 0x7fdce80b89e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdd00079c70 tx=0x7fdcf4005cb0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:56.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.527+0000 7fdce7fff640 1 -- 172.21.15.64:0/2216194412 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdcf0014030 con 0x7fdd00102aa0 2024-09-21T03:55:56.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.716+0000 7fdd055bf640 1 -- 172.21.15.64:0/2216194412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7fdd0002e640 con 0x7fdd00102aa0 2024-09-21T03:55:56.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.717+0000 7fdce7fff640 1 -- 172.21.15.64:0/2216194412 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v33)=0 v33) ==== 108+0+0 (secure 0 0 0) 0x7fdcf00add60 con 0x7fdd00102aa0 2024-09-21T03:55:56.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.719+0000 7fdd055bf640 1 -- 172.21.15.64:0/2216194412 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdce80b64d0 msgr2=0x7fdce80b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:56.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.719+0000 7fdd055bf640 1 --2- 172.21.15.64:0/2216194412 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdce80b64d0 0x7fdce80b89e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdd00079c70 tx=0x7fdcf4005cb0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.719+0000 7fdd055bf640 1 -- 172.21.15.64:0/2216194412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd00102aa0 msgr2=0x7fdd000ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:56.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.719+0000 7fdd055bf640 1 --2- 172.21.15.64:0/2216194412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd00102aa0 0x7fdd000ff670 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000c970 tx=0x7fdcf000ce30 comp rx=0 tx=0).stop 2024-09-21T03:55:56.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.719+0000 7fdd055bf640 1 -- 172.21.15.64:0/2216194412 shutdown_connections 2024-09-21T03:55:56.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.719+0000 7fdd055bf640 1 --2- 172.21.15.64:0/2216194412 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdce80b64d0 0x7fdce80b89e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.719+0000 7fdd055bf640 1 --2- 172.21.15.64:0/2216194412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd00108d30 0x7fdd001000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.719+0000 7fdd055bf640 1 --2- 172.21.15.64:0/2216194412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd001083e0 0x7fdd000ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.719+0000 7fdd055bf640 1 --2- 172.21.15.64:0/2216194412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd00102aa0 0x7fdd000ff670 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:56.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.719+0000 7fdd055bf640 1 -- 172.21.15.64:0/2216194412 >> 172.21.15.64:0/2216194412 conn(0x7fdd00079750 msgr2=0x7fdd00101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:56.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.719+0000 7fdd055bf640 1 -- 172.21.15.64:0/2216194412 shutdown_connections 2024-09-21T03:55:56.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:56.719+0000 7fdd055bf640 1 -- 172.21.15.64:0/2216194412 wait complete. 2024-09-21T03:55:57.424 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:57 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/483903257' entity='client.admin' 2024-09-21T03:55:57.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:57 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/483903257' entity='client.admin' 2024-09-21T03:55:57.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:57 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/483903257' entity='client.admin' 2024-09-21T03:55:57.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.637+0000 7f1b06da8640 1 -- 172.21.15.64:0/86297353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b00108cf0 msgr2=0x7f1b0010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:57.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.637+0000 7f1b06da8640 1 --2- 172.21.15.64:0/86297353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b00108cf0 0x7f1b0010f520 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1af8009ca0 tx=0x7f1af802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:55:57.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.637+0000 7f1b06da8640 1 -- 172.21.15.64:0/86297353 shutdown_connections 2024-09-21T03:55:57.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.637+0000 7f1b06da8640 1 --2- 172.21.15.64:0/86297353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b00108cf0 0x7f1b0010f520 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:57.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.637+0000 7f1b06da8640 1 --2- 172.21.15.64:0/86297353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b001083a0 0x7f1b001087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:57.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.637+0000 7f1b06da8640 1 --2- 172.21.15.64:0/86297353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b00102a60 0x7f1b00102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:57.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.637+0000 7f1b06da8640 1 -- 172.21.15.64:0/86297353 >> 172.21.15.64:0/86297353 conn(0x7f1b00079730 msgr2=0x7f1b00079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:57.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.637+0000 7f1b06da8640 1 -- 172.21.15.64:0/86297353 shutdown_connections 2024-09-21T03:55:57.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.637+0000 7f1b06da8640 1 -- 172.21.15.64:0/86297353 wait complete. 2024-09-21T03:55:57.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.638+0000 7f1b06da8640 1 Processor -- start 2024-09-21T03:55:57.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.638+0000 7f1b06da8640 1 -- start start 2024-09-21T03:55:57.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.638+0000 7f1b06da8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b00102a60 0x7f1b0010ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:57.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.638+0000 7f1b06da8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b001083a0 0x7f1b001104f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:57.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.638+0000 7f1b06da8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b00108cf0 0x7f1b00110a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:57.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.638+0000 7f1b06da8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00113a80 con 0x7f1b00102a60 2024-09-21T03:55:57.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.638+0000 7f1b06da8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00113900 con 0x7f1b001083a0 2024-09-21T03:55:57.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.638+0000 7f1b06da8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1b00113c00 con 0x7f1b00108cf0 2024-09-21T03:55:57.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.638+0000 7f1b04b44640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b00102a60 0x7f1b0010ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:57.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.638+0000 7f1b04b44640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b00102a60 0x7f1b0010ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42104/0 (socket says 172.21.15.64:42104) 2024-09-21T03:55:57.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.638+0000 7f1b04b44640 1 -- 172.21.15.64:0/3044478157 learned_addr learned my addr 172.21.15.64:0/3044478157 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:55:57.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.639+0000 7f1b05345640 1 --2- 172.21.15.64:0/3044478157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b00108cf0 0x7f1b00110a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:57.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.639+0000 7f1afffff640 1 --2- 172.21.15.64:0/3044478157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b001083a0 0x7f1b001104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:57.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.639+0000 7f1b04b44640 1 -- 172.21.15.64:0/3044478157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b00108cf0 msgr2=0x7f1b00110a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:57.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.639+0000 7f1b04b44640 1 --2- 172.21.15.64:0/3044478157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b00108cf0 0x7f1b00110a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:57.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.639+0000 7f1b04b44640 1 -- 172.21.15.64:0/3044478157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b001083a0 msgr2=0x7f1b001104f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:57.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.639+0000 7f1b04b44640 1 --2- 172.21.15.64:0/3044478157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b001083a0 0x7f1b001104f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:57.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.639+0000 7f1b04b44640 1 -- 172.21.15.64:0/3044478157 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b001a8750 con 0x7f1b00102a60 2024-09-21T03:55:57.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.639+0000 7f1afffff640 1 --2- 172.21.15.64:0/3044478157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b001083a0 0x7f1b001104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:55:57.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.639+0000 7f1b04b44640 1 --2- 172.21.15.64:0/3044478157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b00102a60 0x7f1b0010ffb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1af000e9f0 tx=0x7f1af000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:57.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.639+0000 7f1afdffb640 1 -- 172.21.15.64:0/3044478157 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af000cd80 con 0x7f1b00102a60 2024-09-21T03:55:57.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.639+0000 7f1afdffb640 1 -- 172.21.15.64:0/3044478157 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1af00108e0 con 0x7f1b00102a60 2024-09-21T03:55:57.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.639+0000 7f1b06da8640 1 -- 172.21.15.64:0/3044478157 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b001a8a40 con 0x7f1b00102a60 2024-09-21T03:55:57.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.639+0000 7f1afdffb640 1 -- 172.21.15.64:0/3044478157 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af0018980 con 0x7f1b00102a60 2024-09-21T03:55:57.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.639+0000 7f1b06da8640 1 -- 172.21.15.64:0/3044478157 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b001a8fd0 con 0x7f1b00102a60 2024-09-21T03:55:57.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.640+0000 7f1b06da8640 1 -- 172.21.15.64:0/3044478157 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b00102e70 con 0x7f1b00102a60 2024-09-21T03:55:57.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.642+0000 7f1afdffb640 1 -- 172.21.15.64:0/3044478157 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f1af0010430 con 0x7f1b00102a60 2024-09-21T03:55:57.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.643+0000 7f1afdffb640 1 --2- 172.21.15.64:0/3044478157 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f1ae80b6550 0x7f1ae80b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:57.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.643+0000 7f1afffff640 1 --2- 172.21.15.64:0/3044478157 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f1ae80b6550 0x7f1ae80b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:57.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.643+0000 7f1afdffb640 1 -- 172.21.15.64:0/3044478157 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f1af00e5fe0 con 0x7f1b00102a60 2024-09-21T03:55:57.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.644+0000 7f1afffff640 1 --2- 172.21.15.64:0/3044478157 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f1ae80b6550 0x7f1ae80b8a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1af4005fd0 tx=0x7f1af40094d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:57.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.646+0000 7f1afdffb640 1 -- 172.21.15.64:0/3044478157 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1af00ae470 con 0x7f1b00102a60 2024-09-21T03:55:57.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.834+0000 7f1b06da8640 1 -- 172.21.15.64:0/3044478157 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f1b001112b0 con 0x7f1b00102a60 2024-09-21T03:55:57.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.838+0000 7f1afdffb640 1 -- 172.21.15.64:0/3044478157 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v34)=0 v34) ==== 121+0+0 (secure 0 0 0) 0x7f1b001112b0 con 0x7f1b00102a60 2024-09-21T03:55:57.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.841+0000 7f1b06da8640 1 -- 172.21.15.64:0/3044478157 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f1ae80b6550 msgr2=0x7f1ae80b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:57.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.841+0000 7f1b06da8640 1 --2- 172.21.15.64:0/3044478157 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f1ae80b6550 0x7f1ae80b8a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1af4005fd0 tx=0x7f1af40094d0 comp rx=0 tx=0).stop 2024-09-21T03:55:57.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.841+0000 7f1b06da8640 1 -- 172.21.15.64:0/3044478157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b00102a60 msgr2=0x7f1b0010ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:57.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.841+0000 7f1b06da8640 1 --2- 172.21.15.64:0/3044478157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b00102a60 0x7f1b0010ffb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1af000e9f0 tx=0x7f1af000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:55:57.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.841+0000 7f1b06da8640 1 -- 172.21.15.64:0/3044478157 shutdown_connections 2024-09-21T03:55:57.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.841+0000 7f1b06da8640 1 --2- 172.21.15.64:0/3044478157 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f1ae80b6550 0x7f1ae80b8a60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:57.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.841+0000 7f1b06da8640 1 --2- 172.21.15.64:0/3044478157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1b00108cf0 0x7f1b00110a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:57.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.841+0000 7f1b06da8640 1 --2- 172.21.15.64:0/3044478157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b001083a0 0x7f1b001104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:57.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.841+0000 7f1b06da8640 1 --2- 172.21.15.64:0/3044478157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b00102a60 0x7f1b0010ffb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:57.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.841+0000 7f1b06da8640 1 -- 172.21.15.64:0/3044478157 >> 172.21.15.64:0/3044478157 conn(0x7f1b00079730 msgr2=0x7f1b00104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:57.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.841+0000 7f1b06da8640 1 -- 172.21.15.64:0/3044478157 shutdown_connections 2024-09-21T03:55:57.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:57.841+0000 7f1b06da8640 1 -- 172.21.15.64:0/3044478157 wait complete. 2024-09-21T03:55:58.205 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi064') 2024-09-21T03:55:58.205 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi064') 2024-09-21T03:55:58.205 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi064') 2024-09-21T03:55:58.205 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi064') 2024-09-21T03:55:58.206 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi064') 2024-09-21T03:55:58.206 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_cephsqlite', '20'], check_status=True, hostname='smithi064') 2024-09-21T03:55:58.471 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:58 smithi064 ceph-mon[31180]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:58.471 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:58 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3044478157' entity='client.admin' 2024-09-21T03:55:58.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:58 smithi119 ceph-mon[34811]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:58.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:58 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3044478157' entity='client.admin' 2024-09-21T03:55:58.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:58 smithi131 ceph-mon[35193]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:55:58.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:58 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3044478157' entity='client.admin' 2024-09-21T03:55:58.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.766+0000 7f0fa8929640 1 -- 172.21.15.64:0/1516024653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fa4108d00 msgr2=0x7f0fa410f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:58.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.766+0000 7f0fa8929640 1 --2- 172.21.15.64:0/1516024653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fa4108d00 0x7f0fa410f530 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c009ca0 tx=0x7f0f9c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.766+0000 7f0fa8929640 1 -- 172.21.15.64:0/1516024653 shutdown_connections 2024-09-21T03:55:58.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.766+0000 7f0fa8929640 1 --2- 172.21.15.64:0/1516024653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fa4108d00 0x7f0fa410f530 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.766+0000 7f0fa8929640 1 --2- 172.21.15.64:0/1516024653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fa41083b0 0x7f0fa41087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.766+0000 7f0fa8929640 1 --2- 172.21.15.64:0/1516024653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fa4102a70 0x7f0fa4102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.766+0000 7f0fa8929640 1 -- 172.21.15.64:0/1516024653 >> 172.21.15.64:0/1516024653 conn(0x7f0fa4079710 msgr2=0x7f0fa4079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:58.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.766+0000 7f0fa8929640 1 -- 172.21.15.64:0/1516024653 shutdown_connections 2024-09-21T03:55:58.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.766+0000 7f0fa8929640 1 -- 172.21.15.64:0/1516024653 wait complete. 2024-09-21T03:55:58.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.767+0000 7f0fa8929640 1 Processor -- start 2024-09-21T03:55:58.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.767+0000 7f0fa8929640 1 -- start start 2024-09-21T03:55:58.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.768+0000 7f0fa8929640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fa4102a70 0x7f0fa4199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:58.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.768+0000 7f0fa8929640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fa41083b0 0x7f0fa41998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:58.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.768+0000 7f0fa8929640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fa4108d00 0x7f0fa419eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:58.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.768+0000 7f0fa8929640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa41129c0 con 0x7f0fa4102a70 2024-09-21T03:55:58.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.768+0000 7f0fa8929640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa4112840 con 0x7f0fa41083b0 2024-09-21T03:55:58.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.768+0000 7f0fa8929640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa4112b40 con 0x7f0fa4108d00 2024-09-21T03:55:58.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.768+0000 7f0fa259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fa4102a70 0x7f0fa4199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:58.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.768+0000 7f0fa259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fa4102a70 0x7f0fa4199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42124/0 (socket says 172.21.15.64:42124) 2024-09-21T03:55:58.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.768+0000 7f0fa259c640 1 -- 172.21.15.64:0/3500555394 learned_addr learned my addr 172.21.15.64:0/3500555394 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:55:58.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.768+0000 7f0fa1d9b640 1 --2- 172.21.15.64:0/3500555394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fa41083b0 0x7f0fa41998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:58.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.768+0000 7f0fa2d9d640 1 --2- 172.21.15.64:0/3500555394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fa4108d00 0x7f0fa419eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:58.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.768+0000 7f0fa259c640 1 -- 172.21.15.64:0/3500555394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fa4108d00 msgr2=0x7f0fa419eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:58.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.768+0000 7f0fa259c640 1 --2- 172.21.15.64:0/3500555394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fa4108d00 0x7f0fa419eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.768+0000 7f0fa259c640 1 -- 172.21.15.64:0/3500555394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fa41083b0 msgr2=0x7f0fa41998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:58.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.768+0000 7f0fa259c640 1 --2- 172.21.15.64:0/3500555394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fa41083b0 0x7f0fa41998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.768+0000 7f0fa259c640 1 -- 172.21.15.64:0/3500555394 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa419f2c0 con 0x7f0fa4102a70 2024-09-21T03:55:58.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.768+0000 7f0fa259c640 1 --2- 172.21.15.64:0/3500555394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fa4102a70 0x7f0fa4199350 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00d950 tx=0x7f0f8c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:58.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.768+0000 7f0f9b7fe640 1 -- 172.21.15.64:0/3500555394 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c017030 con 0x7f0fa4102a70 2024-09-21T03:55:58.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.769+0000 7f0f9b7fe640 1 -- 172.21.15.64:0/3500555394 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f8c004020 con 0x7f0fa4102a70 2024-09-21T03:55:58.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.769+0000 7f0fa8929640 1 -- 172.21.15.64:0/3500555394 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa419f5b0 con 0x7f0fa4102a70 2024-09-21T03:55:58.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.769+0000 7f0f9b7fe640 1 -- 172.21.15.64:0/3500555394 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c0047a0 con 0x7f0fa4102a70 2024-09-21T03:55:58.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.769+0000 7f0fa8929640 1 -- 172.21.15.64:0/3500555394 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa41a5d10 con 0x7f0fa4102a70 2024-09-21T03:55:58.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.770+0000 7f0fa8929640 1 -- 172.21.15.64:0/3500555394 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fa4103cf0 con 0x7f0fa4102a70 2024-09-21T03:55:58.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.771+0000 7f0f9b7fe640 1 -- 172.21.15.64:0/3500555394 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f0f8c009a30 con 0x7f0fa4102a70 2024-09-21T03:55:58.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.772+0000 7f0f9b7fe640 1 --2- 172.21.15.64:0/3500555394 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0f900b64d0 0x7f0f900b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:58.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.772+0000 7f0fa1d9b640 1 --2- 172.21.15.64:0/3500555394 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0f900b64d0 0x7f0f900b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:58.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.772+0000 7f0f9b7fe640 1 -- 172.21.15.64:0/3500555394 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f0f8c0e5180 con 0x7f0fa4102a70 2024-09-21T03:55:58.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.773+0000 7f0fa1d9b640 1 --2- 172.21.15.64:0/3500555394 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0f900b64d0 0x7f0f900b89e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4107ed0 tx=0x7f0f9400b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:58.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.777+0000 7f0f9b7fe640 1 -- 172.21.15.64:0/3500555394 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0f8c0ad610 con 0x7f0fa4102a70 2024-09-21T03:55:58.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.969+0000 7f0fa8929640 1 -- 172.21.15.64:0/3500555394 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f0fa402e640 con 0x7f0fa4102a70 2024-09-21T03:55:58.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.991+0000 7f0f9b7fe640 1 -- 172.21.15.64:0/3500555394 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v35)=0 v35) ==== 138+0+0 (secure 0 0 0) 0x7f0f8c0b19f0 con 0x7f0fa4102a70 2024-09-21T03:55:58.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.994+0000 7f0fa8929640 1 -- 172.21.15.64:0/3500555394 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0f900b64d0 msgr2=0x7f0f900b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:58.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.994+0000 7f0fa8929640 1 --2- 172.21.15.64:0/3500555394 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0f900b64d0 0x7f0f900b89e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4107ed0 tx=0x7f0f9400b390 comp rx=0 tx=0).stop 2024-09-21T03:55:58.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.994+0000 7f0fa8929640 1 -- 172.21.15.64:0/3500555394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fa4102a70 msgr2=0x7f0fa4199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:58.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.994+0000 7f0fa8929640 1 --2- 172.21.15.64:0/3500555394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fa4102a70 0x7f0fa4199350 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00d950 tx=0x7f0f8c00de10 comp rx=0 tx=0).stop 2024-09-21T03:55:58.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.994+0000 7f0fa8929640 1 -- 172.21.15.64:0/3500555394 shutdown_connections 2024-09-21T03:55:58.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.994+0000 7f0fa8929640 1 --2- 172.21.15.64:0/3500555394 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0f900b64d0 0x7f0f900b89e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.994+0000 7f0fa8929640 1 --2- 172.21.15.64:0/3500555394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0fa4108d00 0x7f0fa419eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.994+0000 7f0fa8929640 1 --2- 172.21.15.64:0/3500555394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fa41083b0 0x7f0fa41998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.994+0000 7f0fa8929640 1 --2- 172.21.15.64:0/3500555394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fa4102a70 0x7f0fa4199350 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:58.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.994+0000 7f0fa8929640 1 -- 172.21.15.64:0/3500555394 >> 172.21.15.64:0/3500555394 conn(0x7f0fa4079710 msgr2=0x7f0fa4106440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:58.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.994+0000 7f0fa8929640 1 -- 172.21.15.64:0/3500555394 shutdown_connections 2024-09-21T03:55:58.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:58.994+0000 7f0fa8929640 1 -- 172.21.15.64:0/3500555394 wait complete. 2024-09-21T03:55:59.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.202+0000 7f6975c8b640 1 -- 172.21.15.64:0/2774819626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69701083d0 msgr2=0x7f69701087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:59.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.202+0000 7f6975c8b640 1 --2- 172.21.15.64:0/2774819626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69701083d0 0x7f69701087e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6964009900 tx=0x7f696402a410 comp rx=0 tx=0).stop 2024-09-21T03:55:59.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.202+0000 7f6975c8b640 1 -- 172.21.15.64:0/2774819626 shutdown_connections 2024-09-21T03:55:59.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.202+0000 7f6975c8b640 1 --2- 172.21.15.64:0/2774819626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6970108d20 0x7f697010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.202+0000 7f6975c8b640 1 --2- 172.21.15.64:0/2774819626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69701083d0 0x7f69701087e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.202+0000 7f6975c8b640 1 --2- 172.21.15.64:0/2774819626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6970102a90 0x7f6970102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.202+0000 7f6975c8b640 1 -- 172.21.15.64:0/2774819626 >> 172.21.15.64:0/2774819626 conn(0x7f6970079750 msgr2=0x7f6970079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:59.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.202+0000 7f6975c8b640 1 -- 172.21.15.64:0/2774819626 shutdown_connections 2024-09-21T03:55:59.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.202+0000 7f6975c8b640 1 -- 172.21.15.64:0/2774819626 wait complete. 2024-09-21T03:55:59.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.202+0000 7f6975c8b640 1 Processor -- start 2024-09-21T03:55:59.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.203+0000 7f6975c8b640 1 -- start start 2024-09-21T03:55:59.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.203+0000 7f6975c8b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6970102a90 0x7f697019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:59.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.203+0000 7f6975c8b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69701083d0 0x7f697019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:59.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.203+0000 7f6975c8b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6970108d20 0x7f697019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:59.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.203+0000 7f6975c8b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6970113ab0 con 0x7f69701083d0 2024-09-21T03:55:59.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.203+0000 7f6975c8b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6970113930 con 0x7f6970108d20 2024-09-21T03:55:59.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.203+0000 7f6975c8b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6970113c30 con 0x7f6970102a90 2024-09-21T03:55:59.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.203+0000 7f696effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69701083d0 0x7f697019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:59.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.203+0000 7f696effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69701083d0 0x7f697019aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42142/0 (socket says 172.21.15.64:42142) 2024-09-21T03:55:59.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.203+0000 7f696effd640 1 -- 172.21.15.64:0/531704805 learned_addr learned my addr 172.21.15.64:0/531704805 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:55:59.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.203+0000 7f696f7fe640 1 --2- 172.21.15.64:0/531704805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6970102a90 0x7f697019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:59.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.203+0000 7f696ffff640 1 --2- 172.21.15.64:0/531704805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6970108d20 0x7f697019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:59.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.203+0000 7f696effd640 1 -- 172.21.15.64:0/531704805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6970102a90 msgr2=0x7f697019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:59.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.203+0000 7f696effd640 1 --2- 172.21.15.64:0/531704805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6970102a90 0x7f697019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.203+0000 7f696effd640 1 -- 172.21.15.64:0/531704805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6970108d20 msgr2=0x7f697019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:59.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.203+0000 7f696effd640 1 --2- 172.21.15.64:0/531704805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6970108d20 0x7f697019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.203+0000 7f696effd640 1 -- 172.21.15.64:0/531704805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69701a03d0 con 0x7f69701083d0 2024-09-21T03:55:59.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.203+0000 7f696ffff640 1 --2- 172.21.15.64:0/531704805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6970108d20 0x7f697019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:55:59.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.204+0000 7f696effd640 1 --2- 172.21.15.64:0/531704805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69701083d0 0x7f697019aa00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6964032fd0 tx=0x7f6964004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:59.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.204+0000 7f696cff9640 1 -- 172.21.15.64:0/531704805 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696403f030 con 0x7f69701083d0 2024-09-21T03:55:59.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.204+0000 7f696cff9640 1 -- 172.21.15.64:0/531704805 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6964004490 con 0x7f69701083d0 2024-09-21T03:55:59.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.204+0000 7f6975c8b640 1 -- 172.21.15.64:0/531704805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69701a0660 con 0x7f69701083d0 2024-09-21T03:55:59.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.204+0000 7f696cff9640 1 -- 172.21.15.64:0/531704805 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6964035070 con 0x7f69701083d0 2024-09-21T03:55:59.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.204+0000 7f6975c8b640 1 -- 172.21.15.64:0/531704805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69701a6e20 con 0x7f69701083d0 2024-09-21T03:55:59.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.205+0000 7f6975c8b640 1 -- 172.21.15.64:0/531704805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6970103d10 con 0x7f69701083d0 2024-09-21T03:55:59.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.206+0000 7f696cff9640 1 -- 172.21.15.64:0/531704805 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f6964042030 con 0x7f69701083d0 2024-09-21T03:55:59.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.207+0000 7f696cff9640 1 --2- 172.21.15.64:0/531704805 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f69580b64d0 0x7f69580b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:55:59.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.207+0000 7f696f7fe640 1 --2- 172.21.15.64:0/531704805 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f69580b64d0 0x7f69580b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:55:59.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.207+0000 7f696cff9640 1 -- 172.21.15.64:0/531704805 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f6964106020 con 0x7f69701083d0 2024-09-21T03:55:59.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.208+0000 7f696f7fe640 1 --2- 172.21.15.64:0/531704805 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f69580b64d0 0x7f69580b89e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6960005d00 tx=0x7f696000b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:55:59.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.210+0000 7f696cff9640 1 -- 172.21.15.64:0/531704805 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f69640ce4b0 con 0x7f69701083d0 2024-09-21T03:55:59.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.402+0000 7f6975c8b640 1 -- 172.21.15.64:0/531704805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f6970102ea0 con 0x7f69701083d0 2024-09-21T03:55:59.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.406+0000 7f696cff9640 1 -- 172.21.15.64:0/531704805 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v36)=0 v36) ==== 121+0+0 (secure 0 0 0) 0x7f69640d2890 con 0x7f69701083d0 2024-09-21T03:55:59.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.410+0000 7f6975c8b640 1 -- 172.21.15.64:0/531704805 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f69580b64d0 msgr2=0x7f69580b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:59.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.410+0000 7f6975c8b640 1 --2- 172.21.15.64:0/531704805 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f69580b64d0 0x7f69580b89e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6960005d00 tx=0x7f696000b410 comp rx=0 tx=0).stop 2024-09-21T03:55:59.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.410+0000 7f6975c8b640 1 -- 172.21.15.64:0/531704805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69701083d0 msgr2=0x7f697019aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:55:59.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.410+0000 7f6975c8b640 1 --2- 172.21.15.64:0/531704805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69701083d0 0x7f697019aa00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6964032fd0 tx=0x7f6964004270 comp rx=0 tx=0).stop 2024-09-21T03:55:59.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.410+0000 7f6975c8b640 1 -- 172.21.15.64:0/531704805 shutdown_connections 2024-09-21T03:55:59.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.410+0000 7f6975c8b640 1 --2- 172.21.15.64:0/531704805 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f69580b64d0 0x7f69580b89e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.410+0000 7f6975c8b640 1 --2- 172.21.15.64:0/531704805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6970108d20 0x7f697019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.410+0000 7f6975c8b640 1 --2- 172.21.15.64:0/531704805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69701083d0 0x7f697019aa00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.410+0000 7f6975c8b640 1 --2- 172.21.15.64:0/531704805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6970102a90 0x7f697019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:55:59.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.410+0000 7f6975c8b640 1 -- 172.21.15.64:0/531704805 >> 172.21.15.64:0/531704805 conn(0x7f6970079750 msgr2=0x7f6970105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:55:59.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.410+0000 7f6975c8b640 1 -- 172.21.15.64:0/531704805 shutdown_connections 2024-09-21T03:55:59.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:55:59.410+0000 7f6975c8b640 1 -- 172.21.15.64:0/531704805 wait complete. 2024-09-21T03:56:00.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:59 smithi064 ceph-mon[31180]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:00.155 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:59 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3500555394' entity='client.admin' 2024-09-21T03:56:00.155 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:59 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:56:00.155 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:59 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:00.155 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:59 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:56:00.155 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:59 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:00.155 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:55:59 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/531704805' entity='client.admin' 2024-09-21T03:56:00.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:59 smithi119 ceph-mon[34811]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:00.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:59 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3500555394' entity='client.admin' 2024-09-21T03:56:00.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:59 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:56:00.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:59 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:00.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:59 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:56:00.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:59 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:00.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:55:59 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/531704805' entity='client.admin' 2024-09-21T03:56:00.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:59 smithi131 ceph-mon[35193]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:00.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:59 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3500555394' entity='client.admin' 2024-09-21T03:56:00.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:59 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:56:00.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:59 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:00.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:59 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:56:00.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:59 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:00.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:55:59 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/531704805' entity='client.admin' 2024-09-21T03:56:00.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.974+0000 7f3e5df46640 1 -- 172.21.15.64:0/2004074450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e581083d0 msgr2=0x7f3e581087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:00.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.974+0000 7f3e5df46640 1 --2- 172.21.15.64:0/2004074450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e581083d0 0x7f3e581087e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3e40009900 tx=0x7f3e4002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:56:00.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.974+0000 7f3e5df46640 1 -- 172.21.15.64:0/2004074450 shutdown_connections 2024-09-21T03:56:00.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.974+0000 7f3e5df46640 1 --2- 172.21.15.64:0/2004074450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e58108d20 0x7f3e5810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:00.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.974+0000 7f3e5df46640 1 --2- 172.21.15.64:0/2004074450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e581083d0 0x7f3e581087e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:00.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.974+0000 7f3e5df46640 1 --2- 172.21.15.64:0/2004074450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e58102ae0 0x7f3e58102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:00.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.974+0000 7f3e5df46640 1 -- 172.21.15.64:0/2004074450 >> 172.21.15.64:0/2004074450 conn(0x7f3e58079760 msgr2=0x7f3e58079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:00.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.974+0000 7f3e5df46640 1 -- 172.21.15.64:0/2004074450 shutdown_connections 2024-09-21T03:56:00.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.974+0000 7f3e5df46640 1 -- 172.21.15.64:0/2004074450 wait complete. 2024-09-21T03:56:00.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.975+0000 7f3e5df46640 1 Processor -- start 2024-09-21T03:56:00.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.975+0000 7f3e5df46640 1 -- start start 2024-09-21T03:56:00.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.975+0000 7f3e5df46640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e58102ae0 0x7f3e580ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:00.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.975+0000 7f3e5df46640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e581083d0 0x7f3e580ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:00.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.975+0000 7f3e5df46640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e58108d20 0x7f3e581000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:00.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.975+0000 7f3e5df46640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3e581129e0 con 0x7f3e58108d20 2024-09-21T03:56:00.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.975+0000 7f3e5df46640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3e58112860 con 0x7f3e58102ae0 2024-09-21T03:56:00.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.975+0000 7f3e5df46640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3e58112b60 con 0x7f3e581083d0 2024-09-21T03:56:00.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.976+0000 7f3e57fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e58108d20 0x7f3e581000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:00.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.976+0000 7f3e57fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e58108d20 0x7f3e581000f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42166/0 (socket says 172.21.15.64:42166) 2024-09-21T03:56:00.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.976+0000 7f3e57fff640 1 -- 172.21.15.64:0/4260335616 learned_addr learned my addr 172.21.15.64:0/4260335616 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:56:00.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.976+0000 7f3e577fe640 1 --2- 172.21.15.64:0/4260335616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e58102ae0 0x7f3e580ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:00.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.976+0000 7f3e4effd640 1 --2- 172.21.15.64:0/4260335616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e581083d0 0x7f3e580ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:00.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.976+0000 7f3e57fff640 1 -- 172.21.15.64:0/4260335616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e581083d0 msgr2=0x7f3e580ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:00.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.976+0000 7f3e57fff640 1 --2- 172.21.15.64:0/4260335616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e581083d0 0x7f3e580ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:00.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.976+0000 7f3e57fff640 1 -- 172.21.15.64:0/4260335616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e58102ae0 msgr2=0x7f3e580ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:00.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.976+0000 7f3e57fff640 1 --2- 172.21.15.64:0/4260335616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e58102ae0 0x7f3e580ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:00.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.976+0000 7f3e57fff640 1 -- 172.21.15.64:0/4260335616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e581a86b0 con 0x7f3e58108d20 2024-09-21T03:56:00.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.976+0000 7f3e57fff640 1 --2- 172.21.15.64:0/4260335616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e58108d20 0x7f3e581000f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000e970 tx=0x7f3e5000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:00.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.976+0000 7f3e557fa640 1 -- 172.21.15.64:0/4260335616 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5000ccc0 con 0x7f3e58108d20 2024-09-21T03:56:00.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.976+0000 7f3e557fa640 1 -- 172.21.15.64:0/4260335616 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e5000ce60 con 0x7f3e58108d20 2024-09-21T03:56:00.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.976+0000 7f3e557fa640 1 -- 172.21.15.64:0/4260335616 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e50010430 con 0x7f3e58108d20 2024-09-21T03:56:00.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.976+0000 7f3e5df46640 1 -- 172.21.15.64:0/4260335616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e581a89a0 con 0x7f3e58108d20 2024-09-21T03:56:00.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.976+0000 7f3e5df46640 1 -- 172.21.15.64:0/4260335616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e581a8e60 con 0x7f3e58108d20 2024-09-21T03:56:00.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.977+0000 7f3e5df46640 1 -- 172.21.15.64:0/4260335616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e58103d10 con 0x7f3e58108d20 2024-09-21T03:56:00.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.979+0000 7f3e557fa640 1 -- 172.21.15.64:0/4260335616 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f3e50004f70 con 0x7f3e58108d20 2024-09-21T03:56:00.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.980+0000 7f3e557fa640 1 --2- 172.21.15.64:0/4260335616 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3e440b6550 0x7f3e440b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:00.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.980+0000 7f3e577fe640 1 --2- 172.21.15.64:0/4260335616 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3e440b6550 0x7f3e440b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:00.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.980+0000 7f3e557fa640 1 -- 172.21.15.64:0/4260335616 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f3e500e5d10 con 0x7f3e58108d20 2024-09-21T03:56:00.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.981+0000 7f3e577fe640 1 --2- 172.21.15.64:0/4260335616 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3e440b6550 0x7f3e440b8a60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3e581010c0 tx=0x7f3e480075b0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:00.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:00.983+0000 7f3e557fa640 1 -- 172.21.15.64:0/4260335616 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3e50056710 con 0x7f3e58108d20 2024-09-21T03:56:01.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:01.172+0000 7f3e5df46640 1 -- 172.21.15.64:0/4260335616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=debug_cephsqlite}] v 0) -- 0x7f3e58100970 con 0x7f3e58108d20 2024-09-21T03:56:01.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:01.177+0000 7f3e557fa640 1 -- 172.21.15.64:0/4260335616 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_cephsqlite}]=0 v37)=0 v37) ==== 115+0+0 (secure 0 0 0) 0x7f3e500ae1a0 con 0x7f3e58108d20 2024-09-21T03:56:01.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:01.179+0000 7f3e5df46640 1 -- 172.21.15.64:0/4260335616 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3e440b6550 msgr2=0x7f3e440b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:01.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:01.179+0000 7f3e5df46640 1 --2- 172.21.15.64:0/4260335616 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3e440b6550 0x7f3e440b8a60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3e581010c0 tx=0x7f3e480075b0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:01.179+0000 7f3e5df46640 1 -- 172.21.15.64:0/4260335616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e58108d20 msgr2=0x7f3e581000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:01.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:01.179+0000 7f3e5df46640 1 --2- 172.21.15.64:0/4260335616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e58108d20 0x7f3e581000f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000e970 tx=0x7f3e5000ed70 comp rx=0 tx=0).stop 2024-09-21T03:56:01.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:01.179+0000 7f3e5df46640 1 -- 172.21.15.64:0/4260335616 shutdown_connections 2024-09-21T03:56:01.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:01.179+0000 7f3e5df46640 1 --2- 172.21.15.64:0/4260335616 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3e440b6550 0x7f3e440b8a60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:01.179+0000 7f3e5df46640 1 --2- 172.21.15.64:0/4260335616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3e58108d20 0x7f3e581000f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:01.179+0000 7f3e5df46640 1 --2- 172.21.15.64:0/4260335616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e581083d0 0x7f3e580ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:01.179+0000 7f3e5df46640 1 --2- 172.21.15.64:0/4260335616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e58102ae0 0x7f3e580ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:01.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:01.179+0000 7f3e5df46640 1 -- 172.21.15.64:0/4260335616 >> 172.21.15.64:0/4260335616 conn(0x7f3e58079760 msgr2=0x7f3e5806be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:01.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:01.179+0000 7f3e5df46640 1 -- 172.21.15.64:0/4260335616 shutdown_connections 2024-09-21T03:56:01.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:01.179+0000 7f3e5df46640 1 -- 172.21.15.64:0/4260335616 wait complete. 2024-09-21T03:56:01.962 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi064') 2024-09-21T03:56:01.962 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi064') 2024-09-21T03:56:01.962 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi064') 2024-09-21T03:56:01.962 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/allow_m_granularity', 'True'], check_status=True, hostname='smithi064') 2024-09-21T03:56:01.962 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/dump_on_update', 'True'], check_status=True, hostname='smithi064') 2024-09-21T03:56:01.962 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/log_level', 'debug'], check_status=True, hostname='smithi064') 2024-09-21T03:56:01.962 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi064') 2024-09-21T03:56:01.963 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi064') 2024-09-21T03:56:01.963 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'e7c927ba-77cc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi064') 2024-09-21T03:56:01.963 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config dump 2024-09-21T03:56:02.215 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:02.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:02 smithi064 ceph-mon[31180]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:02.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:02 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/4260335616' entity='client.admin' 2024-09-21T03:56:02.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:02 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:56:02.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:02 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:02.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:02 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:56:02.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:02 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:02.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:02 smithi119 ceph-mon[34811]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:02.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:02 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/4260335616' entity='client.admin' 2024-09-21T03:56:02.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:02 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:56:02.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:02 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:02.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:02 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:56:02.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:02 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:02.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:02 smithi131 ceph-mon[35193]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:02.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:02 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/4260335616' entity='client.admin' 2024-09-21T03:56:02.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:02 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:56:02.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:02 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:02.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:02 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:56:02.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:02 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:02.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.660+0000 7fbcbb83b640 1 -- 172.21.15.64:0/106860758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcb4103b90 msgr2=0x7fbcb4103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:02.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.660+0000 7fbcbb83b640 1 --2- 172.21.15.64:0/106860758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcb4103b90 0x7fbcb4103fa0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbcac009900 tx=0x7fbcac02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:56:02.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.660+0000 7fbcbb83b640 1 -- 172.21.15.64:0/106860758 shutdown_connections 2024-09-21T03:56:02.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.660+0000 7fbcbb83b640 1 --2- 172.21.15.64:0/106860758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcb4109e20 0x7fbcb4110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:02.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.660+0000 7fbcbb83b640 1 --2- 172.21.15.64:0/106860758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbcb41094d0 0x7fbcb41098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:02.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.660+0000 7fbcbb83b640 1 --2- 172.21.15.64:0/106860758 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcb4103b90 0x7fbcb4103fa0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:02.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.660+0000 7fbcbb83b640 1 -- 172.21.15.64:0/106860758 >> 172.21.15.64:0/106860758 conn(0x7fbcb40ff600 msgr2=0x7fbcb4101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:02.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.660+0000 7fbcbb83b640 1 -- 172.21.15.64:0/106860758 shutdown_connections 2024-09-21T03:56:02.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.660+0000 7fbcbb83b640 1 -- 172.21.15.64:0/106860758 wait complete. 2024-09-21T03:56:02.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.661+0000 7fbcbb83b640 1 Processor -- start 2024-09-21T03:56:02.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.661+0000 7fbcbb83b640 1 -- start start 2024-09-21T03:56:02.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.661+0000 7fbcbb83b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbcb4103b90 0x7fbcb419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:02.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.661+0000 7fbcbb83b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcb41094d0 0x7fbcb419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:02.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.661+0000 7fbcbb83b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcb4109e20 0x7fbcb419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:02.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.661+0000 7fbcbb83b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb4113ae0 con 0x7fbcb4109e20 2024-09-21T03:56:02.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.661+0000 7fbcbb83b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb4113960 con 0x7fbcb41094d0 2024-09-21T03:56:02.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.661+0000 7fbcbb83b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb4113c60 con 0x7fbcb4103b90 2024-09-21T03:56:02.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.662+0000 7fbcb9dd8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcb4109e20 0x7fbcb419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:02.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.662+0000 7fbcb95d7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbcb4103b90 0x7fbcb419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:02.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.662+0000 7fbcb9dd8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcb4109e20 0x7fbcb419fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53056/0 (socket says 172.21.15.64:53056) 2024-09-21T03:56:02.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.662+0000 7fbcb9dd8640 1 -- 172.21.15.64:0/2029293763 learned_addr learned my addr 172.21.15.64:0/2029293763 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:56:02.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.662+0000 7fbcb8dd6640 1 --2- 172.21.15.64:0/2029293763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcb41094d0 0x7fbcb419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:02.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.662+0000 7fbcb9dd8640 1 -- 172.21.15.64:0/2029293763 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbcb4103b90 msgr2=0x7fbcb419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:02.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.662+0000 7fbcb9dd8640 1 --2- 172.21.15.64:0/2029293763 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbcb4103b90 0x7fbcb419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:02.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.662+0000 7fbcb9dd8640 1 -- 172.21.15.64:0/2029293763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcb41094d0 msgr2=0x7fbcb419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:02.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.662+0000 7fbcb9dd8640 1 --2- 172.21.15.64:0/2029293763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcb41094d0 0x7fbcb419aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:02.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.662+0000 7fbcb9dd8640 1 -- 172.21.15.64:0/2029293763 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb41a0360 con 0x7fbcb4109e20 2024-09-21T03:56:02.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.662+0000 7fbcb9dd8640 1 --2- 172.21.15.64:0/2029293763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcb4109e20 0x7fbcb419fcf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbca400d950 tx=0x7fbca400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:02.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.662+0000 7fbcb27fc640 1 -- 172.21.15.64:0/2029293763 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca4017030 con 0x7fbcb4109e20 2024-09-21T03:56:02.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.662+0000 7fbcb27fc640 1 -- 172.21.15.64:0/2029293763 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbca4004020 con 0x7fbcb4109e20 2024-09-21T03:56:02.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.662+0000 7fbcb27fc640 1 -- 172.21.15.64:0/2029293763 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca4003b50 con 0x7fbcb4109e20 2024-09-21T03:56:02.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.662+0000 7fbcbb83b640 1 -- 172.21.15.64:0/2029293763 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb41a0650 con 0x7fbcb4109e20 2024-09-21T03:56:02.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.662+0000 7fbcbb83b640 1 -- 172.21.15.64:0/2029293763 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb41a0b40 con 0x7fbcb4109e20 2024-09-21T03:56:02.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.664+0000 7fbcbb83b640 1 -- 172.21.15.64:0/2029293763 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcb4103fa0 con 0x7fbcb4109e20 2024-09-21T03:56:02.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.665+0000 7fbcb27fc640 1 -- 172.21.15.64:0/2029293763 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fbca401a070 con 0x7fbcb4109e20 2024-09-21T03:56:02.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.666+0000 7fbcb27fc640 1 --2- 172.21.15.64:0/2029293763 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbc9c0b6580 0x7fbc9c0b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:02.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.666+0000 7fbcb95d7640 1 --2- 172.21.15.64:0/2029293763 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbc9c0b6580 0x7fbc9c0b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:02.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.666+0000 7fbcb27fc640 1 -- 172.21.15.64:0/2029293763 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fbca40e4de0 con 0x7fbcb4109e20 2024-09-21T03:56:02.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.666+0000 7fbcb95d7640 1 --2- 172.21.15.64:0/2029293763 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbc9c0b6580 0x7fbc9c0b8a90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbcac031440 tx=0x7fbcac009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:02.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.670+0000 7fbcb27fc640 1 -- 172.21.15.64:0/2029293763 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbcb41a0b40 con 0x7fbcb4109e20 2024-09-21T03:56:02.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.858+0000 7fbcbb83b640 1 -- 172.21.15.64:0/2029293763 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fbcb402e640 con 0x7fbcb4109e20 2024-09-21T03:56:02.862 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.859+0000 7fbcb27fc640 1 -- 172.21.15.64:0/2029293763 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v37) ==== 59+0+22127 (secure 0 0 0) 0x7fbca4021070 con 0x7fbcb4109e20 2024-09-21T03:56:02.863 INFO:teuthology.orchestra.run.smithi064.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T03:56:02.863 INFO:teuthology.orchestra.run.smithi064.stdout:global dev auth_debug true 2024-09-21T03:56:02.863 INFO:teuthology.orchestra.run.smithi064.stdout:global basic container_image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 * 2024-09-21T03:56:02.863 INFO:teuthology.orchestra.run.smithi064.stdout:global dev debug_asserts_on_shutdown true 2024-09-21T03:56:02.863 INFO:teuthology.orchestra.run.smithi064.stdout:global basic log_to_file true 2024-09-21T03:56:02.863 INFO:teuthology.orchestra.run.smithi064.stdout:global basic log_to_journald false 2024-09-21T03:56:02.863 INFO:teuthology.orchestra.run.smithi064.stdout:global basic log_to_stderr false 2024-09-21T03:56:02.863 INFO:teuthology.orchestra.run.smithi064.stdout:global advanced mon_allow_pool_delete true 2024-09-21T03:56:02.863 INFO:teuthology.orchestra.run.smithi064.stdout:global advanced mon_clock_drift_allowed 1.000000 2024-09-21T03:56:02.863 INFO:teuthology.orchestra.run.smithi064.stdout:global advanced mon_cluster_log_to_file true 2024-09-21T03:56:02.864 INFO:teuthology.orchestra.run.smithi064.stdout:global advanced mon_max_pg_per_osd 10000 2024-09-21T03:56:02.864 INFO:teuthology.orchestra.run.smithi064.stdout:global advanced mon_pg_warn_max_object_skew 0.000000 2024-09-21T03:56:02.864 INFO:teuthology.orchestra.run.smithi064.stdout:global advanced mon_warn_on_crush_straw_calc_version_zero false 2024-09-21T03:56:02.864 INFO:teuthology.orchestra.run.smithi064.stdout:global advanced mon_warn_on_legacy_crush_tunables false 2024-09-21T03:56:02.864 INFO:teuthology.orchestra.run.smithi064.stdout:global advanced mon_warn_on_osd_down_out_interval_zero false 2024-09-21T03:56:02.864 INFO:teuthology.orchestra.run.smithi064.stdout:global dev mon_warn_on_pool_pg_num_not_power_of_two false 2024-09-21T03:56:02.864 INFO:teuthology.orchestra.run.smithi064.stdout:global advanced mon_warn_on_too_few_osds false 2024-09-21T03:56:02.864 INFO:teuthology.orchestra.run.smithi064.stdout:global dev ms_die_on_bug true 2024-09-21T03:56:02.864 INFO:teuthology.orchestra.run.smithi064.stdout:global dev ms_die_on_old_message true 2024-09-21T03:56:02.864 INFO:teuthology.orchestra.run.smithi064.stdout:global dev ms_die_on_skipped_message false 2024-09-21T03:56:02.865 INFO:teuthology.orchestra.run.smithi064.stdout:global advanced osd_pool_default_erasure_code_profile plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T03:56:02.865 INFO:teuthology.orchestra.run.smithi064.stdout:global advanced osd_pool_default_pg_autoscale_mode off 2024-09-21T03:56:02.865 INFO:teuthology.orchestra.run.smithi064.stdout:mon advanced auth_allow_insecure_global_id_reclaim false 2024-09-21T03:56:02.865 INFO:teuthology.orchestra.run.smithi064.stdout:mon advanced auth_mon_ticket_ttl 660.000000 2024-09-21T03:56:02.865 INFO:teuthology.orchestra.run.smithi064.stdout:mon advanced auth_service_ticket_ttl 240.000000 2024-09-21T03:56:02.865 INFO:teuthology.orchestra.run.smithi064.stdout:mon advanced debug_mon 20/20 2024-09-21T03:56:02.865 INFO:teuthology.orchestra.run.smithi064.stdout:mon advanced debug_ms 1/1 2024-09-21T03:56:02.865 INFO:teuthology.orchestra.run.smithi064.stdout:mon advanced debug_paxos 20/20 2024-09-21T03:56:02.866 INFO:teuthology.orchestra.run.smithi064.stdout:mon advanced mon_data_avail_warn 5 2024-09-21T03:56:02.866 INFO:teuthology.orchestra.run.smithi064.stdout:mon advanced mon_down_mkfs_grace 300 2024-09-21T03:56:02.866 INFO:teuthology.orchestra.run.smithi064.stdout:mon advanced mon_mgr_mkfs_grace 240 2024-09-21T03:56:02.866 INFO:teuthology.orchestra.run.smithi064.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T03:56:02.866 INFO:teuthology.orchestra.run.smithi064.stdout:mon dev mon_osd_prime_pg_temp true 2024-09-21T03:56:02.866 INFO:teuthology.orchestra.run.smithi064.stdout:mon advanced mon_osd_reporter_subtree_level osd 2024-09-21T03:56:02.866 INFO:teuthology.orchestra.run.smithi064.stdout:mon advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T03:56:02.866 INFO:teuthology.orchestra.run.smithi064.stdout:mon advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T03:56:02.866 INFO:teuthology.orchestra.run.smithi064.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim false 2024-09-21T03:56:02.866 INFO:teuthology.orchestra.run.smithi064.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim_allowed false 2024-09-21T03:56:02.867 INFO:teuthology.orchestra.run.smithi064.stdout:mon advanced public_network 172.21.0.0/20 * 2024-09-21T03:56:02.867 INFO:teuthology.orchestra.run.smithi064.stdout:mgr advanced cephsqlite_lock_renewal_timeout 900000 2024-09-21T03:56:02.867 INFO:teuthology.orchestra.run.smithi064.stdout:mgr advanced client_mount_timeout 30 2024-09-21T03:56:02.867 INFO:teuthology.orchestra.run.smithi064.stdout:mgr advanced debug_cephsqlite 20/20 2024-09-21T03:56:02.867 INFO:teuthology.orchestra.run.smithi064.stdout:mgr advanced debug_client 20/20 2024-09-21T03:56:02.867 INFO:teuthology.orchestra.run.smithi064.stdout:mgr advanced debug_mgr 20/20 2024-09-21T03:56:02.867 INFO:teuthology.orchestra.run.smithi064.stdout:mgr advanced debug_ms 1/1 2024-09-21T03:56:02.867 INFO:teuthology.orchestra.run.smithi064.stdout:mgr advanced mgr/cephadm/allow_ptrace true * 2024-09-21T03:56:02.867 INFO:teuthology.orchestra.run.smithi064.stdout:mgr advanced mgr/cephadm/container_init True * 2024-09-21T03:56:02.867 INFO:teuthology.orchestra.run.smithi064.stdout:mgr advanced mgr/cephadm/migration_current 7 * 2024-09-21T03:56:02.867 INFO:teuthology.orchestra.run.smithi064.stdout:mgr advanced mgr/dashboard/ssl_server_port 8443 * 2024-09-21T03:56:02.867 INFO:teuthology.orchestra.run.smithi064.stdout:mgr advanced mgr/orchestrator/orchestrator cephadm 2024-09-21T03:56:02.867 INFO:teuthology.orchestra.run.smithi064.stdout:mgr advanced mgr/snap_schedule/allow_m_granularity true 2024-09-21T03:56:02.867 INFO:teuthology.orchestra.run.smithi064.stdout:mgr advanced mgr/snap_schedule/dump_on_update true 2024-09-21T03:56:02.868 INFO:teuthology.orchestra.run.smithi064.stdout:mgr advanced mgr/snap_schedule/log_level debug 2024-09-21T03:56:02.868 INFO:teuthology.orchestra.run.smithi064.stdout:mgr advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T03:56:02.868 INFO:teuthology.orchestra.run.smithi064.stdout:mgr advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T03:56:02.868 INFO:teuthology.orchestra.run.smithi064.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T03:56:02.868 INFO:teuthology.orchestra.run.smithi064.stdout:osd dev bdev_debug_aio true 2024-09-21T03:56:02.868 INFO:teuthology.orchestra.run.smithi064.stdout:osd dev bluestore_block_size 96636764160 * 2024-09-21T03:56:02.868 INFO:teuthology.orchestra.run.smithi064.stdout:osd advanced bluestore_compression_mode aggressive 2024-09-21T03:56:02.868 INFO:teuthology.orchestra.run.smithi064.stdout:osd dev bluestore_fsck_on_mount true 2024-09-21T03:56:02.868 INFO:teuthology.orchestra.run.smithi064.stdout:osd advanced debug_bluefs 20/20 2024-09-21T03:56:02.868 INFO:teuthology.orchestra.run.smithi064.stdout:osd advanced debug_bluestore 20/20 2024-09-21T03:56:02.868 INFO:teuthology.orchestra.run.smithi064.stdout:osd advanced debug_ms 1/1 2024-09-21T03:56:02.868 INFO:teuthology.orchestra.run.smithi064.stdout:osd advanced debug_osd 20/20 2024-09-21T03:56:02.868 INFO:teuthology.orchestra.run.smithi064.stdout:osd advanced debug_rocksdb 10/10 2024-09-21T03:56:02.868 INFO:teuthology.orchestra.run.smithi064.stdout:osd dev osd_debug_misdirected_ops true 2024-09-21T03:56:02.869 INFO:teuthology.orchestra.run.smithi064.stdout:osd dev osd_debug_op_order true 2024-09-21T03:56:02.869 INFO:teuthology.orchestra.run.smithi064.stdout:osd dev osd_debug_pg_log_writeout true 2024-09-21T03:56:02.869 INFO:teuthology.orchestra.run.smithi064.stdout:osd dev osd_debug_shutdown true 2024-09-21T03:56:02.869 INFO:teuthology.orchestra.run.smithi064.stdout:osd dev osd_debug_verify_cached_snaps true 2024-09-21T03:56:02.869 INFO:teuthology.orchestra.run.smithi064.stdout:osd dev osd_debug_verify_missing_on_start true 2024-09-21T03:56:02.869 INFO:teuthology.orchestra.run.smithi064.stdout:osd dev osd_debug_verify_stray_on_activate true 2024-09-21T03:56:02.869 INFO:teuthology.orchestra.run.smithi064.stdout:osd advanced osd_deep_scrub_update_digest_min_age 30 2024-09-21T03:56:02.869 INFO:teuthology.orchestra.run.smithi064.stdout:osd advanced osd_failsafe_full_ratio 0.950000 2024-09-21T03:56:02.869 INFO:teuthology.orchestra.run.smithi064.stdout:osd advanced osd_max_omap_entries_per_request 10 2024-09-21T03:56:02.869 INFO:teuthology.orchestra.run.smithi064.stdout:osd advanced osd_mclock_profile high_recovery_ops 2024-09-21T03:56:02.869 INFO:teuthology.orchestra.run.smithi064.stdout:osd advanced osd_memory_target_autotune true 2024-09-21T03:56:02.869 INFO:teuthology.orchestra.run.smithi064.stdout:osd advanced osd_objectstore bluestore * 2024-09-21T03:56:02.869 INFO:teuthology.orchestra.run.smithi064.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T03:56:02.870 INFO:teuthology.orchestra.run.smithi064.stdout:osd advanced osd_op_queue debug_random * 2024-09-21T03:56:02.870 INFO:teuthology.orchestra.run.smithi064.stdout:osd advanced osd_op_queue_cut_off debug_random * 2024-09-21T03:56:02.870 INFO:teuthology.orchestra.run.smithi064.stdout:osd advanced osd_recover_clone_overlap true 2024-09-21T03:56:02.870 INFO:teuthology.orchestra.run.smithi064.stdout:osd advanced osd_recovery_max_chunk 1048576 2024-09-21T03:56:02.870 INFO:teuthology.orchestra.run.smithi064.stdout:osd advanced osd_scrub_load_threshold 5.000000 2024-09-21T03:56:02.870 INFO:teuthology.orchestra.run.smithi064.stdout:osd advanced osd_scrub_max_interval 600.000000 2024-09-21T03:56:02.870 INFO:teuthology.orchestra.run.smithi064.stdout:osd advanced osd_shutdown_pgref_assert true 2024-09-21T03:56:02.870 INFO:teuthology.orchestra.run.smithi064.stdout:mds advanced debug_mds 20/20 2024-09-21T03:56:02.870 INFO:teuthology.orchestra.run.smithi064.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T03:56:02.870 INFO:teuthology.orchestra.run.smithi064.stdout:mds advanced debug_ms 2/2 2024-09-21T03:56:02.870 INFO:teuthology.orchestra.run.smithi064.stdout:mds advanced mds_bal_fragment_size_max 10000 2024-09-21T03:56:02.870 INFO:teuthology.orchestra.run.smithi064.stdout:mds advanced mds_bal_merge_size 5 2024-09-21T03:56:02.870 INFO:teuthology.orchestra.run.smithi064.stdout:mds advanced mds_bal_split_bits 3 2024-09-21T03:56:02.871 INFO:teuthology.orchestra.run.smithi064.stdout:mds advanced mds_bal_split_size 100 2024-09-21T03:56:02.871 INFO:teuthology.orchestra.run.smithi064.stdout:mds dev mds_debug_frag true 2024-09-21T03:56:02.871 INFO:teuthology.orchestra.run.smithi064.stdout:mds dev mds_debug_scatterstat true 2024-09-21T03:56:02.871 INFO:teuthology.orchestra.run.smithi064.stdout:mds advanced mds_export_ephemeral_random_max 0.100000 2024-09-21T03:56:02.871 INFO:teuthology.orchestra.run.smithi064.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T03:56:02.871 INFO:teuthology.orchestra.run.smithi064.stdout:mds dev mds_verify_scatter true 2024-09-21T03:56:02.871 INFO:teuthology.orchestra.run.smithi064.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T03:56:02.871 INFO:teuthology.orchestra.run.smithi064.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T03:56:02.871 INFO:teuthology.orchestra.run.smithi064.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T03:56:02.871 INFO:teuthology.orchestra.run.smithi064.stdout:client advanced client_mount_timeout 600 2024-09-21T03:56:02.871 INFO:teuthology.orchestra.run.smithi064.stdout:client advanced debug_client 20/20 2024-09-21T03:56:02.871 INFO:teuthology.orchestra.run.smithi064.stdout:client advanced debug_ms 1/1 2024-09-21T03:56:02.871 INFO:teuthology.orchestra.run.smithi064.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T03:56:02.871 INFO:teuthology.orchestra.run.smithi064.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T03:56:02.871 INFO:teuthology.orchestra.run.smithi064.stdout:client.rgw advanced rgw_cache_enabled true 2024-09-21T03:56:02.872 INFO:teuthology.orchestra.run.smithi064.stdout:client.rgw advanced rgw_enable_ops_log true 2024-09-21T03:56:02.872 INFO:teuthology.orchestra.run.smithi064.stdout:client.rgw advanced rgw_enable_usage_log true 2024-09-21T03:56:02.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.861+0000 7fbcbb83b640 1 -- 172.21.15.64:0/2029293763 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbc9c0b6580 msgr2=0x7fbc9c0b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:02.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.861+0000 7fbcbb83b640 1 --2- 172.21.15.64:0/2029293763 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbc9c0b6580 0x7fbc9c0b8a90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbcac031440 tx=0x7fbcac009560 comp rx=0 tx=0).stop 2024-09-21T03:56:02.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.861+0000 7fbcbb83b640 1 -- 172.21.15.64:0/2029293763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcb4109e20 msgr2=0x7fbcb419fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:02.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.861+0000 7fbcbb83b640 1 --2- 172.21.15.64:0/2029293763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcb4109e20 0x7fbcb419fcf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbca400d950 tx=0x7fbca400de10 comp rx=0 tx=0).stop 2024-09-21T03:56:02.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.861+0000 7fbcbb83b640 1 -- 172.21.15.64:0/2029293763 shutdown_connections 2024-09-21T03:56:02.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.861+0000 7fbcbb83b640 1 --2- 172.21.15.64:0/2029293763 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbc9c0b6580 0x7fbc9c0b8a90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:02.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.861+0000 7fbcbb83b640 1 --2- 172.21.15.64:0/2029293763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcb4109e20 0x7fbcb419fcf0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:02.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.861+0000 7fbcbb83b640 1 --2- 172.21.15.64:0/2029293763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcb41094d0 0x7fbcb419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:02.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.861+0000 7fbcbb83b640 1 --2- 172.21.15.64:0/2029293763 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbcb4103b90 0x7fbcb419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:02.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.861+0000 7fbcbb83b640 1 -- 172.21.15.64:0/2029293763 >> 172.21.15.64:0/2029293763 conn(0x7fbcb40ff600 msgr2=0x7fbcb4101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:02.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.861+0000 7fbcbb83b640 1 -- 172.21.15.64:0/2029293763 shutdown_connections 2024-09-21T03:56:02.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:02.861+0000 7fbcbb83b640 1 -- 172.21.15.64:0/2029293763 wait complete. 2024-09-21T03:56:03.412 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:03 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2029293763' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T03:56:03.413 INFO:tasks.cephadm:Deploying OSDs... 2024-09-21T03:56:03.413 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-21T03:56:03.414 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T03:56:03.431 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T03:56:03.431 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_1 2024-09-21T03:56:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T03:56:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:56:03.488 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-21T03:56:03.488 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:56:03.488 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:56:03.488 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-09-21 03:54:38.884539750 +0000 2024-09-21T03:56:03.488 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-09-21 03:49:56.508190951 +0000 2024-09-21T03:56:03.488 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-09-21 03:49:56.508190951 +0000 2024-09-21T03:56:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-09-21 03:49:56.508190951 +0000 2024-09-21T03:56:03.489 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T03:56:03.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:03 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2029293763' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T03:56:03.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:03 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2029293763' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T03:56:03.554 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-09-21T03:56:03.554 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-09-21T03:56:03.554 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.00022387 s, 2.3 MB/s 2024-09-21T03:56:03.555 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T03:56:03.612 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_2 2024-09-21T03:56:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T03:56:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:56:03.668 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-21T03:56:03.668 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:56:03.669 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:56:03.669 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-09-21 03:54:38.921539025 +0000 2024-09-21T03:56:03.669 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-09-21 03:49:56.837184607 +0000 2024-09-21T03:56:03.669 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-09-21 03:49:56.837184607 +0000 2024-09-21T03:56:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-09-21 03:49:56.837184607 +0000 2024-09-21T03:56:03.669 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T03:56:03.734 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-09-21T03:56:03.734 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-09-21T03:56:03.734 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000179516 s, 2.9 MB/s 2024-09-21T03:56:03.735 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T03:56:03.792 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_3 2024-09-21T03:56:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T03:56:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:56:03.848 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-21T03:56:03.848 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:56:03.848 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:56:03.848 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-09-21 03:54:38.954538379 +0000 2024-09-21T03:56:03.848 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-09-21 03:49:57.164178302 +0000 2024-09-21T03:56:03.848 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-09-21 03:49:57.164178302 +0000 2024-09-21T03:56:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-09-21 03:49:57.164178302 +0000 2024-09-21T03:56:03.848 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T03:56:03.912 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-09-21T03:56:03.912 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-09-21T03:56:03.912 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000197314 s, 2.6 MB/s 2024-09-21T03:56:03.913 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T03:56:03.970 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_4 2024-09-21T03:56:04.027 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T03:56:04.027 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:56:04.027 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-21T03:56:04.027 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:56:04.027 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:56:04.028 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-09-21 03:54:38.982537830 +0000 2024-09-21T03:56:04.028 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-09-21 03:49:57.508171669 +0000 2024-09-21T03:56:04.028 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-09-21 03:49:57.508171669 +0000 2024-09-21T03:56:04.028 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-09-21 03:49:57.508171669 +0000 2024-09-21T03:56:04.028 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T03:56:04.091 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-09-21T03:56:04.092 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-09-21T03:56:04.092 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000187848 s, 2.7 MB/s 2024-09-21T03:56:04.093 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T03:56:04.150 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-09-21T03:56:04.150 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T03:56:04.169 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T03:56:04.170 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_1 2024-09-21T03:56:04.229 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T03:56:04.229 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:56:04.229 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-21T03:56:04.230 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:56:04.230 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:56:04.230 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-09-21 03:55:36.925555784 +0000 2024-09-21T03:56:04.230 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-09-21 03:50:03.338127923 +0000 2024-09-21T03:56:04.230 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-09-21 03:50:03.338127923 +0000 2024-09-21T03:56:04.230 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-09-21 03:50:03.338127923 +0000 2024-09-21T03:56:04.230 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T03:56:04.300 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-09-21T03:56:04.300 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-09-21T03:56:04.300 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000294912 s, 1.7 MB/s 2024-09-21T03:56:04.302 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T03:56:04.362 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_2 2024-09-21T03:56:04.420 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T03:56:04.420 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:56:04.421 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-21T03:56:04.421 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:56:04.421 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:56:04.421 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-09-21 03:55:36.926555759 +0000 2024-09-21T03:56:04.421 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-09-21 03:50:03.661119750 +0000 2024-09-21T03:56:04.421 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-09-21 03:50:03.661119750 +0000 2024-09-21T03:56:04.421 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-09-21 03:50:03.661119750 +0000 2024-09-21T03:56:04.421 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T03:56:04.485 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:04 smithi119 ceph-mon[34811]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:04.487 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-09-21T03:56:04.487 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-09-21T03:56:04.487 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000231771 s, 2.2 MB/s 2024-09-21T03:56:04.489 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T03:56:04.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:04 smithi064 ceph-mon[31180]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:04.547 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_3 2024-09-21T03:56:04.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:04 smithi131 ceph-mon[35193]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:04.604 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T03:56:04.605 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:56:04.605 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-21T03:56:04.605 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:56:04.605 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:56:04.605 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-09-21 03:55:36.926555759 +0000 2024-09-21T03:56:04.605 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-09-21 03:50:03.980111678 +0000 2024-09-21T03:56:04.605 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-09-21 03:50:03.980111678 +0000 2024-09-21T03:56:04.605 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-09-21 03:50:03.980111678 +0000 2024-09-21T03:56:04.605 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T03:56:04.670 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-09-21T03:56:04.670 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-09-21T03:56:04.670 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000199665 s, 2.6 MB/s 2024-09-21T03:56:04.672 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T03:56:04.729 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_4 2024-09-21T03:56:04.788 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T03:56:04.788 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:56:04.788 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-21T03:56:04.788 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:56:04.788 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:56:04.788 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-09-21 03:55:36.926555759 +0000 2024-09-21T03:56:04.788 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-09-21 03:50:04.305103454 +0000 2024-09-21T03:56:04.788 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-09-21 03:50:04.305103454 +0000 2024-09-21T03:56:04.788 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-09-21 03:50:04.305103454 +0000 2024-09-21T03:56:04.789 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T03:56:04.854 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-09-21T03:56:04.854 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-09-21T03:56:04.854 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000298628 s, 1.7 MB/s 2024-09-21T03:56:04.856 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T03:56:04.913 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-21T03:56:04.913 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T03:56:04.933 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T03:56:04.933 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_1 2024-09-21T03:56:04.991 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T03:56:04.991 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:56:04.991 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-21T03:56:04.991 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:56:04.991 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:56:04.991 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-09-21 03:55:34.790300778 +0000 2024-09-21T03:56:04.991 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-09-21 03:50:11.178732445 +0000 2024-09-21T03:56:04.992 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-09-21 03:50:11.178732445 +0000 2024-09-21T03:56:04.992 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-09-21 03:50:11.178732445 +0000 2024-09-21T03:56:04.992 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T03:56:05.060 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-09-21T03:56:05.060 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-09-21T03:56:05.060 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000288996 s, 1.8 MB/s 2024-09-21T03:56:05.061 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T03:56:05.119 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_2 2024-09-21T03:56:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T03:56:05.176 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:56:05.176 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-21T03:56:05.176 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:56:05.177 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:56:05.177 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-09-21 03:55:34.791300756 +0000 2024-09-21T03:56:05.177 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-09-21 03:50:11.505724948 +0000 2024-09-21T03:56:05.177 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-09-21 03:50:11.505724948 +0000 2024-09-21T03:56:05.177 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-09-21 03:50:11.505724948 +0000 2024-09-21T03:56:05.177 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T03:56:05.241 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-09-21T03:56:05.241 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-09-21T03:56:05.241 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000199389 s, 2.6 MB/s 2024-09-21T03:56:05.243 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T03:56:05.300 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_3 2024-09-21T03:56:05.356 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T03:56:05.356 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:56:05.356 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-21T03:56:05.356 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:56:05.356 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:56:05.356 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-09-21 03:55:34.791300756 +0000 2024-09-21T03:56:05.356 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-09-21 03:50:11.821717704 +0000 2024-09-21T03:56:05.356 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-09-21 03:50:11.821717704 +0000 2024-09-21T03:56:05.356 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-09-21 03:50:11.821717704 +0000 2024-09-21T03:56:05.357 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T03:56:05.420 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-09-21T03:56:05.420 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-09-21T03:56:05.420 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.00016744 s, 3.1 MB/s 2024-09-21T03:56:05.421 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T03:56:05.477 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_4 2024-09-21T03:56:05.536 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T03:56:05.536 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T03:56:05.536 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-21T03:56:05.536 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:56:05.536 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T03:56:05.536 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-09-21 03:55:34.791300756 +0000 2024-09-21T03:56:05.536 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-09-21 03:50:12.147710230 +0000 2024-09-21T03:56:05.536 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-09-21 03:50:12.147710230 +0000 2024-09-21T03:56:05.537 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-09-21 03:50:12.147710230 +0000 2024-09-21T03:56:05.537 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T03:56:05.620 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-09-21T03:56:05.620 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-09-21T03:56:05.620 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000219629 s, 2.3 MB/s 2024-09-21T03:56:05.621 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T03:56:05.691 INFO:tasks.cephadm:Deploying osd.0 on smithi064 with /dev/vg_nvme/lv_4... 2024-09-21T03:56:05.692 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T03:56:05.963 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:06.394 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:06 smithi064 ceph-mon[31180]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:06.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:06 smithi119 ceph-mon[34811]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:06.555 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:06 smithi131 ceph-mon[35193]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:08.298 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:56:08.323 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi064:vg_nvme/lv_4 2024-09-21T03:56:08.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:08 smithi119 ceph-mon[34811]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:08.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:08 smithi064 ceph-mon[31180]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:08.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:08 smithi131 ceph-mon[35193]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:08.575 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:56:08.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.990+0000 7f551ff14640 1 -- 172.21.15.64:0/487397371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5518102a80 msgr2=0x7f5518102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:08.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.990+0000 7f551ff14640 1 --2- 172.21.15.64:0/487397371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5518102a80 0x7f5518102e90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5514009900 tx=0x7f551402a410 comp rx=0 tx=0).stop 2024-09-21T03:56:08.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.990+0000 7f551ff14640 1 -- 172.21.15.64:0/487397371 shutdown_connections 2024-09-21T03:56:08.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.990+0000 7f551ff14640 1 --2- 172.21.15.64:0/487397371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5518108d10 0x7f551810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:08.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.990+0000 7f551ff14640 1 --2- 172.21.15.64:0/487397371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55181083c0 0x7f55181087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:08.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.990+0000 7f551ff14640 1 --2- 172.21.15.64:0/487397371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5518102a80 0x7f5518102e90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:08.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.990+0000 7f551ff14640 1 -- 172.21.15.64:0/487397371 >> 172.21.15.64:0/487397371 conn(0x7f5518079740 msgr2=0x7f5518079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:08.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.990+0000 7f551ff14640 1 -- 172.21.15.64:0/487397371 shutdown_connections 2024-09-21T03:56:08.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.990+0000 7f551ff14640 1 -- 172.21.15.64:0/487397371 wait complete. 2024-09-21T03:56:08.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.991+0000 7f551ff14640 1 Processor -- start 2024-09-21T03:56:08.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.991+0000 7f551ff14640 1 -- start start 2024-09-21T03:56:08.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.991+0000 7f551ff14640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5518102a80 0x7f551819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:08.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.991+0000 7f551ff14640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55181083c0 0x7f551819b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:08.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.991+0000 7f551ff14640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5518108d10 0x7f551819bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:08.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.991+0000 7f551ff14640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f55181129e0 con 0x7f5518108d10 2024-09-21T03:56:08.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.991+0000 7f551ff14640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5518112860 con 0x7f5518102a80 2024-09-21T03:56:08.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.991+0000 7f551ff14640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5518112b60 con 0x7f55181083c0 2024-09-21T03:56:08.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.991+0000 7f551e4b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5518108d10 0x7f551819bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:08.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.991+0000 7f551e4b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5518108d10 0x7f551819bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53076/0 (socket says 172.21.15.64:53076) 2024-09-21T03:56:09.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.991+0000 7f551e4b1640 1 -- 172.21.15.64:0/2089485779 learned_addr learned my addr 172.21.15.64:0/2089485779 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:56:09.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.991+0000 7f551dcb0640 1 --2- 172.21.15.64:0/2089485779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5518102a80 0x7f551819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:09.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.991+0000 7f551d4af640 1 --2- 172.21.15.64:0/2089485779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55181083c0 0x7f551819b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:09.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.991+0000 7f551e4b1640 1 -- 172.21.15.64:0/2089485779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55181083c0 msgr2=0x7f551819b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:09.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.991+0000 7f551e4b1640 1 --2- 172.21.15.64:0/2089485779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55181083c0 0x7f551819b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.992+0000 7f551e4b1640 1 -- 172.21.15.64:0/2089485779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5518102a80 msgr2=0x7f551819b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:09.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.992+0000 7f551e4b1640 1 --2- 172.21.15.64:0/2089485779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5518102a80 0x7f551819b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:09.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.992+0000 7f551e4b1640 1 -- 172.21.15.64:0/2089485779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55181a65e0 con 0x7f5518108d10 2024-09-21T03:56:09.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.992+0000 7f551d4af640 1 --2- 172.21.15.64:0/2089485779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55181083c0 0x7f551819b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:56:09.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.992+0000 7f551e4b1640 1 --2- 172.21.15.64:0/2089485779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5518108d10 0x7f551819bdd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5510009fd0 tx=0x7f551000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:09.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.992+0000 7f550effd640 1 -- 172.21.15.64:0/2089485779 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5510017070 con 0x7f5518108d10 2024-09-21T03:56:09.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.992+0000 7f550effd640 1 -- 172.21.15.64:0/2089485779 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5510003e20 con 0x7f5518108d10 2024-09-21T03:56:09.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.992+0000 7f550effd640 1 -- 172.21.15.64:0/2089485779 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5510010460 con 0x7f5518108d10 2024-09-21T03:56:09.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.992+0000 7f551ff14640 1 -- 172.21.15.64:0/2089485779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55181a68d0 con 0x7f5518108d10 2024-09-21T03:56:09.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.992+0000 7f551ff14640 1 -- 172.21.15.64:0/2089485779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55181a6dc0 con 0x7f5518108d10 2024-09-21T03:56:09.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.993+0000 7f551ff14640 1 -- 172.21.15.64:0/2089485779 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5518103d00 con 0x7f5518108d10 2024-09-21T03:56:09.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.995+0000 7f550effd640 1 -- 172.21.15.64:0/2089485779 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f5510003970 con 0x7f5518108d10 2024-09-21T03:56:09.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.995+0000 7f550effd640 1 --2- 172.21.15.64:0/2089485779 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f55040b6500 0x7f55040b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:09.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.995+0000 7f551dcb0640 1 --2- 172.21.15.64:0/2089485779 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f55040b6500 0x7f55040b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:09.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.996+0000 7f550effd640 1 -- 172.21.15.64:0/2089485779 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f55100e5480 con 0x7f5518108d10 2024-09-21T03:56:09.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:08.996+0000 7f551dcb0640 1 --2- 172.21.15.64:0/2089485779 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f55040b6500 0x7f55040b8a10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5514000c00 tx=0x7f5514031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:09.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:09.000+0000 7f550effd640 1 -- 172.21.15.64:0/2089485779 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f55100ad910 con 0x7f5518108d10 2024-09-21T03:56:09.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:09.203+0000 7f551ff14640 1 -- 172.21.15.64:0/2089485779 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5518063800 con 0x7f55040b6500 2024-09-21T03:56:10.446 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:10 smithi064 ceph-mon[31180]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:10.446 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:10 smithi064 ceph-mon[31180]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:56:10.446 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:10 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:56:10.446 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:10 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:56:10.446 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:10 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:10.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:10 smithi119 ceph-mon[34811]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:10.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:10 smithi119 ceph-mon[34811]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:56:10.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:10 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:56:10.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:10 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:56:10.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:10 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:10.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:10 smithi131 ceph-mon[35193]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:10.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:10 smithi131 ceph-mon[35193]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:56:10.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:10 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:56:10.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:10 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:56:10.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:10 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:12.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:12 smithi119 ceph-mon[34811]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:12.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:12 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1280808772' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fb947c10-98e6-4e3a-8bbd-8753951d870b"} : dispatch 2024-09-21T03:56:12.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:12 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1280808772' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb947c10-98e6-4e3a-8bbd-8753951d870b"}]': finished 2024-09-21T03:56:12.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:12 smithi119 ceph-mon[34811]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T03:56:12.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:12 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:56:12.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:12 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/174671259' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:56:12.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:12 smithi064 ceph-mon[31180]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:12.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:12 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1280808772' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fb947c10-98e6-4e3a-8bbd-8753951d870b"} : dispatch 2024-09-21T03:56:12.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:12 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1280808772' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb947c10-98e6-4e3a-8bbd-8753951d870b"}]': finished 2024-09-21T03:56:12.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:12 smithi064 ceph-mon[31180]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T03:56:12.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:12 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:56:12.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:12 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/174671259' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:56:12.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:12 smithi131 ceph-mon[35193]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:12.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:12 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1280808772' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fb947c10-98e6-4e3a-8bbd-8753951d870b"} : dispatch 2024-09-21T03:56:12.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:12 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1280808772' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb947c10-98e6-4e3a-8bbd-8753951d870b"}]': finished 2024-09-21T03:56:12.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:12 smithi131 ceph-mon[35193]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T03:56:12.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:12 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:56:12.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:12 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/174671259' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:56:14.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:14 smithi119 ceph-mon[34811]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:14.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:14 smithi064 ceph-mon[31180]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:14.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:14 smithi131 ceph-mon[35193]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:16.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:16 smithi119 ceph-mon[34811]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:16.509 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:16 smithi064 ceph-mon[31180]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:16.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:16 smithi131 ceph-mon[35193]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:18.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:18 smithi119 ceph-mon[34811]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:18.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:18 smithi064 ceph-mon[31180]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:18.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:18 smithi131 ceph-mon[35193]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:20.502 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:20 smithi064 ceph-mon[31180]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:20.502 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:20 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T03:56:20.502 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:20 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:20.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:20 smithi119 ceph-mon[34811]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:20.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:20 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T03:56:20.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:20 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:20.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:20 smithi131 ceph-mon[35193]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:20.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:20 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T03:56:20.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:20 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:21.415 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:21 smithi064 ceph-mon[31180]: Deploying daemon osd.0 on smithi064 2024-09-21T03:56:21.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:21 smithi119 ceph-mon[34811]: Deploying daemon osd.0 on smithi064 2024-09-21T03:56:21.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:21 smithi131 ceph-mon[35193]: Deploying daemon osd.0 on smithi064 2024-09-21T03:56:22.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:22 smithi119 ceph-mon[34811]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:22.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:22 smithi064 ceph-mon[31180]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:22.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:22 smithi131 ceph-mon[35193]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:24.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:24 smithi119 ceph-mon[34811]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:24.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:24 smithi064 ceph-mon[31180]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:24.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:24 smithi131 ceph-mon[35193]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:26.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:26 smithi119 ceph-mon[34811]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:26.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:26 smithi064 ceph-mon[31180]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:26.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:26 smithi131 ceph-mon[35193]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:27.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:27 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:56:27.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:27 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:27.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:27 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:27.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:27 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:56:27.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:27 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:27.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:27 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:27.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:27 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:56:27.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:27 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:27.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:27 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:28.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:28 smithi064 ceph-mon[31180]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:28.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:28 smithi119 ceph-mon[34811]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:28.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:28 smithi131 ceph-mon[35193]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:30.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:29 smithi119 ceph-mon[34811]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:30.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:30.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:30.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:30.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:56:30.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:30.007 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:29 smithi064 ceph-mon[31180]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:30.007 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:30.008 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:30.008 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:30.008 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:56:30.008 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:30.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:29 smithi131 ceph-mon[35193]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:30.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:30.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:30.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:30.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:56:30.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:30.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:30.611+0000 7f550effd640 1 -- 172.21.15.64:0/2089485779 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5518063800 con 0x7f55040b6500 2024-09-21T03:56:30.613 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 0 on host 'smithi064' 2024-09-21T03:56:30.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:30.613+0000 7f551ff14640 1 -- 172.21.15.64:0/2089485779 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f55040b6500 msgr2=0x7f55040b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:30.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:30.613+0000 7f551ff14640 1 --2- 172.21.15.64:0/2089485779 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f55040b6500 0x7f55040b8a10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5514000c00 tx=0x7f5514031400 comp rx=0 tx=0).stop 2024-09-21T03:56:30.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:30.613+0000 7f551ff14640 1 -- 172.21.15.64:0/2089485779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5518108d10 msgr2=0x7f551819bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:30.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:30.613+0000 7f551ff14640 1 --2- 172.21.15.64:0/2089485779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5518108d10 0x7f551819bdd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5510009fd0 tx=0x7f551000b260 comp rx=0 tx=0).stop 2024-09-21T03:56:30.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:30.613+0000 7f551ff14640 1 -- 172.21.15.64:0/2089485779 shutdown_connections 2024-09-21T03:56:30.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:30.613+0000 7f551ff14640 1 --2- 172.21.15.64:0/2089485779 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f55040b6500 0x7f55040b8a10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:30.613+0000 7f551ff14640 1 --2- 172.21.15.64:0/2089485779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5518108d10 0x7f551819bdd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:30.613+0000 7f551ff14640 1 --2- 172.21.15.64:0/2089485779 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55181083c0 0x7f551819b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:30.613+0000 7f551ff14640 1 --2- 172.21.15.64:0/2089485779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5518102a80 0x7f551819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:30.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:30.613+0000 7f551ff14640 1 -- 172.21.15.64:0/2089485779 >> 172.21.15.64:0/2089485779 conn(0x7f5518079740 msgr2=0x7f55181014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:30.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:30.614+0000 7f551ff14640 1 -- 172.21.15.64:0/2089485779 shutdown_connections 2024-09-21T03:56:30.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:56:30.614+0000 7f551ff14640 1 -- 172.21.15.64:0/2089485779 wait complete. 2024-09-21T03:56:30.910 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:30 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:56:30.910 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:30 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:30.910 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:30 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:31.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:30 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:56:31.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:30 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:31.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:30 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:31.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:30 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:56:31.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:30 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:31.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:30 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:31.348 DEBUG:teuthology.orchestra.run.smithi064:osd.0> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.0.service 2024-09-21T03:56:31.351 INFO:tasks.cephadm:Deploying osd.1 on smithi119 with /dev/vg_nvme/lv_4... 2024-09-21T03:56:31.351 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T03:56:31.590 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:56:31.989 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:31 smithi119 ceph-mon[34811]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:32.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:31 smithi064 ceph-mon[31180]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:32.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:31 smithi131 ceph-mon[35193]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:32.450 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 21 03:56:32 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-0[45513]: 2024-09-21T03:56:32.160+0000 7fb1ffeec640 -1 osd.0 0 log_to_monitors true 2024-09-21T03:56:32.970 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:32 smithi119 ceph-mon[34811]: from='osd.0 [v2:172.21.15.64:6802/3692023035,v1:172.21.15.64:6803/3692023035]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T03:56:33.003 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:32 smithi064 ceph-mon[31180]: from='osd.0 [v2:172.21.15.64:6802/3692023035,v1:172.21.15.64:6803/3692023035]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T03:56:33.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:32 smithi131 ceph-mon[35193]: from='osd.0 [v2:172.21.15.64:6802/3692023035,v1:172.21.15.64:6803/3692023035]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T03:56:33.848 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 21 03:56:33 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-0[45513]: 2024-09-21T03:56:33.736+0000 7fb1fbe6e640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T03:56:33.849 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 21 03:56:33 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-0[45513]: 2024-09-21T03:56:33.742+0000 7fb1f7c6c640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:56:33.849 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:33 smithi064 ceph-mon[31180]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:33.849 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:33 smithi064 ceph-mon[31180]: from='osd.0 [v2:172.21.15.64:6802/3692023035,v1:172.21.15.64:6803/3692023035]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T03:56:33.849 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:33 smithi064 ceph-mon[31180]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T03:56:33.849 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:33 smithi064 ceph-mon[31180]: from='osd.0 [v2:172.21.15.64:6802/3692023035,v1:172.21.15.64:6803/3692023035]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]} : dispatch 2024-09-21T03:56:33.849 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:33 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:56:33.925 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:33 smithi119 ceph-mon[34811]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:33.925 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:33 smithi119 ceph-mon[34811]: from='osd.0 [v2:172.21.15.64:6802/3692023035,v1:172.21.15.64:6803/3692023035]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T03:56:33.925 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:33 smithi119 ceph-mon[34811]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T03:56:33.925 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:33 smithi119 ceph-mon[34811]: from='osd.0 [v2:172.21.15.64:6802/3692023035,v1:172.21.15.64:6803/3692023035]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]} : dispatch 2024-09-21T03:56:33.925 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:33 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:56:33.927 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T03:56:33.952 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi119:vg_nvme/lv_4 2024-09-21T03:56:34.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:33 smithi131 ceph-mon[35193]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:34.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:33 smithi131 ceph-mon[35193]: from='osd.0 [v2:172.21.15.64:6802/3692023035,v1:172.21.15.64:6803/3692023035]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T03:56:34.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:33 smithi131 ceph-mon[35193]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T03:56:34.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:33 smithi131 ceph-mon[35193]: from='osd.0 [v2:172.21.15.64:6802/3692023035,v1:172.21.15.64:6803/3692023035]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]} : dispatch 2024-09-21T03:56:34.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:33 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:56:34.204 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:56:34.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.705+0000 7fa0743c6640 1 -- 172.21.15.119:0/1821377881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa06c102ae0 msgr2=0x7fa06c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:34.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.705+0000 7fa0743c6640 1 --2- 172.21.15.119:0/1821377881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa06c102ae0 0x7fa06c102ef0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa068009900 tx=0x7fa06802a410 comp rx=0 tx=0).stop 2024-09-21T03:56:34.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.705+0000 7fa0743c6640 1 -- 172.21.15.119:0/1821377881 shutdown_connections 2024-09-21T03:56:34.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.705+0000 7fa0743c6640 1 --2- 172.21.15.119:0/1821377881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa06c108d20 0x7fa06c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:34.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.705+0000 7fa0743c6640 1 --2- 172.21.15.119:0/1821377881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa06c1083d0 0x7fa06c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:34.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.705+0000 7fa0743c6640 1 --2- 172.21.15.119:0/1821377881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa06c102ae0 0x7fa06c102ef0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:34.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.705+0000 7fa0743c6640 1 -- 172.21.15.119:0/1821377881 >> 172.21.15.119:0/1821377881 conn(0x7fa06c079760 msgr2=0x7fa06c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:34.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.705+0000 7fa0743c6640 1 -- 172.21.15.119:0/1821377881 shutdown_connections 2024-09-21T03:56:34.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.705+0000 7fa0743c6640 1 -- 172.21.15.119:0/1821377881 wait complete. 2024-09-21T03:56:34.709 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.705+0000 7fa0743c6640 1 Processor -- start 2024-09-21T03:56:34.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.705+0000 7fa0743c6640 1 -- start start 2024-09-21T03:56:34.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.706+0000 7fa0743c6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa06c102ae0 0x7fa06c19b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:34.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.706+0000 7fa0743c6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa06c1083d0 0x7fa06c19b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:34.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.706+0000 7fa0743c6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa06c108d20 0x7fa06c19bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:34.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.706+0000 7fa0743c6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c112990 con 0x7fa06c108d20 2024-09-21T03:56:34.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.706+0000 7fa0743c6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c112810 con 0x7fa06c1083d0 2024-09-21T03:56:34.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.706+0000 7fa0743c6640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c112b10 con 0x7fa06c102ae0 2024-09-21T03:56:34.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.706+0000 7fa071961640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa06c1083d0 0x7fa06c19b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:34.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.706+0000 7fa071961640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa06c1083d0 0x7fa06c19b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51748/0 (socket says 172.21.15.119:51748) 2024-09-21T03:56:34.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.706+0000 7fa071961640 1 -- 172.21.15.119:0/3752507085 learned_addr learned my addr 172.21.15.119:0/3752507085 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-21T03:56:34.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.706+0000 7fa072162640 1 --2- 172.21.15.119:0/3752507085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa06c102ae0 0x7fa06c19b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:34.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.706+0000 7fa072963640 1 --2- 172.21.15.119:0/3752507085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa06c108d20 0x7fa06c19bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:34.710 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.706+0000 7fa071961640 1 -- 172.21.15.119:0/3752507085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa06c102ae0 msgr2=0x7fa06c19b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:34.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.706+0000 7fa071961640 1 --2- 172.21.15.119:0/3752507085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa06c102ae0 0x7fa06c19b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:34.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.706+0000 7fa071961640 1 -- 172.21.15.119:0/3752507085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa06c108d20 msgr2=0x7fa06c19bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:34.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.706+0000 7fa071961640 1 --2- 172.21.15.119:0/3752507085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa06c108d20 0x7fa06c19bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:34.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.706+0000 7fa071961640 1 -- 172.21.15.119:0/3752507085 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06c1a6540 con 0x7fa06c1083d0 2024-09-21T03:56:34.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.706+0000 7fa072963640 1 --2- 172.21.15.119:0/3752507085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa06c108d20 0x7fa06c19bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:56:34.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.706+0000 7fa071961640 1 --2- 172.21.15.119:0/3752507085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa06c1083d0 0x7fa06c19b800 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00d770 tx=0x7fa05c00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:34.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.706+0000 7fa0637fe640 1 -- 172.21.15.119:0/3752507085 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c017070 con 0x7fa06c1083d0 2024-09-21T03:56:34.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.706+0000 7fa0637fe640 1 -- 172.21.15.119:0/3752507085 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa05c009c30 con 0x7fa06c1083d0 2024-09-21T03:56:34.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.706+0000 7fa0637fe640 1 -- 172.21.15.119:0/3752507085 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c00f620 con 0x7fa06c1083d0 2024-09-21T03:56:34.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.706+0000 7fa0743c6640 1 -- 172.21.15.119:0/3752507085 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa06c1a6830 con 0x7fa06c1083d0 2024-09-21T03:56:34.711 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.707+0000 7fa0743c6640 1 -- 172.21.15.119:0/3752507085 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa06c1a6dc0 con 0x7fa06c1083d0 2024-09-21T03:56:34.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.707+0000 7fa0743c6640 1 -- 172.21.15.119:0/3752507085 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa06c103d10 con 0x7fa06c1083d0 2024-09-21T03:56:34.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.709+0000 7fa0637fe640 1 -- 172.21.15.119:0/3752507085 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fa05c01a070 con 0x7fa06c1083d0 2024-09-21T03:56:34.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.710+0000 7fa0637fe640 1 --2- 172.21.15.119:0/3752507085 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa0580b6550 0x7fa0580b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:34.712 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.710+0000 7fa0637fe640 1 -- 172.21.15.119:0/3752507085 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1702+0+0 (secure 0 0 0) 0x7fa05c0e53c0 con 0x7fa06c1083d0 2024-09-21T03:56:34.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.710+0000 7fa072162640 1 --2- 172.21.15.119:0/3752507085 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa0580b6550 0x7fa0580b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:34.713 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.711+0000 7fa072162640 1 --2- 172.21.15.119:0/3752507085 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa0580b6550 0x7fa0580b8a60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa068000c00 tx=0x7fa068031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:34.716 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.714+0000 7fa0637fe640 1 -- 172.21.15.119:0/3752507085 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa05c009de0 con 0x7fa06c1083d0 2024-09-21T03:56:34.905 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:34.903+0000 7fa0743c6640 1 -- 172.21.15.119:0/3752507085 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa06c063800 con 0x7fa0580b6550 2024-09-21T03:56:34.906 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:34 smithi119 ceph-mon[34811]: from='osd.0 [v2:172.21.15.64:6802/3692023035,v1:172.21.15.64:6803/3692023035]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-21T03:56:34.906 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:34 smithi119 ceph-mon[34811]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T03:56:34.906 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:34 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:56:34.906 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:34 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:56:34.906 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:34 smithi119 ceph-mon[34811]: Detected new or changed devices on smithi064 2024-09-21T03:56:34.906 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:34 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:34.906 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:34 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:34.906 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:34 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:56:34.906 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:34 smithi119 ceph-mon[34811]: Adjusting osd_memory_target on smithi064 to 17088M 2024-09-21T03:56:34.906 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:34 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:34.907 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:34 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:34.907 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:34 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:56:34.907 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:34 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:35.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:34 smithi064 ceph-mon[31180]: from='osd.0 [v2:172.21.15.64:6802/3692023035,v1:172.21.15.64:6803/3692023035]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-21T03:56:35.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:34 smithi064 ceph-mon[31180]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T03:56:35.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:34 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:56:35.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:34 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:56:35.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:34 smithi064 ceph-mon[31180]: Detected new or changed devices on smithi064 2024-09-21T03:56:35.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:34 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:35.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:34 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:35.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:34 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:56:35.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:34 smithi064 ceph-mon[31180]: Adjusting osd_memory_target on smithi064 to 17088M 2024-09-21T03:56:35.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:34 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:35.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:34 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:35.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:34 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:56:35.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:34 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:35.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:34 smithi131 ceph-mon[35193]: from='osd.0 [v2:172.21.15.64:6802/3692023035,v1:172.21.15.64:6803/3692023035]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-21T03:56:35.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:34 smithi131 ceph-mon[35193]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T03:56:35.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:34 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:56:35.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:34 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:56:35.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:34 smithi131 ceph-mon[35193]: Detected new or changed devices on smithi064 2024-09-21T03:56:35.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:34 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:35.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:34 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:35.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:34 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:56:35.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:34 smithi131 ceph-mon[35193]: Adjusting osd_memory_target on smithi064 to 17088M 2024-09-21T03:56:35.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:34 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:35.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:34 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:35.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:34 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:56:35.055 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:34 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:36.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:35 smithi119 ceph-mon[34811]: purged_snaps scrub starts 2024-09-21T03:56:36.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:35 smithi119 ceph-mon[34811]: purged_snaps scrub ok 2024-09-21T03:56:36.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:35 smithi119 ceph-mon[34811]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:36.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:35 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:56:36.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:35 smithi119 ceph-mon[34811]: osd.0 [v2:172.21.15.64:6802/3692023035,v1:172.21.15.64:6803/3692023035] boot 2024-09-21T03:56:36.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:35 smithi119 ceph-mon[34811]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T03:56:36.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:35 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:56:36.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:35 smithi119 ceph-mon[34811]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:56:36.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:35 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:56:36.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:35 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:56:36.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:35 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:36.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:35 smithi064 ceph-mon[31180]: purged_snaps scrub starts 2024-09-21T03:56:36.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:35 smithi064 ceph-mon[31180]: purged_snaps scrub ok 2024-09-21T03:56:36.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:35 smithi064 ceph-mon[31180]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:36.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:35 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:56:36.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:35 smithi064 ceph-mon[31180]: osd.0 [v2:172.21.15.64:6802/3692023035,v1:172.21.15.64:6803/3692023035] boot 2024-09-21T03:56:36.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:35 smithi064 ceph-mon[31180]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T03:56:36.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:35 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:56:36.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:35 smithi064 ceph-mon[31180]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:56:36.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:35 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:56:36.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:35 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:56:36.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:35 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:36.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:35 smithi131 ceph-mon[35193]: purged_snaps scrub starts 2024-09-21T03:56:36.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:35 smithi131 ceph-mon[35193]: purged_snaps scrub ok 2024-09-21T03:56:36.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:35 smithi131 ceph-mon[35193]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T03:56:36.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:35 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:56:36.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:35 smithi131 ceph-mon[35193]: osd.0 [v2:172.21.15.64:6802/3692023035,v1:172.21.15.64:6803/3692023035] boot 2024-09-21T03:56:36.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:35 smithi131 ceph-mon[35193]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T03:56:36.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:35 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T03:56:36.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:35 smithi131 ceph-mon[35193]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:56:36.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:35 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:56:36.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:35 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:56:36.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:35 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:37.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:36 smithi119 ceph-mon[34811]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T03:56:37.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:36 smithi064 ceph-mon[31180]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T03:56:37.057 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:36 smithi131 ceph-mon[35193]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T03:56:38.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:37 smithi119 ceph-mon[34811]: pgmap v32: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:38.008 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:37 smithi119 ceph-mon[34811]: from='client.? 172.21.15.119:0/721047174' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "29b8c92d-82fa-4175-b3e2-711917b7f8c2"} : dispatch 2024-09-21T03:56:38.008 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:37 smithi119 ceph-mon[34811]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "29b8c92d-82fa-4175-b3e2-711917b7f8c2"} : dispatch 2024-09-21T03:56:38.008 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:37 smithi119 ceph-mon[34811]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29b8c92d-82fa-4175-b3e2-711917b7f8c2"}]': finished 2024-09-21T03:56:38.008 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:37 smithi119 ceph-mon[34811]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T03:56:38.008 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:37 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:56:38.008 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:37 smithi119 ceph-mon[34811]: from='client.? 172.21.15.119:0/1741788208' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:56:38.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:37 smithi064 ceph-mon[31180]: pgmap v32: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:38.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:37 smithi064 ceph-mon[31180]: from='client.? 172.21.15.119:0/721047174' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "29b8c92d-82fa-4175-b3e2-711917b7f8c2"} : dispatch 2024-09-21T03:56:38.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:37 smithi064 ceph-mon[31180]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "29b8c92d-82fa-4175-b3e2-711917b7f8c2"} : dispatch 2024-09-21T03:56:38.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:37 smithi064 ceph-mon[31180]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29b8c92d-82fa-4175-b3e2-711917b7f8c2"}]': finished 2024-09-21T03:56:38.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:37 smithi064 ceph-mon[31180]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T03:56:38.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:37 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:56:38.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:37 smithi064 ceph-mon[31180]: from='client.? 172.21.15.119:0/1741788208' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:56:38.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:37 smithi131 ceph-mon[35193]: pgmap v32: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:38.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:37 smithi131 ceph-mon[35193]: from='client.? 172.21.15.119:0/721047174' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "29b8c92d-82fa-4175-b3e2-711917b7f8c2"} : dispatch 2024-09-21T03:56:38.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:37 smithi131 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "29b8c92d-82fa-4175-b3e2-711917b7f8c2"} : dispatch 2024-09-21T03:56:38.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:37 smithi131 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29b8c92d-82fa-4175-b3e2-711917b7f8c2"}]': finished 2024-09-21T03:56:38.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:37 smithi131 ceph-mon[35193]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T03:56:38.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:37 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:56:38.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:37 smithi131 ceph-mon[35193]: from='client.? 172.21.15.119:0/1741788208' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:56:40.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:39 smithi064 ceph-mon[31180]: pgmap v34: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:40.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:39 smithi131 ceph-mon[35193]: pgmap v34: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:40.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:39 smithi119 ceph-mon[34811]: pgmap v34: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:42.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:41 smithi131 ceph-mon[35193]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:42.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:41 smithi119 ceph-mon[34811]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:42.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:41 smithi064 ceph-mon[31180]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:44.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:43 smithi119 ceph-mon[34811]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:44.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:43 smithi131 ceph-mon[35193]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:44.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:43 smithi064 ceph-mon[31180]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:45.951 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:45 smithi119 ceph-mon[34811]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:45.951 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:45 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T03:56:45.951 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:45 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:46.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:45 smithi131 ceph-mon[35193]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:46.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:45 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T03:56:46.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:45 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:46.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:45 smithi064 ceph-mon[31180]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:46.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:45 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T03:56:46.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:45 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:46.982 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:46 smithi119 ceph-mon[34811]: Deploying daemon osd.1 on smithi119 2024-09-21T03:56:47.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:46 smithi131 ceph-mon[35193]: Deploying daemon osd.1 on smithi119 2024-09-21T03:56:47.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:46 smithi064 ceph-mon[31180]: Deploying daemon osd.1 on smithi119 2024-09-21T03:56:48.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:47 smithi131 ceph-mon[35193]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:48.178 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:47 smithi119 ceph-mon[34811]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:48.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:47 smithi064 ceph-mon[31180]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:50.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:49 smithi131 ceph-mon[35193]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:50.118 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:49 smithi119 ceph-mon[34811]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:50.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:49 smithi064 ceph-mon[31180]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:52.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:51 smithi131 ceph-mon[35193]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:52.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:51 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:56:52.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:51 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:52.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:51 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:52.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:51 smithi119 ceph-mon[34811]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:52.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:51 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:56:52.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:51 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:52.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:51 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:52.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:51 smithi064 ceph-mon[31180]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:52.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:51 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:56:52.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:51 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:52.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:51 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:54.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:53 smithi119 ceph-mon[34811]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:54.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:53 smithi131 ceph-mon[35193]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:54.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:53 smithi064 ceph-mon[31180]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:55.229 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:54 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:55.230 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:54 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:55.230 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:54 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:55.230 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:54 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:56:55.230 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:54 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:55.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:54 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:55.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:54 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:55.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:54 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:55.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:54 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:56:55.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:54 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:55.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:54 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:55.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:54 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:55.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:54 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:55.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:54 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:56:55.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:54 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:55.540 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:55.538+0000 7fa0637fe640 1 -- 172.21.15.119:0/3752507085 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa06c063800 con 0x7fa0580b6550 2024-09-21T03:56:55.541 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 1 on host 'smithi119' 2024-09-21T03:56:55.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:55.540+0000 7fa0743c6640 1 -- 172.21.15.119:0/3752507085 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa0580b6550 msgr2=0x7fa0580b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:55.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:55.540+0000 7fa0743c6640 1 --2- 172.21.15.119:0/3752507085 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa0580b6550 0x7fa0580b8a60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa068000c00 tx=0x7fa068031400 comp rx=0 tx=0).stop 2024-09-21T03:56:55.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:55.540+0000 7fa0743c6640 1 -- 172.21.15.119:0/3752507085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa06c1083d0 msgr2=0x7fa06c19b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:55.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:55.540+0000 7fa0743c6640 1 --2- 172.21.15.119:0/3752507085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa06c1083d0 0x7fa06c19b800 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00d770 tx=0x7fa05c00dc30 comp rx=0 tx=0).stop 2024-09-21T03:56:55.543 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:55.540+0000 7fa0743c6640 1 -- 172.21.15.119:0/3752507085 shutdown_connections 2024-09-21T03:56:55.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:55.540+0000 7fa0743c6640 1 --2- 172.21.15.119:0/3752507085 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa0580b6550 0x7fa0580b8a60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:55.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:55.540+0000 7fa0743c6640 1 --2- 172.21.15.119:0/3752507085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa06c108d20 0x7fa06c19bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:55.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:55.540+0000 7fa0743c6640 1 --2- 172.21.15.119:0/3752507085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa06c1083d0 0x7fa06c19b800 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:55.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:55.540+0000 7fa0743c6640 1 --2- 172.21.15.119:0/3752507085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa06c102ae0 0x7fa06c19b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:55.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:55.540+0000 7fa0743c6640 1 -- 172.21.15.119:0/3752507085 >> 172.21.15.119:0/3752507085 conn(0x7fa06c079760 msgr2=0x7fa06c101540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:55.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:55.540+0000 7fa0743c6640 1 -- 172.21.15.119:0/3752507085 shutdown_connections 2024-09-21T03:56:55.544 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:56:55.540+0000 7fa0743c6640 1 -- 172.21.15.119:0/3752507085 wait complete. 2024-09-21T03:56:56.084 DEBUG:teuthology.orchestra.run.smithi119:osd.1> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.1.service 2024-09-21T03:56:56.292 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:55 smithi064 ceph-mon[31180]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:56.292 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:55 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:56:56.292 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:55 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:56.292 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:55 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:56.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:55 smithi119 ceph-mon[34811]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:56.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:55 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:56:56.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:55 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:56.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:55 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:56.295 INFO:tasks.cephadm:Deploying osd.2 on smithi131 with /dev/vg_nvme/lv_4... 2024-09-21T03:56:56.295 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T03:56:56.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:55 smithi131 ceph-mon[35193]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:56.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:55 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:56:56.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:55 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:56.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:55 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:56.549 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:56:58.103 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:57 smithi119 ceph-mon[34811]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:58.103 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:57 smithi119 ceph-mon[34811]: from='osd.1 [v2:172.21.15.119:6800/3363415998,v1:172.21.15.119:6801/3363415998]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:56:58.103 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:57 smithi119 ceph-mon[34811]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:56:58.104 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 03:56:57 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-1[39452]: 2024-09-21T03:56:57.848+0000 7f0758b40640 -1 osd.1 0 log_to_monitors true 2024-09-21T03:56:58.233 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:57 smithi131 ceph-mon[35193]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:58.233 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:57 smithi131 ceph-mon[35193]: from='osd.1 [v2:172.21.15.119:6800/3363415998,v1:172.21.15.119:6801/3363415998]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:56:58.233 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:57 smithi131 ceph-mon[35193]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:56:58.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:57 smithi064 ceph-mon[31180]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:56:58.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:57 smithi064 ceph-mon[31180]: from='osd.1 [v2:172.21.15.119:6800/3363415998,v1:172.21.15.119:6801/3363415998]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:56:58.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:57 smithi064 ceph-mon[31180]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T03:56:58.924 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-21T03:56:58.952 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi131:vg_nvme/lv_4 2024-09-21T03:56:59.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:58 smithi119 ceph-mon[34811]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T03:56:59.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:58 smithi119 ceph-mon[34811]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T03:56:59.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:58 smithi119 ceph-mon[34811]: from='osd.1 [v2:172.21.15.119:6800/3363415998,v1:172.21.15.119:6801/3363415998]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T03:56:59.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:58 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:56:59.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:58 smithi119 ceph-mon[34811]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T03:56:59.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:58 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:59.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:58 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:59.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:58 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:56:59.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:58 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:59.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:58 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:59.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:58 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:56:59.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:58 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:59.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:58 smithi119 ceph-mon[34811]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-09-21T03:56:59.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:58 smithi119 ceph-mon[34811]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T03:56:59.068 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:58 smithi064 ceph-mon[31180]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T03:56:59.068 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:58 smithi064 ceph-mon[31180]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T03:56:59.068 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:58 smithi064 ceph-mon[31180]: from='osd.1 [v2:172.21.15.119:6800/3363415998,v1:172.21.15.119:6801/3363415998]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T03:56:59.068 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:58 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:56:59.068 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:58 smithi064 ceph-mon[31180]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T03:56:59.069 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:58 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:59.069 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:58 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:59.069 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:58 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:56:59.069 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:58 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:59.069 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:58 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:59.069 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:58 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:56:59.069 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:58 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:59.069 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:58 smithi064 ceph-mon[31180]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-09-21T03:56:59.069 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:58 smithi064 ceph-mon[31180]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T03:56:59.252 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:56:59.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:58 smithi131 ceph-mon[35193]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T03:56:59.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:58 smithi131 ceph-mon[35193]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T03:56:59.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:58 smithi131 ceph-mon[35193]: from='osd.1 [v2:172.21.15.119:6800/3363415998,v1:172.21.15.119:6801/3363415998]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T03:56:59.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:58 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:56:59.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:58 smithi131 ceph-mon[35193]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T03:56:59.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:58 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:59.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:58 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:59.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:58 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:56:59.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:58 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:59.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:58 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:56:59.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:58 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:56:59.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:58 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:56:59.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:58 smithi131 ceph-mon[35193]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-09-21T03:56:59.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:58 smithi131 ceph-mon[35193]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T03:56:59.756 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 03:56:59 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-1[39452]: 2024-09-21T03:56:59.321+0000 7f0754ac2640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T03:56:59.756 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 03:56:59 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-1[39452]: 2024-09-21T03:56:59.325+0000 7f074c8b8640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:56:59.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.799+0000 7f286d0eb640 1 -- 172.21.15.131:0/755618812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28681094a0 msgr2=0x7f28681098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:59.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.799+0000 7f286d0eb640 1 --2- 172.21.15.131:0/755618812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28681094a0 0x7f28681098b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2850009900 tx=0x7f285002a410 comp rx=0 tx=0).stop 2024-09-21T03:56:59.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.799+0000 7f286d0eb640 1 -- 172.21.15.131:0/755618812 shutdown_connections 2024-09-21T03:56:59.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.799+0000 7f286d0eb640 1 --2- 172.21.15.131:0/755618812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2868109df0 0x7f2868110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:59.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.799+0000 7f286d0eb640 1 --2- 172.21.15.131:0/755618812 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28681094a0 0x7f28681098b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:59.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.799+0000 7f286d0eb640 1 --2- 172.21.15.131:0/755618812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2868103b60 0x7f2868103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:59.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.799+0000 7f286d0eb640 1 -- 172.21.15.131:0/755618812 >> 172.21.15.131:0/755618812 conn(0x7f286807b850 msgr2=0x7f286807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:56:59.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.799+0000 7f286d0eb640 1 -- 172.21.15.131:0/755618812 shutdown_connections 2024-09-21T03:56:59.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.799+0000 7f286d0eb640 1 -- 172.21.15.131:0/755618812 wait complete. 2024-09-21T03:56:59.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.800+0000 7f286d0eb640 1 Processor -- start 2024-09-21T03:56:59.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.800+0000 7f286d0eb640 1 -- start start 2024-09-21T03:56:59.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.800+0000 7f286d0eb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2868103b60 0x7f286819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:59.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.800+0000 7f286d0eb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28681094a0 0x7f286819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:59.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.800+0000 7f286d0eb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2868109df0 0x7f286819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:59.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.800+0000 7f286d0eb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2868113ab0 con 0x7f2868103b60 2024-09-21T03:56:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.800+0000 7f286d0eb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2868113930 con 0x7f28681094a0 2024-09-21T03:56:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.800+0000 7f286d0eb640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2868113c30 con 0x7f2868109df0 2024-09-21T03:56:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.801+0000 7f2866d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2868103b60 0x7f286819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.801+0000 7f2866d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2868103b60 0x7f286819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.131:45056/0 (socket says 172.21.15.131:45056) 2024-09-21T03:56:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.801+0000 7f285ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28681094a0 0x7f286819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.801+0000 7f286759e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2868109df0 0x7f286819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.801+0000 7f2866d9d640 1 -- 172.21.15.131:0/734085120 learned_addr learned my addr 172.21.15.131:0/734085120 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-21T03:56:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.801+0000 7f2866d9d640 1 -- 172.21.15.131:0/734085120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2868109df0 msgr2=0x7f286819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.801+0000 7f2866d9d640 1 --2- 172.21.15.131:0/734085120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2868109df0 0x7f286819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.801+0000 7f2866d9d640 1 -- 172.21.15.131:0/734085120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28681094a0 msgr2=0x7f286819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:56:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.801+0000 7f2866d9d640 1 --2- 172.21.15.131:0/734085120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28681094a0 0x7f286819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:56:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.801+0000 7f2866d9d640 1 -- 172.21.15.131:0/734085120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28681a0410 con 0x7f2868103b60 2024-09-21T03:56:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.801+0000 7f285ffff640 1 --2- 172.21.15.131:0/734085120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28681094a0 0x7f286819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:56:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.801+0000 7f2866d9d640 1 --2- 172.21.15.131:0/734085120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2868103b60 0x7f286819a4a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f285800e9f0 tx=0x7f285800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.801+0000 7f2864d99640 1 -- 172.21.15.131:0/734085120 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285800cd80 con 0x7f2868103b60 2024-09-21T03:56:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.801+0000 7f286d0eb640 1 -- 172.21.15.131:0/734085120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28681a0700 con 0x7f2868103b60 2024-09-21T03:56:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.801+0000 7f2864d99640 1 -- 172.21.15.131:0/734085120 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28580054d0 con 0x7f2868103b60 2024-09-21T03:56:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.801+0000 7f2864d99640 1 -- 172.21.15.131:0/734085120 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858010690 con 0x7f2868103b60 2024-09-21T03:56:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.801+0000 7f286d0eb640 1 -- 172.21.15.131:0/734085120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28681a6e60 con 0x7f2868103b60 2024-09-21T03:56:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.802+0000 7f286d0eb640 1 -- 172.21.15.131:0/734085120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2868103f70 con 0x7f2868103b60 2024-09-21T03:56:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.805+0000 7f2864d99640 1 -- 172.21.15.131:0/734085120 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f2858004e30 con 0x7f2868103b60 2024-09-21T03:56:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.805+0000 7f2864d99640 1 --2- 172.21.15.131:0/734085120 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f28540b64d0 0x7f28540b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:56:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.805+0000 7f2864d99640 1 -- 172.21.15.131:0/734085120 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2244+0+0 (secure 0 0 0) 0x7f28580e6050 con 0x7f2868103b60 2024-09-21T03:56:59.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.806+0000 7f285ffff640 1 --2- 172.21.15.131:0/734085120 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f28540b64d0 0x7f28540b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:56:59.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.807+0000 7f285ffff640 1 --2- 172.21.15.131:0/734085120 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f28540b64d0 0x7f28540b89e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2850009230 tx=0x7f2850031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:56:59.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:56:59.813+0000 7f2864d99640 1 -- 172.21.15.131:0/734085120 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f28580ae170 con 0x7f2868103b60 2024-09-21T03:57:00.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:57:00.000+0000 7f286d0eb640 1 -- 172.21.15.131:0/734085120 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2868063800 con 0x7f28540b64d0 2024-09-21T03:57:00.002 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:59 smithi131 ceph-mon[35193]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:57:00.003 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:59 smithi131 ceph-mon[35193]: Detected new or changed devices on smithi119 2024-09-21T03:57:00.003 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:59 smithi131 ceph-mon[35193]: Adjusting osd_memory_target on smithi119 to 17088M 2024-09-21T03:57:00.003 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:59 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:57:00.003 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:59 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:57:00.003 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:59 smithi131 ceph-mon[35193]: from='osd.1 ' entity='osd.1' 2024-09-21T03:57:00.003 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:56:59 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:57:00.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:59 smithi119 ceph-mon[34811]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:57:00.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:59 smithi119 ceph-mon[34811]: Detected new or changed devices on smithi119 2024-09-21T03:57:00.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:59 smithi119 ceph-mon[34811]: Adjusting osd_memory_target on smithi119 to 17088M 2024-09-21T03:57:00.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:59 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:57:00.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:59 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:57:00.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:59 smithi119 ceph-mon[34811]: from='osd.1 ' entity='osd.1' 2024-09-21T03:57:00.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:56:59 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:57:00.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:59 smithi064 ceph-mon[31180]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:57:00.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:59 smithi064 ceph-mon[31180]: Detected new or changed devices on smithi119 2024-09-21T03:57:00.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:59 smithi064 ceph-mon[31180]: Adjusting osd_memory_target on smithi119 to 17088M 2024-09-21T03:57:00.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:59 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:57:00.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:59 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:57:00.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:59 smithi064 ceph-mon[31180]: from='osd.1 ' entity='osd.1' 2024-09-21T03:57:00.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:56:59 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:57:01.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:00 smithi119 ceph-mon[34811]: purged_snaps scrub starts 2024-09-21T03:57:01.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:00 smithi119 ceph-mon[34811]: purged_snaps scrub ok 2024-09-21T03:57:01.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:00 smithi119 ceph-mon[34811]: from='client.14484 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:57:01.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:00 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:57:01.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:00 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:57:01.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:00 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:01.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:00 smithi119 ceph-mon[34811]: osd.1 [v2:172.21.15.119:6800/3363415998,v1:172.21.15.119:6801/3363415998] boot 2024-09-21T03:57:01.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:00 smithi119 ceph-mon[34811]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T03:57:01.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:00 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:57:01.257 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:00 smithi131 ceph-mon[35193]: purged_snaps scrub starts 2024-09-21T03:57:01.257 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:00 smithi131 ceph-mon[35193]: purged_snaps scrub ok 2024-09-21T03:57:01.257 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:00 smithi131 ceph-mon[35193]: from='client.14484 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:57:01.257 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:00 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:57:01.257 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:00 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:57:01.258 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:00 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:01.258 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:00 smithi131 ceph-mon[35193]: osd.1 [v2:172.21.15.119:6800/3363415998,v1:172.21.15.119:6801/3363415998] boot 2024-09-21T03:57:01.258 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:00 smithi131 ceph-mon[35193]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T03:57:01.258 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:00 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:57:01.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:00 smithi064 ceph-mon[31180]: purged_snaps scrub starts 2024-09-21T03:57:01.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:00 smithi064 ceph-mon[31180]: purged_snaps scrub ok 2024-09-21T03:57:01.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:00 smithi064 ceph-mon[31180]: from='client.14484 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:57:01.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:00 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:57:01.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:00 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:57:01.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:00 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:01.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:00 smithi064 ceph-mon[31180]: osd.1 [v2:172.21.15.119:6800/3363415998,v1:172.21.15.119:6801/3363415998] boot 2024-09-21T03:57:01.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:00 smithi064 ceph-mon[31180]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T03:57:01.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:00 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T03:57:02.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:01 smithi119 ceph-mon[34811]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:57:02.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:01 smithi119 ceph-mon[34811]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T03:57:02.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:01 smithi064 ceph-mon[31180]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:57:02.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:01 smithi064 ceph-mon[31180]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T03:57:02.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:01 smithi131 ceph-mon[35193]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T03:57:02.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:01 smithi131 ceph-mon[35193]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T03:57:03.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:02 smithi131 ceph-mon[35193]: from='client.? 172.21.15.131:0/3367589510' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a2273fe8-df55-40f7-afda-74ab09c52c55"} : dispatch 2024-09-21T03:57:03.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:02 smithi131 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a2273fe8-df55-40f7-afda-74ab09c52c55"} : dispatch 2024-09-21T03:57:03.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:02 smithi131 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a2273fe8-df55-40f7-afda-74ab09c52c55"}]': finished 2024-09-21T03:57:03.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:02 smithi131 ceph-mon[35193]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T03:57:03.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:02 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:57:03.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:02 smithi131 ceph-mon[35193]: from='client.? 172.21.15.131:0/4126332738' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:57:03.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:02 smithi119 ceph-mon[34811]: from='client.? 172.21.15.131:0/3367589510' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a2273fe8-df55-40f7-afda-74ab09c52c55"} : dispatch 2024-09-21T03:57:03.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:02 smithi119 ceph-mon[34811]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a2273fe8-df55-40f7-afda-74ab09c52c55"} : dispatch 2024-09-21T03:57:03.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:02 smithi119 ceph-mon[34811]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a2273fe8-df55-40f7-afda-74ab09c52c55"}]': finished 2024-09-21T03:57:03.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:02 smithi119 ceph-mon[34811]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T03:57:03.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:02 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:57:03.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:02 smithi119 ceph-mon[34811]: from='client.? 172.21.15.131:0/4126332738' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:57:03.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:02 smithi064 ceph-mon[31180]: from='client.? 172.21.15.131:0/3367589510' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a2273fe8-df55-40f7-afda-74ab09c52c55"} : dispatch 2024-09-21T03:57:03.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:02 smithi064 ceph-mon[31180]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a2273fe8-df55-40f7-afda-74ab09c52c55"} : dispatch 2024-09-21T03:57:03.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:02 smithi064 ceph-mon[31180]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a2273fe8-df55-40f7-afda-74ab09c52c55"}]': finished 2024-09-21T03:57:03.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:02 smithi064 ceph-mon[31180]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T03:57:03.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:02 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:57:03.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:02 smithi064 ceph-mon[31180]: from='client.? 172.21.15.131:0/4126332738' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:57:04.486 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:04 smithi131 ceph-mon[35193]: pgmap v51: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:04.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:04 smithi119 ceph-mon[34811]: pgmap v51: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:04.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:04 smithi064 ceph-mon[31180]: pgmap v51: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:06.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:06 smithi119 ceph-mon[34811]: pgmap v52: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:06.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:06 smithi064 ceph-mon[31180]: pgmap v52: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:06.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:06 smithi131 ceph-mon[35193]: pgmap v52: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:08.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:08 smithi119 ceph-mon[34811]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:08.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:08 smithi064 ceph-mon[31180]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:08.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:08 smithi131 ceph-mon[35193]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:09.470 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:09 smithi131 ceph-mon[35193]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:09.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:09 smithi119 ceph-mon[34811]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:09.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:09 smithi064 ceph-mon[31180]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:10.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:10 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T03:57:10.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:10 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:10.392 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:10 smithi131 ceph-mon[35193]: Deploying daemon osd.2 on smithi131 2024-09-21T03:57:10.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:10 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T03:57:10.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:10 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:10.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:10 smithi119 ceph-mon[34811]: Deploying daemon osd.2 on smithi131 2024-09-21T03:57:10.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:10 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T03:57:10.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:10 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:10.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:10 smithi064 ceph-mon[31180]: Deploying daemon osd.2 on smithi131 2024-09-21T03:57:11.399 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:11 smithi131 ceph-mon[35193]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:11.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:11 smithi119 ceph-mon[34811]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:11.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:11 smithi064 ceph-mon[31180]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:13.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:13 smithi119 ceph-mon[34811]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:13.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:13 smithi064 ceph-mon[31180]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:13.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:13 smithi131 ceph-mon[35193]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:15.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:15 smithi131 ceph-mon[35193]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:15.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:15 smithi119 ceph-mon[34811]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:15.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:15 smithi064 ceph-mon[31180]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:16.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:16 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:57:16.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:16 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:16.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:16 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:16.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:16 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:57:16.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:16 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:16.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:16 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:16.766 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:16 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:57:16.766 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:16 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:16.766 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:16 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:18.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:18 smithi131 ceph-mon[35193]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:18.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:18 smithi119 ceph-mon[34811]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:18.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:18 smithi064 ceph-mon[31180]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:19.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:19 smithi131 ceph-mon[35193]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:19.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:19 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:19.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:19 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:19.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:19 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:19.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:19 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:57:19.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:19 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:19.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:57:19.865+0000 7f2864d99640 1 -- 172.21.15.131:0/734085120 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2868063800 con 0x7f28540b64d0 2024-09-21T03:57:19.869 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 2 on host 'smithi131' 2024-09-21T03:57:19.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:57:19.868+0000 7f286d0eb640 1 -- 172.21.15.131:0/734085120 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f28540b64d0 msgr2=0x7f28540b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:19.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:57:19.868+0000 7f286d0eb640 1 --2- 172.21.15.131:0/734085120 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f28540b64d0 0x7f28540b89e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2850009230 tx=0x7f2850031400 comp rx=0 tx=0).stop 2024-09-21T03:57:19.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:57:19.868+0000 7f286d0eb640 1 -- 172.21.15.131:0/734085120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2868103b60 msgr2=0x7f286819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:19.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:57:19.868+0000 7f286d0eb640 1 --2- 172.21.15.131:0/734085120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2868103b60 0x7f286819a4a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f285800e9f0 tx=0x7f285800edf0 comp rx=0 tx=0).stop 2024-09-21T03:57:19.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:57:19.868+0000 7f286d0eb640 1 -- 172.21.15.131:0/734085120 shutdown_connections 2024-09-21T03:57:19.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:57:19.868+0000 7f286d0eb640 1 --2- 172.21.15.131:0/734085120 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f28540b64d0 0x7f28540b89e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:19.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:57:19.868+0000 7f286d0eb640 1 --2- 172.21.15.131:0/734085120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2868109df0 0x7f286819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:19.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:57:19.868+0000 7f286d0eb640 1 --2- 172.21.15.131:0/734085120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28681094a0 0x7f286819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:19.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:57:19.868+0000 7f286d0eb640 1 --2- 172.21.15.131:0/734085120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2868103b60 0x7f286819a4a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:19.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:57:19.868+0000 7f286d0eb640 1 -- 172.21.15.131:0/734085120 >> 172.21.15.131:0/734085120 conn(0x7f286807b850 msgr2=0x7f2868105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:19.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:57:19.868+0000 7f286d0eb640 1 -- 172.21.15.131:0/734085120 shutdown_connections 2024-09-21T03:57:19.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:57:19.868+0000 7f286d0eb640 1 -- 172.21.15.131:0/734085120 wait complete. 2024-09-21T03:57:20.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:19 smithi119 ceph-mon[34811]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:20.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:19 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:20.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:19 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:20.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:19 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:20.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:19 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:57:20.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:19 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:20.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:19 smithi064 ceph-mon[31180]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:20.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:19 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:20.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:19 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:20.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:19 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:20.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:19 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:57:20.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:19 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:20.478 DEBUG:teuthology.orchestra.run.smithi131:osd.2> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.2.service 2024-09-21T03:57:20.480 INFO:tasks.cephadm:Deploying osd.3 on smithi064 with /dev/vg_nvme/lv_3... 2024-09-21T03:57:20.481 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T03:57:20.748 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:57:20.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:20 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:57:20.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:20 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:20.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:20 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:20.763 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:20 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:57:20.763 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:20 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:20.763 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:20 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:21.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:20 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:57:21.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:20 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:21.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:20 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:21.803 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 21 03:57:21 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-2[40420]: 2024-09-21T03:57:21.676+0000 7f088216d640 -1 osd.2 0 log_to_monitors true 2024-09-21T03:57:22.107 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:21 smithi064 ceph-mon[31180]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:22.107 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:21 smithi064 ceph-mon[31180]: from='osd.2 [v2:172.21.15.131:6800/3422924179,v1:172.21.15.131:6801/3422924179]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:57:22.107 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:21 smithi064 ceph-mon[31180]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:57:22.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:21 smithi119 ceph-mon[34811]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:22.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:21 smithi119 ceph-mon[34811]: from='osd.2 [v2:172.21.15.131:6800/3422924179,v1:172.21.15.131:6801/3422924179]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:57:22.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:21 smithi119 ceph-mon[34811]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:57:22.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:21 smithi131 ceph-mon[35193]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:22.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:21 smithi131 ceph-mon[35193]: from='osd.2 [v2:172.21.15.131:6800/3422924179,v1:172.21.15.131:6801/3422924179]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:57:22.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:21 smithi131 ceph-mon[35193]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T03:57:23.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:22 smithi064 ceph-mon[31180]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T03:57:23.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:22 smithi064 ceph-mon[31180]: from='osd.2 [v2:172.21.15.131:6800/3422924179,v1:172.21.15.131:6801/3422924179]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]} : dispatch 2024-09-21T03:57:23.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:22 smithi064 ceph-mon[31180]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T03:57:23.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:22 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:57:23.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:22 smithi064 ceph-mon[31180]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]} : dispatch 2024-09-21T03:57:23.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:22 smithi064 ceph-mon[31180]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-21T03:57:23.153 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:22 smithi064 ceph-mon[31180]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T03:57:23.154 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:22 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:57:23.175 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:22 smithi131 ceph-mon[35193]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T03:57:23.176 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:22 smithi131 ceph-mon[35193]: from='osd.2 [v2:172.21.15.131:6800/3422924179,v1:172.21.15.131:6801/3422924179]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]} : dispatch 2024-09-21T03:57:23.176 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:22 smithi131 ceph-mon[35193]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T03:57:23.176 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:22 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:57:23.176 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:22 smithi131 ceph-mon[35193]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]} : dispatch 2024-09-21T03:57:23.176 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:22 smithi131 ceph-mon[35193]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-21T03:57:23.176 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:22 smithi131 ceph-mon[35193]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T03:57:23.176 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:22 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:57:23.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:22 smithi119 ceph-mon[34811]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T03:57:23.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:22 smithi119 ceph-mon[34811]: from='osd.2 [v2:172.21.15.131:6800/3422924179,v1:172.21.15.131:6801/3422924179]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]} : dispatch 2024-09-21T03:57:23.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:22 smithi119 ceph-mon[34811]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T03:57:23.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:22 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:57:23.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:22 smithi119 ceph-mon[34811]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]} : dispatch 2024-09-21T03:57:23.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:22 smithi119 ceph-mon[34811]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-21T03:57:23.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:22 smithi119 ceph-mon[34811]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T03:57:23.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:22 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:57:23.448 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:57:23.473 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi064:vg_nvme/lv_3 2024-09-21T03:57:23.553 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 21 03:57:23 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-2[40420]: 2024-09-21T03:57:23.340+0000 7f087e0ef640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T03:57:23.553 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 21 03:57:23 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-2[40420]: 2024-09-21T03:57:23.344+0000 7f0875ee5640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:57:23.724 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:57:24.103 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:23 smithi064 ceph-mon[31180]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:24.104 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:23 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:57:24.104 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:23 smithi064 ceph-mon[31180]: Detected new or changed devices on smithi131 2024-09-21T03:57:24.104 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:23 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:24.104 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:23 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:24.104 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:23 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:57:24.104 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:23 smithi064 ceph-mon[31180]: Adjusting osd_memory_target on smithi131 to 17088M 2024-09-21T03:57:24.104 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:23 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:24.104 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:23 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:24.104 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:23 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:57:24.104 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:23 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:24.104 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:23 smithi064 ceph-mon[31180]: from='osd.2 ' entity='osd.2' 2024-09-21T03:57:24.104 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:23 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:57:24.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:23 smithi119 ceph-mon[34811]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:24.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:23 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:57:24.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:23 smithi119 ceph-mon[34811]: Detected new or changed devices on smithi131 2024-09-21T03:57:24.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:23 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:24.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:23 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:24.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:23 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:57:24.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:23 smithi119 ceph-mon[34811]: Adjusting osd_memory_target on smithi131 to 17088M 2024-09-21T03:57:24.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:23 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:24.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:23 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:24.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:23 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:57:24.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:23 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:24.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:23 smithi119 ceph-mon[34811]: from='osd.2 ' entity='osd.2' 2024-09-21T03:57:24.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:23 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:57:24.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:23 smithi131 ceph-mon[35193]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:24.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:23 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:57:24.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:23 smithi131 ceph-mon[35193]: Detected new or changed devices on smithi131 2024-09-21T03:57:24.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:23 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:24.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:23 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:24.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:23 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:57:24.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:23 smithi131 ceph-mon[35193]: Adjusting osd_memory_target on smithi131 to 17088M 2024-09-21T03:57:24.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:23 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:24.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:23 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:24.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:23 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:57:24.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:23 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:24.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:23 smithi131 ceph-mon[35193]: from='osd.2 ' entity='osd.2' 2024-09-21T03:57:24.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:23 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:57:24.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.302+0000 7f8afc53a640 1 -- 172.21.15.64:0/236613669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8af4105380 msgr2=0x7f8af410d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:24.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.302+0000 7f8afc53a640 1 --2- 172.21.15.64:0/236613669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8af4105380 0x7f8af410d8d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009900 tx=0x7f8ae802a410 comp rx=0 tx=0).stop 2024-09-21T03:57:24.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.302+0000 7f8afc53a640 1 -- 172.21.15.64:0/236613669 shutdown_connections 2024-09-21T03:57:24.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.302+0000 7f8afc53a640 1 --2- 172.21.15.64:0/236613669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8af410de60 0x7f8af4110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:24.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.302+0000 7f8afc53a640 1 --2- 172.21.15.64:0/236613669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8af4105380 0x7f8af410d8d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:24.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.302+0000 7f8afc53a640 1 --2- 172.21.15.64:0/236613669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8af4104a30 0x7f8af4104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:24.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.302+0000 7f8afc53a640 1 -- 172.21.15.64:0/236613669 >> 172.21.15.64:0/236613669 conn(0x7f8af40797b0 msgr2=0x7f8af4079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:24.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.302+0000 7f8afc53a640 1 -- 172.21.15.64:0/236613669 shutdown_connections 2024-09-21T03:57:24.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.302+0000 7f8afc53a640 1 -- 172.21.15.64:0/236613669 wait complete. 2024-09-21T03:57:24.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.303+0000 7f8afc53a640 1 Processor -- start 2024-09-21T03:57:24.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.303+0000 7f8afc53a640 1 -- start start 2024-09-21T03:57:24.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.303+0000 7f8afc53a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8af4104a30 0x7f8af419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:24.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.303+0000 7f8afc53a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8af4105380 0x7f8af419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:24.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.303+0000 7f8afc53a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8af410de60 0x7f8af419fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:24.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.303+0000 7f8afc53a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8af4113aa0 con 0x7f8af4105380 2024-09-21T03:57:24.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.303+0000 7f8afc53a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8af4113920 con 0x7f8af4104a30 2024-09-21T03:57:24.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.303+0000 7f8afc53a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8af4113c20 con 0x7f8af410de60 2024-09-21T03:57:24.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.304+0000 7f8af9ad5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8af4105380 0x7f8af419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:24.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.304+0000 7f8af9ad5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8af4105380 0x7f8af419aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34756/0 (socket says 172.21.15.64:34756) 2024-09-21T03:57:24.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.304+0000 7f8af9ad5640 1 -- 172.21.15.64:0/1814384513 learned_addr learned my addr 172.21.15.64:0/1814384513 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:57:24.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.304+0000 7f8afaad7640 1 --2- 172.21.15.64:0/1814384513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8af410de60 0x7f8af419fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:24.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.304+0000 7f8afa2d6640 1 --2- 172.21.15.64:0/1814384513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8af4104a30 0x7f8af419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:24.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.304+0000 7f8af9ad5640 1 -- 172.21.15.64:0/1814384513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8af410de60 msgr2=0x7f8af419fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:24.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.304+0000 7f8af9ad5640 1 --2- 172.21.15.64:0/1814384513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8af410de60 0x7f8af419fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:24.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.304+0000 7f8af9ad5640 1 -- 172.21.15.64:0/1814384513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8af4104a30 msgr2=0x7f8af419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:24.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.304+0000 7f8af9ad5640 1 --2- 172.21.15.64:0/1814384513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8af4104a30 0x7f8af419a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:24.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.304+0000 7f8af9ad5640 1 -- 172.21.15.64:0/1814384513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af41a04a0 con 0x7f8af4105380 2024-09-21T03:57:24.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.304+0000 7f8af9ad5640 1 --2- 172.21.15.64:0/1814384513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8af4105380 0x7f8af419aad0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8ae802a930 tx=0x7f8ae8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:24.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.304+0000 7f8ae77fe640 1 -- 172.21.15.64:0/1814384513 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae803f030 con 0x7f8af4105380 2024-09-21T03:57:24.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.304+0000 7f8ae77fe640 1 -- 172.21.15.64:0/1814384513 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ae8003aa0 con 0x7f8af4105380 2024-09-21T03:57:24.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.304+0000 7f8afc53a640 1 -- 172.21.15.64:0/1814384513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af41a0730 con 0x7f8af4105380 2024-09-21T03:57:24.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.304+0000 7f8ae77fe640 1 -- 172.21.15.64:0/1814384513 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8035070 con 0x7f8af4105380 2024-09-21T03:57:24.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.304+0000 7f8afc53a640 1 -- 172.21.15.64:0/1814384513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af41a6ef0 con 0x7f8af4105380 2024-09-21T03:57:24.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.305+0000 7f8afc53a640 1 -- 172.21.15.64:0/1814384513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8af406bed0 con 0x7f8af4105380 2024-09-21T03:57:24.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.307+0000 7f8ae77fe640 1 -- 172.21.15.64:0/1814384513 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f8ae8042030 con 0x7f8af4105380 2024-09-21T03:57:24.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.308+0000 7f8ae77fe640 1 --2- 172.21.15.64:0/1814384513 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8ae00b64d0 0x7f8ae00b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:24.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.308+0000 7f8afa2d6640 1 --2- 172.21.15.64:0/1814384513 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8ae00b64d0 0x7f8ae00b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:24.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.308+0000 7f8ae77fe640 1 -- 172.21.15.64:0/1814384513 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2786+0+0 (secure 0 0 0) 0x7f8ae81066a0 con 0x7f8af4105380 2024-09-21T03:57:24.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.308+0000 7f8afa2d6640 1 --2- 172.21.15.64:0/1814384513 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8ae00b64d0 0x7f8ae00b89e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8af0005fd0 tx=0x7f8af0009680 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:24.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.311+0000 7f8ae77fe640 1 -- 172.21.15.64:0/1814384513 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8ae80d2980 con 0x7f8af4105380 2024-09-21T03:57:24.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:24.516+0000 7f8afc53a640 1 -- 172.21.15.64:0/1814384513 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f8af4063800 con 0x7f8ae00b64d0 2024-09-21T03:57:25.427 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:25 smithi064 ceph-mon[31180]: purged_snaps scrub starts 2024-09-21T03:57:25.427 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:25 smithi064 ceph-mon[31180]: purged_snaps scrub ok 2024-09-21T03:57:25.427 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:25 smithi064 ceph-mon[31180]: osd.2 [v2:172.21.15.131:6800/3422924179,v1:172.21.15.131:6801/3422924179] boot 2024-09-21T03:57:25.427 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:25 smithi064 ceph-mon[31180]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T03:57:25.427 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:25 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:57:25.427 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:25 smithi064 ceph-mon[31180]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:25.427 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:25 smithi064 ceph-mon[31180]: from='client.14505 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:57:25.427 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:25 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:57:25.428 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:25 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:57:25.428 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:25 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:25.428 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:25 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:57:25.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:25 smithi119 ceph-mon[34811]: purged_snaps scrub starts 2024-09-21T03:57:25.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:25 smithi119 ceph-mon[34811]: purged_snaps scrub ok 2024-09-21T03:57:25.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:25 smithi119 ceph-mon[34811]: osd.2 [v2:172.21.15.131:6800/3422924179,v1:172.21.15.131:6801/3422924179] boot 2024-09-21T03:57:25.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:25 smithi119 ceph-mon[34811]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T03:57:25.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:25 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:57:25.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:25 smithi119 ceph-mon[34811]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:25.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:25 smithi119 ceph-mon[34811]: from='client.14505 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:57:25.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:25 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:57:25.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:25 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:57:25.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:25 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:25.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:25 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:57:25.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:25 smithi131 ceph-mon[35193]: purged_snaps scrub starts 2024-09-21T03:57:25.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:25 smithi131 ceph-mon[35193]: purged_snaps scrub ok 2024-09-21T03:57:25.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:25 smithi131 ceph-mon[35193]: osd.2 [v2:172.21.15.131:6800/3422924179,v1:172.21.15.131:6801/3422924179] boot 2024-09-21T03:57:25.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:25 smithi131 ceph-mon[35193]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T03:57:25.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:25 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T03:57:25.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:25 smithi131 ceph-mon[35193]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T03:57:25.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:25 smithi131 ceph-mon[35193]: from='client.14505 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:57:25.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:25 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:57:25.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:25 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:57:25.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:25 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:25.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:25 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:57:26.431 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:26 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T03:57:26.431 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:26 smithi064 ceph-mon[31180]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T03:57:26.432 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:26 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:57:26.682 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:26 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T03:57:26.688 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:26 smithi131 ceph-mon[35193]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T03:57:26.697 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:26 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:57:26.698 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 21 03:57:26 smithi131 sudo[43915]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:57:26.698 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 21 03:57:26 smithi131 sudo[43915]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:57:26.698 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 21 03:57:26 smithi131 sudo[43915]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:57:26.699 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 21 03:57:26 smithi131 sudo[43915]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:57:26.699 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 21 03:57:26 smithi131 sudo[43918]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:57:26.699 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 21 03:57:26 smithi131 sudo[43918]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:57:26.699 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 21 03:57:26 smithi131 sudo[43918]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:57:26.699 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 21 03:57:26 smithi131 sudo[43918]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:57:26.722 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:26 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T03:57:26.722 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:26 smithi119 ceph-mon[34811]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T03:57:26.722 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:26 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:57:26.723 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 03:57:26 smithi119 sudo[43433]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:57:26.723 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 03:57:26 smithi119 sudo[43433]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:57:26.723 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 03:57:26 smithi119 sudo[43433]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:57:26.723 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 03:57:26 smithi119 sudo[43433]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:57:26.723 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 03:57:26 smithi119 sudo[43436]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:57:26.723 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 03:57:26 smithi119 sudo[43436]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:57:26.724 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 03:57:26 smithi119 sudo[43436]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:57:26.724 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 03:57:26 smithi119 sudo[43436]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:57:26.724 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:26 smithi064 sudo[49774]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:57:26.724 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:26 smithi064 sudo[49774]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:57:26.724 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:26 smithi064 sudo[49774]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:57:26.725 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:26 smithi064 sudo[49774]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:57:26.725 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:26 smithi064 sudo[49779]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:57:26.725 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:26 smithi064 sudo[49779]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:57:26.725 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:26 smithi064 sudo[49779]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:57:26.725 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:26 smithi064 sudo[49779]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:57:26.725 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 21 03:57:26 smithi064 sudo[49754]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:57:26.725 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 21 03:57:26 smithi064 sudo[49754]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:57:26.726 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 21 03:57:26 smithi064 sudo[49754]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:57:26.726 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 21 03:57:26 smithi064 sudo[49754]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:57:26.726 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 21 03:57:26 smithi064 sudo[49757]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:57:26.726 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 21 03:57:26 smithi064 sudo[49757]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:57:26.726 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 21 03:57:26 smithi064 sudo[49757]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:57:26.726 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 21 03:57:26 smithi064 sudo[49757]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:57:27.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:26 smithi119 sudo[43439]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:57:27.404 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:26 smithi119 sudo[43439]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:57:27.404 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:26 smithi119 sudo[43439]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:57:27.404 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:26 smithi119 sudo[43439]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:57:27.404 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:26 smithi119 sudo[43442]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:57:27.404 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:26 smithi119 sudo[43442]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:57:27.404 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:26 smithi119 sudo[43442]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:57:27.404 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:26 smithi119 sudo[43442]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:57:27.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:27.380+0000 7f8ae77fe640 1 -- 172.21.15.64:0/1814384513 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f8ae80ca1c0 con 0x7f8af4105380 2024-09-21T03:57:27.405 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:26 smithi131 sudo[43921]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T03:57:27.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:26 smithi131 sudo[43921]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:57:27.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:26 smithi131 sudo[43921]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:57:27.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:26 smithi131 sudo[43921]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:57:27.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:26 smithi131 sudo[43924]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T03:57:27.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:26 smithi131 sudo[43924]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T03:57:27.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:26 smithi131 sudo[43924]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T03:57:27.406 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:26 smithi131 sudo[43924]: pam_unix(sudo:session): session closed for user root 2024-09-21T03:57:27.630 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:27 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:57:27.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:27 smithi064 ceph-mon[31180]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T03:57:27.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:27 smithi064 ceph-mon[31180]: pgmap v68: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:27.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:27 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:27.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:27 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:27.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:27 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:27.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:27 smithi064 ceph-mon[31180]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:57:27.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:27 smithi064 ceph-mon[31180]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:57:27.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:27 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:57:27.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:27 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:57:27.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:27 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:57:27.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:27 smithi064 ceph-mon[31180]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:57:27.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:27 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:57:27.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:27 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:57:27.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:27 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:57:27.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:27 smithi064 ceph-mon[31180]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:57:27.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:27 smithi064 ceph-mon[31180]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:57:27.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:27 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:57:27.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:27 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:57:27.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:27 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:57:27.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:27 smithi064 ceph-mon[31180]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:57:27.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:27 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2302385438' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f1634064-b108-41d2-a78f-c33607b08a70"} : dispatch 2024-09-21T03:57:27.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:27 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2302385438' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1634064-b108-41d2-a78f-c33607b08a70"}]': finished 2024-09-21T03:57:27.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:27 smithi064 ceph-mon[31180]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T03:57:27.632 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:27 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:57:27.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:27 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:57:27.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:27 smithi119 ceph-mon[34811]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T03:57:27.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:27 smithi119 ceph-mon[34811]: pgmap v68: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:27.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:27 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:27.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:27 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:27.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:27 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:27.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:27 smithi119 ceph-mon[34811]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:57:27.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:27 smithi119 ceph-mon[34811]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:57:27.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:27 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:57:27.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:27 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:57:27.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:27 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:57:27.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:27 smithi119 ceph-mon[34811]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:57:27.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:27 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:57:27.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:27 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:57:27.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:27 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:57:27.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:27 smithi119 ceph-mon[34811]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:57:27.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:27 smithi119 ceph-mon[34811]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:57:27.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:27 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:57:27.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:27 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:57:27.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:27 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:57:27.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:27 smithi119 ceph-mon[34811]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:57:27.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:27 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2302385438' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f1634064-b108-41d2-a78f-c33607b08a70"} : dispatch 2024-09-21T03:57:27.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:27 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2302385438' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1634064-b108-41d2-a78f-c33607b08a70"}]': finished 2024-09-21T03:57:27.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:27 smithi119 ceph-mon[34811]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T03:57:27.759 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:27 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:57:27.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:27 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:57:27.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:27 smithi131 ceph-mon[35193]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T03:57:27.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:27 smithi131 ceph-mon[35193]: pgmap v68: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:27.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:27 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:27.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:27 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:27.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:27 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:27.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:27 smithi131 ceph-mon[35193]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:57:27.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:27 smithi131 ceph-mon[35193]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:57:27.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:27 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:57:27.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:27 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:57:27.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:27 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:57:27.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:27 smithi131 ceph-mon[35193]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:57:27.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:27 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:57:27.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:27 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:57:27.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:27 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:57:27.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:27 smithi131 ceph-mon[35193]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:57:27.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:27 smithi131 ceph-mon[35193]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T03:57:27.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:27 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T03:57:27.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:27 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T03:57:27.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:27 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T03:57:27.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:27 smithi131 ceph-mon[35193]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T03:57:27.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:27 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2302385438' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f1634064-b108-41d2-a78f-c33607b08a70"} : dispatch 2024-09-21T03:57:27.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:27 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2302385438' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1634064-b108-41d2-a78f-c33607b08a70"}]': finished 2024-09-21T03:57:27.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:27 smithi131 ceph-mon[35193]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T03:57:27.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:27 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:57:28.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:28 smithi131 ceph-mon[35193]: mgrmap e20: x(active, since 118s), standbys: z, y 2024-09-21T03:57:28.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:28 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2572131557' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:57:28.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:28 smithi119 ceph-mon[34811]: mgrmap e20: x(active, since 118s), standbys: z, y 2024-09-21T03:57:28.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:28 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2572131557' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:57:28.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:28 smithi064 ceph-mon[31180]: mgrmap e20: x(active, since 118s), standbys: z, y 2024-09-21T03:57:28.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:28 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2572131557' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:57:29.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:29 smithi119 ceph-mon[34811]: pgmap v70: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:29.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:29 smithi064 ceph-mon[31180]: pgmap v70: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:29.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:29 smithi131 ceph-mon[35193]: pgmap v70: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:31.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:31 smithi119 ceph-mon[34811]: pgmap v71: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:31.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:31 smithi064 ceph-mon[31180]: pgmap v71: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:31.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:31 smithi131 ceph-mon[35193]: pgmap v71: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:33.595 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:33 smithi064 ceph-mon[31180]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:33.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:33 smithi119 ceph-mon[34811]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:33.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:33 smithi131 ceph-mon[35193]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:35.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:35 smithi119 ceph-mon[34811]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:35.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:35 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T03:57:35.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:35 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:35.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:35 smithi119 ceph-mon[34811]: Deploying daemon osd.3 on smithi064 2024-09-21T03:57:35.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:35 smithi064 ceph-mon[31180]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:35.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:35 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T03:57:35.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:35 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:35.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:35 smithi064 ceph-mon[31180]: Deploying daemon osd.3 on smithi064 2024-09-21T03:57:35.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:35 smithi131 ceph-mon[35193]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:35.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:35 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T03:57:35.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:35 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:35.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:35 smithi131 ceph-mon[35193]: Deploying daemon osd.3 on smithi064 2024-09-21T03:57:37.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:37 smithi119 ceph-mon[34811]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:37.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:37 smithi064 ceph-mon[31180]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:37.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:37 smithi131 ceph-mon[35193]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:39.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:39 smithi119 ceph-mon[34811]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:39.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:39 smithi064 ceph-mon[31180]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:39.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:39 smithi131 ceph-mon[35193]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:41.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:41 smithi119 ceph-mon[34811]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:41.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:41 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:57:41.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:41 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:41.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:41 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:41.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:41 smithi064 ceph-mon[31180]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:41.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:41 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:57:41.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:41 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:41.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:41 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:41.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:41 smithi131 ceph-mon[35193]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:41.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:41 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:57:41.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:41 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:41.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:41 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:43.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:43 smithi119 ceph-mon[34811]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:43.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:43 smithi064 ceph-mon[31180]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:43.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:43 smithi131 ceph-mon[35193]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:44.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:44.709+0000 7f8ae77fe640 1 -- 172.21.15.64:0/1814384513 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8af4063800 con 0x7f8ae00b64d0 2024-09-21T03:57:44.712 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 3 on host 'smithi064' 2024-09-21T03:57:44.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:44.712+0000 7f8afc53a640 1 -- 172.21.15.64:0/1814384513 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8ae00b64d0 msgr2=0x7f8ae00b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:44.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:44.712+0000 7f8afc53a640 1 --2- 172.21.15.64:0/1814384513 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8ae00b64d0 0x7f8ae00b89e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8af0005fd0 tx=0x7f8af0009680 comp rx=0 tx=0).stop 2024-09-21T03:57:44.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:44.712+0000 7f8afc53a640 1 -- 172.21.15.64:0/1814384513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8af4105380 msgr2=0x7f8af419aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:44.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:44.712+0000 7f8afc53a640 1 --2- 172.21.15.64:0/1814384513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8af4105380 0x7f8af419aad0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8ae802a930 tx=0x7f8ae8004270 comp rx=0 tx=0).stop 2024-09-21T03:57:44.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:44.712+0000 7f8afc53a640 1 -- 172.21.15.64:0/1814384513 shutdown_connections 2024-09-21T03:57:44.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:44.712+0000 7f8afc53a640 1 --2- 172.21.15.64:0/1814384513 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8ae00b64d0 0x7f8ae00b89e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:44.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:44.712+0000 7f8afc53a640 1 --2- 172.21.15.64:0/1814384513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8af410de60 0x7f8af419fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:44.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:44.712+0000 7f8afc53a640 1 --2- 172.21.15.64:0/1814384513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8af4105380 0x7f8af419aad0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:44.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:44.712+0000 7f8afc53a640 1 --2- 172.21.15.64:0/1814384513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8af4104a30 0x7f8af419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:44.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:44.712+0000 7f8afc53a640 1 -- 172.21.15.64:0/1814384513 >> 172.21.15.64:0/1814384513 conn(0x7f8af40797b0 msgr2=0x7f8af40ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:44.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:44.713+0000 7f8afc53a640 1 -- 172.21.15.64:0/1814384513 shutdown_connections 2024-09-21T03:57:44.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:57:44.713+0000 7f8afc53a640 1 -- 172.21.15.64:0/1814384513 wait complete. 2024-09-21T03:57:45.000 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:44 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:45.000 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:44 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:45.000 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:44 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:45.000 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:44 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:57:45.000 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:44 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:45.000 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:44 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:57:45.000 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:44 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:45.000 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:44 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:45.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:44 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:45.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:44 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:45.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:44 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:45.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:44 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:57:45.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:44 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:45.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:44 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:57:45.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:44 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:45.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:44 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:45.287 DEBUG:teuthology.orchestra.run.smithi064:osd.3> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.3.service 2024-09-21T03:57:45.289 INFO:tasks.cephadm:Deploying osd.4 on smithi119 with /dev/vg_nvme/lv_3... 2024-09-21T03:57:45.289 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T03:57:45.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:44 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:45.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:44 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:45.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:44 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:45.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:44 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:57:45.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:44 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:45.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:44 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:57:45.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:44 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:45.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:44 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:45.541 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:57:45.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:45 smithi119 ceph-mon[34811]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:46.132 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:45 smithi064 ceph-mon[31180]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:46.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:45 smithi131 ceph-mon[35193]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:47.910 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T03:57:47.943 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi119:vg_nvme/lv_3 2024-09-21T03:57:48.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:47 smithi119 ceph-mon[34811]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:48.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:47 smithi119 ceph-mon[34811]: from='osd.3 [v2:172.21.15.64:6810/2022952729,v1:172.21.15.64:6811/2022952729]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T03:57:48.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:47 smithi064 ceph-mon[31180]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:48.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:47 smithi064 ceph-mon[31180]: from='osd.3 [v2:172.21.15.64:6810/2022952729,v1:172.21.15.64:6811/2022952729]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T03:57:48.011 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 21 03:57:47 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-3[51799]: 2024-09-21T03:57:47.708+0000 7fd31ac3f640 -1 osd.3 0 log_to_monitors true 2024-09-21T03:57:48.190 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:57:48.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:47 smithi131 ceph-mon[35193]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:48.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:47 smithi131 ceph-mon[35193]: from='osd.3 [v2:172.21.15.64:6810/2022952729,v1:172.21.15.64:6811/2022952729]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T03:57:48.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.717+0000 7fc20f46b640 1 -- 172.21.15.119:0/387842366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc208102a60 msgr2=0x7fc208102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:48.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.717+0000 7fc20f46b640 1 --2- 172.21.15.119:0/387842366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc208102a60 0x7fc208102e70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc204009900 tx=0x7fc20402a410 comp rx=0 tx=0).stop 2024-09-21T03:57:48.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.718+0000 7fc20f46b640 1 -- 172.21.15.119:0/387842366 shutdown_connections 2024-09-21T03:57:48.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.718+0000 7fc20f46b640 1 --2- 172.21.15.119:0/387842366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc208108cf0 0x7fc20810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:48.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.718+0000 7fc20f46b640 1 --2- 172.21.15.119:0/387842366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2081083a0 0x7fc2081087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:48.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.718+0000 7fc20f46b640 1 --2- 172.21.15.119:0/387842366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc208102a60 0x7fc208102e70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:48.727 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.718+0000 7fc20f46b640 1 -- 172.21.15.119:0/387842366 >> 172.21.15.119:0/387842366 conn(0x7fc208079730 msgr2=0x7fc208079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:57:48.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.718+0000 7fc20f46b640 1 -- 172.21.15.119:0/387842366 shutdown_connections 2024-09-21T03:57:48.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.718+0000 7fc20f46b640 1 -- 172.21.15.119:0/387842366 wait complete. 2024-09-21T03:57:48.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.718+0000 7fc20f46b640 1 Processor -- start 2024-09-21T03:57:48.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.718+0000 7fc20f46b640 1 -- start start 2024-09-21T03:57:48.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.719+0000 7fc20f46b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc208102a60 0x7fc20819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:48.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.719+0000 7fc20f46b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2081083a0 0x7fc20819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:48.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.719+0000 7fc20f46b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc208108cf0 0x7fc20819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:48.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.719+0000 7fc20f46b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc208112a40 con 0x7fc208102a60 2024-09-21T03:57:48.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.719+0000 7fc20f46b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc2081128c0 con 0x7fc2081083a0 2024-09-21T03:57:48.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.719+0000 7fc20f46b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc208112bc0 con 0x7fc208108cf0 2024-09-21T03:57:48.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.719+0000 7fc20ca06640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2081083a0 0x7fc20819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:48.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.719+0000 7fc20d207640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc208102a60 0x7fc20819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:48.728 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.719+0000 7fc20ca06640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2081083a0 0x7fc20819ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41274/0 (socket says 172.21.15.119:41274) 2024-09-21T03:57:48.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.719+0000 7fc20ca06640 1 -- 172.21.15.119:0/2361039769 learned_addr learned my addr 172.21.15.119:0/2361039769 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-21T03:57:48.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.719+0000 7fc20da08640 1 --2- 172.21.15.119:0/2361039769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc208108cf0 0x7fc20819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:48.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.719+0000 7fc20ca06640 1 -- 172.21.15.119:0/2361039769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc208108cf0 msgr2=0x7fc20819fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:48.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.719+0000 7fc20ca06640 1 --2- 172.21.15.119:0/2361039769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc208108cf0 0x7fc20819fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:48.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.719+0000 7fc20ca06640 1 -- 172.21.15.119:0/2361039769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc208102a60 msgr2=0x7fc20819a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:57:48.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.719+0000 7fc20ca06640 1 --2- 172.21.15.119:0/2361039769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc208102a60 0x7fc20819a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:57:48.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.719+0000 7fc20ca06640 1 -- 172.21.15.119:0/2361039769 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2081a0510 con 0x7fc2081083a0 2024-09-21T03:57:48.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.719+0000 7fc20ca06640 1 --2- 172.21.15.119:0/2361039769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2081083a0 0x7fc20819ab40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc1f800b450 tx=0x7fc1f800b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:48.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.719+0000 7fc1fe7fc640 1 -- 172.21.15.119:0/2361039769 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8015020 con 0x7fc2081083a0 2024-09-21T03:57:48.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.719+0000 7fc20f46b640 1 -- 172.21.15.119:0/2361039769 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2081a0800 con 0x7fc2081083a0 2024-09-21T03:57:48.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.719+0000 7fc1fe7fc640 1 -- 172.21.15.119:0/2361039769 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f8011d40 con 0x7fc2081083a0 2024-09-21T03:57:48.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.719+0000 7fc1fe7fc640 1 -- 172.21.15.119:0/2361039769 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f800f340 con 0x7fc2081083a0 2024-09-21T03:57:48.729 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.719+0000 7fc20f46b640 1 -- 172.21.15.119:0/2361039769 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2081a6f60 con 0x7fc2081083a0 2024-09-21T03:57:48.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.720+0000 7fc20f46b640 1 -- 172.21.15.119:0/2361039769 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc208102ef0 con 0x7fc2081083a0 2024-09-21T03:57:48.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.722+0000 7fc1fe7fc640 1 -- 172.21.15.119:0/2361039769 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fc1f8018020 con 0x7fc2081083a0 2024-09-21T03:57:48.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.723+0000 7fc1fe7fc640 1 --2- 172.21.15.119:0/2361039769 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc1f40b6560 0x7fc1f40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:57:48.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.723+0000 7fc1fe7fc640 1 -- 172.21.15.119:0/2361039769 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 3588+0+0 (secure 0 0 0) 0x7fc1f80e6070 con 0x7fc2081083a0 2024-09-21T03:57:48.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.723+0000 7fc20d207640 1 --2- 172.21.15.119:0/2361039769 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc1f40b6560 0x7fc1f40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:57:48.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.723+0000 7fc20d207640 1 --2- 172.21.15.119:0/2361039769 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc1f40b6560 0x7fc1f40b8a70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc204031440 tx=0x7fc204005e80 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:57:48.730 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.726+0000 7fc1fe7fc640 1 -- 172.21.15.119:0/2361039769 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc1f801f020 con 0x7fc2081083a0 2024-09-21T03:57:48.916 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:57:48.914+0000 7fc20f46b640 1 -- 172.21.15.119:0/2361039769 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc20806aeb0 con 0x7fc1f40b6560 2024-09-21T03:57:48.917 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:48 smithi119 ceph-mon[34811]: from='osd.3 [v2:172.21.15.64:6810/2022952729,v1:172.21.15.64:6811/2022952729]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T03:57:48.917 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:48 smithi119 ceph-mon[34811]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T03:57:48.917 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:48 smithi119 ceph-mon[34811]: from='osd.3 [v2:172.21.15.64:6810/2022952729,v1:172.21.15.64:6811/2022952729]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]} : dispatch 2024-09-21T03:57:48.917 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:48 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:57:48.917 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:48 smithi119 ceph-mon[34811]: Detected new or changed devices on smithi064 2024-09-21T03:57:48.917 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:48 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:48.917 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:48 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:48.917 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:48 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:57:48.917 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:48 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:57:48.917 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:48 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:48.917 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:48 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:48.917 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:48 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:57:48.917 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:48 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:48.918 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:48 smithi119 ceph-mon[34811]: from='osd.3 [v2:172.21.15.64:6810/2022952729,v1:172.21.15.64:6811/2022952729]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-21T03:57:49.261 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 21 03:57:48 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-3[51799]: 2024-09-21T03:57:48.863+0000 7fd316bc1640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T03:57:49.261 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 21 03:57:48 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-3[51799]: 2024-09-21T03:57:48.868+0000 7fd3129bf640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:57:49.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:48 smithi064 ceph-mon[31180]: from='osd.3 [v2:172.21.15.64:6810/2022952729,v1:172.21.15.64:6811/2022952729]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T03:57:49.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:48 smithi064 ceph-mon[31180]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T03:57:49.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:48 smithi064 ceph-mon[31180]: from='osd.3 [v2:172.21.15.64:6810/2022952729,v1:172.21.15.64:6811/2022952729]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]} : dispatch 2024-09-21T03:57:49.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:48 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:57:49.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:48 smithi064 ceph-mon[31180]: Detected new or changed devices on smithi064 2024-09-21T03:57:49.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:48 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:49.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:48 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:49.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:48 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:57:49.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:48 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:57:49.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:48 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:49.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:48 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:49.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:48 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:57:49.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:48 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:49.263 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:48 smithi064 ceph-mon[31180]: from='osd.3 [v2:172.21.15.64:6810/2022952729,v1:172.21.15.64:6811/2022952729]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-21T03:57:49.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:48 smithi131 ceph-mon[35193]: from='osd.3 [v2:172.21.15.64:6810/2022952729,v1:172.21.15.64:6811/2022952729]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T03:57:49.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:48 smithi131 ceph-mon[35193]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T03:57:49.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:48 smithi131 ceph-mon[35193]: from='osd.3 [v2:172.21.15.64:6810/2022952729,v1:172.21.15.64:6811/2022952729]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]} : dispatch 2024-09-21T03:57:49.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:48 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:57:49.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:48 smithi131 ceph-mon[35193]: Detected new or changed devices on smithi064 2024-09-21T03:57:49.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:48 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:49.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:48 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:49.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:48 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:57:49.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:48 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:57:49.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:48 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:49.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:48 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:49.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:48 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:57:49.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:48 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:57:49.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:48 smithi131 ceph-mon[35193]: from='osd.3 [v2:172.21.15.64:6810/2022952729,v1:172.21.15.64:6811/2022952729]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-21T03:57:50.242 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:49 smithi119 ceph-mon[34811]: Adjusting osd_memory_target on smithi064 to 8544M 2024-09-21T03:57:50.242 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:49 smithi119 ceph-mon[34811]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:50.242 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:49 smithi119 ceph-mon[34811]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T03:57:50.242 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:49 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:57:50.242 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:49 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:57:50.242 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:49 smithi119 ceph-mon[34811]: from='client.24395 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:57:50.243 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:49 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:57:50.243 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:49 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:57:50.243 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:49 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:50.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:49 smithi064 ceph-mon[31180]: Adjusting osd_memory_target on smithi064 to 8544M 2024-09-21T03:57:50.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:49 smithi064 ceph-mon[31180]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:50.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:49 smithi064 ceph-mon[31180]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T03:57:50.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:49 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:57:50.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:49 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:57:50.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:49 smithi064 ceph-mon[31180]: from='client.24395 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:57:50.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:49 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:57:50.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:49 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:57:50.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:49 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:50.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:49 smithi131 ceph-mon[35193]: Adjusting osd_memory_target on smithi064 to 8544M 2024-09-21T03:57:50.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:49 smithi131 ceph-mon[35193]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:50.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:49 smithi131 ceph-mon[35193]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T03:57:50.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:49 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:57:50.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:49 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:57:50.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:49 smithi131 ceph-mon[35193]: from='client.24395 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:57:50.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:49 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:57:50.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:49 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:57:50.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:49 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:57:51.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:50 smithi119 ceph-mon[34811]: purged_snaps scrub starts 2024-09-21T03:57:51.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:50 smithi119 ceph-mon[34811]: purged_snaps scrub ok 2024-09-21T03:57:51.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:50 smithi119 ceph-mon[34811]: osd.3 [v2:172.21.15.64:6810/2022952729,v1:172.21.15.64:6811/2022952729] boot 2024-09-21T03:57:51.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:50 smithi119 ceph-mon[34811]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T03:57:51.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:50 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:57:51.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:50 smithi119 ceph-mon[34811]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T03:57:51.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:50 smithi064 ceph-mon[31180]: purged_snaps scrub starts 2024-09-21T03:57:51.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:50 smithi064 ceph-mon[31180]: purged_snaps scrub ok 2024-09-21T03:57:51.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:50 smithi064 ceph-mon[31180]: osd.3 [v2:172.21.15.64:6810/2022952729,v1:172.21.15.64:6811/2022952729] boot 2024-09-21T03:57:51.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:50 smithi064 ceph-mon[31180]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T03:57:51.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:50 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:57:51.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:50 smithi064 ceph-mon[31180]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T03:57:51.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:50 smithi131 ceph-mon[35193]: purged_snaps scrub starts 2024-09-21T03:57:51.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:50 smithi131 ceph-mon[35193]: purged_snaps scrub ok 2024-09-21T03:57:51.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:50 smithi131 ceph-mon[35193]: osd.3 [v2:172.21.15.64:6810/2022952729,v1:172.21.15.64:6811/2022952729] boot 2024-09-21T03:57:51.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:50 smithi131 ceph-mon[35193]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T03:57:51.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:50 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T03:57:51.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:50 smithi131 ceph-mon[35193]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T03:57:52.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:51 smithi119 ceph-mon[34811]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:52.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:51 smithi119 ceph-mon[34811]: from='client.? 172.21.15.119:0/1193833286' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "75b81a8d-fa52-411a-bfda-aa69bb70a20f"} : dispatch 2024-09-21T03:57:52.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:51 smithi119 ceph-mon[34811]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "75b81a8d-fa52-411a-bfda-aa69bb70a20f"} : dispatch 2024-09-21T03:57:52.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:51 smithi119 ceph-mon[34811]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "75b81a8d-fa52-411a-bfda-aa69bb70a20f"}]': finished 2024-09-21T03:57:52.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:51 smithi119 ceph-mon[34811]: osdmap e27: 5 total, 4 up, 5 in 2024-09-21T03:57:52.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:51 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:57:52.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:51 smithi119 ceph-mon[34811]: from='client.? 172.21.15.119:0/3942282984' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:57:52.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:51 smithi064 ceph-mon[31180]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:52.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:51 smithi064 ceph-mon[31180]: from='client.? 172.21.15.119:0/1193833286' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "75b81a8d-fa52-411a-bfda-aa69bb70a20f"} : dispatch 2024-09-21T03:57:52.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:51 smithi064 ceph-mon[31180]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "75b81a8d-fa52-411a-bfda-aa69bb70a20f"} : dispatch 2024-09-21T03:57:52.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:51 smithi064 ceph-mon[31180]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "75b81a8d-fa52-411a-bfda-aa69bb70a20f"}]': finished 2024-09-21T03:57:52.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:51 smithi064 ceph-mon[31180]: osdmap e27: 5 total, 4 up, 5 in 2024-09-21T03:57:52.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:51 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:57:52.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:51 smithi064 ceph-mon[31180]: from='client.? 172.21.15.119:0/3942282984' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:57:52.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:51 smithi131 ceph-mon[35193]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T03:57:52.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:51 smithi131 ceph-mon[35193]: from='client.? 172.21.15.119:0/1193833286' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "75b81a8d-fa52-411a-bfda-aa69bb70a20f"} : dispatch 2024-09-21T03:57:52.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:51 smithi131 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "75b81a8d-fa52-411a-bfda-aa69bb70a20f"} : dispatch 2024-09-21T03:57:52.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:51 smithi131 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "75b81a8d-fa52-411a-bfda-aa69bb70a20f"}]': finished 2024-09-21T03:57:52.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:51 smithi131 ceph-mon[35193]: osdmap e27: 5 total, 4 up, 5 in 2024-09-21T03:57:52.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:51 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:57:52.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:51 smithi131 ceph-mon[35193]: from='client.? 172.21.15.119:0/3942282984' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:57:54.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:53 smithi119 ceph-mon[34811]: pgmap v87: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:57:54.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:53 smithi064 ceph-mon[31180]: pgmap v87: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:57:54.306 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:53 smithi131 ceph-mon[35193]: pgmap v87: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:57:56.148 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:55 smithi119 ceph-mon[34811]: pgmap v88: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:57:56.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:55 smithi064 ceph-mon[31180]: pgmap v88: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:57:56.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:55 smithi131 ceph-mon[35193]: pgmap v88: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:57:58.157 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:57 smithi119 ceph-mon[34811]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-21T03:57:58.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:57 smithi064 ceph-mon[31180]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-21T03:57:58.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:57 smithi131 ceph-mon[35193]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-21T03:58:00.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:59 smithi119 ceph-mon[34811]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:58:00.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:59 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T03:58:00.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:57:59 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:00.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:59 smithi064 ceph-mon[31180]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:58:00.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:59 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T03:58:00.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:57:59 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:00.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:59 smithi131 ceph-mon[35193]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T03:58:00.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:59 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T03:58:00.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:57:59 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:01.206 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:00 smithi119 ceph-mon[34811]: Deploying daemon osd.4 on smithi119 2024-09-21T03:58:01.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:00 smithi064 ceph-mon[31180]: Deploying daemon osd.4 on smithi119 2024-09-21T03:58:01.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:00 smithi131 ceph-mon[35193]: Deploying daemon osd.4 on smithi119 2024-09-21T03:58:02.178 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:01 smithi119 ceph-mon[34811]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-21T03:58:02.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:01 smithi064 ceph-mon[31180]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-21T03:58:02.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:01 smithi131 ceph-mon[35193]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-21T03:58:04.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:03 smithi119 ceph-mon[34811]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-21T03:58:04.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:03 smithi064 ceph-mon[31180]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-21T03:58:04.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:03 smithi131 ceph-mon[35193]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-21T03:58:06.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:05 smithi119 ceph-mon[34811]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:58:06.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:05 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:58:06.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:05 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:06.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:05 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:06.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:05 smithi064 ceph-mon[31180]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:58:06.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:05 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:58:06.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:05 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:06.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:05 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:06.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:05 smithi131 ceph-mon[35193]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:58:06.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:05 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:58:06.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:05 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:06.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:05 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:08.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:07 smithi119 ceph-mon[34811]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:58:08.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:07 smithi064 ceph-mon[31180]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:58:08.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:07 smithi131 ceph-mon[35193]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:58:09.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:09 smithi119 ceph-mon[34811]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:58:09.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:09 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:09.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:09 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:09.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:09 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:09.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:09 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:58:09.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:09 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:09.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:09 smithi064 ceph-mon[31180]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:58:09.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:09 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:09.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:09 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:09.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:09 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:09.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:09 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:58:09.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:09 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:09.770 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 4 on host 'smithi119' 2024-09-21T03:58:09.770 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:58:09.768+0000 7fc1fe7fc640 1 -- 172.21.15.119:0/2361039769 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc20806aeb0 con 0x7fc1f40b6560 2024-09-21T03:58:09.772 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:58:09.770+0000 7fc20f46b640 1 -- 172.21.15.119:0/2361039769 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc1f40b6560 msgr2=0x7fc1f40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:09.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:58:09.770+0000 7fc20f46b640 1 --2- 172.21.15.119:0/2361039769 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc1f40b6560 0x7fc1f40b8a70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc204031440 tx=0x7fc204005e80 comp rx=0 tx=0).stop 2024-09-21T03:58:09.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:58:09.770+0000 7fc20f46b640 1 -- 172.21.15.119:0/2361039769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2081083a0 msgr2=0x7fc20819ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:09.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:58:09.770+0000 7fc20f46b640 1 --2- 172.21.15.119:0/2361039769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2081083a0 0x7fc20819ab40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc1f800b450 tx=0x7fc1f800b910 comp rx=0 tx=0).stop 2024-09-21T03:58:09.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:58:09.770+0000 7fc20f46b640 1 -- 172.21.15.119:0/2361039769 shutdown_connections 2024-09-21T03:58:09.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:58:09.770+0000 7fc20f46b640 1 --2- 172.21.15.119:0/2361039769 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc1f40b6560 0x7fc1f40b8a70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:09.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:58:09.770+0000 7fc20f46b640 1 --2- 172.21.15.119:0/2361039769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc208108cf0 0x7fc20819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:09.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:58:09.770+0000 7fc20f46b640 1 --2- 172.21.15.119:0/2361039769 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2081083a0 0x7fc20819ab40 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:09.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:58:09.770+0000 7fc20f46b640 1 --2- 172.21.15.119:0/2361039769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc208102a60 0x7fc20819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:09.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:58:09.770+0000 7fc20f46b640 1 -- 172.21.15.119:0/2361039769 >> 172.21.15.119:0/2361039769 conn(0x7fc208079730 msgr2=0x7fc208101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:09.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:58:09.770+0000 7fc20f46b640 1 -- 172.21.15.119:0/2361039769 shutdown_connections 2024-09-21T03:58:09.773 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:58:09.771+0000 7fc20f46b640 1 -- 172.21.15.119:0/2361039769 wait complete. 2024-09-21T03:58:09.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:09 smithi131 ceph-mon[35193]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:58:09.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:09 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:09.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:09 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:09.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:09 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:09.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:09 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:58:09.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:09 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:10.422 DEBUG:teuthology.orchestra.run.smithi119:osd.4> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.4.service 2024-09-21T03:58:10.424 INFO:tasks.cephadm:Deploying osd.5 on smithi131 with /dev/vg_nvme/lv_3... 2024-09-21T03:58:10.424 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T03:58:10.678 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:58:10.707 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:10 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:58:10.707 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:10 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:10.707 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:10 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:10.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:10 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:58:10.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:10 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:10.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:10 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:10.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:10 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:58:10.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:10 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:10.800 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:10 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:12.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:11 smithi119 ceph-mon[34811]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:58:12.006 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 03:58:11 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-4[46031]: 2024-09-21T03:58:11.875+0000 7fad8f417640 -1 osd.4 0 log_to_monitors true 2024-09-21T03:58:12.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:11 smithi131 ceph-mon[35193]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:58:12.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:11 smithi064 ceph-mon[31180]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:58:13.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:12 smithi131 ceph-mon[35193]: from='osd.4 [v2:172.21.15.119:6808/323030958,v1:172.21.15.119:6809/323030958]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:58:13.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:12 smithi131 ceph-mon[35193]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:58:13.126 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:12 smithi119 ceph-mon[34811]: from='osd.4 [v2:172.21.15.119:6808/323030958,v1:172.21.15.119:6809/323030958]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:58:13.126 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:12 smithi119 ceph-mon[34811]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:58:13.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:12 smithi064 ceph-mon[31180]: from='osd.4 [v2:172.21.15.119:6808/323030958,v1:172.21.15.119:6809/323030958]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:58:13.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:12 smithi064 ceph-mon[31180]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T03:58:13.342 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-21T03:58:13.369 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi131:vg_nvme/lv_3 2024-09-21T03:58:13.613 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:58:14.043 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:13 smithi131 ceph-mon[35193]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:58:14.043 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:13 smithi131 ceph-mon[35193]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T03:58:14.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:13 smithi131 ceph-mon[35193]: from='osd.4 [v2:172.21.15.119:6808/323030958,v1:172.21.15.119:6809/323030958]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T03:58:14.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:13 smithi131 ceph-mon[35193]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T03:58:14.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:13 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:58:14.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:13 smithi131 ceph-mon[35193]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T03:58:14.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:13 smithi131 ceph-mon[35193]: Detected new or changed devices on smithi119 2024-09-21T03:58:14.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:13 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:14.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:13 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:14.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:13 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:58:14.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:13 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:58:14.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:13 smithi131 ceph-mon[35193]: Adjusting osd_memory_target on smithi119 to 8544M 2024-09-21T03:58:14.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:13 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:14.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:13 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:14.044 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:13 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:58:14.045 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:13 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:14.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.058+0000 7fab85db5640 1 -- 172.21.15.131:0/2317507167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab80075730 msgr2=0x7fab80075b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:14.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.058+0000 7fab85db5640 1 --2- 172.21.15.131:0/2317507167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab80075730 0x7fab80075b40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fab70009900 tx=0x7fab7002a410 comp rx=0 tx=0).stop 2024-09-21T03:58:14.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.058+0000 7fab85db5640 1 -- 172.21.15.131:0/2317507167 shutdown_connections 2024-09-21T03:58:14.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.058+0000 7fab85db5640 1 --2- 172.21.15.131:0/2317507167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab80110140 0x7fab80112560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:14.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.058+0000 7fab85db5640 1 --2- 172.21.15.131:0/2317507167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab80076080 0x7fab8010fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:14.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.058+0000 7fab85db5640 1 --2- 172.21.15.131:0/2317507167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab80075730 0x7fab80075b40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:14.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.058+0000 7fab85db5640 1 -- 172.21.15.131:0/2317507167 >> 172.21.15.131:0/2317507167 conn(0x7fab8007b850 msgr2=0x7fab8007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:14.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.059+0000 7fab85db5640 1 -- 172.21.15.131:0/2317507167 shutdown_connections 2024-09-21T03:58:14.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.059+0000 7fab85db5640 1 -- 172.21.15.131:0/2317507167 wait complete. 2024-09-21T03:58:14.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.059+0000 7fab85db5640 1 Processor -- start 2024-09-21T03:58:14.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.059+0000 7fab85db5640 1 -- start start 2024-09-21T03:58:14.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.060+0000 7fab85db5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab80075730 0x7fab8019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:14.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.060+0000 7fab85db5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab80076080 0x7fab8019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:14.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.060+0000 7fab85db5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab80110140 0x7fab801a2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:14.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.060+0000 7fab85db5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fab80115d00 con 0x7fab80110140 2024-09-21T03:58:14.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.060+0000 7fab85db5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fab80115b80 con 0x7fab80075730 2024-09-21T03:58:14.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.060+0000 7fab85db5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fab80115e80 con 0x7fab80076080 2024-09-21T03:58:14.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.060+0000 7fab7effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab80076080 0x7fab8019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:14.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.060+0000 7fab7effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab80076080 0x7fab8019cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36510/0 (socket says 172.21.15.131:36510) 2024-09-21T03:58:14.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.060+0000 7fab7effd640 1 -- 172.21.15.131:0/116566389 learned_addr learned my addr 172.21.15.131:0/116566389 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-21T03:58:14.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.060+0000 7fab7f7fe640 1 --2- 172.21.15.131:0/116566389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab80075730 0x7fab8019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:14.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.060+0000 7fab7ffff640 1 --2- 172.21.15.131:0/116566389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab80110140 0x7fab801a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:14.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.060+0000 7fab7effd640 1 -- 172.21.15.131:0/116566389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab80075730 msgr2=0x7fab8019c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:14.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.060+0000 7fab7effd640 1 --2- 172.21.15.131:0/116566389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab80075730 0x7fab8019c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:14.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.060+0000 7fab7effd640 1 -- 172.21.15.131:0/116566389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab80110140 msgr2=0x7fab801a2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:14.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.060+0000 7fab7effd640 1 --2- 172.21.15.131:0/116566389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab80110140 0x7fab801a2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:14.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.060+0000 7fab7effd640 1 -- 172.21.15.131:0/116566389 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab801a2740 con 0x7fab80076080 2024-09-21T03:58:14.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.060+0000 7fab7f7fe640 1 --2- 172.21.15.131:0/116566389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab80075730 0x7fab8019c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:58:14.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.060+0000 7fab7ffff640 1 --2- 172.21.15.131:0/116566389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab80110140 0x7fab801a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:58:14.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.060+0000 7fab7effd640 1 --2- 172.21.15.131:0/116566389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab80076080 0x7fab8019cd70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fab7400e9f0 tx=0x7fab7400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:14.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.061+0000 7fab7cff9640 1 -- 172.21.15.131:0/116566389 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7400ccc0 con 0x7fab80076080 2024-09-21T03:58:14.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.061+0000 7fab85db5640 1 -- 172.21.15.131:0/116566389 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab801a2a30 con 0x7fab80076080 2024-09-21T03:58:14.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.061+0000 7fab7cff9640 1 -- 172.21.15.131:0/116566389 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab740108e0 con 0x7fab80076080 2024-09-21T03:58:14.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.061+0000 7fab7cff9640 1 -- 172.21.15.131:0/116566389 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab74018860 con 0x7fab80076080 2024-09-21T03:58:14.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.061+0000 7fab85db5640 1 -- 172.21.15.131:0/116566389 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab801a9190 con 0x7fab80076080 2024-09-21T03:58:14.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.062+0000 7fab85db5640 1 -- 172.21.15.131:0/116566389 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab80075b40 con 0x7fab80076080 2024-09-21T03:58:14.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.063+0000 7fab7cff9640 1 -- 172.21.15.131:0/116566389 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fab74010430 con 0x7fab80076080 2024-09-21T03:58:14.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.064+0000 7fab7cff9640 1 --2- 172.21.15.131:0/116566389 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fab680b6560 0x7fab680b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:14.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.064+0000 7fab7cff9640 1 -- 172.21.15.131:0/116566389 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4036+0+0 (secure 0 0 0) 0x7fab740e66a0 con 0x7fab80076080 2024-09-21T03:58:14.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.064+0000 7fab7f7fe640 1 --2- 172.21.15.131:0/116566389 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fab680b6560 0x7fab680b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:14.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.065+0000 7fab7f7fe640 1 --2- 172.21.15.131:0/116566389 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fab680b6560 0x7fab680b8a70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fab70031440 tx=0x7fab70009680 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:14.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.069+0000 7fab7cff9640 1 -- 172.21.15.131:0/116566389 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fab740adfe0 con 0x7fab80076080 2024-09-21T03:58:14.250 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 03:58:13 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-4[46031]: 2024-09-21T03:58:13.782+0000 7fad8b399640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T03:58:14.250 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 03:58:13 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-4[46031]: 2024-09-21T03:58:13.790+0000 7fad87197640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:58:14.250 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:13 smithi119 ceph-mon[34811]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:58:14.250 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:13 smithi119 ceph-mon[34811]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T03:58:14.251 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:13 smithi119 ceph-mon[34811]: from='osd.4 [v2:172.21.15.119:6808/323030958,v1:172.21.15.119:6809/323030958]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T03:58:14.251 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:13 smithi119 ceph-mon[34811]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T03:58:14.251 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:13 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:58:14.251 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:13 smithi119 ceph-mon[34811]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T03:58:14.251 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:13 smithi119 ceph-mon[34811]: Detected new or changed devices on smithi119 2024-09-21T03:58:14.251 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:13 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:14.251 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:13 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:14.251 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:13 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:58:14.251 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:13 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:58:14.251 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:13 smithi119 ceph-mon[34811]: Adjusting osd_memory_target on smithi119 to 8544M 2024-09-21T03:58:14.251 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:13 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:14.251 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:13 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:14.251 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:13 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:58:14.252 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:13 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:14.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:14.254+0000 7fab85db5640 1 -- 172.21.15.131:0/116566389 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fab80063800 con 0x7fab680b6560 2024-09-21T03:58:14.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:13 smithi064 ceph-mon[31180]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:58:14.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:13 smithi064 ceph-mon[31180]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T03:58:14.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:13 smithi064 ceph-mon[31180]: from='osd.4 [v2:172.21.15.119:6808/323030958,v1:172.21.15.119:6809/323030958]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T03:58:14.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:13 smithi064 ceph-mon[31180]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T03:58:14.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:13 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:58:14.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:13 smithi064 ceph-mon[31180]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T03:58:14.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:13 smithi064 ceph-mon[31180]: Detected new or changed devices on smithi119 2024-09-21T03:58:14.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:13 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:14.263 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:13 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:14.263 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:13 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:58:14.263 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:13 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:58:14.263 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:13 smithi064 ceph-mon[31180]: Adjusting osd_memory_target on smithi119 to 8544M 2024-09-21T03:58:14.263 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:13 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:14.263 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:13 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:14.263 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:13 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:58:14.263 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:13 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:14.962 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:14 smithi131 ceph-mon[35193]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-09-21T03:58:14.963 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:14 smithi131 ceph-mon[35193]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T03:58:14.963 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:14 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:58:14.963 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:14 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:58:14.963 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:14 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:58:14.963 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:14 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:58:14.963 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:14 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:14.963 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:14 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:58:15.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:14 smithi119 ceph-mon[34811]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-09-21T03:58:15.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:14 smithi119 ceph-mon[34811]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T03:58:15.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:14 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:58:15.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:14 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:58:15.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:14 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:58:15.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:14 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:58:15.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:14 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:15.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:14 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:58:15.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:14 smithi064 ceph-mon[31180]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-09-21T03:58:15.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:14 smithi064 ceph-mon[31180]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T03:58:15.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:14 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:58:15.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:14 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:58:15.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:14 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:58:15.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:14 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:58:15.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:14 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:15.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:14 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:58:15.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:15 smithi131 ceph-mon[35193]: purged_snaps scrub starts 2024-09-21T03:58:15.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:15 smithi131 ceph-mon[35193]: purged_snaps scrub ok 2024-09-21T03:58:15.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:15 smithi131 ceph-mon[35193]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:58:15.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:15 smithi131 ceph-mon[35193]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:58:15.895 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:15 smithi131 ceph-mon[35193]: osd.4 [v2:172.21.15.119:6808/323030958,v1:172.21.15.119:6809/323030958] boot 2024-09-21T03:58:15.896 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:15 smithi131 ceph-mon[35193]: osdmap e30: 5 total, 5 up, 5 in 2024-09-21T03:58:15.896 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:15 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:58:16.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:15 smithi119 ceph-mon[34811]: purged_snaps scrub starts 2024-09-21T03:58:16.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:15 smithi119 ceph-mon[34811]: purged_snaps scrub ok 2024-09-21T03:58:16.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:15 smithi119 ceph-mon[34811]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:58:16.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:15 smithi119 ceph-mon[34811]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:58:16.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:15 smithi119 ceph-mon[34811]: osd.4 [v2:172.21.15.119:6808/323030958,v1:172.21.15.119:6809/323030958] boot 2024-09-21T03:58:16.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:15 smithi119 ceph-mon[34811]: osdmap e30: 5 total, 5 up, 5 in 2024-09-21T03:58:16.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:15 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:58:16.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:15 smithi064 ceph-mon[31180]: purged_snaps scrub starts 2024-09-21T03:58:16.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:15 smithi064 ceph-mon[31180]: purged_snaps scrub ok 2024-09-21T03:58:16.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:15 smithi064 ceph-mon[31180]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:58:16.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:15 smithi064 ceph-mon[31180]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-21T03:58:16.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:15 smithi064 ceph-mon[31180]: osd.4 [v2:172.21.15.119:6808/323030958,v1:172.21.15.119:6809/323030958] boot 2024-09-21T03:58:16.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:15 smithi064 ceph-mon[31180]: osdmap e30: 5 total, 5 up, 5 in 2024-09-21T03:58:16.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:15 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T03:58:17.238 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:16 smithi131 ceph-mon[35193]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T03:58:17.238 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:16 smithi131 ceph-mon[35193]: from='client.? 172.21.15.131:0/3060236189' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0e4895b5-42e4-4b2c-a9c3-692f287cb841"} : dispatch 2024-09-21T03:58:17.238 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:16 smithi131 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0e4895b5-42e4-4b2c-a9c3-692f287cb841"} : dispatch 2024-09-21T03:58:17.238 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:16 smithi131 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e4895b5-42e4-4b2c-a9c3-692f287cb841"}]': finished 2024-09-21T03:58:17.238 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:16 smithi131 ceph-mon[35193]: osdmap e32: 6 total, 5 up, 6 in 2024-09-21T03:58:17.239 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:16 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:58:17.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:16 smithi119 ceph-mon[34811]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T03:58:17.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:16 smithi119 ceph-mon[34811]: from='client.? 172.21.15.131:0/3060236189' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0e4895b5-42e4-4b2c-a9c3-692f287cb841"} : dispatch 2024-09-21T03:58:17.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:16 smithi119 ceph-mon[34811]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0e4895b5-42e4-4b2c-a9c3-692f287cb841"} : dispatch 2024-09-21T03:58:17.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:16 smithi119 ceph-mon[34811]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e4895b5-42e4-4b2c-a9c3-692f287cb841"}]': finished 2024-09-21T03:58:17.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:16 smithi119 ceph-mon[34811]: osdmap e32: 6 total, 5 up, 6 in 2024-09-21T03:58:17.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:16 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:58:17.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:16 smithi064 ceph-mon[31180]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T03:58:17.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:16 smithi064 ceph-mon[31180]: from='client.? 172.21.15.131:0/3060236189' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0e4895b5-42e4-4b2c-a9c3-692f287cb841"} : dispatch 2024-09-21T03:58:17.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:16 smithi064 ceph-mon[31180]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0e4895b5-42e4-4b2c-a9c3-692f287cb841"} : dispatch 2024-09-21T03:58:17.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:16 smithi064 ceph-mon[31180]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e4895b5-42e4-4b2c-a9c3-692f287cb841"}]': finished 2024-09-21T03:58:17.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:16 smithi064 ceph-mon[31180]: osdmap e32: 6 total, 5 up, 6 in 2024-09-21T03:58:17.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:16 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:58:18.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:17 smithi119 ceph-mon[34811]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:18.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:17 smithi119 ceph-mon[34811]: from='client.? 172.21.15.131:0/4003574386' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:58:18.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:17 smithi064 ceph-mon[31180]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:18.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:17 smithi064 ceph-mon[31180]: from='client.? 172.21.15.131:0/4003574386' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:58:18.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:17 smithi131 ceph-mon[35193]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:18.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:17 smithi131 ceph-mon[35193]: from='client.? 172.21.15.131:0/4003574386' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:58:20.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:19 smithi119 ceph-mon[34811]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:20.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:19 smithi064 ceph-mon[31180]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:20.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:19 smithi131 ceph-mon[35193]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:21.918 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:21 smithi131 ceph-mon[35193]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:22.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:21 smithi119 ceph-mon[34811]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:22.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:21 smithi064 ceph-mon[31180]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:24.181 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:23 smithi131 ceph-mon[35193]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:24.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:23 smithi119 ceph-mon[34811]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:24.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:23 smithi064 ceph-mon[31180]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:26.198 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:25 smithi131 ceph-mon[35193]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:26.198 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:25 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T03:58:26.198 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:25 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:26.198 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:25 smithi131 ceph-mon[35193]: Deploying daemon osd.5 on smithi131 2024-09-21T03:58:26.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:25 smithi119 ceph-mon[34811]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:26.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:25 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T03:58:26.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:25 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:26.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:25 smithi119 ceph-mon[34811]: Deploying daemon osd.5 on smithi131 2024-09-21T03:58:26.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:25 smithi064 ceph-mon[31180]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:26.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:25 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T03:58:26.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:25 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:26.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:25 smithi064 ceph-mon[31180]: Deploying daemon osd.5 on smithi131 2024-09-21T03:58:28.091 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:27 smithi131 ceph-mon[35193]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:28.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:27 smithi119 ceph-mon[34811]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:28.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:27 smithi064 ceph-mon[31180]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:30.171 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:29 smithi131 ceph-mon[35193]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:30.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:29 smithi119 ceph-mon[34811]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:30.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:29 smithi064 ceph-mon[31180]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:32.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:31 smithi131 ceph-mon[35193]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:32.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:31 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:58:32.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:31 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:32.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:31 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:32.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:31 smithi119 ceph-mon[34811]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:32.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:31 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:58:32.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:31 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:32.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:31 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:32.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:31 smithi064 ceph-mon[31180]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:32.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:31 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:58:32.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:31 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:32.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:31 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:34.222 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:33 smithi131 ceph-mon[35193]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:34.222 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:33 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:34.222 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:33 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:34.222 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:33 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:34.222 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:33 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:58:34.222 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:33 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:34.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:33 smithi119 ceph-mon[34811]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:34.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:33 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:34.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:33 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:34.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:33 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:34.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:33 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:58:34.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:33 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:34.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:33 smithi064 ceph-mon[31180]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:34.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:33 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:34.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:33 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:34.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:33 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:34.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:33 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:58:34.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:33 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:34.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:34.843+0000 7fab7cff9640 1 -- 172.21.15.131:0/116566389 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fab80063800 con 0x7fab680b6560 2024-09-21T03:58:34.845 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 5 on host 'smithi131' 2024-09-21T03:58:34.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:34.845+0000 7fab85db5640 1 -- 172.21.15.131:0/116566389 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fab680b6560 msgr2=0x7fab680b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:34.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:34.845+0000 7fab85db5640 1 --2- 172.21.15.131:0/116566389 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fab680b6560 0x7fab680b8a70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fab70031440 tx=0x7fab70009680 comp rx=0 tx=0).stop 2024-09-21T03:58:34.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:34.845+0000 7fab85db5640 1 -- 172.21.15.131:0/116566389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab80076080 msgr2=0x7fab8019cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:34.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:34.845+0000 7fab85db5640 1 --2- 172.21.15.131:0/116566389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab80076080 0x7fab8019cd70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fab7400e9f0 tx=0x7fab7400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:58:34.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:34.845+0000 7fab85db5640 1 -- 172.21.15.131:0/116566389 shutdown_connections 2024-09-21T03:58:34.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:34.845+0000 7fab85db5640 1 --2- 172.21.15.131:0/116566389 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fab680b6560 0x7fab680b8a70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:34.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:34.845+0000 7fab85db5640 1 --2- 172.21.15.131:0/116566389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab80110140 0x7fab801a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:34.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:34.845+0000 7fab85db5640 1 --2- 172.21.15.131:0/116566389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fab80076080 0x7fab8019cd70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:34.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:34.845+0000 7fab85db5640 1 --2- 172.21.15.131:0/116566389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab80075730 0x7fab8019c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:34.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:34.845+0000 7fab85db5640 1 -- 172.21.15.131:0/116566389 >> 172.21.15.131:0/116566389 conn(0x7fab8007b850 msgr2=0x7fab801019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:34.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:34.846+0000 7fab85db5640 1 -- 172.21.15.131:0/116566389 shutdown_connections 2024-09-21T03:58:34.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:58:34.846+0000 7fab85db5640 1 -- 172.21.15.131:0/116566389 wait complete. 2024-09-21T03:58:35.157 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:34 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:58:35.157 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:34 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:35.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:34 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:58:35.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:34 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:35.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:34 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:58:35.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:34 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:35.485 DEBUG:teuthology.orchestra.run.smithi131:osd.5> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.5.service 2024-09-21T03:58:35.487 INFO:tasks.cephadm:Deploying osd.6 on smithi064 with /dev/vg_nvme/lv_2... 2024-09-21T03:58:35.487 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T03:58:35.755 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:58:36.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:35 smithi131 ceph-mon[35193]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:36.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:35 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:36.127 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:35 smithi064 ceph-mon[31180]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:36.127 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:35 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:36.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:35 smithi119 ceph-mon[34811]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:36.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:35 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:37.153 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 03:58:36 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T03:58:36.869+0000 7fb73aee7640 -1 osd.5 0 log_to_monitors true 2024-09-21T03:58:37.931 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:37 smithi131 ceph-mon[35193]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:37.932 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:37 smithi131 ceph-mon[35193]: from='osd.5 [v2:172.21.15.131:6808/3021788088,v1:172.21.15.131:6809/3021788088]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:58:37.932 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:37 smithi131 ceph-mon[35193]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:58:38.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:37 smithi064 ceph-mon[31180]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:38.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:37 smithi064 ceph-mon[31180]: from='osd.5 [v2:172.21.15.131:6808/3021788088,v1:172.21.15.131:6809/3021788088]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:58:38.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:37 smithi064 ceph-mon[31180]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:58:38.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:37 smithi119 ceph-mon[34811]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:38.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:37 smithi119 ceph-mon[34811]: from='osd.5 [v2:172.21.15.131:6808/3021788088,v1:172.21.15.131:6809/3021788088]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:58:38.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:37 smithi119 ceph-mon[34811]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T03:58:38.449 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:58:38.476 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi064:vg_nvme/lv_2 2024-09-21T03:58:38.729 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:58:39.091 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:38 smithi064 ceph-mon[31180]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T03:58:39.091 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:38 smithi064 ceph-mon[31180]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T03:58:39.091 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:38 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:58:39.091 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:38 smithi064 ceph-mon[31180]: from='osd.5 [v2:172.21.15.131:6808/3021788088,v1:172.21.15.131:6809/3021788088]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi131", "root=default"]} : dispatch 2024-09-21T03:58:39.092 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:38 smithi064 ceph-mon[31180]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi131", "root=default"]} : dispatch 2024-09-21T03:58:39.092 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:38 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:39.092 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:38 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:39.092 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:38 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:58:39.092 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:38 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:58:39.092 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:38 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:39.092 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:38 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:39.092 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:38 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:58:39.092 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:38 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:39.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.185+0000 7fe6455e4640 1 -- 172.21.15.64:0/2572684866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe640105270 msgr2=0x7fe64010d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:39.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.185+0000 7fe6455e4640 1 --2- 172.21.15.64:0/2572684866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe640105270 0x7fe64010d8b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe628009900 tx=0x7fe62802a410 comp rx=0 tx=0).stop 2024-09-21T03:58:39.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.185+0000 7fe6455e4640 1 -- 172.21.15.64:0/2572684866 shutdown_connections 2024-09-21T03:58:39.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.185+0000 7fe6455e4640 1 --2- 172.21.15.64:0/2572684866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe64010ddf0 0x7fe640110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:39.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.185+0000 7fe6455e4640 1 --2- 172.21.15.64:0/2572684866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe640105270 0x7fe64010d8b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:39.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.185+0000 7fe6455e4640 1 --2- 172.21.15.64:0/2572684866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe640104840 0x7fe640104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:39.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.185+0000 7fe6455e4640 1 -- 172.21.15.64:0/2572684866 >> 172.21.15.64:0/2572684866 conn(0x7fe640079730 msgr2=0x7fe640079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:58:39.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.185+0000 7fe6455e4640 1 -- 172.21.15.64:0/2572684866 shutdown_connections 2024-09-21T03:58:39.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.185+0000 7fe6455e4640 1 -- 172.21.15.64:0/2572684866 wait complete. 2024-09-21T03:58:39.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.186+0000 7fe6455e4640 1 Processor -- start 2024-09-21T03:58:39.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.186+0000 7fe6455e4640 1 -- start start 2024-09-21T03:58:39.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.186+0000 7fe6455e4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe640104840 0x7fe64019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:39.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.186+0000 7fe6455e4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe640105270 0x7fe64019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:39.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.186+0000 7fe6455e4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe64010ddf0 0x7fe64019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:39.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.187+0000 7fe6455e4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe640113a70 con 0x7fe640105270 2024-09-21T03:58:39.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.187+0000 7fe6455e4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe6401138f0 con 0x7fe64010ddf0 2024-09-21T03:58:39.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.187+0000 7fe6455e4640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe640113bf0 con 0x7fe640104840 2024-09-21T03:58:39.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.187+0000 7fe63e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe640105270 0x7fe64019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:39.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.187+0000 7fe63e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe640105270 0x7fe64019aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:40076/0 (socket says 172.21.15.64:40076) 2024-09-21T03:58:39.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.187+0000 7fe63e7fc640 1 -- 172.21.15.64:0/1716716035 learned_addr learned my addr 172.21.15.64:0/1716716035 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:58:39.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.187+0000 7fe63effd640 1 --2- 172.21.15.64:0/1716716035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe640104840 0x7fe64019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:39.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.187+0000 7fe63f7fe640 1 --2- 172.21.15.64:0/1716716035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe64010ddf0 0x7fe64019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:39.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.187+0000 7fe63e7fc640 1 -- 172.21.15.64:0/1716716035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe640104840 msgr2=0x7fe64019a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:39.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.187+0000 7fe63e7fc640 1 --2- 172.21.15.64:0/1716716035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe640104840 0x7fe64019a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:39.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.187+0000 7fe63e7fc640 1 -- 172.21.15.64:0/1716716035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe64010ddf0 msgr2=0x7fe64019fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:58:39.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.187+0000 7fe63e7fc640 1 --2- 172.21.15.64:0/1716716035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe64010ddf0 0x7fe64019fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:58:39.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.187+0000 7fe63e7fc640 1 -- 172.21.15.64:0/1716716035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6401a0470 con 0x7fe640105270 2024-09-21T03:58:39.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.187+0000 7fe63e7fc640 1 --2- 172.21.15.64:0/1716716035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe640105270 0x7fe64019aaa0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe62802a930 tx=0x7fe628004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:39.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.187+0000 7fe637fff640 1 -- 172.21.15.64:0/1716716035 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62803f030 con 0x7fe640105270 2024-09-21T03:58:39.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.187+0000 7fe637fff640 1 -- 172.21.15.64:0/1716716035 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe628003aa0 con 0x7fe640105270 2024-09-21T03:58:39.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.188+0000 7fe6455e4640 1 -- 172.21.15.64:0/1716716035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6401a0700 con 0x7fe640105270 2024-09-21T03:58:39.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.188+0000 7fe637fff640 1 -- 172.21.15.64:0/1716716035 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe628035070 con 0x7fe640105270 2024-09-21T03:58:39.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.188+0000 7fe6455e4640 1 -- 172.21.15.64:0/1716716035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6401a6ec0 con 0x7fe640105270 2024-09-21T03:58:39.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.189+0000 7fe6455e4640 1 -- 172.21.15.64:0/1716716035 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe640104c50 con 0x7fe640105270 2024-09-21T03:58:39.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.190+0000 7fe637fff640 1 -- 172.21.15.64:0/1716716035 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fe628042030 con 0x7fe640105270 2024-09-21T03:58:39.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.192+0000 7fe637fff640 1 --2- 172.21.15.64:0/1716716035 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe62c0b65e0 0x7fe62c0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:58:39.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.192+0000 7fe63effd640 1 --2- 172.21.15.64:0/1716716035 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe62c0b65e0 0x7fe62c0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:58:39.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.192+0000 7fe637fff640 1 -- 172.21.15.64:0/1716716035 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4468+0+0 (secure 0 0 0) 0x7fe628106d50 con 0x7fe640105270 2024-09-21T03:58:39.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.192+0000 7fe63effd640 1 --2- 172.21.15.64:0/1716716035 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe62c0b65e0 0x7fe62c0b8af0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe64006bc00 tx=0x7fe63800b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:58:39.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.195+0000 7fe637fff640 1 -- 172.21.15.64:0/1716716035 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe6280d28c0 con 0x7fe640105270 2024-09-21T03:58:39.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:38 smithi119 ceph-mon[34811]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T03:58:39.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:38 smithi119 ceph-mon[34811]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T03:58:39.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:38 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:58:39.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:38 smithi119 ceph-mon[34811]: from='osd.5 [v2:172.21.15.131:6808/3021788088,v1:172.21.15.131:6809/3021788088]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi131", "root=default"]} : dispatch 2024-09-21T03:58:39.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:38 smithi119 ceph-mon[34811]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi131", "root=default"]} : dispatch 2024-09-21T03:58:39.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:38 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:39.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:38 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:39.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:38 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:58:39.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:38 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:58:39.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:38 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:39.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:38 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:39.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:38 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:58:39.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:38 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:39.303 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 03:58:38 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T03:58:38.867+0000 7fb736e69640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T03:58:39.303 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 03:58:38 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T03:58:38.874+0000 7fb732c67640 -1 osd.5 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:58:39.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:38 smithi131 ceph-mon[35193]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T03:58:39.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:38 smithi131 ceph-mon[35193]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T03:58:39.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:38 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:58:39.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:38 smithi131 ceph-mon[35193]: from='osd.5 [v2:172.21.15.131:6808/3021788088,v1:172.21.15.131:6809/3021788088]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi131", "root=default"]} : dispatch 2024-09-21T03:58:39.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:38 smithi131 ceph-mon[35193]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi131", "root=default"]} : dispatch 2024-09-21T03:58:39.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:38 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:39.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:38 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:39.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:38 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:58:39.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:38 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:58:39.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:38 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:39.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:38 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:39.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:38 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:58:39.305 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:38 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:39.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:58:39.395+0000 7fe6455e4640 1 -- 172.21.15.64:0/1716716035 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe640063800 con 0x7fe62c0b65e0 2024-09-21T03:58:40.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:39 smithi064 ceph-mon[31180]: Detected new or changed devices on smithi131 2024-09-21T03:58:40.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:39 smithi064 ceph-mon[31180]: Adjusting osd_memory_target on smithi131 to 8544M 2024-09-21T03:58:40.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:39 smithi064 ceph-mon[31180]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:40.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:39 smithi064 ceph-mon[31180]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-21T03:58:40.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:39 smithi064 ceph-mon[31180]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T03:58:40.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:39 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:58:40.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:39 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:58:40.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:39 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:58:40.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:39 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:58:40.111 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:39 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:40.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:39 smithi119 ceph-mon[34811]: Detected new or changed devices on smithi131 2024-09-21T03:58:40.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:39 smithi119 ceph-mon[34811]: Adjusting osd_memory_target on smithi131 to 8544M 2024-09-21T03:58:40.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:39 smithi119 ceph-mon[34811]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:40.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:39 smithi119 ceph-mon[34811]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-21T03:58:40.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:39 smithi119 ceph-mon[34811]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T03:58:40.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:39 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:58:40.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:39 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:58:40.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:39 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:58:40.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:39 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:58:40.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:39 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:40.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:39 smithi131 ceph-mon[35193]: Detected new or changed devices on smithi131 2024-09-21T03:58:40.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:39 smithi131 ceph-mon[35193]: Adjusting osd_memory_target on smithi131 to 8544M 2024-09-21T03:58:40.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:39 smithi131 ceph-mon[35193]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:40.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:39 smithi131 ceph-mon[35193]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-21T03:58:40.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:39 smithi131 ceph-mon[35193]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T03:58:40.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:39 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:58:40.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:39 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:58:40.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:39 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:58:40.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:39 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:58:40.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:39 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:41.166 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:40 smithi064 ceph-mon[31180]: purged_snaps scrub starts 2024-09-21T03:58:41.166 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:40 smithi064 ceph-mon[31180]: purged_snaps scrub ok 2024-09-21T03:58:41.166 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:40 smithi064 ceph-mon[31180]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:58:41.166 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:40 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:58:41.166 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:40 smithi064 ceph-mon[31180]: osd.5 [v2:172.21.15.131:6808/3021788088,v1:172.21.15.131:6809/3021788088] boot 2024-09-21T03:58:41.167 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:40 smithi064 ceph-mon[31180]: osdmap e35: 6 total, 6 up, 6 in 2024-09-21T03:58:41.167 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:40 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:58:41.204 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:40 smithi131 ceph-mon[35193]: purged_snaps scrub starts 2024-09-21T03:58:41.204 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:40 smithi131 ceph-mon[35193]: purged_snaps scrub ok 2024-09-21T03:58:41.205 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:40 smithi131 ceph-mon[35193]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:58:41.205 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:40 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:58:41.205 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:40 smithi131 ceph-mon[35193]: osd.5 [v2:172.21.15.131:6808/3021788088,v1:172.21.15.131:6809/3021788088] boot 2024-09-21T03:58:41.205 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:40 smithi131 ceph-mon[35193]: osdmap e35: 6 total, 6 up, 6 in 2024-09-21T03:58:41.205 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:40 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:58:41.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:40 smithi119 ceph-mon[34811]: purged_snaps scrub starts 2024-09-21T03:58:41.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:40 smithi119 ceph-mon[34811]: purged_snaps scrub ok 2024-09-21T03:58:41.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:40 smithi119 ceph-mon[34811]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:58:41.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:40 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:58:41.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:40 smithi119 ceph-mon[34811]: osd.5 [v2:172.21.15.131:6808/3021788088,v1:172.21.15.131:6809/3021788088] boot 2024-09-21T03:58:41.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:40 smithi119 ceph-mon[34811]: osdmap e35: 6 total, 6 up, 6 in 2024-09-21T03:58:41.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:40 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T03:58:42.149 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:41 smithi064 ceph-mon[31180]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:42.149 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:41 smithi064 ceph-mon[31180]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T03:58:42.220 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:41 smithi131 ceph-mon[35193]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:42.220 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:41 smithi131 ceph-mon[35193]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T03:58:42.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:41 smithi119 ceph-mon[34811]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-21T03:58:42.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:41 smithi119 ceph-mon[34811]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T03:58:43.157 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:42 smithi119 ceph-mon[34811]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T03:58:43.157 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:42 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1529556884' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1fa5c2b7-bce0-43a1-a26b-d029eb3731fd"} : dispatch 2024-09-21T03:58:43.157 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:42 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1529556884' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fa5c2b7-bce0-43a1-a26b-d029eb3731fd"}]': finished 2024-09-21T03:58:43.157 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:42 smithi119 ceph-mon[34811]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T03:58:43.158 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:42 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:58:43.158 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:42 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2891094254' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:58:43.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:42 smithi064 ceph-mon[31180]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T03:58:43.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:42 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1529556884' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1fa5c2b7-bce0-43a1-a26b-d029eb3731fd"} : dispatch 2024-09-21T03:58:43.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:42 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1529556884' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fa5c2b7-bce0-43a1-a26b-d029eb3731fd"}]': finished 2024-09-21T03:58:43.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:42 smithi064 ceph-mon[31180]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T03:58:43.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:42 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:58:43.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:42 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2891094254' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:58:43.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:42 smithi131 ceph-mon[35193]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T03:58:43.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:42 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1529556884' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1fa5c2b7-bce0-43a1-a26b-d029eb3731fd"} : dispatch 2024-09-21T03:58:43.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:42 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1529556884' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fa5c2b7-bce0-43a1-a26b-d029eb3731fd"}]': finished 2024-09-21T03:58:43.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:42 smithi131 ceph-mon[35193]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T03:58:43.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:42 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:58:43.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:42 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2891094254' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:58:44.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:43 smithi119 ceph-mon[34811]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:58:44.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:43 smithi064 ceph-mon[31180]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:58:44.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:43 smithi131 ceph-mon[35193]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:58:46.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:45 smithi119 ceph-mon[34811]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:58:46.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:45 smithi064 ceph-mon[31180]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:58:46.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:45 smithi131 ceph-mon[35193]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:58:48.255 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:47 smithi064 ceph-mon[31180]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:58:48.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:47 smithi119 ceph-mon[34811]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:58:48.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:47 smithi131 ceph-mon[35193]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:58:50.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:49 smithi119 ceph-mon[34811]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-21T03:58:50.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:49 smithi064 ceph-mon[31180]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-21T03:58:50.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:49 smithi131 ceph-mon[35193]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-21T03:58:51.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:50 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T03:58:51.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:50 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:51.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:50 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T03:58:51.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:50 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:51.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:50 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T03:58:51.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:50 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:58:52.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:51 smithi119 ceph-mon[34811]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-21T03:58:52.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:51 smithi119 ceph-mon[34811]: Deploying daemon osd.6 on smithi064 2024-09-21T03:58:52.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:51 smithi064 ceph-mon[31180]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-21T03:58:52.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:51 smithi064 ceph-mon[31180]: Deploying daemon osd.6 on smithi064 2024-09-21T03:58:52.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:51 smithi131 ceph-mon[35193]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-21T03:58:52.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:51 smithi131 ceph-mon[35193]: Deploying daemon osd.6 on smithi064 2024-09-21T03:58:54.177 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:53 smithi064 ceph-mon[31180]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-21T03:58:54.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:53 smithi119 ceph-mon[34811]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-21T03:58:54.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:53 smithi131 ceph-mon[35193]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-21T03:58:56.165 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:55 smithi064 ceph-mon[31180]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:58:56.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:55 smithi119 ceph-mon[34811]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:58:56.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:55 smithi131 ceph-mon[35193]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:58:57.207 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:56 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:58:57.207 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:56 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:57.207 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:56 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:57.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:56 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:58:57.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:56 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:57.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:56 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:57.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:56 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:58:57.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:56 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:57.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:56 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:58:58.247 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:57 smithi064 ceph-mon[31180]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:58:58.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:57 smithi119 ceph-mon[34811]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:58:58.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:57 smithi131 ceph-mon[35193]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T03:59:00.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:59 smithi119 ceph-mon[34811]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:59:00.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:59 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:00.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:59 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:00.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:59 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:00.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:59 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:59:00.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:58:59 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:00.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:59 smithi064 ceph-mon[31180]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:59:00.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:59 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:00.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:59 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:00.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:59 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:00.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:59 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:59:00.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:58:59 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:00.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:59 smithi131 ceph-mon[35193]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:59:00.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:59 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:00.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:59 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:00.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:59 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:00.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:59 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:59:00.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:58:59 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:00.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:00.785+0000 7fe637fff640 1 -- 172.21.15.64:0/1716716035 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe640063800 con 0x7fe62c0b65e0 2024-09-21T03:59:00.788 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 6 on host 'smithi064' 2024-09-21T03:59:00.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:00.788+0000 7fe6455e4640 1 -- 172.21.15.64:0/1716716035 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe62c0b65e0 msgr2=0x7fe62c0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:59:00.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:00.788+0000 7fe6455e4640 1 --2- 172.21.15.64:0/1716716035 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe62c0b65e0 0x7fe62c0b8af0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe64006bc00 tx=0x7fe63800b390 comp rx=0 tx=0).stop 2024-09-21T03:59:00.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:00.788+0000 7fe6455e4640 1 -- 172.21.15.64:0/1716716035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe640105270 msgr2=0x7fe64019aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:59:00.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:00.788+0000 7fe6455e4640 1 --2- 172.21.15.64:0/1716716035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe640105270 0x7fe64019aaa0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe62802a930 tx=0x7fe628004270 comp rx=0 tx=0).stop 2024-09-21T03:59:00.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:00.788+0000 7fe6455e4640 1 -- 172.21.15.64:0/1716716035 shutdown_connections 2024-09-21T03:59:00.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:00.788+0000 7fe6455e4640 1 --2- 172.21.15.64:0/1716716035 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe62c0b65e0 0x7fe62c0b8af0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:59:00.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:00.788+0000 7fe6455e4640 1 --2- 172.21.15.64:0/1716716035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe64010ddf0 0x7fe64019fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:59:00.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:00.788+0000 7fe6455e4640 1 --2- 172.21.15.64:0/1716716035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe640105270 0x7fe64019aaa0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:59:00.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:00.788+0000 7fe6455e4640 1 --2- 172.21.15.64:0/1716716035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe640104840 0x7fe64019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:59:00.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:00.788+0000 7fe6455e4640 1 -- 172.21.15.64:0/1716716035 >> 172.21.15.64:0/1716716035 conn(0x7fe640079730 msgr2=0x7fe640100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:59:00.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:00.789+0000 7fe6455e4640 1 -- 172.21.15.64:0/1716716035 shutdown_connections 2024-09-21T03:59:00.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:00.789+0000 7fe6455e4640 1 -- 172.21.15.64:0/1716716035 wait complete. 2024-09-21T03:59:01.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:00 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:59:01.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:00 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:01.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:00 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:01.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:00 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:59:01.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:00 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:01.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:00 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:01.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:00 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:59:01.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:00 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:01.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:00 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:01.429 DEBUG:teuthology.orchestra.run.smithi064:osd.6> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.6.service 2024-09-21T03:59:01.431 INFO:tasks.cephadm:Deploying osd.7 on smithi119 with /dev/vg_nvme/lv_2... 2024-09-21T03:59:01.431 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T03:59:01.686 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:59:02.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:01 smithi064 ceph-mon[31180]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:59:02.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:01 smithi119 ceph-mon[34811]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:59:02.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:01 smithi131 ceph-mon[35193]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:59:02.915 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 03:59:02 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T03:59:02.590+0000 7f218622d640 -1 osd.6 0 log_to_monitors true 2024-09-21T03:59:03.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:02 smithi119 ceph-mon[34811]: from='osd.6 [v2:172.21.15.64:6818/1913824324,v1:172.21.15.64:6819/1913824324]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T03:59:03.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:02 smithi064 ceph-mon[31180]: from='osd.6 [v2:172.21.15.64:6818/1913824324,v1:172.21.15.64:6819/1913824324]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T03:59:03.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:02 smithi131 ceph-mon[35193]: from='osd.6 [v2:172.21.15.64:6818/1913824324,v1:172.21.15.64:6819/1913824324]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T03:59:04.159 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:03 smithi119 ceph-mon[34811]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:59:04.159 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:03 smithi119 ceph-mon[34811]: from='osd.6 [v2:172.21.15.64:6818/1913824324,v1:172.21.15.64:6819/1913824324]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T03:59:04.159 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:03 smithi119 ceph-mon[34811]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T03:59:04.159 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:03 smithi119 ceph-mon[34811]: from='osd.6 [v2:172.21.15.64:6818/1913824324,v1:172.21.15.64:6819/1913824324]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]} : dispatch 2024-09-21T03:59:04.159 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:03 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:59:04.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:03 smithi064 ceph-mon[31180]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:59:04.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:03 smithi064 ceph-mon[31180]: from='osd.6 [v2:172.21.15.64:6818/1913824324,v1:172.21.15.64:6819/1913824324]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T03:59:04.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:03 smithi064 ceph-mon[31180]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T03:59:04.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:03 smithi064 ceph-mon[31180]: from='osd.6 [v2:172.21.15.64:6818/1913824324,v1:172.21.15.64:6819/1913824324]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]} : dispatch 2024-09-21T03:59:04.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:03 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:59:04.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:03 smithi131 ceph-mon[35193]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:59:04.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:03 smithi131 ceph-mon[35193]: from='osd.6 [v2:172.21.15.64:6818/1913824324,v1:172.21.15.64:6819/1913824324]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T03:59:04.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:03 smithi131 ceph-mon[35193]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T03:59:04.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:03 smithi131 ceph-mon[35193]: from='osd.6 [v2:172.21.15.64:6818/1913824324,v1:172.21.15.64:6819/1913824324]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]} : dispatch 2024-09-21T03:59:04.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:03 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:59:04.459 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T03:59:04.488 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi119:vg_nvme/lv_2 2024-09-21T03:59:04.742 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:59:04.762 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 03:59:04 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T03:59:04.407+0000 7f21821af640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T03:59:04.762 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 03:59:04 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T03:59:04.417+0000 7f2179fa5640 -1 osd.6 40 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:59:05.022 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:04 smithi119 ceph-mon[34811]: from='osd.6 [v2:172.21.15.64:6818/1913824324,v1:172.21.15.64:6819/1913824324]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-21T03:59:05.022 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:04 smithi119 ceph-mon[34811]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T03:59:05.022 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:04 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:59:05.023 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:04 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:59:05.023 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:04 smithi119 ceph-mon[34811]: from='osd.6 [v2:172.21.15.64:6818/1913824324,v1:172.21.15.64:6819/1913824324]' entity='osd.6' 2024-09-21T03:59:05.023 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:04 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:05.023 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:04 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:05.023 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:04 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:59:05.023 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:04 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:59:05.023 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:04 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:59:05.023 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:04 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:05.023 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:04 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:05.023 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:04 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:59:05.023 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:04 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:05.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.211+0000 7f6757239640 1 -- 172.21.15.119:0/946745401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6750102aa0 msgr2=0x7f6750102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:59:05.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.211+0000 7f6757239640 1 --2- 172.21.15.119:0/946745401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6750102aa0 0x7f6750102eb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6744009900 tx=0x7f674402a410 comp rx=0 tx=0).stop 2024-09-21T03:59:05.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.211+0000 7f6757239640 1 -- 172.21.15.119:0/946745401 shutdown_connections 2024-09-21T03:59:05.215 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.211+0000 7f6757239640 1 --2- 172.21.15.119:0/946745401 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6750108d30 0x7f675010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:59:05.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.211+0000 7f6757239640 1 --2- 172.21.15.119:0/946745401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67501083e0 0x7f67501087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:59:05.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.211+0000 7f6757239640 1 --2- 172.21.15.119:0/946745401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6750102aa0 0x7f6750102eb0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:59:05.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.211+0000 7f6757239640 1 -- 172.21.15.119:0/946745401 >> 172.21.15.119:0/946745401 conn(0x7f67500797e0 msgr2=0x7f6750079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:59:05.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.211+0000 7f6757239640 1 -- 172.21.15.119:0/946745401 shutdown_connections 2024-09-21T03:59:05.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.211+0000 7f6757239640 1 -- 172.21.15.119:0/946745401 wait complete. 2024-09-21T03:59:05.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.211+0000 7f6757239640 1 Processor -- start 2024-09-21T03:59:05.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.211+0000 7f6757239640 1 -- start start 2024-09-21T03:59:05.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.212+0000 7f6757239640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6750102aa0 0x7f675019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:59:05.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.212+0000 7f6757239640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67501083e0 0x7f675019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:59:05.216 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.212+0000 7f6757239640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6750108d30 0x7f675019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:59:05.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.212+0000 7f6757239640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f67501129a0 con 0x7f6750108d30 2024-09-21T03:59:05.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.212+0000 7f6757239640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6750112820 con 0x7f6750102aa0 2024-09-21T03:59:05.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.212+0000 7f6757239640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6750112b20 con 0x7f67501083e0 2024-09-21T03:59:05.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.212+0000 7f674ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67501083e0 0x7f675019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:59:05.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.212+0000 7f6754fd5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6750102aa0 0x7f675019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:59:05.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.212+0000 7f67557d6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6750108d30 0x7f675019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:59:05.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.212+0000 7f674ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67501083e0 0x7f675019aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.119:37934/0 (socket says 172.21.15.119:37934) 2024-09-21T03:59:05.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.212+0000 7f6754fd5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6750102aa0 0x7f675019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46432/0 (socket says 172.21.15.119:46432) 2024-09-21T03:59:05.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.212+0000 7f674ffff640 1 -- 172.21.15.119:0/3493618592 learned_addr learned my addr 172.21.15.119:0/3493618592 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-21T03:59:05.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.212+0000 7f67557d6640 1 --2- 172.21.15.119:0/3493618592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6750108d30 0x7f675019fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.119:57226/0 (socket says 172.21.15.119:57226) 2024-09-21T03:59:05.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.212+0000 7f674ffff640 1 -- 172.21.15.119:0/3493618592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6750102aa0 msgr2=0x7f675019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:59:05.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.212+0000 7f674ffff640 1 --2- 172.21.15.119:0/3493618592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6750102aa0 0x7f675019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:59:05.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.212+0000 7f674ffff640 1 -- 172.21.15.119:0/3493618592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6750108d30 msgr2=0x7f675019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:59:05.217 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.212+0000 7f674ffff640 1 --2- 172.21.15.119:0/3493618592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6750108d30 0x7f675019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:59:05.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.212+0000 7f674ffff640 1 -- 172.21.15.119:0/3493618592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67501a0350 con 0x7f67501083e0 2024-09-21T03:59:05.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.212+0000 7f6754fd5640 1 --2- 172.21.15.119:0/3493618592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6750102aa0 0x7f675019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:59:05.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.212+0000 7f67557d6640 1 --2- 172.21.15.119:0/3493618592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6750108d30 0x7f675019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:59:05.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.213+0000 7f674ffff640 1 --2- 172.21.15.119:0/3493618592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67501083e0 0x7f675019aa40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f673c00c970 tx=0x7f673c00ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:59:05.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.213+0000 7f674dffb640 1 -- 172.21.15.119:0/3493618592 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c014070 con 0x7f67501083e0 2024-09-21T03:59:05.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.213+0000 7f674dffb640 1 -- 172.21.15.119:0/3493618592 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f673c004440 con 0x7f67501083e0 2024-09-21T03:59:05.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.213+0000 7f6757239640 1 -- 172.21.15.119:0/3493618592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67501a0640 con 0x7f67501083e0 2024-09-21T03:59:05.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.213+0000 7f674dffb640 1 -- 172.21.15.119:0/3493618592 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c003e50 con 0x7f67501083e0 2024-09-21T03:59:05.218 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.213+0000 7f6757239640 1 -- 172.21.15.119:0/3493618592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67501a6e60 con 0x7f67501083e0 2024-09-21T03:59:05.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.214+0000 7f6757239640 1 -- 172.21.15.119:0/3493618592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6750102eb0 con 0x7f67501083e0 2024-09-21T03:59:05.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.215+0000 7f674dffb640 1 -- 172.21.15.119:0/3493618592 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f673c017070 con 0x7f67501083e0 2024-09-21T03:59:05.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.216+0000 7f674dffb640 1 --2- 172.21.15.119:0/3493618592 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f67400b6730 0x7f67400b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:59:05.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.216+0000 7f674dffb640 1 -- 172.21.15.119:0/3493618592 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4900+0+0 (secure 0 0 0) 0x7f673c0e6120 con 0x7f67501083e0 2024-09-21T03:59:05.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.216+0000 7f6754fd5640 1 --2- 172.21.15.119:0/3493618592 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f67400b6730 0x7f67400b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:59:05.219 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.217+0000 7f6754fd5640 1 --2- 172.21.15.119:0/3493618592 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f67400b6730 0x7f67400b8c40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6744031440 tx=0x7f6744032ed0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:59:05.223 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.221+0000 7f674dffb640 1 -- 172.21.15.119:0/3493618592 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f673c01e030 con 0x7f67501083e0 2024-09-21T03:59:05.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:04 smithi064 ceph-mon[31180]: from='osd.6 [v2:172.21.15.64:6818/1913824324,v1:172.21.15.64:6819/1913824324]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-21T03:59:05.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:04 smithi064 ceph-mon[31180]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T03:59:05.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:04 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:59:05.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:04 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:59:05.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:04 smithi064 ceph-mon[31180]: from='osd.6 [v2:172.21.15.64:6818/1913824324,v1:172.21.15.64:6819/1913824324]' entity='osd.6' 2024-09-21T03:59:05.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:04 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:05.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:04 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:05.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:04 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:59:05.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:04 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:59:05.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:04 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:59:05.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:04 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:05.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:04 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:05.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:04 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:59:05.263 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:04 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:05.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:04 smithi131 ceph-mon[35193]: from='osd.6 [v2:172.21.15.64:6818/1913824324,v1:172.21.15.64:6819/1913824324]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-21T03:59:05.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:04 smithi131 ceph-mon[35193]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T03:59:05.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:04 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:59:05.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:04 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:59:05.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:04 smithi131 ceph-mon[35193]: from='osd.6 [v2:172.21.15.64:6818/1913824324,v1:172.21.15.64:6819/1913824324]' entity='osd.6' 2024-09-21T03:59:05.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:04 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:05.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:04 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:05.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:04 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:59:05.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:04 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:59:05.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:04 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:59:05.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:04 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:05.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:04 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:05.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:04 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:59:05.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:04 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:05.413 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:05.411+0000 7f6757239640 1 -- 172.21.15.119:0/3493618592 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f675006b850 con 0x7f67400b6730 2024-09-21T03:59:06.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:05 smithi119 ceph-mon[34811]: purged_snaps scrub starts 2024-09-21T03:59:06.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:05 smithi119 ceph-mon[34811]: purged_snaps scrub ok 2024-09-21T03:59:06.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:05 smithi119 ceph-mon[34811]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:59:06.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:05 smithi119 ceph-mon[34811]: Detected new or changed devices on smithi064 2024-09-21T03:59:06.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:05 smithi119 ceph-mon[34811]: Adjusting osd_memory_target on smithi064 to 5696M 2024-09-21T03:59:06.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:05 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:59:06.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:05 smithi119 ceph-mon[34811]: osd.6 [v2:172.21.15.64:6818/1913824324,v1:172.21.15.64:6819/1913824324] boot 2024-09-21T03:59:06.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:05 smithi119 ceph-mon[34811]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T03:59:06.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:05 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:59:06.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:05 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:59:06.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:05 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:59:06.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:05 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:06.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:05 smithi064 ceph-mon[31180]: purged_snaps scrub starts 2024-09-21T03:59:06.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:05 smithi064 ceph-mon[31180]: purged_snaps scrub ok 2024-09-21T03:59:06.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:05 smithi064 ceph-mon[31180]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:59:06.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:05 smithi064 ceph-mon[31180]: Detected new or changed devices on smithi064 2024-09-21T03:59:06.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:05 smithi064 ceph-mon[31180]: Adjusting osd_memory_target on smithi064 to 5696M 2024-09-21T03:59:06.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:05 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:59:06.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:05 smithi064 ceph-mon[31180]: osd.6 [v2:172.21.15.64:6818/1913824324,v1:172.21.15.64:6819/1913824324] boot 2024-09-21T03:59:06.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:05 smithi064 ceph-mon[31180]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T03:59:06.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:05 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:59:06.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:05 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:59:06.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:05 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:59:06.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:05 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:06.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:05 smithi131 ceph-mon[35193]: purged_snaps scrub starts 2024-09-21T03:59:06.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:05 smithi131 ceph-mon[35193]: purged_snaps scrub ok 2024-09-21T03:59:06.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:05 smithi131 ceph-mon[35193]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-21T03:59:06.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:05 smithi131 ceph-mon[35193]: Detected new or changed devices on smithi064 2024-09-21T03:59:06.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:05 smithi131 ceph-mon[35193]: Adjusting osd_memory_target on smithi064 to 5696M 2024-09-21T03:59:06.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:05 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:59:06.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:05 smithi131 ceph-mon[35193]: osd.6 [v2:172.21.15.64:6818/1913824324,v1:172.21.15.64:6819/1913824324] boot 2024-09-21T03:59:06.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:05 smithi131 ceph-mon[35193]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T03:59:06.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:05 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T03:59:06.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:05 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:59:06.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:05 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:59:06.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:05 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:07.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:06 smithi119 ceph-mon[34811]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:59:07.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:06 smithi119 ceph-mon[34811]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T03:59:07.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:06 smithi064 ceph-mon[31180]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:59:07.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:06 smithi064 ceph-mon[31180]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T03:59:07.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:06 smithi131 ceph-mon[35193]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:59:07.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:06 smithi131 ceph-mon[35193]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T03:59:08.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:07 smithi119 ceph-mon[34811]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:08.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:07 smithi119 ceph-mon[34811]: from='client.? 172.21.15.119:0/87300543' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c9707260-b581-4a21-b380-f37e16959b02"} : dispatch 2024-09-21T03:59:08.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:07 smithi119 ceph-mon[34811]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c9707260-b581-4a21-b380-f37e16959b02"} : dispatch 2024-09-21T03:59:08.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:07 smithi119 ceph-mon[34811]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9707260-b581-4a21-b380-f37e16959b02"}]': finished 2024-09-21T03:59:08.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:07 smithi119 ceph-mon[34811]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T03:59:08.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:07 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:59:08.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:07 smithi064 ceph-mon[31180]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:08.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:07 smithi064 ceph-mon[31180]: from='client.? 172.21.15.119:0/87300543' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c9707260-b581-4a21-b380-f37e16959b02"} : dispatch 2024-09-21T03:59:08.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:07 smithi064 ceph-mon[31180]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c9707260-b581-4a21-b380-f37e16959b02"} : dispatch 2024-09-21T03:59:08.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:07 smithi064 ceph-mon[31180]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9707260-b581-4a21-b380-f37e16959b02"}]': finished 2024-09-21T03:59:08.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:07 smithi064 ceph-mon[31180]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T03:59:08.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:07 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:59:08.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:07 smithi131 ceph-mon[35193]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:08.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:07 smithi131 ceph-mon[35193]: from='client.? 172.21.15.119:0/87300543' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c9707260-b581-4a21-b380-f37e16959b02"} : dispatch 2024-09-21T03:59:08.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:07 smithi131 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c9707260-b581-4a21-b380-f37e16959b02"} : dispatch 2024-09-21T03:59:08.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:07 smithi131 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9707260-b581-4a21-b380-f37e16959b02"}]': finished 2024-09-21T03:59:08.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:07 smithi131 ceph-mon[35193]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T03:59:08.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:07 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:59:09.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:08 smithi119 ceph-mon[34811]: from='client.? 172.21.15.119:0/2696844226' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:59:09.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:08 smithi064 ceph-mon[31180]: from='client.? 172.21.15.119:0/2696844226' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:59:09.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:08 smithi131 ceph-mon[35193]: from='client.? 172.21.15.119:0/2696844226' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:59:10.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:09 smithi119 ceph-mon[34811]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:10.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:09 smithi064 ceph-mon[31180]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:10.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:09 smithi131 ceph-mon[35193]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:12.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:12 smithi119 ceph-mon[34811]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:12.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:12 smithi064 ceph-mon[31180]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:12.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:12 smithi131 ceph-mon[35193]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:14.115 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:14 smithi119 ceph-mon[34811]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:14.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:14 smithi064 ceph-mon[31180]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:14.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:14 smithi131 ceph-mon[35193]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:16.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:16 smithi064 ceph-mon[31180]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:16.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:16 smithi131 ceph-mon[35193]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:16.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:16 smithi119 ceph-mon[34811]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:17.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:17 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T03:59:17.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:17 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:17.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:17 smithi131 ceph-mon[35193]: Deploying daemon osd.7 on smithi119 2024-09-21T03:59:17.486 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:17 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T03:59:17.486 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:17 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:17.486 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:17 smithi119 ceph-mon[34811]: Deploying daemon osd.7 on smithi119 2024-09-21T03:59:17.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:17 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T03:59:17.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:17 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:17.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:17 smithi064 ceph-mon[31180]: Deploying daemon osd.7 on smithi119 2024-09-21T03:59:18.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:18 smithi119 ceph-mon[34811]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:18.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:18 smithi131 ceph-mon[35193]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:18.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:18 smithi064 ceph-mon[31180]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:20.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:20 smithi119 ceph-mon[34811]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:20.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:20 smithi131 ceph-mon[35193]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:20.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:20 smithi064 ceph-mon[31180]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:22.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:22 smithi131 ceph-mon[35193]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:22.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:22 smithi119 ceph-mon[34811]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:22.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:22 smithi064 ceph-mon[31180]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:23.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:23 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:59:23.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:23 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:23.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:23 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:23.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:23 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:59:23.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:23 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:23.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:23 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:23.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:23 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:59:23.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:23 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:23.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:23 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:24.140 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:24 smithi119 ceph-mon[34811]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:24.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:24 smithi131 ceph-mon[35193]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:24.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:24 smithi064 ceph-mon[31180]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:25.965 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:25 smithi119 ceph-mon[34811]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:25.965 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:25 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:25.965 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:25 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:25.965 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:25 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:25.965 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:25 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:59:25.965 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:25 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:26.012 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:26.010+0000 7f674dffb640 1 -- 172.21.15.119:0/3493618592 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f675006b850 con 0x7f67400b6730 2024-09-21T03:59:26.013 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 7 on host 'smithi119' 2024-09-21T03:59:26.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:26.013+0000 7f6757239640 1 -- 172.21.15.119:0/3493618592 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f67400b6730 msgr2=0x7f67400b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:59:26.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:26.013+0000 7f6757239640 1 --2- 172.21.15.119:0/3493618592 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f67400b6730 0x7f67400b8c40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6744031440 tx=0x7f6744032ed0 comp rx=0 tx=0).stop 2024-09-21T03:59:26.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:26.013+0000 7f6757239640 1 -- 172.21.15.119:0/3493618592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67501083e0 msgr2=0x7f675019aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:59:26.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:26.013+0000 7f6757239640 1 --2- 172.21.15.119:0/3493618592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67501083e0 0x7f675019aa40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f673c00c970 tx=0x7f673c00ce30 comp rx=0 tx=0).stop 2024-09-21T03:59:26.015 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:26.013+0000 7f6757239640 1 -- 172.21.15.119:0/3493618592 shutdown_connections 2024-09-21T03:59:26.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:26.013+0000 7f6757239640 1 --2- 172.21.15.119:0/3493618592 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f67400b6730 0x7f67400b8c40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:59:26.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:26.013+0000 7f6757239640 1 --2- 172.21.15.119:0/3493618592 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6750108d30 0x7f675019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:59:26.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:26.013+0000 7f6757239640 1 --2- 172.21.15.119:0/3493618592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f67501083e0 0x7f675019aa40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:59:26.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:26.013+0000 7f6757239640 1 --2- 172.21.15.119:0/3493618592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6750102aa0 0x7f675019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:59:26.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:26.013+0000 7f6757239640 1 -- 172.21.15.119:0/3493618592 >> 172.21.15.119:0/3493618592 conn(0x7f67500797e0 msgr2=0x7f6750101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:59:26.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:26.013+0000 7f6757239640 1 -- 172.21.15.119:0/3493618592 shutdown_connections 2024-09-21T03:59:26.016 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T03:59:26.013+0000 7f6757239640 1 -- 172.21.15.119:0/3493618592 wait complete. 2024-09-21T03:59:26.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:25 smithi064 ceph-mon[31180]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:26.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:25 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:26.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:25 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:26.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:25 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:26.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:25 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:59:26.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:25 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:26.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:25 smithi131 ceph-mon[35193]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:26.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:25 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:26.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:25 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:26.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:25 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:26.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:25 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:59:26.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:25 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:26.705 DEBUG:teuthology.orchestra.run.smithi119:osd.7> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.7.service 2024-09-21T03:59:26.708 INFO:tasks.cephadm:Deploying osd.8 on smithi131 with /dev/vg_nvme/lv_2... 2024-09-21T03:59:26.708 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T03:59:26.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:26 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:59:26.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:26 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:26.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:26 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:26.963 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:59:27.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:26 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:59:27.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:26 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:27.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:26 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:27.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:26 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:59:27.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:26 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:27.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:26 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:28.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:28 smithi131 ceph-mon[35193]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:28.330 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:28 smithi119 ceph-mon[34811]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:28.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:28 smithi064 ceph-mon[31180]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:28.756 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 03:59:28 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T03:59:28.409+0000 7f0755449640 -1 osd.7 0 log_to_monitors true 2024-09-21T03:59:29.175 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:29 smithi119 ceph-mon[34811]: from='osd.7 [v2:172.21.15.119:6816/2567568166,v1:172.21.15.119:6817/2567568166]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:59:29.175 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:29 smithi119 ceph-mon[34811]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:59:29.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:29 smithi131 ceph-mon[35193]: from='osd.7 [v2:172.21.15.119:6816/2567568166,v1:172.21.15.119:6817/2567568166]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:59:29.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:29 smithi131 ceph-mon[35193]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:59:29.454 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:29 smithi064 ceph-mon[31180]: from='osd.7 [v2:172.21.15.119:6816/2567568166,v1:172.21.15.119:6817/2567568166]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:59:29.454 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:29 smithi064 ceph-mon[31180]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T03:59:29.610 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-21T03:59:29.639 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi131:vg_nvme/lv_2 2024-09-21T03:59:29.887 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:59:30.130 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:30 smithi131 ceph-mon[35193]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:30.131 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:30 smithi131 ceph-mon[35193]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T03:59:30.131 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:30 smithi131 ceph-mon[35193]: from='osd.7 [v2:172.21.15.119:6816/2567568166,v1:172.21.15.119:6817/2567568166]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T03:59:30.131 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:30 smithi131 ceph-mon[35193]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T03:59:30.131 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:30 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:59:30.131 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:30 smithi131 ceph-mon[35193]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T03:59:30.131 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:30 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:30.131 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:30 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:30.131 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:30 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:59:30.131 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:30 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:59:30.131 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:30 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:59:30.131 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:30 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:30.131 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:30 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:30.131 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:30 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:59:30.132 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:30 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:30.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.354+0000 7f5428e99640 1 -- 172.21.15.131:0/3937987083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54241083f0 msgr2=0x7f5424108800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:59:30.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.354+0000 7f5428e99640 1 --2- 172.21.15.131:0/3937987083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54241083f0 0x7f5424108800 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5418009900 tx=0x7f541802a410 comp rx=0 tx=0).stop 2024-09-21T03:59:30.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.354+0000 7f5428e99640 1 -- 172.21.15.131:0/3937987083 shutdown_connections 2024-09-21T03:59:30.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.354+0000 7f5428e99640 1 --2- 172.21.15.131:0/3937987083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5424108d40 0x7f542410f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:59:30.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.354+0000 7f5428e99640 1 --2- 172.21.15.131:0/3937987083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54241083f0 0x7f5424108800 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:59:30.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.354+0000 7f5428e99640 1 --2- 172.21.15.131:0/3937987083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5424102ab0 0x7f5424102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:59:30.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.354+0000 7f5428e99640 1 -- 172.21.15.131:0/3937987083 >> 172.21.15.131:0/3937987083 conn(0x7f54240796b0 msgr2=0x7f5424079ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:59:30.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.354+0000 7f5428e99640 1 -- 172.21.15.131:0/3937987083 shutdown_connections 2024-09-21T03:59:30.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.354+0000 7f5428e99640 1 -- 172.21.15.131:0/3937987083 wait complete. 2024-09-21T03:59:30.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.355+0000 7f5428e99640 1 Processor -- start 2024-09-21T03:59:30.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.355+0000 7f5428e99640 1 -- start start 2024-09-21T03:59:30.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.356+0000 7f5428e99640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5424102ab0 0x7f54241993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:59:30.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.356+0000 7f5428e99640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54241083f0 0x7f5424199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:59:30.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.356+0000 7f5428e99640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5424108d40 0x7f542419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:59:30.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.356+0000 7f5428e99640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f54241129b0 con 0x7f5424102ab0 2024-09-21T03:59:30.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.356+0000 7f5428e99640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5424112830 con 0x7f54241083f0 2024-09-21T03:59:30.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.356+0000 7f5428e99640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5424112b30 con 0x7f5424108d40 2024-09-21T03:59:30.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.356+0000 7f5422d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5424108d40 0x7f542419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:59:30.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.356+0000 7f5422d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5424108d40 0x7f542419ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42624/0 (socket says 172.21.15.131:42624) 2024-09-21T03:59:30.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.356+0000 7f542259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5424102ab0 0x7f54241993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:59:30.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.356+0000 7f5422d9d640 1 -- 172.21.15.131:0/2874268383 learned_addr learned my addr 172.21.15.131:0/2874268383 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-21T03:59:30.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.356+0000 7f5421d9b640 1 --2- 172.21.15.131:0/2874268383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54241083f0 0x7f5424199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:59:30.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.356+0000 7f5422d9d640 1 -- 172.21.15.131:0/2874268383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54241083f0 msgr2=0x7f5424199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:59:30.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.356+0000 7f5422d9d640 1 --2- 172.21.15.131:0/2874268383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54241083f0 0x7f5424199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:59:30.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.356+0000 7f5422d9d640 1 -- 172.21.15.131:0/2874268383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5424102ab0 msgr2=0x7f54241993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:59:30.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.356+0000 7f5422d9d640 1 --2- 172.21.15.131:0/2874268383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5424102ab0 0x7f54241993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:59:30.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.356+0000 7f5422d9d640 1 -- 172.21.15.131:0/2874268383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f542419f270 con 0x7f5424108d40 2024-09-21T03:59:30.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.356+0000 7f542259c640 1 --2- 172.21.15.131:0/2874268383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5424102ab0 0x7f54241993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:59:30.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.356+0000 7f5422d9d640 1 --2- 172.21.15.131:0/2874268383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5424108d40 0x7f542419ec00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f541c00cce0 tx=0x7f541c00a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:59:30.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.357+0000 7f54137fe640 1 -- 172.21.15.131:0/2874268383 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c00f450 con 0x7f5424108d40 2024-09-21T03:59:30.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.357+0000 7f54137fe640 1 -- 172.21.15.131:0/2874268383 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f541c00faa0 con 0x7f5424108d40 2024-09-21T03:59:30.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.357+0000 7f5428e99640 1 -- 172.21.15.131:0/2874268383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f542419f560 con 0x7f5424108d40 2024-09-21T03:59:30.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.357+0000 7f54137fe640 1 -- 172.21.15.131:0/2874268383 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c018b70 con 0x7f5424108d40 2024-09-21T03:59:30.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.357+0000 7f5428e99640 1 -- 172.21.15.131:0/2874268383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f542419fa20 con 0x7f5424108d40 2024-09-21T03:59:30.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.358+0000 7f5428e99640 1 -- 172.21.15.131:0/2874268383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5424103d30 con 0x7f5424108d40 2024-09-21T03:59:30.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.360+0000 7f54137fe640 1 -- 172.21.15.131:0/2874268383 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f541c018d10 con 0x7f5424108d40 2024-09-21T03:59:30.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.361+0000 7f54137fe640 1 --2- 172.21.15.131:0/2874268383 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f540c0b6590 0x7f540c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:59:30.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.361+0000 7f54137fe640 1 -- 172.21.15.131:0/2874268383 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5332+0+0 (secure 0 0 0) 0x7f541c0e6bf0 con 0x7f5424108d40 2024-09-21T03:59:30.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.361+0000 7f542259c640 1 --2- 172.21.15.131:0/2874268383 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f540c0b6590 0x7f540c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:59:30.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.362+0000 7f542259c640 1 --2- 172.21.15.131:0/2874268383 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f540c0b6590 0x7f540c0b8aa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5414009fc0 tx=0x7f5414005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:59:30.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.363+0000 7f54137fe640 1 -- 172.21.15.131:0/2874268383 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f541c014030 con 0x7f5424108d40 2024-09-21T03:59:30.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:30 smithi064 ceph-mon[31180]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:30.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:30 smithi064 ceph-mon[31180]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T03:59:30.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:30 smithi064 ceph-mon[31180]: from='osd.7 [v2:172.21.15.119:6816/2567568166,v1:172.21.15.119:6817/2567568166]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T03:59:30.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:30 smithi064 ceph-mon[31180]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T03:59:30.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:30 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:59:30.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:30 smithi064 ceph-mon[31180]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T03:59:30.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:30 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:30.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:30 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:30.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:30 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:59:30.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:30 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:59:30.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:30 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:59:30.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:30 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:30.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:30 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:30.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:30 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:59:30.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:30 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:30.506 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 03:59:30 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T03:59:30.028+0000 7f07513cb640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T03:59:30.506 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 03:59:30 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T03:59:30.040+0000 7f074d1c9640 -1 osd.7 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:59:30.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:30 smithi119 ceph-mon[34811]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:30.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:30 smithi119 ceph-mon[34811]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T03:59:30.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:30 smithi119 ceph-mon[34811]: from='osd.7 [v2:172.21.15.119:6816/2567568166,v1:172.21.15.119:6817/2567568166]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T03:59:30.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:30 smithi119 ceph-mon[34811]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T03:59:30.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:30 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:59:30.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:30 smithi119 ceph-mon[34811]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T03:59:30.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:30 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:30.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:30 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:30.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:30 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:59:30.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:30 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:59:30.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:30 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:59:30.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:30 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:30.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:30 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:30.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:30 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:59:30.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:30 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:30.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:30.560+0000 7f5428e99640 1 -- 172.21.15.131:0/2874268383 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5424063800 con 0x7f540c0b6590 2024-09-21T03:59:31.147 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:31 smithi131 ceph-mon[35193]: Detected new or changed devices on smithi119 2024-09-21T03:59:31.147 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:31 smithi131 ceph-mon[35193]: Adjusting osd_memory_target on smithi119 to 5696M 2024-09-21T03:59:31.147 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:31 smithi131 ceph-mon[35193]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-09-21T03:59:31.147 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:31 smithi131 ceph-mon[35193]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T03:59:31.147 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:31 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:59:31.147 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:31 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:59:31.148 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:31 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:59:31.148 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:31 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:59:31.148 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:31 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:31.148 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:31 smithi131 ceph-mon[35193]: osd.7 [v2:172.21.15.119:6816/2567568166,v1:172.21.15.119:6817/2567568166] boot 2024-09-21T03:59:31.148 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:31 smithi131 ceph-mon[35193]: osdmap e46: 8 total, 8 up, 8 in 2024-09-21T03:59:31.148 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:31 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:59:31.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:31 smithi119 ceph-mon[34811]: Detected new or changed devices on smithi119 2024-09-21T03:59:31.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:31 smithi119 ceph-mon[34811]: Adjusting osd_memory_target on smithi119 to 5696M 2024-09-21T03:59:31.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:31 smithi119 ceph-mon[34811]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-09-21T03:59:31.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:31 smithi119 ceph-mon[34811]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T03:59:31.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:31 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:59:31.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:31 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:59:31.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:31 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:59:31.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:31 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:59:31.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:31 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:31.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:31 smithi119 ceph-mon[34811]: osd.7 [v2:172.21.15.119:6816/2567568166,v1:172.21.15.119:6817/2567568166] boot 2024-09-21T03:59:31.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:31 smithi119 ceph-mon[34811]: osdmap e46: 8 total, 8 up, 8 in 2024-09-21T03:59:31.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:31 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:59:31.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:31 smithi064 ceph-mon[31180]: Detected new or changed devices on smithi119 2024-09-21T03:59:31.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:31 smithi064 ceph-mon[31180]: Adjusting osd_memory_target on smithi119 to 5696M 2024-09-21T03:59:31.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:31 smithi064 ceph-mon[31180]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-09-21T03:59:31.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:31 smithi064 ceph-mon[31180]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T03:59:31.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:31 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:59:31.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:31 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:59:31.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:31 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:59:31.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:31 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:59:31.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:31 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:31.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:31 smithi064 ceph-mon[31180]: osd.7 [v2:172.21.15.119:6816/2567568166,v1:172.21.15.119:6817/2567568166] boot 2024-09-21T03:59:31.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:31 smithi064 ceph-mon[31180]: osdmap e46: 8 total, 8 up, 8 in 2024-09-21T03:59:31.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:31 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T03:59:32.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:32 smithi131 ceph-mon[35193]: purged_snaps scrub starts 2024-09-21T03:59:32.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:32 smithi131 ceph-mon[35193]: purged_snaps scrub ok 2024-09-21T03:59:32.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:32 smithi131 ceph-mon[35193]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:32.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:32 smithi131 ceph-mon[35193]: from='client.24457 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:59:32.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:32 smithi131 ceph-mon[35193]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T03:59:32.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:32 smithi119 ceph-mon[34811]: purged_snaps scrub starts 2024-09-21T03:59:32.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:32 smithi119 ceph-mon[34811]: purged_snaps scrub ok 2024-09-21T03:59:32.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:32 smithi119 ceph-mon[34811]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:32.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:32 smithi119 ceph-mon[34811]: from='client.24457 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:59:32.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:32 smithi119 ceph-mon[34811]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T03:59:32.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:32 smithi064 ceph-mon[31180]: purged_snaps scrub starts 2024-09-21T03:59:32.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:32 smithi064 ceph-mon[31180]: purged_snaps scrub ok 2024-09-21T03:59:32.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:32 smithi064 ceph-mon[31180]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-21T03:59:32.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:32 smithi064 ceph-mon[31180]: from='client.24457 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:59:32.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:32 smithi064 ceph-mon[31180]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T03:59:33.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:33 smithi131 ceph-mon[35193]: from='client.? 172.21.15.131:0/1010556076' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2dbf8ef2-fc9c-4c6b-bfea-003f41ef191e"} : dispatch 2024-09-21T03:59:33.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:33 smithi131 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2dbf8ef2-fc9c-4c6b-bfea-003f41ef191e"} : dispatch 2024-09-21T03:59:33.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:33 smithi131 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2dbf8ef2-fc9c-4c6b-bfea-003f41ef191e"}]': finished 2024-09-21T03:59:33.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:33 smithi131 ceph-mon[35193]: osdmap e48: 9 total, 8 up, 9 in 2024-09-21T03:59:33.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:33 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:59:33.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:33 smithi119 ceph-mon[34811]: from='client.? 172.21.15.131:0/1010556076' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2dbf8ef2-fc9c-4c6b-bfea-003f41ef191e"} : dispatch 2024-09-21T03:59:33.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:33 smithi119 ceph-mon[34811]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2dbf8ef2-fc9c-4c6b-bfea-003f41ef191e"} : dispatch 2024-09-21T03:59:33.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:33 smithi119 ceph-mon[34811]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2dbf8ef2-fc9c-4c6b-bfea-003f41ef191e"}]': finished 2024-09-21T03:59:33.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:33 smithi119 ceph-mon[34811]: osdmap e48: 9 total, 8 up, 9 in 2024-09-21T03:59:33.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:33 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:59:33.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:33 smithi064 ceph-mon[31180]: from='client.? 172.21.15.131:0/1010556076' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2dbf8ef2-fc9c-4c6b-bfea-003f41ef191e"} : dispatch 2024-09-21T03:59:33.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:33 smithi064 ceph-mon[31180]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2dbf8ef2-fc9c-4c6b-bfea-003f41ef191e"} : dispatch 2024-09-21T03:59:33.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:33 smithi064 ceph-mon[31180]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2dbf8ef2-fc9c-4c6b-bfea-003f41ef191e"}]': finished 2024-09-21T03:59:33.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:33 smithi064 ceph-mon[31180]: osdmap e48: 9 total, 8 up, 9 in 2024-09-21T03:59:33.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:33 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:59:34.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:34 smithi131 ceph-mon[35193]: pgmap v157: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:59:34.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:34 smithi131 ceph-mon[35193]: from='client.? 172.21.15.131:0/3122995361' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:59:34.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:34 smithi119 ceph-mon[34811]: pgmap v157: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:59:34.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:34 smithi119 ceph-mon[34811]: from='client.? 172.21.15.131:0/3122995361' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:59:34.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:34 smithi064 ceph-mon[31180]: pgmap v157: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:59:34.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:34 smithi064 ceph-mon[31180]: from='client.? 172.21.15.131:0/3122995361' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T03:59:36.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:36 smithi131 ceph-mon[35193]: pgmap v159: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:59:36.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:36 smithi119 ceph-mon[34811]: pgmap v159: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:59:36.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:36 smithi064 ceph-mon[31180]: pgmap v159: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:59:38.444 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:38 smithi131 ceph-mon[35193]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:59:38.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:38 smithi119 ceph-mon[34811]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:59:38.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:38 smithi064 ceph-mon[31180]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:59:40.161 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:40 smithi131 ceph-mon[35193]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 155 KiB/s, 0 objects/s recovering 2024-09-21T03:59:40.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:40 smithi064 ceph-mon[31180]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 155 KiB/s, 0 objects/s recovering 2024-09-21T03:59:40.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:40 smithi119 ceph-mon[34811]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 155 KiB/s, 0 objects/s recovering 2024-09-21T03:59:42.215 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:42 smithi131 ceph-mon[35193]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-09-21T03:59:42.215 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:42 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T03:59:42.215 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:42 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:42.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:42 smithi119 ceph-mon[34811]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-09-21T03:59:42.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:42 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T03:59:42.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:42 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:42.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:42 smithi064 ceph-mon[31180]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-09-21T03:59:42.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:42 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T03:59:42.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:42 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:43.197 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:43 smithi131 ceph-mon[35193]: Deploying daemon osd.8 on smithi131 2024-09-21T03:59:43.407 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:43 smithi119 ceph-mon[34811]: Deploying daemon osd.8 on smithi131 2024-09-21T03:59:43.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:43 smithi064 ceph-mon[31180]: Deploying daemon osd.8 on smithi131 2024-09-21T03:59:44.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:44 smithi131 ceph-mon[35193]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-21T03:59:44.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:44 smithi119 ceph-mon[34811]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-21T03:59:44.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:44 smithi064 ceph-mon[31180]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-21T03:59:46.443 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:46 smithi131 ceph-mon[35193]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-21T03:59:46.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:46 smithi119 ceph-mon[34811]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-21T03:59:46.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:46 smithi064 ceph-mon[31180]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-21T03:59:48.336 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:48 smithi131 ceph-mon[35193]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:59:48.337 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:48 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:59:48.337 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:48 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:48.337 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:48 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:48.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:48 smithi119 ceph-mon[34811]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:59:48.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:48 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:59:48.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:48 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:48.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:48 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:48.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:48 smithi064 ceph-mon[31180]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T03:59:48.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:48 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:59:48.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:48 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:48.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:48 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:50.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:50 smithi119 ceph-mon[34811]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:59:50.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:50 smithi064 ceph-mon[31180]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:59:50.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:50 smithi131 ceph-mon[35193]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:59:51.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:51.540+0000 7f54137fe640 1 -- 172.21.15.131:0/2874268383 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5424063800 con 0x7f540c0b6590 2024-09-21T03:59:51.544 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 8 on host 'smithi131' 2024-09-21T03:59:51.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:51.543+0000 7f5428e99640 1 -- 172.21.15.131:0/2874268383 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f540c0b6590 msgr2=0x7f540c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:59:51.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:51.543+0000 7f5428e99640 1 --2- 172.21.15.131:0/2874268383 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f540c0b6590 0x7f540c0b8aa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5414009fc0 tx=0x7f5414005f90 comp rx=0 tx=0).stop 2024-09-21T03:59:51.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:51.543+0000 7f5428e99640 1 -- 172.21.15.131:0/2874268383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5424108d40 msgr2=0x7f542419ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:59:51.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:51.543+0000 7f5428e99640 1 --2- 172.21.15.131:0/2874268383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5424108d40 0x7f542419ec00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f541c00cce0 tx=0x7f541c00a620 comp rx=0 tx=0).stop 2024-09-21T03:59:51.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:51.543+0000 7f5428e99640 1 -- 172.21.15.131:0/2874268383 shutdown_connections 2024-09-21T03:59:51.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:51.543+0000 7f5428e99640 1 --2- 172.21.15.131:0/2874268383 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f540c0b6590 0x7f540c0b8aa0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:59:51.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:51.543+0000 7f5428e99640 1 --2- 172.21.15.131:0/2874268383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5424108d40 0x7f542419ec00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:59:51.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:51.543+0000 7f5428e99640 1 --2- 172.21.15.131:0/2874268383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54241083f0 0x7f5424199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:59:51.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:51.543+0000 7f5428e99640 1 --2- 172.21.15.131:0/2874268383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5424102ab0 0x7f54241993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:59:51.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:51.543+0000 7f5428e99640 1 -- 172.21.15.131:0/2874268383 >> 172.21.15.131:0/2874268383 conn(0x7f54240796b0 msgr2=0x7f54241044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:59:51.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:51.543+0000 7f5428e99640 1 -- 172.21.15.131:0/2874268383 shutdown_connections 2024-09-21T03:59:51.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T03:59:51.543+0000 7f5428e99640 1 -- 172.21.15.131:0/2874268383 wait complete. 2024-09-21T03:59:51.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:51 smithi119 ceph-mon[34811]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:59:51.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:51 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:51.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:51 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:51.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:51 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:51.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:51 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:59:51.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:51 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:51.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:51 smithi064 ceph-mon[31180]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:59:51.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:51 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:51.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:51 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:51.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:51 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:51.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:51 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:59:51.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:51 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:51.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:51 smithi131 ceph-mon[35193]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:59:51.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:51 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:51.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:51 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:51.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:51 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:51.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:51 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:59:51.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:51 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:52.237 DEBUG:teuthology.orchestra.run.smithi131:osd.8> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.8.service 2024-09-21T03:59:52.239 INFO:tasks.cephadm:Deploying osd.9 on smithi064 with /dev/vg_nvme/lv_1... 2024-09-21T03:59:52.239 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T03:59:52.509 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:59:52.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:52 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:59:52.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:52 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:52.633 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:52 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:52.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:52 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:59:52.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:52 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:52.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:52 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:52.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:52 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:59:52.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:52 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:52.771 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:52 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:53.637 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:53 smithi131 ceph-mon[35193]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:59:53.638 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:53 smithi131 ceph-mon[35193]: from='osd.8 [v2:172.21.15.131:6816/199192217,v1:172.21.15.131:6817/199192217]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:59:53.638 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:53 smithi131 ceph-mon[35193]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:59:53.638 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 03:59:53 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T03:59:53.523+0000 7f0dbf3cd640 -1 osd.8 0 log_to_monitors true 2024-09-21T03:59:53.960 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:53 smithi064 ceph-mon[31180]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:59:53.961 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:53 smithi064 ceph-mon[31180]: from='osd.8 [v2:172.21.15.131:6816/199192217,v1:172.21.15.131:6817/199192217]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:59:53.961 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:53 smithi064 ceph-mon[31180]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:59:54.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:53 smithi119 ceph-mon[34811]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:59:54.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:53 smithi119 ceph-mon[34811]: from='osd.8 [v2:172.21.15.131:6816/199192217,v1:172.21.15.131:6817/199192217]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:59:54.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:53 smithi119 ceph-mon[34811]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T03:59:54.716 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:54 smithi131 ceph-mon[35193]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T03:59:54.717 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:54 smithi131 ceph-mon[35193]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T03:59:54.717 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:54 smithi131 ceph-mon[35193]: from='osd.8 [v2:172.21.15.131:6816/199192217,v1:172.21.15.131:6817/199192217]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi131", "root=default"]} : dispatch 2024-09-21T03:59:54.717 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:54 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:59:54.717 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:54 smithi131 ceph-mon[35193]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi131", "root=default"]} : dispatch 2024-09-21T03:59:54.717 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:54 smithi131 ceph-mon[35193]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-21T03:59:55.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:54 smithi119 ceph-mon[34811]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T03:59:55.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:54 smithi119 ceph-mon[34811]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T03:59:55.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:54 smithi119 ceph-mon[34811]: from='osd.8 [v2:172.21.15.131:6816/199192217,v1:172.21.15.131:6817/199192217]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi131", "root=default"]} : dispatch 2024-09-21T03:59:55.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:54 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:59:55.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:54 smithi119 ceph-mon[34811]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi131", "root=default"]} : dispatch 2024-09-21T03:59:55.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:54 smithi119 ceph-mon[34811]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-21T03:59:55.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:54 smithi064 ceph-mon[31180]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T03:59:55.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:54 smithi064 ceph-mon[31180]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T03:59:55.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:54 smithi064 ceph-mon[31180]: from='osd.8 [v2:172.21.15.131:6816/199192217,v1:172.21.15.131:6817/199192217]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi131", "root=default"]} : dispatch 2024-09-21T03:59:55.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:54 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:59:55.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:54 smithi064 ceph-mon[31180]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi131", "root=default"]} : dispatch 2024-09-21T03:59:55.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:54 smithi064 ceph-mon[31180]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-21T03:59:55.303 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 03:59:55 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T03:59:55.001+0000 7f0dbb34f640 -1 osd.8 0 waiting for initial osdmap 2024-09-21T03:59:55.303 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 03:59:55 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T03:59:55.014+0000 7f0db3145640 -1 osd.8 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T03:59:55.431 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T03:59:55.456 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi064:vg_nvme/lv_1 2024-09-21T03:59:55.720 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:59:55.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:55 smithi064 ceph-mon[31180]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:59:55.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:55 smithi064 ceph-mon[31180]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T03:59:55.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:55 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:59:55.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:55 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:59:55.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:55 smithi064 ceph-mon[31180]: from='osd.8 ' entity='osd.8' 2024-09-21T03:59:55.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:55 smithi064 ceph-mon[31180]: Detected new or changed devices on smithi131 2024-09-21T03:59:55.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:55 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:55.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:55 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:55.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:55 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:59:55.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:55 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:59:55.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:55 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:59:55.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:55 smithi064 ceph-mon[31180]: Adjusting osd_memory_target on smithi131 to 5696M 2024-09-21T03:59:55.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:55 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:55.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:55 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:55.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:55 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:59:55.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:55 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:56.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:55 smithi119 ceph-mon[34811]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:59:56.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:55 smithi119 ceph-mon[34811]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T03:59:56.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:55 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:59:56.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:55 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:59:56.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:55 smithi119 ceph-mon[34811]: from='osd.8 ' entity='osd.8' 2024-09-21T03:59:56.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:55 smithi119 ceph-mon[34811]: Detected new or changed devices on smithi131 2024-09-21T03:59:56.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:55 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:56.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:55 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:56.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:55 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:59:56.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:55 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:59:56.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:55 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:59:56.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:55 smithi119 ceph-mon[34811]: Adjusting osd_memory_target on smithi131 to 5696M 2024-09-21T03:59:56.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:55 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:56.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:55 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:56.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:55 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:59:56.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:55 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:56.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:55 smithi131 ceph-mon[35193]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:59:56.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:55 smithi131 ceph-mon[35193]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T03:59:56.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:55 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:59:56.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:55 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:59:56.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:55 smithi131 ceph-mon[35193]: from='osd.8 ' entity='osd.8' 2024-09-21T03:59:56.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:55 smithi131 ceph-mon[35193]: Detected new or changed devices on smithi131 2024-09-21T03:59:56.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:55 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:56.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:55 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:56.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:55 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:59:56.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:55 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:59:56.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:55 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:59:56.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:55 smithi131 ceph-mon[35193]: Adjusting osd_memory_target on smithi131 to 5696M 2024-09-21T03:59:56.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:55 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:56.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:55 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:56.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:55 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:59:56.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:55 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T03:59:56.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.202+0000 7f95b4971640 1 -- 172.21.15.64:0/3401708033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95b0102a60 msgr2=0x7f95b0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:59:56.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.202+0000 7f95b4971640 1 --2- 172.21.15.64:0/3401708033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95b0102a60 0x7f95b0102e70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f95a0009900 tx=0x7f95a002a410 comp rx=0 tx=0).stop 2024-09-21T03:59:56.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.202+0000 7f95b4971640 1 -- 172.21.15.64:0/3401708033 shutdown_connections 2024-09-21T03:59:56.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.202+0000 7f95b4971640 1 --2- 172.21.15.64:0/3401708033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95b0108cf0 0x7f95b010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:59:56.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.202+0000 7f95b4971640 1 --2- 172.21.15.64:0/3401708033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95b01083a0 0x7f95b01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:59:56.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.202+0000 7f95b4971640 1 --2- 172.21.15.64:0/3401708033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95b0102a60 0x7f95b0102e70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:59:56.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.202+0000 7f95b4971640 1 -- 172.21.15.64:0/3401708033 >> 172.21.15.64:0/3401708033 conn(0x7f95b0079730 msgr2=0x7f95b0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:59:56.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.202+0000 7f95b4971640 1 -- 172.21.15.64:0/3401708033 shutdown_connections 2024-09-21T03:59:56.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.202+0000 7f95b4971640 1 -- 172.21.15.64:0/3401708033 wait complete. 2024-09-21T03:59:56.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.203+0000 7f95b4971640 1 Processor -- start 2024-09-21T03:59:56.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.203+0000 7f95b4971640 1 -- start start 2024-09-21T03:59:56.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.203+0000 7f95b4971640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95b0102a60 0x7f95b019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:59:56.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.203+0000 7f95b4971640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95b01083a0 0x7f95b019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:59:56.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.203+0000 7f95b4971640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95b0108cf0 0x7f95b019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:59:56.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.203+0000 7f95b4971640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f95b0112960 con 0x7f95b0102a60 2024-09-21T03:59:56.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.203+0000 7f95b4971640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f95b01127e0 con 0x7f95b01083a0 2024-09-21T03:59:56.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.203+0000 7f95b4971640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f95b0112ae0 con 0x7f95b0108cf0 2024-09-21T03:59:56.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.203+0000 7f95ae59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95b0102a60 0x7f95b019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:59:56.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.203+0000 7f95ae59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95b0102a60 0x7f95b019a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36914/0 (socket says 172.21.15.64:36914) 2024-09-21T03:59:56.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.203+0000 7f95ae59c640 1 -- 172.21.15.64:0/46346472 learned_addr learned my addr 172.21.15.64:0/46346472 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T03:59:56.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.203+0000 7f95aed9d640 1 --2- 172.21.15.64:0/46346472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95b0108cf0 0x7f95b019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:59:56.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.203+0000 7f95add9b640 1 --2- 172.21.15.64:0/46346472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95b01083a0 0x7f95b019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:59:56.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.204+0000 7f95ae59c640 1 -- 172.21.15.64:0/46346472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95b0108cf0 msgr2=0x7f95b019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:59:56.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.204+0000 7f95ae59c640 1 --2- 172.21.15.64:0/46346472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95b0108cf0 0x7f95b019fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:59:56.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.204+0000 7f95ae59c640 1 -- 172.21.15.64:0/46346472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95b01083a0 msgr2=0x7f95b019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:59:56.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.204+0000 7f95ae59c640 1 --2- 172.21.15.64:0/46346472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95b01083a0 0x7f95b019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:59:56.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.204+0000 7f95ae59c640 1 -- 172.21.15.64:0/46346472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b01a03b0 con 0x7f95b0102a60 2024-09-21T03:59:56.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.204+0000 7f95ae59c640 1 --2- 172.21.15.64:0/46346472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95b0102a60 0x7f95b019a440 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f95a0005940 tx=0x7f95a00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:59:56.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.204+0000 7f959f7fe640 1 -- 172.21.15.64:0/46346472 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a003f030 con 0x7f95b0102a60 2024-09-21T03:59:56.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.204+0000 7f959f7fe640 1 -- 172.21.15.64:0/46346472 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95a0003cb0 con 0x7f95b0102a60 2024-09-21T03:59:56.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.204+0000 7f959f7fe640 1 -- 172.21.15.64:0/46346472 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a0035070 con 0x7f95b0102a60 2024-09-21T03:59:56.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.204+0000 7f95b4971640 1 -- 172.21.15.64:0/46346472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b01a0640 con 0x7f95b0102a60 2024-09-21T03:59:56.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.204+0000 7f95b4971640 1 -- 172.21.15.64:0/46346472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b01a6e00 con 0x7f95b0102a60 2024-09-21T03:59:56.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.205+0000 7f95b4971640 1 -- 172.21.15.64:0/46346472 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95b0102e70 con 0x7f95b0102a60 2024-09-21T03:59:56.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.207+0000 7f959f7fe640 1 -- 172.21.15.64:0/46346472 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f95a0042030 con 0x7f95b0102a60 2024-09-21T03:59:56.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.208+0000 7f959f7fe640 1 --2- 172.21.15.64:0/46346472 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f95980b65e0 0x7f95980b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:59:56.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.208+0000 7f95add9b640 1 --2- 172.21.15.64:0/46346472 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f95980b65e0 0x7f95980b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:59:56.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.208+0000 7f959f7fe640 1 -- 172.21.15.64:0/46346472 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6044+0+0 (secure 0 0 0) 0x7f95a0107630 con 0x7f95b0102a60 2024-09-21T03:59:56.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.209+0000 7f95add9b640 1 --2- 172.21.15.64:0/46346472 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f95980b65e0 0x7f95980b8af0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f95b0100cc0 tx=0x7f95a400b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:59:56.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.212+0000 7f959f7fe640 1 -- 172.21.15.64:0/46346472 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f95a00d2ae0 con 0x7f95b0102a60 2024-09-21T03:59:56.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T03:59:56.412+0000 7f95b4971640 1 -- 172.21.15.64:0/46346472 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f95b0106340 con 0x7f95980b65e0 2024-09-21T03:59:56.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:56 smithi064 ceph-mon[31180]: purged_snaps scrub starts 2024-09-21T03:59:56.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:56 smithi064 ceph-mon[31180]: purged_snaps scrub ok 2024-09-21T03:59:56.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:56 smithi064 ceph-mon[31180]: osd.8 [v2:172.21.15.131:6816/199192217,v1:172.21.15.131:6817/199192217] boot 2024-09-21T03:59:56.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:56 smithi064 ceph-mon[31180]: osdmap e51: 9 total, 9 up, 9 in 2024-09-21T03:59:56.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:56 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:59:56.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:56 smithi064 ceph-mon[31180]: osdmap e52: 9 total, 9 up, 9 in 2024-09-21T03:59:56.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:56 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:59:56.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:56 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:59:56.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:56 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:57.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:56 smithi119 ceph-mon[34811]: purged_snaps scrub starts 2024-09-21T03:59:57.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:56 smithi119 ceph-mon[34811]: purged_snaps scrub ok 2024-09-21T03:59:57.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:56 smithi119 ceph-mon[34811]: osd.8 [v2:172.21.15.131:6816/199192217,v1:172.21.15.131:6817/199192217] boot 2024-09-21T03:59:57.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:56 smithi119 ceph-mon[34811]: osdmap e51: 9 total, 9 up, 9 in 2024-09-21T03:59:57.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:56 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:59:57.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:56 smithi119 ceph-mon[34811]: osdmap e52: 9 total, 9 up, 9 in 2024-09-21T03:59:57.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:56 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:59:57.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:56 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:59:57.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:56 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:57.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:56 smithi131 ceph-mon[35193]: purged_snaps scrub starts 2024-09-21T03:59:57.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:56 smithi131 ceph-mon[35193]: purged_snaps scrub ok 2024-09-21T03:59:57.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:56 smithi131 ceph-mon[35193]: osd.8 [v2:172.21.15.131:6816/199192217,v1:172.21.15.131:6817/199192217] boot 2024-09-21T03:59:57.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:56 smithi131 ceph-mon[35193]: osdmap e51: 9 total, 9 up, 9 in 2024-09-21T03:59:57.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:56 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T03:59:57.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:56 smithi131 ceph-mon[35193]: osdmap e52: 9 total, 9 up, 9 in 2024-09-21T03:59:57.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:56 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T03:59:57.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:56 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T03:59:57.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:56 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:59:58.148 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:58 smithi064 ceph-mon[31180]: from='client.14631 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:59:58.148 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:58 smithi064 ceph-mon[31180]: pgmap v174: 1 pgs: 1 peering; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:59:58.149 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:58 smithi064 ceph-mon[31180]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T03:59:58.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:58 smithi131 ceph-mon[35193]: from='client.14631 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:59:58.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:58 smithi131 ceph-mon[35193]: pgmap v174: 1 pgs: 1 peering; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:59:58.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:58 smithi131 ceph-mon[35193]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T03:59:58.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:58 smithi119 ceph-mon[34811]: from='client.14631 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:59:58.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:58 smithi119 ceph-mon[34811]: pgmap v174: 1 pgs: 1 peering; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-21T03:59:58.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:58 smithi119 ceph-mon[34811]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T03:59:59.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:59 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1638541981' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "96f769e5-4783-4558-adfa-4bfaf9da9e9a"} : dispatch 2024-09-21T03:59:59.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:59 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1638541981' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96f769e5-4783-4558-adfa-4bfaf9da9e9a"}]': finished 2024-09-21T03:59:59.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:59 smithi131 ceph-mon[35193]: osdmap e54: 10 total, 9 up, 10 in 2024-09-21T03:59:59.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 03:59:59 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:59:59.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:59 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1638541981' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "96f769e5-4783-4558-adfa-4bfaf9da9e9a"} : dispatch 2024-09-21T03:59:59.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:59 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1638541981' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96f769e5-4783-4558-adfa-4bfaf9da9e9a"}]': finished 2024-09-21T03:59:59.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:59 smithi119 ceph-mon[34811]: osdmap e54: 10 total, 9 up, 10 in 2024-09-21T03:59:59.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 03:59:59 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T03:59:59.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:59 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1638541981' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "96f769e5-4783-4558-adfa-4bfaf9da9e9a"} : dispatch 2024-09-21T03:59:59.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:59 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1638541981' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96f769e5-4783-4558-adfa-4bfaf9da9e9a"}]': finished 2024-09-21T03:59:59.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:59 smithi064 ceph-mon[31180]: osdmap e54: 10 total, 9 up, 10 in 2024-09-21T03:59:59.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 03:59:59 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T04:00:00.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:00 smithi131 ceph-mon[35193]: pgmap v176: 1 pgs: 1 peering; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-09-21T04:00:00.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:00 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1854434724' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T04:00:00.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:00 smithi131 ceph-mon[35193]: overall HEALTH_OK 2024-09-21T04:00:00.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:00 smithi119 ceph-mon[34811]: pgmap v176: 1 pgs: 1 peering; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-09-21T04:00:00.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:00 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1854434724' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T04:00:00.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:00 smithi119 ceph-mon[34811]: overall HEALTH_OK 2024-09-21T04:00:00.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:00 smithi064 ceph-mon[31180]: pgmap v176: 1 pgs: 1 peering; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-09-21T04:00:00.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:00 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1854434724' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T04:00:00.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:00 smithi064 ceph-mon[31180]: overall HEALTH_OK 2024-09-21T04:00:02.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:02 smithi131 ceph-mon[35193]: pgmap v178: 1 pgs: 1 peering; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-09-21T04:00:02.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:02 smithi119 ceph-mon[34811]: pgmap v178: 1 pgs: 1 peering; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-09-21T04:00:02.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:02 smithi064 ceph-mon[31180]: pgmap v178: 1 pgs: 1 peering; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-09-21T04:00:04.224 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:04 smithi064 ceph-mon[31180]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-21T04:00:04.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:04 smithi131 ceph-mon[35193]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-21T04:00:04.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:04 smithi119 ceph-mon[34811]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-21T04:00:06.260 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:06 smithi064 ceph-mon[31180]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T04:00:06.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:06 smithi131 ceph-mon[35193]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T04:00:06.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:06 smithi119 ceph-mon[34811]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T04:00:08.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:08 smithi064 ceph-mon[31180]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-21T04:00:08.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:08 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T04:00:08.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:08 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:00:08.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:08 smithi064 ceph-mon[31180]: Deploying daemon osd.9 on smithi064 2024-09-21T04:00:08.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:08 smithi131 ceph-mon[35193]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-21T04:00:08.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:08 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T04:00:08.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:08 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:00:08.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:08 smithi131 ceph-mon[35193]: Deploying daemon osd.9 on smithi064 2024-09-21T04:00:08.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:08 smithi119 ceph-mon[34811]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-21T04:00:08.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:08 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T04:00:08.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:08 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:00:08.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:08 smithi119 ceph-mon[34811]: Deploying daemon osd.9 on smithi064 2024-09-21T04:00:10.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:10 smithi064 ceph-mon[31180]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T04:00:10.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:10 smithi131 ceph-mon[35193]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T04:00:10.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:10 smithi119 ceph-mon[34811]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T04:00:12.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:12 smithi064 ceph-mon[31180]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-21T04:00:12.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:12 smithi131 ceph-mon[35193]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-21T04:00:12.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:12 smithi119 ceph-mon[34811]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-21T04:00:14.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:14 smithi064 ceph-mon[31180]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T04:00:14.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:14 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:00:14.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:14 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:14.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:14 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:14.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:14 smithi131 ceph-mon[35193]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T04:00:14.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:14 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:00:14.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:14 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:14.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:14 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:14.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:14 smithi119 ceph-mon[34811]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T04:00:14.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:14 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:00:14.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:14 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:14.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:14 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:16.424 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:16 smithi064 ceph-mon[31180]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T04:00:16.424 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:16 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:16.424 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:16 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:16.424 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:16 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:00:16.424 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:16 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:00:16.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:16 smithi119 ceph-mon[34811]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T04:00:16.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:16 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:16.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:16 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:16.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:16 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:00:16.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:16 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:00:16.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:16 smithi131 ceph-mon[35193]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T04:00:16.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:16 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:16.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:16 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:16.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:16 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:00:16.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:16 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:00:16.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:00:16.724+0000 7f959f7fe640 1 -- 172.21.15.64:0/46346472 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f95b0106340 con 0x7f95980b65e0 2024-09-21T04:00:16.727 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 9 on host 'smithi064' 2024-09-21T04:00:16.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:00:16.726+0000 7f95b4971640 1 -- 172.21.15.64:0/46346472 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f95980b65e0 msgr2=0x7f95980b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:00:16.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:00:16.726+0000 7f95b4971640 1 --2- 172.21.15.64:0/46346472 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f95980b65e0 0x7f95980b8af0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f95b0100cc0 tx=0x7f95a400b390 comp rx=0 tx=0).stop 2024-09-21T04:00:16.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:00:16.726+0000 7f95b4971640 1 -- 172.21.15.64:0/46346472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95b0102a60 msgr2=0x7f95b019a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:00:16.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:00:16.726+0000 7f95b4971640 1 --2- 172.21.15.64:0/46346472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95b0102a60 0x7f95b019a440 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f95a0005940 tx=0x7f95a00042f0 comp rx=0 tx=0).stop 2024-09-21T04:00:16.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:00:16.727+0000 7f95b4971640 1 -- 172.21.15.64:0/46346472 shutdown_connections 2024-09-21T04:00:16.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:00:16.727+0000 7f95b4971640 1 --2- 172.21.15.64:0/46346472 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f95980b65e0 0x7f95980b8af0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:00:16.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:00:16.727+0000 7f95b4971640 1 --2- 172.21.15.64:0/46346472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95b0108cf0 0x7f95b019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:00:16.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:00:16.727+0000 7f95b4971640 1 --2- 172.21.15.64:0/46346472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95b01083a0 0x7f95b019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:00:16.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:00:16.727+0000 7f95b4971640 1 --2- 172.21.15.64:0/46346472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95b0102a60 0x7f95b019a440 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:00:16.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:00:16.727+0000 7f95b4971640 1 -- 172.21.15.64:0/46346472 >> 172.21.15.64:0/46346472 conn(0x7f95b0079730 msgr2=0x7f95b0101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:00:16.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:00:16.727+0000 7f95b4971640 1 -- 172.21.15.64:0/46346472 shutdown_connections 2024-09-21T04:00:16.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:00:16.727+0000 7f95b4971640 1 -- 172.21.15.64:0/46346472 wait complete. 2024-09-21T04:00:17.327 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:17 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:17.327 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:17 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:00:17.327 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:17 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:17.327 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:17 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:17.378 DEBUG:teuthology.orchestra.run.smithi064:osd.9> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.9.service 2024-09-21T04:00:17.381 INFO:tasks.cephadm:Deploying osd.10 on smithi119 with /dev/vg_nvme/lv_1... 2024-09-21T04:00:17.381 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T04:00:17.409 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:17 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:17.409 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:17 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:00:17.410 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:17 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:17.410 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:17 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:17.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:17 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:17.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:17 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:00:17.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:17 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:17.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:17 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:17.638 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T04:00:18.326 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:18 smithi064 ceph-mon[31180]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T04:00:18.480 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:18 smithi119 ceph-mon[34811]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T04:00:18.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:18 smithi131 ceph-mon[35193]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T04:00:19.171 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:19 smithi119 ceph-mon[34811]: from='osd.9 [v2:172.21.15.64:6826/3309038990,v1:172.21.15.64:6827/3309038990]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T04:00:19.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:19 smithi064 ceph-mon[31180]: from='osd.9 [v2:172.21.15.64:6826/3309038990,v1:172.21.15.64:6827/3309038990]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T04:00:19.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:00:18 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:00:18.993+0000 7f8c339a4640 -1 osd.9 0 log_to_monitors true 2024-09-21T04:00:19.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:19 smithi131 ceph-mon[35193]: from='osd.9 [v2:172.21.15.64:6826/3309038990,v1:172.21.15.64:6827/3309038990]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T04:00:20.261 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:00:20 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:00:20.076+0000 7f8c2f926640 -1 osd.9 0 waiting for initial osdmap 2024-09-21T04:00:20.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:00:20 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:00:20.090+0000 7f8c2b724640 -1 osd.9 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T04:00:20.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:20 smithi064 ceph-mon[31180]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T04:00:20.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:20 smithi064 ceph-mon[31180]: from='osd.9 [v2:172.21.15.64:6826/3309038990,v1:172.21.15.64:6827/3309038990]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T04:00:20.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:20 smithi064 ceph-mon[31180]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T04:00:20.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:20 smithi064 ceph-mon[31180]: from='osd.9 [v2:172.21.15.64:6826/3309038990,v1:172.21.15.64:6827/3309038990]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]} : dispatch 2024-09-21T04:00:20.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:20 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T04:00:20.409 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:20 smithi119 ceph-mon[34811]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T04:00:20.409 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:20 smithi119 ceph-mon[34811]: from='osd.9 [v2:172.21.15.64:6826/3309038990,v1:172.21.15.64:6827/3309038990]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T04:00:20.409 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:20 smithi119 ceph-mon[34811]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T04:00:20.409 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:20 smithi119 ceph-mon[34811]: from='osd.9 [v2:172.21.15.64:6826/3309038990,v1:172.21.15.64:6827/3309038990]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]} : dispatch 2024-09-21T04:00:20.409 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:20 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T04:00:20.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:20 smithi131 ceph-mon[35193]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T04:00:20.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:20 smithi131 ceph-mon[35193]: from='osd.9 [v2:172.21.15.64:6826/3309038990,v1:172.21.15.64:6827/3309038990]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T04:00:20.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:20 smithi131 ceph-mon[35193]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T04:00:20.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:20 smithi131 ceph-mon[35193]: from='osd.9 [v2:172.21.15.64:6826/3309038990,v1:172.21.15.64:6827/3309038990]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]} : dispatch 2024-09-21T04:00:20.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:20 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T04:00:20.563 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-09-21T04:00:20.588 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi119:vg_nvme/lv_1 2024-09-21T04:00:20.871 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T04:00:21.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:21 smithi119 ceph-mon[34811]: from='osd.9 [v2:172.21.15.64:6826/3309038990,v1:172.21.15.64:6827/3309038990]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-21T04:00:21.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:21 smithi119 ceph-mon[34811]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T04:00:21.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:21 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T04:00:21.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:21 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T04:00:21.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:21 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:21.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:21 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:21.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:21 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T04:00:21.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:21 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T04:00:21.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:21 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T04:00:21.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:21 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T04:00:21.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:21 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:21.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:21 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:00:21.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:21 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:00:21.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:21 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:21.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.377+0000 7fd2c8d96640 1 -- 172.21.15.119:0/3122670089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2c4103bc0 msgr2=0x7fd2c4103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:00:21.385 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.377+0000 7fd2c8d96640 1 --2- 172.21.15.119:0/3122670089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2c4103bc0 0x7fd2c4103fd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009900 tx=0x7fd2bc02a410 comp rx=0 tx=0).stop 2024-09-21T04:00:21.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.377+0000 7fd2c8d96640 1 -- 172.21.15.119:0/3122670089 shutdown_connections 2024-09-21T04:00:21.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.377+0000 7fd2c8d96640 1 --2- 172.21.15.119:0/3122670089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2c4109e00 0x7fd2c4110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:00:21.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.377+0000 7fd2c8d96640 1 --2- 172.21.15.119:0/3122670089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2c41094b0 0x7fd2c41098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:00:21.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.377+0000 7fd2c8d96640 1 --2- 172.21.15.119:0/3122670089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2c4103bc0 0x7fd2c4103fd0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:00:21.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.377+0000 7fd2c8d96640 1 -- 172.21.15.119:0/3122670089 >> 172.21.15.119:0/3122670089 conn(0x7fd2c407b910 msgr2=0x7fd2c407bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:00:21.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.377+0000 7fd2c8d96640 1 -- 172.21.15.119:0/3122670089 shutdown_connections 2024-09-21T04:00:21.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.377+0000 7fd2c8d96640 1 -- 172.21.15.119:0/3122670089 wait complete. 2024-09-21T04:00:21.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.378+0000 7fd2c8d96640 1 Processor -- start 2024-09-21T04:00:21.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.378+0000 7fd2c8d96640 1 -- start start 2024-09-21T04:00:21.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.378+0000 7fd2c8d96640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2c4103bc0 0x7fd2c419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:00:21.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.378+0000 7fd2c8d96640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2c41094b0 0x7fd2c419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:00:21.386 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.378+0000 7fd2c8d96640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2c4109e00 0x7fd2c419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:00:21.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.378+0000 7fd2c8d96640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c4113ac0 con 0x7fd2c41094b0 2024-09-21T04:00:21.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.378+0000 7fd2c8d96640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c4113940 con 0x7fd2c4103bc0 2024-09-21T04:00:21.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.378+0000 7fd2c8d96640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c4113c40 con 0x7fd2c4109e00 2024-09-21T04:00:21.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.378+0000 7fd2c259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2c4103bc0 0x7fd2c419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:00:21.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.378+0000 7fd2c259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2c4103bc0 0x7fd2c419a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45826/0 (socket says 172.21.15.119:45826) 2024-09-21T04:00:21.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.378+0000 7fd2c259c640 1 -- 172.21.15.119:0/2949693894 learned_addr learned my addr 172.21.15.119:0/2949693894 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-09-21T04:00:21.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.378+0000 7fd2c2d9d640 1 --2- 172.21.15.119:0/2949693894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2c4109e00 0x7fd2c419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:00:21.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.379+0000 7fd2c259c640 1 -- 172.21.15.119:0/2949693894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2c4109e00 msgr2=0x7fd2c419fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:00:21.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.379+0000 7fd2c259c640 1 --2- 172.21.15.119:0/2949693894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2c4109e00 0x7fd2c419fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:00:21.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.379+0000 7fd2c259c640 1 -- 172.21.15.119:0/2949693894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2c41094b0 msgr2=0x7fd2c419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:00:21.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.379+0000 7fd2c259c640 1 --2- 172.21.15.119:0/2949693894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2c41094b0 0x7fd2c419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:00:21.387 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.379+0000 7fd2c259c640 1 -- 172.21.15.119:0/2949693894 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c41a03e0 con 0x7fd2c4103bc0 2024-09-21T04:00:21.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.379+0000 7fd2c2d9d640 1 --2- 172.21.15.119:0/2949693894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2c4109e00 0x7fd2c419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:00:21.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.379+0000 7fd2c259c640 1 --2- 172.21.15.119:0/2949693894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2c4103bc0 0x7fd2c419a470 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc005940 tx=0x7fd2bc0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:00:21.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.379+0000 7fd2b37fe640 1 -- 172.21.15.119:0/2949693894 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc03f030 con 0x7fd2c4103bc0 2024-09-21T04:00:21.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.379+0000 7fd2c8d96640 1 -- 172.21.15.119:0/2949693894 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c41a0670 con 0x7fd2c4103bc0 2024-09-21T04:00:21.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.379+0000 7fd2b37fe640 1 -- 172.21.15.119:0/2949693894 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2bc0044c0 con 0x7fd2c4103bc0 2024-09-21T04:00:21.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.379+0000 7fd2b37fe640 1 -- 172.21.15.119:0/2949693894 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc035070 con 0x7fd2c4103bc0 2024-09-21T04:00:21.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.379+0000 7fd2c8d96640 1 -- 172.21.15.119:0/2949693894 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c41a6e30 con 0x7fd2c4103bc0 2024-09-21T04:00:21.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.380+0000 7fd2c8d96640 1 -- 172.21.15.119:0/2949693894 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2c4103fd0 con 0x7fd2c4103bc0 2024-09-21T04:00:21.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.381+0000 7fd2b37fe640 1 -- 172.21.15.119:0/2949693894 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fd2bc042030 con 0x7fd2c4103bc0 2024-09-21T04:00:21.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.382+0000 7fd2b37fe640 1 --2- 172.21.15.119:0/2949693894 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd2ac0b6560 0x7fd2ac0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:00:21.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.382+0000 7fd2b37fe640 1 -- 172.21.15.119:0/2949693894 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6476+0+0 (secure 0 0 0) 0x7fd2bc107350 con 0x7fd2c4103bc0 2024-09-21T04:00:21.388 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.382+0000 7fd2c1d9b640 1 --2- 172.21.15.119:0/2949693894 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd2ac0b6560 0x7fd2ac0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:00:21.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.383+0000 7fd2c1d9b640 1 --2- 172.21.15.119:0/2949693894 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd2ac0b6560 0x7fd2ac0b8a70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8005c40 tx=0x7fd2b8005bd0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:00:21.389 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.385+0000 7fd2b37fe640 1 -- 172.21.15.119:0/2949693894 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd2bc0ce310 con 0x7fd2c4103bc0 2024-09-21T04:00:21.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:21 smithi064 ceph-mon[31180]: from='osd.9 [v2:172.21.15.64:6826/3309038990,v1:172.21.15.64:6827/3309038990]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-21T04:00:21.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:21 smithi064 ceph-mon[31180]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T04:00:21.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:21 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T04:00:21.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:21 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T04:00:21.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:21 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:21.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:21 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:21.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:21 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T04:00:21.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:21 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T04:00:21.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:21 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T04:00:21.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:21 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T04:00:21.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:21 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:21.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:21 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:00:21.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:21 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:00:21.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:21 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:21.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:21 smithi131 ceph-mon[35193]: from='osd.9 [v2:172.21.15.64:6826/3309038990,v1:172.21.15.64:6827/3309038990]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-09-21T04:00:21.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:21 smithi131 ceph-mon[35193]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T04:00:21.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:21 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T04:00:21.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:21 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T04:00:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:21 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:21 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:21 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T04:00:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:21 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T04:00:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:21 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T04:00:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:21 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T04:00:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:21 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:21 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:00:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:21 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:00:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:21 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:21.575 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:21.573+0000 7fd2c8d96640 1 -- 172.21.15.119:0/2949693894 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd2c4063800 con 0x7fd2ac0b6560 2024-09-21T04:00:22.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:22 smithi119 ceph-mon[34811]: purged_snaps scrub starts 2024-09-21T04:00:22.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:22 smithi119 ceph-mon[34811]: purged_snaps scrub ok 2024-09-21T04:00:22.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:22 smithi119 ceph-mon[34811]: Detected new or changed devices on smithi064 2024-09-21T04:00:22.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:22 smithi119 ceph-mon[34811]: Adjusting osd_memory_target on smithi064 to 4272M 2024-09-21T04:00:22.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:22 smithi119 ceph-mon[34811]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T04:00:22.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:22 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T04:00:22.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:22 smithi119 ceph-mon[34811]: osd.9 [v2:172.21.15.64:6826/3309038990,v1:172.21.15.64:6827/3309038990] boot 2024-09-21T04:00:22.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:22 smithi119 ceph-mon[34811]: osdmap e57: 10 total, 10 up, 10 in 2024-09-21T04:00:22.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:22 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T04:00:22.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:22 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T04:00:22.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:22 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T04:00:22.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:22 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:00:22.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:22 smithi064 ceph-mon[31180]: purged_snaps scrub starts 2024-09-21T04:00:22.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:22 smithi064 ceph-mon[31180]: purged_snaps scrub ok 2024-09-21T04:00:22.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:22 smithi064 ceph-mon[31180]: Detected new or changed devices on smithi064 2024-09-21T04:00:22.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:22 smithi064 ceph-mon[31180]: Adjusting osd_memory_target on smithi064 to 4272M 2024-09-21T04:00:22.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:22 smithi064 ceph-mon[31180]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T04:00:22.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:22 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T04:00:22.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:22 smithi064 ceph-mon[31180]: osd.9 [v2:172.21.15.64:6826/3309038990,v1:172.21.15.64:6827/3309038990] boot 2024-09-21T04:00:22.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:22 smithi064 ceph-mon[31180]: osdmap e57: 10 total, 10 up, 10 in 2024-09-21T04:00:22.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:22 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T04:00:22.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:22 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T04:00:22.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:22 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T04:00:22.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:22 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:00:22.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:22 smithi131 ceph-mon[35193]: purged_snaps scrub starts 2024-09-21T04:00:22.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:22 smithi131 ceph-mon[35193]: purged_snaps scrub ok 2024-09-21T04:00:22.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:22 smithi131 ceph-mon[35193]: Detected new or changed devices on smithi064 2024-09-21T04:00:22.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:22 smithi131 ceph-mon[35193]: Adjusting osd_memory_target on smithi064 to 4272M 2024-09-21T04:00:22.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:22 smithi131 ceph-mon[35193]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-21T04:00:22.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:22 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T04:00:22.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:22 smithi131 ceph-mon[35193]: osd.9 [v2:172.21.15.64:6826/3309038990,v1:172.21.15.64:6827/3309038990] boot 2024-09-21T04:00:22.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:22 smithi131 ceph-mon[35193]: osdmap e57: 10 total, 10 up, 10 in 2024-09-21T04:00:22.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:22 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T04:00:22.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:22 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T04:00:22.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:22 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T04:00:22.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:22 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:00:23.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:23 smithi131 ceph-mon[35193]: from='client.24524 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:00:23.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:23 smithi131 ceph-mon[35193]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T04:00:23.368 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:23 smithi119 ceph-mon[34811]: from='client.24524 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:00:23.368 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:23 smithi119 ceph-mon[34811]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T04:00:23.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:23 smithi064 ceph-mon[31180]: from='client.24524 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:00:23.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:23 smithi064 ceph-mon[31180]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T04:00:24.351 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:24 smithi119 ceph-mon[34811]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:24.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:24 smithi064 ceph-mon[31180]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:24.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:24 smithi131 ceph-mon[35193]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:25.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:25 smithi119 ceph-mon[34811]: from='client.? 172.21.15.119:0/2855414963' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "13c1ef5c-2299-440c-af5a-a650b47f4b75"} : dispatch 2024-09-21T04:00:25.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:25 smithi119 ceph-mon[34811]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "13c1ef5c-2299-440c-af5a-a650b47f4b75"} : dispatch 2024-09-21T04:00:25.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:25 smithi119 ceph-mon[34811]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13c1ef5c-2299-440c-af5a-a650b47f4b75"}]': finished 2024-09-21T04:00:25.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:25 smithi119 ceph-mon[34811]: osdmap e59: 11 total, 10 up, 11 in 2024-09-21T04:00:25.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:25 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T04:00:25.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:25 smithi119 ceph-mon[34811]: from='client.? 172.21.15.119:0/741953431' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T04:00:25.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:25 smithi064 ceph-mon[31180]: from='client.? 172.21.15.119:0/2855414963' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "13c1ef5c-2299-440c-af5a-a650b47f4b75"} : dispatch 2024-09-21T04:00:25.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:25 smithi064 ceph-mon[31180]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "13c1ef5c-2299-440c-af5a-a650b47f4b75"} : dispatch 2024-09-21T04:00:25.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:25 smithi064 ceph-mon[31180]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13c1ef5c-2299-440c-af5a-a650b47f4b75"}]': finished 2024-09-21T04:00:25.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:25 smithi064 ceph-mon[31180]: osdmap e59: 11 total, 10 up, 11 in 2024-09-21T04:00:25.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:25 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T04:00:25.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:25 smithi064 ceph-mon[31180]: from='client.? 172.21.15.119:0/741953431' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T04:00:25.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:25 smithi131 ceph-mon[35193]: from='client.? 172.21.15.119:0/2855414963' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "13c1ef5c-2299-440c-af5a-a650b47f4b75"} : dispatch 2024-09-21T04:00:25.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:25 smithi131 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "13c1ef5c-2299-440c-af5a-a650b47f4b75"} : dispatch 2024-09-21T04:00:25.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:25 smithi131 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13c1ef5c-2299-440c-af5a-a650b47f4b75"}]': finished 2024-09-21T04:00:25.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:25 smithi131 ceph-mon[35193]: osdmap e59: 11 total, 10 up, 11 in 2024-09-21T04:00:25.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:25 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T04:00:25.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:25 smithi131 ceph-mon[35193]: from='client.? 172.21.15.119:0/741953431' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T04:00:26.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:26 smithi119 ceph-mon[34811]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:26.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:26 smithi064 ceph-mon[31180]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:26.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:26 smithi131 ceph-mon[35193]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:28.474 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:28 smithi119 ceph-mon[34811]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:28.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:28 smithi064 ceph-mon[31180]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:28.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:28 smithi131 ceph-mon[35193]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:30.231 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:30 smithi119 ceph-mon[34811]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:30.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:30 smithi064 ceph-mon[31180]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:30.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:30 smithi131 ceph-mon[35193]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:32.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:32 smithi119 ceph-mon[34811]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:32.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:32 smithi064 ceph-mon[31180]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:32.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:32 smithi131 ceph-mon[35193]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:33.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:33 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T04:00:33.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:33 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:00:33.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:33 smithi119 ceph-mon[34811]: Deploying daemon osd.10 on smithi119 2024-09-21T04:00:33.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:33 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T04:00:33.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:33 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:00:33.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:33 smithi064 ceph-mon[31180]: Deploying daemon osd.10 on smithi119 2024-09-21T04:00:33.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:33 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T04:00:33.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:33 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:00:33.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:33 smithi131 ceph-mon[35193]: Deploying daemon osd.10 on smithi119 2024-09-21T04:00:34.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:34 smithi119 ceph-mon[34811]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:34.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:34 smithi064 ceph-mon[31180]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:34.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:34 smithi131 ceph-mon[35193]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:36.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:36 smithi119 ceph-mon[34811]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:36.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:36 smithi064 ceph-mon[31180]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:36.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:36 smithi131 ceph-mon[35193]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:38.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:38 smithi119 ceph-mon[34811]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:38.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:38 smithi064 ceph-mon[31180]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:38.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:38 smithi131 ceph-mon[35193]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:39.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:39 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:00:39.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:39 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:39.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:39 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:39.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:39 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:00:39.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:39 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:39.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:39 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:39.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:39 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:00:39.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:39 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:39.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:39 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:40.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:40 smithi119 ceph-mon[34811]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:40.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:40 smithi064 ceph-mon[31180]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:40.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:40 smithi131 ceph-mon[35193]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:41.745 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 10 on host 'smithi119' 2024-09-21T04:00:41.745 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:41.742+0000 7fd2b37fe640 1 -- 172.21.15.119:0/2949693894 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd2c4063800 con 0x7fd2ac0b6560 2024-09-21T04:00:41.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:41.744+0000 7fd2c8d96640 1 -- 172.21.15.119:0/2949693894 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd2ac0b6560 msgr2=0x7fd2ac0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:00:41.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:41.744+0000 7fd2c8d96640 1 --2- 172.21.15.119:0/2949693894 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd2ac0b6560 0x7fd2ac0b8a70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8005c40 tx=0x7fd2b8005bd0 comp rx=0 tx=0).stop 2024-09-21T04:00:41.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:41.744+0000 7fd2c8d96640 1 -- 172.21.15.119:0/2949693894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2c4103bc0 msgr2=0x7fd2c419a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:00:41.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:41.744+0000 7fd2c8d96640 1 --2- 172.21.15.119:0/2949693894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2c4103bc0 0x7fd2c419a470 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc005940 tx=0x7fd2bc0040c0 comp rx=0 tx=0).stop 2024-09-21T04:00:41.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:41.744+0000 7fd2c8d96640 1 -- 172.21.15.119:0/2949693894 shutdown_connections 2024-09-21T04:00:41.747 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:41.744+0000 7fd2c8d96640 1 --2- 172.21.15.119:0/2949693894 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd2ac0b6560 0x7fd2ac0b8a70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:00:41.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:41.744+0000 7fd2c8d96640 1 --2- 172.21.15.119:0/2949693894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2c4109e00 0x7fd2c419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:00:41.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:41.744+0000 7fd2c8d96640 1 --2- 172.21.15.119:0/2949693894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2c41094b0 0x7fd2c419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:00:41.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:41.744+0000 7fd2c8d96640 1 --2- 172.21.15.119:0/2949693894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2c4103bc0 0x7fd2c419a470 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:00:41.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:41.744+0000 7fd2c8d96640 1 -- 172.21.15.119:0/2949693894 >> 172.21.15.119:0/2949693894 conn(0x7fd2c407b910 msgr2=0x7fd2c4107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:00:41.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:41.744+0000 7fd2c8d96640 1 -- 172.21.15.119:0/2949693894 shutdown_connections 2024-09-21T04:00:41.748 INFO:teuthology.orchestra.run.smithi119.stderr:2024-09-21T04:00:41.744+0000 7fd2c8d96640 1 -- 172.21.15.119:0/2949693894 wait complete. 2024-09-21T04:00:42.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:42 smithi131 ceph-mon[35193]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:42.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:42 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:42.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:42 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:42.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:42 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:00:42.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:42 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:00:42.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:42 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:42.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:42 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:00:42.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:42 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:42.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:42 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:42.373 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:42 smithi119 ceph-mon[34811]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:42.373 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:42 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:42.373 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:42 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:42.373 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:42 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:00:42.373 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:42 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:00:42.373 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:42 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:42.373 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:42 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:00:42.373 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:42 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:42.374 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:42 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:42.399 DEBUG:teuthology.orchestra.run.smithi119:osd.10> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.10.service 2024-09-21T04:00:42.401 INFO:tasks.cephadm:Deploying osd.11 on smithi131 with /dev/vg_nvme/lv_1... 2024-09-21T04:00:42.401 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T04:00:42.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:42 smithi064 ceph-mon[31180]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:42.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:42 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:42.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:42 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:42.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:42 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:00:42.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:42 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:00:42.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:42 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:42.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:42 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:00:42.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:42 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:42.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:42 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:42.654 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T04:00:44.221 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:44 smithi131 ceph-mon[35193]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:44.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:44 smithi119 ceph-mon[34811]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:44.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:44 smithi064 ceph-mon[31180]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:44.574 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:00:44 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:00:44.367+0000 7f52db322640 -1 osd.10 0 log_to_monitors true 2024-09-21T04:00:45.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:45 smithi119 ceph-mon[34811]: from='osd.10 [v2:172.21.15.119:6824/1460844681,v1:172.21.15.119:6825/1460844681]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T04:00:45.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:45 smithi119 ceph-mon[34811]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T04:00:45.301 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:45 smithi131 ceph-mon[35193]: from='osd.10 [v2:172.21.15.119:6824/1460844681,v1:172.21.15.119:6825/1460844681]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T04:00:45.301 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:45 smithi131 ceph-mon[35193]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T04:00:45.304 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-21T04:00:45.330 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi131:vg_nvme/lv_1 2024-09-21T04:00:45.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:45 smithi064 ceph-mon[31180]: from='osd.10 [v2:172.21.15.119:6824/1460844681,v1:172.21.15.119:6825/1460844681]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T04:00:45.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:45 smithi064 ceph-mon[31180]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T04:00:45.581 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T04:00:46.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.065+0000 7f8faeceb640 1 -- 172.21.15.131:0/64645339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fa81083a0 msgr2=0x7f8fa81087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:00:46.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.065+0000 7f8faeceb640 1 --2- 172.21.15.131:0/64645339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fa81083a0 0x7f8fa81087b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8f90009900 tx=0x7f8f9002a410 comp rx=0 tx=0).stop 2024-09-21T04:00:46.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.065+0000 7f8faeceb640 1 -- 172.21.15.131:0/64645339 shutdown_connections 2024-09-21T04:00:46.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.065+0000 7f8faeceb640 1 --2- 172.21.15.131:0/64645339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa8108cf0 0x7f8fa810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:00:46.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.065+0000 7f8faeceb640 1 --2- 172.21.15.131:0/64645339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fa81083a0 0x7f8fa81087b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:00:46.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.065+0000 7f8faeceb640 1 --2- 172.21.15.131:0/64645339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fa8102a60 0x7f8fa8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:00:46.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.065+0000 7f8faeceb640 1 -- 172.21.15.131:0/64645339 >> 172.21.15.131:0/64645339 conn(0x7f8fa8079730 msgr2=0x7f8fa8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:00:46.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.065+0000 7f8faeceb640 1 -- 172.21.15.131:0/64645339 shutdown_connections 2024-09-21T04:00:46.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.065+0000 7f8faeceb640 1 -- 172.21.15.131:0/64645339 wait complete. 2024-09-21T04:00:46.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.066+0000 7f8faeceb640 1 Processor -- start 2024-09-21T04:00:46.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.066+0000 7f8faeceb640 1 -- start start 2024-09-21T04:00:46.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.066+0000 7f8faeceb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fa8102a60 0x7f8fa80ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:00:46.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.066+0000 7f8faeceb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa81083a0 0x7f8fa80ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:00:46.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.066+0000 7f8faeceb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fa8108cf0 0x7f8fa81000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:00:46.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.066+0000 7f8faeceb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa8112960 con 0x7f8fa81083a0 2024-09-21T04:00:46.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.066+0000 7f8faeceb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa81127e0 con 0x7f8fa8108cf0 2024-09-21T04:00:46.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.066+0000 7f8faeceb640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa8112ae0 con 0x7f8fa8102a60 2024-09-21T04:00:46.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.066+0000 7f8faca87640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fa8102a60 0x7f8fa80ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:00:46.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.066+0000 7f8faca87640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fa8102a60 0x7f8fa80ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52962/0 (socket says 172.21.15.131:52962) 2024-09-21T04:00:46.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.066+0000 7f8faca87640 1 -- 172.21.15.131:0/148651225 learned_addr learned my addr 172.21.15.131:0/148651225 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-21T04:00:46.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.066+0000 7f8fad288640 1 --2- 172.21.15.131:0/148651225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fa8108cf0 0x7f8fa81000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:00:46.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.066+0000 7f8fa7fff640 1 --2- 172.21.15.131:0/148651225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa81083a0 0x7f8fa80ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:00:46.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.067+0000 7f8faca87640 1 -- 172.21.15.131:0/148651225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fa8108cf0 msgr2=0x7f8fa81000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:00:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.067+0000 7f8faca87640 1 --2- 172.21.15.131:0/148651225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fa8108cf0 0x7f8fa81000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:00:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.067+0000 7f8faca87640 1 -- 172.21.15.131:0/148651225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa81083a0 msgr2=0x7f8fa80ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:00:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.067+0000 7f8faca87640 1 --2- 172.21.15.131:0/148651225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa81083a0 0x7f8fa80ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:00:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.067+0000 7f8faca87640 1 -- 172.21.15.131:0/148651225 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa81a87a0 con 0x7f8fa8102a60 2024-09-21T04:00:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.067+0000 7f8fa7fff640 1 --2- 172.21.15.131:0/148651225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa81083a0 0x7f8fa80ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:00:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.067+0000 7f8faca87640 1 --2- 172.21.15.131:0/148651225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fa8102a60 0x7f8fa80ff670 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800d950 tx=0x7f8f9800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:00:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.067+0000 7f8fa5ffb640 1 -- 172.21.15.131:0/148651225 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f98017070 con 0x7f8fa8102a60 2024-09-21T04:00:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.067+0000 7f8fa5ffb640 1 -- 172.21.15.131:0/148651225 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f98009c30 con 0x7f8fa8102a60 2024-09-21T04:00:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.067+0000 7f8faeceb640 1 -- 172.21.15.131:0/148651225 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fa81a8a90 con 0x7f8fa8102a60 2024-09-21T04:00:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.067+0000 7f8fa5ffb640 1 -- 172.21.15.131:0/148651225 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9800f620 con 0x7f8fa8102a60 2024-09-21T04:00:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.067+0000 7f8faeceb640 1 -- 172.21.15.131:0/148651225 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fa81a8fd0 con 0x7f8fa8102a60 2024-09-21T04:00:46.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.068+0000 7f8faeceb640 1 -- 172.21.15.131:0/148651225 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fa8102e70 con 0x7f8fa8102a60 2024-09-21T04:00:46.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.070+0000 7f8fa5ffb640 1 -- 172.21.15.131:0/148651225 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f8f98010460 con 0x7f8fa8102a60 2024-09-21T04:00:46.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.070+0000 7f8fa5ffb640 1 --2- 172.21.15.131:0/148651225 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8f940b6560 0x7f8f940b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:00:46.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.071+0000 7f8fa5ffb640 1 -- 172.21.15.131:0/148651225 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6629+0+0 (secure 0 0 0) 0x7f8f980e6940 con 0x7f8fa8102a60 2024-09-21T04:00:46.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.071+0000 7f8fa7fff640 1 --2- 172.21.15.131:0/148651225 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8f940b6560 0x7f8f940b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:00:46.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.071+0000 7f8fa7fff640 1 --2- 172.21.15.131:0/148651225 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8f940b6560 0x7f8f940b8a70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8f900096c0 tx=0x7f8f90031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:00:46.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.074+0000 7f8fa5ffb640 1 -- 172.21.15.131:0/148651225 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8f980ad860 con 0x7f8fa8102a60 2024-09-21T04:00:46.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:00:46.264+0000 7f8faeceb640 1 -- 172.21.15.131:0/148651225 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8fa8063800 con 0x7f8f940b6560 2024-09-21T04:00:46.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:46 smithi131 ceph-mon[35193]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:46.266 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:46 smithi131 ceph-mon[35193]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T04:00:46.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:46 smithi131 ceph-mon[35193]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T04:00:46.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:46 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T04:00:46.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:46 smithi131 ceph-mon[35193]: from='osd.10 [v2:172.21.15.119:6824/1460844681,v1:172.21.15.119:6825/1460844681]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T04:00:46.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:46 smithi131 ceph-mon[35193]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T04:00:46.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:46 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:46.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:46 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:46.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:46 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T04:00:46.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:46 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T04:00:46.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:46 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T04:00:46.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:46 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T04:00:46.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:46 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:46.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:46 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:00:46.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:46 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:00:46.267 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:46 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:46.506 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:00:46 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:00:46.041+0000 7f52d72a4640 -1 osd.10 0 waiting for initial osdmap 2024-09-21T04:00:46.506 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:00:46 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:00:46.054+0000 7f52d30a2640 -1 osd.10 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T04:00:46.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:46 smithi119 ceph-mon[34811]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:46.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:46 smithi119 ceph-mon[34811]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T04:00:46.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:46 smithi119 ceph-mon[34811]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T04:00:46.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:46 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T04:00:46.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:46 smithi119 ceph-mon[34811]: from='osd.10 [v2:172.21.15.119:6824/1460844681,v1:172.21.15.119:6825/1460844681]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T04:00:46.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:46 smithi119 ceph-mon[34811]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T04:00:46.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:46 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:46.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:46 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:46.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:46 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T04:00:46.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:46 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T04:00:46.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:46 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T04:00:46.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:46 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T04:00:46.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:46 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:46.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:46 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:00:46.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:46 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:00:46.508 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:46 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:46.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:46 smithi064 ceph-mon[31180]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:46.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:46 smithi064 ceph-mon[31180]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T04:00:46.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:46 smithi064 ceph-mon[31180]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T04:00:46.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:46 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T04:00:46.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:46 smithi064 ceph-mon[31180]: from='osd.10 [v2:172.21.15.119:6824/1460844681,v1:172.21.15.119:6825/1460844681]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T04:00:46.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:46 smithi064 ceph-mon[31180]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]} : dispatch 2024-09-21T04:00:46.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:46 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:46.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:46 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:46.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:46 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T04:00:46.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:46 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T04:00:46.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:46 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T04:00:46.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:46 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T04:00:46.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:46 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:46.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:46 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:00:46.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:46 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:00:46.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:46 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:00:47.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:47 smithi131 ceph-mon[35193]: Detected new or changed devices on smithi119 2024-09-21T04:00:47.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:47 smithi131 ceph-mon[35193]: Adjusting osd_memory_target on smithi119 to 4272M 2024-09-21T04:00:47.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:47 smithi131 ceph-mon[35193]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-09-21T04:00:47.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:47 smithi131 ceph-mon[35193]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T04:00:47.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:47 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T04:00:47.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:47 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T04:00:47.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:47 smithi131 ceph-mon[35193]: from='client.24529 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:00:47.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:47 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T04:00:47.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:47 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T04:00:47.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:47 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:00:47.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:47 smithi131 ceph-mon[35193]: osd.10 [v2:172.21.15.119:6824/1460844681,v1:172.21.15.119:6825/1460844681] boot 2024-09-21T04:00:47.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:47 smithi131 ceph-mon[35193]: osdmap e62: 11 total, 11 up, 11 in 2024-09-21T04:00:47.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:47 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T04:00:47.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:47 smithi119 ceph-mon[34811]: Detected new or changed devices on smithi119 2024-09-21T04:00:47.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:47 smithi119 ceph-mon[34811]: Adjusting osd_memory_target on smithi119 to 4272M 2024-09-21T04:00:47.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:47 smithi119 ceph-mon[34811]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-09-21T04:00:47.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:47 smithi119 ceph-mon[34811]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T04:00:47.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:47 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T04:00:47.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:47 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T04:00:47.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:47 smithi119 ceph-mon[34811]: from='client.24529 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:00:47.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:47 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T04:00:47.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:47 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T04:00:47.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:47 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:00:47.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:47 smithi119 ceph-mon[34811]: osd.10 [v2:172.21.15.119:6824/1460844681,v1:172.21.15.119:6825/1460844681] boot 2024-09-21T04:00:47.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:47 smithi119 ceph-mon[34811]: osdmap e62: 11 total, 11 up, 11 in 2024-09-21T04:00:47.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:47 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T04:00:47.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:47 smithi064 ceph-mon[31180]: Detected new or changed devices on smithi119 2024-09-21T04:00:47.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:47 smithi064 ceph-mon[31180]: Adjusting osd_memory_target on smithi119 to 4272M 2024-09-21T04:00:47.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:47 smithi064 ceph-mon[31180]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-09-21T04:00:47.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:47 smithi064 ceph-mon[31180]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T04:00:47.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:47 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T04:00:47.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:47 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T04:00:47.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:47 smithi064 ceph-mon[31180]: from='client.24529 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:00:47.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:47 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T04:00:47.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:47 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T04:00:47.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:47 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:00:47.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:47 smithi064 ceph-mon[31180]: osd.10 [v2:172.21.15.119:6824/1460844681,v1:172.21.15.119:6825/1460844681] boot 2024-09-21T04:00:47.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:47 smithi064 ceph-mon[31180]: osdmap e62: 11 total, 11 up, 11 in 2024-09-21T04:00:47.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:47 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T04:00:48.235 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:48 smithi131 ceph-mon[35193]: purged_snaps scrub starts 2024-09-21T04:00:48.235 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:48 smithi131 ceph-mon[35193]: purged_snaps scrub ok 2024-09-21T04:00:48.235 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:48 smithi131 ceph-mon[35193]: pgmap v208: 1 pgs: 1 peering; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:48.236 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:48 smithi131 ceph-mon[35193]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T04:00:48.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:48 smithi119 ceph-mon[34811]: purged_snaps scrub starts 2024-09-21T04:00:48.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:48 smithi119 ceph-mon[34811]: purged_snaps scrub ok 2024-09-21T04:00:48.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:48 smithi119 ceph-mon[34811]: pgmap v208: 1 pgs: 1 peering; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:48.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:48 smithi119 ceph-mon[34811]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T04:00:48.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:48 smithi064 ceph-mon[31180]: purged_snaps scrub starts 2024-09-21T04:00:48.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:48 smithi064 ceph-mon[31180]: purged_snaps scrub ok 2024-09-21T04:00:48.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:48 smithi064 ceph-mon[31180]: pgmap v208: 1 pgs: 1 peering; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-21T04:00:48.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:48 smithi064 ceph-mon[31180]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T04:00:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:49 smithi131 ceph-mon[35193]: from='client.? 172.21.15.131:0/1301084753' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c26a5ef9-8585-404e-a54a-852424c4d6fd"} : dispatch 2024-09-21T04:00:49.282 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:49 smithi131 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c26a5ef9-8585-404e-a54a-852424c4d6fd"} : dispatch 2024-09-21T04:00:49.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:49 smithi131 ceph-mon[35193]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c26a5ef9-8585-404e-a54a-852424c4d6fd"}]': finished 2024-09-21T04:00:49.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:49 smithi131 ceph-mon[35193]: osdmap e64: 12 total, 11 up, 12 in 2024-09-21T04:00:49.283 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:49 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T04:00:49.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:49 smithi119 ceph-mon[34811]: from='client.? 172.21.15.131:0/1301084753' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c26a5ef9-8585-404e-a54a-852424c4d6fd"} : dispatch 2024-09-21T04:00:49.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:49 smithi119 ceph-mon[34811]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c26a5ef9-8585-404e-a54a-852424c4d6fd"} : dispatch 2024-09-21T04:00:49.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:49 smithi119 ceph-mon[34811]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c26a5ef9-8585-404e-a54a-852424c4d6fd"}]': finished 2024-09-21T04:00:49.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:49 smithi119 ceph-mon[34811]: osdmap e64: 12 total, 11 up, 12 in 2024-09-21T04:00:49.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:49 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T04:00:49.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:49 smithi064 ceph-mon[31180]: from='client.? 172.21.15.131:0/1301084753' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c26a5ef9-8585-404e-a54a-852424c4d6fd"} : dispatch 2024-09-21T04:00:49.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:49 smithi064 ceph-mon[31180]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c26a5ef9-8585-404e-a54a-852424c4d6fd"} : dispatch 2024-09-21T04:00:49.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:49 smithi064 ceph-mon[31180]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c26a5ef9-8585-404e-a54a-852424c4d6fd"}]': finished 2024-09-21T04:00:49.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:49 smithi064 ceph-mon[31180]: osdmap e64: 12 total, 11 up, 12 in 2024-09-21T04:00:49.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:49 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T04:00:50.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:50 smithi119 ceph-mon[34811]: pgmap v211: 1 pgs: 1 peering; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-09-21T04:00:50.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:50 smithi119 ceph-mon[34811]: from='client.? 172.21.15.131:0/4197692063' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T04:00:50.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:50 smithi064 ceph-mon[31180]: pgmap v211: 1 pgs: 1 peering; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-09-21T04:00:50.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:50 smithi064 ceph-mon[31180]: from='client.? 172.21.15.131:0/4197692063' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T04:00:50.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:50 smithi131 ceph-mon[35193]: pgmap v211: 1 pgs: 1 peering; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-09-21T04:00:50.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:50 smithi131 ceph-mon[35193]: from='client.? 172.21.15.131:0/4197692063' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T04:00:52.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:52 smithi119 ceph-mon[34811]: pgmap v213: 1 pgs: 1 peering; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-09-21T04:00:52.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:52 smithi064 ceph-mon[31180]: pgmap v213: 1 pgs: 1 peering; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-09-21T04:00:52.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:52 smithi131 ceph-mon[35193]: pgmap v213: 1 pgs: 1 peering; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-09-21T04:00:54.223 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:54 smithi131 ceph-mon[35193]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T04:00:54.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:54 smithi119 ceph-mon[34811]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T04:00:54.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:54 smithi064 ceph-mon[31180]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T04:00:56.352 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:56 smithi131 ceph-mon[35193]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-21T04:00:56.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:56 smithi119 ceph-mon[34811]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-21T04:00:56.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:56 smithi064 ceph-mon[31180]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-21T04:00:58.332 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:58 smithi064 ceph-mon[31180]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-21T04:00:58.332 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:58 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T04:00:58.332 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:58 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:00:58.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:58 smithi119 ceph-mon[34811]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-21T04:00:58.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:58 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T04:00:58.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:58 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:00:58.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:58 smithi131 ceph-mon[35193]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-21T04:00:58.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:58 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T04:00:58.535 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:58 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:00:59.414 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:00:59 smithi131 ceph-mon[35193]: Deploying daemon osd.11 on smithi131 2024-09-21T04:00:59.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:00:59 smithi119 ceph-mon[34811]: Deploying daemon osd.11 on smithi131 2024-09-21T04:00:59.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:00:59 smithi064 ceph-mon[31180]: Deploying daemon osd.11 on smithi131 2024-09-21T04:01:00.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:00 smithi131 ceph-mon[35193]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T04:01:00.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:00 smithi119 ceph-mon[34811]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T04:01:00.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:00 smithi064 ceph-mon[31180]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T04:01:02.429 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:02 smithi131 ceph-mon[35193]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-21T04:01:02.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:02 smithi119 ceph-mon[34811]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-21T04:01:02.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:02 smithi064 ceph-mon[31180]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-21T04:01:04.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:04 smithi119 ceph-mon[34811]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T04:01:04.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:04 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:01:04.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:04 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:04.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:04 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:04.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:04 smithi064 ceph-mon[31180]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T04:01:04.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:04 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:01:04.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:04 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:04.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:04 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:04.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:04 smithi131 ceph-mon[35193]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T04:01:04.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:04 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:01:04.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:04 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:04.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:04 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:06.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:06 smithi131 ceph-mon[35193]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T04:01:06.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:06 smithi119 ceph-mon[34811]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T04:01:06.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:06 smithi064 ceph-mon[31180]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T04:01:07.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:07.121+0000 7f8fa5ffb640 1 -- 172.21.15.131:0/148651225 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f8fa8063800 con 0x7f8f940b6560 2024-09-21T04:01:07.124 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 11 on host 'smithi131' 2024-09-21T04:01:07.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:07.123+0000 7f8faeceb640 1 -- 172.21.15.131:0/148651225 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8f940b6560 msgr2=0x7f8f940b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:07.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:07.123+0000 7f8faeceb640 1 --2- 172.21.15.131:0/148651225 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8f940b6560 0x7f8f940b8a70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8f900096c0 tx=0x7f8f90031400 comp rx=0 tx=0).stop 2024-09-21T04:01:07.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:07.123+0000 7f8faeceb640 1 -- 172.21.15.131:0/148651225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fa8102a60 msgr2=0x7f8fa80ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:07.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:07.123+0000 7f8faeceb640 1 --2- 172.21.15.131:0/148651225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fa8102a60 0x7f8fa80ff670 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800d950 tx=0x7f8f9800de10 comp rx=0 tx=0).stop 2024-09-21T04:01:07.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:07.124+0000 7f8faeceb640 1 -- 172.21.15.131:0/148651225 shutdown_connections 2024-09-21T04:01:07.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:07.124+0000 7f8faeceb640 1 --2- 172.21.15.131:0/148651225 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8f940b6560 0x7f8f940b8a70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:07.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:07.124+0000 7f8faeceb640 1 --2- 172.21.15.131:0/148651225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fa8108cf0 0x7f8fa81000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:07.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:07.124+0000 7f8faeceb640 1 --2- 172.21.15.131:0/148651225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fa81083a0 0x7f8fa80ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:07.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:07.124+0000 7f8faeceb640 1 --2- 172.21.15.131:0/148651225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fa8102a60 0x7f8fa80ff670 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:07.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:07.124+0000 7f8faeceb640 1 -- 172.21.15.131:0/148651225 >> 172.21.15.131:0/148651225 conn(0x7f8fa8079730 msgr2=0x7f8fa806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:07.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:07.124+0000 7f8faeceb640 1 -- 172.21.15.131:0/148651225 shutdown_connections 2024-09-21T04:01:07.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:07.124+0000 7f8faeceb640 1 -- 172.21.15.131:0/148651225 wait complete. 2024-09-21T04:01:07.755 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:07 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:07.755 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:07 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:07.755 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:07 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:07.755 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:07 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:01:07.755 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:07 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:07.755 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:07 smithi131 ceph-mon[35193]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T04:01:07.755 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:07 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:01:07.755 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:07 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:07.755 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:07 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:07.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:07 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:07.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:07 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:07.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:07 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:07.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:07 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:01:07.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:07 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:07.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:07 smithi119 ceph-mon[34811]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T04:01:07.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:07 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:01:07.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:07 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:07.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:07 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:07.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:07 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:07.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:07 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:07.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:07 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:07.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:07 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:01:07.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:07 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:07.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:07 smithi064 ceph-mon[31180]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T04:01:07.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:07 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:01:07.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:07 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:07.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:07 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:07.782 DEBUG:teuthology.orchestra.run.smithi131:osd.11> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.11.service 2024-09-21T04:01:07.785 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-21T04:01:07.785 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T04:01:08.059 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:01:08.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.518+0000 7f35a2eb9640 1 -- 172.21.15.64:0/2917410534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f359c1101e0 msgr2=0x7f359c112600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:08.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.518+0000 7f35a2eb9640 1 --2- 172.21.15.64:0/2917410534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f359c1101e0 0x7f359c112600 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3594009ca0 tx=0x7f359402a950 comp rx=0 tx=0).stop 2024-09-21T04:01:08.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.518+0000 7f35a2eb9640 1 -- 172.21.15.64:0/2917410534 shutdown_connections 2024-09-21T04:01:08.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.518+0000 7f35a2eb9640 1 --2- 172.21.15.64:0/2917410534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f359c1101e0 0x7f359c112600 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:08.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.518+0000 7f35a2eb9640 1 --2- 172.21.15.64:0/2917410534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f359c102f80 0x7f359c10fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:08.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.518+0000 7f35a2eb9640 1 --2- 172.21.15.64:0/2917410534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f359c102630 0x7f359c102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:08.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.518+0000 7f35a2eb9640 1 -- 172.21.15.64:0/2917410534 >> 172.21.15.64:0/2917410534 conn(0x7f359c079730 msgr2=0x7f359c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:08.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.518+0000 7f35a2eb9640 1 -- 172.21.15.64:0/2917410534 shutdown_connections 2024-09-21T04:01:08.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.518+0000 7f35a2eb9640 1 -- 172.21.15.64:0/2917410534 wait complete. 2024-09-21T04:01:08.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.519+0000 7f35a2eb9640 1 Processor -- start 2024-09-21T04:01:08.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.519+0000 7f35a2eb9640 1 -- start start 2024-09-21T04:01:08.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.519+0000 7f35a2eb9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f359c102630 0x7f359c19c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:08.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.519+0000 7f35a2eb9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f359c102f80 0x7f359c19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:08.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.520+0000 7f35a2eb9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f359c1101e0 0x7f359c1a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:08.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.520+0000 7f35a2eb9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f359c115ca0 con 0x7f359c102f80 2024-09-21T04:01:08.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.520+0000 7f35a2eb9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f359c115b20 con 0x7f359c102630 2024-09-21T04:01:08.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.520+0000 7f35a2eb9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f359c115e20 con 0x7f359c1101e0 2024-09-21T04:01:08.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.520+0000 7f359bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f359c102f80 0x7f359c19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:08.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.520+0000 7f359bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f359c102f80 0x7f359c19cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37128/0 (socket says 172.21.15.64:37128) 2024-09-21T04:01:08.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.520+0000 7f359bfff640 1 -- 172.21.15.64:0/15125367 learned_addr learned my addr 172.21.15.64:0/15125367 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:01:08.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.520+0000 7f35a0c55640 1 --2- 172.21.15.64:0/15125367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f359c102630 0x7f359c19c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:08.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.520+0000 7f35a1456640 1 --2- 172.21.15.64:0/15125367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f359c1101e0 0x7f359c1a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:08.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.520+0000 7f359bfff640 1 -- 172.21.15.64:0/15125367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f359c1101e0 msgr2=0x7f359c1a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:08.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.520+0000 7f359bfff640 1 --2- 172.21.15.64:0/15125367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f359c1101e0 0x7f359c1a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:08.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.520+0000 7f359bfff640 1 -- 172.21.15.64:0/15125367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f359c102630 msgr2=0x7f359c19c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:08.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.520+0000 7f359bfff640 1 --2- 172.21.15.64:0/15125367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f359c102630 0x7f359c19c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:08.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.520+0000 7f359bfff640 1 -- 172.21.15.64:0/15125367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359c1a2540 con 0x7f359c102f80 2024-09-21T04:01:08.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.520+0000 7f359bfff640 1 --2- 172.21.15.64:0/15125367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f359c102f80 0x7f359c19cb70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f358800e9f0 tx=0x7f358800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:08.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.520+0000 7f3599ffb640 1 -- 172.21.15.64:0/15125367 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358800cd80 con 0x7f359c102f80 2024-09-21T04:01:08.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.520+0000 7f35a2eb9640 1 -- 172.21.15.64:0/15125367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359c1a2830 con 0x7f359c102f80 2024-09-21T04:01:08.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.521+0000 7f3599ffb640 1 -- 172.21.15.64:0/15125367 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35880108e0 con 0x7f359c102f80 2024-09-21T04:01:08.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.521+0000 7f3599ffb640 1 -- 172.21.15.64:0/15125367 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3588018980 con 0x7f359c102f80 2024-09-21T04:01:08.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.521+0000 7f35a2eb9640 1 -- 172.21.15.64:0/15125367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f359c1a8f90 con 0x7f359c102f80 2024-09-21T04:01:08.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.522+0000 7f35a2eb9640 1 -- 172.21.15.64:0/15125367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f359c102a40 con 0x7f359c102f80 2024-09-21T04:01:08.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.524+0000 7f3599ffb640 1 -- 172.21.15.64:0/15125367 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f3588010430 con 0x7f359c102f80 2024-09-21T04:01:08.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.524+0000 7f3599ffb640 1 --2- 172.21.15.64:0/15125367 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f358c0b6560 0x7f358c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:08.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.524+0000 7f35a0c55640 1 --2- 172.21.15.64:0/15125367 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f358c0b6560 0x7f358c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:08.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.525+0000 7f3599ffb640 1 -- 172.21.15.64:0/15125367 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 7024+0+0 (secure 0 0 0) 0x7f35880e7190 con 0x7f359c102f80 2024-09-21T04:01:08.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.525+0000 7f35a0c55640 1 --2- 172.21.15.64:0/15125367 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f358c0b6560 0x7f358c0b8a70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3590009bd0 tx=0x7f3590005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:08.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.533+0000 7f3599ffb640 1 -- 172.21.15.64:0/15125367 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f35880b2310 con 0x7f359c102f80 2024-09-21T04:01:08.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.723+0000 7f35a2eb9640 1 -- 172.21.15.64:0/15125367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f359c1a9270 con 0x7f359c102f80 2024-09-21T04:01:08.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.723+0000 7f3599ffb640 1 -- 172.21.15.64:0/15125367 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v64) ==== 74+0+133 (secure 0 0 0) 0x7f35880b7250 con 0x7f359c102f80 2024-09-21T04:01:08.725 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:01:08.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.725+0000 7f35a2eb9640 1 -- 172.21.15.64:0/15125367 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f358c0b6560 msgr2=0x7f358c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:08.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.725+0000 7f35a2eb9640 1 --2- 172.21.15.64:0/15125367 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f358c0b6560 0x7f358c0b8a70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3590009bd0 tx=0x7f3590005f90 comp rx=0 tx=0).stop 2024-09-21T04:01:08.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.725+0000 7f35a2eb9640 1 -- 172.21.15.64:0/15125367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f359c102f80 msgr2=0x7f359c19cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:08.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.725+0000 7f35a2eb9640 1 --2- 172.21.15.64:0/15125367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f359c102f80 0x7f359c19cb70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f358800e9f0 tx=0x7f358800eeb0 comp rx=0 tx=0).stop 2024-09-21T04:01:08.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.725+0000 7f35a2eb9640 1 -- 172.21.15.64:0/15125367 shutdown_connections 2024-09-21T04:01:08.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.725+0000 7f35a2eb9640 1 --2- 172.21.15.64:0/15125367 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f358c0b6560 0x7f358c0b8a70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:08.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.725+0000 7f35a2eb9640 1 --2- 172.21.15.64:0/15125367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f359c1101e0 0x7f359c1a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:08.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.725+0000 7f35a2eb9640 1 --2- 172.21.15.64:0/15125367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f359c102f80 0x7f359c19cb70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:08.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.725+0000 7f35a2eb9640 1 --2- 172.21.15.64:0/15125367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f359c102630 0x7f359c19c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:08.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.725+0000 7f35a2eb9640 1 -- 172.21.15.64:0/15125367 >> 172.21.15.64:0/15125367 conn(0x7f359c079730 msgr2=0x7f359c111480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:08.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.725+0000 7f35a2eb9640 1 -- 172.21.15.64:0/15125367 shutdown_connections 2024-09-21T04:01:08.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:08.725+0000 7f35a2eb9640 1 -- 172.21.15.64:0/15125367 wait complete. 2024-09-21T04:01:09.528 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:01:09 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:01:09.212+0000 7f6543b45640 -1 osd.11 0 log_to_monitors true 2024-09-21T04:01:09.538 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":64,"num_osds":12,"num_up_osds":11,"osd_up_since":1726891247,"num_in_osds":12,"osd_in_since":1726891248,"num_remapped_pgs":0} 2024-09-21T04:01:09.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:09 smithi064 ceph-mon[31180]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T04:01:09.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:09 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/15125367' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T04:01:09.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:09 smithi064 ceph-mon[31180]: from='osd.11 [v2:172.21.15.131:6824/2294059084,v1:172.21.15.131:6825/2294059084]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T04:01:09.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:09 smithi064 ceph-mon[31180]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T04:01:09.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:09 smithi131 ceph-mon[35193]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T04:01:09.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:09 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/15125367' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T04:01:09.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:09 smithi131 ceph-mon[35193]: from='osd.11 [v2:172.21.15.131:6824/2294059084,v1:172.21.15.131:6825/2294059084]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T04:01:09.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:09 smithi131 ceph-mon[35193]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T04:01:10.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:09 smithi119 ceph-mon[34811]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T04:01:10.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:09 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/15125367' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T04:01:10.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:09 smithi119 ceph-mon[34811]: from='osd.11 [v2:172.21.15.131:6824/2294059084,v1:172.21.15.131:6825/2294059084]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T04:01:10.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:09 smithi119 ceph-mon[34811]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T04:01:10.539 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T04:01:10.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:10 smithi131 ceph-mon[35193]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T04:01:10.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:10 smithi131 ceph-mon[35193]: osdmap e65: 12 total, 11 up, 12 in 2024-09-21T04:01:10.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:10 smithi131 ceph-mon[35193]: from='osd.11 [v2:172.21.15.131:6824/2294059084,v1:172.21.15.131:6825/2294059084]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi131", "root=default"]} : dispatch 2024-09-21T04:01:10.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:10 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T04:01:10.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:10 smithi131 ceph-mon[35193]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi131", "root=default"]} : dispatch 2024-09-21T04:01:10.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:10 smithi131 ceph-mon[35193]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-21T04:01:10.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:10 smithi131 ceph-mon[35193]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T04:01:10.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:10 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T04:01:10.809 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:01:10.931 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:10 smithi064 ceph-mon[31180]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T04:01:10.931 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:10 smithi064 ceph-mon[31180]: osdmap e65: 12 total, 11 up, 12 in 2024-09-21T04:01:10.931 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:10 smithi064 ceph-mon[31180]: from='osd.11 [v2:172.21.15.131:6824/2294059084,v1:172.21.15.131:6825/2294059084]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi131", "root=default"]} : dispatch 2024-09-21T04:01:10.931 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:10 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T04:01:10.931 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:10 smithi064 ceph-mon[31180]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi131", "root=default"]} : dispatch 2024-09-21T04:01:10.931 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:10 smithi064 ceph-mon[31180]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-21T04:01:10.931 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:10 smithi064 ceph-mon[31180]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T04:01:10.931 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:10 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T04:01:11.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:10 smithi119 ceph-mon[34811]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T04:01:11.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:10 smithi119 ceph-mon[34811]: osdmap e65: 12 total, 11 up, 12 in 2024-09-21T04:01:11.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:10 smithi119 ceph-mon[34811]: from='osd.11 [v2:172.21.15.131:6824/2294059084,v1:172.21.15.131:6825/2294059084]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi131", "root=default"]} : dispatch 2024-09-21T04:01:11.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:10 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T04:01:11.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:10 smithi119 ceph-mon[34811]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi131", "root=default"]} : dispatch 2024-09-21T04:01:11.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:10 smithi119 ceph-mon[34811]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-21T04:01:11.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:10 smithi119 ceph-mon[34811]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T04:01:11.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:10 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T04:01:11.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.259+0000 7f373c0fb640 1 -- 172.21.15.64:0/1849208624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3734102600 msgr2=0x7f3734102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:11.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.259+0000 7f373c0fb640 1 --2- 172.21.15.64:0/1849208624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3734102600 0x7f3734102a10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3730009900 tx=0x7f373002a480 comp rx=0 tx=0).stop 2024-09-21T04:01:11.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.259+0000 7f373c0fb640 1 -- 172.21.15.64:0/1849208624 shutdown_connections 2024-09-21T04:01:11.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.259+0000 7f373c0fb640 1 --2- 172.21.15.64:0/1849208624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37341101b0 0x7f37341125d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:11.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.259+0000 7f373c0fb640 1 --2- 172.21.15.64:0/1849208624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3734102f50 0x7f373410fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:11.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.259+0000 7f373c0fb640 1 --2- 172.21.15.64:0/1849208624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3734102600 0x7f3734102a10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:11.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.259+0000 7f373c0fb640 1 -- 172.21.15.64:0/1849208624 >> 172.21.15.64:0/1849208624 conn(0x7f3734079750 msgr2=0x7f3734079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:11.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.259+0000 7f373c0fb640 1 -- 172.21.15.64:0/1849208624 shutdown_connections 2024-09-21T04:01:11.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.260+0000 7f373c0fb640 1 -- 172.21.15.64:0/1849208624 wait complete. 2024-09-21T04:01:11.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.260+0000 7f373c0fb640 1 Processor -- start 2024-09-21T04:01:11.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.260+0000 7f373c0fb640 1 -- start start 2024-09-21T04:01:11.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.261+0000 7f373c0fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3734102600 0x7f373419c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:11.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.261+0000 7f373c0fb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3734102f50 0x7f373419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:11.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.261+0000 7f373c0fb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37341101b0 0x7f37341a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:11.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.261+0000 7f373c0fb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3734115ca0 con 0x7f3734102f50 2024-09-21T04:01:11.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.261+0000 7f373c0fb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3734115b20 con 0x7f3734102600 2024-09-21T04:01:11.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.261+0000 7f373c0fb640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3734115e20 con 0x7f37341101b0 2024-09-21T04:01:11.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.261+0000 7f3739696640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3734102f50 0x7f373419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:11.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.261+0000 7f3739696640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3734102f50 0x7f373419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37146/0 (socket says 172.21.15.64:37146) 2024-09-21T04:01:11.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.261+0000 7f3739696640 1 -- 172.21.15.64:0/3885386805 learned_addr learned my addr 172.21.15.64:0/3885386805 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:01:11.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.261+0000 7f3739e97640 1 --2- 172.21.15.64:0/3885386805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3734102600 0x7f373419c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:11.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.261+0000 7f373a698640 1 --2- 172.21.15.64:0/3885386805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37341101b0 0x7f37341a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:11.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.261+0000 7f3739696640 1 -- 172.21.15.64:0/3885386805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37341101b0 msgr2=0x7f37341a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:11.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.261+0000 7f3739696640 1 --2- 172.21.15.64:0/3885386805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37341101b0 0x7f37341a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:11.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.261+0000 7f3739696640 1 -- 172.21.15.64:0/3885386805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3734102600 msgr2=0x7f373419c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:11.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.261+0000 7f3739696640 1 --2- 172.21.15.64:0/3885386805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3734102600 0x7f373419c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:11.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.261+0000 7f3739696640 1 -- 172.21.15.64:0/3885386805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37341a2580 con 0x7f3734102f50 2024-09-21T04:01:11.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.261+0000 7f3739e97640 1 --2- 172.21.15.64:0/3885386805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3734102600 0x7f373419c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:01:11.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.261+0000 7f3739696640 1 --2- 172.21.15.64:0/3885386805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3734102f50 0x7f373419cbb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f372400d770 tx=0x7f372400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:11.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.262+0000 7f372affd640 1 -- 172.21.15.64:0/3885386805 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724017070 con 0x7f3734102f50 2024-09-21T04:01:11.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.262+0000 7f373c0fb640 1 -- 172.21.15.64:0/3885386805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37341a2870 con 0x7f3734102f50 2024-09-21T04:01:11.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.262+0000 7f372affd640 1 -- 172.21.15.64:0/3885386805 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37240054d0 con 0x7f3734102f50 2024-09-21T04:01:11.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.262+0000 7f372affd640 1 -- 172.21.15.64:0/3885386805 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724004c30 con 0x7f3734102f50 2024-09-21T04:01:11.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.262+0000 7f373c0fb640 1 -- 172.21.15.64:0/3885386805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37341a8fd0 con 0x7f3734102f50 2024-09-21T04:01:11.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.263+0000 7f373c0fb640 1 -- 172.21.15.64:0/3885386805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f373406c280 con 0x7f3734102f50 2024-09-21T04:01:11.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.264+0000 7f372affd640 1 -- 172.21.15.64:0/3885386805 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f3724004e10 con 0x7f3734102f50 2024-09-21T04:01:11.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.265+0000 7f372affd640 1 --2- 172.21.15.64:0/3885386805 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f37200b6560 0x7f37200b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:11.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.265+0000 7f3739e97640 1 --2- 172.21.15.64:0/3885386805 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f37200b6560 0x7f37200b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:11.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.265+0000 7f372affd640 1 -- 172.21.15.64:0/3885386805 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 7062+0+0 (secure 0 0 0) 0x7f37240e69a0 con 0x7f3734102f50 2024-09-21T04:01:11.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.266+0000 7f3739e97640 1 --2- 172.21.15.64:0/3885386805 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f37200b6560 0x7f37200b8a70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3730031440 tx=0x7f3730009680 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:11.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.270+0000 7f372affd640 1 -- 172.21.15.64:0/3885386805 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f37240ad710 con 0x7f3734102f50 2024-09-21T04:01:11.303 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:01:10 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:01:10.984+0000 7f653fac7640 -1 osd.11 0 waiting for initial osdmap 2024-09-21T04:01:11.303 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:01:10 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:01:10.998+0000 7f65378bd640 -1 osd.11 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T04:01:11.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.457+0000 7f373c0fb640 1 -- 172.21.15.64:0/3885386805 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f373419d740 con 0x7f3734102f50 2024-09-21T04:01:11.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.458+0000 7f372affd640 1 -- 172.21.15.64:0/3885386805 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v66) ==== 74+0+133 (secure 0 0 0) 0x7f37240b1af0 con 0x7f3734102f50 2024-09-21T04:01:11.460 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:01:11.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.460+0000 7f373c0fb640 1 -- 172.21.15.64:0/3885386805 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f37200b6560 msgr2=0x7f37200b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:11.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.460+0000 7f373c0fb640 1 --2- 172.21.15.64:0/3885386805 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f37200b6560 0x7f37200b8a70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3730031440 tx=0x7f3730009680 comp rx=0 tx=0).stop 2024-09-21T04:01:11.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.460+0000 7f373c0fb640 1 -- 172.21.15.64:0/3885386805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3734102f50 msgr2=0x7f373419cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:11.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.460+0000 7f373c0fb640 1 --2- 172.21.15.64:0/3885386805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3734102f50 0x7f373419cbb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f372400d770 tx=0x7f372400dc30 comp rx=0 tx=0).stop 2024-09-21T04:01:11.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.460+0000 7f373c0fb640 1 -- 172.21.15.64:0/3885386805 shutdown_connections 2024-09-21T04:01:11.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.460+0000 7f373c0fb640 1 --2- 172.21.15.64:0/3885386805 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f37200b6560 0x7f37200b8a70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:11.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.460+0000 7f373c0fb640 1 --2- 172.21.15.64:0/3885386805 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37341101b0 0x7f37341a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:11.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.460+0000 7f373c0fb640 1 --2- 172.21.15.64:0/3885386805 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3734102f50 0x7f373419cbb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:11.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.460+0000 7f373c0fb640 1 --2- 172.21.15.64:0/3885386805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3734102600 0x7f373419c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:11.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.460+0000 7f373c0fb640 1 -- 172.21.15.64:0/3885386805 >> 172.21.15.64:0/3885386805 conn(0x7f3734079750 msgr2=0x7f3734109d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:11.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.461+0000 7f373c0fb640 1 -- 172.21.15.64:0/3885386805 shutdown_connections 2024-09-21T04:01:11.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:11.461+0000 7f373c0fb640 1 -- 172.21.15.64:0/3885386805 wait complete. 2024-09-21T04:01:11.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:11 smithi131 ceph-mon[35193]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T04:01:11.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T04:01:11.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:11 smithi131 ceph-mon[35193]: Detected new or changed devices on smithi131 2024-09-21T04:01:11.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:11.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:11.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T04:01:11.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T04:01:11.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T04:01:11.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T04:01:11.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:11 smithi131 ceph-mon[35193]: Adjusting osd_memory_target on smithi131 to 4272M 2024-09-21T04:01:11.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:11.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:11.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:01:11.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:11.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:11 smithi131 ceph-mon[35193]: from='osd.11 ' entity='osd.11' 2024-09-21T04:01:11.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:11 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3885386805' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T04:01:11.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T04:01:11.822 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:11 smithi064 ceph-mon[31180]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T04:01:11.822 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T04:01:11.822 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:11 smithi064 ceph-mon[31180]: Detected new or changed devices on smithi131 2024-09-21T04:01:11.822 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:11.822 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:11.822 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T04:01:11.823 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T04:01:11.823 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T04:01:11.823 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T04:01:11.823 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:11 smithi064 ceph-mon[31180]: Adjusting osd_memory_target on smithi131 to 4272M 2024-09-21T04:01:11.823 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:11.823 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:11.823 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:01:11.823 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:11.823 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:11 smithi064 ceph-mon[31180]: from='osd.11 ' entity='osd.11' 2024-09-21T04:01:11.823 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:11 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3885386805' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T04:01:11.823 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T04:01:12.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:11 smithi119 ceph-mon[34811]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T04:01:12.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T04:01:12.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:11 smithi119 ceph-mon[34811]: Detected new or changed devices on smithi131 2024-09-21T04:01:12.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:12.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:12.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T04:01:12.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T04:01:12.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T04:01:12.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T04:01:12.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:11 smithi119 ceph-mon[34811]: Adjusting osd_memory_target on smithi131 to 4272M 2024-09-21T04:01:12.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:12.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:12.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:01:12.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:12.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:11 smithi119 ceph-mon[34811]: from='osd.11 ' entity='osd.11' 2024-09-21T04:01:12.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:11 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3885386805' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T04:01:12.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T04:01:12.144 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":66,"num_osds":12,"num_up_osds":11,"osd_up_since":1726891247,"num_in_osds":12,"osd_in_since":1726891248,"num_remapped_pgs":0} 2024-09-21T04:01:13.146 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T04:01:13.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:13 smithi119 ceph-mon[34811]: purged_snaps scrub starts 2024-09-21T04:01:13.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:13 smithi119 ceph-mon[34811]: purged_snaps scrub ok 2024-09-21T04:01:13.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:13 smithi119 ceph-mon[34811]: osd.11 [v2:172.21.15.131:6824/2294059084,v1:172.21.15.131:6825/2294059084] boot 2024-09-21T04:01:13.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:13 smithi119 ceph-mon[34811]: osdmap e67: 12 total, 12 up, 12 in 2024-09-21T04:01:13.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:13 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T04:01:13.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:13 smithi064 ceph-mon[31180]: purged_snaps scrub starts 2024-09-21T04:01:13.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:13 smithi064 ceph-mon[31180]: purged_snaps scrub ok 2024-09-21T04:01:13.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:13 smithi064 ceph-mon[31180]: osd.11 [v2:172.21.15.131:6824/2294059084,v1:172.21.15.131:6825/2294059084] boot 2024-09-21T04:01:13.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:13 smithi064 ceph-mon[31180]: osdmap e67: 12 total, 12 up, 12 in 2024-09-21T04:01:13.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:13 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T04:01:13.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:13 smithi131 ceph-mon[35193]: purged_snaps scrub starts 2024-09-21T04:01:13.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:13 smithi131 ceph-mon[35193]: purged_snaps scrub ok 2024-09-21T04:01:13.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:13 smithi131 ceph-mon[35193]: osd.11 [v2:172.21.15.131:6824/2294059084,v1:172.21.15.131:6825/2294059084] boot 2024-09-21T04:01:13.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:13 smithi131 ceph-mon[35193]: osdmap e67: 12 total, 12 up, 12 in 2024-09-21T04:01:13.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:13 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T04:01:13.415 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:01:13.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.900+0000 7f0d767d6640 1 -- 172.21.15.64:0/3941082793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d70102ae0 msgr2=0x7f0d70102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:13.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.900+0000 7f0d767d6640 1 --2- 172.21.15.64:0/3941082793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d70102ae0 0x7f0d70102ef0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009900 tx=0x7f0d6002a410 comp rx=0 tx=0).stop 2024-09-21T04:01:13.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.901+0000 7f0d767d6640 1 -- 172.21.15.64:0/3941082793 shutdown_connections 2024-09-21T04:01:13.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.901+0000 7f0d767d6640 1 --2- 172.21.15.64:0/3941082793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d70108d20 0x7f0d7010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:13.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.901+0000 7f0d767d6640 1 --2- 172.21.15.64:0/3941082793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d701083d0 0x7f0d701087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:13.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.901+0000 7f0d767d6640 1 --2- 172.21.15.64:0/3941082793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d70102ae0 0x7f0d70102ef0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:13.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.901+0000 7f0d767d6640 1 -- 172.21.15.64:0/3941082793 >> 172.21.15.64:0/3941082793 conn(0x7f0d70079760 msgr2=0x7f0d70079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:13.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.901+0000 7f0d767d6640 1 -- 172.21.15.64:0/3941082793 shutdown_connections 2024-09-21T04:01:13.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.901+0000 7f0d767d6640 1 -- 172.21.15.64:0/3941082793 wait complete. 2024-09-21T04:01:13.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.901+0000 7f0d767d6640 1 Processor -- start 2024-09-21T04:01:13.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.901+0000 7f0d767d6640 1 -- start start 2024-09-21T04:01:13.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.902+0000 7f0d767d6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d70102ae0 0x7f0d70104280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:13.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.902+0000 7f0d767d6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d701083d0 0x7f0d701047c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:13.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.902+0000 7f0d767d6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d70108d20 0x7f0d70104d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:13.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.902+0000 7f0d767d6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0d70112990 con 0x7f0d70108d20 2024-09-21T04:01:13.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.902+0000 7f0d767d6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0d70112810 con 0x7f0d70102ae0 2024-09-21T04:01:13.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.902+0000 7f0d767d6640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0d70112b10 con 0x7f0d701083d0 2024-09-21T04:01:13.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.902+0000 7f0d74d73640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d70108d20 0x7f0d70104d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:13.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.902+0000 7f0d74d73640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d70108d20 0x7f0d70104d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56680/0 (socket says 172.21.15.64:56680) 2024-09-21T04:01:13.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.902+0000 7f0d74d73640 1 -- 172.21.15.64:0/597046849 learned_addr learned my addr 172.21.15.64:0/597046849 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:01:13.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.902+0000 7f0d6f7fe640 1 --2- 172.21.15.64:0/597046849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d701083d0 0x7f0d701047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:13.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.902+0000 7f0d6ffff640 1 --2- 172.21.15.64:0/597046849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d70102ae0 0x7f0d70104280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:13.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.902+0000 7f0d74d73640 1 -- 172.21.15.64:0/597046849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d701083d0 msgr2=0x7f0d701047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:13.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.902+0000 7f0d74d73640 1 --2- 172.21.15.64:0/597046849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d701083d0 0x7f0d701047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:13.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.902+0000 7f0d74d73640 1 -- 172.21.15.64:0/597046849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d70102ae0 msgr2=0x7f0d70104280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:13.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.902+0000 7f0d74d73640 1 --2- 172.21.15.64:0/597046849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d70102ae0 0x7f0d70104280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:13.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.902+0000 7f0d74d73640 1 -- 172.21.15.64:0/597046849 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d701a8750 con 0x7f0d70108d20 2024-09-21T04:01:13.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.902+0000 7f0d74d73640 1 --2- 172.21.15.64:0/597046849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d70108d20 0x7f0d70104d00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0d68009fd0 tx=0x7f0d6800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:13.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.903+0000 7f0d6d7fa640 1 -- 172.21.15.64:0/597046849 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68017070 con 0x7f0d70108d20 2024-09-21T04:01:13.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.903+0000 7f0d767d6640 1 -- 172.21.15.64:0/597046849 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d701a8a40 con 0x7f0d70108d20 2024-09-21T04:01:13.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.903+0000 7f0d6d7fa640 1 -- 172.21.15.64:0/597046849 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d68003e20 con 0x7f0d70108d20 2024-09-21T04:01:13.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.903+0000 7f0d6d7fa640 1 -- 172.21.15.64:0/597046849 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68010460 con 0x7f0d70108d20 2024-09-21T04:01:13.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.903+0000 7f0d767d6640 1 -- 172.21.15.64:0/597046849 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d701a8f30 con 0x7f0d70108d20 2024-09-21T04:01:13.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.904+0000 7f0d767d6640 1 -- 172.21.15.64:0/597046849 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d70103d10 con 0x7f0d70108d20 2024-09-21T04:01:13.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.905+0000 7f0d6d7fa640 1 -- 172.21.15.64:0/597046849 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f0d68003970 con 0x7f0d70108d20 2024-09-21T04:01:13.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.906+0000 7f0d6d7fa640 1 --2- 172.21.15.64:0/597046849 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0d580b6560 0x7f0d580b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:13.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.906+0000 7f0d6ffff640 1 --2- 172.21.15.64:0/597046849 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0d580b6560 0x7f0d580b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:13.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.906+0000 7f0d6d7fa640 1 -- 172.21.15.64:0/597046849 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7f0d680e6d60 con 0x7f0d70108d20 2024-09-21T04:01:13.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.907+0000 7f0d6ffff640 1 --2- 172.21.15.64:0/597046849 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0d580b6560 0x7f0d580b8a70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0d60000c00 tx=0x7f0d60031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:13.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:13.910+0000 7f0d6d7fa640 1 -- 172.21.15.64:0/597046849 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0d680ad9c0 con 0x7f0d70108d20 2024-09-21T04:01:14.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:14.102+0000 7f0d767d6640 1 -- 172.21.15.64:0/597046849 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f0d70105580 con 0x7f0d70108d20 2024-09-21T04:01:14.104 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:14 smithi064 ceph-mon[31180]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T04:01:14.104 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:14 smithi064 ceph-mon[31180]: osdmap e68: 12 total, 12 up, 12 in 2024-09-21T04:01:14.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:14.102+0000 7f0d6d7fa640 1 -- 172.21.15.64:0/597046849 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v68) ==== 74+0+133 (secure 0 0 0) 0x7f0d680b1da0 con 0x7f0d70108d20 2024-09-21T04:01:14.105 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:01:14.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:14.104+0000 7f0d767d6640 1 -- 172.21.15.64:0/597046849 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0d580b6560 msgr2=0x7f0d580b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:14.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:14.104+0000 7f0d767d6640 1 --2- 172.21.15.64:0/597046849 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0d580b6560 0x7f0d580b8a70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0d60000c00 tx=0x7f0d60031400 comp rx=0 tx=0).stop 2024-09-21T04:01:14.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:14.104+0000 7f0d767d6640 1 -- 172.21.15.64:0/597046849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d70108d20 msgr2=0x7f0d70104d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:14.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:14.104+0000 7f0d767d6640 1 --2- 172.21.15.64:0/597046849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d70108d20 0x7f0d70104d00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0d68009fd0 tx=0x7f0d6800b260 comp rx=0 tx=0).stop 2024-09-21T04:01:14.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:14.104+0000 7f0d767d6640 1 -- 172.21.15.64:0/597046849 shutdown_connections 2024-09-21T04:01:14.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:14.104+0000 7f0d767d6640 1 --2- 172.21.15.64:0/597046849 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0d580b6560 0x7f0d580b8a70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:14.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:14.104+0000 7f0d767d6640 1 --2- 172.21.15.64:0/597046849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d70108d20 0x7f0d70104d00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:14.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:14.104+0000 7f0d767d6640 1 --2- 172.21.15.64:0/597046849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d701083d0 0x7f0d701047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:14.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:14.104+0000 7f0d767d6640 1 --2- 172.21.15.64:0/597046849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d70102ae0 0x7f0d70104280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:14.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:14.104+0000 7f0d767d6640 1 -- 172.21.15.64:0/597046849 >> 172.21.15.64:0/597046849 conn(0x7f0d70079760 msgr2=0x7f0d70101b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:14.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:14.104+0000 7f0d767d6640 1 -- 172.21.15.64:0/597046849 shutdown_connections 2024-09-21T04:01:14.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:14.104+0000 7f0d767d6640 1 -- 172.21.15.64:0/597046849 wait complete. 2024-09-21T04:01:14.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:14 smithi119 ceph-mon[34811]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T04:01:14.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:14 smithi119 ceph-mon[34811]: osdmap e68: 12 total, 12 up, 12 in 2024-09-21T04:01:14.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:14 smithi131 ceph-mon[35193]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-21T04:01:14.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:14 smithi131 ceph-mon[35193]: osdmap e68: 12 total, 12 up, 12 in 2024-09-21T04:01:14.735 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":68,"num_osds":12,"num_up_osds":12,"osd_up_since":1726891271,"num_in_osds":12,"osd_in_since":1726891248,"num_remapped_pgs":0} 2024-09-21T04:01:14.735 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T04:01:14.987 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:01:15.259 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:15 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/597046849' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T04:01:15.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:15 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/597046849' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T04:01:15.430 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:15 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/597046849' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T04:01:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.457+0000 7f198859b640 1 -- 172.21.15.64:0/2409488667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1980103b60 msgr2=0x7f1980103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.457+0000 7f198859b640 1 --2- 172.21.15.64:0/2409488667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1980103b60 0x7f1980103f70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f197c009900 tx=0x7f197c02a410 comp rx=0 tx=0).stop 2024-09-21T04:01:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.457+0000 7f198859b640 1 -- 172.21.15.64:0/2409488667 shutdown_connections 2024-09-21T04:01:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.457+0000 7f198859b640 1 --2- 172.21.15.64:0/2409488667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1980109df0 0x7f1980110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.457+0000 7f198859b640 1 --2- 172.21.15.64:0/2409488667 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19801094a0 0x7f19801098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.457+0000 7f198859b640 1 --2- 172.21.15.64:0/2409488667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1980103b60 0x7f1980103f70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.457+0000 7f198859b640 1 -- 172.21.15.64:0/2409488667 >> 172.21.15.64:0/2409488667 conn(0x7f198007b850 msgr2=0x7f198007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.458+0000 7f198859b640 1 -- 172.21.15.64:0/2409488667 shutdown_connections 2024-09-21T04:01:15.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.458+0000 7f198859b640 1 -- 172.21.15.64:0/2409488667 wait complete. 2024-09-21T04:01:15.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.458+0000 7f198859b640 1 Processor -- start 2024-09-21T04:01:15.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.458+0000 7f198859b640 1 -- start start 2024-09-21T04:01:15.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.459+0000 7f198859b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1980103b60 0x7f198019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:15.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.459+0000 7f198859b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19801094a0 0x7f198019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:15.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.459+0000 7f198859b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1980109df0 0x7f198019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:15.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.459+0000 7f198859b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1980113af0 con 0x7f19801094a0 2024-09-21T04:01:15.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.459+0000 7f198859b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1980113970 con 0x7f1980109df0 2024-09-21T04:01:15.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.459+0000 7f198859b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1980113c70 con 0x7f1980103b60 2024-09-21T04:01:15.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.459+0000 7f1985b36640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19801094a0 0x7f198019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:15.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.459+0000 7f1985b36640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19801094a0 0x7f198019aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56692/0 (socket says 172.21.15.64:56692) 2024-09-21T04:01:15.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.459+0000 7f1985b36640 1 -- 172.21.15.64:0/3192346681 learned_addr learned my addr 172.21.15.64:0/3192346681 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:01:15.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.459+0000 7f1986b38640 1 --2- 172.21.15.64:0/3192346681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1980109df0 0x7f198019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:15.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.459+0000 7f1986337640 1 --2- 172.21.15.64:0/3192346681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1980103b60 0x7f198019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:15.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.459+0000 7f1985b36640 1 -- 172.21.15.64:0/3192346681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1980103b60 msgr2=0x7f198019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:15.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.459+0000 7f1985b36640 1 --2- 172.21.15.64:0/3192346681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1980103b60 0x7f198019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:15.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.459+0000 7f1985b36640 1 -- 172.21.15.64:0/3192346681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1980109df0 msgr2=0x7f198019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:15.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.459+0000 7f1985b36640 1 --2- 172.21.15.64:0/3192346681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1980109df0 0x7f198019fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:15.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.459+0000 7f1985b36640 1 -- 172.21.15.64:0/3192346681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19801a0450 con 0x7f19801094a0 2024-09-21T04:01:15.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.460+0000 7f1985b36640 1 --2- 172.21.15.64:0/3192346681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19801094a0 0x7f198019aa80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1970006fd0 tx=0x7f197000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:15.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.460+0000 7f19777fe640 1 -- 172.21.15.64:0/3192346681 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1970011830 con 0x7f19801094a0 2024-09-21T04:01:15.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.460+0000 7f198859b640 1 -- 172.21.15.64:0/3192346681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19801a0740 con 0x7f19801094a0 2024-09-21T04:01:15.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.460+0000 7f19777fe640 1 -- 172.21.15.64:0/3192346681 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1970016070 con 0x7f19801094a0 2024-09-21T04:01:15.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.460+0000 7f19777fe640 1 -- 172.21.15.64:0/3192346681 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1970010e00 con 0x7f19801094a0 2024-09-21T04:01:15.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.460+0000 7f198859b640 1 -- 172.21.15.64:0/3192346681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19801a6ea0 con 0x7f19801094a0 2024-09-21T04:01:15.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.461+0000 7f198859b640 1 -- 172.21.15.64:0/3192346681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1980103f70 con 0x7f19801094a0 2024-09-21T04:01:15.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.463+0000 7f19777fe640 1 -- 172.21.15.64:0/3192346681 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f19700119d0 con 0x7f19801094a0 2024-09-21T04:01:15.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.463+0000 7f19777fe640 1 --2- 172.21.15.64:0/3192346681 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f196c0b6560 0x7f196c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:15.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.463+0000 7f1986337640 1 --2- 172.21.15.64:0/3192346681 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f196c0b6560 0x7f196c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:15.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.463+0000 7f19777fe640 1 -- 172.21.15.64:0/3192346681 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7f19700e6b50 con 0x7f19801094a0 2024-09-21T04:01:15.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.464+0000 7f1986337640 1 --2- 172.21.15.64:0/3192346681 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f196c0b6560 0x7f196c0b8a70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f197c031440 tx=0x7f197c005d50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:15.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.466+0000 7f19777fe640 1 -- 172.21.15.64:0/3192346681 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f19700ad7b0 con 0x7f19801094a0 2024-09-21T04:01:15.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.656+0000 7f198859b640 1 -- 172.21.15.64:0/3192346681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f198019b610 con 0x7f19801094a0 2024-09-21T04:01:15.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.657+0000 7f19777fe640 1 -- 172.21.15.64:0/3192346681 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+18809 (secure 0 0 0) 0x7f19700b1b90 con 0x7f19801094a0 2024-09-21T04:01:15.661 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:01:15.661 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":68,"fsid":"e7c927ba-77cc-11ef-baf5-efdc52797490","created":"2024-09-21T03:53:41.215073+0000","modified":"2024-09-21T04:01:12.990996+0000","last_up_change":"2024-09-21T04:01:11.986949+0000","last_in_change":"2024-09-21T04:00:48.707791+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:57:24.854438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb947c10-98e6-4e3a-8bbd-8753951d870b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6803","nonce":3692023035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6805","nonce":3692023035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6809","nonce":3692023035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6807","nonce":3692023035}]},"public_addr":"172.21.15.64:6803/3692023035","cluster_addr":"172.21.15.64:6805/3692023035","heartbeat_back_addr":"172.21.15.64:6809/3692023035","heartbeat_front_addr":"172.21.15.64:6807/3692023035","state":["exists","up"]},{"osd":1,"uuid":"29b8c92d-82fa-4175-b3e2-711917b7f8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6801","nonce":3363415998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6803","nonce":3363415998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6807","nonce":3363415998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6805","nonce":3363415998}]},"public_addr":"172.21.15.119:6801/3363415998","cluster_addr":"172.21.15.119:6803/3363415998","heartbeat_back_addr":"172.21.15.119:6807/3363415998","heartbeat_front_addr":"172.21.15.119:6805/3363415998","state":["exists","up"]},{"osd":2,"uuid":"a2273fe8-df55-40f7-afda-74ab09c52c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6801","nonce":3422924179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6803","nonce":3422924179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6807","nonce":3422924179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6805","nonce":3422924179}]},"public_addr":"172.21.15.131:6801/3422924179","cluster_addr":"172.21.15.131:6803/3422924179","heartbeat_back_addr":"172.21.15.131:6807/3422924179","heartbeat_front_addr":"172.21.15.131:6805/3422924179","state":["exists","up"]},{"osd":3,"uuid":"f1634064-b108-41d2-a78f-c33607b08a70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6811","nonce":2022952729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6813","nonce":2022952729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6817","nonce":2022952729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6815","nonce":2022952729}]},"public_addr":"172.21.15.64:6811/2022952729","cluster_addr":"172.21.15.64:6813/2022952729","heartbeat_back_addr":"172.21.15.64:6817/2022952729","heartbeat_front_addr":"172.21.15.64:6815/2022952729","state":["exists","up"]},{"osd":4,"uuid":"75b81a8d-fa52-411a-bfda-aa69bb70a20f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6809","nonce":323030958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6811","nonce":323030958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6815","nonce":323030958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6813","nonce":323030958}]},"public_addr":"172.21.15.119:6809/323030958","cluster_addr":"172.21.15.119:6811/323030958","heartbeat_back_addr":"172.21.15.119:6815/323030958","heartbeat_front_addr":"172.21.15.119:6813/323030958","state":["exists","up"]},{"osd":5,"uuid":"0e4895b5-42e4-4b2c-a9c3-692f287cb841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6809","nonce":3021788088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6811","nonce":3021788088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6815","nonce":3021788088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6813","nonce":3021788088}]},"public_addr":"172.21.15.131:6809/3021788088","cluster_addr":"172.21.15.131:6811/3021788088","heartbeat_back_addr":"172.21.15.131:6815/3021788088","heartbeat_front_addr":"172.21.15.131:6813/3021788088","state":["exists","up"]},{"osd":6,"uuid":"1fa5c2b7-bce0-43a1-a26b-d029eb3731fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6819","nonce":1913824324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6821","nonce":1913824324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6825","nonce":1913824324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6823","nonce":1913824324}]},"public_addr":"172.21.15.64:6819/1913824324","cluster_addr":"172.21.15.64:6821/1913824324","heartbeat_back_addr":"172.21.15.64:6825/1913824324","heartbeat_front_addr":"172.21.15.64:6823/1913824324","state":["exists","up"]},{"osd":7,"uuid":"c9707260-b581-4a21-b380-f37e16959b02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6817","nonce":2567568166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6819","nonce":2567568166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6823","nonce":2567568166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6821","nonce":2567568166}]},"public_addr":"172.21.15.119:6817/2567568166","cluster_addr":"172.21.15.119:6819/2567568166","heartbeat_back_addr":"172.21.15.119:6823/2567568166","heartbeat_front_addr":"172.21.15.119:6821/2567568166","state":["exists","up"]},{"osd":8,"uuid":"2dbf8ef2-fc9c-4c6b-bfea-003f41ef191e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6817","nonce":199192217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6819","nonce":199192217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6823","nonce":199192217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6821","nonce":199192217}]},"public_addr":"172.21.15.131:6817/199192217","cluster_addr":"172.21.15.131:6819/199192217","heartbeat_back_addr":"172.21.15.131:6823/199192217","heartbeat_front_addr":"172.21.15.131:6821/199192217","state":["exists","up"]},{"osd":9,"uuid":"96f769e5-4783-4558-adfa-4bfaf9da9e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6827","nonce":3309038990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6829","nonce":3309038990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6833","nonce":3309038990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6831","nonce":3309038990}]},"public_addr":"172.21.15.64:6827/3309038990","cluster_addr":"172.21.15.64:6829/3309038990","heartbeat_back_addr":"172.21.15.64:6833/3309038990","heartbeat_front_addr":"172.21.15.64:6831/3309038990","state":["exists","up"]},{"osd":10,"uuid":"13c1ef5c-2299-440c-af5a-a650b47f4b75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6825","nonce":1460844681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6827","nonce":1460844681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6831","nonce":1460844681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6829","nonce":1460844681}]},"public_addr":"172.21.15.119:6825/1460844681","cluster_addr":"172.21.15.119:6827/1460844681","heartbeat_back_addr":"172.21.15.119:6831/1460844681","heartbeat_front_addr":"172.21.15.119:6829/1460844681","state":["exists","up"]},{"osd":11,"uuid":"c26a5ef9-8585-404e-a54a-852424c4d6fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6825","nonce":2294059084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6827","nonce":2294059084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6831","nonce":2294059084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6829","nonce":2294059084}]},"public_addr":"172.21.15.131:6825/2294059084","cluster_addr":"172.21.15.131:6827/2294059084","heartbeat_back_addr":"172.21.15.131:6831/2294059084","heartbeat_front_addr":"172.21.15.131:6829/2294059084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:56:33.141102+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:56:58.883062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:57:22.641789+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:57:48.705790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:58:12.871514+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:58:37.893118+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:03.620828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:29.446179+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:55.002978+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:00:19.987463+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:00:45.364158+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:01:10.228248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/3554471670":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6801/3786213009":"2024-09-22T03:55:28.387426+0000","172.21.15.64:0/2562856406":"2024-09-22T03:54:27.180100+0000","172.21.15.64:0/2603015699":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6800/1209932961":"2024-09-22T03:54:02.562380+0000","172.21.15.64:6800/3786213009":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6801/1209932961":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/3407344596":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1233731831":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1686887701":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1598568021":"2024-09-22T03:54:27.180100+0000","172.21.15.64:6800/2013929312":"2024-09-22T03:54:27.180100+0000","172.21.15.64:6801/2013929312":"2024-09-22T03:54:27.180100+0000","172.21.15.64:0/3652036144":"2024-09-22T03:55:28.387426+0000","172.21.15.64:0/105454703":"2024-09-22T03:54:27.180100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T04:01:15.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.660+0000 7f198859b640 1 -- 172.21.15.64:0/3192346681 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f196c0b6560 msgr2=0x7f196c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:15.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.660+0000 7f198859b640 1 --2- 172.21.15.64:0/3192346681 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f196c0b6560 0x7f196c0b8a70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f197c031440 tx=0x7f197c005d50 comp rx=0 tx=0).stop 2024-09-21T04:01:15.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.660+0000 7f198859b640 1 -- 172.21.15.64:0/3192346681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19801094a0 msgr2=0x7f198019aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:15.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.660+0000 7f198859b640 1 --2- 172.21.15.64:0/3192346681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19801094a0 0x7f198019aa80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1970006fd0 tx=0x7f197000b8e0 comp rx=0 tx=0).stop 2024-09-21T04:01:15.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.660+0000 7f198859b640 1 -- 172.21.15.64:0/3192346681 shutdown_connections 2024-09-21T04:01:15.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.660+0000 7f198859b640 1 --2- 172.21.15.64:0/3192346681 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f196c0b6560 0x7f196c0b8a70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:15.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.660+0000 7f198859b640 1 --2- 172.21.15.64:0/3192346681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1980109df0 0x7f198019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:15.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.660+0000 7f198859b640 1 --2- 172.21.15.64:0/3192346681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f19801094a0 0x7f198019aa80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:15.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.660+0000 7f198859b640 1 --2- 172.21.15.64:0/3192346681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1980103b60 0x7f198019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:15.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.660+0000 7f198859b640 1 -- 172.21.15.64:0/3192346681 >> 172.21.15.64:0/3192346681 conn(0x7f198007b850 msgr2=0x7f1980107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:15.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.660+0000 7f198859b640 1 -- 172.21.15.64:0/3192346681 shutdown_connections 2024-09-21T04:01:15.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:15.660+0000 7f198859b640 1 -- 172.21.15.64:0/3192346681 wait complete. 2024-09-21T04:01:16.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:16 smithi064 ceph-mon[31180]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:16.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:16 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3192346681' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:01:16.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:16 smithi131 ceph-mon[35193]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:16.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:16 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3192346681' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:01:16.382 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:16 smithi119 ceph-mon[34811]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:16.382 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:16 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3192346681' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:01:16.406 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T03:57:24.854438+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T04:01:16.406 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd pool get .mgr pg_num 2024-09-21T04:01:16.661 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:01:17.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.114+0000 7fbd40fdc640 1 -- 172.21.15.64:0/311521542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd3c108cf0 msgr2=0x7fbd3c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:17.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.114+0000 7fbd40fdc640 1 --2- 172.21.15.64:0/311521542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd3c108cf0 0x7fbd3c10f520 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009ca0 tx=0x7fbd3402a8c0 comp rx=0 tx=0).stop 2024-09-21T04:01:17.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.114+0000 7fbd40fdc640 1 -- 172.21.15.64:0/311521542 shutdown_connections 2024-09-21T04:01:17.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.114+0000 7fbd40fdc640 1 --2- 172.21.15.64:0/311521542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd3c108cf0 0x7fbd3c10f520 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:17.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.114+0000 7fbd40fdc640 1 --2- 172.21.15.64:0/311521542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd3c1083a0 0x7fbd3c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:17.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.114+0000 7fbd40fdc640 1 --2- 172.21.15.64:0/311521542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd3c102a60 0x7fbd3c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:17.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.114+0000 7fbd40fdc640 1 -- 172.21.15.64:0/311521542 >> 172.21.15.64:0/311521542 conn(0x7fbd3c079730 msgr2=0x7fbd3c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:17.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.114+0000 7fbd40fdc640 1 -- 172.21.15.64:0/311521542 shutdown_connections 2024-09-21T04:01:17.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.114+0000 7fbd40fdc640 1 -- 172.21.15.64:0/311521542 wait complete. 2024-09-21T04:01:17.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.115+0000 7fbd40fdc640 1 Processor -- start 2024-09-21T04:01:17.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.115+0000 7fbd40fdc640 1 -- start start 2024-09-21T04:01:17.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.115+0000 7fbd40fdc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd3c102a60 0x7fbd3c19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:17.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.115+0000 7fbd40fdc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd3c1083a0 0x7fbd3c19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:17.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.115+0000 7fbd40fdc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd3c108cf0 0x7fbd3c19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:17.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.115+0000 7fbd40fdc640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c113a80 con 0x7fbd3c108cf0 2024-09-21T04:01:17.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.115+0000 7fbd40fdc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c113900 con 0x7fbd3c102a60 2024-09-21T04:01:17.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.115+0000 7fbd40fdc640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c113c00 con 0x7fbd3c1083a0 2024-09-21T04:01:17.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.115+0000 7fbd3ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd3c108cf0 0x7fbd3c19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.116+0000 7fbd3ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd3c108cf0 0x7fbd3c19fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56720/0 (socket says 172.21.15.64:56720) 2024-09-21T04:01:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.116+0000 7fbd3ad9d640 1 -- 172.21.15.64:0/3441112627 learned_addr learned my addr 172.21.15.64:0/3441112627 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:01:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.116+0000 7fbd39d9b640 1 --2- 172.21.15.64:0/3441112627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd3c1083a0 0x7fbd3c19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.116+0000 7fbd3a59c640 1 --2- 172.21.15.64:0/3441112627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd3c102a60 0x7fbd3c19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.116+0000 7fbd3ad9d640 1 -- 172.21.15.64:0/3441112627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd3c1083a0 msgr2=0x7fbd3c19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.116+0000 7fbd3ad9d640 1 --2- 172.21.15.64:0/3441112627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd3c1083a0 0x7fbd3c19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.116+0000 7fbd3ad9d640 1 -- 172.21.15.64:0/3441112627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd3c102a60 msgr2=0x7fbd3c19a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.116+0000 7fbd3ad9d640 1 --2- 172.21.15.64:0/3441112627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd3c102a60 0x7fbd3c19a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.116+0000 7fbd3ad9d640 1 -- 172.21.15.64:0/3441112627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3c1a02c0 con 0x7fbd3c108cf0 2024-09-21T04:01:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.116+0000 7fbd3ad9d640 1 --2- 172.21.15.64:0/3441112627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd3c108cf0 0x7fbd3c19fc50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbd340098d0 tx=0x7fbd340317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.116+0000 7fbd2b7fe640 1 -- 172.21.15.64:0/3441112627 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd34003b70 con 0x7fbd3c108cf0 2024-09-21T04:01:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.116+0000 7fbd2b7fe640 1 -- 172.21.15.64:0/3441112627 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd34003d10 con 0x7fbd3c108cf0 2024-09-21T04:01:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.116+0000 7fbd2b7fe640 1 -- 172.21.15.64:0/3441112627 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3402f9d0 con 0x7fbd3c108cf0 2024-09-21T04:01:17.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.116+0000 7fbd40fdc640 1 -- 172.21.15.64:0/3441112627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd3c1a0550 con 0x7fbd3c108cf0 2024-09-21T04:01:17.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.116+0000 7fbd40fdc640 1 -- 172.21.15.64:0/3441112627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd3c1a6dd0 con 0x7fbd3c108cf0 2024-09-21T04:01:17.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.117+0000 7fbd40fdc640 1 -- 172.21.15.64:0/3441112627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd3c102e70 con 0x7fbd3c108cf0 2024-09-21T04:01:17.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.119+0000 7fbd2b7fe640 1 -- 172.21.15.64:0/3441112627 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fbd3402fb70 con 0x7fbd3c108cf0 2024-09-21T04:01:17.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.120+0000 7fbd2b7fe640 1 --2- 172.21.15.64:0/3441112627 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbd240b6610 0x7fbd240b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:17.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.120+0000 7fbd3a59c640 1 --2- 172.21.15.64:0/3441112627 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbd240b6610 0x7fbd240b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:17.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.120+0000 7fbd2b7fe640 1 -- 172.21.15.64:0/3441112627 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7fbd34108240 con 0x7fbd3c108cf0 2024-09-21T04:01:17.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.121+0000 7fbd3a59c640 1 --2- 172.21.15.64:0/3441112627 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbd240b6610 0x7fbd240b8b20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c009bd0 tx=0x7fbd2c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:17.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.123+0000 7fbd2b7fe640 1 -- 172.21.15.64:0/3441112627 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbd340ceea0 con 0x7fbd3c108cf0 2024-09-21T04:01:17.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.311+0000 7fbd40fdc640 1 -- 172.21.15.64:0/3441112627 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbd3c10aac0 con 0x7fbd3c108cf0 2024-09-21T04:01:17.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.311+0000 7fbd2b7fe640 1 -- 172.21.15.64:0/3441112627 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) ==== 93+0+10 (secure 0 0 0) 0x7fbd340d3280 con 0x7fbd3c108cf0 2024-09-21T04:01:17.315 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-09-21T04:01:17.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.313+0000 7fbd40fdc640 1 -- 172.21.15.64:0/3441112627 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbd240b6610 msgr2=0x7fbd240b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:17.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.313+0000 7fbd40fdc640 1 --2- 172.21.15.64:0/3441112627 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbd240b6610 0x7fbd240b8b20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c009bd0 tx=0x7fbd2c005f90 comp rx=0 tx=0).stop 2024-09-21T04:01:17.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.313+0000 7fbd40fdc640 1 -- 172.21.15.64:0/3441112627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd3c108cf0 msgr2=0x7fbd3c19fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:17.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.313+0000 7fbd40fdc640 1 --2- 172.21.15.64:0/3441112627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd3c108cf0 0x7fbd3c19fc50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbd340098d0 tx=0x7fbd340317b0 comp rx=0 tx=0).stop 2024-09-21T04:01:17.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.313+0000 7fbd40fdc640 1 -- 172.21.15.64:0/3441112627 shutdown_connections 2024-09-21T04:01:17.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.313+0000 7fbd40fdc640 1 --2- 172.21.15.64:0/3441112627 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbd240b6610 0x7fbd240b8b20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:17.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.313+0000 7fbd40fdc640 1 --2- 172.21.15.64:0/3441112627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd3c108cf0 0x7fbd3c19fc50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:17.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.313+0000 7fbd40fdc640 1 --2- 172.21.15.64:0/3441112627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbd3c1083a0 0x7fbd3c19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:17.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.313+0000 7fbd40fdc640 1 --2- 172.21.15.64:0/3441112627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd3c102a60 0x7fbd3c19a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:17.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.313+0000 7fbd40fdc640 1 -- 172.21.15.64:0/3441112627 >> 172.21.15.64:0/3441112627 conn(0x7fbd3c079730 msgr2=0x7fbd3c104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:17.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.313+0000 7fbd40fdc640 1 -- 172.21.15.64:0/3441112627 shutdown_connections 2024-09-21T04:01:17.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:17.313+0000 7fbd40fdc640 1 -- 172.21.15.64:0/3441112627 wait complete. 2024-09-21T04:01:18.072 INFO:tasks.cephadm:Adding mds.a on smithi064 2024-09-21T04:01:18.072 INFO:tasks.cephadm:Adding mds.d on smithi064 2024-09-21T04:01:18.072 INFO:tasks.cephadm:Adding mds.g on smithi064 2024-09-21T04:01:18.072 INFO:tasks.cephadm:Adding mds.j on smithi064 2024-09-21T04:01:18.072 INFO:tasks.cephadm:Adding mds.b on smithi119 2024-09-21T04:01:18.073 INFO:tasks.cephadm:Adding mds.e on smithi119 2024-09-21T04:01:18.073 INFO:tasks.cephadm:Adding mds.h on smithi119 2024-09-21T04:01:18.073 INFO:tasks.cephadm:Adding mds.k on smithi119 2024-09-21T04:01:18.073 INFO:tasks.cephadm:Adding mds.c on smithi131 2024-09-21T04:01:18.073 INFO:tasks.cephadm:Adding mds.f on smithi131 2024-09-21T04:01:18.073 INFO:tasks.cephadm:Adding mds.i on smithi131 2024-09-21T04:01:18.073 INFO:tasks.cephadm:Adding mds.l on smithi131 2024-09-21T04:01:18.073 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph orch apply mds all '12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi131=c;smithi131=f;smithi131=i;smithi131=l' 2024-09-21T04:01:18.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:18 smithi131 ceph-mon[35193]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:18.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:18 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3441112627' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T04:01:18.335 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T04:01:18.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:18 smithi119 ceph-mon[34811]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:18.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:18 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3441112627' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T04:01:18.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:18 smithi064 ceph-mon[31180]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:18.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:18 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3441112627' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T04:01:18.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.800+0000 7fbe94db8640 1 -- 172.21.15.131:0/1022930743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe90108450 msgr2=0x7fbe90108860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:18.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.800+0000 7fbe94db8640 1 --2- 172.21.15.131:0/1022930743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe90108450 0x7fbe90108860 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbe78009900 tx=0x7fbe7802a410 comp rx=0 tx=0).stop 2024-09-21T04:01:18.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.801+0000 7fbe94db8640 1 -- 172.21.15.131:0/1022930743 shutdown_connections 2024-09-21T04:01:18.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.801+0000 7fbe94db8640 1 --2- 172.21.15.131:0/1022930743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe90108da0 0x7fbe9010f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:18.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.801+0000 7fbe94db8640 1 --2- 172.21.15.131:0/1022930743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe90108450 0x7fbe90108860 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:18.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.801+0000 7fbe94db8640 1 --2- 172.21.15.131:0/1022930743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe90102b10 0x7fbe90102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:18.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.801+0000 7fbe94db8640 1 -- 172.21.15.131:0/1022930743 >> 172.21.15.131:0/1022930743 conn(0x7fbe900796d0 msgr2=0x7fbe90079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:18.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.801+0000 7fbe94db8640 1 -- 172.21.15.131:0/1022930743 shutdown_connections 2024-09-21T04:01:18.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.801+0000 7fbe94db8640 1 -- 172.21.15.131:0/1022930743 wait complete. 2024-09-21T04:01:18.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.801+0000 7fbe94db8640 1 Processor -- start 2024-09-21T04:01:18.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.802+0000 7fbe94db8640 1 -- start start 2024-09-21T04:01:18.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.802+0000 7fbe94db8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe90102b10 0x7fbe9019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:18.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.802+0000 7fbe94db8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe90108450 0x7fbe9019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:18.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.802+0000 7fbe94db8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe90108da0 0x7fbe9019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:18.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.802+0000 7fbe94db8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbe90112a60 con 0x7fbe90108450 2024-09-21T04:01:18.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.802+0000 7fbe94db8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbe901128e0 con 0x7fbe90108da0 2024-09-21T04:01:18.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.802+0000 7fbe94db8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbe90112be0 con 0x7fbe90102b10 2024-09-21T04:01:18.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.802+0000 7fbe8e59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe90102b10 0x7fbe9019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:18.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.802+0000 7fbe8e59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe90102b10 0x7fbe9019a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:53782/0 (socket says 172.21.15.131:53782) 2024-09-21T04:01:18.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.802+0000 7fbe8e59c640 1 -- 172.21.15.131:0/2613879562 learned_addr learned my addr 172.21.15.131:0/2613879562 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-21T04:01:18.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.802+0000 7fbe8ed9d640 1 --2- 172.21.15.131:0/2613879562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe90108da0 0x7fbe9019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:18.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.802+0000 7fbe85d9b640 1 --2- 172.21.15.131:0/2613879562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe90108450 0x7fbe9019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:18.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.802+0000 7fbe8e59c640 1 -- 172.21.15.131:0/2613879562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe90108da0 msgr2=0x7fbe9019fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:18.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.802+0000 7fbe8e59c640 1 --2- 172.21.15.131:0/2613879562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe90108da0 0x7fbe9019fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:18.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.802+0000 7fbe8e59c640 1 -- 172.21.15.131:0/2613879562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe90108450 msgr2=0x7fbe9019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:18.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.802+0000 7fbe8e59c640 1 --2- 172.21.15.131:0/2613879562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe90108450 0x7fbe9019ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:18.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.802+0000 7fbe8e59c640 1 -- 172.21.15.131:0/2613879562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe901a04d0 con 0x7fbe90102b10 2024-09-21T04:01:18.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.802+0000 7fbe85d9b640 1 --2- 172.21.15.131:0/2613879562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe90108450 0x7fbe9019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:01:18.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.803+0000 7fbe8e59c640 1 --2- 172.21.15.131:0/2613879562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe90102b10 0x7fbe9019a560 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbe8000e9f0 tx=0x7fbe8000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:18.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.803+0000 7fbe87fff640 1 -- 172.21.15.131:0/2613879562 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8000cd80 con 0x7fbe90102b10 2024-09-21T04:01:18.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.803+0000 7fbe94db8640 1 -- 172.21.15.131:0/2613879562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe901a07c0 con 0x7fbe90102b10 2024-09-21T04:01:18.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.803+0000 7fbe87fff640 1 -- 172.21.15.131:0/2613879562 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe800108e0 con 0x7fbe90102b10 2024-09-21T04:01:18.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.803+0000 7fbe87fff640 1 -- 172.21.15.131:0/2613879562 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe80018980 con 0x7fbe90102b10 2024-09-21T04:01:18.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.803+0000 7fbe94db8640 1 -- 172.21.15.131:0/2613879562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe901a6f20 con 0x7fbe90102b10 2024-09-21T04:01:18.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.804+0000 7fbe94db8640 1 -- 172.21.15.131:0/2613879562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe90102f20 con 0x7fbe90102b10 2024-09-21T04:01:18.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.806+0000 7fbe87fff640 1 -- 172.21.15.131:0/2613879562 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fbe80010430 con 0x7fbe90102b10 2024-09-21T04:01:18.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.807+0000 7fbe87fff640 1 --2- 172.21.15.131:0/2613879562 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbe7c0b6560 0x7fbe7c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:18.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.807+0000 7fbe87fff640 1 -- 172.21.15.131:0/2613879562 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7fbe800e78b0 con 0x7fbe90102b10 2024-09-21T04:01:18.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.807+0000 7fbe85d9b640 1 --2- 172.21.15.131:0/2613879562 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbe7c0b6560 0x7fbe7c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:18.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.808+0000 7fbe85d9b640 1 --2- 172.21.15.131:0/2613879562 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbe7c0b6560 0x7fbe7c0b8a70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbe78000c00 tx=0x7fbe78031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:18.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.810+0000 7fbe87fff640 1 -- 172.21.15.131:0/2613879562 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbe800ae510 con 0x7fbe90102b10 2024-09-21T04:01:18.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:18.996+0000 7fbe94db8640 1 -- 172.21.15.131:0/2613879562 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi131=c;smithi131=f;smithi131=i;smithi131=l", "target": ["mon-mgr", ""]}) -- 0x7fbe90063800 con 0x7fbe7c0b6560 2024-09-21T04:01:19.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:19.004+0000 7fbe87fff640 1 -- 172.21.15.131:0/2613879562 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fbe90063800 con 0x7fbe7c0b6560 2024-09-21T04:01:19.007 INFO:teuthology.orchestra.run.smithi131.stdout:Scheduled mds.all update... 2024-09-21T04:01:19.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:19.006+0000 7fbe94db8640 1 -- 172.21.15.131:0/2613879562 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbe7c0b6560 msgr2=0x7fbe7c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:19.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:19.006+0000 7fbe94db8640 1 --2- 172.21.15.131:0/2613879562 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbe7c0b6560 0x7fbe7c0b8a70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbe78000c00 tx=0x7fbe78031400 comp rx=0 tx=0).stop 2024-09-21T04:01:19.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:19.006+0000 7fbe94db8640 1 -- 172.21.15.131:0/2613879562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe90102b10 msgr2=0x7fbe9019a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:19.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:19.006+0000 7fbe94db8640 1 --2- 172.21.15.131:0/2613879562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe90102b10 0x7fbe9019a560 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbe8000e9f0 tx=0x7fbe8000eeb0 comp rx=0 tx=0).stop 2024-09-21T04:01:19.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:19.006+0000 7fbe94db8640 1 -- 172.21.15.131:0/2613879562 shutdown_connections 2024-09-21T04:01:19.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:19.006+0000 7fbe94db8640 1 --2- 172.21.15.131:0/2613879562 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbe7c0b6560 0x7fbe7c0b8a70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:19.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:19.006+0000 7fbe94db8640 1 --2- 172.21.15.131:0/2613879562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe90108da0 0x7fbe9019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:19.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:19.006+0000 7fbe94db8640 1 --2- 172.21.15.131:0/2613879562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe90108450 0x7fbe9019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:19.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:19.006+0000 7fbe94db8640 1 --2- 172.21.15.131:0/2613879562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe90102b10 0x7fbe9019a560 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:19.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:19.006+0000 7fbe94db8640 1 -- 172.21.15.131:0/2613879562 >> 172.21.15.131:0/2613879562 conn(0x7fbe900796d0 msgr2=0x7fbe90101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:19.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:19.007+0000 7fbe94db8640 1 -- 172.21.15.131:0/2613879562 shutdown_connections 2024-09-21T04:01:19.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-21T04:01:19.007+0000 7fbe94db8640 1 -- 172.21.15.131:0/2613879562 wait complete. 2024-09-21T04:01:19.760 DEBUG:teuthology.orchestra.run.smithi064:mds.a> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.a.service 2024-09-21T04:01:19.763 DEBUG:teuthology.orchestra.run.smithi064:mds.d> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.d.service 2024-09-21T04:01:19.807 DEBUG:teuthology.orchestra.run.smithi064:mds.g> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.g.service 2024-09-21T04:01:19.850 DEBUG:teuthology.orchestra.run.smithi064:mds.j> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.j.service 2024-09-21T04:01:19.893 DEBUG:teuthology.orchestra.run.smithi119:mds.b> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.b.service 2024-09-21T04:01:19.896 DEBUG:teuthology.orchestra.run.smithi119:mds.e> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.e.service 2024-09-21T04:01:19.939 DEBUG:teuthology.orchestra.run.smithi119:mds.h> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.h.service 2024-09-21T04:01:19.982 DEBUG:teuthology.orchestra.run.smithi119:mds.k> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.k.service 2024-09-21T04:01:20.025 DEBUG:teuthology.orchestra.run.smithi131:mds.c> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.c.service 2024-09-21T04:01:20.028 DEBUG:teuthology.orchestra.run.smithi131:mds.f> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.f.service 2024-09-21T04:01:20.072 DEBUG:teuthology.orchestra.run.smithi131:mds.i> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.i.service 2024-09-21T04:01:20.115 DEBUG:teuthology.orchestra.run.smithi131:mds.l> sudo journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.l.service 2024-09-21T04:01:20.158 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-21T04:01:20.158 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:01:20.183 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:20 smithi064 ceph-mon[31180]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:20.183 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:20 smithi064 ceph-mon[31180]: from='client.24580 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi131=c;smithi131=f;smithi131=i;smithi131=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:01:20.183 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:20 smithi064 ceph-mon[31180]: Saving service mds.all spec with placement smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi131=c;smithi131=f;smithi131=i;smithi131=l;count:12 2024-09-21T04:01:20.183 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:20 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:20.183 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:20 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:01:20.183 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:20 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:20.183 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:20 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:01:20.183 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:20 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:20.183 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:20 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:20.183 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:20 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:20.184 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:20 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:20.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:20 smithi119 ceph-mon[34811]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:20.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:20 smithi119 ceph-mon[34811]: from='client.24580 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi131=c;smithi131=f;smithi131=i;smithi131=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:01:20.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:20 smithi119 ceph-mon[34811]: Saving service mds.all spec with placement smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi131=c;smithi131=f;smithi131=i;smithi131=l;count:12 2024-09-21T04:01:20.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:20 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:20.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:20 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:01:20.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:20 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:20.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:20 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:01:20.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:20 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:20.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:20 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:20.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:20 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:20.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:20 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:20.258 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:20 smithi131 ceph-mon[35193]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:20.258 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:20 smithi131 ceph-mon[35193]: from='client.24580 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi131=c;smithi131=f;smithi131=i;smithi131=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:01:20.258 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:20 smithi131 ceph-mon[35193]: Saving service mds.all spec with placement smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi131=c;smithi131=f;smithi131=i;smithi131=l;count:12 2024-09-21T04:01:20.258 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:20 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:20.258 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:20 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:01:20.259 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:20 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:20.259 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:20 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:01:20.259 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:20 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:20.259 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:20 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:20.259 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:20 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:20.259 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:20 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:20.415 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:01:20.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.828+0000 7fcba7b92640 1 -- 172.21.15.64:0/902355183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcba0108d10 msgr2=0x7fcba010f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:20.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.828+0000 7fcba7b92640 1 --2- 172.21.15.64:0/902355183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcba0108d10 0x7fcba010f540 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcb98009900 tx=0x7fcb9802a410 comp rx=0 tx=0).stop 2024-09-21T04:01:20.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.828+0000 7fcba7b92640 1 -- 172.21.15.64:0/902355183 shutdown_connections 2024-09-21T04:01:20.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.828+0000 7fcba7b92640 1 --2- 172.21.15.64:0/902355183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcba0108d10 0x7fcba010f540 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:20.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.828+0000 7fcba7b92640 1 --2- 172.21.15.64:0/902355183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcba01083c0 0x7fcba01087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:20.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.828+0000 7fcba7b92640 1 --2- 172.21.15.64:0/902355183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcba0102a80 0x7fcba0102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:20.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.828+0000 7fcba7b92640 1 -- 172.21.15.64:0/902355183 >> 172.21.15.64:0/902355183 conn(0x7fcba0079740 msgr2=0x7fcba0079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:20.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.828+0000 7fcba7b92640 1 -- 172.21.15.64:0/902355183 shutdown_connections 2024-09-21T04:01:20.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.828+0000 7fcba7b92640 1 -- 172.21.15.64:0/902355183 wait complete. 2024-09-21T04:01:20.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.829+0000 7fcba7b92640 1 Processor -- start 2024-09-21T04:01:20.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.829+0000 7fcba7b92640 1 -- start start 2024-09-21T04:01:20.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.829+0000 7fcba7b92640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcba0102a80 0x7fcba010ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:20.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.829+0000 7fcba7b92640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcba01083c0 0x7fcba0110510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:20.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.829+0000 7fcba7b92640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcba0108d10 0x7fcba0110a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:20.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.829+0000 7fcba7b92640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcba0113aa0 con 0x7fcba01083c0 2024-09-21T04:01:20.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.829+0000 7fcba7b92640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcba0113920 con 0x7fcba0102a80 2024-09-21T04:01:20.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.829+0000 7fcba7b92640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcba0113c20 con 0x7fcba0108d10 2024-09-21T04:01:20.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.829+0000 7fcba512d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcba01083c0 0x7fcba0110510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:20.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.829+0000 7fcba512d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcba01083c0 0x7fcba0110510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56732/0 (socket says 172.21.15.64:56732) 2024-09-21T04:01:20.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.829+0000 7fcba512d640 1 -- 172.21.15.64:0/3212283176 learned_addr learned my addr 172.21.15.64:0/3212283176 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:01:20.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.829+0000 7fcba592e640 1 --2- 172.21.15.64:0/3212283176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcba0102a80 0x7fcba010ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:20.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.829+0000 7fcba612f640 1 --2- 172.21.15.64:0/3212283176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcba0108d10 0x7fcba0110a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:20.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.829+0000 7fcba512d640 1 -- 172.21.15.64:0/3212283176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcba0108d10 msgr2=0x7fcba0110a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:20.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.829+0000 7fcba512d640 1 --2- 172.21.15.64:0/3212283176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcba0108d10 0x7fcba0110a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:20.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.829+0000 7fcba512d640 1 -- 172.21.15.64:0/3212283176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcba0102a80 msgr2=0x7fcba010ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:20.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.829+0000 7fcba512d640 1 --2- 172.21.15.64:0/3212283176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcba0102a80 0x7fcba010ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:20.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.829+0000 7fcba512d640 1 -- 172.21.15.64:0/3212283176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba01a8770 con 0x7fcba01083c0 2024-09-21T04:01:20.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.830+0000 7fcba512d640 1 --2- 172.21.15.64:0/3212283176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcba01083c0 0x7fcba0110510 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400c970 tx=0x7fcb9400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:20.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.830+0000 7fcb92ffd640 1 -- 172.21.15.64:0/3212283176 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb94007bb0 con 0x7fcba01083c0 2024-09-21T04:01:20.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.830+0000 7fcb92ffd640 1 -- 172.21.15.64:0/3212283176 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb94007d50 con 0x7fcba01083c0 2024-09-21T04:01:20.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.830+0000 7fcba7b92640 1 -- 172.21.15.64:0/3212283176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba01a8a60 con 0x7fcba01083c0 2024-09-21T04:01:20.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.830+0000 7fcb92ffd640 1 -- 172.21.15.64:0/3212283176 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9400f450 con 0x7fcba01083c0 2024-09-21T04:01:20.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.830+0000 7fcba7b92640 1 -- 172.21.15.64:0/3212283176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba01a8ff0 con 0x7fcba01083c0 2024-09-21T04:01:20.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.831+0000 7fcba7b92640 1 -- 172.21.15.64:0/3212283176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcba0103d00 con 0x7fcba01083c0 2024-09-21T04:01:20.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.834+0000 7fcb92ffd640 1 -- 172.21.15.64:0/3212283176 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fcb94004f20 con 0x7fcba01083c0 2024-09-21T04:01:20.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.835+0000 7fcb92ffd640 1 --2- 172.21.15.64:0/3212283176 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcb8c0b65e0 0x7fcb8c0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:20.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.835+0000 7fcba592e640 1 --2- 172.21.15.64:0/3212283176 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcb8c0b65e0 0x7fcb8c0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:20.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.835+0000 7fcb92ffd640 1 -- 172.21.15.64:0/3212283176 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7fcb940e6fa0 con 0x7fcba01083c0 2024-09-21T04:01:20.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.835+0000 7fcba592e640 1 --2- 172.21.15.64:0/3212283176 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcb8c0b65e0 0x7fcb8c0b8af0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c005c40 tx=0x7fcb9c00a390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:20.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:20.838+0000 7fcb92ffd640 1 -- 172.21.15.64:0/3212283176 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcb94014030 con 0x7fcba01083c0 2024-09-21T04:01:21.071 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:01:21.071 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":1,"btime":"2024-09-21T03:53:41:214835+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-21T04:01:21.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:21.068+0000 7fcba7b92640 1 -- 172.21.15.64:0/3212283176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcba0102e90 con 0x7fcba01083c0 2024-09-21T04:01:21.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:21.068+0000 7fcb92ffd640 1 -- 172.21.15.64:0/3212283176 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fcba0102e90 con 0x7fcba01083c0 2024-09-21T04:01:21.072 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 1 2024-09-21T04:01:21.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:21.071+0000 7fcba7b92640 1 -- 172.21.15.64:0/3212283176 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcb8c0b65e0 msgr2=0x7fcb8c0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:21.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:21.071+0000 7fcba7b92640 1 --2- 172.21.15.64:0/3212283176 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcb8c0b65e0 0x7fcb8c0b8af0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c005c40 tx=0x7fcb9c00a390 comp rx=0 tx=0).stop 2024-09-21T04:01:21.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:21.071+0000 7fcba7b92640 1 -- 172.21.15.64:0/3212283176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcba01083c0 msgr2=0x7fcba0110510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:21.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:21.071+0000 7fcba7b92640 1 --2- 172.21.15.64:0/3212283176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcba01083c0 0x7fcba0110510 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400c970 tx=0x7fcb9400ce30 comp rx=0 tx=0).stop 2024-09-21T04:01:21.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:21.071+0000 7fcba7b92640 1 -- 172.21.15.64:0/3212283176 shutdown_connections 2024-09-21T04:01:21.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:21.071+0000 7fcba7b92640 1 --2- 172.21.15.64:0/3212283176 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcb8c0b65e0 0x7fcb8c0b8af0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:21.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:21.071+0000 7fcba7b92640 1 --2- 172.21.15.64:0/3212283176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcba0108d10 0x7fcba0110a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:21.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:21.071+0000 7fcba7b92640 1 --2- 172.21.15.64:0/3212283176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcba01083c0 0x7fcba0110510 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:21.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:21.071+0000 7fcba7b92640 1 --2- 172.21.15.64:0/3212283176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcba0102a80 0x7fcba010ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:21.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:21.071+0000 7fcba7b92640 1 -- 172.21.15.64:0/3212283176 >> 172.21.15.64:0/3212283176 conn(0x7fcba0079740 msgr2=0x7fcba0104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:21.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:21.071+0000 7fcba7b92640 1 -- 172.21.15.64:0/3212283176 shutdown_connections 2024-09-21T04:01:21.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:21.071+0000 7fcba7b92640 1 -- 172.21.15.64:0/3212283176 wait complete. 2024-09-21T04:01:21.084 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:21 smithi064 ceph-mon[31180]: Deploying daemon mds.l on smithi131 2024-09-21T04:01:21.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:21 smithi131 ceph-mon[35193]: Deploying daemon mds.l on smithi131 2024-09-21T04:01:21.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:21 smithi119 ceph-mon[34811]: Deploying daemon mds.l on smithi131 2024-09-21T04:01:21.744 INFO:journalctl@ceph.mds.l.smithi131.stdout:Sep 21 04:01:21 smithi131 systemd[1]: Starting Ceph mds.l for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:01:22.012 INFO:journalctl@ceph.mds.l.smithi131.stdout:Sep 21 04:01:21 smithi131 podman[63801]: 2024-09-21 04:01:21.644764162 +0000 UTC m=+0.016063956 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:01:22.013 INFO:journalctl@ceph.mds.l.smithi131.stdout:Sep 21 04:01:21 smithi131 podman[63801]: 2024-09-21 04:01:21.764021291 +0000 UTC m=+0.135321123 container create b28956eef11c6464b20b497c7434383c47d172788c73e6b5411dda5eca748f0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-l, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-21T04:01:22.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:22 smithi064 ceph-mon[31180]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:22.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:22 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3212283176' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:22.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:22 smithi131 ceph-mon[35193]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:22.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:22 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3212283176' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:22.303 INFO:journalctl@ceph.mds.l.smithi131.stdout:Sep 21 04:01:22 smithi131 podman[63801]: 2024-09-21 04:01:22.016815405 +0000 UTC m=+0.388115189 container init b28956eef11c6464b20b497c7434383c47d172788c73e6b5411dda5eca748f0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-l, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-09-21T04:01:22.304 INFO:journalctl@ceph.mds.l.smithi131.stdout:Sep 21 04:01:22 smithi131 podman[63801]: 2024-09-21 04:01:22.019803805 +0000 UTC m=+0.391103588 container start b28956eef11c6464b20b497c7434383c47d172788c73e6b5411dda5eca748f0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-21T04:01:22.304 INFO:journalctl@ceph.mds.l.smithi131.stdout:Sep 21 04:01:22 smithi131 ceph-mds[63820]: -- 172.21.15.131:0/2975131176 <== mon.1 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558a5309b860 con 0x558a57243a80 2024-09-21T04:01:22.304 INFO:journalctl@ceph.mds.l.smithi131.stdout:Sep 21 04:01:22 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-l[63816]: starting mds.l at 2024-09-21T04:01:22.304 INFO:journalctl@ceph.mds.l.smithi131.stdout:Sep 21 04:01:22 smithi131 bash[63801]: b28956eef11c6464b20b497c7434383c47d172788c73e6b5411dda5eca748f0b 2024-09-21T04:01:22.304 INFO:journalctl@ceph.mds.l.smithi131.stdout:Sep 21 04:01:22 smithi131 systemd[1]: Started Ceph mds.l for e7c927ba-77cc-11ef-baf5-efdc52797490. 2024-09-21T04:01:22.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:22 smithi119 ceph-mon[34811]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:22.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:22 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3212283176' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:23.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:23 smithi119 ceph-mon[34811]: mds.? [v2:172.21.15.131:6832/1651684499,v1:172.21.15.131:6833/1651684499] up:boot 2024-09-21T04:01:23.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:23 smithi119 ceph-mon[34811]: fsmap 1 up:standby 2024-09-21T04:01:23.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:23 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T04:01:23.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:23 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:23.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:23 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:23.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:23 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:23.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:23 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:23.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:23 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:23.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:23 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:23.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:23 smithi119 ceph-mon[34811]: Deploying daemon mds.k on smithi119 2024-09-21T04:01:23.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:23 smithi064 ceph-mon[31180]: mds.? [v2:172.21.15.131:6832/1651684499,v1:172.21.15.131:6833/1651684499] up:boot 2024-09-21T04:01:23.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:23 smithi064 ceph-mon[31180]: fsmap 1 up:standby 2024-09-21T04:01:23.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:23 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T04:01:23.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:23 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:23.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:23 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:23.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:23 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:23.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:23 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:23.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:23 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:23.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:23 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:23.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:23 smithi064 ceph-mon[31180]: Deploying daemon mds.k on smithi119 2024-09-21T04:01:23.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:23 smithi131 ceph-mon[35193]: mds.? [v2:172.21.15.131:6832/1651684499,v1:172.21.15.131:6833/1651684499] up:boot 2024-09-21T04:01:23.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:23 smithi131 ceph-mon[35193]: fsmap 1 up:standby 2024-09-21T04:01:23.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:23 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T04:01:23.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:23 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:23.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:23 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:23.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:23 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:23.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:23 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:23.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:23 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:23.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:23 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:23.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:23 smithi131 ceph-mon[35193]: Deploying daemon mds.k on smithi119 2024-09-21T04:01:23.960 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:01:24.199 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 04:01:24 smithi119 systemd[1]: Starting Ceph mds.k for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:01:24.229 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:01:24.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:24 smithi064 ceph-mon[31180]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:24.504 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:24 smithi119 ceph-mon[34811]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:24.504 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 04:01:24 smithi119 podman[63319]: 2024-09-21 04:01:24.154480016 +0000 UTC m=+0.015423456 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:01:24.504 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 04:01:24 smithi119 podman[63319]: 2024-09-21 04:01:24.272453371 +0000 UTC m=+0.133396847 container create a9143ff40f362b21fdfaf30252f9915d68ba5dd42c767a96969a98cce21338f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T04:01:24.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:24 smithi131 ceph-mon[35193]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:24.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.714+0000 7f63ae321640 1 -- 172.21.15.64:0/562453164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63a8103b60 msgr2=0x7f63a8103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:24.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.714+0000 7f63ae321640 1 --2- 172.21.15.64:0/562453164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63a8103b60 0x7f63a8103f70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6398009900 tx=0x7f639802a410 comp rx=0 tx=0).stop 2024-09-21T04:01:24.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.714+0000 7f63ae321640 1 -- 172.21.15.64:0/562453164 shutdown_connections 2024-09-21T04:01:24.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.714+0000 7f63ae321640 1 --2- 172.21.15.64:0/562453164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63a8109df0 0x7f63a8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:24.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.714+0000 7f63ae321640 1 --2- 172.21.15.64:0/562453164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63a81094a0 0x7f63a81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:24.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.714+0000 7f63ae321640 1 --2- 172.21.15.64:0/562453164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63a8103b60 0x7f63a8103f70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:24.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.714+0000 7f63ae321640 1 -- 172.21.15.64:0/562453164 >> 172.21.15.64:0/562453164 conn(0x7f63a807b850 msgr2=0x7f63a807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:24.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.714+0000 7f63ae321640 1 -- 172.21.15.64:0/562453164 shutdown_connections 2024-09-21T04:01:24.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.714+0000 7f63ae321640 1 -- 172.21.15.64:0/562453164 wait complete. 2024-09-21T04:01:24.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.715+0000 7f63ae321640 1 Processor -- start 2024-09-21T04:01:24.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.715+0000 7f63ae321640 1 -- start start 2024-09-21T04:01:24.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.715+0000 7f63ae321640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63a8103b60 0x7f63a819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:24.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.715+0000 7f63ae321640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63a81094a0 0x7f63a819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:24.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.715+0000 7f63ae321640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63a8109df0 0x7f63a819fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:24.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.715+0000 7f63ae321640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f63a8113ab0 con 0x7f63a8103b60 2024-09-21T04:01:24.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.715+0000 7f63ae321640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f63a8113930 con 0x7f63a8109df0 2024-09-21T04:01:24.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.715+0000 7f63ae321640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f63a8113c30 con 0x7f63a81094a0 2024-09-21T04:01:24.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.715+0000 7f63a7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63a8103b60 0x7f63a819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:24.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.716+0000 7f63a7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63a8103b60 0x7f63a819a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54568/0 (socket says 172.21.15.64:54568) 2024-09-21T04:01:24.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.716+0000 7f63a7fff640 1 -- 172.21.15.64:0/1439251983 learned_addr learned my addr 172.21.15.64:0/1439251983 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:01:24.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.716+0000 7f63ac8be640 1 --2- 172.21.15.64:0/1439251983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63a8109df0 0x7f63a819fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:24.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.716+0000 7f63a77fe640 1 --2- 172.21.15.64:0/1439251983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63a81094a0 0x7f63a819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:24.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.716+0000 7f63a7fff640 1 -- 172.21.15.64:0/1439251983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63a81094a0 msgr2=0x7f63a819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:24.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.716+0000 7f63a7fff640 1 --2- 172.21.15.64:0/1439251983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63a81094a0 0x7f63a819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:24.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.716+0000 7f63a7fff640 1 -- 172.21.15.64:0/1439251983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63a8109df0 msgr2=0x7f63a819fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:24.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.716+0000 7f63a7fff640 1 --2- 172.21.15.64:0/1439251983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63a8109df0 0x7f63a819fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:24.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.716+0000 7f63a7fff640 1 -- 172.21.15.64:0/1439251983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a81a04f0 con 0x7f63a8103b60 2024-09-21T04:01:24.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.716+0000 7f63a7fff640 1 --2- 172.21.15.64:0/1439251983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63a8103b60 0x7f63a819a580 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6398005940 tx=0x7f6398004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:24.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.716+0000 7f63a57fa640 1 -- 172.21.15.64:0/1439251983 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639803f030 con 0x7f63a8103b60 2024-09-21T04:01:24.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.716+0000 7f63ae321640 1 -- 172.21.15.64:0/1439251983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a81a0780 con 0x7f63a8103b60 2024-09-21T04:01:24.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.716+0000 7f63a57fa640 1 -- 172.21.15.64:0/1439251983 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6398003aa0 con 0x7f63a8103b60 2024-09-21T04:01:24.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.716+0000 7f63a57fa640 1 -- 172.21.15.64:0/1439251983 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6398035070 con 0x7f63a8103b60 2024-09-21T04:01:24.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.716+0000 7f63ae321640 1 -- 172.21.15.64:0/1439251983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a81a6f40 con 0x7f63a8103b60 2024-09-21T04:01:24.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.717+0000 7f63ae321640 1 -- 172.21.15.64:0/1439251983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63a8104de0 con 0x7f63a8103b60 2024-09-21T04:01:24.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.719+0000 7f63a57fa640 1 -- 172.21.15.64:0/1439251983 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f6398042030 con 0x7f63a8103b60 2024-09-21T04:01:24.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.720+0000 7f63a57fa640 1 --2- 172.21.15.64:0/1439251983 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f63900b6560 0x7f63900b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:24.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.720+0000 7f63a77fe640 1 --2- 172.21.15.64:0/1439251983 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f63900b6560 0x7f63900b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:24.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.720+0000 7f63a57fa640 1 -- 172.21.15.64:0/1439251983 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7f63981079a0 con 0x7f63a8103b60 2024-09-21T04:01:24.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.720+0000 7f63a77fe640 1 --2- 172.21.15.64:0/1439251983 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f63900b6560 0x7f63900b8a70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f63a8109200 tx=0x7f639c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:24.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.723+0000 7f63a57fa640 1 -- 172.21.15.64:0/1439251983 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f63980ce600 con 0x7f63a8103b60 2024-09-21T04:01:24.756 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 04:01:24 smithi119 podman[63319]: 2024-09-21 04:01:24.616654555 +0000 UTC m=+0.477597984 container init a9143ff40f362b21fdfaf30252f9915d68ba5dd42c767a96969a98cce21338f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-k, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:01:24.757 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 04:01:24 smithi119 podman[63319]: 2024-09-21 04:01:24.619582078 +0000 UTC m=+0.480525526 container start a9143ff40f362b21fdfaf30252f9915d68ba5dd42c767a96969a98cce21338f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-k, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-21T04:01:24.757 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 04:01:24 smithi119 ceph-mds[63338]: -- 172.21.15.119:0/54895387 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ce93b11860 con 0x55ce97cb9a80 2024-09-21T04:01:24.757 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 04:01:24 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-k[63334]: starting mds.k at 2024-09-21T04:01:24.757 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 04:01:24 smithi119 bash[63319]: a9143ff40f362b21fdfaf30252f9915d68ba5dd42c767a96969a98cce21338f9 2024-09-21T04:01:24.757 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 04:01:24 smithi119 systemd[1]: Started Ceph mds.k for e7c927ba-77cc-11ef-baf5-efdc52797490. 2024-09-21T04:01:24.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.957+0000 7f63ae321640 1 -- 172.21.15.64:0/1439251983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63a810bbc0 con 0x7f63a8103b60 2024-09-21T04:01:24.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.958+0000 7f63a57fa640 1 -- 172.21.15.64:0/1439251983 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f63980d29e0 con 0x7f63a8103b60 2024-09-21T04:01:24.961 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:01:24.961 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":2,"btime":"2024-09-21T04:01:22:058278+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T04:01:24.961 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 2 2024-09-21T04:01:24.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.962+0000 7f63ae321640 1 -- 172.21.15.64:0/1439251983 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f63900b6560 msgr2=0x7f63900b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:24.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.962+0000 7f63ae321640 1 --2- 172.21.15.64:0/1439251983 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f63900b6560 0x7f63900b8a70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f63a8109200 tx=0x7f639c005f90 comp rx=0 tx=0).stop 2024-09-21T04:01:24.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.962+0000 7f63ae321640 1 -- 172.21.15.64:0/1439251983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63a8103b60 msgr2=0x7f63a819a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:24.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.962+0000 7f63ae321640 1 --2- 172.21.15.64:0/1439251983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63a8103b60 0x7f63a819a580 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6398005940 tx=0x7f6398004270 comp rx=0 tx=0).stop 2024-09-21T04:01:24.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.962+0000 7f63ae321640 1 -- 172.21.15.64:0/1439251983 shutdown_connections 2024-09-21T04:01:24.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.962+0000 7f63ae321640 1 --2- 172.21.15.64:0/1439251983 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f63900b6560 0x7f63900b8a70 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:24.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.962+0000 7f63ae321640 1 --2- 172.21.15.64:0/1439251983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63a8109df0 0x7f63a819fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:24.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.962+0000 7f63ae321640 1 --2- 172.21.15.64:0/1439251983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63a81094a0 0x7f63a819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:24.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.962+0000 7f63ae321640 1 --2- 172.21.15.64:0/1439251983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f63a8103b60 0x7f63a819a580 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:24.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.962+0000 7f63ae321640 1 -- 172.21.15.64:0/1439251983 >> 172.21.15.64:0/1439251983 conn(0x7f63a807b850 msgr2=0x7f63a8107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:24.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.962+0000 7f63ae321640 1 -- 172.21.15.64:0/1439251983 shutdown_connections 2024-09-21T04:01:24.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:24.962+0000 7f63ae321640 1 -- 172.21.15.64:0/1439251983 wait complete. 2024-09-21T04:01:26.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:25 smithi064 ceph-mon[31180]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:26.013 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:25 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:26.014 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:25 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:26.014 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:25 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:26.014 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:25 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:26.014 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:25 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:26.014 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:25 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:26.014 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:25 smithi064 ceph-mon[31180]: Deploying daemon mds.j on smithi064 2024-09-21T04:01:26.014 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:25 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1439251983' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:26.014 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:25 smithi064 ceph-mon[31180]: mds.? [v2:172.21.15.119:6832/1724707326,v1:172.21.15.119:6833/1724707326] up:boot 2024-09-21T04:01:26.014 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:25 smithi064 ceph-mon[31180]: fsmap 2 up:standby 2024-09-21T04:01:26.014 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:25 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T04:01:26.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:25 smithi131 ceph-mon[35193]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:26.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:25 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:26.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:25 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:26.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:25 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:26.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:25 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:26.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:25 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:26.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:25 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:26.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:25 smithi131 ceph-mon[35193]: Deploying daemon mds.j on smithi064 2024-09-21T04:01:26.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:25 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1439251983' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:26.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:25 smithi131 ceph-mon[35193]: mds.? [v2:172.21.15.119:6832/1724707326,v1:172.21.15.119:6833/1724707326] up:boot 2024-09-21T04:01:26.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:25 smithi131 ceph-mon[35193]: fsmap 2 up:standby 2024-09-21T04:01:26.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:25 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T04:01:26.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:25 smithi119 ceph-mon[34811]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:26.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:25 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:26.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:25 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:26.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:25 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:26.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:25 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:26.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:25 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:26.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:25 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:26.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:25 smithi119 ceph-mon[34811]: Deploying daemon mds.j on smithi064 2024-09-21T04:01:26.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:25 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1439251983' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:26.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:25 smithi119 ceph-mon[34811]: mds.? [v2:172.21.15.119:6832/1724707326,v1:172.21.15.119:6833/1724707326] up:boot 2024-09-21T04:01:26.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:25 smithi119 ceph-mon[34811]: fsmap 2 up:standby 2024-09-21T04:01:26.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:25 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T04:01:27.703 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:01:27.728 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 21 04:01:27 smithi064 systemd[1]: Starting Ceph mds.j for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:01:28.010 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 21 04:01:27 smithi064 podman[69451]: 2024-09-21 04:01:27.689841324 +0000 UTC m=+0.016252169 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:01:28.010 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 21 04:01:27 smithi064 podman[69451]: 2024-09-21 04:01:27.81394916 +0000 UTC m=+0.140360005 container create da8ece3c8179b88450bbb2cbc8fc851714fd2f98f080306f9d78bcb66f8a3d5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T04:01:28.010 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 21 04:01:28 smithi064 podman[69451]: 2024-09-21 04:01:28.00842449 +0000 UTC m=+0.334835335 container init da8ece3c8179b88450bbb2cbc8fc851714fd2f98f080306f9d78bcb66f8a3d5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-j, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-21T04:01:28.208 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:01:28.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:28 smithi064 ceph-mon[31180]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:28.261 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 21 04:01:28 smithi064 podman[69451]: 2024-09-21 04:01:28.011590528 +0000 UTC m=+0.338001380 container start da8ece3c8179b88450bbb2cbc8fc851714fd2f98f080306f9d78bcb66f8a3d5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-j, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-09-21T04:01:28.261 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 21 04:01:28 smithi064 ceph-mds[69515]: -- 172.21.15.64:0/850607969 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cac31b9860 con 0x55cac7361a80 2024-09-21T04:01:28.261 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 21 04:01:28 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-j[69491]: starting mds.j at 2024-09-21T04:01:28.262 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 21 04:01:28 smithi064 bash[69451]: da8ece3c8179b88450bbb2cbc8fc851714fd2f98f080306f9d78bcb66f8a3d5e 2024-09-21T04:01:28.262 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 21 04:01:28 smithi064 systemd[1]: Started Ceph mds.j for e7c927ba-77cc-11ef-baf5-efdc52797490. 2024-09-21T04:01:28.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:28 smithi131 ceph-mon[35193]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:28.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:28 smithi119 ceph-mon[34811]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:28.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.741+0000 7f8dd8e03640 1 -- 172.21.15.64:0/1624708369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8dd4105280 msgr2=0x7f8dd410d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:28.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.741+0000 7f8dd8e03640 1 --2- 172.21.15.64:0/1624708369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8dd4105280 0x7f8dd410d8c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc009900 tx=0x7f8dbc02a410 comp rx=0 tx=0).stop 2024-09-21T04:01:28.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.741+0000 7f8dd8e03640 1 -- 172.21.15.64:0/1624708369 shutdown_connections 2024-09-21T04:01:28.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.741+0000 7f8dd8e03640 1 --2- 172.21.15.64:0/1624708369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8dd410de00 0x7f8dd4110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:28.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.741+0000 7f8dd8e03640 1 --2- 172.21.15.64:0/1624708369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8dd4105280 0x7f8dd410d8c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:28.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.741+0000 7f8dd8e03640 1 --2- 172.21.15.64:0/1624708369 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8dd4104850 0x7f8dd4104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:28.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.741+0000 7f8dd8e03640 1 -- 172.21.15.64:0/1624708369 >> 172.21.15.64:0/1624708369 conn(0x7f8dd4079730 msgr2=0x7f8dd4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:28.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.741+0000 7f8dd8e03640 1 -- 172.21.15.64:0/1624708369 shutdown_connections 2024-09-21T04:01:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.741+0000 7f8dd8e03640 1 -- 172.21.15.64:0/1624708369 wait complete. 2024-09-21T04:01:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.742+0000 7f8dd8e03640 1 Processor -- start 2024-09-21T04:01:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.742+0000 7f8dd8e03640 1 -- start start 2024-09-21T04:01:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.742+0000 7f8dd8e03640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8dd4104850 0x7f8dd419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.742+0000 7f8dd8e03640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8dd4105280 0x7f8dd419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.742+0000 7f8dd8e03640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8dd410de00 0x7f8dd419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.742+0000 7f8dd8e03640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd4113a80 con 0x7f8dd4104850 2024-09-21T04:01:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.742+0000 7f8dd8e03640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd4113900 con 0x7f8dd410de00 2024-09-21T04:01:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.742+0000 7f8dd8e03640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd4113c00 con 0x7f8dd4105280 2024-09-21T04:01:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.742+0000 7f8dd259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8dd4104850 0x7f8dd419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.743+0000 7f8dd259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8dd4104850 0x7f8dd419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54610/0 (socket says 172.21.15.64:54610) 2024-09-21T04:01:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.743+0000 7f8dd259c640 1 -- 172.21.15.64:0/2930018043 learned_addr learned my addr 172.21.15.64:0/2930018043 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:01:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.743+0000 7f8dd1d9b640 1 --2- 172.21.15.64:0/2930018043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8dd4105280 0x7f8dd419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:28.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.743+0000 7f8dd2d9d640 1 --2- 172.21.15.64:0/2930018043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8dd410de00 0x7f8dd419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.743+0000 7f8dd259c640 1 -- 172.21.15.64:0/2930018043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8dd4105280 msgr2=0x7f8dd419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.743+0000 7f8dd259c640 1 --2- 172.21.15.64:0/2930018043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8dd4105280 0x7f8dd419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.743+0000 7f8dd259c640 1 -- 172.21.15.64:0/2930018043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8dd410de00 msgr2=0x7f8dd419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.743+0000 7f8dd259c640 1 --2- 172.21.15.64:0/2930018043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8dd410de00 0x7f8dd419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.743+0000 7f8dd259c640 1 -- 172.21.15.64:0/2930018043 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd41a0400 con 0x7f8dd4104850 2024-09-21T04:01:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.743+0000 7f8dd2d9d640 1 --2- 172.21.15.64:0/2930018043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8dd410de00 0x7f8dd419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:01:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.743+0000 7f8dd259c640 1 --2- 172.21.15.64:0/2930018043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8dd4104850 0x7f8dd419a490 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc00d770 tx=0x7f8dcc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.743+0000 7f8dcb7fe640 1 -- 172.21.15.64:0/2930018043 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc017070 con 0x7f8dd4104850 2024-09-21T04:01:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.743+0000 7f8dcb7fe640 1 -- 172.21.15.64:0/2930018043 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dcc009c30 con 0x7f8dd4104850 2024-09-21T04:01:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.743+0000 7f8dd8e03640 1 -- 172.21.15.64:0/2930018043 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dd41a06f0 con 0x7f8dd4104850 2024-09-21T04:01:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.743+0000 7f8dcb7fe640 1 -- 172.21.15.64:0/2930018043 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc003cc0 con 0x7f8dd4104850 2024-09-21T04:01:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.743+0000 7f8dd8e03640 1 -- 172.21.15.64:0/2930018043 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dd41a6e50 con 0x7f8dd4104850 2024-09-21T04:01:28.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.744+0000 7f8dd8e03640 1 -- 172.21.15.64:0/2930018043 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dd406c180 con 0x7f8dd4104850 2024-09-21T04:01:28.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.746+0000 7f8dcb7fe640 1 -- 172.21.15.64:0/2930018043 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f8dcc003e60 con 0x7f8dd4104850 2024-09-21T04:01:28.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.747+0000 7f8dcb7fe640 1 --2- 172.21.15.64:0/2930018043 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8dc00b6560 0x7f8dc00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:28.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.747+0000 7f8dd1d9b640 1 --2- 172.21.15.64:0/2930018043 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8dc00b6560 0x7f8dc00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:28.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.747+0000 7f8dcb7fe640 1 -- 172.21.15.64:0/2930018043 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7f8dcc0e6df0 con 0x7f8dd4104850 2024-09-21T04:01:28.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.747+0000 7f8dd1d9b640 1 --2- 172.21.15.64:0/2930018043 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8dc00b6560 0x7f8dc00b8a70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc009230 tx=0x7f8dbc031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:28.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.750+0000 7f8dcb7fe640 1 -- 172.21.15.64:0/2930018043 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8dcc009de0 con 0x7f8dd4104850 2024-09-21T04:01:28.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.989+0000 7f8dd8e03640 1 -- 172.21.15.64:0/2930018043 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8dd4104c60 con 0x7f8dd4104850 2024-09-21T04:01:28.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.990+0000 7f8dcb7fe640 1 -- 172.21.15.64:0/2930018043 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2266 (secure 0 0 0) 0x7f8dcc0b6020 con 0x7f8dd4104850 2024-09-21T04:01:28.992 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:01:28.992 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":3,"btime":"2024-09-21T04:01:25:197983+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T04:01:28.992 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 3 2024-09-21T04:01:28.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.992+0000 7f8dd8e03640 1 -- 172.21.15.64:0/2930018043 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8dc00b6560 msgr2=0x7f8dc00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:28.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.992+0000 7f8dd8e03640 1 --2- 172.21.15.64:0/2930018043 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8dc00b6560 0x7f8dc00b8a70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc009230 tx=0x7f8dbc031400 comp rx=0 tx=0).stop 2024-09-21T04:01:28.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.992+0000 7f8dd8e03640 1 -- 172.21.15.64:0/2930018043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8dd4104850 msgr2=0x7f8dd419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:28.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.992+0000 7f8dd8e03640 1 --2- 172.21.15.64:0/2930018043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8dd4104850 0x7f8dd419a490 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc00d770 tx=0x7f8dcc00dc30 comp rx=0 tx=0).stop 2024-09-21T04:01:28.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.992+0000 7f8dd8e03640 1 -- 172.21.15.64:0/2930018043 shutdown_connections 2024-09-21T04:01:28.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.992+0000 7f8dd8e03640 1 --2- 172.21.15.64:0/2930018043 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8dc00b6560 0x7f8dc00b8a70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:28.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.992+0000 7f8dd8e03640 1 --2- 172.21.15.64:0/2930018043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8dd410de00 0x7f8dd419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:28.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.992+0000 7f8dd8e03640 1 --2- 172.21.15.64:0/2930018043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8dd4105280 0x7f8dd419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:28.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.992+0000 7f8dd8e03640 1 --2- 172.21.15.64:0/2930018043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8dd4104850 0x7f8dd419a490 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:28.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.992+0000 7f8dd8e03640 1 -- 172.21.15.64:0/2930018043 >> 172.21.15.64:0/2930018043 conn(0x7f8dd4079730 msgr2=0x7f8dd4100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:28.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.992+0000 7f8dd8e03640 1 -- 172.21.15.64:0/2930018043 shutdown_connections 2024-09-21T04:01:28.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:28.992+0000 7f8dd8e03640 1 -- 172.21.15.64:0/2930018043 wait complete. 2024-09-21T04:01:29.503 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:29.503 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:29.503 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:29.504 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:29.504 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:29.504 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:29.504 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:29 smithi064 ceph-mon[31180]: Deploying daemon mds.d on smithi064 2024-09-21T04:01:29.504 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:29 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2930018043' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:29.504 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:29 smithi064 ceph-mon[31180]: mds.? [v2:172.21.15.64:6834/2075966415,v1:172.21.15.64:6835/2075966415] up:boot 2024-09-21T04:01:29.504 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:29 smithi064 ceph-mon[31180]: fsmap 3 up:standby 2024-09-21T04:01:29.504 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T04:01:29.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:29.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:29.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:29.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:29.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:29.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:29.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:29 smithi119 ceph-mon[34811]: Deploying daemon mds.d on smithi064 2024-09-21T04:01:29.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:29 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2930018043' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:29.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:29 smithi119 ceph-mon[34811]: mds.? [v2:172.21.15.64:6834/2075966415,v1:172.21.15.64:6835/2075966415] up:boot 2024-09-21T04:01:29.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:29 smithi119 ceph-mon[34811]: fsmap 3 up:standby 2024-09-21T04:01:29.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T04:01:29.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:29.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:29.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:29.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:29.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:29.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:29.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:29 smithi131 ceph-mon[35193]: Deploying daemon mds.d on smithi064 2024-09-21T04:01:29.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:29 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2930018043' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:29.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:29 smithi131 ceph-mon[35193]: mds.? [v2:172.21.15.64:6834/2075966415,v1:172.21.15.64:6835/2075966415] up:boot 2024-09-21T04:01:29.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:29 smithi131 ceph-mon[35193]: fsmap 3 up:standby 2024-09-21T04:01:29.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T04:01:30.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:30 smithi119 ceph-mon[34811]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:30.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:30 smithi064 ceph-mon[31180]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:30.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:30 smithi131 ceph-mon[35193]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:31.261 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 21 04:01:31 smithi064 systemd[1]: Starting Ceph mds.d for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:01:31.578 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 21 04:01:31 smithi064 podman[69849]: 2024-09-21 04:01:31.226106783 +0000 UTC m=+0.016494083 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:01:31.579 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 21 04:01:31 smithi064 podman[69849]: 2024-09-21 04:01:31.341179062 +0000 UTC m=+0.131566404 container create 8e4f2b7a7e88790a1ca6a3e85554c4cab77aa2af19b3c49e16df3c14e978d1f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-d, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T04:01:31.579 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 21 04:01:31 smithi064 podman[69849]: 2024-09-21 04:01:31.577082941 +0000 UTC m=+0.367470248 container init 8e4f2b7a7e88790a1ca6a3e85554c4cab77aa2af19b3c49e16df3c14e978d1f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:01:31.753 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:01:32.011 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 21 04:01:31 smithi064 podman[69849]: 2024-09-21 04:01:31.580226893 +0000 UTC m=+0.370614198 container start 8e4f2b7a7e88790a1ca6a3e85554c4cab77aa2af19b3c49e16df3c14e978d1f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:01:32.011 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 21 04:01:31 smithi064 ceph-mds[69868]: -- 172.21.15.64:0/1191206009 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562bb8dcf860 con 0x562bbcf77a80 2024-09-21T04:01:32.011 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 21 04:01:31 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-d[69864]: starting mds.d at 2024-09-21T04:01:32.012 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 21 04:01:31 smithi064 bash[69849]: 8e4f2b7a7e88790a1ca6a3e85554c4cab77aa2af19b3c49e16df3c14e978d1f6 2024-09-21T04:01:32.012 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 21 04:01:31 smithi064 systemd[1]: Started Ceph mds.d for e7c927ba-77cc-11ef-baf5-efdc52797490. 2024-09-21T04:01:32.014 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:01:32.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:32 smithi119 ceph-mon[34811]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:32.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:32 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:32.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:32 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:32.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:32 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:32.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:32 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:32.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:32 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:32.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:32 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:32.399 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:32 smithi064 ceph-mon[31180]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:32.399 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:32 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:32.399 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:32 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:32.399 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:32 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:32.399 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:32 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:32.399 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:32 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:32.400 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:32 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:32.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.491+0000 7fb34ab4b640 1 -- 172.21.15.64:0/2318435609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb344108530 msgr2=0x7fb34410ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:32.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.491+0000 7fb34ab4b640 1 --2- 172.21.15.64:0/2318435609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb344108530 0x7fb34410ed60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00adb0 tx=0x7fb33c02b7a0 comp rx=0 tx=0).stop 2024-09-21T04:01:32.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.491+0000 7fb34ab4b640 1 -- 172.21.15.64:0/2318435609 shutdown_connections 2024-09-21T04:01:32.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.491+0000 7fb34ab4b640 1 --2- 172.21.15.64:0/2318435609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb344108530 0x7fb34410ed60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:32.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.491+0000 7fb34ab4b640 1 --2- 172.21.15.64:0/2318435609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb344107be0 0x7fb344107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:32.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.491+0000 7fb34ab4b640 1 --2- 172.21.15.64:0/2318435609 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3441022a0 0x7fb3441026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:32.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.491+0000 7fb34ab4b640 1 -- 172.21.15.64:0/2318435609 >> 172.21.15.64:0/2318435609 conn(0x7fb344079ec0 msgr2=0x7fb34407a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:32.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.491+0000 7fb34ab4b640 1 -- 172.21.15.64:0/2318435609 shutdown_connections 2024-09-21T04:01:32.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.491+0000 7fb34ab4b640 1 -- 172.21.15.64:0/2318435609 wait complete. 2024-09-21T04:01:32.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.492+0000 7fb34ab4b640 1 Processor -- start 2024-09-21T04:01:32.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.492+0000 7fb34ab4b640 1 -- start start 2024-09-21T04:01:32.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.492+0000 7fb34ab4b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3441022a0 0x7fb344198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:32.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.492+0000 7fb34ab4b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb344107be0 0x7fb3441988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:32.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.492+0000 7fb34ab4b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb344108530 0x7fb34419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:32.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.492+0000 7fb34ab4b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb344111970 con 0x7fb344107be0 2024-09-21T04:01:32.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.492+0000 7fb34ab4b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb3441117f0 con 0x7fb3441022a0 2024-09-21T04:01:32.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.492+0000 7fb34ab4b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb344111af0 con 0x7fb344108530 2024-09-21T04:01:32.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.492+0000 7fb343fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb344107be0 0x7fb3441988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:32.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.492+0000 7fb343fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb344107be0 0x7fb3441988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41014/0 (socket says 172.21.15.64:41014) 2024-09-21T04:01:32.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.492+0000 7fb343fff640 1 -- 172.21.15.64:0/1607195167 learned_addr learned my addr 172.21.15.64:0/1607195167 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:01:32.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.492+0000 7fb3488e7640 1 --2- 172.21.15.64:0/1607195167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3441022a0 0x7fb344198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:32.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.492+0000 7fb3490e8640 1 --2- 172.21.15.64:0/1607195167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb344108530 0x7fb34419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:32.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.492+0000 7fb343fff640 1 -- 172.21.15.64:0/1607195167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb344108530 msgr2=0x7fb34419db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:32.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.492+0000 7fb343fff640 1 --2- 172.21.15.64:0/1607195167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb344108530 0x7fb34419db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:32.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.492+0000 7fb343fff640 1 -- 172.21.15.64:0/1607195167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3441022a0 msgr2=0x7fb344198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:32.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.492+0000 7fb343fff640 1 --2- 172.21.15.64:0/1607195167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3441022a0 0x7fb344198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:32.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.492+0000 7fb343fff640 1 -- 172.21.15.64:0/1607195167 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34419e1d0 con 0x7fb344107be0 2024-09-21T04:01:32.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.493+0000 7fb343fff640 1 --2- 172.21.15.64:0/1607195167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb344107be0 0x7fb3441988c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb33000d740 tx=0x7fb33000db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:32.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.493+0000 7fb341ffb640 1 -- 172.21.15.64:0/1607195167 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb330017070 con 0x7fb344107be0 2024-09-21T04:01:32.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.493+0000 7fb341ffb640 1 -- 172.21.15.64:0/1607195167 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb330010460 con 0x7fb344107be0 2024-09-21T04:01:32.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.493+0000 7fb34ab4b640 1 -- 172.21.15.64:0/1607195167 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb34419e4c0 con 0x7fb344107be0 2024-09-21T04:01:32.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.493+0000 7fb341ffb640 1 -- 172.21.15.64:0/1607195167 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33000f600 con 0x7fb344107be0 2024-09-21T04:01:32.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.493+0000 7fb34ab4b640 1 -- 172.21.15.64:0/1607195167 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3441a6ef0 con 0x7fb344107be0 2024-09-21T04:01:32.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.494+0000 7fb34ab4b640 1 -- 172.21.15.64:0/1607195167 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3441026b0 con 0x7fb344107be0 2024-09-21T04:01:32.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.496+0000 7fb341ffb640 1 -- 172.21.15.64:0/1607195167 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fb33001a070 con 0x7fb344107be0 2024-09-21T04:01:32.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.496+0000 7fb341ffb640 1 --2- 172.21.15.64:0/1607195167 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb3340b6610 0x7fb3340b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:32.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.497+0000 7fb3488e7640 1 --2- 172.21.15.64:0/1607195167 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb3340b6610 0x7fb3340b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:32.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.497+0000 7fb341ffb640 1 -- 172.21.15.64:0/1607195167 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7fb3300e6880 con 0x7fb344107be0 2024-09-21T04:01:32.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.497+0000 7fb3488e7640 1 --2- 172.21.15.64:0/1607195167 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb3340b6610 0x7fb3340b8b20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb33800abe0 tx=0x7fb33800a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:32.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.500+0000 7fb341ffb640 1 -- 172.21.15.64:0/1607195167 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb3300ad560 con 0x7fb344107be0 2024-09-21T04:01:32.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:32 smithi131 ceph-mon[35193]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:32.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:32 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:32.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:32 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:32.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:32 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:32.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:32 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:32.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:32 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:32.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:32 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:32.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.726+0000 7fb34ab4b640 1 -- 172.21.15.64:0/1607195167 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb344199450 con 0x7fb344107be0 2024-09-21T04:01:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.727+0000 7fb341ffb640 1 -- 172.21.15.64:0/1607195167 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3916 (secure 0 0 0) 0x7fb344199450 con 0x7fb344107be0 2024-09-21T04:01:32.729 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:01:32.729 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":5,"btime":"2024-09-21T04:01:32:197242+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T04:01:32.729 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 5 2024-09-21T04:01:32.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.729+0000 7fb34ab4b640 1 -- 172.21.15.64:0/1607195167 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb3340b6610 msgr2=0x7fb3340b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:32.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.729+0000 7fb34ab4b640 1 --2- 172.21.15.64:0/1607195167 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb3340b6610 0x7fb3340b8b20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb33800abe0 tx=0x7fb33800a2d0 comp rx=0 tx=0).stop 2024-09-21T04:01:32.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.729+0000 7fb34ab4b640 1 -- 172.21.15.64:0/1607195167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb344107be0 msgr2=0x7fb3441988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:32.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.729+0000 7fb34ab4b640 1 --2- 172.21.15.64:0/1607195167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb344107be0 0x7fb3441988c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb33000d740 tx=0x7fb33000db40 comp rx=0 tx=0).stop 2024-09-21T04:01:32.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.729+0000 7fb34ab4b640 1 -- 172.21.15.64:0/1607195167 shutdown_connections 2024-09-21T04:01:32.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.729+0000 7fb34ab4b640 1 --2- 172.21.15.64:0/1607195167 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb3340b6610 0x7fb3340b8b20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:32.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.729+0000 7fb34ab4b640 1 --2- 172.21.15.64:0/1607195167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb344108530 0x7fb34419db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:32.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.729+0000 7fb34ab4b640 1 --2- 172.21.15.64:0/1607195167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb344107be0 0x7fb3441988c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:32.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.729+0000 7fb34ab4b640 1 --2- 172.21.15.64:0/1607195167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3441022a0 0x7fb344198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:32.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.729+0000 7fb34ab4b640 1 -- 172.21.15.64:0/1607195167 >> 172.21.15.64:0/1607195167 conn(0x7fb344079ec0 msgr2=0x7fb344105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:32.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.729+0000 7fb34ab4b640 1 -- 172.21.15.64:0/1607195167 shutdown_connections 2024-09-21T04:01:32.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:32.729+0000 7fb34ab4b640 1 -- 172.21.15.64:0/1607195167 wait complete. 2024-09-21T04:01:33.455 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:33 smithi064 ceph-mon[31180]: Deploying daemon mds.e on smithi119 2024-09-21T04:01:33.455 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:33 smithi064 ceph-mon[31180]: mds.? [v2:172.21.15.64:6836/3072311987,v1:172.21.15.64:6837/3072311987] up:boot 2024-09-21T04:01:33.455 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:33 smithi064 ceph-mon[31180]: fsmap 4 up:standby 2024-09-21T04:01:33.455 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:33 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T04:01:33.455 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:33 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1607195167' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:33.476 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:33 smithi119 ceph-mon[34811]: Deploying daemon mds.e on smithi119 2024-09-21T04:01:33.476 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:33 smithi119 ceph-mon[34811]: mds.? [v2:172.21.15.64:6836/3072311987,v1:172.21.15.64:6837/3072311987] up:boot 2024-09-21T04:01:33.476 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:33 smithi119 ceph-mon[34811]: fsmap 4 up:standby 2024-09-21T04:01:33.476 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:33 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T04:01:33.476 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:33 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1607195167' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:33.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:33 smithi131 ceph-mon[35193]: Deploying daemon mds.e on smithi119 2024-09-21T04:01:33.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:33 smithi131 ceph-mon[35193]: mds.? [v2:172.21.15.64:6836/3072311987,v1:172.21.15.64:6837/3072311987] up:boot 2024-09-21T04:01:33.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:33 smithi131 ceph-mon[35193]: fsmap 4 up:standby 2024-09-21T04:01:33.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:33 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T04:01:33.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:33 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1607195167' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:33.756 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 04:01:33 smithi119 systemd[1]: Starting Ceph mds.e for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:01:34.124 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 04:01:33 smithi119 podman[63602]: 2024-09-21 04:01:33.762073915 +0000 UTC m=+0.015671625 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:01:34.124 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 04:01:33 smithi119 podman[63602]: 2024-09-21 04:01:33.878488511 +0000 UTC m=+0.132086261 container create fc78e5b8b12e3568ba24961189f2ba6630f5270e81dd96957971bcff25ae95b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-e, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-21T04:01:34.124 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 04:01:34 smithi119 podman[63602]: 2024-09-21 04:01:34.123072682 +0000 UTC m=+0.376670385 container init fc78e5b8b12e3568ba24961189f2ba6630f5270e81dd96957971bcff25ae95b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-e, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-21T04:01:34.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:34 smithi119 ceph-mon[34811]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:34.755 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 04:01:34 smithi119 podman[63602]: 2024-09-21 04:01:34.125862957 +0000 UTC m=+0.379460654 container start fc78e5b8b12e3568ba24961189f2ba6630f5270e81dd96957971bcff25ae95b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-e, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-21T04:01:34.755 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 04:01:34 smithi119 ceph-mds[63621]: -- 172.21.15.119:0/981808549 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fb2b983860 con 0x55fb2fb2b180 2024-09-21T04:01:34.755 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 04:01:34 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-e[63617]: starting mds.e at 2024-09-21T04:01:34.755 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 04:01:34 smithi119 bash[63602]: fc78e5b8b12e3568ba24961189f2ba6630f5270e81dd96957971bcff25ae95b4 2024-09-21T04:01:34.755 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 04:01:34 smithi119 systemd[1]: Started Ceph mds.e for e7c927ba-77cc-11ef-baf5-efdc52797490. 2024-09-21T04:01:34.756 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:34 smithi131 ceph-mon[35193]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:34.756 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:34 smithi064 ceph-mon[31180]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:35.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:35 smithi131 ceph-mon[35193]: mds.? [v2:172.21.15.119:6834/3675946142,v1:172.21.15.119:6835/3675946142] up:boot 2024-09-21T04:01:35.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:35 smithi131 ceph-mon[35193]: fsmap 5 up:standby 2024-09-21T04:01:35.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:35 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T04:01:35.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:35 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:35.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:35 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:35.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:35 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:35.305 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:35 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:35.305 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:35 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:35.305 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:35 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:35.305 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:35 smithi131 ceph-mon[35193]: Deploying daemon mds.f on smithi131 2024-09-21T04:01:35.486 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:01:35.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:35 smithi119 ceph-mon[34811]: mds.? [v2:172.21.15.119:6834/3675946142,v1:172.21.15.119:6835/3675946142] up:boot 2024-09-21T04:01:35.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:35 smithi119 ceph-mon[34811]: fsmap 5 up:standby 2024-09-21T04:01:35.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:35 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T04:01:35.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:35 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:35.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:35 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:35.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:35 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:35.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:35 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:35.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:35 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:35.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:35 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:35.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:35 smithi119 ceph-mon[34811]: Deploying daemon mds.f on smithi131 2024-09-21T04:01:35.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:35 smithi064 ceph-mon[31180]: mds.? [v2:172.21.15.119:6834/3675946142,v1:172.21.15.119:6835/3675946142] up:boot 2024-09-21T04:01:35.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:35 smithi064 ceph-mon[31180]: fsmap 5 up:standby 2024-09-21T04:01:35.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:35 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T04:01:35.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:35 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:35.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:35 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:35.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:35 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:35.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:35 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:35.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:35 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:35.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:35 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:35.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:35 smithi064 ceph-mon[31180]: Deploying daemon mds.f on smithi131 2024-09-21T04:01:35.758 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:01:36.285 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 21 04:01:36 smithi131 systemd[1]: Starting Ceph mds.f for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:01:36.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.308+0000 7fda3bf39640 1 -- 172.21.15.64:0/2676959900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda341094c0 msgr2=0x7fda341098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:36.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.308+0000 7fda3bf39640 1 --2- 172.21.15.64:0/2676959900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda341094c0 0x7fda341098d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fda24009900 tx=0x7fda2402a410 comp rx=0 tx=0).stop 2024-09-21T04:01:36.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.309+0000 7fda3bf39640 1 -- 172.21.15.64:0/2676959900 shutdown_connections 2024-09-21T04:01:36.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.309+0000 7fda3bf39640 1 --2- 172.21.15.64:0/2676959900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda34109e10 0x7fda34110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:36.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.309+0000 7fda3bf39640 1 --2- 172.21.15.64:0/2676959900 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda341094c0 0x7fda341098d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:36.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.309+0000 7fda3bf39640 1 --2- 172.21.15.64:0/2676959900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda34103b80 0x7fda34103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:36.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.309+0000 7fda3bf39640 1 -- 172.21.15.64:0/2676959900 >> 172.21.15.64:0/2676959900 conn(0x7fda340ff5d0 msgr2=0x7fda34101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:36.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.309+0000 7fda3bf39640 1 -- 172.21.15.64:0/2676959900 shutdown_connections 2024-09-21T04:01:36.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.309+0000 7fda3bf39640 1 -- 172.21.15.64:0/2676959900 wait complete. 2024-09-21T04:01:36.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.310+0000 7fda3bf39640 1 Processor -- start 2024-09-21T04:01:36.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.310+0000 7fda3bf39640 1 -- start start 2024-09-21T04:01:36.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.310+0000 7fda3bf39640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda34103b80 0x7fda3419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:36.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.310+0000 7fda3bf39640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda341094c0 0x7fda3419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:36.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.310+0000 7fda3bf39640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda34109e10 0x7fda3419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:36.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.310+0000 7fda3bf39640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fda34113ad0 con 0x7fda34103b80 2024-09-21T04:01:36.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.310+0000 7fda3bf39640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fda34113950 con 0x7fda34109e10 2024-09-21T04:01:36.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.310+0000 7fda3bf39640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fda34113c50 con 0x7fda341094c0 2024-09-21T04:01:36.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.310+0000 7fda39cd5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda34103b80 0x7fda3419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:36.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.310+0000 7fda39cd5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda34103b80 0x7fda3419a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41030/0 (socket says 172.21.15.64:41030) 2024-09-21T04:01:36.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.310+0000 7fda39cd5640 1 -- 172.21.15.64:0/877470620 learned_addr learned my addr 172.21.15.64:0/877470620 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:01:36.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.310+0000 7fda394d4640 1 --2- 172.21.15.64:0/877470620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda341094c0 0x7fda3419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:36.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.310+0000 7fda3a4d6640 1 --2- 172.21.15.64:0/877470620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda34109e10 0x7fda3419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:36.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.310+0000 7fda39cd5640 1 -- 172.21.15.64:0/877470620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda341094c0 msgr2=0x7fda3419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:36.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.311+0000 7fda39cd5640 1 --2- 172.21.15.64:0/877470620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda341094c0 0x7fda3419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:36.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.311+0000 7fda39cd5640 1 -- 172.21.15.64:0/877470620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda34109e10 msgr2=0x7fda3419fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:36.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.311+0000 7fda39cd5640 1 --2- 172.21.15.64:0/877470620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda34109e10 0x7fda3419fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:36.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.311+0000 7fda39cd5640 1 -- 172.21.15.64:0/877470620 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda341a0410 con 0x7fda34103b80 2024-09-21T04:01:36.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.311+0000 7fda39cd5640 1 --2- 172.21.15.64:0/877470620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda34103b80 0x7fda3419a4a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fda3000e9f0 tx=0x7fda3000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:36.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.311+0000 7fda2affd640 1 -- 172.21.15.64:0/877470620 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3000cd80 con 0x7fda34103b80 2024-09-21T04:01:36.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.311+0000 7fda2affd640 1 -- 172.21.15.64:0/877470620 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda300054d0 con 0x7fda34103b80 2024-09-21T04:01:36.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.311+0000 7fda3bf39640 1 -- 172.21.15.64:0/877470620 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda341a0700 con 0x7fda34103b80 2024-09-21T04:01:36.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.311+0000 7fda2affd640 1 -- 172.21.15.64:0/877470620 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30010690 con 0x7fda34103b80 2024-09-21T04:01:36.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.311+0000 7fda3bf39640 1 -- 172.21.15.64:0/877470620 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda341a6e60 con 0x7fda34103b80 2024-09-21T04:01:36.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.312+0000 7fda3bf39640 1 -- 172.21.15.64:0/877470620 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda34103f90 con 0x7fda34103b80 2024-09-21T04:01:36.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.314+0000 7fda2affd640 1 -- 172.21.15.64:0/877470620 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fda30004e30 con 0x7fda34103b80 2024-09-21T04:01:36.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.315+0000 7fda2affd640 1 --2- 172.21.15.64:0/877470620 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fda200b6560 0x7fda200b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:36.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.315+0000 7fda394d4640 1 --2- 172.21.15.64:0/877470620 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fda200b6560 0x7fda200b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:36.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.315+0000 7fda2affd640 1 -- 172.21.15.64:0/877470620 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7fda300e7550 con 0x7fda34103b80 2024-09-21T04:01:36.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.316+0000 7fda394d4640 1 --2- 172.21.15.64:0/877470620 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fda200b6560 0x7fda200b8a70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fda24009230 tx=0x7fda24031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:36.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.319+0000 7fda2affd640 1 -- 172.21.15.64:0/877470620 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fda300ae1b0 con 0x7fda34103b80 2024-09-21T04:01:36.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:36 smithi064 ceph-mon[31180]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:36.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.547+0000 7fda3bf39640 1 -- 172.21.15.64:0/877470620 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda3410bc40 con 0x7fda34103b80 2024-09-21T04:01:36.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.548+0000 7fda2affd640 1 -- 172.21.15.64:0/877470620 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4744 (secure 0 0 0) 0x7fda300a9030 con 0x7fda34103b80 2024-09-21T04:01:36.550 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:01:36.550 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":6,"btime":"2024-09-21T04:01:34:211055+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24656,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-21T04:01:36.550 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 6 2024-09-21T04:01:36.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.550+0000 7fda3bf39640 1 -- 172.21.15.64:0/877470620 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fda200b6560 msgr2=0x7fda200b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:36.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.550+0000 7fda3bf39640 1 --2- 172.21.15.64:0/877470620 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fda200b6560 0x7fda200b8a70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fda24009230 tx=0x7fda24031400 comp rx=0 tx=0).stop 2024-09-21T04:01:36.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.550+0000 7fda3bf39640 1 -- 172.21.15.64:0/877470620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda34103b80 msgr2=0x7fda3419a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:36.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.550+0000 7fda3bf39640 1 --2- 172.21.15.64:0/877470620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda34103b80 0x7fda3419a4a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fda3000e9f0 tx=0x7fda3000edf0 comp rx=0 tx=0).stop 2024-09-21T04:01:36.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.550+0000 7fda3bf39640 1 -- 172.21.15.64:0/877470620 shutdown_connections 2024-09-21T04:01:36.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.550+0000 7fda3bf39640 1 --2- 172.21.15.64:0/877470620 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fda200b6560 0x7fda200b8a70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:36.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.550+0000 7fda3bf39640 1 --2- 172.21.15.64:0/877470620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda34109e10 0x7fda3419fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:36.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.550+0000 7fda3bf39640 1 --2- 172.21.15.64:0/877470620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda341094c0 0x7fda3419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:36.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.550+0000 7fda3bf39640 1 --2- 172.21.15.64:0/877470620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda34103b80 0x7fda3419a4a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:36.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.550+0000 7fda3bf39640 1 -- 172.21.15.64:0/877470620 >> 172.21.15.64:0/877470620 conn(0x7fda340ff5d0 msgr2=0x7fda341019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:36.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.550+0000 7fda3bf39640 1 -- 172.21.15.64:0/877470620 shutdown_connections 2024-09-21T04:01:36.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:36.550+0000 7fda3bf39640 1 -- 172.21.15.64:0/877470620 wait complete. 2024-09-21T04:01:36.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:36 smithi131 ceph-mon[35193]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:36.554 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 21 04:01:36 smithi131 podman[64072]: 2024-09-21 04:01:36.264216539 +0000 UTC m=+0.015985716 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:01:36.554 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 21 04:01:36 smithi131 podman[64072]: 2024-09-21 04:01:36.376973073 +0000 UTC m=+0.128742311 container create d2eb13a7058d41882797211d9c1ec725bea3e28bc493b4a8e510d366449d902c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-f, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-21T04:01:36.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:36 smithi119 ceph-mon[34811]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:37.053 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 21 04:01:36 smithi131 podman[64072]: 2024-09-21 04:01:36.621534879 +0000 UTC m=+0.373304051 container init d2eb13a7058d41882797211d9c1ec725bea3e28bc493b4a8e510d366449d902c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-f, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-21T04:01:37.053 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 21 04:01:36 smithi131 podman[64072]: 2024-09-21 04:01:36.624735132 +0000 UTC m=+0.376504313 container start d2eb13a7058d41882797211d9c1ec725bea3e28bc493b4a8e510d366449d902c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-f, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-21T04:01:37.053 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 21 04:01:36 smithi131 ceph-mds[64091]: -- 172.21.15.131:0/486580227 <== mon.1 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556133089860 con 0x556137231180 2024-09-21T04:01:37.053 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 21 04:01:36 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-f[64087]: starting mds.f at 2024-09-21T04:01:37.053 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 21 04:01:36 smithi131 bash[64072]: d2eb13a7058d41882797211d9c1ec725bea3e28bc493b4a8e510d366449d902c 2024-09-21T04:01:37.054 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 21 04:01:36 smithi131 systemd[1]: Started Ceph mds.f for e7c927ba-77cc-11ef-baf5-efdc52797490. 2024-09-21T04:01:37.435 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:37 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/877470620' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:37.435 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:37 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:37.435 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:37 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:37.436 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:37 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:37.436 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:37 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:37.436 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:37 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:37.436 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:37 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:37.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:37 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/877470620' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:37.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:37 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:37.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:37 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:37.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:37 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:37.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:37 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:37.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:37 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:37.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:37 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:37.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:37 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/877470620' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:37.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:37 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:37.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:37 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:37.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:37 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:37.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:37 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:37.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:37 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:37.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:37 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:38.463 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:38 smithi131 ceph-mon[35193]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:38.463 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:38 smithi131 ceph-mon[35193]: Deploying daemon mds.c on smithi131 2024-09-21T04:01:38.463 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:38 smithi131 ceph-mon[35193]: mds.? [v2:172.21.15.131:6834/1422569359,v1:172.21.15.131:6835/1422569359] up:boot 2024-09-21T04:01:38.463 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:38 smithi131 ceph-mon[35193]: fsmap 6 up:standby 2024-09-21T04:01:38.463 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:38 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T04:01:38.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:38 smithi119 ceph-mon[34811]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:38.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:38 smithi119 ceph-mon[34811]: Deploying daemon mds.c on smithi131 2024-09-21T04:01:38.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:38 smithi119 ceph-mon[34811]: mds.? [v2:172.21.15.131:6834/1422569359,v1:172.21.15.131:6835/1422569359] up:boot 2024-09-21T04:01:38.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:38 smithi119 ceph-mon[34811]: fsmap 6 up:standby 2024-09-21T04:01:38.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:38 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T04:01:38.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:38 smithi064 ceph-mon[31180]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:38.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:38 smithi064 ceph-mon[31180]: Deploying daemon mds.c on smithi131 2024-09-21T04:01:38.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:38 smithi064 ceph-mon[31180]: mds.? [v2:172.21.15.131:6834/1422569359,v1:172.21.15.131:6835/1422569359] up:boot 2024-09-21T04:01:38.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:38 smithi064 ceph-mon[31180]: fsmap 6 up:standby 2024-09-21T04:01:38.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:38 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T04:01:38.804 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 21 04:01:38 smithi131 systemd[1]: Starting Ceph mds.c for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:01:39.106 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 21 04:01:38 smithi131 podman[64346]: 2024-09-21 04:01:38.755853415 +0000 UTC m=+0.016056622 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:01:39.106 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 21 04:01:38 smithi131 podman[64346]: 2024-09-21 04:01:38.865706253 +0000 UTC m=+0.125909460 container create 591c2f2438847f370a31a2c67809a54c309c5ddf3e83954b468400288161a770 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-c, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS) 2024-09-21T04:01:39.106 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 21 04:01:39 smithi131 podman[64346]: 2024-09-21 04:01:39.10160711 +0000 UTC m=+0.361810318 container init 591c2f2438847f370a31a2c67809a54c309c5ddf3e83954b468400288161a770 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-21T04:01:39.250 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:01:39.525 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:01:39.553 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 21 04:01:39 smithi131 podman[64346]: 2024-09-21 04:01:39.10453663 +0000 UTC m=+0.364739844 container start 591c2f2438847f370a31a2c67809a54c309c5ddf3e83954b468400288161a770 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-c, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True) 2024-09-21T04:01:39.553 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 21 04:01:39 smithi131 ceph-mds[64365]: -- 172.21.15.131:0/4023794426 <== mon.1 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56102b12f860 con 0x56102f2d7600 2024-09-21T04:01:39.553 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 21 04:01:39 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-c[64361]: starting mds.c at 2024-09-21T04:01:39.554 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 21 04:01:39 smithi131 bash[64346]: 591c2f2438847f370a31a2c67809a54c309c5ddf3e83954b468400288161a770 2024-09-21T04:01:39.554 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 21 04:01:39 smithi131 systemd[1]: Started Ceph mds.c for e7c927ba-77cc-11ef-baf5-efdc52797490. 2024-09-21T04:01:40.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.005+0000 7f62b4ded640 1 -- 172.21.15.64:0/2520698381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b0102a60 msgr2=0x7f62b0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:40.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.005+0000 7f62b4ded640 1 --2- 172.21.15.64:0/2520698381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b0102a60 0x7f62b0102e70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f62a0009900 tx=0x7f62a002a410 comp rx=0 tx=0).stop 2024-09-21T04:01:40.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.005+0000 7f62b4ded640 1 -- 172.21.15.64:0/2520698381 shutdown_connections 2024-09-21T04:01:40.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.005+0000 7f62b4ded640 1 --2- 172.21.15.64:0/2520698381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62b0108cf0 0x7f62b010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:40.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.005+0000 7f62b4ded640 1 --2- 172.21.15.64:0/2520698381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62b01083a0 0x7f62b01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:40.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.005+0000 7f62b4ded640 1 --2- 172.21.15.64:0/2520698381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b0102a60 0x7f62b0102e70 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:40.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.005+0000 7f62b4ded640 1 -- 172.21.15.64:0/2520698381 >> 172.21.15.64:0/2520698381 conn(0x7f62b0079730 msgr2=0x7f62b0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:40.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.005+0000 7f62b4ded640 1 -- 172.21.15.64:0/2520698381 shutdown_connections 2024-09-21T04:01:40.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.005+0000 7f62b4ded640 1 -- 172.21.15.64:0/2520698381 wait complete. 2024-09-21T04:01:40.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.006+0000 7f62b4ded640 1 Processor -- start 2024-09-21T04:01:40.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.006+0000 7f62b4ded640 1 -- start start 2024-09-21T04:01:40.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.006+0000 7f62b4ded640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62b0102a60 0x7f62b019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:40.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.006+0000 7f62b4ded640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b01083a0 0x7f62b019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:40.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.006+0000 7f62b4ded640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62b0108cf0 0x7f62b019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:40.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.006+0000 7f62b4ded640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f62b0112960 con 0x7f62b01083a0 2024-09-21T04:01:40.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.006+0000 7f62b4ded640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f62b01127e0 con 0x7f62b0102a60 2024-09-21T04:01:40.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.006+0000 7f62b4ded640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f62b0112ae0 con 0x7f62b0108cf0 2024-09-21T04:01:40.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.007+0000 7f62add9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b01083a0 0x7f62b019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:40.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.007+0000 7f62add9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b01083a0 0x7f62b019aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41044/0 (socket says 172.21.15.64:41044) 2024-09-21T04:01:40.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.007+0000 7f62add9b640 1 -- 172.21.15.64:0/1714490148 learned_addr learned my addr 172.21.15.64:0/1714490148 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:01:40.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.007+0000 7f62aed9d640 1 --2- 172.21.15.64:0/1714490148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62b0108cf0 0x7f62b019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:40.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.007+0000 7f62ae59c640 1 --2- 172.21.15.64:0/1714490148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62b0102a60 0x7f62b019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:40.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.007+0000 7f62add9b640 1 -- 172.21.15.64:0/1714490148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62b0108cf0 msgr2=0x7f62b019fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:40.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.007+0000 7f62add9b640 1 --2- 172.21.15.64:0/1714490148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62b0108cf0 0x7f62b019fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:40.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.007+0000 7f62add9b640 1 -- 172.21.15.64:0/1714490148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62b0102a60 msgr2=0x7f62b019a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:40.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.007+0000 7f62add9b640 1 --2- 172.21.15.64:0/1714490148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62b0102a60 0x7f62b019a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:40.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.007+0000 7f62add9b640 1 -- 172.21.15.64:0/1714490148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b01a0470 con 0x7f62b01083a0 2024-09-21T04:01:40.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.007+0000 7f62add9b640 1 --2- 172.21.15.64:0/1714490148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b01083a0 0x7f62b019aaa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f62a4005fd0 tx=0x7f62a400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:40.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.007+0000 7f629f7fe640 1 -- 172.21.15.64:0/1714490148 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a4017070 con 0x7f62b01083a0 2024-09-21T04:01:40.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.007+0000 7f62b4ded640 1 -- 172.21.15.64:0/1714490148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b01a0760 con 0x7f62b01083a0 2024-09-21T04:01:40.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.007+0000 7f629f7fe640 1 -- 172.21.15.64:0/1714490148 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62a4009c30 con 0x7f62b01083a0 2024-09-21T04:01:40.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.007+0000 7f62b4ded640 1 -- 172.21.15.64:0/1714490148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b01a6ec0 con 0x7f62b01083a0 2024-09-21T04:01:40.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.008+0000 7f629f7fe640 1 -- 172.21.15.64:0/1714490148 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a4003cd0 con 0x7f62b01083a0 2024-09-21T04:01:40.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.008+0000 7f62b4ded640 1 -- 172.21.15.64:0/1714490148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62b0102e70 con 0x7f62b01083a0 2024-09-21T04:01:40.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.010+0000 7f629f7fe640 1 -- 172.21.15.64:0/1714490148 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f62a4003e70 con 0x7f62b01083a0 2024-09-21T04:01:40.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.011+0000 7f629f7fe640 1 --2- 172.21.15.64:0/1714490148 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f62980b6560 0x7f62980b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:40.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.011+0000 7f62ae59c640 1 --2- 172.21.15.64:0/1714490148 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f62980b6560 0x7f62980b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:40.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.011+0000 7f629f7fe640 1 -- 172.21.15.64:0/1714490148 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7f62a40e75b0 con 0x7f62b01083a0 2024-09-21T04:01:40.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.012+0000 7f62ae59c640 1 --2- 172.21.15.64:0/1714490148 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f62980b6560 0x7f62980b8a70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f62a0031440 tx=0x7f62a0009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:40.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.018+0000 7f629f7fe640 1 -- 172.21.15.64:0/1714490148 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f62a4020030 con 0x7f62b01083a0 2024-09-21T04:01:40.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.245+0000 7f62b4ded640 1 -- 172.21.15.64:0/1714490148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62b019b630 con 0x7f62b01083a0 2024-09-21T04:01:40.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.246+0000 7f629f7fe640 1 -- 172.21.15.64:0/1714490148 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6397 (secure 0 0 0) 0x7f62b019b630 con 0x7f62b01083a0 2024-09-21T04:01:40.248 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:01:40.248 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":8,"btime":"2024-09-21T04:01:39:290992+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24634,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24640,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24656,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-21T04:01:40.248 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 8 2024-09-21T04:01:40.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.248+0000 7f62b4ded640 1 -- 172.21.15.64:0/1714490148 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f62980b6560 msgr2=0x7f62980b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:40.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.248+0000 7f62b4ded640 1 --2- 172.21.15.64:0/1714490148 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f62980b6560 0x7f62980b8a70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f62a0031440 tx=0x7f62a0009560 comp rx=0 tx=0).stop 2024-09-21T04:01:40.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.248+0000 7f62b4ded640 1 -- 172.21.15.64:0/1714490148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b01083a0 msgr2=0x7f62b019aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:40.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.248+0000 7f62b4ded640 1 --2- 172.21.15.64:0/1714490148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b01083a0 0x7f62b019aaa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f62a4005fd0 tx=0x7f62a400dde0 comp rx=0 tx=0).stop 2024-09-21T04:01:40.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.248+0000 7f62b4ded640 1 -- 172.21.15.64:0/1714490148 shutdown_connections 2024-09-21T04:01:40.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.248+0000 7f62b4ded640 1 --2- 172.21.15.64:0/1714490148 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f62980b6560 0x7f62980b8a70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:40.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.248+0000 7f62b4ded640 1 --2- 172.21.15.64:0/1714490148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62b0108cf0 0x7f62b019fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:40.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.248+0000 7f62b4ded640 1 --2- 172.21.15.64:0/1714490148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62b01083a0 0x7f62b019aaa0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:40.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.248+0000 7f62b4ded640 1 --2- 172.21.15.64:0/1714490148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62b0102a60 0x7f62b019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:40.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.248+0000 7f62b4ded640 1 -- 172.21.15.64:0/1714490148 >> 172.21.15.64:0/1714490148 conn(0x7f62b0079730 msgr2=0x7f62b0101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:40.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.248+0000 7f62b4ded640 1 -- 172.21.15.64:0/1714490148 shutdown_connections 2024-09-21T04:01:40.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:40.248+0000 7f62b4ded640 1 -- 172.21.15.64:0/1714490148 wait complete. 2024-09-21T04:01:40.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:40 smithi119 ceph-mon[34811]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:40.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:40 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:40.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:40 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:40.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:40 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:40.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:40 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:40.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:40 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:40.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:40 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:40.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:40 smithi119 ceph-mon[34811]: Deploying daemon mds.h on smithi119 2024-09-21T04:01:40.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:40 smithi119 ceph-mon[34811]: mds.? [v2:172.21.15.131:6836/843209972,v1:172.21.15.131:6837/843209972] up:boot 2024-09-21T04:01:40.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:40 smithi119 ceph-mon[34811]: fsmap 7 up:standby 2024-09-21T04:01:40.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:40 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T04:01:40.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:40 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1714490148' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:40.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:40 smithi064 ceph-mon[31180]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:40.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:40 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:40.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:40 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:40.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:40 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:40.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:40 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:40.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:40 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:40.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:40 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:40.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:40 smithi064 ceph-mon[31180]: Deploying daemon mds.h on smithi119 2024-09-21T04:01:40.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:40 smithi064 ceph-mon[31180]: mds.? [v2:172.21.15.131:6836/843209972,v1:172.21.15.131:6837/843209972] up:boot 2024-09-21T04:01:40.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:40 smithi064 ceph-mon[31180]: fsmap 7 up:standby 2024-09-21T04:01:40.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:40 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T04:01:40.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:40 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1714490148' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:40.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:40 smithi131 ceph-mon[35193]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:40.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:40 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:40.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:40 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:40.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:40 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:40.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:40 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:40.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:40 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:40.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:40 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:40.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:40 smithi131 ceph-mon[35193]: Deploying daemon mds.h on smithi119 2024-09-21T04:01:40.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:40 smithi131 ceph-mon[35193]: mds.? [v2:172.21.15.131:6836/843209972,v1:172.21.15.131:6837/843209972] up:boot 2024-09-21T04:01:40.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:40 smithi131 ceph-mon[35193]: fsmap 7 up:standby 2024-09-21T04:01:40.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:40 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T04:01:40.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:40 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1714490148' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:41.256 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 04:01:41 smithi119 systemd[1]: Starting Ceph mds.h for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:01:41.630 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 04:01:41 smithi119 podman[63868]: 2024-09-21 04:01:41.244862677 +0000 UTC m=+0.015780640 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:01:41.630 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 04:01:41 smithi119 podman[63868]: 2024-09-21 04:01:41.375933986 +0000 UTC m=+0.146852011 container create 78ff7ef87f1841768395fc1ef5b7259f06c0cc97324089b4bc6a79cb25d7b8a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-h, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:01:41.630 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 04:01:41 smithi119 podman[63868]: 2024-09-21 04:01:41.628566046 +0000 UTC m=+0.399484009 container init 78ff7ef87f1841768395fc1ef5b7259f06c0cc97324089b4bc6a79cb25d7b8a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-h, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T04:01:42.006 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 04:01:41 smithi119 podman[63868]: 2024-09-21 04:01:41.631490327 +0000 UTC m=+0.402408301 container start 78ff7ef87f1841768395fc1ef5b7259f06c0cc97324089b4bc6a79cb25d7b8a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-h, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=) 2024-09-21T04:01:42.007 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 04:01:41 smithi119 ceph-mds[63887]: -- 172.21.15.119:0/4018692662 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555611bcb860 con 0x555615d73180 2024-09-21T04:01:42.007 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 04:01:41 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-h[63883]: starting mds.h at 2024-09-21T04:01:42.007 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 04:01:41 smithi119 bash[63868]: 78ff7ef87f1841768395fc1ef5b7259f06c0cc97324089b4bc6a79cb25d7b8a2 2024-09-21T04:01:42.007 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 04:01:41 smithi119 systemd[1]: Started Ceph mds.h for e7c927ba-77cc-11ef-baf5-efdc52797490. 2024-09-21T04:01:42.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:42 smithi064 ceph-mon[31180]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:42.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:42 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:42.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:42 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:42.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:42 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:42.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:42 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:42.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:42 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:42.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:42 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:42.540 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:42 smithi131 ceph-mon[35193]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:42.540 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:42 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:42.540 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:42 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:42.541 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:42 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:42.541 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:42 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:42.541 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:42 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:42.541 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:42 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:42.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:42 smithi119 ceph-mon[34811]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:42.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:42 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:42.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:42 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:42.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:42 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:42.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:42 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:42.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:42 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:42.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:42 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:42.912 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:01:43.185 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:01:43.455 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:43 smithi131 ceph-mon[35193]: Deploying daemon mds.i on smithi131 2024-09-21T04:01:43.455 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:43 smithi131 ceph-mon[35193]: mds.? [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] up:boot 2024-09-21T04:01:43.455 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:43 smithi131 ceph-mon[35193]: fsmap 8 up:standby 2024-09-21T04:01:43.456 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:43 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T04:01:43.603 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:43 smithi064 ceph-mon[31180]: Deploying daemon mds.i on smithi131 2024-09-21T04:01:43.603 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:43 smithi064 ceph-mon[31180]: mds.? [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] up:boot 2024-09-21T04:01:43.603 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:43 smithi064 ceph-mon[31180]: fsmap 8 up:standby 2024-09-21T04:01:43.603 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:43 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T04:01:43.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.692+0000 7f6d1f593640 1 -- 172.21.15.64:0/2751415651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d18109df0 msgr2=0x7f6d18110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:43.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.692+0000 7f6d1f593640 1 --2- 172.21.15.64:0/2751415651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d18109df0 0x7f6d18110620 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6d10009ca0 tx=0x7f6d1002a8c0 comp rx=0 tx=0).stop 2024-09-21T04:01:43.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.692+0000 7f6d1f593640 1 -- 172.21.15.64:0/2751415651 shutdown_connections 2024-09-21T04:01:43.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.692+0000 7f6d1f593640 1 --2- 172.21.15.64:0/2751415651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d18109df0 0x7f6d18110620 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:43.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.692+0000 7f6d1f593640 1 --2- 172.21.15.64:0/2751415651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d181094a0 0x7f6d181098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:43.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.692+0000 7f6d1f593640 1 --2- 172.21.15.64:0/2751415651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d18103b60 0x7f6d18103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:43.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.692+0000 7f6d1f593640 1 -- 172.21.15.64:0/2751415651 >> 172.21.15.64:0/2751415651 conn(0x7f6d1807b850 msgr2=0x7f6d1807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:43.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.693+0000 7f6d1f593640 1 -- 172.21.15.64:0/2751415651 shutdown_connections 2024-09-21T04:01:43.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.693+0000 7f6d1f593640 1 -- 172.21.15.64:0/2751415651 wait complete. 2024-09-21T04:01:43.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.693+0000 7f6d1f593640 1 Processor -- start 2024-09-21T04:01:43.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.693+0000 7f6d1f593640 1 -- start start 2024-09-21T04:01:43.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.694+0000 7f6d1f593640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d18103b60 0x7f6d1819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:43.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.694+0000 7f6d1f593640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d181094a0 0x7f6d1819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:43.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.694+0000 7f6d1f593640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d18109df0 0x7f6d1819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:43.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.694+0000 7f6d1f593640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6d18113b00 con 0x7f6d181094a0 2024-09-21T04:01:43.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.694+0000 7f6d1f593640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6d18113980 con 0x7f6d18103b60 2024-09-21T04:01:43.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.694+0000 7f6d1f593640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6d18113c80 con 0x7f6d18109df0 2024-09-21T04:01:43.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.694+0000 7f6d1cb2e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d181094a0 0x7f6d1819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:43.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.694+0000 7f6d1cb2e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d181094a0 0x7f6d1819a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49382/0 (socket says 172.21.15.64:49382) 2024-09-21T04:01:43.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.694+0000 7f6d1cb2e640 1 -- 172.21.15.64:0/2431450276 learned_addr learned my addr 172.21.15.64:0/2431450276 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:01:43.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.694+0000 7f6d1d32f640 1 --2- 172.21.15.64:0/2431450276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d18103b60 0x7f6d1819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:43.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.694+0000 7f6d1db30640 1 --2- 172.21.15.64:0/2431450276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d18109df0 0x7f6d1819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:43.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.694+0000 7f6d1cb2e640 1 -- 172.21.15.64:0/2431450276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d18109df0 msgr2=0x7f6d1819fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:43.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.694+0000 7f6d1cb2e640 1 --2- 172.21.15.64:0/2431450276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d18109df0 0x7f6d1819fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:43.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.694+0000 7f6d1cb2e640 1 -- 172.21.15.64:0/2431450276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d18103b60 msgr2=0x7f6d1819a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:43.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.694+0000 7f6d1cb2e640 1 --2- 172.21.15.64:0/2431450276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d18103b60 0x7f6d1819a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:43.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.694+0000 7f6d1cb2e640 1 -- 172.21.15.64:0/2431450276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d181a0380 con 0x7f6d181094a0 2024-09-21T04:01:43.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.694+0000 7f6d1cb2e640 1 --2- 172.21.15.64:0/2431450276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d181094a0 0x7f6d1819a9b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00d750 tx=0x7f6d0c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:43.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.694+0000 7f6d0a7fc640 1 -- 172.21.15.64:0/2431450276 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c017070 con 0x7f6d181094a0 2024-09-21T04:01:43.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.694+0000 7f6d0a7fc640 1 -- 172.21.15.64:0/2431450276 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d0c0054d0 con 0x7f6d181094a0 2024-09-21T04:01:43.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.694+0000 7f6d1f593640 1 -- 172.21.15.64:0/2431450276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d181a0670 con 0x7f6d181094a0 2024-09-21T04:01:43.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.695+0000 7f6d0a7fc640 1 -- 172.21.15.64:0/2431450276 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c004c30 con 0x7f6d181094a0 2024-09-21T04:01:43.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.695+0000 7f6d1f593640 1 -- 172.21.15.64:0/2431450276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d181a6dd0 con 0x7f6d181094a0 2024-09-21T04:01:43.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.696+0000 7f6d1f593640 1 -- 172.21.15.64:0/2431450276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d18103f70 con 0x7f6d181094a0 2024-09-21T04:01:43.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.698+0000 7f6d0a7fc640 1 -- 172.21.15.64:0/2431450276 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f6d0c004dd0 con 0x7f6d181094a0 2024-09-21T04:01:43.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.699+0000 7f6d0a7fc640 1 --2- 172.21.15.64:0/2431450276 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6d040b65e0 0x7f6d040b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:43.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.699+0000 7f6d1d32f640 1 --2- 172.21.15.64:0/2431450276 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6d040b65e0 0x7f6d040b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:43.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.699+0000 7f6d0a7fc640 1 -- 172.21.15.64:0/2431450276 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7f6d0c0e6c40 con 0x7f6d181094a0 2024-09-21T04:01:43.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.699+0000 7f6d1d32f640 1 --2- 172.21.15.64:0/2431450276 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6d040b65e0 0x7f6d040b8af0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400bbe0 tx=0x7f6d1400b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:43.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.703+0000 7f6d0a7fc640 1 -- 172.21.15.64:0/2431450276 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6d0c0b1d00 con 0x7f6d181094a0 2024-09-21T04:01:43.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:43 smithi119 ceph-mon[34811]: Deploying daemon mds.i on smithi131 2024-09-21T04:01:43.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:43 smithi119 ceph-mon[34811]: mds.? [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] up:boot 2024-09-21T04:01:43.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:43 smithi119 ceph-mon[34811]: fsmap 8 up:standby 2024-09-21T04:01:43.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:43 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T04:01:43.803 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 21 04:01:43 smithi131 systemd[1]: Starting Ceph mds.i for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:01:43.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.930+0000 7f6d1f593640 1 -- 172.21.15.64:0/2431450276 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d1819b540 con 0x7f6d181094a0 2024-09-21T04:01:43.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.931+0000 7f6d0a7fc640 1 -- 172.21.15.64:0/2431450276 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7225 (secure 0 0 0) 0x7f6d1819b540 con 0x7f6d181094a0 2024-09-21T04:01:43.933 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:01:43.933 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":9,"btime":"2024-09-21T04:01:42:254302+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24634,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24640,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24656,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24680,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-21T04:01:43.933 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 9 2024-09-21T04:01:43.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.933+0000 7f6d1f593640 1 -- 172.21.15.64:0/2431450276 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6d040b65e0 msgr2=0x7f6d040b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:43.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.933+0000 7f6d1f593640 1 --2- 172.21.15.64:0/2431450276 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6d040b65e0 0x7f6d040b8af0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400bbe0 tx=0x7f6d1400b390 comp rx=0 tx=0).stop 2024-09-21T04:01:43.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.933+0000 7f6d1f593640 1 -- 172.21.15.64:0/2431450276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d181094a0 msgr2=0x7f6d1819a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:43.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.933+0000 7f6d1f593640 1 --2- 172.21.15.64:0/2431450276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d181094a0 0x7f6d1819a9b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00d750 tx=0x7f6d0c00dc10 comp rx=0 tx=0).stop 2024-09-21T04:01:43.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.933+0000 7f6d1f593640 1 -- 172.21.15.64:0/2431450276 shutdown_connections 2024-09-21T04:01:43.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.933+0000 7f6d1f593640 1 --2- 172.21.15.64:0/2431450276 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6d040b65e0 0x7f6d040b8af0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:43.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.933+0000 7f6d1f593640 1 --2- 172.21.15.64:0/2431450276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d18109df0 0x7f6d1819fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:43.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.933+0000 7f6d1f593640 1 --2- 172.21.15.64:0/2431450276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d181094a0 0x7f6d1819a9b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:43.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.933+0000 7f6d1f593640 1 --2- 172.21.15.64:0/2431450276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d18103b60 0x7f6d1819a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:43.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.933+0000 7f6d1f593640 1 -- 172.21.15.64:0/2431450276 >> 172.21.15.64:0/2431450276 conn(0x7f6d1807b850 msgr2=0x7f6d18107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:43.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.933+0000 7f6d1f593640 1 -- 172.21.15.64:0/2431450276 shutdown_connections 2024-09-21T04:01:43.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:43.933+0000 7f6d1f593640 1 -- 172.21.15.64:0/2431450276 wait complete. 2024-09-21T04:01:44.133 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 21 04:01:43 smithi131 podman[64616]: 2024-09-21 04:01:43.746558696 +0000 UTC m=+0.015992709 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:01:44.133 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 21 04:01:43 smithi131 podman[64616]: 2024-09-21 04:01:43.870447137 +0000 UTC m=+0.139881140 container create 4394b5bf8a90249c273be0009066850b3e6ed3f0db66a14b3aba49c4bcb56812 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-i, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-21T04:01:44.133 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 21 04:01:44 smithi131 podman[64616]: 2024-09-21 04:01:44.131721617 +0000 UTC m=+0.401155620 container init 4394b5bf8a90249c273be0009066850b3e6ed3f0db66a14b3aba49c4bcb56812 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-i, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-21T04:01:44.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:44 smithi131 ceph-mon[35193]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:44.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:44 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2431450276' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:44.553 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 21 04:01:44 smithi131 podman[64616]: 2024-09-21 04:01:44.134565638 +0000 UTC m=+0.403999660 container start 4394b5bf8a90249c273be0009066850b3e6ed3f0db66a14b3aba49c4bcb56812 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-i, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-21T04:01:44.553 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 21 04:01:44 smithi131 ceph-mds[64635]: -- 172.21.15.131:0/2279514719 <== mon.1 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562421095860 con 0x56242523d600 2024-09-21T04:01:44.553 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 21 04:01:44 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-i[64631]: starting mds.i at 2024-09-21T04:01:44.554 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 21 04:01:44 smithi131 bash[64616]: 4394b5bf8a90249c273be0009066850b3e6ed3f0db66a14b3aba49c4bcb56812 2024-09-21T04:01:44.554 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 21 04:01:44 smithi131 systemd[1]: Started Ceph mds.i for e7c927ba-77cc-11ef-baf5-efdc52797490. 2024-09-21T04:01:44.582 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:44 smithi064 ceph-mon[31180]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:44.583 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:44 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2431450276' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:44.658 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:44 smithi119 ceph-mon[34811]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:44.658 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:44 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2431450276' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:45.552 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:45 smithi131 ceph-mon[35193]: mds.? [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] up:boot 2024-09-21T04:01:45.552 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:45 smithi131 ceph-mon[35193]: fsmap 9 up:standby 2024-09-21T04:01:45.552 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:45 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T04:01:45.552 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:45 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:45.552 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:45 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:45.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:45 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:45.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:45 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:45.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:45 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:45.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:45 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:45.690 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:45 smithi064 ceph-mon[31180]: mds.? [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] up:boot 2024-09-21T04:01:45.690 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:45 smithi064 ceph-mon[31180]: fsmap 9 up:standby 2024-09-21T04:01:45.691 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:45 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T04:01:45.691 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:45 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:45.691 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:45 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:45.691 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:45 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:45.691 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:45 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:45.691 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:45 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:45.691 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:45 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:45.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:45 smithi119 ceph-mon[34811]: mds.? [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] up:boot 2024-09-21T04:01:45.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:45 smithi119 ceph-mon[34811]: fsmap 9 up:standby 2024-09-21T04:01:45.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:45 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T04:01:45.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:45 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:45.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:45 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:45.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:45 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:45.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:45 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:45.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:45 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:45.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:45 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:46.262 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 21 04:01:46 smithi064 systemd[1]: Starting Ceph mds.g for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:01:46.534 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:46 smithi064 ceph-mon[31180]: Deploying daemon mds.g on smithi064 2024-09-21T04:01:46.534 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:46 smithi064 ceph-mon[31180]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:46.534 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 21 04:01:46 smithi064 podman[70578]: 2024-09-21 04:01:46.284035452 +0000 UTC m=+0.016866110 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:01:46.534 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 21 04:01:46 smithi064 podman[70578]: 2024-09-21 04:01:46.399394736 +0000 UTC m=+0.132225387 container create 3bedb67a4c32988bde7c7ce75b2ec95a1ab765efebfe145f122cdb3bdaa6fa08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-g, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-21T04:01:46.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:46 smithi131 ceph-mon[35193]: Deploying daemon mds.g on smithi064 2024-09-21T04:01:46.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:46 smithi131 ceph-mon[35193]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:46.615 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:01:46.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:46 smithi119 ceph-mon[34811]: Deploying daemon mds.g on smithi064 2024-09-21T04:01:46.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:46 smithi119 ceph-mon[34811]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:46.887 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:01:47.009 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 21 04:01:46 smithi064 podman[70578]: 2024-09-21 04:01:46.628074899 +0000 UTC m=+0.360905564 container init 3bedb67a4c32988bde7c7ce75b2ec95a1ab765efebfe145f122cdb3bdaa6fa08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-g, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0) 2024-09-21T04:01:47.010 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 21 04:01:46 smithi064 podman[70578]: 2024-09-21 04:01:46.63108051 +0000 UTC m=+0.363911167 container start 3bedb67a4c32988bde7c7ce75b2ec95a1ab765efebfe145f122cdb3bdaa6fa08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-g, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-21T04:01:47.010 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 21 04:01:46 smithi064 ceph-mds[70613]: -- 172.21.15.64:0/1309137605 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d30fb07860 con 0x55d313caf600 2024-09-21T04:01:47.010 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 21 04:01:46 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-g[70593]: starting mds.g at 2024-09-21T04:01:47.010 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 21 04:01:46 smithi064 bash[70578]: 3bedb67a4c32988bde7c7ce75b2ec95a1ab765efebfe145f122cdb3bdaa6fa08 2024-09-21T04:01:47.010 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 21 04:01:46 smithi064 systemd[1]: Started Ceph mds.g for e7c927ba-77cc-11ef-baf5-efdc52797490. 2024-09-21T04:01:47.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.385+0000 7fec84448640 1 -- 172.21.15.64:0/1719248134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec7c1094c0 msgr2=0x7fec7c1098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:47.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.385+0000 7fec84448640 1 --2- 172.21.15.64:0/1719248134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec7c1094c0 0x7fec7c1098d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fec6c009900 tx=0x7fec6c02a410 comp rx=0 tx=0).stop 2024-09-21T04:01:47.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.385+0000 7fec84448640 1 -- 172.21.15.64:0/1719248134 shutdown_connections 2024-09-21T04:01:47.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.385+0000 7fec84448640 1 --2- 172.21.15.64:0/1719248134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec7c109e10 0x7fec7c110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:47.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.385+0000 7fec84448640 1 --2- 172.21.15.64:0/1719248134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec7c1094c0 0x7fec7c1098d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:47.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.385+0000 7fec84448640 1 --2- 172.21.15.64:0/1719248134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec7c103b80 0x7fec7c103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:47.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.385+0000 7fec84448640 1 -- 172.21.15.64:0/1719248134 >> 172.21.15.64:0/1719248134 conn(0x7fec7c0ff5d0 msgr2=0x7fec7c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:47.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.385+0000 7fec84448640 1 -- 172.21.15.64:0/1719248134 shutdown_connections 2024-09-21T04:01:47.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.385+0000 7fec84448640 1 -- 172.21.15.64:0/1719248134 wait complete. 2024-09-21T04:01:47.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.386+0000 7fec84448640 1 Processor -- start 2024-09-21T04:01:47.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.386+0000 7fec84448640 1 -- start start 2024-09-21T04:01:47.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.386+0000 7fec84448640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec7c103b80 0x7fec7c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:47.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.386+0000 7fec84448640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec7c1094c0 0x7fec7c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:47.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.386+0000 7fec84448640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec7c109e10 0x7fec7c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:47.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.386+0000 7fec84448640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c113ad0 con 0x7fec7c103b80 2024-09-21T04:01:47.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.386+0000 7fec84448640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c113950 con 0x7fec7c109e10 2024-09-21T04:01:47.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.386+0000 7fec84448640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c113c50 con 0x7fec7c1094c0 2024-09-21T04:01:47.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.386+0000 7fec821e4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec7c103b80 0x7fec7c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:47.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.386+0000 7fec821e4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec7c103b80 0x7fec7c075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49424/0 (socket says 172.21.15.64:49424) 2024-09-21T04:01:47.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.386+0000 7fec821e4640 1 -- 172.21.15.64:0/1911689573 learned_addr learned my addr 172.21.15.64:0/1911689573 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:01:47.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.386+0000 7fec829e5640 1 --2- 172.21.15.64:0/1911689573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec7c109e10 0x7fec7c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:47.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.386+0000 7fec819e3640 1 --2- 172.21.15.64:0/1911689573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec7c1094c0 0x7fec7c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:47.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.387+0000 7fec821e4640 1 -- 172.21.15.64:0/1911689573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec7c1094c0 msgr2=0x7fec7c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:47.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.387+0000 7fec821e4640 1 --2- 172.21.15.64:0/1911689573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec7c1094c0 0x7fec7c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:47.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.387+0000 7fec821e4640 1 -- 172.21.15.64:0/1911689573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec7c109e10 msgr2=0x7fec7c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:47.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.387+0000 7fec821e4640 1 --2- 172.21.15.64:0/1911689573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec7c109e10 0x7fec7c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:47.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.387+0000 7fec821e4640 1 -- 172.21.15.64:0/1911689573 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7c1a8740 con 0x7fec7c103b80 2024-09-21T04:01:47.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.387+0000 7fec821e4640 1 --2- 172.21.15.64:0/1911689573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec7c103b80 0x7fec7c075710 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fec7800d950 tx=0x7fec7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:47.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.387+0000 7fec737fe640 1 -- 172.21.15.64:0/1911689573 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec78017030 con 0x7fec7c103b80 2024-09-21T04:01:47.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.387+0000 7fec84448640 1 -- 172.21.15.64:0/1911689573 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec7c1a8a30 con 0x7fec7c103b80 2024-09-21T04:01:47.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.387+0000 7fec737fe640 1 -- 172.21.15.64:0/1911689573 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec78004020 con 0x7fec7c103b80 2024-09-21T04:01:47.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.387+0000 7fec737fe640 1 -- 172.21.15.64:0/1911689573 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec780047a0 con 0x7fec7c103b80 2024-09-21T04:01:47.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.387+0000 7fec84448640 1 -- 172.21.15.64:0/1911689573 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec7c1a8f90 con 0x7fec7c103b80 2024-09-21T04:01:47.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.388+0000 7fec84448640 1 -- 172.21.15.64:0/1911689573 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec7c103f90 con 0x7fec7c103b80 2024-09-21T04:01:47.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.390+0000 7fec737fe640 1 -- 172.21.15.64:0/1911689573 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fec78009a30 con 0x7fec7c103b80 2024-09-21T04:01:47.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.391+0000 7fec737fe640 1 --2- 172.21.15.64:0/1911689573 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fec680b6560 0x7fec680b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:47.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.391+0000 7fec819e3640 1 --2- 172.21.15.64:0/1911689573 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fec680b6560 0x7fec680b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:47.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.391+0000 7fec737fe640 1 -- 172.21.15.64:0/1911689573 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7fec780e6ad0 con 0x7fec7c103b80 2024-09-21T04:01:47.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.391+0000 7fec819e3640 1 --2- 172.21.15.64:0/1911689573 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fec680b6560 0x7fec680b8a70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fec6c009230 tx=0x7fec6c031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:47.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.397+0000 7fec737fe640 1 -- 172.21.15.64:0/1911689573 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fec780ad730 con 0x7fec7c103b80 2024-09-21T04:01:47.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.638+0000 7fec84448640 1 -- 172.21.15.64:0/1911689573 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec7c02e640 con 0x7fec7c103b80 2024-09-21T04:01:47.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.638+0000 7fec737fe640 1 -- 172.21.15.64:0/1911689573 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8872 (secure 0 0 0) 0x7fec780b1b10 con 0x7fec7c103b80 2024-09-21T04:01:47.641 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:01:47.641 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":11,"btime":"2024-09-21T04:01:47:282569+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14805,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/97995119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":97995119},{"type":"v1","addr":"172.21.15.64:6839","nonce":97995119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24634,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24640,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24656,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24661,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6839/782190424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782190424},{"type":"v1","addr":"172.21.15.131:6839","nonce":782190424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24680,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-21T04:01:47.641 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 11 2024-09-21T04:01:47.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.641+0000 7fec84448640 1 -- 172.21.15.64:0/1911689573 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fec680b6560 msgr2=0x7fec680b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:47.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.641+0000 7fec84448640 1 --2- 172.21.15.64:0/1911689573 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fec680b6560 0x7fec680b8a70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fec6c009230 tx=0x7fec6c031400 comp rx=0 tx=0).stop 2024-09-21T04:01:47.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.641+0000 7fec84448640 1 -- 172.21.15.64:0/1911689573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec7c103b80 msgr2=0x7fec7c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:47.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.641+0000 7fec84448640 1 --2- 172.21.15.64:0/1911689573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec7c103b80 0x7fec7c075710 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fec7800d950 tx=0x7fec7800de10 comp rx=0 tx=0).stop 2024-09-21T04:01:47.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.641+0000 7fec84448640 1 -- 172.21.15.64:0/1911689573 shutdown_connections 2024-09-21T04:01:47.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.641+0000 7fec84448640 1 --2- 172.21.15.64:0/1911689573 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fec680b6560 0x7fec680b8a70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:47.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.641+0000 7fec84448640 1 --2- 172.21.15.64:0/1911689573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec7c109e10 0x7fec7c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:47.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.641+0000 7fec84448640 1 --2- 172.21.15.64:0/1911689573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec7c1094c0 0x7fec7c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:47.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.641+0000 7fec84448640 1 --2- 172.21.15.64:0/1911689573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec7c103b80 0x7fec7c075710 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:47.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.641+0000 7fec84448640 1 -- 172.21.15.64:0/1911689573 >> 172.21.15.64:0/1911689573 conn(0x7fec7c0ff5d0 msgr2=0x7fec7c1019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:47.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.641+0000 7fec84448640 1 -- 172.21.15.64:0/1911689573 shutdown_connections 2024-09-21T04:01:47.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:47.641+0000 7fec84448640 1 -- 172.21.15.64:0/1911689573 wait complete. 2024-09-21T04:01:48.235 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:47 smithi064 ceph-mon[31180]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:48.235 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:47 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:48.235 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:47 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:48.235 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:47 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:48.235 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:47 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:48.235 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:47 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:48.235 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:47 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:48.235 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:47 smithi064 ceph-mon[31180]: Deploying daemon mds.a on smithi064 2024-09-21T04:01:48.235 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:47 smithi064 ceph-mon[31180]: mds.? [v2:172.21.15.64:6838/97995119,v1:172.21.15.64:6839/97995119] up:boot 2024-09-21T04:01:48.235 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:47 smithi064 ceph-mon[31180]: fsmap 10 up:standby 2024-09-21T04:01:48.235 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:47 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T04:01:48.235 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:47 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1911689573' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:48.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:47 smithi119 ceph-mon[34811]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:48.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:47 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:48.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:47 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:48.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:47 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:48.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:47 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:48.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:47 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:48.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:47 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:48.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:47 smithi119 ceph-mon[34811]: Deploying daemon mds.a on smithi064 2024-09-21T04:01:48.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:47 smithi119 ceph-mon[34811]: mds.? [v2:172.21.15.64:6838/97995119,v1:172.21.15.64:6839/97995119] up:boot 2024-09-21T04:01:48.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:47 smithi119 ceph-mon[34811]: fsmap 10 up:standby 2024-09-21T04:01:48.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:47 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T04:01:48.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:47 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1911689573' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:48.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:47 smithi131 ceph-mon[35193]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:48.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:47 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:48.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:47 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:48.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:47 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:48.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:47 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:48.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:47 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:48.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:47 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:48.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:47 smithi131 ceph-mon[35193]: Deploying daemon mds.a on smithi064 2024-09-21T04:01:48.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:47 smithi131 ceph-mon[35193]: mds.? [v2:172.21.15.64:6838/97995119,v1:172.21.15.64:6839/97995119] up:boot 2024-09-21T04:01:48.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:47 smithi131 ceph-mon[35193]: fsmap 10 up:standby 2024-09-21T04:01:48.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:47 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T04:01:48.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:47 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1911689573' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:49.761 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 21 04:01:49 smithi064 systemd[1]: Starting Ceph mds.a for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:01:50.064 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:49 smithi064 ceph-mon[31180]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:50.065 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 21 04:01:49 smithi064 podman[70961]: 2024-09-21 04:01:49.681704834 +0000 UTC m=+0.016433695 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:01:50.065 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 21 04:01:49 smithi064 podman[70961]: 2024-09-21 04:01:49.810192601 +0000 UTC m=+0.144921451 container create daa9ea597ce9a7b302261133f4aec2988545f05f14a15aab63d6c3ae2f32414f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-a, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T04:01:50.065 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 21 04:01:50 smithi064 podman[70961]: 2024-09-21 04:01:50.063482432 +0000 UTC m=+0.398211280 container init daa9ea597ce9a7b302261133f4aec2988545f05f14a15aab63d6c3ae2f32414f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-a, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-21T04:01:50.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:49 smithi119 ceph-mon[34811]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:50.264 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:01:50.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:49 smithi131 ceph-mon[35193]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:50.511 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 21 04:01:50 smithi064 podman[70961]: 2024-09-21 04:01:50.066502076 +0000 UTC m=+0.401230928 container start daa9ea597ce9a7b302261133f4aec2988545f05f14a15aab63d6c3ae2f32414f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-a, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T04:01:50.511 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 21 04:01:50 smithi064 ceph-mds[70980]: -- 172.21.15.64:0/1457047108 <== mon.0 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560d8d94b860 con 0x560d91af3a80 2024-09-21T04:01:50.511 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 21 04:01:50 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-a[70976]: starting mds.a at 2024-09-21T04:01:50.511 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 21 04:01:50 smithi064 bash[70961]: daa9ea597ce9a7b302261133f4aec2988545f05f14a15aab63d6c3ae2f32414f 2024-09-21T04:01:50.512 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 21 04:01:50 smithi064 systemd[1]: Started Ceph mds.a for e7c927ba-77cc-11ef-baf5-efdc52797490. 2024-09-21T04:01:50.526 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:01:51.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.010+0000 7fca84d3c640 1 -- 172.21.15.64:0/2623646035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca80102ae0 msgr2=0x7fca80102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:51.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.010+0000 7fca84d3c640 1 --2- 172.21.15.64:0/2623646035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca80102ae0 0x7fca80102ef0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fca70009900 tx=0x7fca7002a410 comp rx=0 tx=0).stop 2024-09-21T04:01:51.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.010+0000 7fca84d3c640 1 -- 172.21.15.64:0/2623646035 shutdown_connections 2024-09-21T04:01:51.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.010+0000 7fca84d3c640 1 --2- 172.21.15.64:0/2623646035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca80108d70 0x7fca8010f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:51.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.010+0000 7fca84d3c640 1 --2- 172.21.15.64:0/2623646035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca80108420 0x7fca80108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:51.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.010+0000 7fca84d3c640 1 --2- 172.21.15.64:0/2623646035 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca80102ae0 0x7fca80102ef0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:51.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.010+0000 7fca84d3c640 1 -- 172.21.15.64:0/2623646035 >> 172.21.15.64:0/2623646035 conn(0x7fca80079770 msgr2=0x7fca80079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:51.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.011+0000 7fca84d3c640 1 -- 172.21.15.64:0/2623646035 shutdown_connections 2024-09-21T04:01:51.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.011+0000 7fca84d3c640 1 -- 172.21.15.64:0/2623646035 wait complete. 2024-09-21T04:01:51.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.011+0000 7fca84d3c640 1 Processor -- start 2024-09-21T04:01:51.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.011+0000 7fca84d3c640 1 -- start start 2024-09-21T04:01:51.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.012+0000 7fca84d3c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca80102ae0 0x7fca8019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:51.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.012+0000 7fca84d3c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca80108420 0x7fca8019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:51.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.012+0000 7fca84d3c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca80108d70 0x7fca8019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:51.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.012+0000 7fca84d3c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fca80113b50 con 0x7fca80108420 2024-09-21T04:01:51.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.012+0000 7fca84d3c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fca801139d0 con 0x7fca80102ae0 2024-09-21T04:01:51.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.012+0000 7fca84d3c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fca80113cd0 con 0x7fca80108d70 2024-09-21T04:01:51.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.012+0000 7fca7dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca80108420 0x7fca8019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:51.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.012+0000 7fca7dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca80108420 0x7fca8019ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49472/0 (socket says 172.21.15.64:49472) 2024-09-21T04:01:51.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.012+0000 7fca7dd9b640 1 -- 172.21.15.64:0/2342229884 learned_addr learned my addr 172.21.15.64:0/2342229884 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:01:51.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.012+0000 7fca7e59c640 1 --2- 172.21.15.64:0/2342229884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca80102ae0 0x7fca8019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:51.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.012+0000 7fca7ed9d640 1 --2- 172.21.15.64:0/2342229884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca80108d70 0x7fca8019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:51.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.012+0000 7fca7dd9b640 1 -- 172.21.15.64:0/2342229884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca80108d70 msgr2=0x7fca8019fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:51.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.012+0000 7fca7dd9b640 1 --2- 172.21.15.64:0/2342229884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca80108d70 0x7fca8019fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:51.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.012+0000 7fca7dd9b640 1 -- 172.21.15.64:0/2342229884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca80102ae0 msgr2=0x7fca8019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:51.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.012+0000 7fca7dd9b640 1 --2- 172.21.15.64:0/2342229884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca80102ae0 0x7fca8019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:51.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.012+0000 7fca7dd9b640 1 -- 172.21.15.64:0/2342229884 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca801a0510 con 0x7fca80108420 2024-09-21T04:01:51.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.012+0000 7fca7dd9b640 1 --2- 172.21.15.64:0/2342229884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca80108420 0x7fca8019ab40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fca7400d750 tx=0x7fca7400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:51.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.013+0000 7fca6f7fe640 1 -- 172.21.15.64:0/2342229884 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74017070 con 0x7fca80108420 2024-09-21T04:01:51.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.013+0000 7fca6f7fe640 1 -- 172.21.15.64:0/2342229884 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca740054d0 con 0x7fca80108420 2024-09-21T04:01:51.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.013+0000 7fca84d3c640 1 -- 172.21.15.64:0/2342229884 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca801a0800 con 0x7fca80108420 2024-09-21T04:01:51.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.013+0000 7fca6f7fe640 1 -- 172.21.15.64:0/2342229884 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74004c30 con 0x7fca80108420 2024-09-21T04:01:51.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.013+0000 7fca84d3c640 1 -- 172.21.15.64:0/2342229884 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca801a6f60 con 0x7fca80108420 2024-09-21T04:01:51.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.014+0000 7fca84d3c640 1 -- 172.21.15.64:0/2342229884 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca80103d60 con 0x7fca80108420 2024-09-21T04:01:51.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.017+0000 7fca6f7fe640 1 -- 172.21.15.64:0/2342229884 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fca74004dd0 con 0x7fca80108420 2024-09-21T04:01:51.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.017+0000 7fca6f7fe640 1 --2- 172.21.15.64:0/2342229884 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fca680b6560 0x7fca680b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:51.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.017+0000 7fca7e59c640 1 --2- 172.21.15.64:0/2342229884 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fca680b6560 0x7fca680b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:51.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.017+0000 7fca6f7fe640 1 -- 172.21.15.64:0/2342229884 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7fca740e6cb0 con 0x7fca80108420 2024-09-21T04:01:51.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.018+0000 7fca7e59c640 1 --2- 172.21.15.64:0/2342229884 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fca680b6560 0x7fca680b8a70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fca70031440 tx=0x7fca70009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:51.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.020+0000 7fca6f7fe640 1 -- 172.21.15.64:0/2342229884 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fca740b1cf0 con 0x7fca80108420 2024-09-21T04:01:51.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.251+0000 7fca84d3c640 1 -- 172.21.15.64:0/2342229884 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca80102ef0 con 0x7fca80108420 2024-09-21T04:01:51.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.251+0000 7fca6f7fe640 1 -- 172.21.15.64:0/2342229884 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9695 (secure 0 0 0) 0x7fca740b6c30 con 0x7fca80108420 2024-09-21T04:01:51.254 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:01:51.254 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":12,"btime":"2024-09-21T04:01:50:809697+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14805,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/97995119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":97995119},{"type":"v1","addr":"172.21.15.64:6839","nonce":97995119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/390891068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":390891068},{"type":"v1","addr":"172.21.15.64:6841","nonce":390891068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24634,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24640,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24656,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24661,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6839/782190424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782190424},{"type":"v1","addr":"172.21.15.131:6839","nonce":782190424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24680,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-21T04:01:51.254 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 12 2024-09-21T04:01:51.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.254+0000 7fca84d3c640 1 -- 172.21.15.64:0/2342229884 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fca680b6560 msgr2=0x7fca680b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.254+0000 7fca84d3c640 1 --2- 172.21.15.64:0/2342229884 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fca680b6560 0x7fca680b8a70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fca70031440 tx=0x7fca70009560 comp rx=0 tx=0).stop 2024-09-21T04:01:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.254+0000 7fca84d3c640 1 -- 172.21.15.64:0/2342229884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca80108420 msgr2=0x7fca8019ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.254+0000 7fca84d3c640 1 --2- 172.21.15.64:0/2342229884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca80108420 0x7fca8019ab40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fca7400d750 tx=0x7fca7400dc10 comp rx=0 tx=0).stop 2024-09-21T04:01:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.254+0000 7fca84d3c640 1 -- 172.21.15.64:0/2342229884 shutdown_connections 2024-09-21T04:01:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.254+0000 7fca84d3c640 1 --2- 172.21.15.64:0/2342229884 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fca680b6560 0x7fca680b8a70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.254+0000 7fca84d3c640 1 --2- 172.21.15.64:0/2342229884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca80108d70 0x7fca8019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.254+0000 7fca84d3c640 1 --2- 172.21.15.64:0/2342229884 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca80108420 0x7fca8019ab40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.254+0000 7fca84d3c640 1 --2- 172.21.15.64:0/2342229884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca80102ae0 0x7fca8019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.254+0000 7fca84d3c640 1 -- 172.21.15.64:0/2342229884 >> 172.21.15.64:0/2342229884 conn(0x7fca80079770 msgr2=0x7fca80106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.254+0000 7fca84d3c640 1 -- 172.21.15.64:0/2342229884 shutdown_connections 2024-09-21T04:01:51.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:51.254+0000 7fca84d3c640 1 -- 172.21.15.64:0/2342229884 wait complete. 2024-09-21T04:01:51.258 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:51 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:51.258 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:51 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:51.258 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:51 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:51.258 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:51 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:51.258 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:51 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:51.258 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:51 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:51.258 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:51 smithi119 ceph-mon[34811]: Deploying daemon mds.b on smithi119 2024-09-21T04:01:51.258 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:51 smithi119 ceph-mon[34811]: mds.? [v2:172.21.15.64:6840/390891068,v1:172.21.15.64:6841/390891068] up:boot 2024-09-21T04:01:51.258 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:51 smithi119 ceph-mon[34811]: fsmap 11 up:standby 2024-09-21T04:01:51.259 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:51 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T04:01:51.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:51 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:51.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:51 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:51.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:51 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:51.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:51 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:51.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:51 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:51.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:51 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:51.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:51 smithi064 ceph-mon[31180]: Deploying daemon mds.b on smithi119 2024-09-21T04:01:51.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:51 smithi064 ceph-mon[31180]: mds.? [v2:172.21.15.64:6840/390891068,v1:172.21.15.64:6841/390891068] up:boot 2024-09-21T04:01:51.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:51 smithi064 ceph-mon[31180]: fsmap 11 up:standby 2024-09-21T04:01:51.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:51 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T04:01:51.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:51 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:51.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:51 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:51.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:51 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:51.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:51 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T04:01:51.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:51 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T04:01:51.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:51 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:51.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:51 smithi131 ceph-mon[35193]: Deploying daemon mds.b on smithi119 2024-09-21T04:01:51.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:51 smithi131 ceph-mon[35193]: mds.? [v2:172.21.15.64:6840/390891068,v1:172.21.15.64:6841/390891068] up:boot 2024-09-21T04:01:51.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:51 smithi131 ceph-mon[35193]: fsmap 11 up:standby 2024-09-21T04:01:51.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:51 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T04:01:52.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:52 smithi119 ceph-mon[34811]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:52.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:52 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2342229884' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:52.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:52 smithi064 ceph-mon[31180]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:52.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:52 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2342229884' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:52.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:52 smithi131 ceph-mon[35193]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:52.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:52 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2342229884' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:53.006 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 04:01:52 smithi119 systemd[1]: Started Ceph mds.b for e7c927ba-77cc-11ef-baf5-efdc52797490. 2024-09-21T04:01:53.914 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:01:54.213 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:01:54.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:53 smithi119 ceph-mon[34811]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:54.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:53 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:54.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:53 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:54.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:53 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:54.258 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:53 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:54.258 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:53 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:54.258 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:53 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:01:54.258 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:53 smithi119 ceph-mon[34811]: mds.? [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] up:boot 2024-09-21T04:01:54.258 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:53 smithi119 ceph-mon[34811]: fsmap 12 up:standby 2024-09-21T04:01:54.258 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:53 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T04:01:54.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:53 smithi064 ceph-mon[31180]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:54.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:53 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:54.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:53 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:54.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:53 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:54.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:53 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:54.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:53 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:54.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:53 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:01:54.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:53 smithi064 ceph-mon[31180]: mds.? [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] up:boot 2024-09-21T04:01:54.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:53 smithi064 ceph-mon[31180]: fsmap 12 up:standby 2024-09-21T04:01:54.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:53 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T04:01:54.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:53 smithi131 ceph-mon[35193]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:54.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:53 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:54.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:53 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:54.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:53 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:54.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:53 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:54.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:53 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:54.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:53 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:01:54.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:53 smithi131 ceph-mon[35193]: mds.? [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] up:boot 2024-09-21T04:01:54.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:53 smithi131 ceph-mon[35193]: fsmap 12 up:standby 2024-09-21T04:01:54.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:53 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T04:01:54.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.725+0000 7f3ec46d3640 1 -- 172.21.15.64:0/181019653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ebc1083d0 msgr2=0x7f3ebc1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:54.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.725+0000 7f3ec46d3640 1 --2- 172.21.15.64:0/181019653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ebc1083d0 0x7f3ebc1087e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0009900 tx=0x7f3eb002a410 comp rx=0 tx=0).stop 2024-09-21T04:01:54.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.725+0000 7f3ec46d3640 1 -- 172.21.15.64:0/181019653 shutdown_connections 2024-09-21T04:01:54.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.725+0000 7f3ec46d3640 1 --2- 172.21.15.64:0/181019653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ebc108d20 0x7f3ebc10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:54.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.725+0000 7f3ec46d3640 1 --2- 172.21.15.64:0/181019653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ebc1083d0 0x7f3ebc1087e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:54.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.725+0000 7f3ec46d3640 1 --2- 172.21.15.64:0/181019653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ebc102ae0 0x7f3ebc102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:54.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.725+0000 7f3ec46d3640 1 -- 172.21.15.64:0/181019653 >> 172.21.15.64:0/181019653 conn(0x7f3ebc079790 msgr2=0x7f3ebc079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:54.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.725+0000 7f3ec46d3640 1 -- 172.21.15.64:0/181019653 shutdown_connections 2024-09-21T04:01:54.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.725+0000 7f3ec46d3640 1 -- 172.21.15.64:0/181019653 wait complete. 2024-09-21T04:01:54.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.726+0000 7f3ec46d3640 1 Processor -- start 2024-09-21T04:01:54.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.726+0000 7f3ec46d3640 1 -- start start 2024-09-21T04:01:54.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.726+0000 7f3ec46d3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ebc102ae0 0x7f3ebc19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:54.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.726+0000 7f3ec46d3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ebc1083d0 0x7f3ebc19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:54.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.726+0000 7f3ec46d3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ebc108d20 0x7f3ebc19fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:54.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.726+0000 7f3ec46d3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3ebc112a00 con 0x7f3ebc108d20 2024-09-21T04:01:54.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.726+0000 7f3ec46d3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3ebc112880 con 0x7f3ebc102ae0 2024-09-21T04:01:54.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.726+0000 7f3ec46d3640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3ebc112b80 con 0x7f3ebc1083d0 2024-09-21T04:01:54.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.726+0000 7f3ec2c70640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ebc108d20 0x7f3ebc19fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:54.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.726+0000 7f3ec2c70640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ebc108d20 0x7f3ebc19fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46592/0 (socket says 172.21.15.64:46592) 2024-09-21T04:01:54.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.727+0000 7f3ec2c70640 1 -- 172.21.15.64:0/3000024370 learned_addr learned my addr 172.21.15.64:0/3000024370 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:01:54.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.727+0000 7f3ec246f640 1 --2- 172.21.15.64:0/3000024370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ebc102ae0 0x7f3ebc19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:54.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.727+0000 7f3ec1c6e640 1 --2- 172.21.15.64:0/3000024370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ebc1083d0 0x7f3ebc19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:54.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.727+0000 7f3ec2c70640 1 -- 172.21.15.64:0/3000024370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ebc1083d0 msgr2=0x7f3ebc19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:54.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.727+0000 7f3ec2c70640 1 --2- 172.21.15.64:0/3000024370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ebc1083d0 0x7f3ebc19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:54.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.727+0000 7f3ec2c70640 1 -- 172.21.15.64:0/3000024370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ebc102ae0 msgr2=0x7f3ebc19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:54.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.727+0000 7f3ec2c70640 1 --2- 172.21.15.64:0/3000024370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ebc102ae0 0x7f3ebc19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:54.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.727+0000 7f3ec2c70640 1 -- 172.21.15.64:0/3000024370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ebc1a0470 con 0x7f3ebc108d20 2024-09-21T04:01:54.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.727+0000 7f3ec2c70640 1 --2- 172.21.15.64:0/3000024370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ebc108d20 0x7f3ebc19fe00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3eb400cce0 tx=0x7f3eb400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:54.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.727+0000 7f3eaf7fe640 1 -- 172.21.15.64:0/3000024370 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb400f450 con 0x7f3ebc108d20 2024-09-21T04:01:54.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.727+0000 7f3eaf7fe640 1 -- 172.21.15.64:0/3000024370 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3eb400faa0 con 0x7f3ebc108d20 2024-09-21T04:01:54.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.727+0000 7f3ec46d3640 1 -- 172.21.15.64:0/3000024370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ebc1a0760 con 0x7f3ebc108d20 2024-09-21T04:01:54.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.727+0000 7f3eaf7fe640 1 -- 172.21.15.64:0/3000024370 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4018a80 con 0x7f3ebc108d20 2024-09-21T04:01:54.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.727+0000 7f3ec46d3640 1 -- 172.21.15.64:0/3000024370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ebc1a0c20 con 0x7f3ebc108d20 2024-09-21T04:01:54.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.728+0000 7f3ec46d3640 1 -- 172.21.15.64:0/3000024370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ebc102ef0 con 0x7f3ebc108d20 2024-09-21T04:01:54.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.730+0000 7f3eaf7fe640 1 -- 172.21.15.64:0/3000024370 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f3eb400fc50 con 0x7f3ebc108d20 2024-09-21T04:01:54.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.731+0000 7f3eaf7fe640 1 --2- 172.21.15.64:0/3000024370 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3ea80b6590 0x7f3ea80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:54.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.731+0000 7f3ec246f640 1 --2- 172.21.15.64:0/3000024370 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3ea80b6590 0x7f3ea80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:54.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.731+0000 7f3eaf7fe640 1 -- 172.21.15.64:0/3000024370 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7f3eb40e7800 con 0x7f3ebc108d20 2024-09-21T04:01:54.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.732+0000 7f3ec246f640 1 --2- 172.21.15.64:0/3000024370 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3ea80b6590 0x7f3ea80b8aa0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009990 tx=0x7f3eb8009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:54.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.734+0000 7f3eaf7fe640 1 -- 172.21.15.64:0/3000024370 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3eb4014030 con 0x7f3ebc108d20 2024-09-21T04:01:54.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.991+0000 7f3ec46d3640 1 -- 172.21.15.64:0/3000024370 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ebc19b6f0 con 0x7f3ebc108d20 2024-09-21T04:01:54.994 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:54 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:54.994 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:54 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:54.994 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:54 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:54.995 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:54 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:54.995 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:54 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:54.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.992+0000 7f3eaf7fe640 1 -- 172.21.15.64:0/3000024370 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10524 (secure 0 0 0) 0x7f3ebc19b6f0 con 0x7f3ebc108d20 2024-09-21T04:01:54.995 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:01:54.995 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":13,"btime":"2024-09-21T04:01:53:239013+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14805,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/97995119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":97995119},{"type":"v1","addr":"172.21.15.64:6839","nonce":97995119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/390891068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":390891068},{"type":"v1","addr":"172.21.15.64:6841","nonce":390891068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24634,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24640,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24656,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24661,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6839/782190424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782190424},{"type":"v1","addr":"172.21.15.131:6839","nonce":782190424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24680,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24710,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6839/1319793636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1319793636},{"type":"v1","addr":"172.21.15.119:6839","nonce":1319793636}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-21T04:01:54.996 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 13 2024-09-21T04:01:54.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.994+0000 7f3ec46d3640 1 -- 172.21.15.64:0/3000024370 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3ea80b6590 msgr2=0x7f3ea80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:54.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.994+0000 7f3ec46d3640 1 --2- 172.21.15.64:0/3000024370 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3ea80b6590 0x7f3ea80b8aa0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009990 tx=0x7f3eb8009110 comp rx=0 tx=0).stop 2024-09-21T04:01:54.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.994+0000 7f3ec46d3640 1 -- 172.21.15.64:0/3000024370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ebc108d20 msgr2=0x7f3ebc19fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:54.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.994+0000 7f3ec46d3640 1 --2- 172.21.15.64:0/3000024370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ebc108d20 0x7f3ebc19fe00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3eb400cce0 tx=0x7f3eb400a620 comp rx=0 tx=0).stop 2024-09-21T04:01:54.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.994+0000 7f3ec46d3640 1 -- 172.21.15.64:0/3000024370 shutdown_connections 2024-09-21T04:01:54.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.994+0000 7f3ec46d3640 1 --2- 172.21.15.64:0/3000024370 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3ea80b6590 0x7f3ea80b8aa0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:54.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.994+0000 7f3ec46d3640 1 --2- 172.21.15.64:0/3000024370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3ebc108d20 0x7f3ebc19fe00 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:54.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.994+0000 7f3ec46d3640 1 --2- 172.21.15.64:0/3000024370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ebc1083d0 0x7f3ebc19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:54.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.994+0000 7f3ec46d3640 1 --2- 172.21.15.64:0/3000024370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ebc102ae0 0x7f3ebc19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:54.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.994+0000 7f3ec46d3640 1 -- 172.21.15.64:0/3000024370 >> 172.21.15.64:0/3000024370 conn(0x7f3ebc079790 msgr2=0x7f3ebc1019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:54.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.995+0000 7f3ec46d3640 1 -- 172.21.15.64:0/3000024370 shutdown_connections 2024-09-21T04:01:54.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:54.995+0000 7f3ec46d3640 1 -- 172.21.15.64:0/3000024370 wait complete. 2024-09-21T04:01:55.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:54 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:55.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:54 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:55.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:54 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:55.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:54 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:55.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:54 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:55.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:54 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:55.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:54 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:55.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:54 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:55.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:54 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:55.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:54 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:55.655 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph --format=json-pretty osd lspools 2024-09-21T04:01:55.933 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:01:56.173 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:55 smithi064 ceph-mon[31180]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:56.173 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:55 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:56.173 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:55 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:56.173 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:55 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3000024370' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:56.173 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:55 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:56.173 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:55 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:01:56.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:55 smithi119 ceph-mon[34811]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:56.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:55 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:56.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:55 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:56.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:55 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3000024370' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:56.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:55 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:56.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:55 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:01:56.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:55 smithi131 ceph-mon[35193]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:56.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:55 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:56.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:55 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:56.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:55 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3000024370' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:01:56.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:55 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:01:56.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:55 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:01:56.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.411+0000 7f5210077640 1 -- 172.21.15.64:0/1332527380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5208102a60 msgr2=0x7f5208102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:56.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.411+0000 7f5210077640 1 --2- 172.21.15.64:0/1332527380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5208102a60 0x7f5208102e70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5200009900 tx=0x7f520002a410 comp rx=0 tx=0).stop 2024-09-21T04:01:56.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.411+0000 7f5210077640 1 -- 172.21.15.64:0/1332527380 shutdown_connections 2024-09-21T04:01:56.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.411+0000 7f5210077640 1 --2- 172.21.15.64:0/1332527380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5208108cf0 0x7f520810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:56.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.411+0000 7f5210077640 1 --2- 172.21.15.64:0/1332527380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52081083a0 0x7f52081087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:56.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.411+0000 7f5210077640 1 --2- 172.21.15.64:0/1332527380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5208102a60 0x7f5208102e70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:56.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.411+0000 7f5210077640 1 -- 172.21.15.64:0/1332527380 >> 172.21.15.64:0/1332527380 conn(0x7f5208079730 msgr2=0x7f5208079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:56.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.411+0000 7f5210077640 1 -- 172.21.15.64:0/1332527380 shutdown_connections 2024-09-21T04:01:56.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.411+0000 7f5210077640 1 -- 172.21.15.64:0/1332527380 wait complete. 2024-09-21T04:01:56.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.412+0000 7f5210077640 1 Processor -- start 2024-09-21T04:01:56.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.412+0000 7f5210077640 1 -- start start 2024-09-21T04:01:56.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.412+0000 7f5210077640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5208102a60 0x7f520819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:56.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.412+0000 7f5210077640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52081083a0 0x7f520819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:56.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.412+0000 7f5210077640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5208108cf0 0x7f520819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:56.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.412+0000 7f5210077640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f52081129c0 con 0x7f5208108cf0 2024-09-21T04:01:56.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.412+0000 7f5210077640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5208112840 con 0x7f52081083a0 2024-09-21T04:01:56.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.412+0000 7f5210077640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5208112b40 con 0x7f5208102a60 2024-09-21T04:01:56.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.412+0000 7f520e614640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5208108cf0 0x7f520819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:56.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.412+0000 7f520e614640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5208108cf0 0x7f520819fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46622/0 (socket says 172.21.15.64:46622) 2024-09-21T04:01:56.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.412+0000 7f520e614640 1 -- 172.21.15.64:0/3313563308 learned_addr learned my addr 172.21.15.64:0/3313563308 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:01:56.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.412+0000 7f520de13640 1 --2- 172.21.15.64:0/3313563308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5208102a60 0x7f520819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:56.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.413+0000 7f520d612640 1 --2- 172.21.15.64:0/3313563308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52081083a0 0x7f520819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:56.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.413+0000 7f520e614640 1 -- 172.21.15.64:0/3313563308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5208102a60 msgr2=0x7f520819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:56.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.413+0000 7f520e614640 1 --2- 172.21.15.64:0/3313563308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5208102a60 0x7f520819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:56.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.413+0000 7f520e614640 1 -- 172.21.15.64:0/3313563308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52081083a0 msgr2=0x7f520819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:56.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.413+0000 7f520e614640 1 --2- 172.21.15.64:0/3313563308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52081083a0 0x7f520819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:56.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.413+0000 7f520e614640 1 -- 172.21.15.64:0/3313563308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52081a0430 con 0x7f5208108cf0 2024-09-21T04:01:56.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.413+0000 7f520e614640 1 --2- 172.21.15.64:0/3313563308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5208108cf0 0x7f520819fd00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f51f800cad0 tx=0x7f51f800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:56.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.413+0000 7f5206ffd640 1 -- 172.21.15.64:0/3313563308 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f800ae20 con 0x7f5208108cf0 2024-09-21T04:01:56.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.413+0000 7f5206ffd640 1 -- 172.21.15.64:0/3313563308 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51f8005020 con 0x7f5208108cf0 2024-09-21T04:01:56.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.413+0000 7f5210077640 1 -- 172.21.15.64:0/3313563308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52081a0720 con 0x7f5208108cf0 2024-09-21T04:01:56.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.413+0000 7f5206ffd640 1 -- 172.21.15.64:0/3313563308 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f800f6b0 con 0x7f5208108cf0 2024-09-21T04:01:56.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.413+0000 7f5210077640 1 -- 172.21.15.64:0/3313563308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52081a6e80 con 0x7f5208108cf0 2024-09-21T04:01:56.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.414+0000 7f5210077640 1 -- 172.21.15.64:0/3313563308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5208102e70 con 0x7f5208108cf0 2024-09-21T04:01:56.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.416+0000 7f5206ffd640 1 -- 172.21.15.64:0/3313563308 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f51f80051d0 con 0x7f5208108cf0 2024-09-21T04:01:56.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.416+0000 7f5206ffd640 1 --2- 172.21.15.64:0/3313563308 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f51f00b6560 0x7f51f00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:56.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.417+0000 7f520de13640 1 --2- 172.21.15.64:0/3313563308 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f51f00b6560 0x7f51f00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:56.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.417+0000 7f5206ffd640 1 -- 172.21.15.64:0/3313563308 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7f51f80e7010 con 0x7f5208108cf0 2024-09-21T04:01:56.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.417+0000 7f520de13640 1 --2- 172.21.15.64:0/3313563308 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f51f00b6560 0x7f51f00b8a70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5200031440 tx=0x7f5200005e80 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:56.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.421+0000 7f5206ffd640 1 -- 172.21.15.64:0/3313563308 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f51f8014030 con 0x7f5208108cf0 2024-09-21T04:01:56.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.623+0000 7f5210077640 1 -- 172.21.15.64:0/3313563308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f52081a7140 con 0x7f5208108cf0 2024-09-21T04:01:56.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.624+0000 7f5206ffd640 1 -- 172.21.15.64:0/3313563308 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v68) ==== 84+0+65 (secure 0 0 0) 0x7f51f80adc70 con 0x7f5208108cf0 2024-09-21T04:01:56.626 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:01:56.627 INFO:teuthology.orchestra.run.smithi064.stdout:[ 2024-09-21T04:01:56.627 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:01:56.627 INFO:teuthology.orchestra.run.smithi064.stdout: "poolnum": 1, 2024-09-21T04:01:56.627 INFO:teuthology.orchestra.run.smithi064.stdout: "poolname": ".mgr" 2024-09-21T04:01:56.627 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:01:56.627 INFO:teuthology.orchestra.run.smithi064.stdout:] 2024-09-21T04:01:56.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.626+0000 7f5210077640 1 -- 172.21.15.64:0/3313563308 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f51f00b6560 msgr2=0x7f51f00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:56.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.626+0000 7f5210077640 1 --2- 172.21.15.64:0/3313563308 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f51f00b6560 0x7f51f00b8a70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5200031440 tx=0x7f5200005e80 comp rx=0 tx=0).stop 2024-09-21T04:01:56.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.626+0000 7f5210077640 1 -- 172.21.15.64:0/3313563308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5208108cf0 msgr2=0x7f520819fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:56.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.626+0000 7f5210077640 1 --2- 172.21.15.64:0/3313563308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5208108cf0 0x7f520819fd00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f51f800cad0 tx=0x7f51f800cf90 comp rx=0 tx=0).stop 2024-09-21T04:01:56.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.626+0000 7f5210077640 1 -- 172.21.15.64:0/3313563308 shutdown_connections 2024-09-21T04:01:56.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.626+0000 7f5210077640 1 --2- 172.21.15.64:0/3313563308 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f51f00b6560 0x7f51f00b8a70 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:56.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.626+0000 7f5210077640 1 --2- 172.21.15.64:0/3313563308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5208108cf0 0x7f520819fd00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:56.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.626+0000 7f5210077640 1 --2- 172.21.15.64:0/3313563308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52081083a0 0x7f520819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:56.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.626+0000 7f5210077640 1 --2- 172.21.15.64:0/3313563308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5208102a60 0x7f520819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:56.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.626+0000 7f5210077640 1 -- 172.21.15.64:0/3313563308 >> 172.21.15.64:0/3313563308 conn(0x7f5208079730 msgr2=0x7f52081019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:56.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.626+0000 7f5210077640 1 -- 172.21.15.64:0/3313563308 shutdown_connections 2024-09-21T04:01:56.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:56.626+0000 7f5210077640 1 -- 172.21.15.64:0/3313563308 wait complete. 2024-09-21T04:01:57.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:56 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:57.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:56 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:01:57.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:56 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3313563308' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T04:01:57.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:56 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:57.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:56 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:01:57.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:56 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3313563308' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T04:01:57.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:56 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:57.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:56 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:01:57.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:56 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3313563308' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T04:01:57.376 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T04:01:57.376 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T04:01:57.969 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:01:58.219 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:57 smithi064 ceph-mon[31180]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:58.219 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:57 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:58.219 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:57 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:58.219 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:57 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:58.219 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:57 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:58.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:57 smithi119 ceph-mon[34811]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:58.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:57 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:58.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:57 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:58.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:57 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:58.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:57 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:58.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:57 smithi131 ceph-mon[35193]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:01:58.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:57 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:58.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:57 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:58.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:57 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:58.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:57 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:01:58.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.465+0000 7f6290c74640 1 -- 172.21.15.64:0/2380798723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f628c1083a0 msgr2=0x7f628c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:58.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.465+0000 7f6290c74640 1 --2- 172.21.15.64:0/2380798723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f628c1083a0 0x7f628c1087b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6280009900 tx=0x7f628002a410 comp rx=0 tx=0).stop 2024-09-21T04:01:58.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.466+0000 7f6290c74640 1 -- 172.21.15.64:0/2380798723 shutdown_connections 2024-09-21T04:01:58.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.466+0000 7f6290c74640 1 --2- 172.21.15.64:0/2380798723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f628c108cf0 0x7f628c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:58.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.466+0000 7f6290c74640 1 --2- 172.21.15.64:0/2380798723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f628c1083a0 0x7f628c1087b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:58.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.466+0000 7f6290c74640 1 --2- 172.21.15.64:0/2380798723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f628c102a60 0x7f628c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:58.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.466+0000 7f6290c74640 1 -- 172.21.15.64:0/2380798723 >> 172.21.15.64:0/2380798723 conn(0x7f628c079730 msgr2=0x7f628c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:58.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.466+0000 7f6290c74640 1 -- 172.21.15.64:0/2380798723 shutdown_connections 2024-09-21T04:01:58.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.466+0000 7f6290c74640 1 -- 172.21.15.64:0/2380798723 wait complete. 2024-09-21T04:01:58.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.466+0000 7f6290c74640 1 Processor -- start 2024-09-21T04:01:58.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.467+0000 7f6290c74640 1 -- start start 2024-09-21T04:01:58.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.467+0000 7f6290c74640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f628c102a60 0x7f628c19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:58.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.467+0000 7f6290c74640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f628c1083a0 0x7f628c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:58.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.467+0000 7f6290c74640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f628c108cf0 0x7f628c19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:58.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.467+0000 7f6290c74640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f628c1129a0 con 0x7f628c108cf0 2024-09-21T04:01:58.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.467+0000 7f6290c74640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f628c112820 con 0x7f628c102a60 2024-09-21T04:01:58.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.467+0000 7f6290c74640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f628c112b20 con 0x7f628c1083a0 2024-09-21T04:01:58.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.467+0000 7f628ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f628c108cf0 0x7f628c19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:58.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.467+0000 7f628ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f628c108cf0 0x7f628c19fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46632/0 (socket says 172.21.15.64:46632) 2024-09-21T04:01:58.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.467+0000 7f628ad9d640 1 -- 172.21.15.64:0/60964883 learned_addr learned my addr 172.21.15.64:0/60964883 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:01:58.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.467+0000 7f628a59c640 1 --2- 172.21.15.64:0/60964883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f628c102a60 0x7f628c19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:58.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.467+0000 7f6289d9b640 1 --2- 172.21.15.64:0/60964883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f628c1083a0 0x7f628c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:58.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.468+0000 7f628ad9d640 1 -- 172.21.15.64:0/60964883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f628c1083a0 msgr2=0x7f628c19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:58.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.468+0000 7f628ad9d640 1 --2- 172.21.15.64:0/60964883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f628c1083a0 0x7f628c19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:58.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.468+0000 7f628ad9d640 1 -- 172.21.15.64:0/60964883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f628c102a60 msgr2=0x7f628c19a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:58.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.468+0000 7f628ad9d640 1 --2- 172.21.15.64:0/60964883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f628c102a60 0x7f628c19a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:58.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.468+0000 7f628ad9d640 1 -- 172.21.15.64:0/60964883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628c1a0430 con 0x7f628c108cf0 2024-09-21T04:01:58.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.468+0000 7f628ad9d640 1 --2- 172.21.15.64:0/60964883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f628c108cf0 0x7f628c19fdc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f628400cce0 tx=0x7f628400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:58.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.468+0000 7f627b7fe640 1 -- 172.21.15.64:0/60964883 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628400f450 con 0x7f628c108cf0 2024-09-21T04:01:58.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.468+0000 7f6290c74640 1 -- 172.21.15.64:0/60964883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f628c1a0720 con 0x7f628c108cf0 2024-09-21T04:01:58.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.468+0000 7f627b7fe640 1 -- 172.21.15.64:0/60964883 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f628400faa0 con 0x7f628c108cf0 2024-09-21T04:01:58.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.468+0000 7f627b7fe640 1 -- 172.21.15.64:0/60964883 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6284018a80 con 0x7f628c108cf0 2024-09-21T04:01:58.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.468+0000 7f6290c74640 1 -- 172.21.15.64:0/60964883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f628c1a6f40 con 0x7f628c108cf0 2024-09-21T04:01:58.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.469+0000 7f6290c74640 1 -- 172.21.15.64:0/60964883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f628c102e70 con 0x7f628c108cf0 2024-09-21T04:01:58.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.471+0000 7f627b7fe640 1 -- 172.21.15.64:0/60964883 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f628400fc50 con 0x7f628c108cf0 2024-09-21T04:01:58.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.472+0000 7f627b7fe640 1 --2- 172.21.15.64:0/60964883 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f62740b6590 0x7f62740b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:01:58.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.472+0000 7f628a59c640 1 --2- 172.21.15.64:0/60964883 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f62740b6590 0x7f62740b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:01:58.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.472+0000 7f627b7fe640 1 -- 172.21.15.64:0/60964883 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7342+0+0 (secure 0 0 0) 0x7f62840e7890 con 0x7f628c108cf0 2024-09-21T04:01:58.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.472+0000 7f628a59c640 1 --2- 172.21.15.64:0/60964883 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f62740b6590 0x7f62740b8aa0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f627c009990 tx=0x7f627c009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:01:58.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.475+0000 7f627b7fe640 1 -- 172.21.15.64:0/60964883 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6284014030 con 0x7f628c108cf0 2024-09-21T04:01:58.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:58.722+0000 7f6290c74640 1 -- 172.21.15.64:0/60964883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f628c1a7110 con 0x7f628c108cf0 2024-09-21T04:01:59.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:59.721+0000 7f627b7fe640 1 -- 172.21.15.64:0/60964883 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v69) ==== 138+0+0 (secure 0 0 0) 0x7f62840ae4f0 con 0x7f628c108cf0 2024-09-21T04:01:59.724 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' created 2024-09-21T04:01:59.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:59.723+0000 7f6290c74640 1 -- 172.21.15.64:0/60964883 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f62740b6590 msgr2=0x7f62740b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:59.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:59.723+0000 7f6290c74640 1 --2- 172.21.15.64:0/60964883 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f62740b6590 0x7f62740b8aa0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f627c009990 tx=0x7f627c009110 comp rx=0 tx=0).stop 2024-09-21T04:01:59.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:59.723+0000 7f6290c74640 1 -- 172.21.15.64:0/60964883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f628c108cf0 msgr2=0x7f628c19fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:01:59.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:59.723+0000 7f6290c74640 1 --2- 172.21.15.64:0/60964883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f628c108cf0 0x7f628c19fdc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f628400cce0 tx=0x7f628400a620 comp rx=0 tx=0).stop 2024-09-21T04:01:59.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:59.723+0000 7f6290c74640 1 -- 172.21.15.64:0/60964883 shutdown_connections 2024-09-21T04:01:59.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:59.723+0000 7f6290c74640 1 --2- 172.21.15.64:0/60964883 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f62740b6590 0x7f62740b8aa0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:59.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:59.723+0000 7f6290c74640 1 --2- 172.21.15.64:0/60964883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f628c108cf0 0x7f628c19fdc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:59.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:59.723+0000 7f6290c74640 1 --2- 172.21.15.64:0/60964883 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f628c1083a0 0x7f628c19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:59.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:59.723+0000 7f6290c74640 1 --2- 172.21.15.64:0/60964883 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f628c102a60 0x7f628c19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:01:59.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:59.723+0000 7f6290c74640 1 -- 172.21.15.64:0/60964883 >> 172.21.15.64:0/60964883 conn(0x7f628c079730 msgr2=0x7f628c0ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:01:59.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:59.724+0000 7f6290c74640 1 -- 172.21.15.64:0/60964883 shutdown_connections 2024-09-21T04:01:59.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:01:59.724+0000 7f6290c74640 1 -- 172.21.15.64:0/60964883 wait complete. 2024-09-21T04:02:00.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:59 smithi119 ceph-mon[34811]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:02:00.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:59 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:02:00.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:59 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:02:00.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:59 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/60964883' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T04:02:00.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:59 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:02:00.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:59 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:02:00.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:01:59 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:02:00.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:59 smithi064 ceph-mon[31180]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:02:00.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:59 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:02:00.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:59 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:02:00.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:59 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/60964883' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T04:02:00.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:59 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:02:00.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:59 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:02:00.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:01:59 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:02:00.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:59 smithi131 ceph-mon[35193]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:02:00.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:59 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:02:00.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:59 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:02:00.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:59 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/60964883' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T04:02:00.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:59 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:02:00.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:59 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:02:00.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:01:59 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:02:00.371 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T04:02:00.630 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:00.996 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:00 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/60964883' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T04:02:00.996 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:00 smithi064 ceph-mon[31180]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T04:02:01.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:00 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/60964883' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T04:02:01.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:00 smithi119 ceph-mon[34811]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T04:02:01.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:00 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/60964883' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T04:02:01.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:00 smithi131 ceph-mon[35193]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T04:02:01.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.094+0000 7f1a90b7b640 1 -- 172.21.15.64:0/331445131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a8c102290 msgr2=0x7f1a8c1026a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:01.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.094+0000 7f1a90b7b640 1 --2- 172.21.15.64:0/331445131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a8c102290 0x7f1a8c1026a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009900 tx=0x7f1a7c02a410 comp rx=0 tx=0).stop 2024-09-21T04:02:01.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.095+0000 7f1a90b7b640 1 -- 172.21.15.64:0/331445131 shutdown_connections 2024-09-21T04:02:01.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.095+0000 7f1a90b7b640 1 --2- 172.21.15.64:0/331445131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a8c108520 0x7f1a8c10ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:01.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.095+0000 7f1a90b7b640 1 --2- 172.21.15.64:0/331445131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a8c107bd0 0x7f1a8c107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:01.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.095+0000 7f1a90b7b640 1 --2- 172.21.15.64:0/331445131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a8c102290 0x7f1a8c1026a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:01.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.095+0000 7f1a90b7b640 1 -- 172.21.15.64:0/331445131 >> 172.21.15.64:0/331445131 conn(0x7f1a8c079ea0 msgr2=0x7f1a8c07a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:01.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.095+0000 7f1a90b7b640 1 -- 172.21.15.64:0/331445131 shutdown_connections 2024-09-21T04:02:01.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.095+0000 7f1a90b7b640 1 -- 172.21.15.64:0/331445131 wait complete. 2024-09-21T04:02:01.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.095+0000 7f1a90b7b640 1 Processor -- start 2024-09-21T04:02:01.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.096+0000 7f1a90b7b640 1 -- start start 2024-09-21T04:02:01.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.096+0000 7f1a90b7b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a8c102290 0x7f1a8c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:01.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.096+0000 7f1a90b7b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a8c107bd0 0x7f1a8c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:01.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.096+0000 7f1a90b7b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a8c108520 0x7f1a8c19db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:01.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.096+0000 7f1a90b7b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c111960 con 0x7f1a8c102290 2024-09-21T04:02:01.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.096+0000 7f1a90b7b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c1117e0 con 0x7f1a8c107bd0 2024-09-21T04:02:01.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.096+0000 7f1a90b7b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c111ae0 con 0x7f1a8c108520 2024-09-21T04:02:01.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.096+0000 7f1a8a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a8c102290 0x7f1a8c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:01.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.096+0000 7f1a8a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a8c102290 0x7f1a8c198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46642/0 (socket says 172.21.15.64:46642) 2024-09-21T04:02:01.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.096+0000 7f1a8a59c640 1 -- 172.21.15.64:0/999864228 learned_addr learned my addr 172.21.15.64:0/999864228 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:01.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.096+0000 7f1a8ad9d640 1 --2- 172.21.15.64:0/999864228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a8c108520 0x7f1a8c19db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:01.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.096+0000 7f1a89d9b640 1 --2- 172.21.15.64:0/999864228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a8c107bd0 0x7f1a8c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:01.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.096+0000 7f1a8a59c640 1 -- 172.21.15.64:0/999864228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a8c108520 msgr2=0x7f1a8c19db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:01.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.096+0000 7f1a8a59c640 1 --2- 172.21.15.64:0/999864228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a8c108520 0x7f1a8c19db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:01.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.096+0000 7f1a8a59c640 1 -- 172.21.15.64:0/999864228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a8c107bd0 msgr2=0x7f1a8c1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:01.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.096+0000 7f1a8a59c640 1 --2- 172.21.15.64:0/999864228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a8c107bd0 0x7f1a8c1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:01.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.096+0000 7f1a8a59c640 1 -- 172.21.15.64:0/999864228 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8c19e2c0 con 0x7f1a8c102290 2024-09-21T04:02:01.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.096+0000 7f1a89d9b640 1 --2- 172.21.15.64:0/999864228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a8c107bd0 0x7f1a8c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:02:01.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.097+0000 7f1a8a59c640 1 --2- 172.21.15.64:0/999864228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a8c102290 0x7f1a8c198350 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c005940 tx=0x7f1a7c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:01.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.097+0000 7f1a7b7fe640 1 -- 172.21.15.64:0/999864228 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c03f030 con 0x7f1a8c102290 2024-09-21T04:02:01.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.097+0000 7f1a90b7b640 1 -- 172.21.15.64:0/999864228 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a8c19e550 con 0x7f1a8c102290 2024-09-21T04:02:01.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.097+0000 7f1a7b7fe640 1 -- 172.21.15.64:0/999864228 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a7c003aa0 con 0x7f1a8c102290 2024-09-21T04:02:01.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.097+0000 7f1a7b7fe640 1 -- 172.21.15.64:0/999864228 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c035070 con 0x7f1a8c102290 2024-09-21T04:02:01.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.097+0000 7f1a90b7b640 1 -- 172.21.15.64:0/999864228 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a8c1a4d10 con 0x7f1a8c102290 2024-09-21T04:02:01.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.098+0000 7f1a90b7b640 1 -- 172.21.15.64:0/999864228 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a8c1026a0 con 0x7f1a8c102290 2024-09-21T04:02:01.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.100+0000 7f1a7b7fe640 1 -- 172.21.15.64:0/999864228 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f1a7c042030 con 0x7f1a8c102290 2024-09-21T04:02:01.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.100+0000 7f1a7b7fe640 1 --2- 172.21.15.64:0/999864228 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f1a740b65e0 0x7f1a740b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:01.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.100+0000 7f1a89d9b640 1 --2- 172.21.15.64:0/999864228 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f1a740b65e0 0x7f1a740b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:01.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.101+0000 7f1a7b7fe640 1 -- 172.21.15.64:0/999864228 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7712+0+0 (secure 0 0 0) 0x7f1a7c107a90 con 0x7f1a8c102290 2024-09-21T04:02:01.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.101+0000 7f1a89d9b640 1 --2- 172.21.15.64:0/999864228 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f1a740b65e0 0x7f1a740b8af0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c107900 tx=0x7f1a80005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:01.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.104+0000 7f1a7b7fe640 1 -- 172.21.15.64:0/999864228 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1a7c0ce580 con 0x7f1a8c102290 2024-09-21T04:02:01.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.295+0000 7f1a90b7b640 1 -- 172.21.15.64:0/999864228 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1a8c10a2f0 con 0x7f1a8c102290 2024-09-21T04:02:01.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.734+0000 7f1a7b7fe640 1 -- 172.21.15.64:0/999864228 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v71) ==== 170+0+0 (secure 0 0 0) 0x7f1a7c0d2960 con 0x7f1a8c102290 2024-09-21T04:02:01.737 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' created 2024-09-21T04:02:01.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.737+0000 7f1a90b7b640 1 -- 172.21.15.64:0/999864228 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f1a740b65e0 msgr2=0x7f1a740b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:01.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.737+0000 7f1a90b7b640 1 --2- 172.21.15.64:0/999864228 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f1a740b65e0 0x7f1a740b8af0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c107900 tx=0x7f1a80005f90 comp rx=0 tx=0).stop 2024-09-21T04:02:01.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.737+0000 7f1a90b7b640 1 -- 172.21.15.64:0/999864228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a8c102290 msgr2=0x7f1a8c198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:01.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.737+0000 7f1a90b7b640 1 --2- 172.21.15.64:0/999864228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a8c102290 0x7f1a8c198350 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c005940 tx=0x7f1a7c004270 comp rx=0 tx=0).stop 2024-09-21T04:02:01.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.737+0000 7f1a90b7b640 1 -- 172.21.15.64:0/999864228 shutdown_connections 2024-09-21T04:02:01.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.737+0000 7f1a90b7b640 1 --2- 172.21.15.64:0/999864228 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f1a740b65e0 0x7f1a740b8af0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:01.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.737+0000 7f1a90b7b640 1 --2- 172.21.15.64:0/999864228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a8c108520 0x7f1a8c19db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:01.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.737+0000 7f1a90b7b640 1 --2- 172.21.15.64:0/999864228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a8c107bd0 0x7f1a8c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:01.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.737+0000 7f1a90b7b640 1 --2- 172.21.15.64:0/999864228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a8c102290 0x7f1a8c198350 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:01.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.737+0000 7f1a90b7b640 1 -- 172.21.15.64:0/999864228 >> 172.21.15.64:0/999864228 conn(0x7f1a8c079ea0 msgr2=0x7f1a8c105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:01.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.737+0000 7f1a90b7b640 1 -- 172.21.15.64:0/999864228 shutdown_connections 2024-09-21T04:02:01.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:01.737+0000 7f1a90b7b640 1 -- 172.21.15.64:0/999864228 wait complete. 2024-09-21T04:02:02.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:01 smithi119 ceph-mon[34811]: pgmap v253: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:02:02.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:01 smithi119 ceph-mon[34811]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T04:02:02.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:01 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/999864228' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T04:02:02.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:01 smithi064 ceph-mon[31180]: pgmap v253: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:02:02.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:01 smithi064 ceph-mon[31180]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T04:02:02.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:01 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/999864228' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T04:02:02.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:01 smithi131 ceph-mon[35193]: pgmap v253: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:02:02.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:01 smithi131 ceph-mon[35193]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T04:02:02.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:01 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/999864228' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T04:02:02.387 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T04:02:02.652 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:03.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:02 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/999864228' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T04:02:03.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:02 smithi119 ceph-mon[34811]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T04:02:03.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:02 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/999864228' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T04:02:03.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:02 smithi064 ceph-mon[31180]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T04:02:03.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:02 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/999864228' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T04:02:03.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:02 smithi131 ceph-mon[35193]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T04:02:03.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.117+0000 7f58ae6e3640 1 -- 172.21.15.64:0/2940173251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58a8108cf0 msgr2=0x7f58a810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:03.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.117+0000 7f58ae6e3640 1 --2- 172.21.15.64:0/2940173251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58a8108cf0 0x7f58a810f520 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f58a0009ca0 tx=0x7f58a002a8c0 comp rx=0 tx=0).stop 2024-09-21T04:02:03.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.117+0000 7f58ae6e3640 1 -- 172.21.15.64:0/2940173251 shutdown_connections 2024-09-21T04:02:03.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.117+0000 7f58ae6e3640 1 --2- 172.21.15.64:0/2940173251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58a8108cf0 0x7f58a810f520 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:03.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.117+0000 7f58ae6e3640 1 --2- 172.21.15.64:0/2940173251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58a81083a0 0x7f58a81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:03.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.117+0000 7f58ae6e3640 1 --2- 172.21.15.64:0/2940173251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58a8102a60 0x7f58a8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:03.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.117+0000 7f58ae6e3640 1 -- 172.21.15.64:0/2940173251 >> 172.21.15.64:0/2940173251 conn(0x7f58a8079730 msgr2=0x7f58a8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:03.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.117+0000 7f58ae6e3640 1 -- 172.21.15.64:0/2940173251 shutdown_connections 2024-09-21T04:02:03.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.117+0000 7f58ae6e3640 1 -- 172.21.15.64:0/2940173251 wait complete. 2024-09-21T04:02:03.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.118+0000 7f58ae6e3640 1 Processor -- start 2024-09-21T04:02:03.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.118+0000 7f58ae6e3640 1 -- start start 2024-09-21T04:02:03.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.118+0000 7f58ae6e3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58a8102a60 0x7f58a819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:03.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.118+0000 7f58ae6e3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58a81083a0 0x7f58a819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:03.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.118+0000 7f58ae6e3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58a8108cf0 0x7f58a819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:03.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.118+0000 7f58ae6e3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f58a8113a70 con 0x7f58a8102a60 2024-09-21T04:02:03.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.118+0000 7f58ae6e3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f58a81138f0 con 0x7f58a81083a0 2024-09-21T04:02:03.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.118+0000 7f58ae6e3640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f58a8113bf0 con 0x7f58a8108cf0 2024-09-21T04:02:03.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.118+0000 7f58a7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58a8102a60 0x7f58a819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:03.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.118+0000 7f58a7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58a8102a60 0x7f58a819a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33600/0 (socket says 172.21.15.64:33600) 2024-09-21T04:02:03.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.118+0000 7f58a7fff640 1 -- 172.21.15.64:0/2260438696 learned_addr learned my addr 172.21.15.64:0/2260438696 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:03.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.119+0000 7f589ffff640 1 --2- 172.21.15.64:0/2260438696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58a81083a0 0x7f58a819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:03.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.119+0000 7f58acc80640 1 --2- 172.21.15.64:0/2260438696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58a8108cf0 0x7f58a819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:03.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.119+0000 7f58a7fff640 1 -- 172.21.15.64:0/2260438696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58a8108cf0 msgr2=0x7f58a819fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:03.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.119+0000 7f58a7fff640 1 --2- 172.21.15.64:0/2260438696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58a8108cf0 0x7f58a819fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:03.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.119+0000 7f58a7fff640 1 -- 172.21.15.64:0/2260438696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58a81083a0 msgr2=0x7f58a819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:03.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.119+0000 7f58a7fff640 1 --2- 172.21.15.64:0/2260438696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58a81083a0 0x7f58a819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:03.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.119+0000 7f58a7fff640 1 -- 172.21.15.64:0/2260438696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a81a03b0 con 0x7f58a8102a60 2024-09-21T04:02:03.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.119+0000 7f58a7fff640 1 --2- 172.21.15.64:0/2260438696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58a8102a60 0x7f58a819a500 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f589800c920 tx=0x7f589800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:03.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.119+0000 7f58a5ffb640 1 -- 172.21.15.64:0/2260438696 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5898007c70 con 0x7f58a8102a60 2024-09-21T04:02:03.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.119+0000 7f58a5ffb640 1 -- 172.21.15.64:0/2260438696 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5898007e10 con 0x7f58a8102a60 2024-09-21T04:02:03.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.119+0000 7f58a5ffb640 1 -- 172.21.15.64:0/2260438696 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5898018820 con 0x7f58a8102a60 2024-09-21T04:02:03.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.119+0000 7f58ae6e3640 1 -- 172.21.15.64:0/2260438696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a81a06a0 con 0x7f58a8102a60 2024-09-21T04:02:03.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.119+0000 7f58ae6e3640 1 -- 172.21.15.64:0/2260438696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a81a6ec0 con 0x7f58a8102a60 2024-09-21T04:02:03.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.120+0000 7f58ae6e3640 1 -- 172.21.15.64:0/2260438696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58a8102e70 con 0x7f58a8102a60 2024-09-21T04:02:03.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.122+0000 7f58a5ffb640 1 -- 172.21.15.64:0/2260438696 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f589800f450 con 0x7f58a8102a60 2024-09-21T04:02:03.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.123+0000 7f58a5ffb640 1 --2- 172.21.15.64:0/2260438696 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f58940b6610 0x7f58940b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:03.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.123+0000 7f58a5ffb640 1 -- 172.21.15.64:0/2260438696 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8094+0+0 (secure 0 0 0) 0x7f58980e7690 con 0x7f58a8102a60 2024-09-21T04:02:03.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.123+0000 7f589ffff640 1 --2- 172.21.15.64:0/2260438696 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f58940b6610 0x7f58940b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:03.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.123+0000 7f589ffff640 1 --2- 172.21.15.64:0/2260438696 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f58940b6610 0x7f58940b8b20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5890005e00 tx=0x7f5890005c50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:03.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.126+0000 7f58a5ffb640 1 -- 172.21.15.64:0/2260438696 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5898014030 con 0x7f58a8102a60 2024-09-21T04:02:03.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.353+0000 7f58ae6e3640 1 -- 172.21.15.64:0/2260438696 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f58a819b630 con 0x7f58a8102a60 2024-09-21T04:02:03.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.742+0000 7f58a5ffb640 1 -- 172.21.15.64:0/2260438696 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f58980ae110 con 0x7f58a8102a60 2024-09-21T04:02:03.744 INFO:teuthology.orchestra.run.smithi064.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T04:02:03.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.744+0000 7f58ae6e3640 1 -- 172.21.15.64:0/2260438696 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f58940b6610 msgr2=0x7f58940b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:03.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.744+0000 7f58ae6e3640 1 --2- 172.21.15.64:0/2260438696 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f58940b6610 0x7f58940b8b20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5890005e00 tx=0x7f5890005c50 comp rx=0 tx=0).stop 2024-09-21T04:02:03.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.744+0000 7f58ae6e3640 1 -- 172.21.15.64:0/2260438696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58a8102a60 msgr2=0x7f58a819a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:03.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.744+0000 7f58ae6e3640 1 --2- 172.21.15.64:0/2260438696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58a8102a60 0x7f58a819a500 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f589800c920 tx=0x7f589800cde0 comp rx=0 tx=0).stop 2024-09-21T04:02:03.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.744+0000 7f58ae6e3640 1 -- 172.21.15.64:0/2260438696 shutdown_connections 2024-09-21T04:02:03.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.744+0000 7f58ae6e3640 1 --2- 172.21.15.64:0/2260438696 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f58940b6610 0x7f58940b8b20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:03.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.744+0000 7f58ae6e3640 1 --2- 172.21.15.64:0/2260438696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58a8108cf0 0x7f58a819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:03.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.744+0000 7f58ae6e3640 1 --2- 172.21.15.64:0/2260438696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58a81083a0 0x7f58a819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:03.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.744+0000 7f58ae6e3640 1 --2- 172.21.15.64:0/2260438696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58a8102a60 0x7f58a819a500 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:03.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.744+0000 7f58ae6e3640 1 -- 172.21.15.64:0/2260438696 >> 172.21.15.64:0/2260438696 conn(0x7f58a8079730 msgr2=0x7f58a8104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:03.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.745+0000 7f58ae6e3640 1 -- 172.21.15.64:0/2260438696 shutdown_connections 2024-09-21T04:02:03.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:03.745+0000 7f58ae6e3640 1 -- 172.21.15.64:0/2260438696 wait complete. 2024-09-21T04:02:04.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:03 smithi119 ceph-mon[34811]: pgmap v256: 97 pgs: 29 active+clean, 1 creating+activating, 9 creating+peering, 58 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:02:04.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:03 smithi119 ceph-mon[34811]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T04:02:04.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:03 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2260438696' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T04:02:04.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:03 smithi119 ceph-mon[34811]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T04:02:04.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:03 smithi064 ceph-mon[31180]: pgmap v256: 97 pgs: 29 active+clean, 1 creating+activating, 9 creating+peering, 58 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:02:04.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:03 smithi064 ceph-mon[31180]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T04:02:04.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:03 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2260438696' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T04:02:04.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:03 smithi064 ceph-mon[31180]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T04:02:04.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:03 smithi131 ceph-mon[35193]: pgmap v256: 97 pgs: 29 active+clean, 1 creating+activating, 9 creating+peering, 58 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:02:04.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:03 smithi131 ceph-mon[35193]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T04:02:04.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:03 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2260438696' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T04:02:04.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:03 smithi131 ceph-mon[35193]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T04:02:04.428 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T04:02:04.689 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:05.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:04 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2260438696' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T04:02:05.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:04 smithi131 ceph-mon[35193]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T04:02:05.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:04 smithi131 ceph-mon[35193]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T04:02:05.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:04 smithi131 ceph-mon[35193]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T04:02:05.065 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:04 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2260438696' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T04:02:05.065 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:04 smithi064 ceph-mon[31180]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T04:02:05.065 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:04 smithi064 ceph-mon[31180]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T04:02:05.065 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:04 smithi064 ceph-mon[31180]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T04:02:05.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.154+0000 7fa4a7da0640 1 -- 172.21.15.64:0/3349075787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4a0108d00 msgr2=0x7fa4a010f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:05.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.154+0000 7fa4a7da0640 1 --2- 172.21.15.64:0/3349075787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4a0108d00 0x7fa4a010f530 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa498009ca0 tx=0x7fa49802a8c0 comp rx=0 tx=0).stop 2024-09-21T04:02:05.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.155+0000 7fa4a7da0640 1 -- 172.21.15.64:0/3349075787 shutdown_connections 2024-09-21T04:02:05.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.155+0000 7fa4a7da0640 1 --2- 172.21.15.64:0/3349075787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4a0108d00 0x7fa4a010f530 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:05.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.155+0000 7fa4a7da0640 1 --2- 172.21.15.64:0/3349075787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4a01083b0 0x7fa4a01087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:05.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.155+0000 7fa4a7da0640 1 --2- 172.21.15.64:0/3349075787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4a0102a70 0x7fa4a0102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:05.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.155+0000 7fa4a7da0640 1 -- 172.21.15.64:0/3349075787 >> 172.21.15.64:0/3349075787 conn(0x7fa4a0079710 msgr2=0x7fa4a0079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:05.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.155+0000 7fa4a7da0640 1 -- 172.21.15.64:0/3349075787 shutdown_connections 2024-09-21T04:02:05.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.155+0000 7fa4a7da0640 1 -- 172.21.15.64:0/3349075787 wait complete. 2024-09-21T04:02:05.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.155+0000 7fa4a7da0640 1 Processor -- start 2024-09-21T04:02:05.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.155+0000 7fa4a7da0640 1 -- start start 2024-09-21T04:02:05.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.156+0000 7fa4a7da0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4a0102a70 0x7fa4a019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:05.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.156+0000 7fa4a7da0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4a01083b0 0x7fa4a019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:05.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.156+0000 7fa4a7da0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4a0108d00 0x7fa4a019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:05.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.156+0000 7fa4a7da0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a0113ae0 con 0x7fa4a0108d00 2024-09-21T04:02:05.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.156+0000 7fa4a7da0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a0113960 con 0x7fa4a01083b0 2024-09-21T04:02:05.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.156+0000 7fa4a7da0640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a0113c60 con 0x7fa4a0102a70 2024-09-21T04:02:05.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.156+0000 7fa4a633d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4a0108d00 0x7fa4a019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:05.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.156+0000 7fa4a633d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4a0108d00 0x7fa4a019fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33620/0 (socket says 172.21.15.64:33620) 2024-09-21T04:02:05.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.156+0000 7fa4a633d640 1 -- 172.21.15.64:0/3774502400 learned_addr learned my addr 172.21.15.64:0/3774502400 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:05.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.156+0000 7fa4a5b3c640 1 --2- 172.21.15.64:0/3774502400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4a0102a70 0x7fa4a019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:05.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.156+0000 7fa4a533b640 1 --2- 172.21.15.64:0/3774502400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4a01083b0 0x7fa4a019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:05.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.156+0000 7fa4a633d640 1 -- 172.21.15.64:0/3774502400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4a0102a70 msgr2=0x7fa4a019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:05.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.156+0000 7fa4a633d640 1 --2- 172.21.15.64:0/3774502400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4a0102a70 0x7fa4a019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:05.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.156+0000 7fa4a633d640 1 -- 172.21.15.64:0/3774502400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4a01083b0 msgr2=0x7fa4a019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:05.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.156+0000 7fa4a633d640 1 --2- 172.21.15.64:0/3774502400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4a01083b0 0x7fa4a019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:05.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.156+0000 7fa4a633d640 1 -- 172.21.15.64:0/3774502400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a01a03a0 con 0x7fa4a0108d00 2024-09-21T04:02:05.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.156+0000 7fa4a633d640 1 --2- 172.21.15.64:0/3774502400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4a0108d00 0x7fa4a019fd30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa498005cc0 tx=0x7fa49802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:05.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.156+0000 7fa492ffd640 1 -- 172.21.15.64:0/3774502400 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498003bf0 con 0x7fa4a0108d00 2024-09-21T04:02:05.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.157+0000 7fa4a7da0640 1 -- 172.21.15.64:0/3774502400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a01a0630 con 0x7fa4a0108d00 2024-09-21T04:02:05.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.157+0000 7fa492ffd640 1 -- 172.21.15.64:0/3774502400 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa498003d90 con 0x7fa4a0108d00 2024-09-21T04:02:05.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.157+0000 7fa492ffd640 1 -- 172.21.15.64:0/3774502400 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49802fb20 con 0x7fa4a0108d00 2024-09-21T04:02:05.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.157+0000 7fa4a7da0640 1 -- 172.21.15.64:0/3774502400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a01a6eb0 con 0x7fa4a0108d00 2024-09-21T04:02:05.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.158+0000 7fa4a7da0640 1 -- 172.21.15.64:0/3774502400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4a0103cf0 con 0x7fa4a0108d00 2024-09-21T04:02:05.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.159+0000 7fa492ffd640 1 -- 172.21.15.64:0/3774502400 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fa498004580 con 0x7fa4a0108d00 2024-09-21T04:02:05.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.160+0000 7fa492ffd640 1 --2- 172.21.15.64:0/3774502400 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa48c0b6610 0x7fa48c0b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:05.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.160+0000 7fa4a5b3c640 1 --2- 172.21.15.64:0/3774502400 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa48c0b6610 0x7fa48c0b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:05.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.160+0000 7fa492ffd640 1 -- 172.21.15.64:0/3774502400 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8194+0+0 (secure 0 0 0) 0x7fa4981086e0 con 0x7fa4a0108d00 2024-09-21T04:02:05.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.161+0000 7fa4a5b3c640 1 --2- 172.21.15.64:0/3774502400 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa48c0b6610 0x7fa48c0b8b20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa49c005fd0 tx=0x7fa49c00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:05.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.163+0000 7fa492ffd640 1 -- 172.21.15.64:0/3774502400 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa4980cef90 con 0x7fa4a0108d00 2024-09-21T04:02:05.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:04 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2260438696' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T04:02:05.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:04 smithi119 ceph-mon[34811]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T04:02:05.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:04 smithi119 ceph-mon[34811]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T04:02:05.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:04 smithi119 ceph-mon[34811]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T04:02:05.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.354+0000 7fa4a7da0640 1 -- 172.21.15.64:0/3774502400 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4a019b620 con 0x7fa4a0108d00 2024-09-21T04:02:05.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.355+0000 7fa492ffd640 1 -- 172.21.15.64:0/3774502400 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+22620 (secure 0 0 0) 0x7fa4980098b0 con 0x7fa4a0108d00 2024-09-21T04:02:05.358 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:02:05.358 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":73,"fsid":"e7c927ba-77cc-11ef-baf5-efdc52797490","created":"2024-09-21T03:53:41.215073+0000","modified":"2024-09-21T04:02:03.354789+0000","last_up_change":"2024-09-21T04:01:11.986949+0000","last_in_change":"2024-09-21T04:00:48.707791+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:57:24.854438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T04:01:58.728355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T04:02:01.300385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb947c10-98e6-4e3a-8bbd-8753951d870b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6803","nonce":3692023035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6805","nonce":3692023035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6809","nonce":3692023035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6807","nonce":3692023035}]},"public_addr":"172.21.15.64:6803/3692023035","cluster_addr":"172.21.15.64:6805/3692023035","heartbeat_back_addr":"172.21.15.64:6809/3692023035","heartbeat_front_addr":"172.21.15.64:6807/3692023035","state":["exists","up"]},{"osd":1,"uuid":"29b8c92d-82fa-4175-b3e2-711917b7f8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6801","nonce":3363415998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6803","nonce":3363415998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6807","nonce":3363415998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6805","nonce":3363415998}]},"public_addr":"172.21.15.119:6801/3363415998","cluster_addr":"172.21.15.119:6803/3363415998","heartbeat_back_addr":"172.21.15.119:6807/3363415998","heartbeat_front_addr":"172.21.15.119:6805/3363415998","state":["exists","up"]},{"osd":2,"uuid":"a2273fe8-df55-40f7-afda-74ab09c52c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6801","nonce":3422924179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6803","nonce":3422924179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6807","nonce":3422924179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6805","nonce":3422924179}]},"public_addr":"172.21.15.131:6801/3422924179","cluster_addr":"172.21.15.131:6803/3422924179","heartbeat_back_addr":"172.21.15.131:6807/3422924179","heartbeat_front_addr":"172.21.15.131:6805/3422924179","state":["exists","up"]},{"osd":3,"uuid":"f1634064-b108-41d2-a78f-c33607b08a70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6811","nonce":2022952729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6813","nonce":2022952729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6817","nonce":2022952729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6815","nonce":2022952729}]},"public_addr":"172.21.15.64:6811/2022952729","cluster_addr":"172.21.15.64:6813/2022952729","heartbeat_back_addr":"172.21.15.64:6817/2022952729","heartbeat_front_addr":"172.21.15.64:6815/2022952729","state":["exists","up"]},{"osd":4,"uuid":"75b81a8d-fa52-411a-bfda-aa69bb70a20f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6809","nonce":323030958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6811","nonce":323030958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6815","nonce":323030958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6813","nonce":323030958}]},"public_addr":"172.21.15.119:6809/323030958","cluster_addr":"172.21.15.119:6811/323030958","heartbeat_back_addr":"172.21.15.119:6815/323030958","heartbeat_front_addr":"172.21.15.119:6813/323030958","state":["exists","up"]},{"osd":5,"uuid":"0e4895b5-42e4-4b2c-a9c3-692f287cb841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6809","nonce":3021788088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6811","nonce":3021788088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6815","nonce":3021788088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6813","nonce":3021788088}]},"public_addr":"172.21.15.131:6809/3021788088","cluster_addr":"172.21.15.131:6811/3021788088","heartbeat_back_addr":"172.21.15.131:6815/3021788088","heartbeat_front_addr":"172.21.15.131:6813/3021788088","state":["exists","up"]},{"osd":6,"uuid":"1fa5c2b7-bce0-43a1-a26b-d029eb3731fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6819","nonce":1913824324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6821","nonce":1913824324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6825","nonce":1913824324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6823","nonce":1913824324}]},"public_addr":"172.21.15.64:6819/1913824324","cluster_addr":"172.21.15.64:6821/1913824324","heartbeat_back_addr":"172.21.15.64:6825/1913824324","heartbeat_front_addr":"172.21.15.64:6823/1913824324","state":["exists","up"]},{"osd":7,"uuid":"c9707260-b581-4a21-b380-f37e16959b02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6817","nonce":2567568166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6819","nonce":2567568166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6823","nonce":2567568166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6821","nonce":2567568166}]},"public_addr":"172.21.15.119:6817/2567568166","cluster_addr":"172.21.15.119:6819/2567568166","heartbeat_back_addr":"172.21.15.119:6823/2567568166","heartbeat_front_addr":"172.21.15.119:6821/2567568166","state":["exists","up"]},{"osd":8,"uuid":"2dbf8ef2-fc9c-4c6b-bfea-003f41ef191e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6817","nonce":199192217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6819","nonce":199192217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6823","nonce":199192217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6821","nonce":199192217}]},"public_addr":"172.21.15.131:6817/199192217","cluster_addr":"172.21.15.131:6819/199192217","heartbeat_back_addr":"172.21.15.131:6823/199192217","heartbeat_front_addr":"172.21.15.131:6821/199192217","state":["exists","up"]},{"osd":9,"uuid":"96f769e5-4783-4558-adfa-4bfaf9da9e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6827","nonce":3309038990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6829","nonce":3309038990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6833","nonce":3309038990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6831","nonce":3309038990}]},"public_addr":"172.21.15.64:6827/3309038990","cluster_addr":"172.21.15.64:6829/3309038990","heartbeat_back_addr":"172.21.15.64:6833/3309038990","heartbeat_front_addr":"172.21.15.64:6831/3309038990","state":["exists","up"]},{"osd":10,"uuid":"13c1ef5c-2299-440c-af5a-a650b47f4b75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6825","nonce":1460844681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6827","nonce":1460844681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6831","nonce":1460844681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6829","nonce":1460844681}]},"public_addr":"172.21.15.119:6825/1460844681","cluster_addr":"172.21.15.119:6827/1460844681","heartbeat_back_addr":"172.21.15.119:6831/1460844681","heartbeat_front_addr":"172.21.15.119:6829/1460844681","state":["exists","up"]},{"osd":11,"uuid":"c26a5ef9-8585-404e-a54a-852424c4d6fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6825","nonce":2294059084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6827","nonce":2294059084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6831","nonce":2294059084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6829","nonce":2294059084}]},"public_addr":"172.21.15.131:6825/2294059084","cluster_addr":"172.21.15.131:6827/2294059084","heartbeat_back_addr":"172.21.15.131:6831/2294059084","heartbeat_front_addr":"172.21.15.131:6829/2294059084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:56:33.141102+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:56:58.883062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:57:22.641789+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:57:48.705790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:58:12.871514+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:58:37.893118+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:03.620828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:29.446179+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:55.002978+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:00:19.987463+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:00:45.364158+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:01:10.228248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/3554471670":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6801/3786213009":"2024-09-22T03:55:28.387426+0000","172.21.15.64:0/2562856406":"2024-09-22T03:54:27.180100+0000","172.21.15.64:0/2603015699":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6800/1209932961":"2024-09-22T03:54:02.562380+0000","172.21.15.64:6800/3786213009":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6801/1209932961":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/3407344596":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1233731831":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1686887701":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1598568021":"2024-09-22T03:54:27.180100+0000","172.21.15.64:6800/2013929312":"2024-09-22T03:54:27.180100+0000","172.21.15.64:6801/2013929312":"2024-09-22T03:54:27.180100+0000","172.21.15.64:0/3652036144":"2024-09-22T03:55:28.387426+0000","172.21.15.64:0/105454703":"2024-09-22T03:54:27.180100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T04:02:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.357+0000 7fa4a7da0640 1 -- 172.21.15.64:0/3774502400 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa48c0b6610 msgr2=0x7fa48c0b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.357+0000 7fa4a7da0640 1 --2- 172.21.15.64:0/3774502400 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa48c0b6610 0x7fa48c0b8b20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa49c005fd0 tx=0x7fa49c00b390 comp rx=0 tx=0).stop 2024-09-21T04:02:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.357+0000 7fa4a7da0640 1 -- 172.21.15.64:0/3774502400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4a0108d00 msgr2=0x7fa4a019fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.357+0000 7fa4a7da0640 1 --2- 172.21.15.64:0/3774502400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4a0108d00 0x7fa4a019fd30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa498005cc0 tx=0x7fa49802f380 comp rx=0 tx=0).stop 2024-09-21T04:02:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.357+0000 7fa4a7da0640 1 -- 172.21.15.64:0/3774502400 shutdown_connections 2024-09-21T04:02:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.357+0000 7fa4a7da0640 1 --2- 172.21.15.64:0/3774502400 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa48c0b6610 0x7fa48c0b8b20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.357+0000 7fa4a7da0640 1 --2- 172.21.15.64:0/3774502400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa4a0108d00 0x7fa4a019fd30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.357+0000 7fa4a7da0640 1 --2- 172.21.15.64:0/3774502400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4a01083b0 0x7fa4a019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.357+0000 7fa4a7da0640 1 --2- 172.21.15.64:0/3774502400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa4a0102a70 0x7fa4a019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.357+0000 7fa4a7da0640 1 -- 172.21.15.64:0/3774502400 >> 172.21.15.64:0/3774502400 conn(0x7fa4a0079710 msgr2=0x7fa4a01043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.357+0000 7fa4a7da0640 1 -- 172.21.15.64:0/3774502400 shutdown_connections 2024-09-21T04:02:05.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:05.358+0000 7fa4a7da0640 1 -- 172.21.15.64:0/3774502400 wait complete. 2024-09-21T04:02:05.987 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:05 smithi064 ceph-mon[31180]: pgmap v259: 97 pgs: 29 active+clean, 1 creating+activating, 16 creating+peering, 51 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:02:05.987 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:05 smithi064 ceph-mon[31180]: mds.? [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] up:active 2024-09-21T04:02:05.987 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:05 smithi064 ceph-mon[31180]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T04:02:05.987 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:05 smithi064 ceph-mon[31180]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T04:02:05.987 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:05 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3774502400' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:02:06.016 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T04:02:06.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:05 smithi131 ceph-mon[35193]: pgmap v259: 97 pgs: 29 active+clean, 1 creating+activating, 16 creating+peering, 51 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:02:06.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:05 smithi131 ceph-mon[35193]: mds.? [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] up:active 2024-09-21T04:02:06.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:05 smithi131 ceph-mon[35193]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T04:02:06.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:05 smithi131 ceph-mon[35193]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T04:02:06.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:05 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3774502400' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:02:06.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:05 smithi119 ceph-mon[34811]: pgmap v259: 97 pgs: 29 active+clean, 1 creating+activating, 16 creating+peering, 51 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:02:06.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:05 smithi119 ceph-mon[34811]: mds.? [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] up:active 2024-09-21T04:02:06.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:05 smithi119 ceph-mon[34811]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T04:02:06.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:05 smithi119 ceph-mon[34811]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T04:02:06.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:05 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3774502400' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:02:06.272 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:06.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.729+0000 7f5a1216f640 1 -- 172.21.15.64:0/3438071141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0c107c40 msgr2=0x7f5a0c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:06.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.729+0000 7f5a1216f640 1 --2- 172.21.15.64:0/3438071141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0c107c40 0x7f5a0c10f520 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5a04009ca0 tx=0x7f5a0402a8c0 comp rx=0 tx=0).stop 2024-09-21T04:02:06.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.729+0000 7f5a1216f640 1 -- 172.21.15.64:0/3438071141 shutdown_connections 2024-09-21T04:02:06.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.729+0000 7f5a1216f640 1 --2- 172.21.15.64:0/3438071141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0c107c40 0x7f5a0c10f520 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:06.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.729+0000 7f5a1216f640 1 --2- 172.21.15.64:0/3438071141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a0c1072f0 0x7f5a0c107700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:06.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.729+0000 7f5a1216f640 1 --2- 172.21.15.64:0/3438071141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a0c108600 0x7f5a0c108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:06.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.729+0000 7f5a1216f640 1 -- 172.21.15.64:0/3438071141 >> 172.21.15.64:0/3438071141 conn(0x7f5a0c077800 msgr2=0x7f5a0c077c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:06.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.729+0000 7f5a1216f640 1 -- 172.21.15.64:0/3438071141 shutdown_connections 2024-09-21T04:02:06.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.729+0000 7f5a1216f640 1 -- 172.21.15.64:0/3438071141 wait complete. 2024-09-21T04:02:06.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.729+0000 7f5a1216f640 1 Processor -- start 2024-09-21T04:02:06.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.730+0000 7f5a1216f640 1 -- start start 2024-09-21T04:02:06.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.730+0000 7f5a1216f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0c1072f0 0x7f5a0c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:06.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.730+0000 7f5a1216f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a0c107c40 0x7f5a0c19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:06.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.730+0000 7f5a1216f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a0c108600 0x7f5a0c19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:06.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.730+0000 7f5a1216f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c113ad0 con 0x7f5a0c1072f0 2024-09-21T04:02:06.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.730+0000 7f5a1216f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c113950 con 0x7f5a0c107c40 2024-09-21T04:02:06.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.730+0000 7f5a1216f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c113c50 con 0x7f5a0c108600 2024-09-21T04:02:06.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.730+0000 7f5a0b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0c1072f0 0x7f5a0c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:06.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.730+0000 7f5a0b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0c1072f0 0x7f5a0c19a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33634/0 (socket says 172.21.15.64:33634) 2024-09-21T04:02:06.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.730+0000 7f5a0b7fe640 1 -- 172.21.15.64:0/1484165849 learned_addr learned my addr 172.21.15.64:0/1484165849 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:06.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.730+0000 7f5a0affd640 1 --2- 172.21.15.64:0/1484165849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a0c107c40 0x7f5a0c19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:06.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.730+0000 7f5a0bfff640 1 --2- 172.21.15.64:0/1484165849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a0c108600 0x7f5a0c19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:06.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.731+0000 7f5a0b7fe640 1 -- 172.21.15.64:0/1484165849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a0c108600 msgr2=0x7f5a0c19fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:06.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.731+0000 7f5a0b7fe640 1 --2- 172.21.15.64:0/1484165849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a0c108600 0x7f5a0c19fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:06.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.731+0000 7f5a0b7fe640 1 -- 172.21.15.64:0/1484165849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a0c107c40 msgr2=0x7f5a0c19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:06.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.731+0000 7f5a0b7fe640 1 --2- 172.21.15.64:0/1484165849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a0c107c40 0x7f5a0c19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:06.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.731+0000 7f5a0b7fe640 1 -- 172.21.15.64:0/1484165849 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a0c1a0390 con 0x7f5a0c1072f0 2024-09-21T04:02:06.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.731+0000 7f5a0b7fe640 1 --2- 172.21.15.64:0/1484165849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0c1072f0 0x7f5a0c19a420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00d950 tx=0x7f59fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:06.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.731+0000 7f5a08ff9640 1 -- 172.21.15.64:0/1484165849 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59fc017070 con 0x7f5a0c1072f0 2024-09-21T04:02:06.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.731+0000 7f5a08ff9640 1 -- 172.21.15.64:0/1484165849 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59fc009c30 con 0x7f5a0c1072f0 2024-09-21T04:02:06.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.731+0000 7f5a1216f640 1 -- 172.21.15.64:0/1484165849 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a0c1a0680 con 0x7f5a0c1072f0 2024-09-21T04:02:06.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.731+0000 7f5a08ff9640 1 -- 172.21.15.64:0/1484165849 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59fc00f620 con 0x7f5a0c1072f0 2024-09-21T04:02:06.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.731+0000 7f5a1216f640 1 -- 172.21.15.64:0/1484165849 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a0c1a6de0 con 0x7f5a0c1072f0 2024-09-21T04:02:06.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.732+0000 7f5a1216f640 1 -- 172.21.15.64:0/1484165849 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a0c107700 con 0x7f5a0c1072f0 2024-09-21T04:02:06.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.734+0000 7f5a08ff9640 1 -- 172.21.15.64:0/1484165849 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f59fc005030 con 0x7f5a0c1072f0 2024-09-21T04:02:06.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.735+0000 7f5a08ff9640 1 --2- 172.21.15.64:0/1484165849 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f59f40b6560 0x7f59f40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:06.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.735+0000 7f5a0affd640 1 --2- 172.21.15.64:0/1484165849 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f59f40b6560 0x7f59f40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:06.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.735+0000 7f5a08ff9640 1 -- 172.21.15.64:0/1484165849 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8194+0+0 (secure 0 0 0) 0x7f59fc0e6f50 con 0x7f5a0c1072f0 2024-09-21T04:02:06.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.735+0000 7f5a0affd640 1 --2- 172.21.15.64:0/1484165849 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f59f40b6560 0x7f59f40b8a70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c107190 tx=0x7f5a0000b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:06.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.738+0000 7f5a08ff9640 1 -- 172.21.15.64:0/1484165849 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f59fc0ad800 con 0x7f5a0c1072f0 2024-09-21T04:02:06.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.930+0000 7f5a1216f640 1 -- 172.21.15.64:0/1484165849 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a0c19b550 con 0x7f5a0c1072f0 2024-09-21T04:02:06.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.932+0000 7f5a08ff9640 1 -- 172.21.15.64:0/1484165849 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+22620 (secure 0 0 0) 0x7f59fc009de0 con 0x7f5a0c1072f0 2024-09-21T04:02:06.935 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:02:06.936 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":73,"fsid":"e7c927ba-77cc-11ef-baf5-efdc52797490","created":"2024-09-21T03:53:41.215073+0000","modified":"2024-09-21T04:02:03.354789+0000","last_up_change":"2024-09-21T04:01:11.986949+0000","last_in_change":"2024-09-21T04:00:48.707791+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:57:24.854438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T04:01:58.728355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T04:02:01.300385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb947c10-98e6-4e3a-8bbd-8753951d870b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6803","nonce":3692023035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6805","nonce":3692023035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6809","nonce":3692023035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6807","nonce":3692023035}]},"public_addr":"172.21.15.64:6803/3692023035","cluster_addr":"172.21.15.64:6805/3692023035","heartbeat_back_addr":"172.21.15.64:6809/3692023035","heartbeat_front_addr":"172.21.15.64:6807/3692023035","state":["exists","up"]},{"osd":1,"uuid":"29b8c92d-82fa-4175-b3e2-711917b7f8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6801","nonce":3363415998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6803","nonce":3363415998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6807","nonce":3363415998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6805","nonce":3363415998}]},"public_addr":"172.21.15.119:6801/3363415998","cluster_addr":"172.21.15.119:6803/3363415998","heartbeat_back_addr":"172.21.15.119:6807/3363415998","heartbeat_front_addr":"172.21.15.119:6805/3363415998","state":["exists","up"]},{"osd":2,"uuid":"a2273fe8-df55-40f7-afda-74ab09c52c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6801","nonce":3422924179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6803","nonce":3422924179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6807","nonce":3422924179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6805","nonce":3422924179}]},"public_addr":"172.21.15.131:6801/3422924179","cluster_addr":"172.21.15.131:6803/3422924179","heartbeat_back_addr":"172.21.15.131:6807/3422924179","heartbeat_front_addr":"172.21.15.131:6805/3422924179","state":["exists","up"]},{"osd":3,"uuid":"f1634064-b108-41d2-a78f-c33607b08a70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6811","nonce":2022952729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6813","nonce":2022952729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6817","nonce":2022952729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6815","nonce":2022952729}]},"public_addr":"172.21.15.64:6811/2022952729","cluster_addr":"172.21.15.64:6813/2022952729","heartbeat_back_addr":"172.21.15.64:6817/2022952729","heartbeat_front_addr":"172.21.15.64:6815/2022952729","state":["exists","up"]},{"osd":4,"uuid":"75b81a8d-fa52-411a-bfda-aa69bb70a20f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6809","nonce":323030958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6811","nonce":323030958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6815","nonce":323030958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6813","nonce":323030958}]},"public_addr":"172.21.15.119:6809/323030958","cluster_addr":"172.21.15.119:6811/323030958","heartbeat_back_addr":"172.21.15.119:6815/323030958","heartbeat_front_addr":"172.21.15.119:6813/323030958","state":["exists","up"]},{"osd":5,"uuid":"0e4895b5-42e4-4b2c-a9c3-692f287cb841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6809","nonce":3021788088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6811","nonce":3021788088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6815","nonce":3021788088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6813","nonce":3021788088}]},"public_addr":"172.21.15.131:6809/3021788088","cluster_addr":"172.21.15.131:6811/3021788088","heartbeat_back_addr":"172.21.15.131:6815/3021788088","heartbeat_front_addr":"172.21.15.131:6813/3021788088","state":["exists","up"]},{"osd":6,"uuid":"1fa5c2b7-bce0-43a1-a26b-d029eb3731fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6819","nonce":1913824324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6821","nonce":1913824324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6825","nonce":1913824324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6823","nonce":1913824324}]},"public_addr":"172.21.15.64:6819/1913824324","cluster_addr":"172.21.15.64:6821/1913824324","heartbeat_back_addr":"172.21.15.64:6825/1913824324","heartbeat_front_addr":"172.21.15.64:6823/1913824324","state":["exists","up"]},{"osd":7,"uuid":"c9707260-b581-4a21-b380-f37e16959b02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6817","nonce":2567568166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6819","nonce":2567568166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6823","nonce":2567568166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6821","nonce":2567568166}]},"public_addr":"172.21.15.119:6817/2567568166","cluster_addr":"172.21.15.119:6819/2567568166","heartbeat_back_addr":"172.21.15.119:6823/2567568166","heartbeat_front_addr":"172.21.15.119:6821/2567568166","state":["exists","up"]},{"osd":8,"uuid":"2dbf8ef2-fc9c-4c6b-bfea-003f41ef191e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6817","nonce":199192217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6819","nonce":199192217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6823","nonce":199192217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6821","nonce":199192217}]},"public_addr":"172.21.15.131:6817/199192217","cluster_addr":"172.21.15.131:6819/199192217","heartbeat_back_addr":"172.21.15.131:6823/199192217","heartbeat_front_addr":"172.21.15.131:6821/199192217","state":["exists","up"]},{"osd":9,"uuid":"96f769e5-4783-4558-adfa-4bfaf9da9e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6827","nonce":3309038990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6829","nonce":3309038990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6833","nonce":3309038990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6831","nonce":3309038990}]},"public_addr":"172.21.15.64:6827/3309038990","cluster_addr":"172.21.15.64:6829/3309038990","heartbeat_back_addr":"172.21.15.64:6833/3309038990","heartbeat_front_addr":"172.21.15.64:6831/3309038990","state":["exists","up"]},{"osd":10,"uuid":"13c1ef5c-2299-440c-af5a-a650b47f4b75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6825","nonce":1460844681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6827","nonce":1460844681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6831","nonce":1460844681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6829","nonce":1460844681}]},"public_addr":"172.21.15.119:6825/1460844681","cluster_addr":"172.21.15.119:6827/1460844681","heartbeat_back_addr":"172.21.15.119:6831/1460844681","heartbeat_front_addr":"172.21.15.119:6829/1460844681","state":["exists","up"]},{"osd":11,"uuid":"c26a5ef9-8585-404e-a54a-852424c4d6fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6825","nonce":2294059084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6827","nonce":2294059084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6831","nonce":2294059084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6829","nonce":2294059084}]},"public_addr":"172.21.15.131:6825/2294059084","cluster_addr":"172.21.15.131:6827/2294059084","heartbeat_back_addr":"172.21.15.131:6831/2294059084","heartbeat_front_addr":"172.21.15.131:6829/2294059084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:56:33.141102+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:56:58.883062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:57:22.641789+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:57:48.705790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:58:12.871514+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:58:37.893118+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:03.620828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:29.446179+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:55.002978+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:00:19.987463+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:00:45.364158+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:01:10.228248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/3554471670":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6801/3786213009":"2024-09-22T03:55:28.387426+0000","172.21.15.64:0/2562856406":"2024-09-22T03:54:27.180100+0000","172.21.15.64:0/2603015699":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6800/1209932961":"2024-09-22T03:54:02.562380+0000","172.21.15.64:6800/3786213009":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6801/1209932961":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/3407344596":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1233731831":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1686887701":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1598568021":"2024-09-22T03:54:27.180100+0000","172.21.15.64:6800/2013929312":"2024-09-22T03:54:27.180100+0000","172.21.15.64:6801/2013929312":"2024-09-22T03:54:27.180100+0000","172.21.15.64:0/3652036144":"2024-09-22T03:55:28.387426+0000","172.21.15.64:0/105454703":"2024-09-22T03:54:27.180100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T04:02:06.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.934+0000 7f5a1216f640 1 -- 172.21.15.64:0/1484165849 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f59f40b6560 msgr2=0x7f59f40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:06.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.934+0000 7f5a1216f640 1 --2- 172.21.15.64:0/1484165849 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f59f40b6560 0x7f59f40b8a70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c107190 tx=0x7f5a0000b390 comp rx=0 tx=0).stop 2024-09-21T04:02:06.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.934+0000 7f5a1216f640 1 -- 172.21.15.64:0/1484165849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0c1072f0 msgr2=0x7f5a0c19a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:06.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.934+0000 7f5a1216f640 1 --2- 172.21.15.64:0/1484165849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0c1072f0 0x7f5a0c19a420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00d950 tx=0x7f59fc00de10 comp rx=0 tx=0).stop 2024-09-21T04:02:06.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.934+0000 7f5a1216f640 1 -- 172.21.15.64:0/1484165849 shutdown_connections 2024-09-21T04:02:06.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.934+0000 7f5a1216f640 1 --2- 172.21.15.64:0/1484165849 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f59f40b6560 0x7f59f40b8a70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:06.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.934+0000 7f5a1216f640 1 --2- 172.21.15.64:0/1484165849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a0c108600 0x7f5a0c19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:06.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.934+0000 7f5a1216f640 1 --2- 172.21.15.64:0/1484165849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a0c107c40 0x7f5a0c19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:06.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.934+0000 7f5a1216f640 1 --2- 172.21.15.64:0/1484165849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a0c1072f0 0x7f5a0c19a420 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:06.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.934+0000 7f5a1216f640 1 -- 172.21.15.64:0/1484165849 >> 172.21.15.64:0/1484165849 conn(0x7f5a0c077800 msgr2=0x7f5a0c1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:06.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.934+0000 7f5a1216f640 1 -- 172.21.15.64:0/1484165849 shutdown_connections 2024-09-21T04:02:06.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:06.935+0000 7f5a1216f640 1 -- 172.21.15.64:0/1484165849 wait complete. 2024-09-21T04:02:07.653 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs set cephfs standby_count_wanted 0 2024-09-21T04:02:07.916 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:08.013 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:07 smithi064 ceph-mon[31180]: pgmap v260: 97 pgs: 80 active+clean, 13 creating+peering, 4 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2024-09-21T04:02:08.014 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:07 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1484165849' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:02:08.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:07 smithi131 ceph-mon[35193]: pgmap v260: 97 pgs: 80 active+clean, 13 creating+peering, 4 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2024-09-21T04:02:08.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:07 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1484165849' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:02:08.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:07 smithi119 ceph-mon[34811]: pgmap v260: 97 pgs: 80 active+clean, 13 creating+peering, 4 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2024-09-21T04:02:08.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:07 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1484165849' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:02:08.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.376+0000 7f2dd9280640 1 -- 172.21.15.64:0/1977658104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd4104a30 msgr2=0x7f2dd4104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:08.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.376+0000 7f2dd9280640 1 --2- 172.21.15.64:0/1977658104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd4104a30 0x7f2dd4104e40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4009900 tx=0x7f2dc402a410 comp rx=0 tx=0).stop 2024-09-21T04:02:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.376+0000 7f2dd9280640 1 -- 172.21.15.64:0/1977658104 shutdown_connections 2024-09-21T04:02:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.376+0000 7f2dd9280640 1 --2- 172.21.15.64:0/1977658104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2dd410de60 0x7f2dd4110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.376+0000 7f2dd9280640 1 --2- 172.21.15.64:0/1977658104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dd4105380 0x7f2dd410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.377+0000 7f2dd9280640 1 --2- 172.21.15.64:0/1977658104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd4104a30 0x7f2dd4104e40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.377+0000 7f2dd9280640 1 -- 172.21.15.64:0/1977658104 >> 172.21.15.64:0/1977658104 conn(0x7f2dd4079760 msgr2=0x7f2dd4079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.377+0000 7f2dd9280640 1 -- 172.21.15.64:0/1977658104 shutdown_connections 2024-09-21T04:02:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.377+0000 7f2dd9280640 1 -- 172.21.15.64:0/1977658104 wait complete. 2024-09-21T04:02:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.377+0000 7f2dd9280640 1 Processor -- start 2024-09-21T04:02:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.378+0000 7f2dd9280640 1 -- start start 2024-09-21T04:02:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.378+0000 7f2dd9280640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2dd4104a30 0x7f2dd419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.378+0000 7f2dd9280640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd4105380 0x7f2dd419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.378+0000 7f2dd9280640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dd410de60 0x7f2dd419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:08.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.378+0000 7f2dd9280640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd4113aa0 con 0x7f2dd4105380 2024-09-21T04:02:08.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.378+0000 7f2dd9280640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd4113920 con 0x7f2dd410de60 2024-09-21T04:02:08.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.378+0000 7f2dd9280640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd4113c20 con 0x7f2dd4104a30 2024-09-21T04:02:08.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.378+0000 7f2dd27fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd4105380 0x7f2dd419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:08.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.378+0000 7f2dd27fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd4105380 0x7f2dd419aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33656/0 (socket says 172.21.15.64:33656) 2024-09-21T04:02:08.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.378+0000 7f2dd27fc640 1 -- 172.21.15.64:0/1139151384 learned_addr learned my addr 172.21.15.64:0/1139151384 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:08.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.378+0000 7f2dd37fe640 1 --2- 172.21.15.64:0/1139151384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dd410de60 0x7f2dd419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:08.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.378+0000 7f2dd2ffd640 1 --2- 172.21.15.64:0/1139151384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2dd4104a30 0x7f2dd419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:08.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.378+0000 7f2dd27fc640 1 -- 172.21.15.64:0/1139151384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2dd4104a30 msgr2=0x7f2dd419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:08.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.378+0000 7f2dd27fc640 1 --2- 172.21.15.64:0/1139151384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2dd4104a30 0x7f2dd419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:08.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.378+0000 7f2dd27fc640 1 -- 172.21.15.64:0/1139151384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dd410de60 msgr2=0x7f2dd419fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:08.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.378+0000 7f2dd27fc640 1 --2- 172.21.15.64:0/1139151384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dd410de60 0x7f2dd419fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:08.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.378+0000 7f2dd27fc640 1 -- 172.21.15.64:0/1139151384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd41a0460 con 0x7f2dd4105380 2024-09-21T04:02:08.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.379+0000 7f2dd27fc640 1 --2- 172.21.15.64:0/1139151384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd4105380 0x7f2dd419aa90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2dc800d740 tx=0x7f2dc800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:08.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.379+0000 7f2dbbfff640 1 -- 172.21.15.64:0/1139151384 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8017070 con 0x7f2dd4105380 2024-09-21T04:02:08.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.379+0000 7f2dd9280640 1 -- 172.21.15.64:0/1139151384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dd41a0750 con 0x7f2dd4105380 2024-09-21T04:02:08.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.379+0000 7f2dbbfff640 1 -- 172.21.15.64:0/1139151384 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dc800de50 con 0x7f2dd4105380 2024-09-21T04:02:08.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.379+0000 7f2dbbfff640 1 -- 172.21.15.64:0/1139151384 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc800f620 con 0x7f2dd4105380 2024-09-21T04:02:08.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.379+0000 7f2dd9280640 1 -- 172.21.15.64:0/1139151384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dd41a6eb0 con 0x7f2dd4105380 2024-09-21T04:02:08.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.380+0000 7f2dd9280640 1 -- 172.21.15.64:0/1139151384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dd406c1f0 con 0x7f2dd4105380 2024-09-21T04:02:08.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.381+0000 7f2dbbfff640 1 -- 172.21.15.64:0/1139151384 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f2dc801a070 con 0x7f2dd4105380 2024-09-21T04:02:08.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.382+0000 7f2dbbfff640 1 --2- 172.21.15.64:0/1139151384 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2dbc0b6560 0x7f2dbc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:08.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.382+0000 7f2dd2ffd640 1 --2- 172.21.15.64:0/1139151384 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2dbc0b6560 0x7f2dbc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:08.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.382+0000 7f2dbbfff640 1 -- 172.21.15.64:0/1139151384 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8194+0+0 (secure 0 0 0) 0x7f2dc80e6cc0 con 0x7f2dd4105380 2024-09-21T04:02:08.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.383+0000 7f2dd2ffd640 1 --2- 172.21.15.64:0/1139151384 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2dbc0b6560 0x7f2dbc0b8a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4000c00 tx=0x7f2dc4031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:08.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.387+0000 7f2dbbfff640 1 -- 172.21.15.64:0/1139151384 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2dc80b1950 con 0x7f2dd4105380 2024-09-21T04:02:08.506 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 04:02:08 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-b[64168]: 2024-09-21T04:02:08.253+0000 7fc539242640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T04:02:08.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.616+0000 7f2dd9280640 1 -- 172.21.15.64:0/1139151384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f2dd419b620 con 0x7f2dd4105380 2024-09-21T04:02:08.772 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.768+0000 7f2dbbfff640 1 -- 172.21.15.64:0/1139151384 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f2dd419b620 con 0x7f2dd4105380 2024-09-21T04:02:08.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.771+0000 7f2dd9280640 1 -- 172.21.15.64:0/1139151384 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2dbc0b6560 msgr2=0x7f2dbc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:08.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.771+0000 7f2dd9280640 1 --2- 172.21.15.64:0/1139151384 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2dbc0b6560 0x7f2dbc0b8a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4000c00 tx=0x7f2dc4031400 comp rx=0 tx=0).stop 2024-09-21T04:02:08.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.771+0000 7f2dd9280640 1 -- 172.21.15.64:0/1139151384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd4105380 msgr2=0x7f2dd419aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:08.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.771+0000 7f2dd9280640 1 --2- 172.21.15.64:0/1139151384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd4105380 0x7f2dd419aa90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2dc800d740 tx=0x7f2dc800dc00 comp rx=0 tx=0).stop 2024-09-21T04:02:08.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.771+0000 7f2dd9280640 1 -- 172.21.15.64:0/1139151384 shutdown_connections 2024-09-21T04:02:08.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.771+0000 7f2dd9280640 1 --2- 172.21.15.64:0/1139151384 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2dbc0b6560 0x7f2dbc0b8a70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:08.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.771+0000 7f2dd9280640 1 --2- 172.21.15.64:0/1139151384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dd410de60 0x7f2dd419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:08.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.771+0000 7f2dd9280640 1 --2- 172.21.15.64:0/1139151384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd4105380 0x7f2dd419aa90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:08.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.771+0000 7f2dd9280640 1 --2- 172.21.15.64:0/1139151384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2dd4104a30 0x7f2dd419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:08.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.771+0000 7f2dd9280640 1 -- 172.21.15.64:0/1139151384 >> 172.21.15.64:0/1139151384 conn(0x7f2dd4079760 msgr2=0x7f2dd4100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:08.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.771+0000 7f2dd9280640 1 -- 172.21.15.64:0/1139151384 shutdown_connections 2024-09-21T04:02:08.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:08.771+0000 7f2dd9280640 1 -- 172.21.15.64:0/1139151384 wait complete. 2024-09-21T04:02:09.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:08 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1139151384' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T04:02:09.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:08 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1139151384' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T04:02:09.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:08 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1139151384' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T04:02:09.449 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-21T04:02:09.449 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs set cephfs max_mds 3 2024-09-21T04:02:09.708 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:10.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:09 smithi131 ceph-mon[35193]: pgmap v261: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-21T04:02:10.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:09 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1139151384' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T04:02:10.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:09 smithi131 ceph-mon[35193]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T04:02:10.093 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:09 smithi064 ceph-mon[31180]: pgmap v261: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-21T04:02:10.093 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:09 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1139151384' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T04:02:10.093 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:09 smithi064 ceph-mon[31180]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T04:02:10.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.187+0000 7f7a6e9e1640 1 -- 172.21.15.64:0/2425788561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a68108cf0 msgr2=0x7f7a6810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:10.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.187+0000 7f7a6e9e1640 1 --2- 172.21.15.64:0/2425788561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a68108cf0 0x7f7a6810f520 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009ca0 tx=0x7f7a6002a8c0 comp rx=0 tx=0).stop 2024-09-21T04:02:10.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.187+0000 7f7a6e9e1640 1 -- 172.21.15.64:0/2425788561 shutdown_connections 2024-09-21T04:02:10.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.187+0000 7f7a6e9e1640 1 --2- 172.21.15.64:0/2425788561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a68108cf0 0x7f7a6810f520 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:10.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.187+0000 7f7a6e9e1640 1 --2- 172.21.15.64:0/2425788561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a681083a0 0x7f7a681087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:10.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.187+0000 7f7a6e9e1640 1 --2- 172.21.15.64:0/2425788561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a68102a60 0x7f7a68102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:10.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.187+0000 7f7a6e9e1640 1 -- 172.21.15.64:0/2425788561 >> 172.21.15.64:0/2425788561 conn(0x7f7a68079730 msgr2=0x7f7a68079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:10.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.187+0000 7f7a6e9e1640 1 -- 172.21.15.64:0/2425788561 shutdown_connections 2024-09-21T04:02:10.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.187+0000 7f7a6e9e1640 1 -- 172.21.15.64:0/2425788561 wait complete. 2024-09-21T04:02:10.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.188+0000 7f7a6e9e1640 1 Processor -- start 2024-09-21T04:02:10.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.188+0000 7f7a6e9e1640 1 -- start start 2024-09-21T04:02:10.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.188+0000 7f7a6e9e1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a68102a60 0x7f7a681990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:10.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.188+0000 7f7a6e9e1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a681083a0 0x7f7a68199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:10.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.188+0000 7f7a6e9e1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a68108cf0 0x7f7a68199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:10.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.188+0000 7f7a6e9e1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7a681129c0 con 0x7f7a68108cf0 2024-09-21T04:02:10.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.188+0000 7f7a6e9e1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7a68112840 con 0x7f7a68102a60 2024-09-21T04:02:10.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.188+0000 7f7a6e9e1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7a68112b40 con 0x7f7a681083a0 2024-09-21T04:02:10.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.188+0000 7f7a6cf7e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a68108cf0 0x7f7a68199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:10.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.189+0000 7f7a6cf7e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a68108cf0 0x7f7a68199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33682/0 (socket says 172.21.15.64:33682) 2024-09-21T04:02:10.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.189+0000 7f7a6cf7e640 1 -- 172.21.15.64:0/2028545376 learned_addr learned my addr 172.21.15.64:0/2028545376 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:10.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.189+0000 7f7a5ffff640 1 --2- 172.21.15.64:0/2028545376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a681083a0 0x7f7a68199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:10.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.189+0000 7f7a67fff640 1 --2- 172.21.15.64:0/2028545376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a68102a60 0x7f7a681990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:10.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.189+0000 7f7a6cf7e640 1 -- 172.21.15.64:0/2028545376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a681083a0 msgr2=0x7f7a68199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:10.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.189+0000 7f7a6cf7e640 1 --2- 172.21.15.64:0/2028545376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a681083a0 0x7f7a68199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:10.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.189+0000 7f7a6cf7e640 1 -- 172.21.15.64:0/2028545376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a68102a60 msgr2=0x7f7a681990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:10.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.189+0000 7f7a6cf7e640 1 --2- 172.21.15.64:0/2028545376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a68102a60 0x7f7a681990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:10.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.189+0000 7f7a6cf7e640 1 -- 172.21.15.64:0/2028545376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a681a8770 con 0x7f7a68108cf0 2024-09-21T04:02:10.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.189+0000 7f7a67fff640 1 --2- 172.21.15.64:0/2028545376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a68102a60 0x7f7a681990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:02:10.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.189+0000 7f7a6cf7e640 1 --2- 172.21.15.64:0/2028545376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a68108cf0 0x7f7a68199b40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7a60005cc0 tx=0x7f7a6002f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:10.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.189+0000 7f7a65ffb640 1 -- 172.21.15.64:0/2028545376 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a60003cf0 con 0x7f7a68108cf0 2024-09-21T04:02:10.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.189+0000 7f7a65ffb640 1 -- 172.21.15.64:0/2028545376 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a600044f0 con 0x7f7a68108cf0 2024-09-21T04:02:10.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.189+0000 7f7a6e9e1640 1 -- 172.21.15.64:0/2028545376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a681a8a00 con 0x7f7a68108cf0 2024-09-21T04:02:10.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.189+0000 7f7a65ffb640 1 -- 172.21.15.64:0/2028545376 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6002fd10 con 0x7f7a68108cf0 2024-09-21T04:02:10.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.189+0000 7f7a6e9e1640 1 -- 172.21.15.64:0/2028545376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a681a8fc0 con 0x7f7a68108cf0 2024-09-21T04:02:10.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.191+0000 7f7a6e9e1640 1 -- 172.21.15.64:0/2028545376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a68102e70 con 0x7f7a68108cf0 2024-09-21T04:02:10.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.192+0000 7f7a65ffb640 1 -- 172.21.15.64:0/2028545376 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f7a60004040 con 0x7f7a68108cf0 2024-09-21T04:02:10.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.193+0000 7f7a65ffb640 1 --2- 172.21.15.64:0/2028545376 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7a540b6560 0x7f7a540b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:10.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.193+0000 7f7a67fff640 1 --2- 172.21.15.64:0/2028545376 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7a540b6560 0x7f7a540b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:10.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.193+0000 7f7a65ffb640 1 -- 172.21.15.64:0/2028545376 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8194+0+0 (secure 0 0 0) 0x7f7a60108980 con 0x7f7a68108cf0 2024-09-21T04:02:10.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.193+0000 7f7a67fff640 1 --2- 172.21.15.64:0/2028545376 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7a540b6560 0x7f7a540b8a70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7a68106ed0 tx=0x7f7a58008040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:10.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.197+0000 7f7a65ffb640 1 -- 172.21.15.64:0/2028545376 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7a600cf230 con 0x7f7a68108cf0 2024-09-21T04:02:10.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:09 smithi119 ceph-mon[34811]: pgmap v261: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-21T04:02:10.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:09 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1139151384' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T04:02:10.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:09 smithi119 ceph-mon[34811]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T04:02:10.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.425+0000 7f7a6e9e1640 1 -- 172.21.15.64:0/2028545376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f7a6802e640 con 0x7f7a68108cf0 2024-09-21T04:02:10.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.776+0000 7f7a65ffb640 1 -- 172.21.15.64:0/2028545376 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f7a600d3610 con 0x7f7a68108cf0 2024-09-21T04:02:10.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.778+0000 7f7a6e9e1640 1 -- 172.21.15.64:0/2028545376 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7a540b6560 msgr2=0x7f7a540b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:10.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.778+0000 7f7a6e9e1640 1 --2- 172.21.15.64:0/2028545376 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7a540b6560 0x7f7a540b8a70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7a68106ed0 tx=0x7f7a58008040 comp rx=0 tx=0).stop 2024-09-21T04:02:10.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.778+0000 7f7a6e9e1640 1 -- 172.21.15.64:0/2028545376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a68108cf0 msgr2=0x7f7a68199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:10.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.778+0000 7f7a6e9e1640 1 --2- 172.21.15.64:0/2028545376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a68108cf0 0x7f7a68199b40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7a60005cc0 tx=0x7f7a6002f470 comp rx=0 tx=0).stop 2024-09-21T04:02:10.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.778+0000 7f7a6e9e1640 1 -- 172.21.15.64:0/2028545376 shutdown_connections 2024-09-21T04:02:10.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.778+0000 7f7a6e9e1640 1 --2- 172.21.15.64:0/2028545376 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7a540b6560 0x7f7a540b8a70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:10.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.778+0000 7f7a6e9e1640 1 --2- 172.21.15.64:0/2028545376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a68108cf0 0x7f7a68199b40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:10.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.778+0000 7f7a6e9e1640 1 --2- 172.21.15.64:0/2028545376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a681083a0 0x7f7a68199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:10.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.778+0000 7f7a6e9e1640 1 --2- 172.21.15.64:0/2028545376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a68102a60 0x7f7a681990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:10.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.778+0000 7f7a6e9e1640 1 -- 172.21.15.64:0/2028545376 >> 172.21.15.64:0/2028545376 conn(0x7f7a68079730 msgr2=0x7f7a681052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:10.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.778+0000 7f7a6e9e1640 1 -- 172.21.15.64:0/2028545376 shutdown_connections 2024-09-21T04:02:10.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:10.778+0000 7f7a6e9e1640 1 -- 172.21.15.64:0/2028545376 wait complete. 2024-09-21T04:02:11.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:10 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2028545376' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-21T04:02:11.096 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:10 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2028545376' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-21T04:02:11.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:10 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2028545376' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-21T04:02:11.424 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs set cephfs allow_standby_replay true 2024-09-21T04:02:11.685 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:12.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:11 smithi131 ceph-mon[35193]: pgmap v262: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-21T04:02:12.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:11 smithi131 ceph-mon[35193]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T04:02:12.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:11 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2028545376' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-21T04:02:12.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:11 smithi131 ceph-mon[35193]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T04:02:12.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:11 smithi131 ceph-mon[35193]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T04:02:12.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:11 smithi131 ceph-mon[35193]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-21T04:02:12.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:11 smithi131 ceph-mon[35193]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-21T04:02:12.070 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:11 smithi064 ceph-mon[31180]: pgmap v262: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-21T04:02:12.070 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:11 smithi064 ceph-mon[31180]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T04:02:12.071 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:11 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2028545376' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-21T04:02:12.071 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:11 smithi064 ceph-mon[31180]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T04:02:12.071 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:11 smithi064 ceph-mon[31180]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T04:02:12.071 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:11 smithi064 ceph-mon[31180]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-21T04:02:12.071 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:11 smithi064 ceph-mon[31180]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-21T04:02:12.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.164+0000 7f6e6d8a5640 1 -- 172.21.15.64:0/1999304787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e681083d0 msgr2=0x7f6e681087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:12.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.164+0000 7f6e6d8a5640 1 --2- 172.21.15.64:0/1999304787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e681083d0 0x7f6e681087e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009900 tx=0x7f6e5c02a410 comp rx=0 tx=0).stop 2024-09-21T04:02:12.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.164+0000 7f6e6d8a5640 1 -- 172.21.15.64:0/1999304787 shutdown_connections 2024-09-21T04:02:12.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.164+0000 7f6e6d8a5640 1 --2- 172.21.15.64:0/1999304787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e68108d20 0x7f6e6810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:12.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.164+0000 7f6e6d8a5640 1 --2- 172.21.15.64:0/1999304787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e681083d0 0x7f6e681087e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.164+0000 7f6e6d8a5640 1 --2- 172.21.15.64:0/1999304787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e68102ae0 0x7f6e68102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.164+0000 7f6e6d8a5640 1 -- 172.21.15.64:0/1999304787 >> 172.21.15.64:0/1999304787 conn(0x7f6e68079760 msgr2=0x7f6e68079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.164+0000 7f6e6d8a5640 1 -- 172.21.15.64:0/1999304787 shutdown_connections 2024-09-21T04:02:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.164+0000 7f6e6d8a5640 1 -- 172.21.15.64:0/1999304787 wait complete. 2024-09-21T04:02:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.165+0000 7f6e6d8a5640 1 Processor -- start 2024-09-21T04:02:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.165+0000 7f6e6d8a5640 1 -- start start 2024-09-21T04:02:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.165+0000 7f6e6d8a5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e68102ae0 0x7f6e6819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.165+0000 7f6e6d8a5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e681083d0 0x7f6e6819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.165+0000 7f6e6d8a5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e68108d20 0x7f6e6819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.165+0000 7f6e6d8a5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6e68112990 con 0x7f6e68102ae0 2024-09-21T04:02:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.165+0000 7f6e6d8a5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6e68112810 con 0x7f6e68108d20 2024-09-21T04:02:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.165+0000 7f6e6d8a5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6e68112b10 con 0x7f6e681083d0 2024-09-21T04:02:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.166+0000 7f6e66ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e68102ae0 0x7f6e6819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:12.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.166+0000 7f6e66ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e68102ae0 0x7f6e6819a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45512/0 (socket says 172.21.15.64:45512) 2024-09-21T04:02:12.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.166+0000 7f6e66ffd640 1 -- 172.21.15.64:0/362350865 learned_addr learned my addr 172.21.15.64:0/362350865 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:12.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.166+0000 7f6e667fc640 1 --2- 172.21.15.64:0/362350865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e681083d0 0x7f6e6819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:12.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.166+0000 7f6e66ffd640 1 -- 172.21.15.64:0/362350865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e681083d0 msgr2=0x7f6e6819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:12.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.166+0000 7f6e677fe640 1 --2- 172.21.15.64:0/362350865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e68108d20 0x7f6e6819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:12.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.166+0000 7f6e66ffd640 1 --2- 172.21.15.64:0/362350865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e681083d0 0x7f6e6819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:12.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.166+0000 7f6e66ffd640 1 -- 172.21.15.64:0/362350865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e68108d20 msgr2=0x7f6e6819fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:12.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.166+0000 7f6e66ffd640 1 --2- 172.21.15.64:0/362350865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e68108d20 0x7f6e6819fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:12.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.166+0000 7f6e66ffd640 1 -- 172.21.15.64:0/362350865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e681a04a0 con 0x7f6e68102ae0 2024-09-21T04:02:12.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.166+0000 7f6e667fc640 1 --2- 172.21.15.64:0/362350865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e681083d0 0x7f6e6819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:02:12.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.166+0000 7f6e66ffd640 1 --2- 172.21.15.64:0/362350865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e68102ae0 0x7f6e6819a530 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800d950 tx=0x7f6e5800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:12.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.166+0000 7f6e6c8a3640 1 -- 172.21.15.64:0/362350865 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e58017070 con 0x7f6e68102ae0 2024-09-21T04:02:12.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.166+0000 7f6e6c8a3640 1 -- 172.21.15.64:0/362350865 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e58009c30 con 0x7f6e68102ae0 2024-09-21T04:02:12.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.166+0000 7f6e6d8a5640 1 -- 172.21.15.64:0/362350865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e681a0790 con 0x7f6e68102ae0 2024-09-21T04:02:12.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.166+0000 7f6e6c8a3640 1 -- 172.21.15.64:0/362350865 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5800f620 con 0x7f6e68102ae0 2024-09-21T04:02:12.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.166+0000 7f6e6d8a5640 1 -- 172.21.15.64:0/362350865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e681a6ef0 con 0x7f6e68102ae0 2024-09-21T04:02:12.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.167+0000 7f6e6d8a5640 1 -- 172.21.15.64:0/362350865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e68103d10 con 0x7f6e68102ae0 2024-09-21T04:02:12.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.169+0000 7f6e6c8a3640 1 -- 172.21.15.64:0/362350865 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f6e58005030 con 0x7f6e68102ae0 2024-09-21T04:02:12.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.170+0000 7f6e6c8a3640 1 --2- 172.21.15.64:0/362350865 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6e500b6560 0x7f6e500b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:12.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.171+0000 7f6e667fc640 1 --2- 172.21.15.64:0/362350865 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6e500b6560 0x7f6e500b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:12.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.171+0000 7f6e6c8a3640 1 -- 172.21.15.64:0/362350865 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8194+0+0 (secure 0 0 0) 0x7f6e580e7000 con 0x7f6e68102ae0 2024-09-21T04:02:12.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.171+0000 7f6e667fc640 1 --2- 172.21.15.64:0/362350865 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6e500b6560 0x7f6e500b8a70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c0096c0 tx=0x7f6e5c031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:12.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.174+0000 7f6e6c8a3640 1 -- 172.21.15.64:0/362350865 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6e580ad8b0 con 0x7f6e68102ae0 2024-09-21T04:02:12.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:11 smithi119 ceph-mon[34811]: pgmap v262: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-21T04:02:12.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:11 smithi119 ceph-mon[34811]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T04:02:12.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:11 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2028545376' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-21T04:02:12.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:11 smithi119 ceph-mon[34811]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T04:02:12.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:11 smithi119 ceph-mon[34811]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T04:02:12.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:11 smithi119 ceph-mon[34811]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-21T04:02:12.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:11 smithi119 ceph-mon[34811]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-21T04:02:12.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.401+0000 7f6e6d8a5640 1 -- 172.21.15.64:0/362350865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f6e68102ef0 con 0x7f6e68102ae0 2024-09-21T04:02:12.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.795+0000 7f6e6c8a3640 1 -- 172.21.15.64:0/362350865 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f6e58009de0 con 0x7f6e68102ae0 2024-09-21T04:02:12.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.798+0000 7f6e6d8a5640 1 -- 172.21.15.64:0/362350865 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6e500b6560 msgr2=0x7f6e500b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:12.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.798+0000 7f6e6d8a5640 1 --2- 172.21.15.64:0/362350865 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6e500b6560 0x7f6e500b8a70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c0096c0 tx=0x7f6e5c031400 comp rx=0 tx=0).stop 2024-09-21T04:02:12.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.798+0000 7f6e6d8a5640 1 -- 172.21.15.64:0/362350865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e68102ae0 msgr2=0x7f6e6819a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:12.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.798+0000 7f6e6d8a5640 1 --2- 172.21.15.64:0/362350865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e68102ae0 0x7f6e6819a530 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6e5800d950 tx=0x7f6e5800de10 comp rx=0 tx=0).stop 2024-09-21T04:02:12.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.798+0000 7f6e6d8a5640 1 -- 172.21.15.64:0/362350865 shutdown_connections 2024-09-21T04:02:12.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.798+0000 7f6e6d8a5640 1 --2- 172.21.15.64:0/362350865 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6e500b6560 0x7f6e500b8a70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:12.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.798+0000 7f6e6d8a5640 1 --2- 172.21.15.64:0/362350865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e68108d20 0x7f6e6819fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:12.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.798+0000 7f6e6d8a5640 1 --2- 172.21.15.64:0/362350865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e681083d0 0x7f6e6819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:12.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.798+0000 7f6e6d8a5640 1 --2- 172.21.15.64:0/362350865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e68102ae0 0x7f6e6819a530 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:12.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.798+0000 7f6e6d8a5640 1 -- 172.21.15.64:0/362350865 >> 172.21.15.64:0/362350865 conn(0x7f6e68079760 msgr2=0x7f6e6806bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:12.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.798+0000 7f6e6d8a5640 1 -- 172.21.15.64:0/362350865 shutdown_connections 2024-09-21T04:02:12.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:12.798+0000 7f6e6d8a5640 1 -- 172.21.15.64:0/362350865 wait complete. 2024-09-21T04:02:13.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:12 smithi131 ceph-mon[35193]: mds.? [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] up:active 2024-09-21T04:02:13.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:12 smithi131 ceph-mon[35193]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T04:02:13.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:12 smithi131 ceph-mon[35193]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T04:02:13.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:12 smithi131 ceph-mon[35193]: Cluster is now healthy 2024-09-21T04:02:13.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:12 smithi131 ceph-mon[35193]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-21T04:02:13.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:12 smithi131 ceph-mon[35193]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-21T04:02:13.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:12 smithi131 ceph-mon[35193]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-21T04:02:13.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:12 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/362350865' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T04:02:13.166 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:12 smithi064 ceph-mon[31180]: mds.? [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] up:active 2024-09-21T04:02:13.166 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:12 smithi064 ceph-mon[31180]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T04:02:13.166 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:12 smithi064 ceph-mon[31180]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T04:02:13.167 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:12 smithi064 ceph-mon[31180]: Cluster is now healthy 2024-09-21T04:02:13.167 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:12 smithi064 ceph-mon[31180]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-21T04:02:13.167 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:12 smithi064 ceph-mon[31180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-21T04:02:13.167 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:12 smithi064 ceph-mon[31180]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-21T04:02:13.167 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:12 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/362350865' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T04:02:13.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:12 smithi119 ceph-mon[34811]: mds.? [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] up:active 2024-09-21T04:02:13.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:12 smithi119 ceph-mon[34811]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T04:02:13.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:12 smithi119 ceph-mon[34811]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T04:02:13.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:12 smithi119 ceph-mon[34811]: Cluster is now healthy 2024-09-21T04:02:13.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:12 smithi119 ceph-mon[34811]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-21T04:02:13.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:12 smithi119 ceph-mon[34811]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-21T04:02:13.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:12 smithi119 ceph-mon[34811]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-21T04:02:13.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:12 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/362350865' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T04:02:13.498 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs set cephfs session_timeout 300 2024-09-21T04:02:13.506 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 04:02:13 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-b[64168]: 2024-09-21T04:02:13.253+0000 7fc539242640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T04:02:13.507 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 04:02:13 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-b[64168]: 2024-09-21T04:02:13.253+0000 7fc539242640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-21T04:02:13.759 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:14.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:13 smithi131 ceph-mon[35193]: pgmap v263: 97 pgs: 97 active+clean; 580 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2024-09-21T04:02:14.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:13 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/362350865' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T04:02:14.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:13 smithi131 ceph-mon[35193]: mds.? [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] up:active 2024-09-21T04:02:14.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:13 smithi131 ceph-mon[35193]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-21T04:02:14.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:13 smithi131 ceph-mon[35193]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:02:14.246 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:13 smithi064 ceph-mon[31180]: pgmap v263: 97 pgs: 97 active+clean; 580 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2024-09-21T04:02:14.246 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:13 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/362350865' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T04:02:14.246 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:13 smithi064 ceph-mon[31180]: mds.? [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] up:active 2024-09-21T04:02:14.246 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:13 smithi064 ceph-mon[31180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-21T04:02:14.246 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:13 smithi064 ceph-mon[31180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:02:14.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:13 smithi119 ceph-mon[34811]: pgmap v263: 97 pgs: 97 active+clean; 580 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2024-09-21T04:02:14.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:13 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/362350865' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T04:02:14.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:13 smithi119 ceph-mon[34811]: mds.? [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] up:active 2024-09-21T04:02:14.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:13 smithi119 ceph-mon[34811]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-21T04:02:14.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:13 smithi119 ceph-mon[34811]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:02:14.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.338+0000 7febad080640 1 -- 172.21.15.64:0/2308281923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feba8103000 msgr2=0x7feba8103410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:14.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.338+0000 7febad080640 1 --2- 172.21.15.64:0/2308281923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feba8103000 0x7feba8103410 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7feb9c009900 tx=0x7feb9c02a410 comp rx=0 tx=0).stop 2024-09-21T04:02:14.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.338+0000 7febad080640 1 -- 172.21.15.64:0/2308281923 shutdown_connections 2024-09-21T04:02:14.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.338+0000 7febad080640 1 --2- 172.21.15.64:0/2308281923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba8103950 0x7feba810f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:14.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.338+0000 7febad080640 1 --2- 172.21.15.64:0/2308281923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feba8103000 0x7feba8103410 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:14.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.338+0000 7febad080640 1 --2- 172.21.15.64:0/2308281923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feba8107690 0x7feba8107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.338+0000 7febad080640 1 -- 172.21.15.64:0/2308281923 >> 172.21.15.64:0/2308281923 conn(0x7feba8079ea0 msgr2=0x7feba807a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.338+0000 7febad080640 1 -- 172.21.15.64:0/2308281923 shutdown_connections 2024-09-21T04:02:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.338+0000 7febad080640 1 -- 172.21.15.64:0/2308281923 wait complete. 2024-09-21T04:02:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.339+0000 7febad080640 1 Processor -- start 2024-09-21T04:02:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.339+0000 7febad080640 1 -- start start 2024-09-21T04:02:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.339+0000 7febad080640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba8103000 0x7feba819b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.339+0000 7febad080640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feba8103950 0x7feba819b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.339+0000 7febad080640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feba8107690 0x7feba819be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.339+0000 7febad080640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7feba8112a40 con 0x7feba8107690 2024-09-21T04:02:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.339+0000 7febad080640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feba81128c0 con 0x7feba8103000 2024-09-21T04:02:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.339+0000 7febad080640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feba8112bc0 con 0x7feba8103950 2024-09-21T04:02:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.339+0000 7feba759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feba8107690 0x7feba819be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.339+0000 7feba759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feba8107690 0x7feba819be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45542/0 (socket says 172.21.15.64:45542) 2024-09-21T04:02:14.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.339+0000 7feba759e640 1 -- 172.21.15.64:0/1742266477 learned_addr learned my addr 172.21.15.64:0/1742266477 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.339+0000 7feba6d9d640 1 --2- 172.21.15.64:0/1742266477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba8103000 0x7feba819b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.339+0000 7feba659c640 1 --2- 172.21.15.64:0/1742266477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feba8103950 0x7feba819b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.340+0000 7feba6d9d640 1 -- 172.21.15.64:0/1742266477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feba8103950 msgr2=0x7feba819b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.340+0000 7feba6d9d640 1 --2- 172.21.15.64:0/1742266477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feba8103950 0x7feba819b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.340+0000 7feba6d9d640 1 -- 172.21.15.64:0/1742266477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feba8107690 msgr2=0x7feba819be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.340+0000 7feba6d9d640 1 --2- 172.21.15.64:0/1742266477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feba8107690 0x7feba819be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.340+0000 7feba6d9d640 1 -- 172.21.15.64:0/1742266477 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba81a6660 con 0x7feba8103000 2024-09-21T04:02:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.340+0000 7feba759e640 1 --2- 172.21.15.64:0/1742266477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feba8107690 0x7feba819be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:02:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.340+0000 7feba6d9d640 1 --2- 172.21.15.64:0/1742266477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba8103000 0x7feba819b3d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7feb9800e9f0 tx=0x7feb9800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.340+0000 7feb8ffff640 1 -- 172.21.15.64:0/1742266477 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9800ccc0 con 0x7feba8103000 2024-09-21T04:02:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.340+0000 7febad080640 1 -- 172.21.15.64:0/1742266477 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba81a6950 con 0x7feba8103000 2024-09-21T04:02:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.340+0000 7feb8ffff640 1 -- 172.21.15.64:0/1742266477 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb98005020 con 0x7feba8103000 2024-09-21T04:02:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.340+0000 7feb8ffff640 1 -- 172.21.15.64:0/1742266477 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98004a00 con 0x7feba8103000 2024-09-21T04:02:14.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.340+0000 7febad080640 1 -- 172.21.15.64:0/1742266477 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba81a6e60 con 0x7feba8103000 2024-09-21T04:02:14.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.341+0000 7febad080640 1 -- 172.21.15.64:0/1742266477 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feba8103410 con 0x7feba8103000 2024-09-21T04:02:14.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.343+0000 7feb8ffff640 1 -- 172.21.15.64:0/1742266477 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7feb98004ba0 con 0x7feba8103000 2024-09-21T04:02:14.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.344+0000 7feb8ffff640 1 --2- 172.21.15.64:0/1742266477 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7feb900b6590 0x7feb900b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:14.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.344+0000 7feba659c640 1 --2- 172.21.15.64:0/1742266477 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7feb900b6590 0x7feb900b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:14.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.344+0000 7feb8ffff640 1 -- 172.21.15.64:0/1742266477 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8194+0+0 (secure 0 0 0) 0x7feb980e72b0 con 0x7feba8103000 2024-09-21T04:02:14.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.344+0000 7feba659c640 1 --2- 172.21.15.64:0/1742266477 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7feb900b6590 0x7feb900b8aa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7feb9c009230 tx=0x7feb9c031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:14.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.347+0000 7feb8ffff640 1 -- 172.21.15.64:0/1742266477 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7feb980adbf0 con 0x7feba8103000 2024-09-21T04:02:14.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.577+0000 7febad080640 1 -- 172.21.15.64:0/1742266477 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7feba8102300 con 0x7feba8103000 2024-09-21T04:02:14.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.808+0000 7feb8ffff640 1 -- 172.21.15.64:0/1742266477 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7feba8102300 con 0x7feba8103000 2024-09-21T04:02:14.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.810+0000 7febad080640 1 -- 172.21.15.64:0/1742266477 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7feb900b6590 msgr2=0x7feb900b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:14.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.810+0000 7febad080640 1 --2- 172.21.15.64:0/1742266477 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7feb900b6590 0x7feb900b8aa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7feb9c009230 tx=0x7feb9c031400 comp rx=0 tx=0).stop 2024-09-21T04:02:14.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.810+0000 7febad080640 1 -- 172.21.15.64:0/1742266477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba8103000 msgr2=0x7feba819b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:14.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.810+0000 7febad080640 1 --2- 172.21.15.64:0/1742266477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba8103000 0x7feba819b3d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7feb9800e9f0 tx=0x7feb9800eeb0 comp rx=0 tx=0).stop 2024-09-21T04:02:14.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.810+0000 7febad080640 1 -- 172.21.15.64:0/1742266477 shutdown_connections 2024-09-21T04:02:14.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.810+0000 7febad080640 1 --2- 172.21.15.64:0/1742266477 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7feb900b6590 0x7feb900b8aa0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:14.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.810+0000 7febad080640 1 --2- 172.21.15.64:0/1742266477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feba8107690 0x7feba819be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:14.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.810+0000 7febad080640 1 --2- 172.21.15.64:0/1742266477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feba8103950 0x7feba819b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:14.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.810+0000 7febad080640 1 --2- 172.21.15.64:0/1742266477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba8103000 0x7feba819b3d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:14.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.810+0000 7febad080640 1 -- 172.21.15.64:0/1742266477 >> 172.21.15.64:0/1742266477 conn(0x7feba8079ea0 msgr2=0x7feba8104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:14.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.810+0000 7febad080640 1 -- 172.21.15.64:0/1742266477 shutdown_connections 2024-09-21T04:02:14.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:14.810+0000 7febad080640 1 -- 172.21.15.64:0/1742266477 wait complete. 2024-09-21T04:02:15.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:14 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1742266477' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T04:02:15.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:14 smithi064 ceph-mon[31180]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T04:02:15.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:14 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1742266477' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T04:02:15.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:14 smithi131 ceph-mon[35193]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T04:02:15.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:14 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1742266477' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T04:02:15.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:14 smithi119 ceph-mon[34811]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T04:02:15.448 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-09-21T04:02:15.448 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs subvolumegroup create cephfs qa 2024-09-21T04:02:15.708 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:16.113 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:15 smithi064 ceph-mon[31180]: pgmap v264: 97 pgs: 97 active+clean; 580 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 8 op/s 2024-09-21T04:02:16.113 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:15 smithi064 ceph-mon[31180]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T04:02:16.113 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:15 smithi064 ceph-mon[31180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:02:16.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.207+0000 7f5c404b6640 1 -- 172.21.15.64:0/3402090846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c38109e40 msgr2=0x7f5c38110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:16.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.207+0000 7f5c404b6640 1 --2- 172.21.15.64:0/3402090846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c38109e40 0x7f5c38110670 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009900 tx=0x7f5c3002a410 comp rx=0 tx=0).stop 2024-09-21T04:02:16.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.207+0000 7f5c404b6640 1 -- 172.21.15.64:0/3402090846 shutdown_connections 2024-09-21T04:02:16.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.207+0000 7f5c404b6640 1 --2- 172.21.15.64:0/3402090846 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c38109e40 0x7f5c38110670 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:16.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.207+0000 7f5c404b6640 1 --2- 172.21.15.64:0/3402090846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c381094f0 0x7f5c38109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:16.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.207+0000 7f5c404b6640 1 --2- 172.21.15.64:0/3402090846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c38103bb0 0x7f5c38103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:16.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.207+0000 7f5c404b6640 1 -- 172.21.15.64:0/3402090846 >> 172.21.15.64:0/3402090846 conn(0x7f5c380ff600 msgr2=0x7f5c38101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:16.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.207+0000 7f5c404b6640 1 -- 172.21.15.64:0/3402090846 shutdown_connections 2024-09-21T04:02:16.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.207+0000 7f5c404b6640 1 -- 172.21.15.64:0/3402090846 wait complete. 2024-09-21T04:02:16.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.208+0000 7f5c404b6640 1 Processor -- start 2024-09-21T04:02:16.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.208+0000 7f5c404b6640 1 -- start start 2024-09-21T04:02:16.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.208+0000 7f5c404b6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c38103bb0 0x7f5c3819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:16.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.208+0000 7f5c404b6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c381094f0 0x7f5c3819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:16.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.208+0000 7f5c404b6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c38109e40 0x7f5c3819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:16.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.208+0000 7f5c404b6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5c38113b00 con 0x7f5c38103bb0 2024-09-21T04:02:16.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.208+0000 7f5c404b6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5c38113980 con 0x7f5c381094f0 2024-09-21T04:02:16.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.208+0000 7f5c404b6640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5c38113c80 con 0x7f5c38109e40 2024-09-21T04:02:16.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.208+0000 7f5c3e252640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c38103bb0 0x7f5c3819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:16.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.208+0000 7f5c3e252640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c38103bb0 0x7f5c3819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45558/0 (socket says 172.21.15.64:45558) 2024-09-21T04:02:16.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.208+0000 7f5c3e252640 1 -- 172.21.15.64:0/3353912493 learned_addr learned my addr 172.21.15.64:0/3353912493 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:16.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.208+0000 7f5c3da51640 1 --2- 172.21.15.64:0/3353912493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c381094f0 0x7f5c3819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:16.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.208+0000 7f5c3ea53640 1 --2- 172.21.15.64:0/3353912493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c38109e40 0x7f5c3819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:16.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.209+0000 7f5c3e252640 1 -- 172.21.15.64:0/3353912493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c38109e40 msgr2=0x7f5c3819fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:16.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.209+0000 7f5c3e252640 1 --2- 172.21.15.64:0/3353912493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c38109e40 0x7f5c3819fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:16.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.209+0000 7f5c3e252640 1 -- 172.21.15.64:0/3353912493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c381094f0 msgr2=0x7f5c3819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:16.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.209+0000 7f5c3e252640 1 --2- 172.21.15.64:0/3353912493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c381094f0 0x7f5c3819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:16.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.209+0000 7f5c3e252640 1 -- 172.21.15.64:0/3353912493 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c381a03e0 con 0x7f5c38103bb0 2024-09-21T04:02:16.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.209+0000 7f5c3da51640 1 --2- 172.21.15.64:0/3353912493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c381094f0 0x7f5c3819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:02:16.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.209+0000 7f5c3e252640 1 --2- 172.21.15.64:0/3353912493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c38103bb0 0x7f5c3819a470 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5c2800ebc0 tx=0x7f5c2800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:16.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.209+0000 7f5c377fe640 1 -- 172.21.15.64:0/3353912493 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28010630 con 0x7f5c38103bb0 2024-09-21T04:02:16.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.209+0000 7f5c377fe640 1 -- 172.21.15.64:0/3353912493 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c28010c80 con 0x7f5c38103bb0 2024-09-21T04:02:16.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.209+0000 7f5c404b6640 1 -- 172.21.15.64:0/3353912493 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c381a06d0 con 0x7f5c38103bb0 2024-09-21T04:02:16.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.209+0000 7f5c377fe640 1 -- 172.21.15.64:0/3353912493 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c28018b60 con 0x7f5c38103bb0 2024-09-21T04:02:16.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.209+0000 7f5c404b6640 1 -- 172.21.15.64:0/3353912493 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c381a6e30 con 0x7f5c38103bb0 2024-09-21T04:02:16.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.210+0000 7f5c404b6640 1 -- 172.21.15.64:0/3353912493 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c38103fc0 con 0x7f5c38103bb0 2024-09-21T04:02:16.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.213+0000 7f5c377fe640 1 -- 172.21.15.64:0/3353912493 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f5c28018d00 con 0x7f5c38103bb0 2024-09-21T04:02:16.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.214+0000 7f5c377fe640 1 --2- 172.21.15.64:0/3353912493 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5c200b6560 0x7f5c200b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:16.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.214+0000 7f5c3da51640 1 --2- 172.21.15.64:0/3353912493 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5c200b6560 0x7f5c200b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:16.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.214+0000 7f5c377fe640 1 -- 172.21.15.64:0/3353912493 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8194+0+0 (secure 0 0 0) 0x7f5c280e7da0 con 0x7f5c38103bb0 2024-09-21T04:02:16.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.215+0000 7f5c3da51640 1 --2- 172.21.15.64:0/3353912493 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5c200b6560 0x7f5c200b8a70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5c38107f10 tx=0x7f5c2c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:16.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.217+0000 7f5c377fe640 1 -- 172.21.15.64:0/3353912493 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5c280b2a30 con 0x7f5c38103bb0 2024-09-21T04:02:16.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:15 smithi119 ceph-mon[34811]: pgmap v264: 97 pgs: 97 active+clean; 580 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 8 op/s 2024-09-21T04:02:16.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:15 smithi119 ceph-mon[34811]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T04:02:16.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:15 smithi119 ceph-mon[34811]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:02:16.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:15 smithi131 ceph-mon[35193]: pgmap v264: 97 pgs: 97 active+clean; 580 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 8 op/s 2024-09-21T04:02:16.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:15 smithi131 ceph-mon[35193]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T04:02:16.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:15 smithi131 ceph-mon[35193]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:02:16.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.431+0000 7f5c404b6640 1 -- 172.21.15.64:0/3353912493 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f5c38063800 con 0x7f5c200b6560 2024-09-21T04:02:16.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.452+0000 7f5c377fe640 1 -- 172.21.15.64:0/3353912493 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5c38063800 con 0x7f5c200b6560 2024-09-21T04:02:16.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.455+0000 7f5c404b6640 1 -- 172.21.15.64:0/3353912493 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5c200b6560 msgr2=0x7f5c200b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:16.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.455+0000 7f5c404b6640 1 --2- 172.21.15.64:0/3353912493 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5c200b6560 0x7f5c200b8a70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5c38107f10 tx=0x7f5c2c005f90 comp rx=0 tx=0).stop 2024-09-21T04:02:16.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.455+0000 7f5c404b6640 1 -- 172.21.15.64:0/3353912493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c38103bb0 msgr2=0x7f5c3819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:16.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.455+0000 7f5c404b6640 1 --2- 172.21.15.64:0/3353912493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c38103bb0 0x7f5c3819a470 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5c2800ebc0 tx=0x7f5c2800c6a0 comp rx=0 tx=0).stop 2024-09-21T04:02:16.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.455+0000 7f5c404b6640 1 -- 172.21.15.64:0/3353912493 shutdown_connections 2024-09-21T04:02:16.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.455+0000 7f5c404b6640 1 --2- 172.21.15.64:0/3353912493 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5c200b6560 0x7f5c200b8a70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:16.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.455+0000 7f5c404b6640 1 --2- 172.21.15.64:0/3353912493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c38109e40 0x7f5c3819fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:16.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.455+0000 7f5c404b6640 1 --2- 172.21.15.64:0/3353912493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c381094f0 0x7f5c3819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:16.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.455+0000 7f5c404b6640 1 --2- 172.21.15.64:0/3353912493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c38103bb0 0x7f5c3819a470 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:16.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.455+0000 7f5c404b6640 1 -- 172.21.15.64:0/3353912493 >> 172.21.15.64:0/3353912493 conn(0x7f5c380ff600 msgr2=0x7f5c38101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:16.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.455+0000 7f5c404b6640 1 -- 172.21.15.64:0/3353912493 shutdown_connections 2024-09-21T04:02:16.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:16.455+0000 7f5c404b6640 1 -- 172.21.15.64:0/3353912493 wait complete. 2024-09-21T04:02:16.761 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 04:02:16 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: 2024-09-21T04:02:16.437+0000 7fa718064640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T04:02:16.761 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 04:02:16 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: 2024-09-21T04:02:16.437+0000 7fa718064640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T04:02:16.762 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 04:02:16 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: 2024-09-21T04:02:16.437+0000 7fa718064640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T04:02:16.762 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 04:02:16 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: 2024-09-21T04:02:16.437+0000 7fa718064640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T04:02:16.762 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 04:02:16 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: 2024-09-21T04:02:16.437+0000 7fa718064640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T04:02:17.139 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-21T04:02:17.396 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:17.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.885+0000 7fb74e48b640 1 -- 172.21.15.64:0/3354306476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb748108d20 msgr2=0x7fb74810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:17.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.885+0000 7fb74e48b640 1 --2- 172.21.15.64:0/3354306476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb748108d20 0x7fb74810f550 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb740009ca0 tx=0x7fb74002a8c0 comp rx=0 tx=0).stop 2024-09-21T04:02:17.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.885+0000 7fb74e48b640 1 -- 172.21.15.64:0/3354306476 shutdown_connections 2024-09-21T04:02:17.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.885+0000 7fb74e48b640 1 --2- 172.21.15.64:0/3354306476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb748108d20 0x7fb74810f550 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:17.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.885+0000 7fb74e48b640 1 --2- 172.21.15.64:0/3354306476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7481083d0 0x7fb7481087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:17.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.885+0000 7fb74e48b640 1 --2- 172.21.15.64:0/3354306476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb748102a90 0x7fb748102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:17.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.885+0000 7fb74e48b640 1 -- 172.21.15.64:0/3354306476 >> 172.21.15.64:0/3354306476 conn(0x7fb748079750 msgr2=0x7fb748079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:17.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.885+0000 7fb74e48b640 1 -- 172.21.15.64:0/3354306476 shutdown_connections 2024-09-21T04:02:17.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.885+0000 7fb74e48b640 1 -- 172.21.15.64:0/3354306476 wait complete. 2024-09-21T04:02:17.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.886+0000 7fb74e48b640 1 Processor -- start 2024-09-21T04:02:17.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.886+0000 7fb74e48b640 1 -- start start 2024-09-21T04:02:17.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.886+0000 7fb74e48b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb748102a90 0x7fb74819b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:17.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.886+0000 7fb74e48b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7481083d0 0x7fb74819b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:17.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.886+0000 7fb74e48b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb748108d20 0x7fb74819bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:17.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.886+0000 7fb74e48b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb7481129f0 con 0x7fb748102a90 2024-09-21T04:02:17.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.886+0000 7fb74e48b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb748112870 con 0x7fb748108d20 2024-09-21T04:02:17.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.886+0000 7fb74e48b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb748112b70 con 0x7fb7481083d0 2024-09-21T04:02:17.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.887+0000 7fb747fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb748102a90 0x7fb74819b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:17.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.887+0000 7fb747fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb748102a90 0x7fb74819b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45592/0 (socket says 172.21.15.64:45592) 2024-09-21T04:02:17.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.887+0000 7fb747fff640 1 -- 172.21.15.64:0/439776005 learned_addr learned my addr 172.21.15.64:0/439776005 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:17.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.887+0000 7fb74ca28640 1 --2- 172.21.15.64:0/439776005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb748108d20 0x7fb74819bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:17.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.887+0000 7fb73ffff640 1 --2- 172.21.15.64:0/439776005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7481083d0 0x7fb74819b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:17.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.887+0000 7fb747fff640 1 -- 172.21.15.64:0/439776005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7481083d0 msgr2=0x7fb74819b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:17.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.887+0000 7fb747fff640 1 --2- 172.21.15.64:0/439776005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7481083d0 0x7fb74819b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:17.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.887+0000 7fb747fff640 1 -- 172.21.15.64:0/439776005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb748108d20 msgr2=0x7fb74819bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:17.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.887+0000 7fb747fff640 1 --2- 172.21.15.64:0/439776005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb748108d20 0x7fb74819bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:17.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.887+0000 7fb747fff640 1 -- 172.21.15.64:0/439776005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7481a6580 con 0x7fb748102a90 2024-09-21T04:02:17.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.887+0000 7fb73ffff640 1 --2- 172.21.15.64:0/439776005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7481083d0 0x7fb74819b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:02:17.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.887+0000 7fb747fff640 1 --2- 172.21.15.64:0/439776005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb748102a90 0x7fb74819b2c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb73800d950 tx=0x7fb73800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:17.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.887+0000 7fb745ffb640 1 -- 172.21.15.64:0/439776005 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb738017070 con 0x7fb748102a90 2024-09-21T04:02:17.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.887+0000 7fb745ffb640 1 -- 172.21.15.64:0/439776005 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb738009c30 con 0x7fb748102a90 2024-09-21T04:02:17.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.887+0000 7fb74e48b640 1 -- 172.21.15.64:0/439776005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7481a6870 con 0x7fb748102a90 2024-09-21T04:02:17.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.887+0000 7fb745ffb640 1 -- 172.21.15.64:0/439776005 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73800f620 con 0x7fb748102a90 2024-09-21T04:02:17.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.887+0000 7fb74e48b640 1 -- 172.21.15.64:0/439776005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7481a6e80 con 0x7fb748102a90 2024-09-21T04:02:17.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.888+0000 7fb74e48b640 1 -- 172.21.15.64:0/439776005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb748103d10 con 0x7fb748102a90 2024-09-21T04:02:17.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.891+0000 7fb745ffb640 1 -- 172.21.15.64:0/439776005 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fb738010460 con 0x7fb748102a90 2024-09-21T04:02:17.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.892+0000 7fb745ffb640 1 --2- 172.21.15.64:0/439776005 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb7340b6670 0x7fb7340b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:17.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.892+0000 7fb73ffff640 1 --2- 172.21.15.64:0/439776005 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb7340b6670 0x7fb7340b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:17.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.892+0000 7fb745ffb640 1 -- 172.21.15.64:0/439776005 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8194+0+0 (secure 0 0 0) 0x7fb7380e7120 con 0x7fb748102a90 2024-09-21T04:02:17.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.892+0000 7fb73ffff640 1 --2- 172.21.15.64:0/439776005 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb7340b6670 0x7fb7340b8b80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb7481079c0 tx=0x7fb730008040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:17.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:17.895+0000 7fb745ffb640 1 -- 172.21.15.64:0/439776005 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb7380ad890 con 0x7fb748102a90 2024-09-21T04:02:18.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:18.109+0000 7fb74e48b640 1 -- 172.21.15.64:0/439776005 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb748063800 con 0x7fb7340b6670 2024-09-21T04:02:18.112 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:17 smithi064 ceph-mon[31180]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:02:18.112 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:17 smithi064 ceph-mon[31180]: pgmap v265: 97 pgs: 97 active+clean; 582 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-09-21T04:02:18.112 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:17 smithi064 ceph-mon[31180]: mgrmap e21: x(active, since 6m), standbys: z, y 2024-09-21T04:02:18.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:18.140+0000 7fb745ffb640 1 -- 172.21.15.64:0/439776005 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb748063800 con 0x7fb7340b6670 2024-09-21T04:02:18.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:18.142+0000 7fb74e48b640 1 -- 172.21.15.64:0/439776005 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb7340b6670 msgr2=0x7fb7340b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:18.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:18.142+0000 7fb74e48b640 1 --2- 172.21.15.64:0/439776005 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb7340b6670 0x7fb7340b8b80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb7481079c0 tx=0x7fb730008040 comp rx=0 tx=0).stop 2024-09-21T04:02:18.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:18.142+0000 7fb74e48b640 1 -- 172.21.15.64:0/439776005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb748102a90 msgr2=0x7fb74819b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:18.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:18.142+0000 7fb74e48b640 1 --2- 172.21.15.64:0/439776005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb748102a90 0x7fb74819b2c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb73800d950 tx=0x7fb73800de10 comp rx=0 tx=0).stop 2024-09-21T04:02:18.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:18.142+0000 7fb74e48b640 1 -- 172.21.15.64:0/439776005 shutdown_connections 2024-09-21T04:02:18.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:18.142+0000 7fb74e48b640 1 --2- 172.21.15.64:0/439776005 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb7340b6670 0x7fb7340b8b80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:18.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:18.142+0000 7fb74e48b640 1 --2- 172.21.15.64:0/439776005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb748108d20 0x7fb74819bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:18.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:18.142+0000 7fb74e48b640 1 --2- 172.21.15.64:0/439776005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb7481083d0 0x7fb74819b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:18.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:18.142+0000 7fb74e48b640 1 --2- 172.21.15.64:0/439776005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb748102a90 0x7fb74819b2c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:18.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:18.142+0000 7fb74e48b640 1 -- 172.21.15.64:0/439776005 >> 172.21.15.64:0/439776005 conn(0x7fb748079750 msgr2=0x7fb748105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:18.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:18.142+0000 7fb74e48b640 1 -- 172.21.15.64:0/439776005 shutdown_connections 2024-09-21T04:02:18.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:18.142+0000 7fb74e48b640 1 -- 172.21.15.64:0/439776005 wait complete. 2024-09-21T04:02:18.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:17 smithi119 ceph-mon[34811]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:02:18.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:17 smithi119 ceph-mon[34811]: pgmap v265: 97 pgs: 97 active+clean; 582 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-09-21T04:02:18.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:17 smithi119 ceph-mon[34811]: mgrmap e21: x(active, since 6m), standbys: z, y 2024-09-21T04:02:18.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:17 smithi131 ceph-mon[35193]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:02:18.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:17 smithi131 ceph-mon[35193]: pgmap v265: 97 pgs: 97 active+clean; 582 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-09-21T04:02:18.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:17 smithi131 ceph-mon[35193]: mgrmap e21: x(active, since 6m), standbys: z, y 2024-09-21T04:02:18.814 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-21T04:02:19.069 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:19.190 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:18 smithi064 ceph-mon[31180]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:02:19.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:18 smithi119 ceph-mon[34811]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:02:19.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:18 smithi131 ceph-mon[35193]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:02:19.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.542+0000 7fbbe4db5640 1 -- 172.21.15.64:0/2351493740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbe01094a0 msgr2=0x7fbbe01098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:19.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.542+0000 7fbbe4db5640 1 --2- 172.21.15.64:0/2351493740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbe01094a0 0x7fbbe01098b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009900 tx=0x7fbbd402a410 comp rx=0 tx=0).stop 2024-09-21T04:02:19.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.542+0000 7fbbe4db5640 1 -- 172.21.15.64:0/2351493740 shutdown_connections 2024-09-21T04:02:19.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.542+0000 7fbbe4db5640 1 --2- 172.21.15.64:0/2351493740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbe0109df0 0x7fbbe0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:19.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.542+0000 7fbbe4db5640 1 --2- 172.21.15.64:0/2351493740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbe01094a0 0x7fbbe01098b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:19.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.542+0000 7fbbe4db5640 1 --2- 172.21.15.64:0/2351493740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbe0103b60 0x7fbbe0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:19.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.542+0000 7fbbe4db5640 1 -- 172.21.15.64:0/2351493740 >> 172.21.15.64:0/2351493740 conn(0x7fbbe007b850 msgr2=0x7fbbe007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:19.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.542+0000 7fbbe4db5640 1 -- 172.21.15.64:0/2351493740 shutdown_connections 2024-09-21T04:02:19.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.542+0000 7fbbe4db5640 1 -- 172.21.15.64:0/2351493740 wait complete. 2024-09-21T04:02:19.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.543+0000 7fbbe4db5640 1 Processor -- start 2024-09-21T04:02:19.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.543+0000 7fbbe4db5640 1 -- start start 2024-09-21T04:02:19.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.543+0000 7fbbe4db5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbe0103b60 0x7fbbe019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:19.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.543+0000 7fbbe4db5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbe01094a0 0x7fbbe019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:19.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.543+0000 7fbbe4db5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbe0109df0 0x7fbbe019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:19.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.543+0000 7fbbe4db5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe0113ab0 con 0x7fbbe0109df0 2024-09-21T04:02:19.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.543+0000 7fbbe4db5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe0113930 con 0x7fbbe0103b60 2024-09-21T04:02:19.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.543+0000 7fbbe4db5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe0113c30 con 0x7fbbe01094a0 2024-09-21T04:02:19.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.543+0000 7fbbded9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbe0109df0 0x7fbbe019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:19.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.543+0000 7fbbded9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbe0109df0 0x7fbbe019fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45598/0 (socket says 172.21.15.64:45598) 2024-09-21T04:02:19.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.543+0000 7fbbded9d640 1 -- 172.21.15.64:0/1161988487 learned_addr learned my addr 172.21.15.64:0/1161988487 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:19.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.543+0000 7fbbde59c640 1 --2- 172.21.15.64:0/1161988487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbe0103b60 0x7fbbe019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:19.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.543+0000 7fbbddd9b640 1 --2- 172.21.15.64:0/1161988487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbe01094a0 0x7fbbe019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:19.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.544+0000 7fbbded9d640 1 -- 172.21.15.64:0/1161988487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbe01094a0 msgr2=0x7fbbe019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:19.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.544+0000 7fbbded9d640 1 --2- 172.21.15.64:0/1161988487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbe01094a0 0x7fbbe019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:19.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.544+0000 7fbbded9d640 1 -- 172.21.15.64:0/1161988487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbe0103b60 msgr2=0x7fbbe019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:19.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.544+0000 7fbbded9d640 1 --2- 172.21.15.64:0/1161988487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbe0103b60 0x7fbbe019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:19.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.544+0000 7fbbded9d640 1 -- 172.21.15.64:0/1161988487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe01a0330 con 0x7fbbe0109df0 2024-09-21T04:02:19.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.544+0000 7fbbded9d640 1 --2- 172.21.15.64:0/1161988487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbe0109df0 0x7fbbe019fcc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800eb10 tx=0x7fbbd800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:19.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.544+0000 7fbbcf7fe640 1 -- 172.21.15.64:0/1161988487 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8004b20 con 0x7fbbe0109df0 2024-09-21T04:02:19.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.544+0000 7fbbcf7fe640 1 -- 172.21.15.64:0/1161988487 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbd8004cc0 con 0x7fbbe0109df0 2024-09-21T04:02:19.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.544+0000 7fbbe4db5640 1 -- 172.21.15.64:0/1161988487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe01a0620 con 0x7fbbe0109df0 2024-09-21T04:02:19.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.544+0000 7fbbcf7fe640 1 -- 172.21.15.64:0/1161988487 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8010670 con 0x7fbbe0109df0 2024-09-21T04:02:19.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.544+0000 7fbbe4db5640 1 -- 172.21.15.64:0/1161988487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe01a6e40 con 0x7fbbe0109df0 2024-09-21T04:02:19.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.545+0000 7fbbe4db5640 1 -- 172.21.15.64:0/1161988487 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbe0104de0 con 0x7fbbe0109df0 2024-09-21T04:02:19.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.547+0000 7fbbcf7fe640 1 -- 172.21.15.64:0/1161988487 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fbbd8005000 con 0x7fbbe0109df0 2024-09-21T04:02:19.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.548+0000 7fbbcf7fe640 1 --2- 172.21.15.64:0/1161988487 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbbc80b6620 0x7fbbc80b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:19.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.548+0000 7fbbde59c640 1 --2- 172.21.15.64:0/1161988487 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbbc80b6620 0x7fbbc80b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:19.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.548+0000 7fbbcf7fe640 1 -- 172.21.15.64:0/1161988487 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8194+0+0 (secure 0 0 0) 0x7fbbd80e7d70 con 0x7fbbe0109df0 2024-09-21T04:02:19.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.548+0000 7fbbde59c640 1 --2- 172.21.15.64:0/1161988487 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbbc80b6620 0x7fbbc80b8b30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0005e10 tx=0x7fbbd0005da0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:19.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.551+0000 7fbbcf7fe640 1 -- 172.21.15.64:0/1161988487 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbbd8056710 con 0x7fbbe0109df0 2024-09-21T04:02:19.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.769+0000 7fbbe4db5640 1 -- 172.21.15.64:0/1161988487 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fbbe010bc40 con 0x7fbbc80b6620 2024-09-21T04:02:19.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.774+0000 7fbbcf7fe640 1 -- 172.21.15.64:0/1161988487 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fbbe010bc40 con 0x7fbbc80b6620 2024-09-21T04:02:19.777 INFO:teuthology.orchestra.run.smithi064.stdout:/volumes/qa/sv_0/e4a2cb47-80b6-4eb6-a064-5fbc590091f4 2024-09-21T04:02:19.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.776+0000 7fbbe4db5640 1 -- 172.21.15.64:0/1161988487 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbbc80b6620 msgr2=0x7fbbc80b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:19.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.776+0000 7fbbe4db5640 1 --2- 172.21.15.64:0/1161988487 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbbc80b6620 0x7fbbc80b8b30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0005e10 tx=0x7fbbd0005da0 comp rx=0 tx=0).stop 2024-09-21T04:02:19.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.776+0000 7fbbe4db5640 1 -- 172.21.15.64:0/1161988487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbe0109df0 msgr2=0x7fbbe019fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:19.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.776+0000 7fbbe4db5640 1 --2- 172.21.15.64:0/1161988487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbe0109df0 0x7fbbe019fcc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800eb10 tx=0x7fbbd800c2d0 comp rx=0 tx=0).stop 2024-09-21T04:02:19.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.776+0000 7fbbe4db5640 1 -- 172.21.15.64:0/1161988487 shutdown_connections 2024-09-21T04:02:19.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.776+0000 7fbbe4db5640 1 --2- 172.21.15.64:0/1161988487 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbbc80b6620 0x7fbbc80b8b30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:19.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.776+0000 7fbbe4db5640 1 --2- 172.21.15.64:0/1161988487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbbe0109df0 0x7fbbe019fcc0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:19.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.776+0000 7fbbe4db5640 1 --2- 172.21.15.64:0/1161988487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbe01094a0 0x7fbbe019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:19.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.776+0000 7fbbe4db5640 1 --2- 172.21.15.64:0/1161988487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbe0103b60 0x7fbbe019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:19.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.777+0000 7fbbe4db5640 1 -- 172.21.15.64:0/1161988487 >> 172.21.15.64:0/1161988487 conn(0x7fbbe007b850 msgr2=0x7fbbe0107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:19.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.777+0000 7fbbe4db5640 1 -- 172.21.15.64:0/1161988487 shutdown_connections 2024-09-21T04:02:19.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:19.777+0000 7fbbe4db5640 1 -- 172.21.15.64:0/1161988487 wait complete. 2024-09-21T04:02:20.222 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:19 smithi064 ceph-mon[31180]: pgmap v266: 97 pgs: 97 active+clean; 588 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-09-21T04:02:20.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:19 smithi119 ceph-mon[34811]: pgmap v266: 97 pgs: 97 active+clean; 588 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-09-21T04:02:20.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:19 smithi131 ceph-mon[35193]: pgmap v266: 97 pgs: 97 active+clean; 588 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-09-21T04:02:20.503 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-21T04:02:20.762 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:21.123 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:20 smithi064 ceph-mon[31180]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:02:21.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.219+0000 7fcb5a546640 1 -- 172.21.15.64:0/2221191795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb54109500 msgr2=0x7fcb54109910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:21.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.219+0000 7fcb5a546640 1 --2- 172.21.15.64:0/2221191795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb54109500 0x7fcb54109910 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009900 tx=0x7fcb4802a410 comp rx=0 tx=0).stop 2024-09-21T04:02:21.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.219+0000 7fcb5a546640 1 -- 172.21.15.64:0/2221191795 shutdown_connections 2024-09-21T04:02:21.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.219+0000 7fcb5a546640 1 --2- 172.21.15.64:0/2221191795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb54109e50 0x7fcb54110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:21.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.219+0000 7fcb5a546640 1 --2- 172.21.15.64:0/2221191795 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb54109500 0x7fcb54109910 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:21.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.219+0000 7fcb5a546640 1 --2- 172.21.15.64:0/2221191795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb54101fc0 0x7fcb541023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:21.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.219+0000 7fcb5a546640 1 -- 172.21.15.64:0/2221191795 >> 172.21.15.64:0/2221191795 conn(0x7fcb54079f90 msgr2=0x7fcb5407a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:21.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.219+0000 7fcb5a546640 1 -- 172.21.15.64:0/2221191795 shutdown_connections 2024-09-21T04:02:21.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.219+0000 7fcb5a546640 1 -- 172.21.15.64:0/2221191795 wait complete. 2024-09-21T04:02:21.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.220+0000 7fcb5a546640 1 Processor -- start 2024-09-21T04:02:21.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.220+0000 7fcb5a546640 1 -- start start 2024-09-21T04:02:21.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.221+0000 7fcb5a546640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb54101fc0 0x7fcb5419c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:21.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.221+0000 7fcb5a546640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb54109500 0x7fcb5419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:21.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.221+0000 7fcb5a546640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb54109e50 0x7fcb541a20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:21.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.221+0000 7fcb5a546640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcb54113b10 con 0x7fcb54109500 2024-09-21T04:02:21.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.221+0000 7fcb5a546640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcb54113990 con 0x7fcb54101fc0 2024-09-21T04:02:21.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.221+0000 7fcb5a546640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcb54113c90 con 0x7fcb54109e50 2024-09-21T04:02:21.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.221+0000 7fcb537fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb54109500 0x7fcb5419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:21.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.221+0000 7fcb537fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb54109500 0x7fcb5419ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45618/0 (socket says 172.21.15.64:45618) 2024-09-21T04:02:21.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.221+0000 7fcb537fe640 1 -- 172.21.15.64:0/3836319756 learned_addr learned my addr 172.21.15.64:0/3836319756 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:21.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.221+0000 7fcb53fff640 1 --2- 172.21.15.64:0/3836319756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb54101fc0 0x7fcb5419c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:21.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.221+0000 7fcb58ae3640 1 --2- 172.21.15.64:0/3836319756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb54109e50 0x7fcb541a20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:21.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.221+0000 7fcb537fe640 1 -- 172.21.15.64:0/3836319756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb54109e50 msgr2=0x7fcb541a20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:21.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.221+0000 7fcb537fe640 1 --2- 172.21.15.64:0/3836319756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb54109e50 0x7fcb541a20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:21.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.221+0000 7fcb537fe640 1 -- 172.21.15.64:0/3836319756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb54101fc0 msgr2=0x7fcb5419c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:21.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.221+0000 7fcb537fe640 1 --2- 172.21.15.64:0/3836319756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb54101fc0 0x7fcb5419c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:21.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.221+0000 7fcb537fe640 1 -- 172.21.15.64:0/3836319756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb541a27f0 con 0x7fcb54109500 2024-09-21T04:02:21.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.221+0000 7fcb58ae3640 1 --2- 172.21.15.64:0/3836319756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb54109e50 0x7fcb541a20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:02:21.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.221+0000 7fcb537fe640 1 --2- 172.21.15.64:0/3836319756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb54109500 0x7fcb5419ce20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcb4802a930 tx=0x7fcb48004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:21.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.221+0000 7fcb517fa640 1 -- 172.21.15.64:0/3836319756 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4803f030 con 0x7fcb54109500 2024-09-21T04:02:21.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.222+0000 7fcb5a546640 1 -- 172.21.15.64:0/3836319756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb541a2a80 con 0x7fcb54109500 2024-09-21T04:02:21.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.222+0000 7fcb517fa640 1 -- 172.21.15.64:0/3836319756 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb48003aa0 con 0x7fcb54109500 2024-09-21T04:02:21.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.222+0000 7fcb517fa640 1 -- 172.21.15.64:0/3836319756 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb48035070 con 0x7fcb54109500 2024-09-21T04:02:21.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.222+0000 7fcb5a546640 1 -- 172.21.15.64:0/3836319756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb541a9240 con 0x7fcb54109500 2024-09-21T04:02:21.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.223+0000 7fcb5a546640 1 -- 172.21.15.64:0/3836319756 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb5406b930 con 0x7fcb54109500 2024-09-21T04:02:21.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.225+0000 7fcb517fa640 1 -- 172.21.15.64:0/3836319756 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fcb48042030 con 0x7fcb54109500 2024-09-21T04:02:21.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.226+0000 7fcb517fa640 1 --2- 172.21.15.64:0/3836319756 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcb3c0b65f0 0x7fcb3c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:21.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.226+0000 7fcb53fff640 1 --2- 172.21.15.64:0/3836319756 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcb3c0b65f0 0x7fcb3c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:21.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.226+0000 7fcb517fa640 1 -- 172.21.15.64:0/3836319756 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8194+0+0 (secure 0 0 0) 0x7fcb48107f80 con 0x7fcb54109500 2024-09-21T04:02:21.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.227+0000 7fcb53fff640 1 --2- 172.21.15.64:0/3836319756 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcb3c0b65f0 0x7fcb3c0b8b00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcb440058c0 tx=0x7fcb4400a4c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:21.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.230+0000 7fcb517fa640 1 -- 172.21.15.64:0/3836319756 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcb480ce770 con 0x7fcb54109500 2024-09-21T04:02:21.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:20 smithi119 ceph-mon[34811]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:02:21.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:20 smithi131 ceph-mon[35193]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:02:21.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.484+0000 7fcb5a546640 1 -- 172.21.15.64:0/3836319756 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fcb5419d9b0 con 0x7fcb3c0b65f0 2024-09-21T04:02:21.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.513+0000 7fcb517fa640 1 -- 172.21.15.64:0/3836319756 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcb5419d9b0 con 0x7fcb3c0b65f0 2024-09-21T04:02:21.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.515+0000 7fcb5a546640 1 -- 172.21.15.64:0/3836319756 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcb3c0b65f0 msgr2=0x7fcb3c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:21.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.515+0000 7fcb5a546640 1 --2- 172.21.15.64:0/3836319756 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcb3c0b65f0 0x7fcb3c0b8b00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcb440058c0 tx=0x7fcb4400a4c0 comp rx=0 tx=0).stop 2024-09-21T04:02:21.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.515+0000 7fcb5a546640 1 -- 172.21.15.64:0/3836319756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb54109500 msgr2=0x7fcb5419ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:21.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.515+0000 7fcb5a546640 1 --2- 172.21.15.64:0/3836319756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb54109500 0x7fcb5419ce20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcb4802a930 tx=0x7fcb48004270 comp rx=0 tx=0).stop 2024-09-21T04:02:21.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.515+0000 7fcb5a546640 1 -- 172.21.15.64:0/3836319756 shutdown_connections 2024-09-21T04:02:21.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.515+0000 7fcb5a546640 1 --2- 172.21.15.64:0/3836319756 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcb3c0b65f0 0x7fcb3c0b8b00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:21.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.515+0000 7fcb5a546640 1 --2- 172.21.15.64:0/3836319756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb54109e50 0x7fcb541a20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:21.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.515+0000 7fcb5a546640 1 --2- 172.21.15.64:0/3836319756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb54109500 0x7fcb5419ce20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:21.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.515+0000 7fcb5a546640 1 --2- 172.21.15.64:0/3836319756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb54101fc0 0x7fcb5419c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:21.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.515+0000 7fcb5a546640 1 -- 172.21.15.64:0/3836319756 >> 172.21.15.64:0/3836319756 conn(0x7fcb54079f90 msgr2=0x7fcb54107310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:21.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.515+0000 7fcb5a546640 1 -- 172.21.15.64:0/3836319756 shutdown_connections 2024-09-21T04:02:21.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:21.515+0000 7fcb5a546640 1 -- 172.21.15.64:0/3836319756 wait complete. 2024-09-21T04:02:22.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:21 smithi119 ceph-mon[34811]: pgmap v267: 97 pgs: 97 active+clean; 588 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-21T04:02:22.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:21 smithi064 ceph-mon[31180]: pgmap v267: 97 pgs: 97 active+clean; 588 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-21T04:02:22.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:21 smithi131 ceph-mon[35193]: pgmap v267: 97 pgs: 97 active+clean; 588 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-21T04:02:22.346 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-21T04:02:22.609 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:22.983 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:22 smithi064 ceph-mon[31180]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:02:23.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.078+0000 7f4417b7d640 1 -- 172.21.15.64:0/3500476418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4410109e20 msgr2=0x7f4410110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:23.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.078+0000 7f4417b7d640 1 --2- 172.21.15.64:0/3500476418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4410109e20 0x7f4410110650 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4408009ca0 tx=0x7f440802a8c0 comp rx=0 tx=0).stop 2024-09-21T04:02:23.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.078+0000 7f4417b7d640 1 -- 172.21.15.64:0/3500476418 shutdown_connections 2024-09-21T04:02:23.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.078+0000 7f4417b7d640 1 --2- 172.21.15.64:0/3500476418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4410109e20 0x7f4410110650 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:23.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.078+0000 7f4417b7d640 1 --2- 172.21.15.64:0/3500476418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44101094d0 0x7f44101098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:23.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.078+0000 7f4417b7d640 1 --2- 172.21.15.64:0/3500476418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4410103b90 0x7f4410103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:23.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.078+0000 7f4417b7d640 1 -- 172.21.15.64:0/3500476418 >> 172.21.15.64:0/3500476418 conn(0x7f44100ff600 msgr2=0x7f4410101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:23.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.078+0000 7f4417b7d640 1 -- 172.21.15.64:0/3500476418 shutdown_connections 2024-09-21T04:02:23.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.078+0000 7f4417b7d640 1 -- 172.21.15.64:0/3500476418 wait complete. 2024-09-21T04:02:23.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.079+0000 7f4417b7d640 1 Processor -- start 2024-09-21T04:02:23.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.079+0000 7f4417b7d640 1 -- start start 2024-09-21T04:02:23.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.079+0000 7f4417b7d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4410103b90 0x7f4410075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:23.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.079+0000 7f4417b7d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44101094d0 0x7f4410075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:23.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.079+0000 7f4417b7d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4410109e20 0x7f4410076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:23.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.079+0000 7f4417b7d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4410113ae0 con 0x7f44101094d0 2024-09-21T04:02:23.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.079+0000 7f4417b7d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4410113960 con 0x7f4410103b90 2024-09-21T04:02:23.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.079+0000 7f4417b7d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4410113c60 con 0x7f4410109e20 2024-09-21T04:02:23.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.079+0000 7f4415118640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44101094d0 0x7f4410075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:23.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.079+0000 7f4415118640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44101094d0 0x7f4410075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34616/0 (socket says 172.21.15.64:34616) 2024-09-21T04:02:23.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.080+0000 7f4415118640 1 -- 172.21.15.64:0/1367054847 learned_addr learned my addr 172.21.15.64:0/1367054847 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:23.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.080+0000 7f4415919640 1 --2- 172.21.15.64:0/1367054847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4410103b90 0x7f4410075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:23.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.080+0000 7f441611a640 1 --2- 172.21.15.64:0/1367054847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4410109e20 0x7f4410076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:23.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.080+0000 7f4415118640 1 -- 172.21.15.64:0/1367054847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4410109e20 msgr2=0x7f4410076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:23.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.080+0000 7f4415118640 1 --2- 172.21.15.64:0/1367054847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4410109e20 0x7f4410076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:23.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.080+0000 7f4415118640 1 -- 172.21.15.64:0/1367054847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4410103b90 msgr2=0x7f4410075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:23.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.080+0000 7f4415118640 1 --2- 172.21.15.64:0/1367054847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4410103b90 0x7f4410075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:23.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.080+0000 7f4415118640 1 -- 172.21.15.64:0/1367054847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44101a8640 con 0x7f44101094d0 2024-09-21T04:02:23.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.080+0000 7f441611a640 1 --2- 172.21.15.64:0/1367054847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4410109e20 0x7f4410076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:02:23.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.080+0000 7f4415118640 1 --2- 172.21.15.64:0/1367054847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44101094d0 0x7f4410075c50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f440000c970 tx=0x7f440000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:23.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.080+0000 7f4406ffd640 1 -- 172.21.15.64:0/1367054847 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400007bb0 con 0x7f44101094d0 2024-09-21T04:02:23.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.080+0000 7f4406ffd640 1 -- 172.21.15.64:0/1367054847 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4400007d50 con 0x7f44101094d0 2024-09-21T04:02:23.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.080+0000 7f4406ffd640 1 -- 172.21.15.64:0/1367054847 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4400004d90 con 0x7f44101094d0 2024-09-21T04:02:23.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.080+0000 7f4417b7d640 1 -- 172.21.15.64:0/1367054847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44101a8930 con 0x7f44101094d0 2024-09-21T04:02:23.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.080+0000 7f4417b7d640 1 -- 172.21.15.64:0/1367054847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44101a8e70 con 0x7f44101094d0 2024-09-21T04:02:23.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.081+0000 7f4417b7d640 1 -- 172.21.15.64:0/1367054847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4410103fa0 con 0x7f44101094d0 2024-09-21T04:02:23.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.083+0000 7f4406ffd640 1 -- 172.21.15.64:0/1367054847 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f4400004f50 con 0x7f44101094d0 2024-09-21T04:02:23.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.084+0000 7f4406ffd640 1 --2- 172.21.15.64:0/1367054847 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f43fc0b6670 0x7f43fc0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:23.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.084+0000 7f4415919640 1 --2- 172.21.15.64:0/1367054847 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f43fc0b6670 0x7f43fc0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:23.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.084+0000 7f4406ffd640 1 -- 172.21.15.64:0/1367054847 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8194+0+0 (secure 0 0 0) 0x7f44000e7530 con 0x7f44101094d0 2024-09-21T04:02:23.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.085+0000 7f4415919640 1 --2- 172.21.15.64:0/1367054847 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f43fc0b6670 0x7f43fc0b8b80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4410077160 tx=0x7f440c00a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:23.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.087+0000 7f4406ffd640 1 -- 172.21.15.64:0/1367054847 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4400014030 con 0x7f44101094d0 2024-09-21T04:02:23.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:22 smithi119 ceph-mon[34811]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:02:23.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:22 smithi131 ceph-mon[35193]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:02:23.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.304+0000 7f4417b7d640 1 -- 172.21.15.64:0/1367054847 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f4410063800 con 0x7f43fc0b6670 2024-09-21T04:02:23.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.308+0000 7f4406ffd640 1 -- 172.21.15.64:0/1367054847 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f4410063800 con 0x7f43fc0b6670 2024-09-21T04:02:23.310 INFO:teuthology.orchestra.run.smithi064.stdout:/volumes/qa/sv_1/071d8841-4290-45c9-ae90-a63e004e386b 2024-09-21T04:02:23.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.310+0000 7f4417b7d640 1 -- 172.21.15.64:0/1367054847 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f43fc0b6670 msgr2=0x7f43fc0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:23.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.310+0000 7f4417b7d640 1 --2- 172.21.15.64:0/1367054847 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f43fc0b6670 0x7f43fc0b8b80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4410077160 tx=0x7f440c00a2d0 comp rx=0 tx=0).stop 2024-09-21T04:02:23.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.310+0000 7f4417b7d640 1 -- 172.21.15.64:0/1367054847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44101094d0 msgr2=0x7f4410075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:23.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.310+0000 7f4417b7d640 1 --2- 172.21.15.64:0/1367054847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44101094d0 0x7f4410075c50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f440000c970 tx=0x7f440000cd70 comp rx=0 tx=0).stop 2024-09-21T04:02:23.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.310+0000 7f4417b7d640 1 -- 172.21.15.64:0/1367054847 shutdown_connections 2024-09-21T04:02:23.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.310+0000 7f4417b7d640 1 --2- 172.21.15.64:0/1367054847 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f43fc0b6670 0x7f43fc0b8b80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:23.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.310+0000 7f4417b7d640 1 --2- 172.21.15.64:0/1367054847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4410109e20 0x7f4410076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:23.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.310+0000 7f4417b7d640 1 --2- 172.21.15.64:0/1367054847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44101094d0 0x7f4410075c50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:23.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.310+0000 7f4417b7d640 1 --2- 172.21.15.64:0/1367054847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4410103b90 0x7f4410075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:23.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.310+0000 7f4417b7d640 1 -- 172.21.15.64:0/1367054847 >> 172.21.15.64:0/1367054847 conn(0x7f44100ff600 msgr2=0x7f4410108b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:23.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.311+0000 7f4417b7d640 1 -- 172.21.15.64:0/1367054847 shutdown_connections 2024-09-21T04:02:23.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:23.311+0000 7f4417b7d640 1 -- 172.21.15.64:0/1367054847 wait complete. 2024-09-21T04:02:24.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:23 smithi064 ceph-mon[31180]: pgmap v268: 97 pgs: 97 active+clean; 624 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.3 KiB/s wr, 14 op/s 2024-09-21T04:02:24.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:23 smithi064 ceph-mon[31180]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:02:24.101 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:02:24.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:23 smithi119 ceph-mon[34811]: pgmap v268: 97 pgs: 97 active+clean; 624 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.3 KiB/s wr, 14 op/s 2024-09-21T04:02:24.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:23 smithi119 ceph-mon[34811]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:02:24.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:23 smithi131 ceph-mon[35193]: pgmap v268: 97 pgs: 97 active+clean; 624 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.3 KiB/s wr, 14 op/s 2024-09-21T04:02:24.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:23 smithi131 ceph-mon[35193]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:02:24.359 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:24.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.891+0000 7f60ab988640 1 -- 172.21.15.64:0/512051841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60a41101a0 msgr2=0x7f60a41125c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:24.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.891+0000 7f60ab988640 1 --2- 172.21.15.64:0/512051841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60a41101a0 0x7f60a41125c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f609c009ca0 tx=0x7f609c02a950 comp rx=0 tx=0).stop 2024-09-21T04:02:24.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.891+0000 7f60ab988640 1 -- 172.21.15.64:0/512051841 shutdown_connections 2024-09-21T04:02:24.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.891+0000 7f60ab988640 1 --2- 172.21.15.64:0/512051841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60a41101a0 0x7f60a41125c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:24.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.891+0000 7f60ab988640 1 --2- 172.21.15.64:0/512051841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60a4102f40 0x7f60a410fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:24.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.891+0000 7f60ab988640 1 --2- 172.21.15.64:0/512051841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60a41025f0 0x7f60a4102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:24.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.891+0000 7f60ab988640 1 -- 172.21.15.64:0/512051841 >> 172.21.15.64:0/512051841 conn(0x7f60a4079740 msgr2=0x7f60a4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:24.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.891+0000 7f60ab988640 1 -- 172.21.15.64:0/512051841 shutdown_connections 2024-09-21T04:02:24.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.891+0000 7f60ab988640 1 -- 172.21.15.64:0/512051841 wait complete. 2024-09-21T04:02:24.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.892+0000 7f60ab988640 1 Processor -- start 2024-09-21T04:02:24.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.892+0000 7f60ab988640 1 -- start start 2024-09-21T04:02:24.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.892+0000 7f60ab988640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60a41025f0 0x7f60a419c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:24.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.892+0000 7f60ab988640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60a4102f40 0x7f60a419cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:24.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.892+0000 7f60ab988640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60a41101a0 0x7f60a41a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:24.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.892+0000 7f60ab988640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f60a4115c90 con 0x7f60a4102f40 2024-09-21T04:02:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.892+0000 7f60ab988640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f60a4115b10 con 0x7f60a41025f0 2024-09-21T04:02:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.892+0000 7f60ab988640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f60a4115e10 con 0x7f60a41101a0 2024-09-21T04:02:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.892+0000 7f60a8f23640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60a4102f40 0x7f60a419cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.892+0000 7f60a8f23640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60a4102f40 0x7f60a419cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34628/0 (socket says 172.21.15.64:34628) 2024-09-21T04:02:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.892+0000 7f60a8f23640 1 -- 172.21.15.64:0/1023266803 learned_addr learned my addr 172.21.15.64:0/1023266803 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.892+0000 7f60a9724640 1 --2- 172.21.15.64:0/1023266803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60a41025f0 0x7f60a419c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.892+0000 7f60a9f25640 1 --2- 172.21.15.64:0/1023266803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60a41101a0 0x7f60a41a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.892+0000 7f60a8f23640 1 -- 172.21.15.64:0/1023266803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60a41101a0 msgr2=0x7f60a41a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.892+0000 7f60a8f23640 1 --2- 172.21.15.64:0/1023266803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60a41101a0 0x7f60a41a1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.892+0000 7f60a8f23640 1 -- 172.21.15.64:0/1023266803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60a41025f0 msgr2=0x7f60a419c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.892+0000 7f60a8f23640 1 --2- 172.21.15.64:0/1023266803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60a41025f0 0x7f60a419c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.892+0000 7f60a8f23640 1 -- 172.21.15.64:0/1023266803 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a41a2530 con 0x7f60a4102f40 2024-09-21T04:02:24.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.893+0000 7f60a8f23640 1 --2- 172.21.15.64:0/1023266803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60a4102f40 0x7f60a419cb60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f609400d950 tx=0x7f609400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:24.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.893+0000 7f609a7fc640 1 -- 172.21.15.64:0/1023266803 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094017070 con 0x7f60a4102f40 2024-09-21T04:02:24.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.893+0000 7f609a7fc640 1 -- 172.21.15.64:0/1023266803 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6094009c30 con 0x7f60a4102f40 2024-09-21T04:02:24.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.893+0000 7f60ab988640 1 -- 172.21.15.64:0/1023266803 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a41a2820 con 0x7f60a4102f40 2024-09-21T04:02:24.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.893+0000 7f609a7fc640 1 -- 172.21.15.64:0/1023266803 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609400f620 con 0x7f60a4102f40 2024-09-21T04:02:24.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.893+0000 7f60ab988640 1 -- 172.21.15.64:0/1023266803 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a41a8f80 con 0x7f60a4102f40 2024-09-21T04:02:24.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.894+0000 7f60ab988640 1 -- 172.21.15.64:0/1023266803 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a406c220 con 0x7f60a4102f40 2024-09-21T04:02:24.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.896+0000 7f609a7fc640 1 -- 172.21.15.64:0/1023266803 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f6094005030 con 0x7f60a4102f40 2024-09-21T04:02:24.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.896+0000 7f609a7fc640 1 --2- 172.21.15.64:0/1023266803 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f60900b65f0 0x7f60900b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:24.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.896+0000 7f60a9724640 1 --2- 172.21.15.64:0/1023266803 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f60900b65f0 0x7f60900b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:24.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.897+0000 7f609a7fc640 1 -- 172.21.15.64:0/1023266803 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8194+0+0 (secure 0 0 0) 0x7f60940e7190 con 0x7f60a4102f40 2024-09-21T04:02:24.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.897+0000 7f60a9724640 1 --2- 172.21.15.64:0/1023266803 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f60900b65f0 0x7f60900b8b00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f60a0005fd0 tx=0x7f60a0005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:24.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:24.899+0000 7f609a7fc640 1 -- 172.21.15.64:0/1023266803 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f60940ad8e0 con 0x7f60a4102f40 2024-09-21T04:02:25.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:25.125+0000 7f60ab988640 1 -- 172.21.15.64:0/1023266803 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60a419d6f0 con 0x7f60a4102f40 2024-09-21T04:02:25.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:25.126+0000 7f609a7fc640 1 -- 172.21.15.64:0/1023266803 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11911 (secure 0 0 0) 0x7f6094009de0 con 0x7f60a4102f40 2024-09-21T04:02:25.128 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:02:25.129 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"btime":"2024-09-21T04:02:14:798588+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14805,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/97995119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":97995119},{"type":"v1","addr":"172.21.15.64:6839","nonce":97995119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/390891068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":390891068},{"type":"v1","addr":"172.21.15.64:6841","nonce":390891068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:02:03.354744+0000","modified":"2024-09-21T04:02:14.579750+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24710,"mds_1":24680,"mds_2":24661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24634":{"gid":24634,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6839/782190424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782190424},{"type":"v1","addr":"172.21.15.131:6839","nonce":782190424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24680":{"gid":24680,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24710":{"gid":24710,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6839/1319793636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1319793636},{"type":"v1","addr":"172.21.15.119:6839","nonce":1319793636}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24710,"qdb_cluster":[24661,24680,24710]},"id":1}]} 2024-09-21T04:02:25.129 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-09-21T04:02:25.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:25.129+0000 7f60ab988640 1 -- 172.21.15.64:0/1023266803 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f60900b65f0 msgr2=0x7f60900b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:25.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:25.129+0000 7f60ab988640 1 --2- 172.21.15.64:0/1023266803 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f60900b65f0 0x7f60900b8b00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f60a0005fd0 tx=0x7f60a0005850 comp rx=0 tx=0).stop 2024-09-21T04:02:25.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:25.129+0000 7f60ab988640 1 -- 172.21.15.64:0/1023266803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60a4102f40 msgr2=0x7f60a419cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:25.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:25.129+0000 7f60ab988640 1 --2- 172.21.15.64:0/1023266803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60a4102f40 0x7f60a419cb60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f609400d950 tx=0x7f609400de10 comp rx=0 tx=0).stop 2024-09-21T04:02:25.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:25.129+0000 7f60ab988640 1 -- 172.21.15.64:0/1023266803 shutdown_connections 2024-09-21T04:02:25.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:25.129+0000 7f60ab988640 1 --2- 172.21.15.64:0/1023266803 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f60900b65f0 0x7f60900b8b00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:25.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:25.129+0000 7f60ab988640 1 --2- 172.21.15.64:0/1023266803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60a41101a0 0x7f60a41a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:25.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:25.129+0000 7f60ab988640 1 --2- 172.21.15.64:0/1023266803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f60a4102f40 0x7f60a419cb60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:25.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:25.129+0000 7f60ab988640 1 --2- 172.21.15.64:0/1023266803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60a41025f0 0x7f60a419c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:25.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:25.129+0000 7f60ab988640 1 -- 172.21.15.64:0/1023266803 >> 172.21.15.64:0/1023266803 conn(0x7f60a4079740 msgr2=0x7f60a4109cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:25.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:25.129+0000 7f60ab988640 1 -- 172.21.15.64:0/1023266803 shutdown_connections 2024-09-21T04:02:25.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:25.129+0000 7f60ab988640 1 -- 172.21.15.64:0/1023266803 wait complete. 2024-09-21T04:02:25.907 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T04:02:26.170 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:26.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:25 smithi119 ceph-mon[34811]: pgmap v269: 97 pgs: 97 active+clean; 624 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-09-21T04:02:26.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:25 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1023266803' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:02:26.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:25 smithi064 ceph-mon[31180]: pgmap v269: 97 pgs: 97 active+clean; 624 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-09-21T04:02:26.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:25 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1023266803' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:02:26.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:25 smithi131 ceph-mon[35193]: pgmap v269: 97 pgs: 97 active+clean; 624 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-09-21T04:02:26.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:25 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1023266803' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:02:26.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.684+0000 7f851be72640 1 -- 172.21.15.64:0/1036684409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85141052b0 msgr2=0x7f851410d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:26.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.684+0000 7f851be72640 1 --2- 172.21.15.64:0/1036684409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85141052b0 0x7f851410d8f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8508009900 tx=0x7f850802a410 comp rx=0 tx=0).stop 2024-09-21T04:02:26.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.684+0000 7f851be72640 1 -- 172.21.15.64:0/1036684409 shutdown_connections 2024-09-21T04:02:26.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.684+0000 7f851be72640 1 --2- 172.21.15.64:0/1036684409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f851410de30 0x7f8514110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:26.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.684+0000 7f851be72640 1 --2- 172.21.15.64:0/1036684409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f85141052b0 0x7f851410d8f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:26.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.684+0000 7f851be72640 1 --2- 172.21.15.64:0/1036684409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8514104880 0x7f8514104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:26.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.684+0000 7f851be72640 1 -- 172.21.15.64:0/1036684409 >> 172.21.15.64:0/1036684409 conn(0x7f8514079730 msgr2=0x7f8514079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:26.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.684+0000 7f851be72640 1 -- 172.21.15.64:0/1036684409 shutdown_connections 2024-09-21T04:02:26.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.684+0000 7f851be72640 1 -- 172.21.15.64:0/1036684409 wait complete. 2024-09-21T04:02:26.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.685+0000 7f851be72640 1 Processor -- start 2024-09-21T04:02:26.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.685+0000 7f851be72640 1 -- start start 2024-09-21T04:02:26.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.685+0000 7f851be72640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8514104880 0x7f851419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:26.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.685+0000 7f851be72640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85141052b0 0x7f851419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:26.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.685+0000 7f851be72640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f851410de30 0x7f851419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:26.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.685+0000 7f851be72640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8514113a70 con 0x7f8514104880 2024-09-21T04:02:26.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.685+0000 7f851be72640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f85141138f0 con 0x7f851410de30 2024-09-21T04:02:26.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.685+0000 7f851be72640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8514113bf0 con 0x7f85141052b0 2024-09-21T04:02:26.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.685+0000 7f8519c0e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8514104880 0x7f851419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:26.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.685+0000 7f8519c0e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8514104880 0x7f851419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34646/0 (socket says 172.21.15.64:34646) 2024-09-21T04:02:26.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.685+0000 7f8519c0e640 1 -- 172.21.15.64:0/260720161 learned_addr learned my addr 172.21.15.64:0/260720161 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:26.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.686+0000 7f851940d640 1 --2- 172.21.15.64:0/260720161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85141052b0 0x7f851419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:26.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.686+0000 7f851a40f640 1 --2- 172.21.15.64:0/260720161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f851410de30 0x7f851419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:26.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.686+0000 7f8519c0e640 1 -- 172.21.15.64:0/260720161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85141052b0 msgr2=0x7f851419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:26.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.686+0000 7f8519c0e640 1 --2- 172.21.15.64:0/260720161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85141052b0 0x7f851419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:26.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.686+0000 7f8519c0e640 1 -- 172.21.15.64:0/260720161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f851410de30 msgr2=0x7f851419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:26.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.686+0000 7f8519c0e640 1 --2- 172.21.15.64:0/260720161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f851410de30 0x7f851419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:26.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.686+0000 7f8519c0e640 1 -- 172.21.15.64:0/260720161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85141a03d0 con 0x7f8514104880 2024-09-21T04:02:26.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.686+0000 7f851a40f640 1 --2- 172.21.15.64:0/260720161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f851410de30 0x7f851419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:02:26.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.686+0000 7f8519c0e640 1 --2- 172.21.15.64:0/260720161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8514104880 0x7f851419a460 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f850c00e9c0 tx=0x7f850c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:26.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.686+0000 7f8512ffd640 1 -- 172.21.15.64:0/260720161 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c00cd80 con 0x7f8514104880 2024-09-21T04:02:26.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.686+0000 7f851be72640 1 -- 172.21.15.64:0/260720161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85141a06c0 con 0x7f8514104880 2024-09-21T04:02:26.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.686+0000 7f8512ffd640 1 -- 172.21.15.64:0/260720161 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f850c0035f0 con 0x7f8514104880 2024-09-21T04:02:26.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.686+0000 7f8512ffd640 1 -- 172.21.15.64:0/260720161 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c010690 con 0x7f8514104880 2024-09-21T04:02:26.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.686+0000 7f851be72640 1 -- 172.21.15.64:0/260720161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85141a6e20 con 0x7f8514104880 2024-09-21T04:02:26.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.687+0000 7f851be72640 1 -- 172.21.15.64:0/260720161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8514104c90 con 0x7f8514104880 2024-09-21T04:02:26.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.689+0000 7f8512ffd640 1 -- 172.21.15.64:0/260720161 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f850c004000 con 0x7f8514104880 2024-09-21T04:02:26.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.689+0000 7f8512ffd640 1 --2- 172.21.15.64:0/260720161 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f84fc0b65f0 0x7f84fc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:26.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.690+0000 7f851940d640 1 --2- 172.21.15.64:0/260720161 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f84fc0b65f0 0x7f84fc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:26.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.690+0000 7f8512ffd640 1 -- 172.21.15.64:0/260720161 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8194+0+0 (secure 0 0 0) 0x7f850c0e7c60 con 0x7f8514104880 2024-09-21T04:02:26.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.690+0000 7f851940d640 1 --2- 172.21.15.64:0/260720161 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f84fc0b65f0 0x7f84fc0b8b00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8508000c00 tx=0x7f8508031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:26.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.693+0000 7f8512ffd640 1 -- 172.21.15.64:0/260720161 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f850c0ae3b0 con 0x7f8514104880 2024-09-21T04:02:26.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.888+0000 7f851be72640 1 -- 172.21.15.64:0/260720161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f851419b590 con 0x7f8514104880 2024-09-21T04:02:26.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.890+0000 7f8512ffd640 1 -- 172.21.15.64:0/260720161 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+22620 (secure 0 0 0) 0x7f850c0b2790 con 0x7f8514104880 2024-09-21T04:02:26.893 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:02:26.893 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":73,"fsid":"e7c927ba-77cc-11ef-baf5-efdc52797490","created":"2024-09-21T03:53:41.215073+0000","modified":"2024-09-21T04:02:03.354789+0000","last_up_change":"2024-09-21T04:01:11.986949+0000","last_in_change":"2024-09-21T04:00:48.707791+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:57:24.854438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T04:01:58.728355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T04:02:01.300385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb947c10-98e6-4e3a-8bbd-8753951d870b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6803","nonce":3692023035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6805","nonce":3692023035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6809","nonce":3692023035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6807","nonce":3692023035}]},"public_addr":"172.21.15.64:6803/3692023035","cluster_addr":"172.21.15.64:6805/3692023035","heartbeat_back_addr":"172.21.15.64:6809/3692023035","heartbeat_front_addr":"172.21.15.64:6807/3692023035","state":["exists","up"]},{"osd":1,"uuid":"29b8c92d-82fa-4175-b3e2-711917b7f8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6801","nonce":3363415998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6803","nonce":3363415998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6807","nonce":3363415998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6805","nonce":3363415998}]},"public_addr":"172.21.15.119:6801/3363415998","cluster_addr":"172.21.15.119:6803/3363415998","heartbeat_back_addr":"172.21.15.119:6807/3363415998","heartbeat_front_addr":"172.21.15.119:6805/3363415998","state":["exists","up"]},{"osd":2,"uuid":"a2273fe8-df55-40f7-afda-74ab09c52c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6801","nonce":3422924179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6803","nonce":3422924179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6807","nonce":3422924179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6805","nonce":3422924179}]},"public_addr":"172.21.15.131:6801/3422924179","cluster_addr":"172.21.15.131:6803/3422924179","heartbeat_back_addr":"172.21.15.131:6807/3422924179","heartbeat_front_addr":"172.21.15.131:6805/3422924179","state":["exists","up"]},{"osd":3,"uuid":"f1634064-b108-41d2-a78f-c33607b08a70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6811","nonce":2022952729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6813","nonce":2022952729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6817","nonce":2022952729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6815","nonce":2022952729}]},"public_addr":"172.21.15.64:6811/2022952729","cluster_addr":"172.21.15.64:6813/2022952729","heartbeat_back_addr":"172.21.15.64:6817/2022952729","heartbeat_front_addr":"172.21.15.64:6815/2022952729","state":["exists","up"]},{"osd":4,"uuid":"75b81a8d-fa52-411a-bfda-aa69bb70a20f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6809","nonce":323030958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6811","nonce":323030958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6815","nonce":323030958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6813","nonce":323030958}]},"public_addr":"172.21.15.119:6809/323030958","cluster_addr":"172.21.15.119:6811/323030958","heartbeat_back_addr":"172.21.15.119:6815/323030958","heartbeat_front_addr":"172.21.15.119:6813/323030958","state":["exists","up"]},{"osd":5,"uuid":"0e4895b5-42e4-4b2c-a9c3-692f287cb841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6809","nonce":3021788088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6811","nonce":3021788088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6815","nonce":3021788088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6813","nonce":3021788088}]},"public_addr":"172.21.15.131:6809/3021788088","cluster_addr":"172.21.15.131:6811/3021788088","heartbeat_back_addr":"172.21.15.131:6815/3021788088","heartbeat_front_addr":"172.21.15.131:6813/3021788088","state":["exists","up"]},{"osd":6,"uuid":"1fa5c2b7-bce0-43a1-a26b-d029eb3731fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6819","nonce":1913824324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6821","nonce":1913824324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6825","nonce":1913824324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6823","nonce":1913824324}]},"public_addr":"172.21.15.64:6819/1913824324","cluster_addr":"172.21.15.64:6821/1913824324","heartbeat_back_addr":"172.21.15.64:6825/1913824324","heartbeat_front_addr":"172.21.15.64:6823/1913824324","state":["exists","up"]},{"osd":7,"uuid":"c9707260-b581-4a21-b380-f37e16959b02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6817","nonce":2567568166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6819","nonce":2567568166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6823","nonce":2567568166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6821","nonce":2567568166}]},"public_addr":"172.21.15.119:6817/2567568166","cluster_addr":"172.21.15.119:6819/2567568166","heartbeat_back_addr":"172.21.15.119:6823/2567568166","heartbeat_front_addr":"172.21.15.119:6821/2567568166","state":["exists","up"]},{"osd":8,"uuid":"2dbf8ef2-fc9c-4c6b-bfea-003f41ef191e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6817","nonce":199192217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6819","nonce":199192217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6823","nonce":199192217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6821","nonce":199192217}]},"public_addr":"172.21.15.131:6817/199192217","cluster_addr":"172.21.15.131:6819/199192217","heartbeat_back_addr":"172.21.15.131:6823/199192217","heartbeat_front_addr":"172.21.15.131:6821/199192217","state":["exists","up"]},{"osd":9,"uuid":"96f769e5-4783-4558-adfa-4bfaf9da9e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6827","nonce":3309038990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6829","nonce":3309038990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6833","nonce":3309038990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6831","nonce":3309038990}]},"public_addr":"172.21.15.64:6827/3309038990","cluster_addr":"172.21.15.64:6829/3309038990","heartbeat_back_addr":"172.21.15.64:6833/3309038990","heartbeat_front_addr":"172.21.15.64:6831/3309038990","state":["exists","up"]},{"osd":10,"uuid":"13c1ef5c-2299-440c-af5a-a650b47f4b75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6825","nonce":1460844681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6827","nonce":1460844681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6831","nonce":1460844681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6829","nonce":1460844681}]},"public_addr":"172.21.15.119:6825/1460844681","cluster_addr":"172.21.15.119:6827/1460844681","heartbeat_back_addr":"172.21.15.119:6831/1460844681","heartbeat_front_addr":"172.21.15.119:6829/1460844681","state":["exists","up"]},{"osd":11,"uuid":"c26a5ef9-8585-404e-a54a-852424c4d6fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6825","nonce":2294059084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6827","nonce":2294059084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6831","nonce":2294059084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6829","nonce":2294059084}]},"public_addr":"172.21.15.131:6825/2294059084","cluster_addr":"172.21.15.131:6827/2294059084","heartbeat_back_addr":"172.21.15.131:6831/2294059084","heartbeat_front_addr":"172.21.15.131:6829/2294059084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:56:33.141102+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:56:58.883062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:57:22.641789+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:57:48.705790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:58:12.871514+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:58:37.893118+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:03.620828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:29.446179+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:55.002978+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:00:19.987463+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:00:45.364158+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:01:10.228248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/3554471670":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6801/3786213009":"2024-09-22T03:55:28.387426+0000","172.21.15.64:0/2562856406":"2024-09-22T03:54:27.180100+0000","172.21.15.64:0/2603015699":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6800/1209932961":"2024-09-22T03:54:02.562380+0000","172.21.15.64:6800/3786213009":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6801/1209932961":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/3407344596":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1233731831":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1686887701":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1598568021":"2024-09-22T03:54:27.180100+0000","172.21.15.64:6800/2013929312":"2024-09-22T03:54:27.180100+0000","172.21.15.64:6801/2013929312":"2024-09-22T03:54:27.180100+0000","172.21.15.64:0/3652036144":"2024-09-22T03:55:28.387426+0000","172.21.15.64:0/105454703":"2024-09-22T03:54:27.180100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T04:02:26.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.892+0000 7f851be72640 1 -- 172.21.15.64:0/260720161 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f84fc0b65f0 msgr2=0x7f84fc0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:26.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.892+0000 7f851be72640 1 --2- 172.21.15.64:0/260720161 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f84fc0b65f0 0x7f84fc0b8b00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8508000c00 tx=0x7f8508031400 comp rx=0 tx=0).stop 2024-09-21T04:02:26.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.892+0000 7f851be72640 1 -- 172.21.15.64:0/260720161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8514104880 msgr2=0x7f851419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:26.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.892+0000 7f851be72640 1 --2- 172.21.15.64:0/260720161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8514104880 0x7f851419a460 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f850c00e9c0 tx=0x7f850c00ee80 comp rx=0 tx=0).stop 2024-09-21T04:02:26.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.892+0000 7f851be72640 1 -- 172.21.15.64:0/260720161 shutdown_connections 2024-09-21T04:02:26.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.892+0000 7f851be72640 1 --2- 172.21.15.64:0/260720161 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f84fc0b65f0 0x7f84fc0b8b00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:26.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.892+0000 7f851be72640 1 --2- 172.21.15.64:0/260720161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f851410de30 0x7f851419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:26.895 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.892+0000 7f851be72640 1 --2- 172.21.15.64:0/260720161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f85141052b0 0x7f851419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:26.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.892+0000 7f851be72640 1 --2- 172.21.15.64:0/260720161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8514104880 0x7f851419a460 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:26.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.892+0000 7f851be72640 1 -- 172.21.15.64:0/260720161 >> 172.21.15.64:0/260720161 conn(0x7f8514079730 msgr2=0x7f8514100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:26.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.893+0000 7f851be72640 1 -- 172.21.15.64:0/260720161 shutdown_connections 2024-09-21T04:02:26.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:26.893+0000 7f851be72640 1 -- 172.21.15.64:0/260720161 wait complete. 2024-09-21T04:02:27.468 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T04:02:27.468 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T04:02:27.724 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:28.163 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:27 smithi064 ceph-mon[31180]: pgmap v270: 97 pgs: 97 active+clean; 624 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-09-21T04:02:28.163 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:27 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/260720161' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:02:28.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.171+0000 7fd58708a640 1 -- 172.21.15.64:0/2788181910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd580102ae0 msgr2=0x7fd580102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:28.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.171+0000 7fd58708a640 1 --2- 172.21.15.64:0/2788181910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd580102ae0 0x7fd580102ef0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd570009900 tx=0x7fd57002a410 comp rx=0 tx=0).stop 2024-09-21T04:02:28.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.171+0000 7fd58708a640 1 -- 172.21.15.64:0/2788181910 shutdown_connections 2024-09-21T04:02:28.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.171+0000 7fd58708a640 1 --2- 172.21.15.64:0/2788181910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd580108d20 0x7fd58010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:28.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.171+0000 7fd58708a640 1 --2- 172.21.15.64:0/2788181910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5801083d0 0x7fd5801087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:28.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.171+0000 7fd58708a640 1 --2- 172.21.15.64:0/2788181910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd580102ae0 0x7fd580102ef0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:28.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.171+0000 7fd58708a640 1 -- 172.21.15.64:0/2788181910 >> 172.21.15.64:0/2788181910 conn(0x7fd580079760 msgr2=0x7fd580079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:28.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.171+0000 7fd58708a640 1 -- 172.21.15.64:0/2788181910 shutdown_connections 2024-09-21T04:02:28.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.171+0000 7fd58708a640 1 -- 172.21.15.64:0/2788181910 wait complete. 2024-09-21T04:02:28.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.171+0000 7fd58708a640 1 Processor -- start 2024-09-21T04:02:28.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.172+0000 7fd58708a640 1 -- start start 2024-09-21T04:02:28.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.172+0000 7fd58708a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd580102ae0 0x7fd58019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:28.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.172+0000 7fd58708a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5801083d0 0x7fd58019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:28.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.172+0000 7fd58708a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd580108d20 0x7fd58019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:28.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.172+0000 7fd58708a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd5801129f0 con 0x7fd580102ae0 2024-09-21T04:02:28.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.172+0000 7fd58708a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd580112870 con 0x7fd5801083d0 2024-09-21T04:02:28.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.172+0000 7fd58708a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd580112b70 con 0x7fd580108d20 2024-09-21T04:02:28.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.172+0000 7fd584e26640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd580102ae0 0x7fd58019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:28.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.172+0000 7fd584e26640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd580102ae0 0x7fd58019a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34670/0 (socket says 172.21.15.64:34670) 2024-09-21T04:02:28.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.172+0000 7fd584e26640 1 -- 172.21.15.64:0/1906664648 learned_addr learned my addr 172.21.15.64:0/1906664648 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:28.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.172+0000 7fd57ffff640 1 --2- 172.21.15.64:0/1906664648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5801083d0 0x7fd58019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:28.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.172+0000 7fd585627640 1 --2- 172.21.15.64:0/1906664648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd580108d20 0x7fd58019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:28.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.172+0000 7fd584e26640 1 -- 172.21.15.64:0/1906664648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd580108d20 msgr2=0x7fd58019fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:28.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.172+0000 7fd584e26640 1 --2- 172.21.15.64:0/1906664648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd580108d20 0x7fd58019fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:28.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.172+0000 7fd584e26640 1 -- 172.21.15.64:0/1906664648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5801083d0 msgr2=0x7fd58019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:28.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.172+0000 7fd584e26640 1 --2- 172.21.15.64:0/1906664648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5801083d0 0x7fd58019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:28.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.172+0000 7fd584e26640 1 -- 172.21.15.64:0/1906664648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5801a0440 con 0x7fd580102ae0 2024-09-21T04:02:28.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.172+0000 7fd585627640 1 --2- 172.21.15.64:0/1906664648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd580108d20 0x7fd58019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:02:28.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.173+0000 7fd584e26640 1 --2- 172.21.15.64:0/1906664648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd580102ae0 0x7fd58019a4d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd570005940 tx=0x7fd5700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:28.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.173+0000 7fd57dffb640 1 -- 172.21.15.64:0/1906664648 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57003f030 con 0x7fd580102ae0 2024-09-21T04:02:28.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.173+0000 7fd57dffb640 1 -- 172.21.15.64:0/1906664648 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd570003cb0 con 0x7fd580102ae0 2024-09-21T04:02:28.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.173+0000 7fd58708a640 1 -- 172.21.15.64:0/1906664648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5801a06d0 con 0x7fd580102ae0 2024-09-21T04:02:28.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.173+0000 7fd57dffb640 1 -- 172.21.15.64:0/1906664648 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570035070 con 0x7fd580102ae0 2024-09-21T04:02:28.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.173+0000 7fd58708a640 1 -- 172.21.15.64:0/1906664648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5801a6e90 con 0x7fd580102ae0 2024-09-21T04:02:28.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.174+0000 7fd58708a640 1 -- 172.21.15.64:0/1906664648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd580103d10 con 0x7fd580102ae0 2024-09-21T04:02:28.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.176+0000 7fd57dffb640 1 -- 172.21.15.64:0/1906664648 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fd570042030 con 0x7fd580102ae0 2024-09-21T04:02:28.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.176+0000 7fd57dffb640 1 --2- 172.21.15.64:0/1906664648 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd5680b6670 0x7fd5680b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:28.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.176+0000 7fd57ffff640 1 --2- 172.21.15.64:0/1906664648 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd5680b6670 0x7fd5680b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:28.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.177+0000 7fd57dffb640 1 -- 172.21.15.64:0/1906664648 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8194+0+0 (secure 0 0 0) 0x7fd570107fa0 con 0x7fd580102ae0 2024-09-21T04:02:28.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.177+0000 7fd57ffff640 1 --2- 172.21.15.64:0/1906664648 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd5680b6670 0x7fd5680b8b80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd580100d30 tx=0x7fd574009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:28.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.179+0000 7fd57dffb640 1 -- 172.21.15.64:0/1906664648 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd5700ce6f0 con 0x7fd580102ae0 2024-09-21T04:02:28.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:27 smithi119 ceph-mon[34811]: pgmap v270: 97 pgs: 97 active+clean; 624 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-09-21T04:02:28.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:27 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/260720161' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:02:28.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:27 smithi131 ceph-mon[35193]: pgmap v270: 97 pgs: 97 active+clean; 624 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-09-21T04:02:28.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:27 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/260720161' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:02:28.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.365+0000 7fd58708a640 1 -- 172.21.15.64:0/1906664648 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd58006bb90 con 0x7fd5680b6670 2024-09-21T04:02:28.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.370+0000 7fd57dffb640 1 -- 172.21.15.64:0/1906664648 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258906 (secure 0 0 0) 0x7fd58006bb90 con 0x7fd5680b6670 2024-09-21T04:02:28.375 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:02:28.376 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-09-21T04:02:28.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.373+0000 7fd58708a640 1 -- 172.21.15.64:0/1906664648 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd5680b6670 msgr2=0x7fd5680b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:28.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.373+0000 7fd58708a640 1 --2- 172.21.15.64:0/1906664648 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd5680b6670 0x7fd5680b8b80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd580100d30 tx=0x7fd574009110 comp rx=0 tx=0).stop 2024-09-21T04:02:28.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.373+0000 7fd58708a640 1 -- 172.21.15.64:0/1906664648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd580102ae0 msgr2=0x7fd58019a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:28.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.373+0000 7fd58708a640 1 --2- 172.21.15.64:0/1906664648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd580102ae0 0x7fd58019a4d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd570005940 tx=0x7fd5700042f0 comp rx=0 tx=0).stop 2024-09-21T04:02:28.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.373+0000 7fd58708a640 1 -- 172.21.15.64:0/1906664648 shutdown_connections 2024-09-21T04:02:28.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.373+0000 7fd58708a640 1 --2- 172.21.15.64:0/1906664648 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd5680b6670 0x7fd5680b8b80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:28.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.373+0000 7fd58708a640 1 --2- 172.21.15.64:0/1906664648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd580108d20 0x7fd58019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:28.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.373+0000 7fd58708a640 1 --2- 172.21.15.64:0/1906664648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5801083d0 0x7fd58019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:28.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.373+0000 7fd58708a640 1 --2- 172.21.15.64:0/1906664648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd580102ae0 0x7fd58019a4d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:28.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.373+0000 7fd58708a640 1 -- 172.21.15.64:0/1906664648 >> 172.21.15.64:0/1906664648 conn(0x7fd580079760 msgr2=0x7fd580101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:28.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.374+0000 7fd58708a640 1 -- 172.21.15.64:0/1906664648 shutdown_connections 2024-09-21T04:02:28.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:28.374+0000 7fd58708a640 1 -- 172.21.15.64:0/1906664648 wait complete. 2024-09-21T04:02:29.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:28 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]} : dispatch 2024-09-21T04:02:29.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:28 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]} : dispatch 2024-09-21T04:02:29.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:28 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]} : dispatch 2024-09-21T04:02:29.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:28 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]} : dispatch 2024-09-21T04:02:29.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:28 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]} : dispatch 2024-09-21T04:02:29.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:28 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]} : dispatch 2024-09-21T04:02:29.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:28 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]} : dispatch 2024-09-21T04:02:29.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:28 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]} : dispatch 2024-09-21T04:02:29.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:28 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]} : dispatch 2024-09-21T04:02:29.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:28 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]} : dispatch 2024-09-21T04:02:29.089 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":270,"stamp":"2024-09-21T04:02:26.480222+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":639415,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":175,"num_read_kb":167,"num_write":268,"num_write_kb":2241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358996,"kb_used_data":9748,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124582828,"statfs":{"total":1151940427776,"available":1151572815872,"internally_reserved":0,"allocated":9981952,"data_stored":6584747,"data_compressed":113454,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":45231,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":94,"num_read_kb":98,"num_write":52,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003015"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.754741+0000","last_change":"2024-09-21T04:02:02.769848+0000","last_active":"2024-09-21T04:02:03.754741+0000","last_peered":"2024-09-21T04:02:03.754741+0000","last_clean":"2024-09-21T04:02:03.754741+0000","last_became_active":"2024-09-21T04:02:02.769508+0000","last_became_peered":"2024-09-21T04:02:02.769508+0000","last_unstale":"2024-09-21T04:02:03.754741+0000","last_undegraded":"2024-09-21T04:02:03.754741+0000","last_fullsized":"2024-09-21T04:02:03.754741+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:00.242233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.759296+0000","last_change":"2024-09-21T04:02:02.769200+0000","last_active":"2024-09-21T04:02:03.759296+0000","last_peered":"2024-09-21T04:02:03.759296+0000","last_clean":"2024-09-21T04:02:03.759296+0000","last_became_active":"2024-09-21T04:02:02.768918+0000","last_became_peered":"2024-09-21T04:02:02.768918+0000","last_unstale":"2024-09-21T04:02:03.759296+0000","last_undegraded":"2024-09-21T04:02:03.759296+0000","last_fullsized":"2024-09-21T04:02:03.759296+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:26.842079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.759579+0000","last_change":"2024-09-21T04:02:02.763511+0000","last_active":"2024-09-21T04:02:03.759579+0000","last_peered":"2024-09-21T04:02:03.759579+0000","last_clean":"2024-09-21T04:02:03.759579+0000","last_became_active":"2024-09-21T04:02:02.763161+0000","last_became_peered":"2024-09-21T04:02:02.763161+0000","last_unstale":"2024-09-21T04:02:03.759579+0000","last_undegraded":"2024-09-21T04:02:03.759579+0000","last_fullsized":"2024-09-21T04:02:03.759579+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:59.834005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.746577+0000","last_change":"2024-09-21T04:02:02.765452+0000","last_active":"2024-09-21T04:02:03.746577+0000","last_peered":"2024-09-21T04:02:03.746577+0000","last_clean":"2024-09-21T04:02:03.746577+0000","last_became_active":"2024-09-21T04:02:02.765065+0000","last_became_peered":"2024-09-21T04:02:02.765065+0000","last_unstale":"2024-09-21T04:02:03.746577+0000","last_undegraded":"2024-09-21T04:02:03.746577+0000","last_fullsized":"2024-09-21T04:02:03.746577+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:33.871366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.752693+0000","last_change":"2024-09-21T04:02:02.767457+0000","last_active":"2024-09-21T04:02:03.752693+0000","last_peered":"2024-09-21T04:02:03.752693+0000","last_clean":"2024-09-21T04:02:03.752693+0000","last_became_active":"2024-09-21T04:02:02.767064+0000","last_became_peered":"2024-09-21T04:02:02.767064+0000","last_unstale":"2024-09-21T04:02:03.752693+0000","last_undegraded":"2024-09-21T04:02:03.752693+0000","last_fullsized":"2024-09-21T04:02:03.752693+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:46.316029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.753553+0000","last_change":"2024-09-21T04:02:02.768795+0000","last_active":"2024-09-21T04:02:03.753553+0000","last_peered":"2024-09-21T04:02:03.753553+0000","last_clean":"2024-09-21T04:02:03.753553+0000","last_became_active":"2024-09-21T04:02:02.768434+0000","last_became_peered":"2024-09-21T04:02:02.768434+0000","last_unstale":"2024-09-21T04:02:03.753553+0000","last_undegraded":"2024-09-21T04:02:03.753553+0000","last_fullsized":"2024-09-21T04:02:03.753553+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:58.991533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.757595+0000","last_change":"2024-09-21T04:02:02.774998+0000","last_active":"2024-09-21T04:02:03.757595+0000","last_peered":"2024-09-21T04:02:03.757595+0000","last_clean":"2024-09-21T04:02:03.757595+0000","last_became_active":"2024-09-21T04:02:02.774344+0000","last_became_peered":"2024-09-21T04:02:02.774344+0000","last_unstale":"2024-09-21T04:02:03.757595+0000","last_undegraded":"2024-09-21T04:02:03.757595+0000","last_fullsized":"2024-09-21T04:02:03.757595+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:26.705126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.754679+0000","last_change":"2024-09-21T04:02:02.772498+0000","last_active":"2024-09-21T04:02:03.754679+0000","last_peered":"2024-09-21T04:02:03.754679+0000","last_clean":"2024-09-21T04:02:03.754679+0000","last_became_active":"2024-09-21T04:02:02.772049+0000","last_became_peered":"2024-09-21T04:02:02.772049+0000","last_unstale":"2024-09-21T04:02:03.754679+0000","last_undegraded":"2024-09-21T04:02:03.754679+0000","last_fullsized":"2024-09-21T04:02:03.754679+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:31:19.088424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.758659+0000","last_change":"2024-09-21T04:02:02.772336+0000","last_active":"2024-09-21T04:02:03.758659+0000","last_peered":"2024-09-21T04:02:03.758659+0000","last_clean":"2024-09-21T04:02:03.758659+0000","last_became_active":"2024-09-21T04:02:02.771964+0000","last_became_peered":"2024-09-21T04:02:02.771964+0000","last_unstale":"2024-09-21T04:02:03.758659+0000","last_undegraded":"2024-09-21T04:02:03.758659+0000","last_fullsized":"2024-09-21T04:02:03.758659+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:46.659690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.754261+0000","last_change":"2024-09-21T04:02:02.758992+0000","last_active":"2024-09-21T04:02:03.754261+0000","last_peered":"2024-09-21T04:02:03.754261+0000","last_clean":"2024-09-21T04:02:03.754261+0000","last_became_active":"2024-09-21T04:02:02.758575+0000","last_became_peered":"2024-09-21T04:02:02.758575+0000","last_unstale":"2024-09-21T04:02:03.754261+0000","last_undegraded":"2024-09-21T04:02:03.754261+0000","last_fullsized":"2024-09-21T04:02:03.754261+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:32:09.543140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.758999+0000","last_change":"2024-09-21T04:02:02.767304+0000","last_active":"2024-09-21T04:02:03.758999+0000","last_peered":"2024-09-21T04:02:03.758999+0000","last_clean":"2024-09-21T04:02:03.758999+0000","last_became_active":"2024-09-21T04:02:02.762804+0000","last_became_peered":"2024-09-21T04:02:02.762804+0000","last_unstale":"2024-09-21T04:02:03.758999+0000","last_undegraded":"2024-09-21T04:02:03.758999+0000","last_fullsized":"2024-09-21T04:02:03.758999+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:32.224071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.746822+0000","last_change":"2024-09-21T04:02:02.762932+0000","last_active":"2024-09-21T04:02:03.746822+0000","last_peered":"2024-09-21T04:02:03.746822+0000","last_clean":"2024-09-21T04:02:03.746822+0000","last_became_active":"2024-09-21T04:02:02.762538+0000","last_became_peered":"2024-09-21T04:02:02.762538+0000","last_unstale":"2024-09-21T04:02:03.746822+0000","last_undegraded":"2024-09-21T04:02:03.746822+0000","last_fullsized":"2024-09-21T04:02:03.746822+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:20.724453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.747667+0000","last_change":"2024-09-21T04:02:02.770313+0000","last_active":"2024-09-21T04:02:03.747667+0000","last_peered":"2024-09-21T04:02:03.747667+0000","last_clean":"2024-09-21T04:02:03.747667+0000","last_became_active":"2024-09-21T04:02:02.769910+0000","last_became_peered":"2024-09-21T04:02:02.769910+0000","last_unstale":"2024-09-21T04:02:03.747667+0000","last_undegraded":"2024-09-21T04:02:03.747667+0000","last_fullsized":"2024-09-21T04:02:03.747667+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:16.228371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.759582+0000","last_change":"2024-09-21T04:02:02.769258+0000","last_active":"2024-09-21T04:02:03.759582+0000","last_peered":"2024-09-21T04:02:03.759582+0000","last_clean":"2024-09-21T04:02:03.759582+0000","last_became_active":"2024-09-21T04:02:02.768914+0000","last_became_peered":"2024-09-21T04:02:02.768914+0000","last_unstale":"2024-09-21T04:02:03.759582+0000","last_undegraded":"2024-09-21T04:02:03.759582+0000","last_fullsized":"2024-09-21T04:02:03.759582+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:37.426810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.759046+0000","last_change":"2024-09-21T04:02:02.755940+0000","last_active":"2024-09-21T04:02:03.759046+0000","last_peered":"2024-09-21T04:02:03.759046+0000","last_clean":"2024-09-21T04:02:03.759046+0000","last_became_active":"2024-09-21T04:02:02.755309+0000","last_became_peered":"2024-09-21T04:02:02.755309+0000","last_unstale":"2024-09-21T04:02:03.759046+0000","last_undegraded":"2024-09-21T04:02:03.759046+0000","last_fullsized":"2024-09-21T04:02:03.759046+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:49.834030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.757784+0000","last_change":"2024-09-21T04:02:02.761068+0000","last_active":"2024-09-21T04:02:03.757784+0000","last_peered":"2024-09-21T04:02:03.757784+0000","last_clean":"2024-09-21T04:02:03.757784+0000","last_became_active":"2024-09-21T04:02:02.760683+0000","last_became_peered":"2024-09-21T04:02:02.760683+0000","last_unstale":"2024-09-21T04:02:03.757784+0000","last_undegraded":"2024-09-21T04:02:03.757784+0000","last_fullsized":"2024-09-21T04:02:03.757784+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:36.627539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.755039+0000","last_change":"2024-09-21T04:02:02.772234+0000","last_active":"2024-09-21T04:02:03.755039+0000","last_peered":"2024-09-21T04:02:03.755039+0000","last_clean":"2024-09-21T04:02:03.755039+0000","last_became_active":"2024-09-21T04:02:02.771825+0000","last_became_peered":"2024-09-21T04:02:02.771825+0000","last_unstale":"2024-09-21T04:02:03.755039+0000","last_undegraded":"2024-09-21T04:02:03.755039+0000","last_fullsized":"2024-09-21T04:02:03.755039+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:33.449745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.753645+0000","last_change":"2024-09-21T04:02:02.758643+0000","last_active":"2024-09-21T04:02:03.753645+0000","last_peered":"2024-09-21T04:02:03.753645+0000","last_clean":"2024-09-21T04:02:03.753645+0000","last_became_active":"2024-09-21T04:02:02.758261+0000","last_became_peered":"2024-09-21T04:02:02.758261+0000","last_unstale":"2024-09-21T04:02:03.753645+0000","last_undegraded":"2024-09-21T04:02:03.753645+0000","last_fullsized":"2024-09-21T04:02:03.753645+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:04.165399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.746552+0000","last_change":"2024-09-21T04:02:02.759405+0000","last_active":"2024-09-21T04:02:03.746552+0000","last_peered":"2024-09-21T04:02:03.746552+0000","last_clean":"2024-09-21T04:02:03.746552+0000","last_became_active":"2024-09-21T04:02:02.758962+0000","last_became_peered":"2024-09-21T04:02:02.758962+0000","last_unstale":"2024-09-21T04:02:03.746552+0000","last_undegraded":"2024-09-21T04:02:03.746552+0000","last_fullsized":"2024-09-21T04:02:03.746552+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:37.203308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.755683+0000","last_change":"2024-09-21T04:02:02.767518+0000","last_active":"2024-09-21T04:02:03.755683+0000","last_peered":"2024-09-21T04:02:03.755683+0000","last_clean":"2024-09-21T04:02:03.755683+0000","last_became_active":"2024-09-21T04:02:02.767147+0000","last_became_peered":"2024-09-21T04:02:02.767147+0000","last_unstale":"2024-09-21T04:02:03.755683+0000","last_undegraded":"2024-09-21T04:02:03.755683+0000","last_fullsized":"2024-09-21T04:02:03.755683+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:34:13.615218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.747768+0000","last_change":"2024-09-21T04:02:02.756407+0000","last_active":"2024-09-21T04:02:03.747768+0000","last_peered":"2024-09-21T04:02:03.747768+0000","last_clean":"2024-09-21T04:02:03.747768+0000","last_became_active":"2024-09-21T04:02:02.756007+0000","last_became_peered":"2024-09-21T04:02:02.756007+0000","last_unstale":"2024-09-21T04:02:03.747768+0000","last_undegraded":"2024-09-21T04:02:03.747768+0000","last_fullsized":"2024-09-21T04:02:03.747768+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:18.798814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.758613+0000","last_change":"2024-09-21T04:02:02.766290+0000","last_active":"2024-09-21T04:02:03.758613+0000","last_peered":"2024-09-21T04:02:03.758613+0000","last_clean":"2024-09-21T04:02:03.758613+0000","last_became_active":"2024-09-21T04:02:02.765660+0000","last_became_peered":"2024-09-21T04:02:02.765660+0000","last_unstale":"2024-09-21T04:02:03.758613+0000","last_undegraded":"2024-09-21T04:02:03.758613+0000","last_fullsized":"2024-09-21T04:02:03.758613+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:23.281476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.754317+0000","last_change":"2024-09-21T04:02:02.767943+0000","last_active":"2024-09-21T04:02:03.754317+0000","last_peered":"2024-09-21T04:02:03.754317+0000","last_clean":"2024-09-21T04:02:03.754317+0000","last_became_active":"2024-09-21T04:02:02.767566+0000","last_became_peered":"2024-09-21T04:02:02.767566+0000","last_unstale":"2024-09-21T04:02:03.754317+0000","last_undegraded":"2024-09-21T04:02:03.754317+0000","last_fullsized":"2024-09-21T04:02:03.754317+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:44.221531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.755758+0000","last_change":"2024-09-21T04:02:02.770114+0000","last_active":"2024-09-21T04:02:03.755758+0000","last_peered":"2024-09-21T04:02:03.755758+0000","last_clean":"2024-09-21T04:02:03.755758+0000","last_became_active":"2024-09-21T04:02:02.769732+0000","last_became_peered":"2024-09-21T04:02:02.769732+0000","last_unstale":"2024-09-21T04:02:03.755758+0000","last_undegraded":"2024-09-21T04:02:03.755758+0000","last_fullsized":"2024-09-21T04:02:03.755758+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:38.396288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.754695+0000","last_change":"2024-09-21T04:02:02.767862+0000","last_active":"2024-09-21T04:02:03.754695+0000","last_peered":"2024-09-21T04:02:03.754695+0000","last_clean":"2024-09-21T04:02:03.754695+0000","last_became_active":"2024-09-21T04:02:02.767475+0000","last_became_peered":"2024-09-21T04:02:02.767475+0000","last_unstale":"2024-09-21T04:02:03.754695+0000","last_undegraded":"2024-09-21T04:02:03.754695+0000","last_fullsized":"2024-09-21T04:02:03.754695+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:26.754666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.746696+0000","last_change":"2024-09-21T04:02:02.764666+0000","last_active":"2024-09-21T04:02:03.746696+0000","last_peered":"2024-09-21T04:02:03.746696+0000","last_clean":"2024-09-21T04:02:03.746696+0000","last_became_active":"2024-09-21T04:02:02.763931+0000","last_became_peered":"2024-09-21T04:02:02.763931+0000","last_unstale":"2024-09-21T04:02:03.746696+0000","last_undegraded":"2024-09-21T04:02:03.746696+0000","last_fullsized":"2024-09-21T04:02:03.746696+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:39.724770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.754152+0000","last_change":"2024-09-21T04:02:02.774276+0000","last_active":"2024-09-21T04:02:03.754152+0000","last_peered":"2024-09-21T04:02:03.754152+0000","last_clean":"2024-09-21T04:02:03.754152+0000","last_became_active":"2024-09-21T04:02:02.773852+0000","last_became_peered":"2024-09-21T04:02:02.773852+0000","last_unstale":"2024-09-21T04:02:03.754152+0000","last_undegraded":"2024-09-21T04:02:03.754152+0000","last_fullsized":"2024-09-21T04:02:03.754152+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:40.072264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.747607+0000","last_change":"2024-09-21T04:02:02.758520+0000","last_active":"2024-09-21T04:02:03.747607+0000","last_peered":"2024-09-21T04:02:03.747607+0000","last_clean":"2024-09-21T04:02:03.747607+0000","last_became_active":"2024-09-21T04:02:02.757656+0000","last_became_peered":"2024-09-21T04:02:02.757656+0000","last_unstale":"2024-09-21T04:02:03.747607+0000","last_undegraded":"2024-09-21T04:02:03.747607+0000","last_fullsized":"2024-09-21T04:02:03.747607+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:30.854336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.759491+0000","last_change":"2024-09-21T04:02:02.756428+0000","last_active":"2024-09-21T04:02:03.759491+0000","last_peered":"2024-09-21T04:02:03.759491+0000","last_clean":"2024-09-21T04:02:03.759491+0000","last_became_active":"2024-09-21T04:02:02.755985+0000","last_became_peered":"2024-09-21T04:02:02.755985+0000","last_unstale":"2024-09-21T04:02:03.759491+0000","last_undegraded":"2024-09-21T04:02:03.759491+0000","last_fullsized":"2024-09-21T04:02:03.759491+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:36.040436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.746155+0000","last_change":"2024-09-21T04:02:02.762893+0000","last_active":"2024-09-21T04:02:03.746155+0000","last_peered":"2024-09-21T04:02:03.746155+0000","last_clean":"2024-09-21T04:02:03.746155+0000","last_became_active":"2024-09-21T04:02:02.762343+0000","last_became_peered":"2024-09-21T04:02:02.762343+0000","last_unstale":"2024-09-21T04:02:03.746155+0000","last_undegraded":"2024-09-21T04:02:03.746155+0000","last_fullsized":"2024-09-21T04:02:03.746155+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:45.074711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.746610+0000","last_change":"2024-09-21T04:02:02.763942+0000","last_active":"2024-09-21T04:02:03.746610+0000","last_peered":"2024-09-21T04:02:03.746610+0000","last_clean":"2024-09-21T04:02:03.746610+0000","last_became_active":"2024-09-21T04:02:02.763433+0000","last_became_peered":"2024-09-21T04:02:02.763433+0000","last_unstale":"2024-09-21T04:02:03.746610+0000","last_undegraded":"2024-09-21T04:02:03.746610+0000","last_fullsized":"2024-09-21T04:02:03.746610+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:36.139440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.753926+0000","last_change":"2024-09-21T04:02:02.773364+0000","last_active":"2024-09-21T04:02:03.753926+0000","last_peered":"2024-09-21T04:02:03.753926+0000","last_clean":"2024-09-21T04:02:03.753926+0000","last_became_active":"2024-09-21T04:02:02.773040+0000","last_became_peered":"2024-09-21T04:02:02.773040+0000","last_unstale":"2024-09-21T04:02:03.753926+0000","last_undegraded":"2024-09-21T04:02:03.753926+0000","last_fullsized":"2024-09-21T04:02:03.753926+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:48.504476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.752962+0000","last_change":"2024-09-21T04:02:02.774951+0000","last_active":"2024-09-21T04:02:03.752962+0000","last_peered":"2024-09-21T04:02:03.752962+0000","last_clean":"2024-09-21T04:02:03.752962+0000","last_became_active":"2024-09-21T04:02:02.774525+0000","last_became_peered":"2024-09-21T04:02:02.774525+0000","last_unstale":"2024-09-21T04:02:03.752962+0000","last_undegraded":"2024-09-21T04:02:03.752962+0000","last_fullsized":"2024-09-21T04:02:03.752962+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:24.447860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"73'5","reported_seq":40,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:24.829799+0000","last_change":"2024-09-21T04:02:00.743470+0000","last_active":"2024-09-21T04:02:24.829799+0000","last_peered":"2024-09-21T04:02:24.829799+0000","last_clean":"2024-09-21T04:02:24.829799+0000","last_became_active":"2024-09-21T04:02:00.743055+0000","last_became_peered":"2024-09-21T04:02:00.743055+0000","last_unstale":"2024-09-21T04:02:24.829799+0000","last_undegraded":"2024-09-21T04:02:24.829799+0000","last_fullsized":"2024-09-21T04:02:24.829799+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:56.101753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.757184+0000","last_change":"2024-09-21T04:02:02.773399+0000","last_active":"2024-09-21T04:02:03.757184+0000","last_peered":"2024-09-21T04:02:03.757184+0000","last_clean":"2024-09-21T04:02:03.757184+0000","last_became_active":"2024-09-21T04:02:02.772996+0000","last_became_peered":"2024-09-21T04:02:02.772996+0000","last_unstale":"2024-09-21T04:02:03.757184+0000","last_undegraded":"2024-09-21T04:02:03.757184+0000","last_fullsized":"2024-09-21T04:02:03.757184+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:07.256226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"73'3","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:11.803438+0000","last_change":"2024-09-21T04:02:00.747383+0000","last_active":"2024-09-21T04:02:11.803438+0000","last_peered":"2024-09-21T04:02:11.803438+0000","last_clean":"2024-09-21T04:02:11.803438+0000","last_became_active":"2024-09-21T04:02:00.746956+0000","last_became_peered":"2024-09-21T04:02:00.746956+0000","last_unstale":"2024-09-21T04:02:11.803438+0000","last_undegraded":"2024-09-21T04:02:11.803438+0000","last_fullsized":"2024-09-21T04:02:11.803438+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:08.320290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"73'2","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:11.801513+0000","last_change":"2024-09-21T04:02:00.736630+0000","last_active":"2024-09-21T04:02:11.801513+0000","last_peered":"2024-09-21T04:02:11.801513+0000","last_clean":"2024-09-21T04:02:11.801513+0000","last_became_active":"2024-09-21T04:02:00.736211+0000","last_became_peered":"2024-09-21T04:02:00.736211+0000","last_unstale":"2024-09-21T04:02:11.801513+0000","last_undegraded":"2024-09-21T04:02:11.801513+0000","last_fullsized":"2024-09-21T04:02:11.801513+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:37.211617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.755272+0000","last_change":"2024-09-21T04:02:02.763323+0000","last_active":"2024-09-21T04:02:03.755272+0000","last_peered":"2024-09-21T04:02:03.755272+0000","last_clean":"2024-09-21T04:02:03.755272+0000","last_became_active":"2024-09-21T04:02:02.762776+0000","last_became_peered":"2024-09-21T04:02:02.762776+0000","last_unstale":"2024-09-21T04:02:03.755272+0000","last_undegraded":"2024-09-21T04:02:03.755272+0000","last_fullsized":"2024-09-21T04:02:03.755272+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:39.891042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"73'1","reported_seq":20,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.756337+0000","last_change":"2024-09-21T04:02:00.743377+0000","last_active":"2024-09-21T04:02:03.756337+0000","last_peered":"2024-09-21T04:02:03.756337+0000","last_clean":"2024-09-21T04:02:03.756337+0000","last_became_active":"2024-09-21T04:02:00.742973+0000","last_became_peered":"2024-09-21T04:02:00.742973+0000","last_unstale":"2024-09-21T04:02:03.756337+0000","last_undegraded":"2024-09-21T04:02:03.756337+0000","last_fullsized":"2024-09-21T04:02:03.756337+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:55:00.655532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.753172+0000","last_change":"2024-09-21T04:02:02.758721+0000","last_active":"2024-09-21T04:02:03.753172+0000","last_peered":"2024-09-21T04:02:03.753172+0000","last_clean":"2024-09-21T04:02:03.753172+0000","last_became_active":"2024-09-21T04:02:02.758352+0000","last_became_peered":"2024-09-21T04:02:02.758352+0000","last_unstale":"2024-09-21T04:02:03.753172+0000","last_undegraded":"2024-09-21T04:02:03.753172+0000","last_fullsized":"2024-09-21T04:02:03.753172+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:56:49.289510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"73'3","reported_seq":32,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:20.822601+0000","last_change":"2024-09-21T04:02:00.744591+0000","last_active":"2024-09-21T04:02:20.822601+0000","last_peered":"2024-09-21T04:02:20.822601+0000","last_clean":"2024-09-21T04:02:20.822601+0000","last_became_active":"2024-09-21T04:02:00.744139+0000","last_became_peered":"2024-09-21T04:02:00.744139+0000","last_unstale":"2024-09-21T04:02:20.822601+0000","last_undegraded":"2024-09-21T04:02:20.822601+0000","last_fullsized":"2024-09-21T04:02:20.822601+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:21.331517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.758125+0000","last_change":"2024-09-21T04:02:02.762779+0000","last_active":"2024-09-21T04:02:03.758125+0000","last_peered":"2024-09-21T04:02:03.758125+0000","last_clean":"2024-09-21T04:02:03.758125+0000","last_became_active":"2024-09-21T04:02:02.760761+0000","last_became_peered":"2024-09-21T04:02:02.760761+0000","last_unstale":"2024-09-21T04:02:03.758125+0000","last_undegraded":"2024-09-21T04:02:03.758125+0000","last_fullsized":"2024-09-21T04:02:03.758125+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:18.789052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"73'2","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:10.795479+0000","last_change":"2024-09-21T04:02:00.743220+0000","last_active":"2024-09-21T04:02:10.795479+0000","last_peered":"2024-09-21T04:02:10.795479+0000","last_clean":"2024-09-21T04:02:10.795479+0000","last_became_active":"2024-09-21T04:02:00.742636+0000","last_became_peered":"2024-09-21T04:02:00.742636+0000","last_unstale":"2024-09-21T04:02:10.795479+0000","last_undegraded":"2024-09-21T04:02:10.795479+0000","last_fullsized":"2024-09-21T04:02:10.795479+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:01.382338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.760136+0000","last_change":"2024-09-21T04:02:02.770744+0000","last_active":"2024-09-21T04:02:03.760136+0000","last_peered":"2024-09-21T04:02:03.760136+0000","last_clean":"2024-09-21T04:02:03.760136+0000","last_became_active":"2024-09-21T04:02:02.769894+0000","last_became_peered":"2024-09-21T04:02:02.769894+0000","last_unstale":"2024-09-21T04:02:03.760136+0000","last_undegraded":"2024-09-21T04:02:03.760136+0000","last_fullsized":"2024-09-21T04:02:03.760136+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:08.726517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"73'4","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:10.795275+0000","last_change":"2024-09-21T04:02:00.853284+0000","last_active":"2024-09-21T04:02:10.795275+0000","last_peered":"2024-09-21T04:02:10.795275+0000","last_clean":"2024-09-21T04:02:10.795275+0000","last_became_active":"2024-09-21T04:02:00.852855+0000","last_became_peered":"2024-09-21T04:02:00.852855+0000","last_unstale":"2024-09-21T04:02:10.795275+0000","last_undegraded":"2024-09-21T04:02:10.795275+0000","last_fullsized":"2024-09-21T04:02:10.795275+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:36.994214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.758427+0000","last_change":"2024-09-21T04:02:02.763577+0000","last_active":"2024-09-21T04:02:03.758427+0000","last_peered":"2024-09-21T04:02:03.758427+0000","last_clean":"2024-09-21T04:02:03.758427+0000","last_became_active":"2024-09-21T04:02:02.762970+0000","last_became_peered":"2024-09-21T04:02:02.762970+0000","last_unstale":"2024-09-21T04:02:03.758427+0000","last_undegraded":"2024-09-21T04:02:03.758427+0000","last_fullsized":"2024-09-21T04:02:03.758427+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:49.956226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"73'20","reported_seq":64,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:21.824560+0000","last_change":"2024-09-21T04:02:00.744203+0000","last_active":"2024-09-21T04:02:21.824560+0000","last_peered":"2024-09-21T04:02:21.824560+0000","last_clean":"2024-09-21T04:02:21.824560+0000","last_became_active":"2024-09-21T04:02:00.743659+0000","last_became_peered":"2024-09-21T04:02:00.743659+0000","last_unstale":"2024-09-21T04:02:21.824560+0000","last_undegraded":"2024-09-21T04:02:21.824560+0000","last_fullsized":"2024-09-21T04:02:21.824560+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:32.108302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45203,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":56,"num_write":21,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.753850+0000","last_change":"2024-09-21T04:02:02.766002+0000","last_active":"2024-09-21T04:02:03.753850+0000","last_peered":"2024-09-21T04:02:03.753850+0000","last_clean":"2024-09-21T04:02:03.753850+0000","last_became_active":"2024-09-21T04:02:02.765628+0000","last_became_peered":"2024-09-21T04:02:02.765628+0000","last_unstale":"2024-09-21T04:02:03.753850+0000","last_undegraded":"2024-09-21T04:02:03.753850+0000","last_fullsized":"2024-09-21T04:02:03.753850+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:22.251586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"73'6","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:11.802292+0000","last_change":"2024-09-21T04:02:00.741086+0000","last_active":"2024-09-21T04:02:11.802292+0000","last_peered":"2024-09-21T04:02:11.802292+0000","last_clean":"2024-09-21T04:02:11.802292+0000","last_became_active":"2024-09-21T04:02:00.740633+0000","last_became_peered":"2024-09-21T04:02:00.740633+0000","last_unstale":"2024-09-21T04:02:11.802292+0000","last_undegraded":"2024-09-21T04:02:11.802292+0000","last_fullsized":"2024-09-21T04:02:11.802292+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:20.392799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.747505+0000","last_change":"2024-09-21T04:02:02.759651+0000","last_active":"2024-09-21T04:02:03.747505+0000","last_peered":"2024-09-21T04:02:03.747505+0000","last_clean":"2024-09-21T04:02:03.747505+0000","last_became_active":"2024-09-21T04:02:02.759290+0000","last_became_peered":"2024-09-21T04:02:02.759290+0000","last_unstale":"2024-09-21T04:02:03.747505+0000","last_undegraded":"2024-09-21T04:02:03.747505+0000","last_fullsized":"2024-09-21T04:02:03.747505+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:32.025703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"73'3","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:10.797863+0000","last_change":"2024-09-21T04:02:00.744005+0000","last_active":"2024-09-21T04:02:10.797863+0000","last_peered":"2024-09-21T04:02:10.797863+0000","last_clean":"2024-09-21T04:02:10.797863+0000","last_became_active":"2024-09-21T04:02:00.743570+0000","last_became_peered":"2024-09-21T04:02:00.743570+0000","last_unstale":"2024-09-21T04:02:10.797863+0000","last_undegraded":"2024-09-21T04:02:10.797863+0000","last_fullsized":"2024-09-21T04:02:10.797863+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:32.158681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.759846+0000","last_change":"2024-09-21T04:02:02.757781+0000","last_active":"2024-09-21T04:02:03.759846+0000","last_peered":"2024-09-21T04:02:03.759846+0000","last_clean":"2024-09-21T04:02:03.759846+0000","last_became_active":"2024-09-21T04:02:02.757138+0000","last_became_peered":"2024-09-21T04:02:02.757138+0000","last_unstale":"2024-09-21T04:02:03.759846+0000","last_undegraded":"2024-09-21T04:02:03.759846+0000","last_fullsized":"2024-09-21T04:02:03.759846+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:17.770006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"73'3","reported_seq":37,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:23.828513+0000","last_change":"2024-09-21T04:02:00.741596+0000","last_active":"2024-09-21T04:02:23.828513+0000","last_peered":"2024-09-21T04:02:23.828513+0000","last_clean":"2024-09-21T04:02:23.828513+0000","last_became_active":"2024-09-21T04:02:00.741185+0000","last_became_peered":"2024-09-21T04:02:00.741185+0000","last_unstale":"2024-09-21T04:02:23.828513+0000","last_undegraded":"2024-09-21T04:02:23.828513+0000","last_fullsized":"2024-09-21T04:02:23.828513+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:31.663133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.757627+0000","last_change":"2024-09-21T04:02:02.773706+0000","last_active":"2024-09-21T04:02:03.757627+0000","last_peered":"2024-09-21T04:02:03.757627+0000","last_clean":"2024-09-21T04:02:03.757627+0000","last_became_active":"2024-09-21T04:02:02.773259+0000","last_became_peered":"2024-09-21T04:02:02.773259+0000","last_unstale":"2024-09-21T04:02:03.757627+0000","last_undegraded":"2024-09-21T04:02:03.757627+0000","last_fullsized":"2024-09-21T04:02:03.757627+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:50.431982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"73'15","reported_seq":35,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:11.804308+0000","last_change":"2024-09-21T04:02:00.851335+0000","last_active":"2024-09-21T04:02:11.804308+0000","last_peered":"2024-09-21T04:02:11.804308+0000","last_clean":"2024-09-21T04:02:11.804308+0000","last_became_active":"2024-09-21T04:02:00.850990+0000","last_became_peered":"2024-09-21T04:02:00.850990+0000","last_unstale":"2024-09-21T04:02:11.804308+0000","last_undegraded":"2024-09-21T04:02:11.804308+0000","last_fullsized":"2024-09-21T04:02:11.804308+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:45.731121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.756010+0000","last_change":"2024-09-21T04:02:02.770985+0000","last_active":"2024-09-21T04:02:03.756010+0000","last_peered":"2024-09-21T04:02:03.756010+0000","last_clean":"2024-09-21T04:02:03.756010+0000","last_became_active":"2024-09-21T04:02:02.770602+0000","last_became_peered":"2024-09-21T04:02:02.770602+0000","last_unstale":"2024-09-21T04:02:03.756010+0000","last_undegraded":"2024-09-21T04:02:03.756010+0000","last_fullsized":"2024-09-21T04:02:03.756010+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:37.131113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"73'2","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:12.809273+0000","last_change":"2024-09-21T04:02:00.740834+0000","last_active":"2024-09-21T04:02:12.809273+0000","last_peered":"2024-09-21T04:02:12.809273+0000","last_clean":"2024-09-21T04:02:12.809273+0000","last_became_active":"2024-09-21T04:02:00.740439+0000","last_became_peered":"2024-09-21T04:02:00.740439+0000","last_unstale":"2024-09-21T04:02:12.809273+0000","last_undegraded":"2024-09-21T04:02:12.809273+0000","last_fullsized":"2024-09-21T04:02:12.809273+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:04:22.159986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.755325+0000","last_change":"2024-09-21T04:02:02.773432+0000","last_active":"2024-09-21T04:02:03.755325+0000","last_peered":"2024-09-21T04:02:03.755325+0000","last_clean":"2024-09-21T04:02:03.755325+0000","last_became_active":"2024-09-21T04:02:02.773092+0000","last_became_peered":"2024-09-21T04:02:02.773092+0000","last_unstale":"2024-09-21T04:02:03.755325+0000","last_undegraded":"2024-09-21T04:02:03.755325+0000","last_fullsized":"2024-09-21T04:02:03.755325+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:36.440497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"73'2","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.755104+0000","last_change":"2024-09-21T04:02:00.738987+0000","last_active":"2024-09-21T04:02:03.755104+0000","last_peered":"2024-09-21T04:02:03.755104+0000","last_clean":"2024-09-21T04:02:03.755104+0000","last_became_active":"2024-09-21T04:02:00.738519+0000","last_became_peered":"2024-09-21T04:02:00.738519+0000","last_unstale":"2024-09-21T04:02:03.755104+0000","last_undegraded":"2024-09-21T04:02:03.755104+0000","last_fullsized":"2024-09-21T04:02:03.755104+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:58.079100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.760230+0000","last_change":"2024-09-21T04:02:02.766709+0000","last_active":"2024-09-21T04:02:03.760230+0000","last_peered":"2024-09-21T04:02:03.760230+0000","last_clean":"2024-09-21T04:02:03.760230+0000","last_became_active":"2024-09-21T04:02:02.766342+0000","last_became_peered":"2024-09-21T04:02:02.766342+0000","last_unstale":"2024-09-21T04:02:03.760230+0000","last_undegraded":"2024-09-21T04:02:03.760230+0000","last_fullsized":"2024-09-21T04:02:03.760230+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:26:35.185368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"73'1","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.762691+0000","last_change":"2024-09-21T04:02:00.745365+0000","last_active":"2024-09-21T04:02:03.762691+0000","last_peered":"2024-09-21T04:02:03.762691+0000","last_clean":"2024-09-21T04:02:03.762691+0000","last_became_active":"2024-09-21T04:02:00.744968+0000","last_became_peered":"2024-09-21T04:02:00.744968+0000","last_unstale":"2024-09-21T04:02:03.762691+0000","last_undegraded":"2024-09-21T04:02:03.762691+0000","last_fullsized":"2024-09-21T04:02:03.762691+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:56.078169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"73'1","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:06.043426+0000","last_change":"2024-09-21T04:02:02.774877+0000","last_active":"2024-09-21T04:02:06.043426+0000","last_peered":"2024-09-21T04:02:06.043426+0000","last_clean":"2024-09-21T04:02:06.043426+0000","last_became_active":"2024-09-21T04:02:02.774464+0000","last_became_peered":"2024-09-21T04:02:02.774464+0000","last_unstale":"2024-09-21T04:02:06.043426+0000","last_undegraded":"2024-09-21T04:02:06.043426+0000","last_fullsized":"2024-09-21T04:02:06.043426+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:23.393239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"73'3","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:11.803150+0000","last_change":"2024-09-21T04:02:01.257649+0000","last_active":"2024-09-21T04:02:11.803150+0000","last_peered":"2024-09-21T04:02:11.803150+0000","last_clean":"2024-09-21T04:02:11.803150+0000","last_became_active":"2024-09-21T04:02:01.257176+0000","last_became_peered":"2024-09-21T04:02:01.257176+0000","last_unstale":"2024-09-21T04:02:11.803150+0000","last_undegraded":"2024-09-21T04:02:11.803150+0000","last_fullsized":"2024-09-21T04:02:11.803150+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:37.257702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.759223+0000","last_change":"2024-09-21T04:02:02.768164+0000","last_active":"2024-09-21T04:02:03.759223+0000","last_peered":"2024-09-21T04:02:03.759223+0000","last_clean":"2024-09-21T04:02:03.759223+0000","last_became_active":"2024-09-21T04:02:02.766925+0000","last_became_peered":"2024-09-21T04:02:02.766925+0000","last_unstale":"2024-09-21T04:02:03.759223+0000","last_undegraded":"2024-09-21T04:02:03.759223+0000","last_fullsized":"2024-09-21T04:02:03.759223+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:59.707637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":91,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.757996+0000","last_change":"2024-09-21T04:00:48.936996+0000","last_active":"2024-09-21T04:02:03.757996+0000","last_peered":"2024-09-21T04:02:03.757996+0000","last_clean":"2024-09-21T04:02:03.757996+0000","last_became_active":"2024-09-21T04:00:48.931310+0000","last_became_peered":"2024-09-21T04:00:48.931310+0000","last_unstale":"2024-09-21T04:02:03.757996+0000","last_undegraded":"2024-09-21T04:02:03.757996+0000","last_fullsized":"2024-09-21T04:02:03.757996+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:57:25.346362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:57:25.346362+0000","last_clean_scrub_stamp":"2024-09-21T03:57:25.346362+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:24.165731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"73'3","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:12.807797+0000","last_change":"2024-09-21T04:02:00.740118+0000","last_active":"2024-09-21T04:02:12.807797+0000","last_peered":"2024-09-21T04:02:12.807797+0000","last_clean":"2024-09-21T04:02:12.807797+0000","last_became_active":"2024-09-21T04:02:00.739677+0000","last_became_peered":"2024-09-21T04:02:00.739677+0000","last_unstale":"2024-09-21T04:02:12.807797+0000","last_undegraded":"2024-09-21T04:02:12.807797+0000","last_fullsized":"2024-09-21T04:02:12.807797+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:46.084118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.755214+0000","last_change":"2024-09-21T04:02:02.771537+0000","last_active":"2024-09-21T04:02:03.755214+0000","last_peered":"2024-09-21T04:02:03.755214+0000","last_clean":"2024-09-21T04:02:03.755214+0000","last_became_active":"2024-09-21T04:02:02.771151+0000","last_became_peered":"2024-09-21T04:02:02.771151+0000","last_unstale":"2024-09-21T04:02:03.755214+0000","last_undegraded":"2024-09-21T04:02:03.755214+0000","last_fullsized":"2024-09-21T04:02:03.755214+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:21.283515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"73'3","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:11.803482+0000","last_change":"2024-09-21T04:02:00.744195+0000","last_active":"2024-09-21T04:02:11.803482+0000","last_peered":"2024-09-21T04:02:11.803482+0000","last_clean":"2024-09-21T04:02:11.803482+0000","last_became_active":"2024-09-21T04:02:00.743779+0000","last_became_peered":"2024-09-21T04:02:00.743779+0000","last_unstale":"2024-09-21T04:02:11.803482+0000","last_undegraded":"2024-09-21T04:02:11.803482+0000","last_fullsized":"2024-09-21T04:02:11.803482+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:31.281347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.747405+0000","last_change":"2024-09-21T04:02:02.755223+0000","last_active":"2024-09-21T04:02:03.747405+0000","last_peered":"2024-09-21T04:02:03.747405+0000","last_clean":"2024-09-21T04:02:03.747405+0000","last_became_active":"2024-09-21T04:02:02.754836+0000","last_became_peered":"2024-09-21T04:02:02.754836+0000","last_unstale":"2024-09-21T04:02:03.747405+0000","last_undegraded":"2024-09-21T04:02:03.747405+0000","last_fullsized":"2024-09-21T04:02:03.747405+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:57.912690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"73'4","reported_seq":37,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:24.829183+0000","last_change":"2024-09-21T04:02:00.851689+0000","last_active":"2024-09-21T04:02:24.829183+0000","last_peered":"2024-09-21T04:02:24.829183+0000","last_clean":"2024-09-21T04:02:24.829183+0000","last_became_active":"2024-09-21T04:02:00.851241+0000","last_became_peered":"2024-09-21T04:02:00.851241+0000","last_unstale":"2024-09-21T04:02:24.829183+0000","last_undegraded":"2024-09-21T04:02:24.829183+0000","last_fullsized":"2024-09-21T04:02:24.829183+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:14.697695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.755846+0000","last_change":"2024-09-21T04:02:02.774750+0000","last_active":"2024-09-21T04:02:03.755846+0000","last_peered":"2024-09-21T04:02:03.755846+0000","last_clean":"2024-09-21T04:02:03.755846+0000","last_became_active":"2024-09-21T04:02:02.774310+0000","last_became_peered":"2024-09-21T04:02:02.774310+0000","last_unstale":"2024-09-21T04:02:03.755846+0000","last_undegraded":"2024-09-21T04:02:03.755846+0000","last_fullsized":"2024-09-21T04:02:03.755846+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:07.909035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"73'1","reported_seq":20,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.758146+0000","last_change":"2024-09-21T04:02:00.741461+0000","last_active":"2024-09-21T04:02:03.758146+0000","last_peered":"2024-09-21T04:02:03.758146+0000","last_clean":"2024-09-21T04:02:03.758146+0000","last_became_active":"2024-09-21T04:02:00.741051+0000","last_became_peered":"2024-09-21T04:02:00.741051+0000","last_unstale":"2024-09-21T04:02:03.758146+0000","last_undegraded":"2024-09-21T04:02:03.758146+0000","last_fullsized":"2024-09-21T04:02:03.758146+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:05.944430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.753784+0000","last_change":"2024-09-21T04:02:02.761703+0000","last_active":"2024-09-21T04:02:03.753784+0000","last_peered":"2024-09-21T04:02:03.753784+0000","last_clean":"2024-09-21T04:02:03.753784+0000","last_became_active":"2024-09-21T04:02:02.761296+0000","last_became_peered":"2024-09-21T04:02:02.761296+0000","last_unstale":"2024-09-21T04:02:03.753784+0000","last_undegraded":"2024-09-21T04:02:03.753784+0000","last_fullsized":"2024-09-21T04:02:03.753784+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:57.337167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.753977+0000","last_change":"2024-09-21T04:02:00.853584+0000","last_active":"2024-09-21T04:02:03.753977+0000","last_peered":"2024-09-21T04:02:03.753977+0000","last_clean":"2024-09-21T04:02:03.753977+0000","last_became_active":"2024-09-21T04:02:00.853019+0000","last_became_peered":"2024-09-21T04:02:00.853019+0000","last_unstale":"2024-09-21T04:02:03.753977+0000","last_undegraded":"2024-09-21T04:02:03.753977+0000","last_fullsized":"2024-09-21T04:02:03.753977+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:56.186791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"73'1","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.754186+0000","last_change":"2024-09-21T04:02:02.758175+0000","last_active":"2024-09-21T04:02:03.754186+0000","last_peered":"2024-09-21T04:02:03.754186+0000","last_clean":"2024-09-21T04:02:03.754186+0000","last_became_active":"2024-09-21T04:02:02.757715+0000","last_became_peered":"2024-09-21T04:02:02.757715+0000","last_unstale":"2024-09-21T04:02:03.754186+0000","last_undegraded":"2024-09-21T04:02:03.754186+0000","last_fullsized":"2024-09-21T04:02:03.754186+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:40.172569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"73'1","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:10.795895+0000","last_change":"2024-09-21T04:02:00.853325+0000","last_active":"2024-09-21T04:02:10.795895+0000","last_peered":"2024-09-21T04:02:10.795895+0000","last_clean":"2024-09-21T04:02:10.795895+0000","last_became_active":"2024-09-21T04:02:00.852834+0000","last_became_peered":"2024-09-21T04:02:00.852834+0000","last_unstale":"2024-09-21T04:02:10.795895+0000","last_undegraded":"2024-09-21T04:02:10.795895+0000","last_fullsized":"2024-09-21T04:02:10.795895+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:16.090493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.747832+0000","last_change":"2024-09-21T04:02:02.752402+0000","last_active":"2024-09-21T04:02:03.747832+0000","last_peered":"2024-09-21T04:02:03.747832+0000","last_clean":"2024-09-21T04:02:03.747832+0000","last_became_active":"2024-09-21T04:02:02.751994+0000","last_became_peered":"2024-09-21T04:02:02.751994+0000","last_unstale":"2024-09-21T04:02:03.747832+0000","last_undegraded":"2024-09-21T04:02:03.747832+0000","last_fullsized":"2024-09-21T04:02:03.747832+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:19.525317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"73'6","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:18.752744+0000","last_change":"2024-09-21T04:02:00.739112+0000","last_active":"2024-09-21T04:02:18.752744+0000","last_peered":"2024-09-21T04:02:18.752744+0000","last_clean":"2024-09-21T04:02:18.752744+0000","last_became_active":"2024-09-21T04:02:00.738613+0000","last_became_peered":"2024-09-21T04:02:00.738613+0000","last_unstale":"2024-09-21T04:02:18.752744+0000","last_undegraded":"2024-09-21T04:02:18.752744+0000","last_fullsized":"2024-09-21T04:02:18.752744+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:34.334576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.755229+0000","last_change":"2024-09-21T04:02:02.763345+0000","last_active":"2024-09-21T04:02:03.755229+0000","last_peered":"2024-09-21T04:02:03.755229+0000","last_clean":"2024-09-21T04:02:03.755229+0000","last_became_active":"2024-09-21T04:02:02.762987+0000","last_became_peered":"2024-09-21T04:02:02.762987+0000","last_unstale":"2024-09-21T04:02:03.755229+0000","last_undegraded":"2024-09-21T04:02:03.755229+0000","last_fullsized":"2024-09-21T04:02:03.755229+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:36.239267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"73'4","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:10.794791+0000","last_change":"2024-09-21T04:02:01.258208+0000","last_active":"2024-09-21T04:02:10.794791+0000","last_peered":"2024-09-21T04:02:10.794791+0000","last_clean":"2024-09-21T04:02:10.794791+0000","last_became_active":"2024-09-21T04:02:01.257780+0000","last_became_peered":"2024-09-21T04:02:01.257780+0000","last_unstale":"2024-09-21T04:02:10.794791+0000","last_undegraded":"2024-09-21T04:02:10.794791+0000","last_fullsized":"2024-09-21T04:02:10.794791+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:20.161792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.759536+0000","last_change":"2024-09-21T04:02:02.761149+0000","last_active":"2024-09-21T04:02:03.759536+0000","last_peered":"2024-09-21T04:02:03.759536+0000","last_clean":"2024-09-21T04:02:03.759536+0000","last_became_active":"2024-09-21T04:02:02.760771+0000","last_became_peered":"2024-09-21T04:02:02.760771+0000","last_unstale":"2024-09-21T04:02:03.759536+0000","last_undegraded":"2024-09-21T04:02:03.759536+0000","last_fullsized":"2024-09-21T04:02:03.759536+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:20.645377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"73'2","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:10.796190+0000","last_change":"2024-09-21T04:02:00.853580+0000","last_active":"2024-09-21T04:02:10.796190+0000","last_peered":"2024-09-21T04:02:10.796190+0000","last_clean":"2024-09-21T04:02:10.796190+0000","last_became_active":"2024-09-21T04:02:00.853091+0000","last_became_peered":"2024-09-21T04:02:00.853091+0000","last_unstale":"2024-09-21T04:02:10.796190+0000","last_undegraded":"2024-09-21T04:02:10.796190+0000","last_fullsized":"2024-09-21T04:02:10.796190+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:32.893695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.753191+0000","last_change":"2024-09-21T04:02:02.769686+0000","last_active":"2024-09-21T04:02:03.753191+0000","last_peered":"2024-09-21T04:02:03.753191+0000","last_clean":"2024-09-21T04:02:03.753191+0000","last_became_active":"2024-09-21T04:02:02.769293+0000","last_became_peered":"2024-09-21T04:02:02.769293+0000","last_unstale":"2024-09-21T04:02:03.753191+0000","last_undegraded":"2024-09-21T04:02:03.753191+0000","last_fullsized":"2024-09-21T04:02:03.753191+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:10.984624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"73'3","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:10.796422+0000","last_change":"2024-09-21T04:02:00.741556+0000","last_active":"2024-09-21T04:02:10.796422+0000","last_peered":"2024-09-21T04:02:10.796422+0000","last_clean":"2024-09-21T04:02:10.796422+0000","last_became_active":"2024-09-21T04:02:00.741188+0000","last_became_peered":"2024-09-21T04:02:00.741188+0000","last_unstale":"2024-09-21T04:02:10.796422+0000","last_undegraded":"2024-09-21T04:02:10.796422+0000","last_fullsized":"2024-09-21T04:02:10.796422+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:23.566814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.754188+0000","last_change":"2024-09-21T04:02:02.769163+0000","last_active":"2024-09-21T04:02:03.754188+0000","last_peered":"2024-09-21T04:02:03.754188+0000","last_clean":"2024-09-21T04:02:03.754188+0000","last_became_active":"2024-09-21T04:02:02.768804+0000","last_became_peered":"2024-09-21T04:02:02.768804+0000","last_unstale":"2024-09-21T04:02:03.754188+0000","last_undegraded":"2024-09-21T04:02:03.754188+0000","last_fullsized":"2024-09-21T04:02:03.754188+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:12.576553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"73'1","reported_seq":20,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.746224+0000","last_change":"2024-09-21T04:02:00.743091+0000","last_active":"2024-09-21T04:02:03.746224+0000","last_peered":"2024-09-21T04:02:03.746224+0000","last_clean":"2024-09-21T04:02:03.746224+0000","last_became_active":"2024-09-21T04:02:00.742665+0000","last_became_peered":"2024-09-21T04:02:00.742665+0000","last_unstale":"2024-09-21T04:02:03.746224+0000","last_undegraded":"2024-09-21T04:02:03.746224+0000","last_fullsized":"2024-09-21T04:02:03.746224+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:09.698421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.754310+0000","last_change":"2024-09-21T04:02:02.761985+0000","last_active":"2024-09-21T04:02:03.754310+0000","last_peered":"2024-09-21T04:02:03.754310+0000","last_clean":"2024-09-21T04:02:03.754310+0000","last_became_active":"2024-09-21T04:02:02.761624+0000","last_became_peered":"2024-09-21T04:02:02.761624+0000","last_unstale":"2024-09-21T04:02:03.754310+0000","last_undegraded":"2024-09-21T04:02:03.754310+0000","last_fullsized":"2024-09-21T04:02:03.754310+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:12.912221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"73'4","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:10.796449+0000","last_change":"2024-09-21T04:02:00.743219+0000","last_active":"2024-09-21T04:02:10.796449+0000","last_peered":"2024-09-21T04:02:10.796449+0000","last_clean":"2024-09-21T04:02:10.796449+0000","last_became_active":"2024-09-21T04:02:00.742740+0000","last_became_peered":"2024-09-21T04:02:00.742740+0000","last_unstale":"2024-09-21T04:02:10.796449+0000","last_undegraded":"2024-09-21T04:02:10.796449+0000","last_fullsized":"2024-09-21T04:02:10.796449+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:54.012137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.757837+0000","last_change":"2024-09-21T04:02:02.761357+0000","last_active":"2024-09-21T04:02:03.757837+0000","last_peered":"2024-09-21T04:02:03.757837+0000","last_clean":"2024-09-21T04:02:03.757837+0000","last_became_active":"2024-09-21T04:02:02.760984+0000","last_became_peered":"2024-09-21T04:02:02.760984+0000","last_unstale":"2024-09-21T04:02:03.757837+0000","last_undegraded":"2024-09-21T04:02:03.757837+0000","last_fullsized":"2024-09-21T04:02:03.757837+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:55.388768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.759536+0000","last_change":"2024-09-21T04:02:02.772272+0000","last_active":"2024-09-21T04:02:03.759536+0000","last_peered":"2024-09-21T04:02:03.759536+0000","last_clean":"2024-09-21T04:02:03.759536+0000","last_became_active":"2024-09-21T04:02:02.771824+0000","last_became_peered":"2024-09-21T04:02:02.771824+0000","last_unstale":"2024-09-21T04:02:03.759536+0000","last_undegraded":"2024-09-21T04:02:03.759536+0000","last_fullsized":"2024-09-21T04:02:03.759536+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:28:56.292477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"73'2","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.760103+0000","last_change":"2024-09-21T04:02:00.742230+0000","last_active":"2024-09-21T04:02:03.760103+0000","last_peered":"2024-09-21T04:02:03.760103+0000","last_clean":"2024-09-21T04:02:03.760103+0000","last_became_active":"2024-09-21T04:02:00.741838+0000","last_became_peered":"2024-09-21T04:02:00.741838+0000","last_unstale":"2024-09-21T04:02:03.760103+0000","last_undegraded":"2024-09-21T04:02:03.760103+0000","last_fullsized":"2024-09-21T04:02:03.760103+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:18.909859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.758167+0000","last_change":"2024-09-21T04:02:02.762336+0000","last_active":"2024-09-21T04:02:03.758167+0000","last_peered":"2024-09-21T04:02:03.758167+0000","last_clean":"2024-09-21T04:02:03.758167+0000","last_became_active":"2024-09-21T04:02:02.761753+0000","last_became_peered":"2024-09-21T04:02:02.761753+0000","last_unstale":"2024-09-21T04:02:03.758167+0000","last_undegraded":"2024-09-21T04:02:03.758167+0000","last_fullsized":"2024-09-21T04:02:03.758167+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:39.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"73'4","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:11.802902+0000","last_change":"2024-09-21T04:02:00.743495+0000","last_active":"2024-09-21T04:02:11.802902+0000","last_peered":"2024-09-21T04:02:11.802902+0000","last_clean":"2024-09-21T04:02:11.802902+0000","last_became_active":"2024-09-21T04:02:00.743135+0000","last_became_peered":"2024-09-21T04:02:00.743135+0000","last_unstale":"2024-09-21T04:02:11.802902+0000","last_undegraded":"2024-09-21T04:02:11.802902+0000","last_fullsized":"2024-09-21T04:02:11.802902+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:16.481148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.760048+0000","last_change":"2024-09-21T04:02:02.766682+0000","last_active":"2024-09-21T04:02:03.760048+0000","last_peered":"2024-09-21T04:02:03.760048+0000","last_clean":"2024-09-21T04:02:03.760048+0000","last_became_active":"2024-09-21T04:02:02.766299+0000","last_became_peered":"2024-09-21T04:02:02.766299+0000","last_unstale":"2024-09-21T04:02:03.760048+0000","last_undegraded":"2024-09-21T04:02:03.760048+0000","last_fullsized":"2024-09-21T04:02:03.760048+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:25.635908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"73'2","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.762819+0000","last_change":"2024-09-21T04:02:00.852281+0000","last_active":"2024-09-21T04:02:03.762819+0000","last_peered":"2024-09-21T04:02:03.762819+0000","last_clean":"2024-09-21T04:02:03.762819+0000","last_became_active":"2024-09-21T04:02:00.851900+0000","last_became_peered":"2024-09-21T04:02:00.851900+0000","last_unstale":"2024-09-21T04:02:03.762819+0000","last_undegraded":"2024-09-21T04:02:03.762819+0000","last_fullsized":"2024-09-21T04:02:03.762819+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:37.115541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.755481+0000","last_change":"2024-09-21T04:02:02.770467+0000","last_active":"2024-09-21T04:02:03.755481+0000","last_peered":"2024-09-21T04:02:03.755481+0000","last_clean":"2024-09-21T04:02:03.755481+0000","last_became_active":"2024-09-21T04:02:02.770099+0000","last_became_peered":"2024-09-21T04:02:02.770099+0000","last_unstale":"2024-09-21T04:02:03.755481+0000","last_undegraded":"2024-09-21T04:02:03.755481+0000","last_fullsized":"2024-09-21T04:02:03.755481+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:38.806298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"73'9","reported_seq":29,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:11.803233+0000","last_change":"2024-09-21T04:02:00.742641+0000","last_active":"2024-09-21T04:02:11.803233+0000","last_peered":"2024-09-21T04:02:11.803233+0000","last_clean":"2024-09-21T04:02:11.803233+0000","last_became_active":"2024-09-21T04:02:00.742380+0000","last_became_peered":"2024-09-21T04:02:00.742380+0000","last_unstale":"2024-09-21T04:02:11.803233+0000","last_undegraded":"2024-09-21T04:02:11.803233+0000","last_fullsized":"2024-09-21T04:02:11.803233+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:27.354314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":48813,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":94,"num_read_kb":98,"num_write":143,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":372736,"data_stored":157913,"data_compressed":5676,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":124,"ondisk_log_size":124,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282548,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808849,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31956,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713196,"statfs":{"total":95995035648,"available":95962312704,"internally_reserved":0,"allocated":790528,"data_stored":388501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":62,"seq":266287972373,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":1025185,"data_compressed":37818,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135899,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":806912,"data_stored":388737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332128,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":806912,"data_stored":388657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495653,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":856064,"data_stored":980196,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659179,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31960,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713192,"statfs":{"total":95995035648,"available":95962308608,"internally_reserved":0,"allocated":794624,"data_stored":388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855408,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":979880,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018933,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717348,"statfs":{"total":95995035648,"available":95966564352,"internally_reserved":0,"allocated":798720,"data_stored":388563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":811008,"data_stored":388437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378687,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":868352,"data_stored":440209,"data_compressed":1892,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542212,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":843776,"data_stored":438859,"data_compressed":1892,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705737,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":815104,"data_stored":389397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34306,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34306,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34306,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":50940,"data_compressed":1892,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":52290,"data_compressed":1892,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":46781,"data_compressed":1892,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T04:02:29.093 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T04:02:29.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:28 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]} : dispatch 2024-09-21T04:02:29.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:28 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]} : dispatch 2024-09-21T04:02:29.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:28 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]} : dispatch 2024-09-21T04:02:29.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:28 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]} : dispatch 2024-09-21T04:02:29.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:28 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]} : dispatch 2024-09-21T04:02:29.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:28 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]} : dispatch 2024-09-21T04:02:29.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:28 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]} : dispatch 2024-09-21T04:02:29.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:28 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]} : dispatch 2024-09-21T04:02:29.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:28 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]} : dispatch 2024-09-21T04:02:29.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:28 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]} : dispatch 2024-09-21T04:02:29.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:28 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]} : dispatch 2024-09-21T04:02:29.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:28 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]} : dispatch 2024-09-21T04:02:29.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:28 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]} : dispatch 2024-09-21T04:02:29.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:28 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]} : dispatch 2024-09-21T04:02:29.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:28 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]} : dispatch 2024-09-21T04:02:29.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:28 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]} : dispatch 2024-09-21T04:02:29.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:28 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]} : dispatch 2024-09-21T04:02:29.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:28 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]} : dispatch 2024-09-21T04:02:29.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:28 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]} : dispatch 2024-09-21T04:02:29.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:28 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]} : dispatch 2024-09-21T04:02:29.364 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:29.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.855+0000 7f8824b3a640 1 -- 172.21.15.64:0/3614102786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88201083b0 msgr2=0x7f88201087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:29.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.855+0000 7f8824b3a640 1 --2- 172.21.15.64:0/3614102786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88201083b0 0x7f88201087c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8808009900 tx=0x7f880802a410 comp rx=0 tx=0).stop 2024-09-21T04:02:29.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.855+0000 7f8824b3a640 1 -- 172.21.15.64:0/3614102786 shutdown_connections 2024-09-21T04:02:29.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.855+0000 7f8824b3a640 1 --2- 172.21.15.64:0/3614102786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8820108d00 0x7f882010f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:29.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.855+0000 7f8824b3a640 1 --2- 172.21.15.64:0/3614102786 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88201083b0 0x7f88201087c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:29.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.855+0000 7f8824b3a640 1 --2- 172.21.15.64:0/3614102786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8820102a70 0x7f8820102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:29.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.855+0000 7f8824b3a640 1 -- 172.21.15.64:0/3614102786 >> 172.21.15.64:0/3614102786 conn(0x7f8820079710 msgr2=0x7f8820079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:29.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.855+0000 7f8824b3a640 1 -- 172.21.15.64:0/3614102786 shutdown_connections 2024-09-21T04:02:29.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.855+0000 7f8824b3a640 1 -- 172.21.15.64:0/3614102786 wait complete. 2024-09-21T04:02:29.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.856+0000 7f8824b3a640 1 Processor -- start 2024-09-21T04:02:29.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.856+0000 7f8824b3a640 1 -- start start 2024-09-21T04:02:29.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.856+0000 7f8824b3a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8820102a70 0x7f882019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:29.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.856+0000 7f8824b3a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88201083b0 0x7f882019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:29.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.857+0000 7f8824b3a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8820108d00 0x7f882019fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:29.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.857+0000 7f8824b3a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8820112a60 con 0x7f88201083b0 2024-09-21T04:02:29.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.857+0000 7f8824b3a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f88201128e0 con 0x7f8820108d00 2024-09-21T04:02:29.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.857+0000 7f8824b3a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8820112be0 con 0x7f8820102a70 2024-09-21T04:02:29.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.857+0000 7f881dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88201083b0 0x7f882019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:29.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.857+0000 7f881dd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88201083b0 0x7f882019abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34688/0 (socket says 172.21.15.64:34688) 2024-09-21T04:02:29.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.857+0000 7f881dd9b640 1 -- 172.21.15.64:0/3854048663 learned_addr learned my addr 172.21.15.64:0/3854048663 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:29.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.857+0000 7f881e59c640 1 --2- 172.21.15.64:0/3854048663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8820102a70 0x7f882019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:29.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.857+0000 7f881ed9d640 1 --2- 172.21.15.64:0/3854048663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8820108d00 0x7f882019fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:29.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.857+0000 7f881dd9b640 1 -- 172.21.15.64:0/3854048663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8820102a70 msgr2=0x7f882019a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:29.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.857+0000 7f881dd9b640 1 --2- 172.21.15.64:0/3854048663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8820102a70 0x7f882019a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:29.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.857+0000 7f881dd9b640 1 -- 172.21.15.64:0/3854048663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8820108d00 msgr2=0x7f882019fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:29.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.857+0000 7f881dd9b640 1 --2- 172.21.15.64:0/3854048663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8820108d00 0x7f882019fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:29.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.857+0000 7f881dd9b640 1 -- 172.21.15.64:0/3854048663 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88201a0590 con 0x7f88201083b0 2024-09-21T04:02:29.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.857+0000 7f881dd9b640 1 --2- 172.21.15.64:0/3854048663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88201083b0 0x7f882019abc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f880802a930 tx=0x7f8808004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:29.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.867+0000 7f88177fe640 1 -- 172.21.15.64:0/3854048663 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880803f030 con 0x7f88201083b0 2024-09-21T04:02:29.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.867+0000 7f8824b3a640 1 -- 172.21.15.64:0/3854048663 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88201a0820 con 0x7f88201083b0 2024-09-21T04:02:29.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.867+0000 7f8824b3a640 1 -- 172.21.15.64:0/3854048663 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88201a6fe0 con 0x7f88201083b0 2024-09-21T04:02:29.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.867+0000 7f88177fe640 1 -- 172.21.15.64:0/3854048663 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8808003cb0 con 0x7f88201083b0 2024-09-21T04:02:29.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.867+0000 7f88177fe640 1 -- 172.21.15.64:0/3854048663 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808049060 con 0x7f88201083b0 2024-09-21T04:02:29.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.869+0000 7f88157fa640 1 -- 172.21.15.64:0/3854048663 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8820103cf0 con 0x7f88201083b0 2024-09-21T04:02:29.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.871+0000 7f88177fe640 1 -- 172.21.15.64:0/3854048663 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f8808042030 con 0x7f88201083b0 2024-09-21T04:02:29.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.872+0000 7f88177fe640 1 --2- 172.21.15.64:0/3854048663 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f880c0b6670 0x7f880c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:29.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.872+0000 7f881e59c640 1 --2- 172.21.15.64:0/3854048663 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f880c0b6670 0x7f880c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:29.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.872+0000 7f88177fe640 1 -- 172.21.15.64:0/3854048663 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f8808107d30 con 0x7f88201083b0 2024-09-21T04:02:29.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.872+0000 7f881e59c640 1 --2- 172.21.15.64:0/3854048663 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f880c0b6670 0x7f880c0b8b80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8818009990 tx=0x7f8818009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:29.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:29.879+0000 7f88177fe640 1 -- 172.21.15.64:0/3854048663 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f880810c070 con 0x7f88201083b0 2024-09-21T04:02:30.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:30.068+0000 7f88157fa640 1 -- 172.21.15.64:0/3854048663 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f882019b750 con 0x7f880c0b6670 2024-09-21T04:02:30.070 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:29 smithi064 ceph-mon[31180]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:02:30.071 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:29 smithi064 ceph-mon[31180]: pgmap v271: 97 pgs: 97 active+clean; 630 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-09-21T04:02:30.071 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-21T04:02:30.071 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-09-21T04:02:30.071 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-21T04:02:30.071 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-09-21T04:02:30.071 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-09-21T04:02:30.071 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-09-21T04:02:30.071 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-21T04:02:30.071 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-09-21T04:02:30.071 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-09-21T04:02:30.071 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-09-21T04:02:30.071 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:29 smithi064 ceph-mon[31180]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T04:02:30.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:30.072+0000 7f88177fe640 1 -- 172.21.15.64:0/3854048663 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258903 (secure 0 0 0) 0x7f882019b750 con 0x7f880c0b6670 2024-09-21T04:02:30.077 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:02:30.078 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-09-21T04:02:30.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:30.076+0000 7f88157fa640 1 -- 172.21.15.64:0/3854048663 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f880c0b6670 msgr2=0x7f880c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:30.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:30.076+0000 7f88157fa640 1 --2- 172.21.15.64:0/3854048663 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f880c0b6670 0x7f880c0b8b80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8818009990 tx=0x7f8818009110 comp rx=0 tx=0).stop 2024-09-21T04:02:30.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:30.076+0000 7f88157fa640 1 -- 172.21.15.64:0/3854048663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88201083b0 msgr2=0x7f882019abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:30.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:30.076+0000 7f88157fa640 1 --2- 172.21.15.64:0/3854048663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88201083b0 0x7f882019abc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f880802a930 tx=0x7f8808004270 comp rx=0 tx=0).stop 2024-09-21T04:02:30.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:30.076+0000 7f88157fa640 1 -- 172.21.15.64:0/3854048663 shutdown_connections 2024-09-21T04:02:30.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:30.076+0000 7f88157fa640 1 --2- 172.21.15.64:0/3854048663 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f880c0b6670 0x7f880c0b8b80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:30.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:30.076+0000 7f88157fa640 1 --2- 172.21.15.64:0/3854048663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8820108d00 0x7f882019fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:30.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:30.076+0000 7f88157fa640 1 --2- 172.21.15.64:0/3854048663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88201083b0 0x7f882019abc0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:30.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:30.076+0000 7f88157fa640 1 --2- 172.21.15.64:0/3854048663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8820102a70 0x7f882019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:30.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:30.076+0000 7f88157fa640 1 -- 172.21.15.64:0/3854048663 >> 172.21.15.64:0/3854048663 conn(0x7f8820079710 msgr2=0x7f8820101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:30.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:30.076+0000 7f88157fa640 1 -- 172.21.15.64:0/3854048663 shutdown_connections 2024-09-21T04:02:30.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:30.076+0000 7f88157fa640 1 -- 172.21.15.64:0/3854048663 wait complete. 2024-09-21T04:02:30.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:29 smithi119 ceph-mon[34811]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:02:30.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:29 smithi119 ceph-mon[34811]: pgmap v271: 97 pgs: 97 active+clean; 630 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-09-21T04:02:30.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-21T04:02:30.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-09-21T04:02:30.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-21T04:02:30.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-09-21T04:02:30.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-09-21T04:02:30.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-09-21T04:02:30.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-21T04:02:30.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-09-21T04:02:30.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-09-21T04:02:30.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-09-21T04:02:30.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:29 smithi119 ceph-mon[34811]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T04:02:30.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:29 smithi131 ceph-mon[35193]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:02:30.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:29 smithi131 ceph-mon[35193]: pgmap v271: 97 pgs: 97 active+clean; 630 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-09-21T04:02:30.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-21T04:02:30.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-09-21T04:02:30.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-09-21T04:02:30.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-09-21T04:02:30.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-09-21T04:02:30.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-09-21T04:02:30.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-21T04:02:30.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-09-21T04:02:30.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-09-21T04:02:30.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-09-21T04:02:30.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:29 smithi131 ceph-mon[35193]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T04:02:30.809 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":273,"stamp":"2024-09-21T04:02:29.861339+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":645152,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":191,"num_read_kb":183,"num_write":276,"num_write_kb":2249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359004,"kb_used_data":9756,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124582820,"statfs":{"total":1151940427776,"available":1151572807680,"internally_reserved":0,"allocated":9990144,"data_stored":6590484,"data_compressed":113454,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42962,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":63,"num_read_kb":73,"num_write":23,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.382494"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.754741+0000","last_change":"2024-09-21T04:02:02.769848+0000","last_active":"2024-09-21T04:02:03.754741+0000","last_peered":"2024-09-21T04:02:03.754741+0000","last_clean":"2024-09-21T04:02:03.754741+0000","last_became_active":"2024-09-21T04:02:02.769508+0000","last_became_peered":"2024-09-21T04:02:02.769508+0000","last_unstale":"2024-09-21T04:02:03.754741+0000","last_undegraded":"2024-09-21T04:02:03.754741+0000","last_fullsized":"2024-09-21T04:02:03.754741+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:00.242233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.759296+0000","last_change":"2024-09-21T04:02:02.769200+0000","last_active":"2024-09-21T04:02:03.759296+0000","last_peered":"2024-09-21T04:02:03.759296+0000","last_clean":"2024-09-21T04:02:03.759296+0000","last_became_active":"2024-09-21T04:02:02.768918+0000","last_became_peered":"2024-09-21T04:02:02.768918+0000","last_unstale":"2024-09-21T04:02:03.759296+0000","last_undegraded":"2024-09-21T04:02:03.759296+0000","last_fullsized":"2024-09-21T04:02:03.759296+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:26.842079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.759579+0000","last_change":"2024-09-21T04:02:02.763511+0000","last_active":"2024-09-21T04:02:03.759579+0000","last_peered":"2024-09-21T04:02:03.759579+0000","last_clean":"2024-09-21T04:02:03.759579+0000","last_became_active":"2024-09-21T04:02:02.763161+0000","last_became_peered":"2024-09-21T04:02:02.763161+0000","last_unstale":"2024-09-21T04:02:03.759579+0000","last_undegraded":"2024-09-21T04:02:03.759579+0000","last_fullsized":"2024-09-21T04:02:03.759579+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:59.834005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.746577+0000","last_change":"2024-09-21T04:02:02.765452+0000","last_active":"2024-09-21T04:02:03.746577+0000","last_peered":"2024-09-21T04:02:03.746577+0000","last_clean":"2024-09-21T04:02:03.746577+0000","last_became_active":"2024-09-21T04:02:02.765065+0000","last_became_peered":"2024-09-21T04:02:02.765065+0000","last_unstale":"2024-09-21T04:02:03.746577+0000","last_undegraded":"2024-09-21T04:02:03.746577+0000","last_fullsized":"2024-09-21T04:02:03.746577+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:33.871366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.752693+0000","last_change":"2024-09-21T04:02:02.767457+0000","last_active":"2024-09-21T04:02:03.752693+0000","last_peered":"2024-09-21T04:02:03.752693+0000","last_clean":"2024-09-21T04:02:03.752693+0000","last_became_active":"2024-09-21T04:02:02.767064+0000","last_became_peered":"2024-09-21T04:02:02.767064+0000","last_unstale":"2024-09-21T04:02:03.752693+0000","last_undegraded":"2024-09-21T04:02:03.752693+0000","last_fullsized":"2024-09-21T04:02:03.752693+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:46.316029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.753553+0000","last_change":"2024-09-21T04:02:02.768795+0000","last_active":"2024-09-21T04:02:03.753553+0000","last_peered":"2024-09-21T04:02:03.753553+0000","last_clean":"2024-09-21T04:02:03.753553+0000","last_became_active":"2024-09-21T04:02:02.768434+0000","last_became_peered":"2024-09-21T04:02:02.768434+0000","last_unstale":"2024-09-21T04:02:03.753553+0000","last_undegraded":"2024-09-21T04:02:03.753553+0000","last_fullsized":"2024-09-21T04:02:03.753553+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:58.991533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.757595+0000","last_change":"2024-09-21T04:02:02.774998+0000","last_active":"2024-09-21T04:02:03.757595+0000","last_peered":"2024-09-21T04:02:03.757595+0000","last_clean":"2024-09-21T04:02:03.757595+0000","last_became_active":"2024-09-21T04:02:02.774344+0000","last_became_peered":"2024-09-21T04:02:02.774344+0000","last_unstale":"2024-09-21T04:02:03.757595+0000","last_undegraded":"2024-09-21T04:02:03.757595+0000","last_fullsized":"2024-09-21T04:02:03.757595+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:26.705126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.754679+0000","last_change":"2024-09-21T04:02:02.772498+0000","last_active":"2024-09-21T04:02:03.754679+0000","last_peered":"2024-09-21T04:02:03.754679+0000","last_clean":"2024-09-21T04:02:03.754679+0000","last_became_active":"2024-09-21T04:02:02.772049+0000","last_became_peered":"2024-09-21T04:02:02.772049+0000","last_unstale":"2024-09-21T04:02:03.754679+0000","last_undegraded":"2024-09-21T04:02:03.754679+0000","last_fullsized":"2024-09-21T04:02:03.754679+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:31:19.088424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.758659+0000","last_change":"2024-09-21T04:02:02.772336+0000","last_active":"2024-09-21T04:02:03.758659+0000","last_peered":"2024-09-21T04:02:03.758659+0000","last_clean":"2024-09-21T04:02:03.758659+0000","last_became_active":"2024-09-21T04:02:02.771964+0000","last_became_peered":"2024-09-21T04:02:02.771964+0000","last_unstale":"2024-09-21T04:02:03.758659+0000","last_undegraded":"2024-09-21T04:02:03.758659+0000","last_fullsized":"2024-09-21T04:02:03.758659+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:46.659690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.754261+0000","last_change":"2024-09-21T04:02:02.758992+0000","last_active":"2024-09-21T04:02:03.754261+0000","last_peered":"2024-09-21T04:02:03.754261+0000","last_clean":"2024-09-21T04:02:03.754261+0000","last_became_active":"2024-09-21T04:02:02.758575+0000","last_became_peered":"2024-09-21T04:02:02.758575+0000","last_unstale":"2024-09-21T04:02:03.754261+0000","last_undegraded":"2024-09-21T04:02:03.754261+0000","last_fullsized":"2024-09-21T04:02:03.754261+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:32:09.543140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.758999+0000","last_change":"2024-09-21T04:02:02.767304+0000","last_active":"2024-09-21T04:02:03.758999+0000","last_peered":"2024-09-21T04:02:03.758999+0000","last_clean":"2024-09-21T04:02:03.758999+0000","last_became_active":"2024-09-21T04:02:02.762804+0000","last_became_peered":"2024-09-21T04:02:02.762804+0000","last_unstale":"2024-09-21T04:02:03.758999+0000","last_undegraded":"2024-09-21T04:02:03.758999+0000","last_fullsized":"2024-09-21T04:02:03.758999+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:32.224071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.746822+0000","last_change":"2024-09-21T04:02:02.762932+0000","last_active":"2024-09-21T04:02:03.746822+0000","last_peered":"2024-09-21T04:02:03.746822+0000","last_clean":"2024-09-21T04:02:03.746822+0000","last_became_active":"2024-09-21T04:02:02.762538+0000","last_became_peered":"2024-09-21T04:02:02.762538+0000","last_unstale":"2024-09-21T04:02:03.746822+0000","last_undegraded":"2024-09-21T04:02:03.746822+0000","last_fullsized":"2024-09-21T04:02:03.746822+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:20.724453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.747667+0000","last_change":"2024-09-21T04:02:02.770313+0000","last_active":"2024-09-21T04:02:03.747667+0000","last_peered":"2024-09-21T04:02:03.747667+0000","last_clean":"2024-09-21T04:02:03.747667+0000","last_became_active":"2024-09-21T04:02:02.769910+0000","last_became_peered":"2024-09-21T04:02:02.769910+0000","last_unstale":"2024-09-21T04:02:03.747667+0000","last_undegraded":"2024-09-21T04:02:03.747667+0000","last_fullsized":"2024-09-21T04:02:03.747667+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:16.228371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.759582+0000","last_change":"2024-09-21T04:02:02.769258+0000","last_active":"2024-09-21T04:02:03.759582+0000","last_peered":"2024-09-21T04:02:03.759582+0000","last_clean":"2024-09-21T04:02:03.759582+0000","last_became_active":"2024-09-21T04:02:02.768914+0000","last_became_peered":"2024-09-21T04:02:02.768914+0000","last_unstale":"2024-09-21T04:02:03.759582+0000","last_undegraded":"2024-09-21T04:02:03.759582+0000","last_fullsized":"2024-09-21T04:02:03.759582+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:37.426810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.759046+0000","last_change":"2024-09-21T04:02:02.755940+0000","last_active":"2024-09-21T04:02:03.759046+0000","last_peered":"2024-09-21T04:02:03.759046+0000","last_clean":"2024-09-21T04:02:03.759046+0000","last_became_active":"2024-09-21T04:02:02.755309+0000","last_became_peered":"2024-09-21T04:02:02.755309+0000","last_unstale":"2024-09-21T04:02:03.759046+0000","last_undegraded":"2024-09-21T04:02:03.759046+0000","last_fullsized":"2024-09-21T04:02:03.759046+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:49.834030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.757784+0000","last_change":"2024-09-21T04:02:02.761068+0000","last_active":"2024-09-21T04:02:03.757784+0000","last_peered":"2024-09-21T04:02:03.757784+0000","last_clean":"2024-09-21T04:02:03.757784+0000","last_became_active":"2024-09-21T04:02:02.760683+0000","last_became_peered":"2024-09-21T04:02:02.760683+0000","last_unstale":"2024-09-21T04:02:03.757784+0000","last_undegraded":"2024-09-21T04:02:03.757784+0000","last_fullsized":"2024-09-21T04:02:03.757784+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:36.627539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.755039+0000","last_change":"2024-09-21T04:02:02.772234+0000","last_active":"2024-09-21T04:02:03.755039+0000","last_peered":"2024-09-21T04:02:03.755039+0000","last_clean":"2024-09-21T04:02:03.755039+0000","last_became_active":"2024-09-21T04:02:02.771825+0000","last_became_peered":"2024-09-21T04:02:02.771825+0000","last_unstale":"2024-09-21T04:02:03.755039+0000","last_undegraded":"2024-09-21T04:02:03.755039+0000","last_fullsized":"2024-09-21T04:02:03.755039+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:33.449745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.753645+0000","last_change":"2024-09-21T04:02:02.758643+0000","last_active":"2024-09-21T04:02:03.753645+0000","last_peered":"2024-09-21T04:02:03.753645+0000","last_clean":"2024-09-21T04:02:03.753645+0000","last_became_active":"2024-09-21T04:02:02.758261+0000","last_became_peered":"2024-09-21T04:02:02.758261+0000","last_unstale":"2024-09-21T04:02:03.753645+0000","last_undegraded":"2024-09-21T04:02:03.753645+0000","last_fullsized":"2024-09-21T04:02:03.753645+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:04.165399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.746552+0000","last_change":"2024-09-21T04:02:02.759405+0000","last_active":"2024-09-21T04:02:03.746552+0000","last_peered":"2024-09-21T04:02:03.746552+0000","last_clean":"2024-09-21T04:02:03.746552+0000","last_became_active":"2024-09-21T04:02:02.758962+0000","last_became_peered":"2024-09-21T04:02:02.758962+0000","last_unstale":"2024-09-21T04:02:03.746552+0000","last_undegraded":"2024-09-21T04:02:03.746552+0000","last_fullsized":"2024-09-21T04:02:03.746552+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:37.203308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.755683+0000","last_change":"2024-09-21T04:02:02.767518+0000","last_active":"2024-09-21T04:02:03.755683+0000","last_peered":"2024-09-21T04:02:03.755683+0000","last_clean":"2024-09-21T04:02:03.755683+0000","last_became_active":"2024-09-21T04:02:02.767147+0000","last_became_peered":"2024-09-21T04:02:02.767147+0000","last_unstale":"2024-09-21T04:02:03.755683+0000","last_undegraded":"2024-09-21T04:02:03.755683+0000","last_fullsized":"2024-09-21T04:02:03.755683+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:34:13.615218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.747768+0000","last_change":"2024-09-21T04:02:02.756407+0000","last_active":"2024-09-21T04:02:03.747768+0000","last_peered":"2024-09-21T04:02:03.747768+0000","last_clean":"2024-09-21T04:02:03.747768+0000","last_became_active":"2024-09-21T04:02:02.756007+0000","last_became_peered":"2024-09-21T04:02:02.756007+0000","last_unstale":"2024-09-21T04:02:03.747768+0000","last_undegraded":"2024-09-21T04:02:03.747768+0000","last_fullsized":"2024-09-21T04:02:03.747768+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:18.798814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.758613+0000","last_change":"2024-09-21T04:02:02.766290+0000","last_active":"2024-09-21T04:02:03.758613+0000","last_peered":"2024-09-21T04:02:03.758613+0000","last_clean":"2024-09-21T04:02:03.758613+0000","last_became_active":"2024-09-21T04:02:02.765660+0000","last_became_peered":"2024-09-21T04:02:02.765660+0000","last_unstale":"2024-09-21T04:02:03.758613+0000","last_undegraded":"2024-09-21T04:02:03.758613+0000","last_fullsized":"2024-09-21T04:02:03.758613+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:23.281476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.754317+0000","last_change":"2024-09-21T04:02:02.767943+0000","last_active":"2024-09-21T04:02:03.754317+0000","last_peered":"2024-09-21T04:02:03.754317+0000","last_clean":"2024-09-21T04:02:03.754317+0000","last_became_active":"2024-09-21T04:02:02.767566+0000","last_became_peered":"2024-09-21T04:02:02.767566+0000","last_unstale":"2024-09-21T04:02:03.754317+0000","last_undegraded":"2024-09-21T04:02:03.754317+0000","last_fullsized":"2024-09-21T04:02:03.754317+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:44.221531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.755758+0000","last_change":"2024-09-21T04:02:02.770114+0000","last_active":"2024-09-21T04:02:03.755758+0000","last_peered":"2024-09-21T04:02:03.755758+0000","last_clean":"2024-09-21T04:02:03.755758+0000","last_became_active":"2024-09-21T04:02:02.769732+0000","last_became_peered":"2024-09-21T04:02:02.769732+0000","last_unstale":"2024-09-21T04:02:03.755758+0000","last_undegraded":"2024-09-21T04:02:03.755758+0000","last_fullsized":"2024-09-21T04:02:03.755758+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:38.396288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.754695+0000","last_change":"2024-09-21T04:02:02.767862+0000","last_active":"2024-09-21T04:02:03.754695+0000","last_peered":"2024-09-21T04:02:03.754695+0000","last_clean":"2024-09-21T04:02:03.754695+0000","last_became_active":"2024-09-21T04:02:02.767475+0000","last_became_peered":"2024-09-21T04:02:02.767475+0000","last_unstale":"2024-09-21T04:02:03.754695+0000","last_undegraded":"2024-09-21T04:02:03.754695+0000","last_fullsized":"2024-09-21T04:02:03.754695+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:26.754666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.746696+0000","last_change":"2024-09-21T04:02:02.764666+0000","last_active":"2024-09-21T04:02:03.746696+0000","last_peered":"2024-09-21T04:02:03.746696+0000","last_clean":"2024-09-21T04:02:03.746696+0000","last_became_active":"2024-09-21T04:02:02.763931+0000","last_became_peered":"2024-09-21T04:02:02.763931+0000","last_unstale":"2024-09-21T04:02:03.746696+0000","last_undegraded":"2024-09-21T04:02:03.746696+0000","last_fullsized":"2024-09-21T04:02:03.746696+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:39.724770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.754152+0000","last_change":"2024-09-21T04:02:02.774276+0000","last_active":"2024-09-21T04:02:03.754152+0000","last_peered":"2024-09-21T04:02:03.754152+0000","last_clean":"2024-09-21T04:02:03.754152+0000","last_became_active":"2024-09-21T04:02:02.773852+0000","last_became_peered":"2024-09-21T04:02:02.773852+0000","last_unstale":"2024-09-21T04:02:03.754152+0000","last_undegraded":"2024-09-21T04:02:03.754152+0000","last_fullsized":"2024-09-21T04:02:03.754152+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:40.072264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.747607+0000","last_change":"2024-09-21T04:02:02.758520+0000","last_active":"2024-09-21T04:02:03.747607+0000","last_peered":"2024-09-21T04:02:03.747607+0000","last_clean":"2024-09-21T04:02:03.747607+0000","last_became_active":"2024-09-21T04:02:02.757656+0000","last_became_peered":"2024-09-21T04:02:02.757656+0000","last_unstale":"2024-09-21T04:02:03.747607+0000","last_undegraded":"2024-09-21T04:02:03.747607+0000","last_fullsized":"2024-09-21T04:02:03.747607+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:30.854336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.759491+0000","last_change":"2024-09-21T04:02:02.756428+0000","last_active":"2024-09-21T04:02:03.759491+0000","last_peered":"2024-09-21T04:02:03.759491+0000","last_clean":"2024-09-21T04:02:03.759491+0000","last_became_active":"2024-09-21T04:02:02.755985+0000","last_became_peered":"2024-09-21T04:02:02.755985+0000","last_unstale":"2024-09-21T04:02:03.759491+0000","last_undegraded":"2024-09-21T04:02:03.759491+0000","last_fullsized":"2024-09-21T04:02:03.759491+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:36.040436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.746155+0000","last_change":"2024-09-21T04:02:02.762893+0000","last_active":"2024-09-21T04:02:03.746155+0000","last_peered":"2024-09-21T04:02:03.746155+0000","last_clean":"2024-09-21T04:02:03.746155+0000","last_became_active":"2024-09-21T04:02:02.762343+0000","last_became_peered":"2024-09-21T04:02:02.762343+0000","last_unstale":"2024-09-21T04:02:03.746155+0000","last_undegraded":"2024-09-21T04:02:03.746155+0000","last_fullsized":"2024-09-21T04:02:03.746155+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:45.074711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.746610+0000","last_change":"2024-09-21T04:02:02.763942+0000","last_active":"2024-09-21T04:02:03.746610+0000","last_peered":"2024-09-21T04:02:03.746610+0000","last_clean":"2024-09-21T04:02:03.746610+0000","last_became_active":"2024-09-21T04:02:02.763433+0000","last_became_peered":"2024-09-21T04:02:02.763433+0000","last_unstale":"2024-09-21T04:02:03.746610+0000","last_undegraded":"2024-09-21T04:02:03.746610+0000","last_fullsized":"2024-09-21T04:02:03.746610+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:36.139440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.753926+0000","last_change":"2024-09-21T04:02:02.773364+0000","last_active":"2024-09-21T04:02:03.753926+0000","last_peered":"2024-09-21T04:02:03.753926+0000","last_clean":"2024-09-21T04:02:03.753926+0000","last_became_active":"2024-09-21T04:02:02.773040+0000","last_became_peered":"2024-09-21T04:02:02.773040+0000","last_unstale":"2024-09-21T04:02:03.753926+0000","last_undegraded":"2024-09-21T04:02:03.753926+0000","last_fullsized":"2024-09-21T04:02:03.753926+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:48.504476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.752962+0000","last_change":"2024-09-21T04:02:02.774951+0000","last_active":"2024-09-21T04:02:03.752962+0000","last_peered":"2024-09-21T04:02:03.752962+0000","last_clean":"2024-09-21T04:02:03.752962+0000","last_became_active":"2024-09-21T04:02:02.774525+0000","last_became_peered":"2024-09-21T04:02:02.774525+0000","last_unstale":"2024-09-21T04:02:03.752962+0000","last_undegraded":"2024-09-21T04:02:03.752962+0000","last_fullsized":"2024-09-21T04:02:03.752962+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:24.447860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"73'5","reported_seq":40,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:24.829799+0000","last_change":"2024-09-21T04:02:00.743470+0000","last_active":"2024-09-21T04:02:24.829799+0000","last_peered":"2024-09-21T04:02:24.829799+0000","last_clean":"2024-09-21T04:02:24.829799+0000","last_became_active":"2024-09-21T04:02:00.743055+0000","last_became_peered":"2024-09-21T04:02:00.743055+0000","last_unstale":"2024-09-21T04:02:24.829799+0000","last_undegraded":"2024-09-21T04:02:24.829799+0000","last_fullsized":"2024-09-21T04:02:24.829799+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:56.101753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.757184+0000","last_change":"2024-09-21T04:02:02.773399+0000","last_active":"2024-09-21T04:02:03.757184+0000","last_peered":"2024-09-21T04:02:03.757184+0000","last_clean":"2024-09-21T04:02:03.757184+0000","last_became_active":"2024-09-21T04:02:02.772996+0000","last_became_peered":"2024-09-21T04:02:02.772996+0000","last_unstale":"2024-09-21T04:02:03.757184+0000","last_undegraded":"2024-09-21T04:02:03.757184+0000","last_fullsized":"2024-09-21T04:02:03.757184+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:07.256226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"73'3","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:11.803438+0000","last_change":"2024-09-21T04:02:00.747383+0000","last_active":"2024-09-21T04:02:11.803438+0000","last_peered":"2024-09-21T04:02:11.803438+0000","last_clean":"2024-09-21T04:02:11.803438+0000","last_became_active":"2024-09-21T04:02:00.746956+0000","last_became_peered":"2024-09-21T04:02:00.746956+0000","last_unstale":"2024-09-21T04:02:11.803438+0000","last_undegraded":"2024-09-21T04:02:11.803438+0000","last_fullsized":"2024-09-21T04:02:11.803438+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:08.320290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"73'2","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:11.801513+0000","last_change":"2024-09-21T04:02:00.736630+0000","last_active":"2024-09-21T04:02:11.801513+0000","last_peered":"2024-09-21T04:02:11.801513+0000","last_clean":"2024-09-21T04:02:11.801513+0000","last_became_active":"2024-09-21T04:02:00.736211+0000","last_became_peered":"2024-09-21T04:02:00.736211+0000","last_unstale":"2024-09-21T04:02:11.801513+0000","last_undegraded":"2024-09-21T04:02:11.801513+0000","last_fullsized":"2024-09-21T04:02:11.801513+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:37.211617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.755272+0000","last_change":"2024-09-21T04:02:02.763323+0000","last_active":"2024-09-21T04:02:03.755272+0000","last_peered":"2024-09-21T04:02:03.755272+0000","last_clean":"2024-09-21T04:02:03.755272+0000","last_became_active":"2024-09-21T04:02:02.762776+0000","last_became_peered":"2024-09-21T04:02:02.762776+0000","last_unstale":"2024-09-21T04:02:03.755272+0000","last_undegraded":"2024-09-21T04:02:03.755272+0000","last_fullsized":"2024-09-21T04:02:03.755272+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:39.891042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"73'1","reported_seq":20,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.756337+0000","last_change":"2024-09-21T04:02:00.743377+0000","last_active":"2024-09-21T04:02:03.756337+0000","last_peered":"2024-09-21T04:02:03.756337+0000","last_clean":"2024-09-21T04:02:03.756337+0000","last_became_active":"2024-09-21T04:02:00.742973+0000","last_became_peered":"2024-09-21T04:02:00.742973+0000","last_unstale":"2024-09-21T04:02:03.756337+0000","last_undegraded":"2024-09-21T04:02:03.756337+0000","last_fullsized":"2024-09-21T04:02:03.756337+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:55:00.655532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.753172+0000","last_change":"2024-09-21T04:02:02.758721+0000","last_active":"2024-09-21T04:02:03.753172+0000","last_peered":"2024-09-21T04:02:03.753172+0000","last_clean":"2024-09-21T04:02:03.753172+0000","last_became_active":"2024-09-21T04:02:02.758352+0000","last_became_peered":"2024-09-21T04:02:02.758352+0000","last_unstale":"2024-09-21T04:02:03.753172+0000","last_undegraded":"2024-09-21T04:02:03.753172+0000","last_fullsized":"2024-09-21T04:02:03.753172+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:56:49.289510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"73'3","reported_seq":37,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:25.830732+0000","last_change":"2024-09-21T04:02:00.744591+0000","last_active":"2024-09-21T04:02:25.830732+0000","last_peered":"2024-09-21T04:02:25.830732+0000","last_clean":"2024-09-21T04:02:25.830732+0000","last_became_active":"2024-09-21T04:02:00.744139+0000","last_became_peered":"2024-09-21T04:02:00.744139+0000","last_unstale":"2024-09-21T04:02:25.830732+0000","last_undegraded":"2024-09-21T04:02:25.830732+0000","last_fullsized":"2024-09-21T04:02:25.830732+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:21.331517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.758125+0000","last_change":"2024-09-21T04:02:02.762779+0000","last_active":"2024-09-21T04:02:03.758125+0000","last_peered":"2024-09-21T04:02:03.758125+0000","last_clean":"2024-09-21T04:02:03.758125+0000","last_became_active":"2024-09-21T04:02:02.760761+0000","last_became_peered":"2024-09-21T04:02:02.760761+0000","last_unstale":"2024-09-21T04:02:03.758125+0000","last_undegraded":"2024-09-21T04:02:03.758125+0000","last_fullsized":"2024-09-21T04:02:03.758125+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:18.789052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"73'2","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:10.795479+0000","last_change":"2024-09-21T04:02:00.743220+0000","last_active":"2024-09-21T04:02:10.795479+0000","last_peered":"2024-09-21T04:02:10.795479+0000","last_clean":"2024-09-21T04:02:10.795479+0000","last_became_active":"2024-09-21T04:02:00.742636+0000","last_became_peered":"2024-09-21T04:02:00.742636+0000","last_unstale":"2024-09-21T04:02:10.795479+0000","last_undegraded":"2024-09-21T04:02:10.795479+0000","last_fullsized":"2024-09-21T04:02:10.795479+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:01.382338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.760136+0000","last_change":"2024-09-21T04:02:02.770744+0000","last_active":"2024-09-21T04:02:03.760136+0000","last_peered":"2024-09-21T04:02:03.760136+0000","last_clean":"2024-09-21T04:02:03.760136+0000","last_became_active":"2024-09-21T04:02:02.769894+0000","last_became_peered":"2024-09-21T04:02:02.769894+0000","last_unstale":"2024-09-21T04:02:03.760136+0000","last_undegraded":"2024-09-21T04:02:03.760136+0000","last_fullsized":"2024-09-21T04:02:03.760136+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:08.726517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"73'4","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:10.795275+0000","last_change":"2024-09-21T04:02:00.853284+0000","last_active":"2024-09-21T04:02:10.795275+0000","last_peered":"2024-09-21T04:02:10.795275+0000","last_clean":"2024-09-21T04:02:10.795275+0000","last_became_active":"2024-09-21T04:02:00.852855+0000","last_became_peered":"2024-09-21T04:02:00.852855+0000","last_unstale":"2024-09-21T04:02:10.795275+0000","last_undegraded":"2024-09-21T04:02:10.795275+0000","last_fullsized":"2024-09-21T04:02:10.795275+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:36.994214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.758427+0000","last_change":"2024-09-21T04:02:02.763577+0000","last_active":"2024-09-21T04:02:03.758427+0000","last_peered":"2024-09-21T04:02:03.758427+0000","last_clean":"2024-09-21T04:02:03.758427+0000","last_became_active":"2024-09-21T04:02:02.762970+0000","last_became_peered":"2024-09-21T04:02:02.762970+0000","last_unstale":"2024-09-21T04:02:03.758427+0000","last_undegraded":"2024-09-21T04:02:03.758427+0000","last_fullsized":"2024-09-21T04:02:03.758427+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:49.956226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"73'22","reported_seq":77,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:26.832417+0000","last_change":"2024-09-21T04:02:00.744203+0000","last_active":"2024-09-21T04:02:26.832417+0000","last_peered":"2024-09-21T04:02:26.832417+0000","last_clean":"2024-09-21T04:02:26.832417+0000","last_became_active":"2024-09-21T04:02:00.743659+0000","last_became_peered":"2024-09-21T04:02:00.743659+0000","last_unstale":"2024-09-21T04:02:26.832417+0000","last_undegraded":"2024-09-21T04:02:26.832417+0000","last_fullsized":"2024-09-21T04:02:26.832417+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:32.108302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50940,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":67,"num_write":23,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.753850+0000","last_change":"2024-09-21T04:02:02.766002+0000","last_active":"2024-09-21T04:02:03.753850+0000","last_peered":"2024-09-21T04:02:03.753850+0000","last_clean":"2024-09-21T04:02:03.753850+0000","last_became_active":"2024-09-21T04:02:02.765628+0000","last_became_peered":"2024-09-21T04:02:02.765628+0000","last_unstale":"2024-09-21T04:02:03.753850+0000","last_undegraded":"2024-09-21T04:02:03.753850+0000","last_fullsized":"2024-09-21T04:02:03.753850+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:22.251586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"73'6","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:11.802292+0000","last_change":"2024-09-21T04:02:00.741086+0000","last_active":"2024-09-21T04:02:11.802292+0000","last_peered":"2024-09-21T04:02:11.802292+0000","last_clean":"2024-09-21T04:02:11.802292+0000","last_became_active":"2024-09-21T04:02:00.740633+0000","last_became_peered":"2024-09-21T04:02:00.740633+0000","last_unstale":"2024-09-21T04:02:11.802292+0000","last_undegraded":"2024-09-21T04:02:11.802292+0000","last_fullsized":"2024-09-21T04:02:11.802292+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:20.392799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.747505+0000","last_change":"2024-09-21T04:02:02.759651+0000","last_active":"2024-09-21T04:02:03.747505+0000","last_peered":"2024-09-21T04:02:03.747505+0000","last_clean":"2024-09-21T04:02:03.747505+0000","last_became_active":"2024-09-21T04:02:02.759290+0000","last_became_peered":"2024-09-21T04:02:02.759290+0000","last_unstale":"2024-09-21T04:02:03.747505+0000","last_undegraded":"2024-09-21T04:02:03.747505+0000","last_fullsized":"2024-09-21T04:02:03.747505+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:32.025703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"73'3","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:10.797863+0000","last_change":"2024-09-21T04:02:00.744005+0000","last_active":"2024-09-21T04:02:10.797863+0000","last_peered":"2024-09-21T04:02:10.797863+0000","last_clean":"2024-09-21T04:02:10.797863+0000","last_became_active":"2024-09-21T04:02:00.743570+0000","last_became_peered":"2024-09-21T04:02:00.743570+0000","last_unstale":"2024-09-21T04:02:10.797863+0000","last_undegraded":"2024-09-21T04:02:10.797863+0000","last_fullsized":"2024-09-21T04:02:10.797863+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:32.158681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.759846+0000","last_change":"2024-09-21T04:02:02.757781+0000","last_active":"2024-09-21T04:02:03.759846+0000","last_peered":"2024-09-21T04:02:03.759846+0000","last_clean":"2024-09-21T04:02:03.759846+0000","last_became_active":"2024-09-21T04:02:02.757138+0000","last_became_peered":"2024-09-21T04:02:02.757138+0000","last_unstale":"2024-09-21T04:02:03.759846+0000","last_undegraded":"2024-09-21T04:02:03.759846+0000","last_fullsized":"2024-09-21T04:02:03.759846+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:17.770006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"73'3","reported_seq":37,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:23.828513+0000","last_change":"2024-09-21T04:02:00.741596+0000","last_active":"2024-09-21T04:02:23.828513+0000","last_peered":"2024-09-21T04:02:23.828513+0000","last_clean":"2024-09-21T04:02:23.828513+0000","last_became_active":"2024-09-21T04:02:00.741185+0000","last_became_peered":"2024-09-21T04:02:00.741185+0000","last_unstale":"2024-09-21T04:02:23.828513+0000","last_undegraded":"2024-09-21T04:02:23.828513+0000","last_fullsized":"2024-09-21T04:02:23.828513+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:31.663133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.757627+0000","last_change":"2024-09-21T04:02:02.773706+0000","last_active":"2024-09-21T04:02:03.757627+0000","last_peered":"2024-09-21T04:02:03.757627+0000","last_clean":"2024-09-21T04:02:03.757627+0000","last_became_active":"2024-09-21T04:02:02.773259+0000","last_became_peered":"2024-09-21T04:02:02.773259+0000","last_unstale":"2024-09-21T04:02:03.757627+0000","last_undegraded":"2024-09-21T04:02:03.757627+0000","last_fullsized":"2024-09-21T04:02:03.757627+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:50.431982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"73'15","reported_seq":35,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:11.804308+0000","last_change":"2024-09-21T04:02:00.851335+0000","last_active":"2024-09-21T04:02:11.804308+0000","last_peered":"2024-09-21T04:02:11.804308+0000","last_clean":"2024-09-21T04:02:11.804308+0000","last_became_active":"2024-09-21T04:02:00.850990+0000","last_became_peered":"2024-09-21T04:02:00.850990+0000","last_unstale":"2024-09-21T04:02:11.804308+0000","last_undegraded":"2024-09-21T04:02:11.804308+0000","last_fullsized":"2024-09-21T04:02:11.804308+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:45.731121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.756010+0000","last_change":"2024-09-21T04:02:02.770985+0000","last_active":"2024-09-21T04:02:03.756010+0000","last_peered":"2024-09-21T04:02:03.756010+0000","last_clean":"2024-09-21T04:02:03.756010+0000","last_became_active":"2024-09-21T04:02:02.770602+0000","last_became_peered":"2024-09-21T04:02:02.770602+0000","last_unstale":"2024-09-21T04:02:03.756010+0000","last_undegraded":"2024-09-21T04:02:03.756010+0000","last_fullsized":"2024-09-21T04:02:03.756010+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:37.131113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"73'2","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:12.809273+0000","last_change":"2024-09-21T04:02:00.740834+0000","last_active":"2024-09-21T04:02:12.809273+0000","last_peered":"2024-09-21T04:02:12.809273+0000","last_clean":"2024-09-21T04:02:12.809273+0000","last_became_active":"2024-09-21T04:02:00.740439+0000","last_became_peered":"2024-09-21T04:02:00.740439+0000","last_unstale":"2024-09-21T04:02:12.809273+0000","last_undegraded":"2024-09-21T04:02:12.809273+0000","last_fullsized":"2024-09-21T04:02:12.809273+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:04:22.159986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.755325+0000","last_change":"2024-09-21T04:02:02.773432+0000","last_active":"2024-09-21T04:02:03.755325+0000","last_peered":"2024-09-21T04:02:03.755325+0000","last_clean":"2024-09-21T04:02:03.755325+0000","last_became_active":"2024-09-21T04:02:02.773092+0000","last_became_peered":"2024-09-21T04:02:02.773092+0000","last_unstale":"2024-09-21T04:02:03.755325+0000","last_undegraded":"2024-09-21T04:02:03.755325+0000","last_fullsized":"2024-09-21T04:02:03.755325+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:36.440497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"73'2","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.755104+0000","last_change":"2024-09-21T04:02:00.738987+0000","last_active":"2024-09-21T04:02:03.755104+0000","last_peered":"2024-09-21T04:02:03.755104+0000","last_clean":"2024-09-21T04:02:03.755104+0000","last_became_active":"2024-09-21T04:02:00.738519+0000","last_became_peered":"2024-09-21T04:02:00.738519+0000","last_unstale":"2024-09-21T04:02:03.755104+0000","last_undegraded":"2024-09-21T04:02:03.755104+0000","last_fullsized":"2024-09-21T04:02:03.755104+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:58.079100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.760230+0000","last_change":"2024-09-21T04:02:02.766709+0000","last_active":"2024-09-21T04:02:03.760230+0000","last_peered":"2024-09-21T04:02:03.760230+0000","last_clean":"2024-09-21T04:02:03.760230+0000","last_became_active":"2024-09-21T04:02:02.766342+0000","last_became_peered":"2024-09-21T04:02:02.766342+0000","last_unstale":"2024-09-21T04:02:03.760230+0000","last_undegraded":"2024-09-21T04:02:03.760230+0000","last_fullsized":"2024-09-21T04:02:03.760230+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:26:35.185368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"73'1","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.762691+0000","last_change":"2024-09-21T04:02:00.745365+0000","last_active":"2024-09-21T04:02:03.762691+0000","last_peered":"2024-09-21T04:02:03.762691+0000","last_clean":"2024-09-21T04:02:03.762691+0000","last_became_active":"2024-09-21T04:02:00.744968+0000","last_became_peered":"2024-09-21T04:02:00.744968+0000","last_unstale":"2024-09-21T04:02:03.762691+0000","last_undegraded":"2024-09-21T04:02:03.762691+0000","last_fullsized":"2024-09-21T04:02:03.762691+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:56.078169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"73'1","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:06.043426+0000","last_change":"2024-09-21T04:02:02.774877+0000","last_active":"2024-09-21T04:02:06.043426+0000","last_peered":"2024-09-21T04:02:06.043426+0000","last_clean":"2024-09-21T04:02:06.043426+0000","last_became_active":"2024-09-21T04:02:02.774464+0000","last_became_peered":"2024-09-21T04:02:02.774464+0000","last_unstale":"2024-09-21T04:02:06.043426+0000","last_undegraded":"2024-09-21T04:02:06.043426+0000","last_fullsized":"2024-09-21T04:02:06.043426+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:23.393239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"73'3","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:11.803150+0000","last_change":"2024-09-21T04:02:01.257649+0000","last_active":"2024-09-21T04:02:11.803150+0000","last_peered":"2024-09-21T04:02:11.803150+0000","last_clean":"2024-09-21T04:02:11.803150+0000","last_became_active":"2024-09-21T04:02:01.257176+0000","last_became_peered":"2024-09-21T04:02:01.257176+0000","last_unstale":"2024-09-21T04:02:11.803150+0000","last_undegraded":"2024-09-21T04:02:11.803150+0000","last_fullsized":"2024-09-21T04:02:11.803150+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:37.257702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.759223+0000","last_change":"2024-09-21T04:02:02.768164+0000","last_active":"2024-09-21T04:02:03.759223+0000","last_peered":"2024-09-21T04:02:03.759223+0000","last_clean":"2024-09-21T04:02:03.759223+0000","last_became_active":"2024-09-21T04:02:02.766925+0000","last_became_peered":"2024-09-21T04:02:02.766925+0000","last_unstale":"2024-09-21T04:02:03.759223+0000","last_undegraded":"2024-09-21T04:02:03.759223+0000","last_fullsized":"2024-09-21T04:02:03.759223+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:59.707637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":91,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.757996+0000","last_change":"2024-09-21T04:00:48.936996+0000","last_active":"2024-09-21T04:02:03.757996+0000","last_peered":"2024-09-21T04:02:03.757996+0000","last_clean":"2024-09-21T04:02:03.757996+0000","last_became_active":"2024-09-21T04:00:48.931310+0000","last_became_peered":"2024-09-21T04:00:48.931310+0000","last_unstale":"2024-09-21T04:02:03.757996+0000","last_undegraded":"2024-09-21T04:02:03.757996+0000","last_fullsized":"2024-09-21T04:02:03.757996+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:57:25.346362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:57:25.346362+0000","last_clean_scrub_stamp":"2024-09-21T03:57:25.346362+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:24.165731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"73'3","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:12.807797+0000","last_change":"2024-09-21T04:02:00.740118+0000","last_active":"2024-09-21T04:02:12.807797+0000","last_peered":"2024-09-21T04:02:12.807797+0000","last_clean":"2024-09-21T04:02:12.807797+0000","last_became_active":"2024-09-21T04:02:00.739677+0000","last_became_peered":"2024-09-21T04:02:00.739677+0000","last_unstale":"2024-09-21T04:02:12.807797+0000","last_undegraded":"2024-09-21T04:02:12.807797+0000","last_fullsized":"2024-09-21T04:02:12.807797+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:46.084118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.755214+0000","last_change":"2024-09-21T04:02:02.771537+0000","last_active":"2024-09-21T04:02:03.755214+0000","last_peered":"2024-09-21T04:02:03.755214+0000","last_clean":"2024-09-21T04:02:03.755214+0000","last_became_active":"2024-09-21T04:02:02.771151+0000","last_became_peered":"2024-09-21T04:02:02.771151+0000","last_unstale":"2024-09-21T04:02:03.755214+0000","last_undegraded":"2024-09-21T04:02:03.755214+0000","last_fullsized":"2024-09-21T04:02:03.755214+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:21.283515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"73'3","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:11.803482+0000","last_change":"2024-09-21T04:02:00.744195+0000","last_active":"2024-09-21T04:02:11.803482+0000","last_peered":"2024-09-21T04:02:11.803482+0000","last_clean":"2024-09-21T04:02:11.803482+0000","last_became_active":"2024-09-21T04:02:00.743779+0000","last_became_peered":"2024-09-21T04:02:00.743779+0000","last_unstale":"2024-09-21T04:02:11.803482+0000","last_undegraded":"2024-09-21T04:02:11.803482+0000","last_fullsized":"2024-09-21T04:02:11.803482+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:31.281347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.747405+0000","last_change":"2024-09-21T04:02:02.755223+0000","last_active":"2024-09-21T04:02:03.747405+0000","last_peered":"2024-09-21T04:02:03.747405+0000","last_clean":"2024-09-21T04:02:03.747405+0000","last_became_active":"2024-09-21T04:02:02.754836+0000","last_became_peered":"2024-09-21T04:02:02.754836+0000","last_unstale":"2024-09-21T04:02:03.747405+0000","last_undegraded":"2024-09-21T04:02:03.747405+0000","last_fullsized":"2024-09-21T04:02:03.747405+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:57.912690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"73'4","reported_seq":37,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:24.829183+0000","last_change":"2024-09-21T04:02:00.851689+0000","last_active":"2024-09-21T04:02:24.829183+0000","last_peered":"2024-09-21T04:02:24.829183+0000","last_clean":"2024-09-21T04:02:24.829183+0000","last_became_active":"2024-09-21T04:02:00.851241+0000","last_became_peered":"2024-09-21T04:02:00.851241+0000","last_unstale":"2024-09-21T04:02:24.829183+0000","last_undegraded":"2024-09-21T04:02:24.829183+0000","last_fullsized":"2024-09-21T04:02:24.829183+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:14.697695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.755846+0000","last_change":"2024-09-21T04:02:02.774750+0000","last_active":"2024-09-21T04:02:03.755846+0000","last_peered":"2024-09-21T04:02:03.755846+0000","last_clean":"2024-09-21T04:02:03.755846+0000","last_became_active":"2024-09-21T04:02:02.774310+0000","last_became_peered":"2024-09-21T04:02:02.774310+0000","last_unstale":"2024-09-21T04:02:03.755846+0000","last_undegraded":"2024-09-21T04:02:03.755846+0000","last_fullsized":"2024-09-21T04:02:03.755846+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:07.909035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"73'1","reported_seq":20,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.758146+0000","last_change":"2024-09-21T04:02:00.741461+0000","last_active":"2024-09-21T04:02:03.758146+0000","last_peered":"2024-09-21T04:02:03.758146+0000","last_clean":"2024-09-21T04:02:03.758146+0000","last_became_active":"2024-09-21T04:02:00.741051+0000","last_became_peered":"2024-09-21T04:02:00.741051+0000","last_unstale":"2024-09-21T04:02:03.758146+0000","last_undegraded":"2024-09-21T04:02:03.758146+0000","last_fullsized":"2024-09-21T04:02:03.758146+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:05.944430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.753784+0000","last_change":"2024-09-21T04:02:02.761703+0000","last_active":"2024-09-21T04:02:03.753784+0000","last_peered":"2024-09-21T04:02:03.753784+0000","last_clean":"2024-09-21T04:02:03.753784+0000","last_became_active":"2024-09-21T04:02:02.761296+0000","last_became_peered":"2024-09-21T04:02:02.761296+0000","last_unstale":"2024-09-21T04:02:03.753784+0000","last_undegraded":"2024-09-21T04:02:03.753784+0000","last_fullsized":"2024-09-21T04:02:03.753784+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:57.337167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.753977+0000","last_change":"2024-09-21T04:02:00.853584+0000","last_active":"2024-09-21T04:02:03.753977+0000","last_peered":"2024-09-21T04:02:03.753977+0000","last_clean":"2024-09-21T04:02:03.753977+0000","last_became_active":"2024-09-21T04:02:00.853019+0000","last_became_peered":"2024-09-21T04:02:00.853019+0000","last_unstale":"2024-09-21T04:02:03.753977+0000","last_undegraded":"2024-09-21T04:02:03.753977+0000","last_fullsized":"2024-09-21T04:02:03.753977+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:56.186791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"73'1","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.754186+0000","last_change":"2024-09-21T04:02:02.758175+0000","last_active":"2024-09-21T04:02:03.754186+0000","last_peered":"2024-09-21T04:02:03.754186+0000","last_clean":"2024-09-21T04:02:03.754186+0000","last_became_active":"2024-09-21T04:02:02.757715+0000","last_became_peered":"2024-09-21T04:02:02.757715+0000","last_unstale":"2024-09-21T04:02:03.754186+0000","last_undegraded":"2024-09-21T04:02:03.754186+0000","last_fullsized":"2024-09-21T04:02:03.754186+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:40.172569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"73'1","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:10.795895+0000","last_change":"2024-09-21T04:02:00.853325+0000","last_active":"2024-09-21T04:02:10.795895+0000","last_peered":"2024-09-21T04:02:10.795895+0000","last_clean":"2024-09-21T04:02:10.795895+0000","last_became_active":"2024-09-21T04:02:00.852834+0000","last_became_peered":"2024-09-21T04:02:00.852834+0000","last_unstale":"2024-09-21T04:02:10.795895+0000","last_undegraded":"2024-09-21T04:02:10.795895+0000","last_fullsized":"2024-09-21T04:02:10.795895+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:16.090493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.747832+0000","last_change":"2024-09-21T04:02:02.752402+0000","last_active":"2024-09-21T04:02:03.747832+0000","last_peered":"2024-09-21T04:02:03.747832+0000","last_clean":"2024-09-21T04:02:03.747832+0000","last_became_active":"2024-09-21T04:02:02.751994+0000","last_became_peered":"2024-09-21T04:02:02.751994+0000","last_unstale":"2024-09-21T04:02:03.747832+0000","last_undegraded":"2024-09-21T04:02:03.747832+0000","last_fullsized":"2024-09-21T04:02:03.747832+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:19.525317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"73'9","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:23.753810+0000","last_change":"2024-09-21T04:02:00.739112+0000","last_active":"2024-09-21T04:02:23.753810+0000","last_peered":"2024-09-21T04:02:23.753810+0000","last_clean":"2024-09-21T04:02:23.753810+0000","last_became_active":"2024-09-21T04:02:00.738613+0000","last_became_peered":"2024-09-21T04:02:00.738613+0000","last_unstale":"2024-09-21T04:02:23.753810+0000","last_undegraded":"2024-09-21T04:02:23.753810+0000","last_fullsized":"2024-09-21T04:02:23.753810+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:34.334576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.755229+0000","last_change":"2024-09-21T04:02:02.763345+0000","last_active":"2024-09-21T04:02:03.755229+0000","last_peered":"2024-09-21T04:02:03.755229+0000","last_clean":"2024-09-21T04:02:03.755229+0000","last_became_active":"2024-09-21T04:02:02.762987+0000","last_became_peered":"2024-09-21T04:02:02.762987+0000","last_unstale":"2024-09-21T04:02:03.755229+0000","last_undegraded":"2024-09-21T04:02:03.755229+0000","last_fullsized":"2024-09-21T04:02:03.755229+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:36.239267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"73'4","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:10.794791+0000","last_change":"2024-09-21T04:02:01.258208+0000","last_active":"2024-09-21T04:02:10.794791+0000","last_peered":"2024-09-21T04:02:10.794791+0000","last_clean":"2024-09-21T04:02:10.794791+0000","last_became_active":"2024-09-21T04:02:01.257780+0000","last_became_peered":"2024-09-21T04:02:01.257780+0000","last_unstale":"2024-09-21T04:02:10.794791+0000","last_undegraded":"2024-09-21T04:02:10.794791+0000","last_fullsized":"2024-09-21T04:02:10.794791+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:20.161792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.759536+0000","last_change":"2024-09-21T04:02:02.761149+0000","last_active":"2024-09-21T04:02:03.759536+0000","last_peered":"2024-09-21T04:02:03.759536+0000","last_clean":"2024-09-21T04:02:03.759536+0000","last_became_active":"2024-09-21T04:02:02.760771+0000","last_became_peered":"2024-09-21T04:02:02.760771+0000","last_unstale":"2024-09-21T04:02:03.759536+0000","last_undegraded":"2024-09-21T04:02:03.759536+0000","last_fullsized":"2024-09-21T04:02:03.759536+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:20.645377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"73'2","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:10.796190+0000","last_change":"2024-09-21T04:02:00.853580+0000","last_active":"2024-09-21T04:02:10.796190+0000","last_peered":"2024-09-21T04:02:10.796190+0000","last_clean":"2024-09-21T04:02:10.796190+0000","last_became_active":"2024-09-21T04:02:00.853091+0000","last_became_peered":"2024-09-21T04:02:00.853091+0000","last_unstale":"2024-09-21T04:02:10.796190+0000","last_undegraded":"2024-09-21T04:02:10.796190+0000","last_fullsized":"2024-09-21T04:02:10.796190+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:32.893695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.753191+0000","last_change":"2024-09-21T04:02:02.769686+0000","last_active":"2024-09-21T04:02:03.753191+0000","last_peered":"2024-09-21T04:02:03.753191+0000","last_clean":"2024-09-21T04:02:03.753191+0000","last_became_active":"2024-09-21T04:02:02.769293+0000","last_became_peered":"2024-09-21T04:02:02.769293+0000","last_unstale":"2024-09-21T04:02:03.753191+0000","last_undegraded":"2024-09-21T04:02:03.753191+0000","last_fullsized":"2024-09-21T04:02:03.753191+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:10.984624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"73'3","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:10.796422+0000","last_change":"2024-09-21T04:02:00.741556+0000","last_active":"2024-09-21T04:02:10.796422+0000","last_peered":"2024-09-21T04:02:10.796422+0000","last_clean":"2024-09-21T04:02:10.796422+0000","last_became_active":"2024-09-21T04:02:00.741188+0000","last_became_peered":"2024-09-21T04:02:00.741188+0000","last_unstale":"2024-09-21T04:02:10.796422+0000","last_undegraded":"2024-09-21T04:02:10.796422+0000","last_fullsized":"2024-09-21T04:02:10.796422+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:23.566814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.754188+0000","last_change":"2024-09-21T04:02:02.769163+0000","last_active":"2024-09-21T04:02:03.754188+0000","last_peered":"2024-09-21T04:02:03.754188+0000","last_clean":"2024-09-21T04:02:03.754188+0000","last_became_active":"2024-09-21T04:02:02.768804+0000","last_became_peered":"2024-09-21T04:02:02.768804+0000","last_unstale":"2024-09-21T04:02:03.754188+0000","last_undegraded":"2024-09-21T04:02:03.754188+0000","last_fullsized":"2024-09-21T04:02:03.754188+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:12.576553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"73'1","reported_seq":20,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.746224+0000","last_change":"2024-09-21T04:02:00.743091+0000","last_active":"2024-09-21T04:02:03.746224+0000","last_peered":"2024-09-21T04:02:03.746224+0000","last_clean":"2024-09-21T04:02:03.746224+0000","last_became_active":"2024-09-21T04:02:00.742665+0000","last_became_peered":"2024-09-21T04:02:00.742665+0000","last_unstale":"2024-09-21T04:02:03.746224+0000","last_undegraded":"2024-09-21T04:02:03.746224+0000","last_fullsized":"2024-09-21T04:02:03.746224+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:09.698421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.754310+0000","last_change":"2024-09-21T04:02:02.761985+0000","last_active":"2024-09-21T04:02:03.754310+0000","last_peered":"2024-09-21T04:02:03.754310+0000","last_clean":"2024-09-21T04:02:03.754310+0000","last_became_active":"2024-09-21T04:02:02.761624+0000","last_became_peered":"2024-09-21T04:02:02.761624+0000","last_unstale":"2024-09-21T04:02:03.754310+0000","last_undegraded":"2024-09-21T04:02:03.754310+0000","last_fullsized":"2024-09-21T04:02:03.754310+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:12.912221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"73'4","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:10.796449+0000","last_change":"2024-09-21T04:02:00.743219+0000","last_active":"2024-09-21T04:02:10.796449+0000","last_peered":"2024-09-21T04:02:10.796449+0000","last_clean":"2024-09-21T04:02:10.796449+0000","last_became_active":"2024-09-21T04:02:00.742740+0000","last_became_peered":"2024-09-21T04:02:00.742740+0000","last_unstale":"2024-09-21T04:02:10.796449+0000","last_undegraded":"2024-09-21T04:02:10.796449+0000","last_fullsized":"2024-09-21T04:02:10.796449+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:54.012137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.757837+0000","last_change":"2024-09-21T04:02:02.761357+0000","last_active":"2024-09-21T04:02:03.757837+0000","last_peered":"2024-09-21T04:02:03.757837+0000","last_clean":"2024-09-21T04:02:03.757837+0000","last_became_active":"2024-09-21T04:02:02.760984+0000","last_became_peered":"2024-09-21T04:02:02.760984+0000","last_unstale":"2024-09-21T04:02:03.757837+0000","last_undegraded":"2024-09-21T04:02:03.757837+0000","last_fullsized":"2024-09-21T04:02:03.757837+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:55.388768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.759536+0000","last_change":"2024-09-21T04:02:02.772272+0000","last_active":"2024-09-21T04:02:03.759536+0000","last_peered":"2024-09-21T04:02:03.759536+0000","last_clean":"2024-09-21T04:02:03.759536+0000","last_became_active":"2024-09-21T04:02:02.771824+0000","last_became_peered":"2024-09-21T04:02:02.771824+0000","last_unstale":"2024-09-21T04:02:03.759536+0000","last_undegraded":"2024-09-21T04:02:03.759536+0000","last_fullsized":"2024-09-21T04:02:03.759536+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:28:56.292477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"73'2","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.760103+0000","last_change":"2024-09-21T04:02:00.742230+0000","last_active":"2024-09-21T04:02:03.760103+0000","last_peered":"2024-09-21T04:02:03.760103+0000","last_clean":"2024-09-21T04:02:03.760103+0000","last_became_active":"2024-09-21T04:02:00.741838+0000","last_became_peered":"2024-09-21T04:02:00.741838+0000","last_unstale":"2024-09-21T04:02:03.760103+0000","last_undegraded":"2024-09-21T04:02:03.760103+0000","last_fullsized":"2024-09-21T04:02:03.760103+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:18.909859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.758167+0000","last_change":"2024-09-21T04:02:02.762336+0000","last_active":"2024-09-21T04:02:03.758167+0000","last_peered":"2024-09-21T04:02:03.758167+0000","last_clean":"2024-09-21T04:02:03.758167+0000","last_became_active":"2024-09-21T04:02:02.761753+0000","last_became_peered":"2024-09-21T04:02:02.761753+0000","last_unstale":"2024-09-21T04:02:03.758167+0000","last_undegraded":"2024-09-21T04:02:03.758167+0000","last_fullsized":"2024-09-21T04:02:03.758167+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:39.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"73'4","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:11.802902+0000","last_change":"2024-09-21T04:02:00.743495+0000","last_active":"2024-09-21T04:02:11.802902+0000","last_peered":"2024-09-21T04:02:11.802902+0000","last_clean":"2024-09-21T04:02:11.802902+0000","last_became_active":"2024-09-21T04:02:00.743135+0000","last_became_peered":"2024-09-21T04:02:00.743135+0000","last_unstale":"2024-09-21T04:02:11.802902+0000","last_undegraded":"2024-09-21T04:02:11.802902+0000","last_fullsized":"2024-09-21T04:02:11.802902+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:16.481148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.760048+0000","last_change":"2024-09-21T04:02:02.766682+0000","last_active":"2024-09-21T04:02:03.760048+0000","last_peered":"2024-09-21T04:02:03.760048+0000","last_clean":"2024-09-21T04:02:03.760048+0000","last_became_active":"2024-09-21T04:02:02.766299+0000","last_became_peered":"2024-09-21T04:02:02.766299+0000","last_unstale":"2024-09-21T04:02:03.760048+0000","last_undegraded":"2024-09-21T04:02:03.760048+0000","last_fullsized":"2024-09-21T04:02:03.760048+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:25.635908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"73'2","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.762819+0000","last_change":"2024-09-21T04:02:00.852281+0000","last_active":"2024-09-21T04:02:03.762819+0000","last_peered":"2024-09-21T04:02:03.762819+0000","last_clean":"2024-09-21T04:02:03.762819+0000","last_became_active":"2024-09-21T04:02:00.851900+0000","last_became_peered":"2024-09-21T04:02:00.851900+0000","last_unstale":"2024-09-21T04:02:03.762819+0000","last_undegraded":"2024-09-21T04:02:03.762819+0000","last_fullsized":"2024-09-21T04:02:03.762819+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:37.115541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:03.755481+0000","last_change":"2024-09-21T04:02:02.770467+0000","last_active":"2024-09-21T04:02:03.755481+0000","last_peered":"2024-09-21T04:02:03.755481+0000","last_clean":"2024-09-21T04:02:03.755481+0000","last_became_active":"2024-09-21T04:02:02.770099+0000","last_became_peered":"2024-09-21T04:02:02.770099+0000","last_unstale":"2024-09-21T04:02:03.755481+0000","last_undegraded":"2024-09-21T04:02:03.755481+0000","last_fullsized":"2024-09-21T04:02:03.755481+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:38.806298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"73'9","reported_seq":29,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T04:02:11.803233+0000","last_change":"2024-09-21T04:02:00.742641+0000","last_active":"2024-09-21T04:02:11.803233+0000","last_peered":"2024-09-21T04:02:11.803233+0000","last_clean":"2024-09-21T04:02:11.803233+0000","last_became_active":"2024-09-21T04:02:00.742380+0000","last_became_peered":"2024-09-21T04:02:00.742380+0000","last_unstale":"2024-09-21T04:02:11.803233+0000","last_undegraded":"2024-09-21T04:02:11.803233+0000","last_fullsized":"2024-09-21T04:02:11.803233+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:27.354314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54550,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":110,"num_read_kb":114,"num_write":151,"num_write_kb":97,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":163650,"data_compressed":5676,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282548,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808850,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31956,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713196,"statfs":{"total":95995035648,"available":95962312704,"internally_reserved":0,"allocated":790528,"data_stored":388501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":62,"seq":266287972374,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":1030922,"data_compressed":37818,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135900,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":806912,"data_stored":388737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332129,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":806912,"data_stored":388657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495653,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":856064,"data_stored":980196,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659179,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31960,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713192,"statfs":{"total":95995035648,"available":95962308608,"internally_reserved":0,"allocated":794624,"data_stored":388126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855408,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":979880,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018933,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717348,"statfs":{"total":95995035648,"available":95966564352,"internally_reserved":0,"allocated":798720,"data_stored":388563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":811008,"data_stored":388437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378687,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":868352,"data_stored":440209,"data_compressed":1892,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542212,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":843776,"data_stored":438859,"data_compressed":1892,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705737,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":815104,"data_stored":389397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34306,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34306,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34306,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":50940,"data_compressed":1892,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":52290,"data_compressed":1892,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":52518,"data_compressed":1892,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T04:02:30.812 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T04:02:30.813 INFO:tasks.cephadm:Setting up client nodes... 2024-09-21T04:02:30.813 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-21T04:02:31.076 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:31.197 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:30 smithi064 ceph-mon[31180]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T04:02:31.197 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:30 smithi064 ceph-mon[31180]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:02:31.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:30 smithi119 ceph-mon[34811]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T04:02:31.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:30 smithi119 ceph-mon[34811]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:02:31.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:30 smithi131 ceph-mon[35193]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T04:02:31.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:30 smithi131 ceph-mon[35193]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:02:31.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.564+0000 7f187094a640 1 -- 172.21.15.64:0/3815429496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f186c102fc0 msgr2=0x7f186c10fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:31.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.564+0000 7f187094a640 1 --2- 172.21.15.64:0/3815429496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f186c102fc0 0x7f186c10fce0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1860009900 tx=0x7f186002a480 comp rx=0 tx=0).stop 2024-09-21T04:02:31.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.564+0000 7f187094a640 1 -- 172.21.15.64:0/3815429496 shutdown_connections 2024-09-21T04:02:31.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.564+0000 7f187094a640 1 --2- 172.21.15.64:0/3815429496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f186c110220 0x7f186c112640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:31.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.564+0000 7f187094a640 1 --2- 172.21.15.64:0/3815429496 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f186c102fc0 0x7f186c10fce0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:31.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.564+0000 7f187094a640 1 --2- 172.21.15.64:0/3815429496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f186c102670 0x7f186c102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:31.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.564+0000 7f187094a640 1 -- 172.21.15.64:0/3815429496 >> 172.21.15.64:0/3815429496 conn(0x7f186c079760 msgr2=0x7f186c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:31.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.564+0000 7f187094a640 1 -- 172.21.15.64:0/3815429496 shutdown_connections 2024-09-21T04:02:31.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.564+0000 7f187094a640 1 -- 172.21.15.64:0/3815429496 wait complete. 2024-09-21T04:02:31.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.565+0000 7f187094a640 1 Processor -- start 2024-09-21T04:02:31.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.565+0000 7f187094a640 1 -- start start 2024-09-21T04:02:31.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.566+0000 7f187094a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f186c102670 0x7f186c19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:31.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.566+0000 7f187094a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f186c102fc0 0x7f186c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:31.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.566+0000 7f187094a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f186c110220 0x7f186c1a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:31.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.566+0000 7f187094a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f186c115c90 con 0x7f186c102670 2024-09-21T04:02:31.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.566+0000 7f187094a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f186c115b10 con 0x7f186c110220 2024-09-21T04:02:31.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.566+0000 7f187094a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f186c115e10 con 0x7f186c102fc0 2024-09-21T04:02:31.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.566+0000 7f186a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f186c102670 0x7f186c19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:31.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.566+0000 7f186a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f186c102670 0x7f186c19c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36528/0 (socket says 172.21.15.64:36528) 2024-09-21T04:02:31.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.566+0000 7f186a59c640 1 -- 172.21.15.64:0/2943549995 learned_addr learned my addr 172.21.15.64:0/2943549995 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:31.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.566+0000 7f1869d9b640 1 --2- 172.21.15.64:0/2943549995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f186c102fc0 0x7f186c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.566+0000 7f186ad9d640 1 --2- 172.21.15.64:0/2943549995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f186c110220 0x7f186c1a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.566+0000 7f186a59c640 1 -- 172.21.15.64:0/2943549995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f186c102fc0 msgr2=0x7f186c19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.566+0000 7f186a59c640 1 --2- 172.21.15.64:0/2943549995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f186c102fc0 0x7f186c19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.566+0000 7f186a59c640 1 -- 172.21.15.64:0/2943549995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f186c110220 msgr2=0x7f186c1a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.566+0000 7f186a59c640 1 --2- 172.21.15.64:0/2943549995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f186c110220 0x7f186c1a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.566+0000 7f186a59c640 1 -- 172.21.15.64:0/2943549995 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186c1a2570 con 0x7f186c102670 2024-09-21T04:02:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.566+0000 7f186a59c640 1 --2- 172.21.15.64:0/2943549995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f186c102670 0x7f186c19c6c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f185c00ea80 tx=0x7f185c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.567+0000 7f185b7fe640 1 -- 172.21.15.64:0/2943549995 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c00ce40 con 0x7f186c102670 2024-09-21T04:02:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.567+0000 7f187094a640 1 -- 172.21.15.64:0/2943549995 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f186c1a2860 con 0x7f186c102670 2024-09-21T04:02:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.567+0000 7f185b7fe640 1 -- 172.21.15.64:0/2943549995 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f185c005020 con 0x7f186c102670 2024-09-21T04:02:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.567+0000 7f185b7fe640 1 -- 172.21.15.64:0/2943549995 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c0116d0 con 0x7f186c102670 2024-09-21T04:02:31.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.567+0000 7f187094a640 1 -- 172.21.15.64:0/2943549995 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f186c1a2d20 con 0x7f186c102670 2024-09-21T04:02:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.568+0000 7f187094a640 1 -- 172.21.15.64:0/2943549995 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f186c102a80 con 0x7f186c102670 2024-09-21T04:02:31.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.569+0000 7f185b7fe640 1 -- 172.21.15.64:0/2943549995 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f185c0054d0 con 0x7f186c102670 2024-09-21T04:02:31.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.570+0000 7f185b7fe640 1 --2- 172.21.15.64:0/2943549995 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f18540b6620 0x7f18540b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:31.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.570+0000 7f1869d9b640 1 --2- 172.21.15.64:0/2943549995 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f18540b6620 0x7f18540b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:31.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.570+0000 7f185b7fe640 1 -- 172.21.15.64:0/2943549995 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f185c0e7c50 con 0x7f186c102670 2024-09-21T04:02:31.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.571+0000 7f1869d9b640 1 --2- 172.21.15.64:0/2943549995 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f18540b6620 0x7f18540b8b30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1860031440 tx=0x7f1860009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:31.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.573+0000 7f185b7fe640 1 -- 172.21.15.64:0/2943549995 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f185c016030 con 0x7f186c102670 2024-09-21T04:02:31.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.832+0000 7f187094a640 1 -- 172.21.15.64:0/2943549995 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f186c1a9280 con 0x7f186c102670 2024-09-21T04:02:31.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.835+0000 7f185b7fe640 1 -- 172.21.15.64:0/2943549995 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f185c0ae280 con 0x7f186c102670 2024-09-21T04:02:31.838 INFO:teuthology.orchestra.run.smithi064.stdout:[client.0] 2024-09-21T04:02:31.838 INFO:teuthology.orchestra.run.smithi064.stdout: key = AQBXRe5m4524MRAA3odQYT9nch2U0Sy8/w+Mtw== 2024-09-21T04:02:31.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.837+0000 7f187094a640 1 -- 172.21.15.64:0/2943549995 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f18540b6620 msgr2=0x7f18540b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:31.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.837+0000 7f187094a640 1 --2- 172.21.15.64:0/2943549995 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f18540b6620 0x7f18540b8b30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1860031440 tx=0x7f1860009560 comp rx=0 tx=0).stop 2024-09-21T04:02:31.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.837+0000 7f187094a640 1 -- 172.21.15.64:0/2943549995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f186c102670 msgr2=0x7f186c19c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:31.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.837+0000 7f187094a640 1 --2- 172.21.15.64:0/2943549995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f186c102670 0x7f186c19c6c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f185c00ea80 tx=0x7f185c00ef40 comp rx=0 tx=0).stop 2024-09-21T04:02:31.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.838+0000 7f187094a640 1 -- 172.21.15.64:0/2943549995 shutdown_connections 2024-09-21T04:02:31.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.838+0000 7f187094a640 1 --2- 172.21.15.64:0/2943549995 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f18540b6620 0x7f18540b8b30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:31.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.838+0000 7f187094a640 1 --2- 172.21.15.64:0/2943549995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f186c110220 0x7f186c1a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:31.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.838+0000 7f187094a640 1 --2- 172.21.15.64:0/2943549995 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f186c102fc0 0x7f186c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:31.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.838+0000 7f187094a640 1 --2- 172.21.15.64:0/2943549995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f186c102670 0x7f186c19c6c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:31.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.838+0000 7f187094a640 1 -- 172.21.15.64:0/2943549995 >> 172.21.15.64:0/2943549995 conn(0x7f186c079760 msgr2=0x7f186c109e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:31.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.838+0000 7f187094a640 1 -- 172.21.15.64:0/2943549995 shutdown_connections 2024-09-21T04:02:31.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:31.838+0000 7f187094a640 1 -- 172.21.15.64:0/2943549995 wait complete. 2024-09-21T04:02:32.210 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:31 smithi064 ceph-mon[31180]: pgmap v274: 97 pgs: 97 active+clean; 630 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-09-21T04:02:32.210 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:31 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2943549995' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T04:02:32.211 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:31 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2943549995' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T04:02:32.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:31 smithi119 ceph-mon[34811]: pgmap v274: 97 pgs: 97 active+clean; 630 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-09-21T04:02:32.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:31 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2943549995' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T04:02:32.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:31 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2943549995' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T04:02:32.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:31 smithi131 ceph-mon[35193]: pgmap v274: 97 pgs: 97 active+clean; 630 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-09-21T04:02:32.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:31 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2943549995' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T04:02:32.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:31 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2943549995' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T04:02:32.621 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-21T04:02:32.621 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-21T04:02:32.621 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T04:02:32.663 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph config log 1 --format=json 2024-09-21T04:02:32.963 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:33.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.423+0000 7f9507776640 1 -- 172.21.15.64:0/1480759827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9500105300 msgr2=0x7f950010d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:33.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.423+0000 7f9507776640 1 --2- 172.21.15.64:0/1480759827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9500105300 0x7f950010d850 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f94f0009900 tx=0x7f94f002a410 comp rx=0 tx=0).stop 2024-09-21T04:02:33.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.423+0000 7f9507776640 1 -- 172.21.15.64:0/1480759827 shutdown_connections 2024-09-21T04:02:33.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.423+0000 7f9507776640 1 --2- 172.21.15.64:0/1480759827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f950010de10 0x7f9500110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:33.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.423+0000 7f9507776640 1 --2- 172.21.15.64:0/1480759827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9500105300 0x7f950010d850 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:33.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.423+0000 7f9507776640 1 --2- 172.21.15.64:0/1480759827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95001049b0 0x7f9500104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:33.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.423+0000 7f9507776640 1 -- 172.21.15.64:0/1480759827 >> 172.21.15.64:0/1480759827 conn(0x7f9500079730 msgr2=0x7f9500079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:33.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.424+0000 7f9507776640 1 -- 172.21.15.64:0/1480759827 shutdown_connections 2024-09-21T04:02:33.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.424+0000 7f9507776640 1 -- 172.21.15.64:0/1480759827 wait complete. 2024-09-21T04:02:33.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.424+0000 7f9507776640 1 Processor -- start 2024-09-21T04:02:33.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.424+0000 7f9507776640 1 -- start start 2024-09-21T04:02:33.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.425+0000 7f9507776640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95001049b0 0x7f950019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:33.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.425+0000 7f9507776640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9500105300 0x7f950019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:33.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.425+0000 7f9507776640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f950010de10 0x7f950019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:33.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.425+0000 7f9507776640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9500113a70 con 0x7f9500105300 2024-09-21T04:02:33.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.425+0000 7f9507776640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f95001138f0 con 0x7f95001049b0 2024-09-21T04:02:33.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.425+0000 7f9507776640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9500113bf0 con 0x7f950010de10 2024-09-21T04:02:33.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.425+0000 7f9504d11640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9500105300 0x7f950019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:33.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.425+0000 7f9504d11640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9500105300 0x7f950019aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36550/0 (socket says 172.21.15.64:36550) 2024-09-21T04:02:33.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.425+0000 7f9504d11640 1 -- 172.21.15.64:0/1374743291 learned_addr learned my addr 172.21.15.64:0/1374743291 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:33.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.425+0000 7f9505512640 1 --2- 172.21.15.64:0/1374743291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95001049b0 0x7f950019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:33.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.425+0000 7f9505d13640 1 --2- 172.21.15.64:0/1374743291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f950010de10 0x7f950019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:33.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.425+0000 7f9504d11640 1 -- 172.21.15.64:0/1374743291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f950010de10 msgr2=0x7f950019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:33.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.425+0000 7f9504d11640 1 --2- 172.21.15.64:0/1374743291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f950010de10 0x7f950019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:33.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.425+0000 7f9504d11640 1 -- 172.21.15.64:0/1374743291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95001049b0 msgr2=0x7f950019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:33.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.425+0000 7f9504d11640 1 --2- 172.21.15.64:0/1374743291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95001049b0 0x7f950019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:33.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.425+0000 7f9504d11640 1 -- 172.21.15.64:0/1374743291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95001a0410 con 0x7f9500105300 2024-09-21T04:02:33.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.425+0000 7f9504d11640 1 --2- 172.21.15.64:0/1374743291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9500105300 0x7f950019aa40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f94f002a930 tx=0x7f94f0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:33.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.425+0000 7f94f67fc640 1 -- 172.21.15.64:0/1374743291 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f003f030 con 0x7f9500105300 2024-09-21T04:02:33.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.426+0000 7f9507776640 1 -- 172.21.15.64:0/1374743291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95001a06a0 con 0x7f9500105300 2024-09-21T04:02:33.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.426+0000 7f94f67fc640 1 -- 172.21.15.64:0/1374743291 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94f0003aa0 con 0x7f9500105300 2024-09-21T04:02:33.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.426+0000 7f94f67fc640 1 -- 172.21.15.64:0/1374743291 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f0035070 con 0x7f9500105300 2024-09-21T04:02:33.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.426+0000 7f9507776640 1 -- 172.21.15.64:0/1374743291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95001a6e60 con 0x7f9500105300 2024-09-21T04:02:33.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.427+0000 7f9507776640 1 -- 172.21.15.64:0/1374743291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f950006b560 con 0x7f9500105300 2024-09-21T04:02:33.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.428+0000 7f94f67fc640 1 -- 172.21.15.64:0/1374743291 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f94f0042030 con 0x7f9500105300 2024-09-21T04:02:33.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.429+0000 7f94f67fc640 1 --2- 172.21.15.64:0/1374743291 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f94ec0b65f0 0x7f94ec0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:33.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.429+0000 7f9505512640 1 --2- 172.21.15.64:0/1374743291 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f94ec0b65f0 0x7f94ec0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:33.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.429+0000 7f94f67fc640 1 -- 172.21.15.64:0/1374743291 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f94f0107fe0 con 0x7f9500105300 2024-09-21T04:02:33.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.429+0000 7f9505512640 1 --2- 172.21.15.64:0/1374743291 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f94ec0b65f0 0x7f94ec0b8b00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f94fc005fd0 tx=0x7f94fc00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:33.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.432+0000 7f94f67fc640 1 -- 172.21.15.64:0/1374743291 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f94f00d29f0 con 0x7f9500105300 2024-09-21T04:02:33.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.622+0000 7f9507776640 1 -- 172.21.15.64:0/1374743291 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f9500108f10 con 0x7f9500105300 2024-09-21T04:02:33.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.622+0000 7f94f67fc640 1 -- 172.21.15.64:0/1374743291 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v55) ==== 86+0+133 (secure 0 0 0) 0x7f94f00d7930 con 0x7f9500105300 2024-09-21T04:02:33.625 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:02:33.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.625+0000 7f9507776640 1 -- 172.21.15.64:0/1374743291 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f94ec0b65f0 msgr2=0x7f94ec0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:33.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.625+0000 7f9507776640 1 --2- 172.21.15.64:0/1374743291 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f94ec0b65f0 0x7f94ec0b8b00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f94fc005fd0 tx=0x7f94fc00b390 comp rx=0 tx=0).stop 2024-09-21T04:02:33.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.625+0000 7f9507776640 1 -- 172.21.15.64:0/1374743291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9500105300 msgr2=0x7f950019aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:33.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.625+0000 7f9507776640 1 --2- 172.21.15.64:0/1374743291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9500105300 0x7f950019aa40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f94f002a930 tx=0x7f94f0004270 comp rx=0 tx=0).stop 2024-09-21T04:02:33.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.625+0000 7f9507776640 1 -- 172.21.15.64:0/1374743291 shutdown_connections 2024-09-21T04:02:33.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.625+0000 7f9507776640 1 --2- 172.21.15.64:0/1374743291 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f94ec0b65f0 0x7f94ec0b8b00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:33.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.625+0000 7f9507776640 1 --2- 172.21.15.64:0/1374743291 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f950010de10 0x7f950019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:33.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.625+0000 7f9507776640 1 --2- 172.21.15.64:0/1374743291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9500105300 0x7f950019aa40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:33.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.625+0000 7f9507776640 1 --2- 172.21.15.64:0/1374743291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95001049b0 0x7f950019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:33.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.625+0000 7f9507776640 1 -- 172.21.15.64:0/1374743291 >> 172.21.15.64:0/1374743291 conn(0x7f9500079730 msgr2=0x7f9500100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:33.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.625+0000 7f9507776640 1 -- 172.21.15.64:0/1374743291 shutdown_connections 2024-09-21T04:02:33.628 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:33.625+0000 7f9507776640 1 -- 172.21.15.64:0/1374743291 wait complete. 2024-09-21T04:02:34.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:33 smithi119 ceph-mon[34811]: pgmap v275: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.6 KiB/s wr, 9 op/s; 72 B/s, 1 objects/s recovering 2024-09-21T04:02:34.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:33 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1374743291' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T04:02:34.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:33 smithi064 ceph-mon[31180]: pgmap v275: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.6 KiB/s wr, 9 op/s; 72 B/s, 1 objects/s recovering 2024-09-21T04:02:34.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:33 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1374743291' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T04:02:34.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:33 smithi131 ceph-mon[35193]: pgmap v275: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.6 KiB/s wr, 9 op/s; 72 B/s, 1 objects/s recovering 2024-09-21T04:02:34.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:33 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1374743291' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T04:02:34.399 INFO:teuthology.orchestra.run.smithi064.stdout:[{"version":55,"timestamp":"2024-09-21T04:01:52.815969+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-09-21T04:02:34.399 INFO:tasks.ceph_manager:config epoch is 55 2024-09-21T04:02:34.400 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T04:02:34.400 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-21T04:02:34.400 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph mgr dump --format=json 2024-09-21T04:02:34.663 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:35.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.164+0000 7f7bbb451640 1 -- 172.21.15.64:0/1033318312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bb4109df0 msgr2=0x7f7bb4110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:35.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.164+0000 7f7bbb451640 1 --2- 172.21.15.64:0/1033318312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bb4109df0 0x7f7bb4110620 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009ca0 tx=0x7f7bac02a8c0 comp rx=0 tx=0).stop 2024-09-21T04:02:35.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.164+0000 7f7bbb451640 1 -- 172.21.15.64:0/1033318312 shutdown_connections 2024-09-21T04:02:35.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.164+0000 7f7bbb451640 1 --2- 172.21.15.64:0/1033318312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bb4109df0 0x7f7bb4110620 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:35.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.164+0000 7f7bbb451640 1 --2- 172.21.15.64:0/1033318312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bb41094a0 0x7f7bb41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:35.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.164+0000 7f7bbb451640 1 --2- 172.21.15.64:0/1033318312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb4103b60 0x7f7bb4103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:35.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.164+0000 7f7bbb451640 1 -- 172.21.15.64:0/1033318312 >> 172.21.15.64:0/1033318312 conn(0x7f7bb407b850 msgr2=0x7f7bb407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:35.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.164+0000 7f7bbb451640 1 -- 172.21.15.64:0/1033318312 shutdown_connections 2024-09-21T04:02:35.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.164+0000 7f7bbb451640 1 -- 172.21.15.64:0/1033318312 wait complete. 2024-09-21T04:02:35.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.165+0000 7f7bbb451640 1 Processor -- start 2024-09-21T04:02:35.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.165+0000 7f7bbb451640 1 -- start start 2024-09-21T04:02:35.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.165+0000 7f7bbb451640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bb4103b60 0x7f7bb419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:35.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.165+0000 7f7bbb451640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bb41094a0 0x7f7bb419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:35.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.165+0000 7f7bbb451640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb4109df0 0x7f7bb419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:35.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.165+0000 7f7bbb451640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb4113b10 con 0x7f7bb41094a0 2024-09-21T04:02:35.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.165+0000 7f7bbb451640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb4113990 con 0x7f7bb4103b60 2024-09-21T04:02:35.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.165+0000 7f7bbb451640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb4113c90 con 0x7f7bb4109df0 2024-09-21T04:02:35.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.165+0000 7f7bb89ec640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bb41094a0 0x7f7bb419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:35.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.165+0000 7f7bb89ec640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bb41094a0 0x7f7bb419aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36570/0 (socket says 172.21.15.64:36570) 2024-09-21T04:02:35.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.165+0000 7f7bb89ec640 1 -- 172.21.15.64:0/3135621169 learned_addr learned my addr 172.21.15.64:0/3135621169 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:35.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.165+0000 7f7bb91ed640 1 --2- 172.21.15.64:0/3135621169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bb4103b60 0x7f7bb419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:35.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.166+0000 7f7bb99ee640 1 --2- 172.21.15.64:0/3135621169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb4109df0 0x7f7bb419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:35.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.166+0000 7f7bb89ec640 1 -- 172.21.15.64:0/3135621169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb4109df0 msgr2=0x7f7bb419fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:35.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.166+0000 7f7bb89ec640 1 --2- 172.21.15.64:0/3135621169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb4109df0 0x7f7bb419fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:35.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.166+0000 7f7bb89ec640 1 -- 172.21.15.64:0/3135621169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bb4103b60 msgr2=0x7f7bb419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:35.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.166+0000 7f7bb89ec640 1 --2- 172.21.15.64:0/3135621169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bb4103b60 0x7f7bb419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:35.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.166+0000 7f7bb89ec640 1 -- 172.21.15.64:0/3135621169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb41a03f0 con 0x7f7bb41094a0 2024-09-21T04:02:35.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.166+0000 7f7bb99ee640 1 --2- 172.21.15.64:0/3135621169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb4109df0 0x7f7bb419fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:02:35.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.166+0000 7f7bb89ec640 1 --2- 172.21.15.64:0/3135621169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bb41094a0 0x7f7bb419aa20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400c970 tx=0x7f7ba400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:35.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.166+0000 7f7baa7fc640 1 -- 172.21.15.64:0/3135621169 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4007bb0 con 0x7f7bb41094a0 2024-09-21T04:02:35.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.166+0000 7f7baa7fc640 1 -- 172.21.15.64:0/3135621169 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ba4007d50 con 0x7f7bb41094a0 2024-09-21T04:02:35.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.166+0000 7f7bbb451640 1 -- 172.21.15.64:0/3135621169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bb41a06e0 con 0x7f7bb41094a0 2024-09-21T04:02:35.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.166+0000 7f7baa7fc640 1 -- 172.21.15.64:0/3135621169 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4004d90 con 0x7f7bb41094a0 2024-09-21T04:02:35.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.166+0000 7f7bbb451640 1 -- 172.21.15.64:0/3135621169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bb41a6ec0 con 0x7f7bb41094a0 2024-09-21T04:02:35.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.167+0000 7f7bbb451640 1 -- 172.21.15.64:0/3135621169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bb4103f70 con 0x7f7bb41094a0 2024-09-21T04:02:35.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.169+0000 7f7baa7fc640 1 -- 172.21.15.64:0/3135621169 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f7ba4004f50 con 0x7f7bb41094a0 2024-09-21T04:02:35.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.170+0000 7f7baa7fc640 1 --2- 172.21.15.64:0/3135621169 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7ba00b6670 0x7f7ba00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:35.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.170+0000 7f7bb91ed640 1 --2- 172.21.15.64:0/3135621169 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7ba00b6670 0x7f7ba00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:35.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.170+0000 7f7baa7fc640 1 -- 172.21.15.64:0/3135621169 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f7ba40e7630 con 0x7f7bb41094a0 2024-09-21T04:02:35.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.170+0000 7f7bb91ed640 1 --2- 172.21.15.64:0/3135621169 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7ba00b6670 0x7f7ba00b8b80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009990 tx=0x7f7bb0009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:35.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.173+0000 7f7baa7fc640 1 -- 172.21.15.64:0/3135621169 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7ba4014030 con 0x7f7bb41094a0 2024-09-21T04:02:35.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.409+0000 7f7bbb451640 1 -- 172.21.15.64:0/3135621169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f7bb419b5b0 con 0x7f7bb41094a0 2024-09-21T04:02:35.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.414+0000 7f7baa7fc640 1 -- 172.21.15.64:0/3135621169 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v21) ==== 74+0+296450 (secure 0 0 0) 0x7f7bb419b5b0 con 0x7f7bb41094a0 2024-09-21T04:02:35.420 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:02:35.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.417+0000 7f7bbb451640 1 -- 172.21.15.64:0/3135621169 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7ba00b6670 msgr2=0x7f7ba00b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:35.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.417+0000 7f7bbb451640 1 --2- 172.21.15.64:0/3135621169 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7ba00b6670 0x7f7ba00b8b80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009990 tx=0x7f7bb0009110 comp rx=0 tx=0).stop 2024-09-21T04:02:35.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.417+0000 7f7bbb451640 1 -- 172.21.15.64:0/3135621169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bb41094a0 msgr2=0x7f7bb419aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:35.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.417+0000 7f7bbb451640 1 --2- 172.21.15.64:0/3135621169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bb41094a0 0x7f7bb419aa20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400c970 tx=0x7f7ba400cd70 comp rx=0 tx=0).stop 2024-09-21T04:02:35.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.418+0000 7f7bbb451640 1 -- 172.21.15.64:0/3135621169 shutdown_connections 2024-09-21T04:02:35.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.418+0000 7f7bbb451640 1 --2- 172.21.15.64:0/3135621169 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7ba00b6670 0x7f7ba00b8b80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:35.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.418+0000 7f7bbb451640 1 --2- 172.21.15.64:0/3135621169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7bb4109df0 0x7f7bb419fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:35.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.418+0000 7f7bbb451640 1 --2- 172.21.15.64:0/3135621169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7bb41094a0 0x7f7bb419aa20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:35.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.418+0000 7f7bbb451640 1 --2- 172.21.15.64:0/3135621169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bb4103b60 0x7f7bb419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:35.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.418+0000 7f7bbb451640 1 -- 172.21.15.64:0/3135621169 >> 172.21.15.64:0/3135621169 conn(0x7f7bb407b850 msgr2=0x7f7bb41074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:35.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.418+0000 7f7bbb451640 1 -- 172.21.15.64:0/3135621169 shutdown_connections 2024-09-21T04:02:35.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:35.418+0000 7f7bbb451640 1 -- 172.21.15.64:0/3135621169 wait complete. 2024-09-21T04:02:36.233 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:35 smithi064 ceph-mon[31180]: pgmap v276: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 7 op/s; 72 B/s, 1 objects/s recovering 2024-09-21T04:02:36.233 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:35 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3135621169' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T04:02:36.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:35 smithi119 ceph-mon[34811]: pgmap v276: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 7 op/s; 72 B/s, 1 objects/s recovering 2024-09-21T04:02:36.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:35 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3135621169' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T04:02:36.266 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":21,"flags":0,"active_gid":14226,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":834288654},{"type":"v1","addr":"172.21.15.64:6801","nonce":834288654}]},"active_addr":"172.21.15.64:6801/834288654","active_change":"2024-09-21T03:55:28.387558+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24151,"name":"z","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24170,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.64:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":2407752174}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":869042251}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":4035775195}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":2999912642}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.64:0","nonce":4250366387}]}]} 2024-09-21T04:02:36.272 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-21T04:02:36.272 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-21T04:02:36.272 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T04:02:36.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:35 smithi131 ceph-mon[35193]: pgmap v276: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 7 op/s; 72 B/s, 1 objects/s recovering 2024-09-21T04:02:36.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:35 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3135621169' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T04:02:36.534 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:37.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.000+0000 7f0363cce640 1 -- 172.21.15.64:0/3489744966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f035c109df0 msgr2=0x7f035c110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:37.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.000+0000 7f0363cce640 1 --2- 172.21.15.64:0/3489744966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f035c109df0 0x7f035c110620 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0354009ca0 tx=0x7f035402a8c0 comp rx=0 tx=0).stop 2024-09-21T04:02:37.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.000+0000 7f0363cce640 1 -- 172.21.15.64:0/3489744966 shutdown_connections 2024-09-21T04:02:37.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.000+0000 7f0363cce640 1 --2- 172.21.15.64:0/3489744966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f035c109df0 0x7f035c110620 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:37.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.000+0000 7f0363cce640 1 --2- 172.21.15.64:0/3489744966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f035c1094a0 0x7f035c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:37.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.000+0000 7f0363cce640 1 --2- 172.21.15.64:0/3489744966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f035c103b60 0x7f035c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:37.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.000+0000 7f0363cce640 1 -- 172.21.15.64:0/3489744966 >> 172.21.15.64:0/3489744966 conn(0x7f035c07b850 msgr2=0x7f035c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:37.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.001+0000 7f0363cce640 1 -- 172.21.15.64:0/3489744966 shutdown_connections 2024-09-21T04:02:37.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.001+0000 7f0363cce640 1 -- 172.21.15.64:0/3489744966 wait complete. 2024-09-21T04:02:37.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.001+0000 7f0363cce640 1 Processor -- start 2024-09-21T04:02:37.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.001+0000 7f0363cce640 1 -- start start 2024-09-21T04:02:37.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.002+0000 7f0363cce640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f035c103b60 0x7f035c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:37.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.002+0000 7f0363cce640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f035c1094a0 0x7f035c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.002+0000 7f0363cce640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f035c109df0 0x7f035c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.002+0000 7f0363cce640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f035c113ab0 con 0x7f035c1094a0 2024-09-21T04:02:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.002+0000 7f0363cce640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f035c113930 con 0x7f035c109df0 2024-09-21T04:02:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.002+0000 7f0363cce640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f035c113c30 con 0x7f035c103b60 2024-09-21T04:02:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.002+0000 7f0361269640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f035c1094a0 0x7f035c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.002+0000 7f0361269640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f035c1094a0 0x7f035c075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36586/0 (socket says 172.21.15.64:36586) 2024-09-21T04:02:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.002+0000 7f0361269640 1 -- 172.21.15.64:0/2986084574 learned_addr learned my addr 172.21.15.64:0/2986084574 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.002+0000 7f036226b640 1 --2- 172.21.15.64:0/2986084574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f035c109df0 0x7f035c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.002+0000 7f0361a6a640 1 --2- 172.21.15.64:0/2986084574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f035c103b60 0x7f035c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.002+0000 7f0361269640 1 -- 172.21.15.64:0/2986084574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f035c103b60 msgr2=0x7f035c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:37.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.002+0000 7f0361269640 1 --2- 172.21.15.64:0/2986084574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f035c103b60 0x7f035c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:37.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.002+0000 7f0361269640 1 -- 172.21.15.64:0/2986084574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f035c109df0 msgr2=0x7f035c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:37.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.002+0000 7f0361269640 1 --2- 172.21.15.64:0/2986084574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f035c109df0 0x7f035c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:37.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.002+0000 7f0361269640 1 -- 172.21.15.64:0/2986084574 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f035c1a8700 con 0x7f035c1094a0 2024-09-21T04:02:37.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.002+0000 7f0361269640 1 --2- 172.21.15.64:0/2986084574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f035c1094a0 0x7f035c075c50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f034c00d950 tx=0x7f034c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:37.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.003+0000 7f0352ffd640 1 -- 172.21.15.64:0/2986084574 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c017070 con 0x7f035c1094a0 2024-09-21T04:02:37.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.003+0000 7f0352ffd640 1 -- 172.21.15.64:0/2986084574 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f034c009c30 con 0x7f035c1094a0 2024-09-21T04:02:37.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.003+0000 7f0363cce640 1 -- 172.21.15.64:0/2986084574 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f035c1a89f0 con 0x7f035c1094a0 2024-09-21T04:02:37.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.003+0000 7f0352ffd640 1 -- 172.21.15.64:0/2986084574 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c0047a0 con 0x7f035c1094a0 2024-09-21T04:02:37.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.003+0000 7f0363cce640 1 -- 172.21.15.64:0/2986084574 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f035c1a8f80 con 0x7f035c1094a0 2024-09-21T04:02:37.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.004+0000 7f0363cce640 1 -- 172.21.15.64:0/2986084574 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f035c103f70 con 0x7f035c1094a0 2024-09-21T04:02:37.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.005+0000 7f0352ffd640 1 -- 172.21.15.64:0/2986084574 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f034c004020 con 0x7f035c1094a0 2024-09-21T04:02:37.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.006+0000 7f0352ffd640 1 --2- 172.21.15.64:0/2986084574 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f03480b6670 0x7f03480b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:37.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.006+0000 7f0352ffd640 1 -- 172.21.15.64:0/2986084574 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f034c0e7510 con 0x7f035c1094a0 2024-09-21T04:02:37.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.006+0000 7f0361a6a640 1 --2- 172.21.15.64:0/2986084574 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f03480b6670 0x7f03480b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:37.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.007+0000 7f0361a6a640 1 --2- 172.21.15.64:0/2986084574 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f03480b6670 0x7f03480b8b80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f035c077160 tx=0x7f035800b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:37.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.009+0000 7f0352ffd640 1 -- 172.21.15.64:0/2986084574 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f034c020020 con 0x7f035c1094a0 2024-09-21T04:02:37.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.201+0000 7f0363cce640 1 -- 172.21.15.64:0/2986084574 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f035c10bc40 con 0x7f035c1094a0 2024-09-21T04:02:37.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.202+0000 7f0352ffd640 1 -- 172.21.15.64:0/2986084574 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+23089 (secure 0 0 0) 0x7f034c0adbc0 con 0x7f035c1094a0 2024-09-21T04:02:37.205 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:02:37.206 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":75,"fsid":"e7c927ba-77cc-11ef-baf5-efdc52797490","created":"2024-09-21T03:53:41.215073+0000","modified":"2024-09-21T04:02:29.856726+0000","last_up_change":"2024-09-21T04:01:11.986949+0000","last_in_change":"2024-09-21T04:00:48.707791+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:57:24.854438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T04:01:58.728355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T04:02:01.300385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb947c10-98e6-4e3a-8bbd-8753951d870b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6803","nonce":3692023035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6805","nonce":3692023035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6809","nonce":3692023035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6807","nonce":3692023035}]},"public_addr":"172.21.15.64:6803/3692023035","cluster_addr":"172.21.15.64:6805/3692023035","heartbeat_back_addr":"172.21.15.64:6809/3692023035","heartbeat_front_addr":"172.21.15.64:6807/3692023035","state":["exists","up"]},{"osd":1,"uuid":"29b8c92d-82fa-4175-b3e2-711917b7f8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6801","nonce":3363415998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6803","nonce":3363415998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6807","nonce":3363415998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6805","nonce":3363415998}]},"public_addr":"172.21.15.119:6801/3363415998","cluster_addr":"172.21.15.119:6803/3363415998","heartbeat_back_addr":"172.21.15.119:6807/3363415998","heartbeat_front_addr":"172.21.15.119:6805/3363415998","state":["exists","up"]},{"osd":2,"uuid":"a2273fe8-df55-40f7-afda-74ab09c52c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6801","nonce":3422924179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6803","nonce":3422924179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6807","nonce":3422924179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6805","nonce":3422924179}]},"public_addr":"172.21.15.131:6801/3422924179","cluster_addr":"172.21.15.131:6803/3422924179","heartbeat_back_addr":"172.21.15.131:6807/3422924179","heartbeat_front_addr":"172.21.15.131:6805/3422924179","state":["exists","up"]},{"osd":3,"uuid":"f1634064-b108-41d2-a78f-c33607b08a70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6811","nonce":2022952729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6813","nonce":2022952729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6817","nonce":2022952729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6815","nonce":2022952729}]},"public_addr":"172.21.15.64:6811/2022952729","cluster_addr":"172.21.15.64:6813/2022952729","heartbeat_back_addr":"172.21.15.64:6817/2022952729","heartbeat_front_addr":"172.21.15.64:6815/2022952729","state":["exists","up"]},{"osd":4,"uuid":"75b81a8d-fa52-411a-bfda-aa69bb70a20f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6809","nonce":323030958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6811","nonce":323030958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6815","nonce":323030958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6813","nonce":323030958}]},"public_addr":"172.21.15.119:6809/323030958","cluster_addr":"172.21.15.119:6811/323030958","heartbeat_back_addr":"172.21.15.119:6815/323030958","heartbeat_front_addr":"172.21.15.119:6813/323030958","state":["exists","up"]},{"osd":5,"uuid":"0e4895b5-42e4-4b2c-a9c3-692f287cb841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6809","nonce":3021788088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6811","nonce":3021788088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6815","nonce":3021788088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6813","nonce":3021788088}]},"public_addr":"172.21.15.131:6809/3021788088","cluster_addr":"172.21.15.131:6811/3021788088","heartbeat_back_addr":"172.21.15.131:6815/3021788088","heartbeat_front_addr":"172.21.15.131:6813/3021788088","state":["exists","up"]},{"osd":6,"uuid":"1fa5c2b7-bce0-43a1-a26b-d029eb3731fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6819","nonce":1913824324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6821","nonce":1913824324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6825","nonce":1913824324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6823","nonce":1913824324}]},"public_addr":"172.21.15.64:6819/1913824324","cluster_addr":"172.21.15.64:6821/1913824324","heartbeat_back_addr":"172.21.15.64:6825/1913824324","heartbeat_front_addr":"172.21.15.64:6823/1913824324","state":["exists","up"]},{"osd":7,"uuid":"c9707260-b581-4a21-b380-f37e16959b02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6817","nonce":2567568166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6819","nonce":2567568166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6823","nonce":2567568166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6821","nonce":2567568166}]},"public_addr":"172.21.15.119:6817/2567568166","cluster_addr":"172.21.15.119:6819/2567568166","heartbeat_back_addr":"172.21.15.119:6823/2567568166","heartbeat_front_addr":"172.21.15.119:6821/2567568166","state":["exists","up"]},{"osd":8,"uuid":"2dbf8ef2-fc9c-4c6b-bfea-003f41ef191e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6817","nonce":199192217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6819","nonce":199192217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6823","nonce":199192217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6821","nonce":199192217}]},"public_addr":"172.21.15.131:6817/199192217","cluster_addr":"172.21.15.131:6819/199192217","heartbeat_back_addr":"172.21.15.131:6823/199192217","heartbeat_front_addr":"172.21.15.131:6821/199192217","state":["exists","up"]},{"osd":9,"uuid":"96f769e5-4783-4558-adfa-4bfaf9da9e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6827","nonce":3309038990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6829","nonce":3309038990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6833","nonce":3309038990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6831","nonce":3309038990}]},"public_addr":"172.21.15.64:6827/3309038990","cluster_addr":"172.21.15.64:6829/3309038990","heartbeat_back_addr":"172.21.15.64:6833/3309038990","heartbeat_front_addr":"172.21.15.64:6831/3309038990","state":["exists","up"]},{"osd":10,"uuid":"13c1ef5c-2299-440c-af5a-a650b47f4b75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6825","nonce":1460844681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6827","nonce":1460844681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6831","nonce":1460844681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6829","nonce":1460844681}]},"public_addr":"172.21.15.119:6825/1460844681","cluster_addr":"172.21.15.119:6827/1460844681","heartbeat_back_addr":"172.21.15.119:6831/1460844681","heartbeat_front_addr":"172.21.15.119:6829/1460844681","state":["exists","up"]},{"osd":11,"uuid":"c26a5ef9-8585-404e-a54a-852424c4d6fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6825","nonce":2294059084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6827","nonce":2294059084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6831","nonce":2294059084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6829","nonce":2294059084}]},"public_addr":"172.21.15.131:6825/2294059084","cluster_addr":"172.21.15.131:6827/2294059084","heartbeat_back_addr":"172.21.15.131:6831/2294059084","heartbeat_front_addr":"172.21.15.131:6829/2294059084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:56:33.141102+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:56:58.883062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:57:22.641789+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:57:48.705790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:58:12.871514+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:58:37.893118+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:03.620828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:29.446179+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:55.002978+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:00:19.987463+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:00:45.364158+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:01:10.228248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/3554471670":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6801/3786213009":"2024-09-22T03:55:28.387426+0000","172.21.15.64:0/2562856406":"2024-09-22T03:54:27.180100+0000","172.21.15.64:0/2603015699":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6800/1209932961":"2024-09-22T03:54:02.562380+0000","172.21.15.64:6800/3786213009":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6801/1209932961":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/3407344596":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1233731831":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1686887701":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1598568021":"2024-09-22T03:54:27.180100+0000","172.21.15.64:6800/2013929312":"2024-09-22T03:54:27.180100+0000","172.21.15.64:6801/2013929312":"2024-09-22T03:54:27.180100+0000","172.21.15.64:0/3652036144":"2024-09-22T03:55:28.387426+0000","172.21.15.64:0/105454703":"2024-09-22T03:54:27.180100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T04:02:37.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.205+0000 7f0363cce640 1 -- 172.21.15.64:0/2986084574 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f03480b6670 msgr2=0x7f03480b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:37.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.205+0000 7f0363cce640 1 --2- 172.21.15.64:0/2986084574 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f03480b6670 0x7f03480b8b80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f035c077160 tx=0x7f035800b410 comp rx=0 tx=0).stop 2024-09-21T04:02:37.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.205+0000 7f0363cce640 1 -- 172.21.15.64:0/2986084574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f035c1094a0 msgr2=0x7f035c075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:37.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.205+0000 7f0363cce640 1 --2- 172.21.15.64:0/2986084574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f035c1094a0 0x7f035c075c50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f034c00d950 tx=0x7f034c00de10 comp rx=0 tx=0).stop 2024-09-21T04:02:37.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.205+0000 7f0363cce640 1 -- 172.21.15.64:0/2986084574 shutdown_connections 2024-09-21T04:02:37.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.205+0000 7f0363cce640 1 --2- 172.21.15.64:0/2986084574 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f03480b6670 0x7f03480b8b80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:37.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.205+0000 7f0363cce640 1 --2- 172.21.15.64:0/2986084574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f035c109df0 0x7f035c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:37.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.205+0000 7f0363cce640 1 --2- 172.21.15.64:0/2986084574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f035c1094a0 0x7f035c075c50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:37.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.205+0000 7f0363cce640 1 --2- 172.21.15.64:0/2986084574 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f035c103b60 0x7f035c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:37.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.205+0000 7f0363cce640 1 -- 172.21.15.64:0/2986084574 >> 172.21.15.64:0/2986084574 conn(0x7f035c07b850 msgr2=0x7f035c1074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:37.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.205+0000 7f0363cce640 1 -- 172.21.15.64:0/2986084574 shutdown_connections 2024-09-21T04:02:37.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:37.205+0000 7f0363cce640 1 -- 172.21.15.64:0/2986084574 wait complete. 2024-09-21T04:02:37.985 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:37 smithi064 ceph-mon[31180]: pgmap v277: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s; 72 B/s, 1 objects/s recovering 2024-09-21T04:02:37.986 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:37 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2986084574' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:02:38.016 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-21T04:02:38.016 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T04:02:38.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:37 smithi119 ceph-mon[34811]: pgmap v277: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s; 72 B/s, 1 objects/s recovering 2024-09-21T04:02:38.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:37 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2986084574' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:02:38.276 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:38.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:37 smithi131 ceph-mon[35193]: pgmap v277: 97 pgs: 97 active+clean; 640 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s; 72 B/s, 1 objects/s recovering 2024-09-21T04:02:38.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:37 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2986084574' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:02:38.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.763+0000 7f289eef6640 1 -- 172.21.15.64:0/2245198752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2898109e10 msgr2=0x7f2898110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:38.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.763+0000 7f289eef6640 1 --2- 172.21.15.64:0/2245198752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2898109e10 0x7f2898110640 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2888009900 tx=0x7f288802a410 comp rx=0 tx=0).stop 2024-09-21T04:02:38.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.763+0000 7f289eef6640 1 -- 172.21.15.64:0/2245198752 shutdown_connections 2024-09-21T04:02:38.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.763+0000 7f289eef6640 1 --2- 172.21.15.64:0/2245198752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2898109e10 0x7f2898110640 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:38.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.763+0000 7f289eef6640 1 --2- 172.21.15.64:0/2245198752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28981094c0 0x7f28981098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:38.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.763+0000 7f289eef6640 1 --2- 172.21.15.64:0/2245198752 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2898103b80 0x7f2898103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:38.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.763+0000 7f289eef6640 1 -- 172.21.15.64:0/2245198752 >> 172.21.15.64:0/2245198752 conn(0x7f28980ff5d0 msgr2=0x7f2898101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:38.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.763+0000 7f289eef6640 1 -- 172.21.15.64:0/2245198752 shutdown_connections 2024-09-21T04:02:38.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.763+0000 7f289eef6640 1 -- 172.21.15.64:0/2245198752 wait complete. 2024-09-21T04:02:38.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.763+0000 7f289eef6640 1 Processor -- start 2024-09-21T04:02:38.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.763+0000 7f289eef6640 1 -- start start 2024-09-21T04:02:38.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.764+0000 7f289eef6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2898103b80 0x7f2898075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:38.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.764+0000 7f289eef6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28981094c0 0x7f2898075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:38.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.764+0000 7f289eef6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2898109e10 0x7f2898076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:38.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.764+0000 7f289eef6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2898113ad0 con 0x7f2898109e10 2024-09-21T04:02:38.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.764+0000 7f289eef6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2898113950 con 0x7f28981094c0 2024-09-21T04:02:38.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.764+0000 7f289eef6640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2898113c50 con 0x7f2898103b80 2024-09-21T04:02:38.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.764+0000 7f289d493640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2898109e10 0x7f2898076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:38.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.764+0000 7f289d493640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2898109e10 0x7f2898076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36592/0 (socket says 172.21.15.64:36592) 2024-09-21T04:02:38.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.764+0000 7f289d493640 1 -- 172.21.15.64:0/2282274050 learned_addr learned my addr 172.21.15.64:0/2282274050 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:38.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.764+0000 7f289cc92640 1 --2- 172.21.15.64:0/2282274050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2898103b80 0x7f2898075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:38.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.764+0000 7f2897fff640 1 --2- 172.21.15.64:0/2282274050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28981094c0 0x7f2898075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:38.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.764+0000 7f289d493640 1 -- 172.21.15.64:0/2282274050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2898103b80 msgr2=0x7f2898075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:38.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.764+0000 7f289d493640 1 --2- 172.21.15.64:0/2282274050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2898103b80 0x7f2898075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:38.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.764+0000 7f289d493640 1 -- 172.21.15.64:0/2282274050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28981094c0 msgr2=0x7f2898075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:38.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.764+0000 7f289d493640 1 --2- 172.21.15.64:0/2282274050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28981094c0 0x7f2898075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:38.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.764+0000 7f289d493640 1 -- 172.21.15.64:0/2282274050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28981a8650 con 0x7f2898109e10 2024-09-21T04:02:38.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.764+0000 7f289d493640 1 --2- 172.21.15.64:0/2282274050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2898109e10 0x7f2898076190 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2888004770 tx=0x7f2888004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:38.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.765+0000 7f2895ffb640 1 -- 172.21.15.64:0/2282274050 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288803f030 con 0x7f2898109e10 2024-09-21T04:02:38.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.765+0000 7f2895ffb640 1 -- 172.21.15.64:0/2282274050 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2888003aa0 con 0x7f2898109e10 2024-09-21T04:02:38.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.765+0000 7f2895ffb640 1 -- 172.21.15.64:0/2282274050 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2888035070 con 0x7f2898109e10 2024-09-21T04:02:38.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.765+0000 7f289eef6640 1 -- 172.21.15.64:0/2282274050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28981a88e0 con 0x7f2898109e10 2024-09-21T04:02:38.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.765+0000 7f289eef6640 1 -- 172.21.15.64:0/2282274050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28981a8d70 con 0x7f2898109e10 2024-09-21T04:02:38.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.766+0000 7f289eef6640 1 -- 172.21.15.64:0/2282274050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2898103f90 con 0x7f2898109e10 2024-09-21T04:02:38.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.769+0000 7f2895ffb640 1 -- 172.21.15.64:0/2282274050 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f2888042030 con 0x7f2898109e10 2024-09-21T04:02:38.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.769+0000 7f2895ffb640 1 --2- 172.21.15.64:0/2282274050 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f28800b6670 0x7f28800b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:38.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.769+0000 7f289cc92640 1 --2- 172.21.15.64:0/2282274050 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f28800b6670 0x7f28800b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:38.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.769+0000 7f2895ffb640 1 -- 172.21.15.64:0/2282274050 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f2888107fb0 con 0x7f2898109e10 2024-09-21T04:02:38.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.770+0000 7f289cc92640 1 --2- 172.21.15.64:0/2282274050 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f28800b6670 0x7f28800b8b80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2898077160 tx=0x7f2890009680 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:38.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.773+0000 7f2895ffb640 1 -- 172.21.15.64:0/2282274050 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f28880ce5e0 con 0x7f2898109e10 2024-09-21T04:02:38.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.963+0000 7f289eef6640 1 -- 172.21.15.64:0/2282274050 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f289802e640 con 0x7f2898109e10 2024-09-21T04:02:39.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.964+0000 7f2895ffb640 1 -- 172.21.15.64:0/2282274050 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+23089 (secure 0 0 0) 0x7f28880d29c0 con 0x7f2898109e10 2024-09-21T04:02:39.154 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:02:39.154 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":75,"fsid":"e7c927ba-77cc-11ef-baf5-efdc52797490","created":"2024-09-21T03:53:41.215073+0000","modified":"2024-09-21T04:02:29.856726+0000","last_up_change":"2024-09-21T04:01:11.986949+0000","last_in_change":"2024-09-21T04:00:48.707791+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:57:24.854438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T04:01:58.728355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T04:02:01.300385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb947c10-98e6-4e3a-8bbd-8753951d870b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6803","nonce":3692023035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6805","nonce":3692023035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6809","nonce":3692023035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6807","nonce":3692023035}]},"public_addr":"172.21.15.64:6803/3692023035","cluster_addr":"172.21.15.64:6805/3692023035","heartbeat_back_addr":"172.21.15.64:6809/3692023035","heartbeat_front_addr":"172.21.15.64:6807/3692023035","state":["exists","up"]},{"osd":1,"uuid":"29b8c92d-82fa-4175-b3e2-711917b7f8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6801","nonce":3363415998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6803","nonce":3363415998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6807","nonce":3363415998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6805","nonce":3363415998}]},"public_addr":"172.21.15.119:6801/3363415998","cluster_addr":"172.21.15.119:6803/3363415998","heartbeat_back_addr":"172.21.15.119:6807/3363415998","heartbeat_front_addr":"172.21.15.119:6805/3363415998","state":["exists","up"]},{"osd":2,"uuid":"a2273fe8-df55-40f7-afda-74ab09c52c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6801","nonce":3422924179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6803","nonce":3422924179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6807","nonce":3422924179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6805","nonce":3422924179}]},"public_addr":"172.21.15.131:6801/3422924179","cluster_addr":"172.21.15.131:6803/3422924179","heartbeat_back_addr":"172.21.15.131:6807/3422924179","heartbeat_front_addr":"172.21.15.131:6805/3422924179","state":["exists","up"]},{"osd":3,"uuid":"f1634064-b108-41d2-a78f-c33607b08a70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6811","nonce":2022952729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6813","nonce":2022952729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6817","nonce":2022952729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6815","nonce":2022952729}]},"public_addr":"172.21.15.64:6811/2022952729","cluster_addr":"172.21.15.64:6813/2022952729","heartbeat_back_addr":"172.21.15.64:6817/2022952729","heartbeat_front_addr":"172.21.15.64:6815/2022952729","state":["exists","up"]},{"osd":4,"uuid":"75b81a8d-fa52-411a-bfda-aa69bb70a20f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6809","nonce":323030958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6811","nonce":323030958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6815","nonce":323030958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6813","nonce":323030958}]},"public_addr":"172.21.15.119:6809/323030958","cluster_addr":"172.21.15.119:6811/323030958","heartbeat_back_addr":"172.21.15.119:6815/323030958","heartbeat_front_addr":"172.21.15.119:6813/323030958","state":["exists","up"]},{"osd":5,"uuid":"0e4895b5-42e4-4b2c-a9c3-692f287cb841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6809","nonce":3021788088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6811","nonce":3021788088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6815","nonce":3021788088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6813","nonce":3021788088}]},"public_addr":"172.21.15.131:6809/3021788088","cluster_addr":"172.21.15.131:6811/3021788088","heartbeat_back_addr":"172.21.15.131:6815/3021788088","heartbeat_front_addr":"172.21.15.131:6813/3021788088","state":["exists","up"]},{"osd":6,"uuid":"1fa5c2b7-bce0-43a1-a26b-d029eb3731fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6819","nonce":1913824324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6821","nonce":1913824324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6825","nonce":1913824324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6823","nonce":1913824324}]},"public_addr":"172.21.15.64:6819/1913824324","cluster_addr":"172.21.15.64:6821/1913824324","heartbeat_back_addr":"172.21.15.64:6825/1913824324","heartbeat_front_addr":"172.21.15.64:6823/1913824324","state":["exists","up"]},{"osd":7,"uuid":"c9707260-b581-4a21-b380-f37e16959b02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6817","nonce":2567568166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6819","nonce":2567568166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6823","nonce":2567568166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6821","nonce":2567568166}]},"public_addr":"172.21.15.119:6817/2567568166","cluster_addr":"172.21.15.119:6819/2567568166","heartbeat_back_addr":"172.21.15.119:6823/2567568166","heartbeat_front_addr":"172.21.15.119:6821/2567568166","state":["exists","up"]},{"osd":8,"uuid":"2dbf8ef2-fc9c-4c6b-bfea-003f41ef191e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6817","nonce":199192217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6819","nonce":199192217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6823","nonce":199192217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6821","nonce":199192217}]},"public_addr":"172.21.15.131:6817/199192217","cluster_addr":"172.21.15.131:6819/199192217","heartbeat_back_addr":"172.21.15.131:6823/199192217","heartbeat_front_addr":"172.21.15.131:6821/199192217","state":["exists","up"]},{"osd":9,"uuid":"96f769e5-4783-4558-adfa-4bfaf9da9e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6827","nonce":3309038990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6829","nonce":3309038990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6833","nonce":3309038990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6831","nonce":3309038990}]},"public_addr":"172.21.15.64:6827/3309038990","cluster_addr":"172.21.15.64:6829/3309038990","heartbeat_back_addr":"172.21.15.64:6833/3309038990","heartbeat_front_addr":"172.21.15.64:6831/3309038990","state":["exists","up"]},{"osd":10,"uuid":"13c1ef5c-2299-440c-af5a-a650b47f4b75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6825","nonce":1460844681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6827","nonce":1460844681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6831","nonce":1460844681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6829","nonce":1460844681}]},"public_addr":"172.21.15.119:6825/1460844681","cluster_addr":"172.21.15.119:6827/1460844681","heartbeat_back_addr":"172.21.15.119:6831/1460844681","heartbeat_front_addr":"172.21.15.119:6829/1460844681","state":["exists","up"]},{"osd":11,"uuid":"c26a5ef9-8585-404e-a54a-852424c4d6fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6825","nonce":2294059084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6827","nonce":2294059084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6831","nonce":2294059084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6829","nonce":2294059084}]},"public_addr":"172.21.15.131:6825/2294059084","cluster_addr":"172.21.15.131:6827/2294059084","heartbeat_back_addr":"172.21.15.131:6831/2294059084","heartbeat_front_addr":"172.21.15.131:6829/2294059084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:56:33.141102+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:56:58.883062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:57:22.641789+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:57:48.705790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:58:12.871514+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:58:37.893118+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:03.620828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:29.446179+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:55.002978+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:00:19.987463+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:00:45.364158+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:01:10.228248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/3554471670":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6801/3786213009":"2024-09-22T03:55:28.387426+0000","172.21.15.64:0/2562856406":"2024-09-22T03:54:27.180100+0000","172.21.15.64:0/2603015699":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6800/1209932961":"2024-09-22T03:54:02.562380+0000","172.21.15.64:6800/3786213009":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6801/1209932961":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/3407344596":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1233731831":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1686887701":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1598568021":"2024-09-22T03:54:27.180100+0000","172.21.15.64:6800/2013929312":"2024-09-22T03:54:27.180100+0000","172.21.15.64:6801/2013929312":"2024-09-22T03:54:27.180100+0000","172.21.15.64:0/3652036144":"2024-09-22T03:55:28.387426+0000","172.21.15.64:0/105454703":"2024-09-22T03:54:27.180100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T04:02:39.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.967+0000 7f289eef6640 1 -- 172.21.15.64:0/2282274050 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f28800b6670 msgr2=0x7f28800b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:39.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.967+0000 7f289eef6640 1 --2- 172.21.15.64:0/2282274050 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f28800b6670 0x7f28800b8b80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2898077160 tx=0x7f2890009680 comp rx=0 tx=0).stop 2024-09-21T04:02:39.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.967+0000 7f289eef6640 1 -- 172.21.15.64:0/2282274050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2898109e10 msgr2=0x7f2898076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:39.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.967+0000 7f289eef6640 1 --2- 172.21.15.64:0/2282274050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2898109e10 0x7f2898076190 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2888004770 tx=0x7f2888004270 comp rx=0 tx=0).stop 2024-09-21T04:02:39.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.967+0000 7f289eef6640 1 -- 172.21.15.64:0/2282274050 shutdown_connections 2024-09-21T04:02:39.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.967+0000 7f289eef6640 1 --2- 172.21.15.64:0/2282274050 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f28800b6670 0x7f28800b8b80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:39.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.967+0000 7f289eef6640 1 --2- 172.21.15.64:0/2282274050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2898109e10 0x7f2898076190 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:39.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.967+0000 7f289eef6640 1 --2- 172.21.15.64:0/2282274050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28981094c0 0x7f2898075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:39.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.967+0000 7f289eef6640 1 --2- 172.21.15.64:0/2282274050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2898103b80 0x7f2898075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:39.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.967+0000 7f289eef6640 1 -- 172.21.15.64:0/2282274050 >> 172.21.15.64:0/2282274050 conn(0x7f28980ff5d0 msgr2=0x7f28981019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:39.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.967+0000 7f289eef6640 1 -- 172.21.15.64:0/2282274050 shutdown_connections 2024-09-21T04:02:39.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:38.967+0000 7f289eef6640 1 -- 172.21.15.64:0/2282274050 wait complete. 2024-09-21T04:02:39.669 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-21T04:02:39.669 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-21T04:02:39.670 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-21T04:02:39.670 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-21T04:02:39.670 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-21T04:02:39.671 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-21T04:02:39.671 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell osd.6 flush_pg_stats 2024-09-21T04:02:39.672 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell osd.7 flush_pg_stats 2024-09-21T04:02:39.672 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell osd.8 flush_pg_stats 2024-09-21T04:02:39.673 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell osd.9 flush_pg_stats 2024-09-21T04:02:39.673 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell osd.10 flush_pg_stats 2024-09-21T04:02:39.673 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell osd.11 flush_pg_stats 2024-09-21T04:02:40.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:39 smithi119 ceph-mon[34811]: pgmap v278: 97 pgs: 97 active+clean; 644 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 7 op/s; 60 B/s, 0 objects/s recovering 2024-09-21T04:02:40.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:39 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2282274050' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:02:40.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:39 smithi064 ceph-mon[31180]: pgmap v278: 97 pgs: 97 active+clean; 644 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 7 op/s; 60 B/s, 0 objects/s recovering 2024-09-21T04:02:40.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:39 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2282274050' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:02:40.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:39 smithi131 ceph-mon[35193]: pgmap v278: 97 pgs: 97 active+clean; 644 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.7 KiB/s wr, 7 op/s; 60 B/s, 0 objects/s recovering 2024-09-21T04:02:40.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:39 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2282274050' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:02:40.364 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:41.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.980+0000 7f9c593e1640 1 -- 172.21.15.64:0/3540118830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c54102aa0 msgr2=0x7f9c54102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:41.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.980+0000 7f9c593e1640 1 --2- 172.21.15.64:0/3540118830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c54102aa0 0x7f9c54102eb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009900 tx=0x7f9c4402a410 comp rx=0 tx=0).stop 2024-09-21T04:02:41.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.980+0000 7f9c593e1640 1 -- 172.21.15.64:0/3540118830 shutdown_connections 2024-09-21T04:02:41.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.980+0000 7f9c593e1640 1 --2- 172.21.15.64:0/3540118830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c54108d30 0x7f9c5410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:41.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.980+0000 7f9c593e1640 1 --2- 172.21.15.64:0/3540118830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c541083e0 0x7f9c541087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:41.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.980+0000 7f9c593e1640 1 --2- 172.21.15.64:0/3540118830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c54102aa0 0x7f9c54102eb0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:41.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.980+0000 7f9c593e1640 1 -- 172.21.15.64:0/3540118830 >> 172.21.15.64:0/3540118830 conn(0x7f9c540797e0 msgr2=0x7f9c54079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:41.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.980+0000 7f9c593e1640 1 -- 172.21.15.64:0/3540118830 shutdown_connections 2024-09-21T04:02:41.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.980+0000 7f9c593e1640 1 -- 172.21.15.64:0/3540118830 wait complete. 2024-09-21T04:02:41.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.981+0000 7f9c593e1640 1 Processor -- start 2024-09-21T04:02:41.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.981+0000 7f9c593e1640 1 -- start start 2024-09-21T04:02:41.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.981+0000 7f9c593e1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c54102aa0 0x7f9c5419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:41.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.981+0000 7f9c593e1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c541083e0 0x7f9c5419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:41.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.981+0000 7f9c593e1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c54108d30 0x7f9c5419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:41.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.981+0000 7f9c593e1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9c541129a0 con 0x7f9c54108d30 2024-09-21T04:02:41.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.981+0000 7f9c593e1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9c54112820 con 0x7f9c54102aa0 2024-09-21T04:02:41.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.981+0000 7f9c593e1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9c54112b20 con 0x7f9c541083e0 2024-09-21T04:02:41.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.982+0000 7f9c537fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c54108d30 0x7f9c5419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:41.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.982+0000 7f9c537fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c54108d30 0x7f9c5419fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36600/0 (socket says 172.21.15.64:36600) 2024-09-21T04:02:41.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.982+0000 7f9c537fe640 1 -- 172.21.15.64:0/1850308009 learned_addr learned my addr 172.21.15.64:0/1850308009 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:41.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.982+0000 7f9c4bfff640 1 --2- 172.21.15.64:0/1850308009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c541083e0 0x7f9c5419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:41.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.982+0000 7f9c52ffd640 1 --2- 172.21.15.64:0/1850308009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c54102aa0 0x7f9c5419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:41.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.982+0000 7f9c537fe640 1 -- 172.21.15.64:0/1850308009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c541083e0 msgr2=0x7f9c5419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:41.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.982+0000 7f9c537fe640 1 --2- 172.21.15.64:0/1850308009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c541083e0 0x7f9c5419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:41.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.982+0000 7f9c537fe640 1 -- 172.21.15.64:0/1850308009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c54102aa0 msgr2=0x7f9c5419a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:41.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.982+0000 7f9c537fe640 1 --2- 172.21.15.64:0/1850308009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c54102aa0 0x7f9c5419a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:41.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.982+0000 7f9c537fe640 1 -- 172.21.15.64:0/1850308009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c541a0470 con 0x7f9c54108d30 2024-09-21T04:02:41.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.982+0000 7f9c52ffd640 1 --2- 172.21.15.64:0/1850308009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9c54102aa0 0x7f9c5419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:02:41.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.982+0000 7f9c537fe640 1 --2- 172.21.15.64:0/1850308009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c54108d30 0x7f9c5419fd40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00eaa0 tx=0x7f9c4c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:41.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.982+0000 7f9c50ff9640 1 -- 172.21.15.64:0/1850308009 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c00cdb0 con 0x7f9c54108d30 2024-09-21T04:02:41.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.982+0000 7f9c50ff9640 1 -- 172.21.15.64:0/1850308009 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c4c005020 con 0x7f9c54108d30 2024-09-21T04:02:41.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.982+0000 7f9c593e1640 1 -- 172.21.15.64:0/1850308009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c541a0760 con 0x7f9c54108d30 2024-09-21T04:02:41.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.982+0000 7f9c50ff9640 1 -- 172.21.15.64:0/1850308009 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c0116d0 con 0x7f9c54108d30 2024-09-21T04:02:41.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.982+0000 7f9c593e1640 1 -- 172.21.15.64:0/1850308009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c541a6ec0 con 0x7f9c54108d30 2024-09-21T04:02:41.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.983+0000 7f9c593e1640 1 -- 172.21.15.64:0/1850308009 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9c541074e0 con 0x7f9c54108d30 2024-09-21T04:02:41.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.985+0000 7f9c50ff9640 1 -- 172.21.15.64:0/1850308009 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f9c4c0051d0 con 0x7f9c54108d30 2024-09-21T04:02:41.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.986+0000 7f9c50ff9640 1 --2- 172.21.15.64:0/1850308009 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f9c400b65f0 0x7f9c400b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:41.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.986+0000 7f9c52ffd640 1 --2- 172.21.15.64:0/1850308009 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f9c400b65f0 0x7f9c400b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:41.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.986+0000 7f9c50ff9640 1 -- 172.21.15.64:0/1850308009 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f9c4c0e7cd0 con 0x7f9c54108d30 2024-09-21T04:02:41.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.986+0000 7f9c50ff9640 1 --2- 172.21.15.64:0/1850308009 >> [v2:172.21.15.64:6810/2022952729,v1:172.21.15.64:6811/2022952729] conn(0x7f9c400c0460 0x7f9c400c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:41.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.986+0000 7f9c50ff9640 1 -- 172.21.15.64:0/1850308009 --> [v2:172.21.15.64:6810/2022952729,v1:172.21.15.64:6811/2022952729] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9c4c014ec0 con 0x7f9c400c0460 2024-09-21T04:02:41.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.986+0000 7f9c50ff9640 1 -- 172.21.15.64:0/1850308009 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f9c4c0e8120 con 0x7f9c54108d30 2024-09-21T04:02:41.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.986+0000 7f9c4bfff640 1 --2- 172.21.15.64:0/1850308009 >> [v2:172.21.15.64:6810/2022952729,v1:172.21.15.64:6811/2022952729] conn(0x7f9c400c0460 0x7f9c400c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:41.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.986+0000 7f9c52ffd640 1 --2- 172.21.15.64:0/1850308009 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f9c400b65f0 0x7f9c400b8b00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9c44031440 tx=0x7f9c44009680 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:41.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.987+0000 7f9c4bfff640 1 --2- 172.21.15.64:0/1850308009 >> [v2:172.21.15.64:6810/2022952729,v1:172.21.15.64:6811/2022952729] conn(0x7f9c400c0460 0x7f9c400c2950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:41.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:40.987+0000 7f9c50ff9640 1 -- 172.21.15.64:0/1850308009 <== osd.3 v2:172.21.15.64:6810/2022952729 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9c4c014ec0 con 0x7f9c400c0460 2024-09-21T04:02:41.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:41.001+0000 7f9c593e1640 1 -- 172.21.15.64:0/1850308009 --> [v2:172.21.15.64:6810/2022952729,v1:172.21.15.64:6811/2022952729] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9c54103d20 con 0x7f9c400c0460 2024-09-21T04:02:41.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:41.002+0000 7f9c50ff9640 1 -- 172.21.15.64:0/1850308009 <== osd.3 v2:172.21.15.64:6810/2022952729 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9c54103d20 con 0x7f9c400c0460 2024-09-21T04:02:41.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:41.002+0000 7f9c593e1640 1 -- 172.21.15.64:0/1850308009 >> [v2:172.21.15.64:6810/2022952729,v1:172.21.15.64:6811/2022952729] conn(0x7f9c400c0460 msgr2=0x7f9c400c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:41.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:41.002+0000 7f9c593e1640 1 --2- 172.21.15.64:0/1850308009 >> [v2:172.21.15.64:6810/2022952729,v1:172.21.15.64:6811/2022952729] conn(0x7f9c400c0460 0x7f9c400c2950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:41.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:41.002+0000 7f9c593e1640 1 -- 172.21.15.64:0/1850308009 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f9c400b65f0 msgr2=0x7f9c400b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:41.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:41.002+0000 7f9c593e1640 1 --2- 172.21.15.64:0/1850308009 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f9c400b65f0 0x7f9c400b8b00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9c44031440 tx=0x7f9c44009680 comp rx=0 tx=0).stop 2024-09-21T04:02:41.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:41.002+0000 7f9c593e1640 1 -- 172.21.15.64:0/1850308009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c54108d30 msgr2=0x7f9c5419fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:41.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:41.002+0000 7f9c593e1640 1 --2- 172.21.15.64:0/1850308009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c54108d30 0x7f9c5419fd40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00eaa0 tx=0x7f9c4c00ef60 comp rx=0 tx=0).stop 2024-09-21T04:02:41.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:41.002+0000 7f9c537fe640 1 -- 172.21.15.64:0/1850308009 reap_dead start 2024-09-21T04:02:41.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:41.002+0000 7f9c593e1640 1 -- 172.21.15.64:0/1850308009 shutdown_connections 2024-09-21T04:02:41.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:41.002+0000 7f9c593e1640 1 -- 172.21.15.64:0/1850308009 >> 172.21.15.64:0/1850308009 conn(0x7f9c540797e0 msgr2=0x7f9c54100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:41.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:41.003+0000 7f9c593e1640 1 -- 172.21.15.64:0/1850308009 shutdown_connections 2024-09-21T04:02:41.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:41.003+0000 7f9c593e1640 1 -- 172.21.15.64:0/1850308009 wait complete. 2024-09-21T04:02:41.754 INFO:teuthology.orchestra.run.smithi064.stdout:107374182462 2024-09-21T04:02:41.755 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-21T04:02:41.833 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:41.836 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:41.836 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:41.844 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:41.844 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:41.850 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:41.855 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:41.855 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:41.859 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:41.861 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:41.890 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:41.895 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:41 smithi064 ceph-mon[31180]: pgmap v279: 97 pgs: 97 active+clean; 644 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 6 op/s; 54 B/s, 0 objects/s recovering 2024-09-21T04:02:42.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:41 smithi119 ceph-mon[34811]: pgmap v279: 97 pgs: 97 active+clean; 644 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 6 op/s; 54 B/s, 0 objects/s recovering 2024-09-21T04:02:42.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:41 smithi131 ceph-mon[35193]: pgmap v279: 97 pgs: 97 active+clean; 644 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 6 op/s; 54 B/s, 0 objects/s recovering 2024-09-21T04:02:42.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.645+0000 7fbe8971b640 1 -- 172.21.15.64:0/3615405918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe84102a60 msgr2=0x7fbe84102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:42.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.645+0000 7fbe8971b640 1 --2- 172.21.15.64:0/3615405918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe84102a60 0x7fbe84102e70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c009900 tx=0x7fbe6c02a410 comp rx=0 tx=0).stop 2024-09-21T04:02:42.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.645+0000 7fbe8971b640 1 -- 172.21.15.64:0/3615405918 shutdown_connections 2024-09-21T04:02:42.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.645+0000 7fbe8971b640 1 --2- 172.21.15.64:0/3615405918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe84108cf0 0x7fbe8410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:42.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.645+0000 7fbe8971b640 1 --2- 172.21.15.64:0/3615405918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe841083a0 0x7fbe841087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:42.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.645+0000 7fbe8971b640 1 --2- 172.21.15.64:0/3615405918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe84102a60 0x7fbe84102e70 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:42.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.645+0000 7fbe8971b640 1 -- 172.21.15.64:0/3615405918 >> 172.21.15.64:0/3615405918 conn(0x7fbe84079730 msgr2=0x7fbe84079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:42.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.645+0000 7fbe8971b640 1 -- 172.21.15.64:0/3615405918 shutdown_connections 2024-09-21T04:02:42.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.645+0000 7fbe8971b640 1 -- 172.21.15.64:0/3615405918 wait complete. 2024-09-21T04:02:42.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.646+0000 7fbe8971b640 1 Processor -- start 2024-09-21T04:02:42.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.646+0000 7fbe8971b640 1 -- start start 2024-09-21T04:02:42.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.646+0000 7fbe8971b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe84102a60 0x7fbe84104250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:42.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.646+0000 7fbe8971b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe841083a0 0x7fbe84104790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:42.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.646+0000 7fbe8971b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe84108cf0 0x7fbe84104cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:42.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.646+0000 7fbe8971b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbe84112960 con 0x7fbe84102a60 2024-09-21T04:02:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.646+0000 7fbe8971b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbe841127e0 con 0x7fbe841083a0 2024-09-21T04:02:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.646+0000 7fbe8971b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbe84112ae0 con 0x7fbe84108cf0 2024-09-21T04:02:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.646+0000 7fbe82ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe84102a60 0x7fbe84104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.647+0000 7fbe82ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe84102a60 0x7fbe84104250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59060/0 (socket says 172.21.15.64:59060) 2024-09-21T04:02:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.647+0000 7fbe82ffd640 1 -- 172.21.15.64:0/578540507 learned_addr learned my addr 172.21.15.64:0/578540507 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.647+0000 7fbe7a7fc640 1 --2- 172.21.15.64:0/578540507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe841083a0 0x7fbe84104790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.647+0000 7fbe837fe640 1 --2- 172.21.15.64:0/578540507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe84108cf0 0x7fbe84104cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.647+0000 7fbe82ffd640 1 -- 172.21.15.64:0/578540507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe84108cf0 msgr2=0x7fbe84104cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.647+0000 7fbe82ffd640 1 --2- 172.21.15.64:0/578540507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe84108cf0 0x7fbe84104cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.647+0000 7fbe82ffd640 1 -- 172.21.15.64:0/578540507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe841083a0 msgr2=0x7fbe84104790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.647+0000 7fbe82ffd640 1 --2- 172.21.15.64:0/578540507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe841083a0 0x7fbe84104790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.647+0000 7fbe82ffd640 1 -- 172.21.15.64:0/578540507 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe841a8700 con 0x7fbe84102a60 2024-09-21T04:02:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.647+0000 7fbe82ffd640 1 --2- 172.21.15.64:0/578540507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe84102a60 0x7fbe84104250 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c005940 tx=0x7fbe6c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.647+0000 7fbe80ff9640 1 -- 172.21.15.64:0/578540507 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c03f030 con 0x7fbe84102a60 2024-09-21T04:02:42.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.647+0000 7fbe8971b640 1 -- 172.21.15.64:0/578540507 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe841a8990 con 0x7fbe84102a60 2024-09-21T04:02:42.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.647+0000 7fbe80ff9640 1 -- 172.21.15.64:0/578540507 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe6c003cb0 con 0x7fbe84102a60 2024-09-21T04:02:42.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.647+0000 7fbe80ff9640 1 -- 172.21.15.64:0/578540507 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c035070 con 0x7fbe84102a60 2024-09-21T04:02:42.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.647+0000 7fbe8971b640 1 -- 172.21.15.64:0/578540507 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe841a8ec0 con 0x7fbe84102a60 2024-09-21T04:02:42.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.648+0000 7fbe8971b640 1 -- 172.21.15.64:0/578540507 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbe84106630 con 0x7fbe84102a60 2024-09-21T04:02:42.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.650+0000 7fbe80ff9640 1 -- 172.21.15.64:0/578540507 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fbe6c042030 con 0x7fbe84102a60 2024-09-21T04:02:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.651+0000 7fbe80ff9640 1 --2- 172.21.15.64:0/578540507 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbe640b65f0 0x7fbe640b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.651+0000 7fbe7a7fc640 1 --2- 172.21.15.64:0/578540507 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbe640b65f0 0x7fbe640b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.651+0000 7fbe80ff9640 1 -- 172.21.15.64:0/578540507 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7fbe6c107f20 con 0x7fbe84102a60 2024-09-21T04:02:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.651+0000 7fbe80ff9640 1 --2- 172.21.15.64:0/578540507 >> [v2:172.21.15.119:6816/2567568166,v1:172.21.15.119:6817/2567568166] conn(0x7fbe640c0460 0x7fbe640c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.651+0000 7fbe80ff9640 1 -- 172.21.15.64:0/578540507 --> [v2:172.21.15.119:6816/2567568166,v1:172.21.15.119:6817/2567568166] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbe640c3020 con 0x7fbe640c0460 2024-09-21T04:02:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.651+0000 7fbe80ff9640 1 -- 172.21.15.64:0/578540507 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fbe6c108370 con 0x7fbe84102a60 2024-09-21T04:02:42.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.651+0000 7fbe837fe640 1 --2- 172.21.15.64:0/578540507 >> [v2:172.21.15.119:6816/2567568166,v1:172.21.15.119:6817/2567568166] conn(0x7fbe640c0460 0x7fbe640c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:42.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.651+0000 7fbe7a7fc640 1 --2- 172.21.15.64:0/578540507 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbe640b65f0 0x7fbe640b8b00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbe84100cc0 tx=0x7fbe7400b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:42.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.652+0000 7fbe837fe640 1 --2- 172.21.15.64:0/578540507 >> [v2:172.21.15.119:6816/2567568166,v1:172.21.15.119:6817/2567568166] conn(0x7fbe640c0460 0x7fbe640c2950 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:42.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.653+0000 7fbe80ff9640 1 -- 172.21.15.64:0/578540507 <== osd.7 v2:172.21.15.119:6816/2567568166 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbe640c3020 con 0x7fbe640c0460 2024-09-21T04:02:42.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.666+0000 7fbe8971b640 1 -- 172.21.15.64:0/578540507 --> [v2:172.21.15.119:6816/2567568166,v1:172.21.15.119:6817/2567568166] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbe84103ce0 con 0x7fbe640c0460 2024-09-21T04:02:42.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.667+0000 7fbe80ff9640 1 -- 172.21.15.64:0/578540507 <== osd.7 v2:172.21.15.119:6816/2567568166 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbe84103ce0 con 0x7fbe640c0460 2024-09-21T04:02:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.667+0000 7fbe8971b640 1 -- 172.21.15.64:0/578540507 >> [v2:172.21.15.119:6816/2567568166,v1:172.21.15.119:6817/2567568166] conn(0x7fbe640c0460 msgr2=0x7fbe640c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.667+0000 7fbe8971b640 1 --2- 172.21.15.64:0/578540507 >> [v2:172.21.15.119:6816/2567568166,v1:172.21.15.119:6817/2567568166] conn(0x7fbe640c0460 0x7fbe640c2950 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.667+0000 7fbe8971b640 1 -- 172.21.15.64:0/578540507 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbe640b65f0 msgr2=0x7fbe640b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.667+0000 7fbe8971b640 1 --2- 172.21.15.64:0/578540507 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbe640b65f0 0x7fbe640b8b00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbe84100cc0 tx=0x7fbe7400b390 comp rx=0 tx=0).stop 2024-09-21T04:02:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.667+0000 7fbe8971b640 1 -- 172.21.15.64:0/578540507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe84102a60 msgr2=0x7fbe84104250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.667+0000 7fbe8971b640 1 --2- 172.21.15.64:0/578540507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe84102a60 0x7fbe84104250 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c005940 tx=0x7fbe6c0042f0 comp rx=0 tx=0).stop 2024-09-21T04:02:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.667+0000 7fbe837fe640 1 -- 172.21.15.64:0/578540507 reap_dead start 2024-09-21T04:02:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.668+0000 7fbe8971b640 1 -- 172.21.15.64:0/578540507 shutdown_connections 2024-09-21T04:02:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.668+0000 7fbe8971b640 1 -- 172.21.15.64:0/578540507 >> 172.21.15.64:0/578540507 conn(0x7fbe84079730 msgr2=0x7fbe84101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.668+0000 7fbe8971b640 1 -- 172.21.15.64:0/578540507 shutdown_connections 2024-09-21T04:02:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.668+0000 7fbe8971b640 1 -- 172.21.15.64:0/578540507 wait complete. 2024-09-21T04:02:43.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.837+0000 7f5547741640 1 -- 172.21.15.64:0/19345017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5540108d10 msgr2=0x7f554010f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:43.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.837+0000 7f5547741640 1 --2- 172.21.15.64:0/19345017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5540108d10 0x7f554010f540 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5538009ca0 tx=0x7f553802a8c0 comp rx=0 tx=0).stop 2024-09-21T04:02:43.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.837+0000 7f5547741640 1 -- 172.21.15.64:0/19345017 shutdown_connections 2024-09-21T04:02:43.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.837+0000 7f5547741640 1 --2- 172.21.15.64:0/19345017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5540108d10 0x7f554010f540 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:43.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.837+0000 7f5547741640 1 --2- 172.21.15.64:0/19345017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55401083c0 0x7f55401087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:43.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.837+0000 7f5547741640 1 --2- 172.21.15.64:0/19345017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5540102a80 0x7f5540102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:43.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.837+0000 7f5547741640 1 -- 172.21.15.64:0/19345017 >> 172.21.15.64:0/19345017 conn(0x7f5540079740 msgr2=0x7f5540079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:43.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.837+0000 7f5547741640 1 -- 172.21.15.64:0/19345017 shutdown_connections 2024-09-21T04:02:43.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.837+0000 7f5547741640 1 -- 172.21.15.64:0/19345017 wait complete. 2024-09-21T04:02:43.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.838+0000 7f5547741640 1 Processor -- start 2024-09-21T04:02:43.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.838+0000 7f5547741640 1 -- start start 2024-09-21T04:02:43.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.838+0000 7f5547741640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5540102a80 0x7f55401982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:43.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.838+0000 7f5547741640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55401083c0 0x7f5540198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:43.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.838+0000 7f5547741640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5540108d10 0x7f554019db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:43.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.838+0000 7f5547741640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5540111880 con 0x7f55401083c0 2024-09-21T04:02:43.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.838+0000 7f5547741640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5540111700 con 0x7f5540102a80 2024-09-21T04:02:43.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.838+0000 7f5547741640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5540111a00 con 0x7f5540108d10 2024-09-21T04:02:43.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.838+0000 7f5544cdc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55401083c0 0x7f5540198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:43.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.838+0000 7f5544cdc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55401083c0 0x7f5540198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59076/0 (socket says 172.21.15.64:59076) 2024-09-21T04:02:43.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.838+0000 7f5544cdc640 1 -- 172.21.15.64:0/5810017 learned_addr learned my addr 172.21.15.64:0/5810017 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:43.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.839+0000 7f55454dd640 1 --2- 172.21.15.64:0/5810017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5540102a80 0x7f55401982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:43.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.839+0000 7f5545cde640 1 --2- 172.21.15.64:0/5810017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5540108d10 0x7f554019db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:43.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.839+0000 7f5544cdc640 1 -- 172.21.15.64:0/5810017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5540108d10 msgr2=0x7f554019db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:43.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.839+0000 7f5544cdc640 1 --2- 172.21.15.64:0/5810017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5540108d10 0x7f554019db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:43.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.839+0000 7f5544cdc640 1 -- 172.21.15.64:0/5810017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5540102a80 msgr2=0x7f55401982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:43.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.839+0000 7f5544cdc640 1 --2- 172.21.15.64:0/5810017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5540102a80 0x7f55401982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:43.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.839+0000 7f5544cdc640 1 -- 172.21.15.64:0/5810017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554019e240 con 0x7f55401083c0 2024-09-21T04:02:43.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.839+0000 7f5544cdc640 1 --2- 172.21.15.64:0/5810017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55401083c0 0x7f5540198870 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f553400d770 tx=0x7f553400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:43.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.839+0000 7f55327fc640 1 -- 172.21.15.64:0/5810017 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5534017070 con 0x7f55401083c0 2024-09-21T04:02:43.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.839+0000 7f55327fc640 1 -- 172.21.15.64:0/5810017 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5534009c30 con 0x7f55401083c0 2024-09-21T04:02:43.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.839+0000 7f55327fc640 1 -- 172.21.15.64:0/5810017 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553400f620 con 0x7f55401083c0 2024-09-21T04:02:43.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.839+0000 7f5547741640 1 -- 172.21.15.64:0/5810017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f554019e530 con 0x7f55401083c0 2024-09-21T04:02:43.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.839+0000 7f5547741640 1 -- 172.21.15.64:0/5810017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55401a4c90 con 0x7f55401083c0 2024-09-21T04:02:43.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.840+0000 7f5547741640 1 -- 172.21.15.64:0/5810017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5540063b50 con 0x7f55401083c0 2024-09-21T04:02:43.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.842+0000 7f55327fc640 1 -- 172.21.15.64:0/5810017 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f553401a070 con 0x7f55401083c0 2024-09-21T04:02:43.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.842+0000 7f55327fc640 1 --2- 172.21.15.64:0/5810017 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f552c0b8880 0x7f552c0bad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:43.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.843+0000 7f55454dd640 1 --2- 172.21.15.64:0/5810017 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f552c0b8880 0x7f552c0bad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:43.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.843+0000 7f55327fc640 1 -- 172.21.15.64:0/5810017 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f55340e74d0 con 0x7f55401083c0 2024-09-21T04:02:43.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.843+0000 7f55327fc640 1 --2- 172.21.15.64:0/5810017 >> [v2:172.21.15.131:6800/3422924179,v1:172.21.15.131:6801/3422924179] conn(0x7f552c0c26f0 0x7f552c0c4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:43.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.843+0000 7f55327fc640 1 -- 172.21.15.64:0/5810017 --> [v2:172.21.15.131:6800/3422924179,v1:172.21.15.131:6801/3422924179] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5534010920 con 0x7f552c0c26f0 2024-09-21T04:02:43.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.843+0000 7f55327fc640 1 -- 172.21.15.64:0/5810017 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f55340e7920 con 0x7f55401083c0 2024-09-21T04:02:43.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.843+0000 7f55454dd640 1 --2- 172.21.15.64:0/5810017 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f552c0b8880 0x7f552c0bad90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f553c005fd0 tx=0x7f553c005e20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:43.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.843+0000 7f5545cde640 1 --2- 172.21.15.64:0/5810017 >> [v2:172.21.15.131:6800/3422924179,v1:172.21.15.131:6801/3422924179] conn(0x7f552c0c26f0 0x7f552c0c4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:43.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.844+0000 7f5545cde640 1 --2- 172.21.15.64:0/5810017 >> [v2:172.21.15.131:6800/3422924179,v1:172.21.15.131:6801/3422924179] conn(0x7f552c0c26f0 0x7f552c0c4be0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:43.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.844+0000 7f55327fc640 1 -- 172.21.15.64:0/5810017 <== osd.2 v2:172.21.15.131:6800/3422924179 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5534010920 con 0x7f552c0c26f0 2024-09-21T04:02:43.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.858+0000 7f5547741640 1 -- 172.21.15.64:0/5810017 --> [v2:172.21.15.131:6800/3422924179,v1:172.21.15.131:6801/3422924179] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5540063f50 con 0x7f552c0c26f0 2024-09-21T04:02:43.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.858+0000 7f55327fc640 1 -- 172.21.15.64:0/5810017 <== osd.2 v2:172.21.15.131:6800/3422924179 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5540063f50 con 0x7f552c0c26f0 2024-09-21T04:02:43.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.859+0000 7f5547741640 1 -- 172.21.15.64:0/5810017 >> [v2:172.21.15.131:6800/3422924179,v1:172.21.15.131:6801/3422924179] conn(0x7f552c0c26f0 msgr2=0x7f552c0c4be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:43.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.859+0000 7f5547741640 1 --2- 172.21.15.64:0/5810017 >> [v2:172.21.15.131:6800/3422924179,v1:172.21.15.131:6801/3422924179] conn(0x7f552c0c26f0 0x7f552c0c4be0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:43.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.859+0000 7f5547741640 1 -- 172.21.15.64:0/5810017 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f552c0b8880 msgr2=0x7f552c0bad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:43.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.859+0000 7f5547741640 1 --2- 172.21.15.64:0/5810017 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f552c0b8880 0x7f552c0bad90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f553c005fd0 tx=0x7f553c005e20 comp rx=0 tx=0).stop 2024-09-21T04:02:43.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.859+0000 7f5547741640 1 -- 172.21.15.64:0/5810017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55401083c0 msgr2=0x7f5540198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:43.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.859+0000 7f5547741640 1 --2- 172.21.15.64:0/5810017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55401083c0 0x7f5540198870 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f553400d770 tx=0x7f553400dc30 comp rx=0 tx=0).stop 2024-09-21T04:02:43.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.859+0000 7f5545cde640 1 -- 172.21.15.64:0/5810017 reap_dead start 2024-09-21T04:02:43.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.859+0000 7f5547741640 1 -- 172.21.15.64:0/5810017 shutdown_connections 2024-09-21T04:02:43.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.859+0000 7f5547741640 1 -- 172.21.15.64:0/5810017 >> 172.21.15.64:0/5810017 conn(0x7f5540079740 msgr2=0x7f5540104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:43.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.859+0000 7f5547741640 1 -- 172.21.15.64:0/5810017 shutdown_connections 2024-09-21T04:02:43.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:42.859+0000 7f5547741640 1 -- 172.21.15.64:0/5810017 wait complete. 2024-09-21T04:02:44.047 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:43 smithi064 ceph-mon[31180]: pgmap v280: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 8 op/s; 48 B/s, 0 objects/s recovering 2024-09-21T04:02:44.258 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:43 smithi119 ceph-mon[34811]: pgmap v280: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 8 op/s; 48 B/s, 0 objects/s recovering 2024-09-21T04:02:44.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:43 smithi131 ceph-mon[35193]: pgmap v280: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 8 op/s; 48 B/s, 0 objects/s recovering 2024-09-21T04:02:44.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.483+0000 7f77fce08640 1 -- 172.21.15.64:0/3846418419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77f810de10 msgr2=0x7f77f8110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:44.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.483+0000 7f77fce08640 1 --2- 172.21.15.64:0/3846418419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77f810de10 0x7f77f8110300 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009ca0 tx=0x7f77f002a8c0 comp rx=0 tx=0).stop 2024-09-21T04:02:44.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.483+0000 7f77fce08640 1 -- 172.21.15.64:0/3846418419 shutdown_connections 2024-09-21T04:02:44.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.483+0000 7f77fce08640 1 --2- 172.21.15.64:0/3846418419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77f810de10 0x7f77f8110300 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:44.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.483+0000 7f77fce08640 1 --2- 172.21.15.64:0/3846418419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77f8105300 0x7f77f810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:44.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.483+0000 7f77fce08640 1 --2- 172.21.15.64:0/3846418419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77f81049b0 0x7f77f8104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:44.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.483+0000 7f77fce08640 1 -- 172.21.15.64:0/3846418419 >> 172.21.15.64:0/3846418419 conn(0x7f77f8079730 msgr2=0x7f77f8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:44.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.484+0000 7f77fce08640 1 -- 172.21.15.64:0/3846418419 shutdown_connections 2024-09-21T04:02:44.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.484+0000 7f77fce08640 1 -- 172.21.15.64:0/3846418419 wait complete. 2024-09-21T04:02:44.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.485+0000 7f77fce08640 1 Processor -- start 2024-09-21T04:02:44.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.485+0000 7f77fce08640 1 -- start start 2024-09-21T04:02:44.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.485+0000 7f77fce08640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77f81049b0 0x7f77f819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:44.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.485+0000 7f77fce08640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77f8105300 0x7f77f819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:44.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.485+0000 7f77fce08640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77f810de10 0x7f77f819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:44.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.485+0000 7f77fce08640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f77f8113a90 con 0x7f77f810de10 2024-09-21T04:02:44.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.485+0000 7f77fce08640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f77f8113910 con 0x7f77f81049b0 2024-09-21T04:02:44.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.485+0000 7f77fce08640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f77f8113c10 con 0x7f77f8105300 2024-09-21T04:02:44.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.485+0000 7f77f6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77f810de10 0x7f77f819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:44.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.485+0000 7f77f6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77f810de10 0x7f77f819fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59114/0 (socket says 172.21.15.64:59114) 2024-09-21T04:02:44.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.485+0000 7f77f6d9d640 1 -- 172.21.15.64:0/540679228 learned_addr learned my addr 172.21.15.64:0/540679228 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:44.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.485+0000 7f77effff640 1 --2- 172.21.15.64:0/540679228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77f8105300 0x7f77f819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:44.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.485+0000 7f77f659c640 1 --2- 172.21.15.64:0/540679228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77f81049b0 0x7f77f819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:44.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.485+0000 7f77f6d9d640 1 -- 172.21.15.64:0/540679228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77f8105300 msgr2=0x7f77f819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:44.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.485+0000 7f77f6d9d640 1 --2- 172.21.15.64:0/540679228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77f8105300 0x7f77f819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:44.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.485+0000 7f77f6d9d640 1 -- 172.21.15.64:0/540679228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77f81049b0 msgr2=0x7f77f819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:44.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.485+0000 7f77f6d9d640 1 --2- 172.21.15.64:0/540679228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77f81049b0 0x7f77f819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:44.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.485+0000 7f77f6d9d640 1 -- 172.21.15.64:0/540679228 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f81a0430 con 0x7f77f810de10 2024-09-21T04:02:44.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.485+0000 7f77effff640 1 --2- 172.21.15.64:0/540679228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77f8105300 0x7f77f819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:02:44.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.486+0000 7f77f6d9d640 1 --2- 172.21.15.64:0/540679228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77f810de10 0x7f77f819fd00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f77f00098d0 tx=0x7f77f00317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:44.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.486+0000 7f77ef7fe640 1 -- 172.21.15.64:0/540679228 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0003b70 con 0x7f77f810de10 2024-09-21T04:02:44.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.486+0000 7f77ef7fe640 1 -- 172.21.15.64:0/540679228 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77f0003d10 con 0x7f77f810de10 2024-09-21T04:02:44.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.486+0000 7f77fce08640 1 -- 172.21.15.64:0/540679228 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f81a06c0 con 0x7f77f810de10 2024-09-21T04:02:44.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.486+0000 7f77ef7fe640 1 -- 172.21.15.64:0/540679228 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f002fa20 con 0x7f77f810de10 2024-09-21T04:02:44.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.486+0000 7f77fce08640 1 -- 172.21.15.64:0/540679228 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f81a6e80 con 0x7f77f810de10 2024-09-21T04:02:44.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.487+0000 7f77fce08640 1 -- 172.21.15.64:0/540679228 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f77f8100f80 con 0x7f77f810de10 2024-09-21T04:02:44.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.488+0000 7f77ef7fe640 1 -- 172.21.15.64:0/540679228 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f77f002fbc0 con 0x7f77f810de10 2024-09-21T04:02:44.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.489+0000 7f77ef7fe640 1 --2- 172.21.15.64:0/540679228 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f77e40b65f0 0x7f77e40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:44.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.489+0000 7f77f659c640 1 --2- 172.21.15.64:0/540679228 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f77e40b65f0 0x7f77e40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:44.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.489+0000 7f77ef7fe640 1 -- 172.21.15.64:0/540679228 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f77f0108ae0 con 0x7f77f810de10 2024-09-21T04:02:44.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.490+0000 7f77ef7fe640 1 --2- 172.21.15.64:0/540679228 >> [v2:172.21.15.131:6816/199192217,v1:172.21.15.131:6817/199192217] conn(0x7f77e40c0460 0x7f77e40c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:44.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.490+0000 7f77ef7fe640 1 -- 172.21.15.64:0/540679228 --> [v2:172.21.15.131:6816/199192217,v1:172.21.15.131:6817/199192217] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f77e40c3020 con 0x7f77e40c0460 2024-09-21T04:02:44.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.490+0000 7f77ef7fe640 1 -- 172.21.15.64:0/540679228 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f77f0108f30 con 0x7f77f810de10 2024-09-21T04:02:44.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.490+0000 7f77f659c640 1 --2- 172.21.15.64:0/540679228 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f77e40b65f0 0x7f77e40b8b00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f77f806bc60 tx=0x7f77e8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:44.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.490+0000 7f77effff640 1 --2- 172.21.15.64:0/540679228 >> [v2:172.21.15.131:6816/199192217,v1:172.21.15.131:6817/199192217] conn(0x7f77e40c0460 0x7f77e40c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:44.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.490+0000 7f77effff640 1 --2- 172.21.15.64:0/540679228 >> [v2:172.21.15.131:6816/199192217,v1:172.21.15.131:6817/199192217] conn(0x7f77e40c0460 0x7f77e40c2950 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:44.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.491+0000 7f77ef7fe640 1 -- 172.21.15.64:0/540679228 <== osd.8 v2:172.21.15.131:6816/199192217 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f77e40c3020 con 0x7f77e40c0460 2024-09-21T04:02:44.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.505+0000 7f77fce08640 1 -- 172.21.15.64:0/540679228 --> [v2:172.21.15.131:6816/199192217,v1:172.21.15.131:6817/199192217] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f77f806b560 con 0x7f77e40c0460 2024-09-21T04:02:44.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.506+0000 7f77ef7fe640 1 -- 172.21.15.64:0/540679228 <== osd.8 v2:172.21.15.131:6816/199192217 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f77f806b560 con 0x7f77e40c0460 2024-09-21T04:02:44.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.506+0000 7f77fce08640 1 -- 172.21.15.64:0/540679228 >> [v2:172.21.15.131:6816/199192217,v1:172.21.15.131:6817/199192217] conn(0x7f77e40c0460 msgr2=0x7f77e40c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:44.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.506+0000 7f77fce08640 1 --2- 172.21.15.64:0/540679228 >> [v2:172.21.15.131:6816/199192217,v1:172.21.15.131:6817/199192217] conn(0x7f77e40c0460 0x7f77e40c2950 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:44.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.506+0000 7f77fce08640 1 -- 172.21.15.64:0/540679228 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f77e40b65f0 msgr2=0x7f77e40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:44.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.506+0000 7f77fce08640 1 --2- 172.21.15.64:0/540679228 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f77e40b65f0 0x7f77e40b8b00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f77f806bc60 tx=0x7f77e8005f90 comp rx=0 tx=0).stop 2024-09-21T04:02:44.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.506+0000 7f77fce08640 1 -- 172.21.15.64:0/540679228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77f810de10 msgr2=0x7f77f819fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:44.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.506+0000 7f77fce08640 1 --2- 172.21.15.64:0/540679228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77f810de10 0x7f77f819fd00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f77f00098d0 tx=0x7f77f00317b0 comp rx=0 tx=0).stop 2024-09-21T04:02:44.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.506+0000 7f77f6d9d640 1 -- 172.21.15.64:0/540679228 reap_dead start 2024-09-21T04:02:44.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.506+0000 7f77fce08640 1 -- 172.21.15.64:0/540679228 shutdown_connections 2024-09-21T04:02:44.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.506+0000 7f77fce08640 1 -- 172.21.15.64:0/540679228 >> 172.21.15.64:0/540679228 conn(0x7f77f8079730 msgr2=0x7f77f80ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:44.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.506+0000 7f77fce08640 1 -- 172.21.15.64:0/540679228 shutdown_connections 2024-09-21T04:02:44.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.506+0000 7f77fce08640 1 -- 172.21.15.64:0/540679228 wait complete. 2024-09-21T04:02:44.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.413+0000 7f692b104640 1 -- 172.21.15.64:0/4067055879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6924102a60 msgr2=0x7f6924102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:44.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.413+0000 7f692b104640 1 --2- 172.21.15.64:0/4067055879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6924102a60 0x7f6924102e70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6918009900 tx=0x7f691802a410 comp rx=0 tx=0).stop 2024-09-21T04:02:44.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.413+0000 7f692b104640 1 -- 172.21.15.64:0/4067055879 shutdown_connections 2024-09-21T04:02:44.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.413+0000 7f692b104640 1 --2- 172.21.15.64:0/4067055879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6924108cf0 0x7f692410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:44.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.413+0000 7f692b104640 1 --2- 172.21.15.64:0/4067055879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69241083a0 0x7f69241087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:44.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.413+0000 7f692b104640 1 --2- 172.21.15.64:0/4067055879 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6924102a60 0x7f6924102e70 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:44.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.414+0000 7f692b104640 1 -- 172.21.15.64:0/4067055879 >> 172.21.15.64:0/4067055879 conn(0x7f6924079730 msgr2=0x7f6924079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:44.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.414+0000 7f692b104640 1 -- 172.21.15.64:0/4067055879 shutdown_connections 2024-09-21T04:02:44.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.414+0000 7f692b104640 1 -- 172.21.15.64:0/4067055879 wait complete. 2024-09-21T04:02:44.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.414+0000 7f692b104640 1 Processor -- start 2024-09-21T04:02:44.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.415+0000 7f692b104640 1 -- start start 2024-09-21T04:02:44.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.415+0000 7f692b104640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6924102a60 0x7f692419b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:44.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.415+0000 7f692b104640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69241083a0 0x7f692419b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:44.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.415+0000 7f692b104640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6924108cf0 0x7f692419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:44.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.415+0000 7f692b104640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6924112960 con 0x7f69241083a0 2024-09-21T04:02:44.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.415+0000 7f692b104640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f69241127e0 con 0x7f6924108cf0 2024-09-21T04:02:44.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.415+0000 7f692b104640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6924112ae0 con 0x7f6924102a60 2024-09-21T04:02:44.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.415+0000 7f6923fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69241083a0 0x7f692419b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:44.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.415+0000 7f6923fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69241083a0 0x7f692419b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59096/0 (socket says 172.21.15.64:59096) 2024-09-21T04:02:44.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.415+0000 7f6923fff640 1 -- 172.21.15.64:0/3848626767 learned_addr learned my addr 172.21.15.64:0/3848626767 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:44.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.415+0000 7f6928ea0640 1 --2- 172.21.15.64:0/3848626767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6924102a60 0x7f692419b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:44.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.415+0000 7f69296a1640 1 --2- 172.21.15.64:0/3848626767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6924108cf0 0x7f692419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:44.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.415+0000 7f6923fff640 1 -- 172.21.15.64:0/3848626767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6924102a60 msgr2=0x7f692419b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:44.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.415+0000 7f6923fff640 1 --2- 172.21.15.64:0/3848626767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6924102a60 0x7f692419b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:44.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.415+0000 7f6923fff640 1 -- 172.21.15.64:0/3848626767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6924108cf0 msgr2=0x7f692419bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:44.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.415+0000 7f6923fff640 1 --2- 172.21.15.64:0/3848626767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6924108cf0 0x7f692419bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:44.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.416+0000 7f6923fff640 1 -- 172.21.15.64:0/3848626767 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69241a6500 con 0x7f69241083a0 2024-09-21T04:02:44.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.416+0000 7f69296a1640 1 --2- 172.21.15.64:0/3848626767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6924108cf0 0x7f692419bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:02:44.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.416+0000 7f6928ea0640 1 --2- 172.21.15.64:0/3848626767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6924102a60 0x7f692419b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:02:44.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.416+0000 7f6923fff640 1 --2- 172.21.15.64:0/3848626767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69241083a0 0x7f692419b730 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6910005fd0 tx=0x7f691000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:44.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.416+0000 7f6921ffb640 1 -- 172.21.15.64:0/3848626767 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6910017070 con 0x7f69241083a0 2024-09-21T04:02:44.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.416+0000 7f6921ffb640 1 -- 172.21.15.64:0/3848626767 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6910009c30 con 0x7f69241083a0 2024-09-21T04:02:44.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.416+0000 7f692b104640 1 -- 172.21.15.64:0/3848626767 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69241a67f0 con 0x7f69241083a0 2024-09-21T04:02:44.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.416+0000 7f6921ffb640 1 -- 172.21.15.64:0/3848626767 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6910003d30 con 0x7f69241083a0 2024-09-21T04:02:44.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.416+0000 7f692b104640 1 -- 172.21.15.64:0/3848626767 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69241a6d30 con 0x7f69241083a0 2024-09-21T04:02:44.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.417+0000 7f692b104640 1 -- 172.21.15.64:0/3848626767 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6924063b50 con 0x7f69241083a0 2024-09-21T04:02:44.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.419+0000 7f6921ffb640 1 -- 172.21.15.64:0/3848626767 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f6910003ed0 con 0x7f69241083a0 2024-09-21T04:02:44.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.420+0000 7f6921ffb640 1 --2- 172.21.15.64:0/3848626767 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f69140b65f0 0x7f69140b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:44.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.420+0000 7f6928ea0640 1 --2- 172.21.15.64:0/3848626767 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f69140b65f0 0x7f69140b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:44.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.420+0000 7f6921ffb640 1 -- 172.21.15.64:0/3848626767 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f69100e76d0 con 0x7f69241083a0 2024-09-21T04:02:44.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.420+0000 7f6921ffb640 1 --2- 172.21.15.64:0/3848626767 >> [v2:172.21.15.119:6800/3363415998,v1:172.21.15.119:6801/3363415998] conn(0x7f69140c0460 0x7f69140c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:44.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.420+0000 7f6921ffb640 1 -- 172.21.15.64:0/3848626767 --> [v2:172.21.15.119:6800/3363415998,v1:172.21.15.119:6801/3363415998] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6910021e70 con 0x7f69140c0460 2024-09-21T04:02:44.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.420+0000 7f6921ffb640 1 -- 172.21.15.64:0/3848626767 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f69100e7b20 con 0x7f69241083a0 2024-09-21T04:02:44.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.420+0000 7f6928ea0640 1 --2- 172.21.15.64:0/3848626767 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f69140b65f0 0x7f69140b8b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6918000c00 tx=0x7f6918031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:44.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.420+0000 7f69296a1640 1 --2- 172.21.15.64:0/3848626767 >> [v2:172.21.15.119:6800/3363415998,v1:172.21.15.119:6801/3363415998] conn(0x7f69140c0460 0x7f69140c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:44.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.421+0000 7f69296a1640 1 --2- 172.21.15.64:0/3848626767 >> [v2:172.21.15.119:6800/3363415998,v1:172.21.15.119:6801/3363415998] conn(0x7f69140c0460 0x7f69140c2950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:44.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.422+0000 7f6921ffb640 1 -- 172.21.15.64:0/3848626767 <== osd.1 v2:172.21.15.119:6800/3363415998 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6910021e70 con 0x7f69140c0460 2024-09-21T04:02:44.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.436+0000 7f692b104640 1 -- 172.21.15.64:0/3848626767 --> [v2:172.21.15.119:6800/3363415998,v1:172.21.15.119:6801/3363415998] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6924063f50 con 0x7f69140c0460 2024-09-21T04:02:44.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.437+0000 7f6921ffb640 1 -- 172.21.15.64:0/3848626767 <== osd.1 v2:172.21.15.119:6800/3363415998 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6924063f50 con 0x7f69140c0460 2024-09-21T04:02:44.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.437+0000 7f692b104640 1 -- 172.21.15.64:0/3848626767 >> [v2:172.21.15.119:6800/3363415998,v1:172.21.15.119:6801/3363415998] conn(0x7f69140c0460 msgr2=0x7f69140c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:44.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.437+0000 7f692b104640 1 --2- 172.21.15.64:0/3848626767 >> [v2:172.21.15.119:6800/3363415998,v1:172.21.15.119:6801/3363415998] conn(0x7f69140c0460 0x7f69140c2950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:44.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.437+0000 7f692b104640 1 -- 172.21.15.64:0/3848626767 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f69140b65f0 msgr2=0x7f69140b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:44.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.437+0000 7f692b104640 1 --2- 172.21.15.64:0/3848626767 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f69140b65f0 0x7f69140b8b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6918000c00 tx=0x7f6918031400 comp rx=0 tx=0).stop 2024-09-21T04:02:44.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.437+0000 7f692b104640 1 -- 172.21.15.64:0/3848626767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69241083a0 msgr2=0x7f692419b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:44.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.437+0000 7f692b104640 1 --2- 172.21.15.64:0/3848626767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f69241083a0 0x7f692419b730 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6910005fd0 tx=0x7f691000dde0 comp rx=0 tx=0).stop 2024-09-21T04:02:44.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.437+0000 7f69296a1640 1 -- 172.21.15.64:0/3848626767 reap_dead start 2024-09-21T04:02:44.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.437+0000 7f692b104640 1 -- 172.21.15.64:0/3848626767 shutdown_connections 2024-09-21T04:02:44.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.437+0000 7f692b104640 1 -- 172.21.15.64:0/3848626767 >> 172.21.15.64:0/3848626767 conn(0x7f6924079730 msgr2=0x7f6924101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:44.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.437+0000 7f692b104640 1 -- 172.21.15.64:0/3848626767 shutdown_connections 2024-09-21T04:02:44.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:44.437+0000 7f692b104640 1 -- 172.21.15.64:0/3848626767 wait complete. 2024-09-21T04:02:45.420 INFO:teuthology.orchestra.run.smithi064.stdout:60129542216 2024-09-21T04:02:45.420 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-21T04:02:45.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.624+0000 7f808223e640 1 -- 172.21.15.64:0/1615316269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f807c109e30 msgr2=0x7f807c110660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:45.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.624+0000 7f808223e640 1 --2- 172.21.15.64:0/1615316269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f807c109e30 0x7f807c110660 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8074009ca0 tx=0x7f807402a8c0 comp rx=0 tx=0).stop 2024-09-21T04:02:45.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.624+0000 7f808223e640 1 -- 172.21.15.64:0/1615316269 shutdown_connections 2024-09-21T04:02:45.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.624+0000 7f808223e640 1 --2- 172.21.15.64:0/1615316269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f807c109e30 0x7f807c110660 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:45.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.624+0000 7f808223e640 1 --2- 172.21.15.64:0/1615316269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f807c1094e0 0x7f807c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:45.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.624+0000 7f808223e640 1 --2- 172.21.15.64:0/1615316269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f807c103ba0 0x7f807c103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:45.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.624+0000 7f808223e640 1 -- 172.21.15.64:0/1615316269 >> 172.21.15.64:0/1615316269 conn(0x7f807c0ff630 msgr2=0x7f807c101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:45.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.624+0000 7f808223e640 1 -- 172.21.15.64:0/1615316269 shutdown_connections 2024-09-21T04:02:45.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.624+0000 7f808223e640 1 -- 172.21.15.64:0/1615316269 wait complete. 2024-09-21T04:02:45.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.625+0000 7f808223e640 1 Processor -- start 2024-09-21T04:02:45.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.625+0000 7f808223e640 1 -- start start 2024-09-21T04:02:45.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.625+0000 7f808223e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f807c103ba0 0x7f807c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:45.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.625+0000 7f808223e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f807c1094e0 0x7f807c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:45.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.625+0000 7f808223e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f807c109e30 0x7f807c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:45.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.625+0000 7f808223e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f807c113af0 con 0x7f807c103ba0 2024-09-21T04:02:45.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.625+0000 7f808223e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f807c113970 con 0x7f807c1094e0 2024-09-21T04:02:45.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.625+0000 7f808223e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f807c113c70 con 0x7f807c109e30 2024-09-21T04:02:45.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.625+0000 7f807b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f807c103ba0 0x7f807c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:45.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.625+0000 7f807b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f807c103ba0 0x7f807c19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59130/0 (socket says 172.21.15.64:59130) 2024-09-21T04:02:45.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.625+0000 7f807b7fe640 1 -- 172.21.15.64:0/3474216223 learned_addr learned my addr 172.21.15.64:0/3474216223 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:45.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.626+0000 7f807bfff640 1 --2- 172.21.15.64:0/3474216223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f807c109e30 0x7f807c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:45.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.626+0000 7f807b7fe640 1 -- 172.21.15.64:0/3474216223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f807c109e30 msgr2=0x7f807c19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:45.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.626+0000 7f807b7fe640 1 --2- 172.21.15.64:0/3474216223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f807c109e30 0x7f807c19fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:45.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.626+0000 7f807b7fe640 1 -- 172.21.15.64:0/3474216223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f807c1094e0 msgr2=0x7f807c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:45.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.626+0000 7f8073fff640 1 --2- 172.21.15.64:0/3474216223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f807c1094e0 0x7f807c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:45.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.626+0000 7f807b7fe640 1 --2- 172.21.15.64:0/3474216223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f807c1094e0 0x7f807c19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:45.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.626+0000 7f807b7fe640 1 -- 172.21.15.64:0/3474216223 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f807c1a03d0 con 0x7f807c103ba0 2024-09-21T04:02:45.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.626+0000 7f807bfff640 1 --2- 172.21.15.64:0/3474216223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f807c109e30 0x7f807c19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:02:45.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.626+0000 7f807b7fe640 1 --2- 172.21.15.64:0/3474216223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f807c103ba0 0x7f807c19a460 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f806c00e9f0 tx=0x7f806c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:45.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.626+0000 7f80797fa640 1 -- 172.21.15.64:0/3474216223 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c00cd80 con 0x7f807c103ba0 2024-09-21T04:02:45.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.626+0000 7f808223e640 1 -- 172.21.15.64:0/3474216223 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f807c1a06c0 con 0x7f807c103ba0 2024-09-21T04:02:45.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.626+0000 7f80797fa640 1 -- 172.21.15.64:0/3474216223 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f806c0054d0 con 0x7f807c103ba0 2024-09-21T04:02:45.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.626+0000 7f80797fa640 1 -- 172.21.15.64:0/3474216223 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c010690 con 0x7f807c103ba0 2024-09-21T04:02:45.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.626+0000 7f808223e640 1 -- 172.21.15.64:0/3474216223 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f807c1a6e20 con 0x7f807c103ba0 2024-09-21T04:02:45.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.627+0000 7f808223e640 1 -- 172.21.15.64:0/3474216223 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f807c063b10 con 0x7f807c103ba0 2024-09-21T04:02:45.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.629+0000 7f80797fa640 1 -- 172.21.15.64:0/3474216223 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f806c004e30 con 0x7f807c103ba0 2024-09-21T04:02:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.630+0000 7f80797fa640 1 --2- 172.21.15.64:0/3474216223 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f80680b65f0 0x7f80680b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.630+0000 7f8073fff640 1 --2- 172.21.15.64:0/3474216223 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f80680b65f0 0x7f80680b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.630+0000 7f80797fa640 1 -- 172.21.15.64:0/3474216223 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f806c0e7f60 con 0x7f807c103ba0 2024-09-21T04:02:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.630+0000 7f80797fa640 1 --2- 172.21.15.64:0/3474216223 >> [v2:172.21.15.131:6824/2294059084,v1:172.21.15.131:6825/2294059084] conn(0x7f80680c0460 0x7f80680c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.630+0000 7f80797fa640 1 -- 172.21.15.64:0/3474216223 --> [v2:172.21.15.131:6824/2294059084,v1:172.21.15.131:6825/2294059084] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f806c015e80 con 0x7f80680c0460 2024-09-21T04:02:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.630+0000 7f80797fa640 1 -- 172.21.15.64:0/3474216223 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f806c0e83b0 con 0x7f807c103ba0 2024-09-21T04:02:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.630+0000 7f8073fff640 1 --2- 172.21.15.64:0/3474216223 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f80680b65f0 0x7f80680b8b00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8064005fd0 tx=0x7f8064005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.631+0000 7f807bfff640 1 --2- 172.21.15.64:0/3474216223 >> [v2:172.21.15.131:6824/2294059084,v1:172.21.15.131:6825/2294059084] conn(0x7f80680c0460 0x7f80680c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.631+0000 7f807bfff640 1 --2- 172.21.15.64:0/3474216223 >> [v2:172.21.15.131:6824/2294059084,v1:172.21.15.131:6825/2294059084] conn(0x7f80680c0460 0x7f80680c2950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:45.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.632+0000 7f80797fa640 1 -- 172.21.15.64:0/3474216223 <== osd.11 v2:172.21.15.131:6824/2294059084 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f806c015e80 con 0x7f80680c0460 2024-09-21T04:02:45.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.647+0000 7f808223e640 1 -- 172.21.15.64:0/3474216223 --> [v2:172.21.15.131:6824/2294059084,v1:172.21.15.131:6825/2294059084] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f807c063f10 con 0x7f80680c0460 2024-09-21T04:02:45.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.647+0000 7f80797fa640 1 -- 172.21.15.64:0/3474216223 <== osd.11 v2:172.21.15.131:6824/2294059084 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f807c063f10 con 0x7f80680c0460 2024-09-21T04:02:45.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.648+0000 7f808223e640 1 -- 172.21.15.64:0/3474216223 >> [v2:172.21.15.131:6824/2294059084,v1:172.21.15.131:6825/2294059084] conn(0x7f80680c0460 msgr2=0x7f80680c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:45.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.648+0000 7f808223e640 1 --2- 172.21.15.64:0/3474216223 >> [v2:172.21.15.131:6824/2294059084,v1:172.21.15.131:6825/2294059084] conn(0x7f80680c0460 0x7f80680c2950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:45.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.648+0000 7f808223e640 1 -- 172.21.15.64:0/3474216223 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f80680b65f0 msgr2=0x7f80680b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:45.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.648+0000 7f808223e640 1 --2- 172.21.15.64:0/3474216223 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f80680b65f0 0x7f80680b8b00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8064005fd0 tx=0x7f8064005850 comp rx=0 tx=0).stop 2024-09-21T04:02:45.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.648+0000 7f808223e640 1 -- 172.21.15.64:0/3474216223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f807c103ba0 msgr2=0x7f807c19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:45.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.648+0000 7f808223e640 1 --2- 172.21.15.64:0/3474216223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f807c103ba0 0x7f807c19a460 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f806c00e9f0 tx=0x7f806c00edf0 comp rx=0 tx=0).stop 2024-09-21T04:02:45.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.648+0000 7f807bfff640 1 -- 172.21.15.64:0/3474216223 reap_dead start 2024-09-21T04:02:45.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.648+0000 7f808223e640 1 -- 172.21.15.64:0/3474216223 shutdown_connections 2024-09-21T04:02:45.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.648+0000 7f808223e640 1 -- 172.21.15.64:0/3474216223 >> 172.21.15.64:0/3474216223 conn(0x7f807c0ff630 msgr2=0x7f807c106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:45.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.648+0000 7f808223e640 1 -- 172.21.15.64:0/3474216223 shutdown_connections 2024-09-21T04:02:45.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:45.648+0000 7f808223e640 1 -- 172.21.15.64:0/3474216223 wait complete. 2024-09-21T04:02:45.917 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:45 smithi064 ceph-mon[31180]: pgmap v281: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-21T04:02:46.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.078+0000 7f9b4c3eb640 1 -- 172.21.15.64:0/268433105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b44105380 msgr2=0x7f9b4410d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:46.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.078+0000 7f9b4c3eb640 1 --2- 172.21.15.64:0/268433105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b44105380 0x7f9b4410d8d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009900 tx=0x7f9b3402a410 comp rx=0 tx=0).stop 2024-09-21T04:02:46.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.078+0000 7f9b4c3eb640 1 -- 172.21.15.64:0/268433105 shutdown_connections 2024-09-21T04:02:46.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.078+0000 7f9b4c3eb640 1 --2- 172.21.15.64:0/268433105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b4410de60 0x7f9b44110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:46.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.078+0000 7f9b4c3eb640 1 --2- 172.21.15.64:0/268433105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b44105380 0x7f9b4410d8d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:46.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.078+0000 7f9b4c3eb640 1 --2- 172.21.15.64:0/268433105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b44104a30 0x7f9b44104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:46.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.078+0000 7f9b4c3eb640 1 -- 172.21.15.64:0/268433105 >> 172.21.15.64:0/268433105 conn(0x7f9b44079760 msgr2=0x7f9b44079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:46.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.078+0000 7f9b4c3eb640 1 -- 172.21.15.64:0/268433105 shutdown_connections 2024-09-21T04:02:46.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.078+0000 7f9b4c3eb640 1 -- 172.21.15.64:0/268433105 wait complete. 2024-09-21T04:02:46.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.079+0000 7f9b4c3eb640 1 Processor -- start 2024-09-21T04:02:46.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.079+0000 7f9b4c3eb640 1 -- start start 2024-09-21T04:02:46.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.079+0000 7f9b4c3eb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b44104a30 0x7f9b4419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:46.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.079+0000 7f9b4c3eb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b44105380 0x7f9b4419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:46.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.079+0000 7f9b4c3eb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b4410de60 0x7f9b4419fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:46.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.080+0000 7f9b4c3eb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9b44113b80 con 0x7f9b44104a30 2024-09-21T04:02:46.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.080+0000 7f9b4c3eb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9b44113a00 con 0x7f9b4410de60 2024-09-21T04:02:46.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.080+0000 7f9b4c3eb640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9b44113d00 con 0x7f9b44105380 2024-09-21T04:02:46.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.080+0000 7f9b4a187640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b44104a30 0x7f9b4419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:46.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.080+0000 7f9b4a187640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b44104a30 0x7f9b4419a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59152/0 (socket says 172.21.15.64:59152) 2024-09-21T04:02:46.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.080+0000 7f9b4a187640 1 -- 172.21.15.64:0/2914013421 learned_addr learned my addr 172.21.15.64:0/2914013421 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:46.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.080+0000 7f9b49986640 1 --2- 172.21.15.64:0/2914013421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b44105380 0x7f9b4419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:46.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.080+0000 7f9b4a988640 1 --2- 172.21.15.64:0/2914013421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b4410de60 0x7f9b4419fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:46.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.080+0000 7f9b4a187640 1 -- 172.21.15.64:0/2914013421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b44105380 msgr2=0x7f9b4419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:46.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.080+0000 7f9b4a187640 1 --2- 172.21.15.64:0/2914013421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b44105380 0x7f9b4419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:46.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.080+0000 7f9b4a187640 1 -- 172.21.15.64:0/2914013421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b4410de60 msgr2=0x7f9b4419fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:46.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.080+0000 7f9b4a187640 1 --2- 172.21.15.64:0/2914013421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b4410de60 0x7f9b4419fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:46.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.080+0000 7f9b4a187640 1 -- 172.21.15.64:0/2914013421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b441a0560 con 0x7f9b44104a30 2024-09-21T04:02:46.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.080+0000 7f9b4a988640 1 --2- 172.21.15.64:0/2914013421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b4410de60 0x7f9b4419fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:02:46.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.080+0000 7f9b4a187640 1 --2- 172.21.15.64:0/2914013421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b44104a30 0x7f9b4419a5f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9b4000b450 tx=0x7f9b4000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:46.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.080+0000 7f9b3b7fe640 1 -- 172.21.15.64:0/2914013421 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40015020 con 0x7f9b44104a30 2024-09-21T04:02:46.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.080+0000 7f9b3b7fe640 1 -- 172.21.15.64:0/2914013421 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b40011d40 con 0x7f9b44104a30 2024-09-21T04:02:46.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.080+0000 7f9b3b7fe640 1 -- 172.21.15.64:0/2914013421 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4000f340 con 0x7f9b44104a30 2024-09-21T04:02:46.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.080+0000 7f9b4c3eb640 1 -- 172.21.15.64:0/2914013421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b441a0850 con 0x7f9b44104a30 2024-09-21T04:02:46.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.081+0000 7f9b4c3eb640 1 -- 172.21.15.64:0/2914013421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b441a6fb0 con 0x7f9b44104a30 2024-09-21T04:02:46.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.081+0000 7f9b4c3eb640 1 -- 172.21.15.64:0/2914013421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9b44063b10 con 0x7f9b44104a30 2024-09-21T04:02:46.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.083+0000 7f9b3b7fe640 1 -- 172.21.15.64:0/2914013421 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f9b40018020 con 0x7f9b44104a30 2024-09-21T04:02:46.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.084+0000 7f9b3b7fe640 1 --2- 172.21.15.64:0/2914013421 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f9b300b65f0 0x7f9b300b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:46.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.084+0000 7f9b3b7fe640 1 -- 172.21.15.64:0/2914013421 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f9b400e75f0 con 0x7f9b44104a30 2024-09-21T04:02:46.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.084+0000 7f9b49986640 1 --2- 172.21.15.64:0/2914013421 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f9b300b65f0 0x7f9b300b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:46.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.084+0000 7f9b3b7fe640 1 --2- 172.21.15.64:0/2914013421 >> [v2:172.21.15.119:6808/323030958,v1:172.21.15.119:6809/323030958] conn(0x7f9b300c0460 0x7f9b300c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:46.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.084+0000 7f9b3b7fe640 1 -- 172.21.15.64:0/2914013421 --> [v2:172.21.15.119:6808/323030958,v1:172.21.15.119:6809/323030958] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9b300c3020 con 0x7f9b300c0460 2024-09-21T04:02:46.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.084+0000 7f9b3b7fe640 1 -- 172.21.15.64:0/2914013421 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f9b400e7a40 con 0x7f9b44104a30 2024-09-21T04:02:46.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.085+0000 7f9b49986640 1 --2- 172.21.15.64:0/2914013421 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f9b300b65f0 0x7f9b300b8b00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009230 tx=0x7f9b34031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:46.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.085+0000 7f9b4a988640 1 --2- 172.21.15.64:0/2914013421 >> [v2:172.21.15.119:6808/323030958,v1:172.21.15.119:6809/323030958] conn(0x7f9b300c0460 0x7f9b300c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:46.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.085+0000 7f9b4a988640 1 --2- 172.21.15.64:0/2914013421 >> [v2:172.21.15.119:6808/323030958,v1:172.21.15.119:6809/323030958] conn(0x7f9b300c0460 0x7f9b300c2950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:46.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.086+0000 7f9b3b7fe640 1 -- 172.21.15.64:0/2914013421 <== osd.4 v2:172.21.15.119:6808/323030958 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9b300c3020 con 0x7f9b300c0460 2024-09-21T04:02:46.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.100+0000 7f9b4c3eb640 1 -- 172.21.15.64:0/2914013421 --> [v2:172.21.15.119:6808/323030958,v1:172.21.15.119:6809/323030958] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9b44063f40 con 0x7f9b300c0460 2024-09-21T04:02:46.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.100+0000 7f9b3b7fe640 1 -- 172.21.15.64:0/2914013421 <== osd.4 v2:172.21.15.119:6808/323030958 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9b44063f40 con 0x7f9b300c0460 2024-09-21T04:02:46.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.100+0000 7f9b4c3eb640 1 -- 172.21.15.64:0/2914013421 >> [v2:172.21.15.119:6808/323030958,v1:172.21.15.119:6809/323030958] conn(0x7f9b300c0460 msgr2=0x7f9b300c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:46.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.100+0000 7f9b4c3eb640 1 --2- 172.21.15.64:0/2914013421 >> [v2:172.21.15.119:6808/323030958,v1:172.21.15.119:6809/323030958] conn(0x7f9b300c0460 0x7f9b300c2950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:46.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.101+0000 7f9b4c3eb640 1 -- 172.21.15.64:0/2914013421 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f9b300b65f0 msgr2=0x7f9b300b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:46.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.101+0000 7f9b4c3eb640 1 --2- 172.21.15.64:0/2914013421 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f9b300b65f0 0x7f9b300b8b00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009230 tx=0x7f9b34031400 comp rx=0 tx=0).stop 2024-09-21T04:02:46.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.101+0000 7f9b4c3eb640 1 -- 172.21.15.64:0/2914013421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b44104a30 msgr2=0x7f9b4419a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:46.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.101+0000 7f9b4c3eb640 1 --2- 172.21.15.64:0/2914013421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9b44104a30 0x7f9b4419a5f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9b4000b450 tx=0x7f9b4000b910 comp rx=0 tx=0).stop 2024-09-21T04:02:46.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.101+0000 7f9b4a988640 1 -- 172.21.15.64:0/2914013421 reap_dead start 2024-09-21T04:02:46.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.101+0000 7f9b4c3eb640 1 -- 172.21.15.64:0/2914013421 shutdown_connections 2024-09-21T04:02:46.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.101+0000 7f9b4c3eb640 1 -- 172.21.15.64:0/2914013421 >> 172.21.15.64:0/2914013421 conn(0x7f9b44079760 msgr2=0x7f9b440ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:46.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.101+0000 7f9b4c3eb640 1 -- 172.21.15.64:0/2914013421 shutdown_connections 2024-09-21T04:02:46.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.101+0000 7f9b4c3eb640 1 -- 172.21.15.64:0/2914013421 wait complete. 2024-09-21T04:02:46.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:45 smithi119 ceph-mon[34811]: pgmap v281: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-21T04:02:46.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:45 smithi131 ceph-mon[35193]: pgmap v281: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-21T04:02:46.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.674+0000 7f2368289640 1 -- 172.21.15.64:0/707034521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2360103bb0 msgr2=0x7f2360103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:46.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.674+0000 7f2368289640 1 --2- 172.21.15.64:0/707034521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2360103bb0 0x7f2360103fc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f235c009900 tx=0x7f235c02a410 comp rx=0 tx=0).stop 2024-09-21T04:02:46.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.674+0000 7f2368289640 1 -- 172.21.15.64:0/707034521 shutdown_connections 2024-09-21T04:02:46.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.674+0000 7f2368289640 1 --2- 172.21.15.64:0/707034521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2360109e40 0x7f2360110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:46.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.674+0000 7f2368289640 1 --2- 172.21.15.64:0/707034521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23601094f0 0x7f2360109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:46.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.674+0000 7f2368289640 1 --2- 172.21.15.64:0/707034521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2360103bb0 0x7f2360103fc0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:46.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.674+0000 7f2368289640 1 -- 172.21.15.64:0/707034521 >> 172.21.15.64:0/707034521 conn(0x7f23600ff600 msgr2=0x7f2360101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:46.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.675+0000 7f2368289640 1 -- 172.21.15.64:0/707034521 shutdown_connections 2024-09-21T04:02:46.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.675+0000 7f2368289640 1 -- 172.21.15.64:0/707034521 wait complete. 2024-09-21T04:02:46.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.675+0000 7f2368289640 1 Processor -- start 2024-09-21T04:02:46.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.676+0000 7f2368289640 1 -- start start 2024-09-21T04:02:46.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.676+0000 7f2368289640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2360103bb0 0x7f236019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:46.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.676+0000 7f2368289640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23601094f0 0x7f236019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:46.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.676+0000 7f2368289640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2360109e40 0x7f236019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:46.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.676+0000 7f2368289640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2360113b50 con 0x7f23601094f0 2024-09-21T04:02:46.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.676+0000 7f2368289640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f23601139d0 con 0x7f2360103bb0 2024-09-21T04:02:46.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.676+0000 7f2368289640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2360113cd0 con 0x7f2360109e40 2024-09-21T04:02:46.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.676+0000 7f2365824640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23601094f0 0x7f236019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:46.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.676+0000 7f2365824640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23601094f0 0x7f236019aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59168/0 (socket says 172.21.15.64:59168) 2024-09-21T04:02:46.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.676+0000 7f2365824640 1 -- 172.21.15.64:0/3145508377 learned_addr learned my addr 172.21.15.64:0/3145508377 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:46.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.676+0000 7f2366025640 1 --2- 172.21.15.64:0/3145508377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2360103bb0 0x7f236019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:46.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.676+0000 7f2366826640 1 --2- 172.21.15.64:0/3145508377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2360109e40 0x7f236019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:46.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.676+0000 7f2365824640 1 -- 172.21.15.64:0/3145508377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2360109e40 msgr2=0x7f236019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:46.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.676+0000 7f2365824640 1 --2- 172.21.15.64:0/3145508377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2360109e40 0x7f236019fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:46.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.676+0000 7f2365824640 1 -- 172.21.15.64:0/3145508377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2360103bb0 msgr2=0x7f236019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:46.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.676+0000 7f2365824640 1 --2- 172.21.15.64:0/3145508377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2360103bb0 0x7f236019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:46.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.676+0000 7f2365824640 1 -- 172.21.15.64:0/3145508377 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23601a0390 con 0x7f23601094f0 2024-09-21T04:02:46.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.676+0000 7f2366826640 1 --2- 172.21.15.64:0/3145508377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2360109e40 0x7f236019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:02:46.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.677+0000 7f2365824640 1 --2- 172.21.15.64:0/3145508377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23601094f0 0x7f236019aa80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f235000d740 tx=0x7f235000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:46.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.677+0000 7f23577fe640 1 -- 172.21.15.64:0/3145508377 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2350017070 con 0x7f23601094f0 2024-09-21T04:02:46.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.677+0000 7f2368289640 1 -- 172.21.15.64:0/3145508377 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23601a0680 con 0x7f23601094f0 2024-09-21T04:02:46.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.677+0000 7f23577fe640 1 -- 172.21.15.64:0/3145508377 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f235000de50 con 0x7f23601094f0 2024-09-21T04:02:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.677+0000 7f23577fe640 1 -- 172.21.15.64:0/3145508377 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235000f620 con 0x7f23601094f0 2024-09-21T04:02:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.677+0000 7f2368289640 1 -- 172.21.15.64:0/3145508377 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23601a6ea0 con 0x7f23601094f0 2024-09-21T04:02:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.678+0000 7f2368289640 1 -- 172.21.15.64:0/3145508377 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2360063b50 con 0x7f23601094f0 2024-09-21T04:02:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.679+0000 7f23577fe640 1 -- 172.21.15.64:0/3145508377 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f235001a070 con 0x7f23601094f0 2024-09-21T04:02:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.680+0000 7f23577fe640 1 --2- 172.21.15.64:0/3145508377 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f234c0b6620 0x7f234c0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.680+0000 7f2366025640 1 --2- 172.21.15.64:0/3145508377 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f234c0b6620 0x7f234c0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.680+0000 7f23577fe640 1 -- 172.21.15.64:0/3145508377 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f23500e73f0 con 0x7f23601094f0 2024-09-21T04:02:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.680+0000 7f23577fe640 1 --2- 172.21.15.64:0/3145508377 >> [v2:172.21.15.64:6818/1913824324,v1:172.21.15.64:6819/1913824324] conn(0x7f234c0c0490 0x7f234c0c2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.680+0000 7f23577fe640 1 -- 172.21.15.64:0/3145508377 --> [v2:172.21.15.64:6818/1913824324,v1:172.21.15.64:6819/1913824324] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f234c0c3050 con 0x7f234c0c0490 2024-09-21T04:02:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.680+0000 7f23577fe640 1 -- 172.21.15.64:0/3145508377 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f23500e7840 con 0x7f23601094f0 2024-09-21T04:02:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.681+0000 7f2366826640 1 --2- 172.21.15.64:0/3145508377 >> [v2:172.21.15.64:6818/1913824324,v1:172.21.15.64:6819/1913824324] conn(0x7f234c0c0490 0x7f234c0c2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.681+0000 7f2366025640 1 --2- 172.21.15.64:0/3145508377 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f234c0b6620 0x7f234c0b8b30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f235c005ab0 tx=0x7f235c005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.681+0000 7f2366826640 1 --2- 172.21.15.64:0/3145508377 >> [v2:172.21.15.64:6818/1913824324,v1:172.21.15.64:6819/1913824324] conn(0x7f234c0c0490 0x7f234c0c2980 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:46.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.682+0000 7f23577fe640 1 -- 172.21.15.64:0/3145508377 <== osd.6 v2:172.21.15.64:6818/1913824324 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f234c0c3050 con 0x7f234c0c0490 2024-09-21T04:02:46.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.696+0000 7f2368289640 1 -- 172.21.15.64:0/3145508377 --> [v2:172.21.15.64:6818/1913824324,v1:172.21.15.64:6819/1913824324] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2360063fd0 con 0x7f234c0c0490 2024-09-21T04:02:46.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.696+0000 7f23577fe640 1 -- 172.21.15.64:0/3145508377 <== osd.6 v2:172.21.15.64:6818/1913824324 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2360063fd0 con 0x7f234c0c0490 2024-09-21T04:02:46.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.696+0000 7f2368289640 1 -- 172.21.15.64:0/3145508377 >> [v2:172.21.15.64:6818/1913824324,v1:172.21.15.64:6819/1913824324] conn(0x7f234c0c0490 msgr2=0x7f234c0c2980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:46.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.696+0000 7f2368289640 1 --2- 172.21.15.64:0/3145508377 >> [v2:172.21.15.64:6818/1913824324,v1:172.21.15.64:6819/1913824324] conn(0x7f234c0c0490 0x7f234c0c2980 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:46.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.696+0000 7f2368289640 1 -- 172.21.15.64:0/3145508377 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f234c0b6620 msgr2=0x7f234c0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:46.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.696+0000 7f2368289640 1 --2- 172.21.15.64:0/3145508377 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f234c0b6620 0x7f234c0b8b30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f235c005ab0 tx=0x7f235c005a20 comp rx=0 tx=0).stop 2024-09-21T04:02:46.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.696+0000 7f2368289640 1 -- 172.21.15.64:0/3145508377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23601094f0 msgr2=0x7f236019aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:46.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.696+0000 7f2368289640 1 --2- 172.21.15.64:0/3145508377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23601094f0 0x7f236019aa80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f235000d740 tx=0x7f235000dc00 comp rx=0 tx=0).stop 2024-09-21T04:02:46.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.696+0000 7f2366826640 1 -- 172.21.15.64:0/3145508377 reap_dead start 2024-09-21T04:02:46.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.696+0000 7f2368289640 1 -- 172.21.15.64:0/3145508377 shutdown_connections 2024-09-21T04:02:46.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.696+0000 7f2368289640 1 -- 172.21.15.64:0/3145508377 >> 172.21.15.64:0/3145508377 conn(0x7f23600ff600 msgr2=0x7f2360101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:46.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.697+0000 7f2368289640 1 -- 172.21.15.64:0/3145508377 shutdown_connections 2024-09-21T04:02:46.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:46.697+0000 7f2368289640 1 -- 172.21.15.64:0/3145508377 wait complete. 2024-09-21T04:02:48.103 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:47 smithi064 ceph-mon[31180]: pgmap v282: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-21T04:02:48.139 INFO:teuthology.orchestra.run.smithi064.stdout:81604378691 2024-09-21T04:02:48.140 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-21T04:02:48.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:47 smithi119 ceph-mon[34811]: pgmap v282: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-21T04:02:48.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:47 smithi131 ceph-mon[35193]: pgmap v282: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-21T04:02:48.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.365+0000 7f32bcccd640 1 -- 172.21.15.64:0/2761057817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32b81083e0 msgr2=0x7f32b81087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.365+0000 7f32bcccd640 1 --2- 172.21.15.64:0/2761057817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32b81083e0 0x7f32b81087f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009900 tx=0x7f32a002a410 comp rx=0 tx=0).stop 2024-09-21T04:02:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.365+0000 7f32bcccd640 1 -- 172.21.15.64:0/2761057817 shutdown_connections 2024-09-21T04:02:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.365+0000 7f32bcccd640 1 --2- 172.21.15.64:0/2761057817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32b8108d30 0x7f32b810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.365+0000 7f32bcccd640 1 --2- 172.21.15.64:0/2761057817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32b81083e0 0x7f32b81087f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.365+0000 7f32bcccd640 1 --2- 172.21.15.64:0/2761057817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32b8102aa0 0x7f32b8102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.365+0000 7f32bcccd640 1 -- 172.21.15.64:0/2761057817 >> 172.21.15.64:0/2761057817 conn(0x7f32b80797e0 msgr2=0x7f32b8079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.365+0000 7f32bcccd640 1 -- 172.21.15.64:0/2761057817 shutdown_connections 2024-09-21T04:02:48.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.365+0000 7f32bcccd640 1 -- 172.21.15.64:0/2761057817 wait complete. 2024-09-21T04:02:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.366+0000 7f32bcccd640 1 Processor -- start 2024-09-21T04:02:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.366+0000 7f32bcccd640 1 -- start start 2024-09-21T04:02:48.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.367+0000 7f32bcccd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32b8102aa0 0x7f32b819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:48.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.367+0000 7f32bcccd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32b81083e0 0x7f32b819b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:48.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.367+0000 7f32bcccd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32b8108d30 0x7f32b819bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:48.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.367+0000 7f32bcccd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f32b81129a0 con 0x7f32b8108d30 2024-09-21T04:02:48.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.367+0000 7f32bcccd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f32b8112820 con 0x7f32b8102aa0 2024-09-21T04:02:48.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.367+0000 7f32bcccd640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f32b8112b20 con 0x7f32b81083e0 2024-09-21T04:02:48.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.367+0000 7f32b659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32b8102aa0 0x7f32b819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:48.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.367+0000 7f32b6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32b8108d30 0x7f32b819bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:48.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.367+0000 7f32b659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32b8102aa0 0x7f32b819b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:57118/0 (socket says 172.21.15.64:57118) 2024-09-21T04:02:48.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.367+0000 7f32b659c640 1 -- 172.21.15.64:0/2842656191 learned_addr learned my addr 172.21.15.64:0/2842656191 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:48.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.367+0000 7f32b5d9b640 1 --2- 172.21.15.64:0/2842656191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32b81083e0 0x7f32b819b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:48.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.367+0000 7f32b6d9d640 1 -- 172.21.15.64:0/2842656191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32b81083e0 msgr2=0x7f32b819b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:48.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.367+0000 7f32b6d9d640 1 --2- 172.21.15.64:0/2842656191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32b81083e0 0x7f32b819b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:48.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.367+0000 7f32b6d9d640 1 -- 172.21.15.64:0/2842656191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32b8102aa0 msgr2=0x7f32b819b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:48.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.367+0000 7f32b6d9d640 1 --2- 172.21.15.64:0/2842656191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32b8102aa0 0x7f32b819b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:48.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.367+0000 7f32b6d9d640 1 -- 172.21.15.64:0/2842656191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b81a6560 con 0x7f32b8108d30 2024-09-21T04:02:48.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.367+0000 7f32b6d9d640 1 --2- 172.21.15.64:0/2842656191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32b8108d30 0x7f32b819bd50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f32a800c710 tx=0x7f32a800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:48.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.368+0000 7f32af7fe640 1 -- 172.21.15.64:0/2842656191 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a8004220 con 0x7f32b8108d30 2024-09-21T04:02:48.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.368+0000 7f32af7fe640 1 -- 172.21.15.64:0/2842656191 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32a800ce20 con 0x7f32b8108d30 2024-09-21T04:02:48.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.368+0000 7f32bcccd640 1 -- 172.21.15.64:0/2842656191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b81a6850 con 0x7f32b8108d30 2024-09-21T04:02:48.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.368+0000 7f32af7fe640 1 -- 172.21.15.64:0/2842656191 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a8003d50 con 0x7f32b8108d30 2024-09-21T04:02:48.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.368+0000 7f32bcccd640 1 -- 172.21.15.64:0/2842656191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b81a6d60 con 0x7f32b8108d30 2024-09-21T04:02:48.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.369+0000 7f32bcccd640 1 -- 172.21.15.64:0/2842656191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f32b8063b10 con 0x7f32b8108d30 2024-09-21T04:02:48.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.370+0000 7f32af7fe640 1 -- 172.21.15.64:0/2842656191 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f32a80079b0 con 0x7f32b8108d30 2024-09-21T04:02:48.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.371+0000 7f32af7fe640 1 --2- 172.21.15.64:0/2842656191 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f32a40b6620 0x7f32a40b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:48.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.371+0000 7f32b659c640 1 --2- 172.21.15.64:0/2842656191 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f32a40b6620 0x7f32a40b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:48.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.371+0000 7f32af7fe640 1 -- 172.21.15.64:0/2842656191 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f32a80e74b0 con 0x7f32b8108d30 2024-09-21T04:02:48.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.372+0000 7f32af7fe640 1 --2- 172.21.15.64:0/2842656191 >> [v2:172.21.15.119:6824/1460844681,v1:172.21.15.119:6825/1460844681] conn(0x7f32a40c0490 0x7f32a40c2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:48.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.372+0000 7f32af7fe640 1 -- 172.21.15.64:0/2842656191 --> [v2:172.21.15.119:6824/1460844681,v1:172.21.15.119:6825/1460844681] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f32a800fe50 con 0x7f32a40c0490 2024-09-21T04:02:48.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.372+0000 7f32af7fe640 1 -- 172.21.15.64:0/2842656191 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f32a80e7900 con 0x7f32b8108d30 2024-09-21T04:02:48.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.372+0000 7f32b659c640 1 --2- 172.21.15.64:0/2842656191 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f32a40b6620 0x7f32a40b8b30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f32b806c380 tx=0x7f32b0009720 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:48.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.372+0000 7f32b5d9b640 1 --2- 172.21.15.64:0/2842656191 >> [v2:172.21.15.119:6824/1460844681,v1:172.21.15.119:6825/1460844681] conn(0x7f32a40c0490 0x7f32a40c2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:48.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.372+0000 7f32b5d9b640 1 --2- 172.21.15.64:0/2842656191 >> [v2:172.21.15.119:6824/1460844681,v1:172.21.15.119:6825/1460844681] conn(0x7f32a40c0490 0x7f32a40c2980 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:48.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.373+0000 7f32af7fe640 1 -- 172.21.15.64:0/2842656191 <== osd.10 v2:172.21.15.119:6824/1460844681 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f32a800fe50 con 0x7f32a40c0490 2024-09-21T04:02:48.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.387+0000 7f32bcccd640 1 -- 172.21.15.64:0/2842656191 --> [v2:172.21.15.119:6824/1460844681,v1:172.21.15.119:6825/1460844681] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f32b8063f90 con 0x7f32a40c0490 2024-09-21T04:02:48.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.387+0000 7f32af7fe640 1 -- 172.21.15.64:0/2842656191 <== osd.10 v2:172.21.15.119:6824/1460844681 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f32b8063f90 con 0x7f32a40c0490 2024-09-21T04:02:48.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.388+0000 7f32bcccd640 1 -- 172.21.15.64:0/2842656191 >> [v2:172.21.15.119:6824/1460844681,v1:172.21.15.119:6825/1460844681] conn(0x7f32a40c0490 msgr2=0x7f32a40c2980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:48.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.388+0000 7f32bcccd640 1 --2- 172.21.15.64:0/2842656191 >> [v2:172.21.15.119:6824/1460844681,v1:172.21.15.119:6825/1460844681] conn(0x7f32a40c0490 0x7f32a40c2980 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:48.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.388+0000 7f32bcccd640 1 -- 172.21.15.64:0/2842656191 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f32a40b6620 msgr2=0x7f32a40b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:48.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.388+0000 7f32bcccd640 1 --2- 172.21.15.64:0/2842656191 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f32a40b6620 0x7f32a40b8b30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f32b806c380 tx=0x7f32b0009720 comp rx=0 tx=0).stop 2024-09-21T04:02:48.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.388+0000 7f32bcccd640 1 -- 172.21.15.64:0/2842656191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32b8108d30 msgr2=0x7f32b819bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:48.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.388+0000 7f32bcccd640 1 --2- 172.21.15.64:0/2842656191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32b8108d30 0x7f32b819bd50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f32a800c710 tx=0x7f32a800cbd0 comp rx=0 tx=0).stop 2024-09-21T04:02:48.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.388+0000 7f32b6d9d640 1 -- 172.21.15.64:0/2842656191 reap_dead start 2024-09-21T04:02:48.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.388+0000 7f32bcccd640 1 -- 172.21.15.64:0/2842656191 shutdown_connections 2024-09-21T04:02:48.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.388+0000 7f32bcccd640 1 -- 172.21.15.64:0/2842656191 >> 172.21.15.64:0/2842656191 conn(0x7f32b80797e0 msgr2=0x7f32b8104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:48.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.388+0000 7f32bcccd640 1 -- 172.21.15.64:0/2842656191 shutdown_connections 2024-09-21T04:02:48.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:48.388+0000 7f32bcccd640 1 -- 172.21.15.64:0/2842656191 wait complete. 2024-09-21T04:02:48.735 INFO:teuthology.orchestra.run.smithi064.stdout:287762808854 2024-09-21T04:02:48.735 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.11 2024-09-21T04:02:49.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.135+0000 7f8e90cfb640 1 -- 172.21.15.64:0/1162419851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e8c109e40 msgr2=0x7f8e8c110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:49.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.135+0000 7f8e90cfb640 1 --2- 172.21.15.64:0/1162419851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e8c109e40 0x7f8e8c110670 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8e84009ca0 tx=0x7f8e8402a8c0 comp rx=0 tx=0).stop 2024-09-21T04:02:49.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.135+0000 7f8e90cfb640 1 -- 172.21.15.64:0/1162419851 shutdown_connections 2024-09-21T04:02:49.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.135+0000 7f8e90cfb640 1 --2- 172.21.15.64:0/1162419851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e8c109e40 0x7f8e8c110670 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:49.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.135+0000 7f8e90cfb640 1 --2- 172.21.15.64:0/1162419851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e8c1094f0 0x7f8e8c109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:49.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.135+0000 7f8e90cfb640 1 --2- 172.21.15.64:0/1162419851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e8c103bb0 0x7f8e8c103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:49.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.135+0000 7f8e90cfb640 1 -- 172.21.15.64:0/1162419851 >> 172.21.15.64:0/1162419851 conn(0x7f8e8c0ff600 msgr2=0x7f8e8c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:49.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.135+0000 7f8e90cfb640 1 -- 172.21.15.64:0/1162419851 shutdown_connections 2024-09-21T04:02:49.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.135+0000 7f8e90cfb640 1 -- 172.21.15.64:0/1162419851 wait complete. 2024-09-21T04:02:49.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.136+0000 7f8e90cfb640 1 Processor -- start 2024-09-21T04:02:49.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.136+0000 7f8e90cfb640 1 -- start start 2024-09-21T04:02:49.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.136+0000 7f8e90cfb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e8c103bb0 0x7f8e8c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:49.145 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.136+0000 7f8e90cfb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e8c1094f0 0x7f8e8c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:49.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.136+0000 7f8e90cfb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e8c109e40 0x7f8e8c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:49.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.136+0000 7f8e90cfb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c113b00 con 0x7f8e8c1094f0 2024-09-21T04:02:49.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.136+0000 7f8e90cfb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c113980 con 0x7f8e8c103bb0 2024-09-21T04:02:49.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.136+0000 7f8e90cfb640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c113c80 con 0x7f8e8c109e40 2024-09-21T04:02:49.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.136+0000 7f8e89d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e8c1094f0 0x7f8e8c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:49.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.136+0000 7f8e89d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e8c1094f0 0x7f8e8c19a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59216/0 (socket says 172.21.15.64:59216) 2024-09-21T04:02:49.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.136+0000 7f8e89d9b640 1 -- 172.21.15.64:0/1089043913 learned_addr learned my addr 172.21.15.64:0/1089043913 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:49.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.136+0000 7f8e8ad9d640 1 --2- 172.21.15.64:0/1089043913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e8c109e40 0x7f8e8c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:49.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.137+0000 7f8e8a59c640 1 --2- 172.21.15.64:0/1089043913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e8c103bb0 0x7f8e8c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:49.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.137+0000 7f8e89d9b640 1 -- 172.21.15.64:0/1089043913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e8c109e40 msgr2=0x7f8e8c19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:49.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.137+0000 7f8e89d9b640 1 --2- 172.21.15.64:0/1089043913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e8c109e40 0x7f8e8c19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:49.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.137+0000 7f8e89d9b640 1 -- 172.21.15.64:0/1089043913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e8c103bb0 msgr2=0x7f8e8c19a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:49.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.137+0000 7f8e89d9b640 1 --2- 172.21.15.64:0/1089043913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e8c103bb0 0x7f8e8c19a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:49.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.137+0000 7f8e89d9b640 1 -- 172.21.15.64:0/1089043913 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e8c1a03c0 con 0x7f8e8c1094f0 2024-09-21T04:02:49.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.137+0000 7f8e89d9b640 1 --2- 172.21.15.64:0/1089043913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e8c1094f0 0x7f8e8c19a9f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8e8000d740 tx=0x7f8e8000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:49.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.137+0000 7f8e7b7fe640 1 -- 172.21.15.64:0/1089043913 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80017070 con 0x7f8e8c1094f0 2024-09-21T04:02:49.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.137+0000 7f8e7b7fe640 1 -- 172.21.15.64:0/1089043913 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e8000de50 con 0x7f8e8c1094f0 2024-09-21T04:02:49.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.137+0000 7f8e7b7fe640 1 -- 172.21.15.64:0/1089043913 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8000f620 con 0x7f8e8c1094f0 2024-09-21T04:02:49.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.137+0000 7f8e90cfb640 1 -- 172.21.15.64:0/1089043913 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e8c1a06b0 con 0x7f8e8c1094f0 2024-09-21T04:02:49.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.137+0000 7f8e90cfb640 1 -- 172.21.15.64:0/1089043913 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e8c1a6e10 con 0x7f8e8c1094f0 2024-09-21T04:02:49.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.138+0000 7f8e90cfb640 1 -- 172.21.15.64:0/1089043913 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8e8c063b50 con 0x7f8e8c1094f0 2024-09-21T04:02:49.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.140+0000 7f8e7b7fe640 1 -- 172.21.15.64:0/1089043913 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f8e8001a070 con 0x7f8e8c1094f0 2024-09-21T04:02:49.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.140+0000 7f8e7b7fe640 1 --2- 172.21.15.64:0/1089043913 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8e740b65f0 0x7f8e740b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:49.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.141+0000 7f8e8a59c640 1 --2- 172.21.15.64:0/1089043913 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8e740b65f0 0x7f8e740b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:49.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.141+0000 7f8e7b7fe640 1 -- 172.21.15.64:0/1089043913 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f8e800e7430 con 0x7f8e8c1094f0 2024-09-21T04:02:49.147 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.141+0000 7f8e7b7fe640 1 --2- 172.21.15.64:0/1089043913 >> [v2:172.21.15.64:6826/3309038990,v1:172.21.15.64:6827/3309038990] conn(0x7f8e740c0460 0x7f8e740c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:49.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.141+0000 7f8e7b7fe640 1 -- 172.21.15.64:0/1089043913 --> [v2:172.21.15.64:6826/3309038990,v1:172.21.15.64:6827/3309038990] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8e740c3020 con 0x7f8e740c0460 2024-09-21T04:02:49.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.141+0000 7f8e7b7fe640 1 -- 172.21.15.64:0/1089043913 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f8e800e7880 con 0x7f8e8c1094f0 2024-09-21T04:02:49.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.141+0000 7f8e8ad9d640 1 --2- 172.21.15.64:0/1089043913 >> [v2:172.21.15.64:6826/3309038990,v1:172.21.15.64:6827/3309038990] conn(0x7f8e740c0460 0x7f8e740c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:49.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.141+0000 7f8e8a59c640 1 --2- 172.21.15.64:0/1089043913 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8e740b65f0 0x7f8e740b8b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c005fd0 tx=0x7f8e7c005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:49.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.141+0000 7f8e8ad9d640 1 --2- 172.21.15.64:0/1089043913 >> [v2:172.21.15.64:6826/3309038990,v1:172.21.15.64:6827/3309038990] conn(0x7f8e740c0460 0x7f8e740c2950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:49.148 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.142+0000 7f8e7b7fe640 1 -- 172.21.15.64:0/1089043913 <== osd.9 v2:172.21.15.64:6826/3309038990 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8e740c3020 con 0x7f8e740c0460 2024-09-21T04:02:49.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.156+0000 7f8e90cfb640 1 -- 172.21.15.64:0/1089043913 --> [v2:172.21.15.64:6826/3309038990,v1:172.21.15.64:6827/3309038990] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8e8c063fd0 con 0x7f8e740c0460 2024-09-21T04:02:49.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.156+0000 7f8e7b7fe640 1 -- 172.21.15.64:0/1089043913 <== osd.9 v2:172.21.15.64:6826/3309038990 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8e8c063fd0 con 0x7f8e740c0460 2024-09-21T04:02:49.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.157+0000 7f8e90cfb640 1 -- 172.21.15.64:0/1089043913 >> [v2:172.21.15.64:6826/3309038990,v1:172.21.15.64:6827/3309038990] conn(0x7f8e740c0460 msgr2=0x7f8e740c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:49.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.157+0000 7f8e90cfb640 1 --2- 172.21.15.64:0/1089043913 >> [v2:172.21.15.64:6826/3309038990,v1:172.21.15.64:6827/3309038990] conn(0x7f8e740c0460 0x7f8e740c2950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:49.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.157+0000 7f8e90cfb640 1 -- 172.21.15.64:0/1089043913 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8e740b65f0 msgr2=0x7f8e740b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:49.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.157+0000 7f8e90cfb640 1 --2- 172.21.15.64:0/1089043913 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8e740b65f0 0x7f8e740b8b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c005fd0 tx=0x7f8e7c005850 comp rx=0 tx=0).stop 2024-09-21T04:02:49.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.157+0000 7f8e90cfb640 1 -- 172.21.15.64:0/1089043913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e8c1094f0 msgr2=0x7f8e8c19a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:49.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.157+0000 7f8e90cfb640 1 --2- 172.21.15.64:0/1089043913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e8c1094f0 0x7f8e8c19a9f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8e8000d740 tx=0x7f8e8000dc00 comp rx=0 tx=0).stop 2024-09-21T04:02:49.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.157+0000 7f8e8ad9d640 1 -- 172.21.15.64:0/1089043913 reap_dead start 2024-09-21T04:02:49.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.157+0000 7f8e90cfb640 1 -- 172.21.15.64:0/1089043913 shutdown_connections 2024-09-21T04:02:49.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.157+0000 7f8e90cfb640 1 -- 172.21.15.64:0/1089043913 >> 172.21.15.64:0/1089043913 conn(0x7f8e8c0ff600 msgr2=0x7f8e8c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:49.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.157+0000 7f8e90cfb640 1 -- 172.21.15.64:0/1089043913 shutdown_connections 2024-09-21T04:02:49.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:49.157+0000 7f8e90cfb640 1 -- 172.21.15.64:0/1089043913 wait complete. 2024-09-21T04:02:49.778 INFO:teuthology.orchestra.run.smithi064.stdout:219043332133 2024-09-21T04:02:49.779 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.8 2024-09-21T04:02:50.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:49 smithi064 ceph-mon[31180]: pgmap v283: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-21T04:02:50.131 INFO:teuthology.orchestra.run.smithi064.stdout:176093659184 2024-09-21T04:02:50.132 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.6 2024-09-21T04:02:50.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:49 smithi119 ceph-mon[34811]: pgmap v283: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-21T04:02:50.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:49 smithi131 ceph-mon[35193]: pgmap v283: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-21T04:02:50.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.717+0000 7fda9142b640 1 -- 172.21.15.64:0/3878205168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda8c105380 msgr2=0x7fda8c10d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:50.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.717+0000 7fda9142b640 1 --2- 172.21.15.64:0/3878205168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda8c105380 0x7fda8c10d8d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fda74009900 tx=0x7fda7402a410 comp rx=0 tx=0).stop 2024-09-21T04:02:50.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.717+0000 7fda9142b640 1 -- 172.21.15.64:0/3878205168 shutdown_connections 2024-09-21T04:02:50.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.717+0000 7fda9142b640 1 --2- 172.21.15.64:0/3878205168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda8c10de60 0x7fda8c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:50.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.717+0000 7fda9142b640 1 --2- 172.21.15.64:0/3878205168 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda8c105380 0x7fda8c10d8d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:50.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.717+0000 7fda9142b640 1 --2- 172.21.15.64:0/3878205168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda8c104a30 0x7fda8c104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:50.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.717+0000 7fda9142b640 1 -- 172.21.15.64:0/3878205168 >> 172.21.15.64:0/3878205168 conn(0x7fda8c0797b0 msgr2=0x7fda8c079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:50.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.717+0000 7fda9142b640 1 -- 172.21.15.64:0/3878205168 shutdown_connections 2024-09-21T04:02:50.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.717+0000 7fda9142b640 1 -- 172.21.15.64:0/3878205168 wait complete. 2024-09-21T04:02:50.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.718+0000 7fda9142b640 1 Processor -- start 2024-09-21T04:02:50.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.718+0000 7fda9142b640 1 -- start start 2024-09-21T04:02:50.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.718+0000 7fda9142b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda8c104a30 0x7fda8c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:50.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.718+0000 7fda9142b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda8c105380 0x7fda8c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:50.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.718+0000 7fda9142b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda8c10de60 0x7fda8c19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:50.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.718+0000 7fda9142b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c113aa0 con 0x7fda8c10de60 2024-09-21T04:02:50.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.718+0000 7fda9142b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c113920 con 0x7fda8c104a30 2024-09-21T04:02:50.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.718+0000 7fda9142b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fda8c113c20 con 0x7fda8c105380 2024-09-21T04:02:50.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.718+0000 7fda8b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda8c10de60 0x7fda8c19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:50.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.718+0000 7fda8b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda8c10de60 0x7fda8c19fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59222/0 (socket says 172.21.15.64:59222) 2024-09-21T04:02:50.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.718+0000 7fda8b7fe640 1 -- 172.21.15.64:0/511674486 learned_addr learned my addr 172.21.15.64:0/511674486 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:50.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.718+0000 7fda8affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda8c104a30 0x7fda8c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:50.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.719+0000 7fda83fff640 1 --2- 172.21.15.64:0/511674486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda8c105380 0x7fda8c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:50.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.719+0000 7fda8b7fe640 1 -- 172.21.15.64:0/511674486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda8c105380 msgr2=0x7fda8c19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:50.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.719+0000 7fda8b7fe640 1 --2- 172.21.15.64:0/511674486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda8c105380 0x7fda8c19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:50.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.719+0000 7fda8b7fe640 1 -- 172.21.15.64:0/511674486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda8c104a30 msgr2=0x7fda8c19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:50.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.719+0000 7fda8b7fe640 1 --2- 172.21.15.64:0/511674486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda8c104a30 0x7fda8c19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:50.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.719+0000 7fda8b7fe640 1 -- 172.21.15.64:0/511674486 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8c1a03e0 con 0x7fda8c10de60 2024-09-21T04:02:50.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.719+0000 7fda8b7fe640 1 --2- 172.21.15.64:0/511674486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda8c10de60 0x7fda8c19fcb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fda8400eaa0 tx=0x7fda8400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:50.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.719+0000 7fda88ff9640 1 -- 172.21.15.64:0/511674486 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8400cdb0 con 0x7fda8c10de60 2024-09-21T04:02:50.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.719+0000 7fda9142b640 1 -- 172.21.15.64:0/511674486 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda8c1a06d0 con 0x7fda8c10de60 2024-09-21T04:02:50.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.719+0000 7fda88ff9640 1 -- 172.21.15.64:0/511674486 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda84005020 con 0x7fda8c10de60 2024-09-21T04:02:50.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.719+0000 7fda88ff9640 1 -- 172.21.15.64:0/511674486 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda840116d0 con 0x7fda8c10de60 2024-09-21T04:02:50.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.719+0000 7fda9142b640 1 -- 172.21.15.64:0/511674486 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda8c102650 con 0x7fda8c10de60 2024-09-21T04:02:50.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.720+0000 7fda9142b640 1 -- 172.21.15.64:0/511674486 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fda8c063b10 con 0x7fda8c10de60 2024-09-21T04:02:50.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.722+0000 7fda88ff9640 1 -- 172.21.15.64:0/511674486 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fda840051d0 con 0x7fda8c10de60 2024-09-21T04:02:50.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.722+0000 7fda88ff9640 1 --2- 172.21.15.64:0/511674486 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fda780b65f0 0x7fda780b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:50.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.723+0000 7fda8affd640 1 --2- 172.21.15.64:0/511674486 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fda780b65f0 0x7fda780b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:50.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.723+0000 7fda88ff9640 1 -- 172.21.15.64:0/511674486 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7fda840e7b80 con 0x7fda8c10de60 2024-09-21T04:02:50.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.723+0000 7fda88ff9640 1 --2- 172.21.15.64:0/511674486 >> [v2:172.21.15.131:6808/3021788088,v1:172.21.15.131:6809/3021788088] conn(0x7fda780c0460 0x7fda780c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:50.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.723+0000 7fda88ff9640 1 -- 172.21.15.64:0/511674486 --> [v2:172.21.15.131:6808/3021788088,v1:172.21.15.131:6809/3021788088] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fda84014ec0 con 0x7fda780c0460 2024-09-21T04:02:50.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.723+0000 7fda88ff9640 1 -- 172.21.15.64:0/511674486 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fda840e7fd0 con 0x7fda8c10de60 2024-09-21T04:02:50.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.723+0000 7fda8affd640 1 --2- 172.21.15.64:0/511674486 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fda780b65f0 0x7fda780b8b00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00bbe0 tx=0x7fda7c00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:50.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.723+0000 7fda83fff640 1 --2- 172.21.15.64:0/511674486 >> [v2:172.21.15.131:6808/3021788088,v1:172.21.15.131:6809/3021788088] conn(0x7fda780c0460 0x7fda780c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:50.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.724+0000 7fda83fff640 1 --2- 172.21.15.64:0/511674486 >> [v2:172.21.15.131:6808/3021788088,v1:172.21.15.131:6809/3021788088] conn(0x7fda780c0460 0x7fda780c2950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:50.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.724+0000 7fda88ff9640 1 -- 172.21.15.64:0/511674486 <== osd.5 v2:172.21.15.131:6808/3021788088 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fda84014ec0 con 0x7fda780c0460 2024-09-21T04:02:50.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.738+0000 7fda9142b640 1 -- 172.21.15.64:0/511674486 --> [v2:172.21.15.131:6808/3021788088,v1:172.21.15.131:6809/3021788088] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fda8c063f90 con 0x7fda780c0460 2024-09-21T04:02:50.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.739+0000 7fda88ff9640 1 -- 172.21.15.64:0/511674486 <== osd.5 v2:172.21.15.131:6808/3021788088 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fda8c063f90 con 0x7fda780c0460 2024-09-21T04:02:50.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.739+0000 7fda9142b640 1 -- 172.21.15.64:0/511674486 >> [v2:172.21.15.131:6808/3021788088,v1:172.21.15.131:6809/3021788088] conn(0x7fda780c0460 msgr2=0x7fda780c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:50.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.739+0000 7fda9142b640 1 --2- 172.21.15.64:0/511674486 >> [v2:172.21.15.131:6808/3021788088,v1:172.21.15.131:6809/3021788088] conn(0x7fda780c0460 0x7fda780c2950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:50.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.739+0000 7fda9142b640 1 -- 172.21.15.64:0/511674486 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fda780b65f0 msgr2=0x7fda780b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:50.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.739+0000 7fda9142b640 1 --2- 172.21.15.64:0/511674486 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fda780b65f0 0x7fda780b8b00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00bbe0 tx=0x7fda7c00b390 comp rx=0 tx=0).stop 2024-09-21T04:02:50.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.739+0000 7fda9142b640 1 -- 172.21.15.64:0/511674486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda8c10de60 msgr2=0x7fda8c19fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:50.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.739+0000 7fda9142b640 1 --2- 172.21.15.64:0/511674486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda8c10de60 0x7fda8c19fcb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fda8400eaa0 tx=0x7fda8400ef60 comp rx=0 tx=0).stop 2024-09-21T04:02:50.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.739+0000 7fda8b7fe640 1 -- 172.21.15.64:0/511674486 reap_dead start 2024-09-21T04:02:50.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.739+0000 7fda9142b640 1 -- 172.21.15.64:0/511674486 shutdown_connections 2024-09-21T04:02:50.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.739+0000 7fda9142b640 1 -- 172.21.15.64:0/511674486 >> 172.21.15.64:0/511674486 conn(0x7fda8c0797b0 msgr2=0x7fda8c100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:50.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.739+0000 7fda9142b640 1 -- 172.21.15.64:0/511674486 shutdown_connections 2024-09-21T04:02:50.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:50.739+0000 7fda9142b640 1 -- 172.21.15.64:0/511674486 wait complete. 2024-09-21T04:02:51.149 INFO:teuthology.orchestra.run.smithi064.stdout:244813135905 2024-09-21T04:02:51.150 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.9 2024-09-21T04:02:51.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.339+0000 7f50d639c640 1 -- 172.21.15.64:0/4211301193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50d01083a0 msgr2=0x7f50d01087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:51.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.339+0000 7f50d639c640 1 --2- 172.21.15.64:0/4211301193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50d01083a0 0x7f50d01087b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f50c4009900 tx=0x7f50c402a410 comp rx=0 tx=0).stop 2024-09-21T04:02:51.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.340+0000 7f50d639c640 1 -- 172.21.15.64:0/4211301193 shutdown_connections 2024-09-21T04:02:51.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.340+0000 7f50d639c640 1 --2- 172.21.15.64:0/4211301193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d0108cf0 0x7f50d010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:51.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.340+0000 7f50d639c640 1 --2- 172.21.15.64:0/4211301193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50d01083a0 0x7f50d01087b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:51.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.340+0000 7f50d639c640 1 --2- 172.21.15.64:0/4211301193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50d0102a60 0x7f50d0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:51.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.340+0000 7f50d639c640 1 -- 172.21.15.64:0/4211301193 >> 172.21.15.64:0/4211301193 conn(0x7f50d0079730 msgr2=0x7f50d0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:51.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.340+0000 7f50d639c640 1 -- 172.21.15.64:0/4211301193 shutdown_connections 2024-09-21T04:02:51.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.340+0000 7f50d639c640 1 -- 172.21.15.64:0/4211301193 wait complete. 2024-09-21T04:02:51.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.340+0000 7f50d639c640 1 Processor -- start 2024-09-21T04:02:51.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.340+0000 7f50d639c640 1 -- start start 2024-09-21T04:02:51.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.341+0000 7f50d639c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50d0102a60 0x7f50d019b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:51.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.341+0000 7f50d639c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50d01083a0 0x7f50d019b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:51.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.341+0000 7f50d639c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d0108cf0 0x7f50d019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:51.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.341+0000 7f50d639c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f50d0112960 con 0x7f50d01083a0 2024-09-21T04:02:51.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.341+0000 7f50d639c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f50d01127e0 con 0x7f50d0102a60 2024-09-21T04:02:51.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.341+0000 7f50d639c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f50d0112ae0 con 0x7f50d0108cf0 2024-09-21T04:02:51.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.341+0000 7f50cf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50d01083a0 0x7f50d019b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:51.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.341+0000 7f50cf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50d01083a0 0x7f50d019b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59226/0 (socket says 172.21.15.64:59226) 2024-09-21T04:02:51.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.341+0000 7f50cf7fe640 1 -- 172.21.15.64:0/240465824 learned_addr learned my addr 172.21.15.64:0/240465824 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:51.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.341+0000 7f50cffff640 1 --2- 172.21.15.64:0/240465824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50d0102a60 0x7f50d019b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:51.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.341+0000 7f50d4939640 1 --2- 172.21.15.64:0/240465824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d0108cf0 0x7f50d019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:51.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.341+0000 7f50cf7fe640 1 -- 172.21.15.64:0/240465824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d0108cf0 msgr2=0x7f50d019bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:51.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.341+0000 7f50cf7fe640 1 --2- 172.21.15.64:0/240465824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d0108cf0 0x7f50d019bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:51.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.341+0000 7f50cf7fe640 1 -- 172.21.15.64:0/240465824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50d0102a60 msgr2=0x7f50d019b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:51.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.341+0000 7f50cf7fe640 1 --2- 172.21.15.64:0/240465824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50d0102a60 0x7f50d019b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:51.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.341+0000 7f50cf7fe640 1 -- 172.21.15.64:0/240465824 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d01a64c0 con 0x7f50d01083a0 2024-09-21T04:02:51.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.342+0000 7f50cf7fe640 1 --2- 172.21.15.64:0/240465824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50d01083a0 0x7f50d019b770 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f50c4032fd0 tx=0x7f50c4031950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:51.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.342+0000 7f50cd7fa640 1 -- 172.21.15.64:0/240465824 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c4035070 con 0x7f50d01083a0 2024-09-21T04:02:51.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.342+0000 7f50cd7fa640 1 -- 172.21.15.64:0/240465824 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50c4041b60 con 0x7f50d01083a0 2024-09-21T04:02:51.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.342+0000 7f50d639c640 1 -- 172.21.15.64:0/240465824 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d0199020 con 0x7f50d01083a0 2024-09-21T04:02:51.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.342+0000 7f50cd7fa640 1 -- 172.21.15.64:0/240465824 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c40496c0 con 0x7f50d01083a0 2024-09-21T04:02:51.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.342+0000 7f50d639c640 1 -- 172.21.15.64:0/240465824 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d0199500 con 0x7f50d01083a0 2024-09-21T04:02:51.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.343+0000 7f50d639c640 1 -- 172.21.15.64:0/240465824 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f50d0063b50 con 0x7f50d01083a0 2024-09-21T04:02:51.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.346+0000 7f50cd7fa640 1 -- 172.21.15.64:0/240465824 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f50c40035f0 con 0x7f50d01083a0 2024-09-21T04:02:51.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.346+0000 7f50cd7fa640 1 --2- 172.21.15.64:0/240465824 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f50b80b6770 0x7f50b80b8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:51.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.347+0000 7f50cffff640 1 --2- 172.21.15.64:0/240465824 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f50b80b6770 0x7f50b80b8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:51.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.347+0000 7f50cd7fa640 1 -- 172.21.15.64:0/240465824 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f50c4110780 con 0x7f50d01083a0 2024-09-21T04:02:51.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.347+0000 7f50cd7fa640 1 --2- 172.21.15.64:0/240465824 >> [v2:172.21.15.64:6802/3692023035,v1:172.21.15.64:6803/3692023035] conn(0x7f50b80c05e0 0x7f50b80c2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:51.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.347+0000 7f50cd7fa640 1 -- 172.21.15.64:0/240465824 --> [v2:172.21.15.64:6802/3692023035,v1:172.21.15.64:6803/3692023035] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f50c4041db0 con 0x7f50b80c05e0 2024-09-21T04:02:51.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.347+0000 7f50cd7fa640 1 -- 172.21.15.64:0/240465824 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f50c4110bd0 con 0x7f50d01083a0 2024-09-21T04:02:51.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.347+0000 7f50d4939640 1 --2- 172.21.15.64:0/240465824 >> [v2:172.21.15.64:6802/3692023035,v1:172.21.15.64:6803/3692023035] conn(0x7f50b80c05e0 0x7f50b80c2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:51.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.347+0000 7f50cffff640 1 --2- 172.21.15.64:0/240465824 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f50b80b6770 0x7f50b80b8c80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f50d006b9c0 tx=0x7f50c000b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:51.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.347+0000 7f50d4939640 1 --2- 172.21.15.64:0/240465824 >> [v2:172.21.15.64:6802/3692023035,v1:172.21.15.64:6803/3692023035] conn(0x7f50b80c05e0 0x7f50b80c2ad0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:51.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.348+0000 7f50cd7fa640 1 -- 172.21.15.64:0/240465824 <== osd.0 v2:172.21.15.64:6802/3692023035 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f50c4041db0 con 0x7f50b80c05e0 2024-09-21T04:02:51.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.363+0000 7f50d639c640 1 -- 172.21.15.64:0/240465824 --> [v2:172.21.15.64:6802/3692023035,v1:172.21.15.64:6803/3692023035] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f50d0063f50 con 0x7f50b80c05e0 2024-09-21T04:02:51.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.364+0000 7f50cd7fa640 1 -- 172.21.15.64:0/240465824 <== osd.0 v2:172.21.15.64:6802/3692023035 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f50d0063f50 con 0x7f50b80c05e0 2024-09-21T04:02:51.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.364+0000 7f50d639c640 1 -- 172.21.15.64:0/240465824 >> [v2:172.21.15.64:6802/3692023035,v1:172.21.15.64:6803/3692023035] conn(0x7f50b80c05e0 msgr2=0x7f50b80c2ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:51.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.364+0000 7f50d639c640 1 --2- 172.21.15.64:0/240465824 >> [v2:172.21.15.64:6802/3692023035,v1:172.21.15.64:6803/3692023035] conn(0x7f50b80c05e0 0x7f50b80c2ad0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:51.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.364+0000 7f50d639c640 1 -- 172.21.15.64:0/240465824 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f50b80b6770 msgr2=0x7f50b80b8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:51.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.364+0000 7f50d639c640 1 --2- 172.21.15.64:0/240465824 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f50b80b6770 0x7f50b80b8c80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f50d006b9c0 tx=0x7f50c000b390 comp rx=0 tx=0).stop 2024-09-21T04:02:51.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.364+0000 7f50d639c640 1 -- 172.21.15.64:0/240465824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50d01083a0 msgr2=0x7f50d019b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:51.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.364+0000 7f50d639c640 1 --2- 172.21.15.64:0/240465824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50d01083a0 0x7f50d019b770 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f50c4032fd0 tx=0x7f50c4031950 comp rx=0 tx=0).stop 2024-09-21T04:02:51.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.364+0000 7f50d4939640 1 -- 172.21.15.64:0/240465824 reap_dead start 2024-09-21T04:02:51.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.365+0000 7f50d639c640 1 -- 172.21.15.64:0/240465824 shutdown_connections 2024-09-21T04:02:51.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.365+0000 7f50d639c640 1 -- 172.21.15.64:0/240465824 >> 172.21.15.64:0/240465824 conn(0x7f50d0079730 msgr2=0x7f50d0104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:51.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.365+0000 7f50d639c640 1 -- 172.21.15.64:0/240465824 shutdown_connections 2024-09-21T04:02:51.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:51.365+0000 7f50d639c640 1 -- 172.21.15.64:0/240465824 wait complete. 2024-09-21T04:02:52.030 INFO:teuthology.orchestra.run.smithi064.stdout:38654705743 2024-09-21T04:02:52.030 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-21T04:02:52.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:51 smithi119 ceph-mon[34811]: pgmap v284: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-21T04:02:52.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:51 smithi064 ceph-mon[31180]: pgmap v284: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-21T04:02:52.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:51 smithi131 ceph-mon[35193]: pgmap v284: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-21T04:02:52.721 INFO:teuthology.orchestra.run.smithi064.stdout:197568495657 2024-09-21T04:02:52.722 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.7 2024-09-21T04:02:53.096 INFO:teuthology.orchestra.run.smithi064.stdout:266287972379 2024-09-21T04:02:53.097 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.10 2024-09-21T04:02:53.751 INFO:teuthology.orchestra.run.smithi064.stdout:150323855413 2024-09-21T04:02:53.752 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-21T04:02:54.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:53 smithi064 ceph-mon[31180]: pgmap v285: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-21T04:02:54.155 INFO:teuthology.orchestra.run.smithi064.stdout:128849018938 2024-09-21T04:02:54.156 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-21T04:02:54.198 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:54.210 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:54.211 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:54.213 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:54.251 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:54.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:53 smithi119 ceph-mon[34811]: pgmap v285: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-21T04:02:54.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:53 smithi131 ceph-mon[35193]: pgmap v285: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-21T04:02:54.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.893+0000 7f441e96b640 1 -- 172.21.15.64:0/79670171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4418102a60 msgr2=0x7f4418102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:54.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.893+0000 7f441e96b640 1 --2- 172.21.15.64:0/79670171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4418102a60 0x7f4418102e70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4408009900 tx=0x7f440802a410 comp rx=0 tx=0).stop 2024-09-21T04:02:54.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.893+0000 7f441e96b640 1 -- 172.21.15.64:0/79670171 shutdown_connections 2024-09-21T04:02:54.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.893+0000 7f441e96b640 1 --2- 172.21.15.64:0/79670171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4418108cf0 0x7f441810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:54.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.893+0000 7f441e96b640 1 --2- 172.21.15.64:0/79670171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44181083a0 0x7f44181087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:54.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.893+0000 7f441e96b640 1 --2- 172.21.15.64:0/79670171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4418102a60 0x7f4418102e70 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:54.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.893+0000 7f441e96b640 1 -- 172.21.15.64:0/79670171 >> 172.21.15.64:0/79670171 conn(0x7f4418079730 msgr2=0x7f4418079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:54.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.893+0000 7f441e96b640 1 -- 172.21.15.64:0/79670171 shutdown_connections 2024-09-21T04:02:54.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.893+0000 7f441e96b640 1 -- 172.21.15.64:0/79670171 wait complete. 2024-09-21T04:02:54.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.894+0000 7f441e96b640 1 Processor -- start 2024-09-21T04:02:54.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.894+0000 7f441e96b640 1 -- start start 2024-09-21T04:02:54.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.895+0000 7f441e96b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4418102a60 0x7f4418199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:54.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.895+0000 7f441e96b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44181083a0 0x7f44181998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:54.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.895+0000 7f441e96b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4418108cf0 0x7f441819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:54.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.895+0000 7f441e96b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4418112960 con 0x7f44181083a0 2024-09-21T04:02:54.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.895+0000 7f441e96b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f44181127e0 con 0x7f4418108cf0 2024-09-21T04:02:54.898 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.895+0000 7f441e96b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4418112ae0 con 0x7f4418102a60 2024-09-21T04:02:54.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.895+0000 7f44177fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44181083a0 0x7f44181998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:54.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.895+0000 7f44177fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44181083a0 0x7f44181998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47896/0 (socket says 172.21.15.64:47896) 2024-09-21T04:02:54.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.895+0000 7f44177fe640 1 -- 172.21.15.64:0/1808805388 learned_addr learned my addr 172.21.15.64:0/1808805388 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:54.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.895+0000 7f441cf08640 1 --2- 172.21.15.64:0/1808805388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4418108cf0 0x7f441819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:54.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.895+0000 7f4417fff640 1 --2- 172.21.15.64:0/1808805388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4418102a60 0x7f4418199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:54.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.895+0000 7f44177fe640 1 -- 172.21.15.64:0/1808805388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4418102a60 msgr2=0x7f4418199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:54.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.895+0000 7f44177fe640 1 --2- 172.21.15.64:0/1808805388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4418102a60 0x7f4418199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:54.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.895+0000 7f44177fe640 1 -- 172.21.15.64:0/1808805388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4418108cf0 msgr2=0x7f441819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:54.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.895+0000 7f44177fe640 1 --2- 172.21.15.64:0/1808805388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4418108cf0 0x7f441819eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:54.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.895+0000 7f44177fe640 1 -- 172.21.15.64:0/1808805388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f441819f2a0 con 0x7f44181083a0 2024-09-21T04:02:54.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.895+0000 7f44177fe640 1 --2- 172.21.15.64:0/1808805388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44181083a0 0x7f44181998d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f440c00d770 tx=0x7f440c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:54.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.896+0000 7f44157fa640 1 -- 172.21.15.64:0/1808805388 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c017070 con 0x7f44181083a0 2024-09-21T04:02:54.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.896+0000 7f44157fa640 1 -- 172.21.15.64:0/1808805388 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f440c009c30 con 0x7f44181083a0 2024-09-21T04:02:54.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.896+0000 7f441e96b640 1 -- 172.21.15.64:0/1808805388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f441819f590 con 0x7f44181083a0 2024-09-21T04:02:54.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.896+0000 7f44157fa640 1 -- 172.21.15.64:0/1808805388 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c00f620 con 0x7f44181083a0 2024-09-21T04:02:54.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.896+0000 7f441e96b640 1 -- 172.21.15.64:0/1808805388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44181a5cf0 con 0x7f44181083a0 2024-09-21T04:02:54.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.897+0000 7f441e96b640 1 -- 172.21.15.64:0/1808805388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4418102e70 con 0x7f44181083a0 2024-09-21T04:02:54.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.898+0000 7f44157fa640 1 -- 172.21.15.64:0/1808805388 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f440c01a070 con 0x7f44181083a0 2024-09-21T04:02:54.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.899+0000 7f44157fa640 1 --2- 172.21.15.64:0/1808805388 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f44000b65f0 0x7f44000b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:54.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.900+0000 7f4417fff640 1 --2- 172.21.15.64:0/1808805388 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f44000b65f0 0x7f44000b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:54.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.900+0000 7f44157fa640 1 -- 172.21.15.64:0/1808805388 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f440c0e7030 con 0x7f44181083a0 2024-09-21T04:02:54.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.900+0000 7f4417fff640 1 --2- 172.21.15.64:0/1808805388 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f44000b65f0 0x7f44000b8b00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4408031440 tx=0x7f4408009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:54.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:54.903+0000 7f44157fa640 1 -- 172.21.15.64:0/1808805388 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f440c009de0 con 0x7f44181083a0 2024-09-21T04:02:55.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.094+0000 7f441e96b640 1 -- 172.21.15.64:0/1808805388 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f441819a460 con 0x7f44181083a0 2024-09-21T04:02:55.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.095+0000 7f44157fa640 1 -- 172.21.15.64:0/1808805388 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f441819a460 con 0x7f44181083a0 2024-09-21T04:02:55.097 INFO:teuthology.orchestra.run.smithi064.stdout:219043332135 2024-09-21T04:02:55.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.097+0000 7f441e96b640 1 -- 172.21.15.64:0/1808805388 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f44000b65f0 msgr2=0x7f44000b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:55.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.097+0000 7f441e96b640 1 --2- 172.21.15.64:0/1808805388 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f44000b65f0 0x7f44000b8b00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4408031440 tx=0x7f4408009560 comp rx=0 tx=0).stop 2024-09-21T04:02:55.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.097+0000 7f441e96b640 1 -- 172.21.15.64:0/1808805388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44181083a0 msgr2=0x7f44181998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:55.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.097+0000 7f441e96b640 1 --2- 172.21.15.64:0/1808805388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44181083a0 0x7f44181998d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f440c00d770 tx=0x7f440c00dc30 comp rx=0 tx=0).stop 2024-09-21T04:02:55.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.098+0000 7f441e96b640 1 -- 172.21.15.64:0/1808805388 shutdown_connections 2024-09-21T04:02:55.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.098+0000 7f441e96b640 1 --2- 172.21.15.64:0/1808805388 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f44000b65f0 0x7f44000b8b00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:55.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.098+0000 7f441e96b640 1 --2- 172.21.15.64:0/1808805388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4418108cf0 0x7f441819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:55.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.098+0000 7f441e96b640 1 --2- 172.21.15.64:0/1808805388 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f44181083a0 0x7f44181998d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:55.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.098+0000 7f441e96b640 1 --2- 172.21.15.64:0/1808805388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4418102a60 0x7f4418199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:55.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.098+0000 7f441e96b640 1 -- 172.21.15.64:0/1808805388 >> 172.21.15.64:0/1808805388 conn(0x7f4418079730 msgr2=0x7f4418101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:55.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.098+0000 7f441e96b640 1 -- 172.21.15.64:0/1808805388 shutdown_connections 2024-09-21T04:02:55.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.098+0000 7f441e96b640 1 -- 172.21.15.64:0/1808805388 wait complete. 2024-09-21T04:02:55.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.118+0000 7f4daf703640 1 -- 172.21.15.64:0/2610737607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da8108d00 msgr2=0x7f4da810f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:55.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.118+0000 7f4daf703640 1 --2- 172.21.15.64:0/2610737607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da8108d00 0x7f4da810f530 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4da0009ca0 tx=0x7f4da002a8c0 comp rx=0 tx=0).stop 2024-09-21T04:02:55.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.118+0000 7f4daf703640 1 -- 172.21.15.64:0/2610737607 shutdown_connections 2024-09-21T04:02:55.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.118+0000 7f4daf703640 1 --2- 172.21.15.64:0/2610737607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da8108d00 0x7f4da810f530 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:55.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.118+0000 7f4daf703640 1 --2- 172.21.15.64:0/2610737607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4da81083b0 0x7f4da81087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:55.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.118+0000 7f4daf703640 1 --2- 172.21.15.64:0/2610737607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4da8102ac0 0x7f4da8102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:55.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.118+0000 7f4daf703640 1 -- 172.21.15.64:0/2610737607 >> 172.21.15.64:0/2610737607 conn(0x7f4da8079790 msgr2=0x7f4da8079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:55.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.118+0000 7f4daf703640 1 -- 172.21.15.64:0/2610737607 shutdown_connections 2024-09-21T04:02:55.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.118+0000 7f4daf703640 1 -- 172.21.15.64:0/2610737607 wait complete. 2024-09-21T04:02:55.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.119+0000 7f4daf703640 1 Processor -- start 2024-09-21T04:02:55.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.119+0000 7f4daf703640 1 -- start start 2024-09-21T04:02:55.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.119+0000 7f4daf703640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4da8102ac0 0x7f4da8199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:55.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.119+0000 7f4daf703640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da81083b0 0x7f4da81998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:55.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.119+0000 7f4daf703640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4da8108d00 0x7f4da819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:55.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.119+0000 7f4daf703640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4da81129c0 con 0x7f4da81083b0 2024-09-21T04:02:55.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.119+0000 7f4daf703640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4da8112840 con 0x7f4da8102ac0 2024-09-21T04:02:55.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.119+0000 7f4daf703640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4da8112b40 con 0x7f4da8108d00 2024-09-21T04:02:55.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.120+0000 7f4dacc9e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da81083b0 0x7f4da81998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:55.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.120+0000 7f4dacc9e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da81083b0 0x7f4da81998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47910/0 (socket says 172.21.15.64:47910) 2024-09-21T04:02:55.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.120+0000 7f4dacc9e640 1 -- 172.21.15.64:0/2465035949 learned_addr learned my addr 172.21.15.64:0/2465035949 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:55.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.120+0000 7f4dad49f640 1 --2- 172.21.15.64:0/2465035949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4da8102ac0 0x7f4da8199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:55.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.120+0000 7f4dadca0640 1 --2- 172.21.15.64:0/2465035949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4da8108d00 0x7f4da819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:55.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.120+0000 7f4dad49f640 1 -- 172.21.15.64:0/2465035949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4da8108d00 msgr2=0x7f4da819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:55.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.120+0000 7f4dad49f640 1 --2- 172.21.15.64:0/2465035949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4da8108d00 0x7f4da819eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:55.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.120+0000 7f4dad49f640 1 -- 172.21.15.64:0/2465035949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da81083b0 msgr2=0x7f4da81998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:55.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.120+0000 7f4dad49f640 1 --2- 172.21.15.64:0/2465035949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da81083b0 0x7f4da81998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:55.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.120+0000 7f4dad49f640 1 -- 172.21.15.64:0/2465035949 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da819f2a0 con 0x7f4da8102ac0 2024-09-21T04:02:55.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.120+0000 7f4dacc9e640 1 --2- 172.21.15.64:0/2465035949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da81083b0 0x7f4da81998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:02:55.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.120+0000 7f4dadca0640 1 --2- 172.21.15.64:0/2465035949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4da8108d00 0x7f4da819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:02:55.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.120+0000 7f4dad49f640 1 --2- 172.21.15.64:0/2465035949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4da8102ac0 0x7f4da8199330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4da400d720 tx=0x7f4da400dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:55.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.121+0000 7f4d9e7fc640 1 -- 172.21.15.64:0/2465035949 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4017070 con 0x7f4da8102ac0 2024-09-21T04:02:55.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.121+0000 7f4d9e7fc640 1 -- 172.21.15.64:0/2465035949 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4da400de30 con 0x7f4da8102ac0 2024-09-21T04:02:55.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.121+0000 7f4d9e7fc640 1 -- 172.21.15.64:0/2465035949 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da40047a0 con 0x7f4da8102ac0 2024-09-21T04:02:55.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.121+0000 7f4daf703640 1 -- 172.21.15.64:0/2465035949 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da819f590 con 0x7f4da8102ac0 2024-09-21T04:02:55.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.121+0000 7f4daf703640 1 -- 172.21.15.64:0/2465035949 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da806b820 con 0x7f4da8102ac0 2024-09-21T04:02:55.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.122+0000 7f4daf703640 1 -- 172.21.15.64:0/2465035949 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da8102ed0 con 0x7f4da8102ac0 2024-09-21T04:02:55.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.124+0000 7f4d9e7fc640 1 -- 172.21.15.64:0/2465035949 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f4da40099d0 con 0x7f4da8102ac0 2024-09-21T04:02:55.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.125+0000 7f4d9e7fc640 1 --2- 172.21.15.64:0/2465035949 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4d940b65f0 0x7f4d940b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:55.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.125+0000 7f4d9e7fc640 1 -- 172.21.15.64:0/2465035949 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f4da40e7400 con 0x7f4da8102ac0 2024-09-21T04:02:55.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.125+0000 7f4dacc9e640 1 --2- 172.21.15.64:0/2465035949 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4d940b65f0 0x7f4d940b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:55.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.126+0000 7f4dacc9e640 1 --2- 172.21.15.64:0/2465035949 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4d940b65f0 0x7f4d940b8b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4da8079d30 tx=0x7f4d9800b440 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:55.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.128+0000 7f4d9e7fc640 1 -- 172.21.15.64:0/2465035949 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4da4020020 con 0x7f4da8102ac0 2024-09-21T04:02:55.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.320+0000 7f4daf703640 1 -- 172.21.15.64:0/2465035949 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f4da806c2d0 con 0x7f4da8102ac0 2024-09-21T04:02:55.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.320+0000 7f4d9e7fc640 1 -- 172.21.15.64:0/2465035949 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4da40ada30 con 0x7f4da8102ac0 2024-09-21T04:02:55.323 INFO:teuthology.orchestra.run.smithi064.stdout:107374182464 2024-09-21T04:02:55.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.322+0000 7f4daf703640 1 -- 172.21.15.64:0/2465035949 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4d940b65f0 msgr2=0x7f4d940b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:55.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.322+0000 7f4daf703640 1 --2- 172.21.15.64:0/2465035949 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4d940b65f0 0x7f4d940b8b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4da8079d30 tx=0x7f4d9800b440 comp rx=0 tx=0).stop 2024-09-21T04:02:55.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.323+0000 7f4daf703640 1 -- 172.21.15.64:0/2465035949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4da8102ac0 msgr2=0x7f4da8199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:55.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.323+0000 7f4daf703640 1 --2- 172.21.15.64:0/2465035949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4da8102ac0 0x7f4da8199330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4da400d720 tx=0x7f4da400dbe0 comp rx=0 tx=0).stop 2024-09-21T04:02:55.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.323+0000 7f4daf703640 1 -- 172.21.15.64:0/2465035949 shutdown_connections 2024-09-21T04:02:55.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.323+0000 7f4daf703640 1 --2- 172.21.15.64:0/2465035949 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4d940b65f0 0x7f4d940b8b00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:55.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.323+0000 7f4daf703640 1 --2- 172.21.15.64:0/2465035949 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4da8108d00 0x7f4da819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:55.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.323+0000 7f4daf703640 1 --2- 172.21.15.64:0/2465035949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4da81083b0 0x7f4da81998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:55.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.323+0000 7f4daf703640 1 --2- 172.21.15.64:0/2465035949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4da8102ac0 0x7f4da8199330 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:55.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.323+0000 7f4daf703640 1 -- 172.21.15.64:0/2465035949 >> 172.21.15.64:0/2465035949 conn(0x7f4da8079790 msgr2=0x7f4da8104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:55.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.323+0000 7f4daf703640 1 -- 172.21.15.64:0/2465035949 shutdown_connections 2024-09-21T04:02:55.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:55.323+0000 7f4daf703640 1 -- 172.21.15.64:0/2465035949 wait complete. 2024-09-21T04:02:55.940 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332133 got 219043332135 for osd.8 2024-09-21T04:02:55.941 DEBUG:teuthology.parallel:result is None 2024-09-21T04:02:56.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:55 smithi119 ceph-mon[34811]: pgmap v286: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:02:56.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:55 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1808805388' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T04:02:56.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:55 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2465035949' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T04:02:56.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:55 smithi064 ceph-mon[31180]: pgmap v286: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:02:56.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:55 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1808805388' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T04:02:56.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:55 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2465035949' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T04:02:56.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:55 smithi131 ceph-mon[35193]: pgmap v286: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:02:56.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:55 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1808805388' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T04:02:56.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:55 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2465035949' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T04:02:56.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.361+0000 7f49d4e7d640 1 -- 172.21.15.64:0/1223401978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49d01094a0 msgr2=0x7f49d01098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:56.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.361+0000 7f49d4e7d640 1 --2- 172.21.15.64:0/1223401978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49d01094a0 0x7f49d01098b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009900 tx=0x7f49c402a410 comp rx=0 tx=0).stop 2024-09-21T04:02:56.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.362+0000 7f49d4e7d640 1 -- 172.21.15.64:0/1223401978 shutdown_connections 2024-09-21T04:02:56.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.362+0000 7f49d4e7d640 1 --2- 172.21.15.64:0/1223401978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49d0109df0 0x7f49d0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:56.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.362+0000 7f49d4e7d640 1 --2- 172.21.15.64:0/1223401978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49d01094a0 0x7f49d01098b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:56.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.362+0000 7f49d4e7d640 1 --2- 172.21.15.64:0/1223401978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49d0103b60 0x7f49d0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:56.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.362+0000 7f49d4e7d640 1 -- 172.21.15.64:0/1223401978 >> 172.21.15.64:0/1223401978 conn(0x7f49d007b850 msgr2=0x7f49d007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:56.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.362+0000 7f49d4e7d640 1 -- 172.21.15.64:0/1223401978 shutdown_connections 2024-09-21T04:02:56.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.362+0000 7f49d4e7d640 1 -- 172.21.15.64:0/1223401978 wait complete. 2024-09-21T04:02:56.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.362+0000 7f49d4e7d640 1 Processor -- start 2024-09-21T04:02:56.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.362+0000 7f49d4e7d640 1 -- start start 2024-09-21T04:02:56.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.363+0000 7f49d4e7d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49d0103b60 0x7f49d019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:56.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.363+0000 7f49d4e7d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49d01094a0 0x7f49d019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:56.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.363+0000 7f49d4e7d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49d0109df0 0x7f49d019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:56.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.363+0000 7f49d4e7d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f49d0113ab0 con 0x7f49d0103b60 2024-09-21T04:02:56.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.363+0000 7f49d4e7d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f49d0113930 con 0x7f49d0109df0 2024-09-21T04:02:56.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.363+0000 7f49d4e7d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f49d0113c30 con 0x7f49d01094a0 2024-09-21T04:02:56.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.363+0000 7f49ce59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49d0103b60 0x7f49d019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:56.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.363+0000 7f49ce59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49d0103b60 0x7f49d019a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47938/0 (socket says 172.21.15.64:47938) 2024-09-21T04:02:56.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.363+0000 7f49ce59c640 1 -- 172.21.15.64:0/2635175082 learned_addr learned my addr 172.21.15.64:0/2635175082 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:56.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.363+0000 7f49cdd9b640 1 --2- 172.21.15.64:0/2635175082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49d01094a0 0x7f49d019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:56.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.363+0000 7f49ced9d640 1 --2- 172.21.15.64:0/2635175082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49d0109df0 0x7f49d019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:56.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.363+0000 7f49ce59c640 1 -- 172.21.15.64:0/2635175082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49d01094a0 msgr2=0x7f49d019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:56.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.363+0000 7f49ce59c640 1 --2- 172.21.15.64:0/2635175082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49d01094a0 0x7f49d019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:56.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.363+0000 7f49ce59c640 1 -- 172.21.15.64:0/2635175082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49d0109df0 msgr2=0x7f49d019fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:56.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.363+0000 7f49ce59c640 1 --2- 172.21.15.64:0/2635175082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49d0109df0 0x7f49d019fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:56.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.363+0000 7f49ce59c640 1 -- 172.21.15.64:0/2635175082 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d01a0470 con 0x7f49d0103b60 2024-09-21T04:02:56.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.363+0000 7f49ced9d640 1 --2- 172.21.15.64:0/2635175082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49d0109df0 0x7f49d019fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:02:56.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.363+0000 7f49ce59c640 1 --2- 172.21.15.64:0/2635175082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49d0103b60 0x7f49d019a500 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f49c000d950 tx=0x7f49c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:56.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.364+0000 7f49bf7fe640 1 -- 172.21.15.64:0/2635175082 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0017070 con 0x7f49d0103b60 2024-09-21T04:02:56.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.364+0000 7f49bf7fe640 1 -- 172.21.15.64:0/2635175082 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49c0009c30 con 0x7f49d0103b60 2024-09-21T04:02:56.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.364+0000 7f49bf7fe640 1 -- 172.21.15.64:0/2635175082 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c000f620 con 0x7f49d0103b60 2024-09-21T04:02:56.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.364+0000 7f49d4e7d640 1 -- 172.21.15.64:0/2635175082 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d01a0760 con 0x7f49d0103b60 2024-09-21T04:02:56.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.364+0000 7f49d4e7d640 1 -- 172.21.15.64:0/2635175082 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d01a6ec0 con 0x7f49d0103b60 2024-09-21T04:02:56.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.365+0000 7f49d4e7d640 1 -- 172.21.15.64:0/2635175082 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d0103f70 con 0x7f49d0103b60 2024-09-21T04:02:56.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.366+0000 7f49bf7fe640 1 -- 172.21.15.64:0/2635175082 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f49c0005030 con 0x7f49d0103b60 2024-09-21T04:02:56.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.367+0000 7f49bf7fe640 1 --2- 172.21.15.64:0/2635175082 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f49b80b6670 0x7f49b80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:56.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.368+0000 7f49cdd9b640 1 --2- 172.21.15.64:0/2635175082 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f49b80b6670 0x7f49b80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:56.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.368+0000 7f49bf7fe640 1 -- 172.21.15.64:0/2635175082 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f49c00e70d0 con 0x7f49d0103b60 2024-09-21T04:02:56.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.368+0000 7f49cdd9b640 1 --2- 172.21.15.64:0/2635175082 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f49b80b6670 0x7f49b80b8b80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009230 tx=0x7f49c4031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:56.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.371+0000 7f49bf7fe640 1 -- 172.21.15.64:0/2635175082 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f49c00ad820 con 0x7f49d0103b60 2024-09-21T04:02:56.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.561+0000 7f49d4e7d640 1 -- 172.21.15.64:0/2635175082 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f49d019b630 con 0x7f49d0103b60 2024-09-21T04:02:56.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.561+0000 7f49bf7fe640 1 -- 172.21.15.64:0/2635175082 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f49c0009de0 con 0x7f49d0103b60 2024-09-21T04:02:56.564 INFO:teuthology.orchestra.run.smithi064.stdout:176093659185 2024-09-21T04:02:56.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.564+0000 7f49d4e7d640 1 -- 172.21.15.64:0/2635175082 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f49b80b6670 msgr2=0x7f49b80b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:56.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.564+0000 7f49d4e7d640 1 --2- 172.21.15.64:0/2635175082 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f49b80b6670 0x7f49b80b8b80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009230 tx=0x7f49c4031400 comp rx=0 tx=0).stop 2024-09-21T04:02:56.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.564+0000 7f49d4e7d640 1 -- 172.21.15.64:0/2635175082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49d0103b60 msgr2=0x7f49d019a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:56.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.564+0000 7f49d4e7d640 1 --2- 172.21.15.64:0/2635175082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49d0103b60 0x7f49d019a500 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f49c000d950 tx=0x7f49c000de10 comp rx=0 tx=0).stop 2024-09-21T04:02:56.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.564+0000 7f49d4e7d640 1 -- 172.21.15.64:0/2635175082 shutdown_connections 2024-09-21T04:02:56.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.564+0000 7f49d4e7d640 1 --2- 172.21.15.64:0/2635175082 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f49b80b6670 0x7f49b80b8b80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:56.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.564+0000 7f49d4e7d640 1 --2- 172.21.15.64:0/2635175082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49d0109df0 0x7f49d019fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:56.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.564+0000 7f49d4e7d640 1 --2- 172.21.15.64:0/2635175082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49d01094a0 0x7f49d019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:56.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.564+0000 7f49d4e7d640 1 --2- 172.21.15.64:0/2635175082 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49d0103b60 0x7f49d019a500 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:56.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.564+0000 7f49d4e7d640 1 -- 172.21.15.64:0/2635175082 >> 172.21.15.64:0/2635175082 conn(0x7f49d007b850 msgr2=0x7f49d0105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:56.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.564+0000 7f49d4e7d640 1 -- 172.21.15.64:0/2635175082 shutdown_connections 2024-09-21T04:02:56.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:56.564+0000 7f49d4e7d640 1 -- 172.21.15.64:0/2635175082 wait complete. 2024-09-21T04:02:57.227 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:56 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2635175082' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T04:02:57.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:56 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2635175082' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T04:02:57.257 INFO:tasks.cephadm.ceph_manager.ceph:need seq 176093659184 got 176093659185 for osd.6 2024-09-21T04:02:57.257 DEBUG:teuthology.parallel:result is None 2024-09-21T04:02:57.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:56 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2635175082' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T04:02:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.464+0000 7f4668b9d640 1 -- 172.21.15.64:0/3383956043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46641081b0 msgr2=0x7f46641085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.464+0000 7f4668b9d640 1 --2- 172.21.15.64:0/3383956043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46641081b0 0x7f46641085c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f464c009900 tx=0x7f464c02a410 comp rx=0 tx=0).stop 2024-09-21T04:02:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.464+0000 7f4668b9d640 1 -- 172.21.15.64:0/3383956043 shutdown_connections 2024-09-21T04:02:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.464+0000 7f4668b9d640 1 --2- 172.21.15.64:0/3383956043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4664108b00 0x7f466410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.464+0000 7f4668b9d640 1 --2- 172.21.15.64:0/3383956043 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46641081b0 0x7f46641085c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.464+0000 7f4668b9d640 1 --2- 172.21.15.64:0/3383956043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4664102870 0x7f4664102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.464+0000 7f4668b9d640 1 -- 172.21.15.64:0/3383956043 >> 172.21.15.64:0/3383956043 conn(0x7f46640fe2c0 msgr2=0x7f4664100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.464+0000 7f4668b9d640 1 -- 172.21.15.64:0/3383956043 shutdown_connections 2024-09-21T04:02:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.464+0000 7f4668b9d640 1 -- 172.21.15.64:0/3383956043 wait complete. 2024-09-21T04:02:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.465+0000 7f4668b9d640 1 Processor -- start 2024-09-21T04:02:57.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.465+0000 7f4668b9d640 1 -- start start 2024-09-21T04:02:57.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.465+0000 7f4668b9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4664102870 0x7f46640fe7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:57.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.465+0000 7f4668b9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46641081b0 0x7f46640fed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:57.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.465+0000 7f4668b9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4664108b00 0x7f46640ff270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:57.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.465+0000 7f4668b9d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4664112770 con 0x7f46641081b0 2024-09-21T04:02:57.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.465+0000 7f4668b9d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f46641125f0 con 0x7f4664102870 2024-09-21T04:02:57.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.465+0000 7f4668b9d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f46641128f0 con 0x7f4664108b00 2024-09-21T04:02:57.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.465+0000 7f4662ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46641081b0 0x7f46640fed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:57.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.465+0000 7f4662ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46641081b0 0x7f46640fed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47962/0 (socket says 172.21.15.64:47962) 2024-09-21T04:02:57.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.465+0000 7f4662ffd640 1 -- 172.21.15.64:0/1307369784 learned_addr learned my addr 172.21.15.64:0/1307369784 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:57.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.466+0000 7f46637fe640 1 --2- 172.21.15.64:0/1307369784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4664102870 0x7f46640fe7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:57.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.466+0000 7f4663fff640 1 --2- 172.21.15.64:0/1307369784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4664108b00 0x7f46640ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:57.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.466+0000 7f4662ffd640 1 -- 172.21.15.64:0/1307369784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4664108b00 msgr2=0x7f46640ff270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:57.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.466+0000 7f4662ffd640 1 --2- 172.21.15.64:0/1307369784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4664108b00 0x7f46640ff270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:57.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.466+0000 7f4662ffd640 1 -- 172.21.15.64:0/1307369784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4664102870 msgr2=0x7f46640fe7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:57.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.466+0000 7f4662ffd640 1 --2- 172.21.15.64:0/1307369784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4664102870 0x7f46640fe7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:57.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.466+0000 7f4662ffd640 1 -- 172.21.15.64:0/1307369784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46641a8580 con 0x7f46641081b0 2024-09-21T04:02:57.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.466+0000 7f46637fe640 1 --2- 172.21.15.64:0/1307369784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4664102870 0x7f46640fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:02:57.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.466+0000 7f4663fff640 1 --2- 172.21.15.64:0/1307369784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4664108b00 0x7f46640ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:02:57.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.466+0000 7f4662ffd640 1 --2- 172.21.15.64:0/1307369784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46641081b0 0x7f46640fed30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f464c02a930 tx=0x7f464c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:57.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.466+0000 7f4660ff9640 1 -- 172.21.15.64:0/1307369784 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464c03f030 con 0x7f46641081b0 2024-09-21T04:02:57.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.466+0000 7f4660ff9640 1 -- 172.21.15.64:0/1307369784 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f464c003aa0 con 0x7f46641081b0 2024-09-21T04:02:57.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.466+0000 7f4668b9d640 1 -- 172.21.15.64:0/1307369784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46641a8810 con 0x7f46641081b0 2024-09-21T04:02:57.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.466+0000 7f4660ff9640 1 -- 172.21.15.64:0/1307369784 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464c035070 con 0x7f46641081b0 2024-09-21T04:02:57.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.466+0000 7f4668b9d640 1 -- 172.21.15.64:0/1307369784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46641a8cf0 con 0x7f46641081b0 2024-09-21T04:02:57.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.467+0000 7f4668b9d640 1 -- 172.21.15.64:0/1307369784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4664102c80 con 0x7f46641081b0 2024-09-21T04:02:57.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.469+0000 7f4660ff9640 1 -- 172.21.15.64:0/1307369784 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f464c042030 con 0x7f46641081b0 2024-09-21T04:02:57.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.470+0000 7f4660ff9640 1 --2- 172.21.15.64:0/1307369784 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f46500b65f0 0x7f46500b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:57.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.470+0000 7f46637fe640 1 --2- 172.21.15.64:0/1307369784 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f46500b65f0 0x7f46500b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:57.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.470+0000 7f4660ff9640 1 -- 172.21.15.64:0/1307369784 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f464c1081d0 con 0x7f46641081b0 2024-09-21T04:02:57.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.470+0000 7f46637fe640 1 --2- 172.21.15.64:0/1307369784 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f46500b65f0 0x7f46500b8b00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4654005fd0 tx=0x7f4654005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:57.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.473+0000 7f4660ff9640 1 -- 172.21.15.64:0/1307369784 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f464c0ce800 con 0x7f46641081b0 2024-09-21T04:02:57.504 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:02:57.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.693+0000 7f4668b9d640 1 -- 172.21.15.64:0/1307369784 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f46641a90b0 con 0x7f46641081b0 2024-09-21T04:02:57.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.694+0000 7f4660ff9640 1 -- 172.21.15.64:0/1307369784 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f464c0d2be0 con 0x7f46641081b0 2024-09-21T04:02:57.696 INFO:teuthology.orchestra.run.smithi064.stdout:60129542219 2024-09-21T04:02:57.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.696+0000 7f4668b9d640 1 -- 172.21.15.64:0/1307369784 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f46500b65f0 msgr2=0x7f46500b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:57.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.696+0000 7f4668b9d640 1 --2- 172.21.15.64:0/1307369784 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f46500b65f0 0x7f46500b8b00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4654005fd0 tx=0x7f4654005850 comp rx=0 tx=0).stop 2024-09-21T04:02:57.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.696+0000 7f4668b9d640 1 -- 172.21.15.64:0/1307369784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46641081b0 msgr2=0x7f46640fed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:57.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.696+0000 7f4668b9d640 1 --2- 172.21.15.64:0/1307369784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46641081b0 0x7f46640fed30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f464c02a930 tx=0x7f464c004270 comp rx=0 tx=0).stop 2024-09-21T04:02:57.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.696+0000 7f4668b9d640 1 -- 172.21.15.64:0/1307369784 shutdown_connections 2024-09-21T04:02:57.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.696+0000 7f4668b9d640 1 --2- 172.21.15.64:0/1307369784 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f46500b65f0 0x7f46500b8b00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:57.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.696+0000 7f4668b9d640 1 --2- 172.21.15.64:0/1307369784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4664108b00 0x7f46640ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:57.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.696+0000 7f4668b9d640 1 --2- 172.21.15.64:0/1307369784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f46641081b0 0x7f46640fed30 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:57.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.696+0000 7f4668b9d640 1 --2- 172.21.15.64:0/1307369784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4664102870 0x7f46640fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:57.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.696+0000 7f4668b9d640 1 -- 172.21.15.64:0/1307369784 >> 172.21.15.64:0/1307369784 conn(0x7f46640fe2c0 msgr2=0x7f46641068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:57.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.696+0000 7f4668b9d640 1 -- 172.21.15.64:0/1307369784 shutdown_connections 2024-09-21T04:02:57.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:57.696+0000 7f4668b9d640 1 -- 172.21.15.64:0/1307369784 wait complete. 2024-09-21T04:02:58.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.108+0000 7f23962c2640 1 -- 172.21.15.64:0/2104157989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2390102a90 msgr2=0x7f2390102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:58.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.108+0000 7f23962c2640 1 --2- 172.21.15.64:0/2104157989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2390102a90 0x7f2390102ea0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2380009900 tx=0x7f238002a410 comp rx=0 tx=0).stop 2024-09-21T04:02:58.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.109+0000 7f23962c2640 1 -- 172.21.15.64:0/2104157989 shutdown_connections 2024-09-21T04:02:58.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.109+0000 7f23962c2640 1 --2- 172.21.15.64:0/2104157989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2390108d20 0x7f239010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:58.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.109+0000 7f23962c2640 1 --2- 172.21.15.64:0/2104157989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23901083d0 0x7f23901087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:58.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.109+0000 7f23962c2640 1 --2- 172.21.15.64:0/2104157989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2390102a90 0x7f2390102ea0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:58.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.109+0000 7f23962c2640 1 -- 172.21.15.64:0/2104157989 >> 172.21.15.64:0/2104157989 conn(0x7f2390079750 msgr2=0x7f2390079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:58.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.109+0000 7f23962c2640 1 -- 172.21.15.64:0/2104157989 shutdown_connections 2024-09-21T04:02:58.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.109+0000 7f23962c2640 1 -- 172.21.15.64:0/2104157989 wait complete. 2024-09-21T04:02:58.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.109+0000 7f23962c2640 1 Processor -- start 2024-09-21T04:02:58.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.109+0000 7f23962c2640 1 -- start start 2024-09-21T04:02:58.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.110+0000 7f23962c2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2390102a90 0x7f239019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:58.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.110+0000 7f23962c2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23901083d0 0x7f239019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:58.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.110+0000 7f23962c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2390108d20 0x7f239019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:58.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.110+0000 7f23962c2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2390112a40 con 0x7f2390102a90 2024-09-21T04:02:58.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.110+0000 7f23962c2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f23901128c0 con 0x7f2390108d20 2024-09-21T04:02:58.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.110+0000 7f23962c2640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2390112bc0 con 0x7f23901083d0 2024-09-21T04:02:58.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.110+0000 7f238ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2390102a90 0x7f239019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:58.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.110+0000 7f238ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2390102a90 0x7f239019a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47976/0 (socket says 172.21.15.64:47976) 2024-09-21T04:02:58.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.110+0000 7f238ffff640 1 -- 172.21.15.64:0/3611363403 learned_addr learned my addr 172.21.15.64:0/3611363403 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:02:58.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.110+0000 7f238f7fe640 1 --2- 172.21.15.64:0/3611363403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23901083d0 0x7f239019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:58.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.110+0000 7f239485f640 1 --2- 172.21.15.64:0/3611363403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2390108d20 0x7f239019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:58.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.110+0000 7f238ffff640 1 -- 172.21.15.64:0/3611363403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23901083d0 msgr2=0x7f239019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:58.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.110+0000 7f238ffff640 1 --2- 172.21.15.64:0/3611363403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23901083d0 0x7f239019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:58.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.110+0000 7f238ffff640 1 -- 172.21.15.64:0/3611363403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2390108d20 msgr2=0x7f239019fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:58.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.110+0000 7f238ffff640 1 --2- 172.21.15.64:0/3611363403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2390108d20 0x7f239019fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:58.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.110+0000 7f238ffff640 1 -- 172.21.15.64:0/3611363403 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23901a03b0 con 0x7f2390102a90 2024-09-21T04:02:58.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.111+0000 7f238ffff640 1 --2- 172.21.15.64:0/3611363403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2390102a90 0x7f239019a500 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2380005ab0 tx=0x7f2380003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:58.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.111+0000 7f238d7fa640 1 -- 172.21.15.64:0/3611363403 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238003f070 con 0x7f2390102a90 2024-09-21T04:02:58.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.111+0000 7f23962c2640 1 -- 172.21.15.64:0/3611363403 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23901a0640 con 0x7f2390102a90 2024-09-21T04:02:58.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.111+0000 7f238d7fa640 1 -- 172.21.15.64:0/3611363403 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2380031d60 con 0x7f2390102a90 2024-09-21T04:02:58.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.111+0000 7f238d7fa640 1 -- 172.21.15.64:0/3611363403 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380035030 con 0x7f2390102a90 2024-09-21T04:02:58.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.111+0000 7f23962c2640 1 -- 172.21.15.64:0/3611363403 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23901a6ec0 con 0x7f2390102a90 2024-09-21T04:02:58.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.112+0000 7f23962c2640 1 -- 172.21.15.64:0/3611363403 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2390103d10 con 0x7f2390102a90 2024-09-21T04:02:58.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.114+0000 7f238d7fa640 1 -- 172.21.15.64:0/3611363403 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f2380042070 con 0x7f2390102a90 2024-09-21T04:02:58.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.115+0000 7f238d7fa640 1 --2- 172.21.15.64:0/3611363403 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f23780b6620 0x7f23780b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:02:58.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.115+0000 7f238f7fe640 1 --2- 172.21.15.64:0/3611363403 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f23780b6620 0x7f23780b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:02:58.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.115+0000 7f238d7fa640 1 -- 172.21.15.64:0/3611363403 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f23801085f0 con 0x7f2390102a90 2024-09-21T04:02:58.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.116+0000 7f238f7fe640 1 --2- 172.21.15.64:0/3611363403 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f23780b6620 0x7f23780b8b30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2390100af0 tx=0x7f238400a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:02:58.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.118+0000 7f238d7fa640 1 -- 172.21.15.64:0/3611363403 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f23800cec20 con 0x7f2390102a90 2024-09-21T04:02:58.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:57 smithi119 ceph-mon[34811]: pgmap v287: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T04:02:58.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:57 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1307369784' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T04:02:58.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:57 smithi064 ceph-mon[31180]: pgmap v287: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T04:02:58.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:57 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1307369784' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T04:02:58.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:57 smithi131 ceph-mon[35193]: pgmap v287: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T04:02:58.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:57 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1307369784' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T04:02:58.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.314+0000 7f23962c2640 1 -- 172.21.15.64:0/3611363403 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f23901a7090 con 0x7f2390102a90 2024-09-21T04:02:58.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.314+0000 7f238d7fa640 1 -- 172.21.15.64:0/3611363403 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f23800d3000 con 0x7f2390102a90 2024-09-21T04:02:58.318 INFO:teuthology.orchestra.run.smithi064.stdout:244813135906 2024-09-21T04:02:58.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.317+0000 7f23962c2640 1 -- 172.21.15.64:0/3611363403 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f23780b6620 msgr2=0x7f23780b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:58.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.317+0000 7f23962c2640 1 --2- 172.21.15.64:0/3611363403 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f23780b6620 0x7f23780b8b30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2390100af0 tx=0x7f238400a2d0 comp rx=0 tx=0).stop 2024-09-21T04:02:58.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.317+0000 7f23962c2640 1 -- 172.21.15.64:0/3611363403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2390102a90 msgr2=0x7f239019a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:02:58.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.317+0000 7f23962c2640 1 --2- 172.21.15.64:0/3611363403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2390102a90 0x7f239019a500 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2380005ab0 tx=0x7f2380003d50 comp rx=0 tx=0).stop 2024-09-21T04:02:58.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.317+0000 7f23962c2640 1 -- 172.21.15.64:0/3611363403 shutdown_connections 2024-09-21T04:02:58.320 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.317+0000 7f23962c2640 1 --2- 172.21.15.64:0/3611363403 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f23780b6620 0x7f23780b8b30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:58.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.317+0000 7f23962c2640 1 --2- 172.21.15.64:0/3611363403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2390108d20 0x7f239019fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:58.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.317+0000 7f23962c2640 1 --2- 172.21.15.64:0/3611363403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23901083d0 0x7f239019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:58.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.317+0000 7f23962c2640 1 --2- 172.21.15.64:0/3611363403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2390102a90 0x7f239019a500 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:02:58.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.317+0000 7f23962c2640 1 -- 172.21.15.64:0/3611363403 >> 172.21.15.64:0/3611363403 conn(0x7f2390079750 msgr2=0x7f2390106710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:02:58.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.317+0000 7f23962c2640 1 -- 172.21.15.64:0/3611363403 shutdown_connections 2024-09-21T04:02:58.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:02:58.317+0000 7f23962c2640 1 -- 172.21.15.64:0/3611363403 wait complete. 2024-09-21T04:02:58.598 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542216 got 60129542219 for osd.1 2024-09-21T04:02:58.598 DEBUG:teuthology.parallel:result is None 2024-09-21T04:02:59.235 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:58 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3611363403' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T04:02:59.235 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:58 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:02:59.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:58 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3611363403' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T04:02:59.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:58 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:02:59.269 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135905 got 244813135906 for osd.9 2024-09-21T04:02:59.269 DEBUG:teuthology.parallel:result is None 2024-09-21T04:02:59.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:58 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3611363403' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T04:02:59.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:58 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:02:59.311 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:00.185 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:02:59 smithi064 ceph-mon[31180]: pgmap v288: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T04:03:00.215 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182462 got 107374182464 for osd.3 2024-09-21T04:03:00.215 DEBUG:teuthology.parallel:result is None 2024-09-21T04:03:00.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:02:59 smithi119 ceph-mon[34811]: pgmap v288: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T04:03:00.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:02:59 smithi131 ceph-mon[35193]: pgmap v288: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T04:03:00.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.515+0000 7feb14116640 1 -- 172.21.15.64:0/3957364387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb0c10de30 msgr2=0x7feb0c110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:00.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.515+0000 7feb14116640 1 --2- 172.21.15.64:0/3957364387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb0c10de30 0x7feb0c110300 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7feb04009ca0 tx=0x7feb0402a8c0 comp rx=0 tx=0).stop 2024-09-21T04:03:00.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.515+0000 7feb14116640 1 -- 172.21.15.64:0/3957364387 shutdown_connections 2024-09-21T04:03:00.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.515+0000 7feb14116640 1 --2- 172.21.15.64:0/3957364387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb0c10de30 0x7feb0c110300 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:00.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.515+0000 7feb14116640 1 --2- 172.21.15.64:0/3957364387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb0c1052b0 0x7feb0c10d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:00.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.515+0000 7feb14116640 1 --2- 172.21.15.64:0/3957364387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb0c104880 0x7feb0c104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:00.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.515+0000 7feb14116640 1 -- 172.21.15.64:0/3957364387 >> 172.21.15.64:0/3957364387 conn(0x7feb0c079730 msgr2=0x7feb0c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:00.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.515+0000 7feb14116640 1 -- 172.21.15.64:0/3957364387 shutdown_connections 2024-09-21T04:03:00.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.515+0000 7feb14116640 1 -- 172.21.15.64:0/3957364387 wait complete. 2024-09-21T04:03:00.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.516+0000 7feb14116640 1 Processor -- start 2024-09-21T04:03:00.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.516+0000 7feb14116640 1 -- start start 2024-09-21T04:03:00.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.517+0000 7feb14116640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb0c104880 0x7feb0c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:00.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.517+0000 7feb14116640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb0c1052b0 0x7feb0c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:00.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.517+0000 7feb14116640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb0c10de30 0x7feb0c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:00.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.517+0000 7feb11eb2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb0c104880 0x7feb0c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:00.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.517+0000 7feb14116640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c113ac0 con 0x7feb0c104880 2024-09-21T04:03:00.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.517+0000 7feb14116640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c113940 con 0x7feb0c1052b0 2024-09-21T04:03:00.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.517+0000 7feb14116640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c113c40 con 0x7feb0c10de30 2024-09-21T04:03:00.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.517+0000 7feb11eb2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb0c104880 0x7feb0c19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48024/0 (socket says 172.21.15.64:48024) 2024-09-21T04:03:00.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.517+0000 7feb11eb2640 1 -- 172.21.15.64:0/2769823067 learned_addr learned my addr 172.21.15.64:0/2769823067 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:00.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.517+0000 7feb116b1640 1 --2- 172.21.15.64:0/2769823067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb0c1052b0 0x7feb0c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:00.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.517+0000 7feb126b3640 1 --2- 172.21.15.64:0/2769823067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb0c10de30 0x7feb0c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:00.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.517+0000 7feb116b1640 1 -- 172.21.15.64:0/2769823067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb0c10de30 msgr2=0x7feb0c19fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:00.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.517+0000 7feb116b1640 1 --2- 172.21.15.64:0/2769823067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb0c10de30 0x7feb0c19fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:00.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.517+0000 7feb116b1640 1 -- 172.21.15.64:0/2769823067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb0c104880 msgr2=0x7feb0c19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:00.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.517+0000 7feb116b1640 1 --2- 172.21.15.64:0/2769823067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb0c104880 0x7feb0c19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:00.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.517+0000 7feb116b1640 1 -- 172.21.15.64:0/2769823067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0c1a0400 con 0x7feb0c1052b0 2024-09-21T04:03:00.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.517+0000 7feb11eb2640 1 --2- 172.21.15.64:0/2769823067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb0c104880 0x7feb0c19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:03:00.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.518+0000 7feb116b1640 1 --2- 172.21.15.64:0/2769823067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb0c1052b0 0x7feb0c19aa30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feafc00e9c0 tx=0x7feafc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:00.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.518+0000 7feb02ffd640 1 -- 172.21.15.64:0/2769823067 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc00ccc0 con 0x7feb0c1052b0 2024-09-21T04:03:00.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.518+0000 7feb02ffd640 1 -- 172.21.15.64:0/2769823067 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feafc005020 con 0x7feb0c1052b0 2024-09-21T04:03:00.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.518+0000 7feb02ffd640 1 -- 172.21.15.64:0/2769823067 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feafc004a00 con 0x7feb0c1052b0 2024-09-21T04:03:00.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.518+0000 7feb14116640 1 -- 172.21.15.64:0/2769823067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb0c1a06f0 con 0x7feb0c1052b0 2024-09-21T04:03:00.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.518+0000 7feb14116640 1 -- 172.21.15.64:0/2769823067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb0c1a0af0 con 0x7feb0c1052b0 2024-09-21T04:03:00.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.520+0000 7feb14116640 1 -- 172.21.15.64:0/2769823067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb0c104c90 con 0x7feb0c1052b0 2024-09-21T04:03:00.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.522+0000 7feb02ffd640 1 -- 172.21.15.64:0/2769823067 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7feafc0051d0 con 0x7feb0c1052b0 2024-09-21T04:03:00.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.523+0000 7feb02ffd640 1 --2- 172.21.15.64:0/2769823067 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7feaf80b65f0 0x7feaf80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:00.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.523+0000 7feb11eb2640 1 --2- 172.21.15.64:0/2769823067 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7feaf80b65f0 0x7feaf80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:00.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.523+0000 7feb02ffd640 1 -- 172.21.15.64:0/2769823067 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7feafc0e7500 con 0x7feb0c1052b0 2024-09-21T04:03:00.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.523+0000 7feb11eb2640 1 --2- 172.21.15.64:0/2769823067 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7feaf80b65f0 0x7feaf80b8b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7feb080058c0 tx=0x7feb08005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:00.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.528+0000 7feb02ffd640 1 -- 172.21.15.64:0/2769823067 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7feafc0adb30 con 0x7feb0c1052b0 2024-09-21T04:03:00.558 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:00.561 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:00.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.430+0000 7f94b136f640 1 -- 172.21.15.64:0/3939353199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94ac10de60 msgr2=0x7f94ac110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:00.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.430+0000 7f94b136f640 1 --2- 172.21.15.64:0/3939353199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94ac10de60 0x7f94ac110330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f94a4009ca0 tx=0x7f94a402a8c0 comp rx=0 tx=0).stop 2024-09-21T04:03:00.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.430+0000 7f94b136f640 1 -- 172.21.15.64:0/3939353199 shutdown_connections 2024-09-21T04:03:00.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.430+0000 7f94b136f640 1 --2- 172.21.15.64:0/3939353199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94ac10de60 0x7f94ac110330 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:00.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.430+0000 7f94b136f640 1 --2- 172.21.15.64:0/3939353199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94ac105380 0x7f94ac10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:00.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.430+0000 7f94b136f640 1 --2- 172.21.15.64:0/3939353199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94ac104a30 0x7f94ac104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:00.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.430+0000 7f94b136f640 1 -- 172.21.15.64:0/3939353199 >> 172.21.15.64:0/3939353199 conn(0x7f94ac0797b0 msgr2=0x7f94ac079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:00.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.430+0000 7f94b136f640 1 -- 172.21.15.64:0/3939353199 shutdown_connections 2024-09-21T04:03:00.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.430+0000 7f94b136f640 1 -- 172.21.15.64:0/3939353199 wait complete. 2024-09-21T04:03:00.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.431+0000 7f94b136f640 1 Processor -- start 2024-09-21T04:03:00.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.432+0000 7f94b136f640 1 -- start start 2024-09-21T04:03:00.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.432+0000 7f94b136f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94ac104a30 0x7f94ac19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:00.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.432+0000 7f94b136f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94ac105380 0x7f94ac19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:00.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.432+0000 7f94b136f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94ac10de60 0x7f94ac19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:00.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.432+0000 7f94b136f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac113aa0 con 0x7f94ac104a30 2024-09-21T04:03:00.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.432+0000 7f94aa7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94ac105380 0x7f94ac19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:00.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.432+0000 7f94b136f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac113920 con 0x7f94ac10de60 2024-09-21T04:03:00.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.432+0000 7f94b136f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac113c20 con 0x7f94ac105380 2024-09-21T04:03:00.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.432+0000 7f94aa7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94ac105380 0x7f94ac19a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:38060/0 (socket says 172.21.15.64:38060) 2024-09-21T04:03:00.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.432+0000 7f94aa7fc640 1 -- 172.21.15.64:0/2465329613 learned_addr learned my addr 172.21.15.64:0/2465329613 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:00.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.432+0000 7f94ab7fe640 1 --2- 172.21.15.64:0/2465329613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94ac10de60 0x7f94ac19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:00.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.432+0000 7f94aaffd640 1 --2- 172.21.15.64:0/2465329613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94ac104a30 0x7f94ac19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:00.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.433+0000 7f94aa7fc640 1 -- 172.21.15.64:0/2465329613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94ac10de60 msgr2=0x7f94ac19fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:00.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.433+0000 7f94aa7fc640 1 --2- 172.21.15.64:0/2465329613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94ac10de60 0x7f94ac19fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:00.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.433+0000 7f94aa7fc640 1 -- 172.21.15.64:0/2465329613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94ac104a30 msgr2=0x7f94ac19a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:00.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.433+0000 7f94aa7fc640 1 --2- 172.21.15.64:0/2465329613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94ac104a30 0x7f94ac19a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:00.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.433+0000 7f94aa7fc640 1 -- 172.21.15.64:0/2465329613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ac1a0380 con 0x7f94ac105380 2024-09-21T04:03:00.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.433+0000 7f94aaffd640 1 --2- 172.21.15.64:0/2465329613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94ac104a30 0x7f94ac19a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:03:00.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.433+0000 7f94aa7fc640 1 --2- 172.21.15.64:0/2465329613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94ac105380 0x7f94ac19a9b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f94a000e9f0 tx=0x7f94a000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:00.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.433+0000 7f9493fff640 1 -- 172.21.15.64:0/2465329613 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a000ccc0 con 0x7f94ac105380 2024-09-21T04:03:00.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.433+0000 7f94b136f640 1 -- 172.21.15.64:0/2465329613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94ac1a0670 con 0x7f94ac105380 2024-09-21T04:03:00.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.433+0000 7f9493fff640 1 -- 172.21.15.64:0/2465329613 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94a00108e0 con 0x7f94ac105380 2024-09-21T04:03:00.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.433+0000 7f9493fff640 1 -- 172.21.15.64:0/2465329613 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0018860 con 0x7f94ac105380 2024-09-21T04:03:00.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.433+0000 7f94b136f640 1 -- 172.21.15.64:0/2465329613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94ac1a6dd0 con 0x7f94ac105380 2024-09-21T04:03:00.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.434+0000 7f94b136f640 1 -- 172.21.15.64:0/2465329613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94ac104e40 con 0x7f94ac105380 2024-09-21T04:03:00.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.436+0000 7f9493fff640 1 -- 172.21.15.64:0/2465329613 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f94a0010430 con 0x7f94ac105380 2024-09-21T04:03:00.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.437+0000 7f9493fff640 1 --2- 172.21.15.64:0/2465329613 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f94940b6670 0x7f94940b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:00.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.437+0000 7f94aaffd640 1 --2- 172.21.15.64:0/2465329613 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f94940b6670 0x7f94940b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:00.700 INFO:teuthology.orchestra.run.smithi064.stdout:197568495660 2024-09-21T04:03:00.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.437+0000 7f9493fff640 1 -- 172.21.15.64:0/2465329613 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f94a00e7db0 con 0x7f94ac105380 2024-09-21T04:03:00.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.438+0000 7f94aaffd640 1 --2- 172.21.15.64:0/2465329613 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f94940b6670 0x7f94940b8b80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f949c009c20 tx=0x7f949c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:00.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.445+0000 7f9493fff640 1 -- 172.21.15.64:0/2465329613 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f94a00ae380 con 0x7f94ac105380 2024-09-21T04:03:00.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.650+0000 7f94b136f640 1 -- 172.21.15.64:0/2465329613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f94ac19b540 con 0x7f94ac105380 2024-09-21T04:03:00.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.651+0000 7f9493fff640 1 -- 172.21.15.64:0/2465329613 <== mon.1 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f94ac19b540 con 0x7f94ac105380 2024-09-21T04:03:00.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.653+0000 7f94b136f640 1 -- 172.21.15.64:0/2465329613 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f94940b6670 msgr2=0x7f94940b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:00.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.653+0000 7f94b136f640 1 --2- 172.21.15.64:0/2465329613 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f94940b6670 0x7f94940b8b80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f949c009c20 tx=0x7f949c005f90 comp rx=0 tx=0).stop 2024-09-21T04:03:00.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.653+0000 7f94b136f640 1 -- 172.21.15.64:0/2465329613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94ac105380 msgr2=0x7f94ac19a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:00.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.653+0000 7f94b136f640 1 --2- 172.21.15.64:0/2465329613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94ac105380 0x7f94ac19a9b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f94a000e9f0 tx=0x7f94a000eeb0 comp rx=0 tx=0).stop 2024-09-21T04:03:00.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.653+0000 7f94b136f640 1 -- 172.21.15.64:0/2465329613 shutdown_connections 2024-09-21T04:03:00.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.653+0000 7f94b136f640 1 --2- 172.21.15.64:0/2465329613 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f94940b6670 0x7f94940b8b80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:00.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.653+0000 7f94b136f640 1 --2- 172.21.15.64:0/2465329613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94ac10de60 0x7f94ac19fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:00.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.653+0000 7f94b136f640 1 --2- 172.21.15.64:0/2465329613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94ac105380 0x7f94ac19a9b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:00.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.653+0000 7f94b136f640 1 --2- 172.21.15.64:0/2465329613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94ac104a30 0x7f94ac19a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:00.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.653+0000 7f94b136f640 1 -- 172.21.15.64:0/2465329613 >> 172.21.15.64:0/2465329613 conn(0x7f94ac0797b0 msgr2=0x7f94ac100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:00.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.653+0000 7f94b136f640 1 -- 172.21.15.64:0/2465329613 shutdown_connections 2024-09-21T04:03:00.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.653+0000 7f94b136f640 1 -- 172.21.15.64:0/2465329613 wait complete. 2024-09-21T04:03:00.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.759+0000 7feb14116640 1 -- 172.21.15.64:0/2769823067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7feb0c02e6d0 con 0x7feb0c1052b0 2024-09-21T04:03:00.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.760+0000 7feb02ffd640 1 -- 172.21.15.64:0/2769823067 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7feafc0b1f10 con 0x7feb0c1052b0 2024-09-21T04:03:00.763 INFO:teuthology.orchestra.run.smithi064.stdout:266287972381 2024-09-21T04:03:00.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.762+0000 7feb14116640 1 -- 172.21.15.64:0/2769823067 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7feaf80b65f0 msgr2=0x7feaf80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:00.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.762+0000 7feb14116640 1 --2- 172.21.15.64:0/2769823067 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7feaf80b65f0 0x7feaf80b8b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7feb080058c0 tx=0x7feb08005f90 comp rx=0 tx=0).stop 2024-09-21T04:03:00.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.762+0000 7feb14116640 1 -- 172.21.15.64:0/2769823067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb0c1052b0 msgr2=0x7feb0c19aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:00.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.762+0000 7feb14116640 1 --2- 172.21.15.64:0/2769823067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb0c1052b0 0x7feb0c19aa30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feafc00e9c0 tx=0x7feafc00ee80 comp rx=0 tx=0).stop 2024-09-21T04:03:00.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.762+0000 7feb14116640 1 -- 172.21.15.64:0/2769823067 shutdown_connections 2024-09-21T04:03:00.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.762+0000 7feb14116640 1 --2- 172.21.15.64:0/2769823067 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7feaf80b65f0 0x7feaf80b8b00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:00.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.762+0000 7feb14116640 1 --2- 172.21.15.64:0/2769823067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb0c10de30 0x7feb0c19fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:00.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.762+0000 7feb14116640 1 --2- 172.21.15.64:0/2769823067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb0c1052b0 0x7feb0c19aa30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:00.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.762+0000 7feb14116640 1 --2- 172.21.15.64:0/2769823067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb0c104880 0x7feb0c19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:00.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.762+0000 7feb14116640 1 -- 172.21.15.64:0/2769823067 >> 172.21.15.64:0/2769823067 conn(0x7feb0c079730 msgr2=0x7feb0c105840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:00.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.762+0000 7feb14116640 1 -- 172.21.15.64:0/2769823067 shutdown_connections 2024-09-21T04:03:00.766 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:00.762+0000 7feb14116640 1 -- 172.21.15.64:0/2769823067 wait complete. 2024-09-21T04:03:00.930 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:00.930 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:01.155 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:01.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:01 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:03:01.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:01 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:03:01.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:01 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:03:01.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:01 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:03:01.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:01 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2465329613' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T04:03:01.463 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:01 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2769823067' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T04:03:01.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:01 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:03:01.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:01 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:03:01.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:01 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:03:01.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:01 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:03:01.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:01 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2465329613' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T04:03:01.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:01 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2769823067' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T04:03:01.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:01 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:03:01.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:01 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:03:01.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:01 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:03:01.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:01 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:03:01.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:01 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2465329613' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T04:03:01.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:01 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2769823067' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T04:03:02.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:02 smithi119 ceph-mon[34811]: pgmap v289: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-09-21T04:03:02.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:02 smithi064 ceph-mon[31180]: pgmap v289: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-09-21T04:03:02.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.512+0000 7f022f8a7640 1 -- 172.21.15.64:0/7613954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02281083a0 msgr2=0x7f02281087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:02.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.512+0000 7f022f8a7640 1 --2- 172.21.15.64:0/7613954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02281083a0 0x7f02281087b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0224009900 tx=0x7f022402a410 comp rx=0 tx=0).stop 2024-09-21T04:03:02.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.512+0000 7f022f8a7640 1 -- 172.21.15.64:0/7613954 shutdown_connections 2024-09-21T04:03:02.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.512+0000 7f022f8a7640 1 --2- 172.21.15.64:0/7613954 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0228108cf0 0x7f022810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:02.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.512+0000 7f022f8a7640 1 --2- 172.21.15.64:0/7613954 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02281083a0 0x7f02281087b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:02.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.512+0000 7f022f8a7640 1 --2- 172.21.15.64:0/7613954 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0228102a60 0x7f0228102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:02.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.512+0000 7f022f8a7640 1 -- 172.21.15.64:0/7613954 >> 172.21.15.64:0/7613954 conn(0x7f0228079730 msgr2=0x7f0228079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:02.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.513+0000 7f022f8a7640 1 -- 172.21.15.64:0/7613954 shutdown_connections 2024-09-21T04:03:02.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.513+0000 7f022f8a7640 1 -- 172.21.15.64:0/7613954 wait complete. 2024-09-21T04:03:02.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.513+0000 7f022f8a7640 1 Processor -- start 2024-09-21T04:03:02.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.513+0000 7f022f8a7640 1 -- start start 2024-09-21T04:03:02.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.513+0000 7f022f8a7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0228102a60 0x7f02280ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:02.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.513+0000 7f022f8a7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02281083a0 0x7f02280ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:02.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.513+0000 7f022f8a7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0228108cf0 0x7f02281000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:02.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.514+0000 7f022f8a7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0228112960 con 0x7f0228102a60 2024-09-21T04:03:02.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.514+0000 7f022f8a7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f02281127e0 con 0x7f0228108cf0 2024-09-21T04:03:02.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.514+0000 7f022f8a7640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0228112ae0 con 0x7f02281083a0 2024-09-21T04:03:02.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.514+0000 7f022d643640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0228102a60 0x7f02280ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:02.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.514+0000 7f022d643640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0228102a60 0x7f02280ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56862/0 (socket says 172.21.15.64:56862) 2024-09-21T04:03:02.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.514+0000 7f022d643640 1 -- 172.21.15.64:0/783617361 learned_addr learned my addr 172.21.15.64:0/783617361 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:02.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.514+0000 7f022de44640 1 --2- 172.21.15.64:0/783617361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0228108cf0 0x7f02281000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:02.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.514+0000 7f022ce42640 1 --2- 172.21.15.64:0/783617361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02281083a0 0x7f02280ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:02.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.514+0000 7f022d643640 1 -- 172.21.15.64:0/783617361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02281083a0 msgr2=0x7f02280ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:02.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.514+0000 7f022d643640 1 --2- 172.21.15.64:0/783617361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02281083a0 0x7f02280ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:02.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.514+0000 7f022d643640 1 -- 172.21.15.64:0/783617361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0228108cf0 msgr2=0x7f02281000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:02.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.514+0000 7f022d643640 1 --2- 172.21.15.64:0/783617361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0228108cf0 0x7f02281000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:02.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.514+0000 7f022d643640 1 -- 172.21.15.64:0/783617361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02281a8700 con 0x7f0228102a60 2024-09-21T04:03:02.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.514+0000 7f022ce42640 1 --2- 172.21.15.64:0/783617361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02281083a0 0x7f02280ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:03:02.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.514+0000 7f022d643640 1 --2- 172.21.15.64:0/783617361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0228102a60 0x7f02280ff670 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f021c00eab0 tx=0x7f021c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:02.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.514+0000 7f021a7fc640 1 -- 172.21.15.64:0/783617361 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021c00ce40 con 0x7f0228102a60 2024-09-21T04:03:02.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.514+0000 7f022f8a7640 1 -- 172.21.15.64:0/783617361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02281a89f0 con 0x7f0228102a60 2024-09-21T04:03:02.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.514+0000 7f021a7fc640 1 -- 172.21.15.64:0/783617361 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f021c011920 con 0x7f0228102a60 2024-09-21T04:03:02.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.514+0000 7f021a7fc640 1 -- 172.21.15.64:0/783617361 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021c010a30 con 0x7f0228102a60 2024-09-21T04:03:02.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.514+0000 7f022f8a7640 1 -- 172.21.15.64:0/783617361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02281a8f30 con 0x7f0228102a60 2024-09-21T04:03:02.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.515+0000 7f022f8a7640 1 -- 172.21.15.64:0/783617361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0228102e70 con 0x7f0228102a60 2024-09-21T04:03:02.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.518+0000 7f021a7fc640 1 -- 172.21.15.64:0/783617361 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f021c010bd0 con 0x7f0228102a60 2024-09-21T04:03:02.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.519+0000 7f021a7fc640 1 --2- 172.21.15.64:0/783617361 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f02140b65f0 0x7f02140b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:02.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.519+0000 7f022ce42640 1 --2- 172.21.15.64:0/783617361 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f02140b65f0 0x7f02140b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:02.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.519+0000 7f021a7fc640 1 -- 172.21.15.64:0/783617361 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f021c0e7cd0 con 0x7f0228102a60 2024-09-21T04:03:02.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.519+0000 7f022ce42640 1 --2- 172.21.15.64:0/783617361 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f02140b65f0 0x7f02140b8b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f02240096c0 tx=0x7f0224031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:02.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.522+0000 7f021a7fc640 1 -- 172.21.15.64:0/783617361 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f021c016030 con 0x7f0228102a60 2024-09-21T04:03:02.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:02 smithi131 ceph-mon[35193]: pgmap v289: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-09-21T04:03:02.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.715+0000 7f022f8a7640 1 -- 172.21.15.64:0/783617361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f0228100970 con 0x7f0228102a60 2024-09-21T04:03:02.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.715+0000 7f021a7fc640 1 -- 172.21.15.64:0/783617361 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0228100970 con 0x7f0228102a60 2024-09-21T04:03:02.719 INFO:teuthology.orchestra.run.smithi064.stdout:150323855415 2024-09-21T04:03:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.717+0000 7f022f8a7640 1 -- 172.21.15.64:0/783617361 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f02140b65f0 msgr2=0x7f02140b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.717+0000 7f022f8a7640 1 --2- 172.21.15.64:0/783617361 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f02140b65f0 0x7f02140b8b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f02240096c0 tx=0x7f0224031400 comp rx=0 tx=0).stop 2024-09-21T04:03:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.717+0000 7f022f8a7640 1 -- 172.21.15.64:0/783617361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0228102a60 msgr2=0x7f02280ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.717+0000 7f022f8a7640 1 --2- 172.21.15.64:0/783617361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0228102a60 0x7f02280ff670 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f021c00eab0 tx=0x7f021c00ef70 comp rx=0 tx=0).stop 2024-09-21T04:03:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.717+0000 7f022f8a7640 1 -- 172.21.15.64:0/783617361 shutdown_connections 2024-09-21T04:03:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.718+0000 7f022f8a7640 1 --2- 172.21.15.64:0/783617361 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f02140b65f0 0x7f02140b8b00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.718+0000 7f022f8a7640 1 --2- 172.21.15.64:0/783617361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0228108cf0 0x7f02281000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.718+0000 7f022f8a7640 1 --2- 172.21.15.64:0/783617361 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02281083a0 0x7f02280ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.718+0000 7f022f8a7640 1 --2- 172.21.15.64:0/783617361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0228102a60 0x7f02280ff670 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.718+0000 7f022f8a7640 1 -- 172.21.15.64:0/783617361 >> 172.21.15.64:0/783617361 conn(0x7f0228079730 msgr2=0x7f022806b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:02.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.718+0000 7f022f8a7640 1 -- 172.21.15.64:0/783617361 shutdown_connections 2024-09-21T04:03:02.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.718+0000 7f022f8a7640 1 -- 172.21.15.64:0/783617361 wait complete. 2024-09-21T04:03:03.388 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:03 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/783617361' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T04:03:03.388 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:03 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/4432289' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T04:03:03.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.477+0000 7ff5862a8640 1 -- 172.21.15.64:0/406383680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5801084c0 msgr2=0x7ff58010ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:03.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.477+0000 7ff5862a8640 1 --2- 172.21.15.64:0/406383680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5801084c0 0x7ff58010ecf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff578009ca0 tx=0x7ff57802a8c0 comp rx=0 tx=0).stop 2024-09-21T04:03:03.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.477+0000 7ff5862a8640 1 -- 172.21.15.64:0/406383680 shutdown_connections 2024-09-21T04:03:03.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.477+0000 7ff5862a8640 1 --2- 172.21.15.64:0/406383680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5801084c0 0x7ff58010ecf0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:03.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.477+0000 7ff5862a8640 1 --2- 172.21.15.64:0/406383680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff580107b70 0x7ff580107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:03.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.477+0000 7ff5862a8640 1 --2- 172.21.15.64:0/406383680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff580102230 0x7ff580102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:03.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.478+0000 7ff5862a8640 1 -- 172.21.15.64:0/406383680 >> 172.21.15.64:0/406383680 conn(0x7ff580079e40 msgr2=0x7ff58007a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:03.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.478+0000 7ff5862a8640 1 -- 172.21.15.64:0/406383680 shutdown_connections 2024-09-21T04:03:03.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.478+0000 7ff5862a8640 1 -- 172.21.15.64:0/406383680 wait complete. 2024-09-21T04:03:03.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.478+0000 7ff5862a8640 1 Processor -- start 2024-09-21T04:03:03.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.478+0000 7ff5862a8640 1 -- start start 2024-09-21T04:03:03.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.479+0000 7ff5862a8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff580102230 0x7ff5800fee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:03.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.479+0000 7ff5862a8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff580107b70 0x7ff5800ff3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:03.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.479+0000 7ff5862a8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5801084c0 0x7ff5800ff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:03.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.479+0000 7ff5862a8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff580111900 con 0x7ff5801084c0 2024-09-21T04:03:03.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.479+0000 7ff5862a8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff580111780 con 0x7ff580102230 2024-09-21T04:03:03.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.479+0000 7ff5862a8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff580111a80 con 0x7ff580107b70 2024-09-21T04:03:03.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.479+0000 7ff584845640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5801084c0 0x7ff5800ff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:03.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.479+0000 7ff584845640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5801084c0 0x7ff5800ff8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56894/0 (socket says 172.21.15.64:56894) 2024-09-21T04:03:03.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.479+0000 7ff584845640 1 -- 172.21.15.64:0/1991944303 learned_addr learned my addr 172.21.15.64:0/1991944303 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:03.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.479+0000 7ff57f7fe640 1 --2- 172.21.15.64:0/1991944303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff580107b70 0x7ff5800ff3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:03.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.479+0000 7ff57ffff640 1 --2- 172.21.15.64:0/1991944303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff580102230 0x7ff5800fee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:03.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.479+0000 7ff584845640 1 -- 172.21.15.64:0/1991944303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff580107b70 msgr2=0x7ff5800ff3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:03.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.479+0000 7ff584845640 1 --2- 172.21.15.64:0/1991944303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff580107b70 0x7ff5800ff3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:03.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.479+0000 7ff584845640 1 -- 172.21.15.64:0/1991944303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff580102230 msgr2=0x7ff5800fee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:03.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.479+0000 7ff584845640 1 --2- 172.21.15.64:0/1991944303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff580102230 0x7ff5800fee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:03.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.479+0000 7ff584845640 1 -- 172.21.15.64:0/1991944303 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5801a8790 con 0x7ff5801084c0 2024-09-21T04:03:03.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.479+0000 7ff57ffff640 1 --2- 172.21.15.64:0/1991944303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff580102230 0x7ff5800fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:03:03.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.479+0000 7ff584845640 1 --2- 172.21.15.64:0/1991944303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5801084c0 0x7ff5800ff8f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff578005cc0 tx=0x7ff57802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:03.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.480+0000 7ff57d7fa640 1 -- 172.21.15.64:0/1991944303 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff578003bf0 con 0x7ff5801084c0 2024-09-21T04:03:03.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.480+0000 7ff57d7fa640 1 -- 172.21.15.64:0/1991944303 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff578003d90 con 0x7ff5801084c0 2024-09-21T04:03:03.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.480+0000 7ff5862a8640 1 -- 172.21.15.64:0/1991944303 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5801a8a20 con 0x7ff5801084c0 2024-09-21T04:03:03.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.480+0000 7ff57d7fa640 1 -- 172.21.15.64:0/1991944303 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57802fbc0 con 0x7ff5801084c0 2024-09-21T04:03:03.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.480+0000 7ff5862a8640 1 -- 172.21.15.64:0/1991944303 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5801a8f60 con 0x7ff5801084c0 2024-09-21T04:03:03.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.481+0000 7ff5862a8640 1 -- 172.21.15.64:0/1991944303 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff580102640 con 0x7ff5801084c0 2024-09-21T04:03:03.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.482+0000 7ff57d7fa640 1 -- 172.21.15.64:0/1991944303 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7ff57802fd60 con 0x7ff5801084c0 2024-09-21T04:03:03.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.483+0000 7ff57d7fa640 1 --2- 172.21.15.64:0/1991944303 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff5680b65f0 0x7ff5680b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:03.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.483+0000 7ff57d7fa640 1 -- 172.21.15.64:0/1991944303 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7ff578108b80 con 0x7ff5801084c0 2024-09-21T04:03:03.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.483+0000 7ff57ffff640 1 --2- 172.21.15.64:0/1991944303 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff5680b65f0 0x7ff5680b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:03.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.484+0000 7ff57ffff640 1 --2- 172.21.15.64:0/1991944303 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff5680b65f0 0x7ff5680b8b00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff570005fd0 tx=0x7ff570005ea0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:03.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.488+0000 7ff57d7fa640 1 -- 172.21.15.64:0/1991944303 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff5780cf1b0 con 0x7ff5801084c0 2024-09-21T04:03:03.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:03 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/783617361' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T04:03:03.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:03 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/4432289' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T04:03:03.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:03 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/783617361' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T04:03:03.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:03 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/4432289' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T04:03:03.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.676+0000 7ff5862a8640 1 -- 172.21.15.64:0/1991944303 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7ff58002e640 con 0x7ff5801084c0 2024-09-21T04:03:03.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.677+0000 7ff57d7fa640 1 -- 172.21.15.64:0/1991944303 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ff5780ca070 con 0x7ff5801084c0 2024-09-21T04:03:03.680 INFO:teuthology.orchestra.run.smithi064.stdout:287762808857 2024-09-21T04:03:03.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.679+0000 7ff5862a8640 1 -- 172.21.15.64:0/1991944303 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff5680b65f0 msgr2=0x7ff5680b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:03.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.679+0000 7ff5862a8640 1 --2- 172.21.15.64:0/1991944303 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff5680b65f0 0x7ff5680b8b00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff570005fd0 tx=0x7ff570005ea0 comp rx=0 tx=0).stop 2024-09-21T04:03:03.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.679+0000 7ff5862a8640 1 -- 172.21.15.64:0/1991944303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5801084c0 msgr2=0x7ff5800ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:03.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.679+0000 7ff5862a8640 1 --2- 172.21.15.64:0/1991944303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5801084c0 0x7ff5800ff8f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff578005cc0 tx=0x7ff57802f380 comp rx=0 tx=0).stop 2024-09-21T04:03:03.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.679+0000 7ff5862a8640 1 -- 172.21.15.64:0/1991944303 shutdown_connections 2024-09-21T04:03:03.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.679+0000 7ff5862a8640 1 --2- 172.21.15.64:0/1991944303 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff5680b65f0 0x7ff5680b8b00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:03.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.679+0000 7ff5862a8640 1 --2- 172.21.15.64:0/1991944303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5801084c0 0x7ff5800ff8f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:03.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.679+0000 7ff5862a8640 1 --2- 172.21.15.64:0/1991944303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff580107b70 0x7ff5800ff3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:03.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.679+0000 7ff5862a8640 1 --2- 172.21.15.64:0/1991944303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff580102230 0x7ff5800fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:03.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.679+0000 7ff5862a8640 1 -- 172.21.15.64:0/1991944303 >> 172.21.15.64:0/1991944303 conn(0x7ff580079e40 msgr2=0x7ff580105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:03.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.679+0000 7ff5862a8640 1 -- 172.21.15.64:0/1991944303 shutdown_connections 2024-09-21T04:03:03.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.679+0000 7ff5862a8640 1 -- 172.21.15.64:0/1991944303 wait complete. 2024-09-21T04:03:03.912 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972379 got 266287972381 for osd.10 2024-09-21T04:03:03.912 DEBUG:teuthology.parallel:result is None 2024-09-21T04:03:04.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:04 smithi119 ceph-mon[34811]: pgmap v290: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-09-21T04:03:04.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:04 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1991944303' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T04:03:04.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:04 smithi064 ceph-mon[31180]: pgmap v290: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-09-21T04:03:04.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:04 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1991944303' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T04:03:04.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:04 smithi131 ceph-mon[35193]: pgmap v290: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-09-21T04:03:04.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:04 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1991944303' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T04:03:04.604 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808854 got 287762808857 for osd.11 2024-09-21T04:03:04.605 DEBUG:teuthology.parallel:result is None 2024-09-21T04:03:05.023 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855413 got 150323855415 for osd.5 2024-09-21T04:03:05.023 DEBUG:teuthology.parallel:result is None 2024-09-21T04:03:05.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.245+0000 7f296f27f640 1 -- 172.21.15.64:0/566270115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29681094a0 msgr2=0x7f29681098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:05.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.245+0000 7f296f27f640 1 --2- 172.21.15.64:0/566270115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29681094a0 0x7f29681098b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f295c009900 tx=0x7f295c02a410 comp rx=0 tx=0).stop 2024-09-21T04:03:05.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.245+0000 7f296f27f640 1 -- 172.21.15.64:0/566270115 shutdown_connections 2024-09-21T04:03:05.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.245+0000 7f296f27f640 1 --2- 172.21.15.64:0/566270115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2968109df0 0x7f2968110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:05.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.245+0000 7f296f27f640 1 --2- 172.21.15.64:0/566270115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29681094a0 0x7f29681098b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:05.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.245+0000 7f296f27f640 1 --2- 172.21.15.64:0/566270115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2968103b60 0x7f2968103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:05.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.245+0000 7f296f27f640 1 -- 172.21.15.64:0/566270115 >> 172.21.15.64:0/566270115 conn(0x7f296807b850 msgr2=0x7f296807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:05.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.245+0000 7f296f27f640 1 -- 172.21.15.64:0/566270115 shutdown_connections 2024-09-21T04:03:05.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.245+0000 7f296f27f640 1 -- 172.21.15.64:0/566270115 wait complete. 2024-09-21T04:03:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.246+0000 7f296f27f640 1 Processor -- start 2024-09-21T04:03:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.246+0000 7f296f27f640 1 -- start start 2024-09-21T04:03:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.246+0000 7f296f27f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2968103b60 0x7f296819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.246+0000 7f296f27f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29681094a0 0x7f296819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.246+0000 7f296f27f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2968109df0 0x7f296819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.246+0000 7f296f27f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2968113ab0 con 0x7f29681094a0 2024-09-21T04:03:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.246+0000 7f296f27f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2968113930 con 0x7f2968109df0 2024-09-21T04:03:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.246+0000 7f296f27f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2968113c30 con 0x7f2968103b60 2024-09-21T04:03:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.246+0000 7f296c81a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29681094a0 0x7f296819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.246+0000 7f296c81a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29681094a0 0x7f296819ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56922/0 (socket says 172.21.15.64:56922) 2024-09-21T04:03:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.247+0000 7f296c81a640 1 -- 172.21.15.64:0/1918874617 learned_addr learned my addr 172.21.15.64:0/1918874617 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.247+0000 7f296d81c640 1 --2- 172.21.15.64:0/1918874617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2968109df0 0x7f296819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.247+0000 7f296d01b640 1 --2- 172.21.15.64:0/1918874617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2968103b60 0x7f296819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:05.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.247+0000 7f296c81a640 1 -- 172.21.15.64:0/1918874617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2968103b60 msgr2=0x7f296819a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.247+0000 7f296c81a640 1 --2- 172.21.15.64:0/1918874617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2968103b60 0x7f296819a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.247+0000 7f296c81a640 1 -- 172.21.15.64:0/1918874617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2968109df0 msgr2=0x7f296819fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.247+0000 7f296c81a640 1 --2- 172.21.15.64:0/1918874617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2968109df0 0x7f296819fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.247+0000 7f296c81a640 1 -- 172.21.15.64:0/1918874617 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29681a0450 con 0x7f29681094a0 2024-09-21T04:03:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.247+0000 7f296c81a640 1 --2- 172.21.15.64:0/1918874617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29681094a0 0x7f296819ab40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f295c02a930 tx=0x7f295c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.247+0000 7f295a7fc640 1 -- 172.21.15.64:0/1918874617 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295c03f030 con 0x7f29681094a0 2024-09-21T04:03:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.247+0000 7f295a7fc640 1 -- 172.21.15.64:0/1918874617 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f295c003aa0 con 0x7f29681094a0 2024-09-21T04:03:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.247+0000 7f295a7fc640 1 -- 172.21.15.64:0/1918874617 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295c035070 con 0x7f29681094a0 2024-09-21T04:03:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.247+0000 7f296f27f640 1 -- 172.21.15.64:0/1918874617 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29681a06e0 con 0x7f29681094a0 2024-09-21T04:03:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.247+0000 7f296f27f640 1 -- 172.21.15.64:0/1918874617 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29681a0c10 con 0x7f29681094a0 2024-09-21T04:03:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.248+0000 7f296f27f640 1 -- 172.21.15.64:0/1918874617 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2968103f70 con 0x7f29681094a0 2024-09-21T04:03:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.250+0000 7f295a7fc640 1 -- 172.21.15.64:0/1918874617 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f295c042030 con 0x7f29681094a0 2024-09-21T04:03:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.251+0000 7f295a7fc640 1 --2- 172.21.15.64:0/1918874617 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f29540b66a0 0x7f29540b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.251+0000 7f296d01b640 1 --2- 172.21.15.64:0/1918874617 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f29540b66a0 0x7f29540b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:05.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.251+0000 7f295a7fc640 1 -- 172.21.15.64:0/1918874617 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f295c1080e0 con 0x7f29681094a0 2024-09-21T04:03:05.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.251+0000 7f296d01b640 1 --2- 172.21.15.64:0/1918874617 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f29540b66a0 0x7f29540b8bb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2964005fd0 tx=0x7f2964009870 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:05.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.254+0000 7f295a7fc640 1 -- 172.21.15.64:0/1918874617 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f29681a0c10 con 0x7f29681094a0 2024-09-21T04:03:05.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.446+0000 7f296f27f640 1 -- 172.21.15.64:0/1918874617 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f296819b6d0 con 0x7f29681094a0 2024-09-21T04:03:05.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.446+0000 7f295a7fc640 1 -- 172.21.15.64:0/1918874617 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f296819b6d0 con 0x7f29681094a0 2024-09-21T04:03:05.449 INFO:teuthology.orchestra.run.smithi064.stdout:81604378695 2024-09-21T04:03:05.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.449+0000 7f296f27f640 1 -- 172.21.15.64:0/1918874617 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f29540b66a0 msgr2=0x7f29540b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:05.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.449+0000 7f296f27f640 1 --2- 172.21.15.64:0/1918874617 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f29540b66a0 0x7f29540b8bb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2964005fd0 tx=0x7f2964009870 comp rx=0 tx=0).stop 2024-09-21T04:03:05.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.449+0000 7f296f27f640 1 -- 172.21.15.64:0/1918874617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29681094a0 msgr2=0x7f296819ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:05.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.449+0000 7f296f27f640 1 --2- 172.21.15.64:0/1918874617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29681094a0 0x7f296819ab40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f295c02a930 tx=0x7f295c004270 comp rx=0 tx=0).stop 2024-09-21T04:03:05.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.449+0000 7f296f27f640 1 -- 172.21.15.64:0/1918874617 shutdown_connections 2024-09-21T04:03:05.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.449+0000 7f296f27f640 1 --2- 172.21.15.64:0/1918874617 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f29540b66a0 0x7f29540b8bb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:05.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.449+0000 7f296f27f640 1 --2- 172.21.15.64:0/1918874617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2968109df0 0x7f296819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:05.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.449+0000 7f296f27f640 1 --2- 172.21.15.64:0/1918874617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29681094a0 0x7f296819ab40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:05.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.449+0000 7f296f27f640 1 --2- 172.21.15.64:0/1918874617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2968103b60 0x7f296819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:05.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.449+0000 7f296f27f640 1 -- 172.21.15.64:0/1918874617 >> 172.21.15.64:0/1918874617 conn(0x7f296807b850 msgr2=0x7f2968105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:05.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.449+0000 7f296f27f640 1 -- 172.21.15.64:0/1918874617 shutdown_connections 2024-09-21T04:03:05.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:05.449+0000 7f296f27f640 1 -- 172.21.15.64:0/1918874617 wait complete. 2024-09-21T04:03:05.759 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495657 got 197568495660 for osd.7 2024-09-21T04:03:05.760 DEBUG:teuthology.parallel:result is None 2024-09-21T04:03:06.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.822+0000 7fd195cba640 1 -- 172.21.15.64:0/493634167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd190108530 msgr2=0x7fd19010ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:06.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.822+0000 7fd195cba640 1 --2- 172.21.15.64:0/493634167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd190108530 0x7fd19010ed60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd188009900 tx=0x7fd18802a410 comp rx=0 tx=0).stop 2024-09-21T04:03:06.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.822+0000 7fd195cba640 1 -- 172.21.15.64:0/493634167 shutdown_connections 2024-09-21T04:03:06.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.822+0000 7fd195cba640 1 --2- 172.21.15.64:0/493634167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd190108530 0x7fd19010ed60 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:06.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.822+0000 7fd195cba640 1 --2- 172.21.15.64:0/493634167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd190107be0 0x7fd190107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:06.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.822+0000 7fd195cba640 1 --2- 172.21.15.64:0/493634167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1901022a0 0x7fd1901026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:06.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.822+0000 7fd195cba640 1 -- 172.21.15.64:0/493634167 >> 172.21.15.64:0/493634167 conn(0x7fd190079ec0 msgr2=0x7fd19007a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:06.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.822+0000 7fd195cba640 1 -- 172.21.15.64:0/493634167 shutdown_connections 2024-09-21T04:03:06.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.822+0000 7fd195cba640 1 -- 172.21.15.64:0/493634167 wait complete. 2024-09-21T04:03:06.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.823+0000 7fd195cba640 1 Processor -- start 2024-09-21T04:03:06.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.823+0000 7fd195cba640 1 -- start start 2024-09-21T04:03:06.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.823+0000 7fd195cba640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1901022a0 0x7fd19019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:06.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.823+0000 7fd195cba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd190107be0 0x7fd19019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:06.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.823+0000 7fd195cba640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd190108530 0x7fd19019fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:06.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.823+0000 7fd195cba640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd190112b00 con 0x7fd190107be0 2024-09-21T04:03:06.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.823+0000 7fd195cba640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd190112980 con 0x7fd1901022a0 2024-09-21T04:03:06.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.823+0000 7fd195cba640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd190112c80 con 0x7fd190108530 2024-09-21T04:03:06.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.823+0000 7fd18effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd190107be0 0x7fd19019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:06.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.823+0000 7fd18effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd190107be0 0x7fd19019aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56878/0 (socket says 172.21.15.64:56878) 2024-09-21T04:03:06.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.823+0000 7fd18effd640 1 -- 172.21.15.64:0/4432289 learned_addr learned my addr 172.21.15.64:0/4432289 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:06.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.823+0000 7fd18f7fe640 1 --2- 172.21.15.64:0/4432289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1901022a0 0x7fd19019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:06.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.824+0000 7fd18ffff640 1 --2- 172.21.15.64:0/4432289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd190108530 0x7fd19019fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:06.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.824+0000 7fd18effd640 1 -- 172.21.15.64:0/4432289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd190108530 msgr2=0x7fd19019fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:06.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.824+0000 7fd18effd640 1 --2- 172.21.15.64:0/4432289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd190108530 0x7fd19019fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:06.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.824+0000 7fd18effd640 1 -- 172.21.15.64:0/4432289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1901022a0 msgr2=0x7fd19019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:06.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.824+0000 7fd18effd640 1 --2- 172.21.15.64:0/4432289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1901022a0 0x7fd19019a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:06.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.824+0000 7fd18effd640 1 -- 172.21.15.64:0/4432289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1901a04b0 con 0x7fd190107be0 2024-09-21T04:03:06.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.824+0000 7fd18effd640 1 --2- 172.21.15.64:0/4432289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd190107be0 0x7fd19019aae0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd18400d750 tx=0x7fd18400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:06.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.824+0000 7fd18cff9640 1 -- 172.21.15.64:0/4432289 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184017070 con 0x7fd190107be0 2024-09-21T04:03:06.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.824+0000 7fd18cff9640 1 -- 172.21.15.64:0/4432289 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1840054d0 con 0x7fd190107be0 2024-09-21T04:03:06.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.824+0000 7fd195cba640 1 -- 172.21.15.64:0/4432289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1901a07a0 con 0x7fd190107be0 2024-09-21T04:03:06.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.824+0000 7fd18cff9640 1 -- 172.21.15.64:0/4432289 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184004c30 con 0x7fd190107be0 2024-09-21T04:03:06.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.824+0000 7fd195cba640 1 -- 172.21.15.64:0/4432289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1901a6f00 con 0x7fd190107be0 2024-09-21T04:03:06.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.825+0000 7fd195cba640 1 -- 172.21.15.64:0/4432289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd190103520 con 0x7fd190107be0 2024-09-21T04:03:06.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.827+0000 7fd18cff9640 1 -- 172.21.15.64:0/4432289 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fd184004dd0 con 0x7fd190107be0 2024-09-21T04:03:06.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.828+0000 7fd18cff9640 1 --2- 172.21.15.64:0/4432289 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd1780b65f0 0x7fd1780b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:06.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.828+0000 7fd18f7fe640 1 --2- 172.21.15.64:0/4432289 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd1780b65f0 0x7fd1780b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:06.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.828+0000 7fd18cff9640 1 -- 172.21.15.64:0/4432289 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7fd1840e73c0 con 0x7fd190107be0 2024-09-21T04:03:06.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.829+0000 7fd18f7fe640 1 --2- 172.21.15.64:0/4432289 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd1780b65f0 0x7fd1780b8b00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd18000a400 tx=0x7fd18000d040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:06.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:02.831+0000 7fd18cff9640 1 -- 172.21.15.64:0/4432289 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd1840ad9f0 con 0x7fd190107be0 2024-09-21T04:03:06.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.030+0000 7fd195cba640 1 -- 172.21.15.64:0/4432289 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fd19010b330 con 0x7fd190107be0 2024-09-21T04:03:06.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.031+0000 7fd18cff9640 1 -- 172.21.15.64:0/4432289 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd1840b1dd0 con 0x7fd190107be0 2024-09-21T04:03:06.062 INFO:teuthology.orchestra.run.smithi064.stdout:38654705744 2024-09-21T04:03:06.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.033+0000 7fd195cba640 1 -- 172.21.15.64:0/4432289 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd1780b65f0 msgr2=0x7fd1780b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:06.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.033+0000 7fd195cba640 1 --2- 172.21.15.64:0/4432289 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd1780b65f0 0x7fd1780b8b00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd18000a400 tx=0x7fd18000d040 comp rx=0 tx=0).stop 2024-09-21T04:03:06.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.033+0000 7fd195cba640 1 -- 172.21.15.64:0/4432289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd190107be0 msgr2=0x7fd19019aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:06.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.033+0000 7fd195cba640 1 --2- 172.21.15.64:0/4432289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd190107be0 0x7fd19019aae0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd18400d750 tx=0x7fd18400dc10 comp rx=0 tx=0).stop 2024-09-21T04:03:06.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.033+0000 7fd195cba640 1 -- 172.21.15.64:0/4432289 shutdown_connections 2024-09-21T04:03:06.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.033+0000 7fd195cba640 1 --2- 172.21.15.64:0/4432289 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd1780b65f0 0x7fd1780b8b00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:06.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.033+0000 7fd195cba640 1 --2- 172.21.15.64:0/4432289 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd190108530 0x7fd19019fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:06.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.033+0000 7fd195cba640 1 --2- 172.21.15.64:0/4432289 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd190107be0 0x7fd19019aae0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:06.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.033+0000 7fd195cba640 1 --2- 172.21.15.64:0/4432289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1901022a0 0x7fd19019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:06.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.033+0000 7fd195cba640 1 -- 172.21.15.64:0/4432289 >> 172.21.15.64:0/4432289 conn(0x7fd190079ec0 msgr2=0x7fd19010f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:06.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.033+0000 7fd195cba640 1 -- 172.21.15.64:0/4432289 shutdown_connections 2024-09-21T04:03:06.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:03.033+0000 7fd195cba640 1 -- 172.21.15.64:0/4432289 wait complete. 2024-09-21T04:03:06.346 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:06 smithi064 ceph-mon[31180]: pgmap v291: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:03:06.346 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:06 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1918874617' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T04:03:06.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:06 smithi119 ceph-mon[34811]: pgmap v291: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:03:06.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:06 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1918874617' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T04:03:06.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:06 smithi131 ceph-mon[35193]: pgmap v291: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:03:06.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:06 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1918874617' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T04:03:06.687 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705743 got 38654705744 for osd.0 2024-09-21T04:03:06.688 DEBUG:teuthology.parallel:result is None 2024-09-21T04:03:07.334 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378691 got 81604378695 for osd.2 2024-09-21T04:03:07.334 DEBUG:teuthology.parallel:result is None 2024-09-21T04:03:07.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.542+0000 7f2f9e032640 1 -- 172.21.15.64:0/4049171708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f98102ae0 msgr2=0x7f2f98102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:07.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.542+0000 7f2f9e032640 1 --2- 172.21.15.64:0/4049171708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f98102ae0 0x7f2f98102ef0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009900 tx=0x7f2f8802a410 comp rx=0 tx=0).stop 2024-09-21T04:03:07.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.542+0000 7f2f9e032640 1 -- 172.21.15.64:0/4049171708 shutdown_connections 2024-09-21T04:03:07.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.542+0000 7f2f9e032640 1 --2- 172.21.15.64:0/4049171708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f98108d20 0x7f2f9810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:07.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.542+0000 7f2f9e032640 1 --2- 172.21.15.64:0/4049171708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f981083d0 0x7f2f981087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:07.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.542+0000 7f2f9e032640 1 --2- 172.21.15.64:0/4049171708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f98102ae0 0x7f2f98102ef0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:07.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.542+0000 7f2f9e032640 1 -- 172.21.15.64:0/4049171708 >> 172.21.15.64:0/4049171708 conn(0x7f2f98079760 msgr2=0x7f2f98079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:07.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.542+0000 7f2f9e032640 1 -- 172.21.15.64:0/4049171708 shutdown_connections 2024-09-21T04:03:07.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.542+0000 7f2f9e032640 1 -- 172.21.15.64:0/4049171708 wait complete. 2024-09-21T04:03:07.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.543+0000 7f2f9e032640 1 Processor -- start 2024-09-21T04:03:07.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.543+0000 7f2f9e032640 1 -- start start 2024-09-21T04:03:07.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.543+0000 7f2f9e032640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f98102ae0 0x7f2f98104280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:07.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.543+0000 7f2f9e032640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f981083d0 0x7f2f981047c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.543+0000 7f2f9e032640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f98108d20 0x7f2f98104d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.543+0000 7f2f9e032640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2f98112990 con 0x7f2f98108d20 2024-09-21T04:03:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.543+0000 7f2f9e032640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2f98112810 con 0x7f2f981083d0 2024-09-21T04:03:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.543+0000 7f2f9e032640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2f98112b10 con 0x7f2f98102ae0 2024-09-21T04:03:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.544+0000 7f2f97fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f98108d20 0x7f2f98104d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.544+0000 7f2f97fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f98108d20 0x7f2f98104d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56936/0 (socket says 172.21.15.64:56936) 2024-09-21T04:03:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.544+0000 7f2f97fff640 1 -- 172.21.15.64:0/2453522555 learned_addr learned my addr 172.21.15.64:0/2453522555 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.544+0000 7f2f977fe640 1 --2- 172.21.15.64:0/2453522555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f98102ae0 0x7f2f98104280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.544+0000 7f2f8ffff640 1 --2- 172.21.15.64:0/2453522555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f981083d0 0x7f2f981047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.544+0000 7f2f97fff640 1 -- 172.21.15.64:0/2453522555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f98102ae0 msgr2=0x7f2f98104280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.544+0000 7f2f97fff640 1 --2- 172.21.15.64:0/2453522555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f98102ae0 0x7f2f98104280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.544+0000 7f2f97fff640 1 -- 172.21.15.64:0/2453522555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f981083d0 msgr2=0x7f2f981047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.544+0000 7f2f97fff640 1 --2- 172.21.15.64:0/2453522555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f981083d0 0x7f2f981047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.544+0000 7f2f97fff640 1 -- 172.21.15.64:0/2453522555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f981a8750 con 0x7f2f98108d20 2024-09-21T04:03:07.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.544+0000 7f2f8ffff640 1 --2- 172.21.15.64:0/2453522555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f981083d0 0x7f2f981047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:03:07.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.544+0000 7f2f97fff640 1 --2- 172.21.15.64:0/2453522555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f98108d20 0x7f2f98104d00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000cad0 tx=0x7f2f9000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:07.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.544+0000 7f2f957fa640 1 -- 172.21.15.64:0/2453522555 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9000ae20 con 0x7f2f98108d20 2024-09-21T04:03:07.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.544+0000 7f2f957fa640 1 -- 172.21.15.64:0/2453522555 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f90005020 con 0x7f2f98108d20 2024-09-21T04:03:07.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.544+0000 7f2f9e032640 1 -- 172.21.15.64:0/2453522555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f981a8a40 con 0x7f2f98108d20 2024-09-21T04:03:07.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.544+0000 7f2f957fa640 1 -- 172.21.15.64:0/2453522555 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9000f6b0 con 0x7f2f98108d20 2024-09-21T04:03:07.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.545+0000 7f2f9e032640 1 -- 172.21.15.64:0/2453522555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f981a8f80 con 0x7f2f98108d20 2024-09-21T04:03:07.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.545+0000 7f2f9e032640 1 -- 172.21.15.64:0/2453522555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f98103d10 con 0x7f2f98108d20 2024-09-21T04:03:07.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.547+0000 7f2f957fa640 1 -- 172.21.15.64:0/2453522555 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f2f900051d0 con 0x7f2f98108d20 2024-09-21T04:03:07.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.548+0000 7f2f957fa640 1 --2- 172.21.15.64:0/2453522555 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2f840b6670 0x7f2f840b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:07.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.548+0000 7f2f957fa640 1 -- 172.21.15.64:0/2453522555 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f2f900e79b0 con 0x7f2f98108d20 2024-09-21T04:03:07.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.548+0000 7f2f977fe640 1 --2- 172.21.15.64:0/2453522555 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2f840b6670 0x7f2f840b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:07.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.549+0000 7f2f977fe640 1 --2- 172.21.15.64:0/2453522555 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2f840b6670 0x7f2f840b8b80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2f88000c00 tx=0x7f2f88031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:07.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.551+0000 7f2f957fa640 1 -- 172.21.15.64:0/2453522555 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2f90014030 con 0x7f2f98108d20 2024-09-21T04:03:07.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.752+0000 7f2f9e032640 1 -- 172.21.15.64:0/2453522555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f2f98105580 con 0x7f2f98108d20 2024-09-21T04:03:07.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.753+0000 7f2f957fa640 1 -- 172.21.15.64:0/2453522555 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2f900ae060 con 0x7f2f98108d20 2024-09-21T04:03:07.755 INFO:teuthology.orchestra.run.smithi064.stdout:128849018942 2024-09-21T04:03:07.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.756+0000 7f2f9e032640 1 -- 172.21.15.64:0/2453522555 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2f840b6670 msgr2=0x7f2f840b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:07.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.756+0000 7f2f9e032640 1 --2- 172.21.15.64:0/2453522555 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2f840b6670 0x7f2f840b8b80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2f88000c00 tx=0x7f2f88031400 comp rx=0 tx=0).stop 2024-09-21T04:03:07.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.756+0000 7f2f9e032640 1 -- 172.21.15.64:0/2453522555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f98108d20 msgr2=0x7f2f98104d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:07.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.756+0000 7f2f9e032640 1 --2- 172.21.15.64:0/2453522555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f98108d20 0x7f2f98104d00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000cad0 tx=0x7f2f9000cf90 comp rx=0 tx=0).stop 2024-09-21T04:03:07.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.756+0000 7f2f9e032640 1 -- 172.21.15.64:0/2453522555 shutdown_connections 2024-09-21T04:03:07.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.756+0000 7f2f9e032640 1 --2- 172.21.15.64:0/2453522555 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2f840b6670 0x7f2f840b8b80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:07.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.756+0000 7f2f9e032640 1 --2- 172.21.15.64:0/2453522555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f98108d20 0x7f2f98104d00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:07.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.756+0000 7f2f9e032640 1 --2- 172.21.15.64:0/2453522555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f981083d0 0x7f2f981047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:07.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.756+0000 7f2f9e032640 1 --2- 172.21.15.64:0/2453522555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2f98102ae0 0x7f2f98104280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:07.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.756+0000 7f2f9e032640 1 -- 172.21.15.64:0/2453522555 >> 172.21.15.64:0/2453522555 conn(0x7f2f98079760 msgr2=0x7f2f98101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:07.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.756+0000 7f2f9e032640 1 -- 172.21.15.64:0/2453522555 shutdown_connections 2024-09-21T04:03:07.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:07.756+0000 7f2f9e032640 1 -- 172.21.15.64:0/2453522555 wait complete. 2024-09-21T04:03:08.407 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:08 smithi064 ceph-mon[31180]: pgmap v292: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T04:03:08.407 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:08 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2453522555' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T04:03:08.440 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018938 got 128849018942 for osd.4 2024-09-21T04:03:08.440 DEBUG:teuthology.parallel:result is None 2024-09-21T04:03:08.440 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T04:03:08.440 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T04:03:08.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:08 smithi119 ceph-mon[34811]: pgmap v292: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T04:03:08.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:08 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2453522555' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T04:03:08.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:08 smithi131 ceph-mon[35193]: pgmap v292: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T04:03:08.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:08 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2453522555' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T04:03:08.728 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:09.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.259+0000 7f87574b8640 1 -- 172.21.15.64:0/2816292077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8750108d20 msgr2=0x7f875010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:09.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.259+0000 7f87574b8640 1 --2- 172.21.15.64:0/2816292077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8750108d20 0x7f875010f550 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8748009ca0 tx=0x7f874802a8c0 comp rx=0 tx=0).stop 2024-09-21T04:03:09.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.259+0000 7f87574b8640 1 -- 172.21.15.64:0/2816292077 shutdown_connections 2024-09-21T04:03:09.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.259+0000 7f87574b8640 1 --2- 172.21.15.64:0/2816292077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8750108d20 0x7f875010f550 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:09.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.259+0000 7f87574b8640 1 --2- 172.21.15.64:0/2816292077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87501083d0 0x7f87501087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:09.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.259+0000 7f87574b8640 1 --2- 172.21.15.64:0/2816292077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8750102ae0 0x7f8750102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:09.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.259+0000 7f87574b8640 1 -- 172.21.15.64:0/2816292077 >> 172.21.15.64:0/2816292077 conn(0x7f8750079760 msgr2=0x7f8750079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:09.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.260+0000 7f87574b8640 1 -- 172.21.15.64:0/2816292077 shutdown_connections 2024-09-21T04:03:09.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.260+0000 7f87574b8640 1 -- 172.21.15.64:0/2816292077 wait complete. 2024-09-21T04:03:09.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.260+0000 7f87574b8640 1 Processor -- start 2024-09-21T04:03:09.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.260+0000 7f87574b8640 1 -- start start 2024-09-21T04:03:09.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.261+0000 7f87574b8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8750102ae0 0x7f875019b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:09.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.261+0000 7f87574b8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87501083d0 0x7f875019b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:09.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.261+0000 7f87574b8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8750108d20 0x7f875019bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:09.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.261+0000 7f87574b8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f87501129f0 con 0x7f8750102ae0 2024-09-21T04:03:09.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.261+0000 7f87574b8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8750112870 con 0x7f8750108d20 2024-09-21T04:03:09.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.261+0000 7f87574b8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8750112b70 con 0x7f87501083d0 2024-09-21T04:03:09.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.261+0000 7f8755254640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8750102ae0 0x7f875019b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:09.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.261+0000 7f8755254640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8750102ae0 0x7f875019b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56944/0 (socket says 172.21.15.64:56944) 2024-09-21T04:03:09.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.261+0000 7f8755254640 1 -- 172.21.15.64:0/4182583197 learned_addr learned my addr 172.21.15.64:0/4182583197 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:09.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.261+0000 7f8755a55640 1 --2- 172.21.15.64:0/4182583197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8750108d20 0x7f875019bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:09.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.261+0000 7f8754a53640 1 --2- 172.21.15.64:0/4182583197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87501083d0 0x7f875019b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:09.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.261+0000 7f8755254640 1 -- 172.21.15.64:0/4182583197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87501083d0 msgr2=0x7f875019b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:09.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.261+0000 7f8755254640 1 --2- 172.21.15.64:0/4182583197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87501083d0 0x7f875019b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:09.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.261+0000 7f8755254640 1 -- 172.21.15.64:0/4182583197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8750108d20 msgr2=0x7f875019bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:09.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.261+0000 7f8755254640 1 --2- 172.21.15.64:0/4182583197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8750108d20 0x7f875019bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:09.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.261+0000 7f8755254640 1 -- 172.21.15.64:0/4182583197 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87501a64b0 con 0x7f8750102ae0 2024-09-21T04:03:09.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.261+0000 7f8754a53640 1 --2- 172.21.15.64:0/4182583197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87501083d0 0x7f875019b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:03:09.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.261+0000 7f8755254640 1 --2- 172.21.15.64:0/4182583197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8750102ae0 0x7f875019b280 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f874c00c920 tx=0x7f874c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:09.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.262+0000 7f87467fc640 1 -- 172.21.15.64:0/4182583197 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c007c70 con 0x7f8750102ae0 2024-09-21T04:03:09.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.262+0000 7f87467fc640 1 -- 172.21.15.64:0/4182583197 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f874c007e10 con 0x7f8750102ae0 2024-09-21T04:03:09.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.262+0000 7f87574b8640 1 -- 172.21.15.64:0/4182583197 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87501a67a0 con 0x7f8750102ae0 2024-09-21T04:03:09.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.262+0000 7f87467fc640 1 -- 172.21.15.64:0/4182583197 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c018820 con 0x7f8750102ae0 2024-09-21T04:03:09.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.262+0000 7f87574b8640 1 -- 172.21.15.64:0/4182583197 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87501a6d60 con 0x7f8750102ae0 2024-09-21T04:03:09.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.263+0000 7f87574b8640 1 -- 172.21.15.64:0/4182583197 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8750103d10 con 0x7f8750102ae0 2024-09-21T04:03:09.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.264+0000 7f87467fc640 1 -- 172.21.15.64:0/4182583197 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f874c00f450 con 0x7f8750102ae0 2024-09-21T04:03:09.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.265+0000 7f87467fc640 1 --2- 172.21.15.64:0/4182583197 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f873c0b65f0 0x7f873c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:09.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.265+0000 7f8754a53640 1 --2- 172.21.15.64:0/4182583197 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f873c0b65f0 0x7f873c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:09.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.265+0000 7f87467fc640 1 -- 172.21.15.64:0/4182583197 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f874c0e7aa0 con 0x7f8750102ae0 2024-09-21T04:03:09.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.266+0000 7f8754a53640 1 --2- 172.21.15.64:0/4182583197 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f873c0b65f0 0x7f873c0b8b00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8740006fd0 tx=0x7f8740008040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:09.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.268+0000 7f87467fc640 1 -- 172.21.15.64:0/4182583197 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f874c014030 con 0x7f8750102ae0 2024-09-21T04:03:09.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.454+0000 7f87574b8640 1 -- 172.21.15.64:0/4182583197 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8750063800 con 0x7f873c0b65f0 2024-09-21T04:03:09.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.459+0000 7f87467fc640 1 -- 172.21.15.64:0/4182583197 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258990 (secure 0 0 0) 0x7f8750063800 con 0x7f873c0b65f0 2024-09-21T04:03:09.463 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:03:09.464 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-09-21T04:03:09.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.461+0000 7f87574b8640 1 -- 172.21.15.64:0/4182583197 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f873c0b65f0 msgr2=0x7f873c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:09.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.461+0000 7f87574b8640 1 --2- 172.21.15.64:0/4182583197 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f873c0b65f0 0x7f873c0b8b00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8740006fd0 tx=0x7f8740008040 comp rx=0 tx=0).stop 2024-09-21T04:03:09.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.461+0000 7f87574b8640 1 -- 172.21.15.64:0/4182583197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8750102ae0 msgr2=0x7f875019b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:09.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.461+0000 7f87574b8640 1 --2- 172.21.15.64:0/4182583197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8750102ae0 0x7f875019b280 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f874c00c920 tx=0x7f874c00cde0 comp rx=0 tx=0).stop 2024-09-21T04:03:09.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.461+0000 7f87574b8640 1 -- 172.21.15.64:0/4182583197 shutdown_connections 2024-09-21T04:03:09.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.461+0000 7f87574b8640 1 --2- 172.21.15.64:0/4182583197 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f873c0b65f0 0x7f873c0b8b00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:09.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.461+0000 7f87574b8640 1 --2- 172.21.15.64:0/4182583197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8750108d20 0x7f875019bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:09.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.461+0000 7f87574b8640 1 --2- 172.21.15.64:0/4182583197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87501083d0 0x7f875019b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:09.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.461+0000 7f87574b8640 1 --2- 172.21.15.64:0/4182583197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8750102ae0 0x7f875019b280 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:09.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.461+0000 7f87574b8640 1 -- 172.21.15.64:0/4182583197 >> 172.21.15.64:0/4182583197 conn(0x7f8750079760 msgr2=0x7f8750105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:09.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.462+0000 7f87574b8640 1 -- 172.21.15.64:0/4182583197 shutdown_connections 2024-09-21T04:03:09.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:09.462+0000 7f87574b8640 1 -- 172.21.15.64:0/4182583197 wait complete. 2024-09-21T04:03:09.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:09 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:03:09.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:09 smithi119 ceph-mon[34811]: pgmap v293: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T04:03:09.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:09 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:03:09.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:09 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:03:09.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:09 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:03:09.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:09 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:03:09.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:09 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:03:09.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:09 smithi064 ceph-mon[31180]: pgmap v293: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T04:03:09.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:09 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:03:09.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:09 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:03:09.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:09 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:03:09.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:09 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:03:09.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:09 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:03:09.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:09 smithi131 ceph-mon[35193]: pgmap v293: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T04:03:09.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:09 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:03:09.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:09 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:03:09.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:09 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:03:09.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:09 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:03:10.161 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-21T04:03:08.492210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660102,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":434,"num_read_kb":319,"num_write":287,"num_write_kb":2268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":2952422,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359384,"kb_used_data":10080,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124582440,"statfs":{"total":1151940427776,"available":1151572418560,"internally_reserved":0,"allocated":10321920,"data_stored":6847998,"data_compressed":182790,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003404"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.868680+0000","last_change":"2024-09-21T04:02:02.769848+0000","last_active":"2024-09-21T04:02:29.868680+0000","last_peered":"2024-09-21T04:02:29.868680+0000","last_clean":"2024-09-21T04:02:29.868680+0000","last_became_active":"2024-09-21T04:02:02.769508+0000","last_became_peered":"2024-09-21T04:02:02.769508+0000","last_unstale":"2024-09-21T04:02:29.868680+0000","last_undegraded":"2024-09-21T04:02:29.868680+0000","last_fullsized":"2024-09-21T04:02:29.868680+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:00.242233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866619+0000","last_change":"2024-09-21T04:02:02.769200+0000","last_active":"2024-09-21T04:02:29.866619+0000","last_peered":"2024-09-21T04:02:29.866619+0000","last_clean":"2024-09-21T04:02:29.866619+0000","last_became_active":"2024-09-21T04:02:02.768918+0000","last_became_peered":"2024-09-21T04:02:02.768918+0000","last_unstale":"2024-09-21T04:02:29.866619+0000","last_undegraded":"2024-09-21T04:02:29.866619+0000","last_fullsized":"2024-09-21T04:02:29.866619+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:26.842079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866563+0000","last_change":"2024-09-21T04:02:02.763511+0000","last_active":"2024-09-21T04:02:29.866563+0000","last_peered":"2024-09-21T04:02:29.866563+0000","last_clean":"2024-09-21T04:02:29.866563+0000","last_became_active":"2024-09-21T04:02:02.763161+0000","last_became_peered":"2024-09-21T04:02:02.763161+0000","last_unstale":"2024-09-21T04:02:29.866563+0000","last_undegraded":"2024-09-21T04:02:29.866563+0000","last_fullsized":"2024-09-21T04:02:29.866563+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:59.834005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.017853+0000","last_change":"2024-09-21T04:02:02.765452+0000","last_active":"2024-09-21T04:02:30.017853+0000","last_peered":"2024-09-21T04:02:30.017853+0000","last_clean":"2024-09-21T04:02:30.017853+0000","last_became_active":"2024-09-21T04:02:02.765065+0000","last_became_peered":"2024-09-21T04:02:02.765065+0000","last_unstale":"2024-09-21T04:02:30.017853+0000","last_undegraded":"2024-09-21T04:02:30.017853+0000","last_fullsized":"2024-09-21T04:02:30.017853+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:33.871366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.863125+0000","last_change":"2024-09-21T04:02:02.767457+0000","last_active":"2024-09-21T04:02:29.863125+0000","last_peered":"2024-09-21T04:02:29.863125+0000","last_clean":"2024-09-21T04:02:29.863125+0000","last_became_active":"2024-09-21T04:02:02.767064+0000","last_became_peered":"2024-09-21T04:02:02.767064+0000","last_unstale":"2024-09-21T04:02:29.863125+0000","last_undegraded":"2024-09-21T04:02:29.863125+0000","last_fullsized":"2024-09-21T04:02:29.863125+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:46.316029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.861263+0000","last_change":"2024-09-21T04:02:02.768795+0000","last_active":"2024-09-21T04:02:29.861263+0000","last_peered":"2024-09-21T04:02:29.861263+0000","last_clean":"2024-09-21T04:02:29.861263+0000","last_became_active":"2024-09-21T04:02:02.768434+0000","last_became_peered":"2024-09-21T04:02:02.768434+0000","last_unstale":"2024-09-21T04:02:29.861263+0000","last_undegraded":"2024-09-21T04:02:29.861263+0000","last_fullsized":"2024-09-21T04:02:29.861263+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:58.991533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.864547+0000","last_change":"2024-09-21T04:02:02.774998+0000","last_active":"2024-09-21T04:02:29.864547+0000","last_peered":"2024-09-21T04:02:29.864547+0000","last_clean":"2024-09-21T04:02:29.864547+0000","last_became_active":"2024-09-21T04:02:02.774344+0000","last_became_peered":"2024-09-21T04:02:02.774344+0000","last_unstale":"2024-09-21T04:02:29.864547+0000","last_undegraded":"2024-09-21T04:02:29.864547+0000","last_fullsized":"2024-09-21T04:02:29.864547+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:26.705126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865509+0000","last_change":"2024-09-21T04:02:02.772498+0000","last_active":"2024-09-21T04:02:29.865509+0000","last_peered":"2024-09-21T04:02:29.865509+0000","last_clean":"2024-09-21T04:02:29.865509+0000","last_became_active":"2024-09-21T04:02:02.772049+0000","last_became_peered":"2024-09-21T04:02:02.772049+0000","last_unstale":"2024-09-21T04:02:29.865509+0000","last_undegraded":"2024-09-21T04:02:29.865509+0000","last_fullsized":"2024-09-21T04:02:29.865509+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:31:19.088424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.016701+0000","last_change":"2024-09-21T04:02:02.772336+0000","last_active":"2024-09-21T04:02:30.016701+0000","last_peered":"2024-09-21T04:02:30.016701+0000","last_clean":"2024-09-21T04:02:30.016701+0000","last_became_active":"2024-09-21T04:02:02.771964+0000","last_became_peered":"2024-09-21T04:02:02.771964+0000","last_unstale":"2024-09-21T04:02:30.016701+0000","last_undegraded":"2024-09-21T04:02:30.016701+0000","last_fullsized":"2024-09-21T04:02:30.016701+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:46.659690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.861686+0000","last_change":"2024-09-21T04:02:02.758992+0000","last_active":"2024-09-21T04:02:29.861686+0000","last_peered":"2024-09-21T04:02:29.861686+0000","last_clean":"2024-09-21T04:02:29.861686+0000","last_became_active":"2024-09-21T04:02:02.758575+0000","last_became_peered":"2024-09-21T04:02:02.758575+0000","last_unstale":"2024-09-21T04:02:29.861686+0000","last_undegraded":"2024-09-21T04:02:29.861686+0000","last_fullsized":"2024-09-21T04:02:29.861686+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:32:09.543140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865235+0000","last_change":"2024-09-21T04:02:02.767304+0000","last_active":"2024-09-21T04:02:29.865235+0000","last_peered":"2024-09-21T04:02:29.865235+0000","last_clean":"2024-09-21T04:02:29.865235+0000","last_became_active":"2024-09-21T04:02:02.762804+0000","last_became_peered":"2024-09-21T04:02:02.762804+0000","last_unstale":"2024-09-21T04:02:29.865235+0000","last_undegraded":"2024-09-21T04:02:29.865235+0000","last_fullsized":"2024-09-21T04:02:29.865235+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:32.224071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.017995+0000","last_change":"2024-09-21T04:02:02.762932+0000","last_active":"2024-09-21T04:02:30.017995+0000","last_peered":"2024-09-21T04:02:30.017995+0000","last_clean":"2024-09-21T04:02:30.017995+0000","last_became_active":"2024-09-21T04:02:02.762538+0000","last_became_peered":"2024-09-21T04:02:02.762538+0000","last_unstale":"2024-09-21T04:02:30.017995+0000","last_undegraded":"2024-09-21T04:02:30.017995+0000","last_fullsized":"2024-09-21T04:02:30.017995+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:20.724453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.863000+0000","last_change":"2024-09-21T04:02:02.770313+0000","last_active":"2024-09-21T04:02:29.863000+0000","last_peered":"2024-09-21T04:02:29.863000+0000","last_clean":"2024-09-21T04:02:29.863000+0000","last_became_active":"2024-09-21T04:02:02.769910+0000","last_became_peered":"2024-09-21T04:02:02.769910+0000","last_unstale":"2024-09-21T04:02:29.863000+0000","last_undegraded":"2024-09-21T04:02:29.863000+0000","last_fullsized":"2024-09-21T04:02:29.863000+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:16.228371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866455+0000","last_change":"2024-09-21T04:02:02.769258+0000","last_active":"2024-09-21T04:02:29.866455+0000","last_peered":"2024-09-21T04:02:29.866455+0000","last_clean":"2024-09-21T04:02:29.866455+0000","last_became_active":"2024-09-21T04:02:02.768914+0000","last_became_peered":"2024-09-21T04:02:02.768914+0000","last_unstale":"2024-09-21T04:02:29.866455+0000","last_undegraded":"2024-09-21T04:02:29.866455+0000","last_fullsized":"2024-09-21T04:02:29.866455+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:37.426810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866413+0000","last_change":"2024-09-21T04:02:02.755940+0000","last_active":"2024-09-21T04:02:29.866413+0000","last_peered":"2024-09-21T04:02:29.866413+0000","last_clean":"2024-09-21T04:02:29.866413+0000","last_became_active":"2024-09-21T04:02:02.755309+0000","last_became_peered":"2024-09-21T04:02:02.755309+0000","last_unstale":"2024-09-21T04:02:29.866413+0000","last_undegraded":"2024-09-21T04:02:29.866413+0000","last_fullsized":"2024-09-21T04:02:29.866413+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:49.834030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.017085+0000","last_change":"2024-09-21T04:02:02.761068+0000","last_active":"2024-09-21T04:02:30.017085+0000","last_peered":"2024-09-21T04:02:30.017085+0000","last_clean":"2024-09-21T04:02:30.017085+0000","last_became_active":"2024-09-21T04:02:02.760683+0000","last_became_peered":"2024-09-21T04:02:02.760683+0000","last_unstale":"2024-09-21T04:02:30.017085+0000","last_undegraded":"2024-09-21T04:02:30.017085+0000","last_fullsized":"2024-09-21T04:02:30.017085+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:36.627539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.868703+0000","last_change":"2024-09-21T04:02:02.772234+0000","last_active":"2024-09-21T04:02:29.868703+0000","last_peered":"2024-09-21T04:02:29.868703+0000","last_clean":"2024-09-21T04:02:29.868703+0000","last_became_active":"2024-09-21T04:02:02.771825+0000","last_became_peered":"2024-09-21T04:02:02.771825+0000","last_unstale":"2024-09-21T04:02:29.868703+0000","last_undegraded":"2024-09-21T04:02:29.868703+0000","last_fullsized":"2024-09-21T04:02:29.868703+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:33.449745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.863704+0000","last_change":"2024-09-21T04:02:02.758643+0000","last_active":"2024-09-21T04:02:29.863704+0000","last_peered":"2024-09-21T04:02:29.863704+0000","last_clean":"2024-09-21T04:02:29.863704+0000","last_became_active":"2024-09-21T04:02:02.758261+0000","last_became_peered":"2024-09-21T04:02:02.758261+0000","last_unstale":"2024-09-21T04:02:29.863704+0000","last_undegraded":"2024-09-21T04:02:29.863704+0000","last_fullsized":"2024-09-21T04:02:29.863704+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:04.165399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.861196+0000","last_change":"2024-09-21T04:02:02.759405+0000","last_active":"2024-09-21T04:02:29.861196+0000","last_peered":"2024-09-21T04:02:29.861196+0000","last_clean":"2024-09-21T04:02:29.861196+0000","last_became_active":"2024-09-21T04:02:02.758962+0000","last_became_peered":"2024-09-21T04:02:02.758962+0000","last_unstale":"2024-09-21T04:02:29.861196+0000","last_undegraded":"2024-09-21T04:02:29.861196+0000","last_fullsized":"2024-09-21T04:02:29.861196+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:37.203308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.867267+0000","last_change":"2024-09-21T04:02:02.767518+0000","last_active":"2024-09-21T04:02:29.867267+0000","last_peered":"2024-09-21T04:02:29.867267+0000","last_clean":"2024-09-21T04:02:29.867267+0000","last_became_active":"2024-09-21T04:02:02.767147+0000","last_became_peered":"2024-09-21T04:02:02.767147+0000","last_unstale":"2024-09-21T04:02:29.867267+0000","last_undegraded":"2024-09-21T04:02:29.867267+0000","last_fullsized":"2024-09-21T04:02:29.867267+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:34:13.615218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.863107+0000","last_change":"2024-09-21T04:02:02.756407+0000","last_active":"2024-09-21T04:02:29.863107+0000","last_peered":"2024-09-21T04:02:29.863107+0000","last_clean":"2024-09-21T04:02:29.863107+0000","last_became_active":"2024-09-21T04:02:02.756007+0000","last_became_peered":"2024-09-21T04:02:02.756007+0000","last_unstale":"2024-09-21T04:02:29.863107+0000","last_undegraded":"2024-09-21T04:02:29.863107+0000","last_fullsized":"2024-09-21T04:02:29.863107+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:18.798814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.017244+0000","last_change":"2024-09-21T04:02:02.766290+0000","last_active":"2024-09-21T04:02:30.017244+0000","last_peered":"2024-09-21T04:02:30.017244+0000","last_clean":"2024-09-21T04:02:30.017244+0000","last_became_active":"2024-09-21T04:02:02.765660+0000","last_became_peered":"2024-09-21T04:02:02.765660+0000","last_unstale":"2024-09-21T04:02:30.017244+0000","last_undegraded":"2024-09-21T04:02:30.017244+0000","last_fullsized":"2024-09-21T04:02:30.017244+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:23.281476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865621+0000","last_change":"2024-09-21T04:02:02.767943+0000","last_active":"2024-09-21T04:02:29.865621+0000","last_peered":"2024-09-21T04:02:29.865621+0000","last_clean":"2024-09-21T04:02:29.865621+0000","last_became_active":"2024-09-21T04:02:02.767566+0000","last_became_peered":"2024-09-21T04:02:02.767566+0000","last_unstale":"2024-09-21T04:02:29.865621+0000","last_undegraded":"2024-09-21T04:02:29.865621+0000","last_fullsized":"2024-09-21T04:02:29.865621+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:44.221531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866695+0000","last_change":"2024-09-21T04:02:02.770114+0000","last_active":"2024-09-21T04:02:29.866695+0000","last_peered":"2024-09-21T04:02:29.866695+0000","last_clean":"2024-09-21T04:02:29.866695+0000","last_became_active":"2024-09-21T04:02:02.769732+0000","last_became_peered":"2024-09-21T04:02:02.769732+0000","last_unstale":"2024-09-21T04:02:29.866695+0000","last_undegraded":"2024-09-21T04:02:29.866695+0000","last_fullsized":"2024-09-21T04:02:29.866695+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:38.396288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.017726+0000","last_change":"2024-09-21T04:02:30.017325+0000","last_active":"2024-09-21T04:02:30.017726+0000","last_peered":"2024-09-21T04:02:30.017726+0000","last_clean":"2024-09-21T04:02:30.017726+0000","last_became_active":"2024-09-21T04:02:30.017014+0000","last_became_peered":"2024-09-21T04:02:30.017014+0000","last_unstale":"2024-09-21T04:02:30.017726+0000","last_undegraded":"2024-09-21T04:02:30.017726+0000","last_fullsized":"2024-09-21T04:02:30.017726+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:28.579436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.861493+0000","last_change":"2024-09-21T04:02:02.764666+0000","last_active":"2024-09-21T04:02:29.861493+0000","last_peered":"2024-09-21T04:02:29.861493+0000","last_clean":"2024-09-21T04:02:29.861493+0000","last_became_active":"2024-09-21T04:02:02.763931+0000","last_became_peered":"2024-09-21T04:02:02.763931+0000","last_unstale":"2024-09-21T04:02:29.861493+0000","last_undegraded":"2024-09-21T04:02:29.861493+0000","last_fullsized":"2024-09-21T04:02:29.861493+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:39.724770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.867617+0000","last_change":"2024-09-21T04:02:02.774276+0000","last_active":"2024-09-21T04:02:29.867617+0000","last_peered":"2024-09-21T04:02:29.867617+0000","last_clean":"2024-09-21T04:02:29.867617+0000","last_became_active":"2024-09-21T04:02:02.773852+0000","last_became_peered":"2024-09-21T04:02:02.773852+0000","last_unstale":"2024-09-21T04:02:29.867617+0000","last_undegraded":"2024-09-21T04:02:29.867617+0000","last_fullsized":"2024-09-21T04:02:29.867617+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:40.072264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.862794+0000","last_change":"2024-09-21T04:02:02.758520+0000","last_active":"2024-09-21T04:02:29.862794+0000","last_peered":"2024-09-21T04:02:29.862794+0000","last_clean":"2024-09-21T04:02:29.862794+0000","last_became_active":"2024-09-21T04:02:02.757656+0000","last_became_peered":"2024-09-21T04:02:02.757656+0000","last_unstale":"2024-09-21T04:02:29.862794+0000","last_undegraded":"2024-09-21T04:02:29.862794+0000","last_fullsized":"2024-09-21T04:02:29.862794+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:30.854336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866958+0000","last_change":"2024-09-21T04:02:02.756428+0000","last_active":"2024-09-21T04:02:29.866958+0000","last_peered":"2024-09-21T04:02:29.866958+0000","last_clean":"2024-09-21T04:02:29.866958+0000","last_became_active":"2024-09-21T04:02:02.755985+0000","last_became_peered":"2024-09-21T04:02:02.755985+0000","last_unstale":"2024-09-21T04:02:29.866958+0000","last_undegraded":"2024-09-21T04:02:29.866958+0000","last_fullsized":"2024-09-21T04:02:29.866958+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:36.040436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.860813+0000","last_change":"2024-09-21T04:02:02.762893+0000","last_active":"2024-09-21T04:02:29.860813+0000","last_peered":"2024-09-21T04:02:29.860813+0000","last_clean":"2024-09-21T04:02:29.860813+0000","last_became_active":"2024-09-21T04:02:02.762343+0000","last_became_peered":"2024-09-21T04:02:02.762343+0000","last_unstale":"2024-09-21T04:02:29.860813+0000","last_undegraded":"2024-09-21T04:02:29.860813+0000","last_fullsized":"2024-09-21T04:02:29.860813+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:45.074711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.860958+0000","last_change":"2024-09-21T04:02:02.763942+0000","last_active":"2024-09-21T04:02:29.860958+0000","last_peered":"2024-09-21T04:02:29.860958+0000","last_clean":"2024-09-21T04:02:29.860958+0000","last_became_active":"2024-09-21T04:02:02.763433+0000","last_became_peered":"2024-09-21T04:02:02.763433+0000","last_unstale":"2024-09-21T04:02:29.860958+0000","last_undegraded":"2024-09-21T04:02:29.860958+0000","last_fullsized":"2024-09-21T04:02:29.860958+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:36.139440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.867052+0000","last_change":"2024-09-21T04:02:02.773364+0000","last_active":"2024-09-21T04:02:29.867052+0000","last_peered":"2024-09-21T04:02:29.867052+0000","last_clean":"2024-09-21T04:02:29.867052+0000","last_became_active":"2024-09-21T04:02:02.773040+0000","last_became_peered":"2024-09-21T04:02:02.773040+0000","last_unstale":"2024-09-21T04:02:29.867052+0000","last_undegraded":"2024-09-21T04:02:29.867052+0000","last_fullsized":"2024-09-21T04:02:29.867052+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:48.504476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.862997+0000","last_change":"2024-09-21T04:02:02.774951+0000","last_active":"2024-09-21T04:02:29.862997+0000","last_peered":"2024-09-21T04:02:29.862997+0000","last_clean":"2024-09-21T04:02:29.862997+0000","last_became_active":"2024-09-21T04:02:02.774525+0000","last_became_peered":"2024-09-21T04:02:02.774525+0000","last_unstale":"2024-09-21T04:02:29.862997+0000","last_undegraded":"2024-09-21T04:02:29.862997+0000","last_fullsized":"2024-09-21T04:02:29.862997+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:24.447860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"73'5","reported_seq":84,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:03:04.891537+0000","last_change":"2024-09-21T04:02:00.743470+0000","last_active":"2024-09-21T04:03:04.891537+0000","last_peered":"2024-09-21T04:03:04.891537+0000","last_clean":"2024-09-21T04:03:04.891537+0000","last_became_active":"2024-09-21T04:02:00.743055+0000","last_became_peered":"2024-09-21T04:02:00.743055+0000","last_unstale":"2024-09-21T04:03:04.891537+0000","last_undegraded":"2024-09-21T04:03:04.891537+0000","last_fullsized":"2024-09-21T04:03:04.891537+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:56.101753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866720+0000","last_change":"2024-09-21T04:02:02.773399+0000","last_active":"2024-09-21T04:02:29.866720+0000","last_peered":"2024-09-21T04:02:29.866720+0000","last_clean":"2024-09-21T04:02:29.866720+0000","last_became_active":"2024-09-21T04:02:02.772996+0000","last_became_peered":"2024-09-21T04:02:02.772996+0000","last_unstale":"2024-09-21T04:02:29.866720+0000","last_undegraded":"2024-09-21T04:02:29.866720+0000","last_fullsized":"2024-09-21T04:02:29.866720+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:07.256226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"73'3","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.026998+0000","last_change":"2024-09-21T04:02:30.026868+0000","last_active":"2024-09-21T04:02:30.026998+0000","last_peered":"2024-09-21T04:02:30.026998+0000","last_clean":"2024-09-21T04:02:30.026998+0000","last_became_active":"2024-09-21T04:02:30.019469+0000","last_became_peered":"2024-09-21T04:02:30.019469+0000","last_unstale":"2024-09-21T04:02:30.026998+0000","last_undegraded":"2024-09-21T04:02:30.026998+0000","last_fullsized":"2024-09-21T04:02:30.026998+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:35.304358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"73'2","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.862695+0000","last_change":"2024-09-21T04:02:00.736630+0000","last_active":"2024-09-21T04:02:29.862695+0000","last_peered":"2024-09-21T04:02:29.862695+0000","last_clean":"2024-09-21T04:02:29.862695+0000","last_became_active":"2024-09-21T04:02:00.736211+0000","last_became_peered":"2024-09-21T04:02:00.736211+0000","last_unstale":"2024-09-21T04:02:29.862695+0000","last_undegraded":"2024-09-21T04:02:29.862695+0000","last_fullsized":"2024-09-21T04:02:29.862695+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:37.211617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865583+0000","last_change":"2024-09-21T04:02:02.763323+0000","last_active":"2024-09-21T04:02:29.865583+0000","last_peered":"2024-09-21T04:02:29.865583+0000","last_clean":"2024-09-21T04:02:29.865583+0000","last_became_active":"2024-09-21T04:02:02.762776+0000","last_became_peered":"2024-09-21T04:02:02.762776+0000","last_unstale":"2024-09-21T04:02:29.865583+0000","last_undegraded":"2024-09-21T04:02:29.865583+0000","last_fullsized":"2024-09-21T04:02:29.865583+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:39.891042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"73'1","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866894+0000","last_change":"2024-09-21T04:02:00.743377+0000","last_active":"2024-09-21T04:02:29.866894+0000","last_peered":"2024-09-21T04:02:29.866894+0000","last_clean":"2024-09-21T04:02:29.866894+0000","last_became_active":"2024-09-21T04:02:00.742973+0000","last_became_peered":"2024-09-21T04:02:00.742973+0000","last_unstale":"2024-09-21T04:02:29.866894+0000","last_undegraded":"2024-09-21T04:02:29.866894+0000","last_fullsized":"2024-09-21T04:02:29.866894+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:55:00.655532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.863390+0000","last_change":"2024-09-21T04:02:02.758721+0000","last_active":"2024-09-21T04:02:29.863390+0000","last_peered":"2024-09-21T04:02:29.863390+0000","last_clean":"2024-09-21T04:02:29.863390+0000","last_became_active":"2024-09-21T04:02:02.758352+0000","last_became_peered":"2024-09-21T04:02:02.758352+0000","last_unstale":"2024-09-21T04:02:29.863390+0000","last_undegraded":"2024-09-21T04:02:29.863390+0000","last_fullsized":"2024-09-21T04:02:29.863390+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:56:49.289510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"73'3","reported_seq":81,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:03:05.892467+0000","last_change":"2024-09-21T04:02:00.744591+0000","last_active":"2024-09-21T04:03:05.892467+0000","last_peered":"2024-09-21T04:03:05.892467+0000","last_clean":"2024-09-21T04:03:05.892467+0000","last_became_active":"2024-09-21T04:02:00.744139+0000","last_became_peered":"2024-09-21T04:02:00.744139+0000","last_unstale":"2024-09-21T04:03:05.892467+0000","last_undegraded":"2024-09-21T04:03:05.892467+0000","last_fullsized":"2024-09-21T04:03:05.892467+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:21.331517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.016835+0000","last_change":"2024-09-21T04:02:02.762779+0000","last_active":"2024-09-21T04:02:30.016835+0000","last_peered":"2024-09-21T04:02:30.016835+0000","last_clean":"2024-09-21T04:02:30.016835+0000","last_became_active":"2024-09-21T04:02:02.760761+0000","last_became_peered":"2024-09-21T04:02:02.760761+0000","last_unstale":"2024-09-21T04:02:30.016835+0000","last_undegraded":"2024-09-21T04:02:30.016835+0000","last_fullsized":"2024-09-21T04:02:30.016835+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:18.789052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"73'2","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865511+0000","last_change":"2024-09-21T04:02:00.743220+0000","last_active":"2024-09-21T04:02:29.865511+0000","last_peered":"2024-09-21T04:02:29.865511+0000","last_clean":"2024-09-21T04:02:29.865511+0000","last_became_active":"2024-09-21T04:02:00.742636+0000","last_became_peered":"2024-09-21T04:02:00.742636+0000","last_unstale":"2024-09-21T04:02:29.865511+0000","last_undegraded":"2024-09-21T04:02:29.865511+0000","last_fullsized":"2024-09-21T04:02:29.865511+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:01.382338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.869766+0000","last_change":"2024-09-21T04:02:29.869663+0000","last_active":"2024-09-21T04:02:29.869766+0000","last_peered":"2024-09-21T04:02:29.869766+0000","last_clean":"2024-09-21T04:02:29.869766+0000","last_became_active":"2024-09-21T04:02:29.869230+0000","last_became_peered":"2024-09-21T04:02:29.869230+0000","last_unstale":"2024-09-21T04:02:29.869766+0000","last_undegraded":"2024-09-21T04:02:29.869766+0000","last_fullsized":"2024-09-21T04:02:29.869766+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:19:59.987267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"73'4","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.867117+0000","last_change":"2024-09-21T04:02:00.853284+0000","last_active":"2024-09-21T04:02:29.867117+0000","last_peered":"2024-09-21T04:02:29.867117+0000","last_clean":"2024-09-21T04:02:29.867117+0000","last_became_active":"2024-09-21T04:02:00.852855+0000","last_became_peered":"2024-09-21T04:02:00.852855+0000","last_unstale":"2024-09-21T04:02:29.867117+0000","last_undegraded":"2024-09-21T04:02:29.867117+0000","last_fullsized":"2024-09-21T04:02:29.867117+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:36.994214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.016563+0000","last_change":"2024-09-21T04:02:02.763577+0000","last_active":"2024-09-21T04:02:30.016563+0000","last_peered":"2024-09-21T04:02:30.016563+0000","last_clean":"2024-09-21T04:02:30.016563+0000","last_became_active":"2024-09-21T04:02:02.762970+0000","last_became_peered":"2024-09-21T04:02:02.762970+0000","last_unstale":"2024-09-21T04:02:30.016563+0000","last_undegraded":"2024-09-21T04:02:30.016563+0000","last_fullsized":"2024-09-21T04:02:30.016563+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:49.956226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'29","reported_seq":171,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:03:06.893840+0000","last_change":"2024-09-21T04:02:00.744203+0000","last_active":"2024-09-21T04:03:06.893840+0000","last_peered":"2024-09-21T04:03:06.893840+0000","last_clean":"2024-09-21T04:03:06.893840+0000","last_became_active":"2024-09-21T04:02:00.743659+0000","last_became_peered":"2024-09-21T04:02:00.743659+0000","last_unstale":"2024-09-21T04:03:06.893840+0000","last_undegraded":"2024-09-21T04:03:06.893840+0000","last_fullsized":"2024-09-21T04:03:06.893840+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:32.108302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65890,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":119,"num_read_kb":123,"num_write":30,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.863164+0000","last_change":"2024-09-21T04:02:02.766002+0000","last_active":"2024-09-21T04:02:29.863164+0000","last_peered":"2024-09-21T04:02:29.863164+0000","last_clean":"2024-09-21T04:02:29.863164+0000","last_became_active":"2024-09-21T04:02:02.765628+0000","last_became_peered":"2024-09-21T04:02:02.765628+0000","last_unstale":"2024-09-21T04:02:29.863164+0000","last_undegraded":"2024-09-21T04:02:29.863164+0000","last_fullsized":"2024-09-21T04:02:29.863164+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:22.251586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"73'6","reported_seq":30,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.861100+0000","last_change":"2024-09-21T04:02:00.741086+0000","last_active":"2024-09-21T04:02:29.861100+0000","last_peered":"2024-09-21T04:02:29.861100+0000","last_clean":"2024-09-21T04:02:29.861100+0000","last_became_active":"2024-09-21T04:02:00.740633+0000","last_became_peered":"2024-09-21T04:02:00.740633+0000","last_unstale":"2024-09-21T04:02:29.861100+0000","last_undegraded":"2024-09-21T04:02:29.861100+0000","last_fullsized":"2024-09-21T04:02:29.861100+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:20.392799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.862759+0000","last_change":"2024-09-21T04:02:02.759651+0000","last_active":"2024-09-21T04:02:29.862759+0000","last_peered":"2024-09-21T04:02:29.862759+0000","last_clean":"2024-09-21T04:02:29.862759+0000","last_became_active":"2024-09-21T04:02:02.759290+0000","last_became_peered":"2024-09-21T04:02:02.759290+0000","last_unstale":"2024-09-21T04:02:29.862759+0000","last_undegraded":"2024-09-21T04:02:29.862759+0000","last_fullsized":"2024-09-21T04:02:29.862759+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:32.025703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"73'3","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866619+0000","last_change":"2024-09-21T04:02:00.744005+0000","last_active":"2024-09-21T04:02:29.866619+0000","last_peered":"2024-09-21T04:02:29.866619+0000","last_clean":"2024-09-21T04:02:29.866619+0000","last_became_active":"2024-09-21T04:02:00.743570+0000","last_became_peered":"2024-09-21T04:02:00.743570+0000","last_unstale":"2024-09-21T04:02:29.866619+0000","last_undegraded":"2024-09-21T04:02:29.866619+0000","last_fullsized":"2024-09-21T04:02:29.866619+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:32.158681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.869292+0000","last_change":"2024-09-21T04:02:29.869200+0000","last_active":"2024-09-21T04:02:29.869292+0000","last_peered":"2024-09-21T04:02:29.869292+0000","last_clean":"2024-09-21T04:02:29.869292+0000","last_became_active":"2024-09-21T04:02:29.868836+0000","last_became_peered":"2024-09-21T04:02:29.868836+0000","last_unstale":"2024-09-21T04:02:29.869292+0000","last_undegraded":"2024-09-21T04:02:29.869292+0000","last_fullsized":"2024-09-21T04:02:29.869292+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:42:49.030499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"73'3","reported_seq":81,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:03:03.889965+0000","last_change":"2024-09-21T04:02:00.741596+0000","last_active":"2024-09-21T04:03:03.889965+0000","last_peered":"2024-09-21T04:03:03.889965+0000","last_clean":"2024-09-21T04:03:03.889965+0000","last_became_active":"2024-09-21T04:02:00.741185+0000","last_became_peered":"2024-09-21T04:02:00.741185+0000","last_unstale":"2024-09-21T04:03:03.889965+0000","last_undegraded":"2024-09-21T04:03:03.889965+0000","last_fullsized":"2024-09-21T04:03:03.889965+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:31.663133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.864575+0000","last_change":"2024-09-21T04:02:02.773706+0000","last_active":"2024-09-21T04:02:29.864575+0000","last_peered":"2024-09-21T04:02:29.864575+0000","last_clean":"2024-09-21T04:02:29.864575+0000","last_became_active":"2024-09-21T04:02:02.773259+0000","last_became_peered":"2024-09-21T04:02:02.773259+0000","last_unstale":"2024-09-21T04:02:29.864575+0000","last_undegraded":"2024-09-21T04:02:29.864575+0000","last_fullsized":"2024-09-21T04:02:29.864575+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:50.431982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"73'15","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.862850+0000","last_change":"2024-09-21T04:02:00.851335+0000","last_active":"2024-09-21T04:02:29.862850+0000","last_peered":"2024-09-21T04:02:29.862850+0000","last_clean":"2024-09-21T04:02:29.862850+0000","last_became_active":"2024-09-21T04:02:00.850990+0000","last_became_peered":"2024-09-21T04:02:00.850990+0000","last_unstale":"2024-09-21T04:02:29.862850+0000","last_undegraded":"2024-09-21T04:02:29.862850+0000","last_fullsized":"2024-09-21T04:02:29.862850+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:45.731121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866618+0000","last_change":"2024-09-21T04:02:02.770985+0000","last_active":"2024-09-21T04:02:29.866618+0000","last_peered":"2024-09-21T04:02:29.866618+0000","last_clean":"2024-09-21T04:02:29.866618+0000","last_became_active":"2024-09-21T04:02:02.770602+0000","last_became_peered":"2024-09-21T04:02:02.770602+0000","last_unstale":"2024-09-21T04:02:29.866618+0000","last_undegraded":"2024-09-21T04:02:29.866618+0000","last_fullsized":"2024-09-21T04:02:29.866618+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:37.131113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"73'2","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.863333+0000","last_change":"2024-09-21T04:02:00.740834+0000","last_active":"2024-09-21T04:02:29.863333+0000","last_peered":"2024-09-21T04:02:29.863333+0000","last_clean":"2024-09-21T04:02:29.863333+0000","last_became_active":"2024-09-21T04:02:00.740439+0000","last_became_peered":"2024-09-21T04:02:00.740439+0000","last_unstale":"2024-09-21T04:02:29.863333+0000","last_undegraded":"2024-09-21T04:02:29.863333+0000","last_fullsized":"2024-09-21T04:02:29.863333+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:04:22.159986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.018787+0000","last_change":"2024-09-21T04:02:30.018671+0000","last_active":"2024-09-21T04:02:30.018787+0000","last_peered":"2024-09-21T04:02:30.018787+0000","last_clean":"2024-09-21T04:02:30.018787+0000","last_became_active":"2024-09-21T04:02:30.018247+0000","last_became_peered":"2024-09-21T04:02:30.018247+0000","last_unstale":"2024-09-21T04:02:30.018787+0000","last_undegraded":"2024-09-21T04:02:30.018787+0000","last_fullsized":"2024-09-21T04:02:30.018787+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:13.501966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"73'2","reported_seq":36,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.024653+0000","last_change":"2024-09-21T04:02:30.024526+0000","last_active":"2024-09-21T04:02:30.024653+0000","last_peered":"2024-09-21T04:02:30.024653+0000","last_clean":"2024-09-21T04:02:30.024653+0000","last_became_active":"2024-09-21T04:02:30.018811+0000","last_became_peered":"2024-09-21T04:02:30.018811+0000","last_unstale":"2024-09-21T04:02:30.024653+0000","last_undegraded":"2024-09-21T04:02:30.024653+0000","last_fullsized":"2024-09-21T04:02:30.024653+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:07.093364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.864925+0000","last_change":"2024-09-21T04:02:02.766709+0000","last_active":"2024-09-21T04:02:29.864925+0000","last_peered":"2024-09-21T04:02:29.864925+0000","last_clean":"2024-09-21T04:02:29.864925+0000","last_became_active":"2024-09-21T04:02:02.766342+0000","last_became_peered":"2024-09-21T04:02:02.766342+0000","last_unstale":"2024-09-21T04:02:29.864925+0000","last_undegraded":"2024-09-21T04:02:29.864925+0000","last_fullsized":"2024-09-21T04:02:29.864925+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:26:35.185368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"73'1","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.867256+0000","last_change":"2024-09-21T04:02:00.745365+0000","last_active":"2024-09-21T04:02:29.867256+0000","last_peered":"2024-09-21T04:02:29.867256+0000","last_clean":"2024-09-21T04:02:29.867256+0000","last_became_active":"2024-09-21T04:02:00.744968+0000","last_became_peered":"2024-09-21T04:02:00.744968+0000","last_unstale":"2024-09-21T04:02:29.867256+0000","last_undegraded":"2024-09-21T04:02:29.867256+0000","last_fullsized":"2024-09-21T04:02:29.867256+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:56.078169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"73'1","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.864785+0000","last_change":"2024-09-21T04:02:02.774877+0000","last_active":"2024-09-21T04:02:29.864785+0000","last_peered":"2024-09-21T04:02:29.864785+0000","last_clean":"2024-09-21T04:02:29.864785+0000","last_became_active":"2024-09-21T04:02:02.774464+0000","last_became_peered":"2024-09-21T04:02:02.774464+0000","last_unstale":"2024-09-21T04:02:29.864785+0000","last_undegraded":"2024-09-21T04:02:29.864785+0000","last_fullsized":"2024-09-21T04:02:29.864785+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:23.393239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"73'3","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.861284+0000","last_change":"2024-09-21T04:02:01.257649+0000","last_active":"2024-09-21T04:02:29.861284+0000","last_peered":"2024-09-21T04:02:29.861284+0000","last_clean":"2024-09-21T04:02:29.861284+0000","last_became_active":"2024-09-21T04:02:01.257176+0000","last_became_peered":"2024-09-21T04:02:01.257176+0000","last_unstale":"2024-09-21T04:02:29.861284+0000","last_undegraded":"2024-09-21T04:02:29.861284+0000","last_fullsized":"2024-09-21T04:02:29.861284+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:37.257702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865292+0000","last_change":"2024-09-21T04:02:02.768164+0000","last_active":"2024-09-21T04:02:29.865292+0000","last_peered":"2024-09-21T04:02:29.865292+0000","last_clean":"2024-09-21T04:02:29.865292+0000","last_became_active":"2024-09-21T04:02:02.766925+0000","last_became_peered":"2024-09-21T04:02:02.766925+0000","last_unstale":"2024-09-21T04:02:29.865292+0000","last_undegraded":"2024-09-21T04:02:29.865292+0000","last_fullsized":"2024-09-21T04:02:29.865292+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:59.707637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":95,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865477+0000","last_change":"2024-09-21T04:00:48.936996+0000","last_active":"2024-09-21T04:02:29.865477+0000","last_peered":"2024-09-21T04:02:29.865477+0000","last_clean":"2024-09-21T04:02:29.865477+0000","last_became_active":"2024-09-21T04:00:48.931310+0000","last_became_peered":"2024-09-21T04:00:48.931310+0000","last_unstale":"2024-09-21T04:02:29.865477+0000","last_undegraded":"2024-09-21T04:02:29.865477+0000","last_fullsized":"2024-09-21T04:02:29.865477+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:57:25.346362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:57:25.346362+0000","last_clean_scrub_stamp":"2024-09-21T03:57:25.346362+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:24.165731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"73'3","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.017671+0000","last_change":"2024-09-21T04:02:00.740118+0000","last_active":"2024-09-21T04:02:30.017671+0000","last_peered":"2024-09-21T04:02:30.017671+0000","last_clean":"2024-09-21T04:02:30.017671+0000","last_became_active":"2024-09-21T04:02:00.739677+0000","last_became_peered":"2024-09-21T04:02:00.739677+0000","last_unstale":"2024-09-21T04:02:30.017671+0000","last_undegraded":"2024-09-21T04:02:30.017671+0000","last_fullsized":"2024-09-21T04:02:30.017671+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:46.084118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865883+0000","last_change":"2024-09-21T04:02:02.771537+0000","last_active":"2024-09-21T04:02:29.865883+0000","last_peered":"2024-09-21T04:02:29.865883+0000","last_clean":"2024-09-21T04:02:29.865883+0000","last_became_active":"2024-09-21T04:02:02.771151+0000","last_became_peered":"2024-09-21T04:02:02.771151+0000","last_unstale":"2024-09-21T04:02:29.865883+0000","last_undegraded":"2024-09-21T04:02:29.865883+0000","last_fullsized":"2024-09-21T04:02:29.865883+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:21.283515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"73'3","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.022770+0000","last_change":"2024-09-21T04:02:30.022634+0000","last_active":"2024-09-21T04:02:30.022770+0000","last_peered":"2024-09-21T04:02:30.022770+0000","last_clean":"2024-09-21T04:02:30.022770+0000","last_became_active":"2024-09-21T04:02:30.018076+0000","last_became_peered":"2024-09-21T04:02:30.018076+0000","last_unstale":"2024-09-21T04:02:30.022770+0000","last_undegraded":"2024-09-21T04:02:30.022770+0000","last_fullsized":"2024-09-21T04:02:30.022770+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:31.650092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.862438+0000","last_change":"2024-09-21T04:02:02.755223+0000","last_active":"2024-09-21T04:02:29.862438+0000","last_peered":"2024-09-21T04:02:29.862438+0000","last_clean":"2024-09-21T04:02:29.862438+0000","last_became_active":"2024-09-21T04:02:02.754836+0000","last_became_peered":"2024-09-21T04:02:02.754836+0000","last_unstale":"2024-09-21T04:02:29.862438+0000","last_undegraded":"2024-09-21T04:02:29.862438+0000","last_fullsized":"2024-09-21T04:02:29.862438+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:57.912690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"73'4","reported_seq":81,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:03:04.891007+0000","last_change":"2024-09-21T04:02:00.851689+0000","last_active":"2024-09-21T04:03:04.891007+0000","last_peered":"2024-09-21T04:03:04.891007+0000","last_clean":"2024-09-21T04:03:04.891007+0000","last_became_active":"2024-09-21T04:02:00.851241+0000","last_became_peered":"2024-09-21T04:02:00.851241+0000","last_unstale":"2024-09-21T04:03:04.891007+0000","last_undegraded":"2024-09-21T04:03:04.891007+0000","last_fullsized":"2024-09-21T04:03:04.891007+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:14.697695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866788+0000","last_change":"2024-09-21T04:02:02.774750+0000","last_active":"2024-09-21T04:02:29.866788+0000","last_peered":"2024-09-21T04:02:29.866788+0000","last_clean":"2024-09-21T04:02:29.866788+0000","last_became_active":"2024-09-21T04:02:02.774310+0000","last_became_peered":"2024-09-21T04:02:02.774310+0000","last_unstale":"2024-09-21T04:02:29.866788+0000","last_undegraded":"2024-09-21T04:02:29.866788+0000","last_fullsized":"2024-09-21T04:02:29.866788+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:07.909035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"73'1","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865053+0000","last_change":"2024-09-21T04:02:00.741461+0000","last_active":"2024-09-21T04:02:29.865053+0000","last_peered":"2024-09-21T04:02:29.865053+0000","last_clean":"2024-09-21T04:02:29.865053+0000","last_became_active":"2024-09-21T04:02:00.741051+0000","last_became_peered":"2024-09-21T04:02:00.741051+0000","last_unstale":"2024-09-21T04:02:29.865053+0000","last_undegraded":"2024-09-21T04:02:29.865053+0000","last_fullsized":"2024-09-21T04:02:29.865053+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:05.944430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.861420+0000","last_change":"2024-09-21T04:02:02.761703+0000","last_active":"2024-09-21T04:02:29.861420+0000","last_peered":"2024-09-21T04:02:29.861420+0000","last_clean":"2024-09-21T04:02:29.861420+0000","last_became_active":"2024-09-21T04:02:02.761296+0000","last_became_peered":"2024-09-21T04:02:02.761296+0000","last_unstale":"2024-09-21T04:02:29.861420+0000","last_undegraded":"2024-09-21T04:02:29.861420+0000","last_fullsized":"2024-09-21T04:02:29.861420+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:57.337167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.863538+0000","last_change":"2024-09-21T04:02:00.853584+0000","last_active":"2024-09-21T04:02:29.863538+0000","last_peered":"2024-09-21T04:02:29.863538+0000","last_clean":"2024-09-21T04:02:29.863538+0000","last_became_active":"2024-09-21T04:02:00.853019+0000","last_became_peered":"2024-09-21T04:02:00.853019+0000","last_unstale":"2024-09-21T04:02:29.863538+0000","last_undegraded":"2024-09-21T04:02:29.863538+0000","last_fullsized":"2024-09-21T04:02:29.863538+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:56.186791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"73'1","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.861430+0000","last_change":"2024-09-21T04:02:02.758175+0000","last_active":"2024-09-21T04:02:29.861430+0000","last_peered":"2024-09-21T04:02:29.861430+0000","last_clean":"2024-09-21T04:02:29.861430+0000","last_became_active":"2024-09-21T04:02:02.757715+0000","last_became_peered":"2024-09-21T04:02:02.757715+0000","last_unstale":"2024-09-21T04:02:29.861430+0000","last_undegraded":"2024-09-21T04:02:29.861430+0000","last_fullsized":"2024-09-21T04:02:29.861430+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:40.172569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"73'1","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.863271+0000","last_change":"2024-09-21T04:02:00.853325+0000","last_active":"2024-09-21T04:02:29.863271+0000","last_peered":"2024-09-21T04:02:29.863271+0000","last_clean":"2024-09-21T04:02:29.863271+0000","last_became_active":"2024-09-21T04:02:00.852834+0000","last_became_peered":"2024-09-21T04:02:00.852834+0000","last_unstale":"2024-09-21T04:02:29.863271+0000","last_undegraded":"2024-09-21T04:02:29.863271+0000","last_fullsized":"2024-09-21T04:02:29.863271+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:16.090493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.862826+0000","last_change":"2024-09-21T04:02:02.752402+0000","last_active":"2024-09-21T04:02:29.862826+0000","last_peered":"2024-09-21T04:02:29.862826+0000","last_clean":"2024-09-21T04:02:29.862826+0000","last_became_active":"2024-09-21T04:02:02.751994+0000","last_became_peered":"2024-09-21T04:02:02.751994+0000","last_unstale":"2024-09-21T04:02:29.862826+0000","last_undegraded":"2024-09-21T04:02:29.862826+0000","last_fullsized":"2024-09-21T04:02:29.862826+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:19.525317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"73'11","reported_seq":34,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.863685+0000","last_change":"2024-09-21T04:02:00.739112+0000","last_active":"2024-09-21T04:02:29.863685+0000","last_peered":"2024-09-21T04:02:29.863685+0000","last_clean":"2024-09-21T04:02:29.863685+0000","last_became_active":"2024-09-21T04:02:00.738613+0000","last_became_peered":"2024-09-21T04:02:00.738613+0000","last_unstale":"2024-09-21T04:02:29.863685+0000","last_undegraded":"2024-09-21T04:02:29.863685+0000","last_fullsized":"2024-09-21T04:02:29.863685+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:34.334576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865906+0000","last_change":"2024-09-21T04:02:02.763345+0000","last_active":"2024-09-21T04:02:29.865906+0000","last_peered":"2024-09-21T04:02:29.865906+0000","last_clean":"2024-09-21T04:02:29.865906+0000","last_became_active":"2024-09-21T04:02:02.762987+0000","last_became_peered":"2024-09-21T04:02:02.762987+0000","last_unstale":"2024-09-21T04:02:29.865906+0000","last_undegraded":"2024-09-21T04:02:29.865906+0000","last_fullsized":"2024-09-21T04:02:29.865906+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:36.239267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"73'4","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.018164+0000","last_change":"2024-09-21T04:02:01.258208+0000","last_active":"2024-09-21T04:02:30.018164+0000","last_peered":"2024-09-21T04:02:30.018164+0000","last_clean":"2024-09-21T04:02:30.018164+0000","last_became_active":"2024-09-21T04:02:01.257780+0000","last_became_peered":"2024-09-21T04:02:01.257780+0000","last_unstale":"2024-09-21T04:02:30.018164+0000","last_undegraded":"2024-09-21T04:02:30.018164+0000","last_fullsized":"2024-09-21T04:02:30.018164+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:20.161792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866715+0000","last_change":"2024-09-21T04:02:02.761149+0000","last_active":"2024-09-21T04:02:29.866715+0000","last_peered":"2024-09-21T04:02:29.866715+0000","last_clean":"2024-09-21T04:02:29.866715+0000","last_became_active":"2024-09-21T04:02:02.760771+0000","last_became_peered":"2024-09-21T04:02:02.760771+0000","last_unstale":"2024-09-21T04:02:29.866715+0000","last_undegraded":"2024-09-21T04:02:29.866715+0000","last_fullsized":"2024-09-21T04:02:29.866715+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:20.645377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"73'2","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.016281+0000","last_change":"2024-09-21T04:02:00.853580+0000","last_active":"2024-09-21T04:02:30.016281+0000","last_peered":"2024-09-21T04:02:30.016281+0000","last_clean":"2024-09-21T04:02:30.016281+0000","last_became_active":"2024-09-21T04:02:00.853091+0000","last_became_peered":"2024-09-21T04:02:00.853091+0000","last_unstale":"2024-09-21T04:02:30.016281+0000","last_undegraded":"2024-09-21T04:02:30.016281+0000","last_fullsized":"2024-09-21T04:02:30.016281+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:32.893695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.862695+0000","last_change":"2024-09-21T04:02:02.769686+0000","last_active":"2024-09-21T04:02:29.862695+0000","last_peered":"2024-09-21T04:02:29.862695+0000","last_clean":"2024-09-21T04:02:29.862695+0000","last_became_active":"2024-09-21T04:02:02.769293+0000","last_became_peered":"2024-09-21T04:02:02.769293+0000","last_unstale":"2024-09-21T04:02:29.862695+0000","last_undegraded":"2024-09-21T04:02:29.862695+0000","last_fullsized":"2024-09-21T04:02:29.862695+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:10.984624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"73'3","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865758+0000","last_change":"2024-09-21T04:02:00.741556+0000","last_active":"2024-09-21T04:02:29.865758+0000","last_peered":"2024-09-21T04:02:29.865758+0000","last_clean":"2024-09-21T04:02:29.865758+0000","last_became_active":"2024-09-21T04:02:00.741188+0000","last_became_peered":"2024-09-21T04:02:00.741188+0000","last_unstale":"2024-09-21T04:02:29.865758+0000","last_undegraded":"2024-09-21T04:02:29.865758+0000","last_fullsized":"2024-09-21T04:02:29.865758+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:23.566814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.867171+0000","last_change":"2024-09-21T04:02:02.769163+0000","last_active":"2024-09-21T04:02:29.867171+0000","last_peered":"2024-09-21T04:02:29.867171+0000","last_clean":"2024-09-21T04:02:29.867171+0000","last_became_active":"2024-09-21T04:02:02.768804+0000","last_became_peered":"2024-09-21T04:02:02.768804+0000","last_unstale":"2024-09-21T04:02:29.867171+0000","last_undegraded":"2024-09-21T04:02:29.867171+0000","last_fullsized":"2024-09-21T04:02:29.867171+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:12.576553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"73'1","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.860913+0000","last_change":"2024-09-21T04:02:00.743091+0000","last_active":"2024-09-21T04:02:29.860913+0000","last_peered":"2024-09-21T04:02:29.860913+0000","last_clean":"2024-09-21T04:02:29.860913+0000","last_became_active":"2024-09-21T04:02:00.742665+0000","last_became_peered":"2024-09-21T04:02:00.742665+0000","last_unstale":"2024-09-21T04:02:29.860913+0000","last_undegraded":"2024-09-21T04:02:29.860913+0000","last_fullsized":"2024-09-21T04:02:29.860913+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:09.698421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865858+0000","last_change":"2024-09-21T04:02:02.761985+0000","last_active":"2024-09-21T04:02:29.865858+0000","last_peered":"2024-09-21T04:02:29.865858+0000","last_clean":"2024-09-21T04:02:29.865858+0000","last_became_active":"2024-09-21T04:02:02.761624+0000","last_became_peered":"2024-09-21T04:02:02.761624+0000","last_unstale":"2024-09-21T04:02:29.865858+0000","last_undegraded":"2024-09-21T04:02:29.865858+0000","last_fullsized":"2024-09-21T04:02:29.865858+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:12.912221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"73'4","reported_seq":41,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.024364+0000","last_change":"2024-09-21T04:02:30.024288+0000","last_active":"2024-09-21T04:02:30.024364+0000","last_peered":"2024-09-21T04:02:30.024364+0000","last_clean":"2024-09-21T04:02:30.024364+0000","last_became_active":"2024-09-21T04:02:30.018929+0000","last_became_peered":"2024-09-21T04:02:30.018929+0000","last_unstale":"2024-09-21T04:02:30.024364+0000","last_undegraded":"2024-09-21T04:02:30.024364+0000","last_fullsized":"2024-09-21T04:02:30.024364+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:19:21.400082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865207+0000","last_change":"2024-09-21T04:02:02.761357+0000","last_active":"2024-09-21T04:02:29.865207+0000","last_peered":"2024-09-21T04:02:29.865207+0000","last_clean":"2024-09-21T04:02:29.865207+0000","last_became_active":"2024-09-21T04:02:02.760984+0000","last_became_peered":"2024-09-21T04:02:02.760984+0000","last_unstale":"2024-09-21T04:02:29.865207+0000","last_undegraded":"2024-09-21T04:02:29.865207+0000","last_fullsized":"2024-09-21T04:02:29.865207+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:55.388768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.019352+0000","last_change":"2024-09-21T04:02:30.019234+0000","last_active":"2024-09-21T04:02:30.019352+0000","last_peered":"2024-09-21T04:02:30.019352+0000","last_clean":"2024-09-21T04:02:30.019352+0000","last_became_active":"2024-09-21T04:02:30.018778+0000","last_became_peered":"2024-09-21T04:02:30.018778+0000","last_unstale":"2024-09-21T04:02:30.019352+0000","last_undegraded":"2024-09-21T04:02:30.019352+0000","last_fullsized":"2024-09-21T04:02:30.019352+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:31.227144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"73'2","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.863023+0000","last_change":"2024-09-21T04:02:00.742230+0000","last_active":"2024-09-21T04:02:29.863023+0000","last_peered":"2024-09-21T04:02:29.863023+0000","last_clean":"2024-09-21T04:02:29.863023+0000","last_became_active":"2024-09-21T04:02:00.741838+0000","last_became_peered":"2024-09-21T04:02:00.741838+0000","last_unstale":"2024-09-21T04:02:29.863023+0000","last_undegraded":"2024-09-21T04:02:29.863023+0000","last_fullsized":"2024-09-21T04:02:29.863023+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:18.909859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.016503+0000","last_change":"2024-09-21T04:02:02.762336+0000","last_active":"2024-09-21T04:02:30.016503+0000","last_peered":"2024-09-21T04:02:30.016503+0000","last_clean":"2024-09-21T04:02:30.016503+0000","last_became_active":"2024-09-21T04:02:02.761753+0000","last_became_peered":"2024-09-21T04:02:02.761753+0000","last_unstale":"2024-09-21T04:02:30.016503+0000","last_undegraded":"2024-09-21T04:02:30.016503+0000","last_fullsized":"2024-09-21T04:02:30.016503+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:39.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"73'4","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.863125+0000","last_change":"2024-09-21T04:02:00.743495+0000","last_active":"2024-09-21T04:02:29.863125+0000","last_peered":"2024-09-21T04:02:29.863125+0000","last_clean":"2024-09-21T04:02:29.863125+0000","last_became_active":"2024-09-21T04:02:00.743135+0000","last_became_peered":"2024-09-21T04:02:00.743135+0000","last_unstale":"2024-09-21T04:02:29.863125+0000","last_undegraded":"2024-09-21T04:02:29.863125+0000","last_fullsized":"2024-09-21T04:02:29.863125+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:16.481148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866766+0000","last_change":"2024-09-21T04:02:02.766682+0000","last_active":"2024-09-21T04:02:29.866766+0000","last_peered":"2024-09-21T04:02:29.866766+0000","last_clean":"2024-09-21T04:02:29.866766+0000","last_became_active":"2024-09-21T04:02:02.766299+0000","last_became_peered":"2024-09-21T04:02:02.766299+0000","last_unstale":"2024-09-21T04:02:29.866766+0000","last_undegraded":"2024-09-21T04:02:29.866766+0000","last_fullsized":"2024-09-21T04:02:29.866766+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:25.635908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"73'2","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866104+0000","last_change":"2024-09-21T04:02:00.852281+0000","last_active":"2024-09-21T04:02:29.866104+0000","last_peered":"2024-09-21T04:02:29.866104+0000","last_clean":"2024-09-21T04:02:29.866104+0000","last_became_active":"2024-09-21T04:02:00.851900+0000","last_became_peered":"2024-09-21T04:02:00.851900+0000","last_unstale":"2024-09-21T04:02:29.866104+0000","last_undegraded":"2024-09-21T04:02:29.866104+0000","last_fullsized":"2024-09-21T04:02:29.866104+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:37.115541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865896+0000","last_change":"2024-09-21T04:02:02.770467+0000","last_active":"2024-09-21T04:02:29.865896+0000","last_peered":"2024-09-21T04:02:29.865896+0000","last_clean":"2024-09-21T04:02:29.865896+0000","last_became_active":"2024-09-21T04:02:02.770099+0000","last_became_peered":"2024-09-21T04:02:02.770099+0000","last_unstale":"2024-09-21T04:02:29.865896+0000","last_undegraded":"2024-09-21T04:02:29.865896+0000","last_fullsized":"2024-09-21T04:02:29.865896+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:38.806298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"73'9","reported_seq":50,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.878120+0000","last_change":"2024-09-21T04:02:29.878053+0000","last_active":"2024-09-21T04:02:29.878120+0000","last_peered":"2024-09-21T04:02:29.878120+0000","last_clean":"2024-09-21T04:02:29.878120+0000","last_became_active":"2024-09-21T04:02:29.869007+0000","last_became_peered":"2024-09-21T04:02:29.869007+0000","last_unstale":"2024-09-21T04:02:29.878120+0000","last_undegraded":"2024-09-21T04:02:29.878120+0000","last_fullsized":"2024-09-21T04:02:29.878120+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:50.509849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69500,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":353,"num_read_kb":250,"num_write":162,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":208500,"data_compressed":5676,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282548,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808859,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31988,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713164,"statfs":{"total":95995035648,"available":95962279936,"internally_reserved":0,"allocated":819200,"data_stored":406805,"data_compressed":5778,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":62,"seq":266287972383,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":1063594,"data_compressed":43596,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135909,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":831488,"data_stored":406459,"data_compressed":5778,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332138,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":831488,"data_stored":406379,"data_compressed":5778,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495662,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":876544,"data_stored":997336,"data_compressed":41704,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659188,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31988,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713164,"statfs":{"total":95995035648,"available":95962279936,"internally_reserved":0,"allocated":819200,"data_stored":405848,"data_compressed":5778,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855417,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":892928,"data_stored":997602,"data_compressed":41704,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018942,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":823296,"data_stored":406285,"data_compressed":5778,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":835584,"data_stored":406159,"data_compressed":5778,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378696,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":905216,"data_stored":472881,"data_compressed":7670,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542221,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":471531,"data_compressed":7670,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705746,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":839680,"data_stored":407119,"data_compressed":5778,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34306,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34306,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34306,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":65890,"data_compressed":1892,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67240,"data_compressed":1892,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":67468,"data_compressed":1892,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T04:03:10.165 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T04:03:10.424 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:10.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:10 smithi119 ceph-mon[34811]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:03:10.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:10 smithi064 ceph-mon[31180]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:03:10.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:10 smithi131 ceph-mon[35193]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:03:10.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.909+0000 7f5c4dc2a640 1 -- 172.21.15.64:0/3912301962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c481083a0 msgr2=0x7f5c481087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:10.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.909+0000 7f5c4dc2a640 1 --2- 172.21.15.64:0/3912301962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c481083a0 0x7f5c481087b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009900 tx=0x7f5c3002a410 comp rx=0 tx=0).stop 2024-09-21T04:03:10.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.909+0000 7f5c4dc2a640 1 -- 172.21.15.64:0/3912301962 shutdown_connections 2024-09-21T04:03:10.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.909+0000 7f5c4dc2a640 1 --2- 172.21.15.64:0/3912301962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c48108cf0 0x7f5c4810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:10.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.909+0000 7f5c4dc2a640 1 --2- 172.21.15.64:0/3912301962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c481083a0 0x7f5c481087b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:10.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.909+0000 7f5c4dc2a640 1 --2- 172.21.15.64:0/3912301962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c48102a60 0x7f5c48102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:10.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.909+0000 7f5c4dc2a640 1 -- 172.21.15.64:0/3912301962 >> 172.21.15.64:0/3912301962 conn(0x7f5c48079730 msgr2=0x7f5c48079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:10.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.909+0000 7f5c4dc2a640 1 -- 172.21.15.64:0/3912301962 shutdown_connections 2024-09-21T04:03:10.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.909+0000 7f5c4dc2a640 1 -- 172.21.15.64:0/3912301962 wait complete. 2024-09-21T04:03:10.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.910+0000 7f5c4dc2a640 1 Processor -- start 2024-09-21T04:03:10.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.910+0000 7f5c4dc2a640 1 -- start start 2024-09-21T04:03:10.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.910+0000 7f5c4dc2a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c48102a60 0x7f5c4819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:10.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.910+0000 7f5c4dc2a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c481083a0 0x7f5c4819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:10.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.910+0000 7f5c4dc2a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c48108cf0 0x7f5c4819fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:10.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.910+0000 7f5c4dc2a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5c48113a80 con 0x7f5c48108cf0 2024-09-21T04:03:10.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.910+0000 7f5c4dc2a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5c48113900 con 0x7f5c481083a0 2024-09-21T04:03:10.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.910+0000 7f5c4dc2a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5c48113c00 con 0x7f5c48102a60 2024-09-21T04:03:10.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.910+0000 7f5c47fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c48108cf0 0x7f5c4819fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:10.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.910+0000 7f5c47fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c48108cf0 0x7f5c4819fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56950/0 (socket says 172.21.15.64:56950) 2024-09-21T04:03:10.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.910+0000 7f5c47fff640 1 -- 172.21.15.64:0/546437267 learned_addr learned my addr 172.21.15.64:0/546437267 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:10.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.911+0000 7f5c46ffd640 1 --2- 172.21.15.64:0/546437267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c481083a0 0x7f5c4819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:10.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.911+0000 7f5c477fe640 1 --2- 172.21.15.64:0/546437267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c48102a60 0x7f5c4819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:10.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.911+0000 7f5c47fff640 1 -- 172.21.15.64:0/546437267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c48102a60 msgr2=0x7f5c4819a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:10.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.911+0000 7f5c47fff640 1 --2- 172.21.15.64:0/546437267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c48102a60 0x7f5c4819a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:10.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.911+0000 7f5c47fff640 1 -- 172.21.15.64:0/546437267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c481083a0 msgr2=0x7f5c4819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:10.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.911+0000 7f5c47fff640 1 --2- 172.21.15.64:0/546437267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c481083a0 0x7f5c4819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:10.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.911+0000 7f5c47fff640 1 -- 172.21.15.64:0/546437267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c481a0480 con 0x7f5c48108cf0 2024-09-21T04:03:10.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.911+0000 7f5c46ffd640 1 --2- 172.21.15.64:0/546437267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c481083a0 0x7f5c4819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:03:10.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.911+0000 7f5c477fe640 1 --2- 172.21.15.64:0/546437267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c48102a60 0x7f5c4819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:03:10.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.911+0000 7f5c47fff640 1 --2- 172.21.15.64:0/546437267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c48108cf0 0x7f5c4819fd50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000d750 tx=0x7f5c4000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:10.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.911+0000 7f5c44ff9640 1 -- 172.21.15.64:0/546437267 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c40017070 con 0x7f5c48108cf0 2024-09-21T04:03:10.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.911+0000 7f5c4dc2a640 1 -- 172.21.15.64:0/546437267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c481a0770 con 0x7f5c48108cf0 2024-09-21T04:03:10.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.911+0000 7f5c44ff9640 1 -- 172.21.15.64:0/546437267 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c400054d0 con 0x7f5c48108cf0 2024-09-21T04:03:10.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.911+0000 7f5c44ff9640 1 -- 172.21.15.64:0/546437267 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c40004c60 con 0x7f5c48108cf0 2024-09-21T04:03:10.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.911+0000 7f5c4dc2a640 1 -- 172.21.15.64:0/546437267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c481a6ed0 con 0x7f5c48108cf0 2024-09-21T04:03:10.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.912+0000 7f5c4dc2a640 1 -- 172.21.15.64:0/546437267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c48102e70 con 0x7f5c48108cf0 2024-09-21T04:03:10.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.914+0000 7f5c44ff9640 1 -- 172.21.15.64:0/546437267 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f5c40009bd0 con 0x7f5c48108cf0 2024-09-21T04:03:10.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.915+0000 7f5c44ff9640 1 --2- 172.21.15.64:0/546437267 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5c340b65f0 0x7f5c340b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:10.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.915+0000 7f5c477fe640 1 --2- 172.21.15.64:0/546437267 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5c340b65f0 0x7f5c340b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:10.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.915+0000 7f5c44ff9640 1 -- 172.21.15.64:0/546437267 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f5c400e7470 con 0x7f5c48108cf0 2024-09-21T04:03:10.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.915+0000 7f5c477fe640 1 --2- 172.21.15.64:0/546437267 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5c340b65f0 0x7f5c340b8b00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5c38005fd0 tx=0x7f5c380093e0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:10.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:10.918+0000 7f5c44ff9640 1 -- 172.21.15.64:0/546437267 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5c400ada40 con 0x7f5c48108cf0 2024-09-21T04:03:11.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:11.105+0000 7f5c4dc2a640 1 -- 172.21.15.64:0/546437267 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5c48063800 con 0x7f5c340b65f0 2024-09-21T04:03:11.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:11.110+0000 7f5c44ff9640 1 -- 172.21.15.64:0/546437267 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258990 (secure 0 0 0) 0x7f5c48063800 con 0x7f5c340b65f0 2024-09-21T04:03:11.114 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:03:11.115 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-09-21T04:03:11.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:11.113+0000 7f5c4dc2a640 1 -- 172.21.15.64:0/546437267 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5c340b65f0 msgr2=0x7f5c340b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:11.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:11.113+0000 7f5c4dc2a640 1 --2- 172.21.15.64:0/546437267 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5c340b65f0 0x7f5c340b8b00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5c38005fd0 tx=0x7f5c380093e0 comp rx=0 tx=0).stop 2024-09-21T04:03:11.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:11.113+0000 7f5c4dc2a640 1 -- 172.21.15.64:0/546437267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c48108cf0 msgr2=0x7f5c4819fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:11.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:11.113+0000 7f5c4dc2a640 1 --2- 172.21.15.64:0/546437267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c48108cf0 0x7f5c4819fd50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000d750 tx=0x7f5c4000dc10 comp rx=0 tx=0).stop 2024-09-21T04:03:11.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:11.113+0000 7f5c4dc2a640 1 -- 172.21.15.64:0/546437267 shutdown_connections 2024-09-21T04:03:11.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:11.113+0000 7f5c4dc2a640 1 --2- 172.21.15.64:0/546437267 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5c340b65f0 0x7f5c340b8b00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:11.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:11.113+0000 7f5c4dc2a640 1 --2- 172.21.15.64:0/546437267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c48108cf0 0x7f5c4819fd50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:11.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:11.113+0000 7f5c4dc2a640 1 --2- 172.21.15.64:0/546437267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c481083a0 0x7f5c4819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:11.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:11.113+0000 7f5c4dc2a640 1 --2- 172.21.15.64:0/546437267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c48102a60 0x7f5c4819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:11.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:11.113+0000 7f5c4dc2a640 1 -- 172.21.15.64:0/546437267 >> 172.21.15.64:0/546437267 conn(0x7f5c48079730 msgr2=0x7f5c481052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:11.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:11.113+0000 7f5c4dc2a640 1 -- 172.21.15.64:0/546437267 shutdown_connections 2024-09-21T04:03:11.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:11.113+0000 7f5c4dc2a640 1 -- 172.21.15.64:0/546437267 wait complete. 2024-09-21T04:03:11.759 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:11 smithi064 ceph-mon[31180]: pgmap v294: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:03:11.759 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:11 smithi064 ceph-mon[31180]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:03:11.794 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-21T04:03:10.492805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660102,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":434,"num_read_kb":319,"num_write":287,"num_write_kb":2268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":2952422,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359384,"kb_used_data":10080,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124582440,"statfs":{"total":1151940427776,"available":1151572418560,"internally_reserved":0,"allocated":10321920,"data_stored":6847998,"data_compressed":182790,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003366"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.868680+0000","last_change":"2024-09-21T04:02:02.769848+0000","last_active":"2024-09-21T04:02:29.868680+0000","last_peered":"2024-09-21T04:02:29.868680+0000","last_clean":"2024-09-21T04:02:29.868680+0000","last_became_active":"2024-09-21T04:02:02.769508+0000","last_became_peered":"2024-09-21T04:02:02.769508+0000","last_unstale":"2024-09-21T04:02:29.868680+0000","last_undegraded":"2024-09-21T04:02:29.868680+0000","last_fullsized":"2024-09-21T04:02:29.868680+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:00.242233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866619+0000","last_change":"2024-09-21T04:02:02.769200+0000","last_active":"2024-09-21T04:02:29.866619+0000","last_peered":"2024-09-21T04:02:29.866619+0000","last_clean":"2024-09-21T04:02:29.866619+0000","last_became_active":"2024-09-21T04:02:02.768918+0000","last_became_peered":"2024-09-21T04:02:02.768918+0000","last_unstale":"2024-09-21T04:02:29.866619+0000","last_undegraded":"2024-09-21T04:02:29.866619+0000","last_fullsized":"2024-09-21T04:02:29.866619+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:26.842079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866563+0000","last_change":"2024-09-21T04:02:02.763511+0000","last_active":"2024-09-21T04:02:29.866563+0000","last_peered":"2024-09-21T04:02:29.866563+0000","last_clean":"2024-09-21T04:02:29.866563+0000","last_became_active":"2024-09-21T04:02:02.763161+0000","last_became_peered":"2024-09-21T04:02:02.763161+0000","last_unstale":"2024-09-21T04:02:29.866563+0000","last_undegraded":"2024-09-21T04:02:29.866563+0000","last_fullsized":"2024-09-21T04:02:29.866563+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:59.834005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.017853+0000","last_change":"2024-09-21T04:02:02.765452+0000","last_active":"2024-09-21T04:02:30.017853+0000","last_peered":"2024-09-21T04:02:30.017853+0000","last_clean":"2024-09-21T04:02:30.017853+0000","last_became_active":"2024-09-21T04:02:02.765065+0000","last_became_peered":"2024-09-21T04:02:02.765065+0000","last_unstale":"2024-09-21T04:02:30.017853+0000","last_undegraded":"2024-09-21T04:02:30.017853+0000","last_fullsized":"2024-09-21T04:02:30.017853+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:33.871366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.863125+0000","last_change":"2024-09-21T04:02:02.767457+0000","last_active":"2024-09-21T04:02:29.863125+0000","last_peered":"2024-09-21T04:02:29.863125+0000","last_clean":"2024-09-21T04:02:29.863125+0000","last_became_active":"2024-09-21T04:02:02.767064+0000","last_became_peered":"2024-09-21T04:02:02.767064+0000","last_unstale":"2024-09-21T04:02:29.863125+0000","last_undegraded":"2024-09-21T04:02:29.863125+0000","last_fullsized":"2024-09-21T04:02:29.863125+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:46.316029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.861263+0000","last_change":"2024-09-21T04:02:02.768795+0000","last_active":"2024-09-21T04:02:29.861263+0000","last_peered":"2024-09-21T04:02:29.861263+0000","last_clean":"2024-09-21T04:02:29.861263+0000","last_became_active":"2024-09-21T04:02:02.768434+0000","last_became_peered":"2024-09-21T04:02:02.768434+0000","last_unstale":"2024-09-21T04:02:29.861263+0000","last_undegraded":"2024-09-21T04:02:29.861263+0000","last_fullsized":"2024-09-21T04:02:29.861263+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:58.991533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.864547+0000","last_change":"2024-09-21T04:02:02.774998+0000","last_active":"2024-09-21T04:02:29.864547+0000","last_peered":"2024-09-21T04:02:29.864547+0000","last_clean":"2024-09-21T04:02:29.864547+0000","last_became_active":"2024-09-21T04:02:02.774344+0000","last_became_peered":"2024-09-21T04:02:02.774344+0000","last_unstale":"2024-09-21T04:02:29.864547+0000","last_undegraded":"2024-09-21T04:02:29.864547+0000","last_fullsized":"2024-09-21T04:02:29.864547+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:26.705126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865509+0000","last_change":"2024-09-21T04:02:02.772498+0000","last_active":"2024-09-21T04:02:29.865509+0000","last_peered":"2024-09-21T04:02:29.865509+0000","last_clean":"2024-09-21T04:02:29.865509+0000","last_became_active":"2024-09-21T04:02:02.772049+0000","last_became_peered":"2024-09-21T04:02:02.772049+0000","last_unstale":"2024-09-21T04:02:29.865509+0000","last_undegraded":"2024-09-21T04:02:29.865509+0000","last_fullsized":"2024-09-21T04:02:29.865509+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:31:19.088424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.016701+0000","last_change":"2024-09-21T04:02:02.772336+0000","last_active":"2024-09-21T04:02:30.016701+0000","last_peered":"2024-09-21T04:02:30.016701+0000","last_clean":"2024-09-21T04:02:30.016701+0000","last_became_active":"2024-09-21T04:02:02.771964+0000","last_became_peered":"2024-09-21T04:02:02.771964+0000","last_unstale":"2024-09-21T04:02:30.016701+0000","last_undegraded":"2024-09-21T04:02:30.016701+0000","last_fullsized":"2024-09-21T04:02:30.016701+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:46.659690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.861686+0000","last_change":"2024-09-21T04:02:02.758992+0000","last_active":"2024-09-21T04:02:29.861686+0000","last_peered":"2024-09-21T04:02:29.861686+0000","last_clean":"2024-09-21T04:02:29.861686+0000","last_became_active":"2024-09-21T04:02:02.758575+0000","last_became_peered":"2024-09-21T04:02:02.758575+0000","last_unstale":"2024-09-21T04:02:29.861686+0000","last_undegraded":"2024-09-21T04:02:29.861686+0000","last_fullsized":"2024-09-21T04:02:29.861686+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:32:09.543140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865235+0000","last_change":"2024-09-21T04:02:02.767304+0000","last_active":"2024-09-21T04:02:29.865235+0000","last_peered":"2024-09-21T04:02:29.865235+0000","last_clean":"2024-09-21T04:02:29.865235+0000","last_became_active":"2024-09-21T04:02:02.762804+0000","last_became_peered":"2024-09-21T04:02:02.762804+0000","last_unstale":"2024-09-21T04:02:29.865235+0000","last_undegraded":"2024-09-21T04:02:29.865235+0000","last_fullsized":"2024-09-21T04:02:29.865235+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:32.224071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.017995+0000","last_change":"2024-09-21T04:02:02.762932+0000","last_active":"2024-09-21T04:02:30.017995+0000","last_peered":"2024-09-21T04:02:30.017995+0000","last_clean":"2024-09-21T04:02:30.017995+0000","last_became_active":"2024-09-21T04:02:02.762538+0000","last_became_peered":"2024-09-21T04:02:02.762538+0000","last_unstale":"2024-09-21T04:02:30.017995+0000","last_undegraded":"2024-09-21T04:02:30.017995+0000","last_fullsized":"2024-09-21T04:02:30.017995+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:20.724453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.863000+0000","last_change":"2024-09-21T04:02:02.770313+0000","last_active":"2024-09-21T04:02:29.863000+0000","last_peered":"2024-09-21T04:02:29.863000+0000","last_clean":"2024-09-21T04:02:29.863000+0000","last_became_active":"2024-09-21T04:02:02.769910+0000","last_became_peered":"2024-09-21T04:02:02.769910+0000","last_unstale":"2024-09-21T04:02:29.863000+0000","last_undegraded":"2024-09-21T04:02:29.863000+0000","last_fullsized":"2024-09-21T04:02:29.863000+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:16.228371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866455+0000","last_change":"2024-09-21T04:02:02.769258+0000","last_active":"2024-09-21T04:02:29.866455+0000","last_peered":"2024-09-21T04:02:29.866455+0000","last_clean":"2024-09-21T04:02:29.866455+0000","last_became_active":"2024-09-21T04:02:02.768914+0000","last_became_peered":"2024-09-21T04:02:02.768914+0000","last_unstale":"2024-09-21T04:02:29.866455+0000","last_undegraded":"2024-09-21T04:02:29.866455+0000","last_fullsized":"2024-09-21T04:02:29.866455+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:37.426810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866413+0000","last_change":"2024-09-21T04:02:02.755940+0000","last_active":"2024-09-21T04:02:29.866413+0000","last_peered":"2024-09-21T04:02:29.866413+0000","last_clean":"2024-09-21T04:02:29.866413+0000","last_became_active":"2024-09-21T04:02:02.755309+0000","last_became_peered":"2024-09-21T04:02:02.755309+0000","last_unstale":"2024-09-21T04:02:29.866413+0000","last_undegraded":"2024-09-21T04:02:29.866413+0000","last_fullsized":"2024-09-21T04:02:29.866413+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:49.834030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.017085+0000","last_change":"2024-09-21T04:02:02.761068+0000","last_active":"2024-09-21T04:02:30.017085+0000","last_peered":"2024-09-21T04:02:30.017085+0000","last_clean":"2024-09-21T04:02:30.017085+0000","last_became_active":"2024-09-21T04:02:02.760683+0000","last_became_peered":"2024-09-21T04:02:02.760683+0000","last_unstale":"2024-09-21T04:02:30.017085+0000","last_undegraded":"2024-09-21T04:02:30.017085+0000","last_fullsized":"2024-09-21T04:02:30.017085+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:36.627539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.868703+0000","last_change":"2024-09-21T04:02:02.772234+0000","last_active":"2024-09-21T04:02:29.868703+0000","last_peered":"2024-09-21T04:02:29.868703+0000","last_clean":"2024-09-21T04:02:29.868703+0000","last_became_active":"2024-09-21T04:02:02.771825+0000","last_became_peered":"2024-09-21T04:02:02.771825+0000","last_unstale":"2024-09-21T04:02:29.868703+0000","last_undegraded":"2024-09-21T04:02:29.868703+0000","last_fullsized":"2024-09-21T04:02:29.868703+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:33.449745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.863704+0000","last_change":"2024-09-21T04:02:02.758643+0000","last_active":"2024-09-21T04:02:29.863704+0000","last_peered":"2024-09-21T04:02:29.863704+0000","last_clean":"2024-09-21T04:02:29.863704+0000","last_became_active":"2024-09-21T04:02:02.758261+0000","last_became_peered":"2024-09-21T04:02:02.758261+0000","last_unstale":"2024-09-21T04:02:29.863704+0000","last_undegraded":"2024-09-21T04:02:29.863704+0000","last_fullsized":"2024-09-21T04:02:29.863704+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:04.165399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.861196+0000","last_change":"2024-09-21T04:02:02.759405+0000","last_active":"2024-09-21T04:02:29.861196+0000","last_peered":"2024-09-21T04:02:29.861196+0000","last_clean":"2024-09-21T04:02:29.861196+0000","last_became_active":"2024-09-21T04:02:02.758962+0000","last_became_peered":"2024-09-21T04:02:02.758962+0000","last_unstale":"2024-09-21T04:02:29.861196+0000","last_undegraded":"2024-09-21T04:02:29.861196+0000","last_fullsized":"2024-09-21T04:02:29.861196+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:37.203308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.867267+0000","last_change":"2024-09-21T04:02:02.767518+0000","last_active":"2024-09-21T04:02:29.867267+0000","last_peered":"2024-09-21T04:02:29.867267+0000","last_clean":"2024-09-21T04:02:29.867267+0000","last_became_active":"2024-09-21T04:02:02.767147+0000","last_became_peered":"2024-09-21T04:02:02.767147+0000","last_unstale":"2024-09-21T04:02:29.867267+0000","last_undegraded":"2024-09-21T04:02:29.867267+0000","last_fullsized":"2024-09-21T04:02:29.867267+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:34:13.615218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.863107+0000","last_change":"2024-09-21T04:02:02.756407+0000","last_active":"2024-09-21T04:02:29.863107+0000","last_peered":"2024-09-21T04:02:29.863107+0000","last_clean":"2024-09-21T04:02:29.863107+0000","last_became_active":"2024-09-21T04:02:02.756007+0000","last_became_peered":"2024-09-21T04:02:02.756007+0000","last_unstale":"2024-09-21T04:02:29.863107+0000","last_undegraded":"2024-09-21T04:02:29.863107+0000","last_fullsized":"2024-09-21T04:02:29.863107+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:18.798814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.017244+0000","last_change":"2024-09-21T04:02:02.766290+0000","last_active":"2024-09-21T04:02:30.017244+0000","last_peered":"2024-09-21T04:02:30.017244+0000","last_clean":"2024-09-21T04:02:30.017244+0000","last_became_active":"2024-09-21T04:02:02.765660+0000","last_became_peered":"2024-09-21T04:02:02.765660+0000","last_unstale":"2024-09-21T04:02:30.017244+0000","last_undegraded":"2024-09-21T04:02:30.017244+0000","last_fullsized":"2024-09-21T04:02:30.017244+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:23.281476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865621+0000","last_change":"2024-09-21T04:02:02.767943+0000","last_active":"2024-09-21T04:02:29.865621+0000","last_peered":"2024-09-21T04:02:29.865621+0000","last_clean":"2024-09-21T04:02:29.865621+0000","last_became_active":"2024-09-21T04:02:02.767566+0000","last_became_peered":"2024-09-21T04:02:02.767566+0000","last_unstale":"2024-09-21T04:02:29.865621+0000","last_undegraded":"2024-09-21T04:02:29.865621+0000","last_fullsized":"2024-09-21T04:02:29.865621+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:44.221531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866695+0000","last_change":"2024-09-21T04:02:02.770114+0000","last_active":"2024-09-21T04:02:29.866695+0000","last_peered":"2024-09-21T04:02:29.866695+0000","last_clean":"2024-09-21T04:02:29.866695+0000","last_became_active":"2024-09-21T04:02:02.769732+0000","last_became_peered":"2024-09-21T04:02:02.769732+0000","last_unstale":"2024-09-21T04:02:29.866695+0000","last_undegraded":"2024-09-21T04:02:29.866695+0000","last_fullsized":"2024-09-21T04:02:29.866695+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:38.396288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.017726+0000","last_change":"2024-09-21T04:02:30.017325+0000","last_active":"2024-09-21T04:02:30.017726+0000","last_peered":"2024-09-21T04:02:30.017726+0000","last_clean":"2024-09-21T04:02:30.017726+0000","last_became_active":"2024-09-21T04:02:30.017014+0000","last_became_peered":"2024-09-21T04:02:30.017014+0000","last_unstale":"2024-09-21T04:02:30.017726+0000","last_undegraded":"2024-09-21T04:02:30.017726+0000","last_fullsized":"2024-09-21T04:02:30.017726+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:28.579436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.861493+0000","last_change":"2024-09-21T04:02:02.764666+0000","last_active":"2024-09-21T04:02:29.861493+0000","last_peered":"2024-09-21T04:02:29.861493+0000","last_clean":"2024-09-21T04:02:29.861493+0000","last_became_active":"2024-09-21T04:02:02.763931+0000","last_became_peered":"2024-09-21T04:02:02.763931+0000","last_unstale":"2024-09-21T04:02:29.861493+0000","last_undegraded":"2024-09-21T04:02:29.861493+0000","last_fullsized":"2024-09-21T04:02:29.861493+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:39.724770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.867617+0000","last_change":"2024-09-21T04:02:02.774276+0000","last_active":"2024-09-21T04:02:29.867617+0000","last_peered":"2024-09-21T04:02:29.867617+0000","last_clean":"2024-09-21T04:02:29.867617+0000","last_became_active":"2024-09-21T04:02:02.773852+0000","last_became_peered":"2024-09-21T04:02:02.773852+0000","last_unstale":"2024-09-21T04:02:29.867617+0000","last_undegraded":"2024-09-21T04:02:29.867617+0000","last_fullsized":"2024-09-21T04:02:29.867617+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:40.072264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.862794+0000","last_change":"2024-09-21T04:02:02.758520+0000","last_active":"2024-09-21T04:02:29.862794+0000","last_peered":"2024-09-21T04:02:29.862794+0000","last_clean":"2024-09-21T04:02:29.862794+0000","last_became_active":"2024-09-21T04:02:02.757656+0000","last_became_peered":"2024-09-21T04:02:02.757656+0000","last_unstale":"2024-09-21T04:02:29.862794+0000","last_undegraded":"2024-09-21T04:02:29.862794+0000","last_fullsized":"2024-09-21T04:02:29.862794+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:30.854336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866958+0000","last_change":"2024-09-21T04:02:02.756428+0000","last_active":"2024-09-21T04:02:29.866958+0000","last_peered":"2024-09-21T04:02:29.866958+0000","last_clean":"2024-09-21T04:02:29.866958+0000","last_became_active":"2024-09-21T04:02:02.755985+0000","last_became_peered":"2024-09-21T04:02:02.755985+0000","last_unstale":"2024-09-21T04:02:29.866958+0000","last_undegraded":"2024-09-21T04:02:29.866958+0000","last_fullsized":"2024-09-21T04:02:29.866958+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:36.040436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.860813+0000","last_change":"2024-09-21T04:02:02.762893+0000","last_active":"2024-09-21T04:02:29.860813+0000","last_peered":"2024-09-21T04:02:29.860813+0000","last_clean":"2024-09-21T04:02:29.860813+0000","last_became_active":"2024-09-21T04:02:02.762343+0000","last_became_peered":"2024-09-21T04:02:02.762343+0000","last_unstale":"2024-09-21T04:02:29.860813+0000","last_undegraded":"2024-09-21T04:02:29.860813+0000","last_fullsized":"2024-09-21T04:02:29.860813+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:45.074711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.860958+0000","last_change":"2024-09-21T04:02:02.763942+0000","last_active":"2024-09-21T04:02:29.860958+0000","last_peered":"2024-09-21T04:02:29.860958+0000","last_clean":"2024-09-21T04:02:29.860958+0000","last_became_active":"2024-09-21T04:02:02.763433+0000","last_became_peered":"2024-09-21T04:02:02.763433+0000","last_unstale":"2024-09-21T04:02:29.860958+0000","last_undegraded":"2024-09-21T04:02:29.860958+0000","last_fullsized":"2024-09-21T04:02:29.860958+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:36.139440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.867052+0000","last_change":"2024-09-21T04:02:02.773364+0000","last_active":"2024-09-21T04:02:29.867052+0000","last_peered":"2024-09-21T04:02:29.867052+0000","last_clean":"2024-09-21T04:02:29.867052+0000","last_became_active":"2024-09-21T04:02:02.773040+0000","last_became_peered":"2024-09-21T04:02:02.773040+0000","last_unstale":"2024-09-21T04:02:29.867052+0000","last_undegraded":"2024-09-21T04:02:29.867052+0000","last_fullsized":"2024-09-21T04:02:29.867052+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:48.504476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.862997+0000","last_change":"2024-09-21T04:02:02.774951+0000","last_active":"2024-09-21T04:02:29.862997+0000","last_peered":"2024-09-21T04:02:29.862997+0000","last_clean":"2024-09-21T04:02:29.862997+0000","last_became_active":"2024-09-21T04:02:02.774525+0000","last_became_peered":"2024-09-21T04:02:02.774525+0000","last_unstale":"2024-09-21T04:02:29.862997+0000","last_undegraded":"2024-09-21T04:02:29.862997+0000","last_fullsized":"2024-09-21T04:02:29.862997+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:24.447860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"73'5","reported_seq":84,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:03:04.891537+0000","last_change":"2024-09-21T04:02:00.743470+0000","last_active":"2024-09-21T04:03:04.891537+0000","last_peered":"2024-09-21T04:03:04.891537+0000","last_clean":"2024-09-21T04:03:04.891537+0000","last_became_active":"2024-09-21T04:02:00.743055+0000","last_became_peered":"2024-09-21T04:02:00.743055+0000","last_unstale":"2024-09-21T04:03:04.891537+0000","last_undegraded":"2024-09-21T04:03:04.891537+0000","last_fullsized":"2024-09-21T04:03:04.891537+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:56.101753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866720+0000","last_change":"2024-09-21T04:02:02.773399+0000","last_active":"2024-09-21T04:02:29.866720+0000","last_peered":"2024-09-21T04:02:29.866720+0000","last_clean":"2024-09-21T04:02:29.866720+0000","last_became_active":"2024-09-21T04:02:02.772996+0000","last_became_peered":"2024-09-21T04:02:02.772996+0000","last_unstale":"2024-09-21T04:02:29.866720+0000","last_undegraded":"2024-09-21T04:02:29.866720+0000","last_fullsized":"2024-09-21T04:02:29.866720+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:07.256226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"73'3","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.026998+0000","last_change":"2024-09-21T04:02:30.026868+0000","last_active":"2024-09-21T04:02:30.026998+0000","last_peered":"2024-09-21T04:02:30.026998+0000","last_clean":"2024-09-21T04:02:30.026998+0000","last_became_active":"2024-09-21T04:02:30.019469+0000","last_became_peered":"2024-09-21T04:02:30.019469+0000","last_unstale":"2024-09-21T04:02:30.026998+0000","last_undegraded":"2024-09-21T04:02:30.026998+0000","last_fullsized":"2024-09-21T04:02:30.026998+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:35.304358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"73'2","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.862695+0000","last_change":"2024-09-21T04:02:00.736630+0000","last_active":"2024-09-21T04:02:29.862695+0000","last_peered":"2024-09-21T04:02:29.862695+0000","last_clean":"2024-09-21T04:02:29.862695+0000","last_became_active":"2024-09-21T04:02:00.736211+0000","last_became_peered":"2024-09-21T04:02:00.736211+0000","last_unstale":"2024-09-21T04:02:29.862695+0000","last_undegraded":"2024-09-21T04:02:29.862695+0000","last_fullsized":"2024-09-21T04:02:29.862695+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:37.211617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865583+0000","last_change":"2024-09-21T04:02:02.763323+0000","last_active":"2024-09-21T04:02:29.865583+0000","last_peered":"2024-09-21T04:02:29.865583+0000","last_clean":"2024-09-21T04:02:29.865583+0000","last_became_active":"2024-09-21T04:02:02.762776+0000","last_became_peered":"2024-09-21T04:02:02.762776+0000","last_unstale":"2024-09-21T04:02:29.865583+0000","last_undegraded":"2024-09-21T04:02:29.865583+0000","last_fullsized":"2024-09-21T04:02:29.865583+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:39.891042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"73'1","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866894+0000","last_change":"2024-09-21T04:02:00.743377+0000","last_active":"2024-09-21T04:02:29.866894+0000","last_peered":"2024-09-21T04:02:29.866894+0000","last_clean":"2024-09-21T04:02:29.866894+0000","last_became_active":"2024-09-21T04:02:00.742973+0000","last_became_peered":"2024-09-21T04:02:00.742973+0000","last_unstale":"2024-09-21T04:02:29.866894+0000","last_undegraded":"2024-09-21T04:02:29.866894+0000","last_fullsized":"2024-09-21T04:02:29.866894+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:55:00.655532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.863390+0000","last_change":"2024-09-21T04:02:02.758721+0000","last_active":"2024-09-21T04:02:29.863390+0000","last_peered":"2024-09-21T04:02:29.863390+0000","last_clean":"2024-09-21T04:02:29.863390+0000","last_became_active":"2024-09-21T04:02:02.758352+0000","last_became_peered":"2024-09-21T04:02:02.758352+0000","last_unstale":"2024-09-21T04:02:29.863390+0000","last_undegraded":"2024-09-21T04:02:29.863390+0000","last_fullsized":"2024-09-21T04:02:29.863390+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:56:49.289510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"73'3","reported_seq":81,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:03:05.892467+0000","last_change":"2024-09-21T04:02:00.744591+0000","last_active":"2024-09-21T04:03:05.892467+0000","last_peered":"2024-09-21T04:03:05.892467+0000","last_clean":"2024-09-21T04:03:05.892467+0000","last_became_active":"2024-09-21T04:02:00.744139+0000","last_became_peered":"2024-09-21T04:02:00.744139+0000","last_unstale":"2024-09-21T04:03:05.892467+0000","last_undegraded":"2024-09-21T04:03:05.892467+0000","last_fullsized":"2024-09-21T04:03:05.892467+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:21.331517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.016835+0000","last_change":"2024-09-21T04:02:02.762779+0000","last_active":"2024-09-21T04:02:30.016835+0000","last_peered":"2024-09-21T04:02:30.016835+0000","last_clean":"2024-09-21T04:02:30.016835+0000","last_became_active":"2024-09-21T04:02:02.760761+0000","last_became_peered":"2024-09-21T04:02:02.760761+0000","last_unstale":"2024-09-21T04:02:30.016835+0000","last_undegraded":"2024-09-21T04:02:30.016835+0000","last_fullsized":"2024-09-21T04:02:30.016835+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:18.789052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"73'2","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865511+0000","last_change":"2024-09-21T04:02:00.743220+0000","last_active":"2024-09-21T04:02:29.865511+0000","last_peered":"2024-09-21T04:02:29.865511+0000","last_clean":"2024-09-21T04:02:29.865511+0000","last_became_active":"2024-09-21T04:02:00.742636+0000","last_became_peered":"2024-09-21T04:02:00.742636+0000","last_unstale":"2024-09-21T04:02:29.865511+0000","last_undegraded":"2024-09-21T04:02:29.865511+0000","last_fullsized":"2024-09-21T04:02:29.865511+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:01.382338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.869766+0000","last_change":"2024-09-21T04:02:29.869663+0000","last_active":"2024-09-21T04:02:29.869766+0000","last_peered":"2024-09-21T04:02:29.869766+0000","last_clean":"2024-09-21T04:02:29.869766+0000","last_became_active":"2024-09-21T04:02:29.869230+0000","last_became_peered":"2024-09-21T04:02:29.869230+0000","last_unstale":"2024-09-21T04:02:29.869766+0000","last_undegraded":"2024-09-21T04:02:29.869766+0000","last_fullsized":"2024-09-21T04:02:29.869766+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:19:59.987267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"73'4","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.867117+0000","last_change":"2024-09-21T04:02:00.853284+0000","last_active":"2024-09-21T04:02:29.867117+0000","last_peered":"2024-09-21T04:02:29.867117+0000","last_clean":"2024-09-21T04:02:29.867117+0000","last_became_active":"2024-09-21T04:02:00.852855+0000","last_became_peered":"2024-09-21T04:02:00.852855+0000","last_unstale":"2024-09-21T04:02:29.867117+0000","last_undegraded":"2024-09-21T04:02:29.867117+0000","last_fullsized":"2024-09-21T04:02:29.867117+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:36.994214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.016563+0000","last_change":"2024-09-21T04:02:02.763577+0000","last_active":"2024-09-21T04:02:30.016563+0000","last_peered":"2024-09-21T04:02:30.016563+0000","last_clean":"2024-09-21T04:02:30.016563+0000","last_became_active":"2024-09-21T04:02:02.762970+0000","last_became_peered":"2024-09-21T04:02:02.762970+0000","last_unstale":"2024-09-21T04:02:30.016563+0000","last_undegraded":"2024-09-21T04:02:30.016563+0000","last_fullsized":"2024-09-21T04:02:30.016563+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:49.956226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'29","reported_seq":171,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:03:06.893840+0000","last_change":"2024-09-21T04:02:00.744203+0000","last_active":"2024-09-21T04:03:06.893840+0000","last_peered":"2024-09-21T04:03:06.893840+0000","last_clean":"2024-09-21T04:03:06.893840+0000","last_became_active":"2024-09-21T04:02:00.743659+0000","last_became_peered":"2024-09-21T04:02:00.743659+0000","last_unstale":"2024-09-21T04:03:06.893840+0000","last_undegraded":"2024-09-21T04:03:06.893840+0000","last_fullsized":"2024-09-21T04:03:06.893840+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:32.108302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65890,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":119,"num_read_kb":123,"num_write":30,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.863164+0000","last_change":"2024-09-21T04:02:02.766002+0000","last_active":"2024-09-21T04:02:29.863164+0000","last_peered":"2024-09-21T04:02:29.863164+0000","last_clean":"2024-09-21T04:02:29.863164+0000","last_became_active":"2024-09-21T04:02:02.765628+0000","last_became_peered":"2024-09-21T04:02:02.765628+0000","last_unstale":"2024-09-21T04:02:29.863164+0000","last_undegraded":"2024-09-21T04:02:29.863164+0000","last_fullsized":"2024-09-21T04:02:29.863164+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:22.251586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"73'6","reported_seq":30,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.861100+0000","last_change":"2024-09-21T04:02:00.741086+0000","last_active":"2024-09-21T04:02:29.861100+0000","last_peered":"2024-09-21T04:02:29.861100+0000","last_clean":"2024-09-21T04:02:29.861100+0000","last_became_active":"2024-09-21T04:02:00.740633+0000","last_became_peered":"2024-09-21T04:02:00.740633+0000","last_unstale":"2024-09-21T04:02:29.861100+0000","last_undegraded":"2024-09-21T04:02:29.861100+0000","last_fullsized":"2024-09-21T04:02:29.861100+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:20.392799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.862759+0000","last_change":"2024-09-21T04:02:02.759651+0000","last_active":"2024-09-21T04:02:29.862759+0000","last_peered":"2024-09-21T04:02:29.862759+0000","last_clean":"2024-09-21T04:02:29.862759+0000","last_became_active":"2024-09-21T04:02:02.759290+0000","last_became_peered":"2024-09-21T04:02:02.759290+0000","last_unstale":"2024-09-21T04:02:29.862759+0000","last_undegraded":"2024-09-21T04:02:29.862759+0000","last_fullsized":"2024-09-21T04:02:29.862759+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:32.025703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"73'3","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866619+0000","last_change":"2024-09-21T04:02:00.744005+0000","last_active":"2024-09-21T04:02:29.866619+0000","last_peered":"2024-09-21T04:02:29.866619+0000","last_clean":"2024-09-21T04:02:29.866619+0000","last_became_active":"2024-09-21T04:02:00.743570+0000","last_became_peered":"2024-09-21T04:02:00.743570+0000","last_unstale":"2024-09-21T04:02:29.866619+0000","last_undegraded":"2024-09-21T04:02:29.866619+0000","last_fullsized":"2024-09-21T04:02:29.866619+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:32.158681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.869292+0000","last_change":"2024-09-21T04:02:29.869200+0000","last_active":"2024-09-21T04:02:29.869292+0000","last_peered":"2024-09-21T04:02:29.869292+0000","last_clean":"2024-09-21T04:02:29.869292+0000","last_became_active":"2024-09-21T04:02:29.868836+0000","last_became_peered":"2024-09-21T04:02:29.868836+0000","last_unstale":"2024-09-21T04:02:29.869292+0000","last_undegraded":"2024-09-21T04:02:29.869292+0000","last_fullsized":"2024-09-21T04:02:29.869292+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:42:49.030499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"73'3","reported_seq":81,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:03:03.889965+0000","last_change":"2024-09-21T04:02:00.741596+0000","last_active":"2024-09-21T04:03:03.889965+0000","last_peered":"2024-09-21T04:03:03.889965+0000","last_clean":"2024-09-21T04:03:03.889965+0000","last_became_active":"2024-09-21T04:02:00.741185+0000","last_became_peered":"2024-09-21T04:02:00.741185+0000","last_unstale":"2024-09-21T04:03:03.889965+0000","last_undegraded":"2024-09-21T04:03:03.889965+0000","last_fullsized":"2024-09-21T04:03:03.889965+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:31.663133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.864575+0000","last_change":"2024-09-21T04:02:02.773706+0000","last_active":"2024-09-21T04:02:29.864575+0000","last_peered":"2024-09-21T04:02:29.864575+0000","last_clean":"2024-09-21T04:02:29.864575+0000","last_became_active":"2024-09-21T04:02:02.773259+0000","last_became_peered":"2024-09-21T04:02:02.773259+0000","last_unstale":"2024-09-21T04:02:29.864575+0000","last_undegraded":"2024-09-21T04:02:29.864575+0000","last_fullsized":"2024-09-21T04:02:29.864575+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:50.431982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"73'15","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.862850+0000","last_change":"2024-09-21T04:02:00.851335+0000","last_active":"2024-09-21T04:02:29.862850+0000","last_peered":"2024-09-21T04:02:29.862850+0000","last_clean":"2024-09-21T04:02:29.862850+0000","last_became_active":"2024-09-21T04:02:00.850990+0000","last_became_peered":"2024-09-21T04:02:00.850990+0000","last_unstale":"2024-09-21T04:02:29.862850+0000","last_undegraded":"2024-09-21T04:02:29.862850+0000","last_fullsized":"2024-09-21T04:02:29.862850+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:45.731121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866618+0000","last_change":"2024-09-21T04:02:02.770985+0000","last_active":"2024-09-21T04:02:29.866618+0000","last_peered":"2024-09-21T04:02:29.866618+0000","last_clean":"2024-09-21T04:02:29.866618+0000","last_became_active":"2024-09-21T04:02:02.770602+0000","last_became_peered":"2024-09-21T04:02:02.770602+0000","last_unstale":"2024-09-21T04:02:29.866618+0000","last_undegraded":"2024-09-21T04:02:29.866618+0000","last_fullsized":"2024-09-21T04:02:29.866618+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:37.131113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"73'2","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.863333+0000","last_change":"2024-09-21T04:02:00.740834+0000","last_active":"2024-09-21T04:02:29.863333+0000","last_peered":"2024-09-21T04:02:29.863333+0000","last_clean":"2024-09-21T04:02:29.863333+0000","last_became_active":"2024-09-21T04:02:00.740439+0000","last_became_peered":"2024-09-21T04:02:00.740439+0000","last_unstale":"2024-09-21T04:02:29.863333+0000","last_undegraded":"2024-09-21T04:02:29.863333+0000","last_fullsized":"2024-09-21T04:02:29.863333+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:04:22.159986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.018787+0000","last_change":"2024-09-21T04:02:30.018671+0000","last_active":"2024-09-21T04:02:30.018787+0000","last_peered":"2024-09-21T04:02:30.018787+0000","last_clean":"2024-09-21T04:02:30.018787+0000","last_became_active":"2024-09-21T04:02:30.018247+0000","last_became_peered":"2024-09-21T04:02:30.018247+0000","last_unstale":"2024-09-21T04:02:30.018787+0000","last_undegraded":"2024-09-21T04:02:30.018787+0000","last_fullsized":"2024-09-21T04:02:30.018787+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:13.501966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"73'2","reported_seq":36,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.024653+0000","last_change":"2024-09-21T04:02:30.024526+0000","last_active":"2024-09-21T04:02:30.024653+0000","last_peered":"2024-09-21T04:02:30.024653+0000","last_clean":"2024-09-21T04:02:30.024653+0000","last_became_active":"2024-09-21T04:02:30.018811+0000","last_became_peered":"2024-09-21T04:02:30.018811+0000","last_unstale":"2024-09-21T04:02:30.024653+0000","last_undegraded":"2024-09-21T04:02:30.024653+0000","last_fullsized":"2024-09-21T04:02:30.024653+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:07.093364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.864925+0000","last_change":"2024-09-21T04:02:02.766709+0000","last_active":"2024-09-21T04:02:29.864925+0000","last_peered":"2024-09-21T04:02:29.864925+0000","last_clean":"2024-09-21T04:02:29.864925+0000","last_became_active":"2024-09-21T04:02:02.766342+0000","last_became_peered":"2024-09-21T04:02:02.766342+0000","last_unstale":"2024-09-21T04:02:29.864925+0000","last_undegraded":"2024-09-21T04:02:29.864925+0000","last_fullsized":"2024-09-21T04:02:29.864925+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:26:35.185368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"73'1","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.867256+0000","last_change":"2024-09-21T04:02:00.745365+0000","last_active":"2024-09-21T04:02:29.867256+0000","last_peered":"2024-09-21T04:02:29.867256+0000","last_clean":"2024-09-21T04:02:29.867256+0000","last_became_active":"2024-09-21T04:02:00.744968+0000","last_became_peered":"2024-09-21T04:02:00.744968+0000","last_unstale":"2024-09-21T04:02:29.867256+0000","last_undegraded":"2024-09-21T04:02:29.867256+0000","last_fullsized":"2024-09-21T04:02:29.867256+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:56.078169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"73'1","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.864785+0000","last_change":"2024-09-21T04:02:02.774877+0000","last_active":"2024-09-21T04:02:29.864785+0000","last_peered":"2024-09-21T04:02:29.864785+0000","last_clean":"2024-09-21T04:02:29.864785+0000","last_became_active":"2024-09-21T04:02:02.774464+0000","last_became_peered":"2024-09-21T04:02:02.774464+0000","last_unstale":"2024-09-21T04:02:29.864785+0000","last_undegraded":"2024-09-21T04:02:29.864785+0000","last_fullsized":"2024-09-21T04:02:29.864785+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:23.393239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"73'3","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.861284+0000","last_change":"2024-09-21T04:02:01.257649+0000","last_active":"2024-09-21T04:02:29.861284+0000","last_peered":"2024-09-21T04:02:29.861284+0000","last_clean":"2024-09-21T04:02:29.861284+0000","last_became_active":"2024-09-21T04:02:01.257176+0000","last_became_peered":"2024-09-21T04:02:01.257176+0000","last_unstale":"2024-09-21T04:02:29.861284+0000","last_undegraded":"2024-09-21T04:02:29.861284+0000","last_fullsized":"2024-09-21T04:02:29.861284+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:37.257702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865292+0000","last_change":"2024-09-21T04:02:02.768164+0000","last_active":"2024-09-21T04:02:29.865292+0000","last_peered":"2024-09-21T04:02:29.865292+0000","last_clean":"2024-09-21T04:02:29.865292+0000","last_became_active":"2024-09-21T04:02:02.766925+0000","last_became_peered":"2024-09-21T04:02:02.766925+0000","last_unstale":"2024-09-21T04:02:29.865292+0000","last_undegraded":"2024-09-21T04:02:29.865292+0000","last_fullsized":"2024-09-21T04:02:29.865292+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:59.707637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":95,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865477+0000","last_change":"2024-09-21T04:00:48.936996+0000","last_active":"2024-09-21T04:02:29.865477+0000","last_peered":"2024-09-21T04:02:29.865477+0000","last_clean":"2024-09-21T04:02:29.865477+0000","last_became_active":"2024-09-21T04:00:48.931310+0000","last_became_peered":"2024-09-21T04:00:48.931310+0000","last_unstale":"2024-09-21T04:02:29.865477+0000","last_undegraded":"2024-09-21T04:02:29.865477+0000","last_fullsized":"2024-09-21T04:02:29.865477+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:57:25.346362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:57:25.346362+0000","last_clean_scrub_stamp":"2024-09-21T03:57:25.346362+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:24.165731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"73'3","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.017671+0000","last_change":"2024-09-21T04:02:00.740118+0000","last_active":"2024-09-21T04:02:30.017671+0000","last_peered":"2024-09-21T04:02:30.017671+0000","last_clean":"2024-09-21T04:02:30.017671+0000","last_became_active":"2024-09-21T04:02:00.739677+0000","last_became_peered":"2024-09-21T04:02:00.739677+0000","last_unstale":"2024-09-21T04:02:30.017671+0000","last_undegraded":"2024-09-21T04:02:30.017671+0000","last_fullsized":"2024-09-21T04:02:30.017671+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:46.084118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865883+0000","last_change":"2024-09-21T04:02:02.771537+0000","last_active":"2024-09-21T04:02:29.865883+0000","last_peered":"2024-09-21T04:02:29.865883+0000","last_clean":"2024-09-21T04:02:29.865883+0000","last_became_active":"2024-09-21T04:02:02.771151+0000","last_became_peered":"2024-09-21T04:02:02.771151+0000","last_unstale":"2024-09-21T04:02:29.865883+0000","last_undegraded":"2024-09-21T04:02:29.865883+0000","last_fullsized":"2024-09-21T04:02:29.865883+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:21.283515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"73'3","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.022770+0000","last_change":"2024-09-21T04:02:30.022634+0000","last_active":"2024-09-21T04:02:30.022770+0000","last_peered":"2024-09-21T04:02:30.022770+0000","last_clean":"2024-09-21T04:02:30.022770+0000","last_became_active":"2024-09-21T04:02:30.018076+0000","last_became_peered":"2024-09-21T04:02:30.018076+0000","last_unstale":"2024-09-21T04:02:30.022770+0000","last_undegraded":"2024-09-21T04:02:30.022770+0000","last_fullsized":"2024-09-21T04:02:30.022770+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:31.650092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.862438+0000","last_change":"2024-09-21T04:02:02.755223+0000","last_active":"2024-09-21T04:02:29.862438+0000","last_peered":"2024-09-21T04:02:29.862438+0000","last_clean":"2024-09-21T04:02:29.862438+0000","last_became_active":"2024-09-21T04:02:02.754836+0000","last_became_peered":"2024-09-21T04:02:02.754836+0000","last_unstale":"2024-09-21T04:02:29.862438+0000","last_undegraded":"2024-09-21T04:02:29.862438+0000","last_fullsized":"2024-09-21T04:02:29.862438+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:57.912690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"73'4","reported_seq":81,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:03:04.891007+0000","last_change":"2024-09-21T04:02:00.851689+0000","last_active":"2024-09-21T04:03:04.891007+0000","last_peered":"2024-09-21T04:03:04.891007+0000","last_clean":"2024-09-21T04:03:04.891007+0000","last_became_active":"2024-09-21T04:02:00.851241+0000","last_became_peered":"2024-09-21T04:02:00.851241+0000","last_unstale":"2024-09-21T04:03:04.891007+0000","last_undegraded":"2024-09-21T04:03:04.891007+0000","last_fullsized":"2024-09-21T04:03:04.891007+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:14.697695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866788+0000","last_change":"2024-09-21T04:02:02.774750+0000","last_active":"2024-09-21T04:02:29.866788+0000","last_peered":"2024-09-21T04:02:29.866788+0000","last_clean":"2024-09-21T04:02:29.866788+0000","last_became_active":"2024-09-21T04:02:02.774310+0000","last_became_peered":"2024-09-21T04:02:02.774310+0000","last_unstale":"2024-09-21T04:02:29.866788+0000","last_undegraded":"2024-09-21T04:02:29.866788+0000","last_fullsized":"2024-09-21T04:02:29.866788+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:07.909035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"73'1","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865053+0000","last_change":"2024-09-21T04:02:00.741461+0000","last_active":"2024-09-21T04:02:29.865053+0000","last_peered":"2024-09-21T04:02:29.865053+0000","last_clean":"2024-09-21T04:02:29.865053+0000","last_became_active":"2024-09-21T04:02:00.741051+0000","last_became_peered":"2024-09-21T04:02:00.741051+0000","last_unstale":"2024-09-21T04:02:29.865053+0000","last_undegraded":"2024-09-21T04:02:29.865053+0000","last_fullsized":"2024-09-21T04:02:29.865053+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:05.944430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.861420+0000","last_change":"2024-09-21T04:02:02.761703+0000","last_active":"2024-09-21T04:02:29.861420+0000","last_peered":"2024-09-21T04:02:29.861420+0000","last_clean":"2024-09-21T04:02:29.861420+0000","last_became_active":"2024-09-21T04:02:02.761296+0000","last_became_peered":"2024-09-21T04:02:02.761296+0000","last_unstale":"2024-09-21T04:02:29.861420+0000","last_undegraded":"2024-09-21T04:02:29.861420+0000","last_fullsized":"2024-09-21T04:02:29.861420+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:57.337167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.863538+0000","last_change":"2024-09-21T04:02:00.853584+0000","last_active":"2024-09-21T04:02:29.863538+0000","last_peered":"2024-09-21T04:02:29.863538+0000","last_clean":"2024-09-21T04:02:29.863538+0000","last_became_active":"2024-09-21T04:02:00.853019+0000","last_became_peered":"2024-09-21T04:02:00.853019+0000","last_unstale":"2024-09-21T04:02:29.863538+0000","last_undegraded":"2024-09-21T04:02:29.863538+0000","last_fullsized":"2024-09-21T04:02:29.863538+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:56.186791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"73'1","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.861430+0000","last_change":"2024-09-21T04:02:02.758175+0000","last_active":"2024-09-21T04:02:29.861430+0000","last_peered":"2024-09-21T04:02:29.861430+0000","last_clean":"2024-09-21T04:02:29.861430+0000","last_became_active":"2024-09-21T04:02:02.757715+0000","last_became_peered":"2024-09-21T04:02:02.757715+0000","last_unstale":"2024-09-21T04:02:29.861430+0000","last_undegraded":"2024-09-21T04:02:29.861430+0000","last_fullsized":"2024-09-21T04:02:29.861430+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:40.172569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"73'1","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.863271+0000","last_change":"2024-09-21T04:02:00.853325+0000","last_active":"2024-09-21T04:02:29.863271+0000","last_peered":"2024-09-21T04:02:29.863271+0000","last_clean":"2024-09-21T04:02:29.863271+0000","last_became_active":"2024-09-21T04:02:00.852834+0000","last_became_peered":"2024-09-21T04:02:00.852834+0000","last_unstale":"2024-09-21T04:02:29.863271+0000","last_undegraded":"2024-09-21T04:02:29.863271+0000","last_fullsized":"2024-09-21T04:02:29.863271+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:16.090493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.862826+0000","last_change":"2024-09-21T04:02:02.752402+0000","last_active":"2024-09-21T04:02:29.862826+0000","last_peered":"2024-09-21T04:02:29.862826+0000","last_clean":"2024-09-21T04:02:29.862826+0000","last_became_active":"2024-09-21T04:02:02.751994+0000","last_became_peered":"2024-09-21T04:02:02.751994+0000","last_unstale":"2024-09-21T04:02:29.862826+0000","last_undegraded":"2024-09-21T04:02:29.862826+0000","last_fullsized":"2024-09-21T04:02:29.862826+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:19.525317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"73'11","reported_seq":34,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.863685+0000","last_change":"2024-09-21T04:02:00.739112+0000","last_active":"2024-09-21T04:02:29.863685+0000","last_peered":"2024-09-21T04:02:29.863685+0000","last_clean":"2024-09-21T04:02:29.863685+0000","last_became_active":"2024-09-21T04:02:00.738613+0000","last_became_peered":"2024-09-21T04:02:00.738613+0000","last_unstale":"2024-09-21T04:02:29.863685+0000","last_undegraded":"2024-09-21T04:02:29.863685+0000","last_fullsized":"2024-09-21T04:02:29.863685+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:34.334576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865906+0000","last_change":"2024-09-21T04:02:02.763345+0000","last_active":"2024-09-21T04:02:29.865906+0000","last_peered":"2024-09-21T04:02:29.865906+0000","last_clean":"2024-09-21T04:02:29.865906+0000","last_became_active":"2024-09-21T04:02:02.762987+0000","last_became_peered":"2024-09-21T04:02:02.762987+0000","last_unstale":"2024-09-21T04:02:29.865906+0000","last_undegraded":"2024-09-21T04:02:29.865906+0000","last_fullsized":"2024-09-21T04:02:29.865906+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:36.239267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"73'4","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.018164+0000","last_change":"2024-09-21T04:02:01.258208+0000","last_active":"2024-09-21T04:02:30.018164+0000","last_peered":"2024-09-21T04:02:30.018164+0000","last_clean":"2024-09-21T04:02:30.018164+0000","last_became_active":"2024-09-21T04:02:01.257780+0000","last_became_peered":"2024-09-21T04:02:01.257780+0000","last_unstale":"2024-09-21T04:02:30.018164+0000","last_undegraded":"2024-09-21T04:02:30.018164+0000","last_fullsized":"2024-09-21T04:02:30.018164+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:20.161792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866715+0000","last_change":"2024-09-21T04:02:02.761149+0000","last_active":"2024-09-21T04:02:29.866715+0000","last_peered":"2024-09-21T04:02:29.866715+0000","last_clean":"2024-09-21T04:02:29.866715+0000","last_became_active":"2024-09-21T04:02:02.760771+0000","last_became_peered":"2024-09-21T04:02:02.760771+0000","last_unstale":"2024-09-21T04:02:29.866715+0000","last_undegraded":"2024-09-21T04:02:29.866715+0000","last_fullsized":"2024-09-21T04:02:29.866715+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:20.645377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"73'2","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.016281+0000","last_change":"2024-09-21T04:02:00.853580+0000","last_active":"2024-09-21T04:02:30.016281+0000","last_peered":"2024-09-21T04:02:30.016281+0000","last_clean":"2024-09-21T04:02:30.016281+0000","last_became_active":"2024-09-21T04:02:00.853091+0000","last_became_peered":"2024-09-21T04:02:00.853091+0000","last_unstale":"2024-09-21T04:02:30.016281+0000","last_undegraded":"2024-09-21T04:02:30.016281+0000","last_fullsized":"2024-09-21T04:02:30.016281+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:32.893695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.862695+0000","last_change":"2024-09-21T04:02:02.769686+0000","last_active":"2024-09-21T04:02:29.862695+0000","last_peered":"2024-09-21T04:02:29.862695+0000","last_clean":"2024-09-21T04:02:29.862695+0000","last_became_active":"2024-09-21T04:02:02.769293+0000","last_became_peered":"2024-09-21T04:02:02.769293+0000","last_unstale":"2024-09-21T04:02:29.862695+0000","last_undegraded":"2024-09-21T04:02:29.862695+0000","last_fullsized":"2024-09-21T04:02:29.862695+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:10.984624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"73'3","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865758+0000","last_change":"2024-09-21T04:02:00.741556+0000","last_active":"2024-09-21T04:02:29.865758+0000","last_peered":"2024-09-21T04:02:29.865758+0000","last_clean":"2024-09-21T04:02:29.865758+0000","last_became_active":"2024-09-21T04:02:00.741188+0000","last_became_peered":"2024-09-21T04:02:00.741188+0000","last_unstale":"2024-09-21T04:02:29.865758+0000","last_undegraded":"2024-09-21T04:02:29.865758+0000","last_fullsized":"2024-09-21T04:02:29.865758+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:23.566814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.867171+0000","last_change":"2024-09-21T04:02:02.769163+0000","last_active":"2024-09-21T04:02:29.867171+0000","last_peered":"2024-09-21T04:02:29.867171+0000","last_clean":"2024-09-21T04:02:29.867171+0000","last_became_active":"2024-09-21T04:02:02.768804+0000","last_became_peered":"2024-09-21T04:02:02.768804+0000","last_unstale":"2024-09-21T04:02:29.867171+0000","last_undegraded":"2024-09-21T04:02:29.867171+0000","last_fullsized":"2024-09-21T04:02:29.867171+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:12.576553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"73'1","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.860913+0000","last_change":"2024-09-21T04:02:00.743091+0000","last_active":"2024-09-21T04:02:29.860913+0000","last_peered":"2024-09-21T04:02:29.860913+0000","last_clean":"2024-09-21T04:02:29.860913+0000","last_became_active":"2024-09-21T04:02:00.742665+0000","last_became_peered":"2024-09-21T04:02:00.742665+0000","last_unstale":"2024-09-21T04:02:29.860913+0000","last_undegraded":"2024-09-21T04:02:29.860913+0000","last_fullsized":"2024-09-21T04:02:29.860913+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:09.698421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865858+0000","last_change":"2024-09-21T04:02:02.761985+0000","last_active":"2024-09-21T04:02:29.865858+0000","last_peered":"2024-09-21T04:02:29.865858+0000","last_clean":"2024-09-21T04:02:29.865858+0000","last_became_active":"2024-09-21T04:02:02.761624+0000","last_became_peered":"2024-09-21T04:02:02.761624+0000","last_unstale":"2024-09-21T04:02:29.865858+0000","last_undegraded":"2024-09-21T04:02:29.865858+0000","last_fullsized":"2024-09-21T04:02:29.865858+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:12.912221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"73'4","reported_seq":41,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.024364+0000","last_change":"2024-09-21T04:02:30.024288+0000","last_active":"2024-09-21T04:02:30.024364+0000","last_peered":"2024-09-21T04:02:30.024364+0000","last_clean":"2024-09-21T04:02:30.024364+0000","last_became_active":"2024-09-21T04:02:30.018929+0000","last_became_peered":"2024-09-21T04:02:30.018929+0000","last_unstale":"2024-09-21T04:02:30.024364+0000","last_undegraded":"2024-09-21T04:02:30.024364+0000","last_fullsized":"2024-09-21T04:02:30.024364+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:19:21.400082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865207+0000","last_change":"2024-09-21T04:02:02.761357+0000","last_active":"2024-09-21T04:02:29.865207+0000","last_peered":"2024-09-21T04:02:29.865207+0000","last_clean":"2024-09-21T04:02:29.865207+0000","last_became_active":"2024-09-21T04:02:02.760984+0000","last_became_peered":"2024-09-21T04:02:02.760984+0000","last_unstale":"2024-09-21T04:02:29.865207+0000","last_undegraded":"2024-09-21T04:02:29.865207+0000","last_fullsized":"2024-09-21T04:02:29.865207+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:55.388768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.019352+0000","last_change":"2024-09-21T04:02:30.019234+0000","last_active":"2024-09-21T04:02:30.019352+0000","last_peered":"2024-09-21T04:02:30.019352+0000","last_clean":"2024-09-21T04:02:30.019352+0000","last_became_active":"2024-09-21T04:02:30.018778+0000","last_became_peered":"2024-09-21T04:02:30.018778+0000","last_unstale":"2024-09-21T04:02:30.019352+0000","last_undegraded":"2024-09-21T04:02:30.019352+0000","last_fullsized":"2024-09-21T04:02:30.019352+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:31.227144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"73'2","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.863023+0000","last_change":"2024-09-21T04:02:00.742230+0000","last_active":"2024-09-21T04:02:29.863023+0000","last_peered":"2024-09-21T04:02:29.863023+0000","last_clean":"2024-09-21T04:02:29.863023+0000","last_became_active":"2024-09-21T04:02:00.741838+0000","last_became_peered":"2024-09-21T04:02:00.741838+0000","last_unstale":"2024-09-21T04:02:29.863023+0000","last_undegraded":"2024-09-21T04:02:29.863023+0000","last_fullsized":"2024-09-21T04:02:29.863023+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:18.909859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:30.016503+0000","last_change":"2024-09-21T04:02:02.762336+0000","last_active":"2024-09-21T04:02:30.016503+0000","last_peered":"2024-09-21T04:02:30.016503+0000","last_clean":"2024-09-21T04:02:30.016503+0000","last_became_active":"2024-09-21T04:02:02.761753+0000","last_became_peered":"2024-09-21T04:02:02.761753+0000","last_unstale":"2024-09-21T04:02:30.016503+0000","last_undegraded":"2024-09-21T04:02:30.016503+0000","last_fullsized":"2024-09-21T04:02:30.016503+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:39.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"73'4","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.863125+0000","last_change":"2024-09-21T04:02:00.743495+0000","last_active":"2024-09-21T04:02:29.863125+0000","last_peered":"2024-09-21T04:02:29.863125+0000","last_clean":"2024-09-21T04:02:29.863125+0000","last_became_active":"2024-09-21T04:02:00.743135+0000","last_became_peered":"2024-09-21T04:02:00.743135+0000","last_unstale":"2024-09-21T04:02:29.863125+0000","last_undegraded":"2024-09-21T04:02:29.863125+0000","last_fullsized":"2024-09-21T04:02:29.863125+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:16.481148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866766+0000","last_change":"2024-09-21T04:02:02.766682+0000","last_active":"2024-09-21T04:02:29.866766+0000","last_peered":"2024-09-21T04:02:29.866766+0000","last_clean":"2024-09-21T04:02:29.866766+0000","last_became_active":"2024-09-21T04:02:02.766299+0000","last_became_peered":"2024-09-21T04:02:02.766299+0000","last_unstale":"2024-09-21T04:02:29.866766+0000","last_undegraded":"2024-09-21T04:02:29.866766+0000","last_fullsized":"2024-09-21T04:02:29.866766+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:25.635908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"73'2","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.866104+0000","last_change":"2024-09-21T04:02:00.852281+0000","last_active":"2024-09-21T04:02:29.866104+0000","last_peered":"2024-09-21T04:02:29.866104+0000","last_clean":"2024-09-21T04:02:29.866104+0000","last_became_active":"2024-09-21T04:02:00.851900+0000","last_became_peered":"2024-09-21T04:02:00.851900+0000","last_unstale":"2024-09-21T04:02:29.866104+0000","last_undegraded":"2024-09-21T04:02:29.866104+0000","last_fullsized":"2024-09-21T04:02:29.866104+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:37.115541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.865896+0000","last_change":"2024-09-21T04:02:02.770467+0000","last_active":"2024-09-21T04:02:29.865896+0000","last_peered":"2024-09-21T04:02:29.865896+0000","last_clean":"2024-09-21T04:02:29.865896+0000","last_became_active":"2024-09-21T04:02:02.770099+0000","last_became_peered":"2024-09-21T04:02:02.770099+0000","last_unstale":"2024-09-21T04:02:29.865896+0000","last_undegraded":"2024-09-21T04:02:29.865896+0000","last_fullsized":"2024-09-21T04:02:29.865896+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:38.806298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"73'9","reported_seq":50,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T04:02:29.878120+0000","last_change":"2024-09-21T04:02:29.878053+0000","last_active":"2024-09-21T04:02:29.878120+0000","last_peered":"2024-09-21T04:02:29.878120+0000","last_clean":"2024-09-21T04:02:29.878120+0000","last_became_active":"2024-09-21T04:02:29.869007+0000","last_became_peered":"2024-09-21T04:02:29.869007+0000","last_unstale":"2024-09-21T04:02:29.878120+0000","last_undegraded":"2024-09-21T04:02:29.878120+0000","last_fullsized":"2024-09-21T04:02:29.878120+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:50.509849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69500,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":353,"num_read_kb":250,"num_write":162,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":208500,"data_compressed":5676,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282548,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808859,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31988,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713164,"statfs":{"total":95995035648,"available":95962279936,"internally_reserved":0,"allocated":819200,"data_stored":406805,"data_compressed":5778,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":62,"seq":266287972383,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":1063594,"data_compressed":43596,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135909,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":831488,"data_stored":406459,"data_compressed":5778,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332138,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32000,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713152,"statfs":{"total":95995035648,"available":95962267648,"internally_reserved":0,"allocated":831488,"data_stored":406379,"data_compressed":5778,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495662,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":876544,"data_stored":997336,"data_compressed":41704,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659188,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31988,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713164,"statfs":{"total":95995035648,"available":95962279936,"internally_reserved":0,"allocated":819200,"data_stored":405848,"data_compressed":5778,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855417,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":892928,"data_stored":997602,"data_compressed":41704,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018942,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":823296,"data_stored":406285,"data_compressed":5778,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":835584,"data_stored":406159,"data_compressed":5778,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378697,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":905216,"data_stored":472881,"data_compressed":7670,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542221,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":471531,"data_compressed":7670,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705746,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":839680,"data_stored":407119,"data_compressed":5778,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34306,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34306,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34306,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":65890,"data_compressed":1892,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67240,"data_compressed":1892,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":67468,"data_compressed":1892,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T04:03:11.798 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T04:03:11.798 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T04:03:11.798 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-21T04:03:11.798 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph health --format=json 2024-09-21T04:03:11.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:11 smithi131 ceph-mon[35193]: pgmap v294: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:03:11.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:11 smithi131 ceph-mon[35193]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:03:12.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:11 smithi119 ceph-mon[34811]: pgmap v294: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:03:12.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:11 smithi119 ceph-mon[34811]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:03:12.060 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:12.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.570+0000 7f7058b4d640 1 -- 172.21.15.64:0/2127360433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7054102a60 msgr2=0x7f7054102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:12.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.570+0000 7f7058b4d640 1 --2- 172.21.15.64:0/2127360433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7054102a60 0x7f7054102e70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7044009900 tx=0x7f704402a410 comp rx=0 tx=0).stop 2024-09-21T04:03:12.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.570+0000 7f7058b4d640 1 -- 172.21.15.64:0/2127360433 shutdown_connections 2024-09-21T04:03:12.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.570+0000 7f7058b4d640 1 --2- 172.21.15.64:0/2127360433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7054108cf0 0x7f705410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:12.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.570+0000 7f7058b4d640 1 --2- 172.21.15.64:0/2127360433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70541083a0 0x7f70541087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:12.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.570+0000 7f7058b4d640 1 --2- 172.21.15.64:0/2127360433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7054102a60 0x7f7054102e70 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:12.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.570+0000 7f7058b4d640 1 -- 172.21.15.64:0/2127360433 >> 172.21.15.64:0/2127360433 conn(0x7f7054079730 msgr2=0x7f7054079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:12.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.571+0000 7f7058b4d640 1 -- 172.21.15.64:0/2127360433 shutdown_connections 2024-09-21T04:03:12.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.571+0000 7f7058b4d640 1 -- 172.21.15.64:0/2127360433 wait complete. 2024-09-21T04:03:12.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.571+0000 7f7058b4d640 1 Processor -- start 2024-09-21T04:03:12.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.571+0000 7f7058b4d640 1 -- start start 2024-09-21T04:03:12.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.572+0000 7f7058b4d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7054102a60 0x7f705419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:12.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.572+0000 7f7058b4d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70541083a0 0x7f705419b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:12.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.572+0000 7f7058b4d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7054108cf0 0x7f705419bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:12.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.572+0000 7f7058b4d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7054112960 con 0x7f7054102a60 2024-09-21T04:03:12.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.572+0000 7f7058b4d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f70541127e0 con 0x7f7054108cf0 2024-09-21T04:03:12.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.572+0000 7f7058b4d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7054112ae0 con 0x7f70541083a0 2024-09-21T04:03:12.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.572+0000 7f705259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7054102a60 0x7f705419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:12.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.572+0000 7f705259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7054102a60 0x7f705419b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49620/0 (socket says 172.21.15.64:49620) 2024-09-21T04:03:12.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.572+0000 7f705259c640 1 -- 172.21.15.64:0/1912806117 learned_addr learned my addr 172.21.15.64:0/1912806117 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:12.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.572+0000 7f7052d9d640 1 --2- 172.21.15.64:0/1912806117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7054108cf0 0x7f705419bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:12.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.572+0000 7f7051d9b640 1 --2- 172.21.15.64:0/1912806117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70541083a0 0x7f705419b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:12.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.572+0000 7f705259c640 1 -- 172.21.15.64:0/1912806117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70541083a0 msgr2=0x7f705419b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:12.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.572+0000 7f705259c640 1 --2- 172.21.15.64:0/1912806117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70541083a0 0x7f705419b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:12.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.572+0000 7f705259c640 1 -- 172.21.15.64:0/1912806117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7054108cf0 msgr2=0x7f705419bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:12.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.572+0000 7f705259c640 1 --2- 172.21.15.64:0/1912806117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7054108cf0 0x7f705419bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:12.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.572+0000 7f705259c640 1 -- 172.21.15.64:0/1912806117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70541a65c0 con 0x7f7054102a60 2024-09-21T04:03:12.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.572+0000 7f7051d9b640 1 --2- 172.21.15.64:0/1912806117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70541083a0 0x7f705419b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:03:12.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.572+0000 7f705259c640 1 --2- 172.21.15.64:0/1912806117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7054102a60 0x7f705419b330 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7044005940 tx=0x7f70440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:12.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.573+0000 7f70437fe640 1 -- 172.21.15.64:0/1912806117 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704403f030 con 0x7f7054102a60 2024-09-21T04:03:12.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.573+0000 7f70437fe640 1 -- 172.21.15.64:0/1912806117 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7044003cb0 con 0x7f7054102a60 2024-09-21T04:03:12.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.573+0000 7f7058b4d640 1 -- 172.21.15.64:0/1912806117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70541a6850 con 0x7f7054102a60 2024-09-21T04:03:12.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.573+0000 7f70437fe640 1 -- 172.21.15.64:0/1912806117 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7044035070 con 0x7f7054102a60 2024-09-21T04:03:12.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.573+0000 7f7058b4d640 1 -- 172.21.15.64:0/1912806117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70541a6d80 con 0x7f7054102a60 2024-09-21T04:03:12.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.574+0000 7f7058b4d640 1 -- 172.21.15.64:0/1912806117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7054102e70 con 0x7f7054102a60 2024-09-21T04:03:12.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.575+0000 7f70437fe640 1 -- 172.21.15.64:0/1912806117 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f7044042030 con 0x7f7054102a60 2024-09-21T04:03:12.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.576+0000 7f70437fe640 1 --2- 172.21.15.64:0/1912806117 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f703c0b66a0 0x7f703c0b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:12.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.576+0000 7f7051d9b640 1 --2- 172.21.15.64:0/1912806117 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f703c0b66a0 0x7f703c0b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:12.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.576+0000 7f70437fe640 1 -- 172.21.15.64:0/1912806117 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f7044107e00 con 0x7f7054102a60 2024-09-21T04:03:12.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.577+0000 7f7051d9b640 1 --2- 172.21.15.64:0/1912806117 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f703c0b66a0 0x7f703c0b8bb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7054100be0 tx=0x7f7048005e50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:12.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.579+0000 7f70437fe640 1 -- 172.21.15.64:0/1912806117 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f70440ce430 con 0x7f7054102a60 2024-09-21T04:03:12.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.805+0000 7f7058b4d640 1 -- 172.21.15.64:0/1912806117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f705406b270 con 0x7f7054102a60 2024-09-21T04:03:12.808 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.806+0000 7f70437fe640 1 -- 172.21.15.64:0/1912806117 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f70440d2810 con 0x7f7054102a60 2024-09-21T04:03:12.808 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:03:12.808 INFO:teuthology.orchestra.run.smithi064.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T04:03:12.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.808+0000 7f7058b4d640 1 -- 172.21.15.64:0/1912806117 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f703c0b66a0 msgr2=0x7f703c0b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:12.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.808+0000 7f7058b4d640 1 --2- 172.21.15.64:0/1912806117 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f703c0b66a0 0x7f703c0b8bb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7054100be0 tx=0x7f7048005e50 comp rx=0 tx=0).stop 2024-09-21T04:03:12.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.808+0000 7f7058b4d640 1 -- 172.21.15.64:0/1912806117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7054102a60 msgr2=0x7f705419b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:12.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.808+0000 7f7058b4d640 1 --2- 172.21.15.64:0/1912806117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7054102a60 0x7f705419b330 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7044005940 tx=0x7f70440042f0 comp rx=0 tx=0).stop 2024-09-21T04:03:12.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.808+0000 7f7058b4d640 1 -- 172.21.15.64:0/1912806117 shutdown_connections 2024-09-21T04:03:12.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.808+0000 7f7058b4d640 1 --2- 172.21.15.64:0/1912806117 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f703c0b66a0 0x7f703c0b8bb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:12.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.808+0000 7f7058b4d640 1 --2- 172.21.15.64:0/1912806117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7054108cf0 0x7f705419bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:12.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.808+0000 7f7058b4d640 1 --2- 172.21.15.64:0/1912806117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f70541083a0 0x7f705419b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:12.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.808+0000 7f7058b4d640 1 --2- 172.21.15.64:0/1912806117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7054102a60 0x7f705419b330 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:12.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.808+0000 7f7058b4d640 1 -- 172.21.15.64:0/1912806117 >> 172.21.15.64:0/1912806117 conn(0x7f7054079730 msgr2=0x7f70541014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:12.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.808+0000 7f7058b4d640 1 -- 172.21.15.64:0/1912806117 shutdown_connections 2024-09-21T04:03:12.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:12.808+0000 7f7058b4d640 1 -- 172.21.15.64:0/1912806117 wait complete. 2024-09-21T04:03:13.472 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-21T04:03:13.472 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:03:13.731 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:13.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:13 smithi064 ceph-mon[31180]: pgmap v295: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T04:03:13.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:13 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1912806117' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T04:03:13.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:13 smithi131 ceph-mon[35193]: pgmap v295: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T04:03:13.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:13 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1912806117' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T04:03:14.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:13 smithi119 ceph-mon[34811]: pgmap v295: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T04:03:14.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:13 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1912806117' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T04:03:14.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.212+0000 7fa9e5ed1640 1 -- 172.21.15.64:0/1839722632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9e0103b60 msgr2=0x7fa9e0103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:14.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.212+0000 7fa9e5ed1640 1 --2- 172.21.15.64:0/1839722632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9e0103b60 0x7fa9e0103f70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0009900 tx=0x7fa9d002a410 comp rx=0 tx=0).stop 2024-09-21T04:03:14.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.212+0000 7fa9e5ed1640 1 -- 172.21.15.64:0/1839722632 shutdown_connections 2024-09-21T04:03:14.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.212+0000 7fa9e5ed1640 1 --2- 172.21.15.64:0/1839722632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9e0109df0 0x7fa9e0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:14.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.212+0000 7fa9e5ed1640 1 --2- 172.21.15.64:0/1839722632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9e01094a0 0x7fa9e01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:14.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.212+0000 7fa9e5ed1640 1 --2- 172.21.15.64:0/1839722632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9e0103b60 0x7fa9e0103f70 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:14.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.212+0000 7fa9e5ed1640 1 -- 172.21.15.64:0/1839722632 >> 172.21.15.64:0/1839722632 conn(0x7fa9e007b850 msgr2=0x7fa9e007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:14.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.212+0000 7fa9e5ed1640 1 -- 172.21.15.64:0/1839722632 shutdown_connections 2024-09-21T04:03:14.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.212+0000 7fa9e5ed1640 1 -- 172.21.15.64:0/1839722632 wait complete. 2024-09-21T04:03:14.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.213+0000 7fa9e5ed1640 1 Processor -- start 2024-09-21T04:03:14.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.213+0000 7fa9e5ed1640 1 -- start start 2024-09-21T04:03:14.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.213+0000 7fa9e5ed1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9e0103b60 0x7fa9e019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:14.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.213+0000 7fa9e5ed1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9e01094a0 0x7fa9e019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:14.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.213+0000 7fa9e5ed1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9e0109df0 0x7fa9e019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:14.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.213+0000 7fa9e5ed1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e0113ab0 con 0x7fa9e0109df0 2024-09-21T04:03:14.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.213+0000 7fa9e5ed1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e0113930 con 0x7fa9e0103b60 2024-09-21T04:03:14.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.213+0000 7fa9e5ed1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e0113c30 con 0x7fa9e01094a0 2024-09-21T04:03:14.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.213+0000 7fa9dffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9e0109df0 0x7fa9e019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:14.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.213+0000 7fa9dffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9e0109df0 0x7fa9e019fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49636/0 (socket says 172.21.15.64:49636) 2024-09-21T04:03:14.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.213+0000 7fa9dffff640 1 -- 172.21.15.64:0/2746600147 learned_addr learned my addr 172.21.15.64:0/2746600147 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:14.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.213+0000 7fa9deffd640 1 --2- 172.21.15.64:0/2746600147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9e01094a0 0x7fa9e019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:14.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.213+0000 7fa9df7fe640 1 --2- 172.21.15.64:0/2746600147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9e0103b60 0x7fa9e019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:14.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.214+0000 7fa9dffff640 1 -- 172.21.15.64:0/2746600147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9e01094a0 msgr2=0x7fa9e019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:14.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.214+0000 7fa9dffff640 1 --2- 172.21.15.64:0/2746600147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9e01094a0 0x7fa9e019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:14.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.214+0000 7fa9dffff640 1 -- 172.21.15.64:0/2746600147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9e0103b60 msgr2=0x7fa9e019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:14.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.214+0000 7fa9dffff640 1 --2- 172.21.15.64:0/2746600147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9e0103b60 0x7fa9e019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:14.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.214+0000 7fa9dffff640 1 -- 172.21.15.64:0/2746600147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e01a0490 con 0x7fa9e0109df0 2024-09-21T04:03:14.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.214+0000 7fa9dffff640 1 --2- 172.21.15.64:0/2746600147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9e0109df0 0x7fa9e019fd60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa9d800cad0 tx=0x7fa9d800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:14.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.214+0000 7fa9dcff9640 1 -- 172.21.15.64:0/2746600147 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d800ae20 con 0x7fa9e0109df0 2024-09-21T04:03:14.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.214+0000 7fa9dcff9640 1 -- 172.21.15.64:0/2746600147 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9d8005020 con 0x7fa9e0109df0 2024-09-21T04:03:14.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.214+0000 7fa9e5ed1640 1 -- 172.21.15.64:0/2746600147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e01a0780 con 0x7fa9e0109df0 2024-09-21T04:03:14.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.214+0000 7fa9dcff9640 1 -- 172.21.15.64:0/2746600147 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d800f6b0 con 0x7fa9e0109df0 2024-09-21T04:03:14.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.214+0000 7fa9e5ed1640 1 -- 172.21.15.64:0/2746600147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e01a6ee0 con 0x7fa9e0109df0 2024-09-21T04:03:14.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.215+0000 7fa9e5ed1640 1 -- 172.21.15.64:0/2746600147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9e0103f70 con 0x7fa9e0109df0 2024-09-21T04:03:14.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.218+0000 7fa9dcff9640 1 -- 172.21.15.64:0/2746600147 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fa9d80051d0 con 0x7fa9e0109df0 2024-09-21T04:03:14.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.218+0000 7fa9dcff9640 1 --2- 172.21.15.64:0/2746600147 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa9cc0b6670 0x7fa9cc0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:14.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.218+0000 7fa9df7fe640 1 --2- 172.21.15.64:0/2746600147 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa9cc0b6670 0x7fa9cc0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:14.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.219+0000 7fa9dcff9640 1 -- 172.21.15.64:0/2746600147 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7fa9d80e7850 con 0x7fa9e0109df0 2024-09-21T04:03:14.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.219+0000 7fa9df7fe640 1 --2- 172.21.15.64:0/2746600147 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa9cc0b6670 0x7fa9cc0b8b80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0031440 tx=0x7fa9d0009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:14.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.222+0000 7fa9dcff9640 1 -- 172.21.15.64:0/2746600147 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa9d8014030 con 0x7fa9e0109df0 2024-09-21T04:03:14.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.447+0000 7fa9e5ed1640 1 -- 172.21.15.64:0/2746600147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9e019b650 con 0x7fa9e0109df0 2024-09-21T04:03:14.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.448+0000 7fa9dcff9640 1 -- 172.21.15.64:0/2746600147 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11911 (secure 0 0 0) 0x7fa9e019b650 con 0x7fa9e0109df0 2024-09-21T04:03:14.450 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:03:14.450 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"btime":"2024-09-21T04:02:14:798588+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14805,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/97995119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":97995119},{"type":"v1","addr":"172.21.15.64:6839","nonce":97995119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/390891068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":390891068},{"type":"v1","addr":"172.21.15.64:6841","nonce":390891068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:02:03.354744+0000","modified":"2024-09-21T04:02:14.579750+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24710,"mds_1":24680,"mds_2":24661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24634":{"gid":24634,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6839/782190424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782190424},{"type":"v1","addr":"172.21.15.131:6839","nonce":782190424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24680":{"gid":24680,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24710":{"gid":24710,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6839/1319793636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1319793636},{"type":"v1","addr":"172.21.15.119:6839","nonce":1319793636}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24710,"qdb_cluster":[24661,24680,24710]},"id":1}]} 2024-09-21T04:03:14.451 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-09-21T04:03:14.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.450+0000 7fa9e5ed1640 1 -- 172.21.15.64:0/2746600147 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa9cc0b6670 msgr2=0x7fa9cc0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:14.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.450+0000 7fa9e5ed1640 1 --2- 172.21.15.64:0/2746600147 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa9cc0b6670 0x7fa9cc0b8b80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0031440 tx=0x7fa9d0009560 comp rx=0 tx=0).stop 2024-09-21T04:03:14.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.451+0000 7fa9e5ed1640 1 -- 172.21.15.64:0/2746600147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9e0109df0 msgr2=0x7fa9e019fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:14.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.451+0000 7fa9e5ed1640 1 --2- 172.21.15.64:0/2746600147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9e0109df0 0x7fa9e019fd60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa9d800cad0 tx=0x7fa9d800cf90 comp rx=0 tx=0).stop 2024-09-21T04:03:14.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.451+0000 7fa9e5ed1640 1 -- 172.21.15.64:0/2746600147 shutdown_connections 2024-09-21T04:03:14.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.451+0000 7fa9e5ed1640 1 --2- 172.21.15.64:0/2746600147 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa9cc0b6670 0x7fa9cc0b8b80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:14.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.451+0000 7fa9e5ed1640 1 --2- 172.21.15.64:0/2746600147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9e0109df0 0x7fa9e019fd60 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:14.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.451+0000 7fa9e5ed1640 1 --2- 172.21.15.64:0/2746600147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9e01094a0 0x7fa9e019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:14.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.451+0000 7fa9e5ed1640 1 --2- 172.21.15.64:0/2746600147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9e0103b60 0x7fa9e019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.451+0000 7fa9e5ed1640 1 -- 172.21.15.64:0/2746600147 >> 172.21.15.64:0/2746600147 conn(0x7fa9e007b850 msgr2=0x7fa9e01075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.451+0000 7fa9e5ed1640 1 -- 172.21.15.64:0/2746600147 shutdown_connections 2024-09-21T04:03:14.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:14.451+0000 7fa9e5ed1640 1 -- 172.21.15.64:0/2746600147 wait complete. 2024-09-21T04:03:14.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:14 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2746600147' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:03:14.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:14 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2746600147' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:03:14.907 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:14 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2746600147' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:03:15.105 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:03:15.363 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:15.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:15 smithi064 ceph-mon[31180]: pgmap v296: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:03:15.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:15 smithi131 ceph-mon[35193]: pgmap v296: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:03:15.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.887+0000 7fc19ed37640 1 -- 172.21.15.64:0/95032420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1981049d0 msgr2=0x7fc198104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:15.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.887+0000 7fc19ed37640 1 --2- 172.21.15.64:0/95032420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1981049d0 0x7fc198104de0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc188009900 tx=0x7fc18802a3f0 comp rx=0 tx=0).stop 2024-09-21T04:03:15.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.887+0000 7fc19ed37640 1 -- 172.21.15.64:0/95032420 shutdown_connections 2024-09-21T04:03:15.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.887+0000 7fc19ed37640 1 --2- 172.21.15.64:0/95032420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc19810de00 0x7fc198110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:15.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.887+0000 7fc19ed37640 1 --2- 172.21.15.64:0/95032420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc198105320 0x7fc19810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:15.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.887+0000 7fc19ed37640 1 --2- 172.21.15.64:0/95032420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1981049d0 0x7fc198104de0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:15.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.887+0000 7fc19ed37640 1 -- 172.21.15.64:0/95032420 >> 172.21.15.64:0/95032420 conn(0x7fc198079730 msgr2=0x7fc198079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:15.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.888+0000 7fc19ed37640 1 -- 172.21.15.64:0/95032420 shutdown_connections 2024-09-21T04:03:15.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.888+0000 7fc19ed37640 1 -- 172.21.15.64:0/95032420 wait complete. 2024-09-21T04:03:15.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.888+0000 7fc19ed37640 1 Processor -- start 2024-09-21T04:03:15.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.888+0000 7fc19ed37640 1 -- start start 2024-09-21T04:03:15.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.888+0000 7fc19ed37640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1981049d0 0x7fc19819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:15.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.888+0000 7fc19ed37640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc198105320 0x7fc19819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:15.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.888+0000 7fc19ed37640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc19810de00 0x7fc19819fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:15.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.888+0000 7fc19ed37640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc198113a80 con 0x7fc198105320 2024-09-21T04:03:15.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.889+0000 7fc19ed37640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc198113900 con 0x7fc19810de00 2024-09-21T04:03:15.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.889+0000 7fc19ed37640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc198113c00 con 0x7fc1981049d0 2024-09-21T04:03:15.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.889+0000 7fc197fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc198105320 0x7fc19819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:15.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.889+0000 7fc197fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc198105320 0x7fc19819ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49656/0 (socket says 172.21.15.64:49656) 2024-09-21T04:03:15.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.889+0000 7fc197fff640 1 -- 172.21.15.64:0/1716788176 learned_addr learned my addr 172.21.15.64:0/1716788176 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:15.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.889+0000 7fc19d2d4640 1 --2- 172.21.15.64:0/1716788176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc19810de00 0x7fc19819fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:15.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.889+0000 7fc19cad3640 1 --2- 172.21.15.64:0/1716788176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1981049d0 0x7fc19819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:15.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.889+0000 7fc197fff640 1 -- 172.21.15.64:0/1716788176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1981049d0 msgr2=0x7fc19819a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:15.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.889+0000 7fc197fff640 1 --2- 172.21.15.64:0/1716788176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1981049d0 0x7fc19819a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:15.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.889+0000 7fc197fff640 1 -- 172.21.15.64:0/1716788176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc19810de00 msgr2=0x7fc19819fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:15.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.889+0000 7fc197fff640 1 --2- 172.21.15.64:0/1716788176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc19810de00 0x7fc19819fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:15.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.889+0000 7fc197fff640 1 -- 172.21.15.64:0/1716788176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1981a0420 con 0x7fc198105320 2024-09-21T04:03:15.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.889+0000 7fc197fff640 1 --2- 172.21.15.64:0/1716788176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc198105320 0x7fc19819ab10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc18000cce0 tx=0x7fc18000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:15.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.889+0000 7fc195ffb640 1 -- 172.21.15.64:0/1716788176 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18000f450 con 0x7fc198105320 2024-09-21T04:03:15.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.889+0000 7fc195ffb640 1 -- 172.21.15.64:0/1716788176 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc18000faa0 con 0x7fc198105320 2024-09-21T04:03:15.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.889+0000 7fc19ed37640 1 -- 172.21.15.64:0/1716788176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1981a0710 con 0x7fc198105320 2024-09-21T04:03:15.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.889+0000 7fc195ffb640 1 -- 172.21.15.64:0/1716788176 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc180018c50 con 0x7fc198105320 2024-09-21T04:03:15.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.890+0000 7fc19ed37640 1 -- 172.21.15.64:0/1716788176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1981a6f30 con 0x7fc198105320 2024-09-21T04:03:15.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.891+0000 7fc19ed37640 1 -- 172.21.15.64:0/1716788176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc198104de0 con 0x7fc198105320 2024-09-21T04:03:15.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.892+0000 7fc195ffb640 1 -- 172.21.15.64:0/1716788176 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fc18000fc50 con 0x7fc198105320 2024-09-21T04:03:15.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.893+0000 7fc195ffb640 1 --2- 172.21.15.64:0/1716788176 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc1840b6620 0x7fc1840b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:15.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.893+0000 7fc19cad3640 1 --2- 172.21.15.64:0/1716788176 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc1840b6620 0x7fc1840b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:15.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.893+0000 7fc195ffb640 1 -- 172.21.15.64:0/1716788176 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7fc1800e7bb0 con 0x7fc198105320 2024-09-21T04:03:15.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.893+0000 7fc19cad3640 1 --2- 172.21.15.64:0/1716788176 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc1840b6620 0x7fc1840b8b30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc188000c00 tx=0x7fc188031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:15.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:15.896+0000 7fc195ffb640 1 -- 172.21.15.64:0/1716788176 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc180056710 con 0x7fc198105320 2024-09-21T04:03:16.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:15 smithi119 ceph-mon[34811]: pgmap v296: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:03:16.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:16.121+0000 7fc19ed37640 1 -- 172.21.15.64:0/1716788176 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc19819b6a0 con 0x7fc198105320 2024-09-21T04:03:16.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:16.122+0000 7fc195ffb640 1 -- 172.21.15.64:0/1716788176 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11911 (secure 0 0 0) 0x7fc180014030 con 0x7fc198105320 2024-09-21T04:03:16.124 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:03:16.124 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"btime":"2024-09-21T04:02:14:798588+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14805,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/97995119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":97995119},{"type":"v1","addr":"172.21.15.64:6839","nonce":97995119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/390891068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":390891068},{"type":"v1","addr":"172.21.15.64:6841","nonce":390891068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:02:03.354744+0000","modified":"2024-09-21T04:02:14.579750+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24710,"mds_1":24680,"mds_2":24661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24634":{"gid":24634,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6839/782190424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782190424},{"type":"v1","addr":"172.21.15.131:6839","nonce":782190424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24680":{"gid":24680,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24710":{"gid":24710,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6839/1319793636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1319793636},{"type":"v1","addr":"172.21.15.119:6839","nonce":1319793636}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24710,"qdb_cluster":[24661,24680,24710]},"id":1}]} 2024-09-21T04:03:16.125 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-09-21T04:03:16.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:16.124+0000 7fc19ed37640 1 -- 172.21.15.64:0/1716788176 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc1840b6620 msgr2=0x7fc1840b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:16.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:16.124+0000 7fc19ed37640 1 --2- 172.21.15.64:0/1716788176 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc1840b6620 0x7fc1840b8b30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc188000c00 tx=0x7fc188031400 comp rx=0 tx=0).stop 2024-09-21T04:03:16.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:16.124+0000 7fc19ed37640 1 -- 172.21.15.64:0/1716788176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc198105320 msgr2=0x7fc19819ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:16.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:16.124+0000 7fc19ed37640 1 --2- 172.21.15.64:0/1716788176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc198105320 0x7fc19819ab10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc18000cce0 tx=0x7fc18000a620 comp rx=0 tx=0).stop 2024-09-21T04:03:16.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:16.124+0000 7fc19ed37640 1 -- 172.21.15.64:0/1716788176 shutdown_connections 2024-09-21T04:03:16.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:16.124+0000 7fc19ed37640 1 --2- 172.21.15.64:0/1716788176 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc1840b6620 0x7fc1840b8b30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:16.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:16.124+0000 7fc19ed37640 1 --2- 172.21.15.64:0/1716788176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc19810de00 0x7fc19819fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:16.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:16.124+0000 7fc19ed37640 1 --2- 172.21.15.64:0/1716788176 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc198105320 0x7fc19819ab10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:16.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:16.124+0000 7fc19ed37640 1 --2- 172.21.15.64:0/1716788176 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1981049d0 0x7fc19819a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:16.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:16.124+0000 7fc19ed37640 1 -- 172.21.15.64:0/1716788176 >> 172.21.15.64:0/1716788176 conn(0x7fc198079730 msgr2=0x7fc1981068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:16.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:16.124+0000 7fc19ed37640 1 -- 172.21.15.64:0/1716788176 shutdown_connections 2024-09-21T04:03:16.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:16.124+0000 7fc19ed37640 1 -- 172.21.15.64:0/1716788176 wait complete. 2024-09-21T04:03:16.782 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:16 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1716788176' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:03:16.783 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T04:03:16.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:16 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1716788176' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:03:17.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:16 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1716788176' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:03:17.042 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:17.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.524+0000 7fa15b0c1640 1 -- 172.21.15.64:0/2682376613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa154076100 msgr2=0x7fa15410fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:17.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.524+0000 7fa15b0c1640 1 --2- 172.21.15.64:0/2682376613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa154076100 0x7fa15410fbd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa140009900 tx=0x7fa14002a410 comp rx=0 tx=0).stop 2024-09-21T04:03:17.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.525+0000 7fa15b0c1640 1 -- 172.21.15.64:0/2682376613 shutdown_connections 2024-09-21T04:03:17.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.525+0000 7fa15b0c1640 1 --2- 172.21.15.64:0/2682376613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa154110110 0x7fa154112550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:17.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.525+0000 7fa15b0c1640 1 --2- 172.21.15.64:0/2682376613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa154076100 0x7fa15410fbd0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:17.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.525+0000 7fa15b0c1640 1 --2- 172.21.15.64:0/2682376613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1540757b0 0x7fa154075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:17.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.525+0000 7fa15b0c1640 1 -- 172.21.15.64:0/2682376613 >> 172.21.15.64:0/2682376613 conn(0x7fa1540ff600 msgr2=0x7fa154101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:17.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.525+0000 7fa15b0c1640 1 -- 172.21.15.64:0/2682376613 shutdown_connections 2024-09-21T04:03:17.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.525+0000 7fa15b0c1640 1 -- 172.21.15.64:0/2682376613 wait complete. 2024-09-21T04:03:17.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.525+0000 7fa15b0c1640 1 Processor -- start 2024-09-21T04:03:17.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.525+0000 7fa15b0c1640 1 -- start start 2024-09-21T04:03:17.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.526+0000 7fa15b0c1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1540757b0 0x7fa15419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:17.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.526+0000 7fa15b0c1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa154076100 0x7fa15419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:17.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.526+0000 7fa15b0c1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa154110110 0x7fa1541a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:17.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.526+0000 7fa15b0c1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa154115cd0 con 0x7fa154110110 2024-09-21T04:03:17.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.526+0000 7fa15b0c1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa154115b50 con 0x7fa154076100 2024-09-21T04:03:17.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.526+0000 7fa15b0c1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa154115e50 con 0x7fa1540757b0 2024-09-21T04:03:17.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.526+0000 7fa15965e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa154110110 0x7fa1541a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:17.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.526+0000 7fa158e5d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1540757b0 0x7fa15419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:17.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.526+0000 7fa15965e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa154110110 0x7fa1541a1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49668/0 (socket says 172.21.15.64:49668) 2024-09-21T04:03:17.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.526+0000 7fa15965e640 1 -- 172.21.15.64:0/3739814616 learned_addr learned my addr 172.21.15.64:0/3739814616 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:17.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.526+0000 7fa153fff640 1 --2- 172.21.15.64:0/3739814616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa154076100 0x7fa15419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:17.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.526+0000 7fa15965e640 1 -- 172.21.15.64:0/3739814616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1540757b0 msgr2=0x7fa15419c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:17.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.526+0000 7fa15965e640 1 --2- 172.21.15.64:0/3739814616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1540757b0 0x7fa15419c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:17.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.526+0000 7fa15965e640 1 -- 172.21.15.64:0/3739814616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa154076100 msgr2=0x7fa15419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:17.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.526+0000 7fa15965e640 1 --2- 172.21.15.64:0/3739814616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa154076100 0x7fa15419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:17.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.526+0000 7fa15965e640 1 -- 172.21.15.64:0/3739814616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1541a2630 con 0x7fa154110110 2024-09-21T04:03:17.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.526+0000 7fa153fff640 1 --2- 172.21.15.64:0/3739814616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa154076100 0x7fa15419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:03:17.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.527+0000 7fa15965e640 1 --2- 172.21.15.64:0/3739814616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa154110110 0x7fa1541a1f00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009fd0 tx=0x7fa14c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:17.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.527+0000 7fa151ffb640 1 -- 172.21.15.64:0/3739814616 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c017070 con 0x7fa154110110 2024-09-21T04:03:17.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.527+0000 7fa151ffb640 1 -- 172.21.15.64:0/3739814616 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa14c003e20 con 0x7fa154110110 2024-09-21T04:03:17.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.527+0000 7fa15b0c1640 1 -- 172.21.15.64:0/3739814616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1541a2920 con 0x7fa154110110 2024-09-21T04:03:17.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.527+0000 7fa151ffb640 1 -- 172.21.15.64:0/3739814616 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c010460 con 0x7fa154110110 2024-09-21T04:03:17.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.527+0000 7fa15b0c1640 1 -- 172.21.15.64:0/3739814616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1541a9080 con 0x7fa154110110 2024-09-21T04:03:17.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.528+0000 7fa15b0c1640 1 -- 172.21.15.64:0/3739814616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa154075bc0 con 0x7fa154110110 2024-09-21T04:03:17.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.530+0000 7fa151ffb640 1 -- 172.21.15.64:0/3739814616 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fa14c003970 con 0x7fa154110110 2024-09-21T04:03:17.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.531+0000 7fa151ffb640 1 --2- 172.21.15.64:0/3739814616 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa1440b6670 0x7fa1440b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:17.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.531+0000 7fa158e5d640 1 --2- 172.21.15.64:0/3739814616 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa1440b6670 0x7fa1440b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:17.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.531+0000 7fa151ffb640 1 -- 172.21.15.64:0/3739814616 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7fa14c0e72b0 con 0x7fa154110110 2024-09-21T04:03:17.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.531+0000 7fa158e5d640 1 --2- 172.21.15.64:0/3739814616 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa1440b6670 0x7fa1440b8b80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa148009bd0 tx=0x7fa148005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:17.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.534+0000 7fa151ffb640 1 -- 172.21.15.64:0/3739814616 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa14c0b1ce0 con 0x7fa154110110 2024-09-21T04:03:17.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.724+0000 7fa15b0c1640 1 -- 172.21.15.64:0/3739814616 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1541a9330 con 0x7fa154110110 2024-09-21T04:03:17.727 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:17 smithi064 ceph-mon[31180]: pgmap v297: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T04:03:17.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.726+0000 7fa151ffb640 1 -- 172.21.15.64:0/3739814616 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+23089 (secure 0 0 0) 0x7fa14c0b6c20 con 0x7fa154110110 2024-09-21T04:03:17.729 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:03:17.729 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":75,"fsid":"e7c927ba-77cc-11ef-baf5-efdc52797490","created":"2024-09-21T03:53:41.215073+0000","modified":"2024-09-21T04:02:29.856726+0000","last_up_change":"2024-09-21T04:01:11.986949+0000","last_in_change":"2024-09-21T04:00:48.707791+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:57:24.854438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T04:01:58.728355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T04:02:01.300385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb947c10-98e6-4e3a-8bbd-8753951d870b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6803","nonce":3692023035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6805","nonce":3692023035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6809","nonce":3692023035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6807","nonce":3692023035}]},"public_addr":"172.21.15.64:6803/3692023035","cluster_addr":"172.21.15.64:6805/3692023035","heartbeat_back_addr":"172.21.15.64:6809/3692023035","heartbeat_front_addr":"172.21.15.64:6807/3692023035","state":["exists","up"]},{"osd":1,"uuid":"29b8c92d-82fa-4175-b3e2-711917b7f8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6801","nonce":3363415998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6803","nonce":3363415998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6807","nonce":3363415998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6805","nonce":3363415998}]},"public_addr":"172.21.15.119:6801/3363415998","cluster_addr":"172.21.15.119:6803/3363415998","heartbeat_back_addr":"172.21.15.119:6807/3363415998","heartbeat_front_addr":"172.21.15.119:6805/3363415998","state":["exists","up"]},{"osd":2,"uuid":"a2273fe8-df55-40f7-afda-74ab09c52c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6801","nonce":3422924179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6803","nonce":3422924179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6807","nonce":3422924179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6805","nonce":3422924179}]},"public_addr":"172.21.15.131:6801/3422924179","cluster_addr":"172.21.15.131:6803/3422924179","heartbeat_back_addr":"172.21.15.131:6807/3422924179","heartbeat_front_addr":"172.21.15.131:6805/3422924179","state":["exists","up"]},{"osd":3,"uuid":"f1634064-b108-41d2-a78f-c33607b08a70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6811","nonce":2022952729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6813","nonce":2022952729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6817","nonce":2022952729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6815","nonce":2022952729}]},"public_addr":"172.21.15.64:6811/2022952729","cluster_addr":"172.21.15.64:6813/2022952729","heartbeat_back_addr":"172.21.15.64:6817/2022952729","heartbeat_front_addr":"172.21.15.64:6815/2022952729","state":["exists","up"]},{"osd":4,"uuid":"75b81a8d-fa52-411a-bfda-aa69bb70a20f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6809","nonce":323030958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6811","nonce":323030958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6815","nonce":323030958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6813","nonce":323030958}]},"public_addr":"172.21.15.119:6809/323030958","cluster_addr":"172.21.15.119:6811/323030958","heartbeat_back_addr":"172.21.15.119:6815/323030958","heartbeat_front_addr":"172.21.15.119:6813/323030958","state":["exists","up"]},{"osd":5,"uuid":"0e4895b5-42e4-4b2c-a9c3-692f287cb841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6809","nonce":3021788088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6811","nonce":3021788088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6815","nonce":3021788088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6813","nonce":3021788088}]},"public_addr":"172.21.15.131:6809/3021788088","cluster_addr":"172.21.15.131:6811/3021788088","heartbeat_back_addr":"172.21.15.131:6815/3021788088","heartbeat_front_addr":"172.21.15.131:6813/3021788088","state":["exists","up"]},{"osd":6,"uuid":"1fa5c2b7-bce0-43a1-a26b-d029eb3731fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6819","nonce":1913824324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6821","nonce":1913824324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6825","nonce":1913824324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6823","nonce":1913824324}]},"public_addr":"172.21.15.64:6819/1913824324","cluster_addr":"172.21.15.64:6821/1913824324","heartbeat_back_addr":"172.21.15.64:6825/1913824324","heartbeat_front_addr":"172.21.15.64:6823/1913824324","state":["exists","up"]},{"osd":7,"uuid":"c9707260-b581-4a21-b380-f37e16959b02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6817","nonce":2567568166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6819","nonce":2567568166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6823","nonce":2567568166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6821","nonce":2567568166}]},"public_addr":"172.21.15.119:6817/2567568166","cluster_addr":"172.21.15.119:6819/2567568166","heartbeat_back_addr":"172.21.15.119:6823/2567568166","heartbeat_front_addr":"172.21.15.119:6821/2567568166","state":["exists","up"]},{"osd":8,"uuid":"2dbf8ef2-fc9c-4c6b-bfea-003f41ef191e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6817","nonce":199192217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6819","nonce":199192217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6823","nonce":199192217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6821","nonce":199192217}]},"public_addr":"172.21.15.131:6817/199192217","cluster_addr":"172.21.15.131:6819/199192217","heartbeat_back_addr":"172.21.15.131:6823/199192217","heartbeat_front_addr":"172.21.15.131:6821/199192217","state":["exists","up"]},{"osd":9,"uuid":"96f769e5-4783-4558-adfa-4bfaf9da9e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6827","nonce":3309038990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6829","nonce":3309038990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6833","nonce":3309038990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6831","nonce":3309038990}]},"public_addr":"172.21.15.64:6827/3309038990","cluster_addr":"172.21.15.64:6829/3309038990","heartbeat_back_addr":"172.21.15.64:6833/3309038990","heartbeat_front_addr":"172.21.15.64:6831/3309038990","state":["exists","up"]},{"osd":10,"uuid":"13c1ef5c-2299-440c-af5a-a650b47f4b75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6825","nonce":1460844681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6827","nonce":1460844681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6831","nonce":1460844681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6829","nonce":1460844681}]},"public_addr":"172.21.15.119:6825/1460844681","cluster_addr":"172.21.15.119:6827/1460844681","heartbeat_back_addr":"172.21.15.119:6831/1460844681","heartbeat_front_addr":"172.21.15.119:6829/1460844681","state":["exists","up"]},{"osd":11,"uuid":"c26a5ef9-8585-404e-a54a-852424c4d6fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6825","nonce":2294059084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6827","nonce":2294059084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6831","nonce":2294059084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6829","nonce":2294059084}]},"public_addr":"172.21.15.131:6825/2294059084","cluster_addr":"172.21.15.131:6827/2294059084","heartbeat_back_addr":"172.21.15.131:6831/2294059084","heartbeat_front_addr":"172.21.15.131:6829/2294059084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:56:33.141102+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:56:58.883062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:57:22.641789+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:57:48.705790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:58:12.871514+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:58:37.893118+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:03.620828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:29.446179+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:55.002978+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:00:19.987463+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:00:45.364158+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:01:10.228248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/3554471670":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6801/3786213009":"2024-09-22T03:55:28.387426+0000","172.21.15.64:0/2562856406":"2024-09-22T03:54:27.180100+0000","172.21.15.64:0/2603015699":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6800/1209932961":"2024-09-22T03:54:02.562380+0000","172.21.15.64:6800/3786213009":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6801/1209932961":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/3407344596":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1233731831":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1686887701":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1598568021":"2024-09-22T03:54:27.180100+0000","172.21.15.64:6800/2013929312":"2024-09-22T03:54:27.180100+0000","172.21.15.64:6801/2013929312":"2024-09-22T03:54:27.180100+0000","172.21.15.64:0/3652036144":"2024-09-22T03:55:28.387426+0000","172.21.15.64:0/105454703":"2024-09-22T03:54:27.180100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T04:03:17.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.729+0000 7fa15b0c1640 1 -- 172.21.15.64:0/3739814616 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa1440b6670 msgr2=0x7fa1440b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:17.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.729+0000 7fa15b0c1640 1 --2- 172.21.15.64:0/3739814616 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa1440b6670 0x7fa1440b8b80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa148009bd0 tx=0x7fa148005f90 comp rx=0 tx=0).stop 2024-09-21T04:03:17.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.729+0000 7fa15b0c1640 1 -- 172.21.15.64:0/3739814616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa154110110 msgr2=0x7fa1541a1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:17.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.729+0000 7fa15b0c1640 1 --2- 172.21.15.64:0/3739814616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa154110110 0x7fa1541a1f00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009fd0 tx=0x7fa14c00b260 comp rx=0 tx=0).stop 2024-09-21T04:03:17.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.729+0000 7fa15b0c1640 1 -- 172.21.15.64:0/3739814616 shutdown_connections 2024-09-21T04:03:17.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.729+0000 7fa15b0c1640 1 --2- 172.21.15.64:0/3739814616 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa1440b6670 0x7fa1440b8b80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:17.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.729+0000 7fa15b0c1640 1 --2- 172.21.15.64:0/3739814616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa154110110 0x7fa1541a1f00 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:17.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.729+0000 7fa15b0c1640 1 --2- 172.21.15.64:0/3739814616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa154076100 0x7fa15419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:17.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.729+0000 7fa15b0c1640 1 --2- 172.21.15.64:0/3739814616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1540757b0 0x7fa15419c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:17.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.729+0000 7fa15b0c1640 1 -- 172.21.15.64:0/3739814616 >> 172.21.15.64:0/3739814616 conn(0x7fa1540ff600 msgr2=0x7fa154109cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:17.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.729+0000 7fa15b0c1640 1 -- 172.21.15.64:0/3739814616 shutdown_connections 2024-09-21T04:03:17.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:17.729+0000 7fa15b0c1640 1 -- 172.21.15.64:0/3739814616 wait complete. 2024-09-21T04:03:17.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:17 smithi131 ceph-mon[35193]: pgmap v297: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T04:03:18.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:17 smithi119 ceph-mon[34811]: pgmap v297: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T04:03:18.444 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:03:18.705 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:18.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:18 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3739814616' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:03:18.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:18 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3739814616' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:03:19.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:18 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3739814616' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:03:19.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.180+0000 7f110dc8b640 1 -- 172.21.15.64:0/4127985557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1108103b80 msgr2=0x7f1108103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:19.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.180+0000 7f110dc8b640 1 --2- 172.21.15.64:0/4127985557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1108103b80 0x7f1108103f90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f10f8009ca0 tx=0x7f10f802a8c0 comp rx=0 tx=0).stop 2024-09-21T04:03:19.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.181+0000 7f110dc8b640 1 -- 172.21.15.64:0/4127985557 shutdown_connections 2024-09-21T04:03:19.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.181+0000 7f110dc8b640 1 --2- 172.21.15.64:0/4127985557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1108109e10 0x7f1108110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:19.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.181+0000 7f110dc8b640 1 --2- 172.21.15.64:0/4127985557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f11081094c0 0x7f11081098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:19.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.181+0000 7f110dc8b640 1 --2- 172.21.15.64:0/4127985557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1108103b80 0x7f1108103f90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:19.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.181+0000 7f110dc8b640 1 -- 172.21.15.64:0/4127985557 >> 172.21.15.64:0/4127985557 conn(0x7f11080ff5d0 msgr2=0x7f1108101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:19.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.181+0000 7f110dc8b640 1 -- 172.21.15.64:0/4127985557 shutdown_connections 2024-09-21T04:03:19.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.181+0000 7f110dc8b640 1 -- 172.21.15.64:0/4127985557 wait complete. 2024-09-21T04:03:19.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.181+0000 7f110dc8b640 1 Processor -- start 2024-09-21T04:03:19.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.181+0000 7f110dc8b640 1 -- start start 2024-09-21T04:03:19.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.182+0000 7f110dc8b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1108103b80 0x7f110819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:19.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.182+0000 7f110dc8b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11081094c0 0x7f110819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:19.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.182+0000 7f110dc8b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1108109e10 0x7f110819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:19.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.182+0000 7f110dc8b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1108113ad0 con 0x7f11081094c0 2024-09-21T04:03:19.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.182+0000 7f110dc8b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1108113950 con 0x7f1108103b80 2024-09-21T04:03:19.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.182+0000 7f110dc8b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1108113c50 con 0x7f1108109e10 2024-09-21T04:03:19.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.182+0000 7f1106ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11081094c0 0x7f110819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:19.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.182+0000 7f1106ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11081094c0 0x7f110819aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49682/0 (socket says 172.21.15.64:49682) 2024-09-21T04:03:19.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.182+0000 7f1106ffd640 1 -- 172.21.15.64:0/3846159133 learned_addr learned my addr 172.21.15.64:0/3846159133 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:19.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.182+0000 7f11077fe640 1 --2- 172.21.15.64:0/3846159133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1108103b80 0x7f110819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:19.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.182+0000 7f1107fff640 1 --2- 172.21.15.64:0/3846159133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1108109e10 0x7f110819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:19.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.182+0000 7f1106ffd640 1 -- 172.21.15.64:0/3846159133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1108109e10 msgr2=0x7f110819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:19.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.182+0000 7f1106ffd640 1 --2- 172.21.15.64:0/3846159133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1108109e10 0x7f110819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:19.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.182+0000 7f1106ffd640 1 -- 172.21.15.64:0/3846159133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1108103b80 msgr2=0x7f110819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:19.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.182+0000 7f1106ffd640 1 --2- 172.21.15.64:0/3846159133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1108103b80 0x7f110819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:19.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.182+0000 7f1106ffd640 1 -- 172.21.15.64:0/3846159133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11081a0430 con 0x7f11081094c0 2024-09-21T04:03:19.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.182+0000 7f1107fff640 1 --2- 172.21.15.64:0/3846159133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1108109e10 0x7f110819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:03:19.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.182+0000 7f1106ffd640 1 --2- 172.21.15.64:0/3846159133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11081094c0 0x7f110819aa60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f10f000d740 tx=0x7f10f000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:19.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.183+0000 7f1104ff9640 1 -- 172.21.15.64:0/3846159133 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0017070 con 0x7f11081094c0 2024-09-21T04:03:19.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.183+0000 7f1104ff9640 1 -- 172.21.15.64:0/3846159133 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10f000de50 con 0x7f11081094c0 2024-09-21T04:03:19.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.183+0000 7f110dc8b640 1 -- 172.21.15.64:0/3846159133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11081a0720 con 0x7f11081094c0 2024-09-21T04:03:19.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.183+0000 7f1104ff9640 1 -- 172.21.15.64:0/3846159133 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f000f620 con 0x7f11081094c0 2024-09-21T04:03:19.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.183+0000 7f110dc8b640 1 -- 172.21.15.64:0/3846159133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11081a6e80 con 0x7f11081094c0 2024-09-21T04:03:19.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.184+0000 7f110dc8b640 1 -- 172.21.15.64:0/3846159133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1108103f90 con 0x7f11081094c0 2024-09-21T04:03:19.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.185+0000 7f1104ff9640 1 -- 172.21.15.64:0/3846159133 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f10f001a070 con 0x7f11081094c0 2024-09-21T04:03:19.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.186+0000 7f1104ff9640 1 --2- 172.21.15.64:0/3846159133 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f10f40b65f0 0x7f10f40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:19.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.187+0000 7f11077fe640 1 --2- 172.21.15.64:0/3846159133 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f10f40b65f0 0x7f10f40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:19.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.187+0000 7f1104ff9640 1 -- 172.21.15.64:0/3846159133 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f10f00e6fd0 con 0x7f11081094c0 2024-09-21T04:03:19.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.187+0000 7f11077fe640 1 --2- 172.21.15.64:0/3846159133 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f10f40b65f0 0x7f10f40b8b00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f10f8005e30 tx=0x7f10f8009360 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:19.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.190+0000 7f1104ff9640 1 -- 172.21.15.64:0/3846159133 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f10f00ad600 con 0x7f11081094c0 2024-09-21T04:03:19.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.417+0000 7f110dc8b640 1 -- 172.21.15.64:0/3846159133 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f110819b5f0 con 0x7f11081094c0 2024-09-21T04:03:19.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.417+0000 7f1104ff9640 1 -- 172.21.15.64:0/3846159133 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11911 (secure 0 0 0) 0x7f110819b5f0 con 0x7f11081094c0 2024-09-21T04:03:19.420 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:03:19.420 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"btime":"2024-09-21T04:02:14:798588+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14805,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/97995119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":97995119},{"type":"v1","addr":"172.21.15.64:6839","nonce":97995119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/390891068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":390891068},{"type":"v1","addr":"172.21.15.64:6841","nonce":390891068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:02:03.354744+0000","modified":"2024-09-21T04:02:14.579750+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24710,"mds_1":24680,"mds_2":24661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24634":{"gid":24634,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6839/782190424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782190424},{"type":"v1","addr":"172.21.15.131:6839","nonce":782190424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24680":{"gid":24680,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24710":{"gid":24710,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6839/1319793636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1319793636},{"type":"v1","addr":"172.21.15.119:6839","nonce":1319793636}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24710,"qdb_cluster":[24661,24680,24710]},"id":1}]} 2024-09-21T04:03:19.420 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-09-21T04:03:19.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.420+0000 7f110dc8b640 1 -- 172.21.15.64:0/3846159133 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f10f40b65f0 msgr2=0x7f10f40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:19.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.420+0000 7f110dc8b640 1 --2- 172.21.15.64:0/3846159133 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f10f40b65f0 0x7f10f40b8b00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f10f8005e30 tx=0x7f10f8009360 comp rx=0 tx=0).stop 2024-09-21T04:03:19.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.420+0000 7f110dc8b640 1 -- 172.21.15.64:0/3846159133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11081094c0 msgr2=0x7f110819aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:19.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.420+0000 7f110dc8b640 1 --2- 172.21.15.64:0/3846159133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11081094c0 0x7f110819aa60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f10f000d740 tx=0x7f10f000dc00 comp rx=0 tx=0).stop 2024-09-21T04:03:19.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.420+0000 7f110dc8b640 1 -- 172.21.15.64:0/3846159133 shutdown_connections 2024-09-21T04:03:19.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.420+0000 7f110dc8b640 1 --2- 172.21.15.64:0/3846159133 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f10f40b65f0 0x7f10f40b8b00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:19.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.420+0000 7f110dc8b640 1 --2- 172.21.15.64:0/3846159133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1108109e10 0x7f110819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:19.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.420+0000 7f110dc8b640 1 --2- 172.21.15.64:0/3846159133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f11081094c0 0x7f110819aa60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:19.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.420+0000 7f110dc8b640 1 --2- 172.21.15.64:0/3846159133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1108103b80 0x7f110819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:19.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.420+0000 7f110dc8b640 1 -- 172.21.15.64:0/3846159133 >> 172.21.15.64:0/3846159133 conn(0x7f11080ff5d0 msgr2=0x7f11081019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:19.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.420+0000 7f110dc8b640 1 -- 172.21.15.64:0/3846159133 shutdown_connections 2024-09-21T04:03:19.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:19.420+0000 7f110dc8b640 1 -- 172.21.15.64:0/3846159133 wait complete. 2024-09-21T04:03:19.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:19 smithi064 ceph-mon[31180]: pgmap v298: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T04:03:19.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:19 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3846159133' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:03:19.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:19 smithi131 ceph-mon[35193]: pgmap v298: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T04:03:19.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:19 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3846159133' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:03:20.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:19 smithi119 ceph-mon[34811]: pgmap v298: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T04:03:20.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:19 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3846159133' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:03:20.097 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T04:02:03.354744+0000', 'modified': '2024-09-21T04:02:14.579750+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24710, 'mds_1': 24680, 'mds_2': 24661}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24634': {'gid': 24634, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.131:6835/1422569359', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 1422569359}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 1422569359}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24640': {'gid': 24640, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.131:6837/843209972', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6836', 'nonce': 843209972}, {'type': 'v1', 'addr': '172.21.15.131:6837', 'nonce': 843209972}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24656': {'gid': 24656, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.119:6835/3675946142', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6834', 'nonce': 3675946142}, {'type': 'v1', 'addr': '172.21.15.119:6835', 'nonce': 3675946142}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24661': {'gid': 24661, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.131:6839/782190424', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6838', 'nonce': 782190424}, {'type': 'v1', 'addr': '172.21.15.131:6839', 'nonce': 782190424}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24680': {'gid': 24680, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.119:6837/2876008744', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6836', 'nonce': 2876008744}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 2876008744}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24710': {'gid': 24710, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.119:6839/1319793636', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6838', 'nonce': 1319793636}, {'type': 'v1', 'addr': '172.21.15.119:6839', 'nonce': 1319793636}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24710, 'qdb_cluster': [24661, 24680, 24710]} 2024-09-21T04:03:20.098 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-21T04:03:20.098 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T04:03:20.358 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:20.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.816+0000 7f627d70b640 1 -- 172.21.15.64:0/676078455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62781083d0 msgr2=0x7f62781087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:20.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.816+0000 7f627d70b640 1 --2- 172.21.15.64:0/676078455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62781083d0 0x7f62781087e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6260009900 tx=0x7f626002a3f0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.816+0000 7f627d70b640 1 -- 172.21.15.64:0/676078455 shutdown_connections 2024-09-21T04:03:20.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.816+0000 7f627d70b640 1 --2- 172.21.15.64:0/676078455 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6278108d20 0x7f627810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.816+0000 7f627d70b640 1 --2- 172.21.15.64:0/676078455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f62781083d0 0x7f62781087e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.816+0000 7f627d70b640 1 --2- 172.21.15.64:0/676078455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6278102ae0 0x7f6278102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.816+0000 7f627d70b640 1 -- 172.21.15.64:0/676078455 >> 172.21.15.64:0/676078455 conn(0x7f6278079760 msgr2=0x7f6278079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:20.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.816+0000 7f627d70b640 1 -- 172.21.15.64:0/676078455 shutdown_connections 2024-09-21T04:03:20.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.816+0000 7f627d70b640 1 -- 172.21.15.64:0/676078455 wait complete. 2024-09-21T04:03:20.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.817+0000 7f627d70b640 1 Processor -- start 2024-09-21T04:03:20.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.817+0000 7f627d70b640 1 -- start start 2024-09-21T04:03:20.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.817+0000 7f627d70b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6278102ae0 0x7f62780ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:20.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.817+0000 7f627d70b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62781083d0 0x7f62780ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:20.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.817+0000 7f627d70b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6278108d20 0x7f62781000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:20.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.818+0000 7f627d70b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6278112990 con 0x7f6278108d20 2024-09-21T04:03:20.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.818+0000 7f627d70b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6278112810 con 0x7f6278102ae0 2024-09-21T04:03:20.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.818+0000 7f627d70b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6278112b10 con 0x7f62781083d0 2024-09-21T04:03:20.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.818+0000 7f62777fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6278108d20 0x7f62781000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:20.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.818+0000 7f62777fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6278108d20 0x7f62781000f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49692/0 (socket says 172.21.15.64:49692) 2024-09-21T04:03:20.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.818+0000 7f62777fe640 1 -- 172.21.15.64:0/3334138686 learned_addr learned my addr 172.21.15.64:0/3334138686 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:20.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.818+0000 7f62767fc640 1 --2- 172.21.15.64:0/3334138686 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62781083d0 0x7f62780ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:20.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.818+0000 7f6276ffd640 1 --2- 172.21.15.64:0/3334138686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6278102ae0 0x7f62780ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:20.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.818+0000 7f62777fe640 1 -- 172.21.15.64:0/3334138686 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62781083d0 msgr2=0x7f62780ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:20.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.818+0000 7f62777fe640 1 --2- 172.21.15.64:0/3334138686 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62781083d0 0x7f62780ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.818+0000 7f62777fe640 1 -- 172.21.15.64:0/3334138686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6278102ae0 msgr2=0x7f62780ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:20.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.818+0000 7f62777fe640 1 --2- 172.21.15.64:0/3334138686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6278102ae0 0x7f62780ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.818+0000 7f62777fe640 1 -- 172.21.15.64:0/3334138686 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62781a8720 con 0x7f6278108d20 2024-09-21T04:03:20.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.818+0000 7f62777fe640 1 --2- 172.21.15.64:0/3334138686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6278108d20 0x7f62781000f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f626800eaa0 tx=0x7f626800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:20.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.818+0000 7f626ffff640 1 -- 172.21.15.64:0/3334138686 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6268004d20 con 0x7f6278108d20 2024-09-21T04:03:20.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.818+0000 7f626ffff640 1 -- 172.21.15.64:0/3334138686 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6268005020 con 0x7f6278108d20 2024-09-21T04:03:20.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.818+0000 7f627d70b640 1 -- 172.21.15.64:0/3334138686 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62781a8a10 con 0x7f6278108d20 2024-09-21T04:03:20.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.819+0000 7f626ffff640 1 -- 172.21.15.64:0/3334138686 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62680116b0 con 0x7f6278108d20 2024-09-21T04:03:20.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.819+0000 7f627d70b640 1 -- 172.21.15.64:0/3334138686 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62781a8f20 con 0x7f6278108d20 2024-09-21T04:03:20.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.821+0000 7f6283c7f740 1 Processor -- start 2024-09-21T04:03:20.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.821+0000 7f626ffff640 1 -- 172.21.15.64:0/3334138686 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f62680051d0 con 0x7f6278108d20 2024-09-21T04:03:20.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.821+0000 7f6283c7f740 1 -- start start 2024-09-21T04:03:20.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.822+0000 7f6283c7f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a79b674f60 0x55a79ba1af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:20.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.822+0000 7f6283c7f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b64c0a0 0x55a79ba1b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:20.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.822+0000 7f6283c7f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a79b654280 0x55a79ba1b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:20.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.822+0000 7f6283c7f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a79b992c10 con 0x55a79b64c0a0 2024-09-21T04:03:20.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.822+0000 7f6283c7f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a79b982040 con 0x55a79b674f60 2024-09-21T04:03:20.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.822+0000 7f6283c7f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a79b985e50 con 0x55a79b654280 2024-09-21T04:03:20.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.822+0000 7f6276ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b64c0a0 0x55a79ba1b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:20.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.822+0000 7f6276ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b64c0a0 0x55a79ba1b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49704/0 (socket says 172.21.15.64:49704) 2024-09-21T04:03:20.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.822+0000 7f6276ffd640 1 -- 172.21.15.64:0/3836610310 learned_addr learned my addr 172.21.15.64:0/3836610310 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:20.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.822+0000 7f62767fc640 1 --2- 172.21.15.64:0/3836610310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a79b654280 0x55a79ba1b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:20.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.822+0000 7f6276ffd640 1 -- 172.21.15.64:0/3836610310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a79b654280 msgr2=0x55a79ba1b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:20.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.822+0000 7f6276ffd640 1 --2- 172.21.15.64:0/3836610310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a79b654280 0x55a79ba1b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.822+0000 7f62767fc640 1 --2- 172.21.15.64:0/3836610310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a79b674f60 0x55a79ba1af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:20.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.822+0000 7f6276ffd640 1 -- 172.21.15.64:0/3836610310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a79b674f60 msgr2=0x55a79ba1af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:20.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.822+0000 7f6276ffd640 1 --2- 172.21.15.64:0/3836610310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a79b674f60 0x55a79ba1af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.822+0000 7f6276ffd640 1 -- 172.21.15.64:0/3836610310 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a79b6e0540 con 0x55a79b64c0a0 2024-09-21T04:03:20.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.822+0000 7f62767fc640 1 --2- 172.21.15.64:0/3836610310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a79b674f60 0x55a79ba1af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:03:20.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.822+0000 7f626ffff640 1 --2- 172.21.15.64:0/3334138686 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f62640b65f0 0x7f62640b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:20.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.823+0000 7f62767fc640 1 --2- 172.21.15.64:0/3334138686 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f62640b65f0 0x7f62640b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:20.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.823+0000 7f6276ffd640 1 --2- 172.21.15.64:0/3836610310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b64c0a0 0x55a79ba1b440 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f627000ac80 tx=0x7f627000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e03f41e9348a56 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:20.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.823+0000 7f626ffff640 1 -- 172.21.15.64:0/3334138686 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f62680e7700 con 0x7f6278108d20 2024-09-21T04:03:20.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.823+0000 7f626d7fa640 1 -- 172.21.15.64:0/3836610310 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627001d070 con 0x55a79b64c0a0 2024-09-21T04:03:20.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.823+0000 7f626d7fa640 1 -- 172.21.15.64:0/3836610310 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6270018af0 con 0x55a79b64c0a0 2024-09-21T04:03:20.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.823+0000 7f626d7fa640 1 -- 172.21.15.64:0/3836610310 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6270010b40 con 0x55a79b64c0a0 2024-09-21T04:03:20.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.824+0000 7f6283c7f740 1 -- 172.21.15.64:0/3836610310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b64c0a0 msgr2=0x55a79ba1b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:20.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.824+0000 7f62767fc640 1 --2- 172.21.15.64:0/3334138686 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f62640b65f0 0x7f62640b8b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6260000c00 tx=0x7f6260031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:20.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.824+0000 7f6283c7f740 1 --2- 172.21.15.64:0/3836610310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b64c0a0 0x55a79ba1b440 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f627000ac80 tx=0x7f627000f790 comp rx=0 tx=0).stop 2024-09-21T04:03:20.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.824+0000 7f6283c7f740 1 -- 172.21.15.64:0/3836610310 shutdown_connections 2024-09-21T04:03:20.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.824+0000 7f6283c7f740 1 --2- 172.21.15.64:0/3836610310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a79b654280 0x55a79ba1b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.824+0000 7f6283c7f740 1 --2- 172.21.15.64:0/3836610310 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b64c0a0 0x55a79ba1b440 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.824+0000 7f6283c7f740 1 --2- 172.21.15.64:0/3836610310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a79b674f60 0x55a79ba1af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.824+0000 7f6283c7f740 1 -- 172.21.15.64:0/3836610310 >> 172.21.15.64:0/3836610310 conn(0x55a79b669350 msgr2=0x55a79ba18df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:20.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.824+0000 7f6283c7f740 1 -- 172.21.15.64:0/3836610310 shutdown_connections 2024-09-21T04:03:20.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.824+0000 7f6283c7f740 1 -- 172.21.15.64:0/3836610310 wait complete. 2024-09-21T04:03:20.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.824+0000 7f6283c7f740 1 -- start start 2024-09-21T04:03:20.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.824+0000 7f6283c7f740 1 Processor -- start 2024-09-21T04:03:20.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.825+0000 7f6283c7f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b64c0a0 0x55a79ba267c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:20.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.825+0000 7f6283c7f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a79b654280 0x55a79ba26da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:20.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.825+0000 7f6283c7f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a79b674f60 0x55a79ba32a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:20.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.825+0000 7f6283c7f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a79b7e35f0 con 0x55a79b64c0a0 2024-09-21T04:03:20.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.825+0000 7f6283c7f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a79b91d010 con 0x55a79b674f60 2024-09-21T04:03:20.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.825+0000 7f6283c7f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a79ba296e0 con 0x55a79b654280 2024-09-21T04:03:20.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.825+0000 7f6276ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b64c0a0 0x55a79ba267c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:20.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.825+0000 7f6283c7f740 10 client.0 _ll_register_callbacks cb 0x55a79b97c340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:03:20.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.825+0000 7f6276ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b64c0a0 0x55a79ba267c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49710/0 (socket says 172.21.15.64:49710) 2024-09-21T04:03:20.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.825+0000 7f6276ffd640 1 -- 172.21.15.64:0/2205752562 learned_addr learned my addr 172.21.15.64:0/2205752562 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:20.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.825+0000 7f62777fe640 1 --2- 172.21.15.64:0/2205752562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a79b674f60 0x55a79ba32a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:20.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.825+0000 7f62767fc640 1 --2- 172.21.15.64:0/2205752562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a79b654280 0x55a79ba26da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:20.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.825+0000 7f6276ffd640 1 -- 172.21.15.64:0/2205752562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a79b654280 msgr2=0x55a79ba26da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:20.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.825+0000 7f6276ffd640 1 --2- 172.21.15.64:0/2205752562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a79b654280 0x55a79ba26da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.825+0000 7f6276ffd640 1 -- 172.21.15.64:0/2205752562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a79b674f60 msgr2=0x55a79ba32a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:20.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.825+0000 7f6276ffd640 1 --2- 172.21.15.64:0/2205752562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a79b674f60 0x55a79ba32a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.825+0000 7f6276ffd640 1 -- 172.21.15.64:0/2205752562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a79b94aa00 con 0x55a79b64c0a0 2024-09-21T04:03:20.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.825+0000 7f62767fc640 1 --2- 172.21.15.64:0/2205752562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a79b654280 0x55a79ba26da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:03:20.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.825+0000 7f6276ffd640 1 --2- 172.21.15.64:0/2205752562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b64c0a0 0x55a79ba267c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f627000a560 tx=0x7f6270005eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89b7cff8dec3e9e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:20.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.825+0000 7f625e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:03:20.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.826+0000 7f625e7fc640 1 -- 172.21.15.64:0/2205752562 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6270027020 con 0x55a79b64c0a0 2024-09-21T04:03:20.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.826+0000 7f625e7fc640 1 -- 172.21.15.64:0/2205752562 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f62700108c0 con 0x55a79b64c0a0 2024-09-21T04:03:20.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.826+0000 7f6283c7f740 1 -- 172.21.15.64:0/2205752562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6270018af0 con 0x55a79b64c0a0 2024-09-21T04:03:20.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.826+0000 7f625e7fc640 1 -- 172.21.15.64:0/2205752562 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6270032050 con 0x55a79b64c0a0 2024-09-21T04:03:20.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.826+0000 7f6283c7f740 10 client.15180 fetch_fsmap learned FSMap version 24 2024-09-21T04:03:20.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.826+0000 7f6283c7f740 1 -- 172.21.15.64:0/2205752562 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a79ba35570 con 0x55a79b64c0a0 2024-09-21T04:03:20.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.826+0000 7f625e7fc640 1 -- 172.21.15.64:0/2205752562 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6270018af0 con 0x55a79b64c0a0 2024-09-21T04:03:20.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.826+0000 7f625e7fc640 1 -- 172.21.15.64:0/2205752562 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f62700046c0 con 0x55a79b64c0a0 2024-09-21T04:03:20.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.826+0000 7f6283c7f740 10 client.15180 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T04:03:20.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.826+0000 7f6283c7f740 10 client.15180 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T04:03:20.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.826+0000 7f6283c7f740 20 client.15180 populate_metadata read hostname 'smithi064' 2024-09-21T04:03:20.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.826+0000 7f6283c7f740 1 --2- 172.21.15.64:0/2205752562 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55a79b6338e0 0x55a79ba38c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:20.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.826+0000 7f6283c7f740 4 client.15180 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:03:20.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.826+0000 7f6283c7f740 1 -- 172.21.15.64:0/2205752562 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a79b994b60 con 0x55a79b6338e0 2024-09-21T04:03:20.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.827+0000 7f62767fc640 1 --2- 172.21.15.64:0/2205752562 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55a79b6338e0 0x55a79ba38c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:20.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.827+0000 7f625e7fc640 10 client.15180 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:03:20.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.827+0000 7f62767fc640 1 --2- 172.21.15.64:0/2205752562 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55a79b6338e0 0x55a79ba38c20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c9eefbbfaa0dc21 server_cookie=f665080fec585380 in_seq=0 out_seq=0 2024-09-21T04:03:20.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.828+0000 7f625e7fc640 1 -- 172.21.15.64:0/2205752562 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a79b994b60 con 0x55a79b6338e0 2024-09-21T04:03:20.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.828+0000 7f625e7fc640 10 client.15180 handle_command_reply: tid=0 2024-09-21T04:03:20.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.828+0000 7f6283c7f740 1 client.15180 shutdown 2024-09-21T04:03:20.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.828+0000 7f6283c7f740 1 -- 172.21.15.64:0/2205752562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b64c0a0 msgr2=0x55a79ba267c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:20.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.828+0000 7f6283c7f740 1 --2- 172.21.15.64:0/2205752562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b64c0a0 0x55a79ba267c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f627000a560 tx=0x7f6270005eb0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.828+0000 7f6283c7f740 1 -- 172.21.15.64:0/2205752562 shutdown_connections 2024-09-21T04:03:20.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.828+0000 7f6283c7f740 1 --2- 172.21.15.64:0/2205752562 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55a79b6338e0 0x55a79ba38c20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.828+0000 7f6283c7f740 1 --2- 172.21.15.64:0/2205752562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a79b674f60 0x55a79ba32a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.828+0000 7f6283c7f740 1 --2- 172.21.15.64:0/2205752562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a79b654280 0x55a79ba26da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.828+0000 7f625e7fc640 0 client.15180 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:03:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.828+0000 7f6283c7f740 1 --2- 172.21.15.64:0/2205752562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b64c0a0 0x55a79ba267c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.828+0000 7f6283c7f740 1 -- 172.21.15.64:0/2205752562 >> 172.21.15.64:0/2205752562 conn(0x55a79b669350 msgr2=0x55a79ba1e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.828+0000 7f6283c7f740 1 -- 172.21.15.64:0/2205752562 shutdown_connections 2024-09-21T04:03:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.828+0000 7f6283c7f740 1 -- 172.21.15.64:0/2205752562 wait complete. 2024-09-21T04:03:20.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.828+0000 7f6283c7f740 20 client.15180 trim_cache size 0 max 16384 2024-09-21T04:03:20.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.847+0000 7f6283c7f740 1 Processor -- start 2024-09-21T04:03:20.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.847+0000 7f6283c7f740 1 -- start start 2024-09-21T04:03:20.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.848+0000 7f6283c7f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a79b6338e0 0x55a79ba21020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:20.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.848+0000 7f6283c7f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b64c0a0 0x55a79ba329a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:20.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.848+0000 7f6283c7f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a79b654280 0x55a79ba32ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:20.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.848+0000 7f6283c7f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a79ba36f60 con 0x55a79b64c0a0 2024-09-21T04:03:20.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.848+0000 7f6283c7f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a79ba37630 con 0x55a79b6338e0 2024-09-21T04:03:20.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.848+0000 7f6283c7f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a79ba37d00 con 0x55a79b654280 2024-09-21T04:03:20.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.848+0000 7f62767fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b64c0a0 0x55a79ba329a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:20.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.848+0000 7f62767fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b64c0a0 0x55a79ba329a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49712/0 (socket says 172.21.15.64:49712) 2024-09-21T04:03:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.848+0000 7f62767fc640 1 -- 172.21.15.64:0/55220489 learned_addr learned my addr 172.21.15.64:0/55220489 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.848+0000 7f6276ffd640 1 --2- 172.21.15.64:0/55220489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a79b6338e0 0x55a79ba21020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.848+0000 7f62777fe640 1 --2- 172.21.15.64:0/55220489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a79b654280 0x55a79ba32ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.848+0000 7f62767fc640 1 -- 172.21.15.64:0/55220489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a79b654280 msgr2=0x55a79ba32ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.848+0000 7f62767fc640 1 --2- 172.21.15.64:0/55220489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a79b654280 0x55a79ba32ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.848+0000 7f62767fc640 1 -- 172.21.15.64:0/55220489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a79b6338e0 msgr2=0x55a79ba21020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.848+0000 7f62767fc640 1 --2- 172.21.15.64:0/55220489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a79b6338e0 0x55a79ba21020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.848+0000 7f62767fc640 1 -- 172.21.15.64:0/55220489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a79ba27a80 con 0x55a79b64c0a0 2024-09-21T04:03:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.848+0000 7f6276ffd640 1 --2- 172.21.15.64:0/55220489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a79b6338e0 0x55a79ba21020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:03:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.849+0000 7f62767fc640 1 --2- 172.21.15.64:0/55220489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b64c0a0 0x55a79ba329a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6260009900 tx=0x7f626003f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639bacd874746e80 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.849+0000 7f626cff9640 1 -- 172.21.15.64:0/55220489 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6260043070 con 0x55a79b64c0a0 2024-09-21T04:03:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.849+0000 7f626cff9640 1 -- 172.21.15.64:0/55220489 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6260004380 con 0x55a79b64c0a0 2024-09-21T04:03:20.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.849+0000 7f626cff9640 1 -- 172.21.15.64:0/55220489 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6260035040 con 0x55a79b64c0a0 2024-09-21T04:03:20.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.850+0000 7f6283c7f740 1 -- 172.21.15.64:0/55220489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b64c0a0 msgr2=0x55a79ba329a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:20.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.850+0000 7f6283c7f740 1 --2- 172.21.15.64:0/55220489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b64c0a0 0x55a79ba329a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6260009900 tx=0x7f626003f070 comp rx=0 tx=0).stop 2024-09-21T04:03:20.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.850+0000 7f6283c7f740 1 -- 172.21.15.64:0/55220489 shutdown_connections 2024-09-21T04:03:20.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.850+0000 7f6283c7f740 1 --2- 172.21.15.64:0/55220489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a79b654280 0x55a79ba32ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.850+0000 7f6283c7f740 1 --2- 172.21.15.64:0/55220489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b64c0a0 0x55a79ba329a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.850+0000 7f6283c7f740 1 --2- 172.21.15.64:0/55220489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a79b6338e0 0x55a79ba21020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.850+0000 7f6283c7f740 1 -- 172.21.15.64:0/55220489 >> 172.21.15.64:0/55220489 conn(0x55a79b669350 msgr2=0x55a79ba219a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:20.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.850+0000 7f6283c7f740 1 -- 172.21.15.64:0/55220489 shutdown_connections 2024-09-21T04:03:20.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.850+0000 7f6283c7f740 1 -- 172.21.15.64:0/55220489 wait complete. 2024-09-21T04:03:20.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.850+0000 7f6283c7f740 1 -- start start 2024-09-21T04:03:20.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.850+0000 7f6283c7f740 1 Processor -- start 2024-09-21T04:03:20.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.851+0000 7f6283c7f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b6338e0 0x55a79ba41880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:20.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.851+0000 7f6283c7f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a79b64c0a0 0x55a79ba3ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:20.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.851+0000 7f6283c7f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a79b654280 0x55a79ba3b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:20.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.851+0000 7f6283c7f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a79ba38aa0 con 0x55a79b6338e0 2024-09-21T04:03:20.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.851+0000 7f6283c7f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a79ba383d0 con 0x55a79b64c0a0 2024-09-21T04:03:20.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.851+0000 7f6283c7f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a79ba39170 con 0x55a79b654280 2024-09-21T04:03:20.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.851+0000 7f6283c7f740 10 client.0 _ll_register_callbacks cb 0x7f627001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:03:20.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.851+0000 7f6276ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b6338e0 0x55a79ba41880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:20.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.851+0000 7f6276ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b6338e0 0x55a79ba41880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49718/0 (socket says 172.21.15.64:49718) 2024-09-21T04:03:20.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.851+0000 7f6276ffd640 1 -- 172.21.15.64:0/1202039160 learned_addr learned my addr 172.21.15.64:0/1202039160 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:20.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.851+0000 7f62767fc640 1 --2- 172.21.15.64:0/1202039160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a79b64c0a0 0x55a79ba3ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:20.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.851+0000 7f62777fe640 1 --2- 172.21.15.64:0/1202039160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a79b654280 0x55a79ba3b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:20.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.851+0000 7f6276ffd640 1 -- 172.21.15.64:0/1202039160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a79b654280 msgr2=0x55a79ba3b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:20.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.851+0000 7f6276ffd640 1 --2- 172.21.15.64:0/1202039160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a79b654280 0x55a79ba3b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.851+0000 7f6276ffd640 1 -- 172.21.15.64:0/1202039160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a79b64c0a0 msgr2=0x55a79ba3ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:20.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.851+0000 7f6276ffd640 1 --2- 172.21.15.64:0/1202039160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a79b64c0a0 0x55a79ba3ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.851+0000 7f6276ffd640 1 -- 172.21.15.64:0/1202039160 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a79ba404c0 con 0x55a79b6338e0 2024-09-21T04:03:20.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.851+0000 7f62777fe640 1 --2- 172.21.15.64:0/1202039160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a79b654280 0x55a79ba3b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:03:20.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.851+0000 7f6276ffd640 1 --2- 172.21.15.64:0/1202039160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b6338e0 0x55a79ba41880 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6270005e80 tx=0x7f627000abf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4e72217315d986 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:20.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.851+0000 7f625e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:03:20.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.852+0000 7f625e7fc640 1 -- 172.21.15.64:0/1202039160 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6270027070 con 0x55a79b6338e0 2024-09-21T04:03:20.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.852+0000 7f625e7fc640 1 -- 172.21.15.64:0/1202039160 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f6270018a10 con 0x55a79b6338e0 2024-09-21T04:03:20.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.852+0000 7f6283c7f740 1 -- 172.21.15.64:0/1202039160 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6260004380 con 0x55a79b6338e0 2024-09-21T04:03:20.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.852+0000 7f625e7fc640 1 -- 172.21.15.64:0/1202039160 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f627001d070 con 0x55a79b6338e0 2024-09-21T04:03:20.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.852+0000 7f6283c7f740 10 client.15186 fetch_fsmap learned FSMap version 24 2024-09-21T04:03:20.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.852+0000 7f6283c7f740 1 -- 172.21.15.64:0/1202039160 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a79baa4240 con 0x55a79b6338e0 2024-09-21T04:03:20.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.852+0000 7f625e7fc640 1 -- 172.21.15.64:0/1202039160 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6260004380 con 0x55a79b6338e0 2024-09-21T04:03:20.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.852+0000 7f625e7fc640 1 -- 172.21.15.64:0/1202039160 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f6270005120 con 0x55a79b6338e0 2024-09-21T04:03:20.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.852+0000 7f6283c7f740 10 client.15186 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T04:03:20.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.852+0000 7f6283c7f740 10 client.15186 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T04:03:20.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.852+0000 7f6283c7f740 20 client.15186 populate_metadata read hostname 'smithi064' 2024-09-21T04:03:20.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.852+0000 7f6283c7f740 1 --2- 172.21.15.64:0/1202039160 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55a79b674f60 0x55a79baa7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:20.859 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.852+0000 7f6283c7f740 4 client.15186 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T04:03:20.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.852+0000 7f6283c7f740 1 -- 172.21.15.64:0/1202039160 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "status"}) -- 0x55a79ba27300 con 0x55a79b674f60 2024-09-21T04:03:20.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.853+0000 7f62767fc640 1 --2- 172.21.15.64:0/1202039160 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55a79b674f60 0x55a79baa7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:20.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.853+0000 7f625e7fc640 10 client.15186 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:03:20.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.853+0000 7f62767fc640 1 --2- 172.21.15.64:0/1202039160 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55a79b674f60 0x55a79baa7aa0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=79c2ec7171583273 server_cookie=4e9e2373a5dbb120 in_seq=0 out_seq=0 2024-09-21T04:03:20.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.854+0000 7f625e7fc640 1 -- 172.21.15.64:0/1202039160 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a79ba27300 con 0x55a79b674f60 2024-09-21T04:03:20.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.854+0000 7f625e7fc640 10 client.15186 handle_command_reply: tid=0 2024-09-21T04:03:20.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.854+0000 7f6283c7f740 1 client.15186 shutdown 2024-09-21T04:03:20.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.854+0000 7f6283c7f740 1 -- 172.21.15.64:0/1202039160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b6338e0 msgr2=0x55a79ba41880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:20.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.854+0000 7f6283c7f740 1 --2- 172.21.15.64:0/1202039160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b6338e0 0x55a79ba41880 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6270005e80 tx=0x7f627000abf0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.854+0000 7f6283c7f740 1 -- 172.21.15.64:0/1202039160 shutdown_connections 2024-09-21T04:03:20.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.854+0000 7f6283c7f740 1 --2- 172.21.15.64:0/1202039160 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55a79b674f60 0x55a79baa7aa0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.854+0000 7f6283c7f740 1 --2- 172.21.15.64:0/1202039160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a79b654280 0x55a79ba3b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.860 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.854+0000 7f6283c7f740 1 --2- 172.21.15.64:0/1202039160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a79b64c0a0 0x55a79ba3ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.854+0000 7f6283c7f740 1 --2- 172.21.15.64:0/1202039160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a79b6338e0 0x55a79ba41880 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.854+0000 7f625e7fc640 0 client.15186 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:03:20.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.854+0000 7f6283c7f740 1 -- 172.21.15.64:0/1202039160 >> 172.21.15.64:0/1202039160 conn(0x55a79b669350 msgr2=0x55a79ba24f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:20.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.854+0000 7f6283c7f740 1 -- 172.21.15.64:0/1202039160 shutdown_connections 2024-09-21T04:03:20.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.854+0000 7f6283c7f740 1 -- 172.21.15.64:0/1202039160 wait complete. 2024-09-21T04:03:20.861 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.854+0000 7f6283c7f740 20 client.15186 trim_cache size 0 max 16384 2024-09-21T04:03:20.861 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:03:20.861 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "e7c927ba-77cc-11ef-baf5-efdc52797490", 2024-09-21T04:03:20.861 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 2, 2024-09-21T04:03:20.861 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24661, 2024-09-21T04:03:20.862 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-09-21T04:03:20.862 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-09-21T04:03:20.862 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-09-21T04:03:20.862 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 69.061989917000005, 2024-09-21T04:03:20.862 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 24, 2024-09-21T04:03:20.862 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 75, 2024-09-21T04:03:20.862 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 73, 2024-09-21T04:03:20.862 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 96.684022424000005 2024-09-21T04:03:20.862 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:03:20.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.855+0000 7f627d70b640 1 -- 172.21.15.64:0/3334138686 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f62640b65f0 msgr2=0x7f62640b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:20.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.855+0000 7f627d70b640 1 --2- 172.21.15.64:0/3334138686 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f62640b65f0 0x7f62640b8b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6260000c00 tx=0x7f6260031400 comp rx=0 tx=0).stop 2024-09-21T04:03:20.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.855+0000 7f627d70b640 1 -- 172.21.15.64:0/3334138686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6278108d20 msgr2=0x7f62781000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:20.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.855+0000 7f627d70b640 1 --2- 172.21.15.64:0/3334138686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6278108d20 0x7f62781000f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f626800eaa0 tx=0x7f626800eea0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.855+0000 7f627d70b640 1 -- 172.21.15.64:0/3334138686 shutdown_connections 2024-09-21T04:03:20.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.855+0000 7f627d70b640 1 --2- 172.21.15.64:0/3334138686 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f62640b65f0 0x7f62640b8b00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.855+0000 7f627d70b640 1 --2- 172.21.15.64:0/3334138686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6278108d20 0x7f62781000f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.855+0000 7f627d70b640 1 --2- 172.21.15.64:0/3334138686 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62781083d0 0x7f62780ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.855+0000 7f627d70b640 1 --2- 172.21.15.64:0/3334138686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6278102ae0 0x7f62780ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:20.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.855+0000 7f627d70b640 1 -- 172.21.15.64:0/3334138686 >> 172.21.15.64:0/3334138686 conn(0x7f6278079760 msgr2=0x7f627806bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:20.863 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.855+0000 7f627d70b640 1 -- 172.21.15.64:0/3334138686 shutdown_connections 2024-09-21T04:03:20.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:20.855+0000 7f627d70b640 1 -- 172.21.15.64:0/3334138686 wait complete. 2024-09-21T04:03:21.655 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T04:03:21.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:21 smithi131 ceph-mon[35193]: pgmap v299: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:03:21.923 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:21.926 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:21 smithi064 ceph-mon[31180]: pgmap v299: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:03:22.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:21 smithi119 ceph-mon[34811]: pgmap v299: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:03:22.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.461+0000 7f718cdf4640 1 -- 172.21.15.64:0/2848710118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7188102a60 msgr2=0x7f7188102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:22.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.461+0000 7f718cdf4640 1 --2- 172.21.15.64:0/2848710118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7188102a60 0x7f7188102e70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7178009900 tx=0x7f717802a410 comp rx=0 tx=0).stop 2024-09-21T04:03:22.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.462+0000 7f718cdf4640 1 -- 172.21.15.64:0/2848710118 shutdown_connections 2024-09-21T04:03:22.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.462+0000 7f718cdf4640 1 --2- 172.21.15.64:0/2848710118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7188108cf0 0x7f718810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.462+0000 7f718cdf4640 1 --2- 172.21.15.64:0/2848710118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71881083a0 0x7f71881087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.462+0000 7f718cdf4640 1 --2- 172.21.15.64:0/2848710118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7188102a60 0x7f7188102e70 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.462+0000 7f718cdf4640 1 -- 172.21.15.64:0/2848710118 >> 172.21.15.64:0/2848710118 conn(0x7f7188079730 msgr2=0x7f7188079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:22.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.462+0000 7f718cdf4640 1 -- 172.21.15.64:0/2848710118 shutdown_connections 2024-09-21T04:03:22.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.462+0000 7f718cdf4640 1 -- 172.21.15.64:0/2848710118 wait complete. 2024-09-21T04:03:22.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.463+0000 7f718cdf4640 1 Processor -- start 2024-09-21T04:03:22.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.463+0000 7f718cdf4640 1 -- start start 2024-09-21T04:03:22.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.463+0000 7f718cdf4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7188102a60 0x7f718819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:22.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.463+0000 7f718cdf4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71881083a0 0x7f718819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:22.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.463+0000 7f718cdf4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7188108cf0 0x7f718819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:22.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.463+0000 7f718cdf4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7188112960 con 0x7f71881083a0 2024-09-21T04:03:22.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.463+0000 7f718cdf4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f71881127e0 con 0x7f7188102a60 2024-09-21T04:03:22.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.463+0000 7f718cdf4640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7188112ae0 con 0x7f7188108cf0 2024-09-21T04:03:22.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.463+0000 7f7185d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71881083a0 0x7f718819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:22.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.463+0000 7f7185d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71881083a0 0x7f718819aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50168/0 (socket says 172.21.15.64:50168) 2024-09-21T04:03:22.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.463+0000 7f7185d9b640 1 -- 172.21.15.64:0/3283974604 learned_addr learned my addr 172.21.15.64:0/3283974604 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:22.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.464+0000 7f718659c640 1 --2- 172.21.15.64:0/3283974604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7188102a60 0x7f718819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:22.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.464+0000 7f7186d9d640 1 --2- 172.21.15.64:0/3283974604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7188108cf0 0x7f718819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.464+0000 7f7185d9b640 1 -- 172.21.15.64:0/3283974604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7188108cf0 msgr2=0x7f718819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.464+0000 7f7185d9b640 1 --2- 172.21.15.64:0/3283974604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7188108cf0 0x7f718819fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.464+0000 7f7185d9b640 1 -- 172.21.15.64:0/3283974604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7188102a60 msgr2=0x7f718819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.464+0000 7f7185d9b640 1 --2- 172.21.15.64:0/3283974604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7188102a60 0x7f718819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.464+0000 7f7185d9b640 1 -- 172.21.15.64:0/3283974604 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71881a03d0 con 0x7f71881083a0 2024-09-21T04:03:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.464+0000 7f7185d9b640 1 --2- 172.21.15.64:0/3283974604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71881083a0 0x7f718819aa00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f717c00c970 tx=0x7f717c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.464+0000 7f71777fe640 1 -- 172.21.15.64:0/3283974604 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717c007bb0 con 0x7f71881083a0 2024-09-21T04:03:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.464+0000 7f71777fe640 1 -- 172.21.15.64:0/3283974604 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f717c007d50 con 0x7f71881083a0 2024-09-21T04:03:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.464+0000 7f718cdf4640 1 -- 172.21.15.64:0/3283974604 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71881a06c0 con 0x7f71881083a0 2024-09-21T04:03:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.464+0000 7f71777fe640 1 -- 172.21.15.64:0/3283974604 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717c00f450 con 0x7f71881083a0 2024-09-21T04:03:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.465+0000 7f718cdf4640 1 -- 172.21.15.64:0/3283974604 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71881a6e20 con 0x7f71881083a0 2024-09-21T04:03:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.467+0000 7f7193368740 1 Processor -- start 2024-09-21T04:03:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.467+0000 7f7193368740 1 -- start start 2024-09-21T04:03:22.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.468+0000 7f7193368740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644d7fa5f60 0x5644d834bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.468+0000 7f7193368740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644d7f7d0a0 0x5644d834c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.468+0000 7f7193368740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644d7f85280 0x5644d834c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.468+0000 7f7193368740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5644d82c3c10 con 0x5644d7f85280 2024-09-21T04:03:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.468+0000 7f7193368740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5644d82b3040 con 0x5644d7f7d0a0 2024-09-21T04:03:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.468+0000 7f7193368740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5644d82b6e50 con 0x5644d7fa5f60 2024-09-21T04:03:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.468+0000 7f71777fe640 1 -- 172.21.15.64:0/3283974604 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f717c00f5f0 con 0x7f71881083a0 2024-09-21T04:03:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.468+0000 7f7186d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644d7fa5f60 0x5644d834bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.468+0000 7f718659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644d7f7d0a0 0x5644d834c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.468+0000 7f7186d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644d7fa5f60 0x5644d834bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:35572/0 (socket says 172.21.15.64:35572) 2024-09-21T04:03:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.468+0000 7f7186d9d640 1 -- 172.21.15.64:0/853488608 learned_addr learned my addr 172.21.15.64:0/853488608 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.468+0000 7f7185d9b640 1 --2- 172.21.15.64:0/853488608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644d7f85280 0x5644d834c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.468+0000 7f7186d9d640 1 -- 172.21.15.64:0/853488608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644d7f7d0a0 msgr2=0x5644d834c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:22.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.468+0000 7f7186d9d640 1 --2- 172.21.15.64:0/853488608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644d7f7d0a0 0x5644d834c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.468+0000 7f7186d9d640 1 -- 172.21.15.64:0/853488608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644d7f85280 msgr2=0x5644d834c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:22.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.468+0000 7f7186d9d640 1 --2- 172.21.15.64:0/853488608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644d7f85280 0x5644d834c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.468+0000 7f7186d9d640 1 -- 172.21.15.64:0/853488608 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644d8011540 con 0x5644d7fa5f60 2024-09-21T04:03:22.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.469+0000 7f7186d9d640 1 --2- 172.21.15.64:0/853488608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644d7fa5f60 0x5644d834bf00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f71800039f0 tx=0x7f7180003eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=657fdb13c411fe7e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:22.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.469+0000 7f7174ff9640 1 -- 172.21.15.64:0/853488608 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718001a470 con 0x5644d7fa5f60 2024-09-21T04:03:22.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.469+0000 7f7174ff9640 1 -- 172.21.15.64:0/853488608 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7180015070 con 0x5644d7fa5f60 2024-09-21T04:03:22.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.469+0000 7f7174ff9640 1 -- 172.21.15.64:0/853488608 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71800107e0 con 0x5644d7fa5f60 2024-09-21T04:03:22.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.469+0000 7f71777fe640 1 --2- 172.21.15.64:0/3283974604 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f71700b6670 0x7f71700b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:22.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.469+0000 7f718659c640 1 --2- 172.21.15.64:0/3283974604 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f71700b6670 0x7f71700b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:22.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.469+0000 7f71777fe640 1 -- 172.21.15.64:0/3283974604 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f717c0e7530 con 0x7f71881083a0 2024-09-21T04:03:22.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.470+0000 7f7193368740 1 -- 172.21.15.64:0/853488608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644d7fa5f60 msgr2=0x5644d834bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:22.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.470+0000 7f718659c640 1 --2- 172.21.15.64:0/3283974604 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f71700b6670 0x7f71700b8b80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7178031440 tx=0x7f7178005db0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:22.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.470+0000 7f7193368740 1 --2- 172.21.15.64:0/853488608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644d7fa5f60 0x5644d834bf00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f71800039f0 tx=0x7f7180003eb0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.470+0000 7f7193368740 1 -- 172.21.15.64:0/853488608 shutdown_connections 2024-09-21T04:03:22.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.470+0000 7f7193368740 1 --2- 172.21.15.64:0/853488608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644d7f85280 0x5644d834c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.470+0000 7f7193368740 1 --2- 172.21.15.64:0/853488608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644d7f7d0a0 0x5644d834c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.470+0000 7f7193368740 1 --2- 172.21.15.64:0/853488608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644d7fa5f60 0x5644d834bf00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.470+0000 7f7193368740 1 -- 172.21.15.64:0/853488608 >> 172.21.15.64:0/853488608 conn(0x5644d7f9a350 msgr2=0x5644d8349df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:22.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.470+0000 7f7193368740 1 -- 172.21.15.64:0/853488608 shutdown_connections 2024-09-21T04:03:22.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.470+0000 7f7193368740 1 -- 172.21.15.64:0/853488608 wait complete. 2024-09-21T04:03:22.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.471+0000 7f7193368740 1 -- start start 2024-09-21T04:03:22.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.471+0000 7f7193368740 1 Processor -- start 2024-09-21T04:03:22.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.471+0000 7f7193368740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644d7f7d0a0 0x5644d83579c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:22.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.471+0000 7f7193368740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644d7f85280 0x5644d8357f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:22.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.471+0000 7f7193368740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644d7fa5f60 0x5644d8363a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:22.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.471+0000 7f7193368740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5644d81145f0 con 0x5644d7f85280 2024-09-21T04:03:22.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.471+0000 7f7193368740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5644d824e010 con 0x5644d7f7d0a0 2024-09-21T04:03:22.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.471+0000 7f7193368740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5644d835a6e0 con 0x5644d7fa5f60 2024-09-21T04:03:22.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.471+0000 7f7193368740 10 client.0 _ll_register_callbacks cb 0x5644d82ad340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:03:22.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.471+0000 7f7185d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644d7f85280 0x5644d8357f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:22.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.471+0000 7f7185d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644d7f85280 0x5644d8357f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50182/0 (socket says 172.21.15.64:50182) 2024-09-21T04:03:22.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.471+0000 7f7185d9b640 1 -- 172.21.15.64:0/2274519984 learned_addr learned my addr 172.21.15.64:0/2274519984 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:22.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.471+0000 7f718659c640 1 --2- 172.21.15.64:0/2274519984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644d7f7d0a0 0x5644d83579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:22.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.472+0000 7f7186d9d640 1 --2- 172.21.15.64:0/2274519984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644d7fa5f60 0x5644d8363a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:22.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.472+0000 7f7185d9b640 1 -- 172.21.15.64:0/2274519984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644d7fa5f60 msgr2=0x5644d8363a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:22.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.472+0000 7f7185d9b640 1 --2- 172.21.15.64:0/2274519984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644d7fa5f60 0x5644d8363a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.472+0000 7f7185d9b640 1 -- 172.21.15.64:0/2274519984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644d7f7d0a0 msgr2=0x5644d83579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:22.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.472+0000 7f7185d9b640 1 --2- 172.21.15.64:0/2274519984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644d7f7d0a0 0x5644d83579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.472+0000 7f7185d9b640 1 -- 172.21.15.64:0/2274519984 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644d827ba00 con 0x5644d7f85280 2024-09-21T04:03:22.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.472+0000 7f7185d9b640 1 --2- 172.21.15.64:0/2274519984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644d7f85280 0x5644d8357f00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f717c0ad210 tx=0x7f717c0a9470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59dd8d1c596ea30 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:22.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.472+0000 7f716dffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:03:22.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.472+0000 7f716dffb640 1 -- 172.21.15.64:0/2274519984 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717c01d040 con 0x5644d7f85280 2024-09-21T04:03:22.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.472+0000 7f716dffb640 1 -- 172.21.15.64:0/2274519984 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f717c0e70d0 con 0x5644d7f85280 2024-09-21T04:03:22.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.472+0000 7f7193368740 1 -- 172.21.15.64:0/2274519984 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7180015070 con 0x5644d7f85280 2024-09-21T04:03:22.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.472+0000 7f716dffb640 1 -- 172.21.15.64:0/2274519984 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f717c0ad020 con 0x5644d7f85280 2024-09-21T04:03:22.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.472+0000 7f7193368740 10 client.15198 fetch_fsmap learned FSMap version 24 2024-09-21T04:03:22.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.472+0000 7f7193368740 1 -- 172.21.15.64:0/2274519984 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5644d8366630 con 0x5644d7f85280 2024-09-21T04:03:22.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.472+0000 7f716dffb640 1 -- 172.21.15.64:0/2274519984 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7180015070 con 0x5644d7f85280 2024-09-21T04:03:22.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.472+0000 7f716dffb640 1 -- 172.21.15.64:0/2274519984 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f717c0e9b00 con 0x5644d7f85280 2024-09-21T04:03:22.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.473+0000 7f7193368740 10 client.15198 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T04:03:22.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.473+0000 7f7193368740 10 client.15198 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T04:03:22.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.473+0000 7f7193368740 20 client.15198 populate_metadata read hostname 'smithi064' 2024-09-21T04:03:22.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.473+0000 7f7193368740 1 --2- 172.21.15.64:0/2274519984 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5644d7f648e0 0x5644d836a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:22.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.473+0000 7f7193368740 4 client.15198 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:03:22.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.473+0000 7f7193368740 1 -- 172.21.15.64:0/2274519984 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644d82c5b60 con 0x5644d7f648e0 2024-09-21T04:03:22.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.473+0000 7f718659c640 1 --2- 172.21.15.64:0/2274519984 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5644d7f648e0 0x5644d836a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:22.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.473+0000 7f716dffb640 10 client.15198 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:03:22.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.473+0000 7f718659c640 1 --2- 172.21.15.64:0/2274519984 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5644d7f648e0 0x5644d836a0b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2947b36f9f8c9a5f server_cookie=84c7b87815230e6e in_seq=0 out_seq=0 2024-09-21T04:03:22.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.474+0000 7f716dffb640 1 -- 172.21.15.64:0/2274519984 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5644d82c5b60 con 0x5644d7f648e0 2024-09-21T04:03:22.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.474+0000 7f716dffb640 10 client.15198 handle_command_reply: tid=0 2024-09-21T04:03:22.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.474+0000 7f7193368740 1 client.15198 shutdown 2024-09-21T04:03:22.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.474+0000 7f7193368740 1 -- 172.21.15.64:0/2274519984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644d7f85280 msgr2=0x5644d8357f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:22.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.474+0000 7f7193368740 1 --2- 172.21.15.64:0/2274519984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644d7f85280 0x5644d8357f00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f717c0ad210 tx=0x7f717c0a9470 comp rx=0 tx=0).stop 2024-09-21T04:03:22.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.474+0000 7f7193368740 1 -- 172.21.15.64:0/2274519984 shutdown_connections 2024-09-21T04:03:22.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.474+0000 7f7193368740 1 --2- 172.21.15.64:0/2274519984 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5644d7f648e0 0x5644d836a0b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.474+0000 7f7193368740 1 --2- 172.21.15.64:0/2274519984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644d7fa5f60 0x5644d8363a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.474+0000 7f7193368740 1 --2- 172.21.15.64:0/2274519984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644d7f85280 0x5644d8357f00 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.474+0000 7f716dffb640 0 client.15198 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:03:22.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.474+0000 7f7193368740 1 --2- 172.21.15.64:0/2274519984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644d7f7d0a0 0x5644d83579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.474+0000 7f7193368740 1 -- 172.21.15.64:0/2274519984 >> 172.21.15.64:0/2274519984 conn(0x5644d7f9a350 msgr2=0x5644d834f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:22.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.475+0000 7f7193368740 1 -- 172.21.15.64:0/2274519984 shutdown_connections 2024-09-21T04:03:22.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.475+0000 7f7193368740 1 -- 172.21.15.64:0/2274519984 wait complete. 2024-09-21T04:03:22.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.475+0000 7f7193368740 20 client.15198 trim_cache size 0 max 16384 2024-09-21T04:03:22.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.493+0000 7f7193368740 1 Processor -- start 2024-09-21T04:03:22.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.493+0000 7f7193368740 1 -- start start 2024-09-21T04:03:22.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.494+0000 7f7193368740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644d7f7d0a0 0x5644d8363970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:22.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.494+0000 7f7193368740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644d7f85280 0x5644d8363eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:22.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.494+0000 7f7193368740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644d7fa5f60 0x5644d8349600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:22.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.494+0000 7f7193368740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5644d83683f0 con 0x5644d7f85280 2024-09-21T04:03:22.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.494+0000 7f7193368740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5644d8368ac0 con 0x5644d7fa5f60 2024-09-21T04:03:22.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.494+0000 7f7193368740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5644d8369190 con 0x5644d7f7d0a0 2024-09-21T04:03:22.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.494+0000 7f7185d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644d7f85280 0x5644d8363eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:22.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.494+0000 7f7185d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644d7f85280 0x5644d8363eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50190/0 (socket says 172.21.15.64:50190) 2024-09-21T04:03:22.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.494+0000 7f7185d9b640 1 -- 172.21.15.64:0/2324749521 learned_addr learned my addr 172.21.15.64:0/2324749521 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:22.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.494+0000 7f7186d9d640 1 --2- 172.21.15.64:0/2324749521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644d7fa5f60 0x5644d8349600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:22.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.494+0000 7f718659c640 1 --2- 172.21.15.64:0/2324749521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644d7f7d0a0 0x5644d8363970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:22.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.494+0000 7f7185d9b640 1 -- 172.21.15.64:0/2324749521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644d7f7d0a0 msgr2=0x5644d8363970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:22.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.494+0000 7f7185d9b640 1 --2- 172.21.15.64:0/2324749521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644d7f7d0a0 0x5644d8363970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.494+0000 7f7185d9b640 1 -- 172.21.15.64:0/2324749521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644d7fa5f60 msgr2=0x5644d8349600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:22.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.494+0000 7f7185d9b640 1 --2- 172.21.15.64:0/2324749521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644d7fa5f60 0x5644d8349600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.494+0000 7f7185d9b640 1 -- 172.21.15.64:0/2324749521 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644d8358a80 con 0x5644d7f85280 2024-09-21T04:03:22.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.494+0000 7f718659c640 1 --2- 172.21.15.64:0/2324749521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644d7f7d0a0 0x5644d8363970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:03:22.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.494+0000 7f7185d9b640 1 --2- 172.21.15.64:0/2324749521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644d7f85280 0x5644d8363eb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f717c0a9a90 tx=0x7f717c0a9be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f5b29593697e7c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:22.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.494+0000 7f716ffff640 1 -- 172.21.15.64:0/2324749521 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717c01d040 con 0x5644d7f85280 2024-09-21T04:03:22.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.495+0000 7f716ffff640 1 -- 172.21.15.64:0/2324749521 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f717c0b1f40 con 0x5644d7f85280 2024-09-21T04:03:22.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.495+0000 7f716ffff640 1 -- 172.21.15.64:0/2324749521 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717c0e7330 con 0x5644d7f85280 2024-09-21T04:03:22.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.495+0000 7f7193368740 1 -- 172.21.15.64:0/2324749521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644d7f85280 msgr2=0x5644d8363eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:22.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.495+0000 7f7193368740 1 --2- 172.21.15.64:0/2324749521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644d7f85280 0x5644d8363eb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f717c0a9a90 tx=0x7f717c0a9be0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.495+0000 7f7193368740 1 -- 172.21.15.64:0/2324749521 shutdown_connections 2024-09-21T04:03:22.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.496+0000 7f7193368740 1 --2- 172.21.15.64:0/2324749521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644d7fa5f60 0x5644d8349600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.496+0000 7f7193368740 1 --2- 172.21.15.64:0/2324749521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644d7f85280 0x5644d8363eb0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.496+0000 7f7193368740 1 --2- 172.21.15.64:0/2324749521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644d7f7d0a0 0x5644d8363970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.496+0000 7f7193368740 1 -- 172.21.15.64:0/2324749521 >> 172.21.15.64:0/2324749521 conn(0x5644d7f9a350 msgr2=0x5644d8356040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:22.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.496+0000 7f7193368740 1 -- 172.21.15.64:0/2324749521 shutdown_connections 2024-09-21T04:03:22.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.496+0000 7f7193368740 1 -- 172.21.15.64:0/2324749521 wait complete. 2024-09-21T04:03:22.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.496+0000 7f7193368740 1 -- start start 2024-09-21T04:03:22.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.496+0000 7f7193368740 1 Processor -- start 2024-09-21T04:03:22.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.496+0000 7f7193368740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644d7f7d0a0 0x5644d8370500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:22.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.497+0000 7f7193368740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644d7f85280 0x5644d8370a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:22.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.497+0000 7f7193368740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644d7fa5f60 0x5644d8370f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:22.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.497+0000 7f7193368740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5644d8369f30 con 0x5644d7fa5f60 2024-09-21T04:03:22.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.497+0000 7f7193368740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5644d8369860 con 0x5644d7f7d0a0 2024-09-21T04:03:22.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.497+0000 7f7193368740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5644d836a600 con 0x5644d7f85280 2024-09-21T04:03:22.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.497+0000 7f7193368740 10 client.0 _ll_register_callbacks cb 0x7f717c0058a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:03:22.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.497+0000 7f7186d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644d7fa5f60 0x5644d8370f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:22.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.497+0000 7f718659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644d7f7d0a0 0x5644d8370500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:22.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.497+0000 7f7186d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644d7fa5f60 0x5644d8370f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50196/0 (socket says 172.21.15.64:50196) 2024-09-21T04:03:22.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.497+0000 7f7186d9d640 1 -- 172.21.15.64:0/701676365 learned_addr learned my addr 172.21.15.64:0/701676365 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:22.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.497+0000 7f718659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644d7f7d0a0 0x5644d8370500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:53288/0 (socket says 172.21.15.64:53288) 2024-09-21T04:03:22.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.497+0000 7f7185d9b640 1 --2- 172.21.15.64:0/701676365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644d7f85280 0x5644d8370a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:22.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.497+0000 7f7186d9d640 1 -- 172.21.15.64:0/701676365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644d7f85280 msgr2=0x5644d8370a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:22.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.497+0000 7f7186d9d640 1 --2- 172.21.15.64:0/701676365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644d7f85280 0x5644d8370a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.497+0000 7f7186d9d640 1 -- 172.21.15.64:0/701676365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644d7f7d0a0 msgr2=0x5644d8370500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:22.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.497+0000 7f7186d9d640 1 --2- 172.21.15.64:0/701676365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644d7f7d0a0 0x5644d8370500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.497+0000 7f7186d9d640 1 -- 172.21.15.64:0/701676365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644d83d8760 con 0x5644d7fa5f60 2024-09-21T04:03:22.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.497+0000 7f718659c640 1 --2- 172.21.15.64:0/701676365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644d7f7d0a0 0x5644d8370500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:03:22.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.497+0000 7f7186d9d640 1 --2- 172.21.15.64:0/701676365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644d7fa5f60 0x5644d8370f80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7180019000 tx=0x7f7180005980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68841e26519fdc6f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:22.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.497+0000 7f716dffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:03:22.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.497+0000 7f716dffb640 1 -- 172.21.15.64:0/701676365 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718001a680 con 0x5644d7fa5f60 2024-09-21T04:03:22.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.498+0000 7f716dffb640 1 -- 172.21.15.64:0/701676365 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f718001a820 con 0x5644d7fa5f60 2024-09-21T04:03:22.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.498+0000 7f7193368740 1 -- 172.21.15.64:0/701676365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f717c0b1f40 con 0x5644d7fa5f60 2024-09-21T04:03:22.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.498+0000 7f716dffb640 1 -- 172.21.15.64:0/701676365 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f718001aa20 con 0x5644d7fa5f60 2024-09-21T04:03:22.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.498+0000 7f7193368740 10 client.15204 fetch_fsmap learned FSMap version 24 2024-09-21T04:03:22.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.498+0000 7f7193368740 1 -- 172.21.15.64:0/701676365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5644d83d99c0 con 0x5644d7fa5f60 2024-09-21T04:03:22.503 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.498+0000 7f716dffb640 1 -- 172.21.15.64:0/701676365 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f717c0b1f40 con 0x5644d7fa5f60 2024-09-21T04:03:22.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.498+0000 7f716dffb640 1 -- 172.21.15.64:0/701676365 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f718002f020 con 0x5644d7fa5f60 2024-09-21T04:03:22.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.498+0000 7f7193368740 10 client.15204 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T04:03:22.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.498+0000 7f7193368740 10 client.15204 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T04:03:22.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.498+0000 7f7193368740 20 client.15204 populate_metadata read hostname 'smithi064' 2024-09-21T04:03:22.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.498+0000 7f7193368740 1 --2- 172.21.15.64:0/701676365 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5644d7f39c90 0x5644d83dcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:22.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.498+0000 7f7193368740 4 client.15204 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T04:03:22.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.498+0000 7f7193368740 1 -- 172.21.15.64:0/701676365 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "status"}) -- 0x5644d82c1610 con 0x5644d7f39c90 2024-09-21T04:03:22.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.499+0000 7f718659c640 1 --2- 172.21.15.64:0/701676365 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5644d7f39c90 0x5644d83dcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:22.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.499+0000 7f718659c640 1 --2- 172.21.15.64:0/701676365 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5644d7f39c90 0x5644d83dcf50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1a7ee20e965b622b server_cookie=11db2de351d7023b in_seq=0 out_seq=0 2024-09-21T04:03:22.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.499+0000 7f716dffb640 10 client.15204 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:03:22.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.499+0000 7f716dffb640 1 -- 172.21.15.64:0/701676365 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5644d82c1610 con 0x5644d7f39c90 2024-09-21T04:03:22.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.499+0000 7f716dffb640 10 client.15204 handle_command_reply: tid=0 2024-09-21T04:03:22.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.499+0000 7f7193368740 1 client.15204 shutdown 2024-09-21T04:03:22.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.500+0000 7f7193368740 1 -- 172.21.15.64:0/701676365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644d7fa5f60 msgr2=0x5644d8370f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:22.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.500+0000 7f7193368740 1 --2- 172.21.15.64:0/701676365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644d7fa5f60 0x5644d8370f80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7180019000 tx=0x7f7180005980 comp rx=0 tx=0).stop 2024-09-21T04:03:22.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.500+0000 7f7193368740 1 -- 172.21.15.64:0/701676365 shutdown_connections 2024-09-21T04:03:22.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.500+0000 7f7193368740 1 --2- 172.21.15.64:0/701676365 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5644d7f39c90 0x5644d83dcf50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.500+0000 7f7193368740 1 --2- 172.21.15.64:0/701676365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5644d7fa5f60 0x5644d8370f80 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.500+0000 7f7193368740 1 --2- 172.21.15.64:0/701676365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5644d7f85280 0x5644d8370a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.500+0000 7f716dffb640 0 client.15204 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:03:22.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.500+0000 7f7193368740 1 --2- 172.21.15.64:0/701676365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5644d7f7d0a0 0x5644d8370500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.500+0000 7f7193368740 1 -- 172.21.15.64:0/701676365 >> 172.21.15.64:0/701676365 conn(0x5644d7f9a350 msgr2=0x5644d83483b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:22.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.500+0000 7f7193368740 1 -- 172.21.15.64:0/701676365 shutdown_connections 2024-09-21T04:03:22.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.500+0000 7f7193368740 1 -- 172.21.15.64:0/701676365 wait complete. 2024-09-21T04:03:22.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.500+0000 7f7193368740 20 client.15204 trim_cache size 0 max 16384 2024-09-21T04:03:22.506 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:03:22.507 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "e7c927ba-77cc-11ef-baf5-efdc52797490", 2024-09-21T04:03:22.507 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 1, 2024-09-21T04:03:22.507 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24680, 2024-09-21T04:03:22.507 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-09-21T04:03:22.507 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-09-21T04:03:22.507 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-09-21T04:03:22.507 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 71.714120343000005, 2024-09-21T04:03:22.507 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 24, 2024-09-21T04:03:22.507 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 75, 2024-09-21T04:03:22.507 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 73, 2024-09-21T04:03:22.507 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 100.83311928000001 2024-09-21T04:03:22.507 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:03:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.500+0000 7f718cdf4640 1 -- 172.21.15.64:0/3283974604 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f71700b6670 msgr2=0x7f71700b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.500+0000 7f718cdf4640 1 --2- 172.21.15.64:0/3283974604 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f71700b6670 0x7f71700b8b80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7178031440 tx=0x7f7178005db0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.500+0000 7f718cdf4640 1 -- 172.21.15.64:0/3283974604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71881083a0 msgr2=0x7f718819aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.500+0000 7f718cdf4640 1 --2- 172.21.15.64:0/3283974604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71881083a0 0x7f718819aa00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f717c00c970 tx=0x7f717c00ce30 comp rx=0 tx=0).stop 2024-09-21T04:03:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.501+0000 7f718cdf4640 1 -- 172.21.15.64:0/3283974604 shutdown_connections 2024-09-21T04:03:22.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.501+0000 7f718cdf4640 1 --2- 172.21.15.64:0/3283974604 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f71700b6670 0x7f71700b8b80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.501+0000 7f718cdf4640 1 --2- 172.21.15.64:0/3283974604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7188108cf0 0x7f718819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.501+0000 7f718cdf4640 1 --2- 172.21.15.64:0/3283974604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f71881083a0 0x7f718819aa00 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.501+0000 7f718cdf4640 1 --2- 172.21.15.64:0/3283974604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7188102a60 0x7f718819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.501+0000 7f718cdf4640 1 -- 172.21.15.64:0/3283974604 >> 172.21.15.64:0/3283974604 conn(0x7f7188079730 msgr2=0x7f7188101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.501+0000 7f718cdf4640 1 -- 172.21.15.64:0/3283974604 shutdown_connections 2024-09-21T04:03:22.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:22.501+0000 7f718cdf4640 1 -- 172.21.15.64:0/3283974604 wait complete. 2024-09-21T04:03:23.196 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T04:03:23.457 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:23.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:23 smithi131 ceph-mon[35193]: pgmap v300: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T04:03:23.861 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:23 smithi064 ceph-mon[31180]: pgmap v300: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T04:03:23.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.952+0000 7fc50a888640 1 -- 172.21.15.64:0/4249193385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc504104a30 msgr2=0x7fc504104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:23.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.952+0000 7fc50a888640 1 --2- 172.21.15.64:0/4249193385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc504104a30 0x7fc504104e40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4009900 tx=0x7fc4f402a410 comp rx=0 tx=0).stop 2024-09-21T04:03:23.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.952+0000 7fc50a888640 1 -- 172.21.15.64:0/4249193385 shutdown_connections 2024-09-21T04:03:23.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.952+0000 7fc50a888640 1 --2- 172.21.15.64:0/4249193385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc50410de60 0x7fc504110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.952+0000 7fc50a888640 1 --2- 172.21.15.64:0/4249193385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc504105380 0x7fc50410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.952+0000 7fc50a888640 1 --2- 172.21.15.64:0/4249193385 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc504104a30 0x7fc504104e40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.952+0000 7fc50a888640 1 -- 172.21.15.64:0/4249193385 >> 172.21.15.64:0/4249193385 conn(0x7fc504079760 msgr2=0x7fc504079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:23.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.952+0000 7fc50a888640 1 -- 172.21.15.64:0/4249193385 shutdown_connections 2024-09-21T04:03:23.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.952+0000 7fc50a888640 1 -- 172.21.15.64:0/4249193385 wait complete. 2024-09-21T04:03:23.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.953+0000 7fc50a888640 1 Processor -- start 2024-09-21T04:03:23.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.953+0000 7fc50a888640 1 -- start start 2024-09-21T04:03:23.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.954+0000 7fc50a888640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc504104a30 0x7fc50419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:23.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.954+0000 7fc50a888640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc504105380 0x7fc50419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:23.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.954+0000 7fc50a888640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc50410de60 0x7fc50419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:23.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.954+0000 7fc50a888640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc504113ac0 con 0x7fc50410de60 2024-09-21T04:03:23.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.954+0000 7fc50a888640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc504113940 con 0x7fc504104a30 2024-09-21T04:03:23.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.954+0000 7fc50a888640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc504113c40 con 0x7fc504105380 2024-09-21T04:03:23.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.954+0000 7fc508e25640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc50410de60 0x7fc50419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:23.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.954+0000 7fc508e25640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc50410de60 0x7fc50419fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50224/0 (socket says 172.21.15.64:50224) 2024-09-21T04:03:23.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.954+0000 7fc508e25640 1 -- 172.21.15.64:0/3692614425 learned_addr learned my addr 172.21.15.64:0/3692614425 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:23.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.954+0000 7fc503fff640 1 --2- 172.21.15.64:0/3692614425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc504104a30 0x7fc50419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:23.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.954+0000 7fc5037fe640 1 --2- 172.21.15.64:0/3692614425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc504105380 0x7fc50419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:23.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.954+0000 7fc508e25640 1 -- 172.21.15.64:0/3692614425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc504105380 msgr2=0x7fc50419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:23.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.954+0000 7fc508e25640 1 --2- 172.21.15.64:0/3692614425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc504105380 0x7fc50419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.954+0000 7fc508e25640 1 -- 172.21.15.64:0/3692614425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc504104a30 msgr2=0x7fc50419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:23.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.954+0000 7fc508e25640 1 --2- 172.21.15.64:0/3692614425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc504104a30 0x7fc50419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.954+0000 7fc508e25640 1 -- 172.21.15.64:0/3692614425 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5041a0400 con 0x7fc50410de60 2024-09-21T04:03:23.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.954+0000 7fc508e25640 1 --2- 172.21.15.64:0/3692614425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc50410de60 0x7fc50419fcd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00eaa0 tx=0x7fc4fc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:23.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.954+0000 7fc5017fa640 1 -- 172.21.15.64:0/3692614425 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc00cdb0 con 0x7fc50410de60 2024-09-21T04:03:23.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.955+0000 7fc50a888640 1 -- 172.21.15.64:0/3692614425 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5041a06f0 con 0x7fc50410de60 2024-09-21T04:03:23.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.955+0000 7fc5017fa640 1 -- 172.21.15.64:0/3692614425 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4fc005020 con 0x7fc50410de60 2024-09-21T04:03:23.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.955+0000 7fc5017fa640 1 -- 172.21.15.64:0/3692614425 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc0116d0 con 0x7fc50410de60 2024-09-21T04:03:23.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.955+0000 7fc50a888640 1 -- 172.21.15.64:0/3692614425 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5041a6e50 con 0x7fc50410de60 2024-09-21T04:03:23.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.957+0000 7fc510dfc740 1 Processor -- start 2024-09-21T04:03:23.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.958+0000 7fc5017fa640 1 -- 172.21.15.64:0/3692614425 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fc4fc004b20 con 0x7fc50410de60 2024-09-21T04:03:23.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.958+0000 7fc510dfc740 1 -- start start 2024-09-21T04:03:23.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.958+0000 7fc510dfc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94fe14f60 0x55a9501baf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:23.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.958+0000 7fc510dfc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fdec0a0 0x55a9501bb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:23.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.958+0000 7fc510dfc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a94fdf4280 0x55a9501bb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:23.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.958+0000 7fc510dfc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a950132c10 con 0x55a94fdec0a0 2024-09-21T04:03:23.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.958+0000 7fc510dfc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a950122040 con 0x55a94fdf4280 2024-09-21T04:03:23.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.958+0000 7fc510dfc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a950125e50 con 0x55a94fe14f60 2024-09-21T04:03:23.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.958+0000 7fc503fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fdec0a0 0x55a9501bb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:23.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.958+0000 7fc503fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fdec0a0 0x55a9501bb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50236/0 (socket says 172.21.15.64:50236) 2024-09-21T04:03:23.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.958+0000 7fc503fff640 1 -- 172.21.15.64:0/192313679 learned_addr learned my addr 172.21.15.64:0/192313679 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:23.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.958+0000 7fc5037fe640 1 --2- 172.21.15.64:0/192313679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94fe14f60 0x55a9501baf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:23.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.959+0000 7fc5017fa640 1 --2- 172.21.15.64:0/3692614425 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc4ec0b65f0 0x7fc4ec0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:23.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.959+0000 7fc5037fe640 1 --2- 172.21.15.64:0/192313679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a94fdf4280 0x55a9501bb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:23.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.959+0000 7fc503fff640 1 -- 172.21.15.64:0/192313679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94fe14f60 msgr2=0x55a9501baf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:23.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.959+0000 7fc503fff640 1 --2- 172.21.15.64:0/192313679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94fe14f60 0x55a9501baf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.959+0000 7fc503fff640 1 -- 172.21.15.64:0/192313679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a94fdf4280 msgr2=0x55a9501bb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:23.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.959+0000 7fc503fff640 1 --2- 172.21.15.64:0/192313679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a94fdf4280 0x55a9501bb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.959+0000 7fc503fff640 1 -- 172.21.15.64:0/192313679 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a94fe80540 con 0x55a94fdec0a0 2024-09-21T04:03:23.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.959+0000 7fc5037fe640 1 --2- 172.21.15.64:0/192313679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94fe14f60 0x55a9501baf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:03:23.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.959+0000 7fc508e25640 1 --2- 172.21.15.64:0/3692614425 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc4ec0b65f0 0x7fc4ec0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:23.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.959+0000 7fc5017fa640 1 -- 172.21.15.64:0/3692614425 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7fc4fc0e78a0 con 0x7fc50410de60 2024-09-21T04:03:23.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.959+0000 7fc503fff640 1 --2- 172.21.15.64:0/192313679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fdec0a0 0x55a9501bb440 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc50406c8f0 tx=0x7fc4f4032ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71bc6fe5cdbcc765 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:23.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.959+0000 7fc4ea7fc640 1 -- 172.21.15.64:0/192313679 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f403f070 con 0x55a94fdec0a0 2024-09-21T04:03:23.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.959+0000 7fc508e25640 1 --2- 172.21.15.64:0/3692614425 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc4ec0b65f0 0x7fc4ec0b8b00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc0ad760 tx=0x7fc4fc000f40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:23.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.959+0000 7fc4ea7fc640 1 -- 172.21.15.64:0/192313679 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4f4004030 con 0x55a94fdec0a0 2024-09-21T04:03:23.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.959+0000 7fc4ea7fc640 1 -- 172.21.15.64:0/192313679 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4035070 con 0x55a94fdec0a0 2024-09-21T04:03:23.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.960+0000 7fc510dfc740 1 -- 172.21.15.64:0/192313679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fdec0a0 msgr2=0x55a9501bb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:23.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.960+0000 7fc510dfc740 1 --2- 172.21.15.64:0/192313679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fdec0a0 0x55a9501bb440 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc50406c8f0 tx=0x7fc4f4032ba0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.960+0000 7fc510dfc740 1 -- 172.21.15.64:0/192313679 shutdown_connections 2024-09-21T04:03:23.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.960+0000 7fc510dfc740 1 --2- 172.21.15.64:0/192313679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a94fdf4280 0x55a9501bb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.960+0000 7fc510dfc740 1 --2- 172.21.15.64:0/192313679 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fdec0a0 0x55a9501bb440 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.960+0000 7fc510dfc740 1 --2- 172.21.15.64:0/192313679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94fe14f60 0x55a9501baf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.960+0000 7fc510dfc740 1 -- 172.21.15.64:0/192313679 >> 172.21.15.64:0/192313679 conn(0x55a94fe09350 msgr2=0x55a9501b8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:23.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.960+0000 7fc510dfc740 1 -- 172.21.15.64:0/192313679 shutdown_connections 2024-09-21T04:03:23.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.960+0000 7fc510dfc740 1 -- 172.21.15.64:0/192313679 wait complete. 2024-09-21T04:03:23.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.961+0000 7fc510dfc740 1 -- start start 2024-09-21T04:03:23.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.961+0000 7fc510dfc740 1 Processor -- start 2024-09-21T04:03:23.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.961+0000 7fc510dfc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fdec0a0 0x55a9501c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:23.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.961+0000 7fc510dfc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a94fdf4280 0x55a9501c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:23.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.961+0000 7fc510dfc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94fe14f60 0x55a9501d29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:23.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.961+0000 7fc510dfc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a94ff835f0 con 0x55a94fdec0a0 2024-09-21T04:03:23.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.961+0000 7fc510dfc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a9500bd010 con 0x55a94fdf4280 2024-09-21T04:03:23.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.961+0000 7fc510dfc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a9501c9680 con 0x55a94fe14f60 2024-09-21T04:03:23.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.961+0000 7fc510dfc740 10 client.0 _ll_register_callbacks cb 0x55a95011c340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:03:23.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.961+0000 7fc5037fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fdec0a0 0x55a9501c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:23.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.961+0000 7fc5037fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fdec0a0 0x55a9501c6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50250/0 (socket says 172.21.15.64:50250) 2024-09-21T04:03:23.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.961+0000 7fc5037fe640 1 -- 172.21.15.64:0/3351015117 learned_addr learned my addr 172.21.15.64:0/3351015117 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:23.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.961+0000 7fc503fff640 1 --2- 172.21.15.64:0/3351015117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a94fdf4280 0x55a9501c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:23.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.961+0000 7fc5037fe640 1 --2- 172.21.15.64:0/3351015117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94fe14f60 0x55a9501d29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:23.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.962+0000 7fc5037fe640 1 -- 172.21.15.64:0/3351015117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94fe14f60 msgr2=0x55a9501d29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:23.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.962+0000 7fc5037fe640 1 --2- 172.21.15.64:0/3351015117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94fe14f60 0x55a9501d29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.962+0000 7fc5037fe640 1 -- 172.21.15.64:0/3351015117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a94fdf4280 msgr2=0x55a9501c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:23.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.962+0000 7fc5037fe640 1 --2- 172.21.15.64:0/3351015117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a94fdf4280 0x55a9501c6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.962+0000 7fc5037fe640 1 -- 172.21.15.64:0/3351015117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9500eaa00 con 0x55a94fdec0a0 2024-09-21T04:03:23.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.962+0000 7fc503fff640 1 --2- 172.21.15.64:0/3351015117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a94fdf4280 0x55a9501c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:03:23.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.962+0000 7fc5037fe640 1 --2- 172.21.15.64:0/3351015117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fdec0a0 0x55a9501c6aa0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8009de0 tx=0x7fc4f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e01f4736ce1f81 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:23.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.962+0000 7fc4e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:03:23.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.962+0000 7fc4e37fe640 1 -- 172.21.15.64:0/3351015117 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f801b040 con 0x55a94fdec0a0 2024-09-21T04:03:23.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.962+0000 7fc4e37fe640 1 -- 172.21.15.64:0/3351015117 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7fc4f8014170 con 0x55a94fdec0a0 2024-09-21T04:03:23.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.962+0000 7fc510dfc740 1 -- 172.21.15.64:0/3351015117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4f4004030 con 0x55a94fdec0a0 2024-09-21T04:03:23.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.962+0000 7fc4e37fe640 1 -- 172.21.15.64:0/3351015117 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc4f8026050 con 0x55a94fdec0a0 2024-09-21T04:03:23.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.962+0000 7fc510dfc740 10 client.15216 fetch_fsmap learned FSMap version 24 2024-09-21T04:03:23.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.962+0000 7fc510dfc740 1 -- 172.21.15.64:0/3351015117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a9501d56a0 con 0x55a94fdec0a0 2024-09-21T04:03:23.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.962+0000 7fc4e37fe640 1 -- 172.21.15.64:0/3351015117 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4f4004030 con 0x55a94fdec0a0 2024-09-21T04:03:23.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.963+0000 7fc4e37fe640 1 -- 172.21.15.64:0/3351015117 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fc4f800f5e0 con 0x55a94fdec0a0 2024-09-21T04:03:23.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.963+0000 7fc510dfc740 10 client.15216 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T04:03:23.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.963+0000 7fc510dfc740 10 client.15216 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T04:03:23.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.963+0000 7fc510dfc740 20 client.15216 populate_metadata read hostname 'smithi064' 2024-09-21T04:03:23.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.963+0000 7fc510dfc740 1 --2- 172.21.15.64:0/3351015117 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a94fdd38e0 0x55a9501d9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:23.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.963+0000 7fc510dfc740 4 client.15216 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:03:23.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.963+0000 7fc510dfc740 1 -- 172.21.15.64:0/3351015117 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a950134b60 con 0x55a94fdd38e0 2024-09-21T04:03:23.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.963+0000 7fc503fff640 1 --2- 172.21.15.64:0/3351015117 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a94fdd38e0 0x55a9501d9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:23.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.963+0000 7fc503fff640 1 --2- 172.21.15.64:0/3351015117 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a94fdd38e0 0x55a9501d9120 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c90f1afb592f9c47 server_cookie=fe9d439a13117178 in_seq=0 out_seq=0 2024-09-21T04:03:23.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.964+0000 7fc4e37fe640 10 client.15216 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:03:23.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.964+0000 7fc4e37fe640 1 -- 172.21.15.64:0/3351015117 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a950134b60 con 0x55a94fdd38e0 2024-09-21T04:03:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.964+0000 7fc4e37fe640 10 client.15216 handle_command_reply: tid=0 2024-09-21T04:03:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.964+0000 7fc510dfc740 1 client.15216 shutdown 2024-09-21T04:03:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.965+0000 7fc510dfc740 1 -- 172.21.15.64:0/3351015117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fdec0a0 msgr2=0x55a9501c6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.965+0000 7fc510dfc740 1 --2- 172.21.15.64:0/3351015117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fdec0a0 0x55a9501c6aa0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8009de0 tx=0x7fc4f80046c0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.965+0000 7fc510dfc740 1 -- 172.21.15.64:0/3351015117 shutdown_connections 2024-09-21T04:03:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.965+0000 7fc510dfc740 1 --2- 172.21.15.64:0/3351015117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94fe14f60 0x55a9501d29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.965+0000 7fc510dfc740 1 --2- 172.21.15.64:0/3351015117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a94fdf4280 0x55a9501c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.965+0000 7fc510dfc740 1 --2- 172.21.15.64:0/3351015117 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a94fdd38e0 0x55a9501d9120 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.965+0000 7fc510dfc740 1 --2- 172.21.15.64:0/3351015117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fdec0a0 0x55a9501c6aa0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.965+0000 7fc510dfc740 1 -- 172.21.15.64:0/3351015117 >> 172.21.15.64:0/3351015117 conn(0x55a94fe09350 msgr2=0x55a9501be090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.965+0000 7fc4e37fe640 0 client.15216 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:03:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.965+0000 7fc510dfc740 1 -- 172.21.15.64:0/3351015117 shutdown_connections 2024-09-21T04:03:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.965+0000 7fc510dfc740 1 -- 172.21.15.64:0/3351015117 wait complete. 2024-09-21T04:03:23.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.965+0000 7fc510dfc740 20 client.15216 trim_cache size 0 max 16384 2024-09-21T04:03:23.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.983+0000 7fc510dfc740 1 Processor -- start 2024-09-21T04:03:23.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.983+0000 7fc510dfc740 1 -- start start 2024-09-21T04:03:23.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.984+0000 7fc510dfc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94fdec0a0 0x55a9501d2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:23.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.984+0000 7fc510dfc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a94fdf4280 0x55a9501c1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:23.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.984+0000 7fc510dfc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fe14f60 0x55a9501c1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:23.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.984+0000 7fc510dfc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a9501d7460 con 0x55a94fe14f60 2024-09-21T04:03:23.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.984+0000 7fc510dfc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a9501d7b30 con 0x55a94fdf4280 2024-09-21T04:03:23.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.984+0000 7fc510dfc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a9501d8200 con 0x55a94fdec0a0 2024-09-21T04:03:23.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.984+0000 7fc5037fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fe14f60 0x55a9501c1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:23.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.984+0000 7fc5037fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fe14f60 0x55a9501c1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50264/0 (socket says 172.21.15.64:50264) 2024-09-21T04:03:23.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.984+0000 7fc5037fe640 1 -- 172.21.15.64:0/3923736382 learned_addr learned my addr 172.21.15.64:0/3923736382 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:23.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.984+0000 7fc503fff640 1 --2- 172.21.15.64:0/3923736382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a94fdf4280 0x55a9501c1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:23.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.984+0000 7fc5037fe640 1 --2- 172.21.15.64:0/3923736382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94fdec0a0 0x55a9501d2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:23.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.984+0000 7fc5037fe640 1 -- 172.21.15.64:0/3923736382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94fdec0a0 msgr2=0x55a9501d2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:23.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.984+0000 7fc5037fe640 1 --2- 172.21.15.64:0/3923736382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94fdec0a0 0x55a9501d2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.984+0000 7fc5037fe640 1 -- 172.21.15.64:0/3923736382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a94fdf4280 msgr2=0x55a9501c1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:23.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.984+0000 7fc5037fe640 1 --2- 172.21.15.64:0/3923736382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a94fdf4280 0x55a9501c1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.984+0000 7fc5037fe640 1 -- 172.21.15.64:0/3923736382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9501b3080 con 0x55a94fe14f60 2024-09-21T04:03:23.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.984+0000 7fc5037fe640 1 --2- 172.21.15.64:0/3923736382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fe14f60 0x55a9501c1f90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8014940 tx=0x7fc4f8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e500a72187bab49 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:23.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.984+0000 7fc4e9ffb640 1 -- 172.21.15.64:0/3923736382 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f802f070 con 0x55a94fe14f60 2024-09-21T04:03:23.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.985+0000 7fc4e9ffb640 1 -- 172.21.15.64:0/3923736382 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4f8010e90 con 0x55a94fe14f60 2024-09-21T04:03:23.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.985+0000 7fc4e9ffb640 1 -- 172.21.15.64:0/3923736382 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8005360 con 0x55a94fe14f60 2024-09-21T04:03:23.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.985+0000 7fc510dfc740 1 -- 172.21.15.64:0/3923736382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fe14f60 msgr2=0x55a9501c1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:23.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.985+0000 7fc510dfc740 1 --2- 172.21.15.64:0/3923736382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fe14f60 0x55a9501c1f90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8014940 tx=0x7fc4f8009f90 comp rx=0 tx=0).stop 2024-09-21T04:03:23.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.985+0000 7fc510dfc740 1 -- 172.21.15.64:0/3923736382 shutdown_connections 2024-09-21T04:03:23.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.985+0000 7fc510dfc740 1 --2- 172.21.15.64:0/3923736382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fe14f60 0x55a9501c1f90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.985+0000 7fc510dfc740 1 --2- 172.21.15.64:0/3923736382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a94fdf4280 0x55a9501c1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.985+0000 7fc510dfc740 1 --2- 172.21.15.64:0/3923736382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94fdec0a0 0x55a9501d2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.985+0000 7fc510dfc740 1 -- 172.21.15.64:0/3923736382 >> 172.21.15.64:0/3923736382 conn(0x55a94fe09350 msgr2=0x55a9501c6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:23.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.986+0000 7fc510dfc740 1 -- 172.21.15.64:0/3923736382 shutdown_connections 2024-09-21T04:03:23.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.986+0000 7fc510dfc740 1 -- 172.21.15.64:0/3923736382 wait complete. 2024-09-21T04:03:23.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.986+0000 7fc510dfc740 1 -- start start 2024-09-21T04:03:23.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.986+0000 7fc510dfc740 1 Processor -- start 2024-09-21T04:03:23.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.986+0000 7fc510dfc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fdec0a0 0x55a9501e1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:23.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.986+0000 7fc510dfc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a94fdf4280 0x55a9501dae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:23.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.986+0000 7fc510dfc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94fe14f60 0x55a9501db370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:23.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.986+0000 7fc510dfc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a9501d8fa0 con 0x55a94fdec0a0 2024-09-21T04:03:23.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.986+0000 7fc510dfc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a9501d88d0 con 0x55a94fdf4280 2024-09-21T04:03:23.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.987+0000 7fc510dfc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a9501d9670 con 0x55a94fe14f60 2024-09-21T04:03:23.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.987+0000 7fc510dfc740 10 client.0 _ll_register_callbacks cb 0x7fc4f80098d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:03:23.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.987+0000 7fc5037fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fdec0a0 0x55a9501e1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:23.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.987+0000 7fc5037fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fdec0a0 0x55a9501e1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50268/0 (socket says 172.21.15.64:50268) 2024-09-21T04:03:23.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.987+0000 7fc5037fe640 1 -- 172.21.15.64:0/229915379 learned_addr learned my addr 172.21.15.64:0/229915379 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:23.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.987+0000 7fc503fff640 1 --2- 172.21.15.64:0/229915379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a94fdf4280 0x55a9501dae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:23.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.987+0000 7fc5037fe640 1 --2- 172.21.15.64:0/229915379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94fe14f60 0x55a9501db370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:23.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.987+0000 7fc5037fe640 1 -- 172.21.15.64:0/229915379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94fe14f60 msgr2=0x55a9501db370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:23.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.987+0000 7fc5037fe640 1 --2- 172.21.15.64:0/229915379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94fe14f60 0x55a9501db370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.987+0000 7fc5037fe640 1 -- 172.21.15.64:0/229915379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a94fdf4280 msgr2=0x55a9501dae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:23.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.987+0000 7fc5037fe640 1 --2- 172.21.15.64:0/229915379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a94fdf4280 0x55a9501dae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.987+0000 7fc5037fe640 1 -- 172.21.15.64:0/229915379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9501db8e0 con 0x55a94fdec0a0 2024-09-21T04:03:23.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.987+0000 7fc503fff640 1 --2- 172.21.15.64:0/229915379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a94fdf4280 0x55a9501dae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:03:23.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.987+0000 7fc5037fe640 1 --2- 172.21.15.64:0/229915379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fdec0a0 0x55a9501e1980 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8009ab0 tx=0x7fc4f8009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3ef0ed3d7ea2a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:23.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.987+0000 7fc4e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:03:23.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.987+0000 7fc4e37fe640 1 -- 172.21.15.64:0/229915379 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f802f030 con 0x55a94fdec0a0 2024-09-21T04:03:23.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.987+0000 7fc510dfc740 1 -- 172.21.15.64:0/229915379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4f8010e90 con 0x55a94fdec0a0 2024-09-21T04:03:23.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.987+0000 7fc4e37fe640 1 -- 172.21.15.64:0/229915379 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7fc4f8010c90 con 0x55a94fdec0a0 2024-09-21T04:03:23.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.988+0000 7fc4e37fe640 1 -- 172.21.15.64:0/229915379 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc4f80050d0 con 0x55a94fdec0a0 2024-09-21T04:03:23.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.988+0000 7fc510dfc740 10 client.15222 fetch_fsmap learned FSMap version 24 2024-09-21T04:03:23.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.988+0000 7fc510dfc740 1 -- 172.21.15.64:0/229915379 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a9501e1330 con 0x55a94fdec0a0 2024-09-21T04:03:23.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.988+0000 7fc4e37fe640 1 -- 172.21.15.64:0/229915379 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4f8010e90 con 0x55a94fdec0a0 2024-09-21T04:03:23.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.988+0000 7fc4e37fe640 1 -- 172.21.15.64:0/229915379 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fc4f80116c0 con 0x55a94fdec0a0 2024-09-21T04:03:23.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.988+0000 7fc510dfc740 10 client.15222 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T04:03:23.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.988+0000 7fc510dfc740 10 client.15222 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T04:03:23.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.988+0000 7fc510dfc740 20 client.15222 populate_metadata read hostname 'smithi064' 2024-09-21T04:03:23.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.988+0000 7fc510dfc740 1 --2- 172.21.15.64:0/229915379 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a94fda8c90 0x55a950247770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:23.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.988+0000 7fc510dfc740 4 client.15222 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T04:03:23.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.988+0000 7fc510dfc740 1 -- 172.21.15.64:0/229915379 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "status"}) -- 0x55a950130610 con 0x55a94fda8c90 2024-09-21T04:03:23.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.988+0000 7fc503fff640 1 --2- 172.21.15.64:0/229915379 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a94fda8c90 0x55a950247770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:23.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.989+0000 7fc4e37fe640 10 client.15222 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:03:23.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.989+0000 7fc503fff640 1 --2- 172.21.15.64:0/229915379 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a94fda8c90 0x55a950247770 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a3b787743d6b813 server_cookie=f07a286205fd9b22 in_seq=0 out_seq=0 2024-09-21T04:03:23.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.989+0000 7fc4e37fe640 1 -- 172.21.15.64:0/229915379 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a950130610 con 0x55a94fda8c90 2024-09-21T04:03:23.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.989+0000 7fc4e37fe640 10 client.15222 handle_command_reply: tid=0 2024-09-21T04:03:23.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.989+0000 7fc510dfc740 1 client.15222 shutdown 2024-09-21T04:03:23.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.989+0000 7fc510dfc740 1 -- 172.21.15.64:0/229915379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fdec0a0 msgr2=0x55a9501e1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:23.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.989+0000 7fc510dfc740 1 --2- 172.21.15.64:0/229915379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fdec0a0 0x55a9501e1980 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8009ab0 tx=0x7fc4f8009a00 comp rx=0 tx=0).stop 2024-09-21T04:03:23.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.989+0000 7fc510dfc740 1 -- 172.21.15.64:0/229915379 shutdown_connections 2024-09-21T04:03:23.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.989+0000 7fc510dfc740 1 --2- 172.21.15.64:0/229915379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a94fe14f60 0x55a9501db370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.989+0000 7fc510dfc740 1 --2- 172.21.15.64:0/229915379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a94fdf4280 0x55a9501dae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.989+0000 7fc510dfc740 1 --2- 172.21.15.64:0/229915379 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a94fda8c90 0x55a950247770 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.989+0000 7fc510dfc740 1 --2- 172.21.15.64:0/229915379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a94fdec0a0 0x55a9501e1980 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.989+0000 7fc4e37fe640 0 client.15222 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:03:23.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.989+0000 7fc510dfc740 1 -- 172.21.15.64:0/229915379 >> 172.21.15.64:0/229915379 conn(0x55a94fe09350 msgr2=0x55a9501c1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:23.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.990+0000 7fc510dfc740 1 -- 172.21.15.64:0/229915379 shutdown_connections 2024-09-21T04:03:23.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.990+0000 7fc510dfc740 1 -- 172.21.15.64:0/229915379 wait complete. 2024-09-21T04:03:23.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.990+0000 7fc510dfc740 20 client.15222 trim_cache size 0 max 16384 2024-09-21T04:03:23.996 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:03:23.996 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "e7c927ba-77cc-11ef-baf5-efdc52797490", 2024-09-21T04:03:23.997 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2024-09-21T04:03:23.997 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24710, 2024-09-21T04:03:23.997 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-09-21T04:03:23.997 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-09-21T04:03:23.997 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-09-21T04:03:23.997 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 80.245544139000003, 2024-09-21T04:03:23.997 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 24, 2024-09-21T04:03:23.997 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 75, 2024-09-21T04:03:23.997 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 73, 2024-09-21T04:03:23.997 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 91.290228870000007 2024-09-21T04:03:23.997 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:03:23.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.990+0000 7fc50a888640 1 -- 172.21.15.64:0/3692614425 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc4ec0b65f0 msgr2=0x7fc4ec0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:23.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.990+0000 7fc50a888640 1 --2- 172.21.15.64:0/3692614425 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc4ec0b65f0 0x7fc4ec0b8b00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc0ad760 tx=0x7fc4fc000f40 comp rx=0 tx=0).stop 2024-09-21T04:03:23.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.990+0000 7fc50a888640 1 -- 172.21.15.64:0/3692614425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc50410de60 msgr2=0x7fc50419fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:23.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.990+0000 7fc50a888640 1 --2- 172.21.15.64:0/3692614425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc50410de60 0x7fc50419fcd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00eaa0 tx=0x7fc4fc00ef60 comp rx=0 tx=0).stop 2024-09-21T04:03:23.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.990+0000 7fc50a888640 1 -- 172.21.15.64:0/3692614425 shutdown_connections 2024-09-21T04:03:23.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.990+0000 7fc50a888640 1 --2- 172.21.15.64:0/3692614425 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc4ec0b65f0 0x7fc4ec0b8b00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.990+0000 7fc50a888640 1 --2- 172.21.15.64:0/3692614425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc50410de60 0x7fc50419fcd0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.990+0000 7fc50a888640 1 --2- 172.21.15.64:0/3692614425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc504105380 0x7fc50419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.990+0000 7fc50a888640 1 --2- 172.21.15.64:0/3692614425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc504104a30 0x7fc50419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:23.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.990+0000 7fc50a888640 1 -- 172.21.15.64:0/3692614425 >> 172.21.15.64:0/3692614425 conn(0x7fc504079760 msgr2=0x7fc504100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:23.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.990+0000 7fc50a888640 1 -- 172.21.15.64:0/3692614425 shutdown_connections 2024-09-21T04:03:23.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:23.990+0000 7fc50a888640 1 -- 172.21.15.64:0/3692614425 wait complete. 2024-09-21T04:03:24.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:23 smithi119 ceph-mon[34811]: pgmap v300: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T04:03:24.700 INFO:tasks.cephadm:Setup complete, yielding 2024-09-21T04:03:24.701 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T04:03:24.709 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-09-21T04:03:24.709 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- bash -c 'ceph orch status' 2024-09-21T04:03:24.967 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:25.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.453+0000 7fd5b7505640 1 -- 172.21.15.64:0/1838963988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5b0102240 msgr2=0x7fd5b0102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:25.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.453+0000 7fd5b7505640 1 --2- 172.21.15.64:0/1838963988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5b0102240 0x7fd5b0102650 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac00aa10 tx=0x7fd5ac02b720 comp rx=0 tx=0).stop 2024-09-21T04:03:25.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.453+0000 7fd5b7505640 1 -- 172.21.15.64:0/1838963988 shutdown_connections 2024-09-21T04:03:25.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.453+0000 7fd5b7505640 1 --2- 172.21.15.64:0/1838963988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5b01084d0 0x7fd5b010ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:25.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.453+0000 7fd5b7505640 1 --2- 172.21.15.64:0/1838963988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5b0107b80 0x7fd5b0107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:25.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.453+0000 7fd5b7505640 1 --2- 172.21.15.64:0/1838963988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5b0102240 0x7fd5b0102650 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:25.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.453+0000 7fd5b7505640 1 -- 172.21.15.64:0/1838963988 >> 172.21.15.64:0/1838963988 conn(0x7fd5b0079f90 msgr2=0x7fd5b007a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:25.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.454+0000 7fd5b7505640 1 -- 172.21.15.64:0/1838963988 shutdown_connections 2024-09-21T04:03:25.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.454+0000 7fd5b7505640 1 -- 172.21.15.64:0/1838963988 wait complete. 2024-09-21T04:03:25.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.454+0000 7fd5b7505640 1 Processor -- start 2024-09-21T04:03:25.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.454+0000 7fd5b7505640 1 -- start start 2024-09-21T04:03:25.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.455+0000 7fd5b7505640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5b0102240 0x7fd5b0199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:25.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.455+0000 7fd5b7505640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5b0107b80 0x7fd5b01980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:25.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.455+0000 7fd5b7505640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5b01084d0 0x7fd5b01985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:25.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.455+0000 7fd5b7505640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b0111910 con 0x7fd5b01084d0 2024-09-21T04:03:25.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.455+0000 7fd5b7505640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b0111790 con 0x7fd5b0107b80 2024-09-21T04:03:25.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.455+0000 7fd5b7505640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b0111a90 con 0x7fd5b0102240 2024-09-21T04:03:25.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.455+0000 7fd5b52a1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5b0102240 0x7fd5b0199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:25.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.455+0000 7fd5b52a1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5b0102240 0x7fd5b0199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:35666/0 (socket says 172.21.15.64:35666) 2024-09-21T04:03:25.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.455+0000 7fd5b52a1640 1 -- 172.21.15.64:0/2317840753 learned_addr learned my addr 172.21.15.64:0/2317840753 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:25.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.455+0000 7fd5b4aa0640 1 --2- 172.21.15.64:0/2317840753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5b0107b80 0x7fd5b01980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:25.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.455+0000 7fd5b5aa2640 1 --2- 172.21.15.64:0/2317840753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5b01084d0 0x7fd5b01985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:25.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.455+0000 7fd5b5aa2640 1 -- 172.21.15.64:0/2317840753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5b0102240 msgr2=0x7fd5b0199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:25.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.455+0000 7fd5b5aa2640 1 --2- 172.21.15.64:0/2317840753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5b0102240 0x7fd5b0199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:25.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.455+0000 7fd5b5aa2640 1 -- 172.21.15.64:0/2317840753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5b0107b80 msgr2=0x7fd5b01980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:25.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.455+0000 7fd5b5aa2640 1 --2- 172.21.15.64:0/2317840753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5b0107b80 0x7fd5b01980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:25.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.455+0000 7fd5b5aa2640 1 -- 172.21.15.64:0/2317840753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b0198e10 con 0x7fd5b01084d0 2024-09-21T04:03:25.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.455+0000 7fd5b52a1640 1 --2- 172.21.15.64:0/2317840753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5b0102240 0x7fd5b0199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:03:25.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.455+0000 7fd5b5aa2640 1 --2- 172.21.15.64:0/2317840753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5b01084d0 0x7fd5b01985f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd5a800e940 tx=0x7fd5a800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:25.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.456+0000 7fd5a67fc640 1 -- 172.21.15.64:0/2317840753 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8004d20 con 0x7fd5b01084d0 2024-09-21T04:03:25.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.456+0000 7fd5a67fc640 1 -- 172.21.15.64:0/2317840753 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5a8005020 con 0x7fd5b01084d0 2024-09-21T04:03:25.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.456+0000 7fd5b7505640 1 -- 172.21.15.64:0/2317840753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b01aceb0 con 0x7fd5b01084d0 2024-09-21T04:03:25.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.456+0000 7fd5a67fc640 1 -- 172.21.15.64:0/2317840753 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a80116d0 con 0x7fd5b01084d0 2024-09-21T04:03:25.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.456+0000 7fd5b7505640 1 -- 172.21.15.64:0/2317840753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b01ad310 con 0x7fd5b01084d0 2024-09-21T04:03:25.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.457+0000 7fd5b7505640 1 -- 172.21.15.64:0/2317840753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5b0102650 con 0x7fd5b01084d0 2024-09-21T04:03:25.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.458+0000 7fd5a67fc640 1 -- 172.21.15.64:0/2317840753 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fd5a8005460 con 0x7fd5b01084d0 2024-09-21T04:03:25.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.459+0000 7fd5a67fc640 1 --2- 172.21.15.64:0/2317840753 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd59c0b65f0 0x7fd59c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:25.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.459+0000 7fd5b52a1640 1 --2- 172.21.15.64:0/2317840753 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd59c0b65f0 0x7fd59c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:25.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.459+0000 7fd5a67fc640 1 -- 172.21.15.64:0/2317840753 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7fd5a80e7680 con 0x7fd5b01084d0 2024-09-21T04:03:25.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.459+0000 7fd5b52a1640 1 --2- 172.21.15.64:0/2317840753 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd59c0b65f0 0x7fd59c0b8b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac00a340 tx=0x7fd5ac0058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:25.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.464+0000 7fd5a67fc640 1 -- 172.21.15.64:0/2317840753 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd5a8016030 con 0x7fd5b01084d0 2024-09-21T04:03:25.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.669+0000 7fd5b7505640 1 -- 172.21.15.64:0/2317840753 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fd5b0063800 con 0x7fd59c0b65f0 2024-09-21T04:03:25.672 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:25 smithi064 ceph-mon[31180]: pgmap v301: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:03:25.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.670+0000 7fd5a67fc640 1 -- 172.21.15.64:0/2317840753 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fd5b0063800 con 0x7fd59c0b65f0 2024-09-21T04:03:25.672 INFO:teuthology.orchestra.run.smithi064.stdout:Backend: cephadm 2024-09-21T04:03:25.673 INFO:teuthology.orchestra.run.smithi064.stdout:Available: Yes 2024-09-21T04:03:25.673 INFO:teuthology.orchestra.run.smithi064.stdout:Paused: No 2024-09-21T04:03:25.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.672+0000 7fd5b7505640 1 -- 172.21.15.64:0/2317840753 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd59c0b65f0 msgr2=0x7fd59c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:25.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.672+0000 7fd5b7505640 1 --2- 172.21.15.64:0/2317840753 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd59c0b65f0 0x7fd59c0b8b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac00a340 tx=0x7fd5ac0058d0 comp rx=0 tx=0).stop 2024-09-21T04:03:25.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.672+0000 7fd5b7505640 1 -- 172.21.15.64:0/2317840753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5b01084d0 msgr2=0x7fd5b01985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:25.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.672+0000 7fd5b7505640 1 --2- 172.21.15.64:0/2317840753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5b01084d0 0x7fd5b01985f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd5a800e940 tx=0x7fd5a800ee00 comp rx=0 tx=0).stop 2024-09-21T04:03:25.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.672+0000 7fd5b7505640 1 -- 172.21.15.64:0/2317840753 shutdown_connections 2024-09-21T04:03:25.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.672+0000 7fd5b7505640 1 --2- 172.21.15.64:0/2317840753 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd59c0b65f0 0x7fd59c0b8b00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:25.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.672+0000 7fd5b7505640 1 --2- 172.21.15.64:0/2317840753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5b01084d0 0x7fd5b01985f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:25.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.672+0000 7fd5b7505640 1 --2- 172.21.15.64:0/2317840753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5b0107b80 0x7fd5b01980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:25.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.672+0000 7fd5b7505640 1 --2- 172.21.15.64:0/2317840753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd5b0102240 0x7fd5b0199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:25.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.672+0000 7fd5b7505640 1 -- 172.21.15.64:0/2317840753 >> 172.21.15.64:0/2317840753 conn(0x7fd5b0079f90 msgr2=0x7fd5b0105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:25.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.672+0000 7fd5b7505640 1 -- 172.21.15.64:0/2317840753 shutdown_connections 2024-09-21T04:03:25.676 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:25.673+0000 7fd5b7505640 1 -- 172.21.15.64:0/2317840753 wait complete. 2024-09-21T04:03:25.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:25 smithi131 ceph-mon[35193]: pgmap v301: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:03:26.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:25 smithi119 ceph-mon[34811]: pgmap v301: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:03:26.348 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ps' 2024-09-21T04:03:26.607 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:26.729 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:26 smithi064 ceph-mon[31180]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:03:26.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:26 smithi131 ceph-mon[35193]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:03:27.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:26 smithi119 ceph-mon[34811]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:03:27.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.104+0000 7f5d0d036640 1 -- 172.21.15.64:0/2881767164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d08105380 msgr2=0x7f5d0810d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:27.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.104+0000 7f5d0d036640 1 --2- 172.21.15.64:0/2881767164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d08105380 0x7f5d0810d8d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009900 tx=0x7f5cfc02a410 comp rx=0 tx=0).stop 2024-09-21T04:03:27.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.105+0000 7f5d0d036640 1 -- 172.21.15.64:0/2881767164 shutdown_connections 2024-09-21T04:03:27.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.105+0000 7f5d0d036640 1 --2- 172.21.15.64:0/2881767164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d0810de60 0x7f5d08110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:27.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.105+0000 7f5d0d036640 1 --2- 172.21.15.64:0/2881767164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d08105380 0x7f5d0810d8d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:27.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.105+0000 7f5d0d036640 1 --2- 172.21.15.64:0/2881767164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d08104a30 0x7f5d08104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:27.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.105+0000 7f5d0d036640 1 -- 172.21.15.64:0/2881767164 >> 172.21.15.64:0/2881767164 conn(0x7f5d08079760 msgr2=0x7f5d08079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:27.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.105+0000 7f5d0d036640 1 -- 172.21.15.64:0/2881767164 shutdown_connections 2024-09-21T04:03:27.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.105+0000 7f5d0d036640 1 -- 172.21.15.64:0/2881767164 wait complete. 2024-09-21T04:03:27.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.105+0000 7f5d0d036640 1 Processor -- start 2024-09-21T04:03:27.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.106+0000 7f5d0d036640 1 -- start start 2024-09-21T04:03:27.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.106+0000 7f5d0d036640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d08104a30 0x7f5d0819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:27.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.106+0000 7f5d0d036640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d08105380 0x7f5d0819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:27.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.106+0000 7f5d0d036640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d0810de60 0x7f5d0819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:27.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.106+0000 7f5d0d036640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5d08113af0 con 0x7f5d0810de60 2024-09-21T04:03:27.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.106+0000 7f5d0d036640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5d08113970 con 0x7f5d08105380 2024-09-21T04:03:27.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.106+0000 7f5d0d036640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5d08113c70 con 0x7f5d08104a30 2024-09-21T04:03:27.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.106+0000 7f5d0759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d0810de60 0x7f5d0819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:27.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.106+0000 7f5d0759e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d0810de60 0x7f5d0819fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50314/0 (socket says 172.21.15.64:50314) 2024-09-21T04:03:27.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.106+0000 7f5d0759e640 1 -- 172.21.15.64:0/2943427569 learned_addr learned my addr 172.21.15.64:0/2943427569 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:27.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.106+0000 7f5d0659c640 1 --2- 172.21.15.64:0/2943427569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d08105380 0x7f5d0819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:27.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.106+0000 7f5d06d9d640 1 --2- 172.21.15.64:0/2943427569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d08104a30 0x7f5d0819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:27.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.107+0000 7f5d0759e640 1 -- 172.21.15.64:0/2943427569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d08104a30 msgr2=0x7f5d0819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:27.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.107+0000 7f5d0759e640 1 --2- 172.21.15.64:0/2943427569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d08104a30 0x7f5d0819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:27.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.107+0000 7f5d0759e640 1 -- 172.21.15.64:0/2943427569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d08105380 msgr2=0x7f5d0819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:27.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.107+0000 7f5d0759e640 1 --2- 172.21.15.64:0/2943427569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d08105380 0x7f5d0819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:27.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.107+0000 7f5d0759e640 1 -- 172.21.15.64:0/2943427569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d081a03f0 con 0x7f5d0810de60 2024-09-21T04:03:27.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.107+0000 7f5d06d9d640 1 --2- 172.21.15.64:0/2943427569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d08104a30 0x7f5d0819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:03:27.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.107+0000 7f5d0759e640 1 --2- 172.21.15.64:0/2943427569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d0810de60 0x7f5d0819fcc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000eaa0 tx=0x7f5d0000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:27.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.107+0000 7f5ceffff640 1 -- 172.21.15.64:0/2943427569 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0000cdb0 con 0x7f5d0810de60 2024-09-21T04:03:27.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.107+0000 7f5ceffff640 1 -- 172.21.15.64:0/2943427569 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d00005020 con 0x7f5d0810de60 2024-09-21T04:03:27.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.107+0000 7f5ceffff640 1 -- 172.21.15.64:0/2943427569 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d000116d0 con 0x7f5d0810de60 2024-09-21T04:03:27.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.107+0000 7f5d0d036640 1 -- 172.21.15.64:0/2943427569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d081a06e0 con 0x7f5d0810de60 2024-09-21T04:03:27.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.107+0000 7f5d0d036640 1 -- 172.21.15.64:0/2943427569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d081a6e40 con 0x7f5d0810de60 2024-09-21T04:03:27.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.108+0000 7f5d0d036640 1 -- 172.21.15.64:0/2943427569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d0806c1f0 con 0x7f5d0810de60 2024-09-21T04:03:27.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.110+0000 7f5ceffff640 1 -- 172.21.15.64:0/2943427569 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f5d000051d0 con 0x7f5d0810de60 2024-09-21T04:03:27.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.110+0000 7f5ceffff640 1 --2- 172.21.15.64:0/2943427569 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5cf00b6670 0x7f5cf00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:27.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.111+0000 7f5d06d9d640 1 --2- 172.21.15.64:0/2943427569 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5cf00b6670 0x7f5cf00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:27.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.111+0000 7f5ceffff640 1 -- 172.21.15.64:0/2943427569 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f5d000e7aa0 con 0x7f5d0810de60 2024-09-21T04:03:27.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.111+0000 7f5d06d9d640 1 --2- 172.21.15.64:0/2943427569 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5cf00b6670 0x7f5cf00b8b80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8005d00 tx=0x7f5cf8005c90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:27.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.114+0000 7f5ceffff640 1 -- 172.21.15.64:0/2943427569 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5d00016030 con 0x7f5d0810de60 2024-09-21T04:03:27.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.312+0000 7f5d0d036640 1 -- 172.21.15.64:0/2943427569 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f5d08063800 con 0x7f5cf00b6670 2024-09-21T04:03:27.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.325+0000 7f5ceffff640 1 -- 172.21.15.64:0/2943427569 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f5d08063800 con 0x7f5cf00b6670 2024-09-21T04:03:27.327 INFO:teuthology.orchestra.run.smithi064.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T04:03:27.328 INFO:teuthology.orchestra.run.smithi064.stdout:mds.a smithi064 running (97s) 18s ago 97s 83.2M - 19.3.0-5120-g18432218 ddce307e1f32 daa9ea597ce9 2024-09-21T04:03:27.328 INFO:teuthology.orchestra.run.smithi064.stdout:mds.b smithi119 running (95s) 26s ago 94s 93.7M - 19.3.0-5120-g18432218 ddce307e1f32 0f0838c747be 2024-09-21T04:03:27.328 INFO:teuthology.orchestra.run.smithi064.stdout:mds.c smithi131 running (108s) 27s ago 108s 85.1M - 19.3.0-5120-g18432218 ddce307e1f32 591c2f243884 2024-09-21T04:03:27.328 INFO:teuthology.orchestra.run.smithi064.stdout:mds.d smithi064 running (116s) 18s ago 115s 84.2M - 19.3.0-5120-g18432218 ddce307e1f32 8e4f2b7a7e88 2024-09-21T04:03:27.328 INFO:teuthology.orchestra.run.smithi064.stdout:mds.e smithi119 running (113s) 26s ago 113s 85.9M - 19.3.0-5120-g18432218 ddce307e1f32 fc78e5b8b12e 2024-09-21T04:03:27.328 INFO:teuthology.orchestra.run.smithi064.stdout:mds.f smithi131 running (111s) 27s ago 110s 84.4M - 19.3.0-5120-g18432218 ddce307e1f32 d2eb13a7058d 2024-09-21T04:03:27.328 INFO:teuthology.orchestra.run.smithi064.stdout:mds.g smithi064 running (101s) 18s ago 100s 82.4M - 19.3.0-5120-g18432218 ddce307e1f32 3bedb67a4c32 2024-09-21T04:03:27.328 INFO:teuthology.orchestra.run.smithi064.stdout:mds.h smithi119 running (106s) 26s ago 105s 87.8M - 19.3.0-5120-g18432218 ddce307e1f32 78ff7ef87f18 2024-09-21T04:03:27.328 INFO:teuthology.orchestra.run.smithi064.stdout:mds.i smithi131 running (103s) 27s ago 103s 85.1M - 19.3.0-5120-g18432218 ddce307e1f32 4394b5bf8a90 2024-09-21T04:03:27.328 INFO:teuthology.orchestra.run.smithi064.stdout:mds.j smithi064 running (119s) 18s ago 119s 82.1M - 19.3.0-5120-g18432218 ddce307e1f32 da8ece3c8179 2024-09-21T04:03:27.328 INFO:teuthology.orchestra.run.smithi064.stdout:mds.k smithi119 running (2m) 26s ago 2m 82.3M - 19.3.0-5120-g18432218 ddce307e1f32 a9143ff40f36 2024-09-21T04:03:27.328 INFO:teuthology.orchestra.run.smithi064.stdout:mds.l smithi131 running (2m) 27s ago 2m 81.7M - 19.3.0-5120-g18432218 ddce307e1f32 b28956eef11c 2024-09-21T04:03:27.329 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.x smithi064 *:9283,8765 running (9m) 18s ago 9m 584M - 19.3.0-5120-g18432218 ddce307e1f32 9323be1996ab 2024-09-21T04:03:27.329 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.y smithi119 *:8443,8765 running (7m) 26s ago 7m 524M - 19.3.0-5120-g18432218 ddce307e1f32 5018b50d1d11 2024-09-21T04:03:27.329 INFO:teuthology.orchestra.run.smithi064.stdout:mgr.z smithi131 *:8443,8765 running (7m) 27s ago 7m 525M - 19.3.0-5120-g18432218 ddce307e1f32 64b0f071a2fd 2024-09-21T04:03:27.329 INFO:teuthology.orchestra.run.smithi064.stdout:mon.a smithi064 running (9m) 18s ago 9m 119M 2048M 19.3.0-5120-g18432218 ddce307e1f32 4084045b6479 2024-09-21T04:03:27.329 INFO:teuthology.orchestra.run.smithi064.stdout:mon.b smithi119 running (8m) 26s ago 8m 110M 2048M 19.3.0-5120-g18432218 ddce307e1f32 4dfbc9648e42 2024-09-21T04:03:27.329 INFO:teuthology.orchestra.run.smithi064.stdout:mon.c smithi131 running (8m) 27s ago 8m 112M 2048M 19.3.0-5120-g18432218 ddce307e1f32 0697ec341458 2024-09-21T04:03:27.329 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 smithi064 running (7m) 18s ago 7m 112M 4272M 19.3.0-5120-g18432218 ddce307e1f32 c3ecbdd2cce3 2024-09-21T04:03:27.329 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 smithi119 running (6m) 26s ago 6m 126M 4272M 19.3.0-5120-g18432218 ddce307e1f32 61e0f8f1664d 2024-09-21T04:03:27.329 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 smithi131 running (6m) 27s ago 6m 124M 4272M 19.3.0-5120-g18432218 ddce307e1f32 9b47f4af9779 2024-09-21T04:03:27.329 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 smithi064 running (5m) 18s ago 5m 112M 4272M 19.3.0-5120-g18432218 ddce307e1f32 f3e18a7ca9f5 2024-09-21T04:03:27.329 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 smithi119 running (5m) 26s ago 5m 109M 4272M 19.3.0-5120-g18432218 ddce307e1f32 90da4eae652d 2024-09-21T04:03:27.329 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 smithi131 running (4m) 27s ago 4m 110M 4272M 19.3.0-5120-g18432218 ddce307e1f32 ca7c302d4c25 2024-09-21T04:03:27.329 INFO:teuthology.orchestra.run.smithi064.stdout:osd.6 smithi064 running (4m) 18s ago 4m 119M 4272M 19.3.0-5120-g18432218 ddce307e1f32 3c4c18ec67af 2024-09-21T04:03:27.330 INFO:teuthology.orchestra.run.smithi064.stdout:osd.7 smithi119 running (4m) 26s ago 4m 111M 4272M 19.3.0-5120-g18432218 ddce307e1f32 bedd7fc191f3 2024-09-21T04:03:27.330 INFO:teuthology.orchestra.run.smithi064.stdout:osd.8 smithi131 running (3m) 27s ago 3m 120M 4272M 19.3.0-5120-g18432218 ddce307e1f32 84e80a27c472 2024-09-21T04:03:27.330 INFO:teuthology.orchestra.run.smithi064.stdout:osd.9 smithi064 running (3m) 18s ago 3m 110M 4272M 19.3.0-5120-g18432218 ddce307e1f32 3b01c43db45e 2024-09-21T04:03:27.330 INFO:teuthology.orchestra.run.smithi064.stdout:osd.10 smithi119 running (2m) 26s ago 2m 111M 4272M 19.3.0-5120-g18432218 ddce307e1f32 884cd453c247 2024-09-21T04:03:27.330 INFO:teuthology.orchestra.run.smithi064.stdout:osd.11 smithi131 running (2m) 27s ago 2m 120M 4272M 19.3.0-5120-g18432218 ddce307e1f32 73777c77b21f 2024-09-21T04:03:27.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.327+0000 7f5d0d036640 1 -- 172.21.15.64:0/2943427569 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5cf00b6670 msgr2=0x7f5cf00b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:27.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.327+0000 7f5d0d036640 1 --2- 172.21.15.64:0/2943427569 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5cf00b6670 0x7f5cf00b8b80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8005d00 tx=0x7f5cf8005c90 comp rx=0 tx=0).stop 2024-09-21T04:03:27.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.327+0000 7f5d0d036640 1 -- 172.21.15.64:0/2943427569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d0810de60 msgr2=0x7f5d0819fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:27.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.327+0000 7f5d0d036640 1 --2- 172.21.15.64:0/2943427569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d0810de60 0x7f5d0819fcc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000eaa0 tx=0x7f5d0000ef60 comp rx=0 tx=0).stop 2024-09-21T04:03:27.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.327+0000 7f5d0d036640 1 -- 172.21.15.64:0/2943427569 shutdown_connections 2024-09-21T04:03:27.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.327+0000 7f5d0d036640 1 --2- 172.21.15.64:0/2943427569 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5cf00b6670 0x7f5cf00b8b80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:27.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.327+0000 7f5d0d036640 1 --2- 172.21.15.64:0/2943427569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d0810de60 0x7f5d0819fcc0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:27.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.327+0000 7f5d0d036640 1 --2- 172.21.15.64:0/2943427569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d08105380 0x7f5d0819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:27.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.327+0000 7f5d0d036640 1 --2- 172.21.15.64:0/2943427569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d08104a30 0x7f5d0819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:27.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.327+0000 7f5d0d036640 1 -- 172.21.15.64:0/2943427569 >> 172.21.15.64:0/2943427569 conn(0x7f5d08079760 msgr2=0x7f5d08100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:27.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.327+0000 7f5d0d036640 1 -- 172.21.15.64:0/2943427569 shutdown_connections 2024-09-21T04:03:27.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:27.327+0000 7f5d0d036640 1 -- 172.21.15.64:0/2943427569 wait complete. 2024-09-21T04:03:27.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:27 smithi064 ceph-mon[31180]: pgmap v302: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T04:03:27.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:27 smithi064 ceph-mon[31180]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:03:27.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:27 smithi131 ceph-mon[35193]: pgmap v302: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T04:03:27.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:27 smithi131 ceph-mon[35193]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:03:27.982 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ls' 2024-09-21T04:03:28.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:27 smithi119 ceph-mon[34811]: pgmap v302: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T04:03:28.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:27 smithi119 ceph-mon[34811]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:03:28.242 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:28.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.755+0000 7f8a10e57640 1 -- 172.21.15.64:0/3239314037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a0c1083a0 msgr2=0x7f8a0c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:28.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.755+0000 7f8a10e57640 1 --2- 172.21.15.64:0/3239314037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a0c1083a0 0x7f8a0c1087b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009900 tx=0x7f89f402a410 comp rx=0 tx=0).stop 2024-09-21T04:03:28.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.755+0000 7f8a10e57640 1 -- 172.21.15.64:0/3239314037 shutdown_connections 2024-09-21T04:03:28.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.755+0000 7f8a10e57640 1 --2- 172.21.15.64:0/3239314037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a0c108cf0 0x7f8a0c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:28.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.755+0000 7f8a10e57640 1 --2- 172.21.15.64:0/3239314037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a0c1083a0 0x7f8a0c1087b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:28.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.755+0000 7f8a10e57640 1 --2- 172.21.15.64:0/3239314037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a0c102a60 0x7f8a0c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:28.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.755+0000 7f8a10e57640 1 -- 172.21.15.64:0/3239314037 >> 172.21.15.64:0/3239314037 conn(0x7f8a0c079730 msgr2=0x7f8a0c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:28.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.755+0000 7f8a10e57640 1 -- 172.21.15.64:0/3239314037 shutdown_connections 2024-09-21T04:03:28.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.755+0000 7f8a10e57640 1 -- 172.21.15.64:0/3239314037 wait complete. 2024-09-21T04:03:28.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.756+0000 7f8a10e57640 1 Processor -- start 2024-09-21T04:03:28.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.756+0000 7f8a10e57640 1 -- start start 2024-09-21T04:03:28.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.756+0000 7f8a10e57640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a0c102a60 0x7f8a0c0ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:28.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.756+0000 7f8a10e57640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a0c1083a0 0x7f8a0c0ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:28.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.756+0000 7f8a10e57640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a0c108cf0 0x7f8a0c1000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:28.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.756+0000 7f8a10e57640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c112960 con 0x7f8a0c1083a0 2024-09-21T04:03:28.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.756+0000 7f8a10e57640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c1127e0 con 0x7f8a0c102a60 2024-09-21T04:03:28.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.756+0000 7f8a10e57640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c112ae0 con 0x7f8a0c108cf0 2024-09-21T04:03:28.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.756+0000 7f8a09d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a0c1083a0 0x7f8a0c0ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:28.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.756+0000 7f8a09d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a0c1083a0 0x7f8a0c0ffbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50324/0 (socket says 172.21.15.64:50324) 2024-09-21T04:03:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.756+0000 7f8a09d9b640 1 -- 172.21.15.64:0/2854094964 learned_addr learned my addr 172.21.15.64:0/2854094964 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.756+0000 7f8a0a59c640 1 --2- 172.21.15.64:0/2854094964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a0c102a60 0x7f8a0c0ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.757+0000 7f8a0ad9d640 1 --2- 172.21.15.64:0/2854094964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a0c108cf0 0x7f8a0c1000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.757+0000 7f8a09d9b640 1 -- 172.21.15.64:0/2854094964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a0c108cf0 msgr2=0x7f8a0c1000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.757+0000 7f8a09d9b640 1 --2- 172.21.15.64:0/2854094964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a0c108cf0 0x7f8a0c1000f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.757+0000 7f8a09d9b640 1 -- 172.21.15.64:0/2854094964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a0c102a60 msgr2=0x7f8a0c0ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.757+0000 7f8a09d9b640 1 --2- 172.21.15.64:0/2854094964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a0c102a60 0x7f8a0c0ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.757+0000 7f8a09d9b640 1 -- 172.21.15.64:0/2854094964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0c1a8740 con 0x7f8a0c1083a0 2024-09-21T04:03:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.757+0000 7f8a09d9b640 1 --2- 172.21.15.64:0/2854094964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a0c1083a0 0x7f8a0c0ffbb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f89f4006010 tx=0x7f89f4004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.757+0000 7f8a037fe640 1 -- 172.21.15.64:0/2854094964 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f403f030 con 0x7f8a0c1083a0 2024-09-21T04:03:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.757+0000 7f8a037fe640 1 -- 172.21.15.64:0/2854094964 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89f4003aa0 con 0x7f8a0c1083a0 2024-09-21T04:03:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.757+0000 7f8a10e57640 1 -- 172.21.15.64:0/2854094964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0c1a89d0 con 0x7f8a0c1083a0 2024-09-21T04:03:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.757+0000 7f8a037fe640 1 -- 172.21.15.64:0/2854094964 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4035070 con 0x7f8a0c1083a0 2024-09-21T04:03:28.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.757+0000 7f8a10e57640 1 -- 172.21.15.64:0/2854094964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a0c1a8eb0 con 0x7f8a0c1083a0 2024-09-21T04:03:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.758+0000 7f8a10e57640 1 -- 172.21.15.64:0/2854094964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a0c102e70 con 0x7f8a0c1083a0 2024-09-21T04:03:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.760+0000 7f8a037fe640 1 -- 172.21.15.64:0/2854094964 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f89f4042030 con 0x7f8a0c1083a0 2024-09-21T04:03:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.761+0000 7f8a037fe640 1 --2- 172.21.15.64:0/2854094964 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f89f80b65f0 0x7f89f80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.761+0000 7f8a0a59c640 1 --2- 172.21.15.64:0/2854094964 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f89f80b65f0 0x7f89f80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.761+0000 7f8a037fe640 1 -- 172.21.15.64:0/2854094964 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8484+0+0 (secure 0 0 0) 0x7f89f4107e90 con 0x7f8a0c1083a0 2024-09-21T04:03:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.761+0000 7f8a0a59c640 1 --2- 172.21.15.64:0/2854094964 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f89f80b65f0 0x7f89f80b8b00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c1010c0 tx=0x7f89fc009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:28.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.764+0000 7f8a037fe640 1 -- 172.21.15.64:0/2854094964 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f89f40ce4c0 con 0x7f8a0c1083a0 2024-09-21T04:03:28.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.993+0000 7f8a10e57640 1 -- 172.21.15.64:0/2854094964 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f8a0c063800 con 0x7f89f80b65f0 2024-09-21T04:03:29.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:28.999+0000 7f8a037fe640 1 -- 172.21.15.64:0/2854094964 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f8a0c063800 con 0x7f89f80b65f0 2024-09-21T04:03:29.002 INFO:teuthology.orchestra.run.smithi064.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-21T04:03:29.002 INFO:teuthology.orchestra.run.smithi064.stdout:mds.all 12/12 28s ago 2m smithi064=a;smithi064=d;smithi064=g;smithi064=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi131=c;smithi131=f;smithi131=i;smithi131=l;count:12 2024-09-21T04:03:29.002 INFO:teuthology.orchestra.run.smithi064.stdout:mgr 3/3 28s ago 7m smithi064=x;smithi119=y;smithi131=z;count:3 2024-09-21T04:03:29.002 INFO:teuthology.orchestra.run.smithi064.stdout:mon 3/3 28s ago 8m smithi064:172.21.15.64=a;smithi119:172.21.15.119=b;smithi131:172.21.15.131=c;count:3 2024-09-21T04:03:29.002 INFO:teuthology.orchestra.run.smithi064.stdout:osd 12 28s ago - 2024-09-21T04:03:29.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:29.002+0000 7f8a10e57640 1 -- 172.21.15.64:0/2854094964 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f89f80b65f0 msgr2=0x7f89f80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:29.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:29.002+0000 7f8a10e57640 1 --2- 172.21.15.64:0/2854094964 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f89f80b65f0 0x7f89f80b8b00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c1010c0 tx=0x7f89fc009500 comp rx=0 tx=0).stop 2024-09-21T04:03:29.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:29.002+0000 7f8a10e57640 1 -- 172.21.15.64:0/2854094964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a0c1083a0 msgr2=0x7f8a0c0ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:29.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:29.002+0000 7f8a10e57640 1 --2- 172.21.15.64:0/2854094964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a0c1083a0 0x7f8a0c0ffbb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f89f4006010 tx=0x7f89f4004060 comp rx=0 tx=0).stop 2024-09-21T04:03:29.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:29.002+0000 7f8a10e57640 1 -- 172.21.15.64:0/2854094964 shutdown_connections 2024-09-21T04:03:29.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:29.002+0000 7f8a10e57640 1 --2- 172.21.15.64:0/2854094964 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f89f80b65f0 0x7f89f80b8b00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:29.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:29.002+0000 7f8a10e57640 1 --2- 172.21.15.64:0/2854094964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a0c108cf0 0x7f8a0c1000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:29.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:29.002+0000 7f8a10e57640 1 --2- 172.21.15.64:0/2854094964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a0c1083a0 0x7f8a0c0ffbb0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:29.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:29.002+0000 7f8a10e57640 1 --2- 172.21.15.64:0/2854094964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a0c102a60 0x7f8a0c0ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:29.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:29.002+0000 7f8a10e57640 1 -- 172.21.15.64:0/2854094964 >> 172.21.15.64:0/2854094964 conn(0x7f8a0c079730 msgr2=0x7f8a0c06b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:29.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:29.002+0000 7f8a10e57640 1 -- 172.21.15.64:0/2854094964 shutdown_connections 2024-09-21T04:03:29.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:29.002+0000 7f8a10e57640 1 -- 172.21.15.64:0/2854094964 wait complete. 2024-09-21T04:03:29.674 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- bash -c 'ceph orch host ls' 2024-09-21T04:03:29.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:29 smithi131 ceph-mon[35193]: pgmap v303: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T04:03:29.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [10, 11]} : dispatch 2024-09-21T04:03:29.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]} : dispatch 2024-09-21T04:03:29.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:29 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [5, 6]} : dispatch 2024-09-21T04:03:29.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:29 smithi131 ceph-mon[35193]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:03:29.940 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:30.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:29 smithi119 ceph-mon[34811]: pgmap v303: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T04:03:30.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [10, 11]} : dispatch 2024-09-21T04:03:30.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]} : dispatch 2024-09-21T04:03:30.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:29 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [5, 6]} : dispatch 2024-09-21T04:03:30.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:29 smithi119 ceph-mon[34811]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:03:30.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:29 smithi064 ceph-mon[31180]: pgmap v303: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T04:03:30.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [10, 11]} : dispatch 2024-09-21T04:03:30.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]} : dispatch 2024-09-21T04:03:30.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:29 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [5, 6]} : dispatch 2024-09-21T04:03:30.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:29 smithi064 ceph-mon[31180]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:03:30.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.467+0000 7f3b6078f640 1 -- 172.21.15.64:0/786923486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b58102a60 msgr2=0x7f3b58102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:30.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.467+0000 7f3b6078f640 1 --2- 172.21.15.64:0/786923486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b58102a60 0x7f3b58102e70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3b50009900 tx=0x7f3b5002a410 comp rx=0 tx=0).stop 2024-09-21T04:03:30.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.467+0000 7f3b6078f640 1 -- 172.21.15.64:0/786923486 shutdown_connections 2024-09-21T04:03:30.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.467+0000 7f3b6078f640 1 --2- 172.21.15.64:0/786923486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b58108cf0 0x7f3b5810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:30.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.467+0000 7f3b6078f640 1 --2- 172.21.15.64:0/786923486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b581083a0 0x7f3b581087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:30.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.467+0000 7f3b6078f640 1 --2- 172.21.15.64:0/786923486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b58102a60 0x7f3b58102e70 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:30.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.467+0000 7f3b6078f640 1 -- 172.21.15.64:0/786923486 >> 172.21.15.64:0/786923486 conn(0x7f3b58079730 msgr2=0x7f3b58079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:30.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.467+0000 7f3b6078f640 1 -- 172.21.15.64:0/786923486 shutdown_connections 2024-09-21T04:03:30.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.467+0000 7f3b6078f640 1 -- 172.21.15.64:0/786923486 wait complete. 2024-09-21T04:03:30.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.468+0000 7f3b6078f640 1 Processor -- start 2024-09-21T04:03:30.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.468+0000 7f3b6078f640 1 -- start start 2024-09-21T04:03:30.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.468+0000 7f3b6078f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b58102a60 0x7f3b58199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:30.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.468+0000 7f3b6078f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b581083a0 0x7f3b581998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:30.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.468+0000 7f3b6078f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b58108cf0 0x7f3b5819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:30.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.468+0000 7f3b6078f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3b58112960 con 0x7f3b58108cf0 2024-09-21T04:03:30.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.468+0000 7f3b6078f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3b581127e0 con 0x7f3b58102a60 2024-09-21T04:03:30.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.468+0000 7f3b6078f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3b58112ae0 con 0x7f3b581083a0 2024-09-21T04:03:30.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.468+0000 7f3b5ed2c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b58108cf0 0x7f3b5819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:30.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.468+0000 7f3b5ed2c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b58108cf0 0x7f3b5819eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50350/0 (socket says 172.21.15.64:50350) 2024-09-21T04:03:30.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.468+0000 7f3b5ed2c640 1 -- 172.21.15.64:0/3490604419 learned_addr learned my addr 172.21.15.64:0/3490604419 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:30.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.468+0000 7f3b5dd2a640 1 --2- 172.21.15.64:0/3490604419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b581083a0 0x7f3b581998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:30.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.469+0000 7f3b5e52b640 1 --2- 172.21.15.64:0/3490604419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b58102a60 0x7f3b58199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:30.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.469+0000 7f3b5ed2c640 1 -- 172.21.15.64:0/3490604419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b581083a0 msgr2=0x7f3b581998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:30.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.469+0000 7f3b5ed2c640 1 --2- 172.21.15.64:0/3490604419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b581083a0 0x7f3b581998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:30.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.469+0000 7f3b5ed2c640 1 -- 172.21.15.64:0/3490604419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b58102a60 msgr2=0x7f3b58199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:30.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.469+0000 7f3b5ed2c640 1 --2- 172.21.15.64:0/3490604419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b58102a60 0x7f3b58199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:30.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.469+0000 7f3b5ed2c640 1 -- 172.21.15.64:0/3490604419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b5819f2c0 con 0x7f3b58108cf0 2024-09-21T04:03:30.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.469+0000 7f3b5e52b640 1 --2- 172.21.15.64:0/3490604419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b58102a60 0x7f3b58199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:03:30.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.469+0000 7f3b5ed2c640 1 --2- 172.21.15.64:0/3490604419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b58108cf0 0x7f3b5819eb90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800eb10 tx=0x7f3b4800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:30.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.469+0000 7f3b577fe640 1 -- 172.21.15.64:0/3490604419 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b48004b20 con 0x7f3b58108cf0 2024-09-21T04:03:30.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.469+0000 7f3b6078f640 1 -- 172.21.15.64:0/3490604419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b5819f5b0 con 0x7f3b58108cf0 2024-09-21T04:03:30.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.469+0000 7f3b577fe640 1 -- 172.21.15.64:0/3490604419 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b48004cc0 con 0x7f3b58108cf0 2024-09-21T04:03:30.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.469+0000 7f3b577fe640 1 -- 172.21.15.64:0/3490604419 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b48010670 con 0x7f3b58108cf0 2024-09-21T04:03:30.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.469+0000 7f3b6078f640 1 -- 172.21.15.64:0/3490604419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b581a5d10 con 0x7f3b58108cf0 2024-09-21T04:03:30.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.470+0000 7f3b6078f640 1 -- 172.21.15.64:0/3490604419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b58102e70 con 0x7f3b58108cf0 2024-09-21T04:03:30.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.472+0000 7f3b577fe640 1 -- 172.21.15.64:0/3490604419 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f3b48005000 con 0x7f3b58108cf0 2024-09-21T04:03:30.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.473+0000 7f3b577fe640 1 --2- 172.21.15.64:0/3490604419 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3b400b65f0 0x7f3b400b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:30.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.473+0000 7f3b5e52b640 1 --2- 172.21.15.64:0/3490604419 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3b400b65f0 0x7f3b400b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:30.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.473+0000 7f3b577fe640 1 -- 172.21.15.64:0/3490604419 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8571+0+0 (secure 0 0 0) 0x7f3b480e7e50 con 0x7f3b58108cf0 2024-09-21T04:03:30.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.473+0000 7f3b5e52b640 1 --2- 172.21.15.64:0/3490604419 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3b400b65f0 0x7f3b400b8b00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3b500096c0 tx=0x7f3b50031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:30.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.476+0000 7f3b577fe640 1 -- 172.21.15.64:0/3490604419 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3b48056710 con 0x7f3b58108cf0 2024-09-21T04:03:30.626 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:30 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [10, 11]}]': finished 2024-09-21T04:03:30.626 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:30 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-21T04:03:30.626 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:30 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [5, 6]}]': finished 2024-09-21T04:03:30.626 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:30 smithi064 ceph-mon[31180]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T04:03:30.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.623+0000 7f3b577fe640 1 -- 172.21.15.64:0/3490604419 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f3b480a9fa0 con 0x7f3b58108cf0 2024-09-21T04:03:30.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.679+0000 7f3b6078f640 1 -- 172.21.15.64:0/3490604419 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f3b58063800 con 0x7f3b400b65f0 2024-09-21T04:03:30.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.680+0000 7f3b577fe640 1 -- 172.21.15.64:0/3490604419 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f3b58063800 con 0x7f3b400b65f0 2024-09-21T04:03:30.682 INFO:teuthology.orchestra.run.smithi064.stdout:HOST ADDR LABELS STATUS 2024-09-21T04:03:30.683 INFO:teuthology.orchestra.run.smithi064.stdout:smithi064 172.21.15.64 2024-09-21T04:03:30.683 INFO:teuthology.orchestra.run.smithi064.stdout:smithi119 172.21.15.119 2024-09-21T04:03:30.683 INFO:teuthology.orchestra.run.smithi064.stdout:smithi131 172.21.15.131 2024-09-21T04:03:30.683 INFO:teuthology.orchestra.run.smithi064.stdout:3 hosts in cluster 2024-09-21T04:03:30.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.683+0000 7f3b6078f640 1 -- 172.21.15.64:0/3490604419 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3b400b65f0 msgr2=0x7f3b400b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:30.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.683+0000 7f3b6078f640 1 --2- 172.21.15.64:0/3490604419 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3b400b65f0 0x7f3b400b8b00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3b500096c0 tx=0x7f3b50031400 comp rx=0 tx=0).stop 2024-09-21T04:03:30.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.683+0000 7f3b6078f640 1 -- 172.21.15.64:0/3490604419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b58108cf0 msgr2=0x7f3b5819eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:30.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.683+0000 7f3b6078f640 1 --2- 172.21.15.64:0/3490604419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b58108cf0 0x7f3b5819eb90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800eb10 tx=0x7f3b4800c2d0 comp rx=0 tx=0).stop 2024-09-21T04:03:30.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.683+0000 7f3b6078f640 1 -- 172.21.15.64:0/3490604419 shutdown_connections 2024-09-21T04:03:30.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.683+0000 7f3b6078f640 1 --2- 172.21.15.64:0/3490604419 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3b400b65f0 0x7f3b400b8b00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:30.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.683+0000 7f3b6078f640 1 --2- 172.21.15.64:0/3490604419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b58108cf0 0x7f3b5819eb90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:30.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.683+0000 7f3b6078f640 1 --2- 172.21.15.64:0/3490604419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b581083a0 0x7f3b581998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:30.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.683+0000 7f3b6078f640 1 --2- 172.21.15.64:0/3490604419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b58102a60 0x7f3b58199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:30.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.683+0000 7f3b6078f640 1 -- 172.21.15.64:0/3490604419 >> 172.21.15.64:0/3490604419 conn(0x7f3b58079730 msgr2=0x7f3b58101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:30.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.683+0000 7f3b6078f640 1 -- 172.21.15.64:0/3490604419 shutdown_connections 2024-09-21T04:03:30.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:30.683+0000 7f3b6078f640 1 -- 172.21.15.64:0/3490604419 wait complete. 2024-09-21T04:03:31.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:30 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [10, 11]}]': finished 2024-09-21T04:03:31.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:30 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-21T04:03:31.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:30 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [5, 6]}]': finished 2024-09-21T04:03:31.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:30 smithi119 ceph-mon[34811]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T04:03:31.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:30 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [10, 11]}]': finished 2024-09-21T04:03:31.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:30 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-21T04:03:31.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:30 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [5, 6]}]': finished 2024-09-21T04:03:31.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:30 smithi131 ceph-mon[35193]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T04:03:31.399 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- bash -c 'ceph orch device ls' 2024-09-21T04:03:31.660 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:31.907 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:31 smithi064 ceph-mon[31180]: pgmap v305: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T04:03:31.907 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:31 smithi064 ceph-mon[31180]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T04:03:31.907 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:31 smithi064 ceph-mon[31180]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-21T04:03:31.907 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:31 smithi064 ceph-mon[31180]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:03:32.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:31 smithi119 ceph-mon[34811]: pgmap v305: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T04:03:32.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:31 smithi119 ceph-mon[34811]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T04:03:32.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:31 smithi119 ceph-mon[34811]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-21T04:03:32.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:31 smithi119 ceph-mon[34811]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:03:32.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:31 smithi131 ceph-mon[35193]: pgmap v305: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T04:03:32.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:31 smithi131 ceph-mon[35193]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T04:03:32.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:31 smithi131 ceph-mon[35193]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-21T04:03:32.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:31 smithi131 ceph-mon[35193]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:03:32.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.160+0000 7fc0fbbef640 1 -- 172.21.15.64:0/4044788351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0f4109e40 msgr2=0x7fc0f4110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:32.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.160+0000 7fc0fbbef640 1 --2- 172.21.15.64:0/4044788351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0f4109e40 0x7fc0f4110670 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec009ca0 tx=0x7fc0ec02a8c0 comp rx=0 tx=0).stop 2024-09-21T04:03:32.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.160+0000 7fc0fbbef640 1 -- 172.21.15.64:0/4044788351 shutdown_connections 2024-09-21T04:03:32.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.160+0000 7fc0fbbef640 1 --2- 172.21.15.64:0/4044788351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0f4109e40 0x7fc0f4110670 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:32.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.160+0000 7fc0fbbef640 1 --2- 172.21.15.64:0/4044788351 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0f41094f0 0x7fc0f4109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:32.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.160+0000 7fc0fbbef640 1 --2- 172.21.15.64:0/4044788351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0f4103bb0 0x7fc0f4103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:32.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.160+0000 7fc0fbbef640 1 -- 172.21.15.64:0/4044788351 >> 172.21.15.64:0/4044788351 conn(0x7fc0f40ff600 msgr2=0x7fc0f4101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:32.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.160+0000 7fc0fbbef640 1 -- 172.21.15.64:0/4044788351 shutdown_connections 2024-09-21T04:03:32.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.160+0000 7fc0fbbef640 1 -- 172.21.15.64:0/4044788351 wait complete. 2024-09-21T04:03:32.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.161+0000 7fc0fbbef640 1 Processor -- start 2024-09-21T04:03:32.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.161+0000 7fc0fbbef640 1 -- start start 2024-09-21T04:03:32.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.161+0000 7fc0fbbef640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0f4103bb0 0x7fc0f419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:32.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.161+0000 7fc0fbbef640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0f41094f0 0x7fc0f419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:32.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.161+0000 7fc0fbbef640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0f4109e40 0x7fc0f419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:32.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.161+0000 7fc0fbbef640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f4113b00 con 0x7fc0f4103bb0 2024-09-21T04:03:32.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.161+0000 7fc0fbbef640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f4113980 con 0x7fc0f4109e40 2024-09-21T04:03:32.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.161+0000 7fc0fbbef640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f4113c80 con 0x7fc0f41094f0 2024-09-21T04:03:32.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.161+0000 7fc0f998b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0f4103bb0 0x7fc0f419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:32.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.161+0000 7fc0f998b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0f4103bb0 0x7fc0f419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47552/0 (socket says 172.21.15.64:47552) 2024-09-21T04:03:32.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.162+0000 7fc0f998b640 1 -- 172.21.15.64:0/2315818446 learned_addr learned my addr 172.21.15.64:0/2315818446 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:32.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.162+0000 7fc0f918a640 1 --2- 172.21.15.64:0/2315818446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0f41094f0 0x7fc0f419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:32.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.162+0000 7fc0fa18c640 1 --2- 172.21.15.64:0/2315818446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0f4109e40 0x7fc0f419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:32.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.162+0000 7fc0f998b640 1 -- 172.21.15.64:0/2315818446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0f41094f0 msgr2=0x7fc0f419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:32.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.162+0000 7fc0f998b640 1 --2- 172.21.15.64:0/2315818446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0f41094f0 0x7fc0f419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:32.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.162+0000 7fc0f998b640 1 -- 172.21.15.64:0/2315818446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0f4109e40 msgr2=0x7fc0f419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:32.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.162+0000 7fc0f998b640 1 --2- 172.21.15.64:0/2315818446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0f4109e40 0x7fc0f419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:32.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.162+0000 7fc0f998b640 1 -- 172.21.15.64:0/2315818446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f41a0340 con 0x7fc0f4103bb0 2024-09-21T04:03:32.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.162+0000 7fc0fa18c640 1 --2- 172.21.15.64:0/2315818446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0f4109e40 0x7fc0f419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:03:32.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.162+0000 7fc0f998b640 1 --2- 172.21.15.64:0/2315818446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0f4103bb0 0x7fc0f419a490 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc0f000d950 tx=0x7fc0f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:32.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.162+0000 7fc0eaffd640 1 -- 172.21.15.64:0/2315818446 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f0017070 con 0x7fc0f4103bb0 2024-09-21T04:03:32.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.162+0000 7fc0eaffd640 1 -- 172.21.15.64:0/2315818446 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0f0009c30 con 0x7fc0f4103bb0 2024-09-21T04:03:32.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.162+0000 7fc0fbbef640 1 -- 172.21.15.64:0/2315818446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f41a0630 con 0x7fc0f4103bb0 2024-09-21T04:03:32.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.162+0000 7fc0eaffd640 1 -- 172.21.15.64:0/2315818446 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f000f620 con 0x7fc0f4103bb0 2024-09-21T04:03:32.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.162+0000 7fc0fbbef640 1 -- 172.21.15.64:0/2315818446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f41a6e50 con 0x7fc0f4103bb0 2024-09-21T04:03:32.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.163+0000 7fc0fbbef640 1 -- 172.21.15.64:0/2315818446 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0f4103fc0 con 0x7fc0f4103bb0 2024-09-21T04:03:32.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.165+0000 7fc0eaffd640 1 -- 172.21.15.64:0/2315818446 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fc0f0010460 con 0x7fc0f4103bb0 2024-09-21T04:03:32.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.166+0000 7fc0eaffd640 1 --2- 172.21.15.64:0/2315818446 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc0e00b6590 0x7fc0e00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:32.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.166+0000 7fc0f918a640 1 --2- 172.21.15.64:0/2315818446 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc0e00b6590 0x7fc0e00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:32.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.166+0000 7fc0eaffd640 1 -- 172.21.15.64:0/2315818446 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fc0f00e7110 con 0x7fc0f4103bb0 2024-09-21T04:03:32.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.166+0000 7fc0f918a640 1 --2- 172.21.15.64:0/2315818446 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc0e00b6590 0x7fc0e00b8aa0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4005e10 tx=0x7fc0e4005da0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:32.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.170+0000 7fc0eaffd640 1 -- 172.21.15.64:0/2315818446 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc0f00ad8f0 con 0x7fc0f4103bb0 2024-09-21T04:03:32.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.367+0000 7fc0fbbef640 1 -- 172.21.15.64:0/2315818446 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fc0f4063800 con 0x7fc0e00b6590 2024-09-21T04:03:32.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.369+0000 7fc0eaffd640 1 -- 172.21.15.64:0/2315818446 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fc0f4063800 con 0x7fc0e00b6590 2024-09-21T04:03:32.371 INFO:teuthology.orchestra.run.smithi064.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-21T04:03:32.371 INFO:teuthology.orchestra.run.smithi064.stdout:smithi064 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT534400C2400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-21T04:03:32.371 INFO:teuthology.orchestra.run.smithi064.stdout:smithi119 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400Y1400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-21T04:03:32.371 INFO:teuthology.orchestra.run.smithi064.stdout:smithi131 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300N4400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-21T04:03:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.371+0000 7fc0fbbef640 1 -- 172.21.15.64:0/2315818446 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc0e00b6590 msgr2=0x7fc0e00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.371+0000 7fc0fbbef640 1 --2- 172.21.15.64:0/2315818446 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc0e00b6590 0x7fc0e00b8aa0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4005e10 tx=0x7fc0e4005da0 comp rx=0 tx=0).stop 2024-09-21T04:03:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.371+0000 7fc0fbbef640 1 -- 172.21.15.64:0/2315818446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0f4103bb0 msgr2=0x7fc0f419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.371+0000 7fc0fbbef640 1 --2- 172.21.15.64:0/2315818446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0f4103bb0 0x7fc0f419a490 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc0f000d950 tx=0x7fc0f000de10 comp rx=0 tx=0).stop 2024-09-21T04:03:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.371+0000 7fc0fbbef640 1 -- 172.21.15.64:0/2315818446 shutdown_connections 2024-09-21T04:03:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.371+0000 7fc0fbbef640 1 --2- 172.21.15.64:0/2315818446 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc0e00b6590 0x7fc0e00b8aa0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.371+0000 7fc0fbbef640 1 --2- 172.21.15.64:0/2315818446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0f4109e40 0x7fc0f419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.371+0000 7fc0fbbef640 1 --2- 172.21.15.64:0/2315818446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0f41094f0 0x7fc0f419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.371+0000 7fc0fbbef640 1 --2- 172.21.15.64:0/2315818446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0f4103bb0 0x7fc0f419a490 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.371+0000 7fc0fbbef640 1 -- 172.21.15.64:0/2315818446 >> 172.21.15.64:0/2315818446 conn(0x7fc0f40ff600 msgr2=0x7fc0f4101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.371+0000 7fc0fbbef640 1 -- 172.21.15.64:0/2315818446 shutdown_connections 2024-09-21T04:03:32.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:32.371+0000 7fc0fbbef640 1 -- 172.21.15.64:0/2315818446 wait complete. 2024-09-21T04:03:33.059 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T04:03:33.069 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-09-21T04:03:33.069 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- bash -c 'ceph fs dump' 2024-09-21T04:03:33.327 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:33.709 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:33 smithi064 ceph-mon[31180]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:03:33.709 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:33 smithi064 ceph-mon[31180]: pgmap v307: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 383 B/s wr, 8 op/s; 76 B/s, 1 keys/s, 0 objects/s recovering 2024-09-21T04:03:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.800+0000 7fa10cf2b640 1 -- 172.21.15.64:0/2571482816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa108108cf0 msgr2=0x7fa10810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.800+0000 7fa10cf2b640 1 --2- 172.21.15.64:0/2571482816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa108108cf0 0x7fa10810f520 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa100009ca0 tx=0x7fa10002a8c0 comp rx=0 tx=0).stop 2024-09-21T04:03:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.800+0000 7fa10cf2b640 1 -- 172.21.15.64:0/2571482816 shutdown_connections 2024-09-21T04:03:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.800+0000 7fa10cf2b640 1 --2- 172.21.15.64:0/2571482816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa108108cf0 0x7fa10810f520 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.800+0000 7fa10cf2b640 1 --2- 172.21.15.64:0/2571482816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa1081083a0 0x7fa1081087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.800+0000 7fa10cf2b640 1 --2- 172.21.15.64:0/2571482816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa108102a60 0x7fa108102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.800+0000 7fa10cf2b640 1 -- 172.21.15.64:0/2571482816 >> 172.21.15.64:0/2571482816 conn(0x7fa108079720 msgr2=0x7fa108079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.800+0000 7fa10cf2b640 1 -- 172.21.15.64:0/2571482816 shutdown_connections 2024-09-21T04:03:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.800+0000 7fa10cf2b640 1 -- 172.21.15.64:0/2571482816 wait complete. 2024-09-21T04:03:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.801+0000 7fa10cf2b640 1 Processor -- start 2024-09-21T04:03:33.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.801+0000 7fa10cf2b640 1 -- start start 2024-09-21T04:03:33.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.801+0000 7fa10cf2b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa108102a60 0x7fa1080ff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:33.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.801+0000 7fa10cf2b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1081083a0 0x7fa1080ffb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:33.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.801+0000 7fa10cf2b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa108108cf0 0x7fa108100070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:33.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.801+0000 7fa10cf2b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa108112960 con 0x7fa108102a60 2024-09-21T04:03:33.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.801+0000 7fa10cf2b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa1081127e0 con 0x7fa1081083a0 2024-09-21T04:03:33.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.801+0000 7fa10cf2b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa108112ae0 con 0x7fa108108cf0 2024-09-21T04:03:33.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.801+0000 7fa10659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa108102a60 0x7fa1080ff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:33.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.801+0000 7fa10659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa108102a60 0x7fa1080ff5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47574/0 (socket says 172.21.15.64:47574) 2024-09-21T04:03:33.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.801+0000 7fa10659c640 1 -- 172.21.15.64:0/935992301 learned_addr learned my addr 172.21.15.64:0/935992301 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:33.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.801+0000 7fa105d9b640 1 --2- 172.21.15.64:0/935992301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1081083a0 0x7fa1080ffb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:33.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.801+0000 7fa10659c640 1 -- 172.21.15.64:0/935992301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa108108cf0 msgr2=0x7fa108100070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:33.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.801+0000 7fa106d9d640 1 --2- 172.21.15.64:0/935992301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa108108cf0 0x7fa108100070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:33.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.801+0000 7fa10659c640 1 --2- 172.21.15.64:0/935992301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa108108cf0 0x7fa108100070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:33.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.801+0000 7fa10659c640 1 -- 172.21.15.64:0/935992301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1081083a0 msgr2=0x7fa1080ffb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:33.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.801+0000 7fa10659c640 1 --2- 172.21.15.64:0/935992301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1081083a0 0x7fa1080ffb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:33.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.801+0000 7fa10659c640 1 -- 172.21.15.64:0/935992301 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1081a8650 con 0x7fa108102a60 2024-09-21T04:03:33.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.801+0000 7fa106d9d640 1 --2- 172.21.15.64:0/935992301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa108108cf0 0x7fa108100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:03:33.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.802+0000 7fa10659c640 1 --2- 172.21.15.64:0/935992301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa108102a60 0x7fa1080ff5f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa0f800d950 tx=0x7fa0f800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:33.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.802+0000 7fa0ff7fe640 1 -- 172.21.15.64:0/935992301 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8017070 con 0x7fa108102a60 2024-09-21T04:03:33.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.802+0000 7fa10cf2b640 1 -- 172.21.15.64:0/935992301 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1081a8940 con 0x7fa108102a60 2024-09-21T04:03:33.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.802+0000 7fa0ff7fe640 1 -- 172.21.15.64:0/935992301 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0f800de80 con 0x7fa108102a60 2024-09-21T04:03:33.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.802+0000 7fa0ff7fe640 1 -- 172.21.15.64:0/935992301 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f800f620 con 0x7fa108102a60 2024-09-21T04:03:33.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.802+0000 7fa10cf2b640 1 -- 172.21.15.64:0/935992301 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1081a8e80 con 0x7fa108102a60 2024-09-21T04:03:33.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.803+0000 7fa10cf2b640 1 -- 172.21.15.64:0/935992301 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa108103ce0 con 0x7fa108102a60 2024-09-21T04:03:33.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.804+0000 7fa0ff7fe640 1 -- 172.21.15.64:0/935992301 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fa0f8009c30 con 0x7fa108102a60 2024-09-21T04:03:33.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.805+0000 7fa0ff7fe640 1 --2- 172.21.15.64:0/935992301 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa0f40b6560 0x7fa0f40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:33.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.805+0000 7fa105d9b640 1 --2- 172.21.15.64:0/935992301 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa0f40b6560 0x7fa0f40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:33.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.806+0000 7fa0ff7fe640 1 -- 172.21.15.64:0/935992301 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fa0f80e7110 con 0x7fa108102a60 2024-09-21T04:03:33.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.806+0000 7fa105d9b640 1 --2- 172.21.15.64:0/935992301 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa0f40b6560 0x7fa0f40b8a70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa10807a0a0 tx=0x7fa0f000a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:33.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:33.810+0000 7fa0ff7fe640 1 -- 172.21.15.64:0/935992301 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa0f80ad850 con 0x7fa108102a60 2024-09-21T04:03:34.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:33 smithi119 ceph-mon[34811]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:03:34.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:33 smithi119 ceph-mon[34811]: pgmap v307: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 383 B/s wr, 8 op/s; 76 B/s, 1 keys/s, 0 objects/s recovering 2024-09-21T04:03:34.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:34.036+0000 7fa10cf2b640 1 -- 172.21.15.64:0/935992301 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fa10802e640 con 0x7fa108102a60 2024-09-21T04:03:34.039 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:34.036+0000 7fa0ff7fe640 1 -- 172.21.15.64:0/935992301 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3086 (secure 0 0 0) 0x7fa0f80b1c30 con 0x7fa108102a60 2024-09-21T04:03:34.039 INFO:teuthology.orchestra.run.smithi064.stdout:e24 2024-09-21T04:03:34.039 INFO:teuthology.orchestra.run.smithi064.stdout:btime 2024-09-21T04:02:14:798588+0000 2024-09-21T04:03:34.039 INFO:teuthology.orchestra.run.smithi064.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T04:03:34.039 INFO:teuthology.orchestra.run.smithi064.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:03:34.039 INFO:teuthology.orchestra.run.smithi064.stdout:legacy client fscid: 1 2024-09-21T04:03:34.039 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:03:34.039 INFO:teuthology.orchestra.run.smithi064.stdout:Filesystem 'cephfs' (1) 2024-09-21T04:03:34.040 INFO:teuthology.orchestra.run.smithi064.stdout:fs_name cephfs 2024-09-21T04:03:34.040 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 24 2024-09-21T04:03:34.040 INFO:teuthology.orchestra.run.smithi064.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T04:03:34.040 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-09-21T04:02:03.354744+0000 2024-09-21T04:03:34.040 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-09-21T04:02:14.579750+0000 2024-09-21T04:03:34.040 INFO:teuthology.orchestra.run.smithi064.stdout:tableserver 0 2024-09-21T04:03:34.040 INFO:teuthology.orchestra.run.smithi064.stdout:root 0 2024-09-21T04:03:34.040 INFO:teuthology.orchestra.run.smithi064.stdout:session_timeout 300 2024-09-21T04:03:34.040 INFO:teuthology.orchestra.run.smithi064.stdout:session_autoclose 300 2024-09-21T04:03:34.040 INFO:teuthology.orchestra.run.smithi064.stdout:max_file_size 1099511627776 2024-09-21T04:03:34.041 INFO:teuthology.orchestra.run.smithi064.stdout:max_xattr_size 65536 2024-09-21T04:03:34.041 INFO:teuthology.orchestra.run.smithi064.stdout:required_client_features {} 2024-09-21T04:03:34.041 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure 0 2024-09-21T04:03:34.041 INFO:teuthology.orchestra.run.smithi064.stdout:last_failure_osd_epoch 0 2024-09-21T04:03:34.041 INFO:teuthology.orchestra.run.smithi064.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T04:03:34.041 INFO:teuthology.orchestra.run.smithi064.stdout:max_mds 3 2024-09-21T04:03:34.041 INFO:teuthology.orchestra.run.smithi064.stdout:in 0,1,2 2024-09-21T04:03:34.041 INFO:teuthology.orchestra.run.smithi064.stdout:up {0=24710,1=24680,2=24661} 2024-09-21T04:03:34.041 INFO:teuthology.orchestra.run.smithi064.stdout:failed 2024-09-21T04:03:34.041 INFO:teuthology.orchestra.run.smithi064.stdout:damaged 2024-09-21T04:03:34.041 INFO:teuthology.orchestra.run.smithi064.stdout:stopped 2024-09-21T04:03:34.041 INFO:teuthology.orchestra.run.smithi064.stdout:data_pools [3] 2024-09-21T04:03:34.041 INFO:teuthology.orchestra.run.smithi064.stdout:metadata_pool 2 2024-09-21T04:03:34.041 INFO:teuthology.orchestra.run.smithi064.stdout:inline_data disabled 2024-09-21T04:03:34.042 INFO:teuthology.orchestra.run.smithi064.stdout:balancer 2024-09-21T04:03:34.042 INFO:teuthology.orchestra.run.smithi064.stdout:bal_rank_mask -1 2024-09-21T04:03:34.042 INFO:teuthology.orchestra.run.smithi064.stdout:standby_count_wanted 0 2024-09-21T04:03:34.042 INFO:teuthology.orchestra.run.smithi064.stdout:qdb_cluster leader: 24710 members: 24661,24680,24710 2024-09-21T04:03:34.042 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.b{0:24710} state up:active seq 4 addr [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:03:34.042 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.e{0:24656} state up:standby-replay seq 1 addr [v2:172.21.15.119:6834/3675946142,v1:172.21.15.119:6835/3675946142] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:03:34.042 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.h{1:24680} state up:active seq 9 addr [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:03:34.042 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.c{1:24640} state up:standby-replay seq 1 addr [v2:172.21.15.131:6836/843209972,v1:172.21.15.131:6837/843209972] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:03:34.042 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.i{2:24661} state up:active seq 8 addr [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:03:34.042 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.f{2:24634} state up:standby-replay seq 1 addr [v2:172.21.15.131:6834/1422569359,v1:172.21.15.131:6835/1422569359] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:03:34.042 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:03:34.042 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:03:34.042 INFO:teuthology.orchestra.run.smithi064.stdout:Standby daemons: 2024-09-21T04:03:34.042 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:03:34.043 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.j{-1:14748} state up:standby seq 1 addr [v2:172.21.15.64:6834/2075966415,v1:172.21.15.64:6835/2075966415] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:03:34.043 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.d{-1:14760} state up:standby seq 1 addr [v2:172.21.15.64:6836/3072311987,v1:172.21.15.64:6837/3072311987] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:03:34.043 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.g{-1:14805} state up:standby seq 1 addr [v2:172.21.15.64:6838/97995119,v1:172.21.15.64:6839/97995119] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:03:34.043 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.a{-1:14817} state up:standby seq 1 addr [v2:172.21.15.64:6840/390891068,v1:172.21.15.64:6841/390891068] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:03:34.043 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.l{-1:24589} state up:standby seq 1 addr [v2:172.21.15.131:6832/1651684499,v1:172.21.15.131:6833/1651684499] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:03:34.043 INFO:teuthology.orchestra.run.smithi064.stdout:[mds.k{-1:24620} state up:standby seq 1 addr [v2:172.21.15.119:6832/1724707326,v1:172.21.15.119:6833/1724707326] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T04:03:34.044 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-09-21T04:03:34.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:34.039+0000 7fa10cf2b640 1 -- 172.21.15.64:0/935992301 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa0f40b6560 msgr2=0x7fa0f40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:34.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:34.039+0000 7fa10cf2b640 1 --2- 172.21.15.64:0/935992301 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa0f40b6560 0x7fa0f40b8a70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa10807a0a0 tx=0x7fa0f000a2d0 comp rx=0 tx=0).stop 2024-09-21T04:03:34.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:34.039+0000 7fa10cf2b640 1 -- 172.21.15.64:0/935992301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa108102a60 msgr2=0x7fa1080ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:34.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:34.039+0000 7fa10cf2b640 1 --2- 172.21.15.64:0/935992301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa108102a60 0x7fa1080ff5f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa0f800d950 tx=0x7fa0f800dd50 comp rx=0 tx=0).stop 2024-09-21T04:03:34.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:34.039+0000 7fa10cf2b640 1 -- 172.21.15.64:0/935992301 shutdown_connections 2024-09-21T04:03:34.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:34.039+0000 7fa10cf2b640 1 --2- 172.21.15.64:0/935992301 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa0f40b6560 0x7fa0f40b8a70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:34.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:34.039+0000 7fa10cf2b640 1 --2- 172.21.15.64:0/935992301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa108108cf0 0x7fa108100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:34.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:34.039+0000 7fa10cf2b640 1 --2- 172.21.15.64:0/935992301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1081083a0 0x7fa1080ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:34.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:34.039+0000 7fa10cf2b640 1 --2- 172.21.15.64:0/935992301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa108102a60 0x7fa1080ff5f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:34.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:34.039+0000 7fa10cf2b640 1 -- 172.21.15.64:0/935992301 >> 172.21.15.64:0/935992301 conn(0x7fa108079720 msgr2=0x7fa10806bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:34.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:34.039+0000 7fa10cf2b640 1 -- 172.21.15.64:0/935992301 shutdown_connections 2024-09-21T04:03:34.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:34.039+0000 7fa10cf2b640 1 -- 172.21.15.64:0/935992301 wait complete. 2024-09-21T04:03:34.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:33 smithi131 ceph-mon[35193]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:03:34.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:33 smithi131 ceph-mon[35193]: pgmap v307: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 383 B/s wr, 8 op/s; 76 B/s, 1 keys/s, 0 objects/s recovering 2024-09-21T04:03:34.723 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:34 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/935992301' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:03:34.751 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- bash -c 'ceph osd dump' 2024-09-21T04:03:35.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:34 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/935992301' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:03:35.007 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:35.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:34 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/935992301' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T04:03:35.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.532+0000 7fed5bb45640 1 -- 172.21.15.64:0/693655123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed54104a30 msgr2=0x7fed54104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:35.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.532+0000 7fed5bb45640 1 --2- 172.21.15.64:0/693655123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed54104a30 0x7fed54104e40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fed50009900 tx=0x7fed5002a410 comp rx=0 tx=0).stop 2024-09-21T04:03:35.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.532+0000 7fed5bb45640 1 -- 172.21.15.64:0/693655123 shutdown_connections 2024-09-21T04:03:35.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.532+0000 7fed5bb45640 1 --2- 172.21.15.64:0/693655123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed5410de60 0x7fed54110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:35.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.532+0000 7fed5bb45640 1 --2- 172.21.15.64:0/693655123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed54105380 0x7fed5410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:35.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.532+0000 7fed5bb45640 1 --2- 172.21.15.64:0/693655123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed54104a30 0x7fed54104e40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:35.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.532+0000 7fed5bb45640 1 -- 172.21.15.64:0/693655123 >> 172.21.15.64:0/693655123 conn(0x7fed54079760 msgr2=0x7fed54079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:35.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.532+0000 7fed5bb45640 1 -- 172.21.15.64:0/693655123 shutdown_connections 2024-09-21T04:03:35.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.533+0000 7fed5bb45640 1 -- 172.21.15.64:0/693655123 wait complete. 2024-09-21T04:03:35.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.533+0000 7fed5bb45640 1 Processor -- start 2024-09-21T04:03:35.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.533+0000 7fed5bb45640 1 -- start start 2024-09-21T04:03:35.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.533+0000 7fed5bb45640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed54104a30 0x7fed5419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:35.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.533+0000 7fed5bb45640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed54105380 0x7fed5419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:35.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.534+0000 7fed5bb45640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed5410de60 0x7fed5419fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:35.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.534+0000 7fed5bb45640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fed54113b00 con 0x7fed54104a30 2024-09-21T04:03:35.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.534+0000 7fed5bb45640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fed54113980 con 0x7fed54105380 2024-09-21T04:03:35.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.534+0000 7fed5bb45640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fed54113c80 con 0x7fed5410de60 2024-09-21T04:03:35.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.534+0000 7fed598e1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed54104a30 0x7fed5419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:35.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.534+0000 7fed598e1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed54104a30 0x7fed5419a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47586/0 (socket says 172.21.15.64:47586) 2024-09-21T04:03:35.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.534+0000 7fed598e1640 1 -- 172.21.15.64:0/2567187339 learned_addr learned my addr 172.21.15.64:0/2567187339 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:35.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.534+0000 7fed5a0e2640 1 --2- 172.21.15.64:0/2567187339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed5410de60 0x7fed5419fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:35.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.534+0000 7fed590e0640 1 --2- 172.21.15.64:0/2567187339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed54105380 0x7fed5419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:35.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.534+0000 7fed598e1640 1 -- 172.21.15.64:0/2567187339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed5410de60 msgr2=0x7fed5419fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:35.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.534+0000 7fed598e1640 1 --2- 172.21.15.64:0/2567187339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed5410de60 0x7fed5419fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:35.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.534+0000 7fed598e1640 1 -- 172.21.15.64:0/2567187339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed54105380 msgr2=0x7fed5419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:35.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.534+0000 7fed598e1640 1 --2- 172.21.15.64:0/2567187339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed54105380 0x7fed5419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:35.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.534+0000 7fed598e1640 1 -- 172.21.15.64:0/2567187339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed541a0400 con 0x7fed54104a30 2024-09-21T04:03:35.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.534+0000 7fed590e0640 1 --2- 172.21.15.64:0/2567187339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed54105380 0x7fed5419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:03:35.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.534+0000 7fed5a0e2640 1 --2- 172.21.15.64:0/2567187339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed5410de60 0x7fed5419fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:03:35.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.534+0000 7fed598e1640 1 --2- 172.21.15.64:0/2567187339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed54104a30 0x7fed5419a550 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fed50031a90 tx=0x7fed50004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:35.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.534+0000 7fed4affd640 1 -- 172.21.15.64:0/2567187339 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5003f030 con 0x7fed54104a30 2024-09-21T04:03:35.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.534+0000 7fed4affd640 1 -- 172.21.15.64:0/2567187339 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed50003cb0 con 0x7fed54104a30 2024-09-21T04:03:35.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.535+0000 7fed5bb45640 1 -- 172.21.15.64:0/2567187339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed541a0690 con 0x7fed54104a30 2024-09-21T04:03:35.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.535+0000 7fed4affd640 1 -- 172.21.15.64:0/2567187339 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50035070 con 0x7fed54104a30 2024-09-21T04:03:35.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.535+0000 7fed5bb45640 1 -- 172.21.15.64:0/2567187339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed541a6f10 con 0x7fed54104a30 2024-09-21T04:03:35.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.535+0000 7fed5bb45640 1 -- 172.21.15.64:0/2567187339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed5406c1f0 con 0x7fed54104a30 2024-09-21T04:03:35.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.537+0000 7fed4affd640 1 -- 172.21.15.64:0/2567187339 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fed50042030 con 0x7fed54104a30 2024-09-21T04:03:35.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.538+0000 7fed4affd640 1 --2- 172.21.15.64:0/2567187339 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fed400b6590 0x7fed400b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:35.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.538+0000 7fed590e0640 1 --2- 172.21.15.64:0/2567187339 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fed400b6590 0x7fed400b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:35.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.538+0000 7fed4affd640 1 -- 172.21.15.64:0/2567187339 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fed50107f60 con 0x7fed54104a30 2024-09-21T04:03:35.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.539+0000 7fed590e0640 1 --2- 172.21.15.64:0/2567187339 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fed400b6590 0x7fed400b8aa0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fed5407a140 tx=0x7fed44008040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:35.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.542+0000 7fed4affd640 1 -- 172.21.15.64:0/2567187339 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fed500ce6a0 con 0x7fed54104a30 2024-09-21T04:03:35.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.732+0000 7fed5bb45640 1 -- 172.21.15.64:0/2567187339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fed54108f40 con 0x7fed54104a30 2024-09-21T04:03:35.735 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:35 smithi064 ceph-mon[31180]: pgmap v308: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 383 B/s wr, 6 op/s; 76 B/s, 1 keys/s, 0 objects/s recovering 2024-09-21T04:03:35.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.734+0000 7fed4affd640 1 -- 172.21.15.64:0/2567187339 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v77) ==== 56+0+5802 (secure 0 0 0) 0x7fed500d2a80 con 0x7fed54104a30 2024-09-21T04:03:35.736 INFO:teuthology.orchestra.run.smithi064.stdout:epoch 77 2024-09-21T04:03:35.736 INFO:teuthology.orchestra.run.smithi064.stdout:fsid e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T04:03:35.736 INFO:teuthology.orchestra.run.smithi064.stdout:created 2024-09-21T03:53:41.215073+0000 2024-09-21T04:03:35.736 INFO:teuthology.orchestra.run.smithi064.stdout:modified 2024-09-21T04:03:30.555854+0000 2024-09-21T04:03:35.737 INFO:teuthology.orchestra.run.smithi064.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T04:03:35.737 INFO:teuthology.orchestra.run.smithi064.stdout:crush_version 26 2024-09-21T04:03:35.737 INFO:teuthology.orchestra.run.smithi064.stdout:full_ratio 0.95 2024-09-21T04:03:35.737 INFO:teuthology.orchestra.run.smithi064.stdout:backfillfull_ratio 0.9 2024-09-21T04:03:35.737 INFO:teuthology.orchestra.run.smithi064.stdout:nearfull_ratio 0.85 2024-09-21T04:03:35.737 INFO:teuthology.orchestra.run.smithi064.stdout:require_min_compat_client luminous 2024-09-21T04:03:35.737 INFO:teuthology.orchestra.run.smithi064.stdout:min_compat_client luminous 2024-09-21T04:03:35.737 INFO:teuthology.orchestra.run.smithi064.stdout:require_osd_release squid 2024-09-21T04:03:35.737 INFO:teuthology.orchestra.run.smithi064.stdout:stretch_mode_enabled false 2024-09-21T04:03:35.737 INFO:teuthology.orchestra.run.smithi064.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-21T04:03:35.737 INFO:teuthology.orchestra.run.smithi064.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 73 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-09-21T04:03:35.737 INFO:teuthology.orchestra.run.smithi064.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 73 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-21T04:03:35.737 INFO:teuthology.orchestra.run.smithi064.stdout:max_osd 12 2024-09-21T04:03:35.738 INFO:teuthology.orchestra.run.smithi064.stdout:osd.0 up in weight 1 up_from 9 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6802/3692023035,v1:172.21.15.64:6803/3692023035] [v2:172.21.15.64:6804/3692023035,v1:172.21.15.64:6805/3692023035] exists,up fb947c10-98e6-4e3a-8bbd-8753951d870b 2024-09-21T04:03:35.738 INFO:teuthology.orchestra.run.smithi064.stdout:osd.1 up in weight 1 up_from 14 up_thru 71 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6800/3363415998,v1:172.21.15.119:6801/3363415998] [v2:172.21.15.119:6802/3363415998,v1:172.21.15.119:6803/3363415998] exists,up 29b8c92d-82fa-4175-b3e2-711917b7f8c2 2024-09-21T04:03:35.738 INFO:teuthology.orchestra.run.smithi064.stdout:osd.2 up in weight 1 up_from 19 up_thru 71 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6800/3422924179,v1:172.21.15.131:6801/3422924179] [v2:172.21.15.131:6802/3422924179,v1:172.21.15.131:6803/3422924179] exists,up a2273fe8-df55-40f7-afda-74ab09c52c55 2024-09-21T04:03:35.738 INFO:teuthology.orchestra.run.smithi064.stdout:osd.3 up in weight 1 up_from 25 up_thru 71 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6810/2022952729,v1:172.21.15.64:6811/2022952729] [v2:172.21.15.64:6812/2022952729,v1:172.21.15.64:6813/2022952729] exists,up f1634064-b108-41d2-a78f-c33607b08a70 2024-09-21T04:03:35.738 INFO:teuthology.orchestra.run.smithi064.stdout:osd.4 up in weight 1 up_from 30 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6808/323030958,v1:172.21.15.119:6809/323030958] [v2:172.21.15.119:6810/323030958,v1:172.21.15.119:6811/323030958] exists,up 75b81a8d-fa52-411a-bfda-aa69bb70a20f 2024-09-21T04:03:35.738 INFO:teuthology.orchestra.run.smithi064.stdout:osd.5 up in weight 1 up_from 35 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6808/3021788088,v1:172.21.15.131:6809/3021788088] [v2:172.21.15.131:6810/3021788088,v1:172.21.15.131:6811/3021788088] exists,up 0e4895b5-42e4-4b2c-a9c3-692f287cb841 2024-09-21T04:03:35.738 INFO:teuthology.orchestra.run.smithi064.stdout:osd.6 up in weight 1 up_from 41 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6818/1913824324,v1:172.21.15.64:6819/1913824324] [v2:172.21.15.64:6820/1913824324,v1:172.21.15.64:6821/1913824324] exists,up 1fa5c2b7-bce0-43a1-a26b-d029eb3731fd 2024-09-21T04:03:35.738 INFO:teuthology.orchestra.run.smithi064.stdout:osd.7 up in weight 1 up_from 46 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6816/2567568166,v1:172.21.15.119:6817/2567568166] [v2:172.21.15.119:6818/2567568166,v1:172.21.15.119:6819/2567568166] exists,up c9707260-b581-4a21-b380-f37e16959b02 2024-09-21T04:03:35.738 INFO:teuthology.orchestra.run.smithi064.stdout:osd.8 up in weight 1 up_from 51 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6816/199192217,v1:172.21.15.131:6817/199192217] [v2:172.21.15.131:6818/199192217,v1:172.21.15.131:6819/199192217] exists,up 2dbf8ef2-fc9c-4c6b-bfea-003f41ef191e 2024-09-21T04:03:35.738 INFO:teuthology.orchestra.run.smithi064.stdout:osd.9 up in weight 1 up_from 57 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6826/3309038990,v1:172.21.15.64:6827/3309038990] [v2:172.21.15.64:6828/3309038990,v1:172.21.15.64:6829/3309038990] exists,up 96f769e5-4783-4558-adfa-4bfaf9da9e9a 2024-09-21T04:03:35.738 INFO:teuthology.orchestra.run.smithi064.stdout:osd.10 up in weight 1 up_from 62 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6824/1460844681,v1:172.21.15.119:6825/1460844681] [v2:172.21.15.119:6826/1460844681,v1:172.21.15.119:6827/1460844681] exists,up 13c1ef5c-2299-440c-af5a-a650b47f4b75 2024-09-21T04:03:35.738 INFO:teuthology.orchestra.run.smithi064.stdout:osd.11 up in weight 1 up_from 67 up_thru 74 down_at 0 last_clean_interval [0,0) [v2:172.21.15.131:6824/2294059084,v1:172.21.15.131:6825/2294059084] [v2:172.21.15.131:6826/2294059084,v1:172.21.15.131:6827/2294059084] exists,up c26a5ef9-8585-404e-a54a-852424c4d6fd 2024-09-21T04:03:35.739 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 2.0 [8,4] 2024-09-21T04:03:35.739 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 2.3 [10,11] 2024-09-21T04:03:35.739 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 2.b [2,1] 2024-09-21T04:03:35.739 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 2.c [2,4] 2024-09-21T04:03:35.739 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 2.e [2,4] 2024-09-21T04:03:35.739 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 2.1a [8,1] 2024-09-21T04:03:35.739 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 2.1c [5,6] 2024-09-21T04:03:35.739 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 2.1f [7,11] 2024-09-21T04:03:35.739 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.e [5,9] 2024-09-21T04:03:35.740 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.13 [4,9] 2024-09-21T04:03:35.740 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.17 [4,8] 2024-09-21T04:03:35.740 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.1d [4,6] 2024-09-21T04:03:35.740 INFO:teuthology.orchestra.run.smithi064.stdout:pg_upmap_items 3.27 [3,9] 2024-09-21T04:03:35.740 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/3554471670 expires 2024-09-22T03:55:28.387426+0000 2024-09-21T04:03:35.740 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:6801/3786213009 expires 2024-09-22T03:55:28.387426+0000 2024-09-21T04:03:35.740 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/2562856406 expires 2024-09-22T03:54:27.180100+0000 2024-09-21T04:03:35.741 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/2603015699 expires 2024-09-22T03:55:28.387426+0000 2024-09-21T04:03:35.741 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:6800/1209932961 expires 2024-09-22T03:54:02.562380+0000 2024-09-21T04:03:35.741 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:6800/3786213009 expires 2024-09-22T03:55:28.387426+0000 2024-09-21T04:03:35.741 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:6801/1209932961 expires 2024-09-22T03:54:02.562380+0000 2024-09-21T04:03:35.741 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/3407344596 expires 2024-09-22T03:54:02.562380+0000 2024-09-21T04:03:35.741 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/1233731831 expires 2024-09-22T03:54:02.562380+0000 2024-09-21T04:03:35.741 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/1686887701 expires 2024-09-22T03:54:02.562380+0000 2024-09-21T04:03:35.742 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/1598568021 expires 2024-09-22T03:54:27.180100+0000 2024-09-21T04:03:35.742 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:6800/2013929312 expires 2024-09-22T03:54:27.180100+0000 2024-09-21T04:03:35.742 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:6801/2013929312 expires 2024-09-22T03:54:27.180100+0000 2024-09-21T04:03:35.742 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/3652036144 expires 2024-09-22T03:55:28.387426+0000 2024-09-21T04:03:35.742 INFO:teuthology.orchestra.run.smithi064.stdout:blocklist 172.21.15.64:0/105454703 expires 2024-09-22T03:54:27.180100+0000 2024-09-21T04:03:35.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.736+0000 7fed5bb45640 1 -- 172.21.15.64:0/2567187339 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fed400b6590 msgr2=0x7fed400b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:35.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.736+0000 7fed5bb45640 1 --2- 172.21.15.64:0/2567187339 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fed400b6590 0x7fed400b8aa0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fed5407a140 tx=0x7fed44008040 comp rx=0 tx=0).stop 2024-09-21T04:03:35.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.736+0000 7fed5bb45640 1 -- 172.21.15.64:0/2567187339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed54104a30 msgr2=0x7fed5419a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:35.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.736+0000 7fed5bb45640 1 --2- 172.21.15.64:0/2567187339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed54104a30 0x7fed5419a550 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fed50031a90 tx=0x7fed50004270 comp rx=0 tx=0).stop 2024-09-21T04:03:35.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.737+0000 7fed5bb45640 1 -- 172.21.15.64:0/2567187339 shutdown_connections 2024-09-21T04:03:35.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.737+0000 7fed5bb45640 1 --2- 172.21.15.64:0/2567187339 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fed400b6590 0x7fed400b8aa0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:35.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.737+0000 7fed5bb45640 1 --2- 172.21.15.64:0/2567187339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed5410de60 0x7fed5419fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:35.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.737+0000 7fed5bb45640 1 --2- 172.21.15.64:0/2567187339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed54105380 0x7fed5419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:35.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.737+0000 7fed5bb45640 1 --2- 172.21.15.64:0/2567187339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fed54104a30 0x7fed5419a550 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:35.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.737+0000 7fed5bb45640 1 -- 172.21.15.64:0/2567187339 >> 172.21.15.64:0/2567187339 conn(0x7fed54079760 msgr2=0x7fed541008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:35.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.737+0000 7fed5bb45640 1 -- 172.21.15.64:0/2567187339 shutdown_connections 2024-09-21T04:03:35.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:35.737+0000 7fed5bb45640 1 -- 172.21.15.64:0/2567187339 wait complete. 2024-09-21T04:03:36.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:35 smithi119 ceph-mon[34811]: pgmap v308: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 383 B/s wr, 6 op/s; 76 B/s, 1 keys/s, 0 objects/s recovering 2024-09-21T04:03:36.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:35 smithi131 ceph-mon[35193]: pgmap v308: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 383 B/s wr, 6 op/s; 76 B/s, 1 keys/s, 0 objects/s recovering 2024-09-21T04:03:36.514 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-21T04:03:36.524 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:03:36.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:36 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2567187339' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T04:03:36.786 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:37.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:36 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2567187339' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T04:03:37.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:36 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2567187339' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T04:03:37.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.235+0000 7fa7f0cf1640 1 -- 172.21.15.64:0/3916771829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ec102a80 msgr2=0x7fa7ec102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:37.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.235+0000 7fa7f0cf1640 1 --2- 172.21.15.64:0/3916771829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ec102a80 0x7fa7ec102e90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc009900 tx=0x7fa7dc02a410 comp rx=0 tx=0).stop 2024-09-21T04:03:37.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.235+0000 7fa7f0cf1640 1 -- 172.21.15.64:0/3916771829 shutdown_connections 2024-09-21T04:03:37.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.235+0000 7fa7f0cf1640 1 --2- 172.21.15.64:0/3916771829 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7ec108d10 0x7fa7ec10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:37.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.235+0000 7fa7f0cf1640 1 --2- 172.21.15.64:0/3916771829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ec1083c0 0x7fa7ec1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:37.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.235+0000 7fa7f0cf1640 1 --2- 172.21.15.64:0/3916771829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ec102a80 0x7fa7ec102e90 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:37.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.235+0000 7fa7f0cf1640 1 -- 172.21.15.64:0/3916771829 >> 172.21.15.64:0/3916771829 conn(0x7fa7ec079740 msgr2=0x7fa7ec079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:37.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.235+0000 7fa7f0cf1640 1 -- 172.21.15.64:0/3916771829 shutdown_connections 2024-09-21T04:03:37.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.235+0000 7fa7f0cf1640 1 -- 172.21.15.64:0/3916771829 wait complete. 2024-09-21T04:03:37.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.236+0000 7fa7f0cf1640 1 Processor -- start 2024-09-21T04:03:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.236+0000 7fa7f0cf1640 1 -- start start 2024-09-21T04:03:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.236+0000 7fa7f0cf1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7ec102a80 0x7fa7ec19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.236+0000 7fa7f0cf1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ec1083c0 0x7fa7ec19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.236+0000 7fa7f0cf1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ec108d10 0x7fa7ec19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.236+0000 7fa7f0cf1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec112980 con 0x7fa7ec1083c0 2024-09-21T04:03:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.236+0000 7fa7f0cf1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec112800 con 0x7fa7ec108d10 2024-09-21T04:03:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.236+0000 7fa7f0cf1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec112b00 con 0x7fa7ec102a80 2024-09-21T04:03:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.236+0000 7fa7e9d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ec1083c0 0x7fa7ec19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.236+0000 7fa7e9d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ec1083c0 0x7fa7ec19aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47596/0 (socket says 172.21.15.64:47596) 2024-09-21T04:03:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.236+0000 7fa7e9d9b640 1 -- 172.21.15.64:0/1607844036 learned_addr learned my addr 172.21.15.64:0/1607844036 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.236+0000 7fa7ead9d640 1 --2- 172.21.15.64:0/1607844036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ec108d10 0x7fa7ec19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:37.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.236+0000 7fa7ea59c640 1 --2- 172.21.15.64:0/1607844036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7ec102a80 0x7fa7ec19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.237+0000 7fa7e9d9b640 1 -- 172.21.15.64:0/1607844036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7ec102a80 msgr2=0x7fa7ec19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.237+0000 7fa7e9d9b640 1 --2- 172.21.15.64:0/1607844036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7ec102a80 0x7fa7ec19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.237+0000 7fa7e9d9b640 1 -- 172.21.15.64:0/1607844036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ec108d10 msgr2=0x7fa7ec19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.237+0000 7fa7e9d9b640 1 --2- 172.21.15.64:0/1607844036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ec108d10 0x7fa7ec19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.237+0000 7fa7e9d9b640 1 -- 172.21.15.64:0/1607844036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7ec1a0470 con 0x7fa7ec1083c0 2024-09-21T04:03:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.237+0000 7fa7ea59c640 1 --2- 172.21.15.64:0/1607844036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7ec102a80 0x7fa7ec19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:03:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.237+0000 7fa7e9d9b640 1 --2- 172.21.15.64:0/1607844036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ec1083c0 0x7fa7ec19aaa0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000e9c0 tx=0x7fa7e000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.237+0000 7fa7db7fe640 1 -- 172.21.15.64:0/1607844036 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e000cd80 con 0x7fa7ec1083c0 2024-09-21T04:03:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.237+0000 7fa7db7fe640 1 -- 172.21.15.64:0/1607844036 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7e00054d0 con 0x7fa7ec1083c0 2024-09-21T04:03:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.237+0000 7fa7f0cf1640 1 -- 172.21.15.64:0/1607844036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7ec1a0760 con 0x7fa7ec1083c0 2024-09-21T04:03:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.237+0000 7fa7db7fe640 1 -- 172.21.15.64:0/1607844036 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0010430 con 0x7fa7ec1083c0 2024-09-21T04:03:37.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.237+0000 7fa7f0cf1640 1 -- 172.21.15.64:0/1607844036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7ec1a6ec0 con 0x7fa7ec1083c0 2024-09-21T04:03:37.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.238+0000 7fa7f0cf1640 1 -- 172.21.15.64:0/1607844036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7ec103d00 con 0x7fa7ec1083c0 2024-09-21T04:03:37.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.241+0000 7fa7db7fe640 1 -- 172.21.15.64:0/1607844036 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fa7e0004fb0 con 0x7fa7ec1083c0 2024-09-21T04:03:37.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.241+0000 7fa7db7fe640 1 --2- 172.21.15.64:0/1607844036 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa7d40b65e0 0x7fa7d40b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:37.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.242+0000 7fa7ea59c640 1 --2- 172.21.15.64:0/1607844036 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa7d40b65e0 0x7fa7d40b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:37.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.242+0000 7fa7db7fe640 1 -- 172.21.15.64:0/1607844036 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fa7e00e7b50 con 0x7fa7ec1083c0 2024-09-21T04:03:37.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.242+0000 7fa7ea59c640 1 --2- 172.21.15.64:0/1607844036 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa7d40b65e0 0x7fa7d40b8af0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc031440 tx=0x7fa7dc009680 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:37.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.245+0000 7fa7db7fe640 1 -- 172.21.15.64:0/1607844036 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa7e0056710 con 0x7fa7ec1083c0 2024-09-21T04:03:37.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.472+0000 7fa7f0cf1640 1 -- 172.21.15.64:0/1607844036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7ec102e90 con 0x7fa7ec1083c0 2024-09-21T04:03:37.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.472+0000 7fa7db7fe640 1 -- 172.21.15.64:0/1607844036 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11911 (secure 0 0 0) 0x7fa7ec102e90 con 0x7fa7ec1083c0 2024-09-21T04:03:37.475 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:03:37.475 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"btime":"2024-09-21T04:02:14:798588+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14805,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/97995119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":97995119},{"type":"v1","addr":"172.21.15.64:6839","nonce":97995119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/390891068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":390891068},{"type":"v1","addr":"172.21.15.64:6841","nonce":390891068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:02:03.354744+0000","modified":"2024-09-21T04:02:14.579750+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24710,"mds_1":24680,"mds_2":24661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24634":{"gid":24634,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6839/782190424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782190424},{"type":"v1","addr":"172.21.15.131:6839","nonce":782190424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24680":{"gid":24680,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24710":{"gid":24710,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6839/1319793636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1319793636},{"type":"v1","addr":"172.21.15.119:6839","nonce":1319793636}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24710,"qdb_cluster":[24661,24680,24710]},"id":1}]} 2024-09-21T04:03:37.475 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-09-21T04:03:37.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.475+0000 7fa7f0cf1640 1 -- 172.21.15.64:0/1607844036 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa7d40b65e0 msgr2=0x7fa7d40b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:37.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.475+0000 7fa7f0cf1640 1 --2- 172.21.15.64:0/1607844036 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa7d40b65e0 0x7fa7d40b8af0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc031440 tx=0x7fa7dc009680 comp rx=0 tx=0).stop 2024-09-21T04:03:37.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.475+0000 7fa7f0cf1640 1 -- 172.21.15.64:0/1607844036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ec1083c0 msgr2=0x7fa7ec19aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:37.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.475+0000 7fa7f0cf1640 1 --2- 172.21.15.64:0/1607844036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ec1083c0 0x7fa7ec19aaa0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000e9c0 tx=0x7fa7e000ee80 comp rx=0 tx=0).stop 2024-09-21T04:03:37.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.475+0000 7fa7f0cf1640 1 -- 172.21.15.64:0/1607844036 shutdown_connections 2024-09-21T04:03:37.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.475+0000 7fa7f0cf1640 1 --2- 172.21.15.64:0/1607844036 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa7d40b65e0 0x7fa7d40b8af0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:37.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.475+0000 7fa7f0cf1640 1 --2- 172.21.15.64:0/1607844036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7ec108d10 0x7fa7ec19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:37.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.475+0000 7fa7f0cf1640 1 --2- 172.21.15.64:0/1607844036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7ec1083c0 0x7fa7ec19aaa0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:37.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.475+0000 7fa7f0cf1640 1 --2- 172.21.15.64:0/1607844036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7ec102a80 0x7fa7ec19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:37.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.475+0000 7fa7f0cf1640 1 -- 172.21.15.64:0/1607844036 >> 172.21.15.64:0/1607844036 conn(0x7fa7ec079740 msgr2=0x7fa7ec101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:37.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.475+0000 7fa7f0cf1640 1 -- 172.21.15.64:0/1607844036 shutdown_connections 2024-09-21T04:03:37.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:37.475+0000 7fa7f0cf1640 1 -- 172.21.15.64:0/1607844036 wait complete. 2024-09-21T04:03:37.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:37 smithi064 ceph-mon[31180]: pgmap v309: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 383 B/s wr, 6 op/s; 76 B/s, 1 keys/s, 0 objects/s recovering 2024-09-21T04:03:37.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:37 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1607844036' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:03:38.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:37 smithi119 ceph-mon[34811]: pgmap v309: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 383 B/s wr, 6 op/s; 76 B/s, 1 keys/s, 0 objects/s recovering 2024-09-21T04:03:38.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:37 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1607844036' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:03:38.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:37 smithi131 ceph-mon[35193]: pgmap v309: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 383 B/s wr, 6 op/s; 76 B/s, 1 keys/s, 0 objects/s recovering 2024-09-21T04:03:38.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:37 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1607844036' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:03:38.273 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:03:38.532 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:39.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.003+0000 7f0bbe2f8640 1 -- 172.21.15.64:0/3714497333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bb81083a0 msgr2=0x7f0bb81087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:39.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.003+0000 7f0bbe2f8640 1 --2- 172.21.15.64:0/3714497333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bb81083a0 0x7f0bb81087b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0bac009900 tx=0x7f0bac02a410 comp rx=0 tx=0).stop 2024-09-21T04:03:39.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.003+0000 7f0bbe2f8640 1 -- 172.21.15.64:0/3714497333 shutdown_connections 2024-09-21T04:03:39.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.003+0000 7f0bbe2f8640 1 --2- 172.21.15.64:0/3714497333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bb8108cf0 0x7f0bb810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:39.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.003+0000 7f0bbe2f8640 1 --2- 172.21.15.64:0/3714497333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bb81083a0 0x7f0bb81087b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:39.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.003+0000 7f0bbe2f8640 1 --2- 172.21.15.64:0/3714497333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bb8102a60 0x7f0bb8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:39.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.003+0000 7f0bbe2f8640 1 -- 172.21.15.64:0/3714497333 >> 172.21.15.64:0/3714497333 conn(0x7f0bb8079730 msgr2=0x7f0bb8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:39.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.003+0000 7f0bbe2f8640 1 -- 172.21.15.64:0/3714497333 shutdown_connections 2024-09-21T04:03:39.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.003+0000 7f0bbe2f8640 1 -- 172.21.15.64:0/3714497333 wait complete. 2024-09-21T04:03:39.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.004+0000 7f0bbe2f8640 1 Processor -- start 2024-09-21T04:03:39.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.004+0000 7f0bbe2f8640 1 -- start start 2024-09-21T04:03:39.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.004+0000 7f0bbe2f8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bb8102a60 0x7f0bb80ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:39.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.004+0000 7f0bbe2f8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bb81083a0 0x7f0bb80ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:39.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.004+0000 7f0bbe2f8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bb8108cf0 0x7f0bb81000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:39.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.004+0000 7f0bbe2f8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb8112960 con 0x7f0bb81083a0 2024-09-21T04:03:39.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.004+0000 7f0bbe2f8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb81127e0 con 0x7f0bb8108cf0 2024-09-21T04:03:39.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.004+0000 7f0bbe2f8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb8112ae0 con 0x7f0bb8102a60 2024-09-21T04:03:39.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.005+0000 7f0bb77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bb81083a0 0x7f0bb80ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:39.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.005+0000 7f0bb77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bb81083a0 0x7f0bb80ffbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47616/0 (socket says 172.21.15.64:47616) 2024-09-21T04:03:39.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.005+0000 7f0bb77fe640 1 -- 172.21.15.64:0/3827670753 learned_addr learned my addr 172.21.15.64:0/3827670753 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:39.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.005+0000 7f0bb7fff640 1 --2- 172.21.15.64:0/3827670753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bb8102a60 0x7f0bb80ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:39.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.005+0000 7f0bbc895640 1 --2- 172.21.15.64:0/3827670753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bb8108cf0 0x7f0bb81000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:39.031 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.005+0000 7f0bb77fe640 1 -- 172.21.15.64:0/3827670753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bb8102a60 msgr2=0x7f0bb80ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:39.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.005+0000 7f0bb77fe640 1 --2- 172.21.15.64:0/3827670753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bb8102a60 0x7f0bb80ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:39.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.005+0000 7f0bb77fe640 1 -- 172.21.15.64:0/3827670753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bb8108cf0 msgr2=0x7f0bb81000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:39.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.005+0000 7f0bb77fe640 1 --2- 172.21.15.64:0/3827670753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bb8108cf0 0x7f0bb81000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:39.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.005+0000 7f0bb77fe640 1 -- 172.21.15.64:0/3827670753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb81a86b0 con 0x7f0bb81083a0 2024-09-21T04:03:39.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.005+0000 7f0bbc895640 1 --2- 172.21.15.64:0/3827670753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bb8108cf0 0x7f0bb81000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:03:39.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.005+0000 7f0bb77fe640 1 --2- 172.21.15.64:0/3827670753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bb81083a0 0x7f0bb80ffbb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0bac031a90 tx=0x7f0bac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:39.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.005+0000 7f0bb57fa640 1 -- 172.21.15.64:0/3827670753 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac03f030 con 0x7f0bb81083a0 2024-09-21T04:03:39.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.005+0000 7f0bbe2f8640 1 -- 172.21.15.64:0/3827670753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb81a8940 con 0x7f0bb81083a0 2024-09-21T04:03:39.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.005+0000 7f0bb57fa640 1 -- 172.21.15.64:0/3827670753 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bac003cb0 con 0x7f0bb81083a0 2024-09-21T04:03:39.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.005+0000 7f0bb57fa640 1 -- 172.21.15.64:0/3827670753 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac035070 con 0x7f0bb81083a0 2024-09-21T04:03:39.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.005+0000 7f0bbe2f8640 1 -- 172.21.15.64:0/3827670753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb81a8e70 con 0x7f0bb81083a0 2024-09-21T04:03:39.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.007+0000 7f0bbe2f8640 1 -- 172.21.15.64:0/3827670753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bb8102e70 con 0x7f0bb81083a0 2024-09-21T04:03:39.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.008+0000 7f0bb57fa640 1 -- 172.21.15.64:0/3827670753 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f0bac042030 con 0x7f0bb81083a0 2024-09-21T04:03:39.032 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.009+0000 7f0bb57fa640 1 --2- 172.21.15.64:0/3827670753 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0ba00b6560 0x7f0ba00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:39.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.009+0000 7f0bb7fff640 1 --2- 172.21.15.64:0/3827670753 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0ba00b6560 0x7f0ba00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:39.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.009+0000 7f0bb57fa640 1 -- 172.21.15.64:0/3827670753 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f0bac107ee0 con 0x7f0bb81083a0 2024-09-21T04:03:39.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.009+0000 7f0bb7fff640 1 --2- 172.21.15.64:0/3827670753 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0ba00b6560 0x7f0ba00b8a70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0ba80058c0 tx=0x7f0ba800a250 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:39.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.012+0000 7f0bb57fa640 1 -- 172.21.15.64:0/3827670753 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0bac0ce620 con 0x7f0bb81083a0 2024-09-21T04:03:39.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.239+0000 7f0bbe2f8640 1 -- 172.21.15.64:0/3827670753 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0bb8100970 con 0x7f0bb81083a0 2024-09-21T04:03:39.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.240+0000 7f0bb57fa640 1 -- 172.21.15.64:0/3827670753 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11911 (secure 0 0 0) 0x7f0bb8100970 con 0x7f0bb81083a0 2024-09-21T04:03:39.243 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:03:39.243 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":24,"btime":"2024-09-21T04:02:14:798588+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14805,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/97995119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":97995119},{"type":"v1","addr":"172.21.15.64:6839","nonce":97995119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/390891068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":390891068},{"type":"v1","addr":"172.21.15.64:6841","nonce":390891068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:02:03.354744+0000","modified":"2024-09-21T04:02:14.579750+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24710,"mds_1":24680,"mds_2":24661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24634":{"gid":24634,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6839/782190424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782190424},{"type":"v1","addr":"172.21.15.131:6839","nonce":782190424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24680":{"gid":24680,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24710":{"gid":24710,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6839/1319793636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1319793636},{"type":"v1","addr":"172.21.15.119:6839","nonce":1319793636}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24710,"qdb_cluster":[24661,24680,24710]},"id":1}]} 2024-09-21T04:03:39.244 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 24 2024-09-21T04:03:39.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.242+0000 7f0bbe2f8640 1 -- 172.21.15.64:0/3827670753 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0ba00b6560 msgr2=0x7f0ba00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:39.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.242+0000 7f0bbe2f8640 1 --2- 172.21.15.64:0/3827670753 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0ba00b6560 0x7f0ba00b8a70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0ba80058c0 tx=0x7f0ba800a250 comp rx=0 tx=0).stop 2024-09-21T04:03:39.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.242+0000 7f0bbe2f8640 1 -- 172.21.15.64:0/3827670753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bb81083a0 msgr2=0x7f0bb80ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:39.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.242+0000 7f0bbe2f8640 1 --2- 172.21.15.64:0/3827670753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bb81083a0 0x7f0bb80ffbb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0bac031a90 tx=0x7f0bac004270 comp rx=0 tx=0).stop 2024-09-21T04:03:39.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.243+0000 7f0bbe2f8640 1 -- 172.21.15.64:0/3827670753 shutdown_connections 2024-09-21T04:03:39.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.243+0000 7f0bbe2f8640 1 --2- 172.21.15.64:0/3827670753 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0ba00b6560 0x7f0ba00b8a70 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:39.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.243+0000 7f0bbe2f8640 1 --2- 172.21.15.64:0/3827670753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bb8108cf0 0x7f0bb81000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:39.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.243+0000 7f0bbe2f8640 1 --2- 172.21.15.64:0/3827670753 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0bb81083a0 0x7f0bb80ffbb0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:39.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.243+0000 7f0bbe2f8640 1 --2- 172.21.15.64:0/3827670753 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0bb8102a60 0x7f0bb80ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:39.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.243+0000 7f0bbe2f8640 1 -- 172.21.15.64:0/3827670753 >> 172.21.15.64:0/3827670753 conn(0x7f0bb8079730 msgr2=0x7f0bb806b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:39.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.243+0000 7f0bbe2f8640 1 -- 172.21.15.64:0/3827670753 shutdown_connections 2024-09-21T04:03:39.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:39.243+0000 7f0bbe2f8640 1 -- 172.21.15.64:0/3827670753 wait complete. 2024-09-21T04:03:39.859 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:39 smithi064 ceph-mon[31180]: pgmap v310: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 343 B/s wr, 7 op/s; 68 B/s, 1 keys/s, 0 objects/s recovering 2024-09-21T04:03:39.860 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:39 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3827670753' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:03:39.888 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T04:03:40.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:39 smithi119 ceph-mon[34811]: pgmap v310: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 343 B/s wr, 7 op/s; 68 B/s, 1 keys/s, 0 objects/s recovering 2024-09-21T04:03:40.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:39 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3827670753' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:03:40.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:39 smithi131 ceph-mon[35193]: pgmap v310: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 343 B/s wr, 7 op/s; 68 B/s, 1 keys/s, 0 objects/s recovering 2024-09-21T04:03:40.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:39 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3827670753' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:03:40.144 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:40.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.640+0000 7f7ab50c5640 1 -- 172.21.15.64:0/938403219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab01094a0 msgr2=0x7f7ab01098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:40.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.640+0000 7f7ab50c5640 1 --2- 172.21.15.64:0/938403219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab01094a0 0x7f7ab01098b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4009900 tx=0x7f7aa402a410 comp rx=0 tx=0).stop 2024-09-21T04:03:40.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.640+0000 7f7ab50c5640 1 -- 172.21.15.64:0/938403219 shutdown_connections 2024-09-21T04:03:40.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.640+0000 7f7ab50c5640 1 --2- 172.21.15.64:0/938403219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ab0109df0 0x7f7ab0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:40.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.640+0000 7f7ab50c5640 1 --2- 172.21.15.64:0/938403219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab01094a0 0x7f7ab01098b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:40.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.640+0000 7f7ab50c5640 1 --2- 172.21.15.64:0/938403219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab0103b60 0x7f7ab0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:40.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.640+0000 7f7ab50c5640 1 -- 172.21.15.64:0/938403219 >> 172.21.15.64:0/938403219 conn(0x7f7ab007b850 msgr2=0x7f7ab007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:40.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.640+0000 7f7ab50c5640 1 -- 172.21.15.64:0/938403219 shutdown_connections 2024-09-21T04:03:40.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.640+0000 7f7ab50c5640 1 -- 172.21.15.64:0/938403219 wait complete. 2024-09-21T04:03:40.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.641+0000 7f7ab50c5640 1 Processor -- start 2024-09-21T04:03:40.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.641+0000 7f7ab50c5640 1 -- start start 2024-09-21T04:03:40.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.641+0000 7f7ab50c5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab0103b60 0x7f7ab0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:40.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.641+0000 7f7ab50c5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ab01094a0 0x7f7ab0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:40.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.641+0000 7f7ab50c5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab0109df0 0x7f7ab0076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:40.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.641+0000 7f7ab50c5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab0113ab0 con 0x7f7ab0109df0 2024-09-21T04:03:40.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.642+0000 7f7ab50c5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab0113930 con 0x7f7ab0103b60 2024-09-21T04:03:40.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.642+0000 7f7ab50c5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab0113c30 con 0x7f7ab01094a0 2024-09-21T04:03:40.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.642+0000 7f7aaf59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab0109df0 0x7f7ab0076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:40.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.642+0000 7f7aaf59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab0109df0 0x7f7ab0076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47640/0 (socket says 172.21.15.64:47640) 2024-09-21T04:03:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.642+0000 7f7aaf59e640 1 -- 172.21.15.64:0/3801930163 learned_addr learned my addr 172.21.15.64:0/3801930163 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.642+0000 7f7aae59c640 1 --2- 172.21.15.64:0/3801930163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ab01094a0 0x7f7ab0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.642+0000 7f7aaed9d640 1 --2- 172.21.15.64:0/3801930163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab0103b60 0x7f7ab0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.642+0000 7f7aaf59e640 1 -- 172.21.15.64:0/3801930163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ab01094a0 msgr2=0x7f7ab0075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.642+0000 7f7aaf59e640 1 --2- 172.21.15.64:0/3801930163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ab01094a0 0x7f7ab0075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.642+0000 7f7aaf59e640 1 -- 172.21.15.64:0/3801930163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab0103b60 msgr2=0x7f7ab0075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.642+0000 7f7aaf59e640 1 --2- 172.21.15.64:0/3801930163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab0103b60 0x7f7ab0075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.642+0000 7f7aaf59e640 1 -- 172.21.15.64:0/3801930163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab01a8740 con 0x7f7ab0109df0 2024-09-21T04:03:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.642+0000 7f7aaed9d640 1 --2- 172.21.15.64:0/3801930163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab0103b60 0x7f7ab0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:03:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.642+0000 7f7aae59c640 1 --2- 172.21.15.64:0/3801930163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ab01094a0 0x7f7ab0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:03:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.642+0000 7f7aaf59e640 1 --2- 172.21.15.64:0/3801930163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab0109df0 0x7f7ab0076190 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800eaa0 tx=0x7f7aa800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.642+0000 7f7a97fff640 1 -- 172.21.15.64:0/3801930163 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa800cdb0 con 0x7f7ab0109df0 2024-09-21T04:03:40.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.642+0000 7f7a97fff640 1 -- 172.21.15.64:0/3801930163 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa8005020 con 0x7f7ab0109df0 2024-09-21T04:03:40.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.642+0000 7f7ab50c5640 1 -- 172.21.15.64:0/3801930163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab01a8a30 con 0x7f7ab0109df0 2024-09-21T04:03:40.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.642+0000 7f7a97fff640 1 -- 172.21.15.64:0/3801930163 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa80116d0 con 0x7f7ab0109df0 2024-09-21T04:03:40.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.642+0000 7f7ab50c5640 1 -- 172.21.15.64:0/3801930163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab01a8ef0 con 0x7f7ab0109df0 2024-09-21T04:03:40.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.644+0000 7f7ab50c5640 1 -- 172.21.15.64:0/3801930163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ab0103f70 con 0x7f7ab0109df0 2024-09-21T04:03:40.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.645+0000 7f7a97fff640 1 -- 172.21.15.64:0/3801930163 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f7aa8004b20 con 0x7f7ab0109df0 2024-09-21T04:03:40.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.646+0000 7f7a97fff640 1 --2- 172.21.15.64:0/3801930163 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7a980b6560 0x7f7a980b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:40.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.646+0000 7f7aaed9d640 1 --2- 172.21.15.64:0/3801930163 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7a980b6560 0x7f7a980b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:40.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.646+0000 7f7a97fff640 1 -- 172.21.15.64:0/3801930163 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f7aa80e7760 con 0x7f7ab0109df0 2024-09-21T04:03:40.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.646+0000 7f7aaed9d640 1 --2- 172.21.15.64:0/3801930163 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7a980b6560 0x7f7a980b8a70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0077160 tx=0x7f7aa0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:40.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.650+0000 7f7a97fff640 1 -- 172.21.15.64:0/3801930163 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7aa8016030 con 0x7f7ab0109df0 2024-09-21T04:03:40.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.837+0000 7f7ab50c5640 1 -- 172.21.15.64:0/3801930163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ab010bc20 con 0x7f7ab0109df0 2024-09-21T04:03:40.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.839+0000 7f7a97fff640 1 -- 172.21.15.64:0/3801930163 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+23230 (secure 0 0 0) 0x7f7aa80adea0 con 0x7f7ab0109df0 2024-09-21T04:03:40.842 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:03:40.842 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":77,"fsid":"e7c927ba-77cc-11ef-baf5-efdc52797490","created":"2024-09-21T03:53:41.215073+0000","modified":"2024-09-21T04:03:30.555854+0000","last_up_change":"2024-09-21T04:01:11.986949+0000","last_in_change":"2024-09-21T04:00:48.707791+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:57:24.854438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T04:01:58.728355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T04:02:01.300385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb947c10-98e6-4e3a-8bbd-8753951d870b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6803","nonce":3692023035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6805","nonce":3692023035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6809","nonce":3692023035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6807","nonce":3692023035}]},"public_addr":"172.21.15.64:6803/3692023035","cluster_addr":"172.21.15.64:6805/3692023035","heartbeat_back_addr":"172.21.15.64:6809/3692023035","heartbeat_front_addr":"172.21.15.64:6807/3692023035","state":["exists","up"]},{"osd":1,"uuid":"29b8c92d-82fa-4175-b3e2-711917b7f8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6801","nonce":3363415998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6803","nonce":3363415998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6807","nonce":3363415998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6805","nonce":3363415998}]},"public_addr":"172.21.15.119:6801/3363415998","cluster_addr":"172.21.15.119:6803/3363415998","heartbeat_back_addr":"172.21.15.119:6807/3363415998","heartbeat_front_addr":"172.21.15.119:6805/3363415998","state":["exists","up"]},{"osd":2,"uuid":"a2273fe8-df55-40f7-afda-74ab09c52c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6801","nonce":3422924179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6803","nonce":3422924179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6807","nonce":3422924179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6805","nonce":3422924179}]},"public_addr":"172.21.15.131:6801/3422924179","cluster_addr":"172.21.15.131:6803/3422924179","heartbeat_back_addr":"172.21.15.131:6807/3422924179","heartbeat_front_addr":"172.21.15.131:6805/3422924179","state":["exists","up"]},{"osd":3,"uuid":"f1634064-b108-41d2-a78f-c33607b08a70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6811","nonce":2022952729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6813","nonce":2022952729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6817","nonce":2022952729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6815","nonce":2022952729}]},"public_addr":"172.21.15.64:6811/2022952729","cluster_addr":"172.21.15.64:6813/2022952729","heartbeat_back_addr":"172.21.15.64:6817/2022952729","heartbeat_front_addr":"172.21.15.64:6815/2022952729","state":["exists","up"]},{"osd":4,"uuid":"75b81a8d-fa52-411a-bfda-aa69bb70a20f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6809","nonce":323030958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6811","nonce":323030958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6815","nonce":323030958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6813","nonce":323030958}]},"public_addr":"172.21.15.119:6809/323030958","cluster_addr":"172.21.15.119:6811/323030958","heartbeat_back_addr":"172.21.15.119:6815/323030958","heartbeat_front_addr":"172.21.15.119:6813/323030958","state":["exists","up"]},{"osd":5,"uuid":"0e4895b5-42e4-4b2c-a9c3-692f287cb841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6809","nonce":3021788088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6811","nonce":3021788088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6815","nonce":3021788088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6813","nonce":3021788088}]},"public_addr":"172.21.15.131:6809/3021788088","cluster_addr":"172.21.15.131:6811/3021788088","heartbeat_back_addr":"172.21.15.131:6815/3021788088","heartbeat_front_addr":"172.21.15.131:6813/3021788088","state":["exists","up"]},{"osd":6,"uuid":"1fa5c2b7-bce0-43a1-a26b-d029eb3731fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6819","nonce":1913824324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6821","nonce":1913824324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6825","nonce":1913824324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6823","nonce":1913824324}]},"public_addr":"172.21.15.64:6819/1913824324","cluster_addr":"172.21.15.64:6821/1913824324","heartbeat_back_addr":"172.21.15.64:6825/1913824324","heartbeat_front_addr":"172.21.15.64:6823/1913824324","state":["exists","up"]},{"osd":7,"uuid":"c9707260-b581-4a21-b380-f37e16959b02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6817","nonce":2567568166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6819","nonce":2567568166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6823","nonce":2567568166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6821","nonce":2567568166}]},"public_addr":"172.21.15.119:6817/2567568166","cluster_addr":"172.21.15.119:6819/2567568166","heartbeat_back_addr":"172.21.15.119:6823/2567568166","heartbeat_front_addr":"172.21.15.119:6821/2567568166","state":["exists","up"]},{"osd":8,"uuid":"2dbf8ef2-fc9c-4c6b-bfea-003f41ef191e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6817","nonce":199192217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6819","nonce":199192217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6823","nonce":199192217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6821","nonce":199192217}]},"public_addr":"172.21.15.131:6817/199192217","cluster_addr":"172.21.15.131:6819/199192217","heartbeat_back_addr":"172.21.15.131:6823/199192217","heartbeat_front_addr":"172.21.15.131:6821/199192217","state":["exists","up"]},{"osd":9,"uuid":"96f769e5-4783-4558-adfa-4bfaf9da9e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6827","nonce":3309038990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6829","nonce":3309038990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6833","nonce":3309038990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6831","nonce":3309038990}]},"public_addr":"172.21.15.64:6827/3309038990","cluster_addr":"172.21.15.64:6829/3309038990","heartbeat_back_addr":"172.21.15.64:6833/3309038990","heartbeat_front_addr":"172.21.15.64:6831/3309038990","state":["exists","up"]},{"osd":10,"uuid":"13c1ef5c-2299-440c-af5a-a650b47f4b75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6825","nonce":1460844681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6827","nonce":1460844681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6831","nonce":1460844681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6829","nonce":1460844681}]},"public_addr":"172.21.15.119:6825/1460844681","cluster_addr":"172.21.15.119:6827/1460844681","heartbeat_back_addr":"172.21.15.119:6831/1460844681","heartbeat_front_addr":"172.21.15.119:6829/1460844681","state":["exists","up"]},{"osd":11,"uuid":"c26a5ef9-8585-404e-a54a-852424c4d6fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6825","nonce":2294059084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6827","nonce":2294059084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6831","nonce":2294059084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6829","nonce":2294059084}]},"public_addr":"172.21.15.131:6825/2294059084","cluster_addr":"172.21.15.131:6827/2294059084","heartbeat_back_addr":"172.21.15.131:6831/2294059084","heartbeat_front_addr":"172.21.15.131:6829/2294059084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:56:33.141102+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:56:58.883062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:57:22.641789+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:57:48.705790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:58:12.871514+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:58:37.893118+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:03.620828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:29.446179+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:55.002978+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:00:19.987463+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:00:45.364158+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:01:10.228248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.3","mappings":[{"from":10,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":5,"to":6}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/3554471670":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6801/3786213009":"2024-09-22T03:55:28.387426+0000","172.21.15.64:0/2562856406":"2024-09-22T03:54:27.180100+0000","172.21.15.64:0/2603015699":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6800/1209932961":"2024-09-22T03:54:02.562380+0000","172.21.15.64:6800/3786213009":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6801/1209932961":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/3407344596":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1233731831":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1686887701":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1598568021":"2024-09-22T03:54:27.180100+0000","172.21.15.64:6800/2013929312":"2024-09-22T03:54:27.180100+0000","172.21.15.64:6801/2013929312":"2024-09-22T03:54:27.180100+0000","172.21.15.64:0/3652036144":"2024-09-22T03:55:28.387426+0000","172.21.15.64:0/105454703":"2024-09-22T03:54:27.180100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T04:03:40.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.841+0000 7f7ab50c5640 1 -- 172.21.15.64:0/3801930163 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7a980b6560 msgr2=0x7f7a980b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:40.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.841+0000 7f7ab50c5640 1 --2- 172.21.15.64:0/3801930163 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7a980b6560 0x7f7a980b8a70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0077160 tx=0x7f7aa0005f90 comp rx=0 tx=0).stop 2024-09-21T04:03:40.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.841+0000 7f7ab50c5640 1 -- 172.21.15.64:0/3801930163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab0109df0 msgr2=0x7f7ab0076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:40.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.841+0000 7f7ab50c5640 1 --2- 172.21.15.64:0/3801930163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab0109df0 0x7f7ab0076190 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800eaa0 tx=0x7f7aa800ef60 comp rx=0 tx=0).stop 2024-09-21T04:03:40.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.841+0000 7f7ab50c5640 1 -- 172.21.15.64:0/3801930163 shutdown_connections 2024-09-21T04:03:40.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.841+0000 7f7ab50c5640 1 --2- 172.21.15.64:0/3801930163 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7a980b6560 0x7f7a980b8a70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:40.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.841+0000 7f7ab50c5640 1 --2- 172.21.15.64:0/3801930163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7ab0109df0 0x7f7ab0076190 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:40.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.841+0000 7f7ab50c5640 1 --2- 172.21.15.64:0/3801930163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ab01094a0 0x7f7ab0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:40.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.841+0000 7f7ab50c5640 1 --2- 172.21.15.64:0/3801930163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab0103b60 0x7f7ab0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:40.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.841+0000 7f7ab50c5640 1 -- 172.21.15.64:0/3801930163 >> 172.21.15.64:0/3801930163 conn(0x7f7ab007b850 msgr2=0x7f7ab0105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:40.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.842+0000 7f7ab50c5640 1 -- 172.21.15.64:0/3801930163 shutdown_connections 2024-09-21T04:03:40.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:40.842+0000 7f7ab50c5640 1 -- 172.21.15.64:0/3801930163 wait complete. 2024-09-21T04:03:41.673 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T04:02:03.354744+0000', 'modified': '2024-09-21T04:02:14.579750+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24710, 'mds_1': 24680, 'mds_2': 24661}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24634': {'gid': 24634, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.131:6835/1422569359', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 1422569359}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 1422569359}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24640': {'gid': 24640, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.131:6837/843209972', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6836', 'nonce': 843209972}, {'type': 'v1', 'addr': '172.21.15.131:6837', 'nonce': 843209972}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24656': {'gid': 24656, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.119:6835/3675946142', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6834', 'nonce': 3675946142}, {'type': 'v1', 'addr': '172.21.15.119:6835', 'nonce': 3675946142}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24661': {'gid': 24661, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.131:6839/782190424', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6838', 'nonce': 782190424}, {'type': 'v1', 'addr': '172.21.15.131:6839', 'nonce': 782190424}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24680': {'gid': 24680, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.119:6837/2876008744', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6836', 'nonce': 2876008744}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 2876008744}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24710': {'gid': 24710, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.119:6839/1319793636', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6838', 'nonce': 1319793636}, {'type': 'v1', 'addr': '172.21.15.119:6839', 'nonce': 1319793636}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24710, 'qdb_cluster': [24661, 24680, 24710]} 2024-09-21T04:03:41.673 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-21T04:03:41.673 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T04:03:41.944 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:41.962 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:41 smithi064 ceph-mon[31180]: pgmap v311: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 307 B/s wr, 6 op/s; 61 B/s, 0 keys/s, 0 objects/s recovering 2024-09-21T04:03:41.962 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:41 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3801930163' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:03:42.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:41 smithi119 ceph-mon[34811]: pgmap v311: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 307 B/s wr, 6 op/s; 61 B/s, 0 keys/s, 0 objects/s recovering 2024-09-21T04:03:42.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:41 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3801930163' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:03:42.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:41 smithi131 ceph-mon[35193]: pgmap v311: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 307 B/s wr, 6 op/s; 61 B/s, 0 keys/s, 0 objects/s recovering 2024-09-21T04:03:42.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:41 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3801930163' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:03:42.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.434+0000 7f73f858f640 1 -- 172.21.15.64:0/3302536201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73f0108d20 msgr2=0x7f73f010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:42.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.434+0000 7f73f858f640 1 --2- 172.21.15.64:0/3302536201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73f0108d20 0x7f73f010f550 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f73e8009ca0 tx=0x7f73e802a8c0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.434+0000 7f73f858f640 1 -- 172.21.15.64:0/3302536201 shutdown_connections 2024-09-21T04:03:42.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.434+0000 7f73f858f640 1 --2- 172.21.15.64:0/3302536201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73f0108d20 0x7f73f010f550 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.434+0000 7f73f858f640 1 --2- 172.21.15.64:0/3302536201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73f01083d0 0x7f73f01087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.434+0000 7f73f858f640 1 --2- 172.21.15.64:0/3302536201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73f0102ae0 0x7f73f0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.434+0000 7f73f858f640 1 -- 172.21.15.64:0/3302536201 >> 172.21.15.64:0/3302536201 conn(0x7f73f0079790 msgr2=0x7f73f0079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:42.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.434+0000 7f73f858f640 1 -- 172.21.15.64:0/3302536201 shutdown_connections 2024-09-21T04:03:42.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.434+0000 7f73f858f640 1 -- 172.21.15.64:0/3302536201 wait complete. 2024-09-21T04:03:42.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.435+0000 7f73f858f640 1 Processor -- start 2024-09-21T04:03:42.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.435+0000 7f73f858f640 1 -- start start 2024-09-21T04:03:42.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.435+0000 7f73f858f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73f0102ae0 0x7f73f019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:42.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.435+0000 7f73f858f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73f01083d0 0x7f73f019b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:42.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.435+0000 7f73f858f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73f0108d20 0x7f73f019bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:42.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.435+0000 7f73f858f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f73f01129e0 con 0x7f73f0108d20 2024-09-21T04:03:42.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.435+0000 7f73f858f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f73f0112860 con 0x7f73f01083d0 2024-09-21T04:03:42.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.435+0000 7f73f858f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f73f0112b60 con 0x7f73f0102ae0 2024-09-21T04:03:42.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.435+0000 7f73f6b2c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73f0108d20 0x7f73f019bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:42.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.435+0000 7f73f6b2c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73f0108d20 0x7f73f019bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52244/0 (socket says 172.21.15.64:52244) 2024-09-21T04:03:42.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.435+0000 7f73f6b2c640 1 -- 172.21.15.64:0/4140624989 learned_addr learned my addr 172.21.15.64:0/4140624989 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:42.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.436+0000 7f73f5b2a640 1 --2- 172.21.15.64:0/4140624989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73f01083d0 0x7f73f019b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:42.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.436+0000 7f73f632b640 1 --2- 172.21.15.64:0/4140624989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73f0102ae0 0x7f73f019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:42.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.436+0000 7f73f6b2c640 1 -- 172.21.15.64:0/4140624989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73f0102ae0 msgr2=0x7f73f019b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:42.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.436+0000 7f73f6b2c640 1 --2- 172.21.15.64:0/4140624989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73f0102ae0 0x7f73f019b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.436+0000 7f73f6b2c640 1 -- 172.21.15.64:0/4140624989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73f01083d0 msgr2=0x7f73f019b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:42.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.436+0000 7f73f6b2c640 1 --2- 172.21.15.64:0/4140624989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73f01083d0 0x7f73f019b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.436+0000 7f73f6b2c640 1 -- 172.21.15.64:0/4140624989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f01a6560 con 0x7f73f0108d20 2024-09-21T04:03:42.451 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.436+0000 7f73f632b640 1 --2- 172.21.15.64:0/4140624989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73f0102ae0 0x7f73f019b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:03:42.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.436+0000 7f73f5b2a640 1 --2- 172.21.15.64:0/4140624989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73f01083d0 0x7f73f019b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:03:42.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.436+0000 7f73f6b2c640 1 --2- 172.21.15.64:0/4140624989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73f0108d20 0x7f73f019bd50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f73e8005cc0 tx=0x7f73e8031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:42.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.436+0000 7f73e77fe640 1 -- 172.21.15.64:0/4140624989 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8003b70 con 0x7f73f0108d20 2024-09-21T04:03:42.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.436+0000 7f73f858f640 1 -- 172.21.15.64:0/4140624989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f01a67f0 con 0x7f73f0108d20 2024-09-21T04:03:42.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.436+0000 7f73e77fe640 1 -- 172.21.15.64:0/4140624989 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73e80318f0 con 0x7f73f0108d20 2024-09-21T04:03:42.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.436+0000 7f73e77fe640 1 -- 172.21.15.64:0/4140624989 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e802f950 con 0x7f73f0108d20 2024-09-21T04:03:42.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.436+0000 7f73f858f640 1 -- 172.21.15.64:0/4140624989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f01a6d80 con 0x7f73f0108d20 2024-09-21T04:03:42.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.439+0000 7f73e77fe640 1 -- 172.21.15.64:0/4140624989 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f73e802faf0 con 0x7f73f0108d20 2024-09-21T04:03:42.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.439+0000 7f73feb03740 1 Processor -- start 2024-09-21T04:03:42.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.439+0000 7f73feb03740 1 -- start start 2024-09-21T04:03:42.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.439+0000 7f73feb03740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555719cbcf60 0x55571a062f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:42.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.439+0000 7f73feb03740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555719c940a0 0x55571a063440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:42.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.439+0000 7f73feb03740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719c9c280 0x55571a063980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:42.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.440+0000 7f73feb03740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555719fdac10 con 0x555719c9c280 2024-09-21T04:03:42.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.440+0000 7f73feb03740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x555719fca040 con 0x555719c940a0 2024-09-21T04:03:42.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.440+0000 7f73feb03740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555719fcde50 con 0x555719cbcf60 2024-09-21T04:03:42.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.440+0000 7f73f5b2a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719c9c280 0x55571a063980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:42.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.440+0000 7f73f5b2a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719c9c280 0x55571a063980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52256/0 (socket says 172.21.15.64:52256) 2024-09-21T04:03:42.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.440+0000 7f73f5b2a640 1 -- 172.21.15.64:0/169610810 learned_addr learned my addr 172.21.15.64:0/169610810 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:42.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.440+0000 7f73f632b640 1 --2- 172.21.15.64:0/169610810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555719c940a0 0x55571a063440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:42.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.440+0000 7f73f5b2a640 1 --2- 172.21.15.64:0/169610810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555719cbcf60 0x55571a062f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:42.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.440+0000 7f73e77fe640 1 --2- 172.21.15.64:0/4140624989 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f73dc0b6590 0x7f73dc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:42.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.440+0000 7f73f5b2a640 1 -- 172.21.15.64:0/169610810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555719cbcf60 msgr2=0x55571a062f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:42.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.440+0000 7f73f5b2a640 1 --2- 172.21.15.64:0/169610810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555719cbcf60 0x55571a062f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.440+0000 7f73f5b2a640 1 -- 172.21.15.64:0/169610810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555719c940a0 msgr2=0x55571a063440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:42.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.440+0000 7f73f5b2a640 1 --2- 172.21.15.64:0/169610810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555719c940a0 0x55571a063440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.440+0000 7f73f6b2c640 1 --2- 172.21.15.64:0/4140624989 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f73dc0b6590 0x7f73dc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:42.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.440+0000 7f73f5b2a640 1 -- 172.21.15.64:0/169610810 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555719d28540 con 0x555719c9c280 2024-09-21T04:03:42.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.440+0000 7f73e77fe640 1 -- 172.21.15.64:0/4140624989 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f73e8108930 con 0x7f73f0108d20 2024-09-21T04:03:42.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.440+0000 7f73f5b2a640 1 --2- 172.21.15.64:0/169610810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719c9c280 0x55571a063980 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f73e00058c0 tx=0x7f73e0009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe8ce32fad37bf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:42.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.441+0000 7f73e4ff9640 1 -- 172.21.15.64:0/169610810 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e0013250 con 0x555719c9c280 2024-09-21T04:03:42.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.441+0000 7f73e4ff9640 1 -- 172.21.15.64:0/169610810 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73e0013e50 con 0x555719c9c280 2024-09-21T04:03:42.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.441+0000 7f73e4ff9640 1 -- 172.21.15.64:0/169610810 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e00129b0 con 0x555719c9c280 2024-09-21T04:03:42.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.442+0000 7f73feb03740 1 -- 172.21.15.64:0/169610810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719c9c280 msgr2=0x55571a063980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:42.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.442+0000 7f73feb03740 1 --2- 172.21.15.64:0/169610810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719c9c280 0x55571a063980 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f73e00058c0 tx=0x7f73e0009670 comp rx=0 tx=0).stop 2024-09-21T04:03:42.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.442+0000 7f73f6b2c640 1 --2- 172.21.15.64:0/4140624989 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f73dc0b6590 0x7f73dc0b8aa0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f73e8005e30 tx=0x7f73e8041000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:42.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.442+0000 7f73feb03740 1 -- 172.21.15.64:0/169610810 shutdown_connections 2024-09-21T04:03:42.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.442+0000 7f73feb03740 1 --2- 172.21.15.64:0/169610810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719c9c280 0x55571a063980 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.442+0000 7f73feb03740 1 --2- 172.21.15.64:0/169610810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555719c940a0 0x55571a063440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.442+0000 7f73feb03740 1 --2- 172.21.15.64:0/169610810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555719cbcf60 0x55571a062f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.442+0000 7f73feb03740 1 -- 172.21.15.64:0/169610810 >> 172.21.15.64:0/169610810 conn(0x555719cb1350 msgr2=0x55571a060df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:42.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.442+0000 7f73feb03740 1 -- 172.21.15.64:0/169610810 shutdown_connections 2024-09-21T04:03:42.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.442+0000 7f73feb03740 1 -- 172.21.15.64:0/169610810 wait complete. 2024-09-21T04:03:42.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.442+0000 7f73feb03740 1 -- start start 2024-09-21T04:03:42.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.442+0000 7f73feb03740 1 Processor -- start 2024-09-21T04:03:42.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.442+0000 7f73feb03740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555719c940a0 0x55571a072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:42.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.442+0000 7f73feb03740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719c9c280 0x55571a073420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:42.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.443+0000 7f73feb03740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555719cbcf60 0x55571a07ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:42.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.443+0000 7f73feb03740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555719e2b5f0 con 0x555719c9c280 2024-09-21T04:03:42.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.443+0000 7f73feb03740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x555719f65010 con 0x555719cbcf60 2024-09-21T04:03:42.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.443+0000 7f73feb03740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55571a0759e0 con 0x555719c940a0 2024-09-21T04:03:42.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.443+0000 7f73feb03740 10 client.0 _ll_register_callbacks cb 0x555719fc4340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:03:42.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.443+0000 7f73f632b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719c9c280 0x55571a073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:42.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.443+0000 7f73f632b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719c9c280 0x55571a073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52264/0 (socket says 172.21.15.64:52264) 2024-09-21T04:03:42.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.443+0000 7f73f632b640 1 -- 172.21.15.64:0/2585953919 learned_addr learned my addr 172.21.15.64:0/2585953919 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:42.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.443+0000 7f73f5b2a640 1 --2- 172.21.15.64:0/2585953919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555719c940a0 0x55571a072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:42.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.443+0000 7f73f5b2a640 1 --2- 172.21.15.64:0/2585953919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555719cbcf60 0x55571a07ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:42.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.443+0000 7f73f632b640 1 -- 172.21.15.64:0/2585953919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555719c940a0 msgr2=0x55571a072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:42.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.443+0000 7f73f632b640 1 --2- 172.21.15.64:0/2585953919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555719c940a0 0x55571a072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.443+0000 7f73f632b640 1 -- 172.21.15.64:0/2585953919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555719cbcf60 msgr2=0x55571a07ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:42.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.443+0000 7f73f632b640 1 --2- 172.21.15.64:0/2585953919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555719cbcf60 0x55571a07ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.443+0000 7f73f632b640 1 -- 172.21.15.64:0/2585953919 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555719f92a00 con 0x555719c9c280 2024-09-21T04:03:42.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.443+0000 7f73f632b640 1 --2- 172.21.15.64:0/2585953919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719c9c280 0x55571a073420 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f73ec013710 tx=0x7f73ec013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f122c18dfd1b6687 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:42.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.443+0000 7f73d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:03:42.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.443+0000 7f73d9ffb640 1 -- 172.21.15.64:0/2585953919 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ec011dd0 con 0x555719c9c280 2024-09-21T04:03:42.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.443+0000 7f73d9ffb640 1 -- 172.21.15.64:0/2585953919 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f73ec01b3e0 con 0x555719c9c280 2024-09-21T04:03:42.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.443+0000 7f73feb03740 1 -- 172.21.15.64:0/2585953919 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73e0013e50 con 0x555719c9c280 2024-09-21T04:03:42.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.444+0000 7f73d9ffb640 1 -- 172.21.15.64:0/2585953919 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f73ec01b5e0 con 0x555719c9c280 2024-09-21T04:03:42.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.444+0000 7f73feb03740 10 client.15294 fetch_fsmap learned FSMap version 24 2024-09-21T04:03:42.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.444+0000 7f73feb03740 1 -- 172.21.15.64:0/2585953919 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55571a081a90 con 0x555719c9c280 2024-09-21T04:03:42.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.444+0000 7f73d9ffb640 1 -- 172.21.15.64:0/2585953919 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73e0013e50 con 0x555719c9c280 2024-09-21T04:03:42.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.444+0000 7f73d9ffb640 1 -- 172.21.15.64:0/2585953919 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f73ec01a070 con 0x555719c9c280 2024-09-21T04:03:42.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.444+0000 7f73feb03740 10 client.15294 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T04:03:42.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.444+0000 7f73feb03740 10 client.15294 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T04:03:42.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.444+0000 7f73feb03740 20 client.15294 populate_metadata read hostname 'smithi064' 2024-09-21T04:03:42.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.444+0000 7f73feb03740 1 --2- 172.21.15.64:0/2585953919 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x555719c7b8e0 0x55571a085580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:42.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.444+0000 7f73feb03740 4 client.15294 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:03:42.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.444+0000 7f73feb03740 1 -- 172.21.15.64:0/2585953919 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555719fdcb60 con 0x555719c7b8e0 2024-09-21T04:03:42.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.444+0000 7f73f5b2a640 1 --2- 172.21.15.64:0/2585953919 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x555719c7b8e0 0x55571a085580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:42.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.445+0000 7f73f5b2a640 1 --2- 172.21.15.64:0/2585953919 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x555719c7b8e0 0x55571a085580 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c2ec30bf355e0ce server_cookie=8844eb6e2867af1e in_seq=0 out_seq=0 2024-09-21T04:03:42.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.445+0000 7f73d9ffb640 10 client.15294 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:03:42.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.446+0000 7f73d9ffb640 1 -- 172.21.15.64:0/2585953919 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555719fdcb60 con 0x555719c7b8e0 2024-09-21T04:03:42.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.446+0000 7f73d9ffb640 10 client.15294 handle_command_reply: tid=0 2024-09-21T04:03:42.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.446+0000 7f73feb03740 1 client.15294 shutdown 2024-09-21T04:03:42.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.446+0000 7f73feb03740 1 -- 172.21.15.64:0/2585953919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719c9c280 msgr2=0x55571a073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:42.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.446+0000 7f73feb03740 1 --2- 172.21.15.64:0/2585953919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719c9c280 0x55571a073420 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f73ec013710 tx=0x7f73ec013bd0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.446+0000 7f73feb03740 1 -- 172.21.15.64:0/2585953919 shutdown_connections 2024-09-21T04:03:42.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.446+0000 7f73feb03740 1 --2- 172.21.15.64:0/2585953919 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x555719c7b8e0 0x55571a085580 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.446+0000 7f73d9ffb640 0 client.15294 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:03:42.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.446+0000 7f73feb03740 1 --2- 172.21.15.64:0/2585953919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555719cbcf60 0x55571a07ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.446+0000 7f73feb03740 1 --2- 172.21.15.64:0/2585953919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719c9c280 0x55571a073420 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.446+0000 7f73feb03740 1 --2- 172.21.15.64:0/2585953919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555719c940a0 0x55571a072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.446+0000 7f73feb03740 1 -- 172.21.15.64:0/2585953919 >> 172.21.15.64:0/2585953919 conn(0x555719cb1350 msgr2=0x55571a05f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:42.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.446+0000 7f73feb03740 1 -- 172.21.15.64:0/2585953919 shutdown_connections 2024-09-21T04:03:42.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.446+0000 7f73feb03740 1 -- 172.21.15.64:0/2585953919 wait complete. 2024-09-21T04:03:42.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.446+0000 7f73feb03740 20 client.15294 trim_cache size 0 max 16384 2024-09-21T04:03:42.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.464+0000 7f73feb03740 1 Processor -- start 2024-09-21T04:03:42.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.465+0000 7f73feb03740 1 -- start start 2024-09-21T04:03:42.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.465+0000 7f73feb03740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555719c940a0 0x55571a084eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:42.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.465+0000 7f73feb03740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555719c9c280 0x55571a082f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:42.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.465+0000 7f73feb03740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719cbcf60 0x55571a06de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:42.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.465+0000 7f73feb03740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55571a0838c0 con 0x555719cbcf60 2024-09-21T04:03:42.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.465+0000 7f73feb03740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55571a083f90 con 0x555719c9c280 2024-09-21T04:03:42.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.465+0000 7f73feb03740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55571a084660 con 0x555719c940a0 2024-09-21T04:03:42.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.465+0000 7f73f5b2a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719cbcf60 0x55571a06de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:42.468 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.465+0000 7f73f5b2a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719cbcf60 0x55571a06de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52272/0 (socket says 172.21.15.64:52272) 2024-09-21T04:03:42.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.465+0000 7f73f5b2a640 1 -- 172.21.15.64:0/1366429270 learned_addr learned my addr 172.21.15.64:0/1366429270 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:42.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.465+0000 7f73f5b2a640 1 --2- 172.21.15.64:0/1366429270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555719c940a0 0x55571a084eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:42.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.465+0000 7f73f632b640 1 --2- 172.21.15.64:0/1366429270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555719c9c280 0x55571a082f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:42.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.465+0000 7f73f5b2a640 1 -- 172.21.15.64:0/1366429270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555719c940a0 msgr2=0x55571a084eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:42.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.465+0000 7f73f5b2a640 1 --2- 172.21.15.64:0/1366429270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555719c940a0 0x55571a084eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.465+0000 7f73f5b2a640 1 -- 172.21.15.64:0/1366429270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555719c9c280 msgr2=0x55571a082f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:42.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.465+0000 7f73f5b2a640 1 --2- 172.21.15.64:0/1366429270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555719c9c280 0x55571a082f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.465+0000 7f73f5b2a640 1 -- 172.21.15.64:0/1366429270 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55571a0711e0 con 0x555719cbcf60 2024-09-21T04:03:42.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.465+0000 7f73f632b640 1 --2- 172.21.15.64:0/1366429270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555719c9c280 0x55571a082f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:03:42.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.465+0000 7f73f5b2a640 1 --2- 172.21.15.64:0/1366429270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719cbcf60 0x55571a06de20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f73e0013810 tx=0x7f73e0009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b2538dee509cb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:42.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.466+0000 7f73dbfff640 1 -- 172.21.15.64:0/1366429270 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e000f980 con 0x555719cbcf60 2024-09-21T04:03:42.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.466+0000 7f73dbfff640 1 -- 172.21.15.64:0/1366429270 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73e0024070 con 0x555719cbcf60 2024-09-21T04:03:42.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.466+0000 7f73dbfff640 1 -- 172.21.15.64:0/1366429270 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e0005460 con 0x555719cbcf60 2024-09-21T04:03:42.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.467+0000 7f73feb03740 1 -- 172.21.15.64:0/1366429270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719cbcf60 msgr2=0x55571a06de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:42.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.467+0000 7f73feb03740 1 --2- 172.21.15.64:0/1366429270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719cbcf60 0x55571a06de20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f73e0013810 tx=0x7f73e0009670 comp rx=0 tx=0).stop 2024-09-21T04:03:42.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.467+0000 7f73feb03740 1 -- 172.21.15.64:0/1366429270 shutdown_connections 2024-09-21T04:03:42.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.467+0000 7f73feb03740 1 --2- 172.21.15.64:0/1366429270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719cbcf60 0x55571a06de20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.467+0000 7f73feb03740 1 --2- 172.21.15.64:0/1366429270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555719c9c280 0x55571a082f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.467+0000 7f73feb03740 1 --2- 172.21.15.64:0/1366429270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555719c940a0 0x55571a084eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.467+0000 7f73feb03740 1 -- 172.21.15.64:0/1366429270 >> 172.21.15.64:0/1366429270 conn(0x555719cb1350 msgr2=0x55571a072ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:42.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.467+0000 7f73feb03740 1 -- 172.21.15.64:0/1366429270 shutdown_connections 2024-09-21T04:03:42.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.467+0000 7f73feb03740 1 -- 172.21.15.64:0/1366429270 wait complete. 2024-09-21T04:03:42.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.467+0000 7f73feb03740 1 -- start start 2024-09-21T04:03:42.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.467+0000 7f73feb03740 1 Processor -- start 2024-09-21T04:03:42.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.468+0000 7f73feb03740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719c940a0 0x55571a08e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:42.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.468+0000 7f73feb03740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555719c9c280 0x55571a087790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:42.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.468+0000 7f73feb03740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555719cbcf60 0x55571a087ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:42.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.468+0000 7f73feb03740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55571a085400 con 0x555719c940a0 2024-09-21T04:03:42.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.468+0000 7f73feb03740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55571a084d30 con 0x555719c9c280 2024-09-21T04:03:42.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.468+0000 7f73feb03740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55571a085ad0 con 0x555719cbcf60 2024-09-21T04:03:42.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.468+0000 7f73feb03740 10 client.0 _ll_register_callbacks cb 0x7f73ec005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:03:42.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.468+0000 7f73f5b2a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719c940a0 0x55571a08e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:42.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.468+0000 7f73f5b2a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719c940a0 0x55571a08e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52278/0 (socket says 172.21.15.64:52278) 2024-09-21T04:03:42.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.468+0000 7f73f5b2a640 1 -- 172.21.15.64:0/3324858924 learned_addr learned my addr 172.21.15.64:0/3324858924 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:42.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.468+0000 7f73f632b640 1 --2- 172.21.15.64:0/3324858924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555719c9c280 0x55571a087790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:42.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.468+0000 7f73f5b2a640 1 -- 172.21.15.64:0/3324858924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555719cbcf60 msgr2=0x55571a087ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:42.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.468+0000 7f73f5b2a640 1 --2- 172.21.15.64:0/3324858924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555719cbcf60 0x55571a087ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.468+0000 7f73f5b2a640 1 -- 172.21.15.64:0/3324858924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555719c9c280 msgr2=0x55571a087790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:42.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.468+0000 7f73f5b2a640 1 --2- 172.21.15.64:0/3324858924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555719c9c280 0x55571a087790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.468+0000 7f73f5b2a640 1 -- 172.21.15.64:0/3324858924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55571a08cea0 con 0x555719c940a0 2024-09-21T04:03:42.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.469+0000 7f73f632b640 1 --2- 172.21.15.64:0/3324858924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555719c9c280 0x55571a087790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:03:42.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.469+0000 7f73f5b2a640 1 --2- 172.21.15.64:0/3324858924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719c940a0 0x55571a08e260 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f73e000f730 tx=0x7f73e0012450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4909213cc8d8e40 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:42.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.469+0000 7f73d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:03:42.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.469+0000 7f73d9ffb640 1 -- 172.21.15.64:0/3324858924 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e00051e0 con 0x555719c940a0 2024-09-21T04:03:42.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.469+0000 7f73d9ffb640 1 -- 172.21.15.64:0/3324858924 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f73e0005380 con 0x555719c940a0 2024-09-21T04:03:42.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.469+0000 7f73feb03740 1 -- 172.21.15.64:0/3324858924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73e0024070 con 0x555719c940a0 2024-09-21T04:03:42.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.469+0000 7f73d9ffb640 1 -- 172.21.15.64:0/3324858924 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f73e0047050 con 0x555719c940a0 2024-09-21T04:03:42.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.469+0000 7f73feb03740 10 client.15300 fetch_fsmap learned FSMap version 24 2024-09-21T04:03:42.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.469+0000 7f73feb03740 1 -- 172.21.15.64:0/3324858924 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55571a0f0ba0 con 0x555719c940a0 2024-09-21T04:03:42.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.469+0000 7f73d9ffb640 1 -- 172.21.15.64:0/3324858924 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73e0024070 con 0x555719c940a0 2024-09-21T04:03:42.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.470+0000 7f73d9ffb640 1 -- 172.21.15.64:0/3324858924 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f73e001f430 con 0x555719c940a0 2024-09-21T04:03:42.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.470+0000 7f73feb03740 10 client.15300 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T04:03:42.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.470+0000 7f73feb03740 10 client.15300 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T04:03:42.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.470+0000 7f73feb03740 20 client.15300 populate_metadata read hostname 'smithi064' 2024-09-21T04:03:42.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.470+0000 7f73feb03740 1 --2- 172.21.15.64:0/3324858924 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x555719c50c90 0x55571a0f3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:42.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.470+0000 7f73feb03740 4 client.15300 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T04:03:42.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.470+0000 7f73feb03740 1 -- 172.21.15.64:0/3324858924 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "status"}) -- 0x555719fd8610 con 0x555719c50c90 2024-09-21T04:03:42.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.470+0000 7f73f632b640 1 --2- 172.21.15.64:0/3324858924 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x555719c50c90 0x55571a0f3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:42.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.471+0000 7f73d9ffb640 10 client.15300 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:03:42.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.471+0000 7f73f632b640 1 --2- 172.21.15.64:0/3324858924 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x555719c50c90 0x55571a0f3e10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c405a5c98d81484 server_cookie=95b59052af58961a in_seq=0 out_seq=0 2024-09-21T04:03:42.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.471+0000 7f73d9ffb640 1 -- 172.21.15.64:0/3324858924 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555719fd8610 con 0x555719c50c90 2024-09-21T04:03:42.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.471+0000 7f73d9ffb640 10 client.15300 handle_command_reply: tid=0 2024-09-21T04:03:42.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.471+0000 7f73feb03740 1 client.15300 shutdown 2024-09-21T04:03:42.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.471+0000 7f73feb03740 1 -- 172.21.15.64:0/3324858924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719c940a0 msgr2=0x55571a08e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:42.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.471+0000 7f73feb03740 1 --2- 172.21.15.64:0/3324858924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719c940a0 0x55571a08e260 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f73e000f730 tx=0x7f73e0012450 comp rx=0 tx=0).stop 2024-09-21T04:03:42.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.471+0000 7f73feb03740 1 -- 172.21.15.64:0/3324858924 shutdown_connections 2024-09-21T04:03:42.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.471+0000 7f73feb03740 1 --2- 172.21.15.64:0/3324858924 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x555719c50c90 0x55571a0f3e10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.471+0000 7f73feb03740 1 --2- 172.21.15.64:0/3324858924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555719cbcf60 0x55571a087ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.471+0000 7f73feb03740 1 --2- 172.21.15.64:0/3324858924 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555719c9c280 0x55571a087790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.471+0000 7f73feb03740 1 --2- 172.21.15.64:0/3324858924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555719c940a0 0x55571a08e260 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.471+0000 7f73feb03740 1 -- 172.21.15.64:0/3324858924 >> 172.21.15.64:0/3324858924 conn(0x555719cb1350 msgr2=0x55571a06de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:42.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.471+0000 7f73d9ffb640 0 client.15300 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:03:42.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.471+0000 7f73feb03740 1 -- 172.21.15.64:0/3324858924 shutdown_connections 2024-09-21T04:03:42.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.471+0000 7f73feb03740 1 -- 172.21.15.64:0/3324858924 wait complete. 2024-09-21T04:03:42.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.472+0000 7f73feb03740 20 client.15300 trim_cache size 0 max 16384 2024-09-21T04:03:42.479 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:03:42.479 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "e7c927ba-77cc-11ef-baf5-efdc52797490", 2024-09-21T04:03:42.479 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 2, 2024-09-21T04:03:42.479 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24661, 2024-09-21T04:03:42.479 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-09-21T04:03:42.479 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-09-21T04:03:42.479 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-09-21T04:03:42.479 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 90.679128765000002, 2024-09-21T04:03:42.480 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 24, 2024-09-21T04:03:42.480 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 77, 2024-09-21T04:03:42.480 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 73, 2024-09-21T04:03:42.480 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 118.301178763 2024-09-21T04:03:42.480 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:03:42.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.472+0000 7f73f858f640 1 -- 172.21.15.64:0/4140624989 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f73dc0b6590 msgr2=0x7f73dc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:42.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.472+0000 7f73f858f640 1 --2- 172.21.15.64:0/4140624989 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f73dc0b6590 0x7f73dc0b8aa0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f73e8005e30 tx=0x7f73e8041000 comp rx=0 tx=0).stop 2024-09-21T04:03:42.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.472+0000 7f73f858f640 1 -- 172.21.15.64:0/4140624989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73f0108d20 msgr2=0x7f73f019bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:42.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.472+0000 7f73f858f640 1 --2- 172.21.15.64:0/4140624989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73f0108d20 0x7f73f019bd50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f73e8005cc0 tx=0x7f73e8031700 comp rx=0 tx=0).stop 2024-09-21T04:03:42.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.472+0000 7f73f858f640 1 -- 172.21.15.64:0/4140624989 shutdown_connections 2024-09-21T04:03:42.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.472+0000 7f73f858f640 1 --2- 172.21.15.64:0/4140624989 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f73dc0b6590 0x7f73dc0b8aa0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.472+0000 7f73f858f640 1 --2- 172.21.15.64:0/4140624989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73f0108d20 0x7f73f019bd50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.472+0000 7f73f858f640 1 --2- 172.21.15.64:0/4140624989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73f01083d0 0x7f73f019b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.472+0000 7f73f858f640 1 --2- 172.21.15.64:0/4140624989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73f0102ae0 0x7f73f019b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:42.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.472+0000 7f73f858f640 1 -- 172.21.15.64:0/4140624989 >> 172.21.15.64:0/4140624989 conn(0x7f73f0079790 msgr2=0x7f73f0106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:42.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.472+0000 7f73f858f640 1 -- 172.21.15.64:0/4140624989 shutdown_connections 2024-09-21T04:03:42.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:42.472+0000 7f73f858f640 1 -- 172.21.15.64:0/4140624989 wait complete. 2024-09-21T04:03:43.190 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T04:03:43.449 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:43.858 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:43 smithi064 ceph-mon[31180]: pgmap v312: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 257 B/s wr, 7 op/s; 51 B/s, 0 keys/s, 0 objects/s recovering 2024-09-21T04:03:43.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.949+0000 7f76492bf640 1 -- 172.21.15.64:0/2747046130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76441094a0 msgr2=0x7f76441098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:43.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.949+0000 7f76492bf640 1 --2- 172.21.15.64:0/2747046130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76441094a0 0x7f76441098b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7638009900 tx=0x7f763802a410 comp rx=0 tx=0).stop 2024-09-21T04:03:43.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.949+0000 7f76492bf640 1 -- 172.21.15.64:0/2747046130 shutdown_connections 2024-09-21T04:03:43.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.949+0000 7f76492bf640 1 --2- 172.21.15.64:0/2747046130 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7644109df0 0x7f7644110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.949+0000 7f76492bf640 1 --2- 172.21.15.64:0/2747046130 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76441094a0 0x7f76441098b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.949+0000 7f76492bf640 1 --2- 172.21.15.64:0/2747046130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7644103b60 0x7f7644103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.949+0000 7f76492bf640 1 -- 172.21.15.64:0/2747046130 >> 172.21.15.64:0/2747046130 conn(0x7f764407b850 msgr2=0x7f764407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:43.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.949+0000 7f76492bf640 1 -- 172.21.15.64:0/2747046130 shutdown_connections 2024-09-21T04:03:43.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.949+0000 7f76492bf640 1 -- 172.21.15.64:0/2747046130 wait complete. 2024-09-21T04:03:43.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.950+0000 7f76492bf640 1 Processor -- start 2024-09-21T04:03:43.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.950+0000 7f76492bf640 1 -- start start 2024-09-21T04:03:43.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.951+0000 7f76492bf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7644103b60 0x7f764419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:43.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.951+0000 7f76492bf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76441094a0 0x7f764419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:43.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.951+0000 7f76492bf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7644109df0 0x7f764419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:43.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.951+0000 7f76492bf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7644113ab0 con 0x7f76441094a0 2024-09-21T04:03:43.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.951+0000 7f76492bf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7644113930 con 0x7f7644109df0 2024-09-21T04:03:43.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.951+0000 7f76492bf640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7644113c30 con 0x7f7644103b60 2024-09-21T04:03:43.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.951+0000 7f76427fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76441094a0 0x7f764419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:43.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.951+0000 7f76427fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76441094a0 0x7f764419aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52302/0 (socket says 172.21.15.64:52302) 2024-09-21T04:03:43.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.951+0000 7f76427fc640 1 -- 172.21.15.64:0/3885204441 learned_addr learned my addr 172.21.15.64:0/3885204441 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:43.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.951+0000 7f76437fe640 1 --2- 172.21.15.64:0/3885204441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7644109df0 0x7f764419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:43.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.951+0000 7f7642ffd640 1 --2- 172.21.15.64:0/3885204441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7644103b60 0x7f764419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:43.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.951+0000 7f76427fc640 1 -- 172.21.15.64:0/3885204441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7644103b60 msgr2=0x7f764419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:43.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.951+0000 7f76427fc640 1 --2- 172.21.15.64:0/3885204441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7644103b60 0x7f764419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.951+0000 7f76427fc640 1 -- 172.21.15.64:0/3885204441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7644109df0 msgr2=0x7f764419fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:43.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.951+0000 7f76427fc640 1 --2- 172.21.15.64:0/3885204441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7644109df0 0x7f764419fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.951+0000 7f76427fc640 1 -- 172.21.15.64:0/3885204441 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76441a03f0 con 0x7f76441094a0 2024-09-21T04:03:43.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.951+0000 7f76427fc640 1 --2- 172.21.15.64:0/3885204441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76441094a0 0x7f764419aa20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f763802a930 tx=0x7f7638004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:43.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.952+0000 7f762bfff640 1 -- 172.21.15.64:0/3885204441 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763803f030 con 0x7f76441094a0 2024-09-21T04:03:43.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.952+0000 7f76492bf640 1 -- 172.21.15.64:0/3885204441 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76441a0680 con 0x7f76441094a0 2024-09-21T04:03:43.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.952+0000 7f762bfff640 1 -- 172.21.15.64:0/3885204441 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7638003aa0 con 0x7f76441094a0 2024-09-21T04:03:43.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.952+0000 7f762bfff640 1 -- 172.21.15.64:0/3885204441 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638035070 con 0x7f76441094a0 2024-09-21T04:03:43.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.952+0000 7f76492bf640 1 -- 172.21.15.64:0/3885204441 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76441a6e40 con 0x7f76441094a0 2024-09-21T04:03:43.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.955+0000 7f764f833740 1 Processor -- start 2024-09-21T04:03:43.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.955+0000 7f764f833740 1 -- start start 2024-09-21T04:03:43.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.955+0000 7f764f833740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563297130f60 0x5632974d6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:43.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.955+0000 7f764f833740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632971080a0 0x5632974d7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:43.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.955+0000 7f764f833740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563297110280 0x5632974d7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:43.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.955+0000 7f764f833740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56329744ec10 con 0x563297110280 2024-09-21T04:03:43.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.955+0000 7f764f833740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56329743e040 con 0x563297130f60 2024-09-21T04:03:43.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.955+0000 7f764f833740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x563297441e50 con 0x5632971080a0 2024-09-21T04:03:43.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.955+0000 7f76437fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563297130f60 0x5632974d6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:43.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.955+0000 7f76437fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563297130f60 0x5632974d6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:49070/0 (socket says 172.21.15.64:49070) 2024-09-21T04:03:43.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.955+0000 7f76437fe640 1 -- 172.21.15.64:0/776027778 learned_addr learned my addr 172.21.15.64:0/776027778 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:43.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.956+0000 7f7642ffd640 1 --2- 172.21.15.64:0/776027778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632971080a0 0x5632974d7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:43.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.956+0000 7f762bfff640 1 -- 172.21.15.64:0/3885204441 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f7638042030 con 0x7f76441094a0 2024-09-21T04:03:43.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.956+0000 7f76427fc640 1 --2- 172.21.15.64:0/776027778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563297110280 0x5632974d7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:43.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.956+0000 7f76437fe640 1 -- 172.21.15.64:0/776027778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632971080a0 msgr2=0x5632974d7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:43.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.956+0000 7f76437fe640 1 --2- 172.21.15.64:0/776027778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632971080a0 0x5632974d7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.956+0000 7f76437fe640 1 -- 172.21.15.64:0/776027778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563297110280 msgr2=0x5632974d7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:43.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.956+0000 7f76437fe640 1 --2- 172.21.15.64:0/776027778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563297110280 0x5632974d7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.956+0000 7f76437fe640 1 -- 172.21.15.64:0/776027778 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56329719c540 con 0x563297130f60 2024-09-21T04:03:43.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.956+0000 7f7642ffd640 1 --2- 172.21.15.64:0/776027778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632971080a0 0x5632974d7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:03:43.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.956+0000 7f76427fc640 1 --2- 172.21.15.64:0/776027778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563297110280 0x5632974d7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:03:43.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.956+0000 7f76437fe640 1 --2- 172.21.15.64:0/776027778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563297130f60 0x5632974d6f00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f763c010370 tx=0x7f763c010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bd329b2d9616768 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:43.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.956+0000 7f76297fa640 1 -- 172.21.15.64:0/776027778 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c016070 con 0x563297130f60 2024-09-21T04:03:43.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.957+0000 7f76297fa640 1 -- 172.21.15.64:0/776027778 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f763c004d20 con 0x563297130f60 2024-09-21T04:03:43.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.957+0000 7f76297fa640 1 -- 172.21.15.64:0/776027778 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c005350 con 0x563297130f60 2024-09-21T04:03:43.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.957+0000 7f762bfff640 1 --2- 172.21.15.64:0/3885204441 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f762c0b6560 0x7f762c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:43.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.957+0000 7f7642ffd640 1 --2- 172.21.15.64:0/3885204441 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f762c0b6560 0x7f762c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:43.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.957+0000 7f762bfff640 1 -- 172.21.15.64:0/3885204441 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f76381077d0 con 0x7f76441094a0 2024-09-21T04:03:43.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.957+0000 7f7642ffd640 1 --2- 172.21.15.64:0/3885204441 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f762c0b6560 0x7f762c0b8a70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x5632974d5b80 tx=0x7f763400b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:43.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.957+0000 7f764f833740 1 -- 172.21.15.64:0/776027778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563297130f60 msgr2=0x5632974d6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:43.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.957+0000 7f764f833740 1 --2- 172.21.15.64:0/776027778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563297130f60 0x5632974d6f00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f763c010370 tx=0x7f763c010830 comp rx=0 tx=0).stop 2024-09-21T04:03:43.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.958+0000 7f764f833740 1 -- 172.21.15.64:0/776027778 shutdown_connections 2024-09-21T04:03:43.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.958+0000 7f764f833740 1 --2- 172.21.15.64:0/776027778 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563297110280 0x5632974d7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.958+0000 7f764f833740 1 --2- 172.21.15.64:0/776027778 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632971080a0 0x5632974d7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.958+0000 7f764f833740 1 --2- 172.21.15.64:0/776027778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563297130f60 0x5632974d6f00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.958+0000 7f764f833740 1 -- 172.21.15.64:0/776027778 >> 172.21.15.64:0/776027778 conn(0x563297125350 msgr2=0x5632974d4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:43.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.958+0000 7f764f833740 1 -- 172.21.15.64:0/776027778 shutdown_connections 2024-09-21T04:03:43.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.958+0000 7f764f833740 1 -- 172.21.15.64:0/776027778 wait complete. 2024-09-21T04:03:43.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.958+0000 7f764f833740 1 -- start start 2024-09-21T04:03:43.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.958+0000 7f764f833740 1 Processor -- start 2024-09-21T04:03:43.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.959+0000 7f764f833740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632971080a0 0x5632974e2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:43.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.959+0000 7f764f833740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563297110280 0x5632974e2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:43.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.959+0000 7f764f833740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563297130f60 0x5632974ee9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:43.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.959+0000 7f764f833740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56329729f5f0 con 0x5632971080a0 2024-09-21T04:03:43.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.959+0000 7f764f833740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5632973d9010 con 0x563297130f60 2024-09-21T04:03:43.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.959+0000 7f764f833740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5632974e56e0 con 0x563297110280 2024-09-21T04:03:43.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.959+0000 7f764f833740 10 client.0 _ll_register_callbacks cb 0x563297438340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:03:43.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.959+0000 7f7642ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632971080a0 0x5632974e2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:43.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.959+0000 7f7642ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632971080a0 0x5632974e2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52316/0 (socket says 172.21.15.64:52316) 2024-09-21T04:03:43.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.959+0000 7f7642ffd640 1 -- 172.21.15.64:0/4211579985 learned_addr learned my addr 172.21.15.64:0/4211579985 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:43.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.959+0000 7f76437fe640 1 --2- 172.21.15.64:0/4211579985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563297130f60 0x5632974ee9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:43.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.959+0000 7f76427fc640 1 --2- 172.21.15.64:0/4211579985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563297110280 0x5632974e2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:43.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.959+0000 7f7642ffd640 1 -- 172.21.15.64:0/4211579985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563297110280 msgr2=0x5632974e2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:43.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.959+0000 7f7642ffd640 1 --2- 172.21.15.64:0/4211579985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563297110280 0x5632974e2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.959+0000 7f7642ffd640 1 -- 172.21.15.64:0/4211579985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563297130f60 msgr2=0x5632974ee9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:43.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.959+0000 7f7642ffd640 1 --2- 172.21.15.64:0/4211579985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563297130f60 0x5632974ee9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.959+0000 7f7642ffd640 1 -- 172.21.15.64:0/4211579985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563297406a00 con 0x5632971080a0 2024-09-21T04:03:43.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.959+0000 7f7642ffd640 1 --2- 172.21.15.64:0/4211579985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632971080a0 0x5632974e2a40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7634003d00 tx=0x7f7634004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84647cbe30e39c44 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:43.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.959+0000 7f7626ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:03:43.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.960+0000 7f7626ff5640 1 -- 172.21.15.64:0/4211579985 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763402c060 con 0x5632971080a0 2024-09-21T04:03:43.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.960+0000 7f7626ff5640 1 -- 172.21.15.64:0/4211579985 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f7634017030 con 0x5632971080a0 2024-09-21T04:03:43.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.960+0000 7f764f833740 1 -- 172.21.15.64:0/4211579985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f763c004d20 con 0x5632971080a0 2024-09-21T04:03:43.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.960+0000 7f7626ff5640 1 -- 172.21.15.64:0/4211579985 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7634030440 con 0x5632971080a0 2024-09-21T04:03:43.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.960+0000 7f764f833740 10 client.15309 fetch_fsmap learned FSMap version 24 2024-09-21T04:03:43.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.960+0000 7f764f833740 1 -- 172.21.15.64:0/4211579985 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5632974f1640 con 0x5632971080a0 2024-09-21T04:03:43.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.960+0000 7f7626ff5640 1 -- 172.21.15.64:0/4211579985 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f763c004d20 con 0x5632971080a0 2024-09-21T04:03:43.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.960+0000 7f7626ff5640 1 -- 172.21.15.64:0/4211579985 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f763401c3f0 con 0x5632971080a0 2024-09-21T04:03:43.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.960+0000 7f764f833740 10 client.15309 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T04:03:43.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.960+0000 7f764f833740 10 client.15309 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T04:03:43.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.960+0000 7f764f833740 20 client.15309 populate_metadata read hostname 'smithi064' 2024-09-21T04:03:43.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.960+0000 7f764f833740 1 --2- 172.21.15.64:0/4211579985 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5632970ef8e0 0x5632974f5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:43.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.960+0000 7f764f833740 4 client.15309 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:03:43.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.960+0000 7f764f833740 1 -- 172.21.15.64:0/4211579985 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563297450b60 con 0x5632970ef8e0 2024-09-21T04:03:43.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.961+0000 7f76427fc640 1 --2- 172.21.15.64:0/4211579985 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5632970ef8e0 0x5632974f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:43.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.961+0000 7f7626ff5640 10 client.15309 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:03:43.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.961+0000 7f76427fc640 1 --2- 172.21.15.64:0/4211579985 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5632970ef8e0 0x5632974f5090 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5cca6e74690af0f4 server_cookie=41191825289d3311 in_seq=0 out_seq=0 2024-09-21T04:03:43.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.962+0000 7f7626ff5640 1 -- 172.21.15.64:0/4211579985 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563297450b60 con 0x5632970ef8e0 2024-09-21T04:03:43.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.962+0000 7f7626ff5640 10 client.15309 handle_command_reply: tid=0 2024-09-21T04:03:43.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.962+0000 7f764f833740 1 client.15309 shutdown 2024-09-21T04:03:43.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.962+0000 7f764f833740 1 -- 172.21.15.64:0/4211579985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632971080a0 msgr2=0x5632974e2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:43.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.962+0000 7f764f833740 1 --2- 172.21.15.64:0/4211579985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632971080a0 0x5632974e2a40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7634003d00 tx=0x7f7634004730 comp rx=0 tx=0).stop 2024-09-21T04:03:43.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.962+0000 7f764f833740 1 -- 172.21.15.64:0/4211579985 shutdown_connections 2024-09-21T04:03:43.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.962+0000 7f764f833740 1 --2- 172.21.15.64:0/4211579985 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5632970ef8e0 0x5632974f5090 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.962+0000 7f764f833740 1 --2- 172.21.15.64:0/4211579985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563297130f60 0x5632974ee9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.962+0000 7f764f833740 1 --2- 172.21.15.64:0/4211579985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563297110280 0x5632974e2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.962+0000 7f764f833740 1 --2- 172.21.15.64:0/4211579985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632971080a0 0x5632974e2a40 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.962+0000 7f764f833740 1 -- 172.21.15.64:0/4211579985 >> 172.21.15.64:0/4211579985 conn(0x563297125350 msgr2=0x5632974da0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:43.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.962+0000 7f7626ff5640 0 client.15309 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:03:43.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.962+0000 7f764f833740 1 -- 172.21.15.64:0/4211579985 shutdown_connections 2024-09-21T04:03:43.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.962+0000 7f764f833740 1 -- 172.21.15.64:0/4211579985 wait complete. 2024-09-21T04:03:43.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.962+0000 7f764f833740 20 client.15309 trim_cache size 0 max 16384 2024-09-21T04:03:43.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.980+0000 7f764f833740 1 Processor -- start 2024-09-21T04:03:43.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.981+0000 7f764f833740 1 -- start start 2024-09-21T04:03:43.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.981+0000 7f764f833740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632971080a0 0x5632974e1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:43.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.981+0000 7f764f833740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563297110280 0x5632974fd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:43.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.981+0000 7f764f833740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563297130f60 0x5632974fdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:43.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.981+0000 7f764f833740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5632974f33d0 con 0x5632971080a0 2024-09-21T04:03:43.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.981+0000 7f764f833740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5632974f3aa0 con 0x563297130f60 2024-09-21T04:03:43.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.981+0000 7f764f833740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5632974f4170 con 0x563297110280 2024-09-21T04:03:43.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.981+0000 7f7642ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632971080a0 0x5632974e1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:43.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.981+0000 7f7642ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632971080a0 0x5632974e1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52326/0 (socket says 172.21.15.64:52326) 2024-09-21T04:03:43.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.981+0000 7f7642ffd640 1 -- 172.21.15.64:0/390838540 learned_addr learned my addr 172.21.15.64:0/390838540 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:43.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.981+0000 7f76437fe640 1 --2- 172.21.15.64:0/390838540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563297130f60 0x5632974fdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:43.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.981+0000 7f76427fc640 1 --2- 172.21.15.64:0/390838540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563297110280 0x5632974fd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:43.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.981+0000 7f7642ffd640 1 -- 172.21.15.64:0/390838540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563297110280 msgr2=0x5632974fd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:43.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.981+0000 7f7642ffd640 1 --2- 172.21.15.64:0/390838540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563297110280 0x5632974fd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.981+0000 7f7642ffd640 1 -- 172.21.15.64:0/390838540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563297130f60 msgr2=0x5632974fdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:43.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.981+0000 7f7642ffd640 1 --2- 172.21.15.64:0/390838540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563297130f60 0x5632974fdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.981+0000 7f7642ffd640 1 -- 172.21.15.64:0/390838540 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632974f4620 con 0x5632971080a0 2024-09-21T04:03:43.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.981+0000 7f76437fe640 1 --2- 172.21.15.64:0/390838540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563297130f60 0x5632974fdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:03:43.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.981+0000 7f7642ffd640 1 --2- 172.21.15.64:0/390838540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632971080a0 0x5632974e1d30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f763400bfd0 tx=0x7f7634003fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b444507110ebd9c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:43.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.982+0000 7f7628ff9640 1 -- 172.21.15.64:0/390838540 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763401ca30 con 0x5632971080a0 2024-09-21T04:03:43.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.982+0000 7f7628ff9640 1 -- 172.21.15.64:0/390838540 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7634004c50 con 0x5632971080a0 2024-09-21T04:03:43.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.982+0000 7f7628ff9640 1 -- 172.21.15.64:0/390838540 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76340053c0 con 0x5632971080a0 2024-09-21T04:03:43.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.982+0000 7f764f833740 1 -- 172.21.15.64:0/390838540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632971080a0 msgr2=0x5632974e1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:43.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.982+0000 7f764f833740 1 --2- 172.21.15.64:0/390838540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632971080a0 0x5632974e1d30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f763400bfd0 tx=0x7f7634003fc0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.982+0000 7f764f833740 1 -- 172.21.15.64:0/390838540 shutdown_connections 2024-09-21T04:03:43.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.982+0000 7f764f833740 1 --2- 172.21.15.64:0/390838540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563297130f60 0x5632974fdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.982+0000 7f764f833740 1 --2- 172.21.15.64:0/390838540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563297110280 0x5632974fd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.982+0000 7f764f833740 1 --2- 172.21.15.64:0/390838540 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632971080a0 0x5632974e1d30 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.982+0000 7f764f833740 1 -- 172.21.15.64:0/390838540 >> 172.21.15.64:0/390838540 conn(0x563297125350 msgr2=0x5632974e2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:43.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.982+0000 7f764f833740 1 -- 172.21.15.64:0/390838540 shutdown_connections 2024-09-21T04:03:43.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.982+0000 7f764f833740 1 -- 172.21.15.64:0/390838540 wait complete. 2024-09-21T04:03:43.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.983+0000 7f764f833740 1 -- start start 2024-09-21T04:03:43.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.983+0000 7f764f833740 1 Processor -- start 2024-09-21T04:03:43.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.983+0000 7f764f833740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632971080a0 0x5632974f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:43.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.983+0000 7f764f833740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563297110280 0x5632974ee3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:43.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.983+0000 7f764f833740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563297130f60 0x5632974ee900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:43.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.983+0000 7f764f833740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5632974f4f10 con 0x5632971080a0 2024-09-21T04:03:43.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.983+0000 7f764f833740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5632974f4840 con 0x563297110280 2024-09-21T04:03:43.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.983+0000 7f764f833740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5632974f55e0 con 0x563297130f60 2024-09-21T04:03:43.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.983+0000 7f764f833740 10 client.0 _ll_register_callbacks cb 0x7f7634003bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:03:43.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.983+0000 7f7642ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632971080a0 0x5632974f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:43.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.983+0000 7f7642ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632971080a0 0x5632974f7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52328/0 (socket says 172.21.15.64:52328) 2024-09-21T04:03:43.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.983+0000 7f7642ffd640 1 -- 172.21.15.64:0/1937745849 learned_addr learned my addr 172.21.15.64:0/1937745849 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:43.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.983+0000 7f76427fc640 1 --2- 172.21.15.64:0/1937745849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563297110280 0x5632974ee3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:43.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.984+0000 7f76437fe640 1 --2- 172.21.15.64:0/1937745849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563297130f60 0x5632974ee900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:43.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.984+0000 7f7642ffd640 1 -- 172.21.15.64:0/1937745849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563297130f60 msgr2=0x5632974ee900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:43.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.984+0000 7f7642ffd640 1 --2- 172.21.15.64:0/1937745849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563297130f60 0x5632974ee900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.984+0000 7f7642ffd640 1 -- 172.21.15.64:0/1937745849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563297110280 msgr2=0x5632974ee3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:43.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.984+0000 7f7642ffd640 1 --2- 172.21.15.64:0/1937745849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563297110280 0x5632974ee3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.984+0000 7f7642ffd640 1 -- 172.21.15.64:0/1937745849 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632974ef070 con 0x5632971080a0 2024-09-21T04:03:43.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.984+0000 7f76437fe640 1 --2- 172.21.15.64:0/1937745849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563297130f60 0x5632974ee900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:03:43.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.984+0000 7f7642ffd640 1 --2- 172.21.15.64:0/1937745849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632971080a0 0x5632974f7b80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f763401c930 tx=0x7f763401c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a5676db1be3a23 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:43.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.984+0000 7f7626ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:03:43.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.984+0000 7f7626ff5640 1 -- 172.21.15.64:0/1937745849 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763403f020 con 0x5632971080a0 2024-09-21T04:03:43.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.984+0000 7f764f833740 1 -- 172.21.15.64:0/1937745849 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7634004c50 con 0x5632971080a0 2024-09-21T04:03:43.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.984+0000 7f7626ff5640 1 -- 172.21.15.64:0/1937745849 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f763401c9b0 con 0x5632971080a0 2024-09-21T04:03:43.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.984+0000 7f7626ff5640 1 -- 172.21.15.64:0/1937745849 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f763404a050 con 0x5632971080a0 2024-09-21T04:03:43.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.984+0000 7f764f833740 10 client.15315 fetch_fsmap learned FSMap version 24 2024-09-21T04:03:43.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.984+0000 7f764f833740 1 -- 172.21.15.64:0/1937745849 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563297564450 con 0x5632971080a0 2024-09-21T04:03:43.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.984+0000 7f7626ff5640 1 -- 172.21.15.64:0/1937745849 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7634004c50 con 0x5632971080a0 2024-09-21T04:03:43.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.985+0000 7f7626ff5640 1 -- 172.21.15.64:0/1937745849 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f76340307a0 con 0x5632971080a0 2024-09-21T04:03:43.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.985+0000 7f764f833740 10 client.15315 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T04:03:43.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.985+0000 7f764f833740 10 client.15315 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T04:03:43.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.985+0000 7f764f833740 20 client.15315 populate_metadata read hostname 'smithi064' 2024-09-21T04:03:43.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.985+0000 7f764f833740 1 --2- 172.21.15.64:0/1937745849 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5632970c4c90 0x563297567960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:43.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.985+0000 7f764f833740 4 client.15315 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T04:03:43.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.985+0000 7f764f833740 1 -- 172.21.15.64:0/1937745849 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "status"}) -- 0x56329744c610 con 0x5632970c4c90 2024-09-21T04:03:43.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.985+0000 7f76427fc640 1 --2- 172.21.15.64:0/1937745849 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5632970c4c90 0x563297567960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:43.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.986+0000 7f7626ff5640 10 client.15315 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:03:43.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.986+0000 7f76427fc640 1 --2- 172.21.15.64:0/1937745849 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5632970c4c90 0x563297567960 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=814f75d08a440ff4 server_cookie=9f90e8f04718d63d in_seq=0 out_seq=0 2024-09-21T04:03:43.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.986+0000 7f7626ff5640 1 -- 172.21.15.64:0/1937745849 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56329744c610 con 0x5632970c4c90 2024-09-21T04:03:43.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.986+0000 7f7626ff5640 10 client.15315 handle_command_reply: tid=0 2024-09-21T04:03:43.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.986+0000 7f764f833740 1 client.15315 shutdown 2024-09-21T04:03:43.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.986+0000 7f764f833740 1 -- 172.21.15.64:0/1937745849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632971080a0 msgr2=0x5632974f7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:43.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.986+0000 7f764f833740 1 --2- 172.21.15.64:0/1937745849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632971080a0 0x5632974f7b80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f763401c930 tx=0x7f763401c620 comp rx=0 tx=0).stop 2024-09-21T04:03:43.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.986+0000 7f764f833740 1 -- 172.21.15.64:0/1937745849 shutdown_connections 2024-09-21T04:03:43.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.986+0000 7f764f833740 1 --2- 172.21.15.64:0/1937745849 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5632970c4c90 0x563297567960 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.986+0000 7f7626ff5640 0 client.15315 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:03:43.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.986+0000 7f764f833740 1 --2- 172.21.15.64:0/1937745849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563297130f60 0x5632974ee900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.986+0000 7f764f833740 1 --2- 172.21.15.64:0/1937745849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563297110280 0x5632974ee3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.986+0000 7f764f833740 1 --2- 172.21.15.64:0/1937745849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5632971080a0 0x5632974f7b80 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.986+0000 7f764f833740 1 -- 172.21.15.64:0/1937745849 >> 172.21.15.64:0/1937745849 conn(0x563297125350 msgr2=0x5632974dd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:43.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.986+0000 7f764f833740 1 -- 172.21.15.64:0/1937745849 shutdown_connections 2024-09-21T04:03:43.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.986+0000 7f764f833740 1 -- 172.21.15.64:0/1937745849 wait complete. 2024-09-21T04:03:43.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.986+0000 7f764f833740 20 client.15315 trim_cache size 0 max 16384 2024-09-21T04:03:43.994 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:03:43.994 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "e7c927ba-77cc-11ef-baf5-efdc52797490", 2024-09-21T04:03:43.994 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 1, 2024-09-21T04:03:43.994 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24680, 2024-09-21T04:03:43.994 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-09-21T04:03:43.994 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-09-21T04:03:43.994 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-09-21T04:03:43.994 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 93.200543597000006, 2024-09-21T04:03:43.994 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 24, 2024-09-21T04:03:43.994 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 77, 2024-09-21T04:03:43.994 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 73, 2024-09-21T04:03:43.994 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 122.319542493 2024-09-21T04:03:43.995 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:03:43.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.987+0000 7f76492bf640 1 -- 172.21.15.64:0/3885204441 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f762c0b6560 msgr2=0x7f762c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:43.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.987+0000 7f76492bf640 1 --2- 172.21.15.64:0/3885204441 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f762c0b6560 0x7f762c0b8a70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x5632974d5b80 tx=0x7f763400b390 comp rx=0 tx=0).stop 2024-09-21T04:03:43.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.987+0000 7f76492bf640 1 -- 172.21.15.64:0/3885204441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76441094a0 msgr2=0x7f764419aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:43.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.987+0000 7f76492bf640 1 --2- 172.21.15.64:0/3885204441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76441094a0 0x7f764419aa20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f763802a930 tx=0x7f7638004270 comp rx=0 tx=0).stop 2024-09-21T04:03:43.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.987+0000 7f76492bf640 1 -- 172.21.15.64:0/3885204441 shutdown_connections 2024-09-21T04:03:43.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.987+0000 7f76492bf640 1 --2- 172.21.15.64:0/3885204441 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f762c0b6560 0x7f762c0b8a70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.987+0000 7f76492bf640 1 --2- 172.21.15.64:0/3885204441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7644109df0 0x7f764419fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.987+0000 7f76492bf640 1 --2- 172.21.15.64:0/3885204441 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76441094a0 0x7f764419aa20 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.987+0000 7f76492bf640 1 --2- 172.21.15.64:0/3885204441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7644103b60 0x7f764419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:43.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.987+0000 7f76492bf640 1 -- 172.21.15.64:0/3885204441 >> 172.21.15.64:0/3885204441 conn(0x7f764407b850 msgr2=0x7f7644105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:43.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.987+0000 7f76492bf640 1 -- 172.21.15.64:0/3885204441 shutdown_connections 2024-09-21T04:03:43.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:43.987+0000 7f76492bf640 1 -- 172.21.15.64:0/3885204441 wait complete. 2024-09-21T04:03:44.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:43 smithi119 ceph-mon[34811]: pgmap v312: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 257 B/s wr, 7 op/s; 51 B/s, 0 keys/s, 0 objects/s recovering 2024-09-21T04:03:44.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:43 smithi131 ceph-mon[35193]: pgmap v312: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 257 B/s wr, 7 op/s; 51 B/s, 0 keys/s, 0 objects/s recovering 2024-09-21T04:03:44.782 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T04:03:45.047 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:45.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.541+0000 7fea0de3a640 1 -- 172.21.15.64:0/2673147519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea081049e0 msgr2=0x7fea08104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:45.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.541+0000 7fea0de3a640 1 --2- 172.21.15.64:0/2673147519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea081049e0 0x7fea08104df0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009900 tx=0x7fe9f802a410 comp rx=0 tx=0).stop 2024-09-21T04:03:45.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.541+0000 7fea0de3a640 1 -- 172.21.15.64:0/2673147519 shutdown_connections 2024-09-21T04:03:45.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.541+0000 7fea0de3a640 1 --2- 172.21.15.64:0/2673147519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea0810de40 0x7fea08110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.541+0000 7fea0de3a640 1 --2- 172.21.15.64:0/2673147519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea08105330 0x7fea0810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.541+0000 7fea0de3a640 1 --2- 172.21.15.64:0/2673147519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea081049e0 0x7fea08104df0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.541+0000 7fea0de3a640 1 -- 172.21.15.64:0/2673147519 >> 172.21.15.64:0/2673147519 conn(0x7fea08079750 msgr2=0x7fea08079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:45.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.541+0000 7fea0de3a640 1 -- 172.21.15.64:0/2673147519 shutdown_connections 2024-09-21T04:03:45.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.541+0000 7fea0de3a640 1 -- 172.21.15.64:0/2673147519 wait complete. 2024-09-21T04:03:45.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.542+0000 7fea0de3a640 1 Processor -- start 2024-09-21T04:03:45.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.542+0000 7fea0de3a640 1 -- start start 2024-09-21T04:03:45.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.542+0000 7fea0de3a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea081049e0 0x7fea08102720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:45.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.542+0000 7fea0de3a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea08105330 0x7fea08102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:45.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.542+0000 7fea0de3a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea0810de40 0x7fea081031a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:45.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.542+0000 7fea0de3a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fea08113aa0 con 0x7fea081049e0 2024-09-21T04:03:45.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.542+0000 7fea0de3a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fea08113920 con 0x7fea0810de40 2024-09-21T04:03:45.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.542+0000 7fea0de3a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fea08113c20 con 0x7fea08105330 2024-09-21T04:03:45.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.542+0000 7fea077fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea081049e0 0x7fea08102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:45.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.542+0000 7fea077fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea081049e0 0x7fea08102720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52348/0 (socket says 172.21.15.64:52348) 2024-09-21T04:03:45.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.542+0000 7fea077fe640 1 -- 172.21.15.64:0/3892948671 learned_addr learned my addr 172.21.15.64:0/3892948671 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:45.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.542+0000 7fea06ffd640 1 --2- 172.21.15.64:0/3892948671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea08105330 0x7fea08102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:45.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.542+0000 7fea07fff640 1 --2- 172.21.15.64:0/3892948671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea0810de40 0x7fea081031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:45.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.542+0000 7fea077fe640 1 -- 172.21.15.64:0/3892948671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea08105330 msgr2=0x7fea08102c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:45.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.543+0000 7fea077fe640 1 --2- 172.21.15.64:0/3892948671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea08105330 0x7fea08102c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.543+0000 7fea077fe640 1 -- 172.21.15.64:0/3892948671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea0810de40 msgr2=0x7fea081031a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:45.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.543+0000 7fea077fe640 1 --2- 172.21.15.64:0/3892948671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea0810de40 0x7fea081031a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.543+0000 7fea077fe640 1 -- 172.21.15.64:0/3892948671 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea081a8840 con 0x7fea081049e0 2024-09-21T04:03:45.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.543+0000 7fea07fff640 1 --2- 172.21.15.64:0/3892948671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea0810de40 0x7fea081031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:03:45.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.543+0000 7fea077fe640 1 --2- 172.21.15.64:0/3892948671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea081049e0 0x7fea08102720 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8032fd0 tx=0x7fe9f8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:45.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.543+0000 7fea04ff9640 1 -- 172.21.15.64:0/3892948671 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f803f030 con 0x7fea081049e0 2024-09-21T04:03:45.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.543+0000 7fea0de3a640 1 -- 172.21.15.64:0/3892948671 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea081a8ad0 con 0x7fea081049e0 2024-09-21T04:03:45.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.543+0000 7fea04ff9640 1 -- 172.21.15.64:0/3892948671 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9f8004490 con 0x7fea081049e0 2024-09-21T04:03:45.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.543+0000 7fea04ff9640 1 -- 172.21.15.64:0/3892948671 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8035070 con 0x7fea081049e0 2024-09-21T04:03:45.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.543+0000 7fea0de3a640 1 -- 172.21.15.64:0/3892948671 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea081a8fb0 con 0x7fea081049e0 2024-09-21T04:03:45.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.546+0000 7fea143ae740 1 Processor -- start 2024-09-21T04:03:45.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.546+0000 7fea143ae740 1 -- start start 2024-09-21T04:03:45.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.546+0000 7fea143ae740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cfaf60 0x5632940a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:45.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.546+0000 7fea143ae740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563293cd20a0 0x5632940a1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:45.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.547+0000 7fea143ae740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563293cda280 0x5632940a1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:45.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.547+0000 7fea143ae740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563294018c10 con 0x563293cfaf60 2024-09-21T04:03:45.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.547+0000 7fea143ae740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563294008040 con 0x563293cd20a0 2024-09-21T04:03:45.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.547+0000 7fea143ae740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56329400be50 con 0x563293cda280 2024-09-21T04:03:45.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.547+0000 7fea07fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cfaf60 0x5632940a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:45.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.547+0000 7fea07fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cfaf60 0x5632940a0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52352/0 (socket says 172.21.15.64:52352) 2024-09-21T04:03:45.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.547+0000 7fea07fff640 1 -- 172.21.15.64:0/3179080870 learned_addr learned my addr 172.21.15.64:0/3179080870 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:45.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.547+0000 7fea06ffd640 1 --2- 172.21.15.64:0/3179080870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563293cda280 0x5632940a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:45.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.547+0000 7fea04ff9640 1 -- 172.21.15.64:0/3892948671 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fe9f8042030 con 0x7fea081049e0 2024-09-21T04:03:45.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.547+0000 7fea07fff640 1 -- 172.21.15.64:0/3179080870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563293cda280 msgr2=0x5632940a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:45.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.547+0000 7fea07fff640 1 --2- 172.21.15.64:0/3179080870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563293cda280 0x5632940a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.547+0000 7fea07fff640 1 -- 172.21.15.64:0/3179080870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563293cd20a0 msgr2=0x5632940a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:45.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.547+0000 7fea07fff640 1 --2- 172.21.15.64:0/3179080870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563293cd20a0 0x5632940a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.547+0000 7fea07fff640 1 -- 172.21.15.64:0/3179080870 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563293d66540 con 0x563293cfaf60 2024-09-21T04:03:45.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.547+0000 7fea07fff640 1 --2- 172.21.15.64:0/3179080870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cfaf60 0x5632940a0f00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fea00008de0 tx=0x7fea0000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b14a502a4a373a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:45.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.547+0000 7fe9edffb640 1 -- 172.21.15.64:0/3179080870 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0000f830 con 0x563293cfaf60 2024-09-21T04:03:45.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.548+0000 7fe9edffb640 1 -- 172.21.15.64:0/3179080870 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea0000de20 con 0x563293cfaf60 2024-09-21T04:03:45.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.548+0000 7fe9edffb640 1 -- 172.21.15.64:0/3179080870 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0000ee10 con 0x563293cfaf60 2024-09-21T04:03:45.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.548+0000 7fea04ff9640 1 --2- 172.21.15.64:0/3892948671 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe9f00b6560 0x7fe9f00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:45.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.548+0000 7fea077fe640 1 --2- 172.21.15.64:0/3892948671 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe9f00b6560 0x7fe9f00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:45.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.548+0000 7fea04ff9640 1 -- 172.21.15.64:0/3892948671 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fe9f8107af0 con 0x7fea081049e0 2024-09-21T04:03:45.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.548+0000 7fea143ae740 1 -- 172.21.15.64:0/3179080870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cfaf60 msgr2=0x5632940a0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:45.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.548+0000 7fea143ae740 1 --2- 172.21.15.64:0/3179080870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cfaf60 0x5632940a0f00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fea00008de0 tx=0x7fea0000d710 comp rx=0 tx=0).stop 2024-09-21T04:03:45.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.548+0000 7fea077fe640 1 --2- 172.21.15.64:0/3892948671 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe9f00b6560 0x7fe9f00b8a70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8105440 tx=0x7fe9f8000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:45.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.549+0000 7fea143ae740 1 -- 172.21.15.64:0/3179080870 shutdown_connections 2024-09-21T04:03:45.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.549+0000 7fea143ae740 1 --2- 172.21.15.64:0/3179080870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563293cda280 0x5632940a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.549+0000 7fea143ae740 1 --2- 172.21.15.64:0/3179080870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563293cd20a0 0x5632940a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.549+0000 7fea143ae740 1 --2- 172.21.15.64:0/3179080870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cfaf60 0x5632940a0f00 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.549+0000 7fea143ae740 1 -- 172.21.15.64:0/3179080870 >> 172.21.15.64:0/3179080870 conn(0x563293cef350 msgr2=0x56329409edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:45.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.549+0000 7fea143ae740 1 -- 172.21.15.64:0/3179080870 shutdown_connections 2024-09-21T04:03:45.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.549+0000 7fea143ae740 1 -- 172.21.15.64:0/3179080870 wait complete. 2024-09-21T04:03:45.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.549+0000 7fea143ae740 1 -- start start 2024-09-21T04:03:45.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.549+0000 7fea143ae740 1 Processor -- start 2024-09-21T04:03:45.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.550+0000 7fea143ae740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563293cd20a0 0x5632940ac7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:45.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.550+0000 7fea143ae740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cda280 0x5632940acd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:45.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.550+0000 7fea143ae740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563293cfaf60 0x5632940b8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:45.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.550+0000 7fea143ae740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563293e695f0 con 0x563293cda280 2024-09-21T04:03:45.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.550+0000 7fea143ae740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563293fa3010 con 0x563293cfaf60 2024-09-21T04:03:45.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.550+0000 7fea143ae740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5632940af6e0 con 0x563293cd20a0 2024-09-21T04:03:45.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.550+0000 7fea143ae740 10 client.0 _ll_register_callbacks cb 0x563294002340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:03:45.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.550+0000 7fea06ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cda280 0x5632940acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:45.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.550+0000 7fea06ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cda280 0x5632940acd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52358/0 (socket says 172.21.15.64:52358) 2024-09-21T04:03:45.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.550+0000 7fea06ffd640 1 -- 172.21.15.64:0/33444834 learned_addr learned my addr 172.21.15.64:0/33444834 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:45.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.550+0000 7fea07fff640 1 --2- 172.21.15.64:0/33444834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563293cd20a0 0x5632940ac7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:45.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.550+0000 7fea06ffd640 1 -- 172.21.15.64:0/33444834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563293cd20a0 msgr2=0x5632940ac7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:45.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.550+0000 7fea06ffd640 1 --2- 172.21.15.64:0/33444834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563293cd20a0 0x5632940ac7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.550+0000 7fea06ffd640 1 -- 172.21.15.64:0/33444834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563293cfaf60 msgr2=0x5632940b8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:45.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.550+0000 7fea06ffd640 1 --2- 172.21.15.64:0/33444834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563293cfaf60 0x5632940b8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.550+0000 7fea06ffd640 1 -- 172.21.15.64:0/33444834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563293fd0a00 con 0x563293cda280 2024-09-21T04:03:45.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.550+0000 7fea06ffd640 1 --2- 172.21.15.64:0/33444834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cda280 0x5632940acd70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc00da10 tx=0x7fe9fc00ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1dfae2660c9d4de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:45.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.550+0000 7fe9e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:03:45.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.550+0000 7fe9e6ffd640 1 -- 172.21.15.64:0/33444834 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc01b640 con 0x563293cda280 2024-09-21T04:03:45.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.551+0000 7fe9e6ffd640 1 -- 172.21.15.64:0/33444834 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fe9fc01b7e0 con 0x563293cda280 2024-09-21T04:03:45.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.551+0000 7fea143ae740 1 -- 172.21.15.64:0/33444834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea0000de20 con 0x563293cda280 2024-09-21T04:03:45.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.551+0000 7fe9e6ffd640 1 -- 172.21.15.64:0/33444834 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe9fc01b9e0 con 0x563293cda280 2024-09-21T04:03:45.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.551+0000 7fea143ae740 10 client.15327 fetch_fsmap learned FSMap version 24 2024-09-21T04:03:45.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.551+0000 7fea143ae740 1 -- 172.21.15.64:0/33444834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5632940bb610 con 0x563293cda280 2024-09-21T04:03:45.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.551+0000 7fe9e6ffd640 1 -- 172.21.15.64:0/33444834 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea0000de20 con 0x563293cda280 2024-09-21T04:03:45.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.551+0000 7fe9e6ffd640 1 -- 172.21.15.64:0/33444834 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fe9fc022070 con 0x563293cda280 2024-09-21T04:03:45.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.551+0000 7fea143ae740 10 client.15327 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T04:03:45.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.551+0000 7fea143ae740 10 client.15327 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T04:03:45.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.551+0000 7fea143ae740 20 client.15327 populate_metadata read hostname 'smithi064' 2024-09-21T04:03:45.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.551+0000 7fea143ae740 1 --2- 172.21.15.64:0/33444834 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x563293cb98e0 0x5632940bf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:45.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.551+0000 7fea143ae740 4 client.15327 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:03:45.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.551+0000 7fea143ae740 1 -- 172.21.15.64:0/33444834 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56329401ab60 con 0x563293cb98e0 2024-09-21T04:03:45.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.552+0000 7fea07fff640 1 --2- 172.21.15.64:0/33444834 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x563293cb98e0 0x5632940bf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:45.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.552+0000 7fe9e6ffd640 10 client.15327 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:03:45.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.552+0000 7fea07fff640 1 --2- 172.21.15.64:0/33444834 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x563293cb98e0 0x5632940bf100 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da7a9cbfe5ab0018 server_cookie=d262fb6577450422 in_seq=0 out_seq=0 2024-09-21T04:03:45.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.553+0000 7fe9e6ffd640 1 -- 172.21.15.64:0/33444834 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56329401ab60 con 0x563293cb98e0 2024-09-21T04:03:45.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.553+0000 7fe9e6ffd640 10 client.15327 handle_command_reply: tid=0 2024-09-21T04:03:45.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.553+0000 7fea143ae740 1 client.15327 shutdown 2024-09-21T04:03:45.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.553+0000 7fea143ae740 1 -- 172.21.15.64:0/33444834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cda280 msgr2=0x5632940acd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:45.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.553+0000 7fea143ae740 1 --2- 172.21.15.64:0/33444834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cda280 0x5632940acd70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc00da10 tx=0x7fe9fc00ded0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.553+0000 7fea143ae740 1 -- 172.21.15.64:0/33444834 shutdown_connections 2024-09-21T04:03:45.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.553+0000 7fea143ae740 1 --2- 172.21.15.64:0/33444834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563293cfaf60 0x5632940b8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.553+0000 7fea143ae740 1 --2- 172.21.15.64:0/33444834 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x563293cb98e0 0x5632940bf100 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.553+0000 7fea143ae740 1 --2- 172.21.15.64:0/33444834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cda280 0x5632940acd70 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.553+0000 7fea143ae740 1 --2- 172.21.15.64:0/33444834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563293cd20a0 0x5632940ac7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.553+0000 7fe9e6ffd640 0 client.15327 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:03:45.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.553+0000 7fea143ae740 1 -- 172.21.15.64:0/33444834 >> 172.21.15.64:0/33444834 conn(0x563293cef350 msgr2=0x5632940a40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:45.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.553+0000 7fea143ae740 1 -- 172.21.15.64:0/33444834 shutdown_connections 2024-09-21T04:03:45.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.553+0000 7fea143ae740 1 -- 172.21.15.64:0/33444834 wait complete. 2024-09-21T04:03:45.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.553+0000 7fea143ae740 20 client.15327 trim_cache size 0 max 16384 2024-09-21T04:03:45.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.572+0000 7fea143ae740 1 Processor -- start 2024-09-21T04:03:45.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.572+0000 7fea143ae740 1 -- start start 2024-09-21T04:03:45.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.572+0000 7fea143ae740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563293cd20a0 0x5632940a7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:45.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.572+0000 7fea143ae740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563293cda280 0x563294099ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:45.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.572+0000 7fea143ae740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cfaf60 0x5632940b8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:45.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.572+0000 7fea143ae740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5632940bd440 con 0x563293cfaf60 2024-09-21T04:03:45.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.572+0000 7fea143ae740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5632940bdb10 con 0x563293cda280 2024-09-21T04:03:45.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.572+0000 7fea143ae740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5632940be1e0 con 0x563293cd20a0 2024-09-21T04:03:45.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.572+0000 7fea07fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cfaf60 0x5632940b8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:45.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.572+0000 7fea07fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cfaf60 0x5632940b8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52372/0 (socket says 172.21.15.64:52372) 2024-09-21T04:03:45.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.572+0000 7fea07fff640 1 -- 172.21.15.64:0/714774013 learned_addr learned my addr 172.21.15.64:0/714774013 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:45.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.572+0000 7fea07fff640 1 --2- 172.21.15.64:0/714774013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563293cd20a0 0x5632940a7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:45.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.572+0000 7fea06ffd640 1 --2- 172.21.15.64:0/714774013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563293cda280 0x563294099ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:45.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.572+0000 7fea07fff640 1 -- 172.21.15.64:0/714774013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563293cd20a0 msgr2=0x5632940a7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:45.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.572+0000 7fea07fff640 1 --2- 172.21.15.64:0/714774013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563293cd20a0 0x5632940a7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.572+0000 7fea07fff640 1 -- 172.21.15.64:0/714774013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563293cda280 msgr2=0x563294099ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:45.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.572+0000 7fea07fff640 1 --2- 172.21.15.64:0/714774013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563293cda280 0x563294099ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.572+0000 7fea07fff640 1 -- 172.21.15.64:0/714774013 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632940a3790 con 0x563293cfaf60 2024-09-21T04:03:45.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.572+0000 7fea06ffd640 1 --2- 172.21.15.64:0/714774013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563293cda280 0x563294099ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:03:45.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.573+0000 7fea07fff640 1 --2- 172.21.15.64:0/714774013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cfaf60 0x5632940b8970 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fea00008b70 tx=0x7fea0000e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81bec15c9e188ade server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:45.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.573+0000 7fe9ed7fa640 1 -- 172.21.15.64:0/714774013 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00004c00 con 0x563293cfaf60 2024-09-21T04:03:45.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.573+0000 7fe9ed7fa640 1 -- 172.21.15.64:0/714774013 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea00028040 con 0x563293cfaf60 2024-09-21T04:03:45.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.573+0000 7fe9ed7fa640 1 -- 172.21.15.64:0/714774013 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0000fb60 con 0x563293cfaf60 2024-09-21T04:03:45.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.573+0000 7fea143ae740 1 -- 172.21.15.64:0/714774013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cfaf60 msgr2=0x5632940b8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:45.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.573+0000 7fea143ae740 1 --2- 172.21.15.64:0/714774013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cfaf60 0x5632940b8970 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fea00008b70 tx=0x7fea0000e9f0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.574+0000 7fea143ae740 1 -- 172.21.15.64:0/714774013 shutdown_connections 2024-09-21T04:03:45.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.574+0000 7fea143ae740 1 --2- 172.21.15.64:0/714774013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cfaf60 0x5632940b8970 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.574+0000 7fea143ae740 1 --2- 172.21.15.64:0/714774013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563293cda280 0x563294099ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.574+0000 7fea143ae740 1 --2- 172.21.15.64:0/714774013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563293cd20a0 0x5632940a7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.574+0000 7fea143ae740 1 -- 172.21.15.64:0/714774013 >> 172.21.15.64:0/714774013 conn(0x563293cef350 msgr2=0x5632940a7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:45.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.574+0000 7fea143ae740 1 -- 172.21.15.64:0/714774013 shutdown_connections 2024-09-21T04:03:45.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.574+0000 7fea143ae740 1 -- 172.21.15.64:0/714774013 wait complete. 2024-09-21T04:03:45.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.574+0000 7fea143ae740 1 -- start start 2024-09-21T04:03:45.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.574+0000 7fea143ae740 1 Processor -- start 2024-09-21T04:03:45.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.575+0000 7fea143ae740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cd20a0 0x5632940c1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:45.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.575+0000 7fea143ae740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563293cda280 0x5632940c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:45.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.575+0000 7fea143ae740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563293cfaf60 0x5632940c6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:45.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.575+0000 7fea143ae740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5632940bef80 con 0x563293cd20a0 2024-09-21T04:03:45.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.575+0000 7fea143ae740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5632940be8b0 con 0x563293cda280 2024-09-21T04:03:45.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.575+0000 7fea143ae740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5632940bf650 con 0x563293cfaf60 2024-09-21T04:03:45.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.575+0000 7fea143ae740 10 client.0 _ll_register_callbacks cb 0x7fe9fc004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:03:45.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.575+0000 7fea07fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cd20a0 0x5632940c1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:45.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.575+0000 7fea07fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cd20a0 0x5632940c1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52378/0 (socket says 172.21.15.64:52378) 2024-09-21T04:03:45.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.575+0000 7fea07fff640 1 -- 172.21.15.64:0/3889151650 learned_addr learned my addr 172.21.15.64:0/3889151650 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:45.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.575+0000 7fea06ffd640 1 --2- 172.21.15.64:0/3889151650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563293cda280 0x5632940c1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:45.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.575+0000 7fea07fff640 1 --2- 172.21.15.64:0/3889151650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563293cfaf60 0x5632940c6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:45.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.575+0000 7fea07fff640 1 -- 172.21.15.64:0/3889151650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563293cfaf60 msgr2=0x5632940c6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:45.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.575+0000 7fea07fff640 1 --2- 172.21.15.64:0/3889151650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563293cfaf60 0x5632940c6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.575+0000 7fea07fff640 1 -- 172.21.15.64:0/3889151650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563293cda280 msgr2=0x5632940c1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:45.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.575+0000 7fea07fff640 1 --2- 172.21.15.64:0/3889151650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563293cda280 0x5632940c1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.575+0000 7fea07fff640 1 -- 172.21.15.64:0/3889151650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632940c7510 con 0x563293cd20a0 2024-09-21T04:03:45.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.575+0000 7fea07fff640 1 --2- 172.21.15.64:0/3889151650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cd20a0 0x5632940c1a40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fea00008d00 tx=0x7fea000086f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325b800060f3f899 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:45.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.575+0000 7fe9e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:03:45.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.576+0000 7fe9e6ffd640 1 -- 172.21.15.64:0/3889151650 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0000e3b0 con 0x563293cd20a0 2024-09-21T04:03:45.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.576+0000 7fe9e6ffd640 1 -- 172.21.15.64:0/3889151650 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fea0000e550 con 0x563293cd20a0 2024-09-21T04:03:45.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.576+0000 7fea143ae740 1 -- 172.21.15.64:0/3889151650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea00028040 con 0x563293cd20a0 2024-09-21T04:03:45.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.576+0000 7fe9e6ffd640 1 -- 172.21.15.64:0/3889151650 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fea0000e750 con 0x563293cd20a0 2024-09-21T04:03:45.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.576+0000 7fea143ae740 10 client.15333 fetch_fsmap learned FSMap version 24 2024-09-21T04:03:45.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.576+0000 7fea143ae740 1 -- 172.21.15.64:0/3889151650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56329412ac80 con 0x563293cd20a0 2024-09-21T04:03:45.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.576+0000 7fe9e6ffd640 1 -- 172.21.15.64:0/3889151650 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea00028040 con 0x563293cd20a0 2024-09-21T04:03:45.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.576+0000 7fe9e6ffd640 1 -- 172.21.15.64:0/3889151650 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fea00043050 con 0x563293cd20a0 2024-09-21T04:03:45.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.576+0000 7fea143ae740 10 client.15333 fetch_fsmap finished waiting for FSMap version 24 2024-09-21T04:03:45.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.576+0000 7fea143ae740 10 client.15333 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T04:03:45.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.576+0000 7fea143ae740 20 client.15333 populate_metadata read hostname 'smithi064' 2024-09-21T04:03:45.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.576+0000 7fea143ae740 1 --2- 172.21.15.64:0/3889151650 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x563293c8ec90 0x56329412dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:45.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.576+0000 7fea143ae740 4 client.15333 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T04:03:45.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.576+0000 7fea143ae740 1 -- 172.21.15.64:0/3889151650 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "status"}) -- 0x563294016610 con 0x563293c8ec90 2024-09-21T04:03:45.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.577+0000 7fea06ffd640 1 --2- 172.21.15.64:0/3889151650 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x563293c8ec90 0x56329412dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:45.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.577+0000 7fea06ffd640 1 --2- 172.21.15.64:0/3889151650 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x563293c8ec90 0x56329412dd70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0f6c6096eb0264 server_cookie=deec4d6ddbf120f3 in_seq=0 out_seq=0 2024-09-21T04:03:45.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.577+0000 7fe9e6ffd640 10 client.15333 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:03:45.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.577+0000 7fe9e6ffd640 1 -- 172.21.15.64:0/3889151650 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x563294016610 con 0x563293c8ec90 2024-09-21T04:03:45.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.578+0000 7fe9e6ffd640 10 client.15333 handle_command_reply: tid=0 2024-09-21T04:03:45.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.578+0000 7fea143ae740 1 client.15333 shutdown 2024-09-21T04:03:45.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.578+0000 7fea143ae740 1 -- 172.21.15.64:0/3889151650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cd20a0 msgr2=0x5632940c1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:45.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.578+0000 7fea143ae740 1 --2- 172.21.15.64:0/3889151650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cd20a0 0x5632940c1a40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fea00008d00 tx=0x7fea000086f0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.578+0000 7fea143ae740 1 -- 172.21.15.64:0/3889151650 shutdown_connections 2024-09-21T04:03:45.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.578+0000 7fea143ae740 1 --2- 172.21.15.64:0/3889151650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563293cfaf60 0x5632940c6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.578+0000 7fea143ae740 1 --2- 172.21.15.64:0/3889151650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563293cda280 0x5632940c1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.578+0000 7fea143ae740 1 --2- 172.21.15.64:0/3889151650 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x563293c8ec90 0x56329412dd70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.578+0000 7fea143ae740 1 --2- 172.21.15.64:0/3889151650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563293cd20a0 0x5632940c1a40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.578+0000 7fea143ae740 1 -- 172.21.15.64:0/3889151650 >> 172.21.15.64:0/3889151650 conn(0x563293cef350 msgr2=0x5632940a7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:45.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.578+0000 7fe9e6ffd640 0 client.15333 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:03:45.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.578+0000 7fea143ae740 1 -- 172.21.15.64:0/3889151650 shutdown_connections 2024-09-21T04:03:45.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.578+0000 7fea143ae740 1 -- 172.21.15.64:0/3889151650 wait complete. 2024-09-21T04:03:45.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.578+0000 7fea143ae740 20 client.15333 trim_cache size 0 max 16384 2024-09-21T04:03:45.585 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:03:45.585 INFO:teuthology.orchestra.run.smithi064.stdout: "cluster_fsid": "e7c927ba-77cc-11ef-baf5-efdc52797490", 2024-09-21T04:03:45.585 INFO:teuthology.orchestra.run.smithi064.stdout: "whoami": 0, 2024-09-21T04:03:45.586 INFO:teuthology.orchestra.run.smithi064.stdout: "id": 24710, 2024-09-21T04:03:45.586 INFO:teuthology.orchestra.run.smithi064.stdout: "want_state": "up:active", 2024-09-21T04:03:45.586 INFO:teuthology.orchestra.run.smithi064.stdout: "state": "up:active", 2024-09-21T04:03:45.586 INFO:teuthology.orchestra.run.smithi064.stdout: "fs_name": "cephfs", 2024-09-21T04:03:45.586 INFO:teuthology.orchestra.run.smithi064.stdout: "rank_uptime": 101.833852542, 2024-09-21T04:03:45.586 INFO:teuthology.orchestra.run.smithi064.stdout: "mdsmap_epoch": 24, 2024-09-21T04:03:45.586 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch": 77, 2024-09-21T04:03:45.586 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_epoch_barrier": 73, 2024-09-21T04:03:45.586 INFO:teuthology.orchestra.run.smithi064.stdout: "uptime": 112.878537804 2024-09-21T04:03:45.586 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:03:45.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.579+0000 7fea0de3a640 1 -- 172.21.15.64:0/3892948671 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe9f00b6560 msgr2=0x7fe9f00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:45.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.579+0000 7fea0de3a640 1 --2- 172.21.15.64:0/3892948671 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe9f00b6560 0x7fe9f00b8a70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8105440 tx=0x7fe9f8000bc0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.579+0000 7fea0de3a640 1 -- 172.21.15.64:0/3892948671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea081049e0 msgr2=0x7fea08102720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:45.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.579+0000 7fea0de3a640 1 --2- 172.21.15.64:0/3892948671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea081049e0 0x7fea08102720 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8032fd0 tx=0x7fe9f8004270 comp rx=0 tx=0).stop 2024-09-21T04:03:45.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.579+0000 7fea0de3a640 1 -- 172.21.15.64:0/3892948671 shutdown_connections 2024-09-21T04:03:45.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.579+0000 7fea0de3a640 1 --2- 172.21.15.64:0/3892948671 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe9f00b6560 0x7fe9f00b8a70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.579+0000 7fea0de3a640 1 --2- 172.21.15.64:0/3892948671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea0810de40 0x7fea081031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.579+0000 7fea0de3a640 1 --2- 172.21.15.64:0/3892948671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea08105330 0x7fea08102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.579+0000 7fea0de3a640 1 --2- 172.21.15.64:0/3892948671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea081049e0 0x7fea08102720 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:45.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.579+0000 7fea0de3a640 1 -- 172.21.15.64:0/3892948671 >> 172.21.15.64:0/3892948671 conn(0x7fea08079750 msgr2=0x7fea080ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:45.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.579+0000 7fea0de3a640 1 -- 172.21.15.64:0/3892948671 shutdown_connections 2024-09-21T04:03:45.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:45.579+0000 7fea0de3a640 1 -- 172.21.15.64:0/3892948671 wait complete. 2024-09-21T04:03:45.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:45 smithi064 ceph-mon[31180]: pgmap v313: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:03:45.855 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:45 smithi131 ceph-mon[35193]: pgmap v313: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:03:46.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:45 smithi119 ceph-mon[34811]: pgmap v313: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:03:46.178 INFO:teuthology.run_tasks:Running task kclient... 2024-09-21T04:03:46.203 INFO:tasks.kclient:Mounting kernel clients... 2024-09-21T04:03:46.203 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-21T04:03:46.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:03:46.203 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-09-21T04:03:46.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:03:46.221 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2024-09-21T04:03:46.291 INFO:teuthology.orchestra.run.smithi064.stderr:Cannot find device "ceph-brx" 2024-09-21T04:03:46.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:03:46.293 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-21T04:03:46.293 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T04:03:46.293 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T04:03:46.294 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T04:03:46.594 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:47.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.016+0000 7f0a82d27640 1 -- 172.21.15.64:0/2247267787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a7c101fc0 msgr2=0x7f0a7c1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:47.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.016+0000 7f0a82d27640 1 --2- 172.21.15.64:0/2247267787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a7c101fc0 0x7f0a7c1023d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009900 tx=0x7f0a7002a410 comp rx=0 tx=0).stop 2024-09-21T04:03:47.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.016+0000 7f0a82d27640 1 -- 172.21.15.64:0/2247267787 shutdown_connections 2024-09-21T04:03:47.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.016+0000 7f0a82d27640 1 --2- 172.21.15.64:0/2247267787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a7c109e50 0x7f0a7c110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:47.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.016+0000 7f0a82d27640 1 --2- 172.21.15.64:0/2247267787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a7c109500 0x7f0a7c109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:47.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.016+0000 7f0a82d27640 1 --2- 172.21.15.64:0/2247267787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a7c101fc0 0x7f0a7c1023d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:47.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.016+0000 7f0a82d27640 1 -- 172.21.15.64:0/2247267787 >> 172.21.15.64:0/2247267787 conn(0x7f0a7c079f90 msgr2=0x7f0a7c07a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:47.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.016+0000 7f0a82d27640 1 -- 172.21.15.64:0/2247267787 shutdown_connections 2024-09-21T04:03:47.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.016+0000 7f0a82d27640 1 -- 172.21.15.64:0/2247267787 wait complete. 2024-09-21T04:03:47.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.017+0000 7f0a82d27640 1 Processor -- start 2024-09-21T04:03:47.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.017+0000 7f0a82d27640 1 -- start start 2024-09-21T04:03:47.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.017+0000 7f0a82d27640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a7c101fc0 0x7f0a7c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:47.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.017+0000 7f0a82d27640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a7c109500 0x7f0a7c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:47.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.017+0000 7f0a82d27640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a7c109e50 0x7f0a7c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:47.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.017+0000 7f0a82d27640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c113b10 con 0x7f0a7c109500 2024-09-21T04:03:47.021 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.017+0000 7f0a82d27640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c113990 con 0x7f0a7c109e50 2024-09-21T04:03:47.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.017+0000 7f0a82d27640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c113c90 con 0x7f0a7c101fc0 2024-09-21T04:03:47.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.017+0000 7f0a7bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a7c109500 0x7f0a7c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:47.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.017+0000 7f0a7bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a7c109500 0x7f0a7c19aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52394/0 (socket says 172.21.15.64:52394) 2024-09-21T04:03:47.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.017+0000 7f0a7bfff640 1 -- 172.21.15.64:0/1976918622 learned_addr learned my addr 172.21.15.64:0/1976918622 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:47.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.018+0000 7f0a812c4640 1 --2- 172.21.15.64:0/1976918622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a7c109e50 0x7f0a7c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:47.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.018+0000 7f0a80ac3640 1 --2- 172.21.15.64:0/1976918622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a7c101fc0 0x7f0a7c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:47.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.018+0000 7f0a7bfff640 1 -- 172.21.15.64:0/1976918622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a7c101fc0 msgr2=0x7f0a7c19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:47.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.018+0000 7f0a7bfff640 1 --2- 172.21.15.64:0/1976918622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a7c101fc0 0x7f0a7c19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:47.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.018+0000 7f0a7bfff640 1 -- 172.21.15.64:0/1976918622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a7c109e50 msgr2=0x7f0a7c19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:47.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.018+0000 7f0a7bfff640 1 --2- 172.21.15.64:0/1976918622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a7c109e50 0x7f0a7c19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:47.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.018+0000 7f0a7bfff640 1 -- 172.21.15.64:0/1976918622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7c1a03b0 con 0x7f0a7c109500 2024-09-21T04:03:47.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.018+0000 7f0a7bfff640 1 --2- 172.21.15.64:0/1976918622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a7c109500 0x7f0a7c19aaa0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800d740 tx=0x7f0a6800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:47.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.018+0000 7f0a79ffb640 1 -- 172.21.15.64:0/1976918622 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a68017070 con 0x7f0a7c109500 2024-09-21T04:03:47.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.018+0000 7f0a82d27640 1 -- 172.21.15.64:0/1976918622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7c1a06a0 con 0x7f0a7c109500 2024-09-21T04:03:47.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.018+0000 7f0a79ffb640 1 -- 172.21.15.64:0/1976918622 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a6800de50 con 0x7f0a7c109500 2024-09-21T04:03:47.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.018+0000 7f0a79ffb640 1 -- 172.21.15.64:0/1976918622 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6800f620 con 0x7f0a7c109500 2024-09-21T04:03:47.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.018+0000 7f0a82d27640 1 -- 172.21.15.64:0/1976918622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a7c1a6ec0 con 0x7f0a7c109500 2024-09-21T04:03:47.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.020+0000 7f0a82d27640 1 -- 172.21.15.64:0/1976918622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a7c06b910 con 0x7f0a7c109500 2024-09-21T04:03:47.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.021+0000 7f0a79ffb640 1 -- 172.21.15.64:0/1976918622 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f0a6801a070 con 0x7f0a7c109500 2024-09-21T04:03:47.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.022+0000 7f0a79ffb640 1 --2- 172.21.15.64:0/1976918622 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0a6c0b6590 0x7f0a6c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:47.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.022+0000 7f0a80ac3640 1 --2- 172.21.15.64:0/1976918622 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0a6c0b6590 0x7f0a6c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:47.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.022+0000 7f0a79ffb640 1 -- 172.21.15.64:0/1976918622 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f0a680e6e90 con 0x7f0a7c109500 2024-09-21T04:03:47.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.023+0000 7f0a80ac3640 1 --2- 172.21.15.64:0/1976918622 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0a6c0b6590 0x7f0a6c0b8aa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0a70031440 tx=0x7f0a70005d50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:47.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.025+0000 7f0a79ffb640 1 -- 172.21.15.64:0/1976918622 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0a680ad5d0 con 0x7f0a7c109500 2024-09-21T04:03:47.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.253+0000 7f0a82d27640 1 -- 172.21.15.64:0/1976918622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0a7c1023d0 con 0x7f0a7c109500 2024-09-21T04:03:47.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.253+0000 7f0a79ffb640 1 -- 172.21.15.64:0/1976918622 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f0a7c1023d0 con 0x7f0a7c109500 2024-09-21T04:03:47.256 INFO:teuthology.orchestra.run.smithi064.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T04:03:47.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.255+0000 7f0a82d27640 1 -- 172.21.15.64:0/1976918622 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0a6c0b6590 msgr2=0x7f0a6c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:47.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.255+0000 7f0a82d27640 1 --2- 172.21.15.64:0/1976918622 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0a6c0b6590 0x7f0a6c0b8aa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0a70031440 tx=0x7f0a70005d50 comp rx=0 tx=0).stop 2024-09-21T04:03:47.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.255+0000 7f0a82d27640 1 -- 172.21.15.64:0/1976918622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a7c109500 msgr2=0x7f0a7c19aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:47.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.255+0000 7f0a82d27640 1 --2- 172.21.15.64:0/1976918622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a7c109500 0x7f0a7c19aaa0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800d740 tx=0x7f0a6800dc00 comp rx=0 tx=0).stop 2024-09-21T04:03:47.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.255+0000 7f0a82d27640 1 -- 172.21.15.64:0/1976918622 shutdown_connections 2024-09-21T04:03:47.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.255+0000 7f0a82d27640 1 --2- 172.21.15.64:0/1976918622 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0a6c0b6590 0x7f0a6c0b8aa0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:47.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.255+0000 7f0a82d27640 1 --2- 172.21.15.64:0/1976918622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a7c109e50 0x7f0a7c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:47.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.256+0000 7f0a82d27640 1 --2- 172.21.15.64:0/1976918622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a7c109500 0x7f0a7c19aaa0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:47.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.256+0000 7f0a82d27640 1 --2- 172.21.15.64:0/1976918622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0a7c101fc0 0x7f0a7c19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:47.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.256+0000 7f0a82d27640 1 -- 172.21.15.64:0/1976918622 >> 172.21.15.64:0/1976918622 conn(0x7f0a7c079f90 msgr2=0x7f0a7c107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:47.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.256+0000 7f0a82d27640 1 -- 172.21.15.64:0/1976918622 shutdown_connections 2024-09-21T04:03:47.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:47.256+0000 7f0a82d27640 1 -- 172.21.15.64:0/1976918622 wait complete. 2024-09-21T04:03:48.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:47 smithi119 ceph-mon[34811]: pgmap v314: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T04:03:48.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:47 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1976918622' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T04:03:48.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:47 smithi064 ceph-mon[31180]: pgmap v314: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T04:03:48.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:47 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1976918622' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T04:03:48.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:47 smithi131 ceph-mon[35193]: pgmap v314: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-21T04:03:48.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:47 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1976918622' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T04:03:48.062 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T04:03:48.062 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T04:03:48.063 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi064.front.sepia.ceph.com 2024-09-21T04:03:48.063 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T04:03:48.063 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T04:03:48.063 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T04:03:48.063 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T04:03:48.063 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T04:03:48.063 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T04:03:48.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:03:48.063 DEBUG:teuthology.orchestra.run.smithi064:> ip addr 2024-09-21T04:03:48.081 INFO:teuthology.orchestra.run.smithi064.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T04:03:48.081 INFO:teuthology.orchestra.run.smithi064.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T04:03:48.082 INFO:teuthology.orchestra.run.smithi064.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T04:03:48.082 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-09-21T04:03:48.082 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 ::1/128 scope host 2024-09-21T04:03:48.082 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-09-21T04:03:48.082 INFO:teuthology.orchestra.run.smithi064.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T04:03:48.082 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:78 brd ff:ff:ff:ff:ff:ff 2024-09-21T04:03:48.082 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f0 2024-09-21T04:03:48.082 INFO:teuthology.orchestra.run.smithi064.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T04:03:48.082 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:a9:a3:79 brd ff:ff:ff:ff:ff:ff 2024-09-21T04:03:48.082 INFO:teuthology.orchestra.run.smithi064.stdout: altname enp7s0f1 2024-09-21T04:03:48.082 INFO:teuthology.orchestra.run.smithi064.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T04:03:48.082 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:de brd ff:ff:ff:ff:ff:ff 2024-09-21T04:03:48.082 INFO:teuthology.orchestra.run.smithi064.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T04:03:48.083 INFO:teuthology.orchestra.run.smithi064.stdout: link/ether 0c:c4:7a:b6:f8:df brd ff:ff:ff:ff:ff:ff 2024-09-21T04:03:48.083 INFO:teuthology.orchestra.run.smithi064.stdout: inet 172.21.15.64/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T04:03:48.083 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft 42148sec preferred_lft 42148sec 2024-09-21T04:03:48.083 INFO:teuthology.orchestra.run.smithi064.stdout: inet6 fe80::ec4:7aff:feb6:f8df/64 scope link noprefixroute 2024-09-21T04:03:48.083 INFO:teuthology.orchestra.run.smithi064.stdout: valid_lft forever preferred_lft forever 2024-09-21T04:03:48.083 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T04:03:48.083 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T04:03:48.083 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-09-21T04:03:48.084 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add name ceph-brx type bridge 2024-09-21T04:03:48.084 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr flush dev ceph-brx 2024-09-21T04:03:48.084 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set ceph-brx up 2024-09-21T04:03:48.084 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T04:03:48.084 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-09-21T04:03:48.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:03:48.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:03:48.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:03:48.304 DEBUG:teuthology.orchestra.run.smithi064:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T04:03:48.331 INFO:teuthology.orchestra.run.smithi064.stdout:1 2024-09-21T04:03:48.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:03:48.333 DEBUG:teuthology.orchestra.run.smithi064:> route 2024-09-21T04:03:48.406 INFO:teuthology.orchestra.run.smithi064.stdout:Kernel IP routing table 2024-09-21T04:03:48.406 INFO:teuthology.orchestra.run.smithi064.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T04:03:48.406 INFO:teuthology.orchestra.run.smithi064.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T04:03:48.406 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T04:03:48.406 INFO:teuthology.orchestra.run.smithi064.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T04:03:48.407 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T04:03:48.407 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-09-21T04:03:48.407 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T04:03:48.407 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T04:03:48.407 DEBUG:teuthology.orchestra.run.smithi064:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T04:03:48.407 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-09-21T04:03:48.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:03:48.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:03:48.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:03:48.738 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-09-21T04:03:48.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:03:48.755 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list-id 2024-09-21T04:03:48.811 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T04:03:48.811 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-09-21T04:03:48.811 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T04:03:48.811 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T04:03:48.811 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-09-21T04:03:48.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:03:48.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:03:48.932 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T04:03:48.932 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T04:03:48.932 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-09-21T04:03:48.932 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T04:03:48.932 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T04:03:48.932 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T04:03:48.932 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T04:03:48.932 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T04:03:48.932 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-09-21T04:03:48.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:03:49.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:03:49.061 DEBUG:teuthology.orchestra.run.smithi064:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T04:03:49.061 DEBUG:teuthology.orchestra.run.smithi064:> set -e 2024-09-21T04:03:49.061 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set brx.0 up 2024-09-21T04:03:49.061 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T04:03:49.062 DEBUG:teuthology.orchestra.run.smithi064:> ') 2024-09-21T04:03:49.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:03:49.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:03:49.142 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T04:03:49.142 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T04:03:49.158 INFO:teuthology.orchestra.run.smithi064.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T04:03:49.158 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T04:03:49.158 DEBUG:teuthology.orchestra.run.smithi064:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T04:03:49.213 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-21T04:03:49.213 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-21T04:03:49.214 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/071d8841-4290-45c9-ae90-a63e004e386b 2024-09-21T04:03:49.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:03:49.214 DEBUG:teuthology.orchestra.run.smithi064:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/071d8841-4290-45c9-ae90-a63e004e386b /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-21T04:03:49.335 INFO:teuthology.orchestra.run.smithi064.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-21T04:03:49.335 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-21T04:03:49.335 INFO:teuthology.orchestra.run.smithi064.stdout:invalid new device string format 2024-09-21T04:03:49.474 INFO:teuthology.orchestra.run.smithi064.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-21T04:03:49.474 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-21T04:03:49.474 INFO:teuthology.orchestra.run.smithi064.stdout:invalid new device string format 2024-09-21T04:03:49.474 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: resolved to: "172.21.15.64:3300,172.21.15.119:3300,172.21.15.131:3300" 2024-09-21T04:03:49.474 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.64:3300,172.21.15.119:3300,172.21.15.131:3300:/volumes/qa/sv_1/071d8841-4290-45c9-ae90-a63e004e386b 2024-09-21T04:03:49.474 INFO:teuthology.orchestra.run.smithi064.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=e7c927ba-77cc-11ef-baf5-efdc52797490" will pass to kernel 2024-09-21T04:03:49.475 INFO:teuthology.orchestra.run.smithi064.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T04:03:49.475 INFO:teuthology.orchestra.run.smithi064.stdout: You just mounted a file system that supports labels which does not 2024-09-21T04:03:49.475 INFO:teuthology.orchestra.run.smithi064.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T04:03:49.476 INFO:teuthology.orchestra.run.smithi064.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T04:03:49.476 INFO:teuthology.orchestra.run.smithi064.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T04:03:49.477 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T04:03:49.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:03:49.477 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T04:03:49.508 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T04:03:49.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T04:03:49.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:03:49.608 INFO:teuthology.orchestra.run.smithi064.stdout:client15342 2024-09-21T04:03:49.609 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-21T04:03:49.609 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T04:03:49.626 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T04:03:49.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T04:03:49.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:03:49.725 INFO:teuthology.orchestra.run.smithi064.stdout:client15342 2024-09-21T04:03:49.725 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T04:03:49.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T04:03:49.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:03:49.783 INFO:teuthology.orchestra.run.smithi064.stdout:client15342 2024-09-21T04:03:49.783 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490.client15342/status') 2024-09-21T04:03:49.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T04:03:49.844 INFO:teuthology.orchestra.run.smithi064.stdout:instance: client.15342 (3)192.168.0.1:0/907018364 2024-09-21T04:03:49.844 INFO:teuthology.orchestra.run.smithi064.stdout:blocklisted: false 2024-09-21T04:03:49.844 INFO:teuthology.orchestra.run.smithi064.stderr:0+1 records in 2024-09-21T04:03:49.844 INFO:teuthology.orchestra.run.smithi064.stderr:0+1 records out 2024-09-21T04:03:49.844 INFO:teuthology.orchestra.run.smithi064.stderr:69 bytes copied, 2.5893e-05 s, 2.7 MB/s 2024-09-21T04:03:49.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T04:03:49.851 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T04:03:49.862 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T04:03:49.862 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-09-21T04:03:49.863 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-21T04:03:49.885 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:49 smithi064 ceph-mon[31180]: pgmap v315: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T04:03:49.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.974+0000 7f907bc93640 1 -- 172.21.15.64:0/4142459050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f907410ddb0 msgr2=0x7f9074110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:49.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.974+0000 7f907bc93640 1 --2- 172.21.15.64:0/4142459050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f907410ddb0 0x7f9074110290 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f906c009ca0 tx=0x7f906c02a8c0 comp rx=0 tx=0).stop 2024-09-21T04:03:49.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.974+0000 7f907bc93640 1 -- 172.21.15.64:0/4142459050 shutdown_connections 2024-09-21T04:03:49.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.974+0000 7f907bc93640 1 --2- 172.21.15.64:0/4142459050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f907410ddb0 0x7f9074110290 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:49.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.974+0000 7f907bc93640 1 --2- 172.21.15.64:0/4142459050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90741052e0 0x7f907410d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:49.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.974+0000 7f907bc93640 1 --2- 172.21.15.64:0/4142459050 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9074104990 0x7f9074104da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:49.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.974+0000 7f907bc93640 1 -- 172.21.15.64:0/4142459050 >> 172.21.15.64:0/4142459050 conn(0x7f90740fe3e0 msgr2=0x7f9074100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:49.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.974+0000 7f907bc93640 1 -- 172.21.15.64:0/4142459050 shutdown_connections 2024-09-21T04:03:49.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.974+0000 7f907bc93640 1 -- 172.21.15.64:0/4142459050 wait complete. 2024-09-21T04:03:49.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.975+0000 7f907bc93640 1 Processor -- start 2024-09-21T04:03:49.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.975+0000 7f907bc93640 1 -- start start 2024-09-21T04:03:49.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.975+0000 7f907bc93640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9074104990 0x7f90741026c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:49.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.975+0000 7f907bc93640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90741052e0 0x7f9074102c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:49.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.975+0000 7f907bc93640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f907410ddb0 0x7f9074103140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:49.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.975+0000 7f907bc93640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9074113a00 con 0x7f907410ddb0 2024-09-21T04:03:49.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.975+0000 7f907bc93640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9074113880 con 0x7f90741052e0 2024-09-21T04:03:49.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.975+0000 7f907bc93640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9074113b80 con 0x7f9074104990 2024-09-21T04:03:49.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.976+0000 7f907a230640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f907410ddb0 0x7f9074103140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:49.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.976+0000 7f907a230640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f907410ddb0 0x7f9074103140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52404/0 (socket says 172.21.15.64:52404) 2024-09-21T04:03:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.976+0000 7f907a230640 1 -- 172.21.15.64:0/2794089605 learned_addr learned my addr 172.21.15.64:0/2794089605 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.976+0000 7f9079a2f640 1 --2- 172.21.15.64:0/2794089605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9074104990 0x7f90741026c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.976+0000 7f907922e640 1 --2- 172.21.15.64:0/2794089605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90741052e0 0x7f9074102c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.976+0000 7f907a230640 1 -- 172.21.15.64:0/2794089605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9074104990 msgr2=0x7f90741026c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.976+0000 7f907a230640 1 --2- 172.21.15.64:0/2794089605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9074104990 0x7f90741026c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.976+0000 7f907a230640 1 -- 172.21.15.64:0/2794089605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90741052e0 msgr2=0x7f9074102c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.976+0000 7f907a230640 1 --2- 172.21.15.64:0/2794089605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90741052e0 0x7f9074102c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.976+0000 7f907a230640 1 -- 172.21.15.64:0/2794089605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90741a8670 con 0x7f907410ddb0 2024-09-21T04:03:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.976+0000 7f907922e640 1 --2- 172.21.15.64:0/2794089605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90741052e0 0x7f9074102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:03:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.976+0000 7f907a230640 1 --2- 172.21.15.64:0/2794089605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f907410ddb0 0x7f9074103140 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f906c005cc0 tx=0x7f906c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.976+0000 7f906affd640 1 -- 172.21.15.64:0/2794089605 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c003bf0 con 0x7f907410ddb0 2024-09-21T04:03:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.976+0000 7f906affd640 1 -- 172.21.15.64:0/2794089605 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f906c003d90 con 0x7f907410ddb0 2024-09-21T04:03:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.976+0000 7f907bc93640 1 -- 172.21.15.64:0/2794089605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90741a8900 con 0x7f907410ddb0 2024-09-21T04:03:49.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.976+0000 7f906affd640 1 -- 172.21.15.64:0/2794089605 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c02fbc0 con 0x7f907410ddb0 2024-09-21T04:03:49.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.976+0000 7f907bc93640 1 -- 172.21.15.64:0/2794089605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90741a8ea0 con 0x7f907410ddb0 2024-09-21T04:03:49.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.977+0000 7f907bc93640 1 -- 172.21.15.64:0/2794089605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9074104da0 con 0x7f907410ddb0 2024-09-21T04:03:49.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.979+0000 7f906affd640 1 -- 172.21.15.64:0/2794089605 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f906c02fd60 con 0x7f907410ddb0 2024-09-21T04:03:49.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.980+0000 7f906affd640 1 --2- 172.21.15.64:0/2794089605 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f90600b6610 0x7f90600b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:49.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.980+0000 7f9079a2f640 1 --2- 172.21.15.64:0/2794089605 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f90600b6610 0x7f90600b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:49.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.980+0000 7f906affd640 1 -- 172.21.15.64:0/2794089605 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f906c108920 con 0x7f907410ddb0 2024-09-21T04:03:49.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.980+0000 7f9079a2f640 1 --2- 172.21.15.64:0/2794089605 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f90600b6610 0x7f90600b8b20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9074104110 tx=0x7f907000b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:49.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:49.983+0000 7f906affd640 1 -- 172.21.15.64:0/2794089605 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f906c0cf060 con 0x7f907410ddb0 2024-09-21T04:03:50.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:49 smithi119 ceph-mon[34811]: pgmap v315: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T04:03:50.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:49 smithi131 ceph-mon[35193]: pgmap v315: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T04:03:50.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.210+0000 7f907bc93640 1 -- 172.21.15.64:0/2794089605 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f90741039c0 con 0x7f907410ddb0 2024-09-21T04:03:50.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.608+0000 7f906affd640 1 -- 172.21.15.64:0/2794089605 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f90741039c0 con 0x7f907410ddb0 2024-09-21T04:03:50.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.610+0000 7f907bc93640 1 -- 172.21.15.64:0/2794089605 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f90600b6610 msgr2=0x7f90600b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:50.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.610+0000 7f907bc93640 1 --2- 172.21.15.64:0/2794089605 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f90600b6610 0x7f90600b8b20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9074104110 tx=0x7f907000b410 comp rx=0 tx=0).stop 2024-09-21T04:03:50.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.610+0000 7f907bc93640 1 -- 172.21.15.64:0/2794089605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f907410ddb0 msgr2=0x7f9074103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:50.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.610+0000 7f907bc93640 1 --2- 172.21.15.64:0/2794089605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f907410ddb0 0x7f9074103140 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f906c005cc0 tx=0x7f906c02f380 comp rx=0 tx=0).stop 2024-09-21T04:03:50.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.610+0000 7f907bc93640 1 -- 172.21.15.64:0/2794089605 shutdown_connections 2024-09-21T04:03:50.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.610+0000 7f907bc93640 1 --2- 172.21.15.64:0/2794089605 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f90600b6610 0x7f90600b8b20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:50.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.610+0000 7f907bc93640 1 --2- 172.21.15.64:0/2794089605 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f907410ddb0 0x7f9074103140 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:50.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.610+0000 7f907bc93640 1 --2- 172.21.15.64:0/2794089605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90741052e0 0x7f9074102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:50.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.610+0000 7f907bc93640 1 --2- 172.21.15.64:0/2794089605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9074104990 0x7f90741026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:50.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.610+0000 7f907bc93640 1 -- 172.21.15.64:0/2794089605 >> 172.21.15.64:0/2794089605 conn(0x7f90740fe3e0 msgr2=0x7f9074100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:50.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.610+0000 7f907bc93640 1 -- 172.21.15.64:0/2794089605 shutdown_connections 2024-09-21T04:03:50.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.610+0000 7f907bc93640 1 -- 172.21.15.64:0/2794089605 wait complete. 2024-09-21T04:03:50.626 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-21T04:03:50.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.781+0000 7fafffdf1640 1 -- 172.21.15.64:0/2666793011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faff8102e90 msgr2=0x7faff810fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:50.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.781+0000 7fafffdf1640 1 --2- 172.21.15.64:0/2666793011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faff8102e90 0x7faff810fbb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fafe8009900 tx=0x7fafe802a480 comp rx=0 tx=0).stop 2024-09-21T04:03:50.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.781+0000 7fafffdf1640 1 -- 172.21.15.64:0/2666793011 shutdown_connections 2024-09-21T04:03:50.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.781+0000 7fafffdf1640 1 --2- 172.21.15.64:0/2666793011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faff81100f0 0x7faff8112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:50.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.781+0000 7fafffdf1640 1 --2- 172.21.15.64:0/2666793011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faff8102e90 0x7faff810fbb0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:50.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.781+0000 7fafffdf1640 1 --2- 172.21.15.64:0/2666793011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faff8102540 0x7faff8102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:50.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.781+0000 7fafffdf1640 1 -- 172.21.15.64:0/2666793011 >> 172.21.15.64:0/2666793011 conn(0x7faff80fe3e0 msgr2=0x7faff8100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:50.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.781+0000 7fafffdf1640 1 -- 172.21.15.64:0/2666793011 shutdown_connections 2024-09-21T04:03:50.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.781+0000 7fafffdf1640 1 -- 172.21.15.64:0/2666793011 wait complete. 2024-09-21T04:03:50.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.781+0000 7fafffdf1640 1 Processor -- start 2024-09-21T04:03:50.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.781+0000 7fafffdf1640 1 -- start start 2024-09-21T04:03:50.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.782+0000 7fafffdf1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faff8102540 0x7faff819c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:50.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.782+0000 7fafffdf1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faff8102e90 0x7faff819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:50.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.782+0000 7fafffdf1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faff81100f0 0x7faff81a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:50.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.782+0000 7fafffdf1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faff8115be0 con 0x7faff81100f0 2024-09-21T04:03:50.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.782+0000 7fafffdf1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faff8115a60 con 0x7faff8102e90 2024-09-21T04:03:50.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.782+0000 7fafffdf1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faff8115d60 con 0x7faff8102540 2024-09-21T04:03:50.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.782+0000 7faffe38e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faff81100f0 0x7faff81a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:50.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.782+0000 7faffe38e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faff81100f0 0x7faff81a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52432/0 (socket says 172.21.15.64:52432) 2024-09-21T04:03:50.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.782+0000 7faffe38e640 1 -- 172.21.15.64:0/1702755128 learned_addr learned my addr 172.21.15.64:0/1702755128 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:50.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.782+0000 7faffd38c640 1 --2- 172.21.15.64:0/1702755128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faff8102e90 0x7faff819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:50.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.782+0000 7faffdb8d640 1 --2- 172.21.15.64:0/1702755128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faff8102540 0x7faff819c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:50.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.782+0000 7faffe38e640 1 -- 172.21.15.64:0/1702755128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faff8102540 msgr2=0x7faff819c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:50.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.782+0000 7faffe38e640 1 --2- 172.21.15.64:0/1702755128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faff8102540 0x7faff819c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:50.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.782+0000 7faffe38e640 1 -- 172.21.15.64:0/1702755128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faff8102e90 msgr2=0x7faff819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:50.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.782+0000 7faffe38e640 1 --2- 172.21.15.64:0/1702755128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faff8102e90 0x7faff819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:50.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.782+0000 7faffe38e640 1 -- 172.21.15.64:0/1702755128 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff81a2570 con 0x7faff81100f0 2024-09-21T04:03:50.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.782+0000 7faffe38e640 1 --2- 172.21.15.64:0/1702755128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faff81100f0 0x7faff81a1e70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7faff000eaa0 tx=0x7faff000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:50.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.783+0000 7fafeeffd640 1 -- 172.21.15.64:0/1702755128 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff000cdb0 con 0x7faff81100f0 2024-09-21T04:03:50.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.783+0000 7fafeeffd640 1 -- 172.21.15.64:0/1702755128 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faff0005020 con 0x7faff81100f0 2024-09-21T04:03:50.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.783+0000 7fafffdf1640 1 -- 172.21.15.64:0/1702755128 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff81a2860 con 0x7faff81100f0 2024-09-21T04:03:50.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.783+0000 7fafeeffd640 1 -- 172.21.15.64:0/1702755128 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff00116d0 con 0x7faff81100f0 2024-09-21T04:03:50.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.783+0000 7fafffdf1640 1 -- 172.21.15.64:0/1702755128 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff81a8ff0 con 0x7faff81100f0 2024-09-21T04:03:50.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.784+0000 7fafffdf1640 1 -- 172.21.15.64:0/1702755128 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faff8102950 con 0x7faff81100f0 2024-09-21T04:03:50.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.785+0000 7fafeeffd640 1 -- 172.21.15.64:0/1702755128 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7faff00051d0 con 0x7faff81100f0 2024-09-21T04:03:50.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.786+0000 7fafeeffd640 1 --2- 172.21.15.64:0/1702755128 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fafe40b65e0 0x7fafe40b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:50.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.786+0000 7faffdb8d640 1 --2- 172.21.15.64:0/1702755128 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fafe40b65e0 0x7fafe40b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:50.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.787+0000 7fafeeffd640 1 -- 172.21.15.64:0/1702755128 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7faff00e75a0 con 0x7faff81100f0 2024-09-21T04:03:50.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.787+0000 7faffdb8d640 1 --2- 172.21.15.64:0/1702755128 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fafe40b65e0 0x7fafe40b8af0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7faff40058c0 tx=0x7faff400a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:50.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:50.791+0000 7fafeeffd640 1 -- 172.21.15.64:0/1702755128 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faff0016030 con 0x7faff81100f0 2024-09-21T04:03:51.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:50 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2794089605' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-21T04:03:51.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.006+0000 7fafffdf1640 1 -- 172.21.15.64:0/1702755128 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7faff8109270 con 0x7fafe40b65e0 2024-09-21T04:03:51.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:50 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2794089605' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-21T04:03:51.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.030+0000 7fafeeffd640 1 -- 172.21.15.64:0/1702755128 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7faff8109270 con 0x7fafe40b65e0 2024-09-21T04:03:51.034 INFO:teuthology.orchestra.run.smithi064.stdout:{} 2024-09-21T04:03:51.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.033+0000 7fafffdf1640 1 -- 172.21.15.64:0/1702755128 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fafe40b65e0 msgr2=0x7fafe40b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:51.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.033+0000 7fafffdf1640 1 --2- 172.21.15.64:0/1702755128 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fafe40b65e0 0x7fafe40b8af0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7faff40058c0 tx=0x7faff400a2d0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.033+0000 7fafffdf1640 1 -- 172.21.15.64:0/1702755128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faff81100f0 msgr2=0x7faff81a1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:51.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.033+0000 7fafffdf1640 1 --2- 172.21.15.64:0/1702755128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faff81100f0 0x7faff81a1e70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7faff000eaa0 tx=0x7faff000ef60 comp rx=0 tx=0).stop 2024-09-21T04:03:51.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.033+0000 7fafffdf1640 1 -- 172.21.15.64:0/1702755128 shutdown_connections 2024-09-21T04:03:51.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.033+0000 7fafffdf1640 1 --2- 172.21.15.64:0/1702755128 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fafe40b65e0 0x7fafe40b8af0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.033+0000 7fafffdf1640 1 --2- 172.21.15.64:0/1702755128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faff81100f0 0x7faff81a1e70 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.033+0000 7fafffdf1640 1 --2- 172.21.15.64:0/1702755128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faff8102e90 0x7faff819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.033+0000 7fafffdf1640 1 --2- 172.21.15.64:0/1702755128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faff8102540 0x7faff819c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.033+0000 7fafffdf1640 1 -- 172.21.15.64:0/1702755128 >> 172.21.15.64:0/1702755128 conn(0x7faff80fe3e0 msgr2=0x7faff8108b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:51.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.033+0000 7fafffdf1640 1 -- 172.21.15.64:0/1702755128 shutdown_connections 2024-09-21T04:03:51.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.033+0000 7fafffdf1640 1 -- 172.21.15.64:0/1702755128 wait complete. 2024-09-21T04:03:51.047 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-21T04:03:51.059 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-21T04:03:51.070 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T04:03:51.071 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:50 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2794089605' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-09-21T04:03:51.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.181+0000 7f0ddb1ad640 1 -- 172.21.15.64:0/223677717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dd4108c30 msgr2=0x7f0dd410f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:51.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.181+0000 7f0ddb1ad640 1 --2- 172.21.15.64:0/223677717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dd4108c30 0x7f0dd410f460 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc009ca0 tx=0x7f0dcc02a8c0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.181+0000 7f0ddb1ad640 1 -- 172.21.15.64:0/223677717 shutdown_connections 2024-09-21T04:03:51.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.181+0000 7f0ddb1ad640 1 --2- 172.21.15.64:0/223677717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dd4108c30 0x7f0dd410f460 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.181+0000 7f0ddb1ad640 1 --2- 172.21.15.64:0/223677717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd41082e0 0x7f0dd41086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.181+0000 7f0ddb1ad640 1 --2- 172.21.15.64:0/223677717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dd41029b0 0x7f0dd4102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.181+0000 7f0ddb1ad640 1 -- 172.21.15.64:0/223677717 >> 172.21.15.64:0/223677717 conn(0x7f0dd40fe3b0 msgr2=0x7f0dd4100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:51.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.182+0000 7f0ddb1ad640 1 -- 172.21.15.64:0/223677717 shutdown_connections 2024-09-21T04:03:51.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.182+0000 7f0ddb1ad640 1 -- 172.21.15.64:0/223677717 wait complete. 2024-09-21T04:03:51.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.182+0000 7f0ddb1ad640 1 Processor -- start 2024-09-21T04:03:51.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.182+0000 7f0ddb1ad640 1 -- start start 2024-09-21T04:03:51.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.182+0000 7f0ddb1ad640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dd41029b0 0x7f0dd41993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:51.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.182+0000 7f0ddb1ad640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd41082e0 0x7f0dd4199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:51.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.182+0000 7f0ddb1ad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dd4108c30 0x7f0dd419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:51.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.182+0000 7f0ddb1ad640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd41129c0 con 0x7f0dd4108c30 2024-09-21T04:03:51.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.182+0000 7f0ddb1ad640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd4112840 con 0x7f0dd41029b0 2024-09-21T04:03:51.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.182+0000 7f0ddb1ad640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd4112b40 con 0x7f0dd41082e0 2024-09-21T04:03:51.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.183+0000 7f0dd974a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dd4108c30 0x7f0dd419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:51.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.183+0000 7f0dd974a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dd4108c30 0x7f0dd419ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52450/0 (socket says 172.21.15.64:52450) 2024-09-21T04:03:51.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.183+0000 7f0dd974a640 1 -- 172.21.15.64:0/3958155868 learned_addr learned my addr 172.21.15.64:0/3958155868 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:51.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.183+0000 7f0dd3fff640 1 --2- 172.21.15.64:0/3958155868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd41082e0 0x7f0dd4199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:51.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.183+0000 7f0dd8f49640 1 --2- 172.21.15.64:0/3958155868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dd41029b0 0x7f0dd41993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:51.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.183+0000 7f0dd974a640 1 -- 172.21.15.64:0/3958155868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd41082e0 msgr2=0x7f0dd4199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:51.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.183+0000 7f0dd974a640 1 --2- 172.21.15.64:0/3958155868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd41082e0 0x7f0dd4199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.183+0000 7f0dd974a640 1 -- 172.21.15.64:0/3958155868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dd41029b0 msgr2=0x7f0dd41993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:51.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.183+0000 7f0dd974a640 1 --2- 172.21.15.64:0/3958155868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dd41029b0 0x7f0dd41993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.183+0000 7f0dd974a640 1 -- 172.21.15.64:0/3958155868 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd419f2d0 con 0x7f0dd4108c30 2024-09-21T04:03:51.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.183+0000 7f0dd8f49640 1 --2- 172.21.15.64:0/3958155868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dd41029b0 0x7f0dd41993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:03:51.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.183+0000 7f0dd974a640 1 --2- 172.21.15.64:0/3958155868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dd4108c30 0x7f0dd419ebd0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc009430 tx=0x7f0dcc0319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:51.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.183+0000 7f0dd1ffb640 1 -- 172.21.15.64:0/3958155868 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc003c30 con 0x7f0dd4108c30 2024-09-21T04:03:51.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.183+0000 7f0dd1ffb640 1 -- 172.21.15.64:0/3958155868 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dcc003dd0 con 0x7f0dd4108c30 2024-09-21T04:03:51.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.183+0000 7f0ddb1ad640 1 -- 172.21.15.64:0/3958155868 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd419f560 con 0x7f0dd4108c30 2024-09-21T04:03:51.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.183+0000 7f0dd1ffb640 1 -- 172.21.15.64:0/3958155868 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc02faa0 con 0x7f0dd4108c30 2024-09-21T04:03:51.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.184+0000 7f0ddb1ad640 1 -- 172.21.15.64:0/3958155868 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd406b910 con 0x7f0dd4108c30 2024-09-21T04:03:51.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.184+0000 7f0ddb1ad640 1 -- 172.21.15.64:0/3958155868 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dd4102dc0 con 0x7f0dd4108c30 2024-09-21T04:03:51.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.186+0000 7f0dd1ffb640 1 -- 172.21.15.64:0/3958155868 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f0dcc02fc40 con 0x7f0dd4108c30 2024-09-21T04:03:51.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.187+0000 7f0dd1ffb640 1 --2- 172.21.15.64:0/3958155868 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0dc40b65e0 0x7f0dc40b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:51.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.187+0000 7f0dd8f49640 1 --2- 172.21.15.64:0/3958155868 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0dc40b65e0 0x7f0dc40b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:51.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.188+0000 7f0dd1ffb640 1 -- 172.21.15.64:0/3958155868 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f0dcc108970 con 0x7f0dd4108c30 2024-09-21T04:03:51.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.188+0000 7f0dd8f49640 1 --2- 172.21.15.64:0/3958155868 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0dc40b65e0 0x7f0dc40b8af0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009930 tx=0x7f0dc8008040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:51.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.191+0000 7f0dd1ffb640 1 -- 172.21.15.64:0/3958155868 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0dcc0cf0b0 con 0x7f0dd4108c30 2024-09-21T04:03:51.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.379+0000 7f0ddb1ad640 1 -- 172.21.15.64:0/3958155868 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0dd406c3f0 con 0x7f0dd4108c30 2024-09-21T04:03:51.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.381+0000 7f0dd1ffb640 1 -- 172.21.15.64:0/3958155868 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+23230 (secure 0 0 0) 0x7f0dcc0d3490 con 0x7f0dd4108c30 2024-09-21T04:03:51.384 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:03:51.385 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":77,"fsid":"e7c927ba-77cc-11ef-baf5-efdc52797490","created":"2024-09-21T03:53:41.215073+0000","modified":"2024-09-21T04:03:30.555854+0000","last_up_change":"2024-09-21T04:01:11.986949+0000","last_in_change":"2024-09-21T04:00:48.707791+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:57:24.854438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T04:01:58.728355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T04:02:01.300385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb947c10-98e6-4e3a-8bbd-8753951d870b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6803","nonce":3692023035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6805","nonce":3692023035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6809","nonce":3692023035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6807","nonce":3692023035}]},"public_addr":"172.21.15.64:6803/3692023035","cluster_addr":"172.21.15.64:6805/3692023035","heartbeat_back_addr":"172.21.15.64:6809/3692023035","heartbeat_front_addr":"172.21.15.64:6807/3692023035","state":["exists","up"]},{"osd":1,"uuid":"29b8c92d-82fa-4175-b3e2-711917b7f8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6801","nonce":3363415998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6803","nonce":3363415998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6807","nonce":3363415998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6805","nonce":3363415998}]},"public_addr":"172.21.15.119:6801/3363415998","cluster_addr":"172.21.15.119:6803/3363415998","heartbeat_back_addr":"172.21.15.119:6807/3363415998","heartbeat_front_addr":"172.21.15.119:6805/3363415998","state":["exists","up"]},{"osd":2,"uuid":"a2273fe8-df55-40f7-afda-74ab09c52c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6801","nonce":3422924179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6803","nonce":3422924179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6807","nonce":3422924179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6805","nonce":3422924179}]},"public_addr":"172.21.15.131:6801/3422924179","cluster_addr":"172.21.15.131:6803/3422924179","heartbeat_back_addr":"172.21.15.131:6807/3422924179","heartbeat_front_addr":"172.21.15.131:6805/3422924179","state":["exists","up"]},{"osd":3,"uuid":"f1634064-b108-41d2-a78f-c33607b08a70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6811","nonce":2022952729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6813","nonce":2022952729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6817","nonce":2022952729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6815","nonce":2022952729}]},"public_addr":"172.21.15.64:6811/2022952729","cluster_addr":"172.21.15.64:6813/2022952729","heartbeat_back_addr":"172.21.15.64:6817/2022952729","heartbeat_front_addr":"172.21.15.64:6815/2022952729","state":["exists","up"]},{"osd":4,"uuid":"75b81a8d-fa52-411a-bfda-aa69bb70a20f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6809","nonce":323030958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6811","nonce":323030958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6815","nonce":323030958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6813","nonce":323030958}]},"public_addr":"172.21.15.119:6809/323030958","cluster_addr":"172.21.15.119:6811/323030958","heartbeat_back_addr":"172.21.15.119:6815/323030958","heartbeat_front_addr":"172.21.15.119:6813/323030958","state":["exists","up"]},{"osd":5,"uuid":"0e4895b5-42e4-4b2c-a9c3-692f287cb841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6809","nonce":3021788088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6811","nonce":3021788088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6815","nonce":3021788088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6813","nonce":3021788088}]},"public_addr":"172.21.15.131:6809/3021788088","cluster_addr":"172.21.15.131:6811/3021788088","heartbeat_back_addr":"172.21.15.131:6815/3021788088","heartbeat_front_addr":"172.21.15.131:6813/3021788088","state":["exists","up"]},{"osd":6,"uuid":"1fa5c2b7-bce0-43a1-a26b-d029eb3731fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6819","nonce":1913824324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6821","nonce":1913824324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6825","nonce":1913824324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6823","nonce":1913824324}]},"public_addr":"172.21.15.64:6819/1913824324","cluster_addr":"172.21.15.64:6821/1913824324","heartbeat_back_addr":"172.21.15.64:6825/1913824324","heartbeat_front_addr":"172.21.15.64:6823/1913824324","state":["exists","up"]},{"osd":7,"uuid":"c9707260-b581-4a21-b380-f37e16959b02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6817","nonce":2567568166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6819","nonce":2567568166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6823","nonce":2567568166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6821","nonce":2567568166}]},"public_addr":"172.21.15.119:6817/2567568166","cluster_addr":"172.21.15.119:6819/2567568166","heartbeat_back_addr":"172.21.15.119:6823/2567568166","heartbeat_front_addr":"172.21.15.119:6821/2567568166","state":["exists","up"]},{"osd":8,"uuid":"2dbf8ef2-fc9c-4c6b-bfea-003f41ef191e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6817","nonce":199192217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6819","nonce":199192217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6823","nonce":199192217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6821","nonce":199192217}]},"public_addr":"172.21.15.131:6817/199192217","cluster_addr":"172.21.15.131:6819/199192217","heartbeat_back_addr":"172.21.15.131:6823/199192217","heartbeat_front_addr":"172.21.15.131:6821/199192217","state":["exists","up"]},{"osd":9,"uuid":"96f769e5-4783-4558-adfa-4bfaf9da9e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6827","nonce":3309038990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6829","nonce":3309038990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6833","nonce":3309038990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6831","nonce":3309038990}]},"public_addr":"172.21.15.64:6827/3309038990","cluster_addr":"172.21.15.64:6829/3309038990","heartbeat_back_addr":"172.21.15.64:6833/3309038990","heartbeat_front_addr":"172.21.15.64:6831/3309038990","state":["exists","up"]},{"osd":10,"uuid":"13c1ef5c-2299-440c-af5a-a650b47f4b75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6825","nonce":1460844681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6827","nonce":1460844681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6831","nonce":1460844681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6829","nonce":1460844681}]},"public_addr":"172.21.15.119:6825/1460844681","cluster_addr":"172.21.15.119:6827/1460844681","heartbeat_back_addr":"172.21.15.119:6831/1460844681","heartbeat_front_addr":"172.21.15.119:6829/1460844681","state":["exists","up"]},{"osd":11,"uuid":"c26a5ef9-8585-404e-a54a-852424c4d6fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6825","nonce":2294059084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6827","nonce":2294059084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6831","nonce":2294059084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6829","nonce":2294059084}]},"public_addr":"172.21.15.131:6825/2294059084","cluster_addr":"172.21.15.131:6827/2294059084","heartbeat_back_addr":"172.21.15.131:6831/2294059084","heartbeat_front_addr":"172.21.15.131:6829/2294059084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:56:33.141102+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:56:58.883062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:57:22.641789+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:57:48.705790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:58:12.871514+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:58:37.893118+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:03.620828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:29.446179+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:55.002978+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:00:19.987463+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:00:45.364158+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:01:10.228248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.3","mappings":[{"from":10,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":5,"to":6}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/3554471670":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6801/3786213009":"2024-09-22T03:55:28.387426+0000","172.21.15.64:0/2562856406":"2024-09-22T03:54:27.180100+0000","172.21.15.64:0/2603015699":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6800/1209932961":"2024-09-22T03:54:02.562380+0000","172.21.15.64:6800/3786213009":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6801/1209932961":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/3407344596":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1233731831":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1686887701":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1598568021":"2024-09-22T03:54:27.180100+0000","172.21.15.64:6800/2013929312":"2024-09-22T03:54:27.180100+0000","172.21.15.64:6801/2013929312":"2024-09-22T03:54:27.180100+0000","172.21.15.64:0/3652036144":"2024-09-22T03:55:28.387426+0000","172.21.15.64:0/105454703":"2024-09-22T03:54:27.180100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T04:03:51.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.383+0000 7f0ddb1ad640 1 -- 172.21.15.64:0/3958155868 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0dc40b65e0 msgr2=0x7f0dc40b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:51.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.383+0000 7f0ddb1ad640 1 --2- 172.21.15.64:0/3958155868 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0dc40b65e0 0x7f0dc40b8af0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009930 tx=0x7f0dc8008040 comp rx=0 tx=0).stop 2024-09-21T04:03:51.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.383+0000 7f0ddb1ad640 1 -- 172.21.15.64:0/3958155868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dd4108c30 msgr2=0x7f0dd419ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:51.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.383+0000 7f0ddb1ad640 1 --2- 172.21.15.64:0/3958155868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dd4108c30 0x7f0dd419ebd0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc009430 tx=0x7f0dcc0319f0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.383+0000 7f0ddb1ad640 1 -- 172.21.15.64:0/3958155868 shutdown_connections 2024-09-21T04:03:51.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.383+0000 7f0ddb1ad640 1 --2- 172.21.15.64:0/3958155868 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0dc40b65e0 0x7f0dc40b8af0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.383+0000 7f0ddb1ad640 1 --2- 172.21.15.64:0/3958155868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0dd4108c30 0x7f0dd419ebd0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.383+0000 7f0ddb1ad640 1 --2- 172.21.15.64:0/3958155868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd41082e0 0x7f0dd4199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.383+0000 7f0ddb1ad640 1 --2- 172.21.15.64:0/3958155868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dd41029b0 0x7f0dd41993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.384+0000 7f0ddb1ad640 1 -- 172.21.15.64:0/3958155868 >> 172.21.15.64:0/3958155868 conn(0x7f0dd40fe3b0 msgr2=0x7f0dd4107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:51.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.384+0000 7f0ddb1ad640 1 -- 172.21.15.64:0/3958155868 shutdown_connections 2024-09-21T04:03:51.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.384+0000 7f0ddb1ad640 1 -- 172.21.15.64:0/3958155868 wait complete. 2024-09-21T04:03:51.397 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 04:03:51 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: 2024-09-21T04:03:51.012+0000 7fa718064640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T04:03:51.397 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 04:03:51 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: 2024-09-21T04:03:51.012+0000 7fa718064640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T04:03:51.397 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 04:03:51 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: 2024-09-21T04:03:51.012+0000 7fa718064640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T04:03:51.398 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 04:03:51 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: 2024-09-21T04:03:51.012+0000 7fa718064640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T04:03:51.398 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 04:03:51 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: 2024-09-21T04:03:51.012+0000 7fa718064640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T04:03:51.399 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T03:57:24.854438+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T04:01:58.728355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T04:02:01.300385+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T04:03:51.399 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T04:03:51.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.512+0000 7eff25d51640 1 -- 172.21.15.64:0/167667755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff201048e0 msgr2=0x7eff20104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.512+0000 7eff25d51640 1 --2- 172.21.15.64:0/167667755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff201048e0 0x7eff20104cf0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7eff10009ca0 tx=0x7eff1002a8c0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.512+0000 7eff25d51640 1 -- 172.21.15.64:0/167667755 shutdown_connections 2024-09-21T04:03:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.512+0000 7eff25d51640 1 --2- 172.21.15.64:0/167667755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff2010dd40 0x7eff20110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.512+0000 7eff25d51640 1 --2- 172.21.15.64:0/167667755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff20105230 0x7eff2010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.512+0000 7eff25d51640 1 --2- 172.21.15.64:0/167667755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff201048e0 0x7eff20104cf0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.512+0000 7eff25d51640 1 -- 172.21.15.64:0/167667755 >> 172.21.15.64:0/167667755 conn(0x7eff200fe3b0 msgr2=0x7eff20100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.512+0000 7eff25d51640 1 -- 172.21.15.64:0/167667755 shutdown_connections 2024-09-21T04:03:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.512+0000 7eff25d51640 1 -- 172.21.15.64:0/167667755 wait complete. 2024-09-21T04:03:51.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.513+0000 7eff25d51640 1 Processor -- start 2024-09-21T04:03:51.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.513+0000 7eff25d51640 1 -- start start 2024-09-21T04:03:51.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.513+0000 7eff25d51640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff201048e0 0x7eff20102610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:51.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.513+0000 7eff25d51640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff20105230 0x7eff20102b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:51.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.513+0000 7eff25d51640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff2010dd40 0x7eff20103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:51.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.513+0000 7eff25d51640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7eff20113a20 con 0x7eff201048e0 2024-09-21T04:03:51.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.513+0000 7eff25d51640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7eff201138a0 con 0x7eff20105230 2024-09-21T04:03:51.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.513+0000 7eff25d51640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7eff20113ba0 con 0x7eff2010dd40 2024-09-21T04:03:51.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.513+0000 7eff1f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff201048e0 0x7eff20102610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:51.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.513+0000 7eff1f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff201048e0 0x7eff20102610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49886/0 (socket says 172.21.15.64:49886) 2024-09-21T04:03:51.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.513+0000 7eff1f7fe640 1 -- 172.21.15.64:0/3183514959 learned_addr learned my addr 172.21.15.64:0/3183514959 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:51.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.514+0000 7eff1effd640 1 --2- 172.21.15.64:0/3183514959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff20105230 0x7eff20102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:51.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.514+0000 7eff1ffff640 1 --2- 172.21.15.64:0/3183514959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff2010dd40 0x7eff20103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:51.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.514+0000 7eff1f7fe640 1 -- 172.21.15.64:0/3183514959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff2010dd40 msgr2=0x7eff20103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:51.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.514+0000 7eff1f7fe640 1 --2- 172.21.15.64:0/3183514959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff2010dd40 0x7eff20103090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.514+0000 7eff1f7fe640 1 -- 172.21.15.64:0/3183514959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff20105230 msgr2=0x7eff20102b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:51.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.514+0000 7eff1f7fe640 1 --2- 172.21.15.64:0/3183514959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff20105230 0x7eff20102b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.514+0000 7eff1f7fe640 1 -- 172.21.15.64:0/3183514959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff201a86b0 con 0x7eff201048e0 2024-09-21T04:03:51.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.514+0000 7eff1f7fe640 1 --2- 172.21.15.64:0/3183514959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff201048e0 0x7eff20102610 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7eff10006010 tx=0x7eff100319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:51.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.514+0000 7eff1cff9640 1 -- 172.21.15.64:0/3183514959 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff10003bf0 con 0x7eff201048e0 2024-09-21T04:03:51.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.514+0000 7eff1cff9640 1 -- 172.21.15.64:0/3183514959 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff10003d90 con 0x7eff201048e0 2024-09-21T04:03:51.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.514+0000 7eff25d51640 1 -- 172.21.15.64:0/3183514959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff201a8940 con 0x7eff201048e0 2024-09-21T04:03:51.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.514+0000 7eff1cff9640 1 -- 172.21.15.64:0/3183514959 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1002fc20 con 0x7eff201048e0 2024-09-21T04:03:51.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.514+0000 7eff25d51640 1 -- 172.21.15.64:0/3183514959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff201a8f10 con 0x7eff201048e0 2024-09-21T04:03:51.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.515+0000 7eff25d51640 1 -- 172.21.15.64:0/3183514959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff20104cf0 con 0x7eff201048e0 2024-09-21T04:03:51.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.517+0000 7eff1cff9640 1 -- 172.21.15.64:0/3183514959 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7eff100046a0 con 0x7eff201048e0 2024-09-21T04:03:51.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.517+0000 7eff1cff9640 1 --2- 172.21.15.64:0/3183514959 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7eff080b6590 0x7eff080b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:51.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.518+0000 7eff1cff9640 1 -- 172.21.15.64:0/3183514959 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7eff10108a60 con 0x7eff201048e0 2024-09-21T04:03:51.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.518+0000 7eff1effd640 1 --2- 172.21.15.64:0/3183514959 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7eff080b6590 0x7eff080b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:51.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.518+0000 7eff1effd640 1 --2- 172.21.15.64:0/3183514959 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7eff080b6590 0x7eff080b8aa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7eff14005fd0 tx=0x7eff14005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:51.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.521+0000 7eff1cff9640 1 -- 172.21.15.64:0/3183514959 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7eff100cf1a0 con 0x7eff201048e0 2024-09-21T04:03:51.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.708+0000 7eff25d51640 1 -- 172.21.15.64:0/3183514959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7eff201a9260 con 0x7eff201048e0 2024-09-21T04:03:51.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.709+0000 7eff1cff9640 1 -- 172.21.15.64:0/3183514959 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7eff100d3580 con 0x7eff201048e0 2024-09-21T04:03:51.712 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-09-21T04:03:51.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.711+0000 7eff25d51640 1 -- 172.21.15.64:0/3183514959 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7eff080b6590 msgr2=0x7eff080b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:51.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.711+0000 7eff25d51640 1 --2- 172.21.15.64:0/3183514959 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7eff080b6590 0x7eff080b8aa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7eff14005fd0 tx=0x7eff14005850 comp rx=0 tx=0).stop 2024-09-21T04:03:51.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.711+0000 7eff25d51640 1 -- 172.21.15.64:0/3183514959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff201048e0 msgr2=0x7eff20102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:51.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.711+0000 7eff25d51640 1 --2- 172.21.15.64:0/3183514959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff201048e0 0x7eff20102610 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7eff10006010 tx=0x7eff100319f0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.711+0000 7eff25d51640 1 -- 172.21.15.64:0/3183514959 shutdown_connections 2024-09-21T04:03:51.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.711+0000 7eff25d51640 1 --2- 172.21.15.64:0/3183514959 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7eff080b6590 0x7eff080b8aa0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.711+0000 7eff25d51640 1 --2- 172.21.15.64:0/3183514959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff2010dd40 0x7eff20103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.711+0000 7eff25d51640 1 --2- 172.21.15.64:0/3183514959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff20105230 0x7eff20102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.711+0000 7eff25d51640 1 --2- 172.21.15.64:0/3183514959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7eff201048e0 0x7eff20102610 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.711+0000 7eff25d51640 1 -- 172.21.15.64:0/3183514959 >> 172.21.15.64:0/3183514959 conn(0x7eff200fe3b0 msgr2=0x7eff20100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:51.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.711+0000 7eff25d51640 1 -- 172.21.15.64:0/3183514959 shutdown_connections 2024-09-21T04:03:51.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.711+0000 7eff25d51640 1 -- 172.21.15.64:0/3183514959 wait complete. 2024-09-21T04:03:51.725 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:51 smithi064 ceph-mon[31180]: pgmap v316: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:03:51.725 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:51 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2794089605' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-21T04:03:51.725 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:51 smithi064 ceph-mon[31180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:03:51.725 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:51 smithi064 ceph-mon[31180]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:03:51.725 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:51 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3958155868' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:03:51.726 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T04:03:51.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.839+0000 7fd6b2c1d640 1 -- 172.21.15.64:0/278810788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6ac1061e0 msgr2=0x7fd6ac1065f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:51.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.839+0000 7fd6b2c1d640 1 --2- 172.21.15.64:0/278810788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6ac1061e0 0x7fd6ac1065f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0009900 tx=0x7fd6a002a410 comp rx=0 tx=0).stop 2024-09-21T04:03:51.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.839+0000 7fd6b2c1d640 1 -- 172.21.15.64:0/278810788 shutdown_connections 2024-09-21T04:03:51.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.839+0000 7fd6b2c1d640 1 --2- 172.21.15.64:0/278810788 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ac106b30 0x7fd6ac1105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.839+0000 7fd6b2c1d640 1 --2- 172.21.15.64:0/278810788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6ac1061e0 0x7fd6ac1065f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.839+0000 7fd6b2c1d640 1 --2- 172.21.15.64:0/278810788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6ac101fa0 0x7fd6ac1023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.839+0000 7fd6b2c1d640 1 -- 172.21.15.64:0/278810788 >> 172.21.15.64:0/278810788 conn(0x7fd6ac079cb0 msgr2=0x7fd6ac07a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:51.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.839+0000 7fd6b2c1d640 1 -- 172.21.15.64:0/278810788 shutdown_connections 2024-09-21T04:03:51.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.839+0000 7fd6b2c1d640 1 -- 172.21.15.64:0/278810788 wait complete. 2024-09-21T04:03:51.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.839+0000 7fd6b2c1d640 1 Processor -- start 2024-09-21T04:03:51.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.839+0000 7fd6b2c1d640 1 -- start start 2024-09-21T04:03:51.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.840+0000 7fd6b2c1d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6ac101fa0 0x7fd6ac19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:51.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.840+0000 7fd6b2c1d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ac1061e0 0x7fd6ac19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:51.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.840+0000 7fd6b2c1d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6ac106b30 0x7fd6ac19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:51.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.840+0000 7fd6b2c1d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac1139e0 con 0x7fd6ac106b30 2024-09-21T04:03:51.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.840+0000 7fd6b2c1d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac113860 con 0x7fd6ac101fa0 2024-09-21T04:03:51.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.840+0000 7fd6b2c1d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac113b60 con 0x7fd6ac1061e0 2024-09-21T04:03:51.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.840+0000 7fd6b11ba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6ac106b30 0x7fd6ac19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:51.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.840+0000 7fd6b11ba640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6ac106b30 0x7fd6ac19fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49908/0 (socket says 172.21.15.64:49908) 2024-09-21T04:03:51.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.840+0000 7fd6b11ba640 1 -- 172.21.15.64:0/3693063265 learned_addr learned my addr 172.21.15.64:0/3693063265 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:51.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.840+0000 7fd6b09b9640 1 --2- 172.21.15.64:0/3693063265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6ac101fa0 0x7fd6ac19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:51.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.840+0000 7fd6abfff640 1 --2- 172.21.15.64:0/3693063265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ac1061e0 0x7fd6ac19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:51.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.840+0000 7fd6b11ba640 1 -- 172.21.15.64:0/3693063265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ac1061e0 msgr2=0x7fd6ac19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:51.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.840+0000 7fd6b11ba640 1 --2- 172.21.15.64:0/3693063265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ac1061e0 0x7fd6ac19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.840+0000 7fd6b11ba640 1 -- 172.21.15.64:0/3693063265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6ac101fa0 msgr2=0x7fd6ac19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:51.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.840+0000 7fd6b11ba640 1 --2- 172.21.15.64:0/3693063265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6ac101fa0 0x7fd6ac19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:51.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.840+0000 7fd6b11ba640 1 -- 172.21.15.64:0/3693063265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6ac1a03f0 con 0x7fd6ac106b30 2024-09-21T04:03:51.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.840+0000 7fd6b11ba640 1 --2- 172.21.15.64:0/3693063265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6ac106b30 0x7fd6ac19fcf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009fd0 tx=0x7fd6a400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:51.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.841+0000 7fd6a9ffb640 1 -- 172.21.15.64:0/3693063265 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4017070 con 0x7fd6ac106b30 2024-09-21T04:03:51.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.841+0000 7fd6b2c1d640 1 -- 172.21.15.64:0/3693063265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6ac1a06e0 con 0x7fd6ac106b30 2024-09-21T04:03:51.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.841+0000 7fd6a9ffb640 1 -- 172.21.15.64:0/3693063265 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6a4003e20 con 0x7fd6ac106b30 2024-09-21T04:03:51.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.841+0000 7fd6a9ffb640 1 -- 172.21.15.64:0/3693063265 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4010460 con 0x7fd6ac106b30 2024-09-21T04:03:51.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.841+0000 7fd6b2c1d640 1 -- 172.21.15.64:0/3693063265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6ac1a6e70 con 0x7fd6ac106b30 2024-09-21T04:03:51.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.842+0000 7fd6b2c1d640 1 -- 172.21.15.64:0/3693063265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6ac1023b0 con 0x7fd6ac106b30 2024-09-21T04:03:51.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.843+0000 7fd6a9ffb640 1 -- 172.21.15.64:0/3693063265 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fd6a4003970 con 0x7fd6ac106b30 2024-09-21T04:03:51.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.844+0000 7fd6a9ffb640 1 --2- 172.21.15.64:0/3693063265 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd6940b6560 0x7fd6940b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:51.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.844+0000 7fd6b09b9640 1 --2- 172.21.15.64:0/3693063265 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd6940b6560 0x7fd6940b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:51.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.844+0000 7fd6a9ffb640 1 -- 172.21.15.64:0/3693063265 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fd6a40e70d0 con 0x7fd6ac106b30 2024-09-21T04:03:51.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.845+0000 7fd6b09b9640 1 --2- 172.21.15.64:0/3693063265 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd6940b6560 0x7fd6940b8a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd69c005fd0 tx=0x7fd69c00b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:51.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:51.847+0000 7fd6a9ffb640 1 -- 172.21.15.64:0/3693063265 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd6a40b1bf0 con 0x7fd6ac106b30 2024-09-21T04:03:52.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:51 smithi119 ceph-mon[34811]: pgmap v316: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:03:52.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:51 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2794089605' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-21T04:03:52.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:51 smithi119 ceph-mon[34811]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:03:52.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:51 smithi119 ceph-mon[34811]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:03:52.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:51 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3958155868' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:03:52.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.035+0000 7fd6b2c1d640 1 -- 172.21.15.64:0/3693063265 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd6ac10bbc0 con 0x7fd6ac106b30 2024-09-21T04:03:52.038 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.035+0000 7fd6a9ffb640 1 -- 172.21.15.64:0/3693063265 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) ==== 104+0+11 (secure 0 0 0) 0x7fd6a40b6b30 con 0x7fd6ac106b30 2024-09-21T04:03:52.039 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-09-21T04:03:52.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.038+0000 7fd6b2c1d640 1 -- 172.21.15.64:0/3693063265 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd6940b6560 msgr2=0x7fd6940b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:52.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.038+0000 7fd6b2c1d640 1 --2- 172.21.15.64:0/3693063265 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd6940b6560 0x7fd6940b8a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd69c005fd0 tx=0x7fd69c00b410 comp rx=0 tx=0).stop 2024-09-21T04:03:52.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.038+0000 7fd6b2c1d640 1 -- 172.21.15.64:0/3693063265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6ac106b30 msgr2=0x7fd6ac19fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:52.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.038+0000 7fd6b2c1d640 1 --2- 172.21.15.64:0/3693063265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6ac106b30 0x7fd6ac19fcf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009fd0 tx=0x7fd6a400b260 comp rx=0 tx=0).stop 2024-09-21T04:03:52.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.038+0000 7fd6b2c1d640 1 -- 172.21.15.64:0/3693063265 shutdown_connections 2024-09-21T04:03:52.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.038+0000 7fd6b2c1d640 1 --2- 172.21.15.64:0/3693063265 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd6940b6560 0x7fd6940b8a70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:52.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.038+0000 7fd6b2c1d640 1 --2- 172.21.15.64:0/3693063265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd6ac106b30 0x7fd6ac19fcf0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:52.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.038+0000 7fd6b2c1d640 1 --2- 172.21.15.64:0/3693063265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6ac1061e0 0x7fd6ac19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:52.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.038+0000 7fd6b2c1d640 1 --2- 172.21.15.64:0/3693063265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6ac101fa0 0x7fd6ac19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:52.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.038+0000 7fd6b2c1d640 1 -- 172.21.15.64:0/3693063265 >> 172.21.15.64:0/3693063265 conn(0x7fd6ac079cb0 msgr2=0x7fd6ac1095f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:52.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.038+0000 7fd6b2c1d640 1 -- 172.21.15.64:0/3693063265 shutdown_connections 2024-09-21T04:03:52.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.038+0000 7fd6b2c1d640 1 -- 172.21.15.64:0/3693063265 wait complete. 2024-09-21T04:03:52.053 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T04:03:52.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:51 smithi131 ceph-mon[35193]: pgmap v316: 97 pgs: 97 active+clean; 645 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:03:52.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:51 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2794089605' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-21T04:03:52.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:51 smithi131 ceph-mon[35193]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:03:52.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:51 smithi131 ceph-mon[35193]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:03:52.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:51 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3958155868' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:03:52.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.167+0000 7f5e5e132640 1 -- 172.21.15.64:0/3103252781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e58102990 msgr2=0x7f5e58102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:52.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.167+0000 7f5e5e132640 1 --2- 172.21.15.64:0/3103252781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e58102990 0x7f5e58102da0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5e48009900 tx=0x7f5e4802a410 comp rx=0 tx=0).stop 2024-09-21T04:03:52.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.167+0000 7f5e5e132640 1 -- 172.21.15.64:0/3103252781 shutdown_connections 2024-09-21T04:03:52.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.167+0000 7f5e5e132640 1 --2- 172.21.15.64:0/3103252781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e58108c10 0x7f5e5810f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:52.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.167+0000 7f5e5e132640 1 --2- 172.21.15.64:0/3103252781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e581082c0 0x7f5e581086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:52.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.167+0000 7f5e5e132640 1 --2- 172.21.15.64:0/3103252781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e58102990 0x7f5e58102da0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:52.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.167+0000 7f5e5e132640 1 -- 172.21.15.64:0/3103252781 >> 172.21.15.64:0/3103252781 conn(0x7f5e580fe3b0 msgr2=0x7f5e58100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:52.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.167+0000 7f5e5e132640 1 -- 172.21.15.64:0/3103252781 shutdown_connections 2024-09-21T04:03:52.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.167+0000 7f5e5e132640 1 -- 172.21.15.64:0/3103252781 wait complete. 2024-09-21T04:03:52.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.168+0000 7f5e5e132640 1 Processor -- start 2024-09-21T04:03:52.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.168+0000 7f5e5e132640 1 -- start start 2024-09-21T04:03:52.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.168+0000 7f5e5e132640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e58102990 0x7f5e5819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.168+0000 7f5e5e132640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e581082c0 0x7f5e5819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.168+0000 7f5e5e132640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e58108c10 0x7f5e5819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.168+0000 7f5e5e132640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5e581139f0 con 0x7f5e58102990 2024-09-21T04:03:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.168+0000 7f5e5e132640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5e58113870 con 0x7f5e581082c0 2024-09-21T04:03:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.168+0000 7f5e5e132640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5e58113b70 con 0x7f5e58108c10 2024-09-21T04:03:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.168+0000 7f5e577fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e58102990 0x7f5e5819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.168+0000 7f5e577fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e58102990 0x7f5e5819a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49936/0 (socket says 172.21.15.64:49936) 2024-09-21T04:03:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.168+0000 7f5e577fe640 1 -- 172.21.15.64:0/127704202 learned_addr learned my addr 172.21.15.64:0/127704202 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.168+0000 7f5e57fff640 1 --2- 172.21.15.64:0/127704202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e58108c10 0x7f5e5819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.169+0000 7f5e56ffd640 1 --2- 172.21.15.64:0/127704202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e581082c0 0x7f5e5819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.169+0000 7f5e577fe640 1 -- 172.21.15.64:0/127704202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e58108c10 msgr2=0x7f5e5819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:52.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.169+0000 7f5e577fe640 1 --2- 172.21.15.64:0/127704202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e58108c10 0x7f5e5819fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.169+0000 7f5e577fe640 1 -- 172.21.15.64:0/127704202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e581082c0 msgr2=0x7f5e5819a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.169+0000 7f5e577fe640 1 --2- 172.21.15.64:0/127704202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e581082c0 0x7f5e5819a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.169+0000 7f5e577fe640 1 -- 172.21.15.64:0/127704202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e581a0300 con 0x7f5e58102990 2024-09-21T04:03:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.169+0000 7f5e56ffd640 1 --2- 172.21.15.64:0/127704202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e581082c0 0x7f5e5819a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:03:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.169+0000 7f5e577fe640 1 --2- 172.21.15.64:0/127704202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e58102990 0x7f5e5819a3d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5e48005940 tx=0x7f5e480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.169+0000 7f5e54ff9640 1 -- 172.21.15.64:0/127704202 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4803f030 con 0x7f5e58102990 2024-09-21T04:03:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.169+0000 7f5e54ff9640 1 -- 172.21.15.64:0/127704202 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e48003cb0 con 0x7f5e58102990 2024-09-21T04:03:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.169+0000 7f5e54ff9640 1 -- 172.21.15.64:0/127704202 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e48035070 con 0x7f5e58102990 2024-09-21T04:03:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.169+0000 7f5e5e132640 1 -- 172.21.15.64:0/127704202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e581a0590 con 0x7f5e58102990 2024-09-21T04:03:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.169+0000 7f5e5e132640 1 -- 172.21.15.64:0/127704202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e581a6d80 con 0x7f5e58102990 2024-09-21T04:03:52.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.170+0000 7f5e5e132640 1 -- 172.21.15.64:0/127704202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e58102da0 con 0x7f5e58102990 2024-09-21T04:03:52.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.172+0000 7f5e54ff9640 1 -- 172.21.15.64:0/127704202 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f5e48042030 con 0x7f5e58102990 2024-09-21T04:03:52.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.173+0000 7f5e54ff9640 1 --2- 172.21.15.64:0/127704202 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5e400b65e0 0x7f5e400b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:52.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.173+0000 7f5e56ffd640 1 --2- 172.21.15.64:0/127704202 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5e400b65e0 0x7f5e400b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:52.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.173+0000 7f5e54ff9640 1 -- 172.21.15.64:0/127704202 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f5e48108030 con 0x7f5e58102990 2024-09-21T04:03:52.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.173+0000 7f5e56ffd640 1 --2- 172.21.15.64:0/127704202 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5e400b65e0 0x7f5e400b8af0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0058c0 tx=0x7f5e4c005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:52.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.176+0000 7f5e54ff9640 1 -- 172.21.15.64:0/127704202 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5e480ce770 con 0x7f5e58102990 2024-09-21T04:03:52.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.364+0000 7f5e5e132640 1 -- 172.21.15.64:0/127704202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5e5802e640 con 0x7f5e58102990 2024-09-21T04:03:52.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.365+0000 7f5e54ff9640 1 -- 172.21.15.64:0/127704202 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) ==== 100+0+11 (secure 0 0 0) 0x7f5e480d2b50 con 0x7f5e58102990 2024-09-21T04:03:52.368 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-09-21T04:03:52.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.367+0000 7f5e5e132640 1 -- 172.21.15.64:0/127704202 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5e400b65e0 msgr2=0x7f5e400b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:52.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.367+0000 7f5e5e132640 1 --2- 172.21.15.64:0/127704202 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5e400b65e0 0x7f5e400b8af0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0058c0 tx=0x7f5e4c005850 comp rx=0 tx=0).stop 2024-09-21T04:03:52.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.367+0000 7f5e5e132640 1 -- 172.21.15.64:0/127704202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e58102990 msgr2=0x7f5e5819a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:52.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.367+0000 7f5e5e132640 1 --2- 172.21.15.64:0/127704202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e58102990 0x7f5e5819a3d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5e48005940 tx=0x7f5e480042f0 comp rx=0 tx=0).stop 2024-09-21T04:03:52.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.367+0000 7f5e5e132640 1 -- 172.21.15.64:0/127704202 shutdown_connections 2024-09-21T04:03:52.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.367+0000 7f5e5e132640 1 --2- 172.21.15.64:0/127704202 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5e400b65e0 0x7f5e400b8af0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:52.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.367+0000 7f5e5e132640 1 --2- 172.21.15.64:0/127704202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e58108c10 0x7f5e5819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:52.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.367+0000 7f5e5e132640 1 --2- 172.21.15.64:0/127704202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e581082c0 0x7f5e5819a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:52.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.367+0000 7f5e5e132640 1 --2- 172.21.15.64:0/127704202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e58102990 0x7f5e5819a3d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:52.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.367+0000 7f5e5e132640 1 -- 172.21.15.64:0/127704202 >> 172.21.15.64:0/127704202 conn(0x7f5e580fe3b0 msgr2=0x7f5e581007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:52.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.367+0000 7f5e5e132640 1 -- 172.21.15.64:0/127704202 shutdown_connections 2024-09-21T04:03:52.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:52.367+0000 7f5e5e132640 1 -- 172.21.15.64:0/127704202 wait complete. 2024-09-21T04:03:52.382 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-21T04:03:52.382 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:03:52.648 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:52.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:52 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3183514959' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T04:03:52.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:52 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3693063265' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T04:03:52.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:52 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/127704202' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T04:03:53.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:52 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3183514959' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T04:03:53.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:52 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3693063265' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T04:03:53.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:52 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/127704202' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T04:03:53.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:52 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3183514959' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T04:03:53.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:52 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3693063265' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T04:03:53.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:52 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/127704202' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T04:03:53.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.148+0000 7f8e2ec20640 1 -- 172.21.15.64:0/4061091165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e28108d10 msgr2=0x7f8e2810f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:53.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.148+0000 7f8e2ec20640 1 --2- 172.21.15.64:0/4061091165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e28108d10 0x7f8e2810f540 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009ca0 tx=0x7f8e2002a8c0 comp rx=0 tx=0).stop 2024-09-21T04:03:53.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.148+0000 7f8e2ec20640 1 -- 172.21.15.64:0/4061091165 shutdown_connections 2024-09-21T04:03:53.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.148+0000 7f8e2ec20640 1 --2- 172.21.15.64:0/4061091165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e28108d10 0x7f8e2810f540 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:53.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.148+0000 7f8e2ec20640 1 --2- 172.21.15.64:0/4061091165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e281083c0 0x7f8e281087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:53.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.148+0000 7f8e2ec20640 1 --2- 172.21.15.64:0/4061091165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e28102a80 0x7f8e28102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:53.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.148+0000 7f8e2ec20640 1 -- 172.21.15.64:0/4061091165 >> 172.21.15.64:0/4061091165 conn(0x7f8e28079740 msgr2=0x7f8e28079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:53.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.149+0000 7f8e2ec20640 1 -- 172.21.15.64:0/4061091165 shutdown_connections 2024-09-21T04:03:53.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.149+0000 7f8e2ec20640 1 -- 172.21.15.64:0/4061091165 wait complete. 2024-09-21T04:03:53.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.149+0000 7f8e2ec20640 1 Processor -- start 2024-09-21T04:03:53.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.149+0000 7f8e2ec20640 1 -- start start 2024-09-21T04:03:53.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.149+0000 7f8e2ec20640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e28102a80 0x7f8e2819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:53.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.149+0000 7f8e2ec20640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e281083c0 0x7f8e2819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:53.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.149+0000 7f8e2ec20640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e28108d10 0x7f8e2819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:53.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.149+0000 7f8e2ec20640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8e281129e0 con 0x7f8e28102a80 2024-09-21T04:03:53.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.149+0000 7f8e2ec20640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8e28112860 con 0x7f8e28108d10 2024-09-21T04:03:53.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.149+0000 7f8e2ec20640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8e28112b60 con 0x7f8e281083c0 2024-09-21T04:03:53.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.150+0000 7f8e2c9bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e28102a80 0x7f8e2819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:53.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.150+0000 7f8e2c9bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e28102a80 0x7f8e2819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49956/0 (socket says 172.21.15.64:49956) 2024-09-21T04:03:53.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.150+0000 7f8e2c9bc640 1 -- 172.21.15.64:0/270505591 learned_addr learned my addr 172.21.15.64:0/270505591 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:53.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.150+0000 7f8e27fff640 1 --2- 172.21.15.64:0/270505591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e281083c0 0x7f8e2819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:53.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.150+0000 7f8e2d1bd640 1 --2- 172.21.15.64:0/270505591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e28108d10 0x7f8e2819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:53.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.150+0000 7f8e2c9bc640 1 -- 172.21.15.64:0/270505591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e281083c0 msgr2=0x7f8e2819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:53.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.150+0000 7f8e2c9bc640 1 --2- 172.21.15.64:0/270505591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e281083c0 0x7f8e2819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:53.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.150+0000 7f8e2c9bc640 1 -- 172.21.15.64:0/270505591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e28108d10 msgr2=0x7f8e2819fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:53.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.150+0000 7f8e2c9bc640 1 --2- 172.21.15.64:0/270505591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e28108d10 0x7f8e2819fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:53.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.150+0000 7f8e2c9bc640 1 -- 172.21.15.64:0/270505591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e281a0330 con 0x7f8e28102a80 2024-09-21T04:03:53.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.150+0000 7f8e2d1bd640 1 --2- 172.21.15.64:0/270505591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e28108d10 0x7f8e2819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:03:53.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.150+0000 7f8e2c9bc640 1 --2- 172.21.15.64:0/270505591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e28102a80 0x7f8e2819a480 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00c920 tx=0x7f8e1c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:53.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.150+0000 7f8e25ffb640 1 -- 172.21.15.64:0/270505591 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c007bb0 con 0x7f8e28102a80 2024-09-21T04:03:53.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.150+0000 7f8e25ffb640 1 -- 172.21.15.64:0/270505591 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e1c007d50 con 0x7f8e28102a80 2024-09-21T04:03:53.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.150+0000 7f8e2ec20640 1 -- 172.21.15.64:0/270505591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e281a0620 con 0x7f8e28102a80 2024-09-21T04:03:53.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.150+0000 7f8e25ffb640 1 -- 172.21.15.64:0/270505591 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c018730 con 0x7f8e28102a80 2024-09-21T04:03:53.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.151+0000 7f8e2ec20640 1 -- 172.21.15.64:0/270505591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e281a6e40 con 0x7f8e28102a80 2024-09-21T04:03:53.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.152+0000 7f8e2ec20640 1 -- 172.21.15.64:0/270505591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e28103d00 con 0x7f8e28102a80 2024-09-21T04:03:53.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.153+0000 7f8e25ffb640 1 -- 172.21.15.64:0/270505591 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) ==== 154580+0+0 (secure 0 0 0) 0x7f8e1c00f450 con 0x7f8e28102a80 2024-09-21T04:03:53.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.154+0000 7f8e25ffb640 1 --2- 172.21.15.64:0/270505591 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8e180b6620 0x7f8e180b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:53.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.154+0000 7f8e27fff640 1 --2- 172.21.15.64:0/270505591 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8e180b6620 0x7f8e180b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:53.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.154+0000 7f8e25ffb640 1 -- 172.21.15.64:0/270505591 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f8e1c0e7780 con 0x7f8e28102a80 2024-09-21T04:03:53.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.154+0000 7f8e27fff640 1 --2- 172.21.15.64:0/270505591 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8e180b6620 0x7f8e180b8b30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8e28107b70 tx=0x7f8e14009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:53.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.158+0000 7f8e25ffb640 1 -- 172.21.15.64:0/270505591 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8e1c014030 con 0x7f8e28102a80 2024-09-21T04:03:53.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.383+0000 7f8e2ec20640 1 -- 172.21.15.64:0/270505591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e28102e90 con 0x7f8e28102a80 2024-09-21T04:03:53.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.384+0000 7f8e25ffb640 1 -- 172.21.15.64:0/270505591 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f8e1c0add60 con 0x7f8e28102a80 2024-09-21T04:03:53.387 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:03:53.388 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-21T04:03:50:600319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14805,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/97995119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":97995119},{"type":"v1","addr":"172.21.15.64:6839","nonce":97995119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/390891068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":390891068},{"type":"v1","addr":"172.21.15.64:6841","nonce":390891068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:02:03.354744+0000","modified":"2024-09-21T04:03:50.212837+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24710,"mds_1":24680,"mds_2":24661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24634":{"gid":24634,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6839/782190424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782190424},{"type":"v1","addr":"172.21.15.131:6839","nonce":782190424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24680":{"gid":24680,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24710":{"gid":24710,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6839/1319793636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1319793636},{"type":"v1","addr":"172.21.15.119:6839","nonce":1319793636}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24710,"qdb_cluster":[24710,24680,24661]},"id":1}]} 2024-09-21T04:03:53.388 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-21T04:03:53.391 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.388+0000 7f8e2ec20640 1 -- 172.21.15.64:0/270505591 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8e180b6620 msgr2=0x7f8e180b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:53.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.388+0000 7f8e2ec20640 1 --2- 172.21.15.64:0/270505591 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8e180b6620 0x7f8e180b8b30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8e28107b70 tx=0x7f8e14009110 comp rx=0 tx=0).stop 2024-09-21T04:03:53.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.388+0000 7f8e2ec20640 1 -- 172.21.15.64:0/270505591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e28102a80 msgr2=0x7f8e2819a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:53.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.388+0000 7f8e2ec20640 1 --2- 172.21.15.64:0/270505591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e28102a80 0x7f8e2819a480 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00c920 tx=0x7f8e1c00cd20 comp rx=0 tx=0).stop 2024-09-21T04:03:53.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.388+0000 7f8e2ec20640 1 -- 172.21.15.64:0/270505591 shutdown_connections 2024-09-21T04:03:53.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.388+0000 7f8e2ec20640 1 --2- 172.21.15.64:0/270505591 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8e180b6620 0x7f8e180b8b30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:53.392 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.388+0000 7f8e2ec20640 1 --2- 172.21.15.64:0/270505591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e28108d10 0x7f8e2819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:53.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.388+0000 7f8e2ec20640 1 --2- 172.21.15.64:0/270505591 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e281083c0 0x7f8e2819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:53.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.388+0000 7f8e2ec20640 1 --2- 172.21.15.64:0/270505591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e28102a80 0x7f8e2819a480 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:53.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.388+0000 7f8e2ec20640 1 -- 172.21.15.64:0/270505591 >> 172.21.15.64:0/270505591 conn(0x7f8e28079740 msgr2=0x7f8e2806be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:53.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.388+0000 7f8e2ec20640 1 -- 172.21.15.64:0/270505591 shutdown_connections 2024-09-21T04:03:53.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:53.388+0000 7f8e2ec20640 1 -- 172.21.15.64:0/270505591 wait complete. 2024-09-21T04:03:53.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:53 smithi064 ceph-mon[31180]: pgmap v317: 97 pgs: 97 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-21T04:03:53.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:53 smithi064 ceph-mon[31180]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-21T04:03:53.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:53 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/270505591' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:03:54.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:53 smithi119 ceph-mon[34811]: pgmap v317: 97 pgs: 97 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-21T04:03:54.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:53 smithi119 ceph-mon[34811]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-21T04:03:54.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:53 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/270505591' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:03:54.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:53 smithi131 ceph-mon[35193]: pgmap v317: 97 pgs: 97 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-21T04:03:54.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:53 smithi131 ceph-mon[35193]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-21T04:03:54.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:53 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/270505591' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:03:54.140 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-21T04:03:54.140 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-21T04:03:54.141 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T04:03:54.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.254+0000 7f085e64a640 1 -- 172.21.15.64:0/3794303791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08581082b0 msgr2=0x7f08581086c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:54.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.254+0000 7f085e64a640 1 --2- 172.21.15.64:0/3794303791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08581082b0 0x7f08581086c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f084c009980 tx=0x7f084c02c620 comp rx=0 tx=0).stop 2024-09-21T04:03:54.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.254+0000 7f085e64a640 1 -- 172.21.15.64:0/3794303791 shutdown_connections 2024-09-21T04:03:54.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.254+0000 7f085e64a640 1 --2- 172.21.15.64:0/3794303791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0858108c00 0x7f085810f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:54.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.254+0000 7f085e64a640 1 --2- 172.21.15.64:0/3794303791 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08581082b0 0x7f08581086c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:54.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.254+0000 7f085e64a640 1 --2- 172.21.15.64:0/3794303791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0858102980 0x7f0858102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:54.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.254+0000 7f085e64a640 1 -- 172.21.15.64:0/3794303791 >> 172.21.15.64:0/3794303791 conn(0x7f08580fe380 msgr2=0x7f08581007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:54.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.254+0000 7f085e64a640 1 -- 172.21.15.64:0/3794303791 shutdown_connections 2024-09-21T04:03:54.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.254+0000 7f085e64a640 1 -- 172.21.15.64:0/3794303791 wait complete. 2024-09-21T04:03:54.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.255+0000 7f085e64a640 1 Processor -- start 2024-09-21T04:03:54.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.255+0000 7f085e64a640 1 -- start start 2024-09-21T04:03:54.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.255+0000 7f085e64a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0858102980 0x7f0858198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:54.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.255+0000 7f085e64a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08581082b0 0x7f08581987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:54.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.255+0000 7f085e64a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0858108c00 0x7f085819da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:54.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.255+0000 7f085e64a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f08581117c0 con 0x7f0858102980 2024-09-21T04:03:54.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.255+0000 7f085e64a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0858111640 con 0x7f0858108c00 2024-09-21T04:03:54.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.255+0000 7f085e64a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0858111940 con 0x7f08581082b0 2024-09-21T04:03:54.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.255+0000 7f0857fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0858102980 0x7f0858198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:54.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.255+0000 7f0857fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0858102980 0x7f0858198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49966/0 (socket says 172.21.15.64:49966) 2024-09-21T04:03:54.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.255+0000 7f0857fff640 1 -- 172.21.15.64:0/2561373491 learned_addr learned my addr 172.21.15.64:0/2561373491 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:54.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.255+0000 7f08577fe640 1 --2- 172.21.15.64:0/2561373491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08581082b0 0x7f08581987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:54.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.256+0000 7f085cbe7640 1 --2- 172.21.15.64:0/2561373491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0858108c00 0x7f085819da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:54.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.256+0000 7f0857fff640 1 -- 172.21.15.64:0/2561373491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08581082b0 msgr2=0x7f08581987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:54.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.256+0000 7f0857fff640 1 --2- 172.21.15.64:0/2561373491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08581082b0 0x7f08581987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:54.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.256+0000 7f0857fff640 1 -- 172.21.15.64:0/2561373491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0858108c00 msgr2=0x7f085819da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:54.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.256+0000 7f0857fff640 1 --2- 172.21.15.64:0/2561373491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0858108c00 0x7f085819da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:54.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.256+0000 7f0857fff640 1 -- 172.21.15.64:0/2561373491 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085819e150 con 0x7f0858102980 2024-09-21T04:03:54.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.256+0000 7f0857fff640 1 --2- 172.21.15.64:0/2561373491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0858102980 0x7f0858198220 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f084800d770 tx=0x7f084800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:54.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.256+0000 7f08557fa640 1 -- 172.21.15.64:0/2561373491 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0848017070 con 0x7f0858102980 2024-09-21T04:03:54.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.256+0000 7f08557fa640 1 -- 172.21.15.64:0/2561373491 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08480054d0 con 0x7f0858102980 2024-09-21T04:03:54.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.256+0000 7f085e64a640 1 -- 172.21.15.64:0/2561373491 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f085819e440 con 0x7f0858102980 2024-09-21T04:03:54.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.256+0000 7f08557fa640 1 -- 172.21.15.64:0/2561373491 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0848004c30 con 0x7f0858102980 2024-09-21T04:03:54.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.256+0000 7f085e64a640 1 -- 172.21.15.64:0/2561373491 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08581a4bd0 con 0x7f0858102980 2024-09-21T04:03:54.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.257+0000 7f085e64a640 1 -- 172.21.15.64:0/2561373491 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0858102d90 con 0x7f0858102980 2024-09-21T04:03:54.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.259+0000 7f08557fa640 1 -- 172.21.15.64:0/2561373491 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) ==== 154580+0+0 (secure 0 0 0) 0x7f0848004dd0 con 0x7f0858102980 2024-09-21T04:03:54.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.260+0000 7f08557fa640 1 --2- 172.21.15.64:0/2561373491 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f08400b65f0 0x7f08400b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:54.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.260+0000 7f08577fe640 1 --2- 172.21.15.64:0/2561373491 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f08400b65f0 0x7f08400b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:54.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.260+0000 7f08557fa640 1 -- 172.21.15.64:0/2561373491 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f08480e7600 con 0x7f0858102980 2024-09-21T04:03:54.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.260+0000 7f08577fe640 1 --2- 172.21.15.64:0/2561373491 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f08400b65f0 0x7f08400b8b00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f084c009230 tx=0x7f084c033400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:54.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.263+0000 7f08557fa640 1 -- 172.21.15.64:0/2561373491 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f08480adbe0 con 0x7f0858102980 2024-09-21T04:03:54.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.447+0000 7f085e64a640 1 -- 172.21.15.64:0/2561373491 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0858063800 con 0x7f08400b65f0 2024-09-21T04:03:54.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.452+0000 7f08557fa640 1 -- 172.21.15.64:0/2561373491 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259137 (secure 0 0 0) 0x7f0858063800 con 0x7f08400b65f0 2024-09-21T04:03:54.458 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:03:54.459 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-09-21T04:03:54.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.455+0000 7f085e64a640 1 -- 172.21.15.64:0/2561373491 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f08400b65f0 msgr2=0x7f08400b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:54.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.455+0000 7f085e64a640 1 --2- 172.21.15.64:0/2561373491 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f08400b65f0 0x7f08400b8b00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f084c009230 tx=0x7f084c033400 comp rx=0 tx=0).stop 2024-09-21T04:03:54.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.455+0000 7f085e64a640 1 -- 172.21.15.64:0/2561373491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0858102980 msgr2=0x7f0858198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:54.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.455+0000 7f085e64a640 1 --2- 172.21.15.64:0/2561373491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0858102980 0x7f0858198220 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f084800d770 tx=0x7f084800dc30 comp rx=0 tx=0).stop 2024-09-21T04:03:54.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.455+0000 7f085e64a640 1 -- 172.21.15.64:0/2561373491 shutdown_connections 2024-09-21T04:03:54.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.455+0000 7f085e64a640 1 --2- 172.21.15.64:0/2561373491 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f08400b65f0 0x7f08400b8b00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:54.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.455+0000 7f085e64a640 1 --2- 172.21.15.64:0/2561373491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0858108c00 0x7f085819da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:54.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.455+0000 7f085e64a640 1 --2- 172.21.15.64:0/2561373491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08581082b0 0x7f08581987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:54.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.455+0000 7f085e64a640 1 --2- 172.21.15.64:0/2561373491 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0858102980 0x7f0858198220 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:54.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.455+0000 7f085e64a640 1 -- 172.21.15.64:0/2561373491 >> 172.21.15.64:0/2561373491 conn(0x7f08580fe380 msgr2=0x7f08581058e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:54.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.455+0000 7f085e64a640 1 -- 172.21.15.64:0/2561373491 shutdown_connections 2024-09-21T04:03:54.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.455+0000 7f085e64a640 1 -- 172.21.15.64:0/2561373491 wait complete. 2024-09-21T04:03:54.470 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-09-21T04:03:52.505606+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665707,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":706,"num_read_kb":461,"num_write":304,"num_write_kb":2284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":2953038,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":745,"ondisk_log_size":745,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359652,"kb_used_data":10384,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124582172,"statfs":{"total":1151940427776,"available":1151572144128,"internally_reserved":0,"allocated":10633216,"data_stored":7068246,"data_compressed":252522,"data_compressed_allocated":344064,"data_compressed_original":2187264,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5356,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":93,"num_read_kb":52,"num_write":12,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003553"},"pg_stats":[{"pgid":"3.3f","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.567004+0000","last_change":"2024-09-21T04:02:02.769848+0000","last_active":"2024-09-21T04:03:30.567004+0000","last_peered":"2024-09-21T04:03:30.567004+0000","last_clean":"2024-09-21T04:03:30.567004+0000","last_became_active":"2024-09-21T04:02:02.769508+0000","last_became_peered":"2024-09-21T04:02:02.769508+0000","last_unstale":"2024-09-21T04:03:30.567004+0000","last_undegraded":"2024-09-21T04:03:30.567004+0000","last_fullsized":"2024-09-21T04:03:30.567004+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:00.242233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"75'4","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569630+0000","last_change":"2024-09-21T04:02:02.769200+0000","last_active":"2024-09-21T04:03:30.569630+0000","last_peered":"2024-09-21T04:03:30.569630+0000","last_clean":"2024-09-21T04:03:30.569630+0000","last_became_active":"2024-09-21T04:02:02.768918+0000","last_became_peered":"2024-09-21T04:02:02.768918+0000","last_unstale":"2024-09-21T04:03:30.569630+0000","last_undegraded":"2024-09-21T04:03:30.569630+0000","last_fullsized":"2024-09-21T04:03:30.569630+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:26.842079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569334+0000","last_change":"2024-09-21T04:02:02.763511+0000","last_active":"2024-09-21T04:03:30.569334+0000","last_peered":"2024-09-21T04:03:30.569334+0000","last_clean":"2024-09-21T04:03:30.569334+0000","last_became_active":"2024-09-21T04:02:02.763161+0000","last_became_peered":"2024-09-21T04:02:02.763161+0000","last_unstale":"2024-09-21T04:03:30.569334+0000","last_undegraded":"2024-09-21T04:03:30.569334+0000","last_fullsized":"2024-09-21T04:03:30.569334+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:59.834005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.567466+0000","last_change":"2024-09-21T04:02:02.765452+0000","last_active":"2024-09-21T04:03:30.567466+0000","last_peered":"2024-09-21T04:03:30.567466+0000","last_clean":"2024-09-21T04:03:30.567466+0000","last_became_active":"2024-09-21T04:02:02.765065+0000","last_became_peered":"2024-09-21T04:02:02.765065+0000","last_unstale":"2024-09-21T04:03:30.567466+0000","last_undegraded":"2024-09-21T04:03:30.567466+0000","last_fullsized":"2024-09-21T04:03:30.567466+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:33.871366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"75'4","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.572434+0000","last_change":"2024-09-21T04:02:02.767457+0000","last_active":"2024-09-21T04:03:30.572434+0000","last_peered":"2024-09-21T04:03:30.572434+0000","last_clean":"2024-09-21T04:03:30.572434+0000","last_became_active":"2024-09-21T04:02:02.767064+0000","last_became_peered":"2024-09-21T04:02:02.767064+0000","last_unstale":"2024-09-21T04:03:30.572434+0000","last_undegraded":"2024-09-21T04:03:30.572434+0000","last_fullsized":"2024-09-21T04:03:30.572434+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:46.316029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"75'6","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569752+0000","last_change":"2024-09-21T04:02:02.768795+0000","last_active":"2024-09-21T04:03:30.569752+0000","last_peered":"2024-09-21T04:03:30.569752+0000","last_clean":"2024-09-21T04:03:30.569752+0000","last_became_active":"2024-09-21T04:02:02.768434+0000","last_became_peered":"2024-09-21T04:02:02.768434+0000","last_unstale":"2024-09-21T04:03:30.569752+0000","last_undegraded":"2024-09-21T04:03:30.569752+0000","last_fullsized":"2024-09-21T04:03:30.569752+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:58.991533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"75'5","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847154+0000","last_change":"2024-09-21T04:02:02.774998+0000","last_active":"2024-09-21T04:03:30.847154+0000","last_peered":"2024-09-21T04:03:30.847154+0000","last_clean":"2024-09-21T04:03:30.847154+0000","last_became_active":"2024-09-21T04:02:02.774344+0000","last_became_peered":"2024-09-21T04:02:02.774344+0000","last_unstale":"2024-09-21T04:03:30.847154+0000","last_undegraded":"2024-09-21T04:03:30.847154+0000","last_fullsized":"2024-09-21T04:03:30.847154+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:26.705126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"75'9","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.565533+0000","last_change":"2024-09-21T04:02:02.772498+0000","last_active":"2024-09-21T04:03:30.565533+0000","last_peered":"2024-09-21T04:03:30.565533+0000","last_clean":"2024-09-21T04:03:30.565533+0000","last_became_active":"2024-09-21T04:02:02.772049+0000","last_became_peered":"2024-09-21T04:02:02.772049+0000","last_unstale":"2024-09-21T04:03:30.565533+0000","last_undegraded":"2024-09-21T04:03:30.565533+0000","last_fullsized":"2024-09-21T04:03:30.565533+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:31:19.088424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"75'11","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575591+0000","last_change":"2024-09-21T04:02:02.772336+0000","last_active":"2024-09-21T04:03:30.575591+0000","last_peered":"2024-09-21T04:03:30.575591+0000","last_clean":"2024-09-21T04:03:30.575591+0000","last_became_active":"2024-09-21T04:02:02.771964+0000","last_became_peered":"2024-09-21T04:02:02.771964+0000","last_unstale":"2024-09-21T04:03:30.575591+0000","last_undegraded":"2024-09-21T04:03:30.575591+0000","last_fullsized":"2024-09-21T04:03:30.575591+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:46.659690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"75'16","reported_seq":38,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.570342+0000","last_change":"2024-09-21T04:02:02.758992+0000","last_active":"2024-09-21T04:03:30.570342+0000","last_peered":"2024-09-21T04:03:30.570342+0000","last_clean":"2024-09-21T04:03:30.570342+0000","last_became_active":"2024-09-21T04:02:02.758575+0000","last_became_peered":"2024-09-21T04:02:02.758575+0000","last_unstale":"2024-09-21T04:03:30.570342+0000","last_undegraded":"2024-09-21T04:03:30.570342+0000","last_fullsized":"2024-09-21T04:03:30.570342+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:32:09.543140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"75'6","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847480+0000","last_change":"2024-09-21T04:02:02.767304+0000","last_active":"2024-09-21T04:03:30.847480+0000","last_peered":"2024-09-21T04:03:30.847480+0000","last_clean":"2024-09-21T04:03:30.847480+0000","last_became_active":"2024-09-21T04:02:02.762804+0000","last_became_peered":"2024-09-21T04:02:02.762804+0000","last_unstale":"2024-09-21T04:03:30.847480+0000","last_undegraded":"2024-09-21T04:03:30.847480+0000","last_fullsized":"2024-09-21T04:03:30.847480+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:32.224071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.567991+0000","last_change":"2024-09-21T04:02:02.762932+0000","last_active":"2024-09-21T04:03:30.567991+0000","last_peered":"2024-09-21T04:03:30.567991+0000","last_clean":"2024-09-21T04:03:30.567991+0000","last_became_active":"2024-09-21T04:02:02.762538+0000","last_became_peered":"2024-09-21T04:02:02.762538+0000","last_unstale":"2024-09-21T04:03:30.567991+0000","last_undegraded":"2024-09-21T04:03:30.567991+0000","last_fullsized":"2024-09-21T04:03:30.567991+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:20.724453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"75'10","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.910867+0000","last_change":"2024-09-21T04:02:02.770313+0000","last_active":"2024-09-21T04:03:30.910867+0000","last_peered":"2024-09-21T04:03:30.910867+0000","last_clean":"2024-09-21T04:03:30.910867+0000","last_became_active":"2024-09-21T04:02:02.769910+0000","last_became_peered":"2024-09-21T04:02:02.769910+0000","last_unstale":"2024-09-21T04:03:30.910867+0000","last_undegraded":"2024-09-21T04:03:30.910867+0000","last_fullsized":"2024-09-21T04:03:30.910867+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:16.228371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569300+0000","last_change":"2024-09-21T04:02:02.769258+0000","last_active":"2024-09-21T04:03:30.569300+0000","last_peered":"2024-09-21T04:03:30.569300+0000","last_clean":"2024-09-21T04:03:30.569300+0000","last_became_active":"2024-09-21T04:02:02.768914+0000","last_became_peered":"2024-09-21T04:02:02.768914+0000","last_unstale":"2024-09-21T04:03:30.569300+0000","last_undegraded":"2024-09-21T04:03:30.569300+0000","last_fullsized":"2024-09-21T04:03:30.569300+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:37.426810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569216+0000","last_change":"2024-09-21T04:02:02.755940+0000","last_active":"2024-09-21T04:03:30.569216+0000","last_peered":"2024-09-21T04:03:30.569216+0000","last_clean":"2024-09-21T04:03:30.569216+0000","last_became_active":"2024-09-21T04:02:02.755309+0000","last_became_peered":"2024-09-21T04:02:02.755309+0000","last_unstale":"2024-09-21T04:03:30.569216+0000","last_undegraded":"2024-09-21T04:03:30.569216+0000","last_fullsized":"2024-09-21T04:03:30.569216+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:49.834030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"75'9","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.574842+0000","last_change":"2024-09-21T04:02:02.761068+0000","last_active":"2024-09-21T04:03:30.574842+0000","last_peered":"2024-09-21T04:03:30.574842+0000","last_clean":"2024-09-21T04:03:30.574842+0000","last_became_active":"2024-09-21T04:02:02.760683+0000","last_became_peered":"2024-09-21T04:02:02.760683+0000","last_unstale":"2024-09-21T04:03:30.574842+0000","last_undegraded":"2024-09-21T04:03:30.574842+0000","last_fullsized":"2024-09-21T04:03:30.574842+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:36.627539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.567055+0000","last_change":"2024-09-21T04:02:02.772234+0000","last_active":"2024-09-21T04:03:30.567055+0000","last_peered":"2024-09-21T04:03:30.567055+0000","last_clean":"2024-09-21T04:03:30.567055+0000","last_became_active":"2024-09-21T04:02:02.771825+0000","last_became_peered":"2024-09-21T04:02:02.771825+0000","last_unstale":"2024-09-21T04:03:30.567055+0000","last_undegraded":"2024-09-21T04:03:30.567055+0000","last_fullsized":"2024-09-21T04:03:30.567055+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:33.449745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"75'10","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569735+0000","last_change":"2024-09-21T04:02:02.758643+0000","last_active":"2024-09-21T04:03:30.569735+0000","last_peered":"2024-09-21T04:03:30.569735+0000","last_clean":"2024-09-21T04:03:30.569735+0000","last_became_active":"2024-09-21T04:02:02.758261+0000","last_became_peered":"2024-09-21T04:02:02.758261+0000","last_unstale":"2024-09-21T04:03:30.569735+0000","last_undegraded":"2024-09-21T04:03:30.569735+0000","last_fullsized":"2024-09-21T04:03:30.569735+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:04.165399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569847+0000","last_change":"2024-09-21T04:02:02.759405+0000","last_active":"2024-09-21T04:03:30.569847+0000","last_peered":"2024-09-21T04:03:30.569847+0000","last_clean":"2024-09-21T04:03:30.569847+0000","last_became_active":"2024-09-21T04:02:02.758962+0000","last_became_peered":"2024-09-21T04:02:02.758962+0000","last_unstale":"2024-09-21T04:03:30.569847+0000","last_undegraded":"2024-09-21T04:03:30.569847+0000","last_fullsized":"2024-09-21T04:03:30.569847+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:37.203308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.566627+0000","last_change":"2024-09-21T04:02:02.767518+0000","last_active":"2024-09-21T04:03:30.566627+0000","last_peered":"2024-09-21T04:03:30.566627+0000","last_clean":"2024-09-21T04:03:30.566627+0000","last_became_active":"2024-09-21T04:02:02.767147+0000","last_became_peered":"2024-09-21T04:02:02.767147+0000","last_unstale":"2024-09-21T04:03:30.566627+0000","last_undegraded":"2024-09-21T04:03:30.566627+0000","last_fullsized":"2024-09-21T04:03:30.566627+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:34:13.615218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"75'11","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.910844+0000","last_change":"2024-09-21T04:02:02.756407+0000","last_active":"2024-09-21T04:03:30.910844+0000","last_peered":"2024-09-21T04:03:30.910844+0000","last_clean":"2024-09-21T04:03:30.910844+0000","last_became_active":"2024-09-21T04:02:02.756007+0000","last_became_peered":"2024-09-21T04:02:02.756007+0000","last_unstale":"2024-09-21T04:03:30.910844+0000","last_undegraded":"2024-09-21T04:03:30.910844+0000","last_fullsized":"2024-09-21T04:03:30.910844+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:18.798814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"75'10","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575347+0000","last_change":"2024-09-21T04:02:02.766290+0000","last_active":"2024-09-21T04:03:30.575347+0000","last_peered":"2024-09-21T04:03:30.575347+0000","last_clean":"2024-09-21T04:03:30.575347+0000","last_became_active":"2024-09-21T04:02:02.765660+0000","last_became_peered":"2024-09-21T04:02:02.765660+0000","last_unstale":"2024-09-21T04:03:30.575347+0000","last_undegraded":"2024-09-21T04:03:30.575347+0000","last_fullsized":"2024-09-21T04:03:30.575347+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:23.281476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.565585+0000","last_change":"2024-09-21T04:02:02.767943+0000","last_active":"2024-09-21T04:03:30.565585+0000","last_peered":"2024-09-21T04:03:30.565585+0000","last_clean":"2024-09-21T04:03:30.565585+0000","last_became_active":"2024-09-21T04:02:02.767566+0000","last_became_peered":"2024-09-21T04:02:02.767566+0000","last_unstale":"2024-09-21T04:03:30.565585+0000","last_undegraded":"2024-09-21T04:03:30.565585+0000","last_fullsized":"2024-09-21T04:03:30.565585+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:44.221531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"75'6","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.574737+0000","last_change":"2024-09-21T04:02:02.770114+0000","last_active":"2024-09-21T04:03:30.574737+0000","last_peered":"2024-09-21T04:03:30.574737+0000","last_clean":"2024-09-21T04:03:30.574737+0000","last_became_active":"2024-09-21T04:02:02.769732+0000","last_became_peered":"2024-09-21T04:02:02.769732+0000","last_unstale":"2024-09-21T04:03:30.574737+0000","last_undegraded":"2024-09-21T04:03:30.574737+0000","last_fullsized":"2024-09-21T04:03:30.574737+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:38.396288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"75'2","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569686+0000","last_change":"2024-09-21T04:02:30.017325+0000","last_active":"2024-09-21T04:03:30.569686+0000","last_peered":"2024-09-21T04:03:30.569686+0000","last_clean":"2024-09-21T04:03:30.569686+0000","last_became_active":"2024-09-21T04:02:30.017014+0000","last_became_peered":"2024-09-21T04:02:30.017014+0000","last_unstale":"2024-09-21T04:03:30.569686+0000","last_undegraded":"2024-09-21T04:03:30.569686+0000","last_fullsized":"2024-09-21T04:03:30.569686+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:28.579436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.26","version":"75'15","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569995+0000","last_change":"2024-09-21T04:02:02.764666+0000","last_active":"2024-09-21T04:03:30.569995+0000","last_peered":"2024-09-21T04:03:30.569995+0000","last_clean":"2024-09-21T04:03:30.569995+0000","last_became_active":"2024-09-21T04:02:02.763931+0000","last_became_peered":"2024-09-21T04:02:02.763931+0000","last_unstale":"2024-09-21T04:03:30.569995+0000","last_undegraded":"2024-09-21T04:03:30.569995+0000","last_fullsized":"2024-09-21T04:03:30.569995+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:39.724770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"75'6","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.566850+0000","last_change":"2024-09-21T04:02:02.774276+0000","last_active":"2024-09-21T04:03:30.566850+0000","last_peered":"2024-09-21T04:03:30.566850+0000","last_clean":"2024-09-21T04:03:30.566850+0000","last_became_active":"2024-09-21T04:02:02.773852+0000","last_became_peered":"2024-09-21T04:02:02.773852+0000","last_unstale":"2024-09-21T04:03:30.566850+0000","last_undegraded":"2024-09-21T04:03:30.566850+0000","last_fullsized":"2024-09-21T04:03:30.566850+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:40.072264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"75'4","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.911090+0000","last_change":"2024-09-21T04:02:02.758520+0000","last_active":"2024-09-21T04:03:30.911090+0000","last_peered":"2024-09-21T04:03:30.911090+0000","last_clean":"2024-09-21T04:03:30.911090+0000","last_became_active":"2024-09-21T04:02:02.757656+0000","last_became_peered":"2024-09-21T04:02:02.757656+0000","last_unstale":"2024-09-21T04:03:30.911090+0000","last_undegraded":"2024-09-21T04:03:30.911090+0000","last_fullsized":"2024-09-21T04:03:30.911090+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:30.854336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569551+0000","last_change":"2024-09-21T04:02:02.756428+0000","last_active":"2024-09-21T04:03:30.569551+0000","last_peered":"2024-09-21T04:03:30.569551+0000","last_clean":"2024-09-21T04:03:30.569551+0000","last_became_active":"2024-09-21T04:02:02.755985+0000","last_became_peered":"2024-09-21T04:02:02.755985+0000","last_unstale":"2024-09-21T04:03:30.569551+0000","last_undegraded":"2024-09-21T04:03:30.569551+0000","last_fullsized":"2024-09-21T04:03:30.569551+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:36.040436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569536+0000","last_change":"2024-09-21T04:02:02.762893+0000","last_active":"2024-09-21T04:03:30.569536+0000","last_peered":"2024-09-21T04:03:30.569536+0000","last_clean":"2024-09-21T04:03:30.569536+0000","last_became_active":"2024-09-21T04:02:02.762343+0000","last_became_peered":"2024-09-21T04:02:02.762343+0000","last_unstale":"2024-09-21T04:03:30.569536+0000","last_undegraded":"2024-09-21T04:03:30.569536+0000","last_fullsized":"2024-09-21T04:03:30.569536+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:45.074711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"75'12","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569534+0000","last_change":"2024-09-21T04:02:02.763942+0000","last_active":"2024-09-21T04:03:30.569534+0000","last_peered":"2024-09-21T04:03:30.569534+0000","last_clean":"2024-09-21T04:03:30.569534+0000","last_became_active":"2024-09-21T04:02:02.763433+0000","last_became_peered":"2024-09-21T04:02:02.763433+0000","last_unstale":"2024-09-21T04:03:30.569534+0000","last_undegraded":"2024-09-21T04:03:30.569534+0000","last_fullsized":"2024-09-21T04:03:30.569534+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:36.139440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"75'9","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.566588+0000","last_change":"2024-09-21T04:02:02.773364+0000","last_active":"2024-09-21T04:03:30.566588+0000","last_peered":"2024-09-21T04:03:30.566588+0000","last_clean":"2024-09-21T04:03:30.566588+0000","last_became_active":"2024-09-21T04:02:02.773040+0000","last_became_peered":"2024-09-21T04:02:02.773040+0000","last_unstale":"2024-09-21T04:03:30.566588+0000","last_undegraded":"2024-09-21T04:03:30.566588+0000","last_fullsized":"2024-09-21T04:03:30.566588+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:48.504476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.572424+0000","last_change":"2024-09-21T04:02:02.774951+0000","last_active":"2024-09-21T04:03:30.572424+0000","last_peered":"2024-09-21T04:03:30.572424+0000","last_clean":"2024-09-21T04:03:30.572424+0000","last_became_active":"2024-09-21T04:02:02.774525+0000","last_became_peered":"2024-09-21T04:02:02.774525+0000","last_unstale":"2024-09-21T04:03:30.572424+0000","last_undegraded":"2024-09-21T04:03:30.572424+0000","last_fullsized":"2024-09-21T04:03:30.572424+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:24.447860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"73'5","reported_seq":132,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:48.958216+0000","last_change":"2024-09-21T04:02:00.743470+0000","last_active":"2024-09-21T04:03:48.958216+0000","last_peered":"2024-09-21T04:03:48.958216+0000","last_clean":"2024-09-21T04:03:48.958216+0000","last_became_active":"2024-09-21T04:02:00.743055+0000","last_became_peered":"2024-09-21T04:02:00.743055+0000","last_unstale":"2024-09-21T04:03:48.958216+0000","last_undegraded":"2024-09-21T04:03:48.958216+0000","last_fullsized":"2024-09-21T04:03:48.958216+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:56.101753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.574871+0000","last_change":"2024-09-21T04:02:02.773399+0000","last_active":"2024-09-21T04:03:30.574871+0000","last_peered":"2024-09-21T04:03:30.574871+0000","last_clean":"2024-09-21T04:03:30.574871+0000","last_became_active":"2024-09-21T04:02:02.772996+0000","last_became_peered":"2024-09-21T04:02:02.772996+0000","last_unstale":"2024-09-21T04:03:30.574871+0000","last_undegraded":"2024-09-21T04:03:30.574871+0000","last_fullsized":"2024-09-21T04:03:30.574871+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:07.256226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"73'3","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569954+0000","last_change":"2024-09-21T04:02:30.026868+0000","last_active":"2024-09-21T04:03:30.569954+0000","last_peered":"2024-09-21T04:03:30.569954+0000","last_clean":"2024-09-21T04:03:30.569954+0000","last_became_active":"2024-09-21T04:02:30.019469+0000","last_became_peered":"2024-09-21T04:02:30.019469+0000","last_unstale":"2024-09-21T04:03:30.569954+0000","last_undegraded":"2024-09-21T04:03:30.569954+0000","last_fullsized":"2024-09-21T04:03:30.569954+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:35.304358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"73'2","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.911058+0000","last_change":"2024-09-21T04:02:00.736630+0000","last_active":"2024-09-21T04:03:30.911058+0000","last_peered":"2024-09-21T04:03:30.911058+0000","last_clean":"2024-09-21T04:03:30.911058+0000","last_became_active":"2024-09-21T04:02:00.736211+0000","last_became_peered":"2024-09-21T04:02:00.736211+0000","last_unstale":"2024-09-21T04:03:30.911058+0000","last_undegraded":"2024-09-21T04:03:30.911058+0000","last_fullsized":"2024-09-21T04:03:30.911058+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:37.211617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.565621+0000","last_change":"2024-09-21T04:02:02.763323+0000","last_active":"2024-09-21T04:03:30.565621+0000","last_peered":"2024-09-21T04:03:30.565621+0000","last_clean":"2024-09-21T04:03:30.565621+0000","last_became_active":"2024-09-21T04:02:02.762776+0000","last_became_peered":"2024-09-21T04:02:02.762776+0000","last_unstale":"2024-09-21T04:03:30.565621+0000","last_undegraded":"2024-09-21T04:03:30.565621+0000","last_fullsized":"2024-09-21T04:03:30.565621+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:39.891042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"73'1","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575049+0000","last_change":"2024-09-21T04:02:00.743377+0000","last_active":"2024-09-21T04:03:30.575049+0000","last_peered":"2024-09-21T04:03:30.575049+0000","last_clean":"2024-09-21T04:03:30.575049+0000","last_became_active":"2024-09-21T04:02:00.742973+0000","last_became_peered":"2024-09-21T04:02:00.742973+0000","last_unstale":"2024-09-21T04:03:30.575049+0000","last_undegraded":"2024-09-21T04:03:30.575049+0000","last_fullsized":"2024-09-21T04:03:30.575049+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:55:00.655532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"75'11","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.570104+0000","last_change":"2024-09-21T04:02:02.758721+0000","last_active":"2024-09-21T04:03:30.570104+0000","last_peered":"2024-09-21T04:03:30.570104+0000","last_clean":"2024-09-21T04:03:30.570104+0000","last_became_active":"2024-09-21T04:02:02.758352+0000","last_became_peered":"2024-09-21T04:02:02.758352+0000","last_unstale":"2024-09-21T04:03:30.570104+0000","last_undegraded":"2024-09-21T04:03:30.570104+0000","last_fullsized":"2024-09-21T04:03:30.570104+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:56:49.289510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'4","reported_seq":131,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:51.018477+0000","last_change":"2024-09-21T04:02:00.744591+0000","last_active":"2024-09-21T04:03:51.018477+0000","last_peered":"2024-09-21T04:03:51.018477+0000","last_clean":"2024-09-21T04:03:51.018477+0000","last_became_active":"2024-09-21T04:02:00.744139+0000","last_became_peered":"2024-09-21T04:02:00.744139+0000","last_unstale":"2024-09-21T04:03:51.018477+0000","last_undegraded":"2024-09-21T04:03:51.018477+0000","last_fullsized":"2024-09-21T04:03:51.018477+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:21.331517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"75'10","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575159+0000","last_change":"2024-09-21T04:02:02.762779+0000","last_active":"2024-09-21T04:03:30.575159+0000","last_peered":"2024-09-21T04:03:30.575159+0000","last_clean":"2024-09-21T04:03:30.575159+0000","last_became_active":"2024-09-21T04:02:02.760761+0000","last_became_peered":"2024-09-21T04:02:02.760761+0000","last_unstale":"2024-09-21T04:03:30.575159+0000","last_undegraded":"2024-09-21T04:03:30.575159+0000","last_fullsized":"2024-09-21T04:03:30.575159+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:18.789052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"73'2","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847588+0000","last_change":"2024-09-21T04:02:00.743220+0000","last_active":"2024-09-21T04:03:30.847588+0000","last_peered":"2024-09-21T04:03:30.847588+0000","last_clean":"2024-09-21T04:03:30.847588+0000","last_became_active":"2024-09-21T04:02:00.742636+0000","last_became_peered":"2024-09-21T04:02:00.742636+0000","last_unstale":"2024-09-21T04:03:30.847588+0000","last_undegraded":"2024-09-21T04:03:30.847588+0000","last_fullsized":"2024-09-21T04:03:30.847588+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:01.382338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"75'9","reported_seq":38,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569960+0000","last_change":"2024-09-21T04:02:29.869663+0000","last_active":"2024-09-21T04:03:30.569960+0000","last_peered":"2024-09-21T04:03:30.569960+0000","last_clean":"2024-09-21T04:03:30.569960+0000","last_became_active":"2024-09-21T04:02:29.869230+0000","last_became_peered":"2024-09-21T04:02:29.869230+0000","last_unstale":"2024-09-21T04:03:30.569960+0000","last_undegraded":"2024-09-21T04:03:30.569960+0000","last_fullsized":"2024-09-21T04:03:30.569960+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:19:59.987267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"73'4","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575489+0000","last_change":"2024-09-21T04:02:00.853284+0000","last_active":"2024-09-21T04:03:30.575489+0000","last_peered":"2024-09-21T04:03:30.575489+0000","last_clean":"2024-09-21T04:03:30.575489+0000","last_became_active":"2024-09-21T04:02:00.852855+0000","last_became_peered":"2024-09-21T04:02:00.852855+0000","last_unstale":"2024-09-21T04:03:30.575489+0000","last_undegraded":"2024-09-21T04:03:30.575489+0000","last_fullsized":"2024-09-21T04:03:30.575489+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:36.994214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"75'11","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575292+0000","last_change":"2024-09-21T04:02:02.763577+0000","last_active":"2024-09-21T04:03:30.575292+0000","last_peered":"2024-09-21T04:03:30.575292+0000","last_clean":"2024-09-21T04:03:30.575292+0000","last_became_active":"2024-09-21T04:02:02.762970+0000","last_became_peered":"2024-09-21T04:02:02.762970+0000","last_unstale":"2024-09-21T04:03:30.575292+0000","last_undegraded":"2024-09-21T04:03:30.575292+0000","last_fullsized":"2024-09-21T04:03:30.575292+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:49.956226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'35","reported_seq":274,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:51.963042+0000","last_change":"2024-09-21T04:02:00.744203+0000","last_active":"2024-09-21T04:03:51.963042+0000","last_peered":"2024-09-21T04:03:51.963042+0000","last_clean":"2024-09-21T04:03:51.963042+0000","last_became_active":"2024-09-21T04:02:00.743659+0000","last_became_peered":"2024-09-21T04:02:00.743659+0000","last_unstale":"2024-09-21T04:03:51.963042+0000","last_undegraded":"2024-09-21T04:03:51.963042+0000","last_fullsized":"2024-09-21T04:03:51.963042+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:32.108302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71187,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":212,"num_read_kb":175,"num_write":36,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"75'6","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.572693+0000","last_change":"2024-09-21T04:02:02.766002+0000","last_active":"2024-09-21T04:03:30.572693+0000","last_peered":"2024-09-21T04:03:30.572693+0000","last_clean":"2024-09-21T04:03:30.572693+0000","last_became_active":"2024-09-21T04:02:02.765628+0000","last_became_peered":"2024-09-21T04:02:02.765628+0000","last_unstale":"2024-09-21T04:03:30.572693+0000","last_undegraded":"2024-09-21T04:03:30.572693+0000","last_fullsized":"2024-09-21T04:03:30.572693+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:22.251586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"73'6","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569569+0000","last_change":"2024-09-21T04:02:00.741086+0000","last_active":"2024-09-21T04:03:30.569569+0000","last_peered":"2024-09-21T04:03:30.569569+0000","last_clean":"2024-09-21T04:03:30.569569+0000","last_became_active":"2024-09-21T04:02:00.740633+0000","last_became_peered":"2024-09-21T04:02:00.740633+0000","last_unstale":"2024-09-21T04:03:30.569569+0000","last_undegraded":"2024-09-21T04:03:30.569569+0000","last_fullsized":"2024-09-21T04:03:30.569569+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:20.392799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"75'9","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.910819+0000","last_change":"2024-09-21T04:02:02.759651+0000","last_active":"2024-09-21T04:03:30.910819+0000","last_peered":"2024-09-21T04:03:30.910819+0000","last_clean":"2024-09-21T04:03:30.910819+0000","last_became_active":"2024-09-21T04:02:02.759290+0000","last_became_peered":"2024-09-21T04:02:02.759290+0000","last_unstale":"2024-09-21T04:03:30.910819+0000","last_undegraded":"2024-09-21T04:03:30.910819+0000","last_fullsized":"2024-09-21T04:03:30.910819+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:32.025703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"73'3","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.574965+0000","last_change":"2024-09-21T04:02:00.744005+0000","last_active":"2024-09-21T04:03:30.574965+0000","last_peered":"2024-09-21T04:03:30.574965+0000","last_clean":"2024-09-21T04:03:30.574965+0000","last_became_active":"2024-09-21T04:02:00.743570+0000","last_became_peered":"2024-09-21T04:02:00.743570+0000","last_unstale":"2024-09-21T04:03:30.574965+0000","last_undegraded":"2024-09-21T04:03:30.574965+0000","last_fullsized":"2024-09-21T04:03:30.574965+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:32.158681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"75'15","reported_seq":44,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847663+0000","last_change":"2024-09-21T04:02:29.869200+0000","last_active":"2024-09-21T04:03:30.847663+0000","last_peered":"2024-09-21T04:03:30.847663+0000","last_clean":"2024-09-21T04:03:30.847663+0000","last_became_active":"2024-09-21T04:02:29.868836+0000","last_became_peered":"2024-09-21T04:02:29.868836+0000","last_unstale":"2024-09-21T04:03:30.847663+0000","last_undegraded":"2024-09-21T04:03:30.847663+0000","last_fullsized":"2024-09-21T04:03:30.847663+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:42:49.030499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'4","reported_seq":132,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:49.959848+0000","last_change":"2024-09-21T04:02:00.741596+0000","last_active":"2024-09-21T04:03:49.959848+0000","last_peered":"2024-09-21T04:03:49.959848+0000","last_clean":"2024-09-21T04:03:49.959848+0000","last_became_active":"2024-09-21T04:02:00.741185+0000","last_became_peered":"2024-09-21T04:02:00.741185+0000","last_unstale":"2024-09-21T04:03:49.959848+0000","last_undegraded":"2024-09-21T04:03:49.959848+0000","last_fullsized":"2024-09-21T04:03:49.959848+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:31.663133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"75'6","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847114+0000","last_change":"2024-09-21T04:02:02.773706+0000","last_active":"2024-09-21T04:03:30.847114+0000","last_peered":"2024-09-21T04:03:30.847114+0000","last_clean":"2024-09-21T04:03:30.847114+0000","last_became_active":"2024-09-21T04:02:02.773259+0000","last_became_peered":"2024-09-21T04:02:02.773259+0000","last_unstale":"2024-09-21T04:03:30.847114+0000","last_undegraded":"2024-09-21T04:03:30.847114+0000","last_fullsized":"2024-09-21T04:03:30.847114+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:50.431982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"73'15","reported_seq":43,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.572386+0000","last_change":"2024-09-21T04:02:00.851335+0000","last_active":"2024-09-21T04:03:30.572386+0000","last_peered":"2024-09-21T04:03:30.572386+0000","last_clean":"2024-09-21T04:03:30.572386+0000","last_became_active":"2024-09-21T04:02:00.850990+0000","last_became_peered":"2024-09-21T04:02:00.850990+0000","last_unstale":"2024-09-21T04:03:30.572386+0000","last_undegraded":"2024-09-21T04:03:30.572386+0000","last_fullsized":"2024-09-21T04:03:30.572386+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:45.731121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"75'6","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.574843+0000","last_change":"2024-09-21T04:02:02.770985+0000","last_active":"2024-09-21T04:03:30.574843+0000","last_peered":"2024-09-21T04:03:30.574843+0000","last_clean":"2024-09-21T04:03:30.574843+0000","last_became_active":"2024-09-21T04:02:02.770602+0000","last_became_peered":"2024-09-21T04:02:02.770602+0000","last_unstale":"2024-09-21T04:03:30.574843+0000","last_undegraded":"2024-09-21T04:03:30.574843+0000","last_fullsized":"2024-09-21T04:03:30.574843+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:37.131113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'3","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.572923+0000","last_change":"2024-09-21T04:02:00.740834+0000","last_active":"2024-09-21T04:03:30.572923+0000","last_peered":"2024-09-21T04:03:30.572923+0000","last_clean":"2024-09-21T04:03:30.572923+0000","last_became_active":"2024-09-21T04:02:00.740439+0000","last_became_peered":"2024-09-21T04:02:00.740439+0000","last_unstale":"2024-09-21T04:03:30.572923+0000","last_undegraded":"2024-09-21T04:03:30.572923+0000","last_fullsized":"2024-09-21T04:03:30.572923+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:04:22.159986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"75'5","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.566818+0000","last_change":"2024-09-21T04:02:30.018671+0000","last_active":"2024-09-21T04:03:30.566818+0000","last_peered":"2024-09-21T04:03:30.566818+0000","last_clean":"2024-09-21T04:03:30.566818+0000","last_became_active":"2024-09-21T04:02:30.018247+0000","last_became_peered":"2024-09-21T04:02:30.018247+0000","last_unstale":"2024-09-21T04:03:30.566818+0000","last_undegraded":"2024-09-21T04:03:30.566818+0000","last_fullsized":"2024-09-21T04:03:30.566818+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:13.501966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"73'2","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.911086+0000","last_change":"2024-09-21T04:02:30.024526+0000","last_active":"2024-09-21T04:03:30.911086+0000","last_peered":"2024-09-21T04:03:30.911086+0000","last_clean":"2024-09-21T04:03:30.911086+0000","last_became_active":"2024-09-21T04:02:30.018811+0000","last_became_peered":"2024-09-21T04:02:30.018811+0000","last_unstale":"2024-09-21T04:03:30.911086+0000","last_undegraded":"2024-09-21T04:03:30.911086+0000","last_fullsized":"2024-09-21T04:03:30.911086+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:07.093364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"75'9","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847230+0000","last_change":"2024-09-21T04:02:02.766709+0000","last_active":"2024-09-21T04:03:30.847230+0000","last_peered":"2024-09-21T04:03:30.847230+0000","last_clean":"2024-09-21T04:03:30.847230+0000","last_became_active":"2024-09-21T04:02:02.766342+0000","last_became_peered":"2024-09-21T04:02:02.766342+0000","last_unstale":"2024-09-21T04:03:30.847230+0000","last_undegraded":"2024-09-21T04:03:30.847230+0000","last_fullsized":"2024-09-21T04:03:30.847230+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:26:35.185368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"73'1","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575295+0000","last_change":"2024-09-21T04:02:00.745365+0000","last_active":"2024-09-21T04:03:30.575295+0000","last_peered":"2024-09-21T04:03:30.575295+0000","last_clean":"2024-09-21T04:03:30.575295+0000","last_became_active":"2024-09-21T04:02:00.744968+0000","last_became_peered":"2024-09-21T04:02:00.744968+0000","last_unstale":"2024-09-21T04:03:30.575295+0000","last_undegraded":"2024-09-21T04:03:30.575295+0000","last_fullsized":"2024-09-21T04:03:30.575295+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:56.078169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847168+0000","last_change":"2024-09-21T04:02:02.774877+0000","last_active":"2024-09-21T04:03:30.847168+0000","last_peered":"2024-09-21T04:03:30.847168+0000","last_clean":"2024-09-21T04:03:30.847168+0000","last_became_active":"2024-09-21T04:02:02.774464+0000","last_became_peered":"2024-09-21T04:02:02.774464+0000","last_unstale":"2024-09-21T04:03:30.847168+0000","last_undegraded":"2024-09-21T04:03:30.847168+0000","last_fullsized":"2024-09-21T04:03:30.847168+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:23.393239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"73'3","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.576129+0000","last_change":"2024-09-21T04:03:30.575520+0000","last_active":"2024-09-21T04:03:30.576129+0000","last_peered":"2024-09-21T04:03:30.576129+0000","last_clean":"2024-09-21T04:03:30.576129+0000","last_became_active":"2024-09-21T04:03:30.570445+0000","last_became_peered":"2024-09-21T04:03:30.570445+0000","last_unstale":"2024-09-21T04:03:30.576129+0000","last_undegraded":"2024-09-21T04:03:30.576129+0000","last_fullsized":"2024-09-21T04:03:30.576129+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:15.571668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847199+0000","last_change":"2024-09-21T04:02:02.768164+0000","last_active":"2024-09-21T04:03:30.847199+0000","last_peered":"2024-09-21T04:03:30.847199+0000","last_clean":"2024-09-21T04:03:30.847199+0000","last_became_active":"2024-09-21T04:02:02.766925+0000","last_became_peered":"2024-09-21T04:02:02.766925+0000","last_unstale":"2024-09-21T04:03:30.847199+0000","last_undegraded":"2024-09-21T04:03:30.847199+0000","last_fullsized":"2024-09-21T04:03:30.847199+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:59.707637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":100,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.848055+0000","last_change":"2024-09-21T04:00:48.936996+0000","last_active":"2024-09-21T04:03:30.848055+0000","last_peered":"2024-09-21T04:03:30.848055+0000","last_clean":"2024-09-21T04:03:30.848055+0000","last_became_active":"2024-09-21T04:00:48.931310+0000","last_became_peered":"2024-09-21T04:00:48.931310+0000","last_unstale":"2024-09-21T04:03:30.848055+0000","last_undegraded":"2024-09-21T04:03:30.848055+0000","last_fullsized":"2024-09-21T04:03:30.848055+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:57:25.346362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:57:25.346362+0000","last_clean_scrub_stamp":"2024-09-21T03:57:25.346362+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:24.165731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"73'3","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.581750+0000","last_change":"2024-09-21T04:03:30.581661+0000","last_active":"2024-09-21T04:03:30.581750+0000","last_peered":"2024-09-21T04:03:30.581750+0000","last_clean":"2024-09-21T04:03:30.581750+0000","last_became_active":"2024-09-21T04:03:30.571414+0000","last_became_peered":"2024-09-21T04:03:30.571414+0000","last_unstale":"2024-09-21T04:03:30.581750+0000","last_undegraded":"2024-09-21T04:03:30.581750+0000","last_fullsized":"2024-09-21T04:03:30.581750+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:49.003453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9],"acting":[4,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.565861+0000","last_change":"2024-09-21T04:02:02.771537+0000","last_active":"2024-09-21T04:03:30.565861+0000","last_peered":"2024-09-21T04:03:30.565861+0000","last_clean":"2024-09-21T04:03:30.565861+0000","last_became_active":"2024-09-21T04:02:02.771151+0000","last_became_peered":"2024-09-21T04:02:02.771151+0000","last_unstale":"2024-09-21T04:03:30.565861+0000","last_undegraded":"2024-09-21T04:03:30.565861+0000","last_fullsized":"2024-09-21T04:03:30.565861+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:21.283515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"73'3","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847919+0000","last_change":"2024-09-21T04:02:30.022634+0000","last_active":"2024-09-21T04:03:30.847919+0000","last_peered":"2024-09-21T04:03:30.847919+0000","last_clean":"2024-09-21T04:03:30.847919+0000","last_became_active":"2024-09-21T04:02:30.018076+0000","last_became_peered":"2024-09-21T04:02:30.018076+0000","last_unstale":"2024-09-21T04:03:30.847919+0000","last_undegraded":"2024-09-21T04:03:30.847919+0000","last_fullsized":"2024-09-21T04:03:30.847919+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:31.650092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'5","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.910802+0000","last_change":"2024-09-21T04:02:02.755223+0000","last_active":"2024-09-21T04:03:30.910802+0000","last_peered":"2024-09-21T04:03:30.910802+0000","last_clean":"2024-09-21T04:03:30.910802+0000","last_became_active":"2024-09-21T04:02:02.754836+0000","last_became_peered":"2024-09-21T04:02:02.754836+0000","last_unstale":"2024-09-21T04:03:30.910802+0000","last_undegraded":"2024-09-21T04:03:30.910802+0000","last_fullsized":"2024-09-21T04:03:30.910802+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:57.912690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'5","reported_seq":131,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:49.959127+0000","last_change":"2024-09-21T04:02:00.851689+0000","last_active":"2024-09-21T04:03:49.959127+0000","last_peered":"2024-09-21T04:03:49.959127+0000","last_clean":"2024-09-21T04:03:49.959127+0000","last_became_active":"2024-09-21T04:02:00.851241+0000","last_became_peered":"2024-09-21T04:02:00.851241+0000","last_unstale":"2024-09-21T04:03:49.959127+0000","last_undegraded":"2024-09-21T04:03:49.959127+0000","last_fullsized":"2024-09-21T04:03:49.959127+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:14.697695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575280+0000","last_change":"2024-09-21T04:02:02.774750+0000","last_active":"2024-09-21T04:03:30.575280+0000","last_peered":"2024-09-21T04:03:30.575280+0000","last_clean":"2024-09-21T04:03:30.575280+0000","last_became_active":"2024-09-21T04:02:02.774310+0000","last_became_peered":"2024-09-21T04:02:02.774310+0000","last_unstale":"2024-09-21T04:03:30.575280+0000","last_undegraded":"2024-09-21T04:03:30.575280+0000","last_fullsized":"2024-09-21T04:03:30.575280+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:07.909035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"73'1","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847647+0000","last_change":"2024-09-21T04:02:00.741461+0000","last_active":"2024-09-21T04:03:30.847647+0000","last_peered":"2024-09-21T04:03:30.847647+0000","last_clean":"2024-09-21T04:03:30.847647+0000","last_became_active":"2024-09-21T04:02:00.741051+0000","last_became_peered":"2024-09-21T04:02:00.741051+0000","last_unstale":"2024-09-21T04:03:30.847647+0000","last_undegraded":"2024-09-21T04:03:30.847647+0000","last_fullsized":"2024-09-21T04:03:30.847647+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:05.944430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.570031+0000","last_change":"2024-09-21T04:02:02.761703+0000","last_active":"2024-09-21T04:03:30.570031+0000","last_peered":"2024-09-21T04:03:30.570031+0000","last_clean":"2024-09-21T04:03:30.570031+0000","last_became_active":"2024-09-21T04:02:02.761296+0000","last_became_peered":"2024-09-21T04:02:02.761296+0000","last_unstale":"2024-09-21T04:03:30.570031+0000","last_undegraded":"2024-09-21T04:03:30.570031+0000","last_fullsized":"2024-09-21T04:03:30.570031+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:57.337167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569828+0000","last_change":"2024-09-21T04:02:00.853584+0000","last_active":"2024-09-21T04:03:30.569828+0000","last_peered":"2024-09-21T04:03:30.569828+0000","last_clean":"2024-09-21T04:03:30.569828+0000","last_became_active":"2024-09-21T04:02:00.853019+0000","last_became_peered":"2024-09-21T04:02:00.853019+0000","last_unstale":"2024-09-21T04:03:30.569828+0000","last_undegraded":"2024-09-21T04:03:30.569828+0000","last_fullsized":"2024-09-21T04:03:30.569828+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:56.186791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'5","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569889+0000","last_change":"2024-09-21T04:02:02.758175+0000","last_active":"2024-09-21T04:03:30.569889+0000","last_peered":"2024-09-21T04:03:30.569889+0000","last_clean":"2024-09-21T04:03:30.569889+0000","last_became_active":"2024-09-21T04:02:02.757715+0000","last_became_peered":"2024-09-21T04:02:02.757715+0000","last_unstale":"2024-09-21T04:03:30.569889+0000","last_undegraded":"2024-09-21T04:03:30.569889+0000","last_fullsized":"2024-09-21T04:03:30.569889+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:40.172569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"73'1","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.572814+0000","last_change":"2024-09-21T04:02:00.853325+0000","last_active":"2024-09-21T04:03:30.572814+0000","last_peered":"2024-09-21T04:03:30.572814+0000","last_clean":"2024-09-21T04:03:30.572814+0000","last_became_active":"2024-09-21T04:02:00.852834+0000","last_became_peered":"2024-09-21T04:02:00.852834+0000","last_unstale":"2024-09-21T04:03:30.572814+0000","last_undegraded":"2024-09-21T04:03:30.572814+0000","last_fullsized":"2024-09-21T04:03:30.572814+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:16.090493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"75'9","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.910869+0000","last_change":"2024-09-21T04:02:02.752402+0000","last_active":"2024-09-21T04:03:30.910869+0000","last_peered":"2024-09-21T04:03:30.910869+0000","last_clean":"2024-09-21T04:03:30.910869+0000","last_became_active":"2024-09-21T04:02:02.751994+0000","last_became_peered":"2024-09-21T04:02:02.751994+0000","last_unstale":"2024-09-21T04:03:30.910869+0000","last_undegraded":"2024-09-21T04:03:30.910869+0000","last_fullsized":"2024-09-21T04:03:30.910869+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:19.525317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'14","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:49.767459+0000","last_change":"2024-09-21T04:02:00.739112+0000","last_active":"2024-09-21T04:03:49.767459+0000","last_peered":"2024-09-21T04:03:49.767459+0000","last_clean":"2024-09-21T04:03:49.767459+0000","last_became_active":"2024-09-21T04:02:00.738613+0000","last_became_peered":"2024-09-21T04:02:00.738613+0000","last_unstale":"2024-09-21T04:03:49.767459+0000","last_undegraded":"2024-09-21T04:03:49.767459+0000","last_fullsized":"2024-09-21T04:03:49.767459+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:34.334576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"75'5","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.565764+0000","last_change":"2024-09-21T04:02:02.763345+0000","last_active":"2024-09-21T04:03:30.565764+0000","last_peered":"2024-09-21T04:03:30.565764+0000","last_clean":"2024-09-21T04:03:30.565764+0000","last_became_active":"2024-09-21T04:02:02.762987+0000","last_became_peered":"2024-09-21T04:02:02.762987+0000","last_unstale":"2024-09-21T04:03:30.565764+0000","last_undegraded":"2024-09-21T04:03:30.565764+0000","last_fullsized":"2024-09-21T04:03:30.565764+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:36.239267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"73'4","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.567663+0000","last_change":"2024-09-21T04:02:01.258208+0000","last_active":"2024-09-21T04:03:30.567663+0000","last_peered":"2024-09-21T04:03:30.567663+0000","last_clean":"2024-09-21T04:03:30.567663+0000","last_became_active":"2024-09-21T04:02:01.257780+0000","last_became_peered":"2024-09-21T04:02:01.257780+0000","last_unstale":"2024-09-21T04:03:30.567663+0000","last_undegraded":"2024-09-21T04:03:30.567663+0000","last_fullsized":"2024-09-21T04:03:30.567663+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:20.161792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569870+0000","last_change":"2024-09-21T04:02:02.761149+0000","last_active":"2024-09-21T04:03:30.569870+0000","last_peered":"2024-09-21T04:03:30.569870+0000","last_clean":"2024-09-21T04:03:30.569870+0000","last_became_active":"2024-09-21T04:02:02.760771+0000","last_became_peered":"2024-09-21T04:02:02.760771+0000","last_unstale":"2024-09-21T04:03:30.569870+0000","last_undegraded":"2024-09-21T04:03:30.569870+0000","last_fullsized":"2024-09-21T04:03:30.569870+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:20.645377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"73'2","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575047+0000","last_change":"2024-09-21T04:02:00.853580+0000","last_active":"2024-09-21T04:03:30.575047+0000","last_peered":"2024-09-21T04:03:30.575047+0000","last_clean":"2024-09-21T04:03:30.575047+0000","last_became_active":"2024-09-21T04:02:00.853091+0000","last_became_peered":"2024-09-21T04:02:00.853091+0000","last_unstale":"2024-09-21T04:03:30.575047+0000","last_undegraded":"2024-09-21T04:03:30.575047+0000","last_fullsized":"2024-09-21T04:03:30.575047+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:32.893695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"75'2","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.572389+0000","last_change":"2024-09-21T04:02:02.769686+0000","last_active":"2024-09-21T04:03:30.572389+0000","last_peered":"2024-09-21T04:03:30.572389+0000","last_clean":"2024-09-21T04:03:30.572389+0000","last_became_active":"2024-09-21T04:02:02.769293+0000","last_became_peered":"2024-09-21T04:02:02.769293+0000","last_unstale":"2024-09-21T04:03:30.572389+0000","last_undegraded":"2024-09-21T04:03:30.572389+0000","last_fullsized":"2024-09-21T04:03:30.572389+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:10.984624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"73'3","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.565759+0000","last_change":"2024-09-21T04:02:00.741556+0000","last_active":"2024-09-21T04:03:30.565759+0000","last_peered":"2024-09-21T04:03:30.565759+0000","last_clean":"2024-09-21T04:03:30.565759+0000","last_became_active":"2024-09-21T04:02:00.741188+0000","last_became_peered":"2024-09-21T04:02:00.741188+0000","last_unstale":"2024-09-21T04:03:30.565759+0000","last_undegraded":"2024-09-21T04:03:30.565759+0000","last_fullsized":"2024-09-21T04:03:30.565759+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:23.566814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"75'9","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.566789+0000","last_change":"2024-09-21T04:02:02.769163+0000","last_active":"2024-09-21T04:03:30.566789+0000","last_peered":"2024-09-21T04:03:30.566789+0000","last_clean":"2024-09-21T04:03:30.566789+0000","last_became_active":"2024-09-21T04:02:02.768804+0000","last_became_peered":"2024-09-21T04:02:02.768804+0000","last_unstale":"2024-09-21T04:03:30.566789+0000","last_undegraded":"2024-09-21T04:03:30.566789+0000","last_fullsized":"2024-09-21T04:03:30.566789+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:12.576553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"73'1","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569475+0000","last_change":"2024-09-21T04:02:00.743091+0000","last_active":"2024-09-21T04:03:30.569475+0000","last_peered":"2024-09-21T04:03:30.569475+0000","last_clean":"2024-09-21T04:03:30.569475+0000","last_became_active":"2024-09-21T04:02:00.742665+0000","last_became_peered":"2024-09-21T04:02:00.742665+0000","last_unstale":"2024-09-21T04:03:30.569475+0000","last_undegraded":"2024-09-21T04:03:30.569475+0000","last_fullsized":"2024-09-21T04:03:30.569475+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:09.698421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.565845+0000","last_change":"2024-09-21T04:02:02.761985+0000","last_active":"2024-09-21T04:03:30.565845+0000","last_peered":"2024-09-21T04:03:30.565845+0000","last_clean":"2024-09-21T04:03:30.565845+0000","last_became_active":"2024-09-21T04:02:02.761624+0000","last_became_peered":"2024-09-21T04:02:02.761624+0000","last_unstale":"2024-09-21T04:03:30.565845+0000","last_undegraded":"2024-09-21T04:03:30.565845+0000","last_fullsized":"2024-09-21T04:03:30.565845+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:12.912221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"73'4","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.572449+0000","last_change":"2024-09-21T04:02:30.024288+0000","last_active":"2024-09-21T04:03:30.572449+0000","last_peered":"2024-09-21T04:03:30.572449+0000","last_clean":"2024-09-21T04:03:30.572449+0000","last_became_active":"2024-09-21T04:02:30.018929+0000","last_became_peered":"2024-09-21T04:02:30.018929+0000","last_unstale":"2024-09-21T04:03:30.572449+0000","last_undegraded":"2024-09-21T04:03:30.572449+0000","last_fullsized":"2024-09-21T04:03:30.572449+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:19:21.400082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"75'5","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847220+0000","last_change":"2024-09-21T04:02:02.761357+0000","last_active":"2024-09-21T04:03:30.847220+0000","last_peered":"2024-09-21T04:03:30.847220+0000","last_clean":"2024-09-21T04:03:30.847220+0000","last_became_active":"2024-09-21T04:02:02.760984+0000","last_became_peered":"2024-09-21T04:02:02.760984+0000","last_unstale":"2024-09-21T04:03:30.847220+0000","last_undegraded":"2024-09-21T04:03:30.847220+0000","last_fullsized":"2024-09-21T04:03:30.847220+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:55.388768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"75'15","reported_seq":44,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575165+0000","last_change":"2024-09-21T04:02:30.019234+0000","last_active":"2024-09-21T04:03:30.575165+0000","last_peered":"2024-09-21T04:03:30.575165+0000","last_clean":"2024-09-21T04:03:30.575165+0000","last_became_active":"2024-09-21T04:02:30.018778+0000","last_became_peered":"2024-09-21T04:02:30.018778+0000","last_unstale":"2024-09-21T04:03:30.575165+0000","last_undegraded":"2024-09-21T04:03:30.575165+0000","last_fullsized":"2024-09-21T04:03:30.575165+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:31.227144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"73'2","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.915768+0000","last_change":"2024-09-21T04:03:30.915685+0000","last_active":"2024-09-21T04:03:30.915768+0000","last_peered":"2024-09-21T04:03:30.915768+0000","last_clean":"2024-09-21T04:03:30.915768+0000","last_became_active":"2024-09-21T04:03:30.912802+0000","last_became_peered":"2024-09-21T04:03:30.912802+0000","last_unstale":"2024-09-21T04:03:30.915768+0000","last_undegraded":"2024-09-21T04:03:30.915768+0000","last_fullsized":"2024-09-21T04:03:30.915768+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:15:58.295051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,6],"acting":[8,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"75'9","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575216+0000","last_change":"2024-09-21T04:02:02.762336+0000","last_active":"2024-09-21T04:03:30.575216+0000","last_peered":"2024-09-21T04:03:30.575216+0000","last_clean":"2024-09-21T04:03:30.575216+0000","last_became_active":"2024-09-21T04:02:02.761753+0000","last_became_peered":"2024-09-21T04:02:02.761753+0000","last_unstale":"2024-09-21T04:03:30.575216+0000","last_undegraded":"2024-09-21T04:03:30.575216+0000","last_fullsized":"2024-09-21T04:03:30.575216+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:39.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"73'4","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.572719+0000","last_change":"2024-09-21T04:02:00.743495+0000","last_active":"2024-09-21T04:03:30.572719+0000","last_peered":"2024-09-21T04:03:30.572719+0000","last_clean":"2024-09-21T04:03:30.572719+0000","last_became_active":"2024-09-21T04:02:00.743135+0000","last_became_peered":"2024-09-21T04:02:00.743135+0000","last_unstale":"2024-09-21T04:03:30.572719+0000","last_undegraded":"2024-09-21T04:03:30.572719+0000","last_fullsized":"2024-09-21T04:03:30.572719+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:16.481148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"75'6","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569827+0000","last_change":"2024-09-21T04:02:02.766682+0000","last_active":"2024-09-21T04:03:30.569827+0000","last_peered":"2024-09-21T04:03:30.569827+0000","last_clean":"2024-09-21T04:03:30.569827+0000","last_became_active":"2024-09-21T04:02:02.766299+0000","last_became_peered":"2024-09-21T04:02:02.766299+0000","last_unstale":"2024-09-21T04:03:30.569827+0000","last_undegraded":"2024-09-21T04:03:30.569827+0000","last_fullsized":"2024-09-21T04:03:30.569827+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:25.635908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"73'2","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.566238+0000","last_change":"2024-09-21T04:02:00.852281+0000","last_active":"2024-09-21T04:03:30.566238+0000","last_peered":"2024-09-21T04:03:30.566238+0000","last_clean":"2024-09-21T04:03:30.566238+0000","last_became_active":"2024-09-21T04:02:00.851900+0000","last_became_peered":"2024-09-21T04:02:00.851900+0000","last_unstale":"2024-09-21T04:03:30.566238+0000","last_undegraded":"2024-09-21T04:03:30.566238+0000","last_fullsized":"2024-09-21T04:03:30.566238+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:37.115541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"75'5","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.566108+0000","last_change":"2024-09-21T04:02:02.770467+0000","last_active":"2024-09-21T04:03:30.566108+0000","last_peered":"2024-09-21T04:03:30.566108+0000","last_clean":"2024-09-21T04:03:30.566108+0000","last_became_active":"2024-09-21T04:02:02.770099+0000","last_became_peered":"2024-09-21T04:02:02.770099+0000","last_unstale":"2024-09-21T04:03:30.566108+0000","last_undegraded":"2024-09-21T04:03:30.566108+0000","last_fullsized":"2024-09-21T04:03:30.566108+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:38.806298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"73'9","reported_seq":54,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.570015+0000","last_change":"2024-09-21T04:02:29.878053+0000","last_active":"2024-09-21T04:03:30.570015+0000","last_peered":"2024-09-21T04:03:30.570015+0000","last_clean":"2024-09-21T04:03:30.570015+0000","last_became_active":"2024-09-21T04:02:29.869007+0000","last_became_peered":"2024-09-21T04:02:29.869007+0000","last_unstale":"2024-09-21T04:03:30.570015+0000","last_undegraded":"2024-09-21T04:03:30.570015+0000","last_fullsized":"2024-09-21T04:03:30.570015+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:50.509849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75105,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":625,"num_read_kb":392,"num_write":179,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1198,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":217008,"data_compressed":5676,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282548,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808867,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":847872,"data_stored":424484,"data_compressed":11589,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":62,"seq":266287972392,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":1087201,"data_compressed":49407,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135918,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":856064,"data_stored":424104,"data_compressed":11589,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332147,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":856064,"data_stored":424024,"data_compressed":11589,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495671,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":1016071,"data_compressed":47515,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659197,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":843776,"data_stored":423493,"data_compressed":11589,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855426,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":1015247,"data_compressed":47515,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018951,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":424820,"data_compressed":11589,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":860160,"data_stored":423804,"data_compressed":11589,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378705,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":929792,"data_stored":490501,"data_compressed":13481,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542230,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":489733,"data_compressed":13481,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705755,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":864256,"data_stored":424764,"data_compressed":11589,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34306,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34306,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34306,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":66447,"data_compressed":1892,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67215,"data_compressed":1892,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":73430,"data_compressed":1892,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T04:03:54.474 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T04:03:54.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.622+0000 7f4c651f1640 1 -- 172.21.15.64:0/2556896607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c60105220 msgr2=0x7f4c6010d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:54.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.622+0000 7f4c651f1640 1 --2- 172.21.15.64:0/2556896607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c60105220 0x7f4c6010d760 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4c54009900 tx=0x7f4c5402a410 comp rx=0 tx=0).stop 2024-09-21T04:03:54.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.622+0000 7f4c651f1640 1 -- 172.21.15.64:0/2556896607 shutdown_connections 2024-09-21T04:03:54.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.622+0000 7f4c651f1640 1 --2- 172.21.15.64:0/2556896607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c6010dd30 0x7f4c60110220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:54.626 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.622+0000 7f4c651f1640 1 --2- 172.21.15.64:0/2556896607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c60105220 0x7f4c6010d760 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:54.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.622+0000 7f4c651f1640 1 --2- 172.21.15.64:0/2556896607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c601048d0 0x7f4c60104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:54.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.622+0000 7f4c651f1640 1 -- 172.21.15.64:0/2556896607 >> 172.21.15.64:0/2556896607 conn(0x7f4c600fe380 msgr2=0x7f4c601007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:54.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.622+0000 7f4c651f1640 1 -- 172.21.15.64:0/2556896607 shutdown_connections 2024-09-21T04:03:54.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.622+0000 7f4c651f1640 1 -- 172.21.15.64:0/2556896607 wait complete. 2024-09-21T04:03:54.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.623+0000 7f4c651f1640 1 Processor -- start 2024-09-21T04:03:54.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.623+0000 7f4c651f1640 1 -- start start 2024-09-21T04:03:54.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.623+0000 7f4c651f1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c601048d0 0x7f4c6019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:54.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.624+0000 7f4c651f1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c60105220 0x7f4c6019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:54.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.624+0000 7f4c651f1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c6010dd30 0x7f4c6019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:54.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.624+0000 7f4c651f1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4c60113990 con 0x7f4c6010dd30 2024-09-21T04:03:54.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.624+0000 7f4c651f1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4c60113810 con 0x7f4c60105220 2024-09-21T04:03:54.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.624+0000 7f4c651f1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4c60113b10 con 0x7f4c601048d0 2024-09-21T04:03:54.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.624+0000 7f4c5f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c6010dd30 0x7f4c6019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:54.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.624+0000 7f4c5f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c6010dd30 0x7f4c6019fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49978/0 (socket says 172.21.15.64:49978) 2024-09-21T04:03:54.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.624+0000 7f4c5f59e640 1 -- 172.21.15.64:0/611616851 learned_addr learned my addr 172.21.15.64:0/611616851 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:54.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.624+0000 7f4c5ed9d640 1 --2- 172.21.15.64:0/611616851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c601048d0 0x7f4c6019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:54.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.624+0000 7f4c5e59c640 1 --2- 172.21.15.64:0/611616851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c60105220 0x7f4c6019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:54.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.624+0000 7f4c5f59e640 1 -- 172.21.15.64:0/611616851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c601048d0 msgr2=0x7f4c6019a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:54.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.624+0000 7f4c5f59e640 1 --2- 172.21.15.64:0/611616851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c601048d0 0x7f4c6019a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:54.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.624+0000 7f4c5f59e640 1 -- 172.21.15.64:0/611616851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c60105220 msgr2=0x7f4c6019a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:54.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.624+0000 7f4c5f59e640 1 --2- 172.21.15.64:0/611616851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c60105220 0x7f4c6019a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:54.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.624+0000 7f4c5f59e640 1 -- 172.21.15.64:0/611616851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c601a0300 con 0x7f4c6010dd30 2024-09-21T04:03:54.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.624+0000 7f4c5f59e640 1 --2- 172.21.15.64:0/611616851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c6010dd30 0x7f4c6019fc00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4c5800eb10 tx=0x7f4c5800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:54.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.624+0000 7f4c47fff640 1 -- 172.21.15.64:0/611616851 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c58004b20 con 0x7f4c6010dd30 2024-09-21T04:03:54.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.624+0000 7f4c47fff640 1 -- 172.21.15.64:0/611616851 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c58004cc0 con 0x7f4c6010dd30 2024-09-21T04:03:54.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.624+0000 7f4c651f1640 1 -- 172.21.15.64:0/611616851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c601a05f0 con 0x7f4c6010dd30 2024-09-21T04:03:54.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.624+0000 7f4c47fff640 1 -- 172.21.15.64:0/611616851 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c58010670 con 0x7f4c6010dd30 2024-09-21T04:03:54.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.625+0000 7f4c651f1640 1 -- 172.21.15.64:0/611616851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c601a6d80 con 0x7f4c6010dd30 2024-09-21T04:03:54.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.626+0000 7f4c651f1640 1 -- 172.21.15.64:0/611616851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c6006ae20 con 0x7f4c6010dd30 2024-09-21T04:03:54.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.627+0000 7f4c47fff640 1 -- 172.21.15.64:0/611616851 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) ==== 154580+0+0 (secure 0 0 0) 0x7f4c58005020 con 0x7f4c6010dd30 2024-09-21T04:03:54.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.628+0000 7f4c47fff640 1 --2- 172.21.15.64:0/611616851 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4c480b65f0 0x7f4c480b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:54.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.628+0000 7f4c5ed9d640 1 --2- 172.21.15.64:0/611616851 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4c480b65f0 0x7f4c480b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:54.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.628+0000 7f4c47fff640 1 -- 172.21.15.64:0/611616851 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f4c580e7bc0 con 0x7f4c6010dd30 2024-09-21T04:03:54.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.629+0000 7f4c5ed9d640 1 --2- 172.21.15.64:0/611616851 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4c480b65f0 0x7f4c480b8b00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4c50005d00 tx=0x7f4c5000b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:54.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.633+0000 7f4c47fff640 1 -- 172.21.15.64:0/611616851 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4c580ae1a0 con 0x7f4c6010dd30 2024-09-21T04:03:54.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.821+0000 7f4c651f1640 1 -- 172.21.15.64:0/611616851 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4c60106e40 con 0x7f4c480b65f0 2024-09-21T04:03:54.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.826+0000 7f4c47fff640 1 -- 172.21.15.64:0/611616851 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259137 (secure 0 0 0) 0x7f4c60106e40 con 0x7f4c480b65f0 2024-09-21T04:03:54.831 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:03:54.833 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-09-21T04:03:54.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.829+0000 7f4c651f1640 1 -- 172.21.15.64:0/611616851 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4c480b65f0 msgr2=0x7f4c480b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:54.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.829+0000 7f4c651f1640 1 --2- 172.21.15.64:0/611616851 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4c480b65f0 0x7f4c480b8b00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4c50005d00 tx=0x7f4c5000b410 comp rx=0 tx=0).stop 2024-09-21T04:03:54.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.829+0000 7f4c651f1640 1 -- 172.21.15.64:0/611616851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c6010dd30 msgr2=0x7f4c6019fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:54.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.829+0000 7f4c651f1640 1 --2- 172.21.15.64:0/611616851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c6010dd30 0x7f4c6019fc00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4c5800eb10 tx=0x7f4c5800c2d0 comp rx=0 tx=0).stop 2024-09-21T04:03:54.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.829+0000 7f4c651f1640 1 -- 172.21.15.64:0/611616851 shutdown_connections 2024-09-21T04:03:54.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.829+0000 7f4c651f1640 1 --2- 172.21.15.64:0/611616851 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4c480b65f0 0x7f4c480b8b00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:54.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.829+0000 7f4c651f1640 1 --2- 172.21.15.64:0/611616851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c6010dd30 0x7f4c6019fc00 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:54.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.829+0000 7f4c651f1640 1 --2- 172.21.15.64:0/611616851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c60105220 0x7f4c6019a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:54.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.829+0000 7f4c651f1640 1 --2- 172.21.15.64:0/611616851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4c601048d0 0x7f4c6019a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:54.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.829+0000 7f4c651f1640 1 -- 172.21.15.64:0/611616851 >> 172.21.15.64:0/611616851 conn(0x7f4c600fe380 msgr2=0x7f4c60106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:54.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.829+0000 7f4c651f1640 1 -- 172.21.15.64:0/611616851 shutdown_connections 2024-09-21T04:03:54.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.829+0000 7f4c651f1640 1 -- 172.21.15.64:0/611616851 wait complete. 2024-09-21T04:03:54.844 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-09-21T04:03:54.506243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665707,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":706,"num_read_kb":461,"num_write":304,"num_write_kb":2284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":2953038,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":745,"ondisk_log_size":745,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359652,"kb_used_data":10384,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124582172,"statfs":{"total":1151940427776,"available":1151572144128,"internally_reserved":0,"allocated":10633216,"data_stored":7068246,"data_compressed":252522,"data_compressed_allocated":344064,"data_compressed_original":2187264,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5356,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":37,"num_write":12,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003590"},"pg_stats":[{"pgid":"3.3f","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.567004+0000","last_change":"2024-09-21T04:02:02.769848+0000","last_active":"2024-09-21T04:03:30.567004+0000","last_peered":"2024-09-21T04:03:30.567004+0000","last_clean":"2024-09-21T04:03:30.567004+0000","last_became_active":"2024-09-21T04:02:02.769508+0000","last_became_peered":"2024-09-21T04:02:02.769508+0000","last_unstale":"2024-09-21T04:03:30.567004+0000","last_undegraded":"2024-09-21T04:03:30.567004+0000","last_fullsized":"2024-09-21T04:03:30.567004+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:00.242233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"75'4","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569630+0000","last_change":"2024-09-21T04:02:02.769200+0000","last_active":"2024-09-21T04:03:30.569630+0000","last_peered":"2024-09-21T04:03:30.569630+0000","last_clean":"2024-09-21T04:03:30.569630+0000","last_became_active":"2024-09-21T04:02:02.768918+0000","last_became_peered":"2024-09-21T04:02:02.768918+0000","last_unstale":"2024-09-21T04:03:30.569630+0000","last_undegraded":"2024-09-21T04:03:30.569630+0000","last_fullsized":"2024-09-21T04:03:30.569630+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:26.842079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569334+0000","last_change":"2024-09-21T04:02:02.763511+0000","last_active":"2024-09-21T04:03:30.569334+0000","last_peered":"2024-09-21T04:03:30.569334+0000","last_clean":"2024-09-21T04:03:30.569334+0000","last_became_active":"2024-09-21T04:02:02.763161+0000","last_became_peered":"2024-09-21T04:02:02.763161+0000","last_unstale":"2024-09-21T04:03:30.569334+0000","last_undegraded":"2024-09-21T04:03:30.569334+0000","last_fullsized":"2024-09-21T04:03:30.569334+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:59.834005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.567466+0000","last_change":"2024-09-21T04:02:02.765452+0000","last_active":"2024-09-21T04:03:30.567466+0000","last_peered":"2024-09-21T04:03:30.567466+0000","last_clean":"2024-09-21T04:03:30.567466+0000","last_became_active":"2024-09-21T04:02:02.765065+0000","last_became_peered":"2024-09-21T04:02:02.765065+0000","last_unstale":"2024-09-21T04:03:30.567466+0000","last_undegraded":"2024-09-21T04:03:30.567466+0000","last_fullsized":"2024-09-21T04:03:30.567466+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:33.871366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"75'4","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.572434+0000","last_change":"2024-09-21T04:02:02.767457+0000","last_active":"2024-09-21T04:03:30.572434+0000","last_peered":"2024-09-21T04:03:30.572434+0000","last_clean":"2024-09-21T04:03:30.572434+0000","last_became_active":"2024-09-21T04:02:02.767064+0000","last_became_peered":"2024-09-21T04:02:02.767064+0000","last_unstale":"2024-09-21T04:03:30.572434+0000","last_undegraded":"2024-09-21T04:03:30.572434+0000","last_fullsized":"2024-09-21T04:03:30.572434+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:46.316029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"75'6","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569752+0000","last_change":"2024-09-21T04:02:02.768795+0000","last_active":"2024-09-21T04:03:30.569752+0000","last_peered":"2024-09-21T04:03:30.569752+0000","last_clean":"2024-09-21T04:03:30.569752+0000","last_became_active":"2024-09-21T04:02:02.768434+0000","last_became_peered":"2024-09-21T04:02:02.768434+0000","last_unstale":"2024-09-21T04:03:30.569752+0000","last_undegraded":"2024-09-21T04:03:30.569752+0000","last_fullsized":"2024-09-21T04:03:30.569752+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:58.991533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"75'5","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847154+0000","last_change":"2024-09-21T04:02:02.774998+0000","last_active":"2024-09-21T04:03:30.847154+0000","last_peered":"2024-09-21T04:03:30.847154+0000","last_clean":"2024-09-21T04:03:30.847154+0000","last_became_active":"2024-09-21T04:02:02.774344+0000","last_became_peered":"2024-09-21T04:02:02.774344+0000","last_unstale":"2024-09-21T04:03:30.847154+0000","last_undegraded":"2024-09-21T04:03:30.847154+0000","last_fullsized":"2024-09-21T04:03:30.847154+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:26.705126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"75'9","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.565533+0000","last_change":"2024-09-21T04:02:02.772498+0000","last_active":"2024-09-21T04:03:30.565533+0000","last_peered":"2024-09-21T04:03:30.565533+0000","last_clean":"2024-09-21T04:03:30.565533+0000","last_became_active":"2024-09-21T04:02:02.772049+0000","last_became_peered":"2024-09-21T04:02:02.772049+0000","last_unstale":"2024-09-21T04:03:30.565533+0000","last_undegraded":"2024-09-21T04:03:30.565533+0000","last_fullsized":"2024-09-21T04:03:30.565533+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:31:19.088424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"75'11","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575591+0000","last_change":"2024-09-21T04:02:02.772336+0000","last_active":"2024-09-21T04:03:30.575591+0000","last_peered":"2024-09-21T04:03:30.575591+0000","last_clean":"2024-09-21T04:03:30.575591+0000","last_became_active":"2024-09-21T04:02:02.771964+0000","last_became_peered":"2024-09-21T04:02:02.771964+0000","last_unstale":"2024-09-21T04:03:30.575591+0000","last_undegraded":"2024-09-21T04:03:30.575591+0000","last_fullsized":"2024-09-21T04:03:30.575591+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:46.659690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"75'16","reported_seq":38,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.570342+0000","last_change":"2024-09-21T04:02:02.758992+0000","last_active":"2024-09-21T04:03:30.570342+0000","last_peered":"2024-09-21T04:03:30.570342+0000","last_clean":"2024-09-21T04:03:30.570342+0000","last_became_active":"2024-09-21T04:02:02.758575+0000","last_became_peered":"2024-09-21T04:02:02.758575+0000","last_unstale":"2024-09-21T04:03:30.570342+0000","last_undegraded":"2024-09-21T04:03:30.570342+0000","last_fullsized":"2024-09-21T04:03:30.570342+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:32:09.543140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"75'6","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847480+0000","last_change":"2024-09-21T04:02:02.767304+0000","last_active":"2024-09-21T04:03:30.847480+0000","last_peered":"2024-09-21T04:03:30.847480+0000","last_clean":"2024-09-21T04:03:30.847480+0000","last_became_active":"2024-09-21T04:02:02.762804+0000","last_became_peered":"2024-09-21T04:02:02.762804+0000","last_unstale":"2024-09-21T04:03:30.847480+0000","last_undegraded":"2024-09-21T04:03:30.847480+0000","last_fullsized":"2024-09-21T04:03:30.847480+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:32.224071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.567991+0000","last_change":"2024-09-21T04:02:02.762932+0000","last_active":"2024-09-21T04:03:30.567991+0000","last_peered":"2024-09-21T04:03:30.567991+0000","last_clean":"2024-09-21T04:03:30.567991+0000","last_became_active":"2024-09-21T04:02:02.762538+0000","last_became_peered":"2024-09-21T04:02:02.762538+0000","last_unstale":"2024-09-21T04:03:30.567991+0000","last_undegraded":"2024-09-21T04:03:30.567991+0000","last_fullsized":"2024-09-21T04:03:30.567991+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:20.724453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"75'10","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.910867+0000","last_change":"2024-09-21T04:02:02.770313+0000","last_active":"2024-09-21T04:03:30.910867+0000","last_peered":"2024-09-21T04:03:30.910867+0000","last_clean":"2024-09-21T04:03:30.910867+0000","last_became_active":"2024-09-21T04:02:02.769910+0000","last_became_peered":"2024-09-21T04:02:02.769910+0000","last_unstale":"2024-09-21T04:03:30.910867+0000","last_undegraded":"2024-09-21T04:03:30.910867+0000","last_fullsized":"2024-09-21T04:03:30.910867+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:16.228371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569300+0000","last_change":"2024-09-21T04:02:02.769258+0000","last_active":"2024-09-21T04:03:30.569300+0000","last_peered":"2024-09-21T04:03:30.569300+0000","last_clean":"2024-09-21T04:03:30.569300+0000","last_became_active":"2024-09-21T04:02:02.768914+0000","last_became_peered":"2024-09-21T04:02:02.768914+0000","last_unstale":"2024-09-21T04:03:30.569300+0000","last_undegraded":"2024-09-21T04:03:30.569300+0000","last_fullsized":"2024-09-21T04:03:30.569300+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:37.426810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569216+0000","last_change":"2024-09-21T04:02:02.755940+0000","last_active":"2024-09-21T04:03:30.569216+0000","last_peered":"2024-09-21T04:03:30.569216+0000","last_clean":"2024-09-21T04:03:30.569216+0000","last_became_active":"2024-09-21T04:02:02.755309+0000","last_became_peered":"2024-09-21T04:02:02.755309+0000","last_unstale":"2024-09-21T04:03:30.569216+0000","last_undegraded":"2024-09-21T04:03:30.569216+0000","last_fullsized":"2024-09-21T04:03:30.569216+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:49.834030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"75'9","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.574842+0000","last_change":"2024-09-21T04:02:02.761068+0000","last_active":"2024-09-21T04:03:30.574842+0000","last_peered":"2024-09-21T04:03:30.574842+0000","last_clean":"2024-09-21T04:03:30.574842+0000","last_became_active":"2024-09-21T04:02:02.760683+0000","last_became_peered":"2024-09-21T04:02:02.760683+0000","last_unstale":"2024-09-21T04:03:30.574842+0000","last_undegraded":"2024-09-21T04:03:30.574842+0000","last_fullsized":"2024-09-21T04:03:30.574842+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:36.627539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.567055+0000","last_change":"2024-09-21T04:02:02.772234+0000","last_active":"2024-09-21T04:03:30.567055+0000","last_peered":"2024-09-21T04:03:30.567055+0000","last_clean":"2024-09-21T04:03:30.567055+0000","last_became_active":"2024-09-21T04:02:02.771825+0000","last_became_peered":"2024-09-21T04:02:02.771825+0000","last_unstale":"2024-09-21T04:03:30.567055+0000","last_undegraded":"2024-09-21T04:03:30.567055+0000","last_fullsized":"2024-09-21T04:03:30.567055+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:33.449745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"75'10","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569735+0000","last_change":"2024-09-21T04:02:02.758643+0000","last_active":"2024-09-21T04:03:30.569735+0000","last_peered":"2024-09-21T04:03:30.569735+0000","last_clean":"2024-09-21T04:03:30.569735+0000","last_became_active":"2024-09-21T04:02:02.758261+0000","last_became_peered":"2024-09-21T04:02:02.758261+0000","last_unstale":"2024-09-21T04:03:30.569735+0000","last_undegraded":"2024-09-21T04:03:30.569735+0000","last_fullsized":"2024-09-21T04:03:30.569735+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:04.165399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569847+0000","last_change":"2024-09-21T04:02:02.759405+0000","last_active":"2024-09-21T04:03:30.569847+0000","last_peered":"2024-09-21T04:03:30.569847+0000","last_clean":"2024-09-21T04:03:30.569847+0000","last_became_active":"2024-09-21T04:02:02.758962+0000","last_became_peered":"2024-09-21T04:02:02.758962+0000","last_unstale":"2024-09-21T04:03:30.569847+0000","last_undegraded":"2024-09-21T04:03:30.569847+0000","last_fullsized":"2024-09-21T04:03:30.569847+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:37.203308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.566627+0000","last_change":"2024-09-21T04:02:02.767518+0000","last_active":"2024-09-21T04:03:30.566627+0000","last_peered":"2024-09-21T04:03:30.566627+0000","last_clean":"2024-09-21T04:03:30.566627+0000","last_became_active":"2024-09-21T04:02:02.767147+0000","last_became_peered":"2024-09-21T04:02:02.767147+0000","last_unstale":"2024-09-21T04:03:30.566627+0000","last_undegraded":"2024-09-21T04:03:30.566627+0000","last_fullsized":"2024-09-21T04:03:30.566627+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:34:13.615218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"75'11","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.910844+0000","last_change":"2024-09-21T04:02:02.756407+0000","last_active":"2024-09-21T04:03:30.910844+0000","last_peered":"2024-09-21T04:03:30.910844+0000","last_clean":"2024-09-21T04:03:30.910844+0000","last_became_active":"2024-09-21T04:02:02.756007+0000","last_became_peered":"2024-09-21T04:02:02.756007+0000","last_unstale":"2024-09-21T04:03:30.910844+0000","last_undegraded":"2024-09-21T04:03:30.910844+0000","last_fullsized":"2024-09-21T04:03:30.910844+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:18.798814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"75'10","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575347+0000","last_change":"2024-09-21T04:02:02.766290+0000","last_active":"2024-09-21T04:03:30.575347+0000","last_peered":"2024-09-21T04:03:30.575347+0000","last_clean":"2024-09-21T04:03:30.575347+0000","last_became_active":"2024-09-21T04:02:02.765660+0000","last_became_peered":"2024-09-21T04:02:02.765660+0000","last_unstale":"2024-09-21T04:03:30.575347+0000","last_undegraded":"2024-09-21T04:03:30.575347+0000","last_fullsized":"2024-09-21T04:03:30.575347+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:23.281476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.565585+0000","last_change":"2024-09-21T04:02:02.767943+0000","last_active":"2024-09-21T04:03:30.565585+0000","last_peered":"2024-09-21T04:03:30.565585+0000","last_clean":"2024-09-21T04:03:30.565585+0000","last_became_active":"2024-09-21T04:02:02.767566+0000","last_became_peered":"2024-09-21T04:02:02.767566+0000","last_unstale":"2024-09-21T04:03:30.565585+0000","last_undegraded":"2024-09-21T04:03:30.565585+0000","last_fullsized":"2024-09-21T04:03:30.565585+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:44.221531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"75'6","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.574737+0000","last_change":"2024-09-21T04:02:02.770114+0000","last_active":"2024-09-21T04:03:30.574737+0000","last_peered":"2024-09-21T04:03:30.574737+0000","last_clean":"2024-09-21T04:03:30.574737+0000","last_became_active":"2024-09-21T04:02:02.769732+0000","last_became_peered":"2024-09-21T04:02:02.769732+0000","last_unstale":"2024-09-21T04:03:30.574737+0000","last_undegraded":"2024-09-21T04:03:30.574737+0000","last_fullsized":"2024-09-21T04:03:30.574737+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:38.396288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"75'2","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569686+0000","last_change":"2024-09-21T04:02:30.017325+0000","last_active":"2024-09-21T04:03:30.569686+0000","last_peered":"2024-09-21T04:03:30.569686+0000","last_clean":"2024-09-21T04:03:30.569686+0000","last_became_active":"2024-09-21T04:02:30.017014+0000","last_became_peered":"2024-09-21T04:02:30.017014+0000","last_unstale":"2024-09-21T04:03:30.569686+0000","last_undegraded":"2024-09-21T04:03:30.569686+0000","last_fullsized":"2024-09-21T04:03:30.569686+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:28.579436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.26","version":"75'15","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569995+0000","last_change":"2024-09-21T04:02:02.764666+0000","last_active":"2024-09-21T04:03:30.569995+0000","last_peered":"2024-09-21T04:03:30.569995+0000","last_clean":"2024-09-21T04:03:30.569995+0000","last_became_active":"2024-09-21T04:02:02.763931+0000","last_became_peered":"2024-09-21T04:02:02.763931+0000","last_unstale":"2024-09-21T04:03:30.569995+0000","last_undegraded":"2024-09-21T04:03:30.569995+0000","last_fullsized":"2024-09-21T04:03:30.569995+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:39.724770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"75'6","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.566850+0000","last_change":"2024-09-21T04:02:02.774276+0000","last_active":"2024-09-21T04:03:30.566850+0000","last_peered":"2024-09-21T04:03:30.566850+0000","last_clean":"2024-09-21T04:03:30.566850+0000","last_became_active":"2024-09-21T04:02:02.773852+0000","last_became_peered":"2024-09-21T04:02:02.773852+0000","last_unstale":"2024-09-21T04:03:30.566850+0000","last_undegraded":"2024-09-21T04:03:30.566850+0000","last_fullsized":"2024-09-21T04:03:30.566850+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:40.072264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"75'4","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.911090+0000","last_change":"2024-09-21T04:02:02.758520+0000","last_active":"2024-09-21T04:03:30.911090+0000","last_peered":"2024-09-21T04:03:30.911090+0000","last_clean":"2024-09-21T04:03:30.911090+0000","last_became_active":"2024-09-21T04:02:02.757656+0000","last_became_peered":"2024-09-21T04:02:02.757656+0000","last_unstale":"2024-09-21T04:03:30.911090+0000","last_undegraded":"2024-09-21T04:03:30.911090+0000","last_fullsized":"2024-09-21T04:03:30.911090+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:30.854336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569551+0000","last_change":"2024-09-21T04:02:02.756428+0000","last_active":"2024-09-21T04:03:30.569551+0000","last_peered":"2024-09-21T04:03:30.569551+0000","last_clean":"2024-09-21T04:03:30.569551+0000","last_became_active":"2024-09-21T04:02:02.755985+0000","last_became_peered":"2024-09-21T04:02:02.755985+0000","last_unstale":"2024-09-21T04:03:30.569551+0000","last_undegraded":"2024-09-21T04:03:30.569551+0000","last_fullsized":"2024-09-21T04:03:30.569551+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:36.040436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569536+0000","last_change":"2024-09-21T04:02:02.762893+0000","last_active":"2024-09-21T04:03:30.569536+0000","last_peered":"2024-09-21T04:03:30.569536+0000","last_clean":"2024-09-21T04:03:30.569536+0000","last_became_active":"2024-09-21T04:02:02.762343+0000","last_became_peered":"2024-09-21T04:02:02.762343+0000","last_unstale":"2024-09-21T04:03:30.569536+0000","last_undegraded":"2024-09-21T04:03:30.569536+0000","last_fullsized":"2024-09-21T04:03:30.569536+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:45.074711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"75'12","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569534+0000","last_change":"2024-09-21T04:02:02.763942+0000","last_active":"2024-09-21T04:03:30.569534+0000","last_peered":"2024-09-21T04:03:30.569534+0000","last_clean":"2024-09-21T04:03:30.569534+0000","last_became_active":"2024-09-21T04:02:02.763433+0000","last_became_peered":"2024-09-21T04:02:02.763433+0000","last_unstale":"2024-09-21T04:03:30.569534+0000","last_undegraded":"2024-09-21T04:03:30.569534+0000","last_fullsized":"2024-09-21T04:03:30.569534+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:36.139440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"75'9","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.566588+0000","last_change":"2024-09-21T04:02:02.773364+0000","last_active":"2024-09-21T04:03:30.566588+0000","last_peered":"2024-09-21T04:03:30.566588+0000","last_clean":"2024-09-21T04:03:30.566588+0000","last_became_active":"2024-09-21T04:02:02.773040+0000","last_became_peered":"2024-09-21T04:02:02.773040+0000","last_unstale":"2024-09-21T04:03:30.566588+0000","last_undegraded":"2024-09-21T04:03:30.566588+0000","last_fullsized":"2024-09-21T04:03:30.566588+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:48.504476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.572424+0000","last_change":"2024-09-21T04:02:02.774951+0000","last_active":"2024-09-21T04:03:30.572424+0000","last_peered":"2024-09-21T04:03:30.572424+0000","last_clean":"2024-09-21T04:03:30.572424+0000","last_became_active":"2024-09-21T04:02:02.774525+0000","last_became_peered":"2024-09-21T04:02:02.774525+0000","last_unstale":"2024-09-21T04:03:30.572424+0000","last_undegraded":"2024-09-21T04:03:30.572424+0000","last_fullsized":"2024-09-21T04:03:30.572424+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:24.447860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"73'5","reported_seq":132,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:48.958216+0000","last_change":"2024-09-21T04:02:00.743470+0000","last_active":"2024-09-21T04:03:48.958216+0000","last_peered":"2024-09-21T04:03:48.958216+0000","last_clean":"2024-09-21T04:03:48.958216+0000","last_became_active":"2024-09-21T04:02:00.743055+0000","last_became_peered":"2024-09-21T04:02:00.743055+0000","last_unstale":"2024-09-21T04:03:48.958216+0000","last_undegraded":"2024-09-21T04:03:48.958216+0000","last_fullsized":"2024-09-21T04:03:48.958216+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:56.101753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.574871+0000","last_change":"2024-09-21T04:02:02.773399+0000","last_active":"2024-09-21T04:03:30.574871+0000","last_peered":"2024-09-21T04:03:30.574871+0000","last_clean":"2024-09-21T04:03:30.574871+0000","last_became_active":"2024-09-21T04:02:02.772996+0000","last_became_peered":"2024-09-21T04:02:02.772996+0000","last_unstale":"2024-09-21T04:03:30.574871+0000","last_undegraded":"2024-09-21T04:03:30.574871+0000","last_fullsized":"2024-09-21T04:03:30.574871+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:07.256226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"73'3","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569954+0000","last_change":"2024-09-21T04:02:30.026868+0000","last_active":"2024-09-21T04:03:30.569954+0000","last_peered":"2024-09-21T04:03:30.569954+0000","last_clean":"2024-09-21T04:03:30.569954+0000","last_became_active":"2024-09-21T04:02:30.019469+0000","last_became_peered":"2024-09-21T04:02:30.019469+0000","last_unstale":"2024-09-21T04:03:30.569954+0000","last_undegraded":"2024-09-21T04:03:30.569954+0000","last_fullsized":"2024-09-21T04:03:30.569954+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:35.304358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"73'2","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.911058+0000","last_change":"2024-09-21T04:02:00.736630+0000","last_active":"2024-09-21T04:03:30.911058+0000","last_peered":"2024-09-21T04:03:30.911058+0000","last_clean":"2024-09-21T04:03:30.911058+0000","last_became_active":"2024-09-21T04:02:00.736211+0000","last_became_peered":"2024-09-21T04:02:00.736211+0000","last_unstale":"2024-09-21T04:03:30.911058+0000","last_undegraded":"2024-09-21T04:03:30.911058+0000","last_fullsized":"2024-09-21T04:03:30.911058+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:37.211617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.565621+0000","last_change":"2024-09-21T04:02:02.763323+0000","last_active":"2024-09-21T04:03:30.565621+0000","last_peered":"2024-09-21T04:03:30.565621+0000","last_clean":"2024-09-21T04:03:30.565621+0000","last_became_active":"2024-09-21T04:02:02.762776+0000","last_became_peered":"2024-09-21T04:02:02.762776+0000","last_unstale":"2024-09-21T04:03:30.565621+0000","last_undegraded":"2024-09-21T04:03:30.565621+0000","last_fullsized":"2024-09-21T04:03:30.565621+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:39.891042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"73'1","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575049+0000","last_change":"2024-09-21T04:02:00.743377+0000","last_active":"2024-09-21T04:03:30.575049+0000","last_peered":"2024-09-21T04:03:30.575049+0000","last_clean":"2024-09-21T04:03:30.575049+0000","last_became_active":"2024-09-21T04:02:00.742973+0000","last_became_peered":"2024-09-21T04:02:00.742973+0000","last_unstale":"2024-09-21T04:03:30.575049+0000","last_undegraded":"2024-09-21T04:03:30.575049+0000","last_fullsized":"2024-09-21T04:03:30.575049+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:55:00.655532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"75'11","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.570104+0000","last_change":"2024-09-21T04:02:02.758721+0000","last_active":"2024-09-21T04:03:30.570104+0000","last_peered":"2024-09-21T04:03:30.570104+0000","last_clean":"2024-09-21T04:03:30.570104+0000","last_became_active":"2024-09-21T04:02:02.758352+0000","last_became_peered":"2024-09-21T04:02:02.758352+0000","last_unstale":"2024-09-21T04:03:30.570104+0000","last_undegraded":"2024-09-21T04:03:30.570104+0000","last_fullsized":"2024-09-21T04:03:30.570104+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:56:49.289510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'4","reported_seq":131,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:51.018477+0000","last_change":"2024-09-21T04:02:00.744591+0000","last_active":"2024-09-21T04:03:51.018477+0000","last_peered":"2024-09-21T04:03:51.018477+0000","last_clean":"2024-09-21T04:03:51.018477+0000","last_became_active":"2024-09-21T04:02:00.744139+0000","last_became_peered":"2024-09-21T04:02:00.744139+0000","last_unstale":"2024-09-21T04:03:51.018477+0000","last_undegraded":"2024-09-21T04:03:51.018477+0000","last_fullsized":"2024-09-21T04:03:51.018477+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:21.331517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"75'10","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575159+0000","last_change":"2024-09-21T04:02:02.762779+0000","last_active":"2024-09-21T04:03:30.575159+0000","last_peered":"2024-09-21T04:03:30.575159+0000","last_clean":"2024-09-21T04:03:30.575159+0000","last_became_active":"2024-09-21T04:02:02.760761+0000","last_became_peered":"2024-09-21T04:02:02.760761+0000","last_unstale":"2024-09-21T04:03:30.575159+0000","last_undegraded":"2024-09-21T04:03:30.575159+0000","last_fullsized":"2024-09-21T04:03:30.575159+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:18.789052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"73'2","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847588+0000","last_change":"2024-09-21T04:02:00.743220+0000","last_active":"2024-09-21T04:03:30.847588+0000","last_peered":"2024-09-21T04:03:30.847588+0000","last_clean":"2024-09-21T04:03:30.847588+0000","last_became_active":"2024-09-21T04:02:00.742636+0000","last_became_peered":"2024-09-21T04:02:00.742636+0000","last_unstale":"2024-09-21T04:03:30.847588+0000","last_undegraded":"2024-09-21T04:03:30.847588+0000","last_fullsized":"2024-09-21T04:03:30.847588+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:01.382338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"75'9","reported_seq":38,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569960+0000","last_change":"2024-09-21T04:02:29.869663+0000","last_active":"2024-09-21T04:03:30.569960+0000","last_peered":"2024-09-21T04:03:30.569960+0000","last_clean":"2024-09-21T04:03:30.569960+0000","last_became_active":"2024-09-21T04:02:29.869230+0000","last_became_peered":"2024-09-21T04:02:29.869230+0000","last_unstale":"2024-09-21T04:03:30.569960+0000","last_undegraded":"2024-09-21T04:03:30.569960+0000","last_fullsized":"2024-09-21T04:03:30.569960+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:19:59.987267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"73'4","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575489+0000","last_change":"2024-09-21T04:02:00.853284+0000","last_active":"2024-09-21T04:03:30.575489+0000","last_peered":"2024-09-21T04:03:30.575489+0000","last_clean":"2024-09-21T04:03:30.575489+0000","last_became_active":"2024-09-21T04:02:00.852855+0000","last_became_peered":"2024-09-21T04:02:00.852855+0000","last_unstale":"2024-09-21T04:03:30.575489+0000","last_undegraded":"2024-09-21T04:03:30.575489+0000","last_fullsized":"2024-09-21T04:03:30.575489+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:36.994214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"75'11","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575292+0000","last_change":"2024-09-21T04:02:02.763577+0000","last_active":"2024-09-21T04:03:30.575292+0000","last_peered":"2024-09-21T04:03:30.575292+0000","last_clean":"2024-09-21T04:03:30.575292+0000","last_became_active":"2024-09-21T04:02:02.762970+0000","last_became_peered":"2024-09-21T04:02:02.762970+0000","last_unstale":"2024-09-21T04:03:30.575292+0000","last_undegraded":"2024-09-21T04:03:30.575292+0000","last_fullsized":"2024-09-21T04:03:30.575292+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:49.956226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'35","reported_seq":274,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:51.963042+0000","last_change":"2024-09-21T04:02:00.744203+0000","last_active":"2024-09-21T04:03:51.963042+0000","last_peered":"2024-09-21T04:03:51.963042+0000","last_clean":"2024-09-21T04:03:51.963042+0000","last_became_active":"2024-09-21T04:02:00.743659+0000","last_became_peered":"2024-09-21T04:02:00.743659+0000","last_unstale":"2024-09-21T04:03:51.963042+0000","last_undegraded":"2024-09-21T04:03:51.963042+0000","last_fullsized":"2024-09-21T04:03:51.963042+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:32.108302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71187,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":212,"num_read_kb":175,"num_write":36,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"75'6","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.572693+0000","last_change":"2024-09-21T04:02:02.766002+0000","last_active":"2024-09-21T04:03:30.572693+0000","last_peered":"2024-09-21T04:03:30.572693+0000","last_clean":"2024-09-21T04:03:30.572693+0000","last_became_active":"2024-09-21T04:02:02.765628+0000","last_became_peered":"2024-09-21T04:02:02.765628+0000","last_unstale":"2024-09-21T04:03:30.572693+0000","last_undegraded":"2024-09-21T04:03:30.572693+0000","last_fullsized":"2024-09-21T04:03:30.572693+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:22.251586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"73'6","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569569+0000","last_change":"2024-09-21T04:02:00.741086+0000","last_active":"2024-09-21T04:03:30.569569+0000","last_peered":"2024-09-21T04:03:30.569569+0000","last_clean":"2024-09-21T04:03:30.569569+0000","last_became_active":"2024-09-21T04:02:00.740633+0000","last_became_peered":"2024-09-21T04:02:00.740633+0000","last_unstale":"2024-09-21T04:03:30.569569+0000","last_undegraded":"2024-09-21T04:03:30.569569+0000","last_fullsized":"2024-09-21T04:03:30.569569+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:20.392799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"75'9","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.910819+0000","last_change":"2024-09-21T04:02:02.759651+0000","last_active":"2024-09-21T04:03:30.910819+0000","last_peered":"2024-09-21T04:03:30.910819+0000","last_clean":"2024-09-21T04:03:30.910819+0000","last_became_active":"2024-09-21T04:02:02.759290+0000","last_became_peered":"2024-09-21T04:02:02.759290+0000","last_unstale":"2024-09-21T04:03:30.910819+0000","last_undegraded":"2024-09-21T04:03:30.910819+0000","last_fullsized":"2024-09-21T04:03:30.910819+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:32.025703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"73'3","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.574965+0000","last_change":"2024-09-21T04:02:00.744005+0000","last_active":"2024-09-21T04:03:30.574965+0000","last_peered":"2024-09-21T04:03:30.574965+0000","last_clean":"2024-09-21T04:03:30.574965+0000","last_became_active":"2024-09-21T04:02:00.743570+0000","last_became_peered":"2024-09-21T04:02:00.743570+0000","last_unstale":"2024-09-21T04:03:30.574965+0000","last_undegraded":"2024-09-21T04:03:30.574965+0000","last_fullsized":"2024-09-21T04:03:30.574965+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:32.158681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"75'15","reported_seq":44,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847663+0000","last_change":"2024-09-21T04:02:29.869200+0000","last_active":"2024-09-21T04:03:30.847663+0000","last_peered":"2024-09-21T04:03:30.847663+0000","last_clean":"2024-09-21T04:03:30.847663+0000","last_became_active":"2024-09-21T04:02:29.868836+0000","last_became_peered":"2024-09-21T04:02:29.868836+0000","last_unstale":"2024-09-21T04:03:30.847663+0000","last_undegraded":"2024-09-21T04:03:30.847663+0000","last_fullsized":"2024-09-21T04:03:30.847663+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:42:49.030499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'4","reported_seq":132,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:49.959848+0000","last_change":"2024-09-21T04:02:00.741596+0000","last_active":"2024-09-21T04:03:49.959848+0000","last_peered":"2024-09-21T04:03:49.959848+0000","last_clean":"2024-09-21T04:03:49.959848+0000","last_became_active":"2024-09-21T04:02:00.741185+0000","last_became_peered":"2024-09-21T04:02:00.741185+0000","last_unstale":"2024-09-21T04:03:49.959848+0000","last_undegraded":"2024-09-21T04:03:49.959848+0000","last_fullsized":"2024-09-21T04:03:49.959848+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:31.663133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"75'6","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847114+0000","last_change":"2024-09-21T04:02:02.773706+0000","last_active":"2024-09-21T04:03:30.847114+0000","last_peered":"2024-09-21T04:03:30.847114+0000","last_clean":"2024-09-21T04:03:30.847114+0000","last_became_active":"2024-09-21T04:02:02.773259+0000","last_became_peered":"2024-09-21T04:02:02.773259+0000","last_unstale":"2024-09-21T04:03:30.847114+0000","last_undegraded":"2024-09-21T04:03:30.847114+0000","last_fullsized":"2024-09-21T04:03:30.847114+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:50.431982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"73'15","reported_seq":43,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.572386+0000","last_change":"2024-09-21T04:02:00.851335+0000","last_active":"2024-09-21T04:03:30.572386+0000","last_peered":"2024-09-21T04:03:30.572386+0000","last_clean":"2024-09-21T04:03:30.572386+0000","last_became_active":"2024-09-21T04:02:00.850990+0000","last_became_peered":"2024-09-21T04:02:00.850990+0000","last_unstale":"2024-09-21T04:03:30.572386+0000","last_undegraded":"2024-09-21T04:03:30.572386+0000","last_fullsized":"2024-09-21T04:03:30.572386+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:45.731121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"75'6","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.574843+0000","last_change":"2024-09-21T04:02:02.770985+0000","last_active":"2024-09-21T04:03:30.574843+0000","last_peered":"2024-09-21T04:03:30.574843+0000","last_clean":"2024-09-21T04:03:30.574843+0000","last_became_active":"2024-09-21T04:02:02.770602+0000","last_became_peered":"2024-09-21T04:02:02.770602+0000","last_unstale":"2024-09-21T04:03:30.574843+0000","last_undegraded":"2024-09-21T04:03:30.574843+0000","last_fullsized":"2024-09-21T04:03:30.574843+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:37.131113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'3","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.572923+0000","last_change":"2024-09-21T04:02:00.740834+0000","last_active":"2024-09-21T04:03:30.572923+0000","last_peered":"2024-09-21T04:03:30.572923+0000","last_clean":"2024-09-21T04:03:30.572923+0000","last_became_active":"2024-09-21T04:02:00.740439+0000","last_became_peered":"2024-09-21T04:02:00.740439+0000","last_unstale":"2024-09-21T04:03:30.572923+0000","last_undegraded":"2024-09-21T04:03:30.572923+0000","last_fullsized":"2024-09-21T04:03:30.572923+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:04:22.159986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"75'5","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.566818+0000","last_change":"2024-09-21T04:02:30.018671+0000","last_active":"2024-09-21T04:03:30.566818+0000","last_peered":"2024-09-21T04:03:30.566818+0000","last_clean":"2024-09-21T04:03:30.566818+0000","last_became_active":"2024-09-21T04:02:30.018247+0000","last_became_peered":"2024-09-21T04:02:30.018247+0000","last_unstale":"2024-09-21T04:03:30.566818+0000","last_undegraded":"2024-09-21T04:03:30.566818+0000","last_fullsized":"2024-09-21T04:03:30.566818+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:13.501966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"73'2","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.911086+0000","last_change":"2024-09-21T04:02:30.024526+0000","last_active":"2024-09-21T04:03:30.911086+0000","last_peered":"2024-09-21T04:03:30.911086+0000","last_clean":"2024-09-21T04:03:30.911086+0000","last_became_active":"2024-09-21T04:02:30.018811+0000","last_became_peered":"2024-09-21T04:02:30.018811+0000","last_unstale":"2024-09-21T04:03:30.911086+0000","last_undegraded":"2024-09-21T04:03:30.911086+0000","last_fullsized":"2024-09-21T04:03:30.911086+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:07.093364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"75'9","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847230+0000","last_change":"2024-09-21T04:02:02.766709+0000","last_active":"2024-09-21T04:03:30.847230+0000","last_peered":"2024-09-21T04:03:30.847230+0000","last_clean":"2024-09-21T04:03:30.847230+0000","last_became_active":"2024-09-21T04:02:02.766342+0000","last_became_peered":"2024-09-21T04:02:02.766342+0000","last_unstale":"2024-09-21T04:03:30.847230+0000","last_undegraded":"2024-09-21T04:03:30.847230+0000","last_fullsized":"2024-09-21T04:03:30.847230+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:26:35.185368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"73'1","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575295+0000","last_change":"2024-09-21T04:02:00.745365+0000","last_active":"2024-09-21T04:03:30.575295+0000","last_peered":"2024-09-21T04:03:30.575295+0000","last_clean":"2024-09-21T04:03:30.575295+0000","last_became_active":"2024-09-21T04:02:00.744968+0000","last_became_peered":"2024-09-21T04:02:00.744968+0000","last_unstale":"2024-09-21T04:03:30.575295+0000","last_undegraded":"2024-09-21T04:03:30.575295+0000","last_fullsized":"2024-09-21T04:03:30.575295+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:56.078169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847168+0000","last_change":"2024-09-21T04:02:02.774877+0000","last_active":"2024-09-21T04:03:30.847168+0000","last_peered":"2024-09-21T04:03:30.847168+0000","last_clean":"2024-09-21T04:03:30.847168+0000","last_became_active":"2024-09-21T04:02:02.774464+0000","last_became_peered":"2024-09-21T04:02:02.774464+0000","last_unstale":"2024-09-21T04:03:30.847168+0000","last_undegraded":"2024-09-21T04:03:30.847168+0000","last_fullsized":"2024-09-21T04:03:30.847168+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:23.393239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"73'3","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.576129+0000","last_change":"2024-09-21T04:03:30.575520+0000","last_active":"2024-09-21T04:03:30.576129+0000","last_peered":"2024-09-21T04:03:30.576129+0000","last_clean":"2024-09-21T04:03:30.576129+0000","last_became_active":"2024-09-21T04:03:30.570445+0000","last_became_peered":"2024-09-21T04:03:30.570445+0000","last_unstale":"2024-09-21T04:03:30.576129+0000","last_undegraded":"2024-09-21T04:03:30.576129+0000","last_fullsized":"2024-09-21T04:03:30.576129+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:15.571668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847199+0000","last_change":"2024-09-21T04:02:02.768164+0000","last_active":"2024-09-21T04:03:30.847199+0000","last_peered":"2024-09-21T04:03:30.847199+0000","last_clean":"2024-09-21T04:03:30.847199+0000","last_became_active":"2024-09-21T04:02:02.766925+0000","last_became_peered":"2024-09-21T04:02:02.766925+0000","last_unstale":"2024-09-21T04:03:30.847199+0000","last_undegraded":"2024-09-21T04:03:30.847199+0000","last_fullsized":"2024-09-21T04:03:30.847199+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:59.707637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":100,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.848055+0000","last_change":"2024-09-21T04:00:48.936996+0000","last_active":"2024-09-21T04:03:30.848055+0000","last_peered":"2024-09-21T04:03:30.848055+0000","last_clean":"2024-09-21T04:03:30.848055+0000","last_became_active":"2024-09-21T04:00:48.931310+0000","last_became_peered":"2024-09-21T04:00:48.931310+0000","last_unstale":"2024-09-21T04:03:30.848055+0000","last_undegraded":"2024-09-21T04:03:30.848055+0000","last_fullsized":"2024-09-21T04:03:30.848055+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:57:25.346362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:57:25.346362+0000","last_clean_scrub_stamp":"2024-09-21T03:57:25.346362+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:24.165731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"73'3","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.581750+0000","last_change":"2024-09-21T04:03:30.581661+0000","last_active":"2024-09-21T04:03:30.581750+0000","last_peered":"2024-09-21T04:03:30.581750+0000","last_clean":"2024-09-21T04:03:30.581750+0000","last_became_active":"2024-09-21T04:03:30.571414+0000","last_became_peered":"2024-09-21T04:03:30.571414+0000","last_unstale":"2024-09-21T04:03:30.581750+0000","last_undegraded":"2024-09-21T04:03:30.581750+0000","last_fullsized":"2024-09-21T04:03:30.581750+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:49.003453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9],"acting":[4,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.565861+0000","last_change":"2024-09-21T04:02:02.771537+0000","last_active":"2024-09-21T04:03:30.565861+0000","last_peered":"2024-09-21T04:03:30.565861+0000","last_clean":"2024-09-21T04:03:30.565861+0000","last_became_active":"2024-09-21T04:02:02.771151+0000","last_became_peered":"2024-09-21T04:02:02.771151+0000","last_unstale":"2024-09-21T04:03:30.565861+0000","last_undegraded":"2024-09-21T04:03:30.565861+0000","last_fullsized":"2024-09-21T04:03:30.565861+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:21.283515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"73'3","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847919+0000","last_change":"2024-09-21T04:02:30.022634+0000","last_active":"2024-09-21T04:03:30.847919+0000","last_peered":"2024-09-21T04:03:30.847919+0000","last_clean":"2024-09-21T04:03:30.847919+0000","last_became_active":"2024-09-21T04:02:30.018076+0000","last_became_peered":"2024-09-21T04:02:30.018076+0000","last_unstale":"2024-09-21T04:03:30.847919+0000","last_undegraded":"2024-09-21T04:03:30.847919+0000","last_fullsized":"2024-09-21T04:03:30.847919+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:31.650092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'5","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.910802+0000","last_change":"2024-09-21T04:02:02.755223+0000","last_active":"2024-09-21T04:03:30.910802+0000","last_peered":"2024-09-21T04:03:30.910802+0000","last_clean":"2024-09-21T04:03:30.910802+0000","last_became_active":"2024-09-21T04:02:02.754836+0000","last_became_peered":"2024-09-21T04:02:02.754836+0000","last_unstale":"2024-09-21T04:03:30.910802+0000","last_undegraded":"2024-09-21T04:03:30.910802+0000","last_fullsized":"2024-09-21T04:03:30.910802+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:57.912690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'5","reported_seq":131,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:49.959127+0000","last_change":"2024-09-21T04:02:00.851689+0000","last_active":"2024-09-21T04:03:49.959127+0000","last_peered":"2024-09-21T04:03:49.959127+0000","last_clean":"2024-09-21T04:03:49.959127+0000","last_became_active":"2024-09-21T04:02:00.851241+0000","last_became_peered":"2024-09-21T04:02:00.851241+0000","last_unstale":"2024-09-21T04:03:49.959127+0000","last_undegraded":"2024-09-21T04:03:49.959127+0000","last_fullsized":"2024-09-21T04:03:49.959127+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:14.697695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575280+0000","last_change":"2024-09-21T04:02:02.774750+0000","last_active":"2024-09-21T04:03:30.575280+0000","last_peered":"2024-09-21T04:03:30.575280+0000","last_clean":"2024-09-21T04:03:30.575280+0000","last_became_active":"2024-09-21T04:02:02.774310+0000","last_became_peered":"2024-09-21T04:02:02.774310+0000","last_unstale":"2024-09-21T04:03:30.575280+0000","last_undegraded":"2024-09-21T04:03:30.575280+0000","last_fullsized":"2024-09-21T04:03:30.575280+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:07.909035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"73'1","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847647+0000","last_change":"2024-09-21T04:02:00.741461+0000","last_active":"2024-09-21T04:03:30.847647+0000","last_peered":"2024-09-21T04:03:30.847647+0000","last_clean":"2024-09-21T04:03:30.847647+0000","last_became_active":"2024-09-21T04:02:00.741051+0000","last_became_peered":"2024-09-21T04:02:00.741051+0000","last_unstale":"2024-09-21T04:03:30.847647+0000","last_undegraded":"2024-09-21T04:03:30.847647+0000","last_fullsized":"2024-09-21T04:03:30.847647+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:05.944430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.570031+0000","last_change":"2024-09-21T04:02:02.761703+0000","last_active":"2024-09-21T04:03:30.570031+0000","last_peered":"2024-09-21T04:03:30.570031+0000","last_clean":"2024-09-21T04:03:30.570031+0000","last_became_active":"2024-09-21T04:02:02.761296+0000","last_became_peered":"2024-09-21T04:02:02.761296+0000","last_unstale":"2024-09-21T04:03:30.570031+0000","last_undegraded":"2024-09-21T04:03:30.570031+0000","last_fullsized":"2024-09-21T04:03:30.570031+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:57.337167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569828+0000","last_change":"2024-09-21T04:02:00.853584+0000","last_active":"2024-09-21T04:03:30.569828+0000","last_peered":"2024-09-21T04:03:30.569828+0000","last_clean":"2024-09-21T04:03:30.569828+0000","last_became_active":"2024-09-21T04:02:00.853019+0000","last_became_peered":"2024-09-21T04:02:00.853019+0000","last_unstale":"2024-09-21T04:03:30.569828+0000","last_undegraded":"2024-09-21T04:03:30.569828+0000","last_fullsized":"2024-09-21T04:03:30.569828+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:56.186791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'5","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569889+0000","last_change":"2024-09-21T04:02:02.758175+0000","last_active":"2024-09-21T04:03:30.569889+0000","last_peered":"2024-09-21T04:03:30.569889+0000","last_clean":"2024-09-21T04:03:30.569889+0000","last_became_active":"2024-09-21T04:02:02.757715+0000","last_became_peered":"2024-09-21T04:02:02.757715+0000","last_unstale":"2024-09-21T04:03:30.569889+0000","last_undegraded":"2024-09-21T04:03:30.569889+0000","last_fullsized":"2024-09-21T04:03:30.569889+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:40.172569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"73'1","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.572814+0000","last_change":"2024-09-21T04:02:00.853325+0000","last_active":"2024-09-21T04:03:30.572814+0000","last_peered":"2024-09-21T04:03:30.572814+0000","last_clean":"2024-09-21T04:03:30.572814+0000","last_became_active":"2024-09-21T04:02:00.852834+0000","last_became_peered":"2024-09-21T04:02:00.852834+0000","last_unstale":"2024-09-21T04:03:30.572814+0000","last_undegraded":"2024-09-21T04:03:30.572814+0000","last_fullsized":"2024-09-21T04:03:30.572814+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:16.090493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"75'9","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.910869+0000","last_change":"2024-09-21T04:02:02.752402+0000","last_active":"2024-09-21T04:03:30.910869+0000","last_peered":"2024-09-21T04:03:30.910869+0000","last_clean":"2024-09-21T04:03:30.910869+0000","last_became_active":"2024-09-21T04:02:02.751994+0000","last_became_peered":"2024-09-21T04:02:02.751994+0000","last_unstale":"2024-09-21T04:03:30.910869+0000","last_undegraded":"2024-09-21T04:03:30.910869+0000","last_fullsized":"2024-09-21T04:03:30.910869+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:19.525317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'14","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:49.767459+0000","last_change":"2024-09-21T04:02:00.739112+0000","last_active":"2024-09-21T04:03:49.767459+0000","last_peered":"2024-09-21T04:03:49.767459+0000","last_clean":"2024-09-21T04:03:49.767459+0000","last_became_active":"2024-09-21T04:02:00.738613+0000","last_became_peered":"2024-09-21T04:02:00.738613+0000","last_unstale":"2024-09-21T04:03:49.767459+0000","last_undegraded":"2024-09-21T04:03:49.767459+0000","last_fullsized":"2024-09-21T04:03:49.767459+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:34.334576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"75'5","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.565764+0000","last_change":"2024-09-21T04:02:02.763345+0000","last_active":"2024-09-21T04:03:30.565764+0000","last_peered":"2024-09-21T04:03:30.565764+0000","last_clean":"2024-09-21T04:03:30.565764+0000","last_became_active":"2024-09-21T04:02:02.762987+0000","last_became_peered":"2024-09-21T04:02:02.762987+0000","last_unstale":"2024-09-21T04:03:30.565764+0000","last_undegraded":"2024-09-21T04:03:30.565764+0000","last_fullsized":"2024-09-21T04:03:30.565764+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:36.239267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"73'4","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.567663+0000","last_change":"2024-09-21T04:02:01.258208+0000","last_active":"2024-09-21T04:03:30.567663+0000","last_peered":"2024-09-21T04:03:30.567663+0000","last_clean":"2024-09-21T04:03:30.567663+0000","last_became_active":"2024-09-21T04:02:01.257780+0000","last_became_peered":"2024-09-21T04:02:01.257780+0000","last_unstale":"2024-09-21T04:03:30.567663+0000","last_undegraded":"2024-09-21T04:03:30.567663+0000","last_fullsized":"2024-09-21T04:03:30.567663+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:20.161792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569870+0000","last_change":"2024-09-21T04:02:02.761149+0000","last_active":"2024-09-21T04:03:30.569870+0000","last_peered":"2024-09-21T04:03:30.569870+0000","last_clean":"2024-09-21T04:03:30.569870+0000","last_became_active":"2024-09-21T04:02:02.760771+0000","last_became_peered":"2024-09-21T04:02:02.760771+0000","last_unstale":"2024-09-21T04:03:30.569870+0000","last_undegraded":"2024-09-21T04:03:30.569870+0000","last_fullsized":"2024-09-21T04:03:30.569870+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:20.645377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"73'2","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575047+0000","last_change":"2024-09-21T04:02:00.853580+0000","last_active":"2024-09-21T04:03:30.575047+0000","last_peered":"2024-09-21T04:03:30.575047+0000","last_clean":"2024-09-21T04:03:30.575047+0000","last_became_active":"2024-09-21T04:02:00.853091+0000","last_became_peered":"2024-09-21T04:02:00.853091+0000","last_unstale":"2024-09-21T04:03:30.575047+0000","last_undegraded":"2024-09-21T04:03:30.575047+0000","last_fullsized":"2024-09-21T04:03:30.575047+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:32.893695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"75'2","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.572389+0000","last_change":"2024-09-21T04:02:02.769686+0000","last_active":"2024-09-21T04:03:30.572389+0000","last_peered":"2024-09-21T04:03:30.572389+0000","last_clean":"2024-09-21T04:03:30.572389+0000","last_became_active":"2024-09-21T04:02:02.769293+0000","last_became_peered":"2024-09-21T04:02:02.769293+0000","last_unstale":"2024-09-21T04:03:30.572389+0000","last_undegraded":"2024-09-21T04:03:30.572389+0000","last_fullsized":"2024-09-21T04:03:30.572389+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:10.984624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"73'3","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.565759+0000","last_change":"2024-09-21T04:02:00.741556+0000","last_active":"2024-09-21T04:03:30.565759+0000","last_peered":"2024-09-21T04:03:30.565759+0000","last_clean":"2024-09-21T04:03:30.565759+0000","last_became_active":"2024-09-21T04:02:00.741188+0000","last_became_peered":"2024-09-21T04:02:00.741188+0000","last_unstale":"2024-09-21T04:03:30.565759+0000","last_undegraded":"2024-09-21T04:03:30.565759+0000","last_fullsized":"2024-09-21T04:03:30.565759+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:23.566814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"75'9","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.566789+0000","last_change":"2024-09-21T04:02:02.769163+0000","last_active":"2024-09-21T04:03:30.566789+0000","last_peered":"2024-09-21T04:03:30.566789+0000","last_clean":"2024-09-21T04:03:30.566789+0000","last_became_active":"2024-09-21T04:02:02.768804+0000","last_became_peered":"2024-09-21T04:02:02.768804+0000","last_unstale":"2024-09-21T04:03:30.566789+0000","last_undegraded":"2024-09-21T04:03:30.566789+0000","last_fullsized":"2024-09-21T04:03:30.566789+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:12.576553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"73'1","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569475+0000","last_change":"2024-09-21T04:02:00.743091+0000","last_active":"2024-09-21T04:03:30.569475+0000","last_peered":"2024-09-21T04:03:30.569475+0000","last_clean":"2024-09-21T04:03:30.569475+0000","last_became_active":"2024-09-21T04:02:00.742665+0000","last_became_peered":"2024-09-21T04:02:00.742665+0000","last_unstale":"2024-09-21T04:03:30.569475+0000","last_undegraded":"2024-09-21T04:03:30.569475+0000","last_fullsized":"2024-09-21T04:03:30.569475+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:09.698421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.565845+0000","last_change":"2024-09-21T04:02:02.761985+0000","last_active":"2024-09-21T04:03:30.565845+0000","last_peered":"2024-09-21T04:03:30.565845+0000","last_clean":"2024-09-21T04:03:30.565845+0000","last_became_active":"2024-09-21T04:02:02.761624+0000","last_became_peered":"2024-09-21T04:02:02.761624+0000","last_unstale":"2024-09-21T04:03:30.565845+0000","last_undegraded":"2024-09-21T04:03:30.565845+0000","last_fullsized":"2024-09-21T04:03:30.565845+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:12.912221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"73'4","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.572449+0000","last_change":"2024-09-21T04:02:30.024288+0000","last_active":"2024-09-21T04:03:30.572449+0000","last_peered":"2024-09-21T04:03:30.572449+0000","last_clean":"2024-09-21T04:03:30.572449+0000","last_became_active":"2024-09-21T04:02:30.018929+0000","last_became_peered":"2024-09-21T04:02:30.018929+0000","last_unstale":"2024-09-21T04:03:30.572449+0000","last_undegraded":"2024-09-21T04:03:30.572449+0000","last_fullsized":"2024-09-21T04:03:30.572449+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:19:21.400082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"75'5","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847220+0000","last_change":"2024-09-21T04:02:02.761357+0000","last_active":"2024-09-21T04:03:30.847220+0000","last_peered":"2024-09-21T04:03:30.847220+0000","last_clean":"2024-09-21T04:03:30.847220+0000","last_became_active":"2024-09-21T04:02:02.760984+0000","last_became_peered":"2024-09-21T04:02:02.760984+0000","last_unstale":"2024-09-21T04:03:30.847220+0000","last_undegraded":"2024-09-21T04:03:30.847220+0000","last_fullsized":"2024-09-21T04:03:30.847220+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:55.388768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"75'15","reported_seq":44,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575165+0000","last_change":"2024-09-21T04:02:30.019234+0000","last_active":"2024-09-21T04:03:30.575165+0000","last_peered":"2024-09-21T04:03:30.575165+0000","last_clean":"2024-09-21T04:03:30.575165+0000","last_became_active":"2024-09-21T04:02:30.018778+0000","last_became_peered":"2024-09-21T04:02:30.018778+0000","last_unstale":"2024-09-21T04:03:30.575165+0000","last_undegraded":"2024-09-21T04:03:30.575165+0000","last_fullsized":"2024-09-21T04:03:30.575165+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:31.227144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"73'2","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.915768+0000","last_change":"2024-09-21T04:03:30.915685+0000","last_active":"2024-09-21T04:03:30.915768+0000","last_peered":"2024-09-21T04:03:30.915768+0000","last_clean":"2024-09-21T04:03:30.915768+0000","last_became_active":"2024-09-21T04:03:30.912802+0000","last_became_peered":"2024-09-21T04:03:30.912802+0000","last_unstale":"2024-09-21T04:03:30.915768+0000","last_undegraded":"2024-09-21T04:03:30.915768+0000","last_fullsized":"2024-09-21T04:03:30.915768+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:15:58.295051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,6],"acting":[8,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"75'9","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575216+0000","last_change":"2024-09-21T04:02:02.762336+0000","last_active":"2024-09-21T04:03:30.575216+0000","last_peered":"2024-09-21T04:03:30.575216+0000","last_clean":"2024-09-21T04:03:30.575216+0000","last_became_active":"2024-09-21T04:02:02.761753+0000","last_became_peered":"2024-09-21T04:02:02.761753+0000","last_unstale":"2024-09-21T04:03:30.575216+0000","last_undegraded":"2024-09-21T04:03:30.575216+0000","last_fullsized":"2024-09-21T04:03:30.575216+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:39.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"73'4","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.572719+0000","last_change":"2024-09-21T04:02:00.743495+0000","last_active":"2024-09-21T04:03:30.572719+0000","last_peered":"2024-09-21T04:03:30.572719+0000","last_clean":"2024-09-21T04:03:30.572719+0000","last_became_active":"2024-09-21T04:02:00.743135+0000","last_became_peered":"2024-09-21T04:02:00.743135+0000","last_unstale":"2024-09-21T04:03:30.572719+0000","last_undegraded":"2024-09-21T04:03:30.572719+0000","last_fullsized":"2024-09-21T04:03:30.572719+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:16.481148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"75'6","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569827+0000","last_change":"2024-09-21T04:02:02.766682+0000","last_active":"2024-09-21T04:03:30.569827+0000","last_peered":"2024-09-21T04:03:30.569827+0000","last_clean":"2024-09-21T04:03:30.569827+0000","last_became_active":"2024-09-21T04:02:02.766299+0000","last_became_peered":"2024-09-21T04:02:02.766299+0000","last_unstale":"2024-09-21T04:03:30.569827+0000","last_undegraded":"2024-09-21T04:03:30.569827+0000","last_fullsized":"2024-09-21T04:03:30.569827+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:25.635908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"73'2","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.566238+0000","last_change":"2024-09-21T04:02:00.852281+0000","last_active":"2024-09-21T04:03:30.566238+0000","last_peered":"2024-09-21T04:03:30.566238+0000","last_clean":"2024-09-21T04:03:30.566238+0000","last_became_active":"2024-09-21T04:02:00.851900+0000","last_became_peered":"2024-09-21T04:02:00.851900+0000","last_unstale":"2024-09-21T04:03:30.566238+0000","last_undegraded":"2024-09-21T04:03:30.566238+0000","last_fullsized":"2024-09-21T04:03:30.566238+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:37.115541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"75'5","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.566108+0000","last_change":"2024-09-21T04:02:02.770467+0000","last_active":"2024-09-21T04:03:30.566108+0000","last_peered":"2024-09-21T04:03:30.566108+0000","last_clean":"2024-09-21T04:03:30.566108+0000","last_became_active":"2024-09-21T04:02:02.770099+0000","last_became_peered":"2024-09-21T04:02:02.770099+0000","last_unstale":"2024-09-21T04:03:30.566108+0000","last_undegraded":"2024-09-21T04:03:30.566108+0000","last_fullsized":"2024-09-21T04:03:30.566108+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:38.806298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"73'9","reported_seq":54,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.570015+0000","last_change":"2024-09-21T04:02:29.878053+0000","last_active":"2024-09-21T04:03:30.570015+0000","last_peered":"2024-09-21T04:03:30.570015+0000","last_clean":"2024-09-21T04:03:30.570015+0000","last_became_active":"2024-09-21T04:02:29.869007+0000","last_became_peered":"2024-09-21T04:02:29.869007+0000","last_unstale":"2024-09-21T04:03:30.570015+0000","last_undegraded":"2024-09-21T04:03:30.570015+0000","last_fullsized":"2024-09-21T04:03:30.570015+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:50.509849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75105,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":625,"num_read_kb":392,"num_write":179,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1198,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":217008,"data_compressed":5676,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282548,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808868,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":847872,"data_stored":424484,"data_compressed":11589,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":62,"seq":266287972392,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":1087201,"data_compressed":49407,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135918,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":856064,"data_stored":424104,"data_compressed":11589,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332147,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":856064,"data_stored":424024,"data_compressed":11589,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495671,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":1016071,"data_compressed":47515,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659197,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":843776,"data_stored":423493,"data_compressed":11589,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855426,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":1015247,"data_compressed":47515,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018951,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":424820,"data_compressed":11589,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":860160,"data_stored":423804,"data_compressed":11589,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378705,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":929792,"data_stored":490501,"data_compressed":13481,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542230,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":489733,"data_compressed":13481,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705755,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":864256,"data_stored":424764,"data_compressed":11589,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34306,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34306,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34306,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":66447,"data_compressed":1892,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67215,"data_compressed":1892,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":73430,"data_compressed":1892,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T04:03:54.848 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-21T04:03:54.848 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T04:03:55.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.995+0000 7f0f1f57b640 1 -- 172.21.15.64:0/59345448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f181100a0 msgr2=0x7f0f181124c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:55.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.995+0000 7f0f1f57b640 1 --2- 172.21.15.64:0/59345448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f181100a0 0x7f0f181124c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0f10009ca0 tx=0x7f0f1002a950 comp rx=0 tx=0).stop 2024-09-21T04:03:55.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.995+0000 7f0f1f57b640 1 -- 172.21.15.64:0/59345448 shutdown_connections 2024-09-21T04:03:55.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.995+0000 7f0f1f57b640 1 --2- 172.21.15.64:0/59345448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f181100a0 0x7f0f181124c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:55.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.995+0000 7f0f1f57b640 1 --2- 172.21.15.64:0/59345448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f18102e40 0x7f0f1810fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:55.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.995+0000 7f0f1f57b640 1 --2- 172.21.15.64:0/59345448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f181024f0 0x7f0f18102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:55.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.995+0000 7f0f1f57b640 1 -- 172.21.15.64:0/59345448 >> 172.21.15.64:0/59345448 conn(0x7f0f180fe350 msgr2=0x7f0f181007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:55.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.995+0000 7f0f1f57b640 1 -- 172.21.15.64:0/59345448 shutdown_connections 2024-09-21T04:03:55.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.995+0000 7f0f1f57b640 1 -- 172.21.15.64:0/59345448 wait complete. 2024-09-21T04:03:55.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.996+0000 7f0f1f57b640 1 Processor -- start 2024-09-21T04:03:55.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.996+0000 7f0f1f57b640 1 -- start start 2024-09-21T04:03:55.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.996+0000 7f0f1f57b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f181024f0 0x7f0f1819c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:55.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.996+0000 7f0f1f57b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f18102e40 0x7f0f1819cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:55.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.996+0000 7f0f1f57b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f181100a0 0x7f0f181a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:55.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.996+0000 7f0f1f57b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0f18115b90 con 0x7f0f181100a0 2024-09-21T04:03:55.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.996+0000 7f0f1f57b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0f18115a10 con 0x7f0f18102e40 2024-09-21T04:03:55.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.996+0000 7f0f1f57b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0f18115d10 con 0x7f0f181024f0 2024-09-21T04:03:55.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.996+0000 7f0f1db18640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f181100a0 0x7f0f181a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:55.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.996+0000 7f0f1d317640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f181024f0 0x7f0f1819c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:55.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.996+0000 7f0f1db18640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f181100a0 0x7f0f181a1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:49998/0 (socket says 172.21.15.64:49998) 2024-09-21T04:03:55.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.996+0000 7f0f1db18640 1 -- 172.21.15.64:0/1243010745 learned_addr learned my addr 172.21.15.64:0/1243010745 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:55.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.996+0000 7f0f1cb16640 1 --2- 172.21.15.64:0/1243010745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f18102e40 0x7f0f1819cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:55.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.996+0000 7f0f1db18640 1 -- 172.21.15.64:0/1243010745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f181024f0 msgr2=0x7f0f1819c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:55.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.996+0000 7f0f1db18640 1 --2- 172.21.15.64:0/1243010745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f181024f0 0x7f0f1819c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:55.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.996+0000 7f0f1db18640 1 -- 172.21.15.64:0/1243010745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f18102e40 msgr2=0x7f0f1819cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:55.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.996+0000 7f0f1db18640 1 --2- 172.21.15.64:0/1243010745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f18102e40 0x7f0f1819cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:55.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.996+0000 7f0f1db18640 1 -- 172.21.15.64:0/1243010745 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f181a24a0 con 0x7f0f181100a0 2024-09-21T04:03:55.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.996+0000 7f0f1cb16640 1 --2- 172.21.15.64:0/1243010745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f18102e40 0x7f0f1819cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:03:55.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.997+0000 7f0f1db18640 1 --2- 172.21.15.64:0/1243010745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f181100a0 0x7f0f181a1da0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0f1002ae70 tx=0x7f0f1002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:55.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.997+0000 7f0f0e7fc640 1 -- 172.21.15.64:0/1243010745 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f10034040 con 0x7f0f181100a0 2024-09-21T04:03:55.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.997+0000 7f0f0e7fc640 1 -- 172.21.15.64:0/1243010745 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f100308f0 con 0x7f0f181100a0 2024-09-21T04:03:55.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.997+0000 7f0f1f57b640 1 -- 172.21.15.64:0/1243010745 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f181a2730 con 0x7f0f181100a0 2024-09-21T04:03:55.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.997+0000 7f0f0e7fc640 1 -- 172.21.15.64:0/1243010745 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f100047f0 con 0x7f0f181100a0 2024-09-21T04:03:55.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.997+0000 7f0f1f57b640 1 -- 172.21.15.64:0/1243010745 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f181a8f20 con 0x7f0f181100a0 2024-09-21T04:03:55.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.998+0000 7f0f1f57b640 1 -- 172.21.15.64:0/1243010745 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f18102900 con 0x7f0f181100a0 2024-09-21T04:03:55.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:54.999+0000 7f0f0e7fc640 1 -- 172.21.15.64:0/1243010745 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) ==== 154580+0+0 (secure 0 0 0) 0x7f0f10037020 con 0x7f0f181100a0 2024-09-21T04:03:55.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.000+0000 7f0f0e7fc640 1 --2- 172.21.15.64:0/1243010745 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0f040b6670 0x7f0f040b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:55.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.000+0000 7f0f1d317640 1 --2- 172.21.15.64:0/1243010745 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0f040b6670 0x7f0f040b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:55.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.000+0000 7f0f0e7fc640 1 -- 172.21.15.64:0/1243010745 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f0f10108760 con 0x7f0f181100a0 2024-09-21T04:03:55.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.001+0000 7f0f1d317640 1 --2- 172.21.15.64:0/1243010745 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0f040b6670 0x7f0f040b8b80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0f14005fd0 tx=0x7f0f1400b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:55.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.003+0000 7f0f0e7fc640 1 -- 172.21.15.64:0/1243010745 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0f10083710 con 0x7f0f181100a0 2024-09-21T04:03:55.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.189+0000 7f0f1f57b640 1 -- 172.21.15.64:0/1243010745 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0f18063800 con 0x7f0f040b6670 2024-09-21T04:03:55.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.193+0000 7f0f0e7fc640 1 -- 172.21.15.64:0/1243010745 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259137 (secure 0 0 0) 0x7f0f18063800 con 0x7f0f040b6670 2024-09-21T04:03:55.199 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:03:55.200 INFO:teuthology.orchestra.run.smithi064.stderr:dumped all 2024-09-21T04:03:55.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.197+0000 7f0f1f57b640 1 -- 172.21.15.64:0/1243010745 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0f040b6670 msgr2=0x7f0f040b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:55.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.197+0000 7f0f1f57b640 1 --2- 172.21.15.64:0/1243010745 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0f040b6670 0x7f0f040b8b80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0f14005fd0 tx=0x7f0f1400b390 comp rx=0 tx=0).stop 2024-09-21T04:03:55.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.197+0000 7f0f1f57b640 1 -- 172.21.15.64:0/1243010745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f181100a0 msgr2=0x7f0f181a1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:55.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.197+0000 7f0f1f57b640 1 --2- 172.21.15.64:0/1243010745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f181100a0 0x7f0f181a1da0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0f1002ae70 tx=0x7f0f1002f5d0 comp rx=0 tx=0).stop 2024-09-21T04:03:55.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.197+0000 7f0f1f57b640 1 -- 172.21.15.64:0/1243010745 shutdown_connections 2024-09-21T04:03:55.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.197+0000 7f0f1f57b640 1 --2- 172.21.15.64:0/1243010745 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0f040b6670 0x7f0f040b8b80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:55.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.197+0000 7f0f1f57b640 1 --2- 172.21.15.64:0/1243010745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f181100a0 0x7f0f181a1da0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:55.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.197+0000 7f0f1f57b640 1 --2- 172.21.15.64:0/1243010745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f18102e40 0x7f0f1819cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:55.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.197+0000 7f0f1f57b640 1 --2- 172.21.15.64:0/1243010745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f181024f0 0x7f0f1819c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:55.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.197+0000 7f0f1f57b640 1 -- 172.21.15.64:0/1243010745 >> 172.21.15.64:0/1243010745 conn(0x7f0f180fe350 msgr2=0x7f0f181120b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:55.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.197+0000 7f0f1f57b640 1 -- 172.21.15.64:0/1243010745 shutdown_connections 2024-09-21T04:03:55.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.197+0000 7f0f1f57b640 1 -- 172.21.15.64:0/1243010745 wait complete. 2024-09-21T04:03:55.211 INFO:teuthology.orchestra.run.smithi064.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-09-21T04:03:54.506243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665707,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":706,"num_read_kb":461,"num_write":304,"num_write_kb":2284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":2953038,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":745,"ondisk_log_size":745,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359652,"kb_used_data":10384,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124582172,"statfs":{"total":1151940427776,"available":1151572144128,"internally_reserved":0,"allocated":10633216,"data_stored":7068246,"data_compressed":252522,"data_compressed_allocated":344064,"data_compressed_original":2187264,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5356,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":37,"num_write":12,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003590"},"pg_stats":[{"pgid":"3.3f","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.567004+0000","last_change":"2024-09-21T04:02:02.769848+0000","last_active":"2024-09-21T04:03:30.567004+0000","last_peered":"2024-09-21T04:03:30.567004+0000","last_clean":"2024-09-21T04:03:30.567004+0000","last_became_active":"2024-09-21T04:02:02.769508+0000","last_became_peered":"2024-09-21T04:02:02.769508+0000","last_unstale":"2024-09-21T04:03:30.567004+0000","last_undegraded":"2024-09-21T04:03:30.567004+0000","last_fullsized":"2024-09-21T04:03:30.567004+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:00.242233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"75'4","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569630+0000","last_change":"2024-09-21T04:02:02.769200+0000","last_active":"2024-09-21T04:03:30.569630+0000","last_peered":"2024-09-21T04:03:30.569630+0000","last_clean":"2024-09-21T04:03:30.569630+0000","last_became_active":"2024-09-21T04:02:02.768918+0000","last_became_peered":"2024-09-21T04:02:02.768918+0000","last_unstale":"2024-09-21T04:03:30.569630+0000","last_undegraded":"2024-09-21T04:03:30.569630+0000","last_fullsized":"2024-09-21T04:03:30.569630+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:26.842079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569334+0000","last_change":"2024-09-21T04:02:02.763511+0000","last_active":"2024-09-21T04:03:30.569334+0000","last_peered":"2024-09-21T04:03:30.569334+0000","last_clean":"2024-09-21T04:03:30.569334+0000","last_became_active":"2024-09-21T04:02:02.763161+0000","last_became_peered":"2024-09-21T04:02:02.763161+0000","last_unstale":"2024-09-21T04:03:30.569334+0000","last_undegraded":"2024-09-21T04:03:30.569334+0000","last_fullsized":"2024-09-21T04:03:30.569334+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:59.834005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.567466+0000","last_change":"2024-09-21T04:02:02.765452+0000","last_active":"2024-09-21T04:03:30.567466+0000","last_peered":"2024-09-21T04:03:30.567466+0000","last_clean":"2024-09-21T04:03:30.567466+0000","last_became_active":"2024-09-21T04:02:02.765065+0000","last_became_peered":"2024-09-21T04:02:02.765065+0000","last_unstale":"2024-09-21T04:03:30.567466+0000","last_undegraded":"2024-09-21T04:03:30.567466+0000","last_fullsized":"2024-09-21T04:03:30.567466+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:19:33.871366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"75'4","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.572434+0000","last_change":"2024-09-21T04:02:02.767457+0000","last_active":"2024-09-21T04:03:30.572434+0000","last_peered":"2024-09-21T04:03:30.572434+0000","last_clean":"2024-09-21T04:03:30.572434+0000","last_became_active":"2024-09-21T04:02:02.767064+0000","last_became_peered":"2024-09-21T04:02:02.767064+0000","last_unstale":"2024-09-21T04:03:30.572434+0000","last_undegraded":"2024-09-21T04:03:30.572434+0000","last_fullsized":"2024-09-21T04:03:30.572434+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:21:46.316029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"75'6","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569752+0000","last_change":"2024-09-21T04:02:02.768795+0000","last_active":"2024-09-21T04:03:30.569752+0000","last_peered":"2024-09-21T04:03:30.569752+0000","last_clean":"2024-09-21T04:03:30.569752+0000","last_became_active":"2024-09-21T04:02:02.768434+0000","last_became_peered":"2024-09-21T04:02:02.768434+0000","last_unstale":"2024-09-21T04:03:30.569752+0000","last_undegraded":"2024-09-21T04:03:30.569752+0000","last_fullsized":"2024-09-21T04:03:30.569752+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:58.991533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"75'5","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847154+0000","last_change":"2024-09-21T04:02:02.774998+0000","last_active":"2024-09-21T04:03:30.847154+0000","last_peered":"2024-09-21T04:03:30.847154+0000","last_clean":"2024-09-21T04:03:30.847154+0000","last_became_active":"2024-09-21T04:02:02.774344+0000","last_became_peered":"2024-09-21T04:02:02.774344+0000","last_unstale":"2024-09-21T04:03:30.847154+0000","last_undegraded":"2024-09-21T04:03:30.847154+0000","last_fullsized":"2024-09-21T04:03:30.847154+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:26.705126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"75'9","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.565533+0000","last_change":"2024-09-21T04:02:02.772498+0000","last_active":"2024-09-21T04:03:30.565533+0000","last_peered":"2024-09-21T04:03:30.565533+0000","last_clean":"2024-09-21T04:03:30.565533+0000","last_became_active":"2024-09-21T04:02:02.772049+0000","last_became_peered":"2024-09-21T04:02:02.772049+0000","last_unstale":"2024-09-21T04:03:30.565533+0000","last_undegraded":"2024-09-21T04:03:30.565533+0000","last_fullsized":"2024-09-21T04:03:30.565533+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:31:19.088424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"75'11","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575591+0000","last_change":"2024-09-21T04:02:02.772336+0000","last_active":"2024-09-21T04:03:30.575591+0000","last_peered":"2024-09-21T04:03:30.575591+0000","last_clean":"2024-09-21T04:03:30.575591+0000","last_became_active":"2024-09-21T04:02:02.771964+0000","last_became_peered":"2024-09-21T04:02:02.771964+0000","last_unstale":"2024-09-21T04:03:30.575591+0000","last_undegraded":"2024-09-21T04:03:30.575591+0000","last_fullsized":"2024-09-21T04:03:30.575591+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:23:46.659690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"75'16","reported_seq":38,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.570342+0000","last_change":"2024-09-21T04:02:02.758992+0000","last_active":"2024-09-21T04:03:30.570342+0000","last_peered":"2024-09-21T04:03:30.570342+0000","last_clean":"2024-09-21T04:03:30.570342+0000","last_became_active":"2024-09-21T04:02:02.758575+0000","last_became_peered":"2024-09-21T04:02:02.758575+0000","last_unstale":"2024-09-21T04:03:30.570342+0000","last_undegraded":"2024-09-21T04:03:30.570342+0000","last_fullsized":"2024-09-21T04:03:30.570342+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:32:09.543140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"75'6","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847480+0000","last_change":"2024-09-21T04:02:02.767304+0000","last_active":"2024-09-21T04:03:30.847480+0000","last_peered":"2024-09-21T04:03:30.847480+0000","last_clean":"2024-09-21T04:03:30.847480+0000","last_became_active":"2024-09-21T04:02:02.762804+0000","last_became_peered":"2024-09-21T04:02:02.762804+0000","last_unstale":"2024-09-21T04:03:30.847480+0000","last_undegraded":"2024-09-21T04:03:30.847480+0000","last_fullsized":"2024-09-21T04:03:30.847480+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:32.224071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.567991+0000","last_change":"2024-09-21T04:02:02.762932+0000","last_active":"2024-09-21T04:03:30.567991+0000","last_peered":"2024-09-21T04:03:30.567991+0000","last_clean":"2024-09-21T04:03:30.567991+0000","last_became_active":"2024-09-21T04:02:02.762538+0000","last_became_peered":"2024-09-21T04:02:02.762538+0000","last_unstale":"2024-09-21T04:03:30.567991+0000","last_undegraded":"2024-09-21T04:03:30.567991+0000","last_fullsized":"2024-09-21T04:03:30.567991+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:20.724453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"75'10","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.910867+0000","last_change":"2024-09-21T04:02:02.770313+0000","last_active":"2024-09-21T04:03:30.910867+0000","last_peered":"2024-09-21T04:03:30.910867+0000","last_clean":"2024-09-21T04:03:30.910867+0000","last_became_active":"2024-09-21T04:02:02.769910+0000","last_became_peered":"2024-09-21T04:02:02.769910+0000","last_unstale":"2024-09-21T04:03:30.910867+0000","last_undegraded":"2024-09-21T04:03:30.910867+0000","last_fullsized":"2024-09-21T04:03:30.910867+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:16.228371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569300+0000","last_change":"2024-09-21T04:02:02.769258+0000","last_active":"2024-09-21T04:03:30.569300+0000","last_peered":"2024-09-21T04:03:30.569300+0000","last_clean":"2024-09-21T04:03:30.569300+0000","last_became_active":"2024-09-21T04:02:02.768914+0000","last_became_peered":"2024-09-21T04:02:02.768914+0000","last_unstale":"2024-09-21T04:03:30.569300+0000","last_undegraded":"2024-09-21T04:03:30.569300+0000","last_fullsized":"2024-09-21T04:03:30.569300+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:37.426810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569216+0000","last_change":"2024-09-21T04:02:02.755940+0000","last_active":"2024-09-21T04:03:30.569216+0000","last_peered":"2024-09-21T04:03:30.569216+0000","last_clean":"2024-09-21T04:03:30.569216+0000","last_became_active":"2024-09-21T04:02:02.755309+0000","last_became_peered":"2024-09-21T04:02:02.755309+0000","last_unstale":"2024-09-21T04:03:30.569216+0000","last_undegraded":"2024-09-21T04:03:30.569216+0000","last_fullsized":"2024-09-21T04:03:30.569216+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:49.834030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"75'9","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.574842+0000","last_change":"2024-09-21T04:02:02.761068+0000","last_active":"2024-09-21T04:03:30.574842+0000","last_peered":"2024-09-21T04:03:30.574842+0000","last_clean":"2024-09-21T04:03:30.574842+0000","last_became_active":"2024-09-21T04:02:02.760683+0000","last_became_peered":"2024-09-21T04:02:02.760683+0000","last_unstale":"2024-09-21T04:03:30.574842+0000","last_undegraded":"2024-09-21T04:03:30.574842+0000","last_fullsized":"2024-09-21T04:03:30.574842+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:36.627539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.567055+0000","last_change":"2024-09-21T04:02:02.772234+0000","last_active":"2024-09-21T04:03:30.567055+0000","last_peered":"2024-09-21T04:03:30.567055+0000","last_clean":"2024-09-21T04:03:30.567055+0000","last_became_active":"2024-09-21T04:02:02.771825+0000","last_became_peered":"2024-09-21T04:02:02.771825+0000","last_unstale":"2024-09-21T04:03:30.567055+0000","last_undegraded":"2024-09-21T04:03:30.567055+0000","last_fullsized":"2024-09-21T04:03:30.567055+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:33.449745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"75'10","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569735+0000","last_change":"2024-09-21T04:02:02.758643+0000","last_active":"2024-09-21T04:03:30.569735+0000","last_peered":"2024-09-21T04:03:30.569735+0000","last_clean":"2024-09-21T04:03:30.569735+0000","last_became_active":"2024-09-21T04:02:02.758261+0000","last_became_peered":"2024-09-21T04:02:02.758261+0000","last_unstale":"2024-09-21T04:03:30.569735+0000","last_undegraded":"2024-09-21T04:03:30.569735+0000","last_fullsized":"2024-09-21T04:03:30.569735+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:04.165399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569847+0000","last_change":"2024-09-21T04:02:02.759405+0000","last_active":"2024-09-21T04:03:30.569847+0000","last_peered":"2024-09-21T04:03:30.569847+0000","last_clean":"2024-09-21T04:03:30.569847+0000","last_became_active":"2024-09-21T04:02:02.758962+0000","last_became_peered":"2024-09-21T04:02:02.758962+0000","last_unstale":"2024-09-21T04:03:30.569847+0000","last_undegraded":"2024-09-21T04:03:30.569847+0000","last_fullsized":"2024-09-21T04:03:30.569847+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:37.203308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.566627+0000","last_change":"2024-09-21T04:02:02.767518+0000","last_active":"2024-09-21T04:03:30.566627+0000","last_peered":"2024-09-21T04:03:30.566627+0000","last_clean":"2024-09-21T04:03:30.566627+0000","last_became_active":"2024-09-21T04:02:02.767147+0000","last_became_peered":"2024-09-21T04:02:02.767147+0000","last_unstale":"2024-09-21T04:03:30.566627+0000","last_undegraded":"2024-09-21T04:03:30.566627+0000","last_fullsized":"2024-09-21T04:03:30.566627+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:34:13.615218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"75'11","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.910844+0000","last_change":"2024-09-21T04:02:02.756407+0000","last_active":"2024-09-21T04:03:30.910844+0000","last_peered":"2024-09-21T04:03:30.910844+0000","last_clean":"2024-09-21T04:03:30.910844+0000","last_became_active":"2024-09-21T04:02:02.756007+0000","last_became_peered":"2024-09-21T04:02:02.756007+0000","last_unstale":"2024-09-21T04:03:30.910844+0000","last_undegraded":"2024-09-21T04:03:30.910844+0000","last_fullsized":"2024-09-21T04:03:30.910844+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:18.798814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"75'10","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575347+0000","last_change":"2024-09-21T04:02:02.766290+0000","last_active":"2024-09-21T04:03:30.575347+0000","last_peered":"2024-09-21T04:03:30.575347+0000","last_clean":"2024-09-21T04:03:30.575347+0000","last_became_active":"2024-09-21T04:02:02.765660+0000","last_became_peered":"2024-09-21T04:02:02.765660+0000","last_unstale":"2024-09-21T04:03:30.575347+0000","last_undegraded":"2024-09-21T04:03:30.575347+0000","last_fullsized":"2024-09-21T04:03:30.575347+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:55:23.281476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.565585+0000","last_change":"2024-09-21T04:02:02.767943+0000","last_active":"2024-09-21T04:03:30.565585+0000","last_peered":"2024-09-21T04:03:30.565585+0000","last_clean":"2024-09-21T04:03:30.565585+0000","last_became_active":"2024-09-21T04:02:02.767566+0000","last_became_peered":"2024-09-21T04:02:02.767566+0000","last_unstale":"2024-09-21T04:03:30.565585+0000","last_undegraded":"2024-09-21T04:03:30.565585+0000","last_fullsized":"2024-09-21T04:03:30.565585+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:44.221531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"75'6","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.574737+0000","last_change":"2024-09-21T04:02:02.770114+0000","last_active":"2024-09-21T04:03:30.574737+0000","last_peered":"2024-09-21T04:03:30.574737+0000","last_clean":"2024-09-21T04:03:30.574737+0000","last_became_active":"2024-09-21T04:02:02.769732+0000","last_became_peered":"2024-09-21T04:02:02.769732+0000","last_unstale":"2024-09-21T04:03:30.574737+0000","last_undegraded":"2024-09-21T04:03:30.574737+0000","last_fullsized":"2024-09-21T04:03:30.574737+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:39:38.396288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"75'2","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569686+0000","last_change":"2024-09-21T04:02:30.017325+0000","last_active":"2024-09-21T04:03:30.569686+0000","last_peered":"2024-09-21T04:03:30.569686+0000","last_clean":"2024-09-21T04:03:30.569686+0000","last_became_active":"2024-09-21T04:02:30.017014+0000","last_became_peered":"2024-09-21T04:02:30.017014+0000","last_unstale":"2024-09-21T04:03:30.569686+0000","last_undegraded":"2024-09-21T04:03:30.569686+0000","last_fullsized":"2024-09-21T04:03:30.569686+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:28.579436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.26","version":"75'15","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569995+0000","last_change":"2024-09-21T04:02:02.764666+0000","last_active":"2024-09-21T04:03:30.569995+0000","last_peered":"2024-09-21T04:03:30.569995+0000","last_clean":"2024-09-21T04:03:30.569995+0000","last_became_active":"2024-09-21T04:02:02.763931+0000","last_became_peered":"2024-09-21T04:02:02.763931+0000","last_unstale":"2024-09-21T04:03:30.569995+0000","last_undegraded":"2024-09-21T04:03:30.569995+0000","last_fullsized":"2024-09-21T04:03:30.569995+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:39.724770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"75'6","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.566850+0000","last_change":"2024-09-21T04:02:02.774276+0000","last_active":"2024-09-21T04:03:30.566850+0000","last_peered":"2024-09-21T04:03:30.566850+0000","last_clean":"2024-09-21T04:03:30.566850+0000","last_became_active":"2024-09-21T04:02:02.773852+0000","last_became_peered":"2024-09-21T04:02:02.773852+0000","last_unstale":"2024-09-21T04:03:30.566850+0000","last_undegraded":"2024-09-21T04:03:30.566850+0000","last_fullsized":"2024-09-21T04:03:30.566850+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:40.072264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"75'4","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.911090+0000","last_change":"2024-09-21T04:02:02.758520+0000","last_active":"2024-09-21T04:03:30.911090+0000","last_peered":"2024-09-21T04:03:30.911090+0000","last_clean":"2024-09-21T04:03:30.911090+0000","last_became_active":"2024-09-21T04:02:02.757656+0000","last_became_peered":"2024-09-21T04:02:02.757656+0000","last_unstale":"2024-09-21T04:03:30.911090+0000","last_undegraded":"2024-09-21T04:03:30.911090+0000","last_fullsized":"2024-09-21T04:03:30.911090+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:30.854336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569551+0000","last_change":"2024-09-21T04:02:02.756428+0000","last_active":"2024-09-21T04:03:30.569551+0000","last_peered":"2024-09-21T04:03:30.569551+0000","last_clean":"2024-09-21T04:03:30.569551+0000","last_became_active":"2024-09-21T04:02:02.755985+0000","last_became_peered":"2024-09-21T04:02:02.755985+0000","last_unstale":"2024-09-21T04:03:30.569551+0000","last_undegraded":"2024-09-21T04:03:30.569551+0000","last_fullsized":"2024-09-21T04:03:30.569551+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:36.040436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569536+0000","last_change":"2024-09-21T04:02:02.762893+0000","last_active":"2024-09-21T04:03:30.569536+0000","last_peered":"2024-09-21T04:03:30.569536+0000","last_clean":"2024-09-21T04:03:30.569536+0000","last_became_active":"2024-09-21T04:02:02.762343+0000","last_became_peered":"2024-09-21T04:02:02.762343+0000","last_unstale":"2024-09-21T04:03:30.569536+0000","last_undegraded":"2024-09-21T04:03:30.569536+0000","last_fullsized":"2024-09-21T04:03:30.569536+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:45.074711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"75'12","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569534+0000","last_change":"2024-09-21T04:02:02.763942+0000","last_active":"2024-09-21T04:03:30.569534+0000","last_peered":"2024-09-21T04:03:30.569534+0000","last_clean":"2024-09-21T04:03:30.569534+0000","last_became_active":"2024-09-21T04:02:02.763433+0000","last_became_peered":"2024-09-21T04:02:02.763433+0000","last_unstale":"2024-09-21T04:03:30.569534+0000","last_undegraded":"2024-09-21T04:03:30.569534+0000","last_fullsized":"2024-09-21T04:03:30.569534+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:36.139440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"75'9","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.566588+0000","last_change":"2024-09-21T04:02:02.773364+0000","last_active":"2024-09-21T04:03:30.566588+0000","last_peered":"2024-09-21T04:03:30.566588+0000","last_clean":"2024-09-21T04:03:30.566588+0000","last_became_active":"2024-09-21T04:02:02.773040+0000","last_became_peered":"2024-09-21T04:02:02.773040+0000","last_unstale":"2024-09-21T04:03:30.566588+0000","last_undegraded":"2024-09-21T04:03:30.566588+0000","last_fullsized":"2024-09-21T04:03:30.566588+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:48.504476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.572424+0000","last_change":"2024-09-21T04:02:02.774951+0000","last_active":"2024-09-21T04:03:30.572424+0000","last_peered":"2024-09-21T04:03:30.572424+0000","last_clean":"2024-09-21T04:03:30.572424+0000","last_became_active":"2024-09-21T04:02:02.774525+0000","last_became_peered":"2024-09-21T04:02:02.774525+0000","last_unstale":"2024-09-21T04:03:30.572424+0000","last_undegraded":"2024-09-21T04:03:30.572424+0000","last_fullsized":"2024-09-21T04:03:30.572424+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:24.447860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"73'5","reported_seq":132,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:48.958216+0000","last_change":"2024-09-21T04:02:00.743470+0000","last_active":"2024-09-21T04:03:48.958216+0000","last_peered":"2024-09-21T04:03:48.958216+0000","last_clean":"2024-09-21T04:03:48.958216+0000","last_became_active":"2024-09-21T04:02:00.743055+0000","last_became_peered":"2024-09-21T04:02:00.743055+0000","last_unstale":"2024-09-21T04:03:48.958216+0000","last_undegraded":"2024-09-21T04:03:48.958216+0000","last_fullsized":"2024-09-21T04:03:48.958216+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:56.101753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.574871+0000","last_change":"2024-09-21T04:02:02.773399+0000","last_active":"2024-09-21T04:03:30.574871+0000","last_peered":"2024-09-21T04:03:30.574871+0000","last_clean":"2024-09-21T04:03:30.574871+0000","last_became_active":"2024-09-21T04:02:02.772996+0000","last_became_peered":"2024-09-21T04:02:02.772996+0000","last_unstale":"2024-09-21T04:03:30.574871+0000","last_undegraded":"2024-09-21T04:03:30.574871+0000","last_fullsized":"2024-09-21T04:03:30.574871+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:07.256226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"73'3","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569954+0000","last_change":"2024-09-21T04:02:30.026868+0000","last_active":"2024-09-21T04:03:30.569954+0000","last_peered":"2024-09-21T04:03:30.569954+0000","last_clean":"2024-09-21T04:03:30.569954+0000","last_became_active":"2024-09-21T04:02:30.019469+0000","last_became_peered":"2024-09-21T04:02:30.019469+0000","last_unstale":"2024-09-21T04:03:30.569954+0000","last_undegraded":"2024-09-21T04:03:30.569954+0000","last_fullsized":"2024-09-21T04:03:30.569954+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:35.304358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"73'2","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.911058+0000","last_change":"2024-09-21T04:02:00.736630+0000","last_active":"2024-09-21T04:03:30.911058+0000","last_peered":"2024-09-21T04:03:30.911058+0000","last_clean":"2024-09-21T04:03:30.911058+0000","last_became_active":"2024-09-21T04:02:00.736211+0000","last_became_peered":"2024-09-21T04:02:00.736211+0000","last_unstale":"2024-09-21T04:03:30.911058+0000","last_undegraded":"2024-09-21T04:03:30.911058+0000","last_fullsized":"2024-09-21T04:03:30.911058+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:35:37.211617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.565621+0000","last_change":"2024-09-21T04:02:02.763323+0000","last_active":"2024-09-21T04:03:30.565621+0000","last_peered":"2024-09-21T04:03:30.565621+0000","last_clean":"2024-09-21T04:03:30.565621+0000","last_became_active":"2024-09-21T04:02:02.762776+0000","last_became_peered":"2024-09-21T04:02:02.762776+0000","last_unstale":"2024-09-21T04:03:30.565621+0000","last_undegraded":"2024-09-21T04:03:30.565621+0000","last_fullsized":"2024-09-21T04:03:30.565621+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:39.891042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"73'1","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575049+0000","last_change":"2024-09-21T04:02:00.743377+0000","last_active":"2024-09-21T04:03:30.575049+0000","last_peered":"2024-09-21T04:03:30.575049+0000","last_clean":"2024-09-21T04:03:30.575049+0000","last_became_active":"2024-09-21T04:02:00.742973+0000","last_became_peered":"2024-09-21T04:02:00.742973+0000","last_unstale":"2024-09-21T04:03:30.575049+0000","last_undegraded":"2024-09-21T04:03:30.575049+0000","last_fullsized":"2024-09-21T04:03:30.575049+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:55:00.655532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"75'11","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.570104+0000","last_change":"2024-09-21T04:02:02.758721+0000","last_active":"2024-09-21T04:03:30.570104+0000","last_peered":"2024-09-21T04:03:30.570104+0000","last_clean":"2024-09-21T04:03:30.570104+0000","last_became_active":"2024-09-21T04:02:02.758352+0000","last_became_peered":"2024-09-21T04:02:02.758352+0000","last_unstale":"2024-09-21T04:03:30.570104+0000","last_undegraded":"2024-09-21T04:03:30.570104+0000","last_fullsized":"2024-09-21T04:03:30.570104+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:56:49.289510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'4","reported_seq":131,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:51.018477+0000","last_change":"2024-09-21T04:02:00.744591+0000","last_active":"2024-09-21T04:03:51.018477+0000","last_peered":"2024-09-21T04:03:51.018477+0000","last_clean":"2024-09-21T04:03:51.018477+0000","last_became_active":"2024-09-21T04:02:00.744139+0000","last_became_peered":"2024-09-21T04:02:00.744139+0000","last_unstale":"2024-09-21T04:03:51.018477+0000","last_undegraded":"2024-09-21T04:03:51.018477+0000","last_fullsized":"2024-09-21T04:03:51.018477+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:21.331517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"75'10","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575159+0000","last_change":"2024-09-21T04:02:02.762779+0000","last_active":"2024-09-21T04:03:30.575159+0000","last_peered":"2024-09-21T04:03:30.575159+0000","last_clean":"2024-09-21T04:03:30.575159+0000","last_became_active":"2024-09-21T04:02:02.760761+0000","last_became_peered":"2024-09-21T04:02:02.760761+0000","last_unstale":"2024-09-21T04:03:30.575159+0000","last_undegraded":"2024-09-21T04:03:30.575159+0000","last_fullsized":"2024-09-21T04:03:30.575159+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:18.789052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"73'2","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847588+0000","last_change":"2024-09-21T04:02:00.743220+0000","last_active":"2024-09-21T04:03:30.847588+0000","last_peered":"2024-09-21T04:03:30.847588+0000","last_clean":"2024-09-21T04:03:30.847588+0000","last_became_active":"2024-09-21T04:02:00.742636+0000","last_became_peered":"2024-09-21T04:02:00.742636+0000","last_unstale":"2024-09-21T04:03:30.847588+0000","last_undegraded":"2024-09-21T04:03:30.847588+0000","last_fullsized":"2024-09-21T04:03:30.847588+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:10:01.382338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"75'9","reported_seq":38,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569960+0000","last_change":"2024-09-21T04:02:29.869663+0000","last_active":"2024-09-21T04:03:30.569960+0000","last_peered":"2024-09-21T04:03:30.569960+0000","last_clean":"2024-09-21T04:03:30.569960+0000","last_became_active":"2024-09-21T04:02:29.869230+0000","last_became_peered":"2024-09-21T04:02:29.869230+0000","last_unstale":"2024-09-21T04:03:30.569960+0000","last_undegraded":"2024-09-21T04:03:30.569960+0000","last_fullsized":"2024-09-21T04:03:30.569960+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:19:59.987267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"73'4","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575489+0000","last_change":"2024-09-21T04:02:00.853284+0000","last_active":"2024-09-21T04:03:30.575489+0000","last_peered":"2024-09-21T04:03:30.575489+0000","last_clean":"2024-09-21T04:03:30.575489+0000","last_became_active":"2024-09-21T04:02:00.852855+0000","last_became_peered":"2024-09-21T04:02:00.852855+0000","last_unstale":"2024-09-21T04:03:30.575489+0000","last_undegraded":"2024-09-21T04:03:30.575489+0000","last_fullsized":"2024-09-21T04:03:30.575489+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:36.994214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"75'11","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575292+0000","last_change":"2024-09-21T04:02:02.763577+0000","last_active":"2024-09-21T04:03:30.575292+0000","last_peered":"2024-09-21T04:03:30.575292+0000","last_clean":"2024-09-21T04:03:30.575292+0000","last_became_active":"2024-09-21T04:02:02.762970+0000","last_became_peered":"2024-09-21T04:02:02.762970+0000","last_unstale":"2024-09-21T04:03:30.575292+0000","last_undegraded":"2024-09-21T04:03:30.575292+0000","last_fullsized":"2024-09-21T04:03:30.575292+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:49.956226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'35","reported_seq":274,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:51.963042+0000","last_change":"2024-09-21T04:02:00.744203+0000","last_active":"2024-09-21T04:03:51.963042+0000","last_peered":"2024-09-21T04:03:51.963042+0000","last_clean":"2024-09-21T04:03:51.963042+0000","last_became_active":"2024-09-21T04:02:00.743659+0000","last_became_peered":"2024-09-21T04:02:00.743659+0000","last_unstale":"2024-09-21T04:03:51.963042+0000","last_undegraded":"2024-09-21T04:03:51.963042+0000","last_fullsized":"2024-09-21T04:03:51.963042+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:32.108302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71187,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":212,"num_read_kb":175,"num_write":36,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"75'6","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.572693+0000","last_change":"2024-09-21T04:02:02.766002+0000","last_active":"2024-09-21T04:03:30.572693+0000","last_peered":"2024-09-21T04:03:30.572693+0000","last_clean":"2024-09-21T04:03:30.572693+0000","last_became_active":"2024-09-21T04:02:02.765628+0000","last_became_peered":"2024-09-21T04:02:02.765628+0000","last_unstale":"2024-09-21T04:03:30.572693+0000","last_undegraded":"2024-09-21T04:03:30.572693+0000","last_fullsized":"2024-09-21T04:03:30.572693+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:22.251586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"73'6","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569569+0000","last_change":"2024-09-21T04:02:00.741086+0000","last_active":"2024-09-21T04:03:30.569569+0000","last_peered":"2024-09-21T04:03:30.569569+0000","last_clean":"2024-09-21T04:03:30.569569+0000","last_became_active":"2024-09-21T04:02:00.740633+0000","last_became_peered":"2024-09-21T04:02:00.740633+0000","last_unstale":"2024-09-21T04:03:30.569569+0000","last_undegraded":"2024-09-21T04:03:30.569569+0000","last_fullsized":"2024-09-21T04:03:30.569569+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:20.392799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"75'9","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.910819+0000","last_change":"2024-09-21T04:02:02.759651+0000","last_active":"2024-09-21T04:03:30.910819+0000","last_peered":"2024-09-21T04:03:30.910819+0000","last_clean":"2024-09-21T04:03:30.910819+0000","last_became_active":"2024-09-21T04:02:02.759290+0000","last_became_peered":"2024-09-21T04:02:02.759290+0000","last_unstale":"2024-09-21T04:03:30.910819+0000","last_undegraded":"2024-09-21T04:03:30.910819+0000","last_fullsized":"2024-09-21T04:03:30.910819+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:51:32.025703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"73'3","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.574965+0000","last_change":"2024-09-21T04:02:00.744005+0000","last_active":"2024-09-21T04:03:30.574965+0000","last_peered":"2024-09-21T04:03:30.574965+0000","last_clean":"2024-09-21T04:03:30.574965+0000","last_became_active":"2024-09-21T04:02:00.743570+0000","last_became_peered":"2024-09-21T04:02:00.743570+0000","last_unstale":"2024-09-21T04:03:30.574965+0000","last_undegraded":"2024-09-21T04:03:30.574965+0000","last_fullsized":"2024-09-21T04:03:30.574965+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:32.158681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"75'15","reported_seq":44,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847663+0000","last_change":"2024-09-21T04:02:29.869200+0000","last_active":"2024-09-21T04:03:30.847663+0000","last_peered":"2024-09-21T04:03:30.847663+0000","last_clean":"2024-09-21T04:03:30.847663+0000","last_became_active":"2024-09-21T04:02:29.868836+0000","last_became_peered":"2024-09-21T04:02:29.868836+0000","last_unstale":"2024-09-21T04:03:30.847663+0000","last_undegraded":"2024-09-21T04:03:30.847663+0000","last_fullsized":"2024-09-21T04:03:30.847663+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:42:49.030499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'4","reported_seq":132,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:49.959848+0000","last_change":"2024-09-21T04:02:00.741596+0000","last_active":"2024-09-21T04:03:49.959848+0000","last_peered":"2024-09-21T04:03:49.959848+0000","last_clean":"2024-09-21T04:03:49.959848+0000","last_became_active":"2024-09-21T04:02:00.741185+0000","last_became_peered":"2024-09-21T04:02:00.741185+0000","last_unstale":"2024-09-21T04:03:49.959848+0000","last_undegraded":"2024-09-21T04:03:49.959848+0000","last_fullsized":"2024-09-21T04:03:49.959848+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:31.663133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"75'6","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847114+0000","last_change":"2024-09-21T04:02:02.773706+0000","last_active":"2024-09-21T04:03:30.847114+0000","last_peered":"2024-09-21T04:03:30.847114+0000","last_clean":"2024-09-21T04:03:30.847114+0000","last_became_active":"2024-09-21T04:02:02.773259+0000","last_became_peered":"2024-09-21T04:02:02.773259+0000","last_unstale":"2024-09-21T04:03:30.847114+0000","last_undegraded":"2024-09-21T04:03:30.847114+0000","last_fullsized":"2024-09-21T04:03:30.847114+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:50.431982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"73'15","reported_seq":43,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.572386+0000","last_change":"2024-09-21T04:02:00.851335+0000","last_active":"2024-09-21T04:03:30.572386+0000","last_peered":"2024-09-21T04:03:30.572386+0000","last_clean":"2024-09-21T04:03:30.572386+0000","last_became_active":"2024-09-21T04:02:00.850990+0000","last_became_peered":"2024-09-21T04:02:00.850990+0000","last_unstale":"2024-09-21T04:03:30.572386+0000","last_undegraded":"2024-09-21T04:03:30.572386+0000","last_fullsized":"2024-09-21T04:03:30.572386+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:45.731121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"75'6","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.574843+0000","last_change":"2024-09-21T04:02:02.770985+0000","last_active":"2024-09-21T04:03:30.574843+0000","last_peered":"2024-09-21T04:03:30.574843+0000","last_clean":"2024-09-21T04:03:30.574843+0000","last_became_active":"2024-09-21T04:02:02.770602+0000","last_became_peered":"2024-09-21T04:02:02.770602+0000","last_unstale":"2024-09-21T04:03:30.574843+0000","last_undegraded":"2024-09-21T04:03:30.574843+0000","last_fullsized":"2024-09-21T04:03:30.574843+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:37.131113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'3","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.572923+0000","last_change":"2024-09-21T04:02:00.740834+0000","last_active":"2024-09-21T04:03:30.572923+0000","last_peered":"2024-09-21T04:03:30.572923+0000","last_clean":"2024-09-21T04:03:30.572923+0000","last_became_active":"2024-09-21T04:02:00.740439+0000","last_became_peered":"2024-09-21T04:02:00.740439+0000","last_unstale":"2024-09-21T04:03:30.572923+0000","last_undegraded":"2024-09-21T04:03:30.572923+0000","last_fullsized":"2024-09-21T04:03:30.572923+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:04:22.159986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"75'5","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.566818+0000","last_change":"2024-09-21T04:02:30.018671+0000","last_active":"2024-09-21T04:03:30.566818+0000","last_peered":"2024-09-21T04:03:30.566818+0000","last_clean":"2024-09-21T04:03:30.566818+0000","last_became_active":"2024-09-21T04:02:30.018247+0000","last_became_peered":"2024-09-21T04:02:30.018247+0000","last_unstale":"2024-09-21T04:03:30.566818+0000","last_undegraded":"2024-09-21T04:03:30.566818+0000","last_fullsized":"2024-09-21T04:03:30.566818+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:13.501966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"73'2","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.911086+0000","last_change":"2024-09-21T04:02:30.024526+0000","last_active":"2024-09-21T04:03:30.911086+0000","last_peered":"2024-09-21T04:03:30.911086+0000","last_clean":"2024-09-21T04:03:30.911086+0000","last_became_active":"2024-09-21T04:02:30.018811+0000","last_became_peered":"2024-09-21T04:02:30.018811+0000","last_unstale":"2024-09-21T04:03:30.911086+0000","last_undegraded":"2024-09-21T04:03:30.911086+0000","last_fullsized":"2024-09-21T04:03:30.911086+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:43:07.093364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"75'9","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847230+0000","last_change":"2024-09-21T04:02:02.766709+0000","last_active":"2024-09-21T04:03:30.847230+0000","last_peered":"2024-09-21T04:03:30.847230+0000","last_clean":"2024-09-21T04:03:30.847230+0000","last_became_active":"2024-09-21T04:02:02.766342+0000","last_became_peered":"2024-09-21T04:02:02.766342+0000","last_unstale":"2024-09-21T04:03:30.847230+0000","last_undegraded":"2024-09-21T04:03:30.847230+0000","last_fullsized":"2024-09-21T04:03:30.847230+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:26:35.185368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"73'1","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575295+0000","last_change":"2024-09-21T04:02:00.745365+0000","last_active":"2024-09-21T04:03:30.575295+0000","last_peered":"2024-09-21T04:03:30.575295+0000","last_clean":"2024-09-21T04:03:30.575295+0000","last_became_active":"2024-09-21T04:02:00.744968+0000","last_became_peered":"2024-09-21T04:02:00.744968+0000","last_unstale":"2024-09-21T04:03:30.575295+0000","last_undegraded":"2024-09-21T04:03:30.575295+0000","last_fullsized":"2024-09-21T04:03:30.575295+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:56.078169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847168+0000","last_change":"2024-09-21T04:02:02.774877+0000","last_active":"2024-09-21T04:03:30.847168+0000","last_peered":"2024-09-21T04:03:30.847168+0000","last_clean":"2024-09-21T04:03:30.847168+0000","last_became_active":"2024-09-21T04:02:02.774464+0000","last_became_peered":"2024-09-21T04:02:02.774464+0000","last_unstale":"2024-09-21T04:03:30.847168+0000","last_undegraded":"2024-09-21T04:03:30.847168+0000","last_fullsized":"2024-09-21T04:03:30.847168+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:23.393239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"73'3","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.576129+0000","last_change":"2024-09-21T04:03:30.575520+0000","last_active":"2024-09-21T04:03:30.576129+0000","last_peered":"2024-09-21T04:03:30.576129+0000","last_clean":"2024-09-21T04:03:30.576129+0000","last_became_active":"2024-09-21T04:03:30.570445+0000","last_became_peered":"2024-09-21T04:03:30.570445+0000","last_unstale":"2024-09-21T04:03:30.576129+0000","last_undegraded":"2024-09-21T04:03:30.576129+0000","last_fullsized":"2024-09-21T04:03:30.576129+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:15.571668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847199+0000","last_change":"2024-09-21T04:02:02.768164+0000","last_active":"2024-09-21T04:03:30.847199+0000","last_peered":"2024-09-21T04:03:30.847199+0000","last_clean":"2024-09-21T04:03:30.847199+0000","last_became_active":"2024-09-21T04:02:02.766925+0000","last_became_peered":"2024-09-21T04:02:02.766925+0000","last_unstale":"2024-09-21T04:03:30.847199+0000","last_undegraded":"2024-09-21T04:03:30.847199+0000","last_fullsized":"2024-09-21T04:03:30.847199+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:59.707637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":100,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.848055+0000","last_change":"2024-09-21T04:00:48.936996+0000","last_active":"2024-09-21T04:03:30.848055+0000","last_peered":"2024-09-21T04:03:30.848055+0000","last_clean":"2024-09-21T04:03:30.848055+0000","last_became_active":"2024-09-21T04:00:48.931310+0000","last_became_peered":"2024-09-21T04:00:48.931310+0000","last_unstale":"2024-09-21T04:03:30.848055+0000","last_undegraded":"2024-09-21T04:03:30.848055+0000","last_fullsized":"2024-09-21T04:03:30.848055+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T03:57:25.346362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T03:57:25.346362+0000","last_clean_scrub_stamp":"2024-09-21T03:57:25.346362+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:24.165731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"73'3","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.581750+0000","last_change":"2024-09-21T04:03:30.581661+0000","last_active":"2024-09-21T04:03:30.581750+0000","last_peered":"2024-09-21T04:03:30.581750+0000","last_clean":"2024-09-21T04:03:30.581750+0000","last_became_active":"2024-09-21T04:03:30.571414+0000","last_became_peered":"2024-09-21T04:03:30.571414+0000","last_unstale":"2024-09-21T04:03:30.581750+0000","last_undegraded":"2024-09-21T04:03:30.581750+0000","last_fullsized":"2024-09-21T04:03:30.581750+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:49.003453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9],"acting":[4,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.565861+0000","last_change":"2024-09-21T04:02:02.771537+0000","last_active":"2024-09-21T04:03:30.565861+0000","last_peered":"2024-09-21T04:03:30.565861+0000","last_clean":"2024-09-21T04:03:30.565861+0000","last_became_active":"2024-09-21T04:02:02.771151+0000","last_became_peered":"2024-09-21T04:02:02.771151+0000","last_unstale":"2024-09-21T04:03:30.565861+0000","last_undegraded":"2024-09-21T04:03:30.565861+0000","last_fullsized":"2024-09-21T04:03:30.565861+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:21.283515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"73'3","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847919+0000","last_change":"2024-09-21T04:02:30.022634+0000","last_active":"2024-09-21T04:03:30.847919+0000","last_peered":"2024-09-21T04:03:30.847919+0000","last_clean":"2024-09-21T04:03:30.847919+0000","last_became_active":"2024-09-21T04:02:30.018076+0000","last_became_peered":"2024-09-21T04:02:30.018076+0000","last_unstale":"2024-09-21T04:03:30.847919+0000","last_undegraded":"2024-09-21T04:03:30.847919+0000","last_fullsized":"2024-09-21T04:03:30.847919+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:31.650092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'5","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.910802+0000","last_change":"2024-09-21T04:02:02.755223+0000","last_active":"2024-09-21T04:03:30.910802+0000","last_peered":"2024-09-21T04:03:30.910802+0000","last_clean":"2024-09-21T04:03:30.910802+0000","last_became_active":"2024-09-21T04:02:02.754836+0000","last_became_peered":"2024-09-21T04:02:02.754836+0000","last_unstale":"2024-09-21T04:03:30.910802+0000","last_undegraded":"2024-09-21T04:03:30.910802+0000","last_fullsized":"2024-09-21T04:03:30.910802+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:57.912690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'5","reported_seq":131,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:49.959127+0000","last_change":"2024-09-21T04:02:00.851689+0000","last_active":"2024-09-21T04:03:49.959127+0000","last_peered":"2024-09-21T04:03:49.959127+0000","last_clean":"2024-09-21T04:03:49.959127+0000","last_became_active":"2024-09-21T04:02:00.851241+0000","last_became_peered":"2024-09-21T04:02:00.851241+0000","last_unstale":"2024-09-21T04:03:49.959127+0000","last_undegraded":"2024-09-21T04:03:49.959127+0000","last_fullsized":"2024-09-21T04:03:49.959127+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:14.697695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575280+0000","last_change":"2024-09-21T04:02:02.774750+0000","last_active":"2024-09-21T04:03:30.575280+0000","last_peered":"2024-09-21T04:03:30.575280+0000","last_clean":"2024-09-21T04:03:30.575280+0000","last_became_active":"2024-09-21T04:02:02.774310+0000","last_became_peered":"2024-09-21T04:02:02.774310+0000","last_unstale":"2024-09-21T04:03:30.575280+0000","last_undegraded":"2024-09-21T04:03:30.575280+0000","last_fullsized":"2024-09-21T04:03:30.575280+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:07.909035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"73'1","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847647+0000","last_change":"2024-09-21T04:02:00.741461+0000","last_active":"2024-09-21T04:03:30.847647+0000","last_peered":"2024-09-21T04:03:30.847647+0000","last_clean":"2024-09-21T04:03:30.847647+0000","last_became_active":"2024-09-21T04:02:00.741051+0000","last_became_peered":"2024-09-21T04:02:00.741051+0000","last_unstale":"2024-09-21T04:03:30.847647+0000","last_undegraded":"2024-09-21T04:03:30.847647+0000","last_fullsized":"2024-09-21T04:03:30.847647+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:05.944430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.570031+0000","last_change":"2024-09-21T04:02:02.761703+0000","last_active":"2024-09-21T04:03:30.570031+0000","last_peered":"2024-09-21T04:03:30.570031+0000","last_clean":"2024-09-21T04:03:30.570031+0000","last_became_active":"2024-09-21T04:02:02.761296+0000","last_became_peered":"2024-09-21T04:02:02.761296+0000","last_unstale":"2024-09-21T04:03:30.570031+0000","last_undegraded":"2024-09-21T04:03:30.570031+0000","last_fullsized":"2024-09-21T04:03:30.570031+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:57.337167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569828+0000","last_change":"2024-09-21T04:02:00.853584+0000","last_active":"2024-09-21T04:03:30.569828+0000","last_peered":"2024-09-21T04:03:30.569828+0000","last_clean":"2024-09-21T04:03:30.569828+0000","last_became_active":"2024-09-21T04:02:00.853019+0000","last_became_peered":"2024-09-21T04:02:00.853019+0000","last_unstale":"2024-09-21T04:03:30.569828+0000","last_undegraded":"2024-09-21T04:03:30.569828+0000","last_fullsized":"2024-09-21T04:03:30.569828+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:56.186791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'5","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569889+0000","last_change":"2024-09-21T04:02:02.758175+0000","last_active":"2024-09-21T04:03:30.569889+0000","last_peered":"2024-09-21T04:03:30.569889+0000","last_clean":"2024-09-21T04:03:30.569889+0000","last_became_active":"2024-09-21T04:02:02.757715+0000","last_became_peered":"2024-09-21T04:02:02.757715+0000","last_unstale":"2024-09-21T04:03:30.569889+0000","last_undegraded":"2024-09-21T04:03:30.569889+0000","last_fullsized":"2024-09-21T04:03:30.569889+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:40.172569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"73'1","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.572814+0000","last_change":"2024-09-21T04:02:00.853325+0000","last_active":"2024-09-21T04:03:30.572814+0000","last_peered":"2024-09-21T04:03:30.572814+0000","last_clean":"2024-09-21T04:03:30.572814+0000","last_became_active":"2024-09-21T04:02:00.852834+0000","last_became_peered":"2024-09-21T04:02:00.852834+0000","last_unstale":"2024-09-21T04:03:30.572814+0000","last_undegraded":"2024-09-21T04:03:30.572814+0000","last_fullsized":"2024-09-21T04:03:30.572814+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:16.090493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"75'9","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.910869+0000","last_change":"2024-09-21T04:02:02.752402+0000","last_active":"2024-09-21T04:03:30.910869+0000","last_peered":"2024-09-21T04:03:30.910869+0000","last_clean":"2024-09-21T04:03:30.910869+0000","last_became_active":"2024-09-21T04:02:02.751994+0000","last_became_peered":"2024-09-21T04:02:02.751994+0000","last_unstale":"2024-09-21T04:03:30.910869+0000","last_undegraded":"2024-09-21T04:03:30.910869+0000","last_fullsized":"2024-09-21T04:03:30.910869+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:19.525317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'14","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:49.767459+0000","last_change":"2024-09-21T04:02:00.739112+0000","last_active":"2024-09-21T04:03:49.767459+0000","last_peered":"2024-09-21T04:03:49.767459+0000","last_clean":"2024-09-21T04:03:49.767459+0000","last_became_active":"2024-09-21T04:02:00.738613+0000","last_became_peered":"2024-09-21T04:02:00.738613+0000","last_unstale":"2024-09-21T04:03:49.767459+0000","last_undegraded":"2024-09-21T04:03:49.767459+0000","last_fullsized":"2024-09-21T04:03:49.767459+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:18:34.334576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"75'5","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.565764+0000","last_change":"2024-09-21T04:02:02.763345+0000","last_active":"2024-09-21T04:03:30.565764+0000","last_peered":"2024-09-21T04:03:30.565764+0000","last_clean":"2024-09-21T04:03:30.565764+0000","last_became_active":"2024-09-21T04:02:02.762987+0000","last_became_peered":"2024-09-21T04:02:02.762987+0000","last_unstale":"2024-09-21T04:03:30.565764+0000","last_undegraded":"2024-09-21T04:03:30.565764+0000","last_fullsized":"2024-09-21T04:03:30.565764+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:36.239267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"73'4","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.567663+0000","last_change":"2024-09-21T04:02:01.258208+0000","last_active":"2024-09-21T04:03:30.567663+0000","last_peered":"2024-09-21T04:03:30.567663+0000","last_clean":"2024-09-21T04:03:30.567663+0000","last_became_active":"2024-09-21T04:02:01.257780+0000","last_became_peered":"2024-09-21T04:02:01.257780+0000","last_unstale":"2024-09-21T04:03:30.567663+0000","last_undegraded":"2024-09-21T04:03:30.567663+0000","last_fullsized":"2024-09-21T04:03:30.567663+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:20.161792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"75'7","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569870+0000","last_change":"2024-09-21T04:02:02.761149+0000","last_active":"2024-09-21T04:03:30.569870+0000","last_peered":"2024-09-21T04:03:30.569870+0000","last_clean":"2024-09-21T04:03:30.569870+0000","last_became_active":"2024-09-21T04:02:02.760771+0000","last_became_peered":"2024-09-21T04:02:02.760771+0000","last_unstale":"2024-09-21T04:03:30.569870+0000","last_undegraded":"2024-09-21T04:03:30.569870+0000","last_fullsized":"2024-09-21T04:03:30.569870+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:20.645377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"73'2","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575047+0000","last_change":"2024-09-21T04:02:00.853580+0000","last_active":"2024-09-21T04:03:30.575047+0000","last_peered":"2024-09-21T04:03:30.575047+0000","last_clean":"2024-09-21T04:03:30.575047+0000","last_became_active":"2024-09-21T04:02:00.853091+0000","last_became_peered":"2024-09-21T04:02:00.853091+0000","last_unstale":"2024-09-21T04:03:30.575047+0000","last_undegraded":"2024-09-21T04:03:30.575047+0000","last_fullsized":"2024-09-21T04:03:30.575047+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:32.893695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"75'2","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.572389+0000","last_change":"2024-09-21T04:02:02.769686+0000","last_active":"2024-09-21T04:03:30.572389+0000","last_peered":"2024-09-21T04:03:30.572389+0000","last_clean":"2024-09-21T04:03:30.572389+0000","last_became_active":"2024-09-21T04:02:02.769293+0000","last_became_peered":"2024-09-21T04:02:02.769293+0000","last_unstale":"2024-09-21T04:03:30.572389+0000","last_undegraded":"2024-09-21T04:03:30.572389+0000","last_fullsized":"2024-09-21T04:03:30.572389+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:10.984624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"73'3","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.565759+0000","last_change":"2024-09-21T04:02:00.741556+0000","last_active":"2024-09-21T04:03:30.565759+0000","last_peered":"2024-09-21T04:03:30.565759+0000","last_clean":"2024-09-21T04:03:30.565759+0000","last_became_active":"2024-09-21T04:02:00.741188+0000","last_became_peered":"2024-09-21T04:02:00.741188+0000","last_unstale":"2024-09-21T04:03:30.565759+0000","last_undegraded":"2024-09-21T04:03:30.565759+0000","last_fullsized":"2024-09-21T04:03:30.565759+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:23.566814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"75'9","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.566789+0000","last_change":"2024-09-21T04:02:02.769163+0000","last_active":"2024-09-21T04:03:30.566789+0000","last_peered":"2024-09-21T04:03:30.566789+0000","last_clean":"2024-09-21T04:03:30.566789+0000","last_became_active":"2024-09-21T04:02:02.768804+0000","last_became_peered":"2024-09-21T04:02:02.768804+0000","last_unstale":"2024-09-21T04:03:30.566789+0000","last_undegraded":"2024-09-21T04:03:30.566789+0000","last_fullsized":"2024-09-21T04:03:30.566789+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:38:12.576553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"73'1","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569475+0000","last_change":"2024-09-21T04:02:00.743091+0000","last_active":"2024-09-21T04:03:30.569475+0000","last_peered":"2024-09-21T04:03:30.569475+0000","last_clean":"2024-09-21T04:03:30.569475+0000","last_became_active":"2024-09-21T04:02:00.742665+0000","last_became_peered":"2024-09-21T04:02:00.742665+0000","last_unstale":"2024-09-21T04:03:30.569475+0000","last_undegraded":"2024-09-21T04:03:30.569475+0000","last_fullsized":"2024-09-21T04:03:30.569475+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:09.698421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"75'8","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.565845+0000","last_change":"2024-09-21T04:02:02.761985+0000","last_active":"2024-09-21T04:03:30.565845+0000","last_peered":"2024-09-21T04:03:30.565845+0000","last_clean":"2024-09-21T04:03:30.565845+0000","last_became_active":"2024-09-21T04:02:02.761624+0000","last_became_peered":"2024-09-21T04:02:02.761624+0000","last_unstale":"2024-09-21T04:03:30.565845+0000","last_undegraded":"2024-09-21T04:03:30.565845+0000","last_fullsized":"2024-09-21T04:03:30.565845+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:12.912221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"73'4","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.572449+0000","last_change":"2024-09-21T04:02:30.024288+0000","last_active":"2024-09-21T04:03:30.572449+0000","last_peered":"2024-09-21T04:03:30.572449+0000","last_clean":"2024-09-21T04:03:30.572449+0000","last_became_active":"2024-09-21T04:02:30.018929+0000","last_became_peered":"2024-09-21T04:02:30.018929+0000","last_unstale":"2024-09-21T04:03:30.572449+0000","last_undegraded":"2024-09-21T04:03:30.572449+0000","last_fullsized":"2024-09-21T04:03:30.572449+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:19:21.400082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"75'5","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.847220+0000","last_change":"2024-09-21T04:02:02.761357+0000","last_active":"2024-09-21T04:03:30.847220+0000","last_peered":"2024-09-21T04:03:30.847220+0000","last_clean":"2024-09-21T04:03:30.847220+0000","last_became_active":"2024-09-21T04:02:02.760984+0000","last_became_peered":"2024-09-21T04:02:02.760984+0000","last_unstale":"2024-09-21T04:03:30.847220+0000","last_undegraded":"2024-09-21T04:03:30.847220+0000","last_fullsized":"2024-09-21T04:03:30.847220+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:55.388768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"75'15","reported_seq":44,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575165+0000","last_change":"2024-09-21T04:02:30.019234+0000","last_active":"2024-09-21T04:03:30.575165+0000","last_peered":"2024-09-21T04:03:30.575165+0000","last_clean":"2024-09-21T04:03:30.575165+0000","last_became_active":"2024-09-21T04:02:30.018778+0000","last_became_peered":"2024-09-21T04:02:30.018778+0000","last_unstale":"2024-09-21T04:03:30.575165+0000","last_undegraded":"2024-09-21T04:03:30.575165+0000","last_fullsized":"2024-09-21T04:03:30.575165+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:31.227144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"73'2","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.915768+0000","last_change":"2024-09-21T04:03:30.915685+0000","last_active":"2024-09-21T04:03:30.915768+0000","last_peered":"2024-09-21T04:03:30.915768+0000","last_clean":"2024-09-21T04:03:30.915768+0000","last_became_active":"2024-09-21T04:03:30.912802+0000","last_became_peered":"2024-09-21T04:03:30.912802+0000","last_unstale":"2024-09-21T04:03:30.915768+0000","last_undegraded":"2024-09-21T04:03:30.915768+0000","last_fullsized":"2024-09-21T04:03:30.915768+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:15:58.295051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,6],"acting":[8,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"75'9","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.575216+0000","last_change":"2024-09-21T04:02:02.762336+0000","last_active":"2024-09-21T04:03:30.575216+0000","last_peered":"2024-09-21T04:03:30.575216+0000","last_clean":"2024-09-21T04:03:30.575216+0000","last_became_active":"2024-09-21T04:02:02.761753+0000","last_became_peered":"2024-09-21T04:02:02.761753+0000","last_unstale":"2024-09-21T04:03:30.575216+0000","last_undegraded":"2024-09-21T04:03:30.575216+0000","last_fullsized":"2024-09-21T04:03:30.575216+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:39.229760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"73'4","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.572719+0000","last_change":"2024-09-21T04:02:00.743495+0000","last_active":"2024-09-21T04:03:30.572719+0000","last_peered":"2024-09-21T04:03:30.572719+0000","last_clean":"2024-09-21T04:03:30.572719+0000","last_became_active":"2024-09-21T04:02:00.743135+0000","last_became_peered":"2024-09-21T04:02:00.743135+0000","last_unstale":"2024-09-21T04:03:30.572719+0000","last_undegraded":"2024-09-21T04:03:30.572719+0000","last_fullsized":"2024-09-21T04:03:30.572719+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:16.481148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"75'6","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.569827+0000","last_change":"2024-09-21T04:02:02.766682+0000","last_active":"2024-09-21T04:03:30.569827+0000","last_peered":"2024-09-21T04:03:30.569827+0000","last_clean":"2024-09-21T04:03:30.569827+0000","last_became_active":"2024-09-21T04:02:02.766299+0000","last_became_peered":"2024-09-21T04:02:02.766299+0000","last_unstale":"2024-09-21T04:03:30.569827+0000","last_undegraded":"2024-09-21T04:03:30.569827+0000","last_fullsized":"2024-09-21T04:03:30.569827+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:07:25.635908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"73'2","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.566238+0000","last_change":"2024-09-21T04:02:00.852281+0000","last_active":"2024-09-21T04:03:30.566238+0000","last_peered":"2024-09-21T04:03:30.566238+0000","last_clean":"2024-09-21T04:03:30.566238+0000","last_became_active":"2024-09-21T04:02:00.851900+0000","last_became_peered":"2024-09-21T04:02:00.851900+0000","last_unstale":"2024-09-21T04:03:30.566238+0000","last_undegraded":"2024-09-21T04:03:30.566238+0000","last_fullsized":"2024-09-21T04:03:30.566238+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:37.115541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"75'5","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.566108+0000","last_change":"2024-09-21T04:02:02.770467+0000","last_active":"2024-09-21T04:03:30.566108+0000","last_peered":"2024-09-21T04:03:30.566108+0000","last_clean":"2024-09-21T04:03:30.566108+0000","last_became_active":"2024-09-21T04:02:02.770099+0000","last_became_peered":"2024-09-21T04:02:02.770099+0000","last_unstale":"2024-09-21T04:03:30.566108+0000","last_undegraded":"2024-09-21T04:03:30.566108+0000","last_fullsized":"2024-09-21T04:03:30.566108+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:02:01.728685+0000","last_clean_scrub_stamp":"2024-09-21T04:02:01.728685+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:38.806298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"73'9","reported_seq":54,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T04:03:30.570015+0000","last_change":"2024-09-21T04:02:29.878053+0000","last_active":"2024-09-21T04:03:30.570015+0000","last_peered":"2024-09-21T04:03:30.570015+0000","last_clean":"2024-09-21T04:03:30.570015+0000","last_became_active":"2024-09-21T04:02:29.869007+0000","last_became_peered":"2024-09-21T04:02:29.869007+0000","last_unstale":"2024-09-21T04:03:30.570015+0000","last_undegraded":"2024-09-21T04:03:30.570015+0000","last_fullsized":"2024-09-21T04:03:30.570015+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T04:01:59.720589+0000","last_clean_scrub_stamp":"2024-09-21T04:01:59.720589+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:50.509849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75105,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":625,"num_read_kb":392,"num_write":179,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1198,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":217008,"data_compressed":5676,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282548,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":67,"seq":287762808868,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":847872,"data_stored":424484,"data_compressed":11589,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":62,"seq":266287972392,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":1087201,"data_compressed":49407,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":57,"seq":244813135918,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":856064,"data_stored":424104,"data_compressed":11589,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332147,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":856064,"data_stored":424024,"data_compressed":11589,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495671,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":1016071,"data_compressed":47515,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659197,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":843776,"data_stored":423493,"data_compressed":11589,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855426,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":1015247,"data_compressed":47515,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018951,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":424820,"data_compressed":11589,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":860160,"data_stored":423804,"data_compressed":11589,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378705,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":929792,"data_stored":490501,"data_compressed":13481,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542230,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":489733,"data_compressed":13481,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705755,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":864256,"data_stored":424764,"data_compressed":11589,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34306,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34306,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34306,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35926,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":66447,"data_compressed":1892,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67215,"data_compressed":1892,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":73430,"data_compressed":1892,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T04:03:55.215 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:03:55.509 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:55.889 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:55 smithi064 ceph-mon[31180]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:03:55.890 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:55 smithi064 ceph-mon[31180]: pgmap v318: 97 pgs: 97 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-21T04:03:55.890 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:55 smithi064 ceph-mon[31180]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:03:55.890 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:55 smithi064 ceph-mon[31180]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:03:55.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.980+0000 7f09bd26e640 1 -- 172.21.15.64:0/281677781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09b8105380 msgr2=0x7f09b810d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:55.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.980+0000 7f09bd26e640 1 --2- 172.21.15.64:0/281677781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09b8105380 0x7f09b810d8d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009900 tx=0x7f09a002a3f0 comp rx=0 tx=0).stop 2024-09-21T04:03:55.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.980+0000 7f09bd26e640 1 -- 172.21.15.64:0/281677781 shutdown_connections 2024-09-21T04:03:55.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.980+0000 7f09bd26e640 1 --2- 172.21.15.64:0/281677781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09b810de60 0x7f09b8110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:55.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.980+0000 7f09bd26e640 1 --2- 172.21.15.64:0/281677781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09b8105380 0x7f09b810d8d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:55.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.980+0000 7f09bd26e640 1 --2- 172.21.15.64:0/281677781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09b8104a30 0x7f09b8104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:55.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.980+0000 7f09bd26e640 1 -- 172.21.15.64:0/281677781 >> 172.21.15.64:0/281677781 conn(0x7f09b80797b0 msgr2=0x7f09b8079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:55.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.980+0000 7f09bd26e640 1 -- 172.21.15.64:0/281677781 shutdown_connections 2024-09-21T04:03:55.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.980+0000 7f09bd26e640 1 -- 172.21.15.64:0/281677781 wait complete. 2024-09-21T04:03:55.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.981+0000 7f09bd26e640 1 Processor -- start 2024-09-21T04:03:55.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.981+0000 7f09bd26e640 1 -- start start 2024-09-21T04:03:55.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.981+0000 7f09bd26e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09b8104a30 0x7f09b8102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:55.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.981+0000 7f09bd26e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09b8105380 0x7f09b8102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:55.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.981+0000 7f09bd26e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09b810de60 0x7f09b81031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:55.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.981+0000 7f09bd26e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f09b8113aa0 con 0x7f09b8104a30 2024-09-21T04:03:55.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.981+0000 7f09bd26e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f09b8113920 con 0x7f09b8105380 2024-09-21T04:03:55.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.981+0000 7f09bd26e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f09b8113c20 con 0x7f09b810de60 2024-09-21T04:03:55.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.981+0000 7f09b6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09b8104a30 0x7f09b8102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:55.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.981+0000 7f09b6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09b8104a30 0x7f09b8102770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50028/0 (socket says 172.21.15.64:50028) 2024-09-21T04:03:55.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.981+0000 7f09b6ffd640 1 -- 172.21.15.64:0/3580368729 learned_addr learned my addr 172.21.15.64:0/3580368729 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:55.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.981+0000 7f09b67fc640 1 --2- 172.21.15.64:0/3580368729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09b8105380 0x7f09b8102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:55.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.982+0000 7f09b77fe640 1 --2- 172.21.15.64:0/3580368729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09b810de60 0x7f09b81031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:55.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.982+0000 7f09b6ffd640 1 -- 172.21.15.64:0/3580368729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09b810de60 msgr2=0x7f09b81031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:55.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.982+0000 7f09b6ffd640 1 --2- 172.21.15.64:0/3580368729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09b810de60 0x7f09b81031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:55.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.982+0000 7f09b6ffd640 1 -- 172.21.15.64:0/3580368729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09b8105380 msgr2=0x7f09b8102cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:55.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.982+0000 7f09b6ffd640 1 --2- 172.21.15.64:0/3580368729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09b8105380 0x7f09b8102cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:55.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.982+0000 7f09b6ffd640 1 -- 172.21.15.64:0/3580368729 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b81a8720 con 0x7f09b8104a30 2024-09-21T04:03:55.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.982+0000 7f09b77fe640 1 --2- 172.21.15.64:0/3580368729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09b810de60 0x7f09b81031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:03:55.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.982+0000 7f09b6ffd640 1 --2- 172.21.15.64:0/3580368729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09b8104a30 0x7f09b8102770 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f09b000c920 tx=0x7f09b000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:55.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.982+0000 7f09affff640 1 -- 172.21.15.64:0/3580368729 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b0007c70 con 0x7f09b8104a30 2024-09-21T04:03:55.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.982+0000 7f09affff640 1 -- 172.21.15.64:0/3580368729 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09b0007e10 con 0x7f09b8104a30 2024-09-21T04:03:55.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.982+0000 7f09bd26e640 1 -- 172.21.15.64:0/3580368729 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b81a8a10 con 0x7f09b8104a30 2024-09-21T04:03:55.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.982+0000 7f09affff640 1 -- 172.21.15.64:0/3580368729 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b0018820 con 0x7f09b8104a30 2024-09-21T04:03:55.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.982+0000 7f09bd26e640 1 -- 172.21.15.64:0/3580368729 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b81a8f70 con 0x7f09b8104a30 2024-09-21T04:03:55.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.983+0000 7f09bd26e640 1 -- 172.21.15.64:0/3580368729 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09b8104e40 con 0x7f09b8104a30 2024-09-21T04:03:55.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.985+0000 7f09affff640 1 -- 172.21.15.64:0/3580368729 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) ==== 154580+0+0 (secure 0 0 0) 0x7f09b000f450 con 0x7f09b8104a30 2024-09-21T04:03:55.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.986+0000 7f09affff640 1 --2- 172.21.15.64:0/3580368729 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f09a40b6670 0x7f09a40b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:55.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.986+0000 7f09b67fc640 1 --2- 172.21.15.64:0/3580368729 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f09a40b6670 0x7f09a40b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:55.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.986+0000 7f09affff640 1 -- 172.21.15.64:0/3580368729 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f09b00e7a70 con 0x7f09b8104a30 2024-09-21T04:03:55.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.986+0000 7f09b67fc640 1 --2- 172.21.15.64:0/3580368729 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f09a40b6670 0x7f09a40b8b80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009230 tx=0x7f09a0031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:55.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:55.989+0000 7f09affff640 1 -- 172.21.15.64:0/3580368729 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f09b0014030 con 0x7f09b8104a30 2024-09-21T04:03:56.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:55 smithi119 ceph-mon[34811]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:03:56.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:55 smithi119 ceph-mon[34811]: pgmap v318: 97 pgs: 97 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-21T04:03:56.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:55 smithi119 ceph-mon[34811]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:03:56.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:55 smithi119 ceph-mon[34811]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:03:56.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:55 smithi131 ceph-mon[35193]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:03:56.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:55 smithi131 ceph-mon[35193]: pgmap v318: 97 pgs: 97 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-21T04:03:56.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:55 smithi131 ceph-mon[35193]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:03:56.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:55 smithi131 ceph-mon[35193]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T04:03:56.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:56.215+0000 7f09bd26e640 1 -- 172.21.15.64:0/3580368729 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09b8103a70 con 0x7f09b8104a30 2024-09-21T04:03:56.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:56.215+0000 7f09affff640 1 -- 172.21.15.64:0/3580368729 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f09b00ae160 con 0x7f09b8104a30 2024-09-21T04:03:56.218 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:03:56.218 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-21T04:03:50:600319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14805,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/97995119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":97995119},{"type":"v1","addr":"172.21.15.64:6839","nonce":97995119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/390891068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":390891068},{"type":"v1","addr":"172.21.15.64:6841","nonce":390891068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:02:03.354744+0000","modified":"2024-09-21T04:03:50.212837+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24710,"mds_1":24680,"mds_2":24661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24634":{"gid":24634,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6839/782190424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782190424},{"type":"v1","addr":"172.21.15.131:6839","nonce":782190424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24680":{"gid":24680,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24710":{"gid":24710,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6839/1319793636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1319793636},{"type":"v1","addr":"172.21.15.119:6839","nonce":1319793636}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24710,"qdb_cluster":[24710,24680,24661]},"id":1}]} 2024-09-21T04:03:56.218 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-21T04:03:56.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:56.218+0000 7f09bd26e640 1 -- 172.21.15.64:0/3580368729 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f09a40b6670 msgr2=0x7f09a40b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:56.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:56.218+0000 7f09bd26e640 1 --2- 172.21.15.64:0/3580368729 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f09a40b6670 0x7f09a40b8b80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009230 tx=0x7f09a0031400 comp rx=0 tx=0).stop 2024-09-21T04:03:56.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:56.218+0000 7f09bd26e640 1 -- 172.21.15.64:0/3580368729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09b8104a30 msgr2=0x7f09b8102770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:56.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:56.218+0000 7f09bd26e640 1 --2- 172.21.15.64:0/3580368729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09b8104a30 0x7f09b8102770 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f09b000c920 tx=0x7f09b000cde0 comp rx=0 tx=0).stop 2024-09-21T04:03:56.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:56.218+0000 7f09bd26e640 1 -- 172.21.15.64:0/3580368729 shutdown_connections 2024-09-21T04:03:56.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:56.218+0000 7f09bd26e640 1 --2- 172.21.15.64:0/3580368729 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f09a40b6670 0x7f09a40b8b80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:56.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:56.218+0000 7f09bd26e640 1 --2- 172.21.15.64:0/3580368729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09b810de60 0x7f09b81031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:56.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:56.218+0000 7f09bd26e640 1 --2- 172.21.15.64:0/3580368729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09b8105380 0x7f09b8102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:56.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:56.218+0000 7f09bd26e640 1 --2- 172.21.15.64:0/3580368729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f09b8104a30 0x7f09b8102770 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:56.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:56.218+0000 7f09bd26e640 1 -- 172.21.15.64:0/3580368729 >> 172.21.15.64:0/3580368729 conn(0x7f09b80797b0 msgr2=0x7f09b8100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:56.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:56.218+0000 7f09bd26e640 1 -- 172.21.15.64:0/3580368729 shutdown_connections 2024-09-21T04:03:56.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:56.218+0000 7f09bd26e640 1 -- 172.21.15.64:0/3580368729 wait complete. 2024-09-21T04:03:57.004 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:56 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3580368729' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:03:57.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:56 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3580368729' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:03:57.038 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T04:03:57.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:56 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3580368729' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:03:57.301 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:57.683 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:57 smithi064 ceph-mon[31180]: pgmap v319: 97 pgs: 97 active+clean; 651 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-21T04:03:57.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.774+0000 7fe522bb3640 1 -- 172.21.15.64:0/2606250631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe51c103b60 msgr2=0x7fe51c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:57.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.774+0000 7fe522bb3640 1 --2- 172.21.15.64:0/2606250631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe51c103b60 0x7fe51c103f70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe504009900 tx=0x7fe50402a410 comp rx=0 tx=0).stop 2024-09-21T04:03:57.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.774+0000 7fe522bb3640 1 -- 172.21.15.64:0/2606250631 shutdown_connections 2024-09-21T04:03:57.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.774+0000 7fe522bb3640 1 --2- 172.21.15.64:0/2606250631 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe51c109df0 0x7fe51c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:57.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.774+0000 7fe522bb3640 1 --2- 172.21.15.64:0/2606250631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe51c1094a0 0x7fe51c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:57.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.774+0000 7fe522bb3640 1 --2- 172.21.15.64:0/2606250631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe51c103b60 0x7fe51c103f70 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:57.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.774+0000 7fe522bb3640 1 -- 172.21.15.64:0/2606250631 >> 172.21.15.64:0/2606250631 conn(0x7fe51c07b850 msgr2=0x7fe51c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:57.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.774+0000 7fe522bb3640 1 -- 172.21.15.64:0/2606250631 shutdown_connections 2024-09-21T04:03:57.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.774+0000 7fe522bb3640 1 -- 172.21.15.64:0/2606250631 wait complete. 2024-09-21T04:03:57.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.775+0000 7fe522bb3640 1 Processor -- start 2024-09-21T04:03:57.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.775+0000 7fe522bb3640 1 -- start start 2024-09-21T04:03:57.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.775+0000 7fe522bb3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe51c103b60 0x7fe51c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:57.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.775+0000 7fe522bb3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe51c1094a0 0x7fe51c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:57.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.775+0000 7fe522bb3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe51c109df0 0x7fe51c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:57.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.775+0000 7fe522bb3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c113ab0 con 0x7fe51c103b60 2024-09-21T04:03:57.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.775+0000 7fe522bb3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c113930 con 0x7fe51c1094a0 2024-09-21T04:03:57.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.775+0000 7fe522bb3640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c113c30 con 0x7fe51c109df0 2024-09-21T04:03:57.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.775+0000 7fe52094f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe51c103b60 0x7fe51c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:57.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.775+0000 7fe52094f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe51c103b60 0x7fe51c075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50038/0 (socket says 172.21.15.64:50038) 2024-09-21T04:03:57.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.776+0000 7fe52094f640 1 -- 172.21.15.64:0/760038410 learned_addr learned my addr 172.21.15.64:0/760038410 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:57.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.776+0000 7fe51bfff640 1 --2- 172.21.15.64:0/760038410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe51c1094a0 0x7fe51c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:57.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.776+0000 7fe521150640 1 --2- 172.21.15.64:0/760038410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe51c109df0 0x7fe51c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:57.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.776+0000 7fe52094f640 1 -- 172.21.15.64:0/760038410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe51c109df0 msgr2=0x7fe51c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:57.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.776+0000 7fe52094f640 1 --2- 172.21.15.64:0/760038410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe51c109df0 0x7fe51c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:57.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.776+0000 7fe52094f640 1 -- 172.21.15.64:0/760038410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe51c1094a0 msgr2=0x7fe51c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:57.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.776+0000 7fe52094f640 1 --2- 172.21.15.64:0/760038410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe51c1094a0 0x7fe51c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:57.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.776+0000 7fe52094f640 1 -- 172.21.15.64:0/760038410 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51c1a87a0 con 0x7fe51c103b60 2024-09-21T04:03:57.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.776+0000 7fe521150640 1 --2- 172.21.15.64:0/760038410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe51c109df0 0x7fe51c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:03:57.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.776+0000 7fe52094f640 1 --2- 172.21.15.64:0/760038410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe51c103b60 0x7fe51c075710 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe504005940 tx=0x7fe5040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:57.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.776+0000 7fe519ffb640 1 -- 172.21.15.64:0/760038410 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50403f030 con 0x7fe51c103b60 2024-09-21T04:03:57.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.776+0000 7fe522bb3640 1 -- 172.21.15.64:0/760038410 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe51c1a8a30 con 0x7fe51c103b60 2024-09-21T04:03:57.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.776+0000 7fe519ffb640 1 -- 172.21.15.64:0/760038410 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe504003cb0 con 0x7fe51c103b60 2024-09-21T04:03:57.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.776+0000 7fe519ffb640 1 -- 172.21.15.64:0/760038410 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe504035070 con 0x7fe51c103b60 2024-09-21T04:03:57.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.777+0000 7fe522bb3640 1 -- 172.21.15.64:0/760038410 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe51c1a8f10 con 0x7fe51c103b60 2024-09-21T04:03:57.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.777+0000 7fe522bb3640 1 -- 172.21.15.64:0/760038410 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe51c103f70 con 0x7fe51c103b60 2024-09-21T04:03:57.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.779+0000 7fe519ffb640 1 -- 172.21.15.64:0/760038410 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) ==== 154580+0+0 (secure 0 0 0) 0x7fe504042030 con 0x7fe51c103b60 2024-09-21T04:03:57.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.780+0000 7fe519ffb640 1 --2- 172.21.15.64:0/760038410 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe5080b65f0 0x7fe5080b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:57.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.780+0000 7fe51bfff640 1 --2- 172.21.15.64:0/760038410 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe5080b65f0 0x7fe5080b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:57.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.780+0000 7fe519ffb640 1 -- 172.21.15.64:0/760038410 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fe504108210 con 0x7fe51c103b60 2024-09-21T04:03:57.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.780+0000 7fe51bfff640 1 --2- 172.21.15.64:0/760038410 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe5080b65f0 0x7fe5080b8b00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe50c005fd0 tx=0x7fe50c005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:57.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.783+0000 7fe519ffb640 1 -- 172.21.15.64:0/760038410 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe5040ce7f0 con 0x7fe51c103b60 2024-09-21T04:03:57.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.972+0000 7fe522bb3640 1 -- 172.21.15.64:0/760038410 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe51c02e640 con 0x7fe51c103b60 2024-09-21T04:03:57.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.973+0000 7fe519ffb640 1 -- 172.21.15.64:0/760038410 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+23230 (secure 0 0 0) 0x7fe5040d2bd0 con 0x7fe51c103b60 2024-09-21T04:03:57.976 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:03:57.976 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":77,"fsid":"e7c927ba-77cc-11ef-baf5-efdc52797490","created":"2024-09-21T03:53:41.215073+0000","modified":"2024-09-21T04:03:30.555854+0000","last_up_change":"2024-09-21T04:01:11.986949+0000","last_in_change":"2024-09-21T04:00:48.707791+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:57:24.854438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T04:01:58.728355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T04:02:01.300385+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb947c10-98e6-4e3a-8bbd-8753951d870b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6803","nonce":3692023035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6805","nonce":3692023035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6809","nonce":3692023035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6807","nonce":3692023035}]},"public_addr":"172.21.15.64:6803/3692023035","cluster_addr":"172.21.15.64:6805/3692023035","heartbeat_back_addr":"172.21.15.64:6809/3692023035","heartbeat_front_addr":"172.21.15.64:6807/3692023035","state":["exists","up"]},{"osd":1,"uuid":"29b8c92d-82fa-4175-b3e2-711917b7f8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6801","nonce":3363415998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6803","nonce":3363415998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6807","nonce":3363415998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6805","nonce":3363415998}]},"public_addr":"172.21.15.119:6801/3363415998","cluster_addr":"172.21.15.119:6803/3363415998","heartbeat_back_addr":"172.21.15.119:6807/3363415998","heartbeat_front_addr":"172.21.15.119:6805/3363415998","state":["exists","up"]},{"osd":2,"uuid":"a2273fe8-df55-40f7-afda-74ab09c52c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6801","nonce":3422924179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6803","nonce":3422924179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6807","nonce":3422924179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6805","nonce":3422924179}]},"public_addr":"172.21.15.131:6801/3422924179","cluster_addr":"172.21.15.131:6803/3422924179","heartbeat_back_addr":"172.21.15.131:6807/3422924179","heartbeat_front_addr":"172.21.15.131:6805/3422924179","state":["exists","up"]},{"osd":3,"uuid":"f1634064-b108-41d2-a78f-c33607b08a70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6811","nonce":2022952729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6813","nonce":2022952729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6817","nonce":2022952729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6815","nonce":2022952729}]},"public_addr":"172.21.15.64:6811/2022952729","cluster_addr":"172.21.15.64:6813/2022952729","heartbeat_back_addr":"172.21.15.64:6817/2022952729","heartbeat_front_addr":"172.21.15.64:6815/2022952729","state":["exists","up"]},{"osd":4,"uuid":"75b81a8d-fa52-411a-bfda-aa69bb70a20f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6809","nonce":323030958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6811","nonce":323030958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6815","nonce":323030958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6813","nonce":323030958}]},"public_addr":"172.21.15.119:6809/323030958","cluster_addr":"172.21.15.119:6811/323030958","heartbeat_back_addr":"172.21.15.119:6815/323030958","heartbeat_front_addr":"172.21.15.119:6813/323030958","state":["exists","up"]},{"osd":5,"uuid":"0e4895b5-42e4-4b2c-a9c3-692f287cb841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6809","nonce":3021788088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6811","nonce":3021788088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6815","nonce":3021788088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6813","nonce":3021788088}]},"public_addr":"172.21.15.131:6809/3021788088","cluster_addr":"172.21.15.131:6811/3021788088","heartbeat_back_addr":"172.21.15.131:6815/3021788088","heartbeat_front_addr":"172.21.15.131:6813/3021788088","state":["exists","up"]},{"osd":6,"uuid":"1fa5c2b7-bce0-43a1-a26b-d029eb3731fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6819","nonce":1913824324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6821","nonce":1913824324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6825","nonce":1913824324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6823","nonce":1913824324}]},"public_addr":"172.21.15.64:6819/1913824324","cluster_addr":"172.21.15.64:6821/1913824324","heartbeat_back_addr":"172.21.15.64:6825/1913824324","heartbeat_front_addr":"172.21.15.64:6823/1913824324","state":["exists","up"]},{"osd":7,"uuid":"c9707260-b581-4a21-b380-f37e16959b02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6817","nonce":2567568166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6819","nonce":2567568166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6823","nonce":2567568166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6821","nonce":2567568166}]},"public_addr":"172.21.15.119:6817/2567568166","cluster_addr":"172.21.15.119:6819/2567568166","heartbeat_back_addr":"172.21.15.119:6823/2567568166","heartbeat_front_addr":"172.21.15.119:6821/2567568166","state":["exists","up"]},{"osd":8,"uuid":"2dbf8ef2-fc9c-4c6b-bfea-003f41ef191e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6817","nonce":199192217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6819","nonce":199192217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6823","nonce":199192217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6821","nonce":199192217}]},"public_addr":"172.21.15.131:6817/199192217","cluster_addr":"172.21.15.131:6819/199192217","heartbeat_back_addr":"172.21.15.131:6823/199192217","heartbeat_front_addr":"172.21.15.131:6821/199192217","state":["exists","up"]},{"osd":9,"uuid":"96f769e5-4783-4558-adfa-4bfaf9da9e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6827","nonce":3309038990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6829","nonce":3309038990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6833","nonce":3309038990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6831","nonce":3309038990}]},"public_addr":"172.21.15.64:6827/3309038990","cluster_addr":"172.21.15.64:6829/3309038990","heartbeat_back_addr":"172.21.15.64:6833/3309038990","heartbeat_front_addr":"172.21.15.64:6831/3309038990","state":["exists","up"]},{"osd":10,"uuid":"13c1ef5c-2299-440c-af5a-a650b47f4b75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6825","nonce":1460844681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6827","nonce":1460844681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6831","nonce":1460844681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6829","nonce":1460844681}]},"public_addr":"172.21.15.119:6825/1460844681","cluster_addr":"172.21.15.119:6827/1460844681","heartbeat_back_addr":"172.21.15.119:6831/1460844681","heartbeat_front_addr":"172.21.15.119:6829/1460844681","state":["exists","up"]},{"osd":11,"uuid":"c26a5ef9-8585-404e-a54a-852424c4d6fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6825","nonce":2294059084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6827","nonce":2294059084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6831","nonce":2294059084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6829","nonce":2294059084}]},"public_addr":"172.21.15.131:6825/2294059084","cluster_addr":"172.21.15.131:6827/2294059084","heartbeat_back_addr":"172.21.15.131:6831/2294059084","heartbeat_front_addr":"172.21.15.131:6829/2294059084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:56:33.141102+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:56:58.883062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:57:22.641789+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:57:48.705790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:58:12.871514+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:58:37.893118+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:03.620828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:29.446179+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:55.002978+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:00:19.987463+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:00:45.364158+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:01:10.228248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.3","mappings":[{"from":10,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":5,"to":6}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/3554471670":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6801/3786213009":"2024-09-22T03:55:28.387426+0000","172.21.15.64:0/2562856406":"2024-09-22T03:54:27.180100+0000","172.21.15.64:0/2603015699":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6800/1209932961":"2024-09-22T03:54:02.562380+0000","172.21.15.64:6800/3786213009":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6801/1209932961":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/3407344596":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1233731831":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1686887701":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1598568021":"2024-09-22T03:54:27.180100+0000","172.21.15.64:6800/2013929312":"2024-09-22T03:54:27.180100+0000","172.21.15.64:6801/2013929312":"2024-09-22T03:54:27.180100+0000","172.21.15.64:0/3652036144":"2024-09-22T03:55:28.387426+0000","172.21.15.64:0/105454703":"2024-09-22T03:54:27.180100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T04:03:57.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.976+0000 7fe522bb3640 1 -- 172.21.15.64:0/760038410 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe5080b65f0 msgr2=0x7fe5080b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:57.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.976+0000 7fe522bb3640 1 --2- 172.21.15.64:0/760038410 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe5080b65f0 0x7fe5080b8b00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe50c005fd0 tx=0x7fe50c005850 comp rx=0 tx=0).stop 2024-09-21T04:03:57.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.976+0000 7fe522bb3640 1 -- 172.21.15.64:0/760038410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe51c103b60 msgr2=0x7fe51c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:57.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.976+0000 7fe522bb3640 1 --2- 172.21.15.64:0/760038410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe51c103b60 0x7fe51c075710 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe504005940 tx=0x7fe5040042f0 comp rx=0 tx=0).stop 2024-09-21T04:03:57.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.976+0000 7fe522bb3640 1 -- 172.21.15.64:0/760038410 shutdown_connections 2024-09-21T04:03:57.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.976+0000 7fe522bb3640 1 --2- 172.21.15.64:0/760038410 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe5080b65f0 0x7fe5080b8b00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:57.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.976+0000 7fe522bb3640 1 --2- 172.21.15.64:0/760038410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe51c109df0 0x7fe51c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:57.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.976+0000 7fe522bb3640 1 --2- 172.21.15.64:0/760038410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe51c1094a0 0x7fe51c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:57.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.976+0000 7fe522bb3640 1 --2- 172.21.15.64:0/760038410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe51c103b60 0x7fe51c075710 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:57.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.976+0000 7fe522bb3640 1 -- 172.21.15.64:0/760038410 >> 172.21.15.64:0/760038410 conn(0x7fe51c07b850 msgr2=0x7fe51c107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:57.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.976+0000 7fe522bb3640 1 -- 172.21.15.64:0/760038410 shutdown_connections 2024-09-21T04:03:57.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:57.976+0000 7fe522bb3640 1 -- 172.21.15.64:0/760038410 wait complete. 2024-09-21T04:03:58.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:57 smithi119 ceph-mon[34811]: pgmap v319: 97 pgs: 97 active+clean; 651 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-21T04:03:58.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:57 smithi131 ceph-mon[35193]: pgmap v319: 97 pgs: 97 active+clean; 651 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-21T04:03:58.744 DEBUG:tasks.fwd_scrub:Yielding 2024-09-21T04:03:58.744 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T04:03:58.755 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T04:03:58.755 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-09-21T04:03:58.755 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-21T04:03:58.758 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-21T04:03:58.758 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T04:03:58.758 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T04:03:58.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.877+0000 7f37625f1640 1 -- 172.21.15.64:0/2385723142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f375c1082e0 msgr2=0x7f375c1086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:58.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.877+0000 7f37625f1640 1 --2- 172.21.15.64:0/2385723142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f375c1082e0 0x7f375c1086f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3750009900 tx=0x7f375002a410 comp rx=0 tx=0).stop 2024-09-21T04:03:58.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.877+0000 7f37625f1640 1 -- 172.21.15.64:0/2385723142 shutdown_connections 2024-09-21T04:03:58.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.877+0000 7f37625f1640 1 --2- 172.21.15.64:0/2385723142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f375c108c30 0x7f375c10f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:58.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.877+0000 7f37625f1640 1 --2- 172.21.15.64:0/2385723142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f375c1082e0 0x7f375c1086f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:58.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.877+0000 7f37625f1640 1 --2- 172.21.15.64:0/2385723142 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f375c1029b0 0x7f375c102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:58.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.877+0000 7f37625f1640 1 -- 172.21.15.64:0/2385723142 >> 172.21.15.64:0/2385723142 conn(0x7f375c07a740 msgr2=0x7f375c07ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:58.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.877+0000 7f37625f1640 1 -- 172.21.15.64:0/2385723142 shutdown_connections 2024-09-21T04:03:58.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.877+0000 7f37625f1640 1 -- 172.21.15.64:0/2385723142 wait complete. 2024-09-21T04:03:58.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.878+0000 7f37625f1640 1 Processor -- start 2024-09-21T04:03:58.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.878+0000 7f37625f1640 1 -- start start 2024-09-21T04:03:58.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.878+0000 7f37625f1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f375c1029b0 0x7f375c199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:58.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.878+0000 7f37625f1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f375c1082e0 0x7f375c1998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:58.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.878+0000 7f37625f1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f375c108c30 0x7f375c19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:58.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.878+0000 7f37625f1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f375c1128f0 con 0x7f375c1082e0 2024-09-21T04:03:58.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.878+0000 7f37625f1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f375c112770 con 0x7f375c1029b0 2024-09-21T04:03:58.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.878+0000 7f37625f1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f375c112a70 con 0x7f375c108c30 2024-09-21T04:03:58.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.878+0000 7f375b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f375c1082e0 0x7f375c1998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:58.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.878+0000 7f375b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f375c1082e0 0x7f375c1998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50054/0 (socket says 172.21.15.64:50054) 2024-09-21T04:03:58.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.878+0000 7f375b7fe640 1 -- 172.21.15.64:0/2340370103 learned_addr learned my addr 172.21.15.64:0/2340370103 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:58.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.878+0000 7f375bfff640 1 --2- 172.21.15.64:0/2340370103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f375c1029b0 0x7f375c199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:58.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.879+0000 7f3760b8e640 1 --2- 172.21.15.64:0/2340370103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f375c108c30 0x7f375c19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:58.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.879+0000 7f375b7fe640 1 -- 172.21.15.64:0/2340370103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f375c108c30 msgr2=0x7f375c19eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:58.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.879+0000 7f375b7fe640 1 --2- 172.21.15.64:0/2340370103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f375c108c30 0x7f375c19eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:58.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.879+0000 7f375b7fe640 1 -- 172.21.15.64:0/2340370103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f375c1029b0 msgr2=0x7f375c199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:58.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.879+0000 7f375b7fe640 1 --2- 172.21.15.64:0/2340370103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f375c1029b0 0x7f375c199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:58.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.879+0000 7f375b7fe640 1 -- 172.21.15.64:0/2340370103 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375c19f280 con 0x7f375c1082e0 2024-09-21T04:03:58.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.879+0000 7f375b7fe640 1 --2- 172.21.15.64:0/2340370103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f375c1082e0 0x7f375c1998f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f375002a930 tx=0x7f3750004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:58.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.879+0000 7f37597fa640 1 -- 172.21.15.64:0/2340370103 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375003f030 con 0x7f375c1082e0 2024-09-21T04:03:58.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.879+0000 7f37597fa640 1 -- 172.21.15.64:0/2340370103 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3750003cb0 con 0x7f375c1082e0 2024-09-21T04:03:58.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.879+0000 7f37625f1640 1 -- 172.21.15.64:0/2340370103 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f375c19f510 con 0x7f375c1082e0 2024-09-21T04:03:58.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.879+0000 7f37597fa640 1 -- 172.21.15.64:0/2340370103 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3750035070 con 0x7f375c1082e0 2024-09-21T04:03:58.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.879+0000 7f37625f1640 1 -- 172.21.15.64:0/2340370103 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f375c1a5d00 con 0x7f375c1082e0 2024-09-21T04:03:58.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.880+0000 7f37625f1640 1 -- 172.21.15.64:0/2340370103 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f375c102dc0 con 0x7f375c1082e0 2024-09-21T04:03:58.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.883+0000 7f37597fa640 1 -- 172.21.15.64:0/2340370103 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) ==== 154580+0+0 (secure 0 0 0) 0x7f3750042030 con 0x7f375c1082e0 2024-09-21T04:03:58.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.884+0000 7f37597fa640 1 --2- 172.21.15.64:0/2340370103 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f37440b6670 0x7f37440b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:58.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.884+0000 7f37597fa640 1 -- 172.21.15.64:0/2340370103 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f3750108030 con 0x7f375c1082e0 2024-09-21T04:03:58.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.884+0000 7f375bfff640 1 --2- 172.21.15.64:0/2340370103 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f37440b6670 0x7f37440b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:58.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.884+0000 7f375bfff640 1 --2- 172.21.15.64:0/2340370103 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f37440b6670 0x7f37440b8b80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f374c0058c0 tx=0x7f374c00a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:58.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:58.887+0000 7f37597fa640 1 -- 172.21.15.64:0/2340370103 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f37500ce610 con 0x7f375c1082e0 2024-09-21T04:03:59.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:58 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/760038410' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:03:59.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:58 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/760038410' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:03:59.034 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:03:59.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:58 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/760038410' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:03:59.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.118+0000 7f37625f1640 1 -- 172.21.15.64:0/2340370103 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f375c10aa80 con 0x7f37440b6670 2024-09-21T04:03:59.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.290+0000 7f37597fa640 1 -- 172.21.15.64:0/2340370103 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f375c10aa80 con 0x7f37440b6670 2024-09-21T04:03:59.292 INFO:teuthology.orchestra.run.smithi064.stdout:Schedule set for path / 2024-09-21T04:03:59.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.292+0000 7f37625f1640 1 -- 172.21.15.64:0/2340370103 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f37440b6670 msgr2=0x7f37440b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:59.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.292+0000 7f37625f1640 1 --2- 172.21.15.64:0/2340370103 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f37440b6670 0x7f37440b8b80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f374c0058c0 tx=0x7f374c00a2d0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.292+0000 7f37625f1640 1 -- 172.21.15.64:0/2340370103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f375c1082e0 msgr2=0x7f375c1998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:59.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.292+0000 7f37625f1640 1 --2- 172.21.15.64:0/2340370103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f375c1082e0 0x7f375c1998f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f375002a930 tx=0x7f3750004270 comp rx=0 tx=0).stop 2024-09-21T04:03:59.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.292+0000 7f37625f1640 1 -- 172.21.15.64:0/2340370103 shutdown_connections 2024-09-21T04:03:59.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.292+0000 7f37625f1640 1 --2- 172.21.15.64:0/2340370103 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f37440b6670 0x7f37440b8b80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.292+0000 7f37625f1640 1 --2- 172.21.15.64:0/2340370103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f375c108c30 0x7f375c19eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.292+0000 7f37625f1640 1 --2- 172.21.15.64:0/2340370103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f375c1082e0 0x7f375c1998f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.292+0000 7f37625f1640 1 --2- 172.21.15.64:0/2340370103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f375c1029b0 0x7f375c199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.292+0000 7f37625f1640 1 -- 172.21.15.64:0/2340370103 >> 172.21.15.64:0/2340370103 conn(0x7f375c07a740 msgr2=0x7f375c106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:59.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.292+0000 7f37625f1640 1 -- 172.21.15.64:0/2340370103 shutdown_connections 2024-09-21T04:03:59.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.292+0000 7f37625f1640 1 -- 172.21.15.64:0/2340370103 wait complete. 2024-09-21T04:03:59.308 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-21T04:03:59.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.428+0000 7f98a7da1640 1 -- 172.21.15.64:0/1584840103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98a0107ab0 msgr2=0x7f98a0107ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:59.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.428+0000 7f98a7da1640 1 --2- 172.21.15.64:0/1584840103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98a0107ab0 0x7f98a0107ec0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f989000aa10 tx=0x7f989002b720 comp rx=0 tx=0).stop 2024-09-21T04:03:59.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.428+0000 7f98a7da1640 1 -- 172.21.15.64:0/1584840103 shutdown_connections 2024-09-21T04:03:59.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.428+0000 7f98a7da1640 1 --2- 172.21.15.64:0/1584840103 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98a0108400 0x7f98a010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.428+0000 7f98a7da1640 1 --2- 172.21.15.64:0/1584840103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98a0107ab0 0x7f98a0107ec0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.428+0000 7f98a7da1640 1 --2- 172.21.15.64:0/1584840103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98a0102180 0x7f98a0102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.428+0000 7f98a7da1640 1 -- 172.21.15.64:0/1584840103 >> 172.21.15.64:0/1584840103 conn(0x7f98a0079e90 msgr2=0x7f98a007a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:59.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.428+0000 7f98a7da1640 1 -- 172.21.15.64:0/1584840103 shutdown_connections 2024-09-21T04:03:59.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.429+0000 7f98a7da1640 1 -- 172.21.15.64:0/1584840103 wait complete. 2024-09-21T04:03:59.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.429+0000 7f98a7da1640 1 Processor -- start 2024-09-21T04:03:59.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.430+0000 7f98a7da1640 1 -- start start 2024-09-21T04:03:59.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.430+0000 7f98a7da1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98a0102180 0x7f98a019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:59.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.430+0000 7f98a7da1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98a0107ab0 0x7f98a019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:59.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.430+0000 7f98a7da1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98a0108400 0x7f98a019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:59.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.430+0000 7f98a5b3d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98a0102180 0x7f98a019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:59.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.430+0000 7f98a7da1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f98a0111890 con 0x7f98a0102180 2024-09-21T04:03:59.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.430+0000 7f98a7da1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f98a0111710 con 0x7f98a0107ab0 2024-09-21T04:03:59.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.430+0000 7f98a5b3d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98a0102180 0x7f98a019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50080/0 (socket says 172.21.15.64:50080) 2024-09-21T04:03:59.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.430+0000 7f98a7da1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f98a0111a10 con 0x7f98a0108400 2024-09-21T04:03:59.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.430+0000 7f98a5b3d640 1 -- 172.21.15.64:0/1754550007 learned_addr learned my addr 172.21.15.64:0/1754550007 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:59.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.430+0000 7f98a533c640 1 --2- 172.21.15.64:0/1754550007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98a0107ab0 0x7f98a019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:59.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.430+0000 7f98a633e640 1 --2- 172.21.15.64:0/1754550007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98a0108400 0x7f98a019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:59.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.431+0000 7f98a5b3d640 1 -- 172.21.15.64:0/1754550007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98a0108400 msgr2=0x7f98a019fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:59.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.431+0000 7f98a5b3d640 1 --2- 172.21.15.64:0/1754550007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98a0108400 0x7f98a019fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.431+0000 7f98a5b3d640 1 -- 172.21.15.64:0/1754550007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98a0107ab0 msgr2=0x7f98a019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.431+0000 7f98a5b3d640 1 --2- 172.21.15.64:0/1754550007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98a0107ab0 0x7f98a019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.431+0000 7f98a5b3d640 1 -- 172.21.15.64:0/1754550007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a01a03b0 con 0x7f98a0102180 2024-09-21T04:03:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.431+0000 7f98a533c640 1 --2- 172.21.15.64:0/1754550007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98a0107ab0 0x7f98a019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:03:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.431+0000 7f98a5b3d640 1 --2- 172.21.15.64:0/1754550007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98a0102180 0x7f98a019a480 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f989c00d950 tx=0x7f989c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.431+0000 7f9896ffd640 1 -- 172.21.15.64:0/1754550007 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c017070 con 0x7f98a0102180 2024-09-21T04:03:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.431+0000 7f98a7da1640 1 -- 172.21.15.64:0/1754550007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a01a06a0 con 0x7f98a0102180 2024-09-21T04:03:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.431+0000 7f9896ffd640 1 -- 172.21.15.64:0/1754550007 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f989c009c30 con 0x7f98a0102180 2024-09-21T04:03:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.431+0000 7f9896ffd640 1 -- 172.21.15.64:0/1754550007 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c00f620 con 0x7f98a0102180 2024-09-21T04:03:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.431+0000 7f98a7da1640 1 -- 172.21.15.64:0/1754550007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a01a6e30 con 0x7f98a0102180 2024-09-21T04:03:59.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.432+0000 7f9894ff9640 1 -- 172.21.15.64:0/1754550007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9888005290 con 0x7f98a0102180 2024-09-21T04:03:59.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.434+0000 7f9896ffd640 1 -- 172.21.15.64:0/1754550007 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) ==== 154580+0+0 (secure 0 0 0) 0x7f989c010460 con 0x7f98a0102180 2024-09-21T04:03:59.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.435+0000 7f9896ffd640 1 --2- 172.21.15.64:0/1754550007 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f988c0b65f0 0x7f988c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:59.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.436+0000 7f98a533c640 1 --2- 172.21.15.64:0/1754550007 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f988c0b65f0 0x7f988c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:59.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.436+0000 7f9896ffd640 1 -- 172.21.15.64:0/1754550007 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f989c0e7200 con 0x7f98a0102180 2024-09-21T04:03:59.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.436+0000 7f98a533c640 1 --2- 172.21.15.64:0/1754550007 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f988c0b65f0 0x7f988c0b8b00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f989002bd10 tx=0x7f9890005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:59.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.439+0000 7f9896ffd640 1 -- 172.21.15.64:0/1754550007 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f989c009de0 con 0x7f98a0102180 2024-09-21T04:03:59.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.516+0000 7f40357b1640 1 -- 172.21.15.64:0/274686002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f403010ddf0 msgr2=0x7f4030110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:59.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.516+0000 7f40357b1640 1 --2- 172.21.15.64:0/274686002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f403010ddf0 0x7f4030110300 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4028009ca0 tx=0x7f402802a8c0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.516+0000 7f40357b1640 1 -- 172.21.15.64:0/274686002 shutdown_connections 2024-09-21T04:03:59.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.516+0000 7f40357b1640 1 --2- 172.21.15.64:0/274686002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f403010ddf0 0x7f4030110300 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.516+0000 7f40357b1640 1 --2- 172.21.15.64:0/274686002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4030105270 0x7f403010d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.517+0000 7f40357b1640 1 --2- 172.21.15.64:0/274686002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4030104840 0x7f4030104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.517+0000 7f40357b1640 1 -- 172.21.15.64:0/274686002 >> 172.21.15.64:0/274686002 conn(0x7f4030079730 msgr2=0x7f4030079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:59.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.517+0000 7f40357b1640 1 -- 172.21.15.64:0/274686002 shutdown_connections 2024-09-21T04:03:59.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.517+0000 7f40357b1640 1 -- 172.21.15.64:0/274686002 wait complete. 2024-09-21T04:03:59.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.517+0000 7f40357b1640 1 Processor -- start 2024-09-21T04:03:59.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.518+0000 7f40357b1640 1 -- start start 2024-09-21T04:03:59.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.518+0000 7f40357b1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4030104840 0x7f403019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:59.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.518+0000 7f40357b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4030105270 0x7f403019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:59.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.518+0000 7f40357b1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f403010ddf0 0x7f40301a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:59.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.518+0000 7f40357b1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4030113a70 con 0x7f4030105270 2024-09-21T04:03:59.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.518+0000 7f40357b1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f40301138f0 con 0x7f403010ddf0 2024-09-21T04:03:59.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.518+0000 7f40357b1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4030113bf0 con 0x7f4030104840 2024-09-21T04:03:59.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.518+0000 7f402e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4030105270 0x7f403019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:59.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.518+0000 7f402e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4030105270 0x7f403019cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50098/0 (socket says 172.21.15.64:50098) 2024-09-21T04:03:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.518+0000 7f402e7fc640 1 -- 172.21.15.64:0/525554029 learned_addr learned my addr 172.21.15.64:0/525554029 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.518+0000 7f402effd640 1 --2- 172.21.15.64:0/525554029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4030104840 0x7f403019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.518+0000 7f402f7fe640 1 --2- 172.21.15.64:0/525554029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f403010ddf0 0x7f40301a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.518+0000 7f402effd640 1 -- 172.21.15.64:0/525554029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f403010ddf0 msgr2=0x7f40301a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.518+0000 7f402effd640 1 --2- 172.21.15.64:0/525554029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f403010ddf0 0x7f40301a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.518+0000 7f402effd640 1 -- 172.21.15.64:0/525554029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4030105270 msgr2=0x7f403019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.518+0000 7f402effd640 1 --2- 172.21.15.64:0/525554029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4030105270 0x7f403019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.518+0000 7f402effd640 1 -- 172.21.15.64:0/525554029 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40301a2650 con 0x7f4030104840 2024-09-21T04:03:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.518+0000 7f402e7fc640 1 --2- 172.21.15.64:0/525554029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4030105270 0x7f403019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:03:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.518+0000 7f402f7fe640 1 --2- 172.21.15.64:0/525554029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f403010ddf0 0x7f40301a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:03:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.519+0000 7f402effd640 1 --2- 172.21.15.64:0/525554029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4030104840 0x7f403019c6e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f402000e9c0 tx=0x7f402000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.519+0000 7f4017fff640 1 -- 172.21.15.64:0/525554029 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402000ccc0 con 0x7f4030104840 2024-09-21T04:03:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.519+0000 7f40357b1640 1 -- 172.21.15.64:0/525554029 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40301a2940 con 0x7f4030104840 2024-09-21T04:03:59.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.519+0000 7f4017fff640 1 -- 172.21.15.64:0/525554029 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40200035f0 con 0x7f4030104840 2024-09-21T04:03:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.519+0000 7f4017fff640 1 -- 172.21.15.64:0/525554029 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402000ce60 con 0x7f4030104840 2024-09-21T04:03:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.519+0000 7f40357b1640 1 -- 172.21.15.64:0/525554029 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40301a90a0 con 0x7f4030104840 2024-09-21T04:03:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.523+0000 7f403bd25740 1 Processor -- start 2024-09-21T04:03:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.523+0000 7f4017fff640 1 -- 172.21.15.64:0/525554029 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 23) ==== 154580+0+0 (secure 0 0 0) 0x7f4020010690 con 0x7f4030104840 2024-09-21T04:03:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.523+0000 7f403bd25740 1 -- start start 2024-09-21T04:03:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.523+0000 7f403bd25740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5e47b9350 0x55c5e4b60430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.523+0000 7f403bd25740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5e4790490 0x55c5e4b60970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.523+0000 7f403bd25740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e4798670 0x55c5e4b60eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.523+0000 7f403bd25740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c5e4ad6e60 con 0x55c5e4798670 2024-09-21T04:03:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.523+0000 7f403bd25740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c5e4ac6290 con 0x55c5e4790490 2024-09-21T04:03:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.523+0000 7f403bd25740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c5e4aca0a0 con 0x55c5e47b9350 2024-09-21T04:03:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.523+0000 7f402e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e4798670 0x55c5e4b60eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:59.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.523+0000 7f402f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5e47b9350 0x55c5e4b60430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:59.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.523+0000 7f402e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e4798670 0x55c5e4b60eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50106/0 (socket says 172.21.15.64:50106) 2024-09-21T04:03:59.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.523+0000 7f402e7fc640 1 -- 172.21.15.64:0/3041524638 learned_addr learned my addr 172.21.15.64:0/3041524638 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:59.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.524+0000 7f402effd640 1 --2- 172.21.15.64:0/3041524638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5e4790490 0x55c5e4b60970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:59.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.524+0000 7f402e7fc640 1 -- 172.21.15.64:0/3041524638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5e47b9350 msgr2=0x55c5e4b60430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:59.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.524+0000 7f402e7fc640 1 --2- 172.21.15.64:0/3041524638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5e47b9350 0x55c5e4b60430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.524+0000 7f402e7fc640 1 -- 172.21.15.64:0/3041524638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5e4790490 msgr2=0x55c5e4b60970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:59.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.524+0000 7f402e7fc640 1 --2- 172.21.15.64:0/3041524638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5e4790490 0x55c5e4b60970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.524+0000 7f4017fff640 1 --2- 172.21.15.64:0/525554029 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f40180b65f0 0x7f40180b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:59.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.524+0000 7f402e7fc640 1 -- 172.21.15.64:0/3041524638 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5e4824930 con 0x55c5e4798670 2024-09-21T04:03:59.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.524+0000 7f402f7fe640 1 --2- 172.21.15.64:0/525554029 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f40180b65f0 0x7f40180b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:59.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.524+0000 7f4017fff640 1 -- 172.21.15.64:0/525554029 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f40200e7b90 con 0x7f4030104840 2024-09-21T04:03:59.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.524+0000 7f402e7fc640 1 --2- 172.21.15.64:0/3041524638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e4798670 0x55c5e4b60eb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f40240114c0 tx=0x7f4024011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b9bfcaccd8c31d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:59.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.525+0000 7f40157fa640 1 -- 172.21.15.64:0/3041524638 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402400f5c0 con 0x55c5e4798670 2024-09-21T04:03:59.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.525+0000 7f40157fa640 1 -- 172.21.15.64:0/3041524638 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f402400fc10 con 0x55c5e4798670 2024-09-21T04:03:59.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.525+0000 7f40157fa640 1 -- 172.21.15.64:0/3041524638 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4024019c60 con 0x55c5e4798670 2024-09-21T04:03:59.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.526+0000 7f403bd25740 1 -- 172.21.15.64:0/3041524638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e4798670 msgr2=0x55c5e4b60eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:59.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.526+0000 7f402f7fe640 1 --2- 172.21.15.64:0/525554029 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f40180b65f0 0x7f40180b8b00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4028004af0 tx=0x7f4028005c30 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:59.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.526+0000 7f403bd25740 1 --2- 172.21.15.64:0/3041524638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e4798670 0x55c5e4b60eb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f40240114c0 tx=0x7f4024011980 comp rx=0 tx=0).stop 2024-09-21T04:03:59.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.526+0000 7f403bd25740 1 -- 172.21.15.64:0/3041524638 shutdown_connections 2024-09-21T04:03:59.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.526+0000 7f403bd25740 1 --2- 172.21.15.64:0/3041524638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e4798670 0x55c5e4b60eb0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.526+0000 7f403bd25740 1 --2- 172.21.15.64:0/3041524638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5e4790490 0x55c5e4b60970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.526+0000 7f403bd25740 1 --2- 172.21.15.64:0/3041524638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5e47b9350 0x55c5e4b60430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.526+0000 7f403bd25740 1 -- 172.21.15.64:0/3041524638 >> 172.21.15.64:0/3041524638 conn(0x55c5e47ad740 msgr2=0x55c5e4b5e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:59.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.526+0000 7f403bd25740 1 -- 172.21.15.64:0/3041524638 shutdown_connections 2024-09-21T04:03:59.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.526+0000 7f403bd25740 1 -- 172.21.15.64:0/3041524638 wait complete. 2024-09-21T04:03:59.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.526+0000 7f403bd25740 1 -- start start 2024-09-21T04:03:59.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.526+0000 7f403bd25740 1 Processor -- start 2024-09-21T04:03:59.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.527+0000 7f403bd25740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e4790490 0x55c5e4b6bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:59.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.527+0000 7f403bd25740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5e4798670 0x55c5e4b6c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:59.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.527+0000 7f403bd25740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5e47b9350 0x55c5e4b77f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:59.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.527+0000 7f403bd25740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c5e4927890 con 0x55c5e4790490 2024-09-21T04:03:59.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.527+0000 7f403bd25740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c5e4a60d40 con 0x55c5e47b9350 2024-09-21T04:03:59.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.527+0000 7f403bd25740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c5e4b6ec10 con 0x55c5e4798670 2024-09-21T04:03:59.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.527+0000 7f403bd25740 10 client.0 _ll_register_callbacks cb 0x55c5e4ac4c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:03:59.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.527+0000 7f402effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e4790490 0x55c5e4b6bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:59.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.527+0000 7f402effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e4790490 0x55c5e4b6bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50118/0 (socket says 172.21.15.64:50118) 2024-09-21T04:03:59.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.527+0000 7f402effd640 1 -- 172.21.15.64:0/3326261384 learned_addr learned my addr 172.21.15.64:0/3326261384 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:59.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.527+0000 7f402f7fe640 1 --2- 172.21.15.64:0/3326261384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5e47b9350 0x55c5e4b77f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:59.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.527+0000 7f402e7fc640 1 --2- 172.21.15.64:0/3326261384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5e4798670 0x55c5e4b6c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:59.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.527+0000 7f402effd640 1 -- 172.21.15.64:0/3326261384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5e4798670 msgr2=0x55c5e4b6c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:59.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.527+0000 7f402effd640 1 --2- 172.21.15.64:0/3326261384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5e4798670 0x55c5e4b6c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.527+0000 7f402effd640 1 -- 172.21.15.64:0/3326261384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5e47b9350 msgr2=0x55c5e4b77f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:59.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.527+0000 7f402effd640 1 --2- 172.21.15.64:0/3326261384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5e47b9350 0x55c5e4b77f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.527+0000 7f402effd640 1 -- 172.21.15.64:0/3326261384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5e4b78440 con 0x55c5e4790490 2024-09-21T04:03:59.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.528+0000 7f402effd640 1 --2- 172.21.15.64:0/3326261384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e4790490 0x55c5e4b6bf10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f40200ada00 tx=0x7f4020003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522805cf3a8c375a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:59.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.528+0000 7f4012ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:03:59.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.528+0000 7f4012ff5640 1 -- 172.21.15.64:0/3326261384 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40200a9050 con 0x55c5e4790490 2024-09-21T04:03:59.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.528+0000 7f4012ff5640 1 -- 172.21.15.64:0/3326261384 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f40200b21b0 con 0x55c5e4790490 2024-09-21T04:03:59.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.528+0000 7f403bd25740 1 -- 172.21.15.64:0/3326261384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f402400fc10 con 0x55c5e4790490 2024-09-21T04:03:59.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.528+0000 7f4012ff5640 1 -- 172.21.15.64:0/3326261384 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f40200ad580 con 0x55c5e4790490 2024-09-21T04:03:59.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.528+0000 7f403bd25740 10 client.15450 fetch_fsmap learned FSMap version 25 2024-09-21T04:03:59.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.528+0000 7f403bd25740 1 -- 172.21.15.64:0/3326261384 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c5e4b7ad80 con 0x55c5e4790490 2024-09-21T04:03:59.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.528+0000 7f4012ff5640 1 -- 172.21.15.64:0/3326261384 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f402400fc10 con 0x55c5e4790490 2024-09-21T04:03:59.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.529+0000 7f4012ff5640 1 -- 172.21.15.64:0/3326261384 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f40200a93c0 con 0x55c5e4790490 2024-09-21T04:03:59.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.529+0000 7f403bd25740 10 client.15450 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:03:59.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.529+0000 7f403bd25740 10 client.15450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:03:59.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.529+0000 7f403bd25740 20 client.15450 populate_metadata read hostname 'smithi064' 2024-09-21T04:03:59.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.529+0000 7f403bd25740 1 --2- 172.21.15.64:0/3326261384 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55c5e4777cd0 0x55c5e4b7e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:59.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.529+0000 7f403bd25740 4 client.15450 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:03:59.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.529+0000 7f403bd25740 1 -- 172.21.15.64:0/3326261384 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5e4ad8db0 con 0x55c5e4777cd0 2024-09-21T04:03:59.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.529+0000 7f402e7fc640 1 --2- 172.21.15.64:0/3326261384 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55c5e4777cd0 0x55c5e4b7e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:59.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.530+0000 7f402e7fc640 1 --2- 172.21.15.64:0/3326261384 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55c5e4777cd0 0x55c5e4b7e3e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eb3fd67bab54246 server_cookie=4d42ce8f0504382f in_seq=0 out_seq=0 2024-09-21T04:03:59.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.530+0000 7f4012ff5640 10 client.15450 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:03:59.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.530+0000 7f4012ff5640 1 -- 172.21.15.64:0/3326261384 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c5e4ad8db0 con 0x55c5e4777cd0 2024-09-21T04:03:59.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.530+0000 7f4012ff5640 10 client.15450 handle_command_reply: tid=0 2024-09-21T04:03:59.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.530+0000 7f403bd25740 1 client.15450 shutdown 2024-09-21T04:03:59.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.531+0000 7f403bd25740 1 -- 172.21.15.64:0/3326261384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e4790490 msgr2=0x55c5e4b6bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:59.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.531+0000 7f403bd25740 1 --2- 172.21.15.64:0/3326261384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e4790490 0x55c5e4b6bf10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f40200ada00 tx=0x7f4020003cd0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.531+0000 7f403bd25740 1 -- 172.21.15.64:0/3326261384 shutdown_connections 2024-09-21T04:03:59.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.531+0000 7f403bd25740 1 --2- 172.21.15.64:0/3326261384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5e47b9350 0x55c5e4b77f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.531+0000 7f403bd25740 1 --2- 172.21.15.64:0/3326261384 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5e4798670 0x55c5e4b6c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.531+0000 7f403bd25740 1 --2- 172.21.15.64:0/3326261384 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55c5e4777cd0 0x55c5e4b7e3e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.531+0000 7f403bd25740 1 --2- 172.21.15.64:0/3326261384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e4790490 0x55c5e4b6bf10 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.531+0000 7f403bd25740 1 -- 172.21.15.64:0/3326261384 >> 172.21.15.64:0/3326261384 conn(0x55c5e47ad740 msgr2=0x55c5e4b635c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:59.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.531+0000 7f4012ff5640 0 client.15450 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:03:59.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.531+0000 7f403bd25740 1 -- 172.21.15.64:0/3326261384 shutdown_connections 2024-09-21T04:03:59.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.531+0000 7f403bd25740 1 -- 172.21.15.64:0/3326261384 wait complete. 2024-09-21T04:03:59.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.531+0000 7f403bd25740 20 client.15450 trim_cache size 0 max 16384 2024-09-21T04:03:59.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.564+0000 7f403bd25740 1 Processor -- start 2024-09-21T04:03:59.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.564+0000 7f403bd25740 1 -- start start 2024-09-21T04:03:59.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.564+0000 7f403bd25740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5e4790490 0x55c5e4b66550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:59.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.564+0000 7f403bd25740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5e4798670 0x55c5e4b77e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:59.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.564+0000 7f403bd25740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e47b9350 0x55c5e4b68f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:59.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.564+0000 7f403bd25740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f40301b43f0 con 0x55c5e47b9350 2024-09-21T04:03:59.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.564+0000 7f403bd25740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f40301b1cc0 con 0x55c5e4798670 2024-09-21T04:03:59.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.564+0000 7f403bd25740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f40301afb10 con 0x55c5e4790490 2024-09-21T04:03:59.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.564+0000 7f402f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e47b9350 0x55c5e4b68f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:59.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.564+0000 7f402f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e47b9350 0x55c5e4b68f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50120/0 (socket says 172.21.15.64:50120) 2024-09-21T04:03:59.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.564+0000 7f402f7fe640 1 -- 172.21.15.64:0/3211349568 learned_addr learned my addr 172.21.15.64:0/3211349568 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:59.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.564+0000 7f402e7fc640 1 --2- 172.21.15.64:0/3211349568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5e4798670 0x55c5e4b77e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:59.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.565+0000 7f402effd640 1 --2- 172.21.15.64:0/3211349568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5e4790490 0x55c5e4b66550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:59.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.565+0000 7f402f7fe640 1 -- 172.21.15.64:0/3211349568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5e4790490 msgr2=0x55c5e4b66550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:59.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.565+0000 7f402f7fe640 1 --2- 172.21.15.64:0/3211349568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5e4790490 0x55c5e4b66550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.565+0000 7f402f7fe640 1 -- 172.21.15.64:0/3211349568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5e4798670 msgr2=0x55c5e4b77e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:59.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.565+0000 7f402f7fe640 1 --2- 172.21.15.64:0/3211349568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5e4798670 0x55c5e4b77e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.565+0000 7f402f7fe640 1 -- 172.21.15.64:0/3211349568 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5e4b675a0 con 0x55c5e47b9350 2024-09-21T04:03:59.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.565+0000 7f402f7fe640 1 --2- 172.21.15.64:0/3211349568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e47b9350 0x55c5e4b68f00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4028009ca0 tx=0x7f4028044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dde16420796f7d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:59.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.565+0000 7f4014ff9640 1 -- 172.21.15.64:0/3211349568 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4028003970 con 0x55c5e47b9350 2024-09-21T04:03:59.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.565+0000 7f4014ff9640 1 -- 172.21.15.64:0/3211349568 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4028036040 con 0x55c5e47b9350 2024-09-21T04:03:59.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.565+0000 7f4014ff9640 1 -- 172.21.15.64:0/3211349568 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402802f680 con 0x55c5e47b9350 2024-09-21T04:03:59.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.566+0000 7f403bd25740 1 -- 172.21.15.64:0/3211349568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e47b9350 msgr2=0x55c5e4b68f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:59.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.566+0000 7f403bd25740 1 --2- 172.21.15.64:0/3211349568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e47b9350 0x55c5e4b68f00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4028009ca0 tx=0x7f4028044000 comp rx=0 tx=0).stop 2024-09-21T04:03:59.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.566+0000 7f403bd25740 1 -- 172.21.15.64:0/3211349568 shutdown_connections 2024-09-21T04:03:59.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.566+0000 7f403bd25740 1 --2- 172.21.15.64:0/3211349568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e47b9350 0x55c5e4b68f00 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.566+0000 7f403bd25740 1 --2- 172.21.15.64:0/3211349568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5e4798670 0x55c5e4b77e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.566+0000 7f403bd25740 1 --2- 172.21.15.64:0/3211349568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5e4790490 0x55c5e4b66550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.566+0000 7f403bd25740 1 -- 172.21.15.64:0/3211349568 >> 172.21.15.64:0/3211349568 conn(0x55c5e47ad740 msgr2=0x55c5e4b6a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:59.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.566+0000 7f403bd25740 1 -- 172.21.15.64:0/3211349568 shutdown_connections 2024-09-21T04:03:59.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.566+0000 7f403bd25740 1 -- 172.21.15.64:0/3211349568 wait complete. 2024-09-21T04:03:59.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.567+0000 7f403bd25740 1 -- start start 2024-09-21T04:03:59.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.567+0000 7f403bd25740 1 Processor -- start 2024-09-21T04:03:59.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.567+0000 7f403bd25740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e4790490 0x55c5e4b70260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:59.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.567+0000 7f403bd25740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5e4798670 0x55c5e4b70800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:59.575 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.568+0000 7f403bd25740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5e47b9350 0x55c5e4be7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:59.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.568+0000 7f403bd25740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f40301b0180 con 0x55c5e4790490 2024-09-21T04:03:59.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.568+0000 7f403bd25740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f40301b15f0 con 0x55c5e4798670 2024-09-21T04:03:59.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.568+0000 7f403bd25740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f40301b0850 con 0x55c5e47b9350 2024-09-21T04:03:59.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.568+0000 7f403bd25740 10 client.0 _ll_register_callbacks cb 0x7f40200a9980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:03:59.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.568+0000 7f402effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e4790490 0x55c5e4b70260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:59.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.568+0000 7f402effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e4790490 0x55c5e4b70260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50130/0 (socket says 172.21.15.64:50130) 2024-09-21T04:03:59.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.568+0000 7f402effd640 1 -- 172.21.15.64:0/3217596498 learned_addr learned my addr 172.21.15.64:0/3217596498 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:59.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.568+0000 7f402f7fe640 1 --2- 172.21.15.64:0/3217596498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5e47b9350 0x55c5e4be7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:59.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.568+0000 7f402e7fc640 1 --2- 172.21.15.64:0/3217596498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5e4798670 0x55c5e4b70800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:59.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.568+0000 7f402effd640 1 -- 172.21.15.64:0/3217596498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5e47b9350 msgr2=0x55c5e4be7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:59.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.568+0000 7f402effd640 1 --2- 172.21.15.64:0/3217596498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5e47b9350 0x55c5e4be7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.568+0000 7f402effd640 1 -- 172.21.15.64:0/3217596498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5e4798670 msgr2=0x55c5e4b70800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:03:59.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.568+0000 7f402effd640 1 --2- 172.21.15.64:0/3217596498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5e4798670 0x55c5e4b70800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.568+0000 7f402effd640 1 -- 172.21.15.64:0/3217596498 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5e4be8200 con 0x55c5e4790490 2024-09-21T04:03:59.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.568+0000 7f402e7fc640 1 --2- 172.21.15.64:0/3217596498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5e4798670 0x55c5e4b70800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:03:59.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.568+0000 7f402effd640 1 --2- 172.21.15.64:0/3217596498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e4790490 0x55c5e4b70260 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f40200ad720 tx=0x7f40200a95f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231f1a3d916bb59a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:59.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.568+0000 7f4012ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:03:59.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.569+0000 7f4012ff5640 1 -- 172.21.15.64:0/3217596498 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40200ac740 con 0x55c5e4790490 2024-09-21T04:03:59.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.569+0000 7f4012ff5640 1 -- 172.21.15.64:0/3217596498 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f40200eb100 con 0x55c5e4790490 2024-09-21T04:03:59.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.569+0000 7f403bd25740 1 -- 172.21.15.64:0/3217596498 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4028036040 con 0x55c5e4790490 2024-09-21T04:03:59.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.569+0000 7f4012ff5640 1 -- 172.21.15.64:0/3217596498 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f40200ad840 con 0x55c5e4790490 2024-09-21T04:03:59.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.569+0000 7f403bd25740 10 client.15456 fetch_fsmap learned FSMap version 25 2024-09-21T04:03:59.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.569+0000 7f403bd25740 1 -- 172.21.15.64:0/3217596498 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c5e4be93c0 con 0x55c5e4790490 2024-09-21T04:03:59.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.569+0000 7f4012ff5640 1 -- 172.21.15.64:0/3217596498 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40200a9040 con 0x55c5e4790490 2024-09-21T04:03:59.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.569+0000 7f4012ff5640 1 -- 172.21.15.64:0/3217596498 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f40200b21b0 con 0x55c5e4790490 2024-09-21T04:03:59.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.569+0000 7f403bd25740 10 client.15456 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:03:59.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.569+0000 7f403bd25740 10 client.15456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:03:59.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.569+0000 7f403bd25740 20 client.15456 populate_metadata read hostname 'smithi064' 2024-09-21T04:03:59.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.569+0000 7f403bd25740 1 --2- 172.21.15.64:0/3217596498 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55c5e474d080 0x55c5e4bebb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:59.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.569+0000 7f403bd25740 4 client.15456 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T04:03:59.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.569+0000 7f403bd25740 1 -- 172.21.15.64:0/3217596498 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f4028036040 con 0x55c5e474d080 2024-09-21T04:03:59.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.570+0000 7f402e7fc640 1 --2- 172.21.15.64:0/3217596498 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55c5e474d080 0x55c5e4bebb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:59.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.570+0000 7f402e7fc640 1 --2- 172.21.15.64:0/3217596498 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55c5e474d080 0x55c5e4bebb50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74a7d4a09a5357cc server_cookie=a4727da91f6883b9 in_seq=0 out_seq=0 2024-09-21T04:03:59.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.570+0000 7f4012ff5640 10 client.15456 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:03:59.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.572+0000 7f4012ff5640 1 -- 172.21.15.64:0/3217596498 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f4028036040 con 0x55c5e474d080 2024-09-21T04:03:59.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.572+0000 7f4012ff5640 10 client.15456 handle_command_reply: tid=0 2024-09-21T04:03:59.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.572+0000 7f403bd25740 1 client.15456 shutdown 2024-09-21T04:03:59.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.572+0000 7f403bd25740 1 -- 172.21.15.64:0/3217596498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e4790490 msgr2=0x55c5e4b70260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:59.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.572+0000 7f403bd25740 1 --2- 172.21.15.64:0/3217596498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e4790490 0x55c5e4b70260 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f40200ad720 tx=0x7f40200a95f0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.573+0000 7f403bd25740 1 -- 172.21.15.64:0/3217596498 shutdown_connections 2024-09-21T04:03:59.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.573+0000 7f403bd25740 1 --2- 172.21.15.64:0/3217596498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c5e47b9350 0x55c5e4be7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.573+0000 7f403bd25740 1 --2- 172.21.15.64:0/3217596498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c5e4798670 0x55c5e4b70800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.573+0000 7f403bd25740 1 --2- 172.21.15.64:0/3217596498 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55c5e474d080 0x55c5e4bebb50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.573+0000 7f403bd25740 1 --2- 172.21.15.64:0/3217596498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c5e4790490 0x55c5e4b70260 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.573+0000 7f403bd25740 1 -- 172.21.15.64:0/3217596498 >> 172.21.15.64:0/3217596498 conn(0x55c5e47ad740 msgr2=0x55c5e4b6d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:59.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.573+0000 7f4012ff5640 0 client.15456 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:03:59.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.573+0000 7f403bd25740 1 -- 172.21.15.64:0/3217596498 shutdown_connections 2024-09-21T04:03:59.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.573+0000 7f403bd25740 1 -- 172.21.15.64:0/3217596498 wait complete. 2024-09-21T04:03:59.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.573+0000 7f403bd25740 20 client.15456 trim_cache size 0 max 16384 2024-09-21T04:03:59.579 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:03:59.579 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-21T04:03:59.580 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "cb0c7943-c962-4f34-86e2-8e678a340f05", 2024-09-21T04:03:59.580 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-21T04:03:59.580 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:03:59.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.574+0000 7f40357b1640 1 -- 172.21.15.64:0/525554029 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f40180b65f0 msgr2=0x7f40180b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:59.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.574+0000 7f40357b1640 1 --2- 172.21.15.64:0/525554029 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f40180b65f0 0x7f40180b8b00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4028004af0 tx=0x7f4028005c30 comp rx=0 tx=0).stop 2024-09-21T04:03:59.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.574+0000 7f40357b1640 1 -- 172.21.15.64:0/525554029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4030104840 msgr2=0x7f403019c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:59.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.574+0000 7f40357b1640 1 --2- 172.21.15.64:0/525554029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4030104840 0x7f403019c6e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f402000e9c0 tx=0x7f402000ee80 comp rx=0 tx=0).stop 2024-09-21T04:03:59.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.574+0000 7f40357b1640 1 -- 172.21.15.64:0/525554029 shutdown_connections 2024-09-21T04:03:59.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.574+0000 7f40357b1640 1 --2- 172.21.15.64:0/525554029 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f40180b65f0 0x7f40180b8b00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.574+0000 7f40357b1640 1 --2- 172.21.15.64:0/525554029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f403010ddf0 0x7f40301a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.574+0000 7f40357b1640 1 --2- 172.21.15.64:0/525554029 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4030105270 0x7f403019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.574+0000 7f40357b1640 1 --2- 172.21.15.64:0/525554029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4030104840 0x7f403019c6e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.574+0000 7f40357b1640 1 -- 172.21.15.64:0/525554029 >> 172.21.15.64:0/525554029 conn(0x7f4030079730 msgr2=0x7f4030100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:59.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.574+0000 7f40357b1640 1 -- 172.21.15.64:0/525554029 shutdown_connections 2024-09-21T04:03:59.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.574+0000 7f40357b1640 1 -- 172.21.15.64:0/525554029 wait complete. 2024-09-21T04:03:59.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.662+0000 7f9894ff9640 1 -- 172.21.15.64:0/1754550007 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f9888000c60 con 0x7f988c0b65f0 2024-09-21T04:03:59.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.730+0000 7f9896ffd640 1 -- 172.21.15.64:0/1754550007 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f9888000c60 con 0x7f988c0b65f0 2024-09-21T04:03:59.733 INFO:teuthology.orchestra.run.smithi064.stdout:Retention added to path / 2024-09-21T04:03:59.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.733+0000 7f9894ff9640 1 -- 172.21.15.64:0/1754550007 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f988c0b65f0 msgr2=0x7f988c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:59.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.733+0000 7f9894ff9640 1 --2- 172.21.15.64:0/1754550007 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f988c0b65f0 0x7f988c0b8b00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f989002bd10 tx=0x7f9890005a20 comp rx=0 tx=0).stop 2024-09-21T04:03:59.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.733+0000 7f9894ff9640 1 -- 172.21.15.64:0/1754550007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98a0102180 msgr2=0x7f98a019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:59.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.733+0000 7f9894ff9640 1 --2- 172.21.15.64:0/1754550007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98a0102180 0x7f98a019a480 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f989c00d950 tx=0x7f989c00de10 comp rx=0 tx=0).stop 2024-09-21T04:03:59.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.733+0000 7f9894ff9640 1 -- 172.21.15.64:0/1754550007 shutdown_connections 2024-09-21T04:03:59.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.733+0000 7f9894ff9640 1 --2- 172.21.15.64:0/1754550007 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f988c0b65f0 0x7f988c0b8b00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.733+0000 7f9894ff9640 1 --2- 172.21.15.64:0/1754550007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98a0108400 0x7f98a019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.733+0000 7f9894ff9640 1 --2- 172.21.15.64:0/1754550007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98a0107ab0 0x7f98a019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.733+0000 7f9894ff9640 1 --2- 172.21.15.64:0/1754550007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98a0102180 0x7f98a019a480 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.733+0000 7f9894ff9640 1 -- 172.21.15.64:0/1754550007 >> 172.21.15.64:0/1754550007 conn(0x7f98a0079e90 msgr2=0x7f98a0105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:59.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.733+0000 7f9894ff9640 1 -- 172.21.15.64:0/1754550007 shutdown_connections 2024-09-21T04:03:59.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.733+0000 7f9894ff9640 1 -- 172.21.15.64:0/1754550007 wait complete. 2024-09-21T04:03:59.746 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:59 smithi064 ceph-mon[31180]: pgmap v320: 97 pgs: 97 active+clean; 653 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-21T04:03:59.746 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:03:59 smithi064 ceph-mon[31180]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:03:59.747 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-21T04:03:59.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.866+0000 7f35f4a63640 1 -- 172.21.15.64:0/4162722379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f0109430 msgr2=0x7f35f0109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:59.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.866+0000 7f35f4a63640 1 --2- 172.21.15.64:0/4162722379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f0109430 0x7f35f0109840 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009900 tx=0x7f35d802a410 comp rx=0 tx=0).stop 2024-09-21T04:03:59.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.866+0000 7f35f4a63640 1 -- 172.21.15.64:0/4162722379 shutdown_connections 2024-09-21T04:03:59.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.866+0000 7f35f4a63640 1 --2- 172.21.15.64:0/4162722379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35f0109d80 0x7f35f01105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.866+0000 7f35f4a63640 1 --2- 172.21.15.64:0/4162722379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f0109430 0x7f35f0109840 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.866+0000 7f35f4a63640 1 --2- 172.21.15.64:0/4162722379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35f0102060 0x7f35f0102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.866+0000 7f35f4a63640 1 -- 172.21.15.64:0/4162722379 >> 172.21.15.64:0/4162722379 conn(0x7f35f0079e90 msgr2=0x7f35f007a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:03:59.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.866+0000 7f35f4a63640 1 -- 172.21.15.64:0/4162722379 shutdown_connections 2024-09-21T04:03:59.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.866+0000 7f35f4a63640 1 -- 172.21.15.64:0/4162722379 wait complete. 2024-09-21T04:03:59.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.867+0000 7f35f4a63640 1 Processor -- start 2024-09-21T04:03:59.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.867+0000 7f35f4a63640 1 -- start start 2024-09-21T04:03:59.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.867+0000 7f35f4a63640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35f0102060 0x7f35f0102ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:59.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.867+0000 7f35f4a63640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f0109430 0x7f35f0103530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:59.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.867+0000 7f35f4a63640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35f0109d80 0x7f35f0103a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:59.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.867+0000 7f35f4a63640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f35f0113a40 con 0x7f35f0109430 2024-09-21T04:03:59.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.867+0000 7f35f4a63640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f35f01138c0 con 0x7f35f0109d80 2024-09-21T04:03:59.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.867+0000 7f35f4a63640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f35f0113bc0 con 0x7f35f0102060 2024-09-21T04:03:59.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.867+0000 7f35edd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f0109430 0x7f35f0103530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:59.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.868+0000 7f35edd9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f0109430 0x7f35f0103530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50146/0 (socket says 172.21.15.64:50146) 2024-09-21T04:03:59.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.868+0000 7f35edd9b640 1 -- 172.21.15.64:0/2390585782 learned_addr learned my addr 172.21.15.64:0/2390585782 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:03:59.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.868+0000 7f35eed9d640 1 --2- 172.21.15.64:0/2390585782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35f0109d80 0x7f35f0103a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:59.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.868+0000 7f35ee59c640 1 --2- 172.21.15.64:0/2390585782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35f0102060 0x7f35f0102ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:59.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.868+0000 7f35edd9b640 1 -- 172.21.15.64:0/2390585782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35f0102060 msgr2=0x7f35f0102ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:59.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.868+0000 7f35edd9b640 1 --2- 172.21.15.64:0/2390585782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35f0102060 0x7f35f0102ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.868+0000 7f35edd9b640 1 -- 172.21.15.64:0/2390585782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35f0109d80 msgr2=0x7f35f0103a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:03:59.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.868+0000 7f35edd9b640 1 --2- 172.21.15.64:0/2390585782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35f0109d80 0x7f35f0103a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:03:59.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.868+0000 7f35edd9b640 1 -- 172.21.15.64:0/2390585782 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f01a8730 con 0x7f35f0109430 2024-09-21T04:03:59.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.868+0000 7f35edd9b640 1 --2- 172.21.15.64:0/2390585782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f0109430 0x7f35f0103530 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f35d802a930 tx=0x7f35d8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:59.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.868+0000 7f35e77fe640 1 -- 172.21.15.64:0/2390585782 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d803f030 con 0x7f35f0109430 2024-09-21T04:03:59.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.868+0000 7f35e77fe640 1 -- 172.21.15.64:0/2390585782 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35d8003aa0 con 0x7f35f0109430 2024-09-21T04:03:59.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.868+0000 7f35f4a63640 1 -- 172.21.15.64:0/2390585782 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f01a89c0 con 0x7f35f0109430 2024-09-21T04:03:59.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.868+0000 7f35e77fe640 1 -- 172.21.15.64:0/2390585782 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8035070 con 0x7f35f0109430 2024-09-21T04:03:59.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.869+0000 7f35f4a63640 1 -- 172.21.15.64:0/2390585782 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f01a8eb0 con 0x7f35f0109430 2024-09-21T04:03:59.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.870+0000 7f35f4a63640 1 -- 172.21.15.64:0/2390585782 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35f0102470 con 0x7f35f0109430 2024-09-21T04:03:59.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.872+0000 7f35e77fe640 1 -- 172.21.15.64:0/2390585782 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) ==== 154580+0+0 (secure 0 0 0) 0x7f35d8042030 con 0x7f35f0109430 2024-09-21T04:03:59.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.874+0000 7f35e77fe640 1 --2- 172.21.15.64:0/2390585782 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f35dc0b6620 0x7f35dc0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:03:59.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.874+0000 7f35ee59c640 1 --2- 172.21.15.64:0/2390585782 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f35dc0b6620 0x7f35dc0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:03:59.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.874+0000 7f35e77fe640 1 -- 172.21.15.64:0/2390585782 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f35d81080a0 con 0x7f35f0109430 2024-09-21T04:03:59.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.875+0000 7f35ee59c640 1 --2- 172.21.15.64:0/2390585782 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f35dc0b6620 0x7f35dc0b8b30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f35f0104a40 tx=0x7f35e8009760 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:03:59.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:03:59.876+0000 7f35e77fe640 1 -- 172.21.15.64:0/2390585782 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f35d80d2a60 con 0x7f35f0109430 2024-09-21T04:04:00.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:59 smithi119 ceph-mon[34811]: pgmap v320: 97 pgs: 97 active+clean; 653 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-21T04:04:00.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:03:59 smithi119 ceph-mon[34811]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:04:00.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:59 smithi131 ceph-mon[35193]: pgmap v320: 97 pgs: 97 active+clean; 653 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-09-21T04:04:00.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:03:59 smithi131 ceph-mon[35193]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:04:00.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.091+0000 7f35f4a63640 1 -- 172.21.15.64:0/2390585782 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f35f0063800 con 0x7f35dc0b6620 2024-09-21T04:04:00.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.099+0000 7f35e77fe640 1 -- 172.21.15.64:0/2390585782 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f35f0063800 con 0x7f35dc0b6620 2024-09-21T04:04:00.101 INFO:teuthology.orchestra.run.smithi064.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-21T00:00:00", "created": "2024-09-21T04:03:59", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-21T04:04:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.101+0000 7f35f4a63640 1 -- 172.21.15.64:0/2390585782 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f35dc0b6620 msgr2=0x7f35dc0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.101+0000 7f35f4a63640 1 --2- 172.21.15.64:0/2390585782 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f35dc0b6620 0x7f35dc0b8b30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f35f0104a40 tx=0x7f35e8009760 comp rx=0 tx=0).stop 2024-09-21T04:04:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.101+0000 7f35f4a63640 1 -- 172.21.15.64:0/2390585782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f0109430 msgr2=0x7f35f0103530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.101+0000 7f35f4a63640 1 --2- 172.21.15.64:0/2390585782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f0109430 0x7f35f0103530 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f35d802a930 tx=0x7f35d8004270 comp rx=0 tx=0).stop 2024-09-21T04:04:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.101+0000 7f35f4a63640 1 -- 172.21.15.64:0/2390585782 shutdown_connections 2024-09-21T04:04:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.101+0000 7f35f4a63640 1 --2- 172.21.15.64:0/2390585782 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f35dc0b6620 0x7f35dc0b8b30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.101+0000 7f35f4a63640 1 --2- 172.21.15.64:0/2390585782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35f0109d80 0x7f35f0103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.101+0000 7f35f4a63640 1 --2- 172.21.15.64:0/2390585782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f0109430 0x7f35f0103530 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.101+0000 7f35f4a63640 1 --2- 172.21.15.64:0/2390585782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35f0102060 0x7f35f0102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.101+0000 7f35f4a63640 1 -- 172.21.15.64:0/2390585782 >> 172.21.15.64:0/2390585782 conn(0x7f35f0079e90 msgr2=0x7f35f0107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.101+0000 7f35f4a63640 1 -- 172.21.15.64:0/2390585782 shutdown_connections 2024-09-21T04:04:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.101+0000 7f35f4a63640 1 -- 172.21.15.64:0/2390585782 wait complete. 2024-09-21T04:04:00.115 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-21T04:04:00.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.230+0000 7f756ed90640 1 -- 172.21.15.64:0/3285562260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75681082e0 msgr2=0x7f75681086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:00.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.230+0000 7f756ed90640 1 --2- 172.21.15.64:0/3285562260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75681082e0 0x7f75681086f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7554009900 tx=0x7f755402a410 comp rx=0 tx=0).stop 2024-09-21T04:04:00.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.230+0000 7f756ed90640 1 -- 172.21.15.64:0/3285562260 shutdown_connections 2024-09-21T04:04:00.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.230+0000 7f756ed90640 1 --2- 172.21.15.64:0/3285562260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7568108c30 0x7f756810f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:00.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.230+0000 7f756ed90640 1 --2- 172.21.15.64:0/3285562260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75681082e0 0x7f75681086f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:00.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.230+0000 7f756ed90640 1 --2- 172.21.15.64:0/3285562260 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75681029b0 0x7f7568102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:00.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.230+0000 7f756ed90640 1 -- 172.21.15.64:0/3285562260 >> 172.21.15.64:0/3285562260 conn(0x7f756807a740 msgr2=0x7f756807ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:00.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.230+0000 7f756ed90640 1 -- 172.21.15.64:0/3285562260 shutdown_connections 2024-09-21T04:04:00.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.230+0000 7f756ed90640 1 -- 172.21.15.64:0/3285562260 wait complete. 2024-09-21T04:04:00.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.231+0000 7f756ed90640 1 Processor -- start 2024-09-21T04:04:00.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.231+0000 7f756ed90640 1 -- start start 2024-09-21T04:04:00.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.231+0000 7f756ed90640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75681029b0 0x7f75680ff550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:00.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.231+0000 7f756ed90640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75681082e0 0x7f75680ffa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:00.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.231+0000 7f756ed90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7568108c30 0x7f75680fffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:00.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.231+0000 7f756ed90640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f75681128f0 con 0x7f75681082e0 2024-09-21T04:04:00.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.231+0000 7f756ed90640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7568112770 con 0x7f7568108c30 2024-09-21T04:04:00.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.231+0000 7f756ed90640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7568112a70 con 0x7f75681029b0 2024-09-21T04:04:00.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.231+0000 7f7567fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75681082e0 0x7f75680ffa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:00.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.231+0000 7f7567fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75681082e0 0x7f75680ffa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50156/0 (socket says 172.21.15.64:50156) 2024-09-21T04:04:00.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.231+0000 7f7567fff640 1 -- 172.21.15.64:0/3704541792 learned_addr learned my addr 172.21.15.64:0/3704541792 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:00.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.231+0000 7f756cb2c640 1 --2- 172.21.15.64:0/3704541792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75681029b0 0x7f75680ff550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:00.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.232+0000 7f756d32d640 1 --2- 172.21.15.64:0/3704541792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7568108c30 0x7f75680fffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:00.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.232+0000 7f7567fff640 1 -- 172.21.15.64:0/3704541792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75681029b0 msgr2=0x7f75680ff550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:00.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.232+0000 7f7567fff640 1 --2- 172.21.15.64:0/3704541792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75681029b0 0x7f75680ff550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:00.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.232+0000 7f7567fff640 1 -- 172.21.15.64:0/3704541792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7568108c30 msgr2=0x7f75680fffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:00.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.232+0000 7f7567fff640 1 --2- 172.21.15.64:0/3704541792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7568108c30 0x7f75680fffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:00.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.232+0000 7f7567fff640 1 -- 172.21.15.64:0/3704541792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75681a8770 con 0x7f75681082e0 2024-09-21T04:04:00.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.232+0000 7f7567fff640 1 --2- 172.21.15.64:0/3704541792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75681082e0 0x7f75680ffa90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f755402a930 tx=0x7f7554031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:00.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.232+0000 7f7565ffb640 1 -- 172.21.15.64:0/3704541792 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755403f030 con 0x7f75681082e0 2024-09-21T04:04:00.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.232+0000 7f7565ffb640 1 -- 172.21.15.64:0/3704541792 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7554004490 con 0x7f75681082e0 2024-09-21T04:04:00.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.232+0000 7f756ed90640 1 -- 172.21.15.64:0/3704541792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75681a8a00 con 0x7f75681082e0 2024-09-21T04:04:00.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.232+0000 7f7565ffb640 1 -- 172.21.15.64:0/3704541792 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554035070 con 0x7f75681082e0 2024-09-21T04:04:00.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.232+0000 7f756ed90640 1 -- 172.21.15.64:0/3704541792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75681a8ef0 con 0x7f75681082e0 2024-09-21T04:04:00.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.234+0000 7f756ed90640 1 -- 172.21.15.64:0/3704541792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7568102dc0 con 0x7f75681082e0 2024-09-21T04:04:00.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.235+0000 7f7565ffb640 1 -- 172.21.15.64:0/3704541792 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) ==== 154580+0+0 (secure 0 0 0) 0x7f7554042030 con 0x7f75681082e0 2024-09-21T04:04:00.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.236+0000 7f7565ffb640 1 --2- 172.21.15.64:0/3704541792 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f75580b6620 0x7f75580b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:00.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.236+0000 7f756cb2c640 1 --2- 172.21.15.64:0/3704541792 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f75580b6620 0x7f75580b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:00.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.236+0000 7f7565ffb640 1 -- 172.21.15.64:0/3704541792 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f75541079d0 con 0x7f75681082e0 2024-09-21T04:04:00.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.236+0000 7f756cb2c640 1 --2- 172.21.15.64:0/3704541792 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f75580b6620 0x7f75580b8b30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7568100fa0 tx=0x7f755c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:00.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.240+0000 7f7565ffb640 1 -- 172.21.15.64:0/3704541792 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f75540cdfb0 con 0x7f75681082e0 2024-09-21T04:04:00.372 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T04:04:00.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.454+0000 7f756ed90640 1 -- 172.21.15.64:0/3704541792 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f7568063800 con 0x7f75580b6620 2024-09-21T04:04:00.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.461+0000 7f7565ffb640 1 -- 172.21.15.64:0/3704541792 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f7568063800 con 0x7f75580b6620 2024-09-21T04:04:00.464 INFO:teuthology.orchestra.run.smithi064.stdout:/ 1m 6m3h 2024-09-21T04:04:00.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.464+0000 7f756ed90640 1 -- 172.21.15.64:0/3704541792 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f75580b6620 msgr2=0x7f75580b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:00.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.464+0000 7f756ed90640 1 --2- 172.21.15.64:0/3704541792 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f75580b6620 0x7f75580b8b30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7568100fa0 tx=0x7f755c005f90 comp rx=0 tx=0).stop 2024-09-21T04:04:00.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.464+0000 7f756ed90640 1 -- 172.21.15.64:0/3704541792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75681082e0 msgr2=0x7f75680ffa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:00.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.464+0000 7f756ed90640 1 --2- 172.21.15.64:0/3704541792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75681082e0 0x7f75680ffa90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f755402a930 tx=0x7f7554031a20 comp rx=0 tx=0).stop 2024-09-21T04:04:00.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.464+0000 7f756ed90640 1 -- 172.21.15.64:0/3704541792 shutdown_connections 2024-09-21T04:04:00.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.464+0000 7f756ed90640 1 --2- 172.21.15.64:0/3704541792 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f75580b6620 0x7f75580b8b30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:00.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.464+0000 7f756ed90640 1 --2- 172.21.15.64:0/3704541792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7568108c30 0x7f75680fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:00.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.464+0000 7f756ed90640 1 --2- 172.21.15.64:0/3704541792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f75681082e0 0x7f75680ffa90 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:00.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.464+0000 7f756ed90640 1 --2- 172.21.15.64:0/3704541792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75681029b0 0x7f75680ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:00.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.464+0000 7f756ed90640 1 -- 172.21.15.64:0/3704541792 >> 172.21.15.64:0/3704541792 conn(0x7f756807a740 msgr2=0x7f7568101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:00.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.464+0000 7f756ed90640 1 -- 172.21.15.64:0/3704541792 shutdown_connections 2024-09-21T04:04:00.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:00.464+0000 7f756ed90640 1 -- 172.21.15.64:0/3704541792 wait complete. 2024-09-21T04:04:00.478 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-21T04:04:00.505 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-21T04:04:00.529 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-21T04:04:00.568 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-09-21T04:04:00.568 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-21T04:04:00.570 INFO:teuthology.run_tasks:Running task workunit... 2024-09-21T04:04:00.593 INFO:tasks.workunit:Pulling workunits from ref 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T04:04:00.593 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-21T04:04:00.593 INFO:tasks.workunit:timeout=3h 2024-09-21T04:04:00.594 INFO:tasks.workunit:cleanup=True 2024-09-21T04:04:00.594 DEBUG:teuthology.orchestra.run.smithi064:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-21T04:04:00.610 INFO:teuthology.orchestra.run.smithi064.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-21T04:04:00.611 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-21T04:04:00.611 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-21T04:04:00.611 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T04:04:00.611 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-21T04:04:00.611 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-09-21 04:02:21.490350324 +0000 2024-09-21T04:04:00.611 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-09-21 04:02:21.490350324 +0000 2024-09-21T04:04:00.611 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-09-21 04:03:49.498992543 +0000 2024-09-21T04:04:00.611 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-09-21 04:02:21.490350324 +0000 2024-09-21T04:04:00.611 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-21T04:04:00.611 DEBUG:teuthology.orchestra.run.smithi064:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-21T04:04:00.643 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:04:00.684 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T04:04:00.742 INFO:tasks.workunit.client.0.smithi064.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-21T04:04:00.941 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:00 smithi064 ceph-mon[31180]: scrub queued for path: / 2024-09-21T04:04:00.941 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:00 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:04:00.941 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:00 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:04:00.941 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:00 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:04:00.942 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:00 smithi064 ceph-mon[31180]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:04:00.942 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:00 smithi064 ceph-mon[31180]: from='client.15462 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:04:00.942 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 04:04:00 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: 2024-09-21T04:04:00.741+0000 7fa6f7429640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T04:04:00.942 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 04:04:00 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: 2024-09-21T04:04:00.741+0000 7fa6f7429640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T04:04:00.942 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 04:04:00 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: 2024-09-21T04:04:00.741+0000 7fa6f7429640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T04:04:00.942 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 04:04:00 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: 2024-09-21T04:04:00.741+0000 7fa6f7429640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T04:04:00.942 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 04:04:00 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: 2024-09-21T04:04:00.741+0000 7fa6f7429640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T04:04:01.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:00 smithi119 ceph-mon[34811]: scrub queued for path: / 2024-09-21T04:04:01.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:00 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:04:01.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:00 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:04:01.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:00 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:04:01.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:00 smithi119 ceph-mon[34811]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:04:01.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:00 smithi119 ceph-mon[34811]: from='client.15462 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:04:01.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:00 smithi131 ceph-mon[35193]: scrub queued for path: / 2024-09-21T04:04:01.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:00 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:04:01.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:00 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:04:01.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:00 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:04:01.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:00 smithi131 ceph-mon[35193]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:04:01.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:00 smithi131 ceph-mon[35193]: from='client.15462 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:04:01.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.174+0000 7f1591b7b640 1 -- 172.21.15.64:0/3160245662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f158c105300 msgr2=0x7f158c10d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:01.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.174+0000 7f1591b7b640 1 --2- 172.21.15.64:0/3160245662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f158c105300 0x7f158c10d850 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1580009900 tx=0x7f158002a410 comp rx=0 tx=0).stop 2024-09-21T04:04:01.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.174+0000 7f1591b7b640 1 -- 172.21.15.64:0/3160245662 shutdown_connections 2024-09-21T04:04:01.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.174+0000 7f1591b7b640 1 --2- 172.21.15.64:0/3160245662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f158c10de10 0x7f158c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.174+0000 7f1591b7b640 1 --2- 172.21.15.64:0/3160245662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f158c105300 0x7f158c10d850 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.174+0000 7f1591b7b640 1 --2- 172.21.15.64:0/3160245662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f158c1049b0 0x7f158c104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.174+0000 7f1591b7b640 1 -- 172.21.15.64:0/3160245662 >> 172.21.15.64:0/3160245662 conn(0x7f158c079730 msgr2=0x7f158c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:01.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.174+0000 7f1591b7b640 1 -- 172.21.15.64:0/3160245662 shutdown_connections 2024-09-21T04:04:01.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.174+0000 7f1591b7b640 1 -- 172.21.15.64:0/3160245662 wait complete. 2024-09-21T04:04:01.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.175+0000 7f1591b7b640 1 Processor -- start 2024-09-21T04:04:01.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.175+0000 7f1591b7b640 1 -- start start 2024-09-21T04:04:01.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.176+0000 7f1591b7b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f158c1049b0 0x7f158c1026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:01.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.176+0000 7f1591b7b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f158c105300 0x7f158c102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:01.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.176+0000 7f1591b7b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f158c10de10 0x7f158c103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:01.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.176+0000 7f1591b7b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f158c113a70 con 0x7f158c1049b0 2024-09-21T04:04:01.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.176+0000 7f1591b7b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f158c1138f0 con 0x7f158c105300 2024-09-21T04:04:01.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.176+0000 7f1591b7b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f158c113bf0 con 0x7f158c10de10 2024-09-21T04:04:01.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.176+0000 7f158b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f158c1049b0 0x7f158c1026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:01.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.176+0000 7f158b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f158c1049b0 0x7f158c1026f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50194/0 (socket says 172.21.15.64:50194) 2024-09-21T04:04:01.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.176+0000 7f158b7fe640 1 -- 172.21.15.64:0/1202289159 learned_addr learned my addr 172.21.15.64:0/1202289159 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:01.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.176+0000 7f158affd640 1 --2- 172.21.15.64:0/1202289159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f158c105300 0x7f158c102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:01.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.176+0000 7f158bfff640 1 --2- 172.21.15.64:0/1202289159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f158c10de10 0x7f158c103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:01.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.176+0000 7f158b7fe640 1 -- 172.21.15.64:0/1202289159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f158c10de10 msgr2=0x7f158c103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:01.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.176+0000 7f158b7fe640 1 --2- 172.21.15.64:0/1202289159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f158c10de10 0x7f158c103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.176+0000 7f158b7fe640 1 -- 172.21.15.64:0/1202289159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f158c105300 msgr2=0x7f158c102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:01.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.176+0000 7f158b7fe640 1 --2- 172.21.15.64:0/1202289159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f158c105300 0x7f158c102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.176+0000 7f158b7fe640 1 -- 172.21.15.64:0/1202289159 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158c1a8760 con 0x7f158c1049b0 2024-09-21T04:04:01.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.176+0000 7f158affd640 1 --2- 172.21.15.64:0/1202289159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f158c105300 0x7f158c102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:01.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.176+0000 7f158b7fe640 1 --2- 172.21.15.64:0/1202289159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f158c1049b0 0x7f158c1026f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f157c00eaa0 tx=0x7f157c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:01.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.177+0000 7f1588ff9640 1 -- 172.21.15.64:0/1202289159 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c016070 con 0x7f158c1049b0 2024-09-21T04:04:01.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.177+0000 7f1591b7b640 1 -- 172.21.15.64:0/1202289159 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f158c1a8a50 con 0x7f158c1049b0 2024-09-21T04:04:01.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.177+0000 7f1588ff9640 1 -- 172.21.15.64:0/1202289159 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f157c003e20 con 0x7f158c1049b0 2024-09-21T04:04:01.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.177+0000 7f1588ff9640 1 -- 172.21.15.64:0/1202289159 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c0116d0 con 0x7f158c1049b0 2024-09-21T04:04:01.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.177+0000 7f1591b7b640 1 -- 172.21.15.64:0/1202289159 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f158c1a8f10 con 0x7f158c1049b0 2024-09-21T04:04:01.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.179+0000 7f15980ef740 1 Processor -- start 2024-09-21T04:04:01.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.180+0000 7f15980ef740 1 -- start start 2024-09-21T04:04:01.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.180+0000 7f1588ff9640 1 -- 172.21.15.64:0/1202289159 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) ==== 154637+0+0 (secure 0 0 0) 0x7f157c019070 con 0x7f158c1049b0 2024-09-21T04:04:01.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.180+0000 7f15980ef740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4db070 0x56445d882650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:01.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.180+0000 7f15980ef740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56445d4b21b0 0x56445d882b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:01.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.180+0000 7f15980ef740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56445d4ba390 0x56445d8830d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:01.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.180+0000 7f15980ef740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56445d7f8e50 con 0x56445d4db070 2024-09-21T04:04:01.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.180+0000 7f15980ef740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56445d7e8280 con 0x56445d4b21b0 2024-09-21T04:04:01.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.180+0000 7f15980ef740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56445d7ec090 con 0x56445d4ba390 2024-09-21T04:04:01.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.180+0000 7f158bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4db070 0x56445d882650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:01.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.180+0000 7f158bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4db070 0x56445d882650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50212/0 (socket says 172.21.15.64:50212) 2024-09-21T04:04:01.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.180+0000 7f158bfff640 1 -- 172.21.15.64:0/1329556156 learned_addr learned my addr 172.21.15.64:0/1329556156 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:01.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.180+0000 7f158b7fe640 1 --2- 172.21.15.64:0/1329556156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56445d4b21b0 0x56445d882b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:01.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.180+0000 7f158affd640 1 --2- 172.21.15.64:0/1329556156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56445d4ba390 0x56445d8830d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:01.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.180+0000 7f158bfff640 1 -- 172.21.15.64:0/1329556156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56445d4ba390 msgr2=0x56445d8830d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:01.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.180+0000 7f158bfff640 1 --2- 172.21.15.64:0/1329556156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56445d4ba390 0x56445d8830d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.180+0000 7f158bfff640 1 -- 172.21.15.64:0/1329556156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56445d4b21b0 msgr2=0x56445d882b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:01.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.180+0000 7f158bfff640 1 --2- 172.21.15.64:0/1329556156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56445d4b21b0 0x56445d882b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.180+0000 7f158bfff640 1 -- 172.21.15.64:0/1329556156 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56445d546650 con 0x56445d4db070 2024-09-21T04:04:01.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.180+0000 7f158b7fe640 1 --2- 172.21.15.64:0/1329556156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56445d4b21b0 0x56445d882b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:01.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.181+0000 7f1588ff9640 1 --2- 172.21.15.64:0/1202289159 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f15740b6680 0x7f15740b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:01.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.181+0000 7f158bfff640 1 --2- 172.21.15.64:0/1329556156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4db070 0x56445d882650 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1584009c10 tx=0x7f15840117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b457d95b15acf75 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:01.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.181+0000 7f158b7fe640 1 --2- 172.21.15.64:0/1202289159 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f15740b6680 0x7f15740b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:01.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.181+0000 7f1571ffb640 1 -- 172.21.15.64:0/1329556156 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1584012450 con 0x56445d4db070 2024-09-21T04:04:01.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.181+0000 7f1571ffb640 1 -- 172.21.15.64:0/1329556156 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1584012aa0 con 0x56445d4db070 2024-09-21T04:04:01.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.181+0000 7f1571ffb640 1 -- 172.21.15.64:0/1329556156 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158400fac0 con 0x56445d4db070 2024-09-21T04:04:01.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.181+0000 7f1588ff9640 1 -- 172.21.15.64:0/1202289159 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f157c0e7840 con 0x7f158c1049b0 2024-09-21T04:04:01.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.182+0000 7f15980ef740 1 -- 172.21.15.64:0/1329556156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4db070 msgr2=0x56445d882650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:01.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.182+0000 7f15980ef740 1 --2- 172.21.15.64:0/1329556156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4db070 0x56445d882650 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1584009c10 tx=0x7f15840117b0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.182+0000 7f158b7fe640 1 --2- 172.21.15.64:0/1202289159 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f15740b6680 0x7f15740b8b90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f157c0e5010 tx=0x7f157c0ed550 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:01.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.182+0000 7f15980ef740 1 -- 172.21.15.64:0/1329556156 shutdown_connections 2024-09-21T04:04:01.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.182+0000 7f15980ef740 1 --2- 172.21.15.64:0/1329556156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56445d4ba390 0x56445d8830d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.182+0000 7f15980ef740 1 --2- 172.21.15.64:0/1329556156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56445d4b21b0 0x56445d882b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.182+0000 7f15980ef740 1 --2- 172.21.15.64:0/1329556156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4db070 0x56445d882650 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.182+0000 7f15980ef740 1 -- 172.21.15.64:0/1329556156 >> 172.21.15.64:0/1329556156 conn(0x56445d4cf460 msgr2=0x56445d880540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:01.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.182+0000 7f15980ef740 1 -- 172.21.15.64:0/1329556156 shutdown_connections 2024-09-21T04:04:01.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.182+0000 7f15980ef740 1 -- 172.21.15.64:0/1329556156 wait complete. 2024-09-21T04:04:01.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.182+0000 7f15980ef740 1 -- start start 2024-09-21T04:04:01.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.183+0000 7f15980ef740 1 Processor -- start 2024-09-21T04:04:01.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.183+0000 7f15980ef740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56445d4b21b0 0x56445d88dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:01.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.183+0000 7f15980ef740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4ba390 0x56445d88e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:01.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.183+0000 7f15980ef740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56445d4db070 0x56445d89a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:01.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.183+0000 7f15980ef740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56445d649700 con 0x56445d4ba390 2024-09-21T04:04:01.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.183+0000 7f15980ef740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56445d783110 con 0x56445d4b21b0 2024-09-21T04:04:01.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.183+0000 7f15980ef740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56445d890e30 con 0x56445d4db070 2024-09-21T04:04:01.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.183+0000 7f15980ef740 10 client.0 _ll_register_callbacks cb 0x56445d79c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:01.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.183+0000 7f158affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4ba390 0x56445d88e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:01.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.183+0000 7f158affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4ba390 0x56445d88e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50226/0 (socket says 172.21.15.64:50226) 2024-09-21T04:04:01.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.183+0000 7f158affd640 1 -- 172.21.15.64:0/208602110 learned_addr learned my addr 172.21.15.64:0/208602110 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:01.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.183+0000 7f158bfff640 1 --2- 172.21.15.64:0/208602110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56445d4db070 0x56445d89a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:01.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.183+0000 7f158bfff640 1 --2- 172.21.15.64:0/208602110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56445d4b21b0 0x56445d88dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:01.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.184+0000 7f158affd640 1 -- 172.21.15.64:0/208602110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56445d4db070 msgr2=0x56445d89a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:01.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.184+0000 7f158affd640 1 --2- 172.21.15.64:0/208602110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56445d4db070 0x56445d89a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.184+0000 7f158affd640 1 -- 172.21.15.64:0/208602110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56445d4b21b0 msgr2=0x56445d88dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:01.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.184+0000 7f158affd640 1 --2- 172.21.15.64:0/208602110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56445d4b21b0 0x56445d88dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.184+0000 7f158affd640 1 -- 172.21.15.64:0/208602110 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56445d7b0c40 con 0x56445d4ba390 2024-09-21T04:04:01.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.184+0000 7f158bfff640 1 --2- 172.21.15.64:0/208602110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56445d4db070 0x56445d89a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:04:01.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.184+0000 7f158affd640 1 --2- 172.21.15.64:0/208602110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4ba390 0x56445d88e260 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1580009900 tx=0x7f15800316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe27d14a579f75df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:01.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.184+0000 7f156affd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:04:01.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.184+0000 7f156affd640 1 -- 172.21.15.64:0/208602110 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1580035040 con 0x56445d4ba390 2024-09-21T04:04:01.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.184+0000 7f156affd640 1 -- 172.21.15.64:0/208602110 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f1580004040 con 0x56445d4ba390 2024-09-21T04:04:01.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.184+0000 7f15980ef740 1 -- 172.21.15.64:0/208602110 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1584012aa0 con 0x56445d4ba390 2024-09-21T04:04:01.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.184+0000 7f156affd640 1 -- 172.21.15.64:0/208602110 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1580004240 con 0x56445d4ba390 2024-09-21T04:04:01.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.184+0000 7f15980ef740 10 client.15486 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:01.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.184+0000 7f15980ef740 1 -- 172.21.15.64:0/208602110 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56445d89cb00 con 0x56445d4ba390 2024-09-21T04:04:01.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.184+0000 7f156affd640 1 -- 172.21.15.64:0/208602110 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1584012aa0 con 0x56445d4ba390 2024-09-21T04:04:01.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.185+0000 7f156affd640 1 -- 172.21.15.64:0/208602110 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1580004ae0 con 0x56445d4ba390 2024-09-21T04:04:01.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.185+0000 7f15980ef740 10 client.15486 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:01.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.185+0000 7f15980ef740 10 client.15486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:04:01.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.185+0000 7f15980ef740 20 client.15486 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:01.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.185+0000 7f15980ef740 1 --2- 172.21.15.64:0/208602110 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56445d4999f0 0x56445d8a0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:01.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.185+0000 7f15980ef740 4 client.15486 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:04:01.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.185+0000 7f15980ef740 1 -- 172.21.15.64:0/208602110 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56445d7fada0 con 0x56445d4999f0 2024-09-21T04:04:01.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.185+0000 7f158bfff640 1 --2- 172.21.15.64:0/208602110 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56445d4999f0 0x56445d8a0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:01.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.185+0000 7f156affd640 10 client.15486 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:01.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.185+0000 7f158bfff640 1 --2- 172.21.15.64:0/208602110 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56445d4999f0 0x56445d8a0060 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b8ba7f7cd538821 server_cookie=fe628b4675c52cf3 in_seq=0 out_seq=0 2024-09-21T04:04:01.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.186+0000 7f156affd640 1 -- 172.21.15.64:0/208602110 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56445d7fada0 con 0x56445d4999f0 2024-09-21T04:04:01.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.186+0000 7f156affd640 10 client.15486 handle_command_reply: tid=0 2024-09-21T04:04:01.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.186+0000 7f15980ef740 1 client.15486 shutdown 2024-09-21T04:04:01.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.186+0000 7f15980ef740 1 -- 172.21.15.64:0/208602110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4ba390 msgr2=0x56445d88e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:01.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.186+0000 7f15980ef740 1 --2- 172.21.15.64:0/208602110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4ba390 0x56445d88e260 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1580009900 tx=0x7f15800316d0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.186+0000 7f15980ef740 1 -- 172.21.15.64:0/208602110 shutdown_connections 2024-09-21T04:04:01.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.186+0000 7f15980ef740 1 --2- 172.21.15.64:0/208602110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56445d4db070 0x56445d89a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.205 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.186+0000 7f15980ef740 1 --2- 172.21.15.64:0/208602110 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56445d4999f0 0x56445d8a0060 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.186+0000 7f15980ef740 1 --2- 172.21.15.64:0/208602110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4ba390 0x56445d88e260 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.186+0000 7f15980ef740 1 --2- 172.21.15.64:0/208602110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56445d4b21b0 0x56445d88dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.186+0000 7f15980ef740 1 -- 172.21.15.64:0/208602110 >> 172.21.15.64:0/208602110 conn(0x56445d4cf460 msgr2=0x56445d885830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:01.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.186+0000 7f156affd640 0 client.15486 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:01.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.187+0000 7f15980ef740 1 -- 172.21.15.64:0/208602110 shutdown_connections 2024-09-21T04:04:01.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.187+0000 7f15980ef740 1 -- 172.21.15.64:0/208602110 wait complete. 2024-09-21T04:04:01.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.187+0000 7f15980ef740 20 client.15486 trim_cache size 0 max 16384 2024-09-21T04:04:01.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.204+0000 7f15980ef740 1 Processor -- start 2024-09-21T04:04:01.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.204+0000 7f15980ef740 1 -- start start 2024-09-21T04:04:01.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.204+0000 7f15980ef740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4b21b0 0x56445d883d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:01.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.204+0000 7f15980ef740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56445d4ba390 0x56445d87fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:01.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.204+0000 7f15980ef740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56445d4db070 0x56445d8800b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:01.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.204+0000 7f15980ef740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f158c1b4330 con 0x56445d4b21b0 2024-09-21T04:04:01.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.204+0000 7f15980ef740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f158c1b1c00 con 0x56445d4db070 2024-09-21T04:04:01.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.204+0000 7f15980ef740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f158c1afa50 con 0x56445d4ba390 2024-09-21T04:04:01.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.205+0000 7f158bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4b21b0 0x56445d883d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:01.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.205+0000 7f158bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4b21b0 0x56445d883d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50238/0 (socket says 172.21.15.64:50238) 2024-09-21T04:04:01.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.205+0000 7f158bfff640 1 -- 172.21.15.64:0/1678276610 learned_addr learned my addr 172.21.15.64:0/1678276610 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:01.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.205+0000 7f158affd640 1 --2- 172.21.15.64:0/1678276610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56445d4ba390 0x56445d87fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:01.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.205+0000 7f158bfff640 1 -- 172.21.15.64:0/1678276610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56445d4ba390 msgr2=0x56445d87fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:01.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.205+0000 7f158bfff640 1 --2- 172.21.15.64:0/1678276610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56445d4ba390 0x56445d87fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.205+0000 7f158bfff640 1 -- 172.21.15.64:0/1678276610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56445d4db070 msgr2=0x56445d8800b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:01.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.205+0000 7f158bfff640 1 --2- 172.21.15.64:0/1678276610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56445d4db070 0x56445d8800b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.205+0000 7f158bfff640 1 -- 172.21.15.64:0/1678276610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56445d88efe0 con 0x56445d4b21b0 2024-09-21T04:04:01.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.205+0000 7f158affd640 1 --2- 172.21.15.64:0/1678276610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56445d4ba390 0x56445d87fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:04:01.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.205+0000 7f158bfff640 1 --2- 172.21.15.64:0/1678276610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4b21b0 0x56445d883d90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1584011cb0 tx=0x7f158400f660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c894a54c0b9bcf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:01.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.205+0000 7f15717fa640 1 -- 172.21.15.64:0/1678276610 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1584036070 con 0x56445d4b21b0 2024-09-21T04:04:01.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.205+0000 7f15717fa640 1 -- 172.21.15.64:0/1678276610 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f158400f790 con 0x56445d4b21b0 2024-09-21T04:04:01.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.205+0000 7f15717fa640 1 -- 172.21.15.64:0/1678276610 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15840037f0 con 0x56445d4b21b0 2024-09-21T04:04:01.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.206+0000 7f15980ef740 1 -- 172.21.15.64:0/1678276610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4b21b0 msgr2=0x56445d883d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:01.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.206+0000 7f15980ef740 1 --2- 172.21.15.64:0/1678276610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4b21b0 0x56445d883d90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1584011cb0 tx=0x7f158400f660 comp rx=0 tx=0).stop 2024-09-21T04:04:01.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.206+0000 7f15980ef740 1 -- 172.21.15.64:0/1678276610 shutdown_connections 2024-09-21T04:04:01.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.206+0000 7f15980ef740 1 --2- 172.21.15.64:0/1678276610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56445d4db070 0x56445d8800b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.206+0000 7f15980ef740 1 --2- 172.21.15.64:0/1678276610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56445d4ba390 0x56445d87fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.206+0000 7f15980ef740 1 --2- 172.21.15.64:0/1678276610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4b21b0 0x56445d883d90 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.206+0000 7f15980ef740 1 -- 172.21.15.64:0/1678276610 >> 172.21.15.64:0/1678276610 conn(0x56445d4cf460 msgr2=0x56445d883680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:01.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.206+0000 7f15980ef740 1 -- 172.21.15.64:0/1678276610 shutdown_connections 2024-09-21T04:04:01.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.206+0000 7f15980ef740 1 -- 172.21.15.64:0/1678276610 wait complete. 2024-09-21T04:04:01.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.207+0000 7f15980ef740 1 -- start start 2024-09-21T04:04:01.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.207+0000 7f15980ef740 1 Processor -- start 2024-09-21T04:04:01.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.207+0000 7f15980ef740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4b21b0 0x56445d892140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:01.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.207+0000 7f15980ef740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56445d4ba390 0x56445d892680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:01.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.207+0000 7f15980ef740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56445d4db070 0x56445d892bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:01.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.207+0000 7f15980ef740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f158c1b00c0 con 0x56445d4b21b0 2024-09-21T04:04:01.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.207+0000 7f15980ef740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f158c1b1530 con 0x56445d4ba390 2024-09-21T04:04:01.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.207+0000 7f15980ef740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f158c1b0790 con 0x56445d4db070 2024-09-21T04:04:01.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.207+0000 7f15980ef740 10 client.0 _ll_register_callbacks cb 0x7f1580009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:01.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.207+0000 7f158bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4b21b0 0x56445d892140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:01.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.207+0000 7f158bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4b21b0 0x56445d892140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50240/0 (socket says 172.21.15.64:50240) 2024-09-21T04:04:01.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.207+0000 7f158bfff640 1 -- 172.21.15.64:0/3574313047 learned_addr learned my addr 172.21.15.64:0/3574313047 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:01.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.208+0000 7f158affd640 1 --2- 172.21.15.64:0/3574313047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56445d4ba390 0x56445d892680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:01.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.208+0000 7f158bfff640 1 --2- 172.21.15.64:0/3574313047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56445d4db070 0x56445d892bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:01.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.208+0000 7f158bfff640 1 -- 172.21.15.64:0/3574313047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56445d4db070 msgr2=0x56445d892bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:01.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.208+0000 7f158bfff640 1 --2- 172.21.15.64:0/3574313047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56445d4db070 0x56445d892bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.208+0000 7f158bfff640 1 -- 172.21.15.64:0/3574313047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56445d4ba390 msgr2=0x56445d892680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:01.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.208+0000 7f158bfff640 1 --2- 172.21.15.64:0/3574313047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56445d4ba390 0x56445d892680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.208+0000 7f158bfff640 1 -- 172.21.15.64:0/3574313047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56445d912320 con 0x56445d4b21b0 2024-09-21T04:04:01.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.208+0000 7f158bfff640 1 --2- 172.21.15.64:0/3574313047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4b21b0 0x56445d892140 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1584006010 tx=0x7f158400f6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d49557cbe0a48a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:01.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.208+0000 7f156affd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:04:01.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.208+0000 7f156affd640 1 -- 172.21.15.64:0/3574313047 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1584036040 con 0x56445d4b21b0 2024-09-21T04:04:01.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.208+0000 7f156affd640 1 -- 172.21.15.64:0/3574313047 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f158400f940 con 0x56445d4b21b0 2024-09-21T04:04:01.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.208+0000 7f15980ef740 1 -- 172.21.15.64:0/3574313047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f158400f790 con 0x56445d4b21b0 2024-09-21T04:04:01.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.209+0000 7f156affd640 1 -- 172.21.15.64:0/3574313047 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f15840055d0 con 0x56445d4b21b0 2024-09-21T04:04:01.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.209+0000 7f15980ef740 10 client.15492 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:01.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.209+0000 7f15980ef740 1 -- 172.21.15.64:0/3574313047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56445d913590 con 0x56445d4b21b0 2024-09-21T04:04:01.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.209+0000 7f156affd640 1 -- 172.21.15.64:0/3574313047 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f158400f790 con 0x56445d4b21b0 2024-09-21T04:04:01.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.209+0000 7f156affd640 1 -- 172.21.15.64:0/3574313047 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1584045070 con 0x56445d4b21b0 2024-09-21T04:04:01.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.209+0000 7f15980ef740 10 client.15492 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:01.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.209+0000 7f15980ef740 10 client.15492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:04:01.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.209+0000 7f15980ef740 20 client.15492 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:01.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.209+0000 7f15980ef740 1 --2- 172.21.15.64:0/3574313047 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56445d46eda0 0x56445d915a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:01.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.209+0000 7f15980ef740 4 client.15492 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T04:04:01.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.209+0000 7f15980ef740 1 -- 172.21.15.64:0/3574313047 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56445d7f6850 con 0x56445d46eda0 2024-09-21T04:04:01.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.209+0000 7f158affd640 1 --2- 172.21.15.64:0/3574313047 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56445d46eda0 0x56445d915a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:01.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.210+0000 7f156affd640 10 client.15492 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:01.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.210+0000 7f158affd640 1 --2- 172.21.15.64:0/3574313047 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56445d46eda0 0x56445d915a10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96dff88bb8efe2c6 server_cookie=aaaa1447dd8d7110 in_seq=0 out_seq=0 2024-09-21T04:04:01.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.210+0000 7f156affd640 1 -- 172.21.15.64:0/3574313047 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+570 (crc 0 0 0) 0x56445d7f6850 con 0x56445d46eda0 2024-09-21T04:04:01.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.210+0000 7f156affd640 10 client.15492 handle_command_reply: tid=0 2024-09-21T04:04:01.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.210+0000 7f15980ef740 1 client.15492 shutdown 2024-09-21T04:04:01.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.210+0000 7f15980ef740 1 -- 172.21.15.64:0/3574313047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4b21b0 msgr2=0x56445d892140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:01.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.210+0000 7f15980ef740 1 --2- 172.21.15.64:0/3574313047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4b21b0 0x56445d892140 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1584006010 tx=0x7f158400f6d0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.210+0000 7f15980ef740 1 -- 172.21.15.64:0/3574313047 shutdown_connections 2024-09-21T04:04:01.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.210+0000 7f15980ef740 1 --2- 172.21.15.64:0/3574313047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56445d4db070 0x56445d892bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.210+0000 7f15980ef740 1 --2- 172.21.15.64:0/3574313047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56445d4ba390 0x56445d892680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.210+0000 7f15980ef740 1 --2- 172.21.15.64:0/3574313047 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56445d46eda0 0x56445d915a10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.210+0000 7f15980ef740 1 --2- 172.21.15.64:0/3574313047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56445d4b21b0 0x56445d892140 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.210+0000 7f156affd640 0 client.15492 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:01.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.210+0000 7f15980ef740 1 -- 172.21.15.64:0/3574313047 >> 172.21.15.64:0/3574313047 conn(0x56445d4cf460 msgr2=0x56445d883680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:01.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.210+0000 7f15980ef740 1 -- 172.21.15.64:0/3574313047 shutdown_connections 2024-09-21T04:04:01.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.210+0000 7f15980ef740 1 -- 172.21.15.64:0/3574313047 wait complete. 2024-09-21T04:04:01.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.210+0000 7f15980ef740 20 client.15492 trim_cache size 0 max 16384 2024-09-21T04:04:01.219 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:04:01.219 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T04:04:01.219 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-09-21T04:04:01.219 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:04:01.219 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:04:01.219 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "cb0c7943-c962-4f34-86e2-8e678a340f05", 2024-09-21T04:04:01.219 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-09-21T04:04:01.219 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:04:01.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:04:01.220 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_stats": { 2024-09-21T04:04:01.220 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:04:01.220 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:03:59.570174+0000", 2024-09-21T04:04:01.220 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:04:01.220 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:04:01.220 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:04:01.220 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:04:01.220 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:04:01.220 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:04:01.220 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:04:01.220 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:04:01.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.211+0000 7f1591b7b640 1 -- 172.21.15.64:0/1202289159 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f15740b6680 msgr2=0x7f15740b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:01.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.211+0000 7f1591b7b640 1 --2- 172.21.15.64:0/1202289159 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f15740b6680 0x7f15740b8b90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f157c0e5010 tx=0x7f157c0ed550 comp rx=0 tx=0).stop 2024-09-21T04:04:01.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.211+0000 7f1591b7b640 1 -- 172.21.15.64:0/1202289159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f158c1049b0 msgr2=0x7f158c1026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:01.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.211+0000 7f1591b7b640 1 --2- 172.21.15.64:0/1202289159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f158c1049b0 0x7f158c1026f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f157c00eaa0 tx=0x7f157c00ef60 comp rx=0 tx=0).stop 2024-09-21T04:04:01.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.211+0000 7f1591b7b640 1 -- 172.21.15.64:0/1202289159 shutdown_connections 2024-09-21T04:04:01.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.211+0000 7f1591b7b640 1 --2- 172.21.15.64:0/1202289159 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f15740b6680 0x7f15740b8b90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.211+0000 7f1591b7b640 1 --2- 172.21.15.64:0/1202289159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f158c10de10 0x7f158c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.211+0000 7f1591b7b640 1 --2- 172.21.15.64:0/1202289159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f158c105300 0x7f158c102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.211+0000 7f1591b7b640 1 --2- 172.21.15.64:0/1202289159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f158c1049b0 0x7f158c1026f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:01.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.211+0000 7f1591b7b640 1 -- 172.21.15.64:0/1202289159 >> 172.21.15.64:0/1202289159 conn(0x7f158c079730 msgr2=0x7f158c0ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:01.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.211+0000 7f1591b7b640 1 -- 172.21.15.64:0/1202289159 shutdown_connections 2024-09-21T04:04:01.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:01.211+0000 7f1591b7b640 1 -- 172.21.15.64:0/1202289159 wait complete. 2024-09-21T04:04:01.987 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:01 smithi064 ceph-mon[31180]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:04:01.987 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:01 smithi064 ceph-mon[31180]: pgmap v321: 97 pgs: 97 active+clean; 653 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-21T04:04:01.987 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:01 smithi064 ceph-mon[31180]: mgrmap e24: x(active, since 8m), standbys: z, y 2024-09-21T04:04:02.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:01 smithi119 ceph-mon[34811]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:04:02.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:01 smithi119 ceph-mon[34811]: pgmap v321: 97 pgs: 97 active+clean; 653 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-21T04:04:02.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:01 smithi119 ceph-mon[34811]: mgrmap e24: x(active, since 8m), standbys: z, y 2024-09-21T04:04:02.021 INFO:tasks.cephfs.filesystem:scrub status for tag:cb0c7943-c962-4f34-86e2-8e678a340f05 - {'path': '/', 'tag': 'cb0c7943-c962-4f34-86e2-8e678a340f05', 'options': 'recursive,force'} 2024-09-21T04:04:02.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:01 smithi131 ceph-mon[35193]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:04:02.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:01 smithi131 ceph-mon[35193]: pgmap v321: 97 pgs: 97 active+clean; 653 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-21T04:04:02.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:01 smithi131 ceph-mon[35193]: mgrmap e24: x(active, since 8m), standbys: z, y 2024-09-21T04:04:04.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:03 smithi119 ceph-mon[34811]: pgmap v322: 97 pgs: 97 active+clean; 1.4 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 198 KiB/s wr, 71 op/s 2024-09-21T04:04:04.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:03 smithi119 ceph-mon[34811]: mgrmap e25: x(active, since 8m), standbys: z, y 2024-09-21T04:04:04.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:03 smithi064 ceph-mon[31180]: pgmap v322: 97 pgs: 97 active+clean; 1.4 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 198 KiB/s wr, 71 op/s 2024-09-21T04:04:04.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:03 smithi064 ceph-mon[31180]: mgrmap e25: x(active, since 8m), standbys: z, y 2024-09-21T04:04:04.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:03 smithi131 ceph-mon[35193]: pgmap v322: 97 pgs: 97 active+clean; 1.4 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 198 KiB/s wr, 71 op/s 2024-09-21T04:04:04.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:03 smithi131 ceph-mon[35193]: mgrmap e25: x(active, since 8m), standbys: z, y 2024-09-21T04:04:06.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:05 smithi119 ceph-mon[34811]: pgmap v323: 97 pgs: 97 active+clean; 1.4 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 197 KiB/s wr, 67 op/s 2024-09-21T04:04:06.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:05 smithi064 ceph-mon[31180]: pgmap v323: 97 pgs: 97 active+clean; 1.4 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 197 KiB/s wr, 67 op/s 2024-09-21T04:04:06.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:05 smithi131 ceph-mon[35193]: pgmap v323: 97 pgs: 97 active+clean; 1.4 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 197 KiB/s wr, 67 op/s 2024-09-21T04:04:08.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:07 smithi119 ceph-mon[34811]: pgmap v324: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 197 KiB/s wr, 68 op/s 2024-09-21T04:04:08.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:07 smithi064 ceph-mon[31180]: pgmap v324: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 197 KiB/s wr, 68 op/s 2024-09-21T04:04:08.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:07 smithi131 ceph-mon[35193]: pgmap v324: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 197 KiB/s wr, 68 op/s 2024-09-21T04:04:09.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:09 smithi064 ceph-mon[31180]: pgmap v325: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 197 KiB/s wr, 68 op/s 2024-09-21T04:04:09.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:09 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:04:10.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:09 smithi119 ceph-mon[34811]: pgmap v325: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 197 KiB/s wr, 68 op/s 2024-09-21T04:04:10.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:09 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:04:10.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:09 smithi131 ceph-mon[35193]: pgmap v325: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 197 KiB/s wr, 68 op/s 2024-09-21T04:04:10.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:09 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:04:11.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:04:11.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:04:11.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:04:11.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:04:11.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:04:11.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:04:11.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:11 smithi119 ceph-mon[34811]: pgmap v326: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 197 KiB/s wr, 67 op/s 2024-09-21T04:04:11.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:04:11.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:04:11.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:04:11.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:04:11.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:04:11.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:04:11.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:04:11.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:04:11.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:04:11.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:11 smithi064 ceph-mon[31180]: pgmap v326: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 197 KiB/s wr, 67 op/s 2024-09-21T04:04:11.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:04:11.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:04:11.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:04:11.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:04:11.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:04:11.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:04:11.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:04:11.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:04:11.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:04:11.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:11 smithi131 ceph-mon[35193]: pgmap v326: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 197 KiB/s wr, 67 op/s 2024-09-21T04:04:11.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:04:11.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:04:11.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:04:13.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.733+0000 7f0b05b51640 1 -- 172.21.15.64:0/2511718714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b00108310 msgr2=0x7f0b00108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:13.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.733+0000 7f0b05b51640 1 --2- 172.21.15.64:0/2511718714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b00108310 0x7f0b00108720 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009900 tx=0x7f0ae802a410 comp rx=0 tx=0).stop 2024-09-21T04:04:13.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.733+0000 7f0b05b51640 1 -- 172.21.15.64:0/2511718714 shutdown_connections 2024-09-21T04:04:13.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.733+0000 7f0b05b51640 1 --2- 172.21.15.64:0/2511718714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b00108c60 0x7f0b0010f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.733+0000 7f0b05b51640 1 --2- 172.21.15.64:0/2511718714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b00108310 0x7f0b00108720 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.733+0000 7f0b05b51640 1 --2- 172.21.15.64:0/2511718714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b001029e0 0x7f0b00102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.733+0000 7f0b05b51640 1 -- 172.21.15.64:0/2511718714 >> 172.21.15.64:0/2511718714 conn(0x7f0b000fe3e0 msgr2=0x7f0b00100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:13.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.733+0000 7f0b05b51640 1 -- 172.21.15.64:0/2511718714 shutdown_connections 2024-09-21T04:04:13.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.733+0000 7f0b05b51640 1 -- 172.21.15.64:0/2511718714 wait complete. 2024-09-21T04:04:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.733+0000 7f0b05b51640 1 Processor -- start 2024-09-21T04:04:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.734+0000 7f0b05b51640 1 -- start start 2024-09-21T04:04:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.734+0000 7f0b05b51640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b001029e0 0x7f0b001993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.734+0000 7f0b05b51640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b00108310 0x7f0b00199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.734+0000 7f0b05b51640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b00108c60 0x7f0b0019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.734+0000 7f0b05b51640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0b00112920 con 0x7f0b00108c60 2024-09-21T04:04:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.734+0000 7f0b05b51640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0b001127a0 con 0x7f0b00108310 2024-09-21T04:04:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.734+0000 7f0b05b51640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0b00112aa0 con 0x7f0b001029e0 2024-09-21T04:04:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.734+0000 7f0afffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b00108c60 0x7f0b0019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.734+0000 7f0afffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b00108c60 0x7f0b0019ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53000/0 (socket says 172.21.15.64:53000) 2024-09-21T04:04:13.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.734+0000 7f0afffff640 1 -- 172.21.15.64:0/1334945810 learned_addr learned my addr 172.21.15.64:0/1334945810 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:13.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.734+0000 7f0aff7fe640 1 --2- 172.21.15.64:0/1334945810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b001029e0 0x7f0b001993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:13.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.734+0000 7f0af6ffd640 1 --2- 172.21.15.64:0/1334945810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b00108310 0x7f0b00199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:13.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.735+0000 7f0afffff640 1 -- 172.21.15.64:0/1334945810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b001029e0 msgr2=0x7f0b001993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:13.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.735+0000 7f0afffff640 1 --2- 172.21.15.64:0/1334945810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b001029e0 0x7f0b001993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.735+0000 7f0afffff640 1 -- 172.21.15.64:0/1334945810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b00108310 msgr2=0x7f0b00199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:13.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.735+0000 7f0afffff640 1 --2- 172.21.15.64:0/1334945810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b00108310 0x7f0b00199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.735+0000 7f0afffff640 1 -- 172.21.15.64:0/1334945810 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0019f2d0 con 0x7f0b00108c60 2024-09-21T04:04:13.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.735+0000 7f0af6ffd640 1 --2- 172.21.15.64:0/1334945810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b00108310 0x7f0b00199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:04:13.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.735+0000 7f0afffff640 1 --2- 172.21.15.64:0/1334945810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b00108c60 0x7f0b0019ebd0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0af8009fd0 tx=0x7f0af800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:13.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.735+0000 7f0afd7fa640 1 -- 172.21.15.64:0/1334945810 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af8017070 con 0x7f0b00108c60 2024-09-21T04:04:13.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.735+0000 7f0b05b51640 1 -- 172.21.15.64:0/1334945810 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b0019f5c0 con 0x7f0b00108c60 2024-09-21T04:04:13.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.735+0000 7f0afd7fa640 1 -- 172.21.15.64:0/1334945810 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0af8003e20 con 0x7f0b00108c60 2024-09-21T04:04:13.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.735+0000 7f0b05b51640 1 -- 172.21.15.64:0/1334945810 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b001a5d50 con 0x7f0b00108c60 2024-09-21T04:04:13.740 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.735+0000 7f0afd7fa640 1 -- 172.21.15.64:0/1334945810 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af8020060 con 0x7f0b00108c60 2024-09-21T04:04:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.738+0000 7f0b0cab9740 1 Processor -- start 2024-09-21T04:04:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.738+0000 7f0afd7fa640 1 -- 172.21.15.64:0/1334945810 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 154694+0+0 (secure 0 0 0) 0x7f0af8003970 con 0x7f0b00108c60 2024-09-21T04:04:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.738+0000 7f0b0cab9740 1 -- start start 2024-09-21T04:04:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.738+0000 7f0b0cab9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a20de41b0 0x557a2118c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.738+0000 7f0b0cab9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a20dbb2f0 0x557a211899b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.738+0000 7f0b0cab9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dc34d0 0x557a2118a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.738+0000 7f0b0cab9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557a210fe6f0 con 0x557a20dc34d0 2024-09-21T04:04:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.738+0000 7f0b0cab9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557a210edb20 con 0x557a20dbb2f0 2024-09-21T04:04:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.738+0000 7f0b0cab9740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557a210f1930 con 0x557a20de41b0 2024-09-21T04:04:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.739+0000 7f0af6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dc34d0 0x557a2118a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.739+0000 7f0af6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dc34d0 0x557a2118a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53008/0 (socket says 172.21.15.64:53008) 2024-09-21T04:04:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.739+0000 7f0af6ffd640 1 -- 172.21.15.64:0/827979673 learned_addr learned my addr 172.21.15.64:0/827979673 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.739+0000 7f0aff7fe640 1 --2- 172.21.15.64:0/827979673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a20dbb2f0 0x557a211899b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.739+0000 7f0af6ffd640 1 --2- 172.21.15.64:0/827979673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a20de41b0 0x557a2118c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.739+0000 7f0afd7fa640 1 --2- 172.21.15.64:0/1334945810 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0aec0b6710 0x7f0aec0b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.739+0000 7f0aff7fe640 1 -- 172.21.15.64:0/827979673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a20de41b0 msgr2=0x557a2118c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.739+0000 7f0aff7fe640 1 --2- 172.21.15.64:0/827979673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a20de41b0 0x557a2118c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.739+0000 7f0aff7fe640 1 -- 172.21.15.64:0/827979673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dc34d0 msgr2=0x557a2118a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.739+0000 7f0aff7fe640 1 --2- 172.21.15.64:0/827979673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dc34d0 0x557a2118a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.739+0000 7f0aff7fe640 1 -- 172.21.15.64:0/827979673 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a210b3530 con 0x557a20dbb2f0 2024-09-21T04:04:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.739+0000 7f0afffff640 1 --2- 172.21.15.64:0/1334945810 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0aec0b6710 0x7f0aec0b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.739+0000 7f0af6ffd640 1 --2- 172.21.15.64:0/827979673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dc34d0 0x557a2118a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.739+0000 7f0afd7fa640 1 -- 172.21.15.64:0/1334945810 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f0af80e7a00 con 0x7f0b00108c60 2024-09-21T04:04:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.739+0000 7f0aff7fe640 1 --2- 172.21.15.64:0/827979673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a20dbb2f0 0x557a211899b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0af0009c00 tx=0x7f0af0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5157b777663e8914 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.740+0000 7f0af5ffb640 1 -- 172.21.15.64:0/827979673 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af0004230 con 0x557a20dbb2f0 2024-09-21T04:04:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.740+0000 7f0af5ffb640 1 -- 172.21.15.64:0/827979673 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0af00043d0 con 0x557a20dbb2f0 2024-09-21T04:04:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.740+0000 7f0af5ffb640 1 -- 172.21.15.64:0/827979673 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af0004b90 con 0x557a20dbb2f0 2024-09-21T04:04:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.740+0000 7f0b0cab9740 1 -- 172.21.15.64:0/827979673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a20dbb2f0 msgr2=0x557a211899b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.740+0000 7f0afffff640 1 --2- 172.21.15.64:0/1334945810 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0aec0b6710 0x7f0aec0b8c20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0af80e50c0 tx=0x7f0af80761d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.740+0000 7f0b0cab9740 1 --2- 172.21.15.64:0/827979673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a20dbb2f0 0x557a211899b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0af0009c00 tx=0x7f0af0011810 comp rx=0 tx=0).stop 2024-09-21T04:04:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.740+0000 7f0b0cab9740 1 -- 172.21.15.64:0/827979673 shutdown_connections 2024-09-21T04:04:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.740+0000 7f0b0cab9740 1 --2- 172.21.15.64:0/827979673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dc34d0 0x557a2118a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.740+0000 7f0b0cab9740 1 --2- 172.21.15.64:0/827979673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a20dbb2f0 0x557a211899b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.740+0000 7f0b0cab9740 1 --2- 172.21.15.64:0/827979673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a20de41b0 0x557a2118c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.740+0000 7f0b0cab9740 1 -- 172.21.15.64:0/827979673 >> 172.21.15.64:0/827979673 conn(0x557a20dd8300 msgr2=0x557a211878a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.741+0000 7f0b0cab9740 1 -- 172.21.15.64:0/827979673 shutdown_connections 2024-09-21T04:04:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.741+0000 7f0b0cab9740 1 -- 172.21.15.64:0/827979673 wait complete. 2024-09-21T04:04:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.741+0000 7f0b0cab9740 1 -- start start 2024-09-21T04:04:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.741+0000 7f0b0cab9740 1 Processor -- start 2024-09-21T04:04:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.742+0000 7f0b0cab9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dbb2f0 0x557a211995b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.742+0000 7f0b0cab9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a20dc34d0 0x557a21199af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.742+0000 7f0b0cab9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a20de41b0 0x557a2119a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.742+0000 7f0b0cab9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557a21025ab0 con 0x557a20dbb2f0 2024-09-21T04:04:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.742+0000 7f0b0cab9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557a21093730 con 0x557a20de41b0 2024-09-21T04:04:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.742+0000 7f0b0cab9740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557a2119c570 con 0x557a20dc34d0 2024-09-21T04:04:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.742+0000 7f0b0cab9740 10 client.0 _ll_register_callbacks cb 0x557a210e76b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.742+0000 7f0af6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dbb2f0 0x557a211995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.742+0000 7f0af6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dbb2f0 0x557a211995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53014/0 (socket says 172.21.15.64:53014) 2024-09-21T04:04:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.742+0000 7f0af6ffd640 1 -- 172.21.15.64:0/196853717 learned_addr learned my addr 172.21.15.64:0/196853717 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.742+0000 7f0aff7fe640 1 --2- 172.21.15.64:0/196853717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a20dc34d0 0x557a21199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.742+0000 7f0af6ffd640 1 --2- 172.21.15.64:0/196853717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a20de41b0 0x557a2119a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.742+0000 7f0af6ffd640 1 -- 172.21.15.64:0/196853717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a20dc34d0 msgr2=0x557a21199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.742+0000 7f0af6ffd640 1 --2- 172.21.15.64:0/196853717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a20dc34d0 0x557a21199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.742+0000 7f0af6ffd640 1 -- 172.21.15.64:0/196853717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a20de41b0 msgr2=0x557a2119a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.742+0000 7f0af6ffd640 1 --2- 172.21.15.64:0/196853717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a20de41b0 0x557a2119a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.742+0000 7f0af6ffd640 1 -- 172.21.15.64:0/196853717 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a20e4f790 con 0x557a20dbb2f0 2024-09-21T04:04:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.742+0000 7f0af6ffd640 1 --2- 172.21.15.64:0/196853717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dbb2f0 0x557a211995b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8032fd0 tx=0x7f0ae802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f96459e073dbf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.742+0000 7f0adeffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:04:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.743+0000 7f0adeffd640 1 -- 172.21.15.64:0/196853717 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8035040 con 0x557a20dbb2f0 2024-09-21T04:04:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.743+0000 7f0adeffd640 1 -- 172.21.15.64:0/196853717 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f0ae8004450 con 0x557a20dbb2f0 2024-09-21T04:04:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.743+0000 7f0b0cab9740 1 -- 172.21.15.64:0/196853717 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0af00043d0 con 0x557a20dbb2f0 2024-09-21T04:04:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.743+0000 7f0adeffd640 1 -- 172.21.15.64:0/196853717 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0ae8031b10 con 0x557a20dbb2f0 2024-09-21T04:04:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.743+0000 7f0b0cab9740 10 client.15504 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.743+0000 7f0b0cab9740 1 -- 172.21.15.64:0/196853717 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557a20ec8620 con 0x557a20dbb2f0 2024-09-21T04:04:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.743+0000 7f0adeffd640 1 -- 172.21.15.64:0/196853717 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0af00043d0 con 0x557a20dbb2f0 2024-09-21T04:04:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.743+0000 7f0adeffd640 1 -- 172.21.15.64:0/196853717 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0ae803a040 con 0x557a20dbb2f0 2024-09-21T04:04:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.743+0000 7f0b0cab9740 10 client.15504 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.743+0000 7f0b0cab9740 10 client.15504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:04:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.743+0000 7f0b0cab9740 20 client.15504 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.743+0000 7f0b0cab9740 1 --2- 172.21.15.64:0/196853717 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x557a20da2d80 0x557a211ab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.743+0000 7f0b0cab9740 4 client.15504 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:04:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.743+0000 7f0b0cab9740 1 -- 172.21.15.64:0/196853717 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a21100640 con 0x557a20da2d80 2024-09-21T04:04:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.744+0000 7f0aff7fe640 1 --2- 172.21.15.64:0/196853717 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x557a20da2d80 0x557a211ab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.744+0000 7f0aff7fe640 1 --2- 172.21.15.64:0/196853717 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x557a20da2d80 0x557a211ab4f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=669ddd3a71d893ea server_cookie=3316b943655fcfb7 in_seq=0 out_seq=0 2024-09-21T04:04:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.744+0000 7f0adeffd640 10 client.15504 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.745+0000 7f0adeffd640 1 -- 172.21.15.64:0/196853717 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557a21100640 con 0x557a20da2d80 2024-09-21T04:04:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.745+0000 7f0adeffd640 10 client.15504 handle_command_reply: tid=0 2024-09-21T04:04:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.745+0000 7f0b0cab9740 1 client.15504 shutdown 2024-09-21T04:04:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.745+0000 7f0b0cab9740 1 -- 172.21.15.64:0/196853717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dbb2f0 msgr2=0x557a211995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.745+0000 7f0b0cab9740 1 --2- 172.21.15.64:0/196853717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dbb2f0 0x557a211995b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8032fd0 tx=0x7f0ae802a410 comp rx=0 tx=0).stop 2024-09-21T04:04:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.745+0000 7f0b0cab9740 1 -- 172.21.15.64:0/196853717 shutdown_connections 2024-09-21T04:04:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.745+0000 7f0b0cab9740 1 --2- 172.21.15.64:0/196853717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a20de41b0 0x557a2119a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.745+0000 7f0b0cab9740 1 --2- 172.21.15.64:0/196853717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a20dc34d0 0x557a21199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.745+0000 7f0b0cab9740 1 --2- 172.21.15.64:0/196853717 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x557a20da2d80 0x557a211ab4f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.745+0000 7f0b0cab9740 1 --2- 172.21.15.64:0/196853717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dbb2f0 0x557a211995b0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.745+0000 7f0adeffd640 0 client.15504 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.745+0000 7f0b0cab9740 1 -- 172.21.15.64:0/196853717 >> 172.21.15.64:0/196853717 conn(0x557a20dd8300 msgr2=0x557a2118b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.745+0000 7f0b0cab9740 1 -- 172.21.15.64:0/196853717 shutdown_connections 2024-09-21T04:04:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.745+0000 7f0b0cab9740 1 -- 172.21.15.64:0/196853717 wait complete. 2024-09-21T04:04:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.745+0000 7f0b0cab9740 20 client.15504 trim_cache size 0 max 16384 2024-09-21T04:04:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.760+0000 7f0b0cab9740 1 Processor -- start 2024-09-21T04:04:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.760+0000 7f0b0cab9740 1 -- start start 2024-09-21T04:04:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.761+0000 7f0b0cab9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dbb2f0 0x557a21199a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.761+0000 7f0b0cab9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a20dc34d0 0x557a21186eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.761+0000 7f0b0cab9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a20de41b0 0x557a211873f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.761+0000 7f0b0cab9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0b001afe40 con 0x557a20dbb2f0 2024-09-21T04:04:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.761+0000 7f0b0cab9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0b001ad710 con 0x557a20de41b0 2024-09-21T04:04:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.761+0000 7f0b0cab9740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0b001ab560 con 0x557a20dc34d0 2024-09-21T04:04:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.761+0000 7f0af6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dbb2f0 0x557a21199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.761+0000 7f0af6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dbb2f0 0x557a21199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53030/0 (socket says 172.21.15.64:53030) 2024-09-21T04:04:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.761+0000 7f0af6ffd640 1 -- 172.21.15.64:0/984893975 learned_addr learned my addr 172.21.15.64:0/984893975 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.761+0000 7f0aff7fe640 1 --2- 172.21.15.64:0/984893975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a20dc34d0 0x557a21186eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.761+0000 7f0af6ffd640 1 -- 172.21.15.64:0/984893975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a20dc34d0 msgr2=0x557a21186eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:13.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.761+0000 7f0af6ffd640 1 --2- 172.21.15.64:0/984893975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a20dc34d0 0x557a21186eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.761+0000 7f0af6ffd640 1 -- 172.21.15.64:0/984893975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a20de41b0 msgr2=0x557a211873f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.761+0000 7f0af6ffd640 1 --2- 172.21.15.64:0/984893975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a20de41b0 0x557a211873f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.761+0000 7f0af6ffd640 1 -- 172.21.15.64:0/984893975 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a211a5760 con 0x557a20dbb2f0 2024-09-21T04:04:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.761+0000 7f0af6ffd640 1 --2- 172.21.15.64:0/984893975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dbb2f0 0x557a21199a90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0ae802a930 tx=0x7f0ae8031880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44f395f0945eec9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.761+0000 7f0af57fa640 1 -- 172.21.15.64:0/984893975 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8056030 con 0x557a20dbb2f0 2024-09-21T04:04:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.761+0000 7f0af57fa640 1 -- 172.21.15.64:0/984893975 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ae8003aa0 con 0x557a20dbb2f0 2024-09-21T04:04:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.761+0000 7f0af57fa640 1 -- 172.21.15.64:0/984893975 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8035040 con 0x557a20dbb2f0 2024-09-21T04:04:13.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.762+0000 7f0b0cab9740 1 -- 172.21.15.64:0/984893975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dbb2f0 msgr2=0x557a21199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:13.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.762+0000 7f0b0cab9740 1 --2- 172.21.15.64:0/984893975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dbb2f0 0x557a21199a90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0ae802a930 tx=0x7f0ae8031880 comp rx=0 tx=0).stop 2024-09-21T04:04:13.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.762+0000 7f0b0cab9740 1 -- 172.21.15.64:0/984893975 shutdown_connections 2024-09-21T04:04:13.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.762+0000 7f0b0cab9740 1 --2- 172.21.15.64:0/984893975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a20de41b0 0x557a211873f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.762+0000 7f0b0cab9740 1 --2- 172.21.15.64:0/984893975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a20dc34d0 0x557a21186eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.762+0000 7f0b0cab9740 1 --2- 172.21.15.64:0/984893975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dbb2f0 0x557a21199a90 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.762+0000 7f0b0cab9740 1 -- 172.21.15.64:0/984893975 >> 172.21.15.64:0/984893975 conn(0x557a20dd8300 msgr2=0x557a21197e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.762+0000 7f0b0cab9740 1 -- 172.21.15.64:0/984893975 shutdown_connections 2024-09-21T04:04:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.762+0000 7f0b0cab9740 1 -- 172.21.15.64:0/984893975 wait complete. 2024-09-21T04:04:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.763+0000 7f0b0cab9740 1 -- start start 2024-09-21T04:04:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.763+0000 7f0b0cab9740 1 Processor -- start 2024-09-21T04:04:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.763+0000 7f0b0cab9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a20dbb2f0 0x557a211a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.763+0000 7f0b0cab9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dc34d0 0x557a211a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.763+0000 7f0b0cab9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a20de41b0 0x557a2119e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.763+0000 7f0b0cab9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0b001abbd0 con 0x557a20dc34d0 2024-09-21T04:04:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.763+0000 7f0b0cab9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0b001ad040 con 0x557a20dbb2f0 2024-09-21T04:04:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.763+0000 7f0b0cab9740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0b001ac2a0 con 0x557a20de41b0 2024-09-21T04:04:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.763+0000 7f0b0cab9740 10 client.0 _ll_register_callbacks cb 0x7f0ae8031530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.763+0000 7f0aff7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dc34d0 0x557a211a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.763+0000 7f0aff7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dc34d0 0x557a211a1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53034/0 (socket says 172.21.15.64:53034) 2024-09-21T04:04:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.763+0000 7f0aff7fe640 1 -- 172.21.15.64:0/1566261809 learned_addr learned my addr 172.21.15.64:0/1566261809 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.763+0000 7f0af6ffd640 1 --2- 172.21.15.64:0/1566261809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a20de41b0 0x557a2119e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.764+0000 7f0af6ffd640 1 --2- 172.21.15.64:0/1566261809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a20dbb2f0 0x557a211a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.764+0000 7f0aff7fe640 1 -- 172.21.15.64:0/1566261809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a20de41b0 msgr2=0x557a2119e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.764+0000 7f0aff7fe640 1 --2- 172.21.15.64:0/1566261809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a20de41b0 0x557a2119e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.764+0000 7f0aff7fe640 1 -- 172.21.15.64:0/1566261809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a20dbb2f0 msgr2=0x557a211a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.764+0000 7f0aff7fe640 1 --2- 172.21.15.64:0/1566261809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a20dbb2f0 0x557a211a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.764+0000 7f0aff7fe640 1 -- 172.21.15.64:0/1566261809 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a2119ee80 con 0x557a20dc34d0 2024-09-21T04:04:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.764+0000 7f0aff7fe640 1 --2- 172.21.15.64:0/1566261809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dc34d0 0x557a211a1f50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0af0014040 tx=0x7f0af00092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a356959488e602 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.764+0000 7f0adeffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:04:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.764+0000 7f0adeffd640 1 -- 172.21.15.64:0/1566261809 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af001e040 con 0x557a20dc34d0 2024-09-21T04:04:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.764+0000 7f0adeffd640 1 -- 172.21.15.64:0/1566261809 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f0af00196f0 con 0x557a20dc34d0 2024-09-21T04:04:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.764+0000 7f0b0cab9740 1 -- 172.21.15.64:0/1566261809 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ae8003aa0 con 0x557a20dc34d0 2024-09-21T04:04:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.764+0000 7f0adeffd640 1 -- 172.21.15.64:0/1566261809 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0af003b050 con 0x557a20dc34d0 2024-09-21T04:04:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.764+0000 7f0b0cab9740 10 client.15510 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.764+0000 7f0b0cab9740 1 -- 172.21.15.64:0/1566261809 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557a21216810 con 0x557a20dc34d0 2024-09-21T04:04:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.765+0000 7f0adeffd640 1 -- 172.21.15.64:0/1566261809 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ae8003aa0 con 0x557a20dc34d0 2024-09-21T04:04:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.765+0000 7f0adeffd640 1 -- 172.21.15.64:0/1566261809 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0af0024040 con 0x557a20dc34d0 2024-09-21T04:04:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.765+0000 7f0b0cab9740 10 client.15510 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.765+0000 7f0b0cab9740 10 client.15510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:04:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.765+0000 7f0b0cab9740 20 client.15510 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.765+0000 7f0b0cab9740 1 --2- 172.21.15.64:0/1566261809 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x557a20d781c0 0x557a21218de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.765+0000 7f0b0cab9740 4 client.15510 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:04:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.765+0000 7f0b0cab9740 1 -- 172.21.15.64:0/1566261809 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a210fc0f0 con 0x557a20d781c0 2024-09-21T04:04:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.765+0000 7f0af6ffd640 1 --2- 172.21.15.64:0/1566261809 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x557a20d781c0 0x557a21218de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:13.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.765+0000 7f0adeffd640 10 client.15510 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.765+0000 7f0af6ffd640 1 --2- 172.21.15.64:0/1566261809 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x557a20d781c0 0x557a21218de0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21969222424e4994 server_cookie=d94eeb773351a670 in_seq=0 out_seq=0 2024-09-21T04:04:13.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.777+0000 7f0adeffd640 1 -- 172.21.15.64:0/1566261809 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557a210fc0f0 con 0x557a20d781c0 2024-09-21T04:04:13.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.777+0000 7f0adeffd640 10 client.15510 handle_command_reply: tid=0 2024-09-21T04:04:13.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.777+0000 7f0b0cab9740 1 client.15510 shutdown 2024-09-21T04:04:13.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.777+0000 7f0b0cab9740 1 -- 172.21.15.64:0/1566261809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dc34d0 msgr2=0x557a211a1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:13.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.777+0000 7f0b0cab9740 1 --2- 172.21.15.64:0/1566261809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dc34d0 0x557a211a1f50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0af0014040 tx=0x7f0af00092a0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.778+0000 7f0b0cab9740 1 -- 172.21.15.64:0/1566261809 shutdown_connections 2024-09-21T04:04:13.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.778+0000 7f0b0cab9740 1 --2- 172.21.15.64:0/1566261809 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a20de41b0 0x557a2119e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.778+0000 7f0b0cab9740 1 --2- 172.21.15.64:0/1566261809 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x557a20d781c0 0x557a21218de0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.778+0000 7f0b0cab9740 1 --2- 172.21.15.64:0/1566261809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557a20dc34d0 0x557a211a1f50 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.778+0000 7f0b0cab9740 1 --2- 172.21.15.64:0/1566261809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557a20dbb2f0 0x557a211a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.778+0000 7f0adeffd640 0 client.15510 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.778+0000 7f0b0cab9740 1 -- 172.21.15.64:0/1566261809 >> 172.21.15.64:0/1566261809 conn(0x557a20dd8300 msgr2=0x557a21197dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.778+0000 7f0b0cab9740 1 -- 172.21.15.64:0/1566261809 shutdown_connections 2024-09-21T04:04:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.778+0000 7f0b0cab9740 1 -- 172.21.15.64:0/1566261809 wait complete. 2024-09-21T04:04:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.778+0000 7f0b0cab9740 20 client.15510 trim_cache size 0 max 16384 2024-09-21T04:04:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:04:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:04:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:04:13.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:04:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.778+0000 7f0b05b51640 1 -- 172.21.15.64:0/1334945810 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0aec0b6710 msgr2=0x7f0aec0b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.778+0000 7f0b05b51640 1 --2- 172.21.15.64:0/1334945810 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0aec0b6710 0x7f0aec0b8c20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0af80e50c0 tx=0x7f0af80761d0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.778+0000 7f0b05b51640 1 -- 172.21.15.64:0/1334945810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b00108c60 msgr2=0x7f0b0019ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:13.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.778+0000 7f0b05b51640 1 --2- 172.21.15.64:0/1334945810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b00108c60 0x7f0b0019ebd0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0af8009fd0 tx=0x7f0af800b260 comp rx=0 tx=0).stop 2024-09-21T04:04:13.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.778+0000 7f0b05b51640 1 -- 172.21.15.64:0/1334945810 shutdown_connections 2024-09-21T04:04:13.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.778+0000 7f0b05b51640 1 --2- 172.21.15.64:0/1334945810 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0aec0b6710 0x7f0aec0b8c20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.778+0000 7f0b05b51640 1 --2- 172.21.15.64:0/1334945810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b00108c60 0x7f0b0019ebd0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.778+0000 7f0b05b51640 1 --2- 172.21.15.64:0/1334945810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b00108310 0x7f0b00199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.778+0000 7f0b05b51640 1 --2- 172.21.15.64:0/1334945810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0b001029e0 0x7f0b001993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:13.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.778+0000 7f0b05b51640 1 -- 172.21.15.64:0/1334945810 >> 172.21.15.64:0/1334945810 conn(0x7f0b000fe3e0 msgr2=0x7f0b00105980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:13.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.779+0000 7f0b05b51640 1 -- 172.21.15.64:0/1334945810 shutdown_connections 2024-09-21T04:04:13.783 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:13.779+0000 7f0b05b51640 1 -- 172.21.15.64:0/1334945810 wait complete. 2024-09-21T04:04:14.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:13 smithi119 ceph-mon[34811]: pgmap v327: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 198 KiB/s wr, 70 op/s 2024-09-21T04:04:14.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:13 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:04:14.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:13 smithi064 ceph-mon[31180]: pgmap v327: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 198 KiB/s wr, 70 op/s 2024-09-21T04:04:14.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:13 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:04:14.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:13 smithi131 ceph-mon[35193]: pgmap v327: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 198 KiB/s wr, 70 op/s 2024-09-21T04:04:14.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:13 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:04:15.855 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:15 smithi131 ceph-mon[35193]: pgmap v328: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-21T04:04:16.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:15 smithi119 ceph-mon[34811]: pgmap v328: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-21T04:04:16.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:15 smithi064 ceph-mon[31180]: pgmap v328: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-21T04:04:18.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:17 smithi119 ceph-mon[34811]: pgmap v329: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-21T04:04:18.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:17 smithi064 ceph-mon[31180]: pgmap v329: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-21T04:04:18.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:17 smithi131 ceph-mon[35193]: pgmap v329: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-21T04:04:20.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:19 smithi119 ceph-mon[34811]: pgmap v330: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-09-21T04:04:20.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:19 smithi064 ceph-mon[31180]: pgmap v330: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-09-21T04:04:20.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:19 smithi131 ceph-mon[35193]: pgmap v330: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-09-21T04:04:22.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:21 smithi119 ceph-mon[34811]: pgmap v331: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-21T04:04:22.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:21 smithi064 ceph-mon[31180]: pgmap v331: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-21T04:04:22.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:21 smithi131 ceph-mon[35193]: pgmap v331: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-21T04:04:24.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:23 smithi119 ceph-mon[34811]: pgmap v332: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-21T04:04:24.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:23 smithi064 ceph-mon[31180]: pgmap v332: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-21T04:04:24.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:23 smithi131 ceph-mon[35193]: pgmap v332: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-21T04:04:26.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:25 smithi119 ceph-mon[34811]: pgmap v333: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-09-21T04:04:26.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:25 smithi064 ceph-mon[31180]: pgmap v333: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-09-21T04:04:26.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:25 smithi131 ceph-mon[35193]: pgmap v333: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-09-21T04:04:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.882+0000 7fa8003b0640 1 -- 172.21.15.64:0/3737191253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7f8104900 msgr2=0x7fa7f8104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.882+0000 7fa8003b0640 1 --2- 172.21.15.64:0/3737191253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7f8104900 0x7fa7f8104d10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4009900 tx=0x7fa7f402a410 comp rx=0 tx=0).stop 2024-09-21T04:04:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.882+0000 7fa8003b0640 1 -- 172.21.15.64:0/3737191253 shutdown_connections 2024-09-21T04:04:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.882+0000 7fa8003b0640 1 --2- 172.21.15.64:0/3737191253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7f810dd60 0x7fa7f8110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.882+0000 7fa8003b0640 1 --2- 172.21.15.64:0/3737191253 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7f8105250 0x7fa7f810d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.882+0000 7fa8003b0640 1 --2- 172.21.15.64:0/3737191253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7f8104900 0x7fa7f8104d10 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.882+0000 7fa8003b0640 1 -- 172.21.15.64:0/3737191253 >> 172.21.15.64:0/3737191253 conn(0x7fa7f807a740 msgr2=0x7fa7f807ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.882+0000 7fa8003b0640 1 -- 172.21.15.64:0/3737191253 shutdown_connections 2024-09-21T04:04:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.882+0000 7fa8003b0640 1 -- 172.21.15.64:0/3737191253 wait complete. 2024-09-21T04:04:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.883+0000 7fa8003b0640 1 Processor -- start 2024-09-21T04:04:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.883+0000 7fa8003b0640 1 -- start start 2024-09-21T04:04:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.883+0000 7fa8003b0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7f8104900 0x7fa7f819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.883+0000 7fa8003b0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7f8105250 0x7fa7f819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.883+0000 7fa8003b0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7f810dd60 0x7fa7f81a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.883+0000 7fa8003b0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f8113a30 con 0x7fa7f8105250 2024-09-21T04:04:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.883+0000 7fa8003b0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f81138b0 con 0x7fa7f8104900 2024-09-21T04:04:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.883+0000 7fa8003b0640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f8113bb0 con 0x7fa7f810dd60 2024-09-21T04:04:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.883+0000 7fa7fd94b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7f8105250 0x7fa7f819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.883+0000 7fa7fd94b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7f8105250 0x7fa7f819cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43340/0 (socket says 172.21.15.64:43340) 2024-09-21T04:04:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.883+0000 7fa7fd94b640 1 -- 172.21.15.64:0/3610550427 learned_addr learned my addr 172.21.15.64:0/3610550427 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.884+0000 7fa7fe94d640 1 --2- 172.21.15.64:0/3610550427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7f810dd60 0x7fa7f81a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.884+0000 7fa7fe14c640 1 --2- 172.21.15.64:0/3610550427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7f8104900 0x7fa7f819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.884+0000 7fa7fd94b640 1 -- 172.21.15.64:0/3610550427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7f810dd60 msgr2=0x7fa7f81a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.884+0000 7fa7fd94b640 1 --2- 172.21.15.64:0/3610550427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7f810dd60 0x7fa7f81a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.884+0000 7fa7fd94b640 1 -- 172.21.15.64:0/3610550427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7f8104900 msgr2=0x7fa7f819c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.884+0000 7fa7fd94b640 1 --2- 172.21.15.64:0/3610550427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7f8104900 0x7fa7f819c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.884+0000 7fa7fd94b640 1 -- 172.21.15.64:0/3610550427 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f81a2560 con 0x7fa7f8105250 2024-09-21T04:04:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.884+0000 7fa7fe14c640 1 --2- 172.21.15.64:0/3610550427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7f8104900 0x7fa7f819c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:04:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.884+0000 7fa7fd94b640 1 --2- 172.21.15.64:0/3610550427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7f8105250 0x7fa7f819cbd0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00d750 tx=0x7fa7ec00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.884+0000 7fa7eb7fe640 1 -- 172.21.15.64:0/3610550427 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec017070 con 0x7fa7f8105250 2024-09-21T04:04:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.884+0000 7fa7eb7fe640 1 -- 172.21.15.64:0/3610550427 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7ec0054d0 con 0x7fa7f8105250 2024-09-21T04:04:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.884+0000 7fa8003b0640 1 -- 172.21.15.64:0/3610550427 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7f81a2850 con 0x7fa7f8105250 2024-09-21T04:04:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.884+0000 7fa7eb7fe640 1 -- 172.21.15.64:0/3610550427 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec004c30 con 0x7fa7f8105250 2024-09-21T04:04:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.884+0000 7fa8003b0640 1 -- 172.21.15.64:0/3610550427 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7f81a8fe0 con 0x7fa7f8105250 2024-09-21T04:04:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.887+0000 7fa7eb7fe640 1 -- 172.21.15.64:0/3610550427 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 154694+0+0 (secure 0 0 0) 0x7fa7ec010460 con 0x7fa7f8105250 2024-09-21T04:04:26.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.887+0000 7fa8073f4740 1 Processor -- start 2024-09-21T04:04:26.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.888+0000 7fa8073f4740 1 -- start start 2024-09-21T04:04:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.888+0000 7fa8073f4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb693e1b0 0x55bcb6ce39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.888+0000 7fa8073f4740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcb69152f0 0x55bcb6ce3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.888+0000 7fa8073f4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bcb691d4d0 0x55bcb6ce4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.888+0000 7fa8073f4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bcb6c586f0 con 0x55bcb693e1b0 2024-09-21T04:04:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.888+0000 7fa8073f4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bcb6c47b20 con 0x55bcb691d4d0 2024-09-21T04:04:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.888+0000 7fa8073f4740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bcb6c4b930 con 0x55bcb69152f0 2024-09-21T04:04:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.888+0000 7fa7fe94d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb693e1b0 0x55bcb6ce39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.888+0000 7fa7fe94d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb693e1b0 0x55bcb6ce39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43348/0 (socket says 172.21.15.64:43348) 2024-09-21T04:04:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.888+0000 7fa7fe94d640 1 -- 172.21.15.64:0/2508489451 learned_addr learned my addr 172.21.15.64:0/2508489451 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.888+0000 7fa7fd94b640 1 --2- 172.21.15.64:0/2508489451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bcb691d4d0 0x55bcb6ce4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.888+0000 7fa7fe14c640 1 --2- 172.21.15.64:0/2508489451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcb69152f0 0x55bcb6ce3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.888+0000 7fa7eb7fe640 1 --2- 172.21.15.64:0/3610550427 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa7e40b6790 0x7fa7e40b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:26.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.889+0000 7fa7fe94d640 1 -- 172.21.15.64:0/2508489451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcb69152f0 msgr2=0x55bcb6ce3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:26.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.889+0000 7fa7fe94d640 1 --2- 172.21.15.64:0/2508489451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcb69152f0 0x55bcb6ce3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.889+0000 7fa7fe94d640 1 -- 172.21.15.64:0/2508489451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bcb691d4d0 msgr2=0x55bcb6ce4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:26.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.889+0000 7fa7fe94d640 1 --2- 172.21.15.64:0/2508489451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bcb691d4d0 0x55bcb6ce4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.889+0000 7fa7fe94d640 1 -- 172.21.15.64:0/2508489451 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcb6c0d530 con 0x55bcb693e1b0 2024-09-21T04:04:26.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.889+0000 7fa7fe14c640 1 --2- 172.21.15.64:0/2508489451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcb69152f0 0x55bcb6ce3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:26.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.889+0000 7fa7fd94b640 1 --2- 172.21.15.64:0/2508489451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bcb691d4d0 0x55bcb6ce4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:26.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.889+0000 7fa7fe94d640 1 --2- 172.21.15.64:0/3610550427 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa7e40b6790 0x7fa7e40b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:26.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.889+0000 7fa7eb7fe640 1 -- 172.21.15.64:0/3610550427 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fa7ec0e73a0 con 0x7fa7f8105250 2024-09-21T04:04:26.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.889+0000 7fa7fe94d640 1 --2- 172.21.15.64:0/2508489451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb693e1b0 0x55bcb6ce39b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000f480 tx=0x7fa7f000f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36be3a54aad71706 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:26.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.889+0000 7fa7e8bfe640 1 -- 172.21.15.64:0/2508489451 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f001c400 con 0x55bcb693e1b0 2024-09-21T04:04:26.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.889+0000 7fa7fe94d640 1 --2- 172.21.15.64:0/3610550427 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa7e40b6790 0x7fa7e40b8ca0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0020000 tx=0x7fa7f001d000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:26.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.889+0000 7fa7e8bfe640 1 -- 172.21.15.64:0/2508489451 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7f001ca50 con 0x55bcb693e1b0 2024-09-21T04:04:26.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.889+0000 7fa7e8bfe640 1 -- 172.21.15.64:0/2508489451 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0010850 con 0x55bcb693e1b0 2024-09-21T04:04:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.890+0000 7fa8073f4740 1 -- 172.21.15.64:0/2508489451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb693e1b0 msgr2=0x55bcb6ce39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.890+0000 7fa8073f4740 1 --2- 172.21.15.64:0/2508489451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb693e1b0 0x55bcb6ce39b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000f480 tx=0x7fa7f000f940 comp rx=0 tx=0).stop 2024-09-21T04:04:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.890+0000 7fa8073f4740 1 -- 172.21.15.64:0/2508489451 shutdown_connections 2024-09-21T04:04:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.890+0000 7fa8073f4740 1 --2- 172.21.15.64:0/2508489451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bcb691d4d0 0x55bcb6ce4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.890+0000 7fa8073f4740 1 --2- 172.21.15.64:0/2508489451 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcb69152f0 0x55bcb6ce3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.890+0000 7fa8073f4740 1 --2- 172.21.15.64:0/2508489451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb693e1b0 0x55bcb6ce39b0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.890+0000 7fa8073f4740 1 -- 172.21.15.64:0/2508489451 >> 172.21.15.64:0/2508489451 conn(0x55bcb6932300 msgr2=0x55bcb6ce18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.890+0000 7fa8073f4740 1 -- 172.21.15.64:0/2508489451 shutdown_connections 2024-09-21T04:04:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.890+0000 7fa8073f4740 1 -- 172.21.15.64:0/2508489451 wait complete. 2024-09-21T04:04:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.891+0000 7fa8073f4740 1 -- start start 2024-09-21T04:04:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.891+0000 7fa8073f4740 1 Processor -- start 2024-09-21T04:04:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.891+0000 7fa8073f4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bcb69152f0 0x55bcb6cef0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.891+0000 7fa8073f4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb691d4d0 0x55bcb6cef5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.891+0000 7fa8073f4740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcb693e1b0 0x55bcb6cfb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.891+0000 7fa8073f4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bcb6b7fab0 con 0x55bcb691d4d0 2024-09-21T04:04:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.891+0000 7fa8073f4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bcb6bed730 con 0x55bcb69152f0 2024-09-21T04:04:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.891+0000 7fa8073f4740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bcb6cf2130 con 0x55bcb693e1b0 2024-09-21T04:04:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.891+0000 7fa8073f4740 10 client.0 _ll_register_callbacks cb 0x55bcb6c416b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.891+0000 7fa7fd94b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb691d4d0 0x55bcb6cef5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.891+0000 7fa7fd94b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb691d4d0 0x55bcb6cef5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43352/0 (socket says 172.21.15.64:43352) 2024-09-21T04:04:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.891+0000 7fa7fd94b640 1 -- 172.21.15.64:0/1194988248 learned_addr learned my addr 172.21.15.64:0/1194988248 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.892+0000 7fa7fe14c640 1 --2- 172.21.15.64:0/1194988248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bcb69152f0 0x55bcb6cef0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.892+0000 7fa7fe94d640 1 --2- 172.21.15.64:0/1194988248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcb693e1b0 0x55bcb6cfb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.892+0000 7fa7fd94b640 1 -- 172.21.15.64:0/1194988248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcb693e1b0 msgr2=0x55bcb6cfb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.892+0000 7fa7fd94b640 1 --2- 172.21.15.64:0/1194988248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcb693e1b0 0x55bcb6cfb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.892+0000 7fa7fd94b640 1 -- 172.21.15.64:0/1194988248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bcb69152f0 msgr2=0x55bcb6cef0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.892+0000 7fa7fd94b640 1 --2- 172.21.15.64:0/1194988248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bcb69152f0 0x55bcb6cef0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.892+0000 7fa7fd94b640 1 -- 172.21.15.64:0/1194988248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcb69a9790 con 0x55bcb691d4d0 2024-09-21T04:04:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.892+0000 7fa7fe94d640 1 --2- 172.21.15.64:0/1194988248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcb693e1b0 0x55bcb6cfb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:04:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.892+0000 7fa7fe14c640 1 --2- 172.21.15.64:0/1194988248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bcb69152f0 0x55bcb6cef0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:04:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.892+0000 7fa7fd94b640 1 --2- 172.21.15.64:0/1194988248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb691d4d0 0x55bcb6cef5e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0e4a90 tx=0x7fa7ec0e76a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2062d5b1f0277b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.892+0000 7fa7e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:04:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.892+0000 7fa7e1ffb640 1 -- 172.21.15.64:0/1194988248 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec020050 con 0x55bcb691d4d0 2024-09-21T04:04:26.899 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.892+0000 7fa8073f4740 1 -- 172.21.15.64:0/1194988248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7f001ca50 con 0x55bcb691d4d0 2024-09-21T04:04:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.892+0000 7fa7e1ffb640 1 -- 172.21.15.64:0/1194988248 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fa7ec0a89a0 con 0x55bcb691d4d0 2024-09-21T04:04:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.892+0000 7fa7e1ffb640 1 -- 172.21.15.64:0/1194988248 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa7ec0ac9a0 con 0x55bcb691d4d0 2024-09-21T04:04:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.892+0000 7fa8073f4740 10 client.15522 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.892+0000 7fa8073f4740 1 -- 172.21.15.64:0/1194988248 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bcb6a22620 con 0x55bcb691d4d0 2024-09-21T04:04:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.893+0000 7fa7e1ffb640 1 -- 172.21.15.64:0/1194988248 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7f001ca50 con 0x55bcb691d4d0 2024-09-21T04:04:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.893+0000 7fa7e1ffb640 1 -- 172.21.15.64:0/1194988248 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa7ec0a8ba0 con 0x55bcb691d4d0 2024-09-21T04:04:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.893+0000 7fa8073f4740 10 client.15522 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.893+0000 7fa8073f4740 10 client.15522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:04:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.893+0000 7fa8073f4740 20 client.15522 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.893+0000 7fa8073f4740 1 --2- 172.21.15.64:0/1194988248 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55bcb68fcd80 0x55bcb6d011d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.893+0000 7fa8073f4740 4 client.15522 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:04:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.893+0000 7fa8073f4740 1 -- 172.21.15.64:0/1194988248 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcb6c5a640 con 0x55bcb68fcd80 2024-09-21T04:04:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.893+0000 7fa7fe14c640 1 --2- 172.21.15.64:0/1194988248 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55bcb68fcd80 0x55bcb6d011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.894+0000 7fa7e1ffb640 10 client.15522 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.894+0000 7fa7fe14c640 1 --2- 172.21.15.64:0/1194988248 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55bcb68fcd80 0x55bcb6d011d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6370ffb9118744b9 server_cookie=dd04698ea54e2389 in_seq=0 out_seq=0 2024-09-21T04:04:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.894+0000 7fa7e1ffb640 1 -- 172.21.15.64:0/1194988248 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bcb6c5a640 con 0x55bcb68fcd80 2024-09-21T04:04:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.894+0000 7fa7e1ffb640 10 client.15522 handle_command_reply: tid=0 2024-09-21T04:04:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.894+0000 7fa8073f4740 1 client.15522 shutdown 2024-09-21T04:04:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.894+0000 7fa8073f4740 1 -- 172.21.15.64:0/1194988248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb691d4d0 msgr2=0x55bcb6cef5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.894+0000 7fa8073f4740 1 --2- 172.21.15.64:0/1194988248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb691d4d0 0x55bcb6cef5e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0e4a90 tx=0x7fa7ec0e76a0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.895+0000 7fa8073f4740 1 -- 172.21.15.64:0/1194988248 shutdown_connections 2024-09-21T04:04:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.895+0000 7fa8073f4740 1 --2- 172.21.15.64:0/1194988248 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcb693e1b0 0x55bcb6cfb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.895+0000 7fa8073f4740 1 --2- 172.21.15.64:0/1194988248 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55bcb68fcd80 0x55bcb6d011d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.895+0000 7fa8073f4740 1 --2- 172.21.15.64:0/1194988248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb691d4d0 0x55bcb6cef5e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.895+0000 7fa8073f4740 1 --2- 172.21.15.64:0/1194988248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bcb69152f0 0x55bcb6cef0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.895+0000 7fa7e1ffb640 0 client.15522 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.895+0000 7fa8073f4740 1 -- 172.21.15.64:0/1194988248 >> 172.21.15.64:0/1194988248 conn(0x55bcb6932300 msgr2=0x55bcb6ce49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.895+0000 7fa8073f4740 1 -- 172.21.15.64:0/1194988248 shutdown_connections 2024-09-21T04:04:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.895+0000 7fa8073f4740 1 -- 172.21.15.64:0/1194988248 wait complete. 2024-09-21T04:04:26.901 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.895+0000 7fa8073f4740 20 client.15522 trim_cache size 0 max 16384 2024-09-21T04:04:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.912+0000 7fa8073f4740 1 Processor -- start 2024-09-21T04:04:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.912+0000 7fa8073f4740 1 -- start start 2024-09-21T04:04:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.912+0000 7fa8073f4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb69152f0 0x55bcb6cef580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.912+0000 7fa8073f4740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcb691d4d0 0x55bcb6ce50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.912+0000 7fa8073f4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bcb693e1b0 0x55bcb6cfb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.912+0000 7fa8073f4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f81b4310 con 0x55bcb69152f0 2024-09-21T04:04:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.912+0000 7fa8073f4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f81b1be0 con 0x55bcb693e1b0 2024-09-21T04:04:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.912+0000 7fa8073f4740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f81afa30 con 0x55bcb691d4d0 2024-09-21T04:04:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.912+0000 7fa7fe14c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb69152f0 0x55bcb6cef580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.912+0000 7fa7fe14c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb69152f0 0x55bcb6cef580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43366/0 (socket says 172.21.15.64:43366) 2024-09-21T04:04:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.912+0000 7fa7fe14c640 1 -- 172.21.15.64:0/1225877653 learned_addr learned my addr 172.21.15.64:0/1225877653 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.913+0000 7fa7fe94d640 1 --2- 172.21.15.64:0/1225877653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bcb693e1b0 0x55bcb6cfb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.913+0000 7fa7fd94b640 1 --2- 172.21.15.64:0/1225877653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcb691d4d0 0x55bcb6ce50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.913+0000 7fa7fe14c640 1 -- 172.21.15.64:0/1225877653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcb691d4d0 msgr2=0x55bcb6ce50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.913+0000 7fa7fe14c640 1 --2- 172.21.15.64:0/1225877653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcb691d4d0 0x55bcb6ce50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.913+0000 7fa7fe14c640 1 -- 172.21.15.64:0/1225877653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bcb693e1b0 msgr2=0x55bcb6cfb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.913+0000 7fa7fe14c640 1 --2- 172.21.15.64:0/1225877653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bcb693e1b0 0x55bcb6cfb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.913+0000 7fa7fe14c640 1 -- 172.21.15.64:0/1225877653 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcb6cf0360 con 0x55bcb69152f0 2024-09-21T04:04:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.913+0000 7fa7fe94d640 1 --2- 172.21.15.64:0/1225877653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bcb693e1b0 0x55bcb6cfb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:04:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.913+0000 7fa7fe14c640 1 --2- 172.21.15.64:0/1225877653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb69152f0 0x55bcb6cef580 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa7f402a9d0 tx=0x7fa7f4009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c5345332ca950 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.913+0000 7fa7e3fff640 1 -- 172.21.15.64:0/1225877653 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4048020 con 0x55bcb69152f0 2024-09-21T04:04:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.913+0000 7fa7e3fff640 1 -- 172.21.15.64:0/1225877653 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7f4005430 con 0x55bcb69152f0 2024-09-21T04:04:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.913+0000 7fa7e3fff640 1 -- 172.21.15.64:0/1225877653 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4035040 con 0x55bcb69152f0 2024-09-21T04:04:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.914+0000 7fa8073f4740 1 -- 172.21.15.64:0/1225877653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb69152f0 msgr2=0x55bcb6cef580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.914+0000 7fa8073f4740 1 --2- 172.21.15.64:0/1225877653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb69152f0 0x55bcb6cef580 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa7f402a9d0 tx=0x7fa7f4009560 comp rx=0 tx=0).stop 2024-09-21T04:04:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.914+0000 7fa8073f4740 1 -- 172.21.15.64:0/1225877653 shutdown_connections 2024-09-21T04:04:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.914+0000 7fa8073f4740 1 --2- 172.21.15.64:0/1225877653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bcb693e1b0 0x55bcb6cfb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.914+0000 7fa8073f4740 1 --2- 172.21.15.64:0/1225877653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcb691d4d0 0x55bcb6ce50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.914+0000 7fa8073f4740 1 --2- 172.21.15.64:0/1225877653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb69152f0 0x55bcb6cef580 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.914+0000 7fa8073f4740 1 -- 172.21.15.64:0/1225877653 >> 172.21.15.64:0/1225877653 conn(0x55bcb6932300 msgr2=0x55bcb6ce49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.914+0000 7fa8073f4740 1 -- 172.21.15.64:0/1225877653 shutdown_connections 2024-09-21T04:04:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.914+0000 7fa8073f4740 1 -- 172.21.15.64:0/1225877653 wait complete. 2024-09-21T04:04:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.915+0000 7fa8073f4740 1 -- start start 2024-09-21T04:04:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.915+0000 7fa8073f4740 1 Processor -- start 2024-09-21T04:04:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.915+0000 7fa8073f4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bcb69152f0 0x55bcb6cf7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.915+0000 7fa8073f4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb691d4d0 0x55bcb6cf2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.915+0000 7fa8073f4740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcb693e1b0 0x55bcb6cf3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.915+0000 7fa8073f4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f81b00a0 con 0x55bcb691d4d0 2024-09-21T04:04:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.915+0000 7fa8073f4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f81b1510 con 0x55bcb69152f0 2024-09-21T04:04:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.915+0000 7fa8073f4740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f81b0770 con 0x55bcb693e1b0 2024-09-21T04:04:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.915+0000 7fa8073f4740 10 client.0 _ll_register_callbacks cb 0x7fa7ec0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.915+0000 7fa7fd94b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb691d4d0 0x55bcb6cf2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.915+0000 7fa7fd94b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb691d4d0 0x55bcb6cf2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43380/0 (socket says 172.21.15.64:43380) 2024-09-21T04:04:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.915+0000 7fa7fd94b640 1 -- 172.21.15.64:0/2981312100 learned_addr learned my addr 172.21.15.64:0/2981312100 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.915+0000 7fa7fe94d640 1 --2- 172.21.15.64:0/2981312100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcb693e1b0 0x55bcb6cf3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.916+0000 7fa7fe14c640 1 --2- 172.21.15.64:0/2981312100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bcb69152f0 0x55bcb6cf7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.916+0000 7fa7fd94b640 1 -- 172.21.15.64:0/2981312100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcb693e1b0 msgr2=0x55bcb6cf3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.916+0000 7fa7fd94b640 1 --2- 172.21.15.64:0/2981312100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcb693e1b0 0x55bcb6cf3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.916+0000 7fa7fd94b640 1 -- 172.21.15.64:0/2981312100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bcb69152f0 msgr2=0x55bcb6cf7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.916+0000 7fa7fd94b640 1 --2- 172.21.15.64:0/2981312100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bcb69152f0 0x55bcb6cf7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.916+0000 7fa7fd94b640 1 -- 172.21.15.64:0/2981312100 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcb6cf38b0 con 0x55bcb691d4d0 2024-09-21T04:04:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.916+0000 7fa7fd94b640 1 --2- 172.21.15.64:0/2981312100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb691d4d0 0x55bcb6cf2e00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0aa2c0 tx=0x7fa7ec0b6a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab8f042fc6a78ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.916+0000 7fa7e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:04:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.916+0000 7fa7e1ffb640 1 -- 172.21.15.64:0/2981312100 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec020070 con 0x55bcb691d4d0 2024-09-21T04:04:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.916+0000 7fa7e1ffb640 1 -- 172.21.15.64:0/2981312100 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fa7ec0abd20 con 0x55bcb691d4d0 2024-09-21T04:04:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.916+0000 7fa8073f4740 1 -- 172.21.15.64:0/2981312100 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7f4005430 con 0x55bcb691d4d0 2024-09-21T04:04:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.916+0000 7fa7e1ffb640 1 -- 172.21.15.64:0/2981312100 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa7ec0ac7c0 con 0x55bcb691d4d0 2024-09-21T04:04:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.916+0000 7fa8073f4740 10 client.15528 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.916+0000 7fa8073f4740 1 -- 172.21.15.64:0/2981312100 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bcb6d6b860 con 0x55bcb691d4d0 2024-09-21T04:04:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.916+0000 7fa7e1ffb640 1 -- 172.21.15.64:0/2981312100 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7f4005430 con 0x55bcb691d4d0 2024-09-21T04:04:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.917+0000 7fa7e1ffb640 1 -- 172.21.15.64:0/2981312100 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa7ec0abf20 con 0x55bcb691d4d0 2024-09-21T04:04:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.917+0000 7fa8073f4740 10 client.15528 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.917+0000 7fa8073f4740 10 client.15528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:04:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.917+0000 7fa8073f4740 20 client.15528 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.917+0000 7fa8073f4740 1 --2- 172.21.15.64:0/2981312100 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55bcb68d21c0 0x55bcb6d6dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.917+0000 7fa8073f4740 4 client.15528 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:04:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.917+0000 7fa8073f4740 1 -- 172.21.15.64:0/2981312100 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bcb6c560f0 con 0x55bcb68d21c0 2024-09-21T04:04:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.917+0000 7fa7fe14c640 1 --2- 172.21.15.64:0/2981312100 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55bcb68d21c0 0x55bcb6d6dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.918+0000 7fa7fe14c640 1 --2- 172.21.15.64:0/2981312100 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55bcb68d21c0 0x55bcb6d6dfc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0866bccd008d91d server_cookie=ef79646057782d2c in_seq=0 out_seq=0 2024-09-21T04:04:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.918+0000 7fa7e1ffb640 10 client.15528 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.923+0000 7fa7e1ffb640 1 -- 172.21.15.64:0/2981312100 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bcb6c560f0 con 0x55bcb68d21c0 2024-09-21T04:04:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.923+0000 7fa7e1ffb640 10 client.15528 handle_command_reply: tid=0 2024-09-21T04:04:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.923+0000 7fa8073f4740 1 client.15528 shutdown 2024-09-21T04:04:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.923+0000 7fa8073f4740 1 -- 172.21.15.64:0/2981312100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb691d4d0 msgr2=0x55bcb6cf2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.923+0000 7fa8073f4740 1 --2- 172.21.15.64:0/2981312100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb691d4d0 0x55bcb6cf2e00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0aa2c0 tx=0x7fa7ec0b6a70 comp rx=0 tx=0).stop 2024-09-21T04:04:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.923+0000 7fa8073f4740 1 -- 172.21.15.64:0/2981312100 shutdown_connections 2024-09-21T04:04:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.923+0000 7fa8073f4740 1 --2- 172.21.15.64:0/2981312100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bcb693e1b0 0x55bcb6cf3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.923+0000 7fa8073f4740 1 --2- 172.21.15.64:0/2981312100 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55bcb68d21c0 0x55bcb6d6dfc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.923+0000 7fa8073f4740 1 --2- 172.21.15.64:0/2981312100 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bcb691d4d0 0x55bcb6cf2e00 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.923+0000 7fa8073f4740 1 --2- 172.21.15.64:0/2981312100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bcb69152f0 0x55bcb6cf7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.923+0000 7fa7e1ffb640 0 client.15528 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.923+0000 7fa8073f4740 1 -- 172.21.15.64:0/2981312100 >> 172.21.15.64:0/2981312100 conn(0x55bcb6932300 msgr2=0x55bcb6ce52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.923+0000 7fa8073f4740 1 -- 172.21.15.64:0/2981312100 shutdown_connections 2024-09-21T04:04:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.924+0000 7fa8073f4740 1 -- 172.21.15.64:0/2981312100 wait complete. 2024-09-21T04:04:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.924+0000 7fa8073f4740 20 client.15528 trim_cache size 0 max 16384 2024-09-21T04:04:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:04:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:04:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:04:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:04:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.924+0000 7fa8003b0640 1 -- 172.21.15.64:0/3610550427 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa7e40b6790 msgr2=0x7fa7e40b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.924+0000 7fa8003b0640 1 --2- 172.21.15.64:0/3610550427 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa7e40b6790 0x7fa7e40b8ca0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0020000 tx=0x7fa7f001d000 comp rx=0 tx=0).stop 2024-09-21T04:04:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.924+0000 7fa8003b0640 1 -- 172.21.15.64:0/3610550427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7f8105250 msgr2=0x7fa7f819cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.924+0000 7fa8003b0640 1 --2- 172.21.15.64:0/3610550427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7f8105250 0x7fa7f819cbd0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00d750 tx=0x7fa7ec00dc10 comp rx=0 tx=0).stop 2024-09-21T04:04:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.924+0000 7fa8003b0640 1 -- 172.21.15.64:0/3610550427 shutdown_connections 2024-09-21T04:04:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.924+0000 7fa8003b0640 1 --2- 172.21.15.64:0/3610550427 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa7e40b6790 0x7fa7e40b8ca0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.924+0000 7fa8003b0640 1 --2- 172.21.15.64:0/3610550427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7f810dd60 0x7fa7f81a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.924+0000 7fa8003b0640 1 --2- 172.21.15.64:0/3610550427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa7f8105250 0x7fa7f819cbd0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.924+0000 7fa8003b0640 1 --2- 172.21.15.64:0/3610550427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7f8104900 0x7fa7f819c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.924+0000 7fa8003b0640 1 -- 172.21.15.64:0/3610550427 >> 172.21.15.64:0/3610550427 conn(0x7fa7f807a740 msgr2=0x7fa7f810cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.924+0000 7fa8003b0640 1 -- 172.21.15.64:0/3610550427 shutdown_connections 2024-09-21T04:04:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:26.924+0000 7fa8003b0640 1 -- 172.21.15.64:0/3610550427 wait complete. 2024-09-21T04:04:28.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:27 smithi119 ceph-mon[34811]: pgmap v334: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-21T04:04:28.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:27 smithi064 ceph-mon[31180]: pgmap v334: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-21T04:04:28.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:27 smithi131 ceph-mon[35193]: pgmap v334: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-21T04:04:30.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:29 smithi119 ceph-mon[34811]: pgmap v335: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-21T04:04:30.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:29 smithi064 ceph-mon[31180]: pgmap v335: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-21T04:04:30.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:29 smithi131 ceph-mon[35193]: pgmap v335: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-21T04:04:32.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:31 smithi119 ceph-mon[34811]: pgmap v336: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T04:04:32.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:31 smithi064 ceph-mon[31180]: pgmap v336: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T04:04:32.022 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T04:04:32.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:31 smithi131 ceph-mon[35193]: pgmap v336: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T04:04:32.324 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:04:32.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.861+0000 7fcc3faa9640 1 -- 172.21.15.64:0/434766817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38102a70 msgr2=0x7fcc38102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:32.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.861+0000 7fcc3faa9640 1 --2- 172.21.15.64:0/434766817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38102a70 0x7fcc38102e80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009900 tx=0x7fcc3402a3f0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.864 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.861+0000 7fcc3faa9640 1 -- 172.21.15.64:0/434766817 shutdown_connections 2024-09-21T04:04:32.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.861+0000 7fcc3faa9640 1 --2- 172.21.15.64:0/434766817 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc38108d00 0x7fcc3810f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.861+0000 7fcc3faa9640 1 --2- 172.21.15.64:0/434766817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc381083b0 0x7fcc381087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.861+0000 7fcc3faa9640 1 --2- 172.21.15.64:0/434766817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38102a70 0x7fcc38102e80 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.861+0000 7fcc3faa9640 1 -- 172.21.15.64:0/434766817 >> 172.21.15.64:0/434766817 conn(0x7fcc38079730 msgr2=0x7fcc38079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:32.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.861+0000 7fcc3faa9640 1 -- 172.21.15.64:0/434766817 shutdown_connections 2024-09-21T04:04:32.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.861+0000 7fcc3faa9640 1 -- 172.21.15.64:0/434766817 wait complete. 2024-09-21T04:04:32.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.862+0000 7fcc3faa9640 1 Processor -- start 2024-09-21T04:04:32.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.862+0000 7fcc3faa9640 1 -- start start 2024-09-21T04:04:32.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.863+0000 7fcc3faa9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc38102a70 0x7fcc38105c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:32.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.863+0000 7fcc3faa9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc381083b0 0x7fcc38104260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:32.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.863+0000 7fcc3faa9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38108d00 0x7fcc381047a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:32.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.863+0000 7fcc3faa9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcc38112970 con 0x7fcc38108d00 2024-09-21T04:04:32.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.863+0000 7fcc3faa9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcc381127f0 con 0x7fcc38102a70 2024-09-21T04:04:32.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.863+0000 7fcc3faa9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcc38112af0 con 0x7fcc381083b0 2024-09-21T04:04:32.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.863+0000 7fcc3e046640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38108d00 0x7fcc381047a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:32.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.863+0000 7fcc3d845640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc38102a70 0x7fcc38105c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:32.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.863+0000 7fcc3e046640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38108d00 0x7fcc381047a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34348/0 (socket says 172.21.15.64:34348) 2024-09-21T04:04:32.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.863+0000 7fcc3e046640 1 -- 172.21.15.64:0/2755094457 learned_addr learned my addr 172.21.15.64:0/2755094457 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:32.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.863+0000 7fcc3d044640 1 --2- 172.21.15.64:0/2755094457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc381083b0 0x7fcc38104260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:32.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.864+0000 7fcc3e046640 1 -- 172.21.15.64:0/2755094457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc381083b0 msgr2=0x7fcc38104260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:32.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.864+0000 7fcc3e046640 1 --2- 172.21.15.64:0/2755094457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc381083b0 0x7fcc38104260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.864+0000 7fcc3e046640 1 -- 172.21.15.64:0/2755094457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc38102a70 msgr2=0x7fcc38105c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:32.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.864+0000 7fcc3e046640 1 --2- 172.21.15.64:0/2755094457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc38102a70 0x7fcc38105c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.864+0000 7fcc3e046640 1 -- 172.21.15.64:0/2755094457 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc38104fc0 con 0x7fcc38108d00 2024-09-21T04:04:32.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.864+0000 7fcc3d845640 1 --2- 172.21.15.64:0/2755094457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc38102a70 0x7fcc38105c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:32.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.864+0000 7fcc3e046640 1 --2- 172.21.15.64:0/2755094457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38108d00 0x7fcc381047a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000eb10 tx=0x7fcc3000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:32.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.864+0000 7fcc2effd640 1 -- 172.21.15.64:0/2755094457 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc30004b20 con 0x7fcc38108d00 2024-09-21T04:04:32.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.864+0000 7fcc2effd640 1 -- 172.21.15.64:0/2755094457 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc30004cc0 con 0x7fcc38108d00 2024-09-21T04:04:32.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.864+0000 7fcc3faa9640 1 -- 172.21.15.64:0/2755094457 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc381ace30 con 0x7fcc38108d00 2024-09-21T04:04:32.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.864+0000 7fcc2effd640 1 -- 172.21.15.64:0/2755094457 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc30010670 con 0x7fcc38108d00 2024-09-21T04:04:32.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.864+0000 7fcc3faa9640 1 -- 172.21.15.64:0/2755094457 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc381ad290 con 0x7fcc38108d00 2024-09-21T04:04:32.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.868+0000 7fcc4601d740 1 Processor -- start 2024-09-21T04:04:32.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.868+0000 7fcc2effd640 1 -- 172.21.15.64:0/2755094457 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 154694+0+0 (secure 0 0 0) 0x7fcc30005020 con 0x7fcc38108d00 2024-09-21T04:04:32.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.868+0000 7fcc4601d740 1 -- start start 2024-09-21T04:04:32.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.869+0000 7fcc4601d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c073c1070 0x557c07768650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:32.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.869+0000 7fcc4601d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c073981b0 0x557c07768b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:32.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.869+0000 7fcc4601d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c073a0390 0x557c077690d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:32.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.869+0000 7fcc4601d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557c076dee50 con 0x557c073981b0 2024-09-21T04:04:32.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.869+0000 7fcc4601d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557c076ce280 con 0x557c073c1070 2024-09-21T04:04:32.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.869+0000 7fcc4601d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557c076d2090 con 0x557c073a0390 2024-09-21T04:04:32.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.869+0000 7fcc3d845640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c073981b0 0x557c07768b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:32.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.869+0000 7fcc2effd640 1 --2- 172.21.15.64:0/2755094457 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcc240b6710 0x7fcc240b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:32.880 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.869+0000 7fcc3d845640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c073981b0 0x557c07768b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34352/0 (socket says 172.21.15.64:34352) 2024-09-21T04:04:32.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.870+0000 7fcc3d845640 1 -- 172.21.15.64:0/2443126548 learned_addr learned my addr 172.21.15.64:0/2443126548 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:32.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.870+0000 7fcc3e046640 1 --2- 172.21.15.64:0/2755094457 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcc240b6710 0x7fcc240b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:32.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.870+0000 7fcc3d044640 1 --2- 172.21.15.64:0/2443126548 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c073c1070 0x557c07768650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:32.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.870+0000 7fcc3d044640 1 --2- 172.21.15.64:0/2443126548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c073a0390 0x557c077690d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:32.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.870+0000 7fcc2effd640 1 -- 172.21.15.64:0/2755094457 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fcc300e7f20 con 0x7fcc38108d00 2024-09-21T04:04:32.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.870+0000 7fcc3d845640 1 -- 172.21.15.64:0/2443126548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c073a0390 msgr2=0x557c077690d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:32.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.870+0000 7fcc3d845640 1 --2- 172.21.15.64:0/2443126548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c073a0390 0x557c077690d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.870+0000 7fcc3d845640 1 -- 172.21.15.64:0/2443126548 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c073c1070 msgr2=0x557c07768650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:32.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.870+0000 7fcc3d845640 1 --2- 172.21.15.64:0/2443126548 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c073c1070 0x557c07768650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.870+0000 7fcc3d845640 1 -- 172.21.15.64:0/2443126548 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c0742c650 con 0x557c073981b0 2024-09-21T04:04:32.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.870+0000 7fcc3d044640 1 --2- 172.21.15.64:0/2443126548 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c073c1070 0x557c07768650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:04:32.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.870+0000 7fcc3e046640 1 --2- 172.21.15.64:0/2755094457 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcc240b6710 0x7fcc240b8c20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcc300e5580 tx=0x7fcc300a9af0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:32.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.871+0000 7fcc3d845640 1 --2- 172.21.15.64:0/2443126548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c073981b0 0x557c07768b90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcc3402a910 tx=0x7fcc34005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c926eda710e08618 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:32.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.871+0000 7fcc23fff640 1 -- 172.21.15.64:0/2443126548 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34040070 con 0x557c073981b0 2024-09-21T04:04:32.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.871+0000 7fcc23fff640 1 -- 172.21.15.64:0/2443126548 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc34031720 con 0x557c073981b0 2024-09-21T04:04:32.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.871+0000 7fcc23fff640 1 -- 172.21.15.64:0/2443126548 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34035040 con 0x557c073981b0 2024-09-21T04:04:32.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.872+0000 7fcc4601d740 1 -- 172.21.15.64:0/2443126548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c073981b0 msgr2=0x557c07768b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:32.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.872+0000 7fcc4601d740 1 --2- 172.21.15.64:0/2443126548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c073981b0 0x557c07768b90 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcc3402a910 tx=0x7fcc34005e20 comp rx=0 tx=0).stop 2024-09-21T04:04:32.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.872+0000 7fcc4601d740 1 -- 172.21.15.64:0/2443126548 shutdown_connections 2024-09-21T04:04:32.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.872+0000 7fcc4601d740 1 --2- 172.21.15.64:0/2443126548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c073a0390 0x557c077690d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.872+0000 7fcc4601d740 1 --2- 172.21.15.64:0/2443126548 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c073981b0 0x557c07768b90 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.872+0000 7fcc4601d740 1 --2- 172.21.15.64:0/2443126548 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c073c1070 0x557c07768650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.872+0000 7fcc4601d740 1 -- 172.21.15.64:0/2443126548 >> 172.21.15.64:0/2443126548 conn(0x557c073b5460 msgr2=0x557c07766540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:32.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.872+0000 7fcc4601d740 1 -- 172.21.15.64:0/2443126548 shutdown_connections 2024-09-21T04:04:32.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.872+0000 7fcc4601d740 1 -- 172.21.15.64:0/2443126548 wait complete. 2024-09-21T04:04:32.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.873+0000 7fcc4601d740 1 -- start start 2024-09-21T04:04:32.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.873+0000 7fcc4601d740 1 Processor -- start 2024-09-21T04:04:32.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.873+0000 7fcc4601d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c073a0390 0x557c07773e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:32.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.873+0000 7fcc4601d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c073c1070 0x557c077743c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:32.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.873+0000 7fcc4601d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c0737f9f0 0x557c077800f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:32.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.873+0000 7fcc4601d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557c0752f700 con 0x557c073c1070 2024-09-21T04:04:32.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.873+0000 7fcc4601d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557c07669110 con 0x557c0737f9f0 2024-09-21T04:04:32.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.873+0000 7fcc4601d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557c07776dd0 con 0x557c073a0390 2024-09-21T04:04:32.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.873+0000 7fcc4601d740 10 client.0 _ll_register_callbacks cb 0x557c07682990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:32.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.874+0000 7fcc3d044640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c073a0390 0x557c07773e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:32.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.874+0000 7fcc3d044640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c073a0390 0x557c07773e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:43358/0 (socket says 172.21.15.64:43358) 2024-09-21T04:04:32.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.874+0000 7fcc3d044640 1 -- 172.21.15.64:0/2428158646 learned_addr learned my addr 172.21.15.64:0/2428158646 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:32.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.874+0000 7fcc3d044640 1 --2- 172.21.15.64:0/2428158646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c0737f9f0 0x557c077800f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:32.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.874+0000 7fcc3d845640 1 --2- 172.21.15.64:0/2428158646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c073c1070 0x557c077743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:32.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.874+0000 7fcc3d044640 1 -- 172.21.15.64:0/2428158646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c0737f9f0 msgr2=0x557c077800f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:32.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.874+0000 7fcc3d044640 1 --2- 172.21.15.64:0/2428158646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c0737f9f0 0x557c077800f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.874+0000 7fcc3d044640 1 -- 172.21.15.64:0/2428158646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c073c1070 msgr2=0x557c077743c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:32.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.874+0000 7fcc3d044640 1 --2- 172.21.15.64:0/2428158646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c073c1070 0x557c077743c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.874+0000 7fcc3d044640 1 -- 172.21.15.64:0/2428158646 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c07696c40 con 0x557c073a0390 2024-09-21T04:04:32.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.874+0000 7fcc3d845640 1 --2- 172.21.15.64:0/2428158646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c073c1070 0x557c077743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:04:32.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.874+0000 7fcc3d044640 1 --2- 172.21.15.64:0/2428158646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c073a0390 0x557c07773e80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcc28010ec0 tx=0x7fcc280046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d771ae7d732ce87 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:32.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.874+0000 7fcc217fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-21T04:04:32.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.875+0000 7fcc217fa640 1 -- 172.21.15.64:0/2428158646 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2801a040 con 0x557c073a0390 2024-09-21T04:04:32.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.875+0000 7fcc217fa640 1 -- 172.21.15.64:0/2428158646 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fcc28004050 con 0x557c073a0390 2024-09-21T04:04:32.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.875+0000 7fcc4601d740 1 -- 172.21.15.64:0/2428158646 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc34031720 con 0x557c073a0390 2024-09-21T04:04:32.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.875+0000 7fcc217fa640 1 -- 172.21.15.64:0/2428158646 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcc28004250 con 0x557c073a0390 2024-09-21T04:04:32.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.875+0000 7fcc4601d740 10 client.25207 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:32.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.875+0000 7fcc4601d740 1 -- 172.21.15.64:0/2428158646 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557c07782c90 con 0x557c073a0390 2024-09-21T04:04:32.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.876+0000 7fcc217fa640 1 -- 172.21.15.64:0/2428158646 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc34031720 con 0x557c073a0390 2024-09-21T04:04:32.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.876+0000 7fcc217fa640 1 -- 172.21.15.64:0/2428158646 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcc28021430 con 0x557c073a0390 2024-09-21T04:04:32.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.876+0000 7fcc4601d740 10 client.25207 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:32.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.876+0000 7fcc4601d740 10 client.25207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:04:32.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.876+0000 7fcc4601d740 20 client.25207 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:32.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.876+0000 7fcc4601d740 1 --2- 172.21.15.64:0/2428158646 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x557c07354da0 0x557c077860f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:32.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.876+0000 7fcc4601d740 4 client.25207 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:04:32.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.876+0000 7fcc4601d740 1 -- 172.21.15.64:0/2428158646 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c076e0da0 con 0x557c07354da0 2024-09-21T04:04:32.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.876+0000 7fcc3d845640 1 --2- 172.21.15.64:0/2428158646 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x557c07354da0 0x557c077860f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:32.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.877+0000 7fcc217fa640 10 client.25207 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:32.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.877+0000 7fcc3d845640 1 --2- 172.21.15.64:0/2428158646 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x557c07354da0 0x557c077860f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2c174de6abd0c7d server_cookie=873750ade448b28a in_seq=0 out_seq=0 2024-09-21T04:04:32.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.878+0000 7fcc217fa640 1 -- 172.21.15.64:0/2428158646 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557c076e0da0 con 0x557c07354da0 2024-09-21T04:04:32.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.878+0000 7fcc217fa640 10 client.25207 handle_command_reply: tid=0 2024-09-21T04:04:32.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.878+0000 7fcc4601d740 1 client.25207 shutdown 2024-09-21T04:04:32.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.878+0000 7fcc4601d740 1 -- 172.21.15.64:0/2428158646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c073a0390 msgr2=0x557c07773e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:32.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.878+0000 7fcc4601d740 1 --2- 172.21.15.64:0/2428158646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c073a0390 0x557c07773e80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcc28010ec0 tx=0x7fcc280046c0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.878+0000 7fcc4601d740 1 -- 172.21.15.64:0/2428158646 shutdown_connections 2024-09-21T04:04:32.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.878+0000 7fcc4601d740 1 --2- 172.21.15.64:0/2428158646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c0737f9f0 0x557c077800f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.878+0000 7fcc4601d740 1 --2- 172.21.15.64:0/2428158646 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x557c07354da0 0x557c077860f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.878+0000 7fcc217fa640 0 client.25207 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:32.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.878+0000 7fcc4601d740 1 --2- 172.21.15.64:0/2428158646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c073c1070 0x557c077743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.878+0000 7fcc4601d740 1 --2- 172.21.15.64:0/2428158646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c073a0390 0x557c07773e80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.878+0000 7fcc4601d740 1 -- 172.21.15.64:0/2428158646 >> 172.21.15.64:0/2428158646 conn(0x557c073b5460 msgr2=0x557c0776b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:32.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.879+0000 7fcc4601d740 1 -- 172.21.15.64:0/2428158646 shutdown_connections 2024-09-21T04:04:32.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.879+0000 7fcc4601d740 1 -- 172.21.15.64:0/2428158646 wait complete. 2024-09-21T04:04:32.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.879+0000 7fcc4601d740 20 client.25207 trim_cache size 0 max 16384 2024-09-21T04:04:32.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.914+0000 7fcc4601d740 1 Processor -- start 2024-09-21T04:04:32.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.914+0000 7fcc4601d740 1 -- start start 2024-09-21T04:04:32.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.915+0000 7fcc4601d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c0737f9f0 0x557c0776b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:32.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.915+0000 7fcc4601d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c073a0390 0x557c0776bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:32.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.915+0000 7fcc4601d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c073c1070 0x557c07771070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:32.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.915+0000 7fcc4601d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcc381b8740 con 0x557c073c1070 2024-09-21T04:04:32.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.915+0000 7fcc4601d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcc381b6010 con 0x557c0737f9f0 2024-09-21T04:04:32.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.915+0000 7fcc4601d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcc381b3e60 con 0x557c073a0390 2024-09-21T04:04:32.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.915+0000 7fcc3d044640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c073c1070 0x557c07771070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.915+0000 7fcc3d044640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c073c1070 0x557c07771070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34368/0 (socket says 172.21.15.64:34368) 2024-09-21T04:04:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.915+0000 7fcc3d044640 1 -- 172.21.15.64:0/1636915927 learned_addr learned my addr 172.21.15.64:0/1636915927 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.915+0000 7fcc3d044640 1 --2- 172.21.15.64:0/1636915927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c0737f9f0 0x557c0776b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.915+0000 7fcc3d845640 1 --2- 172.21.15.64:0/1636915927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c073a0390 0x557c0776bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.915+0000 7fcc3d044640 1 -- 172.21.15.64:0/1636915927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c073a0390 msgr2=0x557c0776bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.915+0000 7fcc3d044640 1 --2- 172.21.15.64:0/1636915927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c073a0390 0x557c0776bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.915+0000 7fcc3d044640 1 -- 172.21.15.64:0/1636915927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c0737f9f0 msgr2=0x557c0776b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.915+0000 7fcc3d044640 1 --2- 172.21.15.64:0/1636915927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c0737f9f0 0x557c0776b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.915+0000 7fcc3d044640 1 -- 172.21.15.64:0/1636915927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c0776eae0 con 0x557c073c1070 2024-09-21T04:04:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.916+0000 7fcc3d845640 1 --2- 172.21.15.64:0/1636915927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c073a0390 0x557c0776bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:04:32.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.916+0000 7fcc3d044640 1 --2- 172.21.15.64:0/1636915927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c073c1070 0x557c07771070 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fcc280059f0 tx=0x7fcc28009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f3d582c0be15f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:32.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.916+0000 7fcc237fe640 1 -- 172.21.15.64:0/1636915927 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2802d070 con 0x557c073c1070 2024-09-21T04:04:32.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.916+0000 7fcc237fe640 1 -- 172.21.15.64:0/1636915927 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc2802e040 con 0x557c073c1070 2024-09-21T04:04:32.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.916+0000 7fcc237fe640 1 -- 172.21.15.64:0/1636915927 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc28037b90 con 0x557c073c1070 2024-09-21T04:04:32.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.917+0000 7fcc4601d740 1 -- 172.21.15.64:0/1636915927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c073c1070 msgr2=0x557c07771070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:32.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.917+0000 7fcc4601d740 1 --2- 172.21.15.64:0/1636915927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c073c1070 0x557c07771070 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fcc280059f0 tx=0x7fcc28009ba0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.917+0000 7fcc4601d740 1 -- 172.21.15.64:0/1636915927 shutdown_connections 2024-09-21T04:04:32.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.917+0000 7fcc4601d740 1 --2- 172.21.15.64:0/1636915927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c073c1070 0x557c07771070 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.918+0000 7fcc4601d740 1 --2- 172.21.15.64:0/1636915927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c073a0390 0x557c0776bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.918+0000 7fcc4601d740 1 --2- 172.21.15.64:0/1636915927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c0737f9f0 0x557c0776b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.918+0000 7fcc4601d740 1 -- 172.21.15.64:0/1636915927 >> 172.21.15.64:0/1636915927 conn(0x557c073b5460 msgr2=0x557c07772700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:32.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.918+0000 7fcc4601d740 1 -- 172.21.15.64:0/1636915927 shutdown_connections 2024-09-21T04:04:32.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.918+0000 7fcc4601d740 1 -- 172.21.15.64:0/1636915927 wait complete. 2024-09-21T04:04:32.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.918+0000 7fcc4601d740 1 -- start start 2024-09-21T04:04:32.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.918+0000 7fcc4601d740 1 Processor -- start 2024-09-21T04:04:32.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.919+0000 7fcc4601d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c0737f9f0 0x557c07778300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:32.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.919+0000 7fcc4601d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c073a0390 0x557c07778840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:32.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.919+0000 7fcc4601d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c073c1070 0x557c077efb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:32.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.919+0000 7fcc4601d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcc381b44d0 con 0x557c073a0390 2024-09-21T04:04:32.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.919+0000 7fcc4601d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcc381b5940 con 0x557c073c1070 2024-09-21T04:04:32.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.919+0000 7fcc4601d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcc381b4ba0 con 0x557c0737f9f0 2024-09-21T04:04:32.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.919+0000 7fcc4601d740 10 client.0 _ll_register_callbacks cb 0x7fcc28009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.919+0000 7fcc3d845640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c073a0390 0x557c07778840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.919+0000 7fcc3d845640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c073a0390 0x557c07778840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34380/0 (socket says 172.21.15.64:34380) 2024-09-21T04:04:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.919+0000 7fcc3d845640 1 -- 172.21.15.64:0/4017863037 learned_addr learned my addr 172.21.15.64:0/4017863037 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.919+0000 7fcc3d044640 1 --2- 172.21.15.64:0/4017863037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c0737f9f0 0x557c07778300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.919+0000 7fcc3d044640 1 --2- 172.21.15.64:0/4017863037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c073c1070 0x557c077efb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.920+0000 7fcc3d845640 1 -- 172.21.15.64:0/4017863037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c0737f9f0 msgr2=0x557c07778300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.920+0000 7fcc3d845640 1 --2- 172.21.15.64:0/4017863037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c0737f9f0 0x557c07778300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.920+0000 7fcc3d845640 1 -- 172.21.15.64:0/4017863037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c073c1070 msgr2=0x557c077efb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.920+0000 7fcc3d845640 1 --2- 172.21.15.64:0/4017863037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c073c1070 0x557c077efb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.920+0000 7fcc3d845640 1 -- 172.21.15.64:0/4017863037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c077790c0 con 0x557c073a0390 2024-09-21T04:04:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.920+0000 7fcc3d845640 1 --2- 172.21.15.64:0/4017863037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c073a0390 0x557c07778840 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcc340054d0 tx=0x7fcc34000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7227a796ca3a73fd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.920+0000 7fcc217fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:04:32.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.920+0000 7fcc217fa640 1 -- 172.21.15.64:0/4017863037 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34040040 con 0x557c073a0390 2024-09-21T04:04:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.920+0000 7fcc217fa640 1 -- 172.21.15.64:0/4017863037 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fcc3403c440 con 0x557c073a0390 2024-09-21T04:04:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.920+0000 7fcc4601d740 1 -- 172.21.15.64:0/4017863037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc2802e040 con 0x557c073a0390 2024-09-21T04:04:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.921+0000 7fcc217fa640 1 -- 172.21.15.64:0/4017863037 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcc34035040 con 0x557c073a0390 2024-09-21T04:04:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.921+0000 7fcc4601d740 10 client.15543 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.921+0000 7fcc4601d740 1 -- 172.21.15.64:0/4017863037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557c077f1100 con 0x557c073a0390 2024-09-21T04:04:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.921+0000 7fcc217fa640 1 -- 172.21.15.64:0/4017863037 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc3403caf0 con 0x557c073a0390 2024-09-21T04:04:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.921+0000 7fcc217fa640 1 -- 172.21.15.64:0/4017863037 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcc3405c070 con 0x557c073a0390 2024-09-21T04:04:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.921+0000 7fcc4601d740 10 client.15543 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.921+0000 7fcc4601d740 10 client.15543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:04:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.921+0000 7fcc4601d740 20 client.15543 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.921+0000 7fcc4601d740 1 --2- 172.21.15.64:0/4017863037 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x557c077f1440 0x557c077f3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.921+0000 7fcc4601d740 4 client.15543 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T04:04:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.921+0000 7fcc4601d740 1 -- 172.21.15.64:0/4017863037 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fcc2802e040 con 0x557c077f1440 2024-09-21T04:04:32.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.922+0000 7fcc3d044640 1 --2- 172.21.15.64:0/4017863037 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x557c077f1440 0x557c077f3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.922+0000 7fcc3d044640 1 --2- 172.21.15.64:0/4017863037 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x557c077f1440 0x557c077f3860 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fb65b42fb6b1324 server_cookie=ed28aca0de1c0ef1 in_seq=0 out_seq=0 2024-09-21T04:04:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.922+0000 7fcc217fa640 10 client.15543 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.923+0000 7fcc217fa640 1 -- 172.21.15.64:0/4017863037 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x7fcc2802e040 con 0x557c077f1440 2024-09-21T04:04:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.923+0000 7fcc217fa640 10 client.15543 handle_command_reply: tid=0 2024-09-21T04:04:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.923+0000 7fcc4601d740 1 client.15543 shutdown 2024-09-21T04:04:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.923+0000 7fcc4601d740 1 -- 172.21.15.64:0/4017863037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c073a0390 msgr2=0x557c07778840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.923+0000 7fcc4601d740 1 --2- 172.21.15.64:0/4017863037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c073a0390 0x557c07778840 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcc340054d0 tx=0x7fcc34000bc0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.923+0000 7fcc4601d740 1 -- 172.21.15.64:0/4017863037 shutdown_connections 2024-09-21T04:04:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.923+0000 7fcc4601d740 1 --2- 172.21.15.64:0/4017863037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557c073c1070 0x557c077efb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.923+0000 7fcc4601d740 1 --2- 172.21.15.64:0/4017863037 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x557c077f1440 0x557c077f3860 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.923+0000 7fcc4601d740 1 --2- 172.21.15.64:0/4017863037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557c073a0390 0x557c07778840 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.923+0000 7fcc4601d740 1 --2- 172.21.15.64:0/4017863037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c0737f9f0 0x557c07778300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.923+0000 7fcc4601d740 1 -- 172.21.15.64:0/4017863037 >> 172.21.15.64:0/4017863037 conn(0x557c073b5460 msgr2=0x557c0776b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:32.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.923+0000 7fcc217fa640 0 client.15543 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:32.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.923+0000 7fcc4601d740 1 -- 172.21.15.64:0/4017863037 shutdown_connections 2024-09-21T04:04:32.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.923+0000 7fcc4601d740 1 -- 172.21.15.64:0/4017863037 wait complete. 2024-09-21T04:04:32.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.923+0000 7fcc4601d740 20 client.15543 trim_cache size 0 max 16384 2024-09-21T04:04:32.930 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:04:32.930 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-21T04:04:32.931 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {}, 2024-09-21T04:04:32.931 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_stats": { 2024-09-21T04:04:32.931 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:04:32.931 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:03:59.570174+0000", 2024-09-21T04:04:32.931 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:04:32.931 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:04:32.931 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:04:32.931 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:04:32.931 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:04:32.931 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:04:32.931 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:04:32.931 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:04:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.924+0000 7fcc3faa9640 1 -- 172.21.15.64:0/2755094457 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcc240b6710 msgr2=0x7fcc240b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.924+0000 7fcc3faa9640 1 --2- 172.21.15.64:0/2755094457 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcc240b6710 0x7fcc240b8c20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcc300e5580 tx=0x7fcc300a9af0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.924+0000 7fcc3faa9640 1 -- 172.21.15.64:0/2755094457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38108d00 msgr2=0x7fcc381047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.924+0000 7fcc3faa9640 1 --2- 172.21.15.64:0/2755094457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38108d00 0x7fcc381047a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000eb10 tx=0x7fcc3000c2d0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.924+0000 7fcc3faa9640 1 -- 172.21.15.64:0/2755094457 shutdown_connections 2024-09-21T04:04:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.924+0000 7fcc3faa9640 1 --2- 172.21.15.64:0/2755094457 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcc240b6710 0x7fcc240b8c20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.924+0000 7fcc3faa9640 1 --2- 172.21.15.64:0/2755094457 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc38108d00 0x7fcc381047a0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.924+0000 7fcc3faa9640 1 --2- 172.21.15.64:0/2755094457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc381083b0 0x7fcc38104260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.924+0000 7fcc3faa9640 1 --2- 172.21.15.64:0/2755094457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc38102a70 0x7fcc38105c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:32.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.924+0000 7fcc3faa9640 1 -- 172.21.15.64:0/2755094457 >> 172.21.15.64:0/2755094457 conn(0x7fcc38079730 msgr2=0x7fcc38101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:32.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.924+0000 7fcc3faa9640 1 -- 172.21.15.64:0/2755094457 shutdown_connections 2024-09-21T04:04:32.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:32.925+0000 7fcc3faa9640 1 -- 172.21.15.64:0/2755094457 wait complete. 2024-09-21T04:04:33.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:33 smithi064 ceph-mon[31180]: pgmap v337: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-21T04:04:33.806 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T04:04:33.806 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:04:34.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:33 smithi119 ceph-mon[34811]: pgmap v337: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-21T04:04:34.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:33 smithi131 ceph-mon[35193]: pgmap v337: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-21T04:04:34.101 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:04:34.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.642+0000 7f6fce753640 1 -- 172.21.15.64:0/1242417285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fc8102a60 msgr2=0x7f6fc8102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:34.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.642+0000 7f6fce753640 1 --2- 172.21.15.64:0/1242417285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fc8102a60 0x7f6fc8102e70 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8009900 tx=0x7f6fb802a410 comp rx=0 tx=0).stop 2024-09-21T04:04:34.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.642+0000 7f6fce753640 1 -- 172.21.15.64:0/1242417285 shutdown_connections 2024-09-21T04:04:34.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.642+0000 7f6fce753640 1 --2- 172.21.15.64:0/1242417285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fc8108cf0 0x7f6fc810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:34.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.642+0000 7f6fce753640 1 --2- 172.21.15.64:0/1242417285 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc81083a0 0x7f6fc81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:34.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.642+0000 7f6fce753640 1 --2- 172.21.15.64:0/1242417285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fc8102a60 0x7f6fc8102e70 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:34.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.642+0000 7f6fce753640 1 -- 172.21.15.64:0/1242417285 >> 172.21.15.64:0/1242417285 conn(0x7f6fc8079730 msgr2=0x7f6fc8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:34.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.642+0000 7f6fce753640 1 -- 172.21.15.64:0/1242417285 shutdown_connections 2024-09-21T04:04:34.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.642+0000 7f6fce753640 1 -- 172.21.15.64:0/1242417285 wait complete. 2024-09-21T04:04:34.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.643+0000 7f6fce753640 1 Processor -- start 2024-09-21T04:04:34.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.643+0000 7f6fce753640 1 -- start start 2024-09-21T04:04:34.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.643+0000 7f6fce753640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fc8102a60 0x7f6fc819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:34.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.644+0000 7f6fce753640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fc81083a0 0x7f6fc819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:34.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.644+0000 7f6fce753640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc8108cf0 0x7f6fc819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.644+0000 7f6fce753640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc8112960 con 0x7f6fc81083a0 2024-09-21T04:04:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.644+0000 7f6fce753640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc81127e0 con 0x7f6fc8102a60 2024-09-21T04:04:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.644+0000 7f6fce753640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc8112ae0 con 0x7f6fc8108cf0 2024-09-21T04:04:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.644+0000 7f6fc7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fc8102a60 0x7f6fc819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.644+0000 7f6fc77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fc81083a0 0x7f6fc819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.644+0000 7f6fc77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fc81083a0 0x7f6fc819a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34406/0 (socket says 172.21.15.64:34406) 2024-09-21T04:04:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.644+0000 7f6fc77fe640 1 -- 172.21.15.64:0/2542269288 learned_addr learned my addr 172.21.15.64:0/2542269288 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:34.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.644+0000 7f6fcccf0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc8108cf0 0x7f6fc819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.644+0000 7f6fcccf0640 1 -- 172.21.15.64:0/2542269288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fc8102a60 msgr2=0x7f6fc819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.644+0000 7f6fcccf0640 1 --2- 172.21.15.64:0/2542269288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fc8102a60 0x7f6fc819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.644+0000 7f6fcccf0640 1 -- 172.21.15.64:0/2542269288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fc81083a0 msgr2=0x7f6fc819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.644+0000 7f6fcccf0640 1 --2- 172.21.15.64:0/2542269288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fc81083a0 0x7f6fc819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.644+0000 7f6fcccf0640 1 -- 172.21.15.64:0/2542269288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc81a03b0 con 0x7f6fc8108cf0 2024-09-21T04:04:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.645+0000 7f6fcccf0640 1 --2- 172.21.15.64:0/2542269288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc8108cf0 0x7f6fc819fc80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6fc000e940 tx=0x7f6fc000ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.645+0000 7f6fc57fa640 1 -- 172.21.15.64:0/2542269288 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0004d20 con 0x7f6fc8108cf0 2024-09-21T04:04:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.645+0000 7f6fc57fa640 1 -- 172.21.15.64:0/2542269288 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fc0005020 con 0x7f6fc8108cf0 2024-09-21T04:04:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.645+0000 7f6fc57fa640 1 -- 172.21.15.64:0/2542269288 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc00116d0 con 0x7f6fc8108cf0 2024-09-21T04:04:34.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.645+0000 7f6fce753640 1 -- 172.21.15.64:0/2542269288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fc81a06a0 con 0x7f6fc8108cf0 2024-09-21T04:04:34.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.645+0000 7f6fce753640 1 -- 172.21.15.64:0/2542269288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc81a6e00 con 0x7f6fc8108cf0 2024-09-21T04:04:34.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.646+0000 7f6fce753640 1 -- 172.21.15.64:0/2542269288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fc8102e70 con 0x7f6fc8108cf0 2024-09-21T04:04:34.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.649+0000 7f6fc57fa640 1 -- 172.21.15.64:0/2542269288 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) ==== 154694+0+0 (secure 0 0 0) 0x7f6fc0005440 con 0x7f6fc8108cf0 2024-09-21T04:04:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.650+0000 7f6fc57fa640 1 --2- 172.21.15.64:0/2542269288 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6fb00b6790 0x7f6fb00b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.650+0000 7f6fc57fa640 1 -- 172.21.15.64:0/2542269288 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f6fc00e78b0 con 0x7f6fc8108cf0 2024-09-21T04:04:34.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.650+0000 7f6fc7fff640 1 --2- 172.21.15.64:0/2542269288 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6fb00b6790 0x7f6fb00b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:34.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.651+0000 7f6fc7fff640 1 --2- 172.21.15.64:0/2542269288 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6fb00b6790 0x7f6fb00b8ca0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8031440 tx=0x7f6fb8009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:34.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.652+0000 7f6fc57fa640 1 -- 172.21.15.64:0/2542269288 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6fc0016030 con 0x7f6fc8108cf0 2024-09-21T04:04:34.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.979+0000 7f6fce753640 1 -- 172.21.15.64:0/2542269288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6fc819b570 con 0x7f6fc8108cf0 2024-09-21T04:04:34.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.980+0000 7f6fc57fa640 1 -- 172.21.15.64:0/2542269288 <== mon.1 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f6fc819b570 con 0x7f6fc8108cf0 2024-09-21T04:04:34.983 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:04:34.983 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-21T04:03:50:600319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14805,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/97995119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":97995119},{"type":"v1","addr":"172.21.15.64:6839","nonce":97995119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/390891068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":390891068},{"type":"v1","addr":"172.21.15.64:6841","nonce":390891068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:02:03.354744+0000","modified":"2024-09-21T04:03:50.212837+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24710,"mds_1":24680,"mds_2":24661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24634":{"gid":24634,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6839/782190424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782190424},{"type":"v1","addr":"172.21.15.131:6839","nonce":782190424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24680":{"gid":24680,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24710":{"gid":24710,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6839/1319793636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1319793636},{"type":"v1","addr":"172.21.15.119:6839","nonce":1319793636}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24710,"qdb_cluster":[24710,24680,24661]},"id":1}]} 2024-09-21T04:04:34.983 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-21T04:04:34.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.983+0000 7f6fce753640 1 -- 172.21.15.64:0/2542269288 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6fb00b6790 msgr2=0x7f6fb00b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:34.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.983+0000 7f6fce753640 1 --2- 172.21.15.64:0/2542269288 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6fb00b6790 0x7f6fb00b8ca0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8031440 tx=0x7f6fb8009560 comp rx=0 tx=0).stop 2024-09-21T04:04:34.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.983+0000 7f6fce753640 1 -- 172.21.15.64:0/2542269288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc8108cf0 msgr2=0x7f6fc819fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:34.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.983+0000 7f6fce753640 1 --2- 172.21.15.64:0/2542269288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc8108cf0 0x7f6fc819fc80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6fc000e940 tx=0x7f6fc000ee00 comp rx=0 tx=0).stop 2024-09-21T04:04:34.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.983+0000 7f6fce753640 1 -- 172.21.15.64:0/2542269288 shutdown_connections 2024-09-21T04:04:34.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.983+0000 7f6fce753640 1 --2- 172.21.15.64:0/2542269288 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6fb00b6790 0x7f6fb00b8ca0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:34.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.983+0000 7f6fce753640 1 --2- 172.21.15.64:0/2542269288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fc8108cf0 0x7f6fc819fc80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:34.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.983+0000 7f6fce753640 1 --2- 172.21.15.64:0/2542269288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6fc81083a0 0x7f6fc819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:34.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.983+0000 7f6fce753640 1 --2- 172.21.15.64:0/2542269288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fc8102a60 0x7f6fc819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:34.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.983+0000 7f6fce753640 1 -- 172.21.15.64:0/2542269288 >> 172.21.15.64:0/2542269288 conn(0x7f6fc8079730 msgr2=0x7f6fc8101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:34.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.983+0000 7f6fce753640 1 -- 172.21.15.64:0/2542269288 shutdown_connections 2024-09-21T04:04:34.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:34.983+0000 7f6fce753640 1 -- 172.21.15.64:0/2542269288 wait complete. 2024-09-21T04:04:35.673 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T04:04:35.948 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:04:36.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:35 smithi119 ceph-mon[34811]: pgmap v338: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T04:04:36.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:35 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2542269288' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:04:36.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:35 smithi064 ceph-mon[31180]: pgmap v338: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T04:04:36.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:35 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2542269288' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:04:36.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:35 smithi131 ceph-mon[35193]: pgmap v338: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T04:04:36.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:35 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2542269288' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:04:36.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.507+0000 7fa6cfa13640 1 -- 172.21.15.64:0/3958639321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6c8101f70 msgr2=0x7fa6c8102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:36.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.507+0000 7fa6cfa13640 1 --2- 172.21.15.64:0/3958639321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6c8101f70 0x7fa6c8102380 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009900 tx=0x7fa6c402a410 comp rx=0 tx=0).stop 2024-09-21T04:04:36.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.507+0000 7fa6cfa13640 1 -- 172.21.15.64:0/3958639321 shutdown_connections 2024-09-21T04:04:36.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.507+0000 7fa6cfa13640 1 --2- 172.21.15.64:0/3958639321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6c8109e00 0x7fa6c8110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.507+0000 7fa6cfa13640 1 --2- 172.21.15.64:0/3958639321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6c81094b0 0x7fa6c81098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.507+0000 7fa6cfa13640 1 --2- 172.21.15.64:0/3958639321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6c8101f70 0x7fa6c8102380 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.507+0000 7fa6cfa13640 1 -- 172.21.15.64:0/3958639321 >> 172.21.15.64:0/3958639321 conn(0x7fa6c8077c60 msgr2=0x7fa6c80780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:36.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.508+0000 7fa6cfa13640 1 -- 172.21.15.64:0/3958639321 shutdown_connections 2024-09-21T04:04:36.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.508+0000 7fa6cfa13640 1 -- 172.21.15.64:0/3958639321 wait complete. 2024-09-21T04:04:36.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.508+0000 7fa6cfa13640 1 Processor -- start 2024-09-21T04:04:36.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.508+0000 7fa6cfa13640 1 -- start start 2024-09-21T04:04:36.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.509+0000 7fa6cfa13640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6c8101f70 0x7fa6c819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:36.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.509+0000 7fa6cfa13640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6c81094b0 0x7fa6c819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:36.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.509+0000 7fa6cfa13640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6c8109e00 0x7fa6c819fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:36.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.509+0000 7fa6cfa13640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c8113ac0 con 0x7fa6c8109e00 2024-09-21T04:04:36.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.509+0000 7fa6cfa13640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c8113940 con 0x7fa6c81094b0 2024-09-21T04:04:36.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.509+0000 7fa6cfa13640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c8113c40 con 0x7fa6c8101f70 2024-09-21T04:04:36.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.509+0000 7fa6cdfb0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6c8109e00 0x7fa6c819fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:36.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.509+0000 7fa6cdfb0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6c8109e00 0x7fa6c819fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34428/0 (socket says 172.21.15.64:34428) 2024-09-21T04:04:36.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.509+0000 7fa6cdfb0640 1 -- 172.21.15.64:0/615198274 learned_addr learned my addr 172.21.15.64:0/615198274 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:36.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.509+0000 7fa6ccfae640 1 --2- 172.21.15.64:0/615198274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6c81094b0 0x7fa6c819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:36.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.509+0000 7fa6cd7af640 1 --2- 172.21.15.64:0/615198274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6c8101f70 0x7fa6c819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:36.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.509+0000 7fa6cdfb0640 1 -- 172.21.15.64:0/615198274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6c8101f70 msgr2=0x7fa6c819a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:36.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.509+0000 7fa6cdfb0640 1 --2- 172.21.15.64:0/615198274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6c8101f70 0x7fa6c819a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.509+0000 7fa6cdfb0640 1 -- 172.21.15.64:0/615198274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6c81094b0 msgr2=0x7fa6c819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:36.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.509+0000 7fa6cdfb0640 1 --2- 172.21.15.64:0/615198274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6c81094b0 0x7fa6c819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.509+0000 7fa6cdfb0640 1 -- 172.21.15.64:0/615198274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c81a0480 con 0x7fa6c8109e00 2024-09-21T04:04:36.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.510+0000 7fa6cdfb0640 1 --2- 172.21.15.64:0/615198274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6c8109e00 0x7fa6c819fd50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009fd0 tx=0x7fa6c000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:36.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.510+0000 7fa6be7fc640 1 -- 172.21.15.64:0/615198274 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0017070 con 0x7fa6c8109e00 2024-09-21T04:04:36.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.510+0000 7fa6be7fc640 1 -- 172.21.15.64:0/615198274 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6c0003e20 con 0x7fa6c8109e00 2024-09-21T04:04:36.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.510+0000 7fa6be7fc640 1 -- 172.21.15.64:0/615198274 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0010460 con 0x7fa6c8109e00 2024-09-21T04:04:36.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.510+0000 7fa6cfa13640 1 -- 172.21.15.64:0/615198274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c81a0770 con 0x7fa6c8109e00 2024-09-21T04:04:36.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.510+0000 7fa6cfa13640 1 -- 172.21.15.64:0/615198274 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c8102df0 con 0x7fa6c8109e00 2024-09-21T04:04:36.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.513+0000 7fa6d5f87740 1 Processor -- start 2024-09-21T04:04:36.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.515+0000 7fa6be7fc640 1 -- 172.21.15.64:0/615198274 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 154694+0+0 (secure 0 0 0) 0x7fa6c0003970 con 0x7fa6c8109e00 2024-09-21T04:04:36.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.515+0000 7fa6d5f87740 1 -- start start 2024-09-21T04:04:36.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.515+0000 7fa6d5f87740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bd6072070 0x562bd64192c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:36.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.515+0000 7fa6d5f87740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bd60491b0 0x562bd6419800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:36.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.515+0000 7fa6d5f87740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd6051390 0x562bd6419d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:36.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.515+0000 7fa6d5f87740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562bd638fea0 con 0x562bd6051390 2024-09-21T04:04:36.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.516+0000 7fa6d5f87740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562bd637f2d0 con 0x562bd60491b0 2024-09-21T04:04:36.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.516+0000 7fa6d5f87740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562bd63830e0 con 0x562bd6072070 2024-09-21T04:04:36.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.516+0000 7fa6ccfae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd6051390 0x562bd6419d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:36.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.516+0000 7fa6ccfae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd6051390 0x562bd6419d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34444/0 (socket says 172.21.15.64:34444) 2024-09-21T04:04:36.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.516+0000 7fa6ccfae640 1 -- 172.21.15.64:0/3867388459 learned_addr learned my addr 172.21.15.64:0/3867388459 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:36.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.516+0000 7fa6cd7af640 1 --2- 172.21.15.64:0/3867388459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bd60491b0 0x562bd6419800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:36.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.516+0000 7fa6ccfae640 1 --2- 172.21.15.64:0/3867388459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bd6072070 0x562bd64192c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:36.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.516+0000 7fa6cd7af640 1 -- 172.21.15.64:0/3867388459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bd6072070 msgr2=0x562bd64192c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:36.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.516+0000 7fa6cd7af640 1 --2- 172.21.15.64:0/3867388459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bd6072070 0x562bd64192c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.516+0000 7fa6cd7af640 1 -- 172.21.15.64:0/3867388459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd6051390 msgr2=0x562bd6419d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:36.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.516+0000 7fa6cd7af640 1 --2- 172.21.15.64:0/3867388459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd6051390 0x562bd6419d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.516+0000 7fa6cd7af640 1 -- 172.21.15.64:0/3867388459 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bd60dd650 con 0x562bd60491b0 2024-09-21T04:04:36.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.516+0000 7fa6ccfae640 1 --2- 172.21.15.64:0/3867388459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd6051390 0x562bd6419d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:36.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.517+0000 7fa6cd7af640 1 --2- 172.21.15.64:0/3867388459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bd60491b0 0x562bd6419800 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009230 tx=0x7fa6c4032c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c183b7fa5783c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.517+0000 7fa6be7fc640 1 --2- 172.21.15.64:0/615198274 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa6b40b6790 0x7fa6b40b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.517+0000 7fa6ccfae640 1 --2- 172.21.15.64:0/615198274 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa6b40b6790 0x7fa6b40b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.517+0000 7fa6be7fc640 1 -- 172.21.15.64:0/615198274 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fa6c00e75a0 con 0x7fa6c8109e00 2024-09-21T04:04:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.518+0000 7fa6b37fe640 1 -- 172.21.15.64:0/3867388459 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4050070 con 0x562bd60491b0 2024-09-21T04:04:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.518+0000 7fa6b37fe640 1 -- 172.21.15.64:0/3867388459 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6c4032e40 con 0x562bd60491b0 2024-09-21T04:04:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.518+0000 7fa6b37fe640 1 -- 172.21.15.64:0/3867388459 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4035070 con 0x562bd60491b0 2024-09-21T04:04:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.519+0000 7fa6d5f87740 1 -- 172.21.15.64:0/3867388459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bd60491b0 msgr2=0x562bd6419800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.519+0000 7fa6ccfae640 1 --2- 172.21.15.64:0/615198274 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa6b40b6790 0x7fa6b40b8ca0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800bfd0 tx=0x7fa6b8004730 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.519+0000 7fa6d5f87740 1 --2- 172.21.15.64:0/3867388459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bd60491b0 0x562bd6419800 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009230 tx=0x7fa6c4032c50 comp rx=0 tx=0).stop 2024-09-21T04:04:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.519+0000 7fa6d5f87740 1 -- 172.21.15.64:0/3867388459 shutdown_connections 2024-09-21T04:04:36.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.519+0000 7fa6d5f87740 1 --2- 172.21.15.64:0/3867388459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd6051390 0x562bd6419d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.519+0000 7fa6d5f87740 1 --2- 172.21.15.64:0/3867388459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bd60491b0 0x562bd6419800 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.519+0000 7fa6d5f87740 1 --2- 172.21.15.64:0/3867388459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bd6072070 0x562bd64192c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.519+0000 7fa6d5f87740 1 -- 172.21.15.64:0/3867388459 >> 172.21.15.64:0/3867388459 conn(0x562bd6066460 msgr2=0x562bd64171b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:36.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.519+0000 7fa6d5f87740 1 -- 172.21.15.64:0/3867388459 shutdown_connections 2024-09-21T04:04:36.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.519+0000 7fa6d5f87740 1 -- 172.21.15.64:0/3867388459 wait complete. 2024-09-21T04:04:36.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.520+0000 7fa6d5f87740 1 -- start start 2024-09-21T04:04:36.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.520+0000 7fa6d5f87740 1 Processor -- start 2024-09-21T04:04:36.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.521+0000 7fa6d5f87740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd60491b0 0x562bd64249a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:36.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.521+0000 7fa6d5f87740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bd6051390 0x562bd6424ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:36.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.521+0000 7fa6d5f87740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bd6072070 0x562bd6430d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:36.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.521+0000 7fa6cd7af640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd60491b0 0x562bd64249a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:36.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.521+0000 7fa6cd7af640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd60491b0 0x562bd64249a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34452/0 (socket says 172.21.15.64:34452) 2024-09-21T04:04:36.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.521+0000 7fa6cd7af640 1 -- 172.21.15.64:0/2512700450 learned_addr learned my addr 172.21.15.64:0/2512700450 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:36.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.521+0000 7fa6d5f87740 1 -- 172.21.15.64:0/2512700450 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562bd61e0700 con 0x562bd60491b0 2024-09-21T04:04:36.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.521+0000 7fa6d5f87740 1 -- 172.21.15.64:0/2512700450 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562bd631a160 con 0x562bd6072070 2024-09-21T04:04:36.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.521+0000 7fa6d5f87740 1 -- 172.21.15.64:0/2512700450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562bd6427a40 con 0x562bd6051390 2024-09-21T04:04:36.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.521+0000 7fa6ccfae640 1 --2- 172.21.15.64:0/2512700450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bd6051390 0x562bd6424ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:36.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.521+0000 7fa6cdfb0640 1 --2- 172.21.15.64:0/2512700450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bd6072070 0x562bd6430d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:36.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.521+0000 7fa6d5f87740 10 client.0 _ll_register_callbacks cb 0x562bd63339e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:36.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.522+0000 7fa6ccfae640 1 -- 172.21.15.64:0/2512700450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bd6072070 msgr2=0x562bd6430d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:36.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.522+0000 7fa6ccfae640 1 --2- 172.21.15.64:0/2512700450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bd6072070 0x562bd6430d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.522+0000 7fa6ccfae640 1 -- 172.21.15.64:0/2512700450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd60491b0 msgr2=0x562bd64249a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:36.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.522+0000 7fa6ccfae640 1 --2- 172.21.15.64:0/2512700450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd60491b0 0x562bd64249a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.522+0000 7fa6ccfae640 1 -- 172.21.15.64:0/2512700450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bd6347c90 con 0x562bd6051390 2024-09-21T04:04:36.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.522+0000 7fa6cd7af640 1 --2- 172.21.15.64:0/2512700450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd60491b0 0x562bd64249a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:36.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.522+0000 7fa6ccfae640 1 --2- 172.21.15.64:0/2512700450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bd6051390 0x562bd6424ee0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8005fd0 tx=0x7fa6b800b410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93713af30fe7c606 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:36.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.522+0000 7fa6b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-21T04:04:36.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.522+0000 7fa6b0ff9640 1 -- 172.21.15.64:0/2512700450 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b800ee90 con 0x562bd6051390 2024-09-21T04:04:36.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.522+0000 7fa6d5f87740 1 -- 172.21.15.64:0/2512700450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562bd6391df0 con 0x562bd6051390 2024-09-21T04:04:36.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.522+0000 7fa6b0ff9640 1 -- 172.21.15.64:0/2512700450 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fa6b801b030 con 0x562bd6051390 2024-09-21T04:04:36.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.523+0000 7fa6b0ff9640 1 -- 172.21.15.64:0/2512700450 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa6b8021b70 con 0x562bd6051390 2024-09-21T04:04:36.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.523+0000 7fa6d5f87740 10 client.25231 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:36.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.523+0000 7fa6d5f87740 1 -- 172.21.15.64:0/2512700450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562bd6433760 con 0x562bd6051390 2024-09-21T04:04:36.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.523+0000 7fa6b0ff9640 1 -- 172.21.15.64:0/2512700450 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6b8021d10 con 0x562bd6051390 2024-09-21T04:04:36.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.523+0000 7fa6b0ff9640 1 -- 172.21.15.64:0/2512700450 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa6b8005410 con 0x562bd6051390 2024-09-21T04:04:36.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.523+0000 7fa6d5f87740 10 client.25231 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:36.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.523+0000 7fa6d5f87740 10 client.25231 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:04:36.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.523+0000 7fa6d5f87740 20 client.25231 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:36.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.523+0000 7fa6d5f87740 1 --2- 172.21.15.64:0/2512700450 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562bd60309f0 0x562bd6436d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:36.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.523+0000 7fa6d5f87740 4 client.25231 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:04:36.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.523+0000 7fa6d5f87740 1 -- 172.21.15.64:0/2512700450 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bd6391df0 con 0x562bd60309f0 2024-09-21T04:04:36.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.524+0000 7fa6cd7af640 1 --2- 172.21.15.64:0/2512700450 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562bd60309f0 0x562bd6436d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:36.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.524+0000 7fa6b0ff9640 10 client.25231 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:04:36.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.524+0000 7fa6cd7af640 1 --2- 172.21.15.64:0/2512700450 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562bd60309f0 0x562bd6436d20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d542c8c4d20e4330 server_cookie=87c61f95a170f887 in_seq=0 out_seq=0 2024-09-21T04:04:36.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.525+0000 7fa6b0ff9640 1 -- 172.21.15.64:0/2512700450 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562bd6391df0 con 0x562bd60309f0 2024-09-21T04:04:36.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.525+0000 7fa6b0ff9640 10 client.25231 handle_command_reply: tid=0 2024-09-21T04:04:36.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.525+0000 7fa6d5f87740 1 client.25231 shutdown 2024-09-21T04:04:36.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.525+0000 7fa6d5f87740 1 -- 172.21.15.64:0/2512700450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bd6051390 msgr2=0x562bd6424ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:36.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.525+0000 7fa6d5f87740 1 --2- 172.21.15.64:0/2512700450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bd6051390 0x562bd6424ee0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8005fd0 tx=0x7fa6b800b410 comp rx=0 tx=0).stop 2024-09-21T04:04:36.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.525+0000 7fa6d5f87740 1 -- 172.21.15.64:0/2512700450 shutdown_connections 2024-09-21T04:04:36.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.525+0000 7fa6d5f87740 1 --2- 172.21.15.64:0/2512700450 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562bd60309f0 0x562bd6436d20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.525+0000 7fa6d5f87740 1 --2- 172.21.15.64:0/2512700450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bd6072070 0x562bd6430d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.525+0000 7fa6d5f87740 1 --2- 172.21.15.64:0/2512700450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bd6051390 0x562bd6424ee0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.525+0000 7fa6d5f87740 1 --2- 172.21.15.64:0/2512700450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd60491b0 0x562bd64249a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.525+0000 7fa6d5f87740 1 -- 172.21.15.64:0/2512700450 >> 172.21.15.64:0/2512700450 conn(0x562bd6066460 msgr2=0x562bd641c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:36.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.525+0000 7fa6b0ff9640 0 client.25231 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:04:36.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.526+0000 7fa6d5f87740 1 -- 172.21.15.64:0/2512700450 shutdown_connections 2024-09-21T04:04:36.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.526+0000 7fa6d5f87740 1 -- 172.21.15.64:0/2512700450 wait complete. 2024-09-21T04:04:36.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.526+0000 7fa6d5f87740 20 client.25231 trim_cache size 0 max 16384 2024-09-21T04:04:36.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.546+0000 7fa6d5f87740 1 Processor -- start 2024-09-21T04:04:36.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.547+0000 7fa6d5f87740 1 -- start start 2024-09-21T04:04:36.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.547+0000 7fa6d5f87740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bd60491b0 0x562bd6426880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:36.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.547+0000 7fa6d5f87740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bd6051390 0x562bd641aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:36.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.547+0000 7fa6d5f87740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd6072070 0x562bd6430c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:36.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.547+0000 7fa6d5f87740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c81abb40 con 0x562bd6072070 2024-09-21T04:04:36.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.547+0000 7fa6d5f87740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c8073830 con 0x562bd60491b0 2024-09-21T04:04:36.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.547+0000 7fa6d5f87740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c8071680 con 0x562bd6051390 2024-09-21T04:04:36.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.547+0000 7fa6cdfb0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd6072070 0x562bd6430c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:36.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.547+0000 7fa6cdfb0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd6072070 0x562bd6430c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34462/0 (socket says 172.21.15.64:34462) 2024-09-21T04:04:36.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.547+0000 7fa6cdfb0640 1 -- 172.21.15.64:0/3271434647 learned_addr learned my addr 172.21.15.64:0/3271434647 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:36.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.547+0000 7fa6cd7af640 1 --2- 172.21.15.64:0/3271434647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bd60491b0 0x562bd6426880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:36.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.548+0000 7fa6ccfae640 1 --2- 172.21.15.64:0/3271434647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bd6051390 0x562bd641aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:36.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.548+0000 7fa6cdfb0640 1 -- 172.21.15.64:0/3271434647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bd6051390 msgr2=0x562bd641aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:36.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.548+0000 7fa6cdfb0640 1 --2- 172.21.15.64:0/3271434647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bd6051390 0x562bd641aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.548+0000 7fa6cdfb0640 1 -- 172.21.15.64:0/3271434647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bd60491b0 msgr2=0x562bd6426880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:36.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.548+0000 7fa6cdfb0640 1 --2- 172.21.15.64:0/3271434647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bd60491b0 0x562bd6426880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.548+0000 7fa6cdfb0640 1 -- 172.21.15.64:0/3271434647 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bd641c120 con 0x562bd6072070 2024-09-21T04:04:36.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.548+0000 7fa6cd7af640 1 --2- 172.21.15.64:0/3271434647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bd60491b0 0x562bd6426880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:36.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.548+0000 7fa6cdfb0640 1 --2- 172.21.15.64:0/3271434647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd6072070 0x562bd6430c30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000b760 tx=0x7fa6c00acc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b701b396c5a0ed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:36.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.548+0000 7fa6b2ffd640 1 -- 172.21.15.64:0/3271434647 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0020040 con 0x562bd6072070 2024-09-21T04:04:36.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.548+0000 7fa6b2ffd640 1 -- 172.21.15.64:0/3271434647 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6c00acd60 con 0x562bd6072070 2024-09-21T04:04:36.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.548+0000 7fa6b2ffd640 1 -- 172.21.15.64:0/3271434647 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c00a8ba0 con 0x562bd6072070 2024-09-21T04:04:36.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.549+0000 7fa6d5f87740 1 -- 172.21.15.64:0/3271434647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd6072070 msgr2=0x562bd6430c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:36.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.549+0000 7fa6d5f87740 1 --2- 172.21.15.64:0/3271434647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd6072070 0x562bd6430c30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000b760 tx=0x7fa6c00acc60 comp rx=0 tx=0).stop 2024-09-21T04:04:36.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.549+0000 7fa6d5f87740 1 -- 172.21.15.64:0/3271434647 shutdown_connections 2024-09-21T04:04:36.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.549+0000 7fa6d5f87740 1 --2- 172.21.15.64:0/3271434647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd6072070 0x562bd6430c30 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.549+0000 7fa6d5f87740 1 --2- 172.21.15.64:0/3271434647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bd6051390 0x562bd641aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.549+0000 7fa6d5f87740 1 --2- 172.21.15.64:0/3271434647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bd60491b0 0x562bd6426880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.549+0000 7fa6d5f87740 1 -- 172.21.15.64:0/3271434647 >> 172.21.15.64:0/3271434647 conn(0x562bd6066460 msgr2=0x562bd641a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:36.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.549+0000 7fa6d5f87740 1 -- 172.21.15.64:0/3271434647 shutdown_connections 2024-09-21T04:04:36.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.549+0000 7fa6d5f87740 1 -- 172.21.15.64:0/3271434647 wait complete. 2024-09-21T04:04:36.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.550+0000 7fa6d5f87740 1 -- start start 2024-09-21T04:04:36.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.550+0000 7fa6d5f87740 1 Processor -- start 2024-09-21T04:04:36.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.550+0000 7fa6d5f87740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bd60491b0 0x562bd642d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:36.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.551+0000 7fa6d5f87740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd6051390 0x562bd6428b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:36.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.551+0000 7fa6d5f87740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bd6072070 0x562bd64291a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:36.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.551+0000 7fa6d5f87740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c8071cf0 con 0x562bd6051390 2024-09-21T04:04:36.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.551+0000 7fa6ccfae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd6051390 0x562bd6428b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:36.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.551+0000 7fa6cd7af640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bd60491b0 0x562bd642d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:36.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.551+0000 7fa6cd7af640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bd60491b0 0x562bd642d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:43480/0 (socket says 172.21.15.64:43480) 2024-09-21T04:04:36.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.551+0000 7fa6ccfae640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd6051390 0x562bd6428b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34478/0 (socket says 172.21.15.64:34478) 2024-09-21T04:04:36.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.551+0000 7fa6cd7af640 1 -- 172.21.15.64:0/4193601480 learned_addr learned my addr 172.21.15.64:0/4193601480 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:36.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.551+0000 7fa6d5f87740 1 -- 172.21.15.64:0/4193601480 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c8073160 con 0x562bd6072070 2024-09-21T04:04:36.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.551+0000 7fa6d5f87740 1 -- 172.21.15.64:0/4193601480 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c80723c0 con 0x562bd60491b0 2024-09-21T04:04:36.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.551+0000 7fa6cdfb0640 1 --2- 172.21.15.64:0/4193601480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bd6072070 0x562bd64291a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:36.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.551+0000 7fa6d5f87740 10 client.0 _ll_register_callbacks cb 0x7fa6b8012d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:36.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.551+0000 7fa6cdfb0640 1 -- 172.21.15.64:0/4193601480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bd60491b0 msgr2=0x562bd642d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:36.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.551+0000 7fa6cdfb0640 1 --2- 172.21.15.64:0/4193601480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bd60491b0 0x562bd642d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.551+0000 7fa6cdfb0640 1 -- 172.21.15.64:0/4193601480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd6051390 msgr2=0x562bd6428b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:36.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.551+0000 7fa6cdfb0640 1 --2- 172.21.15.64:0/4193601480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd6051390 0x562bd6428b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.551+0000 7fa6cdfb0640 1 -- 172.21.15.64:0/4193601480 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bd6429910 con 0x562bd6072070 2024-09-21T04:04:36.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.552+0000 7fa6cdfb0640 1 --2- 172.21.15.64:0/4193601480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bd6072070 0x562bd64291a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009a30 tx=0x7fa6c0004480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db35fda57e6a73af server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:36.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.552+0000 7fa6b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:04:36.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.552+0000 7fa6b0ff9640 1 -- 172.21.15.64:0/4193601480 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0020070 con 0x562bd6072070 2024-09-21T04:04:36.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.552+0000 7fa6ccfae640 1 --2- 172.21.15.64:0/4193601480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd6051390 0x562bd6428b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:36.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.552+0000 7fa6b0ff9640 1 -- 172.21.15.64:0/4193601480 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fa6c00d0dd0 con 0x562bd6072070 2024-09-21T04:04:36.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.552+0000 7fa6d5f87740 1 -- 172.21.15.64:0/4193601480 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6c00acd60 con 0x562bd6072070 2024-09-21T04:04:36.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.552+0000 7fa6b0ff9640 1 -- 172.21.15.64:0/4193601480 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa6c00e50c0 con 0x562bd6072070 2024-09-21T04:04:36.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.552+0000 7fa6d5f87740 10 client.25313 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:36.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.552+0000 7fa6d5f87740 1 -- 172.21.15.64:0/4193601480 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562bd64a9f70 con 0x562bd6072070 2024-09-21T04:04:36.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.552+0000 7fa6b0ff9640 1 -- 172.21.15.64:0/4193601480 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6c00b1d30 con 0x562bd6072070 2024-09-21T04:04:36.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.553+0000 7fa6b0ff9640 1 -- 172.21.15.64:0/4193601480 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa6c00a8cc0 con 0x562bd6072070 2024-09-21T04:04:36.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.553+0000 7fa6d5f87740 10 client.25313 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:36.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.553+0000 7fa6d5f87740 10 client.25313 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:04:36.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.553+0000 7fa6d5f87740 20 client.25313 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:36.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.553+0000 7fa6d5f87740 1 --2- 172.21.15.64:0/4193601480 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562bd6005da0 0x562bd64ac620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:36.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.553+0000 7fa6d5f87740 4 client.25313 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:04:36.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.553+0000 7fa6d5f87740 1 -- 172.21.15.64:0/4193601480 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa6c00acd60 con 0x562bd6005da0 2024-09-21T04:04:36.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.553+0000 7fa6cd7af640 1 --2- 172.21.15.64:0/4193601480 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562bd6005da0 0x562bd64ac620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:36.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.554+0000 7fa6cd7af640 1 --2- 172.21.15.64:0/4193601480 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562bd6005da0 0x562bd64ac620 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2abfb79af66012f4 server_cookie=112200fd5a565501 in_seq=0 out_seq=0 2024-09-21T04:04:36.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.554+0000 7fa6b0ff9640 10 client.25313 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:04:36.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.554+0000 7fa6b0ff9640 1 -- 172.21.15.64:0/4193601480 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa6c00acd60 con 0x562bd6005da0 2024-09-21T04:04:36.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.554+0000 7fa6b0ff9640 10 client.25313 handle_command_reply: tid=0 2024-09-21T04:04:36.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.554+0000 7fa6d5f87740 1 client.25313 shutdown 2024-09-21T04:04:36.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.554+0000 7fa6d5f87740 1 -- 172.21.15.64:0/4193601480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bd6072070 msgr2=0x562bd64291a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:36.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.554+0000 7fa6d5f87740 1 --2- 172.21.15.64:0/4193601480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bd6072070 0x562bd64291a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009a30 tx=0x7fa6c0004480 comp rx=0 tx=0).stop 2024-09-21T04:04:36.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.555+0000 7fa6d5f87740 1 -- 172.21.15.64:0/4193601480 shutdown_connections 2024-09-21T04:04:36.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.555+0000 7fa6d5f87740 1 --2- 172.21.15.64:0/4193601480 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562bd6005da0 0x562bd64ac620 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.555+0000 7fa6b0ff9640 0 client.25313 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:04:36.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.555+0000 7fa6d5f87740 1 --2- 172.21.15.64:0/4193601480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562bd6072070 0x562bd64291a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.555+0000 7fa6d5f87740 1 --2- 172.21.15.64:0/4193601480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562bd6051390 0x562bd6428b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.555+0000 7fa6d5f87740 1 --2- 172.21.15.64:0/4193601480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bd60491b0 0x562bd642d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.555+0000 7fa6d5f87740 1 -- 172.21.15.64:0/4193601480 >> 172.21.15.64:0/4193601480 conn(0x562bd6066460 msgr2=0x562bd6426810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:36.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.555+0000 7fa6d5f87740 1 -- 172.21.15.64:0/4193601480 shutdown_connections 2024-09-21T04:04:36.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.555+0000 7fa6d5f87740 1 -- 172.21.15.64:0/4193601480 wait complete. 2024-09-21T04:04:36.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.555+0000 7fa6d5f87740 20 client.25313 trim_cache size 0 max 16384 2024-09-21T04:04:36.561 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:04:36.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.556+0000 7fa6cfa13640 1 -- 172.21.15.64:0/615198274 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa6b40b6790 msgr2=0x7fa6b40b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:36.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.556+0000 7fa6cfa13640 1 --2- 172.21.15.64:0/615198274 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa6b40b6790 0x7fa6b40b8ca0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800bfd0 tx=0x7fa6b8004730 comp rx=0 tx=0).stop 2024-09-21T04:04:36.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.556+0000 7fa6cfa13640 1 -- 172.21.15.64:0/615198274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6c8109e00 msgr2=0x7fa6c819fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:36.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.556+0000 7fa6cfa13640 1 --2- 172.21.15.64:0/615198274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6c8109e00 0x7fa6c819fd50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009fd0 tx=0x7fa6c000b260 comp rx=0 tx=0).stop 2024-09-21T04:04:36.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.556+0000 7fa6cfa13640 1 -- 172.21.15.64:0/615198274 shutdown_connections 2024-09-21T04:04:36.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.556+0000 7fa6cfa13640 1 --2- 172.21.15.64:0/615198274 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa6b40b6790 0x7fa6b40b8ca0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.556+0000 7fa6cfa13640 1 --2- 172.21.15.64:0/615198274 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6c8109e00 0x7fa6c819fd50 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.556+0000 7fa6cfa13640 1 --2- 172.21.15.64:0/615198274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6c81094b0 0x7fa6c819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.556+0000 7fa6cfa13640 1 --2- 172.21.15.64:0/615198274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6c8101f70 0x7fa6c819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:36.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.556+0000 7fa6cfa13640 1 -- 172.21.15.64:0/615198274 >> 172.21.15.64:0/615198274 conn(0x7fa6c8077c60 msgr2=0x7fa6c81075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:36.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.556+0000 7fa6cfa13640 1 -- 172.21.15.64:0/615198274 shutdown_connections 2024-09-21T04:04:36.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:36.556+0000 7fa6cfa13640 1 -- 172.21.15.64:0/615198274 wait complete. 2024-09-21T04:04:37.275 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T04:04:37.639 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:04:37.895 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:37 smithi064 ceph-mon[31180]: pgmap v339: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T04:04:38.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:37 smithi119 ceph-mon[34811]: pgmap v339: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T04:04:38.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:37 smithi131 ceph-mon[35193]: pgmap v339: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T04:04:38.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.210+0000 7fadb089f640 1 -- 172.21.15.64:0/3230027630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadac108d30 msgr2=0x7fadac10f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:38.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.210+0000 7fadb089f640 1 --2- 172.21.15.64:0/3230027630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadac108d30 0x7fadac10f560 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fada4009ca0 tx=0x7fada402a8c0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.210+0000 7fadb089f640 1 -- 172.21.15.64:0/3230027630 shutdown_connections 2024-09-21T04:04:38.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.210+0000 7fadb089f640 1 --2- 172.21.15.64:0/3230027630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadac108d30 0x7fadac10f560 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.210+0000 7fadb089f640 1 --2- 172.21.15.64:0/3230027630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadac1083e0 0x7fadac1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.210+0000 7fadb089f640 1 --2- 172.21.15.64:0/3230027630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadac102aa0 0x7fadac102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.210+0000 7fadb089f640 1 -- 172.21.15.64:0/3230027630 >> 172.21.15.64:0/3230027630 conn(0x7fadac079750 msgr2=0x7fadac079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:38.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.210+0000 7fadb089f640 1 -- 172.21.15.64:0/3230027630 shutdown_connections 2024-09-21T04:04:38.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.210+0000 7fadb089f640 1 -- 172.21.15.64:0/3230027630 wait complete. 2024-09-21T04:04:38.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.211+0000 7fadb089f640 1 Processor -- start 2024-09-21T04:04:38.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.212+0000 7fadb089f640 1 -- start start 2024-09-21T04:04:38.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.212+0000 7fadb089f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadac102aa0 0x7fadac1a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:38.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.212+0000 7fadb089f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadac1083e0 0x7fadac1a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:38.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.212+0000 7fadb089f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadac108d30 0x7fadac1a6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:38.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.212+0000 7fadb089f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fadac1118f0 con 0x7fadac1083e0 2024-09-21T04:04:38.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.212+0000 7fadb089f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fadac111770 con 0x7fadac102aa0 2024-09-21T04:04:38.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.212+0000 7fadb089f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fadac111a70 con 0x7fadac108d30 2024-09-21T04:04:38.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.212+0000 7fadaa59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadac102aa0 0x7fadac1a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:38.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.212+0000 7fadaa59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadac102aa0 0x7fadac1a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:51924/0 (socket says 172.21.15.64:51924) 2024-09-21T04:04:38.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.212+0000 7fadaa59c640 1 -- 172.21.15.64:0/1732738917 learned_addr learned my addr 172.21.15.64:0/1732738917 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:38.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.212+0000 7fada3fff640 1 --2- 172.21.15.64:0/1732738917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadac1083e0 0x7fadac1a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:38.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.213+0000 7fadaad9d640 1 --2- 172.21.15.64:0/1732738917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadac108d30 0x7fadac1a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:38.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.213+0000 7fadaa59c640 1 -- 172.21.15.64:0/1732738917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadac108d30 msgr2=0x7fadac1a6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:38.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.213+0000 7fadaa59c640 1 --2- 172.21.15.64:0/1732738917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadac108d30 0x7fadac1a6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.213+0000 7fadaa59c640 1 -- 172.21.15.64:0/1732738917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadac1083e0 msgr2=0x7fadac1a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:38.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.213+0000 7fadaa59c640 1 --2- 172.21.15.64:0/1732738917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadac1083e0 0x7fadac1a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.213+0000 7fadaa59c640 1 -- 172.21.15.64:0/1732738917 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadac1a6a80 con 0x7fadac102aa0 2024-09-21T04:04:38.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.213+0000 7fadaad9d640 1 --2- 172.21.15.64:0/1732738917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadac108d30 0x7fadac1a6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:04:38.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.213+0000 7fada3fff640 1 --2- 172.21.15.64:0/1732738917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadac1083e0 0x7fadac1a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:04:38.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.213+0000 7fadaa59c640 1 --2- 172.21.15.64:0/1732738917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadac102aa0 0x7fadac1a0b10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fad9c009990 tx=0x7fad9c009e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:38.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.214+0000 7fada37fe640 1 -- 172.21.15.64:0/1732738917 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c00ca80 con 0x7fadac102aa0 2024-09-21T04:04:38.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.214+0000 7fadb089f640 1 -- 172.21.15.64:0/1732738917 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadac1a6d70 con 0x7fadac102aa0 2024-09-21T04:04:38.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.214+0000 7fada37fe640 1 -- 172.21.15.64:0/1732738917 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad9c00cc20 con 0x7fadac102aa0 2024-09-21T04:04:38.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.214+0000 7fadb089f640 1 -- 172.21.15.64:0/1732738917 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadac1ad4d0 con 0x7fadac102aa0 2024-09-21T04:04:38.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.214+0000 7fada37fe640 1 -- 172.21.15.64:0/1732738917 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c003ce0 con 0x7fadac102aa0 2024-09-21T04:04:38.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.218+0000 7fadb6e13740 1 Processor -- start 2024-09-21T04:04:38.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.218+0000 7fadb6e13740 1 -- start start 2024-09-21T04:04:38.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.219+0000 7fadb6e13740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620312f1070 0x5620316982c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:38.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.219+0000 7fadb6e13740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5620312c81b0 0x562031698800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:38.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.219+0000 7fadaad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620312f1070 0x5620316982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:38.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.219+0000 7fadaad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620312f1070 0x5620316982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34492/0 (socket says 172.21.15.64:34492) 2024-09-21T04:04:38.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.219+0000 7fadb6e13740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620312d0390 0x562031698d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:38.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.219+0000 7fada37fe640 1 -- 172.21.15.64:0/1732738917 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 25) ==== 154694+0+0 (secure 0 0 0) 0x7fad9c016410 con 0x7fadac102aa0 2024-09-21T04:04:38.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.219+0000 7fadaad9d640 1 -- 172.21.15.64:0/1444610708 learned_addr learned my addr 172.21.15.64:0/1444610708 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:38.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.219+0000 7fadb6e13740 1 -- 172.21.15.64:0/1444610708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56203160eea0 con 0x5620312f1070 2024-09-21T04:04:38.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.219+0000 7fadb6e13740 1 -- 172.21.15.64:0/1444610708 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5620315fe2d0 con 0x5620312c81b0 2024-09-21T04:04:38.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.220+0000 7fadb6e13740 1 -- 172.21.15.64:0/1444610708 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5620316020e0 con 0x5620312d0390 2024-09-21T04:04:38.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.220+0000 7fadaa59c640 1 --2- 172.21.15.64:0/1444610708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5620312c81b0 0x562031698800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:38.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.220+0000 7fada3fff640 1 --2- 172.21.15.64:0/1444610708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620312d0390 0x562031698d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:38.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.220+0000 7fadaad9d640 1 -- 172.21.15.64:0/1444610708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620312d0390 msgr2=0x562031698d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:38.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.220+0000 7fadaad9d640 1 --2- 172.21.15.64:0/1444610708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620312d0390 0x562031698d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.220+0000 7fadaad9d640 1 -- 172.21.15.64:0/1444610708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5620312c81b0 msgr2=0x562031698800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:38.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.220+0000 7fadaad9d640 1 --2- 172.21.15.64:0/1444610708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5620312c81b0 0x562031698800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.220+0000 7fadaad9d640 1 -- 172.21.15.64:0/1444610708 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56203135c650 con 0x5620312f1070 2024-09-21T04:04:38.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.220+0000 7fadaad9d640 1 --2- 172.21.15.64:0/1444610708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620312f1070 0x5620316982c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fada4009470 tx=0x7fada40094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a030086ce2e7e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:38.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.221+0000 7fada0ff9640 1 -- 172.21.15.64:0/1444610708 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4031cf0 con 0x5620312f1070 2024-09-21T04:04:38.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.221+0000 7fada0ff9640 1 -- 172.21.15.64:0/1444610708 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fada4036040 con 0x5620312f1070 2024-09-21T04:04:38.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.221+0000 7fada0ff9640 1 -- 172.21.15.64:0/1444610708 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada40042a0 con 0x5620312f1070 2024-09-21T04:04:38.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.222+0000 7fada37fe640 1 --2- 172.21.15.64:0/1732738917 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fad980b6710 0x7fad980b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:38.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.222+0000 7fada37fe640 1 -- 172.21.15.64:0/1732738917 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fad9c0e6de0 con 0x7fadac102aa0 2024-09-21T04:04:38.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.223+0000 7fadb6e13740 1 -- 172.21.15.64:0/1444610708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620312f1070 msgr2=0x5620316982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:38.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.223+0000 7fadb6e13740 1 --2- 172.21.15.64:0/1444610708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620312f1070 0x5620316982c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fada4009470 tx=0x7fada40094a0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.223+0000 7fadb6e13740 1 -- 172.21.15.64:0/1444610708 shutdown_connections 2024-09-21T04:04:38.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.223+0000 7fadb6e13740 1 --2- 172.21.15.64:0/1444610708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620312d0390 0x562031698d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.223+0000 7fadb6e13740 1 --2- 172.21.15.64:0/1444610708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5620312c81b0 0x562031698800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.223+0000 7fadb6e13740 1 --2- 172.21.15.64:0/1444610708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620312f1070 0x5620316982c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.223+0000 7fadb6e13740 1 -- 172.21.15.64:0/1444610708 >> 172.21.15.64:0/1444610708 conn(0x5620312e5460 msgr2=0x5620316961b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:38.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.223+0000 7fadaa59c640 1 --2- 172.21.15.64:0/1732738917 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fad980b6710 0x7fad980b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:38.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.223+0000 7fadb6e13740 1 -- 172.21.15.64:0/1444610708 shutdown_connections 2024-09-21T04:04:38.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.224+0000 7fadb6e13740 1 -- 172.21.15.64:0/1444610708 wait complete. 2024-09-21T04:04:38.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.224+0000 7fadaa59c640 1 --2- 172.21.15.64:0/1732738917 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fad980b6710 0x7fad980b8c20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0e44a0 tx=0x7fad9c000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:38.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.224+0000 7fadb6e13740 1 -- start start 2024-09-21T04:04:38.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.224+0000 7fadb6e13740 1 Processor -- start 2024-09-21T04:04:38.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.225+0000 7fadb6e13740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5620312c81b0 0x5620316a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:38.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.225+0000 7fadb6e13740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620312d0390 0x5620316a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:38.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.225+0000 7fadb6e13740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620312f1070 0x5620316afd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:38.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.225+0000 7fadb6e13740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56203145f700 con 0x5620312f1070 2024-09-21T04:04:38.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.225+0000 7fadb6e13740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562031599160 con 0x5620312c81b0 2024-09-21T04:04:38.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.225+0000 7fadb6e13740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5620316a6a40 con 0x5620312d0390 2024-09-21T04:04:38.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.225+0000 7fadb6e13740 10 client.0 _ll_register_callbacks cb 0x5620315b29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:38.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.225+0000 7fadaad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620312f1070 0x5620316afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:38.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.225+0000 7fadaad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5620312c81b0 0x5620316a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:38.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.225+0000 7fada3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620312d0390 0x5620316a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:38.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.225+0000 7fadaad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620312f1070 0x5620316afd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34502/0 (socket says 172.21.15.64:34502) 2024-09-21T04:04:38.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.225+0000 7fadaad9d640 1 -- 172.21.15.64:0/425028039 learned_addr learned my addr 172.21.15.64:0/425028039 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:38.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.226+0000 7fada3fff640 1 -- 172.21.15.64:0/425028039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5620312c81b0 msgr2=0x5620316a39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:38.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.226+0000 7fada3fff640 1 --2- 172.21.15.64:0/425028039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5620312c81b0 0x5620316a39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.226+0000 7fada3fff640 1 -- 172.21.15.64:0/425028039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620312f1070 msgr2=0x5620316afd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:38.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.226+0000 7fada3fff640 1 --2- 172.21.15.64:0/425028039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620312f1070 0x5620316afd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.226+0000 7fada3fff640 1 -- 172.21.15.64:0/425028039 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620315c6c90 con 0x5620312d0390 2024-09-21T04:04:38.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.226+0000 7fadaad9d640 1 --2- 172.21.15.64:0/425028039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5620312c81b0 0x5620316a39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:38.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.226+0000 7fada3fff640 1 --2- 172.21.15.64:0/425028039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620312d0390 0x5620316a3ef0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fadac06a890 tx=0x7fad94013bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=537a5b303e0989bf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:38.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.226+0000 7fad91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-21T04:04:38.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.226+0000 7fad91ffb640 1 -- 172.21.15.64:0/425028039 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94020060 con 0x5620312d0390 2024-09-21T04:04:38.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.226+0000 7fad91ffb640 1 -- 172.21.15.64:0/425028039 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fad94011da0 con 0x5620312d0390 2024-09-21T04:04:38.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.226+0000 7fadb6e13740 1 -- 172.21.15.64:0/425028039 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fada4036040 con 0x5620312d0390 2024-09-21T04:04:38.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.227+0000 7fad91ffb640 1 -- 172.21.15.64:0/425028039 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fad9400f450 con 0x5620312d0390 2024-09-21T04:04:38.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.227+0000 7fadb6e13740 10 client.25249 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:38.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.227+0000 7fadb6e13740 1 -- 172.21.15.64:0/425028039 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5620316b2810 con 0x5620312d0390 2024-09-21T04:04:38.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.227+0000 7fad91ffb640 1 -- 172.21.15.64:0/425028039 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad9400faa0 con 0x5620312d0390 2024-09-21T04:04:38.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.228+0000 7fad91ffb640 1 -- 172.21.15.64:0/425028039 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fad9401a030 con 0x5620312d0390 2024-09-21T04:04:38.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.228+0000 7fadb6e13740 10 client.25249 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:38.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.228+0000 7fadb6e13740 10 client.25249 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:04:38.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.229+0000 7fadb6e13740 20 client.25249 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:38.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.229+0000 7fadb6e13740 1 --2- 172.21.15.64:0/425028039 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5620312af9f0 0x5620316b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:38.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.229+0000 7fadb6e13740 4 client.25249 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:04:38.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.229+0000 7fadb6e13740 1 -- 172.21.15.64:0/425028039 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fada4036040 con 0x5620312af9f0 2024-09-21T04:04:38.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.229+0000 7fadaad9d640 1 --2- 172.21.15.64:0/425028039 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5620312af9f0 0x5620316b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:38.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.230+0000 7fad91ffb640 10 client.25249 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:04:38.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.230+0000 7fadaad9d640 1 --2- 172.21.15.64:0/425028039 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5620312af9f0 0x5620316b5d00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=25257acc6aac8c5e server_cookie=c8b656ecd333843b in_seq=0 out_seq=0 2024-09-21T04:04:38.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.231+0000 7fad91ffb640 1 -- 172.21.15.64:0/425028039 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fada4036040 con 0x5620312af9f0 2024-09-21T04:04:38.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.231+0000 7fad91ffb640 10 client.25249 handle_command_reply: tid=0 2024-09-21T04:04:38.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.231+0000 7fadb6e13740 1 client.25249 shutdown 2024-09-21T04:04:38.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.231+0000 7fadb6e13740 1 -- 172.21.15.64:0/425028039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620312d0390 msgr2=0x5620316a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:38.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.231+0000 7fadb6e13740 1 --2- 172.21.15.64:0/425028039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620312d0390 0x5620316a3ef0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fadac06a890 tx=0x7fad94013bf0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.231+0000 7fadb6e13740 1 -- 172.21.15.64:0/425028039 shutdown_connections 2024-09-21T04:04:38.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.231+0000 7fadb6e13740 1 --2- 172.21.15.64:0/425028039 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5620312af9f0 0x5620316b5d00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.231+0000 7fadb6e13740 1 --2- 172.21.15.64:0/425028039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620312f1070 0x5620316afd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.231+0000 7fadb6e13740 1 --2- 172.21.15.64:0/425028039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620312d0390 0x5620316a3ef0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.231+0000 7fadb6e13740 1 --2- 172.21.15.64:0/425028039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5620312c81b0 0x5620316a39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.231+0000 7fad91ffb640 0 client.25249 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:04:38.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.231+0000 7fadb6e13740 1 -- 172.21.15.64:0/425028039 >> 172.21.15.64:0/425028039 conn(0x5620312e5460 msgr2=0x56203169b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:38.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.231+0000 7fadb6e13740 1 -- 172.21.15.64:0/425028039 shutdown_connections 2024-09-21T04:04:38.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.231+0000 7fadb6e13740 1 -- 172.21.15.64:0/425028039 wait complete. 2024-09-21T04:04:38.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.231+0000 7fadb6e13740 20 client.25249 trim_cache size 0 max 16384 2024-09-21T04:04:38.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.258+0000 7fadb6e13740 1 Processor -- start 2024-09-21T04:04:38.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.258+0000 7fadb6e13740 1 -- start start 2024-09-21T04:04:38.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.258+0000 7fadb6e13740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5620312af9f0 0x5620316a3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:38.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.258+0000 7fadb6e13740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620312c81b0 0x56203169b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:38.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.258+0000 7fadb6e13740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620312d0390 0x56203169b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:38.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.258+0000 7fadb6e13740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fad8c0111e0 con 0x5620312c81b0 2024-09-21T04:04:38.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.258+0000 7fadb6e13740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fad8c00eab0 con 0x5620312af9f0 2024-09-21T04:04:38.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.258+0000 7fadb6e13740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fad8c00c900 con 0x5620312d0390 2024-09-21T04:04:38.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.259+0000 7fadaad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5620312af9f0 0x5620316a3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:38.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.259+0000 7fada3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620312c81b0 0x56203169b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:38.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.259+0000 7fadaad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5620312af9f0 0x5620316a3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:51948/0 (socket says 172.21.15.64:51948) 2024-09-21T04:04:38.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.259+0000 7fadaad9d640 1 -- 172.21.15.64:0/1133117514 learned_addr learned my addr 172.21.15.64:0/1133117514 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:38.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.259+0000 7fadaad9d640 1 --2- 172.21.15.64:0/1133117514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620312d0390 0x56203169b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:38.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.259+0000 7fada3fff640 1 -- 172.21.15.64:0/1133117514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620312d0390 msgr2=0x56203169b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:38.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.259+0000 7fada3fff640 1 --2- 172.21.15.64:0/1133117514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620312d0390 0x56203169b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.259+0000 7fada3fff640 1 -- 172.21.15.64:0/1133117514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5620312af9f0 msgr2=0x5620316a3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:38.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.259+0000 7fada3fff640 1 --2- 172.21.15.64:0/1133117514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5620312af9f0 0x5620316a3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.259+0000 7fada3fff640 1 -- 172.21.15.64:0/1133117514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620316a1eb0 con 0x5620312c81b0 2024-09-21T04:04:38.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.259+0000 7fadaad9d640 1 --2- 172.21.15.64:0/1133117514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5620312af9f0 0x5620316a3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:38.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.260+0000 7fada3fff640 1 --2- 172.21.15.64:0/1133117514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620312c81b0 0x56203169b3c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fad94009c00 tx=0x7fad9401bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2473574b8f038703 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:38.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.260+0000 7fad93fff640 1 -- 172.21.15.64:0/1133117514 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94035070 con 0x5620312c81b0 2024-09-21T04:04:38.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.260+0000 7fad93fff640 1 -- 172.21.15.64:0/1133117514 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad94011da0 con 0x5620312c81b0 2024-09-21T04:04:38.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.260+0000 7fad93fff640 1 -- 172.21.15.64:0/1133117514 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9402a080 con 0x5620312c81b0 2024-09-21T04:04:38.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.261+0000 7fadb6e13740 1 -- 172.21.15.64:0/1133117514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620312c81b0 msgr2=0x56203169b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:38.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.261+0000 7fadb6e13740 1 --2- 172.21.15.64:0/1133117514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620312c81b0 0x56203169b3c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fad94009c00 tx=0x7fad9401bd80 comp rx=0 tx=0).stop 2024-09-21T04:04:38.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.261+0000 7fadb6e13740 1 -- 172.21.15.64:0/1133117514 shutdown_connections 2024-09-21T04:04:38.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.261+0000 7fadb6e13740 1 --2- 172.21.15.64:0/1133117514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620312d0390 0x56203169b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.261+0000 7fadb6e13740 1 --2- 172.21.15.64:0/1133117514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620312c81b0 0x56203169b3c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.261+0000 7fadb6e13740 1 --2- 172.21.15.64:0/1133117514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5620312af9f0 0x5620316a3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.261+0000 7fadb6e13740 1 -- 172.21.15.64:0/1133117514 >> 172.21.15.64:0/1133117514 conn(0x5620312e5460 msgr2=0x5620316a3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:38.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.261+0000 7fadb6e13740 1 -- 172.21.15.64:0/1133117514 shutdown_connections 2024-09-21T04:04:38.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.261+0000 7fadb6e13740 1 -- 172.21.15.64:0/1133117514 wait complete. 2024-09-21T04:04:38.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.261+0000 7fadb6e13740 1 -- start start 2024-09-21T04:04:38.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.262+0000 7fadb6e13740 1 Processor -- start 2024-09-21T04:04:38.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.262+0000 7fadb6e13740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5620312af9f0 0x5620316a7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:38.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.262+0000 7fadb6e13740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620312c81b0 0x5620316a8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:38.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.262+0000 7fada3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620312c81b0 0x5620316a8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:38.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.263+0000 7fadaad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5620312af9f0 0x5620316a7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:38.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.263+0000 7fada3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620312c81b0 0x5620316a8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34518/0 (socket says 172.21.15.64:34518) 2024-09-21T04:04:38.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.263+0000 7fada3fff640 1 -- 172.21.15.64:0/3289065580 learned_addr learned my addr 172.21.15.64:0/3289065580 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:38.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.263+0000 7fadb6e13740 1 --2- 172.21.15.64:0/3289065580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620312d0390 0x5620316a8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:38.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.263+0000 7fadb6e13740 1 -- 172.21.15.64:0/3289065580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fad8c00cf70 con 0x5620312c81b0 2024-09-21T04:04:38.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.263+0000 7fadb6e13740 1 -- 172.21.15.64:0/3289065580 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fad8c00e3e0 con 0x5620312af9f0 2024-09-21T04:04:38.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.263+0000 7fadb6e13740 1 -- 172.21.15.64:0/3289065580 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fad8c00d640 con 0x5620312d0390 2024-09-21T04:04:38.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.263+0000 7fadb6e13740 10 client.0 _ll_register_callbacks cb 0x7fad94009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:38.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.263+0000 7fadaad9d640 1 --2- 172.21.15.64:0/3289065580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620312d0390 0x5620316a8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:38.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.263+0000 7fada3fff640 1 -- 172.21.15.64:0/3289065580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620312d0390 msgr2=0x5620316a8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:38.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.263+0000 7fada3fff640 1 --2- 172.21.15.64:0/3289065580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620312d0390 0x5620316a8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.263+0000 7fada3fff640 1 -- 172.21.15.64:0/3289065580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5620312af9f0 msgr2=0x5620316a7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:38.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.263+0000 7fada3fff640 1 --2- 172.21.15.64:0/3289065580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5620312af9f0 0x5620316a7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.263+0000 7fada3fff640 1 -- 172.21.15.64:0/3289065580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56203171f6c0 con 0x5620312c81b0 2024-09-21T04:04:38.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.263+0000 7fadaad9d640 1 --2- 172.21.15.64:0/3289065580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620312d0390 0x5620316a8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:04:38.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.263+0000 7fada3fff640 1 --2- 172.21.15.64:0/3289065580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620312c81b0 0x5620316a8130 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fad940056f0 tx=0x7fad94000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a264281c341f93 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:38.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.263+0000 7fad91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:04:38.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.264+0000 7fad91ffb640 1 -- 172.21.15.64:0/3289065580 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad94035030 con 0x5620312c81b0 2024-09-21T04:04:38.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.264+0000 7fad91ffb640 1 -- 172.21.15.64:0/3289065580 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fad94011ba0 con 0x5620312c81b0 2024-09-21T04:04:38.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.264+0000 7fadb6e13740 1 -- 172.21.15.64:0/3289065580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad94011da0 con 0x5620312c81b0 2024-09-21T04:04:38.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.264+0000 7fad91ffb640 1 -- 172.21.15.64:0/3289065580 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fad9402a040 con 0x5620312c81b0 2024-09-21T04:04:38.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.264+0000 7fadb6e13740 10 client.15585 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:38.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.264+0000 7fadb6e13740 1 -- 172.21.15.64:0/3289065580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562031720840 con 0x5620312c81b0 2024-09-21T04:04:38.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.265+0000 7fad91ffb640 1 -- 172.21.15.64:0/3289065580 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad94011da0 con 0x5620312c81b0 2024-09-21T04:04:38.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.265+0000 7fad91ffb640 1 -- 172.21.15.64:0/3289065580 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fad94041020 con 0x5620312c81b0 2024-09-21T04:04:38.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.265+0000 7fadb6e13740 10 client.15585 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.265+0000 7fadb6e13740 10 client.15585 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:04:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.265+0000 7fadb6e13740 20 client.15585 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.265+0000 7fadb6e13740 1 --2- 172.21.15.64:0/3289065580 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5620312f1070 0x562031722ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.265+0000 7fadb6e13740 4 client.15585 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:04:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.265+0000 7fadb6e13740 1 -- 172.21.15.64:0/3289065580 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562031610df0 con 0x5620312f1070 2024-09-21T04:04:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.265+0000 7fadaad9d640 1 --2- 172.21.15.64:0/3289065580 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5620312f1070 0x562031722ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.266+0000 7fad91ffb640 10 client.15585 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:04:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.266+0000 7fadaad9d640 1 --2- 172.21.15.64:0/3289065580 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5620312f1070 0x562031722ec0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5ee88b3cee79de95 server_cookie=b345399fbe824d54 in_seq=0 out_seq=0 2024-09-21T04:04:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.266+0000 7fad91ffb640 1 -- 172.21.15.64:0/3289065580 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562031610df0 con 0x5620312f1070 2024-09-21T04:04:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.266+0000 7fad91ffb640 10 client.15585 handle_command_reply: tid=0 2024-09-21T04:04:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.266+0000 7fadb6e13740 1 client.15585 shutdown 2024-09-21T04:04:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.267+0000 7fadb6e13740 1 -- 172.21.15.64:0/3289065580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620312c81b0 msgr2=0x5620316a8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.267+0000 7fadb6e13740 1 --2- 172.21.15.64:0/3289065580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620312c81b0 0x5620316a8130 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fad940056f0 tx=0x7fad94000bc0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.267+0000 7fadb6e13740 1 -- 172.21.15.64:0/3289065580 shutdown_connections 2024-09-21T04:04:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.267+0000 7fadb6e13740 1 --2- 172.21.15.64:0/3289065580 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5620312f1070 0x562031722ec0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.267+0000 7fadb6e13740 1 --2- 172.21.15.64:0/3289065580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620312d0390 0x5620316a8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.267+0000 7fadb6e13740 1 --2- 172.21.15.64:0/3289065580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620312c81b0 0x5620316a8130 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.267+0000 7fadb6e13740 1 --2- 172.21.15.64:0/3289065580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5620312af9f0 0x5620316a7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.267+0000 7fadb6e13740 1 -- 172.21.15.64:0/3289065580 >> 172.21.15.64:0/3289065580 conn(0x5620312e5460 msgr2=0x56203169b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.267+0000 7fad91ffb640 0 client.15585 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:04:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.267+0000 7fadb6e13740 1 -- 172.21.15.64:0/3289065580 shutdown_connections 2024-09-21T04:04:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.267+0000 7fadb6e13740 1 -- 172.21.15.64:0/3289065580 wait complete. 2024-09-21T04:04:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.267+0000 7fadb6e13740 20 client.15585 trim_cache size 0 max 16384 2024-09-21T04:04:38.271 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:04:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.268+0000 7fadb089f640 1 -- 172.21.15.64:0/1732738917 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fad980b6710 msgr2=0x7fad980b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.268+0000 7fadb089f640 1 --2- 172.21.15.64:0/1732738917 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fad980b6710 0x7fad980b8c20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0e44a0 tx=0x7fad9c000bc0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.268+0000 7fadb089f640 1 -- 172.21.15.64:0/1732738917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadac102aa0 msgr2=0x7fadac1a0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.268+0000 7fadb089f640 1 --2- 172.21.15.64:0/1732738917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadac102aa0 0x7fadac1a0b10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fad9c009990 tx=0x7fad9c009e50 comp rx=0 tx=0).stop 2024-09-21T04:04:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.268+0000 7fadb089f640 1 -- 172.21.15.64:0/1732738917 shutdown_connections 2024-09-21T04:04:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.268+0000 7fadb089f640 1 --2- 172.21.15.64:0/1732738917 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fad980b6710 0x7fad980b8c20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.268+0000 7fadb089f640 1 --2- 172.21.15.64:0/1732738917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadac108d30 0x7fadac1a6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.268+0000 7fadb089f640 1 --2- 172.21.15.64:0/1732738917 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fadac1083e0 0x7fadac1a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.268+0000 7fadb089f640 1 --2- 172.21.15.64:0/1732738917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadac102aa0 0x7fadac1a0b10 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fad9c009990 tx=0x7fad9c009e50 comp rx=0 tx=0).stop 2024-09-21T04:04:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.268+0000 7fadb089f640 1 -- 172.21.15.64:0/1732738917 >> 172.21.15.64:0/1732738917 conn(0x7fadac079750 msgr2=0x7fadac106230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.268+0000 7fadb089f640 1 -- 172.21.15.64:0/1732738917 shutdown_connections 2024-09-21T04:04:38.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:38.268+0000 7fadb089f640 1 -- 172.21.15.64:0/1732738917 wait complete. 2024-09-21T04:04:38.994 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T04:04:39.297 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:04:39.753 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:39 smithi064 ceph-mon[31180]: pgmap v340: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-21T04:04:39.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.896+0000 7fdcdf7fe640 1 -- 172.21.15.64:0/2968404221 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd80313a0 con 0x7fdce0108480 2024-09-21T04:04:39.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.897+0000 7fdce6af7640 1 -- 172.21.15.64:0/2968404221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdce0108480 msgr2=0x7fdce010ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:39.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.897+0000 7fdce6af7640 1 --2- 172.21.15.64:0/2968404221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdce0108480 0x7fdce010ecb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fdcd800adb0 tx=0x7fdcd802b7a0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.897+0000 7fdce6af7640 1 -- 172.21.15.64:0/2968404221 shutdown_connections 2024-09-21T04:04:39.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.897+0000 7fdce6af7640 1 --2- 172.21.15.64:0/2968404221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdce0108480 0x7fdce010ecb0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.897+0000 7fdce6af7640 1 --2- 172.21.15.64:0/2968404221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdce0107b30 0x7fdce0107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.897+0000 7fdce6af7640 1 --2- 172.21.15.64:0/2968404221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdce01021f0 0x7fdce0102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.897+0000 7fdce6af7640 1 -- 172.21.15.64:0/2968404221 >> 172.21.15.64:0/2968404221 conn(0x7fdce0077c60 msgr2=0x7fdce00780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:39.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.898+0000 7fdce6af7640 1 -- 172.21.15.64:0/2968404221 shutdown_connections 2024-09-21T04:04:39.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.898+0000 7fdce6af7640 1 -- 172.21.15.64:0/2968404221 wait complete. 2024-09-21T04:04:39.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.899+0000 7fdce6af7640 1 Processor -- start 2024-09-21T04:04:39.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.899+0000 7fdce6af7640 1 -- start start 2024-09-21T04:04:39.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.899+0000 7fdce6af7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdce01021f0 0x7fdce0198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:39.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.899+0000 7fdce6af7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdce0107b30 0x7fdce01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:39.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.899+0000 7fdce6af7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdce0108480 0x7fdce0198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:39.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.899+0000 7fdce6af7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdce0111910 con 0x7fdce0107b30 2024-09-21T04:04:39.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.899+0000 7fdce6af7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdce0111790 con 0x7fdce01021f0 2024-09-21T04:04:39.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.899+0000 7fdce6af7640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdce0111a90 con 0x7fdce0108480 2024-09-21T04:04:39.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.899+0000 7fdce4893640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdce01021f0 0x7fdce0198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:39.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.900+0000 7fdce4893640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdce01021f0 0x7fdce0198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:51964/0 (socket says 172.21.15.64:51964) 2024-09-21T04:04:39.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.900+0000 7fdce4893640 1 -- 172.21.15.64:0/1030194696 learned_addr learned my addr 172.21.15.64:0/1030194696 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:39.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.900+0000 7fdce5094640 1 --2- 172.21.15.64:0/1030194696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdce0108480 0x7fdce0198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:39.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.900+0000 7fdcdffff640 1 --2- 172.21.15.64:0/1030194696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdce0107b30 0x7fdce01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:39.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.900+0000 7fdce5094640 1 -- 172.21.15.64:0/1030194696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdce01021f0 msgr2=0x7fdce0198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:39.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.900+0000 7fdce5094640 1 --2- 172.21.15.64:0/1030194696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdce01021f0 0x7fdce0198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.900+0000 7fdce5094640 1 -- 172.21.15.64:0/1030194696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdce0107b30 msgr2=0x7fdce01985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:39.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.900+0000 7fdce5094640 1 --2- 172.21.15.64:0/1030194696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdce0107b30 0x7fdce01985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.900+0000 7fdce5094640 1 -- 172.21.15.64:0/1030194696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce01a8710 con 0x7fdce0108480 2024-09-21T04:04:39.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.900+0000 7fdce4893640 1 --2- 172.21.15.64:0/1030194696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdce01021f0 0x7fdce0198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:39.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.900+0000 7fdcdffff640 1 --2- 172.21.15.64:0/1030194696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdce0107b30 0x7fdce01985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:04:39.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.901+0000 7fdce5094640 1 --2- 172.21.15.64:0/1030194696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdce0108480 0x7fdce0198af0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdcd802bcc0 tx=0x7fdcd8033d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:39.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.901+0000 7fdcddffb640 1 -- 172.21.15.64:0/1030194696 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8040070 con 0x7fdce0108480 2024-09-21T04:04:39.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.901+0000 7fdce6af7640 1 -- 172.21.15.64:0/1030194696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdce01a89a0 con 0x7fdce0108480 2024-09-21T04:04:39.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.901+0000 7fdcddffb640 1 -- 172.21.15.64:0/1030194696 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcd8003d80 con 0x7fdce0108480 2024-09-21T04:04:39.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.901+0000 7fdcddffb640 1 -- 172.21.15.64:0/1030194696 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd80046e0 con 0x7fdce0108480 2024-09-21T04:04:39.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.901+0000 7fdce6af7640 1 -- 172.21.15.64:0/1030194696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdce01a8ed0 con 0x7fdce0108480 2024-09-21T04:04:39.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.905+0000 7fdced06b740 1 Processor -- start 2024-09-21T04:04:39.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.907+0000 7fdcddffb640 1 -- 172.21.15.64:0/1030194696 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) ==== 154694+0+0 (secure 0 0 0) 0x7fdcd8030510 con 0x7fdce0108480 2024-09-21T04:04:39.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.907+0000 7fdced06b740 1 -- start start 2024-09-21T04:04:39.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.907+0000 7fdced06b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387d811070 0x56387dbb86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:39.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.907+0000 7fdced06b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56387d7e81b0 0x56387dbb8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:39.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.907+0000 7fdced06b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56387d7f0390 0x56387dbb9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:39.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.907+0000 7fdced06b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56387db2eea0 con 0x56387d7e81b0 2024-09-21T04:04:39.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.907+0000 7fdced06b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56387db1e2d0 con 0x56387d7f0390 2024-09-21T04:04:39.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.907+0000 7fdced06b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56387db220e0 con 0x56387d811070 2024-09-21T04:04:39.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.908+0000 7fdce4893640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56387d7e81b0 0x56387dbb8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:39.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.908+0000 7fdce4893640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56387d7e81b0 0x56387dbb8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34534/0 (socket says 172.21.15.64:34534) 2024-09-21T04:04:39.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.908+0000 7fdce4893640 1 -- 172.21.15.64:0/2146552472 learned_addr learned my addr 172.21.15.64:0/2146552472 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:39.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.908+0000 7fdcdffff640 1 --2- 172.21.15.64:0/2146552472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387d811070 0x56387dbb86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:39.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.908+0000 7fdcdffff640 1 --2- 172.21.15.64:0/2146552472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56387d7f0390 0x56387dbb9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:39.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.908+0000 7fdcdffff640 1 -- 172.21.15.64:0/2146552472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56387d7f0390 msgr2=0x56387dbb9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:39.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.908+0000 7fdcdffff640 1 --2- 172.21.15.64:0/2146552472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56387d7f0390 0x56387dbb9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.908+0000 7fdcdffff640 1 -- 172.21.15.64:0/2146552472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56387d7e81b0 msgr2=0x56387dbb8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:39.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.908+0000 7fdcdffff640 1 --2- 172.21.15.64:0/2146552472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56387d7e81b0 0x56387dbb8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.908+0000 7fdcdffff640 1 -- 172.21.15.64:0/2146552472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56387d87c650 con 0x56387d811070 2024-09-21T04:04:39.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.908+0000 7fdcddffb640 1 --2- 172.21.15.64:0/1030194696 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdcd00c7820 0x7fdcd00c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:39.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.908+0000 7fdce4893640 1 --2- 172.21.15.64:0/2146552472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56387d7e81b0 0x56387dbb8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:39.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.908+0000 7fdcdffff640 1 --2- 172.21.15.64:0/2146552472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387d811070 0x56387dbb86a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdce0105760 tx=0x7fdccc00c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34cfb12e8bb4bc69 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:39.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.908+0000 7fdcddffb640 1 -- 172.21.15.64:0/1030194696 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fdcd8108cb0 con 0x7fdce0108480 2024-09-21T04:04:39.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.909+0000 7fdce4893640 1 --2- 172.21.15.64:0/1030194696 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdcd00c7820 0x7fdcd00c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:39.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.909+0000 7fdccaffd640 1 -- 172.21.15.64:0/2146552472 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc004230 con 0x56387d811070 2024-09-21T04:04:39.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.909+0000 7fdccaffd640 1 -- 172.21.15.64:0/2146552472 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdccc017070 con 0x56387d811070 2024-09-21T04:04:39.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.909+0000 7fdccaffd640 1 -- 172.21.15.64:0/2146552472 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc003850 con 0x56387d811070 2024-09-21T04:04:39.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.910+0000 7fdced06b740 1 -- 172.21.15.64:0/2146552472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387d811070 msgr2=0x56387dbb86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:39.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.910+0000 7fdced06b740 1 --2- 172.21.15.64:0/2146552472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387d811070 0x56387dbb86a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdce0105760 tx=0x7fdccc00c510 comp rx=0 tx=0).stop 2024-09-21T04:04:39.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.910+0000 7fdce4893640 1 --2- 172.21.15.64:0/1030194696 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdcd00c7820 0x7fdcd00c9d30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x56387dbb7320 tx=0x7fdcd4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:39.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.910+0000 7fdced06b740 1 -- 172.21.15.64:0/2146552472 shutdown_connections 2024-09-21T04:04:39.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.910+0000 7fdced06b740 1 --2- 172.21.15.64:0/2146552472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56387d7f0390 0x56387dbb9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.910+0000 7fdced06b740 1 --2- 172.21.15.64:0/2146552472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56387d7e81b0 0x56387dbb8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.910+0000 7fdced06b740 1 --2- 172.21.15.64:0/2146552472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387d811070 0x56387dbb86a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.910+0000 7fdced06b740 1 -- 172.21.15.64:0/2146552472 >> 172.21.15.64:0/2146552472 conn(0x56387d805460 msgr2=0x56387dbb6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:39.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.910+0000 7fdced06b740 1 -- 172.21.15.64:0/2146552472 shutdown_connections 2024-09-21T04:04:39.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.910+0000 7fdced06b740 1 -- 172.21.15.64:0/2146552472 wait complete. 2024-09-21T04:04:39.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.911+0000 7fdced06b740 1 -- start start 2024-09-21T04:04:39.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.911+0000 7fdced06b740 1 Processor -- start 2024-09-21T04:04:39.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.911+0000 7fdced06b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56387d7e81b0 0x56387dbc3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:39.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.911+0000 7fdced06b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387d7f0390 0x56387dbc4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:39.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.912+0000 7fdced06b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56387d811070 0x56387dbd0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:39.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.912+0000 7fdced06b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56387d97f700 con 0x56387d7e81b0 2024-09-21T04:04:39.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.912+0000 7fdced06b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56387dab9160 con 0x56387d811070 2024-09-21T04:04:39.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.912+0000 7fdced06b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56387dbc6e20 con 0x56387d7f0390 2024-09-21T04:04:39.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.912+0000 7fdced06b740 10 client.0 _ll_register_callbacks cb 0x56387dad29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:39.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.912+0000 7fdce4893640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56387d7e81b0 0x56387dbc3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:39.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.912+0000 7fdce4893640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56387d7e81b0 0x56387dbc3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34538/0 (socket says 172.21.15.64:34538) 2024-09-21T04:04:39.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.912+0000 7fdcdffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387d7f0390 0x56387dbc4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:39.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.912+0000 7fdce4893640 1 -- 172.21.15.64:0/1365006182 learned_addr learned my addr 172.21.15.64:0/1365006182 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:39.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.912+0000 7fdce5094640 1 --2- 172.21.15.64:0/1365006182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56387d811070 0x56387dbd0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:39.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.912+0000 7fdcdffff640 1 -- 172.21.15.64:0/1365006182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56387d811070 msgr2=0x56387dbd0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:39.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.912+0000 7fdcdffff640 1 --2- 172.21.15.64:0/1365006182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56387d811070 0x56387dbd0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.912+0000 7fdcdffff640 1 -- 172.21.15.64:0/1365006182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56387d7e81b0 msgr2=0x56387dbc3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:39.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.912+0000 7fdcdffff640 1 --2- 172.21.15.64:0/1365006182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56387d7e81b0 0x56387dbc3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.912+0000 7fdcdffff640 1 -- 172.21.15.64:0/1365006182 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56387dae6c90 con 0x56387d7f0390 2024-09-21T04:04:39.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.912+0000 7fdce4893640 1 --2- 172.21.15.64:0/1365006182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56387d7e81b0 0x56387dbc3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:39.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.913+0000 7fdcdffff640 1 --2- 172.21.15.64:0/1365006182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387d7f0390 0x56387dbc4320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdccc016000 tx=0x7fdccc011f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3d222d8a296356f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:39.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.913+0000 7fdcbffff640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-21T04:04:39.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.913+0000 7fdcbffff640 1 -- 172.21.15.64:0/1365006182 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc004440 con 0x56387d7f0390 2024-09-21T04:04:39.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.913+0000 7fdcbffff640 1 -- 172.21.15.64:0/1365006182 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fdccc01c400 con 0x56387d7f0390 2024-09-21T04:04:39.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.913+0000 7fdced06b740 1 -- 172.21.15.64:0/1365006182 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdccc017070 con 0x56387d7f0390 2024-09-21T04:04:39.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.913+0000 7fdcbffff640 1 -- 172.21.15.64:0/1365006182 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdccc01c600 con 0x56387d7f0390 2024-09-21T04:04:39.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.913+0000 7fdced06b740 10 client.25264 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:39.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.913+0000 7fdced06b740 1 -- 172.21.15.64:0/1365006182 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56387dbd2b90 con 0x56387d7f0390 2024-09-21T04:04:39.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.914+0000 7fdcbffff640 1 -- 172.21.15.64:0/1365006182 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdccc00f730 con 0x56387d7f0390 2024-09-21T04:04:39.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.915+0000 7fdcbffff640 1 -- 172.21.15.64:0/1365006182 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdccc02d020 con 0x56387d7f0390 2024-09-21T04:04:39.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.915+0000 7fdced06b740 10 client.25264 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:39.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.915+0000 7fdced06b740 10 client.25264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:04:39.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.915+0000 7fdced06b740 20 client.25264 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:39.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.915+0000 7fdced06b740 1 --2- 172.21.15.64:0/1365006182 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56387d7cf9f0 0x56387dbd5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:39.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.915+0000 7fdced06b740 4 client.25264 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:04:39.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.915+0000 7fdced06b740 1 -- 172.21.15.64:0/1365006182 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdccc017070 con 0x56387d7cf9f0 2024-09-21T04:04:39.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.915+0000 7fdce4893640 1 --2- 172.21.15.64:0/1365006182 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56387d7cf9f0 0x56387dbd5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:39.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.916+0000 7fdce4893640 1 --2- 172.21.15.64:0/1365006182 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56387d7cf9f0 0x56387dbd5fb0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39ac416b5145aa08 server_cookie=69b28904e9badf1c in_seq=0 out_seq=0 2024-09-21T04:04:39.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.916+0000 7fdcbffff640 10 client.25264 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:39.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.916+0000 7fdcbffff640 1 -- 172.21.15.64:0/1365006182 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fdccc017070 con 0x56387d7cf9f0 2024-09-21T04:04:39.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.916+0000 7fdcbffff640 10 client.25264 handle_command_reply: tid=0 2024-09-21T04:04:39.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.916+0000 7fdced06b740 1 client.25264 shutdown 2024-09-21T04:04:39.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.917+0000 7fdced06b740 1 -- 172.21.15.64:0/1365006182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387d7f0390 msgr2=0x56387dbc4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:39.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.917+0000 7fdced06b740 1 --2- 172.21.15.64:0/1365006182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387d7f0390 0x56387dbc4320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdccc016000 tx=0x7fdccc011f00 comp rx=0 tx=0).stop 2024-09-21T04:04:39.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.917+0000 7fdced06b740 1 -- 172.21.15.64:0/1365006182 shutdown_connections 2024-09-21T04:04:39.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.917+0000 7fdced06b740 1 --2- 172.21.15.64:0/1365006182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56387d811070 0x56387dbd0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.917+0000 7fdced06b740 1 --2- 172.21.15.64:0/1365006182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387d7f0390 0x56387dbc4320 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.917+0000 7fdced06b740 1 --2- 172.21.15.64:0/1365006182 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56387d7cf9f0 0x56387dbd5fb0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.917+0000 7fdced06b740 1 --2- 172.21.15.64:0/1365006182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56387d7e81b0 0x56387dbc3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.917+0000 7fdced06b740 1 -- 172.21.15.64:0/1365006182 >> 172.21.15.64:0/1365006182 conn(0x56387d805460 msgr2=0x56387dbbb830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:39.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.917+0000 7fdcbffff640 0 client.25264 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:39.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.917+0000 7fdced06b740 1 -- 172.21.15.64:0/1365006182 shutdown_connections 2024-09-21T04:04:39.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.917+0000 7fdced06b740 1 -- 172.21.15.64:0/1365006182 wait complete. 2024-09-21T04:04:39.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.917+0000 7fdced06b740 20 client.25264 trim_cache size 0 max 16384 2024-09-21T04:04:39.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.944+0000 7fdced06b740 1 Processor -- start 2024-09-21T04:04:39.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.944+0000 7fdced06b740 1 -- start start 2024-09-21T04:04:39.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.944+0000 7fdced06b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56387d7e81b0 0x56387dbc5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:39.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.944+0000 7fdced06b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387d7f0390 0x56387dbb9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:39.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.944+0000 7fdced06b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56387d811070 0x56387dbb5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:39.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.945+0000 7fdced06b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdce0073c10 con 0x56387d811070 2024-09-21T04:04:39.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.945+0000 7fdced06b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdce00714e0 con 0x56387d7e81b0 2024-09-21T04:04:39.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.945+0000 7fdced06b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdce006f330 con 0x56387d7f0390 2024-09-21T04:04:39.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.945+0000 7fdce5094640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56387d811070 0x56387dbb5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:39.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.945+0000 7fdce5094640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56387d811070 0x56387dbb5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34550/0 (socket says 172.21.15.64:34550) 2024-09-21T04:04:39.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.945+0000 7fdce5094640 1 -- 172.21.15.64:0/1319075377 learned_addr learned my addr 172.21.15.64:0/1319075377 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:39.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.945+0000 7fdcdffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387d7f0390 0x56387dbb9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:39.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.945+0000 7fdcdffff640 1 -- 172.21.15.64:0/1319075377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56387d7e81b0 msgr2=0x56387dbc5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:39.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.945+0000 7fdcdffff640 1 --2- 172.21.15.64:0/1319075377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56387d7e81b0 0x56387dbc5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.945+0000 7fdcdffff640 1 -- 172.21.15.64:0/1319075377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56387d811070 msgr2=0x56387dbb5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.945+0000 7fdcdffff640 1 --2- 172.21.15.64:0/1319075377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56387d811070 0x56387dbb5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.945+0000 7fdcdffff640 1 -- 172.21.15.64:0/1319075377 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56387dbc50a0 con 0x56387d7f0390 2024-09-21T04:04:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.945+0000 7fdcdffff640 1 --2- 172.21.15.64:0/1319075377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387d7f0390 0x56387dbb9de0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdccc016000 tx=0x7fdccc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63ed503752a703a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.946+0000 7fdcca7fc640 1 -- 172.21.15.64:0/1319075377 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc00f730 con 0x56387d7f0390 2024-09-21T04:04:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.946+0000 7fdcca7fc640 1 -- 172.21.15.64:0/1319075377 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdccc017030 con 0x56387d7f0390 2024-09-21T04:04:39.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.946+0000 7fdcca7fc640 1 -- 172.21.15.64:0/1319075377 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc004680 con 0x56387d7f0390 2024-09-21T04:04:39.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.947+0000 7fdced06b740 1 -- 172.21.15.64:0/1319075377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387d7f0390 msgr2=0x56387dbb9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:39.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.947+0000 7fdced06b740 1 --2- 172.21.15.64:0/1319075377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387d7f0390 0x56387dbb9de0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdccc016000 tx=0x7fdccc000bc0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.947+0000 7fdced06b740 1 -- 172.21.15.64:0/1319075377 shutdown_connections 2024-09-21T04:04:39.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.947+0000 7fdced06b740 1 --2- 172.21.15.64:0/1319075377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56387d811070 0x56387dbb5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.947+0000 7fdced06b740 1 --2- 172.21.15.64:0/1319075377 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387d7f0390 0x56387dbb9de0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.947+0000 7fdced06b740 1 --2- 172.21.15.64:0/1319075377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56387d7e81b0 0x56387dbc5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.947+0000 7fdced06b740 1 -- 172.21.15.64:0/1319075377 >> 172.21.15.64:0/1319075377 conn(0x56387d805460 msgr2=0x56387dbb96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:39.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.948+0000 7fdced06b740 1 -- 172.21.15.64:0/1319075377 shutdown_connections 2024-09-21T04:04:39.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.948+0000 7fdced06b740 1 -- 172.21.15.64:0/1319075377 wait complete. 2024-09-21T04:04:39.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.948+0000 7fdced06b740 1 -- start start 2024-09-21T04:04:39.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.948+0000 7fdced06b740 1 Processor -- start 2024-09-21T04:04:39.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.949+0000 7fdced06b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387d7e81b0 0x56387dbcc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:39.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.949+0000 7fdced06b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56387d7f0390 0x56387dbcca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:39.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.949+0000 7fdcdffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56387d7f0390 0x56387dbcca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:39.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.949+0000 7fdcdffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56387d7f0390 0x56387dbcca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34560/0 (socket says 172.21.15.64:34560) 2024-09-21T04:04:39.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.950+0000 7fdcdffff640 1 -- 172.21.15.64:0/1415503996 learned_addr learned my addr 172.21.15.64:0/1415503996 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:39.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.950+0000 7fdced06b740 1 --2- 172.21.15.64:0/1415503996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56387d811070 0x56387dbc7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:39.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.950+0000 7fdced06b740 1 -- 172.21.15.64:0/1415503996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdce006f9a0 con 0x56387d7f0390 2024-09-21T04:04:39.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.950+0000 7fdced06b740 1 -- 172.21.15.64:0/1415503996 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdce0070e10 con 0x56387d811070 2024-09-21T04:04:39.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.950+0000 7fdced06b740 1 -- 172.21.15.64:0/1415503996 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdce0070070 con 0x56387d7e81b0 2024-09-21T04:04:39.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.950+0000 7fdced06b740 10 client.0 _ll_register_callbacks cb 0x7fdcd4009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:39.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.950+0000 7fdce4893640 1 --2- 172.21.15.64:0/1415503996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387d7e81b0 0x56387dbcc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:39.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.950+0000 7fdce5094640 1 --2- 172.21.15.64:0/1415503996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56387d811070 0x56387dbc7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:39.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.950+0000 7fdcdffff640 1 -- 172.21.15.64:0/1415503996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387d7e81b0 msgr2=0x56387dbcc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:39.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.950+0000 7fdcdffff640 1 --2- 172.21.15.64:0/1415503996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387d7e81b0 0x56387dbcc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.950+0000 7fdcdffff640 1 -- 172.21.15.64:0/1415503996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56387d811070 msgr2=0x56387dbc7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:39.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.950+0000 7fdcdffff640 1 --2- 172.21.15.64:0/1415503996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56387d811070 0x56387dbc7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.950+0000 7fdcdffff640 1 -- 172.21.15.64:0/1415503996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56387dbc8260 con 0x56387d7f0390 2024-09-21T04:04:39.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.950+0000 7fdce4893640 1 --2- 172.21.15.64:0/1415503996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387d7e81b0 0x56387dbcc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:04:39.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.951+0000 7fdcdffff640 1 --2- 172.21.15.64:0/1415503996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56387d7f0390 0x56387dbcca50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fdccc01ca50 tx=0x7fdccc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958d5ece1263367e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:39.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.951+0000 7fdcbffff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:04:39.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.951+0000 7fdcbffff640 1 -- 172.21.15.64:0/1415503996 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc00fe70 con 0x56387d7f0390 2024-09-21T04:04:39.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.951+0000 7fdcbffff640 1 -- 172.21.15.64:0/1415503996 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fdccc004890 con 0x56387d7f0390 2024-09-21T04:04:39.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.951+0000 7fdced06b740 1 -- 172.21.15.64:0/1415503996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdccc017030 con 0x56387d7f0390 2024-09-21T04:04:39.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.952+0000 7fdcbffff640 1 -- 172.21.15.64:0/1415503996 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdccc047050 con 0x56387d7f0390 2024-09-21T04:04:39.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.952+0000 7fdced06b740 10 client.15603 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:39.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.952+0000 7fdced06b740 1 -- 172.21.15.64:0/1415503996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56387dc61e40 con 0x56387d7f0390 2024-09-21T04:04:39.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.952+0000 7fdcbffff640 1 -- 172.21.15.64:0/1415503996 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdccc0035f0 con 0x56387d7f0390 2024-09-21T04:04:39.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.952+0000 7fdcbffff640 1 -- 172.21.15.64:0/1415503996 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdccc040060 con 0x56387d7f0390 2024-09-21T04:04:39.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.952+0000 7fdced06b740 10 client.15603 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:39.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.952+0000 7fdced06b740 10 client.15603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:04:39.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.952+0000 7fdced06b740 20 client.15603 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:39.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.952+0000 7fdced06b740 1 --2- 172.21.15.64:0/1415503996 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56387d7a4da0 0x56387dc644f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:39.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.952+0000 7fdced06b740 4 client.15603 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:04:39.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.952+0000 7fdced06b740 1 -- 172.21.15.64:0/1415503996 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fdccc017030 con 0x56387d7a4da0 2024-09-21T04:04:39.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.953+0000 7fdce4893640 1 --2- 172.21.15.64:0/1415503996 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56387d7a4da0 0x56387dc644f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:39.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.953+0000 7fdcbffff640 10 client.15603 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:39.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.953+0000 7fdce4893640 1 --2- 172.21.15.64:0/1415503996 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56387d7a4da0 0x56387dc644f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f1bce2d788cac47 server_cookie=20a1441bf4786737 in_seq=0 out_seq=0 2024-09-21T04:04:39.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.954+0000 7fdcbffff640 1 -- 172.21.15.64:0/1415503996 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fdccc017030 con 0x56387d7a4da0 2024-09-21T04:04:39.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.954+0000 7fdcbffff640 10 client.15603 handle_command_reply: tid=0 2024-09-21T04:04:39.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.954+0000 7fdced06b740 1 client.15603 shutdown 2024-09-21T04:04:39.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.954+0000 7fdced06b740 1 -- 172.21.15.64:0/1415503996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56387d7f0390 msgr2=0x56387dbcca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:39.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.954+0000 7fdced06b740 1 --2- 172.21.15.64:0/1415503996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56387d7f0390 0x56387dbcca50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fdccc01ca50 tx=0x7fdccc001e70 comp rx=0 tx=0).stop 2024-09-21T04:04:39.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.954+0000 7fdced06b740 1 -- 172.21.15.64:0/1415503996 shutdown_connections 2024-09-21T04:04:39.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.954+0000 7fdced06b740 1 --2- 172.21.15.64:0/1415503996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56387d811070 0x56387dbc7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.954+0000 7fdced06b740 1 --2- 172.21.15.64:0/1415503996 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56387d7a4da0 0x56387dc644f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.954+0000 7fdced06b740 1 --2- 172.21.15.64:0/1415503996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56387d7f0390 0x56387dbcca50 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.954+0000 7fdcbffff640 0 client.15603 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:39.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.954+0000 7fdced06b740 1 --2- 172.21.15.64:0/1415503996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387d7e81b0 0x56387dbcc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.954+0000 7fdced06b740 1 -- 172.21.15.64:0/1415503996 >> 172.21.15.64:0/1415503996 conn(0x56387d805460 msgr2=0x56387dbc5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:39.959 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:04:39.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.955+0000 7fdced06b740 1 -- 172.21.15.64:0/1415503996 shutdown_connections 2024-09-21T04:04:39.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.955+0000 7fdced06b740 1 -- 172.21.15.64:0/1415503996 wait complete. 2024-09-21T04:04:39.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.955+0000 7fdced06b740 20 client.15603 trim_cache size 0 max 16384 2024-09-21T04:04:39.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.956+0000 7fdce6af7640 1 -- 172.21.15.64:0/1030194696 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdcd00c7820 msgr2=0x7fdcd00c9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:39.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.956+0000 7fdce6af7640 1 --2- 172.21.15.64:0/1030194696 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdcd00c7820 0x7fdcd00c9d30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x56387dbb7320 tx=0x7fdcd4005f90 comp rx=0 tx=0).stop 2024-09-21T04:04:39.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.956+0000 7fdce6af7640 1 -- 172.21.15.64:0/1030194696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdce0108480 msgr2=0x7fdce0198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:39.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.956+0000 7fdce6af7640 1 --2- 172.21.15.64:0/1030194696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdce0108480 0x7fdce0198af0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdcd802bcc0 tx=0x7fdcd8033d50 comp rx=0 tx=0).stop 2024-09-21T04:04:39.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.956+0000 7fdce6af7640 1 -- 172.21.15.64:0/1030194696 shutdown_connections 2024-09-21T04:04:39.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.956+0000 7fdce6af7640 1 --2- 172.21.15.64:0/1030194696 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdcd00c7820 0x7fdcd00c9d30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.956+0000 7fdce6af7640 1 --2- 172.21.15.64:0/1030194696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdce0108480 0x7fdce0198af0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.956+0000 7fdce6af7640 1 --2- 172.21.15.64:0/1030194696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdce0107b30 0x7fdce01985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.956+0000 7fdce6af7640 1 --2- 172.21.15.64:0/1030194696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdce01021f0 0x7fdce0198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:39.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.956+0000 7fdce6af7640 1 -- 172.21.15.64:0/1030194696 >> 172.21.15.64:0/1030194696 conn(0x7fdce0077c60 msgr2=0x7fdce0103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:39.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.956+0000 7fdce6af7640 1 -- 172.21.15.64:0/1030194696 shutdown_connections 2024-09-21T04:04:39.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:39.956+0000 7fdce6af7640 1 -- 172.21.15.64:0/1030194696 wait complete. 2024-09-21T04:04:40.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:39 smithi119 ceph-mon[34811]: pgmap v340: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-21T04:04:40.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:39 smithi131 ceph-mon[35193]: pgmap v340: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-09-21T04:04:40.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.051+0000 7f50db33a640 1 -- 172.21.15.64:0/1159644104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50d4102060 msgr2=0x7f50d4102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:40.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.051+0000 7f50db33a640 1 --2- 172.21.15.64:0/1159644104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50d4102060 0x7f50d4102470 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f50d0009900 tx=0x7f50d0032c20 comp rx=0 tx=0).stop 2024-09-21T04:04:40.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.051+0000 7f50db33a640 1 -- 172.21.15.64:0/1159644104 shutdown_connections 2024-09-21T04:04:40.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.051+0000 7f50db33a640 1 --2- 172.21.15.64:0/1159644104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50d4109d80 0x7f50d41105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.051+0000 7f50db33a640 1 --2- 172.21.15.64:0/1159644104 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d4109430 0x7f50d4109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.051+0000 7f50db33a640 1 --2- 172.21.15.64:0/1159644104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50d4102060 0x7f50d4102470 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.051+0000 7f50db33a640 1 -- 172.21.15.64:0/1159644104 >> 172.21.15.64:0/1159644104 conn(0x7f50d4079e90 msgr2=0x7f50d407a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:40.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.051+0000 7f50db33a640 1 -- 172.21.15.64:0/1159644104 shutdown_connections 2024-09-21T04:04:40.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.051+0000 7f50db33a640 1 -- 172.21.15.64:0/1159644104 wait complete. 2024-09-21T04:04:40.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.052+0000 7f50db33a640 1 Processor -- start 2024-09-21T04:04:40.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.052+0000 7f50db33a640 1 -- start start 2024-09-21T04:04:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.052+0000 7f50db33a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50d4102060 0x7f50d4198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.052+0000 7f50db33a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50d4109430 0x7f50d41988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.052+0000 7f50db33a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d4109d80 0x7f50d419db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.052+0000 7f50db33a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f50d4111880 con 0x7f50d4102060 2024-09-21T04:04:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.052+0000 7f50db33a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f50d4111700 con 0x7f50d4109430 2024-09-21T04:04:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.052+0000 7f50db33a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f50d4111a00 con 0x7f50d4109d80 2024-09-21T04:04:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.052+0000 7f50d90d6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50d4102060 0x7f50d4198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.052+0000 7f50d90d6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50d4102060 0x7f50d4198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34574/0 (socket says 172.21.15.64:34574) 2024-09-21T04:04:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.052+0000 7f50d90d6640 1 -- 172.21.15.64:0/694238470 learned_addr learned my addr 172.21.15.64:0/694238470 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.053+0000 7f50d88d5640 1 --2- 172.21.15.64:0/694238470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50d4109430 0x7f50d41988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.053+0000 7f50d90d6640 1 -- 172.21.15.64:0/694238470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d4109d80 msgr2=0x7f50d419db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.053+0000 7f50d90d6640 1 --2- 172.21.15.64:0/694238470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d4109d80 0x7f50d419db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.053+0000 7f50d90d6640 1 -- 172.21.15.64:0/694238470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50d4109430 msgr2=0x7f50d41988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.053+0000 7f50d90d6640 1 --2- 172.21.15.64:0/694238470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50d4109430 0x7f50d41988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.053+0000 7f50d90d6640 1 -- 172.21.15.64:0/694238470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d419e230 con 0x7f50d4102060 2024-09-21T04:04:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.053+0000 7f50d90d6640 1 --2- 172.21.15.64:0/694238470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50d4102060 0x7f50d4198300 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f50d0000c00 tx=0x7f50d00335c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.053+0000 7f50ca7fc640 1 -- 172.21.15.64:0/694238470 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d004a020 con 0x7f50d4102060 2024-09-21T04:04:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.053+0000 7f50ca7fc640 1 -- 172.21.15.64:0/694238470 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50d0033ac0 con 0x7f50d4102060 2024-09-21T04:04:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.053+0000 7f50ca7fc640 1 -- 172.21.15.64:0/694238470 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d00039e0 con 0x7f50d4102060 2024-09-21T04:04:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.053+0000 7f50db33a640 1 -- 172.21.15.64:0/694238470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d419e4c0 con 0x7f50d4102060 2024-09-21T04:04:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.054+0000 7f50db33a640 1 -- 172.21.15.64:0/694238470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d41a4cb0 con 0x7f50d4102060 2024-09-21T04:04:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.057+0000 7f50ca7fc640 1 -- 172.21.15.64:0/694238470 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 154694+0+0 (secure 0 0 0) 0x7f50d004d020 con 0x7f50d4102060 2024-09-21T04:04:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.057+0000 7f50e22a4740 1 Processor -- start 2024-09-21T04:04:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.058+0000 7f50e22a4740 1 -- start start 2024-09-21T04:04:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.058+0000 7f50e22a4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0b17451b0 0x55e0b1aea9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.058+0000 7f50e22a4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e0b171c2f0 0x55e0b1aeaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.058+0000 7f50e22a4740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0b17244d0 0x55e0b1aeb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.058+0000 7f50e22a4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e0b1a5f6f0 con 0x55e0b17451b0 2024-09-21T04:04:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.058+0000 7f50e22a4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e0b1a4eb20 con 0x55e0b171c2f0 2024-09-21T04:04:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.058+0000 7f50e22a4740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e0b1a52930 con 0x55e0b17244d0 2024-09-21T04:04:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.058+0000 7f50ca7fc640 1 --2- 172.21.15.64:0/694238470 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f50c00b6810 0x7f50c00b8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.058+0000 7f50d98d7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0b17451b0 0x55e0b1aea9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.058+0000 7f50d98d7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0b17451b0 0x55e0b1aea9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34578/0 (socket says 172.21.15.64:34578) 2024-09-21T04:04:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.058+0000 7f50d98d7640 1 -- 172.21.15.64:0/2434209568 learned_addr learned my addr 172.21.15.64:0/2434209568 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.058+0000 7f50d90d6640 1 --2- 172.21.15.64:0/2434209568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e0b171c2f0 0x55e0b1aeaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.058+0000 7f50d98d7640 1 --2- 172.21.15.64:0/694238470 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f50c00b6810 0x7f50c00b8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.058+0000 7f50d88d5640 1 --2- 172.21.15.64:0/2434209568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0b17244d0 0x55e0b1aeb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.059+0000 7f50ca7fc640 1 -- 172.21.15.64:0/694238470 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f50d01109b0 con 0x7f50d4102060 2024-09-21T04:04:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.059+0000 7f50d90d6640 1 -- 172.21.15.64:0/2434209568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0b17244d0 msgr2=0x55e0b1aeb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.059+0000 7f50d90d6640 1 --2- 172.21.15.64:0/2434209568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0b17244d0 0x55e0b1aeb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.059+0000 7f50d90d6640 1 -- 172.21.15.64:0/2434209568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0b17451b0 msgr2=0x55e0b1aea9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.059+0000 7f50d90d6640 1 --2- 172.21.15.64:0/2434209568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0b17451b0 0x55e0b1aea9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.059+0000 7f50d90d6640 1 -- 172.21.15.64:0/2434209568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0b1a14530 con 0x55e0b171c2f0 2024-09-21T04:04:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.059+0000 7f50d90d6640 1 --2- 172.21.15.64:0/2434209568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e0b171c2f0 0x55e0b1aeaef0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f50d010df80 tx=0x7f50d0005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1769d4f96c93964a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.059+0000 7f50d98d7640 1 --2- 172.21.15.64:0/694238470 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f50c00b6810 0x7f50c00b8d20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f50cc005c40 tx=0x7f50cc009b50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.059+0000 7f50bf7fe640 1 -- 172.21.15.64:0/2434209568 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d00d6460 con 0x55e0b171c2f0 2024-09-21T04:04:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.059+0000 7f50bf7fe640 1 -- 172.21.15.64:0/2434209568 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50d00d61a0 con 0x55e0b171c2f0 2024-09-21T04:04:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.059+0000 7f50bf7fe640 1 -- 172.21.15.64:0/2434209568 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d00d5e00 con 0x55e0b171c2f0 2024-09-21T04:04:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.061+0000 7f50e22a4740 1 -- 172.21.15.64:0/2434209568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e0b171c2f0 msgr2=0x55e0b1aeaef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.061+0000 7f50e22a4740 1 --2- 172.21.15.64:0/2434209568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e0b171c2f0 0x55e0b1aeaef0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f50d010df80 tx=0x7f50d0005e20 comp rx=0 tx=0).stop 2024-09-21T04:04:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.061+0000 7f50e22a4740 1 -- 172.21.15.64:0/2434209568 shutdown_connections 2024-09-21T04:04:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.061+0000 7f50e22a4740 1 --2- 172.21.15.64:0/2434209568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0b17244d0 0x55e0b1aeb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.061+0000 7f50e22a4740 1 --2- 172.21.15.64:0/2434209568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e0b171c2f0 0x55e0b1aeaef0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.061+0000 7f50e22a4740 1 --2- 172.21.15.64:0/2434209568 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0b17451b0 0x55e0b1aea9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.061+0000 7f50e22a4740 1 -- 172.21.15.64:0/2434209568 >> 172.21.15.64:0/2434209568 conn(0x55e0b1739300 msgr2=0x55e0b1ae88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.061+0000 7f50e22a4740 1 -- 172.21.15.64:0/2434209568 shutdown_connections 2024-09-21T04:04:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.061+0000 7f50e22a4740 1 -- 172.21.15.64:0/2434209568 wait complete. 2024-09-21T04:04:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.061+0000 7f50e22a4740 1 -- start start 2024-09-21T04:04:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.061+0000 7f50e22a4740 1 Processor -- start 2024-09-21T04:04:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.062+0000 7f50e22a4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0b171c2f0 0x55e0b1af6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.062+0000 7f50e22a4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e0b17244d0 0x55e0b1af6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.062+0000 7f50e22a4740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0b17451b0 0x55e0b1b01fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.062+0000 7f50d90d6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0b171c2f0 0x55e0b1af6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.062+0000 7f50e22a4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e0b1986ab0 con 0x55e0b171c2f0 2024-09-21T04:04:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.062+0000 7f50d90d6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0b171c2f0 0x55e0b1af6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34594/0 (socket says 172.21.15.64:34594) 2024-09-21T04:04:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.062+0000 7f50d90d6640 1 -- 172.21.15.64:0/1400281058 learned_addr learned my addr 172.21.15.64:0/1400281058 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.062+0000 7f50e22a4740 1 -- 172.21.15.64:0/1400281058 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e0b19f4730 con 0x55e0b17244d0 2024-09-21T04:04:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.062+0000 7f50e22a4740 1 -- 172.21.15.64:0/1400281058 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e0b1a4eb20 con 0x55e0b17451b0 2024-09-21T04:04:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.062+0000 7f50e22a4740 10 client.0 _ll_register_callbacks cb 0x55e0b1a486b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.063+0000 7f50d88d5640 1 --2- 172.21.15.64:0/1400281058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e0b17244d0 0x55e0b1af6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.063+0000 7f50d98d7640 1 --2- 172.21.15.64:0/1400281058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0b17451b0 0x55e0b1b01fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.063+0000 7f50d90d6640 1 -- 172.21.15.64:0/1400281058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0b17451b0 msgr2=0x55e0b1b01fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.063+0000 7f50d90d6640 1 --2- 172.21.15.64:0/1400281058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0b17451b0 0x55e0b1b01fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.063+0000 7f50d90d6640 1 -- 172.21.15.64:0/1400281058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e0b17244d0 msgr2=0x55e0b1af6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.063+0000 7f50d90d6640 1 --2- 172.21.15.64:0/1400281058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e0b17244d0 0x55e0b1af6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.063+0000 7f50d90d6640 1 -- 172.21.15.64:0/1400281058 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0b17b0790 con 0x55e0b171c2f0 2024-09-21T04:04:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.063+0000 7f50d88d5640 1 --2- 172.21.15.64:0/1400281058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e0b17244d0 0x55e0b1af6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:04:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.063+0000 7f50d90d6640 1 --2- 172.21.15.64:0/1400281058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0b171c2f0 0x55e0b1af6150 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f50d00d3900 tx=0x7f50d00384d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507aee162b5f6d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.063+0000 7f50bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:04:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.063+0000 7f50bcff9640 1 -- 172.21.15.64:0/1400281058 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d00d5fe0 con 0x55e0b171c2f0 2024-09-21T04:04:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.063+0000 7f50bcff9640 1 -- 172.21.15.64:0/1400281058 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f50d0003d80 con 0x55e0b171c2f0 2024-09-21T04:04:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.063+0000 7f50e22a4740 1 -- 172.21.15.64:0/1400281058 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50d00d61a0 con 0x55e0b171c2f0 2024-09-21T04:04:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.063+0000 7f50bcff9640 1 -- 172.21.15.64:0/1400281058 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f50d00d6460 con 0x55e0b171c2f0 2024-09-21T04:04:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.063+0000 7f50e22a4740 10 client.15615 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.063+0000 7f50e22a4740 1 -- 172.21.15.64:0/1400281058 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e0b1829620 con 0x55e0b171c2f0 2024-09-21T04:04:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.064+0000 7f50bcff9640 1 -- 172.21.15.64:0/1400281058 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50d00d61a0 con 0x55e0b171c2f0 2024-09-21T04:04:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.064+0000 7f50bcff9640 1 -- 172.21.15.64:0/1400281058 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f50d00dd360 con 0x55e0b171c2f0 2024-09-21T04:04:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.064+0000 7f50e22a4740 10 client.15615 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.064+0000 7f50e22a4740 10 client.15615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:04:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.064+0000 7f50e22a4740 20 client.15615 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.064+0000 7f50e22a4740 1 --2- 172.21.15.64:0/1400281058 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55e0b1703d80 0x55e0b1b07c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.065+0000 7f50e22a4740 4 client.15615 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:04:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.065+0000 7f50d88d5640 1 --2- 172.21.15.64:0/1400281058 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55e0b1703d80 0x55e0b1b07c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.065+0000 7f50e22a4740 1 -- 172.21.15.64:0/1400281058 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0b1a61640 con 0x55e0b1703d80 2024-09-21T04:04:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.065+0000 7f50bcff9640 10 client.15615 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.065+0000 7f50d88d5640 1 --2- 172.21.15.64:0/1400281058 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55e0b1703d80 0x55e0b1b07c00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21e99e2ed97a2b06 server_cookie=466bcea985ebc854 in_seq=0 out_seq=0 2024-09-21T04:04:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.066+0000 7f50bcff9640 1 -- 172.21.15.64:0/1400281058 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e0b1a61640 con 0x55e0b1703d80 2024-09-21T04:04:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.066+0000 7f50bcff9640 10 client.15615 handle_command_reply: tid=0 2024-09-21T04:04:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.066+0000 7f50e22a4740 1 client.15615 shutdown 2024-09-21T04:04:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.067+0000 7f50e22a4740 1 -- 172.21.15.64:0/1400281058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0b171c2f0 msgr2=0x55e0b1af6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.067+0000 7f50e22a4740 1 --2- 172.21.15.64:0/1400281058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0b171c2f0 0x55e0b1af6150 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f50d00d3900 tx=0x7f50d00384d0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.067+0000 7f50e22a4740 1 -- 172.21.15.64:0/1400281058 shutdown_connections 2024-09-21T04:04:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.067+0000 7f50e22a4740 1 --2- 172.21.15.64:0/1400281058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0b17451b0 0x55e0b1b01fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.067+0000 7f50e22a4740 1 --2- 172.21.15.64:0/1400281058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e0b17244d0 0x55e0b1af6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.067+0000 7f50e22a4740 1 --2- 172.21.15.64:0/1400281058 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55e0b1703d80 0x55e0b1b07c00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.067+0000 7f50e22a4740 1 --2- 172.21.15.64:0/1400281058 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0b171c2f0 0x55e0b1af6150 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.067+0000 7f50bcff9640 0 client.15615 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.067+0000 7f50e22a4740 1 -- 172.21.15.64:0/1400281058 >> 172.21.15.64:0/1400281058 conn(0x55e0b1739300 msgr2=0x55e0b1aeb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.067+0000 7f50e22a4740 1 -- 172.21.15.64:0/1400281058 shutdown_connections 2024-09-21T04:04:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.067+0000 7f50e22a4740 1 -- 172.21.15.64:0/1400281058 wait complete. 2024-09-21T04:04:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.067+0000 7f50e22a4740 20 client.15615 trim_cache size 0 max 16384 2024-09-21T04:04:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.096+0000 7f50e22a4740 1 Processor -- start 2024-09-21T04:04:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.096+0000 7f50e22a4740 1 -- start start 2024-09-21T04:04:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.097+0000 7f50e22a4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e0b1703d80 0x55e0b1af6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.097+0000 7f50e22a4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0b171c2f0 0x55e0b1aec3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.097+0000 7f50e22a4740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0b17244d0 0x55e0b1af7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.097+0000 7f50d88d5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0b171c2f0 0x55e0b1aec3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.097+0000 7f50d88d5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0b171c2f0 0x55e0b1aec3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34608/0 (socket says 172.21.15.64:34608) 2024-09-21T04:04:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.097+0000 7f50d88d5640 1 -- 172.21.15.64:0/3660288728 learned_addr learned my addr 172.21.15.64:0/3660288728 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.097+0000 7f50e22a4740 1 -- 172.21.15.64:0/3660288728 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f50d41b0050 con 0x55e0b171c2f0 2024-09-21T04:04:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.097+0000 7f50e22a4740 1 -- 172.21.15.64:0/3660288728 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f50d41ad920 con 0x55e0b1703d80 2024-09-21T04:04:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.097+0000 7f50e22a4740 1 -- 172.21.15.64:0/3660288728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f50d41ab770 con 0x55e0b17244d0 2024-09-21T04:04:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.097+0000 7f50d90d6640 1 --2- 172.21.15.64:0/3660288728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e0b1703d80 0x55e0b1af6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.097+0000 7f50d98d7640 1 --2- 172.21.15.64:0/3660288728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0b17244d0 0x55e0b1af7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.097+0000 7f50d90d6640 1 -- 172.21.15.64:0/3660288728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0b17244d0 msgr2=0x55e0b1af7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.097+0000 7f50d90d6640 1 --2- 172.21.15.64:0/3660288728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0b17244d0 0x55e0b1af7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.097+0000 7f50d90d6640 1 -- 172.21.15.64:0/3660288728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0b171c2f0 msgr2=0x55e0b1aec3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.097+0000 7f50d90d6640 1 --2- 172.21.15.64:0/3660288728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0b171c2f0 0x55e0b1aec3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.097+0000 7f50d90d6640 1 -- 172.21.15.64:0/3660288728 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0b1aed7c0 con 0x55e0b1703d80 2024-09-21T04:04:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.097+0000 7f50d98d7640 1 --2- 172.21.15.64:0/3660288728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0b17244d0 0x55e0b1af7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:04:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.097+0000 7f50d88d5640 1 --2- 172.21.15.64:0/3660288728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0b171c2f0 0x55e0b1aec3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.098+0000 7f50d90d6640 1 --2- 172.21.15.64:0/3660288728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e0b1703d80 0x55e0b1af6630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f50d00dd780 tx=0x7f50d0032c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd4f9a5af5030546 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.098+0000 7f50beffd640 1 -- 172.21.15.64:0/3660288728 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d00d5fe0 con 0x55e0b1703d80 2024-09-21T04:04:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.098+0000 7f50beffd640 1 -- 172.21.15.64:0/3660288728 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50d00d61a0 con 0x55e0b1703d80 2024-09-21T04:04:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.098+0000 7f50beffd640 1 -- 172.21.15.64:0/3660288728 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d00e0020 con 0x55e0b1703d80 2024-09-21T04:04:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.099+0000 7f50e22a4740 1 -- 172.21.15.64:0/3660288728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e0b1703d80 msgr2=0x55e0b1af6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.099+0000 7f50e22a4740 1 --2- 172.21.15.64:0/3660288728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e0b1703d80 0x55e0b1af6630 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f50d00dd780 tx=0x7f50d0032c20 comp rx=0 tx=0).stop 2024-09-21T04:04:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.099+0000 7f50e22a4740 1 -- 172.21.15.64:0/3660288728 shutdown_connections 2024-09-21T04:04:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.099+0000 7f50e22a4740 1 --2- 172.21.15.64:0/3660288728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0b17244d0 0x55e0b1af7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.099+0000 7f50e22a4740 1 --2- 172.21.15.64:0/3660288728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0b171c2f0 0x55e0b1aec3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.099+0000 7f50e22a4740 1 --2- 172.21.15.64:0/3660288728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e0b1703d80 0x55e0b1af6630 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.099+0000 7f50e22a4740 1 -- 172.21.15.64:0/3660288728 >> 172.21.15.64:0/3660288728 conn(0x55e0b1739300 msgr2=0x55e0b1af7410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.099+0000 7f50e22a4740 1 -- 172.21.15.64:0/3660288728 shutdown_connections 2024-09-21T04:04:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.099+0000 7f50e22a4740 1 -- 172.21.15.64:0/3660288728 wait complete. 2024-09-21T04:04:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.100+0000 7f50e22a4740 1 -- start start 2024-09-21T04:04:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.100+0000 7f50e22a4740 1 Processor -- start 2024-09-21T04:04:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.100+0000 7f50e22a4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0b1703d80 0x55e0b1af9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.101+0000 7f50d90d6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0b1703d80 0x55e0b1af9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.101+0000 7f50d90d6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0b1703d80 0x55e0b1af9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34618/0 (socket says 172.21.15.64:34618) 2024-09-21T04:04:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.101+0000 7f50e22a4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e0b171c2f0 0x55e0b1af9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.101+0000 7f50d90d6640 1 -- 172.21.15.64:0/2626774283 learned_addr learned my addr 172.21.15.64:0/2626774283 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.101+0000 7f50e22a4740 1 --2- 172.21.15.64:0/2626774283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0b17244d0 0x55e0b1b6a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.101+0000 7f50e22a4740 1 -- 172.21.15.64:0/2626774283 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f50d41abde0 con 0x55e0b1703d80 2024-09-21T04:04:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.101+0000 7f50e22a4740 1 -- 172.21.15.64:0/2626774283 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f50d41ad250 con 0x55e0b171c2f0 2024-09-21T04:04:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.101+0000 7f50d88d5640 1 --2- 172.21.15.64:0/2626774283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e0b171c2f0 0x55e0b1af9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.101+0000 7f50e22a4740 1 -- 172.21.15.64:0/2626774283 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f50d41ac4b0 con 0x55e0b17244d0 2024-09-21T04:04:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.101+0000 7f50d98d7640 1 --2- 172.21.15.64:0/2626774283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0b17244d0 0x55e0b1b6a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.101+0000 7f50d88d5640 1 -- 172.21.15.64:0/2626774283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0b17244d0 msgr2=0x55e0b1b6a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.101+0000 7f50d88d5640 1 --2- 172.21.15.64:0/2626774283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0b17244d0 0x55e0b1b6a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.101+0000 7f50d88d5640 1 -- 172.21.15.64:0/2626774283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0b1703d80 msgr2=0x55e0b1af9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.101+0000 7f50d88d5640 1 --2- 172.21.15.64:0/2626774283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0b1703d80 0x55e0b1af9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.102+0000 7f50d88d5640 1 -- 172.21.15.64:0/2626774283 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0b1b6aa90 con 0x55e0b171c2f0 2024-09-21T04:04:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.102+0000 7f50d90d6640 1 --2- 172.21.15.64:0/2626774283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0b1703d80 0x55e0b1af9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.102+0000 7f50e22a4740 10 client.0 _ll_register_callbacks cb 0x7f50c400f460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.102+0000 7f50e22a4740 1 -- 172.21.15.64:0/2626774283 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50d00d61a0 con 0x55e0b171c2f0 2024-09-21T04:04:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.102+0000 7f50d88d5640 1 --2- 172.21.15.64:0/2626774283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e0b171c2f0 0x55e0b1af9d30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f50c4004650 tx=0x7f50c4005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f74f0336eb7fe2b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.102+0000 7f50bcff9640 10 client.25361 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:04:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.102+0000 7f50bcff9640 1 -- 172.21.15.64:0/2626774283 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c400fe50 con 0x55e0b171c2f0 2024-09-21T04:04:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.102+0000 7f50bcff9640 1 -- 172.21.15.64:0/2626774283 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f50c402b470 con 0x55e0b171c2f0 2024-09-21T04:04:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.102+0000 7f50bcff9640 1 -- 172.21.15.64:0/2626774283 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f50c402b6d0 con 0x55e0b171c2f0 2024-09-21T04:04:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.102+0000 7f50e22a4740 10 client.25361 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.102+0000 7f50e22a4740 1 -- 172.21.15.64:0/2626774283 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55e0b1b71be0 con 0x55e0b171c2f0 2024-09-21T04:04:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.103+0000 7f50bcff9640 1 -- 172.21.15.64:0/2626774283 <== mon.2 v2:172.21.15.119:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f50c401b450 con 0x55e0b171c2f0 2024-09-21T04:04:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.103+0000 7f50e22a4740 10 client.25361 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.103+0000 7f50e22a4740 10 client.25361 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:04:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.103+0000 7f50e22a4740 20 client.25361 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.103+0000 7f50e22a4740 1 --2- 172.21.15.64:0/2626774283 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55e0b17451b0 0x55e0b1b74310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.103+0000 7f50e22a4740 4 client.25361 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:04:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.103+0000 7f50e22a4740 1 -- 172.21.15.64:0/2626774283 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e0b1a5d0f0 con 0x55e0b17451b0 2024-09-21T04:04:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.103+0000 7f50d90d6640 1 --2- 172.21.15.64:0/2626774283 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55e0b17451b0 0x55e0b1b74310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.104+0000 7f50d90d6640 1 --2- 172.21.15.64:0/2626774283 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55e0b17451b0 0x55e0b1b74310 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d71890d9bf427e7 server_cookie=72c94ce5b08b68d3 in_seq=0 out_seq=0 2024-09-21T04:04:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.104+0000 7f50bcff9640 10 client.25361 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.112+0000 7f50bcff9640 1 -- 172.21.15.64:0/2626774283 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e0b1a5d0f0 con 0x55e0b17451b0 2024-09-21T04:04:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.112+0000 7f50bcff9640 10 client.25361 handle_command_reply: tid=0 2024-09-21T04:04:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.112+0000 7f50e22a4740 1 client.25361 shutdown 2024-09-21T04:04:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.112+0000 7f50e22a4740 1 -- 172.21.15.64:0/2626774283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e0b171c2f0 msgr2=0x55e0b1af9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.112+0000 7f50e22a4740 1 --2- 172.21.15.64:0/2626774283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e0b171c2f0 0x55e0b1af9d30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f50c4004650 tx=0x7f50c4005880 comp rx=0 tx=0).stop 2024-09-21T04:04:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.112+0000 7f50e22a4740 1 -- 172.21.15.64:0/2626774283 shutdown_connections 2024-09-21T04:04:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.112+0000 7f50e22a4740 1 --2- 172.21.15.64:0/2626774283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e0b17244d0 0x55e0b1b6a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.112+0000 7f50e22a4740 1 --2- 172.21.15.64:0/2626774283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e0b171c2f0 0x55e0b1af9d30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.112+0000 7f50e22a4740 1 --2- 172.21.15.64:0/2626774283 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55e0b17451b0 0x55e0b1b74310 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.112+0000 7f50e22a4740 1 --2- 172.21.15.64:0/2626774283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0b1703d80 0x55e0b1af9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.112+0000 7f50e22a4740 1 -- 172.21.15.64:0/2626774283 >> 172.21.15.64:0/2626774283 conn(0x55e0b1739300 msgr2=0x55e0b1af7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.112+0000 7f50bcff9640 0 client.25361 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.112+0000 7f50e22a4740 1 -- 172.21.15.64:0/2626774283 shutdown_connections 2024-09-21T04:04:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.112+0000 7f50e22a4740 1 -- 172.21.15.64:0/2626774283 wait complete. 2024-09-21T04:04:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.112+0000 7f50e22a4740 20 client.25361 trim_cache size 0 max 16384 2024-09-21T04:04:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:04:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:04:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:04:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:04:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.113+0000 7f50db33a640 1 -- 172.21.15.64:0/694238470 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f50c00b6810 msgr2=0x7f50c00b8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.113+0000 7f50db33a640 1 --2- 172.21.15.64:0/694238470 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f50c00b6810 0x7f50c00b8d20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f50cc005c40 tx=0x7f50cc009b50 comp rx=0 tx=0).stop 2024-09-21T04:04:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.113+0000 7f50db33a640 1 -- 172.21.15.64:0/694238470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50d4102060 msgr2=0x7f50d4198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.113+0000 7f50db33a640 1 --2- 172.21.15.64:0/694238470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50d4102060 0x7f50d4198300 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f50d0000c00 tx=0x7f50d00335c0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.113+0000 7f50db33a640 1 -- 172.21.15.64:0/694238470 shutdown_connections 2024-09-21T04:04:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.113+0000 7f50db33a640 1 --2- 172.21.15.64:0/694238470 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f50c00b6810 0x7f50c00b8d20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.113+0000 7f50db33a640 1 --2- 172.21.15.64:0/694238470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f50d4109d80 0x7f50d419db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.113+0000 7f50db33a640 1 --2- 172.21.15.64:0/694238470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50d4109430 0x7f50d41988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.113+0000 7f50db33a640 1 --2- 172.21.15.64:0/694238470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f50d4102060 0x7f50d4198300 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.113+0000 7f50db33a640 1 -- 172.21.15.64:0/694238470 >> 172.21.15.64:0/694238470 conn(0x7f50d4079e90 msgr2=0x7f50d4107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.114+0000 7f50db33a640 1 -- 172.21.15.64:0/694238470 shutdown_connections 2024-09-21T04:04:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:40.114+0000 7f50db33a640 1 -- 172.21.15.64:0/694238470 wait complete. 2024-09-21T04:04:41.706 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T04:04:41.706 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T04:04:42.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:41 smithi119 ceph-mon[34811]: pgmap v341: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-09-21T04:04:42.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:41 smithi064 ceph-mon[31180]: pgmap v341: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-09-21T04:04:42.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:41 smithi131 ceph-mon[35193]: pgmap v341: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-09-21T04:04:42.111 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:04:42.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.646+0000 7fb695511640 1 -- 172.21.15.64:0/1894959391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb69010de30 msgr2=0x7fb690110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:42.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.646+0000 7fb695511640 1 --2- 172.21.15.64:0/1894959391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb69010de30 0x7fb690110320 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb688009ca0 tx=0x7fb68802a8c0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.646+0000 7fb695511640 1 -- 172.21.15.64:0/1894959391 shutdown_connections 2024-09-21T04:04:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.646+0000 7fb695511640 1 --2- 172.21.15.64:0/1894959391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb69010de30 0x7fb690110320 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.646+0000 7fb695511640 1 --2- 172.21.15.64:0/1894959391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb690105320 0x7fb69010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.646+0000 7fb695511640 1 --2- 172.21.15.64:0/1894959391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6901049d0 0x7fb690104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.646+0000 7fb695511640 1 -- 172.21.15.64:0/1894959391 >> 172.21.15.64:0/1894959391 conn(0x7fb690079740 msgr2=0x7fb690079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.647+0000 7fb695511640 1 -- 172.21.15.64:0/1894959391 shutdown_connections 2024-09-21T04:04:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.647+0000 7fb695511640 1 -- 172.21.15.64:0/1894959391 wait complete. 2024-09-21T04:04:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.648+0000 7fb695511640 1 Processor -- start 2024-09-21T04:04:42.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.648+0000 7fb695511640 1 -- start start 2024-09-21T04:04:42.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.649+0000 7fb695511640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6901049d0 0x7fb69019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:42.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.649+0000 7fb695511640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb690105320 0x7fb69019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.649+0000 7fb695511640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb69010de30 0x7fb69019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.649+0000 7fb695511640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb690113a90 con 0x7fb69010de30 2024-09-21T04:04:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.649+0000 7fb695511640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb690113910 con 0x7fb690105320 2024-09-21T04:04:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.649+0000 7fb695511640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb690113c10 con 0x7fb6901049d0 2024-09-21T04:04:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.649+0000 7fb68effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6901049d0 0x7fb69019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.649+0000 7fb68f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb69010de30 0x7fb69019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.649+0000 7fb68f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb69010de30 0x7fb69019fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57096/0 (socket says 172.21.15.64:57096) 2024-09-21T04:04:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.649+0000 7fb68f7fe640 1 -- 172.21.15.64:0/3542117543 learned_addr learned my addr 172.21.15.64:0/3542117543 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.649+0000 7fb68e7fc640 1 --2- 172.21.15.64:0/3542117543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb690105320 0x7fb69019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:42.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.650+0000 7fb68effd640 1 -- 172.21.15.64:0/3542117543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb690105320 msgr2=0x7fb69019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:42.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.650+0000 7fb68effd640 1 --2- 172.21.15.64:0/3542117543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb690105320 0x7fb69019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.650+0000 7fb68effd640 1 -- 172.21.15.64:0/3542117543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb69010de30 msgr2=0x7fb69019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:42.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.650+0000 7fb68effd640 1 --2- 172.21.15.64:0/3542117543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb69010de30 0x7fb69019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.650+0000 7fb68effd640 1 -- 172.21.15.64:0/3542117543 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6901a03f0 con 0x7fb6901049d0 2024-09-21T04:04:42.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.650+0000 7fb68f7fe640 1 --2- 172.21.15.64:0/3542117543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb69010de30 0x7fb69019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:42.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.650+0000 7fb68effd640 1 --2- 172.21.15.64:0/3542117543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6901049d0 0x7fb69019a480 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb69006c620 tx=0x7fb68000dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:42.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.651+0000 7fb677fff640 1 -- 172.21.15.64:0/3542117543 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb680017070 con 0x7fb6901049d0 2024-09-21T04:04:42.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.651+0000 7fb677fff640 1 -- 172.21.15.64:0/3542117543 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb680009c30 con 0x7fb6901049d0 2024-09-21T04:04:42.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.651+0000 7fb695511640 1 -- 172.21.15.64:0/3542117543 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6901a06e0 con 0x7fb6901049d0 2024-09-21T04:04:42.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.651+0000 7fb677fff640 1 -- 172.21.15.64:0/3542117543 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68000f620 con 0x7fb6901049d0 2024-09-21T04:04:42.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.652+0000 7fb695511640 1 -- 172.21.15.64:0/3542117543 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6901a6e40 con 0x7fb6901049d0 2024-09-21T04:04:42.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.655+0000 7fb677fff640 1 -- 172.21.15.64:0/3542117543 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 25) ==== 154694+0+0 (secure 0 0 0) 0x7fb68001a070 con 0x7fb6901049d0 2024-09-21T04:04:42.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.656+0000 7fb69ba85740 1 Processor -- start 2024-09-21T04:04:42.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.656+0000 7fb69ba85740 1 -- start start 2024-09-21T04:04:42.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.656+0000 7fb677fff640 1 --2- 172.21.15.64:0/3542117543 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb6780befa0 0x7fb6780c14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:42.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.656+0000 7fb69ba85740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9b990e350 0x55f9b9cb5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:42.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.656+0000 7fb69ba85740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9b98e5490 0x55f9b9cb5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:42.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.656+0000 7fb69ba85740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9b98ed670 0x55f9b9cb5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:42.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.656+0000 7fb677fff640 1 -- 172.21.15.64:0/3542117543 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fb6800e6a70 con 0x7fb6901049d0 2024-09-21T04:04:42.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.656+0000 7fb69ba85740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f9b9c2be60 con 0x55f9b98e5490 2024-09-21T04:04:42.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.656+0000 7fb69ba85740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f9b9c1b290 con 0x55f9b98ed670 2024-09-21T04:04:42.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.656+0000 7fb69ba85740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f9b9c1f0a0 con 0x55f9b990e350 2024-09-21T04:04:42.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.657+0000 7fb68effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9b990e350 0x55f9b9cb5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:42.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.657+0000 7fb68effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9b990e350 0x55f9b9cb5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:33438/0 (socket says 172.21.15.64:33438) 2024-09-21T04:04:42.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.657+0000 7fb68effd640 1 -- 172.21.15.64:0/4287890952 learned_addr learned my addr 172.21.15.64:0/4287890952 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:42.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.657+0000 7fb68e7fc640 1 --2- 172.21.15.64:0/4287890952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9b98e5490 0x55f9b9cb5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:42.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.657+0000 7fb68effd640 1 -- 172.21.15.64:0/4287890952 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9b98ed670 msgr2=0x55f9b9cb5eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T04:04:42.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.657+0000 7fb68effd640 1 --2- 172.21.15.64:0/4287890952 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9b98ed670 0x55f9b9cb5eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.657+0000 7fb68effd640 1 -- 172.21.15.64:0/4287890952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9b98e5490 msgr2=0x55f9b9cb5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:42.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.657+0000 7fb68effd640 1 --2- 172.21.15.64:0/4287890952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9b98e5490 0x55f9b9cb5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.657+0000 7fb68f7fe640 1 --2- 172.21.15.64:0/3542117543 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb6780befa0 0x7fb6780c14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:42.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.657+0000 7fb68effd640 1 -- 172.21.15.64:0/4287890952 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9b9979930 con 0x55f9b990e350 2024-09-21T04:04:42.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.658+0000 7fb68effd640 1 --2- 172.21.15.64:0/4287890952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9b990e350 0x55f9b9cb5430 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb6800e4100 tx=0x7fb6800abe30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db22ee93b832240d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:42.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.658+0000 7fb68f7fe640 1 --2- 172.21.15.64:0/3542117543 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb6780befa0 0x7fb6780c14b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb688005e30 tx=0x7fb688005c50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:42.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.658+0000 7fb6757fa640 1 -- 172.21.15.64:0/4287890952 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6800b11a0 con 0x55f9b990e350 2024-09-21T04:04:42.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.658+0000 7fb6757fa640 1 -- 172.21.15.64:0/4287890952 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6800b60e0 con 0x55f9b990e350 2024-09-21T04:04:42.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.659+0000 7fb6757fa640 1 -- 172.21.15.64:0/4287890952 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6800ab5f0 con 0x55f9b990e350 2024-09-21T04:04:42.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.659+0000 7fb69ba85740 1 -- 172.21.15.64:0/4287890952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9b990e350 msgr2=0x55f9b9cb5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:42.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.659+0000 7fb69ba85740 1 --2- 172.21.15.64:0/4287890952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9b990e350 0x55f9b9cb5430 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb6800e4100 tx=0x7fb6800abe30 comp rx=0 tx=0).stop 2024-09-21T04:04:42.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.659+0000 7fb69ba85740 1 -- 172.21.15.64:0/4287890952 shutdown_connections 2024-09-21T04:04:42.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.659+0000 7fb69ba85740 1 --2- 172.21.15.64:0/4287890952 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9b98ed670 0x55f9b9cb5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.659+0000 7fb69ba85740 1 --2- 172.21.15.64:0/4287890952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9b98e5490 0x55f9b9cb5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.659+0000 7fb69ba85740 1 --2- 172.21.15.64:0/4287890952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9b990e350 0x55f9b9cb5430 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.659+0000 7fb69ba85740 1 -- 172.21.15.64:0/4287890952 >> 172.21.15.64:0/4287890952 conn(0x55f9b9902740 msgr2=0x55f9b9cb3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:42.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.659+0000 7fb69ba85740 1 -- 172.21.15.64:0/4287890952 shutdown_connections 2024-09-21T04:04:42.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.659+0000 7fb69ba85740 1 -- 172.21.15.64:0/4287890952 wait complete. 2024-09-21T04:04:42.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.659+0000 7fb69ba85740 1 -- start start 2024-09-21T04:04:42.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.659+0000 7fb69ba85740 1 Processor -- start 2024-09-21T04:04:42.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.660+0000 7fb69ba85740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9b98e5490 0x55f9b9cc0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:42.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.660+0000 7fb69ba85740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9b98ed670 0x55f9b9cc1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:42.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.660+0000 7fb69ba85740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9b990e350 0x55f9b9cccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:42.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.660+0000 7fb69ba85740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f9b9a7c890 con 0x55f9b98e5490 2024-09-21T04:04:42.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.660+0000 7fb69ba85740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f9b9bb5d40 con 0x55f9b990e350 2024-09-21T04:04:42.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.660+0000 7fb69ba85740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f9b9cc3cb0 con 0x55f9b98ed670 2024-09-21T04:04:42.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.660+0000 7fb69ba85740 10 client.0 _ll_register_callbacks cb 0x55f9b9c19c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:42.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.660+0000 7fb68e7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9b98ed670 0x55f9b9cc1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:42.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.660+0000 7fb68e7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9b98ed670 0x55f9b9cc1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:33452/0 (socket says 172.21.15.64:33452) 2024-09-21T04:04:42.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.660+0000 7fb68e7fc640 1 -- 172.21.15.64:0/2793245871 learned_addr learned my addr 172.21.15.64:0/2793245871 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:42.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.660+0000 7fb68effd640 1 --2- 172.21.15.64:0/2793245871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9b98e5490 0x55f9b9cc0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:42.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.660+0000 7fb68f7fe640 1 --2- 172.21.15.64:0/2793245871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9b990e350 0x55f9b9cccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:42.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.660+0000 7fb68effd640 1 -- 172.21.15.64:0/2793245871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9b98ed670 msgr2=0x55f9b9cc1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:42.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.660+0000 7fb68effd640 1 --2- 172.21.15.64:0/2793245871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9b98ed670 0x55f9b9cc1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.660+0000 7fb68effd640 1 -- 172.21.15.64:0/2793245871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9b990e350 msgr2=0x55f9b9cccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.660+0000 7fb68effd640 1 --2- 172.21.15.64:0/2793245871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9b990e350 0x55f9b9cccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.660+0000 7fb68effd640 1 -- 172.21.15.64:0/2793245871 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9b9ccd530 con 0x55f9b98e5490 2024-09-21T04:04:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.661+0000 7fb68effd640 1 --2- 172.21.15.64:0/2793245871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9b98e5490 0x55f9b9cc0f40 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb680009610 tx=0x7fb6800a8b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360a11e9461464ba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.661+0000 7fb672ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:04:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.661+0000 7fb672ff5640 1 -- 172.21.15.64:0/2793245871 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6800a8360 con 0x55f9b98e5490 2024-09-21T04:04:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.661+0000 7fb672ff5640 1 -- 172.21.15.64:0/2793245871 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fb680009de0 con 0x55f9b98e5490 2024-09-21T04:04:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.661+0000 7fb69ba85740 1 -- 172.21.15.64:0/2793245871 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6800b60e0 con 0x55f9b98e5490 2024-09-21T04:04:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.661+0000 7fb672ff5640 1 -- 172.21.15.64:0/2793245871 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb6800b11a0 con 0x55f9b98e5490 2024-09-21T04:04:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.661+0000 7fb69ba85740 10 client.15633 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.661+0000 7fb69ba85740 1 -- 172.21.15.64:0/2793245871 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f9b9ccfe30 con 0x55f9b98e5490 2024-09-21T04:04:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.662+0000 7fb672ff5640 1 -- 172.21.15.64:0/2793245871 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6800b60e0 con 0x55f9b98e5490 2024-09-21T04:04:42.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.662+0000 7fb672ff5640 1 -- 172.21.15.64:0/2793245871 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb6800b2330 con 0x55f9b98e5490 2024-09-21T04:04:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.662+0000 7fb69ba85740 10 client.15633 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.662+0000 7fb69ba85740 10 client.15633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:04:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.662+0000 7fb69ba85740 20 client.15633 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.662+0000 7fb69ba85740 1 --2- 172.21.15.64:0/2793245871 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f9b98cccd0 0x55f9b9cd3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.662+0000 7fb69ba85740 4 client.15633 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:04:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.662+0000 7fb69ba85740 1 -- 172.21.15.64:0/2793245871 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9b9c2ddb0 con 0x55f9b98cccd0 2024-09-21T04:04:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.662+0000 7fb68e7fc640 1 --2- 172.21.15.64:0/2793245871 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f9b98cccd0 0x55f9b9cd3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.663+0000 7fb68e7fc640 1 --2- 172.21.15.64:0/2793245871 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f9b98cccd0 0x55f9b9cd3190 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68918527313525c7 server_cookie=1ce1c122df171c65 in_seq=0 out_seq=0 2024-09-21T04:04:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.663+0000 7fb672ff5640 10 client.15633 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.663+0000 7fb672ff5640 1 -- 172.21.15.64:0/2793245871 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f9b9c2ddb0 con 0x55f9b98cccd0 2024-09-21T04:04:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.663+0000 7fb672ff5640 10 client.15633 handle_command_reply: tid=0 2024-09-21T04:04:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.664+0000 7fb69ba85740 1 client.15633 shutdown 2024-09-21T04:04:42.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.664+0000 7fb69ba85740 1 -- 172.21.15.64:0/2793245871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9b98e5490 msgr2=0x55f9b9cc0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:42.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.664+0000 7fb69ba85740 1 --2- 172.21.15.64:0/2793245871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9b98e5490 0x55f9b9cc0f40 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb680009610 tx=0x7fb6800a8b40 comp rx=0 tx=0).stop 2024-09-21T04:04:42.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.664+0000 7fb69ba85740 1 -- 172.21.15.64:0/2793245871 shutdown_connections 2024-09-21T04:04:42.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.664+0000 7fb69ba85740 1 --2- 172.21.15.64:0/2793245871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9b990e350 0x55f9b9cccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.664+0000 7fb69ba85740 1 --2- 172.21.15.64:0/2793245871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9b98ed670 0x55f9b9cc1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.664+0000 7fb69ba85740 1 --2- 172.21.15.64:0/2793245871 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f9b98cccd0 0x55f9b9cd3190 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.664+0000 7fb69ba85740 1 --2- 172.21.15.64:0/2793245871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9b98e5490 0x55f9b9cc0f40 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.664+0000 7fb69ba85740 1 -- 172.21.15.64:0/2793245871 >> 172.21.15.64:0/2793245871 conn(0x55f9b9902740 msgr2=0x55f9b9cb8610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:42.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.664+0000 7fb672ff5640 0 client.15633 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:42.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.664+0000 7fb69ba85740 1 -- 172.21.15.64:0/2793245871 shutdown_connections 2024-09-21T04:04:42.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.664+0000 7fb69ba85740 1 -- 172.21.15.64:0/2793245871 wait complete. 2024-09-21T04:04:42.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.664+0000 7fb69ba85740 20 client.15633 trim_cache size 0 max 16384 2024-09-21T04:04:42.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.682+0000 7fb69ba85740 1 Processor -- start 2024-09-21T04:04:42.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.682+0000 7fb69ba85740 1 -- start start 2024-09-21T04:04:42.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.683+0000 7fb69ba85740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9b98e5490 0x55f9b9cbb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:42.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.683+0000 7fb69ba85740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9b98ed670 0x55f9b9cb6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:42.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.683+0000 7fb69ba85740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9b990e350 0x55f9b9cc05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:42.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.683+0000 7fb69ba85740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb6901a7780 con 0x55f9b990e350 2024-09-21T04:04:42.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.683+0000 7fb69ba85740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb690073630 con 0x55f9b98e5490 2024-09-21T04:04:42.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.683+0000 7fb69ba85740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb690071480 con 0x55f9b98ed670 2024-09-21T04:04:42.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.683+0000 7fb68f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9b990e350 0x55f9b9cc05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:42.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.683+0000 7fb68f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9b990e350 0x55f9b9cc05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57132/0 (socket says 172.21.15.64:57132) 2024-09-21T04:04:42.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.683+0000 7fb68f7fe640 1 -- 172.21.15.64:0/2228366406 learned_addr learned my addr 172.21.15.64:0/2228366406 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:42.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.683+0000 7fb68effd640 1 --2- 172.21.15.64:0/2228366406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9b98e5490 0x55f9b9cbb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:42.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.683+0000 7fb68e7fc640 1 --2- 172.21.15.64:0/2228366406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9b98ed670 0x55f9b9cb6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:42.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.684+0000 7fb68f7fe640 1 -- 172.21.15.64:0/2228366406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9b98ed670 msgr2=0x55f9b9cb6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:42.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.684+0000 7fb68f7fe640 1 --2- 172.21.15.64:0/2228366406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9b98ed670 0x55f9b9cb6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.684+0000 7fb68f7fe640 1 -- 172.21.15.64:0/2228366406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9b98e5490 msgr2=0x55f9b9cbb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:42.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.684+0000 7fb68f7fe640 1 --2- 172.21.15.64:0/2228366406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9b98e5490 0x55f9b9cbb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.684+0000 7fb68f7fe640 1 -- 172.21.15.64:0/2228366406 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9b9caeac0 con 0x55f9b990e350 2024-09-21T04:04:42.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.684+0000 7fb68f7fe640 1 --2- 172.21.15.64:0/2228366406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9b990e350 0x55f9b9cc05d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb6880316e0 tx=0x7fb688040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c6f840ba9118d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:42.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.684+0000 7fb674ff9640 1 -- 172.21.15.64:0/2228366406 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb688031710 con 0x55f9b990e350 2024-09-21T04:04:42.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.684+0000 7fb674ff9640 1 -- 172.21.15.64:0/2228366406 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb688036040 con 0x55f9b990e350 2024-09-21T04:04:42.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.684+0000 7fb674ff9640 1 -- 172.21.15.64:0/2228366406 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68802fa90 con 0x55f9b990e350 2024-09-21T04:04:42.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.685+0000 7fb69ba85740 1 -- 172.21.15.64:0/2228366406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9b990e350 msgr2=0x55f9b9cc05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:42.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.685+0000 7fb69ba85740 1 --2- 172.21.15.64:0/2228366406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9b990e350 0x55f9b9cc05d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb6880316e0 tx=0x7fb688040000 comp rx=0 tx=0).stop 2024-09-21T04:04:42.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.685+0000 7fb69ba85740 1 -- 172.21.15.64:0/2228366406 shutdown_connections 2024-09-21T04:04:42.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.685+0000 7fb69ba85740 1 --2- 172.21.15.64:0/2228366406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9b990e350 0x55f9b9cc05d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.685+0000 7fb69ba85740 1 --2- 172.21.15.64:0/2228366406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9b98ed670 0x55f9b9cb6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.685+0000 7fb69ba85740 1 --2- 172.21.15.64:0/2228366406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9b98e5490 0x55f9b9cbb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.685+0000 7fb69ba85740 1 -- 172.21.15.64:0/2228366406 >> 172.21.15.64:0/2228366406 conn(0x55f9b9902740 msgr2=0x55f9b9cc28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:42.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.686+0000 7fb69ba85740 1 -- 172.21.15.64:0/2228366406 shutdown_connections 2024-09-21T04:04:42.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.686+0000 7fb69ba85740 1 -- 172.21.15.64:0/2228366406 wait complete. 2024-09-21T04:04:42.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.686+0000 7fb69ba85740 1 -- start start 2024-09-21T04:04:42.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.686+0000 7fb69ba85740 1 Processor -- start 2024-09-21T04:04:42.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.686+0000 7fb69ba85740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9b98e5490 0x55f9b9cc9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:42.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.686+0000 7fb69ba85740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9b98ed670 0x55f9b9cc9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:42.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.687+0000 7fb68e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9b98ed670 0x55f9b9cc9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:42.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.687+0000 7fb68e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9b98ed670 0x55f9b9cc9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57140/0 (socket says 172.21.15.64:57140) 2024-09-21T04:04:42.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.687+0000 7fb68e7fc640 1 -- 172.21.15.64:0/1279349886 learned_addr learned my addr 172.21.15.64:0/1279349886 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:42.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.687+0000 7fb69ba85740 1 --2- 172.21.15.64:0/1279349886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9b990e350 0x55f9b9cc3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:42.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.687+0000 7fb69ba85740 1 -- 172.21.15.64:0/1279349886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb690071af0 con 0x55f9b98ed670 2024-09-21T04:04:42.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.687+0000 7fb69ba85740 1 -- 172.21.15.64:0/1279349886 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb690072f60 con 0x55f9b98e5490 2024-09-21T04:04:42.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.687+0000 7fb69ba85740 1 -- 172.21.15.64:0/1279349886 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb6900721c0 con 0x55f9b990e350 2024-09-21T04:04:42.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.687+0000 7fb69ba85740 10 client.0 _ll_register_callbacks cb 0x7fb68400b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:42.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.687+0000 7fb68effd640 1 --2- 172.21.15.64:0/1279349886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9b98e5490 0x55f9b9cc9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:42.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.687+0000 7fb68f7fe640 1 --2- 172.21.15.64:0/1279349886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9b990e350 0x55f9b9cc3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:42.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.687+0000 7fb68e7fc640 1 -- 172.21.15.64:0/1279349886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9b990e350 msgr2=0x55f9b9cc3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:42.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.687+0000 7fb68e7fc640 1 --2- 172.21.15.64:0/1279349886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9b990e350 0x55f9b9cc3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.687+0000 7fb68e7fc640 1 -- 172.21.15.64:0/1279349886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9b98e5490 msgr2=0x55f9b9cc9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:42.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.687+0000 7fb68e7fc640 1 --2- 172.21.15.64:0/1279349886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9b98e5490 0x55f9b9cc9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.687+0000 7fb68e7fc640 1 -- 172.21.15.64:0/1279349886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9b9cc4620 con 0x55f9b98ed670 2024-09-21T04:04:42.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.688+0000 7fb68e7fc640 1 --2- 172.21.15.64:0/1279349886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9b98ed670 0x55f9b9cc9740 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb68400b7b0 tx=0x7fb6840044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4781e6e224ddc1a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:42.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.688+0000 7fb672ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:04:42.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.688+0000 7fb672ff5640 1 -- 172.21.15.64:0/1279349886 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb684033650 con 0x55f9b98ed670 2024-09-21T04:04:42.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.688+0000 7fb672ff5640 1 -- 172.21.15.64:0/1279349886 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fb6840337f0 con 0x55f9b98ed670 2024-09-21T04:04:42.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.688+0000 7fb69ba85740 1 -- 172.21.15.64:0/1279349886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb688036040 con 0x55f9b98ed670 2024-09-21T04:04:42.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.688+0000 7fb672ff5640 1 -- 172.21.15.64:0/1279349886 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb6840339f0 con 0x55f9b98ed670 2024-09-21T04:04:42.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.688+0000 7fb69ba85740 10 client.15639 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:42.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.688+0000 7fb69ba85740 1 -- 172.21.15.64:0/1279349886 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f9b9d5fc00 con 0x55f9b98ed670 2024-09-21T04:04:42.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.688+0000 7fb672ff5640 1 -- 172.21.15.64:0/1279349886 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb688036040 con 0x55f9b98ed670 2024-09-21T04:04:42.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.689+0000 7fb672ff5640 1 -- 172.21.15.64:0/1279349886 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb68404c070 con 0x55f9b98ed670 2024-09-21T04:04:42.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.689+0000 7fb69ba85740 10 client.15639 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:42.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.689+0000 7fb69ba85740 10 client.15639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:04:42.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.689+0000 7fb69ba85740 20 client.15639 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:42.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.689+0000 7fb69ba85740 1 --2- 172.21.15.64:0/1279349886 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f9b98a2080 0x55f9b9d62420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:42.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.689+0000 7fb69ba85740 4 client.15639 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T04:04:42.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.689+0000 7fb69ba85740 1 -- 172.21.15.64:0/1279349886 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f9b9c29860 con 0x55f9b98a2080 2024-09-21T04:04:42.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.689+0000 7fb68effd640 1 --2- 172.21.15.64:0/1279349886 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f9b98a2080 0x55f9b9d62420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:42.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.690+0000 7fb672ff5640 10 client.15639 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:42.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.690+0000 7fb68effd640 1 --2- 172.21.15.64:0/1279349886 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f9b98a2080 0x55f9b9d62420 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51fe443c506c09b1 server_cookie=1f53d3b3f390afd1 in_seq=0 out_seq=0 2024-09-21T04:04:42.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.692+0000 7fb672ff5640 1 -- 172.21.15.64:0/1279349886 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f9b9c29860 con 0x55f9b98a2080 2024-09-21T04:04:42.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.692+0000 7fb672ff5640 10 client.15639 handle_command_reply: tid=0 2024-09-21T04:04:42.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.692+0000 7fb69ba85740 1 client.15639 shutdown 2024-09-21T04:04:42.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.692+0000 7fb69ba85740 1 -- 172.21.15.64:0/1279349886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9b98ed670 msgr2=0x55f9b9cc9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:42.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.692+0000 7fb69ba85740 1 --2- 172.21.15.64:0/1279349886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9b98ed670 0x55f9b9cc9740 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb68400b7b0 tx=0x7fb6840044d0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.692+0000 7fb69ba85740 1 -- 172.21.15.64:0/1279349886 shutdown_connections 2024-09-21T04:04:42.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.692+0000 7fb69ba85740 1 --2- 172.21.15.64:0/1279349886 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9b990e350 0x55f9b9cc3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.692+0000 7fb69ba85740 1 --2- 172.21.15.64:0/1279349886 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f9b98a2080 0x55f9b9d62420 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.692+0000 7fb672ff5640 0 client.15639 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:42.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.692+0000 7fb69ba85740 1 --2- 172.21.15.64:0/1279349886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9b98ed670 0x55f9b9cc9740 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.692+0000 7fb69ba85740 1 --2- 172.21.15.64:0/1279349886 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9b98e5490 0x55f9b9cc9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.692+0000 7fb69ba85740 1 -- 172.21.15.64:0/1279349886 >> 172.21.15.64:0/1279349886 conn(0x55f9b9902740 msgr2=0x55f9b9cba540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:42.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.693+0000 7fb69ba85740 1 -- 172.21.15.64:0/1279349886 shutdown_connections 2024-09-21T04:04:42.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.693+0000 7fb69ba85740 1 -- 172.21.15.64:0/1279349886 wait complete. 2024-09-21T04:04:42.697 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.693+0000 7fb69ba85740 20 client.15639 trim_cache size 0 max 16384 2024-09-21T04:04:42.697 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:04:42.697 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-21T04:04:42.697 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "0ab717a7-36c6-4dca-94e7-af0bdf2aa112", 2024-09-21T04:04:42.698 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-21T04:04:42.698 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:04:42.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.693+0000 7fb695511640 1 -- 172.21.15.64:0/3542117543 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb6780befa0 msgr2=0x7fb6780c14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:42.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.693+0000 7fb695511640 1 --2- 172.21.15.64:0/3542117543 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb6780befa0 0x7fb6780c14b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb688005e30 tx=0x7fb688005c50 comp rx=0 tx=0).stop 2024-09-21T04:04:42.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.693+0000 7fb695511640 1 -- 172.21.15.64:0/3542117543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6901049d0 msgr2=0x7fb69019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:42.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.693+0000 7fb695511640 1 --2- 172.21.15.64:0/3542117543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6901049d0 0x7fb69019a480 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb69006c620 tx=0x7fb68000dde0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.694+0000 7fb695511640 1 -- 172.21.15.64:0/3542117543 shutdown_connections 2024-09-21T04:04:42.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.694+0000 7fb695511640 1 --2- 172.21.15.64:0/3542117543 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb6780befa0 0x7fb6780c14b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.694+0000 7fb695511640 1 --2- 172.21.15.64:0/3542117543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb69010de30 0x7fb69019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.694+0000 7fb695511640 1 --2- 172.21.15.64:0/3542117543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb690105320 0x7fb69019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.694+0000 7fb695511640 1 --2- 172.21.15.64:0/3542117543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6901049d0 0x7fb69019a480 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:42.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.694+0000 7fb695511640 1 -- 172.21.15.64:0/3542117543 >> 172.21.15.64:0/3542117543 conn(0x7fb690079740 msgr2=0x7fb6900ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:42.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.694+0000 7fb695511640 1 -- 172.21.15.64:0/3542117543 shutdown_connections 2024-09-21T04:04:42.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:42.694+0000 7fb695511640 1 -- 172.21.15.64:0/3542117543 wait complete. 2024-09-21T04:04:43.445 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T04:04:43.842 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:04:43.975 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:43 smithi064 ceph-mon[31180]: pgmap v342: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-21T04:04:43.975 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:43 smithi064 ceph-mon[31180]: scrub queued for path: / 2024-09-21T04:04:43.975 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:43 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:04:43.975 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:43 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:04:43.975 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:43 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:04:43.975 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:43 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:04:44.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:43 smithi119 ceph-mon[34811]: pgmap v342: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-21T04:04:44.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:43 smithi119 ceph-mon[34811]: scrub queued for path: / 2024-09-21T04:04:44.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:43 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:04:44.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:43 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:04:44.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:43 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:04:44.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:43 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:04:44.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:43 smithi131 ceph-mon[35193]: pgmap v342: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-21T04:04:44.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:43 smithi131 ceph-mon[35193]: scrub queued for path: / 2024-09-21T04:04:44.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:43 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:04:44.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:43 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:04:44.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:43 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:04:44.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:43 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:04:44.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.398+0000 7f7dbbebc640 1 -- 172.21.15.64:0/2541008616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7db4106c20 msgr2=0x7f7db4110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:44.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.398+0000 7f7dbbebc640 1 --2- 172.21.15.64:0/2541008616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7db4106c20 0x7f7db4110670 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7dac009ca0 tx=0x7f7dac02a8c0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.398+0000 7f7dbbebc640 1 -- 172.21.15.64:0/2541008616 shutdown_connections 2024-09-21T04:04:44.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.398+0000 7f7dbbebc640 1 --2- 172.21.15.64:0/2541008616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7db4106c20 0x7f7db4110670 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.398+0000 7f7dbbebc640 1 --2- 172.21.15.64:0/2541008616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7db41062d0 0x7f7db41066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.398+0000 7f7dbbebc640 1 --2- 172.21.15.64:0/2541008616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db4101ff0 0x7f7db4102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.398+0000 7f7dbbebc640 1 -- 172.21.15.64:0/2541008616 >> 172.21.15.64:0/2541008616 conn(0x7f7db4077cc0 msgr2=0x7f7db4078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:44.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.398+0000 7f7dbbebc640 1 -- 172.21.15.64:0/2541008616 shutdown_connections 2024-09-21T04:04:44.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.398+0000 7f7dbbebc640 1 -- 172.21.15.64:0/2541008616 wait complete. 2024-09-21T04:04:44.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.399+0000 7f7dbbebc640 1 Processor -- start 2024-09-21T04:04:44.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.399+0000 7f7dbbebc640 1 -- start start 2024-09-21T04:04:44.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.400+0000 7f7dbbebc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7db4101ff0 0x7f7db4102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:44.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.400+0000 7f7dbbebc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7db41062d0 0x7f7db41034d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:44.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.400+0000 7f7db9457640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7db41062d0 0x7f7db41034d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:44.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.400+0000 7f7dbbebc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db4106c20 0x7f7db4103a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:44.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.400+0000 7f7db9c58640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7db4101ff0 0x7f7db4102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:44.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.400+0000 7f7db9c58640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7db4101ff0 0x7f7db4102f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:33488/0 (socket says 172.21.15.64:33488) 2024-09-21T04:04:44.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.400+0000 7f7db9c58640 1 -- 172.21.15.64:0/1415096448 learned_addr learned my addr 172.21.15.64:0/1415096448 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:44.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.400+0000 7f7dbbebc640 1 -- 172.21.15.64:0/1415096448 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7db4113ac0 con 0x7f7db41062d0 2024-09-21T04:04:44.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.400+0000 7f7dbbebc640 1 -- 172.21.15.64:0/1415096448 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7db4113940 con 0x7f7db4106c20 2024-09-21T04:04:44.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.400+0000 7f7dbbebc640 1 -- 172.21.15.64:0/1415096448 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7db4113c40 con 0x7f7db4101ff0 2024-09-21T04:04:44.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.400+0000 7f7dba459640 1 --2- 172.21.15.64:0/1415096448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db4106c20 0x7f7db4103a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:44.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.400+0000 7f7db9457640 1 -- 172.21.15.64:0/1415096448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7db4101ff0 msgr2=0x7f7db4102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:44.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.400+0000 7f7db9457640 1 --2- 172.21.15.64:0/1415096448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7db4101ff0 0x7f7db4102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.401+0000 7f7db9457640 1 -- 172.21.15.64:0/1415096448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db4106c20 msgr2=0x7f7db4103a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:44.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.401+0000 7f7db9457640 1 --2- 172.21.15.64:0/1415096448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db4106c20 0x7f7db4103a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.401+0000 7f7db9457640 1 -- 172.21.15.64:0/1415096448 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db41a8710 con 0x7f7db41062d0 2024-09-21T04:04:44.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.401+0000 7f7db9457640 1 --2- 172.21.15.64:0/1415096448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7db41062d0 0x7f7db41034d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7db4075ad0 tx=0x7f7da400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:44.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.401+0000 7f7daaffd640 1 -- 172.21.15.64:0/1415096448 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da400cc40 con 0x7f7db41062d0 2024-09-21T04:04:44.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.401+0000 7f7daaffd640 1 -- 172.21.15.64:0/1415096448 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7da400cde0 con 0x7f7db41062d0 2024-09-21T04:04:44.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.402+0000 7f7daaffd640 1 -- 172.21.15.64:0/1415096448 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da4018780 con 0x7f7db41062d0 2024-09-21T04:04:44.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.402+0000 7f7dbbebc640 1 -- 172.21.15.64:0/1415096448 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db41a8a00 con 0x7f7db41062d0 2024-09-21T04:04:44.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.403+0000 7f7dbbebc640 1 -- 172.21.15.64:0/1415096448 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db41a8ec0 con 0x7f7db41062d0 2024-09-21T04:04:44.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.405+0000 7f7daaffd640 1 -- 172.21.15.64:0/1415096448 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 154694+0+0 (secure 0 0 0) 0x7f7da401f020 con 0x7f7db41062d0 2024-09-21T04:04:44.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.406+0000 7f7daaffd640 1 --2- 172.21.15.64:0/1415096448 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7da00b65f0 0x7f7da00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:44.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.407+0000 7f7daaffd640 1 -- 172.21.15.64:0/1415096448 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f7da40e7b50 con 0x7f7db41062d0 2024-09-21T04:04:44.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.407+0000 7f7db9c58640 1 --2- 172.21.15.64:0/1415096448 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7da00b65f0 0x7f7da00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:44.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.407+0000 7f7dc2430740 1 Processor -- start 2024-09-21T04:04:44.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.407+0000 7f7dc2430740 1 -- start start 2024-09-21T04:04:44.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.407+0000 7f7dc2430740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617176aa070 0x561717a51610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:44.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.407+0000 7f7dc2430740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617176811b0 0x561717a51b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:44.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.407+0000 7f7dc2430740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561717689390 0x561717a52090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:44.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.407+0000 7f7dc2430740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5617179c7e50 con 0x561717689390 2024-09-21T04:04:44.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.407+0000 7f7dc2430740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5617179b7280 con 0x5617176811b0 2024-09-21T04:04:44.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.407+0000 7f7dc2430740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5617179bb090 con 0x5617176aa070 2024-09-21T04:04:44.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.408+0000 7f7dba459640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561717689390 0x561717a52090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:44.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.408+0000 7f7dba459640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561717689390 0x561717a52090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57168/0 (socket says 172.21.15.64:57168) 2024-09-21T04:04:44.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.408+0000 7f7dba459640 1 -- 172.21.15.64:0/3552866347 learned_addr learned my addr 172.21.15.64:0/3552866347 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:44.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.408+0000 7f7db9457640 1 --2- 172.21.15.64:0/3552866347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617176811b0 0x561717a51b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:44.430 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.408+0000 7f7db9c58640 1 --2- 172.21.15.64:0/1415096448 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7da00b65f0 0x7f7da00b8b00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7db41049e0 tx=0x7f7db0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:44.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.408+0000 7f7dba459640 1 -- 172.21.15.64:0/3552866347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617176aa070 msgr2=0x561717a51610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T04:04:44.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.408+0000 7f7dba459640 1 --2- 172.21.15.64:0/3552866347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617176aa070 0x561717a51610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.408+0000 7f7dba459640 1 -- 172.21.15.64:0/3552866347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617176811b0 msgr2=0x561717a51b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:44.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.408+0000 7f7dba459640 1 --2- 172.21.15.64:0/3552866347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617176811b0 0x561717a51b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.408+0000 7f7dba459640 1 -- 172.21.15.64:0/3552866347 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561717715650 con 0x561717689390 2024-09-21T04:04:44.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.408+0000 7f7dba459640 1 --2- 172.21.15.64:0/3552866347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561717689390 0x561717a52090 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7dac005cc0 tx=0x7f7dac02f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da3c806c0b58235 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:44.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.409+0000 7f7d9ffff640 1 -- 172.21.15.64:0/3552866347 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac031cf0 con 0x561717689390 2024-09-21T04:04:44.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.409+0000 7f7d9ffff640 1 -- 172.21.15.64:0/3552866347 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dac036040 con 0x561717689390 2024-09-21T04:04:44.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.409+0000 7f7d9ffff640 1 -- 172.21.15.64:0/3552866347 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac004040 con 0x561717689390 2024-09-21T04:04:44.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.410+0000 7f7dc2430740 1 -- 172.21.15.64:0/3552866347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561717689390 msgr2=0x561717a52090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:44.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.410+0000 7f7dc2430740 1 --2- 172.21.15.64:0/3552866347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561717689390 0x561717a52090 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7dac005cc0 tx=0x7f7dac02f420 comp rx=0 tx=0).stop 2024-09-21T04:04:44.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.410+0000 7f7dc2430740 1 -- 172.21.15.64:0/3552866347 shutdown_connections 2024-09-21T04:04:44.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.410+0000 7f7dc2430740 1 --2- 172.21.15.64:0/3552866347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561717689390 0x561717a52090 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.410+0000 7f7dc2430740 1 --2- 172.21.15.64:0/3552866347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617176811b0 0x561717a51b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.410+0000 7f7dc2430740 1 --2- 172.21.15.64:0/3552866347 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617176aa070 0x561717a51610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.410+0000 7f7dc2430740 1 -- 172.21.15.64:0/3552866347 >> 172.21.15.64:0/3552866347 conn(0x56171769e460 msgr2=0x561717a4f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:44.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.410+0000 7f7dc2430740 1 -- 172.21.15.64:0/3552866347 shutdown_connections 2024-09-21T04:04:44.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.410+0000 7f7dc2430740 1 -- 172.21.15.64:0/3552866347 wait complete. 2024-09-21T04:04:44.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.411+0000 7f7dc2430740 1 -- start start 2024-09-21T04:04:44.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.411+0000 7f7dc2430740 1 Processor -- start 2024-09-21T04:04:44.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.412+0000 7f7dc2430740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617176811b0 0x561717a5cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:44.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.412+0000 7f7dc2430740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561717689390 0x561717a5d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:44.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.412+0000 7f7dc2430740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617176aa070 0x561717a69080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:44.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.412+0000 7f7dc2430740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561717818700 con 0x5617176811b0 2024-09-21T04:04:44.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.412+0000 7f7dc2430740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561717952110 con 0x561717689390 2024-09-21T04:04:44.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.412+0000 7f7dc2430740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x561717a5fd90 con 0x5617176aa070 2024-09-21T04:04:44.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.412+0000 7f7dc2430740 10 client.0 _ll_register_callbacks cb 0x56171796b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:44.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.412+0000 7f7db9c58640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617176811b0 0x561717a5cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:44.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.412+0000 7f7db9c58640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617176811b0 0x561717a5cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57184/0 (socket says 172.21.15.64:57184) 2024-09-21T04:04:44.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.412+0000 7f7db9c58640 1 -- 172.21.15.64:0/1709360800 learned_addr learned my addr 172.21.15.64:0/1709360800 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:44.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.412+0000 7f7dba459640 1 --2- 172.21.15.64:0/1709360800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617176aa070 0x561717a69080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:44.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.412+0000 7f7db9457640 1 --2- 172.21.15.64:0/1709360800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561717689390 0x561717a5d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:44.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.412+0000 7f7db9c58640 1 -- 172.21.15.64:0/1709360800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617176aa070 msgr2=0x561717a69080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:44.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.412+0000 7f7db9c58640 1 --2- 172.21.15.64:0/1709360800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617176aa070 0x561717a69080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.412+0000 7f7db9c58640 1 -- 172.21.15.64:0/1709360800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561717689390 msgr2=0x561717a5d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:44.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.412+0000 7f7db9c58640 1 --2- 172.21.15.64:0/1709360800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561717689390 0x561717a5d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.412+0000 7f7db9c58640 1 -- 172.21.15.64:0/1709360800 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56171797fc40 con 0x5617176811b0 2024-09-21T04:04:44.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.412+0000 7f7dba459640 1 --2- 172.21.15.64:0/1709360800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617176aa070 0x561717a69080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:04:44.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.412+0000 7f7db9457640 1 --2- 172.21.15.64:0/1709360800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561717689390 0x561717a5d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:04:44.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.413+0000 7f7dc2430740 1 -- 172.21.15.64:0/1709360800 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dac036040 con 0x5617176811b0 2024-09-21T04:04:44.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.413+0000 7f7db9c58640 1 --2- 172.21.15.64:0/1709360800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617176811b0 0x561717a5cd30 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7db001c970 tx=0x7f7db0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e15e531a6a06ba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:44.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.413+0000 7f7d9d7fa640 10 client.15651 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:04:44.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.413+0000 7f7d9d7fa640 1 -- 172.21.15.64:0/1709360800 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db000e8b0 con 0x5617176811b0 2024-09-21T04:04:44.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.413+0000 7f7d9d7fa640 1 -- 172.21.15.64:0/1709360800 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f7db000ea50 con 0x5617176811b0 2024-09-21T04:04:44.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.413+0000 7f7d9d7fa640 1 -- 172.21.15.64:0/1709360800 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7db000ecb0 con 0x5617176811b0 2024-09-21T04:04:44.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.413+0000 7f7dc2430740 10 client.15651 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:44.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.413+0000 7f7dc2430740 1 -- 172.21.15.64:0/1709360800 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({fsmap=25}) -- 0x561717a6bab0 con 0x5617176811b0 2024-09-21T04:04:44.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.414+0000 7f7d9d7fa640 1 -- 172.21.15.64:0/1709360800 <== mon.0 v2:172.21.15.64:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7db0026420 con 0x5617176811b0 2024-09-21T04:04:44.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.414+0000 7f7dc2430740 10 client.15651 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:44.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.414+0000 7f7dc2430740 10 client.15651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:04:44.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.414+0000 7f7dc2430740 20 client.15651 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:44.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.414+0000 7f7dc2430740 1 --2- 172.21.15.64:0/1709360800 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5617176689f0 0x561717a6f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:44.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.414+0000 7f7dc2430740 4 client.15651 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:04:44.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.414+0000 7f7dc2430740 1 -- 172.21.15.64:0/1709360800 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617179c9da0 con 0x5617176689f0 2024-09-21T04:04:44.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.414+0000 7f7db9457640 1 --2- 172.21.15.64:0/1709360800 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5617176689f0 0x561717a6f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:44.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.415+0000 7f7db9457640 1 --2- 172.21.15.64:0/1709360800 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5617176689f0 0x561717a6f370 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40674b8ef9870b59 server_cookie=3dff0cecef57c666 in_seq=0 out_seq=0 2024-09-21T04:04:44.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.415+0000 7f7d9d7fa640 10 client.15651 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:44.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.416+0000 7f7d9d7fa640 1 -- 172.21.15.64:0/1709360800 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5617179c9da0 con 0x5617176689f0 2024-09-21T04:04:44.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.416+0000 7f7d9d7fa640 10 client.15651 handle_command_reply: tid=0 2024-09-21T04:04:44.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.416+0000 7f7dc2430740 1 client.15651 shutdown 2024-09-21T04:04:44.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.416+0000 7f7dc2430740 1 -- 172.21.15.64:0/1709360800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617176811b0 msgr2=0x561717a5cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:44.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.416+0000 7f7dc2430740 1 --2- 172.21.15.64:0/1709360800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617176811b0 0x561717a5cd30 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7db001c970 tx=0x7f7db0004730 comp rx=0 tx=0).stop 2024-09-21T04:04:44.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.416+0000 7f7dc2430740 1 -- 172.21.15.64:0/1709360800 shutdown_connections 2024-09-21T04:04:44.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.416+0000 7f7dc2430740 1 --2- 172.21.15.64:0/1709360800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617176aa070 0x561717a69080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.416+0000 7f7dc2430740 1 --2- 172.21.15.64:0/1709360800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561717689390 0x561717a5d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.416+0000 7f7dc2430740 1 --2- 172.21.15.64:0/1709360800 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5617176689f0 0x561717a6f370 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.416+0000 7f7dc2430740 1 --2- 172.21.15.64:0/1709360800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617176811b0 0x561717a5cd30 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.416+0000 7f7dc2430740 1 -- 172.21.15.64:0/1709360800 >> 172.21.15.64:0/1709360800 conn(0x56171769e460 msgr2=0x561717a547c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:44.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.416+0000 7f7d9d7fa640 0 client.15651 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:44.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.416+0000 7f7dc2430740 1 -- 172.21.15.64:0/1709360800 shutdown_connections 2024-09-21T04:04:44.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.416+0000 7f7dc2430740 1 -- 172.21.15.64:0/1709360800 wait complete. 2024-09-21T04:04:44.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.416+0000 7f7dc2430740 20 client.15651 trim_cache size 0 max 16384 2024-09-21T04:04:44.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.452+0000 7f7dc2430740 1 Processor -- start 2024-09-21T04:04:44.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.452+0000 7f7dc2430740 1 -- start start 2024-09-21T04:04:44.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.452+0000 7f7dc2430740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617176811b0 0x561717a5d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:44.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.452+0000 7f7dc2430740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561717689390 0x561717a54710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:44.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.452+0000 7f7db9457640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561717689390 0x561717a54710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:44.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.452+0000 7f7db9457640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561717689390 0x561717a54710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57190/0 (socket says 172.21.15.64:57190) 2024-09-21T04:04:44.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.452+0000 7f7db9457640 1 -- 172.21.15.64:0/1545976076 learned_addr learned my addr 172.21.15.64:0/1545976076 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:44.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.453+0000 7f7dc2430740 1 --2- 172.21.15.64:0/1545976076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617176aa070 0x561717a54c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:44.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.453+0000 7f7dc2430740 1 -- 172.21.15.64:0/1545976076 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7db41b4200 con 0x561717689390 2024-09-21T04:04:44.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.453+0000 7f7dc2430740 1 -- 172.21.15.64:0/1545976076 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7db41b1ad0 con 0x5617176811b0 2024-09-21T04:04:44.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.453+0000 7f7dc2430740 1 -- 172.21.15.64:0/1545976076 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7db41af920 con 0x5617176aa070 2024-09-21T04:04:44.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.453+0000 7f7db9c58640 1 --2- 172.21.15.64:0/1545976076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617176811b0 0x561717a5d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:44.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.453+0000 7f7dba459640 1 --2- 172.21.15.64:0/1545976076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617176aa070 0x561717a54c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:44.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.453+0000 7f7db9457640 1 -- 172.21.15.64:0/1545976076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617176aa070 msgr2=0x561717a54c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:44.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.453+0000 7f7db9457640 1 --2- 172.21.15.64:0/1545976076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617176aa070 0x561717a54c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.453+0000 7f7db9457640 1 -- 172.21.15.64:0/1545976076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617176811b0 msgr2=0x561717a5d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:44.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.453+0000 7f7db9457640 1 --2- 172.21.15.64:0/1545976076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617176811b0 0x561717a5d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.453+0000 7f7db9457640 1 -- 172.21.15.64:0/1545976076 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561717a5bc90 con 0x561717689390 2024-09-21T04:04:44.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.453+0000 7f7db9c58640 1 --2- 172.21.15.64:0/1545976076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617176811b0 0x561717a5d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:44.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.454+0000 7f7db9457640 1 --2- 172.21.15.64:0/1545976076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561717689390 0x561717a54710 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7da400c6a0 tx=0x7f7da40e7a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2889215663b39362 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:44.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.454+0000 7f7d9f7fe640 1 -- 172.21.15.64:0/1545976076 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da40d3030 con 0x561717689390 2024-09-21T04:04:44.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.454+0000 7f7d9f7fe640 1 -- 172.21.15.64:0/1545976076 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7da40b2070 con 0x561717689390 2024-09-21T04:04:44.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.454+0000 7f7d9f7fe640 1 -- 172.21.15.64:0/1545976076 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da40a9350 con 0x561717689390 2024-09-21T04:04:44.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.455+0000 7f7dc2430740 1 -- 172.21.15.64:0/1545976076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561717689390 msgr2=0x561717a54710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:44.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.455+0000 7f7dc2430740 1 --2- 172.21.15.64:0/1545976076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561717689390 0x561717a54710 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7da400c6a0 tx=0x7f7da40e7a40 comp rx=0 tx=0).stop 2024-09-21T04:04:44.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.455+0000 7f7dc2430740 1 -- 172.21.15.64:0/1545976076 shutdown_connections 2024-09-21T04:04:44.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.455+0000 7f7dc2430740 1 --2- 172.21.15.64:0/1545976076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617176aa070 0x561717a54c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.455+0000 7f7dc2430740 1 --2- 172.21.15.64:0/1545976076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561717689390 0x561717a54710 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.455+0000 7f7dc2430740 1 --2- 172.21.15.64:0/1545976076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5617176811b0 0x561717a5d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.456+0000 7f7dc2430740 1 -- 172.21.15.64:0/1545976076 >> 172.21.15.64:0/1545976076 conn(0x56171769e460 msgr2=0x561717a5cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:44.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.456+0000 7f7dc2430740 1 -- 172.21.15.64:0/1545976076 shutdown_connections 2024-09-21T04:04:44.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.456+0000 7f7dc2430740 1 -- 172.21.15.64:0/1545976076 wait complete. 2024-09-21T04:04:44.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.456+0000 7f7dc2430740 1 -- start start 2024-09-21T04:04:44.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.456+0000 7f7dc2430740 1 Processor -- start 2024-09-21T04:04:44.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.457+0000 7f7dc2430740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617176811b0 0x561717a65880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:44.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.457+0000 7f7dc2430740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617176aa070 0x561717a60d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:44.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.457+0000 7f7db9c58640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617176811b0 0x561717a65880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:44.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.457+0000 7f7dc2430740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56171763dda0 0x561717a61330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:44.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.457+0000 7f7dc2430740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7db41aff90 con 0x5617176811b0 2024-09-21T04:04:44.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.457+0000 7f7db9c58640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617176811b0 0x561717a65880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57198/0 (socket says 172.21.15.64:57198) 2024-09-21T04:04:44.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.457+0000 7f7db9c58640 1 -- 172.21.15.64:0/2315219312 learned_addr learned my addr 172.21.15.64:0/2315219312 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:44.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.457+0000 7f7dc2430740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7db41b1400 con 0x56171763dda0 2024-09-21T04:04:44.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.458+0000 7f7dc2430740 1 -- 172.21.15.64:0/2315219312 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7db41b0660 con 0x5617176aa070 2024-09-21T04:04:44.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.458+0000 7f7db9457640 1 --2- 172.21.15.64:0/2315219312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617176aa070 0x561717a60d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:44.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.458+0000 7f7dba459640 1 --2- 172.21.15.64:0/2315219312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56171763dda0 0x561717a61330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:44.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.458+0000 7f7dc2430740 10 client.0 _ll_register_callbacks cb 0x7f7db0009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:44.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.458+0000 7f7db9457640 1 -- 172.21.15.64:0/2315219312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56171763dda0 msgr2=0x561717a61330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:44.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.458+0000 7f7db9457640 1 --2- 172.21.15.64:0/2315219312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56171763dda0 0x561717a61330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.458+0000 7f7db9457640 1 -- 172.21.15.64:0/2315219312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617176811b0 msgr2=0x561717a65880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:44.465 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.458+0000 7f7db9457640 1 --2- 172.21.15.64:0/2315219312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617176811b0 0x561717a65880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.458+0000 7f7db9457640 1 -- 172.21.15.64:0/2315219312 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561717a6f940 con 0x5617176aa070 2024-09-21T04:04:44.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.459+0000 7f7dba459640 1 --2- 172.21.15.64:0/2315219312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56171763dda0 0x561717a61330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:44.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.459+0000 7f7db9c58640 1 --2- 172.21.15.64:0/2315219312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617176811b0 0x561717a65880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:44.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.459+0000 7f7dc2430740 1 -- 172.21.15.64:0/2315219312 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7da40b2070 con 0x5617176aa070 2024-09-21T04:04:44.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.459+0000 7f7db9457640 1 --2- 172.21.15.64:0/2315219312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617176aa070 0x561717a60d30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7da40b2440 tx=0x7f7da40b2340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5473ba5b21d1cc87 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:44.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.459+0000 7f7d9d7fa640 10 client.25315 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-21T04:04:44.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.459+0000 7f7d9d7fa640 1 -- 172.21.15.64:0/2315219312 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da40c8050 con 0x5617176aa070 2024-09-21T04:04:44.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.459+0000 7f7d9d7fa640 1 -- 172.21.15.64:0/2315219312 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f7da40a90f0 con 0x5617176aa070 2024-09-21T04:04:44.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.459+0000 7f7d9d7fa640 1 -- 172.21.15.64:0/2315219312 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7da40d3020 con 0x5617176aa070 2024-09-21T04:04:44.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.459+0000 7f7dc2430740 10 client.25315 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:44.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.459+0000 7f7dc2430740 1 -- 172.21.15.64:0/2315219312 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=25}) -- 0x561717ad9a90 con 0x5617176aa070 2024-09-21T04:04:44.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.460+0000 7f7d9d7fa640 1 -- 172.21.15.64:0/2315219312 <== mon.1 v2:172.21.15.131:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7da40aead0 con 0x5617176aa070 2024-09-21T04:04:44.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.460+0000 7f7dc2430740 10 client.25315 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:44.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.460+0000 7f7dc2430740 10 client.25315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:04:44.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.460+0000 7f7dc2430740 20 client.25315 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:44.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.460+0000 7f7dc2430740 1 --2- 172.21.15.64:0/2315219312 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x561717ad9f60 0x561717adc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:44.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.460+0000 7f7dc2430740 4 client.25315 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T04:04:44.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.460+0000 7f7dc2430740 1 -- 172.21.15.64:0/2315219312 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5617179c5850 con 0x561717ad9f60 2024-09-21T04:04:44.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.460+0000 7f7db9c58640 1 --2- 172.21.15.64:0/2315219312 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x561717ad9f60 0x561717adc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:44.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.461+0000 7f7db9c58640 1 --2- 172.21.15.64:0/2315219312 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x561717ad9f60 0x561717adc380 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbe9e8762394f5c3 server_cookie=96dc1dc83ee09bbf in_seq=0 out_seq=0 2024-09-21T04:04:44.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.461+0000 7f7d9d7fa640 10 client.25315 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:44.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.461+0000 7f7d9d7fa640 1 -- 172.21.15.64:0/2315219312 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+658 (crc 0 0 0) 0x5617179c5850 con 0x561717ad9f60 2024-09-21T04:04:44.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.461+0000 7f7d9d7fa640 10 client.25315 handle_command_reply: tid=0 2024-09-21T04:04:44.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.461+0000 7f7dc2430740 1 client.25315 shutdown 2024-09-21T04:04:44.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.461+0000 7f7dc2430740 1 -- 172.21.15.64:0/2315219312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617176aa070 msgr2=0x561717a60d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:44.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.461+0000 7f7dc2430740 1 --2- 172.21.15.64:0/2315219312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617176aa070 0x561717a60d30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7da40b2440 tx=0x7f7da40b2340 comp rx=0 tx=0).stop 2024-09-21T04:04:44.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.461+0000 7f7dc2430740 1 -- 172.21.15.64:0/2315219312 shutdown_connections 2024-09-21T04:04:44.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.462+0000 7f7dc2430740 1 --2- 172.21.15.64:0/2315219312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56171763dda0 0x561717a61330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.462+0000 7f7dc2430740 1 --2- 172.21.15.64:0/2315219312 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5617176aa070 0x561717a60d30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.462+0000 7f7dc2430740 1 --2- 172.21.15.64:0/2315219312 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x561717ad9f60 0x561717adc380 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.462+0000 7f7d9d7fa640 0 client.25315 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:44.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.462+0000 7f7dc2430740 1 --2- 172.21.15.64:0/2315219312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617176811b0 0x561717a65880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.462+0000 7f7dc2430740 1 -- 172.21.15.64:0/2315219312 >> 172.21.15.64:0/2315219312 conn(0x56171769e460 msgr2=0x561717a54710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:44.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.462+0000 7f7dc2430740 1 -- 172.21.15.64:0/2315219312 shutdown_connections 2024-09-21T04:04:44.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.462+0000 7f7dc2430740 1 -- 172.21.15.64:0/2315219312 wait complete. 2024-09-21T04:04:44.470 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.462+0000 7f7dc2430740 20 client.25315 trim_cache size 0 max 16384 2024-09-21T04:04:44.470 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:04:44.470 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-21T04:04:44.470 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {}, 2024-09-21T04:04:44.471 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_stats": { 2024-09-21T04:04:44.471 INFO:teuthology.orchestra.run.smithi064.stdout: "0ab717a7-36c6-4dca-94e7-af0bdf2aa112": { 2024-09-21T04:04:44.471 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:42.691086+0000", 2024-09-21T04:04:44.471 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:04:44.471 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:04:44.471 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:04:44.471 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:04:44.471 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:04:44.471 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:04:44.471 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:04:44.471 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:03:59.570174+0000", 2024-09-21T04:04:44.471 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:04:44.471 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:04:44.471 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:04:44.472 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:04:44.472 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:04:44.472 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:04:44.472 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:04:44.472 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:04:44.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.463+0000 7f7dbbebc640 1 -- 172.21.15.64:0/1415096448 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7da00b65f0 msgr2=0x7f7da00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:44.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.463+0000 7f7dbbebc640 1 --2- 172.21.15.64:0/1415096448 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7da00b65f0 0x7f7da00b8b00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7db41049e0 tx=0x7f7db0005f90 comp rx=0 tx=0).stop 2024-09-21T04:04:44.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.463+0000 7f7dbbebc640 1 -- 172.21.15.64:0/1415096448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7db41062d0 msgr2=0x7f7db41034d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:44.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.463+0000 7f7dbbebc640 1 --2- 172.21.15.64:0/1415096448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7db41062d0 0x7f7db41034d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7db4075ad0 tx=0x7f7da400ee50 comp rx=0 tx=0).stop 2024-09-21T04:04:44.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.463+0000 7f7dbbebc640 1 -- 172.21.15.64:0/1415096448 shutdown_connections 2024-09-21T04:04:44.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.463+0000 7f7dbbebc640 1 --2- 172.21.15.64:0/1415096448 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7da00b65f0 0x7f7da00b8b00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.463+0000 7f7dbbebc640 1 --2- 172.21.15.64:0/1415096448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7db4106c20 0x7f7db4103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.463+0000 7f7dbbebc640 1 --2- 172.21.15.64:0/1415096448 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7db41062d0 0x7f7db41034d0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.463+0000 7f7dbbebc640 1 --2- 172.21.15.64:0/1415096448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7db4101ff0 0x7f7db4102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:44.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.463+0000 7f7dbbebc640 1 -- 172.21.15.64:0/1415096448 >> 172.21.15.64:0/1415096448 conn(0x7f7db4077cc0 msgr2=0x7f7db41074a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:44.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.463+0000 7f7dbbebc640 1 -- 172.21.15.64:0/1415096448 shutdown_connections 2024-09-21T04:04:44.473 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:44.464+0000 7f7dbbebc640 1 -- 172.21.15.64:0/1415096448 wait complete. 2024-09-21T04:04:45.233 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T04:04:45.233 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:04:45.639 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:04:45.913 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:45 smithi064 ceph-mon[31180]: pgmap v343: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-21T04:04:46.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:45 smithi119 ceph-mon[34811]: pgmap v343: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-21T04:04:46.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:45 smithi131 ceph-mon[35193]: pgmap v343: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-21T04:04:46.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.152+0000 7f0806c96640 1 -- 172.21.15.64:0/2929540747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08001083e0 msgr2=0x7f08001087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:46.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.152+0000 7f0806c96640 1 --2- 172.21.15.64:0/2929540747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08001083e0 0x7f08001087f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009900 tx=0x7f07ec02a410 comp rx=0 tx=0).stop 2024-09-21T04:04:46.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.152+0000 7f0806c96640 1 -- 172.21.15.64:0/2929540747 shutdown_connections 2024-09-21T04:04:46.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.152+0000 7f0806c96640 1 --2- 172.21.15.64:0/2929540747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0800108d30 0x7f080010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:46.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.152+0000 7f0806c96640 1 --2- 172.21.15.64:0/2929540747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08001083e0 0x7f08001087f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:46.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.152+0000 7f0806c96640 1 --2- 172.21.15.64:0/2929540747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0800102aa0 0x7f0800102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:46.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.152+0000 7f0806c96640 1 -- 172.21.15.64:0/2929540747 >> 172.21.15.64:0/2929540747 conn(0x7f0800079750 msgr2=0x7f0800079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:46.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.152+0000 7f0806c96640 1 -- 172.21.15.64:0/2929540747 shutdown_connections 2024-09-21T04:04:46.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.152+0000 7f0806c96640 1 -- 172.21.15.64:0/2929540747 wait complete. 2024-09-21T04:04:46.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.153+0000 7f0806c96640 1 Processor -- start 2024-09-21T04:04:46.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.153+0000 7f0806c96640 1 -- start start 2024-09-21T04:04:46.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.153+0000 7f0806c96640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0800102aa0 0x7f0800199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:46.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.153+0000 7f0806c96640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08001083e0 0x7f08001999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:46.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.154+0000 7f0806c96640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0800108d30 0x7f080019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:46.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.154+0000 7f0806c96640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f08001129f0 con 0x7f08001083e0 2024-09-21T04:04:46.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.154+0000 7f0806c96640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0800112870 con 0x7f0800102aa0 2024-09-21T04:04:46.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.154+0000 7f0806c96640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0800112b70 con 0x7f0800108d30 2024-09-21T04:04:46.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.154+0000 7f07fffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08001083e0 0x7f08001999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:46.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.154+0000 7f07fffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08001083e0 0x7f08001999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57212/0 (socket says 172.21.15.64:57212) 2024-09-21T04:04:46.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.154+0000 7f07fffff640 1 -- 172.21.15.64:0/3096540023 learned_addr learned my addr 172.21.15.64:0/3096540023 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:46.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.154+0000 7f0804a32640 1 --2- 172.21.15.64:0/3096540023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0800102aa0 0x7f0800199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:46.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.154+0000 7f0805233640 1 --2- 172.21.15.64:0/3096540023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0800108d30 0x7f080019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:46.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.154+0000 7f07fffff640 1 -- 172.21.15.64:0/3096540023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0800108d30 msgr2=0x7f080019ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:46.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.154+0000 7f07fffff640 1 --2- 172.21.15.64:0/3096540023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0800108d30 0x7f080019ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:46.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.154+0000 7f07fffff640 1 -- 172.21.15.64:0/3096540023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0800102aa0 msgr2=0x7f0800199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:46.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.154+0000 7f07fffff640 1 --2- 172.21.15.64:0/3096540023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0800102aa0 0x7f0800199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:46.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.154+0000 7f07fffff640 1 -- 172.21.15.64:0/3096540023 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080019f3b0 con 0x7f08001083e0 2024-09-21T04:04:46.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.154+0000 7f07fffff640 1 --2- 172.21.15.64:0/3096540023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08001083e0 0x7f08001999e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f07ec006010 tx=0x7f07ec004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:46.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.155+0000 7f07fdffb640 1 -- 172.21.15.64:0/3096540023 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec03f030 con 0x7f08001083e0 2024-09-21T04:04:46.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.155+0000 7f07fdffb640 1 -- 172.21.15.64:0/3096540023 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07ec003aa0 con 0x7f08001083e0 2024-09-21T04:04:46.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.155+0000 7f0806c96640 1 -- 172.21.15.64:0/3096540023 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f080019f640 con 0x7f08001083e0 2024-09-21T04:04:46.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.155+0000 7f07fdffb640 1 -- 172.21.15.64:0/3096540023 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec035070 con 0x7f08001083e0 2024-09-21T04:04:46.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.155+0000 7f0806c96640 1 -- 172.21.15.64:0/3096540023 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f080006b820 con 0x7f08001083e0 2024-09-21T04:04:46.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.157+0000 7f07eb7fe640 1 -- 172.21.15.64:0/3096540023 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07e0005290 con 0x7f08001083e0 2024-09-21T04:04:46.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.159+0000 7f07fdffb640 1 -- 172.21.15.64:0/3096540023 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 154694+0+0 (secure 0 0 0) 0x7f07ec042030 con 0x7f08001083e0 2024-09-21T04:04:46.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.160+0000 7f07fdffb640 1 --2- 172.21.15.64:0/3096540023 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f07f00b6790 0x7f07f00b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:46.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.161+0000 7f07fdffb640 1 -- 172.21.15.64:0/3096540023 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f07ec107d10 con 0x7f08001083e0 2024-09-21T04:04:46.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.161+0000 7f0804a32640 1 --2- 172.21.15.64:0/3096540023 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f07f00b6790 0x7f07f00b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:46.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.161+0000 7f0804a32640 1 --2- 172.21.15.64:0/3096540023 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f07f00b6790 0x7f07f00b8ca0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f07f4009bd0 tx=0x7f07f4010040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:46.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.167+0000 7f07fdffb640 1 -- 172.21.15.64:0/3096540023 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f07ec0d24a0 con 0x7f08001083e0 2024-09-21T04:04:46.425 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.421+0000 7f07eb7fe640 1 -- 172.21.15.64:0/3096540023 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07e0005850 con 0x7f08001083e0 2024-09-21T04:04:46.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.422+0000 7f07fdffb640 1 -- 172.21.15.64:0/3096540023 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11911 (secure 0 0 0) 0x7f07ec0d73e0 con 0x7f08001083e0 2024-09-21T04:04:46.426 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:04:46.426 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":25,"btime":"2024-09-21T04:03:50:600319+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14805,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/97995119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":97995119},{"type":"v1","addr":"172.21.15.64:6839","nonce":97995119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/390891068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":390891068},{"type":"v1","addr":"172.21.15.64:6841","nonce":390891068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:02:03.354744+0000","modified":"2024-09-21T04:03:50.212837+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24710,"mds_1":24680,"mds_2":24661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24634":{"gid":24634,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6839/782190424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782190424},{"type":"v1","addr":"172.21.15.131:6839","nonce":782190424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24680":{"gid":24680,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24710":{"gid":24710,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6839/1319793636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1319793636},{"type":"v1","addr":"172.21.15.119:6839","nonce":1319793636}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24710,"qdb_cluster":[24710,24680,24661]},"id":1}]} 2024-09-21T04:04:46.426 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 25 2024-09-21T04:04:46.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.425+0000 7f07eb7fe640 1 -- 172.21.15.64:0/3096540023 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f07f00b6790 msgr2=0x7f07f00b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:46.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.425+0000 7f07eb7fe640 1 --2- 172.21.15.64:0/3096540023 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f07f00b6790 0x7f07f00b8ca0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f07f4009bd0 tx=0x7f07f4010040 comp rx=0 tx=0).stop 2024-09-21T04:04:46.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.425+0000 7f07eb7fe640 1 -- 172.21.15.64:0/3096540023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08001083e0 msgr2=0x7f08001999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:46.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.425+0000 7f07eb7fe640 1 --2- 172.21.15.64:0/3096540023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08001083e0 0x7f08001999e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f07ec006010 tx=0x7f07ec004060 comp rx=0 tx=0).stop 2024-09-21T04:04:46.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.425+0000 7f07eb7fe640 1 -- 172.21.15.64:0/3096540023 shutdown_connections 2024-09-21T04:04:46.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.425+0000 7f07eb7fe640 1 --2- 172.21.15.64:0/3096540023 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f07f00b6790 0x7f07f00b8ca0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:46.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.425+0000 7f07eb7fe640 1 --2- 172.21.15.64:0/3096540023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0800108d30 0x7f080019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:46.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.425+0000 7f07eb7fe640 1 --2- 172.21.15.64:0/3096540023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f08001083e0 0x7f08001999e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:46.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.425+0000 7f07eb7fe640 1 --2- 172.21.15.64:0/3096540023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0800102aa0 0x7f0800199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:46.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.425+0000 7f07eb7fe640 1 -- 172.21.15.64:0/3096540023 >> 172.21.15.64:0/3096540023 conn(0x7f0800079750 msgr2=0x7f08001064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:46.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.425+0000 7f07eb7fe640 1 -- 172.21.15.64:0/3096540023 shutdown_connections 2024-09-21T04:04:46.429 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:46.425+0000 7f07eb7fe640 1 -- 172.21.15.64:0/3096540023 wait complete. 2024-09-21T04:04:46.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:46 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3096540023' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:04:47.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:46 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3096540023' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:04:47.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:46 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3096540023' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:04:47.122 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T04:04:47.499 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:04:47.952 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:47 smithi064 ceph-mon[31180]: pgmap v344: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-09-21T04:04:48.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:47 smithi119 ceph-mon[34811]: pgmap v344: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-09-21T04:04:48.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:47 smithi131 ceph-mon[35193]: pgmap v344: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-09-21T04:04:48.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.059+0000 7fec9e407640 1 -- 172.21.15.64:0/389696207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec98108cf0 msgr2=0x7fec9810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:48.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.059+0000 7fec9e407640 1 --2- 172.21.15.64:0/389696207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec98108cf0 0x7fec9810f520 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fec90009ca0 tx=0x7fec9002a8c0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.059+0000 7fec9e407640 1 -- 172.21.15.64:0/389696207 shutdown_connections 2024-09-21T04:04:48.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.059+0000 7fec9e407640 1 --2- 172.21.15.64:0/389696207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec98108cf0 0x7fec9810f520 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.059+0000 7fec9e407640 1 --2- 172.21.15.64:0/389696207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec981083a0 0x7fec981087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.059+0000 7fec9e407640 1 --2- 172.21.15.64:0/389696207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec98102a60 0x7fec98102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.059+0000 7fec9e407640 1 -- 172.21.15.64:0/389696207 >> 172.21.15.64:0/389696207 conn(0x7fec98079730 msgr2=0x7fec98079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:48.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.060+0000 7fec9e407640 1 -- 172.21.15.64:0/389696207 shutdown_connections 2024-09-21T04:04:48.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.060+0000 7fec9e407640 1 -- 172.21.15.64:0/389696207 wait complete. 2024-09-21T04:04:48.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.061+0000 7fec9e407640 1 Processor -- start 2024-09-21T04:04:48.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.061+0000 7fec9e407640 1 -- start start 2024-09-21T04:04:48.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.061+0000 7fec9e407640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec98102a60 0x7fec9819c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:48.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.061+0000 7fec9e407640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec981083a0 0x7fec9819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:48.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.061+0000 7fec9e407640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec98108cf0 0x7fec981a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:48.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.061+0000 7fec9e407640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fec981129b0 con 0x7fec98102a60 2024-09-21T04:04:48.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.061+0000 7fec9e407640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fec98112830 con 0x7fec981083a0 2024-09-21T04:04:48.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.061+0000 7fec9e407640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fec98112b30 con 0x7fec98108cf0 2024-09-21T04:04:48.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.061+0000 7fec97fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec98102a60 0x7fec9819c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:48.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.061+0000 7fec97fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec98102a60 0x7fec9819c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57226/0 (socket says 172.21.15.64:57226) 2024-09-21T04:04:48.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.061+0000 7fec97fff640 1 -- 172.21.15.64:0/3156678245 learned_addr learned my addr 172.21.15.64:0/3156678245 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:48.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.062+0000 7fec977fe640 1 --2- 172.21.15.64:0/3156678245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec981083a0 0x7fec9819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:48.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.062+0000 7fec97fff640 1 -- 172.21.15.64:0/3156678245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec98108cf0 msgr2=0x7fec981a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:48.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.062+0000 7fec97fff640 1 --2- 172.21.15.64:0/3156678245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec98108cf0 0x7fec981a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.062+0000 7fec97fff640 1 -- 172.21.15.64:0/3156678245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec981083a0 msgr2=0x7fec9819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:48.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.062+0000 7fec97fff640 1 --2- 172.21.15.64:0/3156678245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec981083a0 0x7fec9819cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.062+0000 7fec97fff640 1 -- 172.21.15.64:0/3156678245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec981a25e0 con 0x7fec98102a60 2024-09-21T04:04:48.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.062+0000 7fec97fff640 1 --2- 172.21.15.64:0/3156678245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec98102a60 0x7fec9819c670 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fec8800e9f0 tx=0x7fec8800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:48.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.062+0000 7fec957fa640 1 -- 172.21.15.64:0/3156678245 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8800cd80 con 0x7fec98102a60 2024-09-21T04:04:48.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.062+0000 7fec957fa640 1 -- 172.21.15.64:0/3156678245 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec880108e0 con 0x7fec98102a60 2024-09-21T04:04:48.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.062+0000 7fec957fa640 1 -- 172.21.15.64:0/3156678245 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88018980 con 0x7fec98102a60 2024-09-21T04:04:48.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.063+0000 7fec9e407640 1 -- 172.21.15.64:0/3156678245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec981a28d0 con 0x7fec98102a60 2024-09-21T04:04:48.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.063+0000 7fec9e407640 1 -- 172.21.15.64:0/3156678245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec981a2cd0 con 0x7fec98102a60 2024-09-21T04:04:48.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.066+0000 7feca497b740 1 Processor -- start 2024-09-21T04:04:48.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.066+0000 7feca497b740 1 -- start start 2024-09-21T04:04:48.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.067+0000 7feca497b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990aa1a070 0x55990adc16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:48.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.067+0000 7feca497b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55990a9f11b0 0x55990adc1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:48.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.067+0000 7feca497b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55990a9f9390 0x55990adc2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:48.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.067+0000 7feca497b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55990ad37ea0 con 0x55990aa1a070 2024-09-21T04:04:48.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.067+0000 7feca497b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55990ad272d0 con 0x55990a9f11b0 2024-09-21T04:04:48.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.067+0000 7feca497b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55990ad2b0e0 con 0x55990a9f9390 2024-09-21T04:04:48.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.067+0000 7fec9c9a4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990aa1a070 0x55990adc16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:48.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.067+0000 7fec9c9a4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990aa1a070 0x55990adc16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57228/0 (socket says 172.21.15.64:57228) 2024-09-21T04:04:48.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.067+0000 7fec9c9a4640 1 -- 172.21.15.64:0/2347100713 learned_addr learned my addr 172.21.15.64:0/2347100713 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:48.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.067+0000 7fec957fa640 1 -- 172.21.15.64:0/3156678245 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 154694+0+0 (secure 0 0 0) 0x7fec88010430 con 0x7fec98102a60 2024-09-21T04:04:48.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.067+0000 7fec977fe640 1 --2- 172.21.15.64:0/2347100713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55990a9f9390 0x55990adc2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:48.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.068+0000 7fec9c9a4640 1 -- 172.21.15.64:0/2347100713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55990a9f9390 msgr2=0x55990adc2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:48.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.068+0000 7fec9c9a4640 1 --2- 172.21.15.64:0/2347100713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55990a9f9390 0x55990adc2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.068+0000 7fec9c9a4640 1 -- 172.21.15.64:0/2347100713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55990a9f11b0 msgr2=0x55990adc1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:48.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.068+0000 7fec9c9a4640 1 --2- 172.21.15.64:0/2347100713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55990a9f11b0 0x55990adc1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.068+0000 7fec9c9a4640 1 -- 172.21.15.64:0/2347100713 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55990aa85650 con 0x55990aa1a070 2024-09-21T04:04:48.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.068+0000 7fec9c9a4640 1 --2- 172.21.15.64:0/2347100713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990aa1a070 0x55990adc16a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fec90005e30 tx=0x7fec90009460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d68b32433f0ece0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:48.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.069+0000 7fec957fa640 1 --2- 172.21.15.64:0/3156678245 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fec800b6860 0x7fec800b8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:48.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.069+0000 7fec7e7fc640 1 -- 172.21.15.64:0/2347100713 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90031cf0 con 0x55990aa1a070 2024-09-21T04:04:48.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.069+0000 7fec7e7fc640 1 -- 172.21.15.64:0/2347100713 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec9003f070 con 0x55990aa1a070 2024-09-21T04:04:48.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.069+0000 7fec7e7fc640 1 -- 172.21.15.64:0/2347100713 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec900042a0 con 0x55990aa1a070 2024-09-21T04:04:48.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.069+0000 7fec957fa640 1 -- 172.21.15.64:0/3156678245 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fec880e7cc0 con 0x7fec98102a60 2024-09-21T04:04:48.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.070+0000 7fec97fff640 1 --2- 172.21.15.64:0/3156678245 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fec800b6860 0x7fec800b8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:48.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.070+0000 7fec97fff640 1 --2- 172.21.15.64:0/3156678245 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fec800b6860 0x7fec800b8d70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fec880e5320 tx=0x7fec880ad2b0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:48.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.070+0000 7feca497b740 1 -- 172.21.15.64:0/2347100713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990aa1a070 msgr2=0x55990adc16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:48.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.070+0000 7feca497b740 1 --2- 172.21.15.64:0/2347100713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990aa1a070 0x55990adc16a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fec90005e30 tx=0x7fec90009460 comp rx=0 tx=0).stop 2024-09-21T04:04:48.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.070+0000 7feca497b740 1 -- 172.21.15.64:0/2347100713 shutdown_connections 2024-09-21T04:04:48.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.070+0000 7feca497b740 1 --2- 172.21.15.64:0/2347100713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55990a9f9390 0x55990adc2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.070+0000 7feca497b740 1 --2- 172.21.15.64:0/2347100713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55990a9f11b0 0x55990adc1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.070+0000 7feca497b740 1 --2- 172.21.15.64:0/2347100713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990aa1a070 0x55990adc16a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.070+0000 7feca497b740 1 -- 172.21.15.64:0/2347100713 >> 172.21.15.64:0/2347100713 conn(0x55990aa0e460 msgr2=0x55990adbf590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:48.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.071+0000 7feca497b740 1 -- 172.21.15.64:0/2347100713 shutdown_connections 2024-09-21T04:04:48.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.071+0000 7feca497b740 1 -- 172.21.15.64:0/2347100713 wait complete. 2024-09-21T04:04:48.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.071+0000 7feca497b740 1 -- start start 2024-09-21T04:04:48.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.071+0000 7feca497b740 1 Processor -- start 2024-09-21T04:04:48.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.071+0000 7feca497b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990a9f11b0 0x55990adcccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:48.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.072+0000 7feca497b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55990a9f9390 0x55990adcd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:48.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.072+0000 7feca497b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55990aa1a070 0x55990add9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:48.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.072+0000 7feca497b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55990ab88700 con 0x55990a9f11b0 2024-09-21T04:04:48.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.072+0000 7feca497b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55990acc2160 con 0x55990a9f9390 2024-09-21T04:04:48.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.072+0000 7feca497b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55990adcfe20 con 0x55990aa1a070 2024-09-21T04:04:48.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.072+0000 7feca497b740 10 client.0 _ll_register_callbacks cb 0x55990acdb9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:48.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.072+0000 7fec977fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55990a9f9390 0x55990adcd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:48.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.072+0000 7fec9c9a4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990a9f11b0 0x55990adcccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:48.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.072+0000 7fec9c9a4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55990aa1a070 0x55990add9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:48.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.072+0000 7fec977fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55990a9f9390 0x55990adcd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:57478/0 (socket says 172.21.15.64:57478) 2024-09-21T04:04:48.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.072+0000 7fec977fe640 1 -- 172.21.15.64:0/4159515370 learned_addr learned my addr 172.21.15.64:0/4159515370 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:48.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.072+0000 7fec977fe640 1 -- 172.21.15.64:0/4159515370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55990aa1a070 msgr2=0x55990add9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:48.076 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.072+0000 7fec977fe640 1 --2- 172.21.15.64:0/4159515370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55990aa1a070 0x55990add9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.072+0000 7fec977fe640 1 -- 172.21.15.64:0/4159515370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990a9f11b0 msgr2=0x55990adcccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:48.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.072+0000 7fec977fe640 1 --2- 172.21.15.64:0/4159515370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990a9f11b0 0x55990adcccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.072+0000 7fec977fe640 1 -- 172.21.15.64:0/4159515370 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55990acefc90 con 0x55990a9f9390 2024-09-21T04:04:48.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.072+0000 7fec9c9a4640 1 --2- 172.21.15.64:0/4159515370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990a9f11b0 0x55990adcccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:04:48.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.073+0000 7fec977fe640 1 --2- 172.21.15.64:0/4159515370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55990a9f9390 0x55990adcd230 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00fe80 tx=0x7fec8c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b22a2a3293cace0a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:48.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.073+0000 7fec777fe640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:04:48.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.073+0000 7fec777fe640 1 -- 172.21.15.64:0/4159515370 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c004220 con 0x55990a9f9390 2024-09-21T04:04:48.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.073+0000 7feca497b740 1 -- 172.21.15.64:0/4159515370 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec9003f070 con 0x55990a9f9390 2024-09-21T04:04:48.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.073+0000 7fec777fe640 1 -- 172.21.15.64:0/4159515370 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fec8c0043c0 con 0x55990a9f9390 2024-09-21T04:04:48.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.073+0000 7fec777fe640 1 -- 172.21.15.64:0/4159515370 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fec8c012660 con 0x55990a9f9390 2024-09-21T04:04:48.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.073+0000 7feca497b740 10 client.25406 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:48.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.073+0000 7feca497b740 1 -- 172.21.15.64:0/4159515370 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55990addbb80 con 0x55990a9f9390 2024-09-21T04:04:48.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.074+0000 7fec777fe640 1 -- 172.21.15.64:0/4159515370 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec9003f070 con 0x55990a9f9390 2024-09-21T04:04:48.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.074+0000 7fec777fe640 1 -- 172.21.15.64:0/4159515370 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fec8c003ab0 con 0x55990a9f9390 2024-09-21T04:04:48.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.075+0000 7feca497b740 10 client.25406 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:48.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.075+0000 7feca497b740 10 client.25406 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:04:48.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.075+0000 7feca497b740 20 client.25406 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:48.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.075+0000 7feca497b740 1 --2- 172.21.15.64:0/4159515370 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55990a9d89f0 0x55990addf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:48.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.075+0000 7feca497b740 4 client.25406 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:04:48.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.075+0000 7feca497b740 1 -- 172.21.15.64:0/4159515370 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55990ad39df0 con 0x55990a9d89f0 2024-09-21T04:04:48.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.075+0000 7fec9c9a4640 1 --2- 172.21.15.64:0/4159515370 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55990a9d89f0 0x55990addf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:48.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.075+0000 7fec777fe640 10 client.25406 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:04:48.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.075+0000 7fec9c9a4640 1 --2- 172.21.15.64:0/4159515370 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55990a9d89f0 0x55990addf0a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a3368229125f4f75 server_cookie=210195cbda1d147a in_seq=0 out_seq=0 2024-09-21T04:04:48.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.076+0000 7fec777fe640 1 -- 172.21.15.64:0/4159515370 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55990ad39df0 con 0x55990a9d89f0 2024-09-21T04:04:48.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.076+0000 7fec777fe640 10 client.25406 handle_command_reply: tid=0 2024-09-21T04:04:48.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.076+0000 7feca497b740 1 client.25406 shutdown 2024-09-21T04:04:48.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.076+0000 7feca497b740 1 -- 172.21.15.64:0/4159515370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55990a9f9390 msgr2=0x55990adcd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:48.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.076+0000 7feca497b740 1 --2- 172.21.15.64:0/4159515370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55990a9f9390 0x55990adcd230 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00fe80 tx=0x7fec8c0117b0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.076+0000 7feca497b740 1 -- 172.21.15.64:0/4159515370 shutdown_connections 2024-09-21T04:04:48.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.076+0000 7feca497b740 1 --2- 172.21.15.64:0/4159515370 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55990a9d89f0 0x55990addf0a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.077+0000 7fec777fe640 0 client.25406 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:04:48.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.077+0000 7feca497b740 1 --2- 172.21.15.64:0/4159515370 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55990aa1a070 0x55990add9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.077+0000 7feca497b740 1 --2- 172.21.15.64:0/4159515370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55990a9f9390 0x55990adcd230 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.077+0000 7feca497b740 1 --2- 172.21.15.64:0/4159515370 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990a9f11b0 0x55990adcccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.077+0000 7feca497b740 1 -- 172.21.15.64:0/4159515370 >> 172.21.15.64:0/4159515370 conn(0x55990aa0e460 msgr2=0x55990adc4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:48.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.077+0000 7feca497b740 1 -- 172.21.15.64:0/4159515370 shutdown_connections 2024-09-21T04:04:48.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.077+0000 7feca497b740 1 -- 172.21.15.64:0/4159515370 wait complete. 2024-09-21T04:04:48.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.077+0000 7feca497b740 20 client.25406 trim_cache size 0 max 16384 2024-09-21T04:04:48.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.090+0000 7feca497b740 1 Processor -- start 2024-09-21T04:04:48.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.090+0000 7feca497b740 1 -- start start 2024-09-21T04:04:48.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.090+0000 7feca497b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990a9d89f0 0x55990adce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:48.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.090+0000 7feca497b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55990a9f11b0 0x55990add9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:48.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.090+0000 7feca497b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55990a9f9390 0x55990adc9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:48.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.090+0000 7feca497b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fec981b8800 con 0x55990a9d89f0 2024-09-21T04:04:48.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.090+0000 7feca497b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fec981b60d0 con 0x55990a9f9390 2024-09-21T04:04:48.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.090+0000 7feca497b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fec981b3f40 con 0x55990a9f11b0 2024-09-21T04:04:48.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.091+0000 7fec9c9a4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990a9d89f0 0x55990adce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:48.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.091+0000 7fec9c9a4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990a9d89f0 0x55990adce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57234/0 (socket says 172.21.15.64:57234) 2024-09-21T04:04:48.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.091+0000 7fec9c9a4640 1 -- 172.21.15.64:0/2222228090 learned_addr learned my addr 172.21.15.64:0/2222228090 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:48.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.091+0000 7fec977fe640 1 --2- 172.21.15.64:0/2222228090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55990a9f11b0 0x55990add9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:48.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.091+0000 7fec9c9a4640 1 --2- 172.21.15.64:0/2222228090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55990a9f9390 0x55990adc9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:48.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.091+0000 7fec9c9a4640 1 -- 172.21.15.64:0/2222228090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55990a9f11b0 msgr2=0x55990add9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:48.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.091+0000 7fec9c9a4640 1 --2- 172.21.15.64:0/2222228090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55990a9f11b0 0x55990add9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.091+0000 7fec9c9a4640 1 -- 172.21.15.64:0/2222228090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55990a9f9390 msgr2=0x55990adc9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:48.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.091+0000 7fec9c9a4640 1 --2- 172.21.15.64:0/2222228090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55990a9f9390 0x55990adc9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.094 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.091+0000 7fec9c9a4640 1 -- 172.21.15.64:0/2222228090 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55990adc4500 con 0x55990a9d89f0 2024-09-21T04:04:48.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.092+0000 7fec9c9a4640 1 --2- 172.21.15.64:0/2222228090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990a9d89f0 0x55990adce920 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fec90003c70 tx=0x7fec90003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b6695cc6a66e81 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:48.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.092+0000 7fec7dffb640 1 -- 172.21.15.64:0/2222228090 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90031450 con 0x55990a9d89f0 2024-09-21T04:04:48.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.092+0000 7fec7dffb640 1 -- 172.21.15.64:0/2222228090 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec9003f040 con 0x55990a9d89f0 2024-09-21T04:04:48.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.092+0000 7fec7dffb640 1 -- 172.21.15.64:0/2222228090 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec900305c0 con 0x55990a9d89f0 2024-09-21T04:04:48.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.093+0000 7feca497b740 1 -- 172.21.15.64:0/2222228090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990a9d89f0 msgr2=0x55990adce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:48.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.093+0000 7feca497b740 1 --2- 172.21.15.64:0/2222228090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990a9d89f0 0x55990adce920 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fec90003c70 tx=0x7fec90003d40 comp rx=0 tx=0).stop 2024-09-21T04:04:48.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.093+0000 7feca497b740 1 -- 172.21.15.64:0/2222228090 shutdown_connections 2024-09-21T04:04:48.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.093+0000 7feca497b740 1 --2- 172.21.15.64:0/2222228090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55990a9f9390 0x55990adc9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.093+0000 7feca497b740 1 --2- 172.21.15.64:0/2222228090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55990a9f11b0 0x55990add9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.093+0000 7feca497b740 1 --2- 172.21.15.64:0/2222228090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990a9d89f0 0x55990adce920 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.093+0000 7feca497b740 1 -- 172.21.15.64:0/2222228090 >> 172.21.15.64:0/2222228090 conn(0x55990aa0e460 msgr2=0x55990adce210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:48.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.093+0000 7feca497b740 1 -- 172.21.15.64:0/2222228090 shutdown_connections 2024-09-21T04:04:48.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.093+0000 7feca497b740 1 -- 172.21.15.64:0/2222228090 wait complete. 2024-09-21T04:04:48.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.094+0000 7feca497b740 1 -- start start 2024-09-21T04:04:48.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.094+0000 7feca497b740 1 Processor -- start 2024-09-21T04:04:48.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.094+0000 7feca497b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55990a9d89f0 0x55990add1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:48.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.094+0000 7feca497b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55990a9f11b0 0x55990add1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:48.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.094+0000 7feca497b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990a9f9390 0x55990ae48a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:48.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.095+0000 7fec9c9a4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990a9f9390 0x55990ae48a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:48.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.095+0000 7feca497b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fec981b4590 con 0x55990a9f9390 2024-09-21T04:04:48.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.095+0000 7fec9c9a4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990a9f9390 0x55990ae48a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57244/0 (socket says 172.21.15.64:57244) 2024-09-21T04:04:48.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.095+0000 7fec9c9a4640 1 -- 172.21.15.64:0/862582684 learned_addr learned my addr 172.21.15.64:0/862582684 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:48.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.095+0000 7feca497b740 1 -- 172.21.15.64:0/862582684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fec981b5a00 con 0x55990a9d89f0 2024-09-21T04:04:48.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.095+0000 7feca497b740 1 -- 172.21.15.64:0/862582684 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fec981b4c60 con 0x55990a9f11b0 2024-09-21T04:04:48.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.095+0000 7fec9c9a4640 1 --2- 172.21.15.64:0/862582684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55990a9d89f0 0x55990add1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:48.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.095+0000 7feca497b740 10 client.0 _ll_register_callbacks cb 0x7fec8c00f3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:48.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.095+0000 7fec977fe640 1 --2- 172.21.15.64:0/862582684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55990a9f11b0 0x55990add1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:48.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.095+0000 7fec9c9a4640 1 -- 172.21.15.64:0/862582684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55990a9f11b0 msgr2=0x55990add1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:48.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.095+0000 7fec9c9a4640 1 --2- 172.21.15.64:0/862582684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55990a9f11b0 0x55990add1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.095+0000 7fec9c9a4640 1 -- 172.21.15.64:0/862582684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55990a9d89f0 msgr2=0x55990add1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:48.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.095+0000 7fec9c9a4640 1 --2- 172.21.15.64:0/862582684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55990a9d89f0 0x55990add1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.095+0000 7fec9c9a4640 1 -- 172.21.15.64:0/862582684 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55990add2030 con 0x55990a9f9390 2024-09-21T04:04:48.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.096+0000 7fec777fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:04:48.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.096+0000 7fec9c9a4640 1 --2- 172.21.15.64:0/862582684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990a9f9390 0x55990ae48a60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fec90005e00 tx=0x7fec90000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6434051215a35c3b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:48.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.096+0000 7fec777fe640 1 -- 172.21.15.64:0/862582684 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90031660 con 0x55990a9f9390 2024-09-21T04:04:48.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.096+0000 7fec777fe640 1 -- 172.21.15.64:0/862582684 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fec90031800 con 0x55990a9f9390 2024-09-21T04:04:48.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.096+0000 7feca497b740 1 -- 172.21.15.64:0/862582684 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec9003f040 con 0x55990a9f9390 2024-09-21T04:04:48.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.096+0000 7fec777fe640 1 -- 172.21.15.64:0/862582684 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fec9006f050 con 0x55990a9f9390 2024-09-21T04:04:48.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.096+0000 7feca497b740 10 client.15678 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:48.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.096+0000 7feca497b740 1 -- 172.21.15.64:0/862582684 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55990ae49f20 con 0x55990a9f9390 2024-09-21T04:04:48.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.097+0000 7fec777fe640 1 -- 172.21.15.64:0/862582684 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec90036030 con 0x55990a9f9390 2024-09-21T04:04:48.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.097+0000 7fec777fe640 1 -- 172.21.15.64:0/862582684 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fec90030930 con 0x55990a9f9390 2024-09-21T04:04:48.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.097+0000 7feca497b740 10 client.15678 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:48.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.097+0000 7feca497b740 10 client.15678 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:04:48.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.097+0000 7feca497b740 20 client.15678 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:48.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.097+0000 7feca497b740 1 --2- 172.21.15.64:0/862582684 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55990aa1a070 0x55990ae4c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:48.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.097+0000 7feca497b740 4 client.15678 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:04:48.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.097+0000 7feca497b740 1 -- 172.21.15.64:0/862582684 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fec9003f040 con 0x55990aa1a070 2024-09-21T04:04:48.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.098+0000 7fec977fe640 1 --2- 172.21.15.64:0/862582684 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55990aa1a070 0x55990ae4c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:48.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.098+0000 7fec977fe640 1 --2- 172.21.15.64:0/862582684 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55990aa1a070 0x55990ae4c100 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=332d274ee62b81a1 server_cookie=349d6ad9432ee8a4 in_seq=0 out_seq=0 2024-09-21T04:04:48.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.098+0000 7fec777fe640 10 client.15678 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:04:48.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.098+0000 7fec777fe640 1 -- 172.21.15.64:0/862582684 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fec9003f040 con 0x55990aa1a070 2024-09-21T04:04:48.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.098+0000 7fec777fe640 10 client.15678 handle_command_reply: tid=0 2024-09-21T04:04:48.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.098+0000 7feca497b740 1 client.15678 shutdown 2024-09-21T04:04:48.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.099+0000 7feca497b740 1 -- 172.21.15.64:0/862582684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990a9f9390 msgr2=0x55990ae48a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:48.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.099+0000 7feca497b740 1 --2- 172.21.15.64:0/862582684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990a9f9390 0x55990ae48a60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fec90005e00 tx=0x7fec90000f40 comp rx=0 tx=0).stop 2024-09-21T04:04:48.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.099+0000 7feca497b740 1 -- 172.21.15.64:0/862582684 shutdown_connections 2024-09-21T04:04:48.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.099+0000 7feca497b740 1 --2- 172.21.15.64:0/862582684 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55990aa1a070 0x55990ae4c100 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.099+0000 7fec777fe640 0 client.15678 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:04:48.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.099+0000 7feca497b740 1 --2- 172.21.15.64:0/862582684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55990a9f9390 0x55990ae48a60 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.099+0000 7feca497b740 1 --2- 172.21.15.64:0/862582684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55990a9f11b0 0x55990add1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.099+0000 7feca497b740 1 --2- 172.21.15.64:0/862582684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55990a9d89f0 0x55990add1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.099+0000 7feca497b740 1 -- 172.21.15.64:0/862582684 >> 172.21.15.64:0/862582684 conn(0x55990aa0e460 msgr2=0x55990adc26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:48.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.099+0000 7feca497b740 1 -- 172.21.15.64:0/862582684 shutdown_connections 2024-09-21T04:04:48.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.099+0000 7feca497b740 1 -- 172.21.15.64:0/862582684 wait complete. 2024-09-21T04:04:48.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.099+0000 7feca497b740 20 client.15678 trim_cache size 0 max 16384 2024-09-21T04:04:48.104 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:04:48.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.100+0000 7fec9e407640 1 -- 172.21.15.64:0/3156678245 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fec800b6860 msgr2=0x7fec800b8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:48.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.100+0000 7fec9e407640 1 --2- 172.21.15.64:0/3156678245 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fec800b6860 0x7fec800b8d70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fec880e5320 tx=0x7fec880ad2b0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.100+0000 7fec9e407640 1 -- 172.21.15.64:0/3156678245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec98102a60 msgr2=0x7fec9819c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:48.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.100+0000 7fec9e407640 1 --2- 172.21.15.64:0/3156678245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec98102a60 0x7fec9819c670 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fec8800e9f0 tx=0x7fec8800eeb0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.100+0000 7fec9e407640 1 -- 172.21.15.64:0/3156678245 shutdown_connections 2024-09-21T04:04:48.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.100+0000 7fec9e407640 1 --2- 172.21.15.64:0/3156678245 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fec800b6860 0x7fec800b8d70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.100+0000 7fec9e407640 1 --2- 172.21.15.64:0/3156678245 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fec98108cf0 0x7fec981a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.100+0000 7fec9e407640 1 --2- 172.21.15.64:0/3156678245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec981083a0 0x7fec9819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.100+0000 7fec9e407640 1 --2- 172.21.15.64:0/3156678245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fec98102a60 0x7fec9819c670 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:48.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.100+0000 7fec9e407640 1 -- 172.21.15.64:0/3156678245 >> 172.21.15.64:0/3156678245 conn(0x7fec98079730 msgr2=0x7fec9806aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:48.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.100+0000 7fec9e407640 1 -- 172.21.15.64:0/3156678245 shutdown_connections 2024-09-21T04:04:48.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:48.100+0000 7fec9e407640 1 -- 172.21.15.64:0/3156678245 wait complete. 2024-09-21T04:04:48.815 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T04:04:49.079 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:04:49.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.609+0000 7f4f2aab7640 1 -- 172.21.15.64:0/2719385927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f24105190 msgr2=0x7f4f241055a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:49.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.609+0000 7f4f2aab7640 1 --2- 172.21.15.64:0/2719385927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f24105190 0x7f4f241055a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4f10009900 tx=0x7f4f1002a410 comp rx=0 tx=0).stop 2024-09-21T04:04:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.609+0000 7f4f2aab7640 1 -- 172.21.15.64:0/2719385927 shutdown_connections 2024-09-21T04:04:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.609+0000 7f4f2aab7640 1 --2- 172.21.15.64:0/2719385927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f24105ae0 0x7f4f2410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.609+0000 7f4f2aab7640 1 --2- 172.21.15.64:0/2719385927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f24105190 0x7f4f241055a0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.609+0000 7f4f2aab7640 1 --2- 172.21.15.64:0/2719385927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f241031c0 0x7f4f241035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.609+0000 7f4f2aab7640 1 -- 172.21.15.64:0/2719385927 >> 172.21.15.64:0/2719385927 conn(0x7f4f24077c60 msgr2=0x7f4f240780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.609+0000 7f4f2aab7640 1 -- 172.21.15.64:0/2719385927 shutdown_connections 2024-09-21T04:04:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.609+0000 7f4f2aab7640 1 -- 172.21.15.64:0/2719385927 wait complete. 2024-09-21T04:04:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.610+0000 7f4f2aab7640 1 Processor -- start 2024-09-21T04:04:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.610+0000 7f4f2aab7640 1 -- start start 2024-09-21T04:04:49.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.610+0000 7f4f2aab7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f241031c0 0x7f4f241993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:49.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.610+0000 7f4f2aab7640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f24105190 0x7f4f24199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:49.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.610+0000 7f4f2aab7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f24105ae0 0x7f4f2419ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:49.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.610+0000 7f4f2aab7640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4f24112980 con 0x7f4f24105190 2024-09-21T04:04:49.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.610+0000 7f4f2aab7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4f24112800 con 0x7f4f241031c0 2024-09-21T04:04:49.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.610+0000 7f4f2aab7640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4f24112b00 con 0x7f4f24105ae0 2024-09-21T04:04:49.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.611+0000 7f4f23fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f24105190 0x7f4f24199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:49.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.611+0000 7f4f23fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f24105190 0x7f4f24199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57272/0 (socket says 172.21.15.64:57272) 2024-09-21T04:04:49.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.611+0000 7f4f23fff640 1 -- 172.21.15.64:0/2072320867 learned_addr learned my addr 172.21.15.64:0/2072320867 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:49.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.611+0000 7f4f28853640 1 --2- 172.21.15.64:0/2072320867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f241031c0 0x7f4f241993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:49.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.611+0000 7f4f29054640 1 --2- 172.21.15.64:0/2072320867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f24105ae0 0x7f4f2419ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:49.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.611+0000 7f4f23fff640 1 -- 172.21.15.64:0/2072320867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f24105ae0 msgr2=0x7f4f2419ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:49.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.611+0000 7f4f23fff640 1 --2- 172.21.15.64:0/2072320867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f24105ae0 0x7f4f2419ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.611+0000 7f4f23fff640 1 -- 172.21.15.64:0/2072320867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f241031c0 msgr2=0x7f4f241993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:49.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.611+0000 7f4f23fff640 1 --2- 172.21.15.64:0/2072320867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f241031c0 0x7f4f241993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.611+0000 7f4f23fff640 1 -- 172.21.15.64:0/2072320867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f2419f360 con 0x7f4f24105190 2024-09-21T04:04:49.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.611+0000 7f4f23fff640 1 --2- 172.21.15.64:0/2072320867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f24105190 0x7f4f24199990 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4f10032fd0 tx=0x7f4f10004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:49.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.612+0000 7f4f21ffb640 1 -- 172.21.15.64:0/2072320867 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1003f030 con 0x7f4f24105190 2024-09-21T04:04:49.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.612+0000 7f4f21ffb640 1 -- 172.21.15.64:0/2072320867 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f10004490 con 0x7f4f24105190 2024-09-21T04:04:49.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.612+0000 7f4f21ffb640 1 -- 172.21.15.64:0/2072320867 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10035070 con 0x7f4f24105190 2024-09-21T04:04:49.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.612+0000 7f4f2aab7640 1 -- 172.21.15.64:0/2072320867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f2419f5f0 con 0x7f4f24105190 2024-09-21T04:04:49.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.612+0000 7f4f2aab7640 1 -- 172.21.15.64:0/2072320867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f2419f9c0 con 0x7f4f24105190 2024-09-21T04:04:49.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.615+0000 7f4f3102b740 1 Processor -- start 2024-09-21T04:04:49.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.615+0000 7f4f3102b740 1 -- start start 2024-09-21T04:04:49.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.615+0000 7f4f3102b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583d74dd070 0x5583d78842c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:49.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.615+0000 7f4f3102b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5583d74b41b0 0x5583d7884800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:49.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.615+0000 7f4f3102b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d74bc390 0x5583d7884d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:49.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.616+0000 7f4f3102b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5583d77faea0 con 0x5583d74bc390 2024-09-21T04:04:49.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.616+0000 7f4f3102b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5583d77ea2d0 con 0x5583d74dd070 2024-09-21T04:04:49.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.616+0000 7f4f3102b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5583d77ee0e0 con 0x5583d74b41b0 2024-09-21T04:04:49.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.616+0000 7f4f29054640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583d74dd070 0x5583d78842c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:49.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.616+0000 7f4f28853640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5583d74b41b0 0x5583d7884800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:49.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.616+0000 7f4f29054640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583d74dd070 0x5583d78842c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:57524/0 (socket says 172.21.15.64:57524) 2024-09-21T04:04:49.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.616+0000 7f4f29054640 1 -- 172.21.15.64:0/735563781 learned_addr learned my addr 172.21.15.64:0/735563781 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:49.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.616+0000 7f4f29054640 1 -- 172.21.15.64:0/735563781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5583d74b41b0 msgr2=0x5583d7884800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:49.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.616+0000 7f4f29054640 1 --2- 172.21.15.64:0/735563781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5583d74b41b0 0x5583d7884800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.616+0000 7f4f29054640 1 -- 172.21.15.64:0/735563781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d74bc390 msgr2=0x5583d7884d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:04:49.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.616+0000 7f4f29054640 1 --2- 172.21.15.64:0/735563781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d74bc390 0x5583d7884d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.616+0000 7f4f29054640 1 -- 172.21.15.64:0/735563781 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583d7548650 con 0x5583d74dd070 2024-09-21T04:04:49.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.617+0000 7f4f29054640 1 --2- 172.21.15.64:0/735563781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583d74dd070 0x5583d78842c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c009f90 tx=0x7f4f1c010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=555d851708249bcf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:49.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.617+0000 7f4f21ffb640 1 -- 172.21.15.64:0/2072320867 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 154694+0+0 (secure 0 0 0) 0x7f4f10042030 con 0x7f4f24105190 2024-09-21T04:04:49.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.617+0000 7f4f0effd640 1 -- 172.21.15.64:0/735563781 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c011700 con 0x5583d74dd070 2024-09-21T04:04:49.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.617+0000 7f4f0effd640 1 -- 172.21.15.64:0/735563781 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f1c011d50 con 0x5583d74dd070 2024-09-21T04:04:49.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.618+0000 7f4f21ffb640 1 --2- 172.21.15.64:0/2072320867 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4f140b6790 0x7f4f140b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:49.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.618+0000 7f4f21ffb640 1 -- 172.21.15.64:0/2072320867 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f4f10107df0 con 0x7f4f24105190 2024-09-21T04:04:49.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.618+0000 7f4f28853640 1 --2- 172.21.15.64:0/2072320867 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4f140b6790 0x7f4f140b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:49.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.618+0000 7f4f3102b740 1 -- 172.21.15.64:0/735563781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583d74dd070 msgr2=0x5583d78842c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:49.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.618+0000 7f4f3102b740 1 --2- 172.21.15.64:0/735563781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583d74dd070 0x5583d78842c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c009f90 tx=0x7f4f1c010860 comp rx=0 tx=0).stop 2024-09-21T04:04:49.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.618+0000 7f4f28853640 1 --2- 172.21.15.64:0/2072320867 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4f140b6790 0x7f4f140b8ca0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5583d7882f40 tx=0x7f4f1800c430 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:49.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.619+0000 7f4f3102b740 1 -- 172.21.15.64:0/735563781 shutdown_connections 2024-09-21T04:04:49.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.619+0000 7f4f3102b740 1 --2- 172.21.15.64:0/735563781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d74bc390 0x5583d7884d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.619+0000 7f4f3102b740 1 --2- 172.21.15.64:0/735563781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5583d74b41b0 0x5583d7884800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.619+0000 7f4f3102b740 1 --2- 172.21.15.64:0/735563781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583d74dd070 0x5583d78842c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.619+0000 7f4f3102b740 1 -- 172.21.15.64:0/735563781 >> 172.21.15.64:0/735563781 conn(0x5583d74d1460 msgr2=0x5583d78821b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:49.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.619+0000 7f4f3102b740 1 -- 172.21.15.64:0/735563781 shutdown_connections 2024-09-21T04:04:49.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.619+0000 7f4f3102b740 1 -- 172.21.15.64:0/735563781 wait complete. 2024-09-21T04:04:49.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.620+0000 7f4f3102b740 1 -- start start 2024-09-21T04:04:49.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.620+0000 7f4f3102b740 1 Processor -- start 2024-09-21T04:04:49.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.621+0000 7f4f3102b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5583d74b41b0 0x5583d788f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:49.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.621+0000 7f4f3102b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583d74bc390 0x5583d788fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:49.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.621+0000 7f4f3102b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d74dd070 0x5583d7890380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:49.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.621+0000 7f4f29054640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d74dd070 0x5583d7890380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.621+0000 7f4f29054640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d74dd070 0x5583d7890380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57276/0 (socket says 172.21.15.64:57276) 2024-09-21T04:04:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.621+0000 7f4f29054640 1 -- 172.21.15.64:0/2893125251 learned_addr learned my addr 172.21.15.64:0/2893125251 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.621+0000 7f4f28853640 1 --2- 172.21.15.64:0/2893125251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5583d74b41b0 0x5583d788f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.621+0000 7f4f23fff640 1 --2- 172.21.15.64:0/2893125251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583d74bc390 0x5583d788fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.621+0000 7f4f3102b740 1 -- 172.21.15.64:0/2893125251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5583d764b700 con 0x5583d74dd070 2024-09-21T04:04:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.622+0000 7f4f3102b740 1 -- 172.21.15.64:0/2893125251 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5583d7785160 con 0x5583d74bc390 2024-09-21T04:04:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.622+0000 7f4f3102b740 1 -- 172.21.15.64:0/2893125251 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5583d7892b40 con 0x5583d74b41b0 2024-09-21T04:04:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.622+0000 7f4f23fff640 1 -- 172.21.15.64:0/2893125251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5583d74b41b0 msgr2=0x5583d788f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.622+0000 7f4f23fff640 1 --2- 172.21.15.64:0/2893125251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5583d74b41b0 0x5583d788f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.622+0000 7f4f23fff640 1 -- 172.21.15.64:0/2893125251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d74dd070 msgr2=0x5583d7890380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:49.632 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.622+0000 7f4f23fff640 1 --2- 172.21.15.64:0/2893125251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d74dd070 0x5583d7890380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.622+0000 7f4f3102b740 10 client.0 _ll_register_callbacks cb 0x5583d779e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.622+0000 7f4f23fff640 1 -- 172.21.15.64:0/2893125251 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583d77b2c90 con 0x5583d74bc390 2024-09-21T04:04:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.622+0000 7f4f29054640 1 --2- 172.21.15.64:0/2893125251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d74dd070 0x5583d7890380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.622+0000 7f4f3102b740 1 -- 172.21.15.64:0/2893125251 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f1c011d50 con 0x5583d74bc390 2024-09-21T04:04:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.623+0000 7f4f23fff640 1 --2- 172.21.15.64:0/2893125251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583d74bc390 0x5583d788fe20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4f101053f0 tx=0x7f4f100c98e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3267f59dc4bd90ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.623+0000 7f4f03fff640 10 client.25424 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:04:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.623+0000 7f4f03fff640 1 -- 172.21.15.64:0/2893125251 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10049040 con 0x5583d74bc390 2024-09-21T04:04:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.623+0000 7f4f03fff640 1 -- 172.21.15.64:0/2893125251 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f4f101056a0 con 0x5583d74bc390 2024-09-21T04:04:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.623+0000 7f4f03fff640 1 -- 172.21.15.64:0/2893125251 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4f100cd1b0 con 0x5583d74bc390 2024-09-21T04:04:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.623+0000 7f4f3102b740 10 client.25424 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:49.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.623+0000 7f4f3102b740 1 -- 172.21.15.64:0/2893125251 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5583d789e670 con 0x5583d74bc390 2024-09-21T04:04:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.623+0000 7f4f03fff640 1 -- 172.21.15.64:0/2893125251 <== mon.2 v2:172.21.15.119:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4f1003bc20 con 0x5583d74bc390 2024-09-21T04:04:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.624+0000 7f4f3102b740 10 client.25424 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.624+0000 7f4f3102b740 10 client.25424 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:04:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.624+0000 7f4f3102b740 20 client.25424 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.624+0000 7f4f3102b740 1 --2- 172.21.15.64:0/2893125251 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5583d749b9f0 0x5583d78a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.624+0000 7f4f3102b740 4 client.25424 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:04:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.624+0000 7f4f3102b740 1 -- 172.21.15.64:0/2893125251 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583d77fcdf0 con 0x5583d749b9f0 2024-09-21T04:04:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.624+0000 7f4f28853640 1 --2- 172.21.15.64:0/2893125251 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5583d749b9f0 0x5583d78a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.625+0000 7f4f28853640 1 --2- 172.21.15.64:0/2893125251 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5583d749b9f0 0x5583d78a1e50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bd3a6d6698e22411 server_cookie=bd41e9656f7f3729 in_seq=0 out_seq=0 2024-09-21T04:04:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.625+0000 7f4f03fff640 10 client.25424 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:04:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.625+0000 7f4f03fff640 1 -- 172.21.15.64:0/2893125251 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5583d77fcdf0 con 0x5583d749b9f0 2024-09-21T04:04:49.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.625+0000 7f4f03fff640 10 client.25424 handle_command_reply: tid=0 2024-09-21T04:04:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.625+0000 7f4f3102b740 1 client.25424 shutdown 2024-09-21T04:04:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.626+0000 7f4f3102b740 1 -- 172.21.15.64:0/2893125251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583d74bc390 msgr2=0x5583d788fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.626+0000 7f4f3102b740 1 --2- 172.21.15.64:0/2893125251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583d74bc390 0x5583d788fe20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4f101053f0 tx=0x7f4f100c98e0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.626+0000 7f4f3102b740 1 -- 172.21.15.64:0/2893125251 shutdown_connections 2024-09-21T04:04:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.626+0000 7f4f3102b740 1 --2- 172.21.15.64:0/2893125251 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5583d749b9f0 0x5583d78a1e50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.626+0000 7f4f3102b740 1 --2- 172.21.15.64:0/2893125251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d74dd070 0x5583d7890380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.626+0000 7f4f3102b740 1 --2- 172.21.15.64:0/2893125251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583d74bc390 0x5583d788fe20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.626+0000 7f4f3102b740 1 --2- 172.21.15.64:0/2893125251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5583d74b41b0 0x5583d788f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.626+0000 7f4f3102b740 1 -- 172.21.15.64:0/2893125251 >> 172.21.15.64:0/2893125251 conn(0x5583d74d1460 msgr2=0x5583d7887450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.626+0000 7f4f03fff640 0 client.25424 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:04:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.626+0000 7f4f3102b740 1 -- 172.21.15.64:0/2893125251 shutdown_connections 2024-09-21T04:04:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.626+0000 7f4f3102b740 1 -- 172.21.15.64:0/2893125251 wait complete. 2024-09-21T04:04:49.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.626+0000 7f4f3102b740 20 client.25424 trim_cache size 0 max 16384 2024-09-21T04:04:49.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.653+0000 7f4f3102b740 1 Processor -- start 2024-09-21T04:04:49.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.653+0000 7f4f3102b740 1 -- start start 2024-09-21T04:04:49.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.653+0000 7f4f3102b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5583d74b41b0 0x5583d788fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:49.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.653+0000 7f4f3102b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583d74bc390 0x5583d7881760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:49.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.653+0000 7f4f3102b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d74dd070 0x5583d7881ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:49.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.653+0000 7f4f3102b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4f241b4300 con 0x5583d74dd070 2024-09-21T04:04:49.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.653+0000 7f4f3102b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4f241b1bd0 con 0x5583d74bc390 2024-09-21T04:04:49.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.653+0000 7f4f3102b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4f241afa20 con 0x5583d74b41b0 2024-09-21T04:04:49.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.654+0000 7f4f29054640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d74dd070 0x5583d7881ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:49.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.654+0000 7f4f23fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583d74bc390 0x5583d7881760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:49.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.654+0000 7f4f29054640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d74dd070 0x5583d7881ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57282/0 (socket says 172.21.15.64:57282) 2024-09-21T04:04:49.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.654+0000 7f4f29054640 1 -- 172.21.15.64:0/1833879885 learned_addr learned my addr 172.21.15.64:0/1833879885 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:49.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.654+0000 7f4f28853640 1 --2- 172.21.15.64:0/1833879885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5583d74b41b0 0x5583d788fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:49.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.654+0000 7f4f23fff640 1 -- 172.21.15.64:0/1833879885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5583d74b41b0 msgr2=0x5583d788fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:49.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.654+0000 7f4f23fff640 1 --2- 172.21.15.64:0/1833879885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5583d74b41b0 0x5583d788fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.654+0000 7f4f23fff640 1 -- 172.21.15.64:0/1833879885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d74dd070 msgr2=0x5583d7881ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:49.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.654+0000 7f4f23fff640 1 --2- 172.21.15.64:0/1833879885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d74dd070 0x5583d7881ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.654+0000 7f4f23fff640 1 -- 172.21.15.64:0/1833879885 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583d7886800 con 0x5583d74bc390 2024-09-21T04:04:49.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.654+0000 7f4f29054640 1 --2- 172.21.15.64:0/1833879885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d74dd070 0x5583d7881ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:49.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.655+0000 7f4f23fff640 1 --2- 172.21.15.64:0/1833879885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583d74bc390 0x5583d7881760 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4f10005d00 tx=0x7f4f100c9a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1924baf9904c1410 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:49.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.656+0000 7f4f0e7fc640 1 -- 172.21.15.64:0/1833879885 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10049070 con 0x5583d74bc390 2024-09-21T04:04:49.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.656+0000 7f4f0e7fc640 1 -- 172.21.15.64:0/1833879885 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f1c011d50 con 0x5583d74bc390 2024-09-21T04:04:49.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.656+0000 7f4f0e7fc640 1 -- 172.21.15.64:0/1833879885 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f100d2520 con 0x5583d74bc390 2024-09-21T04:04:49.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.657+0000 7f4f3102b740 1 -- 172.21.15.64:0/1833879885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583d74bc390 msgr2=0x5583d7881760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:49.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.657+0000 7f4f3102b740 1 --2- 172.21.15.64:0/1833879885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583d74bc390 0x5583d7881760 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4f10005d00 tx=0x7f4f100c9a10 comp rx=0 tx=0).stop 2024-09-21T04:04:49.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.657+0000 7f4f3102b740 1 -- 172.21.15.64:0/1833879885 shutdown_connections 2024-09-21T04:04:49.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.657+0000 7f4f3102b740 1 --2- 172.21.15.64:0/1833879885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d74dd070 0x5583d7881ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.657+0000 7f4f3102b740 1 --2- 172.21.15.64:0/1833879885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583d74bc390 0x5583d7881760 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.657+0000 7f4f3102b740 1 --2- 172.21.15.64:0/1833879885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5583d74b41b0 0x5583d788fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.657+0000 7f4f3102b740 1 -- 172.21.15.64:0/1833879885 >> 172.21.15.64:0/1833879885 conn(0x5583d74d1460 msgr2=0x5583d788f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:49.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.657+0000 7f4f3102b740 1 -- 172.21.15.64:0/1833879885 shutdown_connections 2024-09-21T04:04:49.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.657+0000 7f4f3102b740 1 -- 172.21.15.64:0/1833879885 wait complete. 2024-09-21T04:04:49.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.658+0000 7f4f3102b740 1 -- start start 2024-09-21T04:04:49.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.658+0000 7f4f3102b740 1 Processor -- start 2024-09-21T04:04:49.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.659+0000 7f4f3102b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d74b41b0 0x5583d78a5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:49.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.659+0000 7f4f28853640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d74b41b0 0x5583d78a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:49.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.659+0000 7f4f28853640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d74b41b0 0x5583d78a5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57292/0 (socket says 172.21.15.64:57292) 2024-09-21T04:04:49.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.659+0000 7f4f28853640 1 -- 172.21.15.64:0/3948757610 learned_addr learned my addr 172.21.15.64:0/3948757610 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:49.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.659+0000 7f4f3102b740 1 --2- 172.21.15.64:0/3948757610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5583d74bc390 0x5583d78a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:49.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.659+0000 7f4f3102b740 1 --2- 172.21.15.64:0/3948757610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583d74dd070 0x5583d78a3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:49.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.659+0000 7f4f3102b740 1 -- 172.21.15.64:0/3948757610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4f241b0090 con 0x5583d74b41b0 2024-09-21T04:04:49.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.659+0000 7f4f3102b740 1 -- 172.21.15.64:0/3948757610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4f241b1500 con 0x5583d74dd070 2024-09-21T04:04:49.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.659+0000 7f4f3102b740 1 -- 172.21.15.64:0/3948757610 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4f241b0760 con 0x5583d74bc390 2024-09-21T04:04:49.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.660+0000 7f4f23fff640 1 --2- 172.21.15.64:0/3948757610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5583d74bc390 0x5583d78a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:49.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.660+0000 7f4f23fff640 1 -- 172.21.15.64:0/3948757610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583d74dd070 msgr2=0x5583d78a3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:49.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.660+0000 7f4f23fff640 1 --2- 172.21.15.64:0/3948757610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583d74dd070 0x5583d78a3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.660+0000 7f4f23fff640 1 -- 172.21.15.64:0/3948757610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d74b41b0 msgr2=0x5583d78a5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:49.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.660+0000 7f4f23fff640 1 --2- 172.21.15.64:0/3948757610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d74b41b0 0x5583d78a5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.660+0000 7f4f23fff640 1 -- 172.21.15.64:0/3948757610 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583d790b6d0 con 0x5583d74bc390 2024-09-21T04:04:49.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.660+0000 7f4f28853640 1 --2- 172.21.15.64:0/3948757610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d74b41b0 0x5583d78a5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:49.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.660+0000 7f4f3102b740 10 client.0 _ll_register_callbacks cb 0x7f4f10105280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:49.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.660+0000 7f4f3102b740 1 -- 172.21.15.64:0/3948757610 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f1c011d50 con 0x5583d74bc390 2024-09-21T04:04:49.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.661+0000 7f4f23fff640 1 --2- 172.21.15.64:0/3948757610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5583d74bc390 0x5583d78a2c10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4f100c9320 tx=0x7f4f100c98e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b4858e808691ac2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:49.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.661+0000 7f4f03fff640 10 client.25348 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-21T04:04:49.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.661+0000 7f4f03fff640 1 -- 172.21.15.64:0/3948757610 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10049070 con 0x5583d74bc390 2024-09-21T04:04:49.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.661+0000 7f4f03fff640 1 -- 172.21.15.64:0/3948757610 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f4f101056a0 con 0x5583d74bc390 2024-09-21T04:04:49.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.661+0000 7f4f03fff640 1 -- 172.21.15.64:0/3948757610 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4f100cd810 con 0x5583d74bc390 2024-09-21T04:04:49.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.661+0000 7f4f3102b740 10 client.25348 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:49.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.661+0000 7f4f3102b740 1 -- 172.21.15.64:0/3948757610 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5583d790b6d0 con 0x5583d74bc390 2024-09-21T04:04:49.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.662+0000 7f4f03fff640 1 -- 172.21.15.64:0/3948757610 <== mon.1 v2:172.21.15.131:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4f100ccca0 con 0x5583d74bc390 2024-09-21T04:04:49.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.662+0000 7f4f3102b740 10 client.25348 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:49.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.662+0000 7f4f3102b740 10 client.25348 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:04:49.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.662+0000 7f4f3102b740 20 client.25348 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:49.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.662+0000 7f4f3102b740 1 --2- 172.21.15.64:0/3948757610 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5583d7470da0 0x5583d790e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:49.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.662+0000 7f4f3102b740 4 client.25348 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:04:49.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.662+0000 7f4f3102b740 1 -- 172.21.15.64:0/3948757610 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5583d77f88a0 con 0x5583d7470da0 2024-09-21T04:04:49.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.662+0000 7f4f28853640 1 --2- 172.21.15.64:0/3948757610 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5583d7470da0 0x5583d790e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:49.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.663+0000 7f4f03fff640 10 client.25348 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:04:49.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.663+0000 7f4f28853640 1 --2- 172.21.15.64:0/3948757610 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5583d7470da0 0x5583d790e790 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8f7d0c4b77728f0d server_cookie=ef1d3f4395d79b6b in_seq=0 out_seq=0 2024-09-21T04:04:49.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.663+0000 7f4f03fff640 1 -- 172.21.15.64:0/3948757610 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5583d77f88a0 con 0x5583d7470da0 2024-09-21T04:04:49.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.663+0000 7f4f03fff640 10 client.25348 handle_command_reply: tid=0 2024-09-21T04:04:49.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.663+0000 7f4f3102b740 1 client.25348 shutdown 2024-09-21T04:04:49.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.663+0000 7f4f3102b740 1 -- 172.21.15.64:0/3948757610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5583d74bc390 msgr2=0x5583d78a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:49.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.663+0000 7f4f3102b740 1 --2- 172.21.15.64:0/3948757610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5583d74bc390 0x5583d78a2c10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4f100c9320 tx=0x7f4f100c98e0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.663+0000 7f4f3102b740 1 -- 172.21.15.64:0/3948757610 shutdown_connections 2024-09-21T04:04:49.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.663+0000 7f4f3102b740 1 --2- 172.21.15.64:0/3948757610 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5583d7470da0 0x5583d790e790 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.663+0000 7f4f3102b740 1 --2- 172.21.15.64:0/3948757610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5583d74dd070 0x5583d78a3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.663+0000 7f4f3102b740 1 --2- 172.21.15.64:0/3948757610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5583d74bc390 0x5583d78a2c10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.669 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:04:49.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.663+0000 7f4f03fff640 0 client.25348 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:04:49.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.663+0000 7f4f3102b740 1 --2- 172.21.15.64:0/3948757610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583d74b41b0 0x5583d78a5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.663+0000 7f4f3102b740 1 -- 172.21.15.64:0/3948757610 >> 172.21.15.64:0/3948757610 conn(0x5583d74d1460 msgr2=0x5583d7885250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:49.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.664+0000 7f4f3102b740 1 -- 172.21.15.64:0/3948757610 shutdown_connections 2024-09-21T04:04:49.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.664+0000 7f4f3102b740 1 -- 172.21.15.64:0/3948757610 wait complete. 2024-09-21T04:04:49.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.664+0000 7f4f3102b740 20 client.25348 trim_cache size 0 max 16384 2024-09-21T04:04:49.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.665+0000 7f4f2aab7640 1 -- 172.21.15.64:0/2072320867 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4f140b6790 msgr2=0x7f4f140b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:49.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.665+0000 7f4f2aab7640 1 --2- 172.21.15.64:0/2072320867 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4f140b6790 0x7f4f140b8ca0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5583d7882f40 tx=0x7f4f1800c430 comp rx=0 tx=0).stop 2024-09-21T04:04:49.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.665+0000 7f4f2aab7640 1 -- 172.21.15.64:0/2072320867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f24105190 msgr2=0x7f4f24199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:49.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.665+0000 7f4f2aab7640 1 --2- 172.21.15.64:0/2072320867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f24105190 0x7f4f24199990 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4f10032fd0 tx=0x7f4f10004270 comp rx=0 tx=0).stop 2024-09-21T04:04:49.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.665+0000 7f4f2aab7640 1 -- 172.21.15.64:0/2072320867 shutdown_connections 2024-09-21T04:04:49.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.665+0000 7f4f2aab7640 1 --2- 172.21.15.64:0/2072320867 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4f140b6790 0x7f4f140b8ca0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.665+0000 7f4f2aab7640 1 --2- 172.21.15.64:0/2072320867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f24105ae0 0x7f4f2419ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.665+0000 7f4f2aab7640 1 --2- 172.21.15.64:0/2072320867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f24105190 0x7f4f24199990 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.665+0000 7f4f2aab7640 1 --2- 172.21.15.64:0/2072320867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f241031c0 0x7f4f241993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:49.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.665+0000 7f4f2aab7640 1 -- 172.21.15.64:0/2072320867 >> 172.21.15.64:0/2072320867 conn(0x7f4f24077c60 msgr2=0x7f4f241065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:49.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.665+0000 7f4f2aab7640 1 -- 172.21.15.64:0/2072320867 shutdown_connections 2024-09-21T04:04:49.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:49.665+0000 7f4f2aab7640 1 -- 172.21.15.64:0/2072320867 wait complete. 2024-09-21T04:04:50.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:49 smithi119 ceph-mon[34811]: pgmap v345: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-09-21T04:04:50.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:49 smithi064 ceph-mon[31180]: pgmap v345: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-09-21T04:04:50.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:49 smithi131 ceph-mon[35193]: pgmap v345: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-09-21T04:04:50.424 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T04:04:50.721 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:04:51.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.287+0000 7fcdd2e85640 1 -- 172.21.15.64:0/1020299533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdcc109e00 msgr2=0x7fcdcc110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:51.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.287+0000 7fcdd2e85640 1 --2- 172.21.15.64:0/1020299533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdcc109e00 0x7fcdcc110630 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4009ca0 tx=0x7fcdc402a8c0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.288+0000 7fcdd2e85640 1 -- 172.21.15.64:0/1020299533 shutdown_connections 2024-09-21T04:04:51.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.288+0000 7fcdd2e85640 1 --2- 172.21.15.64:0/1020299533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdcc109e00 0x7fcdcc110630 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.288+0000 7fcdd2e85640 1 --2- 172.21.15.64:0/1020299533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdcc1094b0 0x7fcdcc1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.288+0000 7fcdd2e85640 1 --2- 172.21.15.64:0/1020299533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdcc103bc0 0x7fcdcc103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.288+0000 7fcdd2e85640 1 -- 172.21.15.64:0/1020299533 >> 172.21.15.64:0/1020299533 conn(0x7fcdcc07b910 msgr2=0x7fcdcc07bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:51.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.288+0000 7fcdd2e85640 1 -- 172.21.15.64:0/1020299533 shutdown_connections 2024-09-21T04:04:51.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.288+0000 7fcdd2e85640 1 -- 172.21.15.64:0/1020299533 wait complete. 2024-09-21T04:04:51.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.289+0000 7fcdd2e85640 1 Processor -- start 2024-09-21T04:04:51.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.289+0000 7fcdd2e85640 1 -- start start 2024-09-21T04:04:51.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.290+0000 7fcdd2e85640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdcc103bc0 0x7fcdcc19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:51.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.290+0000 7fcdd2e85640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdcc1094b0 0x7fcdcc19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:51.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.290+0000 7fcdd0c21640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdcc103bc0 0x7fcdcc19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:51.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.290+0000 7fcdd0c21640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdcc103bc0 0x7fcdcc19a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57310/0 (socket says 172.21.15.64:57310) 2024-09-21T04:04:51.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.290+0000 7fcdd0c21640 1 -- 172.21.15.64:0/1791591942 learned_addr learned my addr 172.21.15.64:0/1791591942 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:51.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.290+0000 7fcdd2e85640 1 --2- 172.21.15.64:0/1791591942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdcc109e00 0x7fcdcc19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:51.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.290+0000 7fcdd2e85640 1 -- 172.21.15.64:0/1791591942 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc113ac0 con 0x7fcdcc103bc0 2024-09-21T04:04:51.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.290+0000 7fcdd2e85640 1 -- 172.21.15.64:0/1791591942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc113940 con 0x7fcdcc1094b0 2024-09-21T04:04:51.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.290+0000 7fcdd2e85640 1 -- 172.21.15.64:0/1791591942 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc113c40 con 0x7fcdcc109e00 2024-09-21T04:04:51.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.290+0000 7fcdcbfff640 1 --2- 172.21.15.64:0/1791591942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdcc1094b0 0x7fcdcc19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:51.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.290+0000 7fcdd1422640 1 --2- 172.21.15.64:0/1791591942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdcc109e00 0x7fcdcc19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:51.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.290+0000 7fcdcbfff640 1 -- 172.21.15.64:0/1791591942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdcc109e00 msgr2=0x7fcdcc19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:51.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.290+0000 7fcdcbfff640 1 --2- 172.21.15.64:0/1791591942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdcc109e00 0x7fcdcc19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.290+0000 7fcdcbfff640 1 -- 172.21.15.64:0/1791591942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdcc103bc0 msgr2=0x7fcdcc19a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:51.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.291+0000 7fcdcbfff640 1 --2- 172.21.15.64:0/1791591942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdcc103bc0 0x7fcdcc19a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.291+0000 7fcdcbfff640 1 -- 172.21.15.64:0/1791591942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdcc1a03c0 con 0x7fcdcc1094b0 2024-09-21T04:04:51.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.291+0000 7fcdd1422640 1 --2- 172.21.15.64:0/1791591942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdcc109e00 0x7fcdcc19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:04:51.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.291+0000 7fcdd0c21640 1 --2- 172.21.15.64:0/1791591942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdcc103bc0 0x7fcdcc19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:51.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.291+0000 7fcdcbfff640 1 --2- 172.21.15.64:0/1791591942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdcc1094b0 0x7fcdcc19a9f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc06bb30 tx=0x7fcdb800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:51.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.291+0000 7fcdc9ffb640 1 -- 172.21.15.64:0/1791591942 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb800cc40 con 0x7fcdcc1094b0 2024-09-21T04:04:51.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.291+0000 7fcdd2e85640 1 -- 172.21.15.64:0/1791591942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdcc1a06b0 con 0x7fcdcc1094b0 2024-09-21T04:04:51.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.291+0000 7fcdc9ffb640 1 -- 172.21.15.64:0/1791591942 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdb800cde0 con 0x7fcdcc1094b0 2024-09-21T04:04:51.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.291+0000 7fcdc9ffb640 1 -- 172.21.15.64:0/1791591942 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8018760 con 0x7fcdcc1094b0 2024-09-21T04:04:51.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.291+0000 7fcdd2e85640 1 -- 172.21.15.64:0/1791591942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdcc1a6e10 con 0x7fcdcc1094b0 2024-09-21T04:04:51.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.295+0000 7fcdd93f9740 1 Processor -- start 2024-09-21T04:04:51.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.295+0000 7fcdd93f9740 1 -- start start 2024-09-21T04:04:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.295+0000 7fcdd93f9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564325e31070 0x5643261d86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.295+0000 7fcdd93f9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564325e081b0 0x5643261d8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.295+0000 7fcdd93f9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564325e10390 0x5643261d9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:51.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.295+0000 7fcdd93f9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56432614eea0 con 0x564325e081b0 2024-09-21T04:04:51.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.295+0000 7fcdd93f9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56432613e2d0 con 0x564325e10390 2024-09-21T04:04:51.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.295+0000 7fcdd93f9740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5643261420e0 con 0x564325e31070 2024-09-21T04:04:51.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.296+0000 7fcdd1422640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564325e31070 0x5643261d86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:51.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.296+0000 7fcdd1422640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564325e31070 0x5643261d86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:33716/0 (socket says 172.21.15.64:33716) 2024-09-21T04:04:51.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.296+0000 7fcdd1422640 1 -- 172.21.15.64:0/2167448467 learned_addr learned my addr 172.21.15.64:0/2167448467 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:51.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.296+0000 7fcdd0c21640 1 --2- 172.21.15.64:0/2167448467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564325e081b0 0x5643261d8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:51.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.296+0000 7fcdc9ffb640 1 -- 172.21.15.64:0/1791591942 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 25) ==== 154694+0+0 (secure 0 0 0) 0x7fcdb8018900 con 0x7fcdcc1094b0 2024-09-21T04:04:51.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.296+0000 7fcdcbfff640 1 --2- 172.21.15.64:0/2167448467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564325e10390 0x5643261d9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:51.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.297+0000 7fcdd0c21640 1 -- 172.21.15.64:0/2167448467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564325e31070 msgr2=0x5643261d86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:51.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.297+0000 7fcdd0c21640 1 --2- 172.21.15.64:0/2167448467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564325e31070 0x5643261d86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.297+0000 7fcdd0c21640 1 -- 172.21.15.64:0/2167448467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564325e10390 msgr2=0x5643261d9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:51.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.297+0000 7fcdd0c21640 1 --2- 172.21.15.64:0/2167448467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564325e10390 0x5643261d9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.297+0000 7fcdd0c21640 1 -- 172.21.15.64:0/2167448467 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564325e9c650 con 0x564325e081b0 2024-09-21T04:04:51.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.297+0000 7fcdd0c21640 1 --2- 172.21.15.64:0/2167448467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564325e081b0 0x5643261d8be0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0011560 tx=0x7fcdc0011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd4dcda4c8c6962 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:51.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.298+0000 7fcdc9ffb640 1 --2- 172.21.15.64:0/1791591942 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcdbc0b6710 0x7fcdbc0b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:51.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.298+0000 7fcdd1422640 1 --2- 172.21.15.64:0/1791591942 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcdbc0b6710 0x7fcdbc0b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:51.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.298+0000 7fcdc9ffb640 1 -- 172.21.15.64:0/1791591942 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fcdb80e7cc0 con 0x7fcdcc1094b0 2024-09-21T04:04:51.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.298+0000 7fcdb6ffd640 1 -- 172.21.15.64:0/2167448467 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc001e070 con 0x564325e081b0 2024-09-21T04:04:51.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.298+0000 7fcdb6ffd640 1 -- 172.21.15.64:0/2167448467 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdc000faf0 con 0x564325e081b0 2024-09-21T04:04:51.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.298+0000 7fcdb6ffd640 1 -- 172.21.15.64:0/2167448467 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0019c00 con 0x564325e081b0 2024-09-21T04:04:51.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.300+0000 7fcdd93f9740 1 -- 172.21.15.64:0/2167448467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564325e081b0 msgr2=0x5643261d8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:51.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.300+0000 7fcdd1422640 1 --2- 172.21.15.64:0/1791591942 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcdbc0b6710 0x7fcdbc0b8c20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4004af0 tx=0x7fcdc4005c30 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:51.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.300+0000 7fcdd93f9740 1 --2- 172.21.15.64:0/2167448467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564325e081b0 0x5643261d8be0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0011560 tx=0x7fcdc0011a20 comp rx=0 tx=0).stop 2024-09-21T04:04:51.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.300+0000 7fcdd93f9740 1 -- 172.21.15.64:0/2167448467 shutdown_connections 2024-09-21T04:04:51.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.300+0000 7fcdd93f9740 1 --2- 172.21.15.64:0/2167448467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564325e10390 0x5643261d9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.300+0000 7fcdd93f9740 1 --2- 172.21.15.64:0/2167448467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564325e081b0 0x5643261d8be0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.300+0000 7fcdd93f9740 1 --2- 172.21.15.64:0/2167448467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564325e31070 0x5643261d86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.300+0000 7fcdd93f9740 1 -- 172.21.15.64:0/2167448467 >> 172.21.15.64:0/2167448467 conn(0x564325e25460 msgr2=0x5643261d6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:51.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.300+0000 7fcdd93f9740 1 -- 172.21.15.64:0/2167448467 shutdown_connections 2024-09-21T04:04:51.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.300+0000 7fcdd93f9740 1 -- 172.21.15.64:0/2167448467 wait complete. 2024-09-21T04:04:51.303 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.301+0000 7fcdd93f9740 1 -- start start 2024-09-21T04:04:51.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.301+0000 7fcdd93f9740 1 Processor -- start 2024-09-21T04:04:51.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.302+0000 7fcdd93f9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564325e081b0 0x5643261dfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:51.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.302+0000 7fcdd93f9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564325e10390 0x5643261e0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:51.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.302+0000 7fcdd93f9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564325e31070 0x5643261e5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:51.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.302+0000 7fcdd93f9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564325f9f700 con 0x564325e10390 2024-09-21T04:04:51.304 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.302+0000 7fcdd93f9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5643260d9160 con 0x564325e081b0 2024-09-21T04:04:51.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.302+0000 7fcdd93f9740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5643261e6cc0 con 0x564325e31070 2024-09-21T04:04:51.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.302+0000 7fcdcbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564325e10390 0x5643261e0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:51.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.302+0000 7fcdcbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564325e10390 0x5643261e0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57314/0 (socket says 172.21.15.64:57314) 2024-09-21T04:04:51.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.302+0000 7fcdcbfff640 1 -- 172.21.15.64:0/4147428191 learned_addr learned my addr 172.21.15.64:0/4147428191 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:51.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.302+0000 7fcdd0c21640 1 --2- 172.21.15.64:0/4147428191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564325e081b0 0x5643261dfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.302+0000 7fcdcbfff640 1 -- 172.21.15.64:0/4147428191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564325e31070 msgr2=0x5643261e5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.302+0000 7fcdcbfff640 1 --2- 172.21.15.64:0/4147428191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564325e31070 0x5643261e5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.302+0000 7fcdcbfff640 1 -- 172.21.15.64:0/4147428191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564325e081b0 msgr2=0x5643261dfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.302+0000 7fcdcbfff640 1 --2- 172.21.15.64:0/4147428191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564325e081b0 0x5643261dfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.302+0000 7fcdcbfff640 1 -- 172.21.15.64:0/4147428191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564326106c90 con 0x564325e10390 2024-09-21T04:04:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.302+0000 7fcdd93f9740 10 client.0 _ll_register_callbacks cb 0x5643260f29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.303+0000 7fcdcbfff640 1 --2- 172.21.15.64:0/4147428191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564325e10390 0x5643261e0340 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fcdb800c6a0 tx=0x7fcdb800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b549451fe5902265 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.303+0000 7fcdabfff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:04:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.303+0000 7fcdabfff640 1 -- 172.21.15.64:0/4147428191 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb80ad260 con 0x564325e10390 2024-09-21T04:04:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.303+0000 7fcdabfff640 1 -- 172.21.15.64:0/4147428191 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fcdb80e5ad0 con 0x564325e10390 2024-09-21T04:04:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.303+0000 7fcdd93f9740 1 -- 172.21.15.64:0/4147428191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdc000faf0 con 0x564325e10390 2024-09-21T04:04:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.303+0000 7fcdabfff640 1 -- 172.21.15.64:0/4147428191 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcdb80ad080 con 0x564325e10390 2024-09-21T04:04:51.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.303+0000 7fcdd93f9740 10 client.15705 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.303+0000 7fcdd93f9740 1 -- 172.21.15.64:0/4147428191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5643261f2a80 con 0x564325e10390 2024-09-21T04:04:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.304+0000 7fcdabfff640 1 -- 172.21.15.64:0/4147428191 <== mon.0 v2:172.21.15.64:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcdb80e5cd0 con 0x564325e10390 2024-09-21T04:04:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.304+0000 7fcdd93f9740 10 client.15705 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.304+0000 7fcdd93f9740 10 client.15705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:04:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.304+0000 7fcdd93f9740 20 client.15705 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.304+0000 7fcdd93f9740 1 --2- 172.21.15.64:0/4147428191 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564325def9f0 0x5643261f5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.304+0000 7fcdd93f9740 4 client.15705 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:04:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.304+0000 7fcdd93f9740 1 -- 172.21.15.64:0/4147428191 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcdc000faf0 con 0x564325def9f0 2024-09-21T04:04:51.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.304+0000 7fcdd0c21640 1 --2- 172.21.15.64:0/4147428191 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564325def9f0 0x5643261f5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.305+0000 7fcdabfff640 10 client.15705 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.305+0000 7fcdd0c21640 1 --2- 172.21.15.64:0/4147428191 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564325def9f0 0x5643261f5e90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d738f8485d99fd8 server_cookie=836be2abb22f9a6c in_seq=0 out_seq=0 2024-09-21T04:04:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.306+0000 7fcdabfff640 1 -- 172.21.15.64:0/4147428191 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fcdc000faf0 con 0x564325def9f0 2024-09-21T04:04:51.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.306+0000 7fcdabfff640 10 client.15705 handle_command_reply: tid=0 2024-09-21T04:04:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.306+0000 7fcdd93f9740 1 client.15705 shutdown 2024-09-21T04:04:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.306+0000 7fcdd93f9740 1 -- 172.21.15.64:0/4147428191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564325e10390 msgr2=0x5643261e0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.306+0000 7fcdd93f9740 1 --2- 172.21.15.64:0/4147428191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564325e10390 0x5643261e0340 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fcdb800c6a0 tx=0x7fcdb800cf90 comp rx=0 tx=0).stop 2024-09-21T04:04:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.306+0000 7fcdd93f9740 1 -- 172.21.15.64:0/4147428191 shutdown_connections 2024-09-21T04:04:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.306+0000 7fcdd93f9740 1 --2- 172.21.15.64:0/4147428191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564325e31070 0x5643261e5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.306+0000 7fcdd93f9740 1 --2- 172.21.15.64:0/4147428191 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564325def9f0 0x5643261f5e90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.306+0000 7fcdd93f9740 1 --2- 172.21.15.64:0/4147428191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564325e10390 0x5643261e0340 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.306+0000 7fcdd93f9740 1 --2- 172.21.15.64:0/4147428191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564325e081b0 0x5643261dfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.306+0000 7fcdd93f9740 1 -- 172.21.15.64:0/4147428191 >> 172.21.15.64:0/4147428191 conn(0x564325e25460 msgr2=0x5643261db830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.306+0000 7fcdabfff640 0 client.15705 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.307+0000 7fcdd93f9740 1 -- 172.21.15.64:0/4147428191 shutdown_connections 2024-09-21T04:04:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.307+0000 7fcdd93f9740 1 -- 172.21.15.64:0/4147428191 wait complete. 2024-09-21T04:04:51.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.307+0000 7fcdd93f9740 20 client.15705 trim_cache size 0 max 16384 2024-09-21T04:04:51.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.333+0000 7fcdd93f9740 1 Processor -- start 2024-09-21T04:04:51.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.333+0000 7fcdd93f9740 1 -- start start 2024-09-21T04:04:51.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.334+0000 7fcdd93f9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564325e081b0 0x5643261dfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:51.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.334+0000 7fcdd93f9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564325e10390 0x5643261e02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:51.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.334+0000 7fcdd93f9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564325e31070 0x5643261de680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:51.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.334+0000 7fcdd93f9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc1aff10 con 0x564325e10390 2024-09-21T04:04:51.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.334+0000 7fcdd93f9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc1ad7e0 con 0x564325e31070 2024-09-21T04:04:51.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.334+0000 7fcdd93f9740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc1ab630 con 0x564325e081b0 2024-09-21T04:04:51.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.334+0000 7fcdcbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564325e10390 0x5643261e02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:51.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.334+0000 7fcdcbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564325e10390 0x5643261e02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57322/0 (socket says 172.21.15.64:57322) 2024-09-21T04:04:51.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.334+0000 7fcdcbfff640 1 -- 172.21.15.64:0/1114323941 learned_addr learned my addr 172.21.15.64:0/1114323941 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:51.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.334+0000 7fcdd0c21640 1 --2- 172.21.15.64:0/1114323941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564325e081b0 0x5643261dfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:51.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.335+0000 7fcdcbfff640 1 -- 172.21.15.64:0/1114323941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564325e081b0 msgr2=0x5643261dfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:51.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.335+0000 7fcdcbfff640 1 --2- 172.21.15.64:0/1114323941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564325e081b0 0x5643261dfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.335+0000 7fcdcbfff640 1 -- 172.21.15.64:0/1114323941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564325e31070 msgr2=0x5643261de680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:51.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.335+0000 7fcdcbfff640 1 --2- 172.21.15.64:0/1114323941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564325e31070 0x5643261de680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.335+0000 7fcdcbfff640 1 -- 172.21.15.64:0/1114323941 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643261d7350 con 0x564325e10390 2024-09-21T04:04:51.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.335+0000 7fcdcbfff640 1 --2- 172.21.15.64:0/1114323941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564325e10390 0x5643261e02b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80a9e90 tx=0x7fcdb80d1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca86885b62d9771 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:51.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.335+0000 7fcdb67fc640 1 -- 172.21.15.64:0/1114323941 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb80a9080 con 0x564325e10390 2024-09-21T04:04:51.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.335+0000 7fcdb67fc640 1 -- 172.21.15.64:0/1114323941 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdb80ae010 con 0x564325e10390 2024-09-21T04:04:51.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.335+0000 7fcdb67fc640 1 -- 172.21.15.64:0/1114323941 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb80c3ac0 con 0x564325e10390 2024-09-21T04:04:51.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.336+0000 7fcdd93f9740 1 -- 172.21.15.64:0/1114323941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564325e10390 msgr2=0x5643261e02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:51.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.336+0000 7fcdd93f9740 1 --2- 172.21.15.64:0/1114323941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564325e10390 0x5643261e02b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80a9e90 tx=0x7fcdb80d1040 comp rx=0 tx=0).stop 2024-09-21T04:04:51.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.337+0000 7fcdd93f9740 1 -- 172.21.15.64:0/1114323941 shutdown_connections 2024-09-21T04:04:51.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.337+0000 7fcdd93f9740 1 --2- 172.21.15.64:0/1114323941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564325e31070 0x5643261de680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.337+0000 7fcdd93f9740 1 --2- 172.21.15.64:0/1114323941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564325e10390 0x5643261e02b0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.337+0000 7fcdd93f9740 1 --2- 172.21.15.64:0/1114323941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564325e081b0 0x5643261dfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.337+0000 7fcdd93f9740 1 -- 172.21.15.64:0/1114323941 >> 172.21.15.64:0/1114323941 conn(0x564325e25460 msgr2=0x5643261e94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:51.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.337+0000 7fcdd93f9740 1 -- 172.21.15.64:0/1114323941 shutdown_connections 2024-09-21T04:04:51.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.337+0000 7fcdd93f9740 1 -- 172.21.15.64:0/1114323941 wait complete. 2024-09-21T04:04:51.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.337+0000 7fcdd93f9740 1 -- start start 2024-09-21T04:04:51.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.338+0000 7fcdd93f9740 1 Processor -- start 2024-09-21T04:04:51.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.338+0000 7fcdd93f9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564325e081b0 0x5643261f02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:51.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.338+0000 7fcdd93f9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564325e10390 0x5643261f07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:51.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.338+0000 7fcdcbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564325e10390 0x5643261f07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:51.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.339+0000 7fcdcbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564325e10390 0x5643261f07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:57330/0 (socket says 172.21.15.64:57330) 2024-09-21T04:04:51.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.339+0000 7fcdcbfff640 1 -- 172.21.15.64:0/2533401367 learned_addr learned my addr 172.21.15.64:0/2533401367 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:51.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.339+0000 7fcdd93f9740 1 --2- 172.21.15.64:0/2533401367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564325e31070 0x5643261eb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:51.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.339+0000 7fcdd0c21640 1 --2- 172.21.15.64:0/2533401367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564325e081b0 0x5643261f02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:51.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.339+0000 7fcdd93f9740 1 -- 172.21.15.64:0/2533401367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc1abca0 con 0x564325e10390 2024-09-21T04:04:51.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.339+0000 7fcdd93f9740 1 -- 172.21.15.64:0/2533401367 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc1ad110 con 0x564325e31070 2024-09-21T04:04:51.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.339+0000 7fcdd93f9740 1 -- 172.21.15.64:0/2533401367 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc1ac370 con 0x564325e081b0 2024-09-21T04:04:51.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.339+0000 7fcdd1422640 1 --2- 172.21.15.64:0/2533401367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564325e31070 0x5643261eb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:51.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.339+0000 7fcdcbfff640 1 -- 172.21.15.64:0/2533401367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564325e081b0 msgr2=0x5643261f02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:51.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.339+0000 7fcdcbfff640 1 --2- 172.21.15.64:0/2533401367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564325e081b0 0x5643261f02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.339+0000 7fcdcbfff640 1 -- 172.21.15.64:0/2533401367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564325e31070 msgr2=0x5643261eb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:51.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.339+0000 7fcdcbfff640 1 --2- 172.21.15.64:0/2533401367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564325e31070 0x5643261eb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.339+0000 7fcdcbfff640 1 -- 172.21.15.64:0/2533401367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643261eb640 con 0x564325e10390 2024-09-21T04:04:51.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.339+0000 7fcdd1422640 1 --2- 172.21.15.64:0/2533401367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564325e31070 0x5643261eb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:51.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.340+0000 7fcdd93f9740 10 client.0 _ll_register_callbacks cb 0x7fcdb80e5150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:51.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.340+0000 7fcdd93f9740 1 -- 172.21.15.64:0/2533401367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdb80ae010 con 0x564325e10390 2024-09-21T04:04:51.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.340+0000 7fcdcbfff640 1 --2- 172.21.15.64:0/2533401367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564325e10390 0x5643261f07e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80e9040 tx=0x7fcdb8010e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa494d275baf56c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:51.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.340+0000 7fcdabfff640 10 client.15711 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:04:51.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.340+0000 7fcdabfff640 1 -- 172.21.15.64:0/2533401367 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb80a9080 con 0x564325e10390 2024-09-21T04:04:51.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.340+0000 7fcdabfff640 1 -- 172.21.15.64:0/2533401367 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fcdb80c3cc0 con 0x564325e10390 2024-09-21T04:04:51.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.340+0000 7fcdabfff640 1 -- 172.21.15.64:0/2533401367 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcdb80b7330 con 0x564325e10390 2024-09-21T04:04:51.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.340+0000 7fcdd93f9740 10 client.15711 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:51.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.340+0000 7fcdd93f9740 1 -- 172.21.15.64:0/2533401367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({fsmap=25}) -- 0x564326260e80 con 0x564325e10390 2024-09-21T04:04:51.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.341+0000 7fcdabfff640 1 -- 172.21.15.64:0/2533401367 <== mon.0 v2:172.21.15.64:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcdb80ac5e0 con 0x564325e10390 2024-09-21T04:04:51.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.341+0000 7fcdd93f9740 10 client.15711 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:51.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.341+0000 7fcdd93f9740 10 client.15711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:04:51.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.341+0000 7fcdd93f9740 20 client.15711 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:51.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.341+0000 7fcdd93f9740 1 --2- 172.21.15.64:0/2533401367 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564325dc4da0 0x564326263310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:51.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.341+0000 7fcdd93f9740 4 client.15711 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:04:51.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.341+0000 7fcdd93f9740 1 -- 172.21.15.64:0/2533401367 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564326150df0 con 0x564325dc4da0 2024-09-21T04:04:51.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.341+0000 7fcdd0c21640 1 --2- 172.21.15.64:0/2533401367 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564325dc4da0 0x564326263310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:51.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.342+0000 7fcdabfff640 10 client.15711 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:51.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.342+0000 7fcdd0c21640 1 --2- 172.21.15.64:0/2533401367 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564325dc4da0 0x564326263310 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9d17459c519b3b server_cookie=f1273a09480d6cbf in_seq=0 out_seq=0 2024-09-21T04:04:51.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.342+0000 7fcdabfff640 1 -- 172.21.15.64:0/2533401367 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564326150df0 con 0x564325dc4da0 2024-09-21T04:04:51.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.342+0000 7fcdabfff640 10 client.15711 handle_command_reply: tid=0 2024-09-21T04:04:51.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.342+0000 7fcdd93f9740 1 client.15711 shutdown 2024-09-21T04:04:51.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.342+0000 7fcdd93f9740 1 -- 172.21.15.64:0/2533401367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564325e10390 msgr2=0x5643261f07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:51.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.342+0000 7fcdd93f9740 1 --2- 172.21.15.64:0/2533401367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564325e10390 0x5643261f07e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80e9040 tx=0x7fcdb8010e30 comp rx=0 tx=0).stop 2024-09-21T04:04:51.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.343+0000 7fcdd93f9740 1 -- 172.21.15.64:0/2533401367 shutdown_connections 2024-09-21T04:04:51.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.343+0000 7fcdd93f9740 1 --2- 172.21.15.64:0/2533401367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564325e31070 0x5643261eb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.343+0000 7fcdd93f9740 1 --2- 172.21.15.64:0/2533401367 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564325dc4da0 0x564326263310 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.343+0000 7fcdd93f9740 1 --2- 172.21.15.64:0/2533401367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564325e10390 0x5643261f07e0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.343+0000 7fcdd93f9740 1 --2- 172.21.15.64:0/2533401367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564325e081b0 0x5643261f02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.343+0000 7fcdd93f9740 1 -- 172.21.15.64:0/2533401367 >> 172.21.15.64:0/2533401367 conn(0x564325e25460 msgr2=0x5643261d5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:51.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.343+0000 7fcdabfff640 0 client.15711 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:51.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.343+0000 7fcdd93f9740 1 -- 172.21.15.64:0/2533401367 shutdown_connections 2024-09-21T04:04:51.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.343+0000 7fcdd93f9740 1 -- 172.21.15.64:0/2533401367 wait complete. 2024-09-21T04:04:51.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.343+0000 7fcdd93f9740 20 client.15711 trim_cache size 0 max 16384 2024-09-21T04:04:51.348 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:04:51.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.344+0000 7fcdd2e85640 1 -- 172.21.15.64:0/1791591942 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcdbc0b6710 msgr2=0x7fcdbc0b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:51.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.344+0000 7fcdd2e85640 1 --2- 172.21.15.64:0/1791591942 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcdbc0b6710 0x7fcdbc0b8c20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4004af0 tx=0x7fcdc4005c30 comp rx=0 tx=0).stop 2024-09-21T04:04:51.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.344+0000 7fcdd2e85640 1 -- 172.21.15.64:0/1791591942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdcc1094b0 msgr2=0x7fcdcc19a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:51.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.344+0000 7fcdd2e85640 1 --2- 172.21.15.64:0/1791591942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdcc1094b0 0x7fcdcc19a9f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc06bb30 tx=0x7fcdb800ee50 comp rx=0 tx=0).stop 2024-09-21T04:04:51.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.344+0000 7fcdd2e85640 1 -- 172.21.15.64:0/1791591942 shutdown_connections 2024-09-21T04:04:51.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.344+0000 7fcdd2e85640 1 --2- 172.21.15.64:0/1791591942 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcdbc0b6710 0x7fcdbc0b8c20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.344+0000 7fcdd2e85640 1 --2- 172.21.15.64:0/1791591942 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcdcc109e00 0x7fcdcc19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.344+0000 7fcdd2e85640 1 --2- 172.21.15.64:0/1791591942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdcc1094b0 0x7fcdcc19a9f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.344+0000 7fcdd2e85640 1 --2- 172.21.15.64:0/1791591942 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdcc103bc0 0x7fcdcc19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:51.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.344+0000 7fcdd2e85640 1 -- 172.21.15.64:0/1791591942 >> 172.21.15.64:0/1791591942 conn(0x7fcdcc07b910 msgr2=0x7fcdcc105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:51.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.345+0000 7fcdd2e85640 1 -- 172.21.15.64:0/1791591942 shutdown_connections 2024-09-21T04:04:51.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:51.345+0000 7fcdd2e85640 1 -- 172.21.15.64:0/1791591942 wait complete. 2024-09-21T04:04:51.961 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:51 smithi064 ceph-mon[31180]: pgmap v346: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-21T04:04:52.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:51 smithi119 ceph-mon[34811]: pgmap v346: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-21T04:04:52.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:51 smithi131 ceph-mon[35193]: pgmap v346: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-21T04:04:53.001 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T04:04:53.002 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T04:04:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.256+0000 7fa94d6b4640 1 -- 172.21.15.64:0/3932920184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa948108400 msgr2=0x7fa94810ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.256+0000 7fa94d6b4640 1 --2- 172.21.15.64:0/3932920184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa948108400 0x7fa94810ec30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa940009be0 tx=0x7fa94002c9f0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.257+0000 7fa94d6b4640 1 -- 172.21.15.64:0/3932920184 shutdown_connections 2024-09-21T04:04:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.257+0000 7fa94d6b4640 1 --2- 172.21.15.64:0/3932920184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa948108400 0x7fa94810ec30 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.257+0000 7fa94d6b4640 1 --2- 172.21.15.64:0/3932920184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa948107ab0 0x7fa948107ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.257+0000 7fa94d6b4640 1 --2- 172.21.15.64:0/3932920184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa948102180 0x7fa948102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.257+0000 7fa94d6b4640 1 -- 172.21.15.64:0/3932920184 >> 172.21.15.64:0/3932920184 conn(0x7fa948079e90 msgr2=0x7fa94807a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.257+0000 7fa94d6b4640 1 -- 172.21.15.64:0/3932920184 shutdown_connections 2024-09-21T04:04:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.257+0000 7fa94d6b4640 1 -- 172.21.15.64:0/3932920184 wait complete. 2024-09-21T04:04:53.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.258+0000 7fa94d6b4640 1 Processor -- start 2024-09-21T04:04:53.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.258+0000 7fa94d6b4640 1 -- start start 2024-09-21T04:04:53.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.258+0000 7fa94d6b4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa948102180 0x7fa9481981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.258+0000 7fa94d6b4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa948107ab0 0x7fa948198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.258+0000 7fa94d6b4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa948108400 0x7fa94819da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.259+0000 7fa94d6b4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa948111840 con 0x7fa948107ab0 2024-09-21T04:04:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.259+0000 7fa94d6b4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa9481116c0 con 0x7fa948102180 2024-09-21T04:04:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.259+0000 7fa94d6b4640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa9481119c0 con 0x7fa948108400 2024-09-21T04:04:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.259+0000 7fa946ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa948102180 0x7fa9481981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.259+0000 7fa946ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa948102180 0x7fa9481981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:47484/0 (socket says 172.21.15.64:47484) 2024-09-21T04:04:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.259+0000 7fa946ffd640 1 -- 172.21.15.64:0/2426876331 learned_addr learned my addr 172.21.15.64:0/2426876331 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.259+0000 7fa9477fe640 1 --2- 172.21.15.64:0/2426876331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa948108400 0x7fa94819da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.259+0000 7fa9467fc640 1 --2- 172.21.15.64:0/2426876331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa948107ab0 0x7fa948198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.259+0000 7fa946ffd640 1 -- 172.21.15.64:0/2426876331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa948108400 msgr2=0x7fa94819da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.259+0000 7fa946ffd640 1 --2- 172.21.15.64:0/2426876331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa948108400 0x7fa94819da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.259+0000 7fa946ffd640 1 -- 172.21.15.64:0/2426876331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa948107ab0 msgr2=0x7fa948198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.259+0000 7fa946ffd640 1 --2- 172.21.15.64:0/2426876331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa948107ab0 0x7fa948198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.259+0000 7fa946ffd640 1 -- 172.21.15.64:0/2426876331 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94819e110 con 0x7fa948102180 2024-09-21T04:04:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.260+0000 7fa946ffd640 1 --2- 172.21.15.64:0/2426876331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa948102180 0x7fa9481981e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00c890 tx=0x7fa93c00cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.260+0000 7fa92ffff640 1 -- 172.21.15.64:0/2426876331 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c004220 con 0x7fa948102180 2024-09-21T04:04:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.260+0000 7fa92ffff640 1 -- 172.21.15.64:0/2426876331 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa93c0043c0 con 0x7fa948102180 2024-09-21T04:04:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.260+0000 7fa94d6b4640 1 -- 172.21.15.64:0/2426876331 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94819e400 con 0x7fa948102180 2024-09-21T04:04:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.260+0000 7fa92ffff640 1 -- 172.21.15.64:0/2426876331 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c004a00 con 0x7fa948102180 2024-09-21T04:04:53.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.260+0000 7fa94d6b4640 1 -- 172.21.15.64:0/2426876331 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9481a4b90 con 0x7fa948102180 2024-09-21T04:04:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.264+0000 7fa954626740 1 Processor -- start 2024-09-21T04:04:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.264+0000 7fa954626740 1 -- start start 2024-09-21T04:04:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.264+0000 7fa92ffff640 1 -- 172.21.15.64:0/2426876331 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 25) ==== 154694+0+0 (secure 0 0 0) 0x7fa93c004ba0 con 0x7fa948102180 2024-09-21T04:04:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.264+0000 7fa954626740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56372aa541b0 0x56372adfddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.264+0000 7fa954626740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372aa2b2f0 0x56372adfe300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.264+0000 7fa954626740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56372aa334d0 0x56372adfe840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.264+0000 7fa954626740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56372ad6e6f0 con 0x56372aa334d0 2024-09-21T04:04:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.264+0000 7fa954626740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56372ad5db20 con 0x56372aa2b2f0 2024-09-21T04:04:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.264+0000 7fa954626740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56372ad61930 con 0x56372aa541b0 2024-09-21T04:04:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.265+0000 7fa946ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372aa2b2f0 0x56372adfe300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.265+0000 7fa9477fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56372aa541b0 0x56372adfddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.265+0000 7fa9467fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56372aa334d0 0x56372adfe840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.265+0000 7fa946ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372aa2b2f0 0x56372adfe300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:47486/0 (socket says 172.21.15.64:47486) 2024-09-21T04:04:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.265+0000 7fa946ffd640 1 -- 172.21.15.64:0/1767436391 learned_addr learned my addr 172.21.15.64:0/1767436391 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.265+0000 7fa9477fe640 1 -- 172.21.15.64:0/1767436391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372aa2b2f0 msgr2=0x56372adfe300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.265+0000 7fa9477fe640 1 --2- 172.21.15.64:0/1767436391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372aa2b2f0 0x56372adfe300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.265+0000 7fa9477fe640 1 -- 172.21.15.64:0/1767436391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56372aa334d0 msgr2=0x56372adfe840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.265+0000 7fa9477fe640 1 --2- 172.21.15.64:0/1767436391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56372aa334d0 0x56372adfe840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.265+0000 7fa9477fe640 1 -- 172.21.15.64:0/1767436391 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56372ad23530 con 0x56372aa541b0 2024-09-21T04:04:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.266+0000 7fa9467fc640 1 --2- 172.21.15.64:0/1767436391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56372aa334d0 0x56372adfe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.266+0000 7fa92ffff640 1 --2- 172.21.15.64:0/2426876331 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa9300b6710 0x7fa9300b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.266+0000 7fa9477fe640 1 --2- 172.21.15.64:0/1767436391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56372aa541b0 0x56372adfddc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa940006010 tx=0x7fa94000f4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ccadcdbe1a92d40 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.266+0000 7fa946ffd640 1 --2- 172.21.15.64:0/2426876331 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa9300b6710 0x7fa9300b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.271 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.266+0000 7fa92ffff640 1 -- 172.21.15.64:0/2426876331 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fa93c0e7640 con 0x7fa948102180 2024-09-21T04:04:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.266+0000 7fa92d7fa640 1 -- 172.21.15.64:0/1767436391 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940049070 con 0x56372aa541b0 2024-09-21T04:04:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.266+0000 7fa92d7fa640 1 -- 172.21.15.64:0/1767436391 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa940003e20 con 0x56372aa541b0 2024-09-21T04:04:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.266+0000 7fa92d7fa640 1 -- 172.21.15.64:0/1767436391 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940004280 con 0x56372aa541b0 2024-09-21T04:04:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.267+0000 7fa954626740 1 -- 172.21.15.64:0/1767436391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56372aa541b0 msgr2=0x56372adfddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.267+0000 7fa946ffd640 1 --2- 172.21.15.64:0/2426876331 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa9300b6710 0x7fa9300b8c20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa93c005fd0 tx=0x7fa93c0ed480 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.267+0000 7fa954626740 1 --2- 172.21.15.64:0/1767436391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56372aa541b0 0x56372adfddc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa940006010 tx=0x7fa94000f4f0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.267+0000 7fa954626740 1 -- 172.21.15.64:0/1767436391 shutdown_connections 2024-09-21T04:04:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.267+0000 7fa954626740 1 --2- 172.21.15.64:0/1767436391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56372aa334d0 0x56372adfe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.267+0000 7fa954626740 1 --2- 172.21.15.64:0/1767436391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372aa2b2f0 0x56372adfe300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.267+0000 7fa954626740 1 --2- 172.21.15.64:0/1767436391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56372aa541b0 0x56372adfddc0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.267+0000 7fa954626740 1 -- 172.21.15.64:0/1767436391 >> 172.21.15.64:0/1767436391 conn(0x56372aa48300 msgr2=0x56372adfbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.267+0000 7fa954626740 1 -- 172.21.15.64:0/1767436391 shutdown_connections 2024-09-21T04:04:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.267+0000 7fa954626740 1 -- 172.21.15.64:0/1767436391 wait complete. 2024-09-21T04:04:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.268+0000 7fa954626740 1 -- start start 2024-09-21T04:04:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.268+0000 7fa954626740 1 Processor -- start 2024-09-21T04:04:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.268+0000 7fa954626740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56372aa2b2f0 0x56372adf8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.268+0000 7fa954626740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56372aa334d0 0x56372adf8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.268+0000 7fa954626740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372aa541b0 0x56372ae19bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.268+0000 7fa954626740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56372ac95ab0 con 0x56372aa2b2f0 2024-09-21T04:04:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.268+0000 7fa954626740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56372ad03730 con 0x56372aa541b0 2024-09-21T04:04:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.268+0000 7fa954626740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56372ae10920 con 0x56372aa334d0 2024-09-21T04:04:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.268+0000 7fa954626740 10 client.0 _ll_register_callbacks cb 0x56372ad576b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.268+0000 7fa9477fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56372aa2b2f0 0x56372adf8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.268+0000 7fa9477fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56372aa2b2f0 0x56372adf8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34382/0 (socket says 172.21.15.64:34382) 2024-09-21T04:04:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.268+0000 7fa9477fe640 1 -- 172.21.15.64:0/2725736734 learned_addr learned my addr 172.21.15.64:0/2725736734 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.269+0000 7fa9467fc640 1 --2- 172.21.15.64:0/2725736734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56372aa334d0 0x56372adf8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.269+0000 7fa9477fe640 1 -- 172.21.15.64:0/2725736734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56372aa334d0 msgr2=0x56372adf8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.269+0000 7fa9477fe640 1 --2- 172.21.15.64:0/2725736734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56372aa334d0 0x56372adf8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.269+0000 7fa9477fe640 1 -- 172.21.15.64:0/2725736734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372aa541b0 msgr2=0x56372ae19bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.269+0000 7fa9477fe640 1 --2- 172.21.15.64:0/2725736734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372aa541b0 0x56372ae19bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.269+0000 7fa9477fe640 1 -- 172.21.15.64:0/2725736734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56372aabf790 con 0x56372aa2b2f0 2024-09-21T04:04:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.269+0000 7fa9477fe640 1 --2- 172.21.15.64:0/2725736734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56372aa2b2f0 0x56372adf8410 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa940009370 tx=0x7fa940005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c00fc55b96a683e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.269+0000 7fa92aff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:04:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.269+0000 7fa92aff5640 1 -- 172.21.15.64:0/2725736734 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940049040 con 0x56372aa2b2f0 2024-09-21T04:04:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.270+0000 7fa92aff5640 1 -- 172.21.15.64:0/2725736734 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fa940030b30 con 0x56372aa2b2f0 2024-09-21T04:04:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.270+0000 7fa954626740 1 -- 172.21.15.64:0/2725736734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa940003e20 con 0x56372aa2b2f0 2024-09-21T04:04:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.270+0000 7fa92aff5640 1 -- 172.21.15.64:0/2725736734 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa94005c050 con 0x56372aa2b2f0 2024-09-21T04:04:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.270+0000 7fa954626740 10 client.15723 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.270+0000 7fa954626740 1 -- 172.21.15.64:0/2725736734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56372ab38620 con 0x56372aa2b2f0 2024-09-21T04:04:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.270+0000 7fa92aff5640 1 -- 172.21.15.64:0/2725736734 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa940003e20 con 0x56372aa2b2f0 2024-09-21T04:04:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.270+0000 7fa92aff5640 1 -- 172.21.15.64:0/2725736734 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa940057030 con 0x56372aa2b2f0 2024-09-21T04:04:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.270+0000 7fa954626740 10 client.15723 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.270+0000 7fa954626740 10 client.15723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:04:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.270+0000 7fa954626740 20 client.15723 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.270+0000 7fa954626740 1 --2- 172.21.15.64:0/2725736734 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56372aa12d80 0x56372ae1f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.270+0000 7fa954626740 4 client.15723 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:04:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.270+0000 7fa954626740 1 -- 172.21.15.64:0/2725736734 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56372ad70640 con 0x56372aa12d80 2024-09-21T04:04:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.271+0000 7fa9467fc640 1 --2- 172.21.15.64:0/2725736734 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56372aa12d80 0x56372ae1f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.271+0000 7fa9467fc640 1 --2- 172.21.15.64:0/2725736734 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56372aa12d80 0x56372ae1f780 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ece8b7ac1145e0dd server_cookie=1c32c1fd1ff12a5b in_seq=0 out_seq=0 2024-09-21T04:04:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.271+0000 7fa92aff5640 10 client.15723 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.272+0000 7fa92aff5640 1 -- 172.21.15.64:0/2725736734 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56372ad70640 con 0x56372aa12d80 2024-09-21T04:04:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.272+0000 7fa92aff5640 10 client.15723 handle_command_reply: tid=0 2024-09-21T04:04:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.272+0000 7fa954626740 1 client.15723 shutdown 2024-09-21T04:04:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.272+0000 7fa954626740 1 -- 172.21.15.64:0/2725736734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56372aa2b2f0 msgr2=0x56372adf8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.272+0000 7fa954626740 1 --2- 172.21.15.64:0/2725736734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56372aa2b2f0 0x56372adf8410 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa940009370 tx=0x7fa940005cf0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.273+0000 7fa954626740 1 -- 172.21.15.64:0/2725736734 shutdown_connections 2024-09-21T04:04:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.273+0000 7fa954626740 1 --2- 172.21.15.64:0/2725736734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372aa541b0 0x56372ae19bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.273+0000 7fa954626740 1 --2- 172.21.15.64:0/2725736734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56372aa334d0 0x56372adf8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.273+0000 7fa954626740 1 --2- 172.21.15.64:0/2725736734 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56372aa12d80 0x56372ae1f780 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.273+0000 7fa954626740 1 --2- 172.21.15.64:0/2725736734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56372aa2b2f0 0x56372adf8410 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.273+0000 7fa954626740 1 -- 172.21.15.64:0/2725736734 >> 172.21.15.64:0/2725736734 conn(0x56372aa48300 msgr2=0x56372ae00f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.273+0000 7fa92aff5640 0 client.15723 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.273+0000 7fa954626740 1 -- 172.21.15.64:0/2725736734 shutdown_connections 2024-09-21T04:04:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.273+0000 7fa954626740 1 -- 172.21.15.64:0/2725736734 wait complete. 2024-09-21T04:04:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.273+0000 7fa954626740 20 client.15723 trim_cache size 0 max 16384 2024-09-21T04:04:53.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.300+0000 7fa954626740 1 Processor -- start 2024-09-21T04:04:53.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.300+0000 7fa954626740 1 -- start start 2024-09-21T04:04:53.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.301+0000 7fa954626740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372aa2b2f0 0x56372adfbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.301+0000 7fa954626740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56372aa334d0 0x56372ae19b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.301+0000 7fa954626740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56372aa541b0 0x56372ae10060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.301+0000 7fa954626740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa9481afee0 con 0x56372aa541b0 2024-09-21T04:04:53.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.301+0000 7fa954626740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa9481ad7b0 con 0x56372aa2b2f0 2024-09-21T04:04:53.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.301+0000 7fa954626740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa9481ab600 con 0x56372aa334d0 2024-09-21T04:04:53.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.301+0000 7fa9477fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372aa2b2f0 0x56372adfbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.301+0000 7fa9477fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56372aa541b0 0x56372ae10060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.301+0000 7fa9477fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372aa2b2f0 0x56372adfbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:47506/0 (socket says 172.21.15.64:47506) 2024-09-21T04:04:53.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.301+0000 7fa9477fe640 1 -- 172.21.15.64:0/1720669212 learned_addr learned my addr 172.21.15.64:0/1720669212 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:53.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.301+0000 7fa9467fc640 1 --2- 172.21.15.64:0/1720669212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56372aa334d0 0x56372ae19b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.302+0000 7fa9477fe640 1 -- 172.21.15.64:0/1720669212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56372aa334d0 msgr2=0x56372ae19b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.302+0000 7fa9477fe640 1 --2- 172.21.15.64:0/1720669212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56372aa334d0 0x56372ae19b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.302+0000 7fa9477fe640 1 -- 172.21.15.64:0/1720669212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56372aa541b0 msgr2=0x56372ae10060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.302+0000 7fa9477fe640 1 --2- 172.21.15.64:0/1720669212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56372aa541b0 0x56372ae10060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.302+0000 7fa9477fe640 1 -- 172.21.15.64:0/1720669212 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56372adfafd0 con 0x56372aa2b2f0 2024-09-21T04:04:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.302+0000 7fa9467fc640 1 --2- 172.21.15.64:0/1720669212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56372aa334d0 0x56372ae19b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.302+0000 7fa9477fe640 1 --2- 172.21.15.64:0/1720669212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372aa2b2f0 0x56372adfbcc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa940002230 tx=0x7fa94002c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19577a7c16f7377d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.302+0000 7fa92cff9640 1 -- 172.21.15.64:0/1720669212 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940067070 con 0x56372aa2b2f0 2024-09-21T04:04:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.302+0000 7fa92cff9640 1 -- 172.21.15.64:0/1720669212 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa94002ed60 con 0x56372aa2b2f0 2024-09-21T04:04:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.302+0000 7fa92cff9640 1 -- 172.21.15.64:0/1720669212 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940049070 con 0x56372aa2b2f0 2024-09-21T04:04:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.303+0000 7fa954626740 1 -- 172.21.15.64:0/1720669212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372aa2b2f0 msgr2=0x56372adfbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.303+0000 7fa954626740 1 --2- 172.21.15.64:0/1720669212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372aa2b2f0 0x56372adfbcc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa940002230 tx=0x7fa94002c9f0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.303+0000 7fa954626740 1 -- 172.21.15.64:0/1720669212 shutdown_connections 2024-09-21T04:04:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.303+0000 7fa954626740 1 --2- 172.21.15.64:0/1720669212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56372aa541b0 0x56372ae10060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.303+0000 7fa954626740 1 --2- 172.21.15.64:0/1720669212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56372aa334d0 0x56372ae19b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.303+0000 7fa954626740 1 --2- 172.21.15.64:0/1720669212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372aa2b2f0 0x56372adfbcc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.303+0000 7fa954626740 1 -- 172.21.15.64:0/1720669212 >> 172.21.15.64:0/1720669212 conn(0x56372aa48300 msgr2=0x56372adfb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.303+0000 7fa954626740 1 -- 172.21.15.64:0/1720669212 shutdown_connections 2024-09-21T04:04:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.303+0000 7fa954626740 1 -- 172.21.15.64:0/1720669212 wait complete. 2024-09-21T04:04:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.304+0000 7fa954626740 1 -- start start 2024-09-21T04:04:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.304+0000 7fa954626740 1 Processor -- start 2024-09-21T04:04:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.304+0000 7fa954626740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56372aa2b2f0 0x56372ae220b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.304+0000 7fa954626740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372aa334d0 0x56372ae22650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.304+0000 7fa954626740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56372aa541b0 0x56372ae81ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.304+0000 7fa954626740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa9481abc70 con 0x56372aa2b2f0 2024-09-21T04:04:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.304+0000 7fa954626740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa9481ad0e0 con 0x56372aa334d0 2024-09-21T04:04:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.304+0000 7fa954626740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa9481ac340 con 0x56372aa541b0 2024-09-21T04:04:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.304+0000 7fa954626740 10 client.0 _ll_register_callbacks cb 0x7fa940047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.305+0000 7fa9477fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56372aa2b2f0 0x56372ae220b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.305+0000 7fa9467fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372aa334d0 0x56372ae22650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.305+0000 7fa9477fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56372aa2b2f0 0x56372ae220b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34398/0 (socket says 172.21.15.64:34398) 2024-09-21T04:04:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.305+0000 7fa9477fe640 1 -- 172.21.15.64:0/27220204 learned_addr learned my addr 172.21.15.64:0/27220204 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.305+0000 7fa9477fe640 1 --2- 172.21.15.64:0/27220204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56372aa541b0 0x56372ae81ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.305+0000 7fa9467fc640 1 -- 172.21.15.64:0/27220204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56372aa541b0 msgr2=0x56372ae81ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.305+0000 7fa9467fc640 1 --2- 172.21.15.64:0/27220204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56372aa541b0 0x56372ae81ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.305+0000 7fa9467fc640 1 -- 172.21.15.64:0/27220204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56372aa2b2f0 msgr2=0x56372ae220b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.305+0000 7fa9467fc640 1 --2- 172.21.15.64:0/27220204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56372aa2b2f0 0x56372ae220b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.305+0000 7fa9467fc640 1 -- 172.21.15.64:0/27220204 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56372ae82510 con 0x56372aa334d0 2024-09-21T04:04:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.305+0000 7fa9477fe640 1 --2- 172.21.15.64:0/27220204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56372aa541b0 0x56372ae81ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:04:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.306+0000 7fa9467fc640 1 --2- 172.21.15.64:0/27220204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372aa334d0 0x56372ae22650 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x56372ae109d0 tx=0x7fa93800ddc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4da5e1c854b9708 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.306+0000 7fa92aff5640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:04:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.306+0000 7fa92aff5640 1 -- 172.21.15.64:0/27220204 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93801d410 con 0x56372aa334d0 2024-09-21T04:04:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.306+0000 7fa92aff5640 1 -- 172.21.15.64:0/27220204 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fa93801d5b0 con 0x56372aa334d0 2024-09-21T04:04:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.306+0000 7fa954626740 1 -- 172.21.15.64:0/27220204 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa94002ed60 con 0x56372aa334d0 2024-09-21T04:04:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.306+0000 7fa92aff5640 1 -- 172.21.15.64:0/27220204 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa93802e050 con 0x56372aa334d0 2024-09-21T04:04:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.306+0000 7fa954626740 10 client.25454 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.306+0000 7fa954626740 1 -- 172.21.15.64:0/27220204 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56372ae898b0 con 0x56372aa334d0 2024-09-21T04:04:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.306+0000 7fa92aff5640 1 -- 172.21.15.64:0/27220204 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa94002ed60 con 0x56372aa334d0 2024-09-21T04:04:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.307+0000 7fa92aff5640 1 -- 172.21.15.64:0/27220204 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa938027910 con 0x56372aa334d0 2024-09-21T04:04:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.307+0000 7fa954626740 10 client.25454 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.307+0000 7fa954626740 10 client.25454 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:04:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.307+0000 7fa954626740 20 client.25454 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.307+0000 7fa954626740 1 --2- 172.21.15.64:0/27220204 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56372a9e81c0 0x56372ae8c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.307+0000 7fa954626740 4 client.25454 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:04:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.307+0000 7fa954626740 1 -- 172.21.15.64:0/27220204 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56372ad6c0f0 con 0x56372a9e81c0 2024-09-21T04:04:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.307+0000 7fa9477fe640 1 --2- 172.21.15.64:0/27220204 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56372a9e81c0 0x56372ae8c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.308+0000 7fa92aff5640 10 client.25454 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.308+0000 7fa9477fe640 1 --2- 172.21.15.64:0/27220204 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56372a9e81c0 0x56372ae8c0a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac2b963c127cdf23 server_cookie=833da06d78eaba51 in_seq=0 out_seq=0 2024-09-21T04:04:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.314+0000 7fa92aff5640 1 -- 172.21.15.64:0/27220204 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56372ad6c0f0 con 0x56372a9e81c0 2024-09-21T04:04:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.314+0000 7fa92aff5640 10 client.25454 handle_command_reply: tid=0 2024-09-21T04:04:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.314+0000 7fa954626740 1 client.25454 shutdown 2024-09-21T04:04:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.314+0000 7fa954626740 1 -- 172.21.15.64:0/27220204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372aa334d0 msgr2=0x56372ae22650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.315+0000 7fa954626740 1 --2- 172.21.15.64:0/27220204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372aa334d0 0x56372ae22650 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x56372ae109d0 tx=0x7fa93800ddc0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.315+0000 7fa954626740 1 -- 172.21.15.64:0/27220204 shutdown_connections 2024-09-21T04:04:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.315+0000 7fa954626740 1 --2- 172.21.15.64:0/27220204 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56372aa541b0 0x56372ae81ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.315+0000 7fa954626740 1 --2- 172.21.15.64:0/27220204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56372aa334d0 0x56372ae22650 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.315+0000 7fa954626740 1 --2- 172.21.15.64:0/27220204 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56372a9e81c0 0x56372ae8c0a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.315+0000 7fa954626740 1 --2- 172.21.15.64:0/27220204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56372aa2b2f0 0x56372ae220b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.315+0000 7fa954626740 1 -- 172.21.15.64:0/27220204 >> 172.21.15.64:0/27220204 conn(0x56372aa48300 msgr2=0x56372adfedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.315+0000 7fa92aff5640 0 client.25454 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.315+0000 7fa954626740 1 -- 172.21.15.64:0/27220204 shutdown_connections 2024-09-21T04:04:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.315+0000 7fa954626740 1 -- 172.21.15.64:0/27220204 wait complete. 2024-09-21T04:04:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.315+0000 7fa954626740 20 client.25454 trim_cache size 0 max 16384 2024-09-21T04:04:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:04:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:04:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:04:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:04:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.316+0000 7fa94d6b4640 1 -- 172.21.15.64:0/2426876331 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa9300b6710 msgr2=0x7fa9300b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.316+0000 7fa94d6b4640 1 --2- 172.21.15.64:0/2426876331 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa9300b6710 0x7fa9300b8c20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa93c005fd0 tx=0x7fa93c0ed480 comp rx=0 tx=0).stop 2024-09-21T04:04:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.316+0000 7fa94d6b4640 1 -- 172.21.15.64:0/2426876331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa948102180 msgr2=0x7fa9481981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.316+0000 7fa94d6b4640 1 --2- 172.21.15.64:0/2426876331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa948102180 0x7fa9481981e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00c890 tx=0x7fa93c00cd50 comp rx=0 tx=0).stop 2024-09-21T04:04:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.316+0000 7fa94d6b4640 1 -- 172.21.15.64:0/2426876331 shutdown_connections 2024-09-21T04:04:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.316+0000 7fa94d6b4640 1 --2- 172.21.15.64:0/2426876331 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa9300b6710 0x7fa9300b8c20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.316+0000 7fa94d6b4640 1 --2- 172.21.15.64:0/2426876331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa948108400 0x7fa94819da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.316+0000 7fa94d6b4640 1 --2- 172.21.15.64:0/2426876331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa948107ab0 0x7fa948198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.317+0000 7fa94d6b4640 1 --2- 172.21.15.64:0/2426876331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa948102180 0x7fa9481981e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.317+0000 7fa94d6b4640 1 -- 172.21.15.64:0/2426876331 >> 172.21.15.64:0/2426876331 conn(0x7fa948079e90 msgr2=0x7fa948103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.317+0000 7fa94d6b4640 1 -- 172.21.15.64:0/2426876331 shutdown_connections 2024-09-21T04:04:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:04:53.317+0000 7fa94d6b4640 1 -- 172.21.15.64:0/2426876331 wait complete. 2024-09-21T04:04:53.347 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:04:53.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:53 smithi064 ceph-mon[31180]: pgmap v347: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-21T04:04:53.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.936+0000 7f45d6e88640 1 -- 172.21.15.64:0/1362159331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d0102230 msgr2=0x7f45d0102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:53.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.936+0000 7f45d6e88640 1 --2- 172.21.15.64:0/1362159331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d0102230 0x7f45d0102640 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f45c400aa10 tx=0x7f45c402b750 comp rx=0 tx=0).stop 2024-09-21T04:04:53.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.936+0000 7f45d6e88640 1 -- 172.21.15.64:0/1362159331 shutdown_connections 2024-09-21T04:04:53.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.936+0000 7f45d6e88640 1 --2- 172.21.15.64:0/1362159331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45d01084c0 0x7f45d010ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.936+0000 7f45d6e88640 1 --2- 172.21.15.64:0/1362159331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d0107b70 0x7f45d0107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.936+0000 7f45d6e88640 1 --2- 172.21.15.64:0/1362159331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d0102230 0x7f45d0102640 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.937+0000 7f45d6e88640 1 -- 172.21.15.64:0/1362159331 >> 172.21.15.64:0/1362159331 conn(0x7f45d0079e10 msgr2=0x7f45d007a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:53.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.937+0000 7f45d6e88640 1 -- 172.21.15.64:0/1362159331 shutdown_connections 2024-09-21T04:04:53.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.937+0000 7f45d6e88640 1 -- 172.21.15.64:0/1362159331 wait complete. 2024-09-21T04:04:53.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.939+0000 7f45d6e88640 1 Processor -- start 2024-09-21T04:04:53.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.939+0000 7f45d6e88640 1 -- start start 2024-09-21T04:04:53.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.940+0000 7f45d6e88640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45d0102230 0x7f45d0198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.940+0000 7f45d6e88640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d0107b70 0x7f45d01989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.940+0000 7f45d6e88640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d01084c0 0x7f45d019dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.940+0000 7f45d6e88640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f45d0111940 con 0x7f45d0107b70 2024-09-21T04:04:53.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.940+0000 7f45d6e88640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f45d01117c0 con 0x7f45d0102230 2024-09-21T04:04:53.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.940+0000 7f45d6e88640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f45d0111ac0 con 0x7f45d01084c0 2024-09-21T04:04:53.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.940+0000 7f45d4c24640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45d0102230 0x7f45d0198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.940+0000 7f45d4c24640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45d0102230 0x7f45d0198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:47530/0 (socket says 172.21.15.64:47530) 2024-09-21T04:04:53.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.940+0000 7f45d4c24640 1 -- 172.21.15.64:0/3492976918 learned_addr learned my addr 172.21.15.64:0/3492976918 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:53.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.940+0000 7f45cffff640 1 --2- 172.21.15.64:0/3492976918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d0107b70 0x7f45d01989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.940+0000 7f45d5425640 1 --2- 172.21.15.64:0/3492976918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d01084c0 0x7f45d019dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.941+0000 7f45d4c24640 1 -- 172.21.15.64:0/3492976918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d01084c0 msgr2=0x7f45d019dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:53.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.941+0000 7f45d4c24640 1 --2- 172.21.15.64:0/3492976918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d01084c0 0x7f45d019dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.941+0000 7f45d4c24640 1 -- 172.21.15.64:0/3492976918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d0107b70 msgr2=0x7f45d01989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:53.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.941+0000 7f45d4c24640 1 --2- 172.21.15.64:0/3492976918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d0107b70 0x7f45d01989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.941+0000 7f45d4c24640 1 -- 172.21.15.64:0/3492976918 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d019e380 con 0x7f45d0102230 2024-09-21T04:04:53.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.941+0000 7f45cffff640 1 --2- 172.21.15.64:0/3492976918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d0107b70 0x7f45d01989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:04:53.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.941+0000 7f45d4c24640 1 --2- 172.21.15.64:0/3492976918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45d0102230 0x7f45d0198410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f45c4006010 tx=0x7f45c40042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:53.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.942+0000 7f45cdffb640 1 -- 172.21.15.64:0/3492976918 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c403f070 con 0x7f45d0102230 2024-09-21T04:04:53.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.942+0000 7f45cdffb640 1 -- 172.21.15.64:0/3492976918 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45c4032400 con 0x7f45d0102230 2024-09-21T04:04:53.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.942+0000 7f45d6e88640 1 -- 172.21.15.64:0/3492976918 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d019e610 con 0x7f45d0102230 2024-09-21T04:04:53.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.942+0000 7f45cdffb640 1 -- 172.21.15.64:0/3492976918 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4036040 con 0x7f45d0102230 2024-09-21T04:04:53.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.942+0000 7f45d6e88640 1 -- 172.21.15.64:0/3492976918 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d01a6fe0 con 0x7f45d0102230 2024-09-21T04:04:53.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.946+0000 7f45dd3fc740 1 Processor -- start 2024-09-21T04:04:53.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.946+0000 7f45dd3fc740 1 -- start start 2024-09-21T04:04:53.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.946+0000 7f45dd3fc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1c27350 0x5582e1fce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.946+0000 7f45dd3fc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e1bfe490 0x5582e1fce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.946+0000 7f45dd3fc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5582e1c06670 0x5582e1fceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.946+0000 7f45dd3fc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5582e1f44e60 con 0x5582e1c27350 2024-09-21T04:04:53.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.946+0000 7f45dd3fc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5582e1f34290 con 0x5582e1bfe490 2024-09-21T04:04:53.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.946+0000 7f45dd3fc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5582e1f380a0 con 0x5582e1c06670 2024-09-21T04:04:53.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.947+0000 7f45d5425640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1c27350 0x5582e1fce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.947+0000 7f45d5425640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1c27350 0x5582e1fce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34410/0 (socket says 172.21.15.64:34410) 2024-09-21T04:04:53.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.947+0000 7f45d5425640 1 -- 172.21.15.64:0/3402933014 learned_addr learned my addr 172.21.15.64:0/3402933014 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:53.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.947+0000 7f45cffff640 1 --2- 172.21.15.64:0/3402933014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5582e1c06670 0x5582e1fceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.947+0000 7f45d5425640 1 -- 172.21.15.64:0/3402933014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5582e1c06670 msgr2=0x5582e1fceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:53.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.947+0000 7f45d5425640 1 --2- 172.21.15.64:0/3402933014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5582e1c06670 0x5582e1fceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.947+0000 7f45d5425640 1 -- 172.21.15.64:0/3402933014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e1bfe490 msgr2=0x5582e1fce970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:04:53.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.947+0000 7f45d5425640 1 --2- 172.21.15.64:0/3402933014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e1bfe490 0x5582e1fce970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.947+0000 7f45d5425640 1 -- 172.21.15.64:0/3402933014 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582e1c92930 con 0x5582e1c27350 2024-09-21T04:04:53.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.948+0000 7f45d5425640 1 --2- 172.21.15.64:0/3402933014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1c27350 0x5582e1fce430 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009fc0 tx=0x7f45c8010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91865fbed3d8cb07 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:53.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.948+0000 7f45baffd640 1 -- 172.21.15.64:0/3402933014 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8003b70 con 0x5582e1c27350 2024-09-21T04:04:53.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.948+0000 7f45baffd640 1 -- 172.21.15.64:0/3402933014 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45c8003d10 con 0x5582e1c27350 2024-09-21T04:04:53.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.949+0000 7f45baffd640 1 -- 172.21.15.64:0/3402933014 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8011a00 con 0x5582e1c27350 2024-09-21T04:04:53.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.949+0000 7f45dd3fc740 1 -- 172.21.15.64:0/3402933014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1c27350 msgr2=0x5582e1fce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:53.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.949+0000 7f45dd3fc740 1 --2- 172.21.15.64:0/3402933014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1c27350 0x5582e1fce430 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009fc0 tx=0x7f45c8010860 comp rx=0 tx=0).stop 2024-09-21T04:04:53.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.949+0000 7f45dd3fc740 1 -- 172.21.15.64:0/3402933014 shutdown_connections 2024-09-21T04:04:53.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.949+0000 7f45dd3fc740 1 --2- 172.21.15.64:0/3402933014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5582e1c06670 0x5582e1fceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.949+0000 7f45dd3fc740 1 --2- 172.21.15.64:0/3402933014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e1bfe490 0x5582e1fce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.949+0000 7f45dd3fc740 1 --2- 172.21.15.64:0/3402933014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1c27350 0x5582e1fce430 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.949+0000 7f45dd3fc740 1 -- 172.21.15.64:0/3402933014 >> 172.21.15.64:0/3402933014 conn(0x5582e1c1b740 msgr2=0x5582e1fcc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:53.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.950+0000 7f45cdffb640 1 -- 172.21.15.64:0/3492976918 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 25) ==== 154694+0+0 (secure 0 0 0) 0x7f45c4042070 con 0x7f45d0102230 2024-09-21T04:04:53.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.950+0000 7f45dd3fc740 1 -- 172.21.15.64:0/3402933014 shutdown_connections 2024-09-21T04:04:53.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.950+0000 7f45dd3fc740 1 -- 172.21.15.64:0/3402933014 wait complete. 2024-09-21T04:04:53.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.951+0000 7f45dd3fc740 1 -- start start 2024-09-21T04:04:53.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.951+0000 7f45cdffb640 1 --2- 172.21.15.64:0/3492976918 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f45c00b66a0 0x7f45c00b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.951+0000 7f45d5425640 1 --2- 172.21.15.64:0/3492976918 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f45c00b66a0 0x7f45c00b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.951+0000 7f45cdffb640 1 -- 172.21.15.64:0/3492976918 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f45c4108010 con 0x7f45d0102230 2024-09-21T04:04:53.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.952+0000 7f45dd3fc740 1 Processor -- start 2024-09-21T04:04:53.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.952+0000 7f45d5425640 1 --2- 172.21.15.64:0/3492976918 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f45c00b66a0 0x7f45c00b8bb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f45c8004980 tx=0x7f45c8005fd0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:53.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.953+0000 7f45dd3fc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1c06670 0x5582e1fd9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.953+0000 7f45dd3fc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e1c27350 0x5582e1fda4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.953+0000 7f45dd3fc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5582e1be5cd0 0x5582e1fe6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.953+0000 7f45d4c24640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1c06670 0x5582e1fd9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.953+0000 7f45dd3fc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5582e1d95890 con 0x5582e1c06670 2024-09-21T04:04:53.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.953+0000 7f45dd3fc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5582e1eced40 con 0x5582e1c27350 2024-09-21T04:04:53.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.953+0000 7f45dd3fc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5582e1fdccb0 con 0x5582e1be5cd0 2024-09-21T04:04:53.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.953+0000 7f45d4c24640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1c06670 0x5582e1fd9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34424/0 (socket says 172.21.15.64:34424) 2024-09-21T04:04:53.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.953+0000 7f45d4c24640 1 -- 172.21.15.64:0/2019439661 learned_addr learned my addr 172.21.15.64:0/2019439661 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:53.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.953+0000 7f45cffff640 1 --2- 172.21.15.64:0/2019439661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e1c27350 0x5582e1fda4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.953+0000 7f45d5425640 1 --2- 172.21.15.64:0/2019439661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5582e1be5cd0 0x5582e1fe6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.954+0000 7f45d4c24640 1 -- 172.21.15.64:0/2019439661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5582e1be5cd0 msgr2=0x5582e1fe6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:53.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.954+0000 7f45d4c24640 1 --2- 172.21.15.64:0/2019439661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5582e1be5cd0 0x5582e1fe6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.954+0000 7f45d4c24640 1 -- 172.21.15.64:0/2019439661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e1c27350 msgr2=0x5582e1fda4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:53.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.954+0000 7f45d4c24640 1 --2- 172.21.15.64:0/2019439661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e1c27350 0x5582e1fda4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.954+0000 7f45d4c24640 1 -- 172.21.15.64:0/2019439661 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582e1fe6560 con 0x5582e1c06670 2024-09-21T04:04:53.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.954+0000 7f45d5425640 1 --2- 172.21.15.64:0/2019439661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5582e1be5cd0 0x5582e1fe6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:04:53.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.954+0000 7f45cffff640 1 --2- 172.21.15.64:0/2019439661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e1c27350 0x5582e1fda4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:53.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.954+0000 7f45dd3fc740 10 client.0 _ll_register_callbacks cb 0x5582e1f32c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:53.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.954+0000 7f45dd3fc740 1 -- 172.21.15.64:0/2019439661 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45c8003d10 con 0x5582e1c06670 2024-09-21T04:04:53.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.954+0000 7f45d4c24640 1 --2- 172.21.15.64:0/2019439661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1c06670 0x5582e1fd9f70 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f45c4105d30 tx=0x7f45c40cdb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9e25c8490e9e91 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:53.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.954+0000 7f45affff640 10 client.15741 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:04:53.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.954+0000 7f45affff640 1 -- 172.21.15.64:0/2019439661 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4049040 con 0x5582e1c06670 2024-09-21T04:04:53.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.954+0000 7f45affff640 1 -- 172.21.15.64:0/2019439661 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f45c40328d0 con 0x5582e1c06670 2024-09-21T04:04:53.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.954+0000 7f45affff640 1 -- 172.21.15.64:0/2019439661 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f45c40cd430 con 0x5582e1c06670 2024-09-21T04:04:53.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.955+0000 7f45dd3fc740 10 client.15741 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:53.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.955+0000 7f45dd3fc740 1 -- 172.21.15.64:0/2019439661 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5582e1fe8bd0 con 0x5582e1c06670 2024-09-21T04:04:53.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.955+0000 7f45affff640 1 -- 172.21.15.64:0/2019439661 <== mon.0 v2:172.21.15.64:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f45c400b040 con 0x5582e1c06670 2024-09-21T04:04:53.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.955+0000 7f45dd3fc740 10 client.15741 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:53.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.955+0000 7f45dd3fc740 10 client.15741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:04:53.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.955+0000 7f45dd3fc740 20 client.15741 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:53.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.955+0000 7f45dd3fc740 1 --2- 172.21.15.64:0/2019439661 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5582e1bbb080 0x5582e1fec0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.955+0000 7f45dd3fc740 4 client.15741 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:04:53.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.955+0000 7f45dd3fc740 1 -- 172.21.15.64:0/2019439661 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582e1f46db0 con 0x5582e1bbb080 2024-09-21T04:04:53.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.956+0000 7f45cffff640 1 --2- 172.21.15.64:0/2019439661 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5582e1bbb080 0x5582e1fec0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.956+0000 7f45cffff640 1 --2- 172.21.15.64:0/2019439661 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5582e1bbb080 0x5582e1fec0a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99d8131a8a3e7e82 server_cookie=1afa777277867bec in_seq=0 out_seq=0 2024-09-21T04:04:53.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.956+0000 7f45affff640 10 client.15741 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:53.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.957+0000 7f45affff640 1 -- 172.21.15.64:0/2019439661 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5582e1f46db0 con 0x5582e1bbb080 2024-09-21T04:04:53.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.957+0000 7f45affff640 10 client.15741 handle_command_reply: tid=0 2024-09-21T04:04:53.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.957+0000 7f45dd3fc740 1 client.15741 shutdown 2024-09-21T04:04:53.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.957+0000 7f45dd3fc740 1 -- 172.21.15.64:0/2019439661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1c06670 msgr2=0x5582e1fd9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:53.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.957+0000 7f45dd3fc740 1 --2- 172.21.15.64:0/2019439661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1c06670 0x5582e1fd9f70 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f45c4105d30 tx=0x7f45c40cdb50 comp rx=0 tx=0).stop 2024-09-21T04:04:53.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.957+0000 7f45dd3fc740 1 -- 172.21.15.64:0/2019439661 shutdown_connections 2024-09-21T04:04:53.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.957+0000 7f45dd3fc740 1 --2- 172.21.15.64:0/2019439661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5582e1be5cd0 0x5582e1fe6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.957+0000 7f45dd3fc740 1 --2- 172.21.15.64:0/2019439661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e1c27350 0x5582e1fda4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.957+0000 7f45dd3fc740 1 --2- 172.21.15.64:0/2019439661 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5582e1bbb080 0x5582e1fec0a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.957+0000 7f45affff640 0 client.15741 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:53.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.957+0000 7f45dd3fc740 1 --2- 172.21.15.64:0/2019439661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1c06670 0x5582e1fd9f70 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.957+0000 7f45dd3fc740 1 -- 172.21.15.64:0/2019439661 >> 172.21.15.64:0/2019439661 conn(0x5582e1c1b740 msgr2=0x5582e1fd1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:53.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.958+0000 7f45dd3fc740 1 -- 172.21.15.64:0/2019439661 shutdown_connections 2024-09-21T04:04:53.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.958+0000 7f45dd3fc740 1 -- 172.21.15.64:0/2019439661 wait complete. 2024-09-21T04:04:53.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.958+0000 7f45dd3fc740 20 client.15741 trim_cache size 0 max 16384 2024-09-21T04:04:53.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.994+0000 7f45dd3fc740 1 Processor -- start 2024-09-21T04:04:53.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.994+0000 7f45dd3fc740 1 -- start start 2024-09-21T04:04:53.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.994+0000 7f45dd3fc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e1bbb080 0x5582e1fe5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.994+0000 7f45dd3fc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1be5cd0 0x5582e1fd8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.994+0000 7f45dd3fc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5582e1c27350 0x5582e1fd6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:53.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.994+0000 7f45dd3fc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f45d01afff0 con 0x5582e1be5cd0 2024-09-21T04:04:53.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.994+0000 7f45dd3fc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f45d01ad8c0 con 0x5582e1bbb080 2024-09-21T04:04:53.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.994+0000 7f45dd3fc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f45d01ab710 con 0x5582e1c27350 2024-09-21T04:04:53.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.994+0000 7f45cffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1be5cd0 0x5582e1fd8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.994+0000 7f45cffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1be5cd0 0x5582e1fd8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34438/0 (socket says 172.21.15.64:34438) 2024-09-21T04:04:53.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.994+0000 7f45cffff640 1 -- 172.21.15.64:0/937414649 learned_addr learned my addr 172.21.15.64:0/937414649 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:53.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.994+0000 7f45d4c24640 1 --2- 172.21.15.64:0/937414649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e1bbb080 0x5582e1fe5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.995+0000 7f45d5425640 1 --2- 172.21.15.64:0/937414649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5582e1c27350 0x5582e1fd6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:53.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.995+0000 7f45d4c24640 1 -- 172.21.15.64:0/937414649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5582e1c27350 msgr2=0x5582e1fd6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:53.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.995+0000 7f45d4c24640 1 --2- 172.21.15.64:0/937414649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5582e1c27350 0x5582e1fd6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:53.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.995+0000 7f45d4c24640 1 -- 172.21.15.64:0/937414649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1be5cd0 msgr2=0x5582e1fd8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:54.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.995+0000 7f45d4c24640 1 --2- 172.21.15.64:0/937414649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1be5cd0 0x5582e1fd8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:54.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.995+0000 7f45d4c24640 1 -- 172.21.15.64:0/937414649 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582e1fd82b0 con 0x5582e1bbb080 2024-09-21T04:04:54.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.995+0000 7f45d5425640 1 --2- 172.21.15.64:0/937414649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5582e1c27350 0x5582e1fd6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:04:54.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.995+0000 7f45cffff640 1 --2- 172.21.15.64:0/937414649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1be5cd0 0x5582e1fd8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:54.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.995+0000 7f45d4c24640 1 --2- 172.21.15.64:0/937414649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e1bbb080 0x5582e1fe5f20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f45c40ca290 tx=0x7f45c40f0000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=313e44d9193ebb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:54.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.995+0000 7f45ba7fc640 1 -- 172.21.15.64:0/937414649 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4049020 con 0x5582e1bbb080 2024-09-21T04:04:54.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.996+0000 7f45ba7fc640 1 -- 172.21.15.64:0/937414649 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45c8003d10 con 0x5582e1bbb080 2024-09-21T04:04:54.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.996+0000 7f45ba7fc640 1 -- 172.21.15.64:0/937414649 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c40044e0 con 0x5582e1bbb080 2024-09-21T04:04:54.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.997+0000 7f45dd3fc740 1 -- 172.21.15.64:0/937414649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e1bbb080 msgr2=0x5582e1fe5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:54.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.997+0000 7f45dd3fc740 1 --2- 172.21.15.64:0/937414649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e1bbb080 0x5582e1fe5f20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f45c40ca290 tx=0x7f45c40f0000 comp rx=0 tx=0).stop 2024-09-21T04:04:54.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.997+0000 7f45dd3fc740 1 -- 172.21.15.64:0/937414649 shutdown_connections 2024-09-21T04:04:54.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.997+0000 7f45dd3fc740 1 --2- 172.21.15.64:0/937414649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5582e1c27350 0x5582e1fd6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:54.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.997+0000 7f45dd3fc740 1 --2- 172.21.15.64:0/937414649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1be5cd0 0x5582e1fd8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:54.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.997+0000 7f45dd3fc740 1 --2- 172.21.15.64:0/937414649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e1bbb080 0x5582e1fe5f20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:54.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.997+0000 7f45dd3fc740 1 -- 172.21.15.64:0/937414649 >> 172.21.15.64:0/937414649 conn(0x5582e1c1b740 msgr2=0x5582e1fd8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:54.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.997+0000 7f45dd3fc740 1 -- 172.21.15.64:0/937414649 shutdown_connections 2024-09-21T04:04:54.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.997+0000 7f45dd3fc740 1 -- 172.21.15.64:0/937414649 wait complete. 2024-09-21T04:04:54.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.998+0000 7f45dd3fc740 1 -- start start 2024-09-21T04:04:54.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.998+0000 7f45dd3fc740 1 Processor -- start 2024-09-21T04:04:54.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.998+0000 7f45dd3fc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e1bbb080 0x5582e1fde860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:54.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.999+0000 7f45dd3fc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1be5cd0 0x5582e1fdee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:54.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.999+0000 7f45dd3fc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5582e1c27350 0x5582e204fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:54.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.999+0000 7f45dd3fc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f45d01abd80 con 0x5582e1be5cd0 2024-09-21T04:04:54.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.999+0000 7f45dd3fc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f45d01ad1f0 con 0x5582e1bbb080 2024-09-21T04:04:54.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.999+0000 7f45dd3fc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f45d01ac450 con 0x5582e1c27350 2024-09-21T04:04:54.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.999+0000 7f45cffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1be5cd0 0x5582e1fdee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:54.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.999+0000 7f45d4c24640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e1bbb080 0x5582e1fde860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:54.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.999+0000 7f45cffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1be5cd0 0x5582e1fdee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34446/0 (socket says 172.21.15.64:34446) 2024-09-21T04:04:54.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.999+0000 7f45cffff640 1 -- 172.21.15.64:0/2908459617 learned_addr learned my addr 172.21.15.64:0/2908459617 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:54.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.999+0000 7f45d5425640 1 --2- 172.21.15.64:0/2908459617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5582e1c27350 0x5582e204fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:54.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.999+0000 7f45dd3fc740 10 client.0 _ll_register_callbacks cb 0x7f45c4105500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:54.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.999+0000 7f45d4c24640 1 -- 172.21.15.64:0/2908459617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5582e1c27350 msgr2=0x5582e204fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:54.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.999+0000 7f45d4c24640 1 --2- 172.21.15.64:0/2908459617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5582e1c27350 0x5582e204fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:54.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.999+0000 7f45d4c24640 1 -- 172.21.15.64:0/2908459617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1be5cd0 msgr2=0x5582e1fdee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:54.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.999+0000 7f45d4c24640 1 --2- 172.21.15.64:0/2908459617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1be5cd0 0x5582e1fdee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:54.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.999+0000 7f45d4c24640 1 -- 172.21.15.64:0/2908459617 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582e20500a0 con 0x5582e1bbb080 2024-09-21T04:04:54.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:53.999+0000 7f45cffff640 1 --2- 172.21.15.64:0/2908459617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1be5cd0 0x5582e1fdee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:54.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.000+0000 7f45d4c24640 1 --2- 172.21.15.64:0/2908459617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e1bbb080 0x5582e1fde860 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f45c40ca3f0 tx=0x7f45c40ca310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0a0229500000707 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:54.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.000+0000 7f45affff640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:04:54.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.000+0000 7f45affff640 1 -- 172.21.15.64:0/2908459617 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4049040 con 0x5582e1bbb080 2024-09-21T04:04:54.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.000+0000 7f45affff640 1 -- 172.21.15.64:0/2908459617 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f45c4105920 con 0x5582e1bbb080 2024-09-21T04:04:54.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.000+0000 7f45dd3fc740 1 -- 172.21.15.64:0/2908459617 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45c8003d10 con 0x5582e1bbb080 2024-09-21T04:04:54.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.000+0000 7f45affff640 1 -- 172.21.15.64:0/2908459617 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f45c40044e0 con 0x5582e1bbb080 2024-09-21T04:04:54.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.000+0000 7f45dd3fc740 10 client.25469 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:54.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.000+0000 7f45dd3fc740 1 -- 172.21.15.64:0/2908459617 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5582e20500a0 con 0x5582e1bbb080 2024-09-21T04:04:54.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.000+0000 7f45affff640 1 -- 172.21.15.64:0/2908459617 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45c4105b20 con 0x5582e1bbb080 2024-09-21T04:04:54.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.002+0000 7f45affff640 1 -- 172.21.15.64:0/2908459617 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f45c40f10d0 con 0x5582e1bbb080 2024-09-21T04:04:54.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.002+0000 7f45dd3fc740 10 client.25469 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:54.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.002+0000 7f45dd3fc740 10 client.25469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:04:54.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.002+0000 7f45dd3fc740 20 client.25469 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:54.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.002+0000 7f45dd3fc740 1 --2- 172.21.15.64:0/2908459617 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5582e2057800 0x5582e2059c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:54.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.002+0000 7f45dd3fc740 4 client.25469 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T04:04:54.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.002+0000 7f45dd3fc740 1 -- 172.21.15.64:0/2908459617 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f45c8003d10 con 0x5582e2057800 2024-09-21T04:04:54.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.002+0000 7f45cffff640 1 --2- 172.21.15.64:0/2908459617 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5582e2057800 0x5582e2059c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:54.008 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:53 smithi119 ceph-mon[34811]: pgmap v347: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-21T04:04:54.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.003+0000 7f45cffff640 1 --2- 172.21.15.64:0/2908459617 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5582e2057800 0x5582e2059c20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=313f5d85292ccd42 server_cookie=fe28148efa1ef52c in_seq=0 out_seq=0 2024-09-21T04:04:54.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.003+0000 7f45affff640 10 client.25469 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:54.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.004+0000 7f45affff640 1 -- 172.21.15.64:0/2908459617 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f45c8003d10 con 0x5582e2057800 2024-09-21T04:04:54.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.004+0000 7f45affff640 10 client.25469 handle_command_reply: tid=0 2024-09-21T04:04:54.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.004+0000 7f45dd3fc740 1 client.25469 shutdown 2024-09-21T04:04:54.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.005+0000 7f45dd3fc740 1 -- 172.21.15.64:0/2908459617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e1bbb080 msgr2=0x5582e1fde860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:54.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.005+0000 7f45dd3fc740 1 --2- 172.21.15.64:0/2908459617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e1bbb080 0x5582e1fde860 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f45c40ca3f0 tx=0x7f45c40ca310 comp rx=0 tx=0).stop 2024-09-21T04:04:54.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.005+0000 7f45dd3fc740 1 -- 172.21.15.64:0/2908459617 shutdown_connections 2024-09-21T04:04:54.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.005+0000 7f45dd3fc740 1 --2- 172.21.15.64:0/2908459617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5582e1c27350 0x5582e204fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:54.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.005+0000 7f45dd3fc740 1 --2- 172.21.15.64:0/2908459617 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5582e2057800 0x5582e2059c20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:54.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.005+0000 7f45dd3fc740 1 --2- 172.21.15.64:0/2908459617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5582e1be5cd0 0x5582e1fdee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:54.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.005+0000 7f45dd3fc740 1 --2- 172.21.15.64:0/2908459617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582e1bbb080 0x5582e1fde860 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:54.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.005+0000 7f45dd3fc740 1 -- 172.21.15.64:0/2908459617 >> 172.21.15.64:0/2908459617 conn(0x5582e1c1b740 msgr2=0x5582e1fcf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:54.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.005+0000 7f45affff640 0 client.25469 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:54.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.006+0000 7f45dd3fc740 1 -- 172.21.15.64:0/2908459617 shutdown_connections 2024-09-21T04:04:54.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.006+0000 7f45dd3fc740 1 -- 172.21.15.64:0/2908459617 wait complete. 2024-09-21T04:04:54.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.006+0000 7f45dd3fc740 20 client.25469 trim_cache size 0 max 16384 2024-09-21T04:04:54.011 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:04:54.011 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-21T04:04:54.011 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "b52b950c-0f97-49ec-a37c-1a8e04348ce8", 2024-09-21T04:04:54.011 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-21T04:04:54.011 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:04:54.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.007+0000 7f45d6e88640 1 -- 172.21.15.64:0/3492976918 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f45c00b66a0 msgr2=0x7f45c00b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:54.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.007+0000 7f45d6e88640 1 --2- 172.21.15.64:0/3492976918 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f45c00b66a0 0x7f45c00b8bb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f45c8004980 tx=0x7f45c8005fd0 comp rx=0 tx=0).stop 2024-09-21T04:04:54.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.007+0000 7f45d6e88640 1 -- 172.21.15.64:0/3492976918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45d0102230 msgr2=0x7f45d0198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:54.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.007+0000 7f45d6e88640 1 --2- 172.21.15.64:0/3492976918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45d0102230 0x7f45d0198410 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f45c4006010 tx=0x7f45c40042f0 comp rx=0 tx=0).stop 2024-09-21T04:04:54.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.007+0000 7f45d6e88640 1 -- 172.21.15.64:0/3492976918 shutdown_connections 2024-09-21T04:04:54.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.007+0000 7f45d6e88640 1 --2- 172.21.15.64:0/3492976918 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f45c00b66a0 0x7f45c00b8bb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:54.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.007+0000 7f45d6e88640 1 --2- 172.21.15.64:0/3492976918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45d01084c0 0x7f45d019dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:54.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.007+0000 7f45d6e88640 1 --2- 172.21.15.64:0/3492976918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45d0107b70 0x7f45d01989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:54.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.007+0000 7f45d6e88640 1 --2- 172.21.15.64:0/3492976918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45d0102230 0x7f45d0198410 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:54.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.007+0000 7f45d6e88640 1 -- 172.21.15.64:0/3492976918 >> 172.21.15.64:0/3492976918 conn(0x7f45d0079e10 msgr2=0x7f45d0105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:54.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.008+0000 7f45d6e88640 1 -- 172.21.15.64:0/3492976918 shutdown_connections 2024-09-21T04:04:54.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:54.008+0000 7f45d6e88640 1 -- 172.21.15.64:0/3492976918 wait complete. 2024-09-21T04:04:54.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:53 smithi131 ceph-mon[35193]: pgmap v347: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-21T04:04:54.864 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T04:04:55.155 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:04:55.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.702+0000 7ff1cad0e640 1 -- 172.21.15.64:0/3853845544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1c4108d20 msgr2=0x7ff1c410f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:55.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.702+0000 7ff1cad0e640 1 --2- 172.21.15.64:0/3853845544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1c4108d20 0x7ff1c410f550 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009ca0 tx=0x7ff1bc02a8c0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.702+0000 7ff1cad0e640 1 -- 172.21.15.64:0/3853845544 shutdown_connections 2024-09-21T04:04:55.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.702+0000 7ff1cad0e640 1 --2- 172.21.15.64:0/3853845544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1c4108d20 0x7ff1c410f550 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.702+0000 7ff1cad0e640 1 --2- 172.21.15.64:0/3853845544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1c41083d0 0x7ff1c41087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.702+0000 7ff1cad0e640 1 --2- 172.21.15.64:0/3853845544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1c4102ae0 0x7ff1c4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.702+0000 7ff1cad0e640 1 -- 172.21.15.64:0/3853845544 >> 172.21.15.64:0/3853845544 conn(0x7ff1c4079760 msgr2=0x7ff1c4079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:55.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.702+0000 7ff1cad0e640 1 -- 172.21.15.64:0/3853845544 shutdown_connections 2024-09-21T04:04:55.705 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.702+0000 7ff1cad0e640 1 -- 172.21.15.64:0/3853845544 wait complete. 2024-09-21T04:04:55.706 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.703+0000 7ff1cad0e640 1 Processor -- start 2024-09-21T04:04:55.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.703+0000 7ff1cad0e640 1 -- start start 2024-09-21T04:04:55.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.704+0000 7ff1cad0e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1c4102ae0 0x7ff1c419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:55.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.704+0000 7ff1cad0e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1c41083d0 0x7ff1c419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:55.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.704+0000 7ff1cad0e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1c4108d20 0x7ff1c419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:55.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.704+0000 7ff1c3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1c41083d0 0x7ff1c419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:55.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.704+0000 7ff1c3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1c41083d0 0x7ff1c419a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34456/0 (socket says 172.21.15.64:34456) 2024-09-21T04:04:55.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.704+0000 7ff1c3fff640 1 -- 172.21.15.64:0/972385877 learned_addr learned my addr 172.21.15.64:0/972385877 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:55.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.704+0000 7ff1c8aaa640 1 --2- 172.21.15.64:0/972385877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1c4102ae0 0x7ff1c419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:55.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.704+0000 7ff1cad0e640 1 -- 172.21.15.64:0/972385877 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c4113ab0 con 0x7ff1c41083d0 2024-09-21T04:04:55.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.704+0000 7ff1cad0e640 1 -- 172.21.15.64:0/972385877 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c4113930 con 0x7ff1c4102ae0 2024-09-21T04:04:55.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.704+0000 7ff1cad0e640 1 -- 172.21.15.64:0/972385877 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c4113c30 con 0x7ff1c4108d20 2024-09-21T04:04:55.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.704+0000 7ff1c92ab640 1 --2- 172.21.15.64:0/972385877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1c4108d20 0x7ff1c419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:55.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.705+0000 7ff1c8aaa640 1 -- 172.21.15.64:0/972385877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1c4108d20 msgr2=0x7ff1c419fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:55.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.705+0000 7ff1c8aaa640 1 --2- 172.21.15.64:0/972385877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1c4108d20 0x7ff1c419fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.705+0000 7ff1c8aaa640 1 -- 172.21.15.64:0/972385877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1c41083d0 msgr2=0x7ff1c419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:55.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.705+0000 7ff1c8aaa640 1 --2- 172.21.15.64:0/972385877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1c41083d0 0x7ff1c419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.705+0000 7ff1c8aaa640 1 -- 172.21.15.64:0/972385877 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c41a03b0 con 0x7ff1c4102ae0 2024-09-21T04:04:55.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.705+0000 7ff1c3fff640 1 --2- 172.21.15.64:0/972385877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1c41083d0 0x7ff1c419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:04:55.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.705+0000 7ff1c8aaa640 1 --2- 172.21.15.64:0/972385877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1c4102ae0 0x7ff1c419a440 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800e9f0 tx=0x7ff1b800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:55.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.705+0000 7ff1c1ffb640 1 -- 172.21.15.64:0/972385877 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b800ccc0 con 0x7ff1c4102ae0 2024-09-21T04:04:55.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.705+0000 7ff1c1ffb640 1 -- 172.21.15.64:0/972385877 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1b80108e0 con 0x7ff1c4102ae0 2024-09-21T04:04:55.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.705+0000 7ff1c1ffb640 1 -- 172.21.15.64:0/972385877 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8018880 con 0x7ff1c4102ae0 2024-09-21T04:04:55.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.705+0000 7ff1cad0e640 1 -- 172.21.15.64:0/972385877 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c41a06a0 con 0x7ff1c4102ae0 2024-09-21T04:04:55.709 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.706+0000 7ff1cad0e640 1 -- 172.21.15.64:0/972385877 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c41a6e00 con 0x7ff1c4102ae0 2024-09-21T04:04:55.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.709+0000 7ff1c1ffb640 1 -- 172.21.15.64:0/972385877 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 25) ==== 154694+0+0 (secure 0 0 0) 0x7ff1b8010430 con 0x7ff1c4102ae0 2024-09-21T04:04:55.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.709+0000 7ff1d1282740 1 Processor -- start 2024-09-21T04:04:55.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.709+0000 7ff1d1282740 1 -- start start 2024-09-21T04:04:55.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.710+0000 7ff1d1282740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9379af070 0x55f937d56270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:55.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.710+0000 7ff1d1282740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9379861b0 0x55f937d567b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:55.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.710+0000 7ff1d1282740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f93798e390 0x55f937d56cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:55.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.710+0000 7ff1d1282740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f937ccce50 con 0x55f93798e390 2024-09-21T04:04:55.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.710+0000 7ff1d1282740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f937cbc280 con 0x55f9379861b0 2024-09-21T04:04:55.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.710+0000 7ff1d1282740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f937cc0090 con 0x55f9379af070 2024-09-21T04:04:55.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.710+0000 7ff1c3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f93798e390 0x55f937d56cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:55.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.710+0000 7ff1c3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f93798e390 0x55f937d56cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34464/0 (socket says 172.21.15.64:34464) 2024-09-21T04:04:55.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.710+0000 7ff1c3fff640 1 -- 172.21.15.64:0/3387757406 learned_addr learned my addr 172.21.15.64:0/3387757406 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:55.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.710+0000 7ff1c1ffb640 1 --2- 172.21.15.64:0/972385877 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff1b40b6710 0x7ff1b40b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:55.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.710+0000 7ff1c92ab640 1 --2- 172.21.15.64:0/3387757406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9379af070 0x55f937d56270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:55.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.710+0000 7ff1c8aaa640 1 --2- 172.21.15.64:0/3387757406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9379861b0 0x55f937d567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:55.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.710+0000 7ff1c92ab640 1 --2- 172.21.15.64:0/972385877 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff1b40b6710 0x7ff1b40b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:55.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.711+0000 7ff1c3fff640 1 -- 172.21.15.64:0/3387757406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9379af070 msgr2=0x55f937d56270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:55.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.711+0000 7ff1c3fff640 1 --2- 172.21.15.64:0/3387757406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9379af070 0x55f937d56270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.711+0000 7ff1c3fff640 1 -- 172.21.15.64:0/3387757406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9379861b0 msgr2=0x55f937d567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:55.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.711+0000 7ff1c3fff640 1 --2- 172.21.15.64:0/3387757406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9379861b0 0x55f937d567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.711+0000 7ff1c3fff640 1 -- 172.21.15.64:0/3387757406 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f937a1a650 con 0x55f93798e390 2024-09-21T04:04:55.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.711+0000 7ff1c3fff640 1 --2- 172.21.15.64:0/3387757406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f93798e390 0x55f937d56cf0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0011570 tx=0x7ff1b0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6a33f443499370 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:55.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.712+0000 7ff1aeffd640 1 -- 172.21.15.64:0/3387757406 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b001e070 con 0x55f93798e390 2024-09-21T04:04:55.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.712+0000 7ff1c1ffb640 1 -- 172.21.15.64:0/972385877 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7ff1b80e7bf0 con 0x7ff1c4102ae0 2024-09-21T04:04:55.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.712+0000 7ff1c92ab640 1 --2- 172.21.15.64:0/972385877 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff1b40b6710 0x7ff1b40b8c20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc006010 tx=0x7ff1bc02a8c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:55.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.712+0000 7ff1aeffd640 1 -- 172.21.15.64:0/3387757406 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1b000faf0 con 0x55f93798e390 2024-09-21T04:04:55.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.713+0000 7ff1d1282740 1 -- 172.21.15.64:0/3387757406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f93798e390 msgr2=0x55f937d56cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:55.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.713+0000 7ff1d1282740 1 --2- 172.21.15.64:0/3387757406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f93798e390 0x55f937d56cf0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0011570 tx=0x7ff1b0011a30 comp rx=0 tx=0).stop 2024-09-21T04:04:55.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.713+0000 7ff1d1282740 1 -- 172.21.15.64:0/3387757406 shutdown_connections 2024-09-21T04:04:55.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.713+0000 7ff1d1282740 1 --2- 172.21.15.64:0/3387757406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f93798e390 0x55f937d56cf0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.713+0000 7ff1d1282740 1 --2- 172.21.15.64:0/3387757406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9379861b0 0x55f937d567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.713+0000 7ff1d1282740 1 --2- 172.21.15.64:0/3387757406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9379af070 0x55f937d56270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.713+0000 7ff1d1282740 1 -- 172.21.15.64:0/3387757406 >> 172.21.15.64:0/3387757406 conn(0x55f9379a3460 msgr2=0x55f937d54160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:55.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.713+0000 7ff1d1282740 1 -- 172.21.15.64:0/3387757406 shutdown_connections 2024-09-21T04:04:55.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.713+0000 7ff1d1282740 1 -- 172.21.15.64:0/3387757406 wait complete. 2024-09-21T04:04:55.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.714+0000 7ff1d1282740 1 -- start start 2024-09-21T04:04:55.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.714+0000 7ff1d1282740 1 Processor -- start 2024-09-21T04:04:55.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.714+0000 7ff1d1282740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9379861b0 0x55f937d617d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:55.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.714+0000 7ff1d1282740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f93798e390 0x55f937d61d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:55.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.714+0000 7ff1d1282740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9379af070 0x55f937d62250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:55.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.714+0000 7ff1d1282740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f937b1d700 con 0x55f93798e390 2024-09-21T04:04:55.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.714+0000 7ff1d1282740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f937c57110 con 0x55f9379861b0 2024-09-21T04:04:55.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.714+0000 7ff1d1282740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f937d64af0 con 0x55f9379af070 2024-09-21T04:04:55.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.714+0000 7ff1d1282740 10 client.0 _ll_register_callbacks cb 0x55f937c70990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:55.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.715+0000 7ff1c3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f93798e390 0x55f937d61d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:55.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.715+0000 7ff1c3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f93798e390 0x55f937d61d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34466/0 (socket says 172.21.15.64:34466) 2024-09-21T04:04:55.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.715+0000 7ff1c3fff640 1 -- 172.21.15.64:0/2126014689 learned_addr learned my addr 172.21.15.64:0/2126014689 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:55.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.715+0000 7ff1c3fff640 1 -- 172.21.15.64:0/2126014689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9379af070 msgr2=0x55f937d62250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:04:55.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.715+0000 7ff1c3fff640 1 --2- 172.21.15.64:0/2126014689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9379af070 0x55f937d62250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.715+0000 7ff1c3fff640 1 -- 172.21.15.64:0/2126014689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9379861b0 msgr2=0x55f937d617d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:55.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.715+0000 7ff1c8aaa640 1 --2- 172.21.15.64:0/2126014689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9379861b0 0x55f937d617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:55.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.715+0000 7ff1c3fff640 1 --2- 172.21.15.64:0/2126014689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9379861b0 0x55f937d617d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.715+0000 7ff1c3fff640 1 -- 172.21.15.64:0/2126014689 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f937c84c40 con 0x55f93798e390 2024-09-21T04:04:55.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.715+0000 7ff1c3fff640 1 --2- 172.21.15.64:0/2126014689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f93798e390 0x55f937d61d10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0019770 tx=0x7ff1b00095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=811ef111284fb1e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:55.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.715+0000 7ff1a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:04:55.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.716+0000 7ff1a3fff640 1 -- 172.21.15.64:0/2126014689 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b001e040 con 0x55f93798e390 2024-09-21T04:04:55.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.716+0000 7ff1a3fff640 1 -- 172.21.15.64:0/2126014689 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7ff1b0005040 con 0x55f93798e390 2024-09-21T04:04:55.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.716+0000 7ff1d1282740 1 -- 172.21.15.64:0/2126014689 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1b000faf0 con 0x55f93798e390 2024-09-21T04:04:55.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.716+0000 7ff1a3fff640 1 -- 172.21.15.64:0/2126014689 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff1b0032050 con 0x55f93798e390 2024-09-21T04:04:55.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.716+0000 7ff1d1282740 10 client.15759 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:55.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.716+0000 7ff1d1282740 1 -- 172.21.15.64:0/2126014689 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f937d706f0 con 0x55f93798e390 2024-09-21T04:04:55.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.717+0000 7ff1a3fff640 1 -- 172.21.15.64:0/2126014689 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1b000faf0 con 0x55f93798e390 2024-09-21T04:04:55.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.717+0000 7ff1a3fff640 1 -- 172.21.15.64:0/2126014689 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff1b0004ab0 con 0x55f93798e390 2024-09-21T04:04:55.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.717+0000 7ff1d1282740 10 client.15759 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:55.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.717+0000 7ff1d1282740 10 client.15759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:04:55.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.717+0000 7ff1d1282740 20 client.15759 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:55.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.717+0000 7ff1d1282740 1 --2- 172.21.15.64:0/2126014689 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f93796d9f0 0x55f937d73a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:55.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.717+0000 7ff1d1282740 4 client.15759 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:04:55.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.717+0000 7ff1d1282740 1 -- 172.21.15.64:0/2126014689 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f937cceda0 con 0x55f93796d9f0 2024-09-21T04:04:55.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.717+0000 7ff1c8aaa640 1 --2- 172.21.15.64:0/2126014689 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f93796d9f0 0x55f937d73a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:55.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.718+0000 7ff1a3fff640 10 client.15759 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:55.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.718+0000 7ff1c8aaa640 1 --2- 172.21.15.64:0/2126014689 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f93796d9f0 0x55f937d73a50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29fc9c66e70e9924 server_cookie=9b1726702d5f18a6 in_seq=0 out_seq=0 2024-09-21T04:04:55.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.719+0000 7ff1a3fff640 1 -- 172.21.15.64:0/2126014689 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f937cceda0 con 0x55f93796d9f0 2024-09-21T04:04:55.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.719+0000 7ff1a3fff640 10 client.15759 handle_command_reply: tid=0 2024-09-21T04:04:55.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.719+0000 7ff1d1282740 1 client.15759 shutdown 2024-09-21T04:04:55.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.719+0000 7ff1d1282740 1 -- 172.21.15.64:0/2126014689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f93798e390 msgr2=0x55f937d61d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:55.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.719+0000 7ff1d1282740 1 --2- 172.21.15.64:0/2126014689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f93798e390 0x55f937d61d10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0019770 tx=0x7ff1b00095a0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.719+0000 7ff1d1282740 1 -- 172.21.15.64:0/2126014689 shutdown_connections 2024-09-21T04:04:55.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.719+0000 7ff1d1282740 1 --2- 172.21.15.64:0/2126014689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9379af070 0x55f937d62250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.719+0000 7ff1d1282740 1 --2- 172.21.15.64:0/2126014689 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f93796d9f0 0x55f937d73a50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.719+0000 7ff1d1282740 1 --2- 172.21.15.64:0/2126014689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f93798e390 0x55f937d61d10 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.719+0000 7ff1d1282740 1 --2- 172.21.15.64:0/2126014689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9379861b0 0x55f937d617d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.719+0000 7ff1d1282740 1 -- 172.21.15.64:0/2126014689 >> 172.21.15.64:0/2126014689 conn(0x55f9379a3460 msgr2=0x55f937d59450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:55.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.719+0000 7ff1a3fff640 0 client.15759 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:55.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.720+0000 7ff1d1282740 1 -- 172.21.15.64:0/2126014689 shutdown_connections 2024-09-21T04:04:55.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.720+0000 7ff1d1282740 1 -- 172.21.15.64:0/2126014689 wait complete. 2024-09-21T04:04:55.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.720+0000 7ff1d1282740 20 client.15759 trim_cache size 0 max 16384 2024-09-21T04:04:55.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.756+0000 7ff1d1282740 1 Processor -- start 2024-09-21T04:04:55.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.756+0000 7ff1d1282740 1 -- start start 2024-09-21T04:04:55.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.756+0000 7ff1d1282740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9379861b0 0x55f937d57d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:55.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.756+0000 7ff1d1282740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f93798e390 0x55f937d636b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:55.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.756+0000 7ff1c8aaa640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9379861b0 0x55f937d57d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:55.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.756+0000 7ff1c8aaa640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9379861b0 0x55f937d57d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34476/0 (socket says 172.21.15.64:34476) 2024-09-21T04:04:55.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.756+0000 7ff1c8aaa640 1 -- 172.21.15.64:0/1206902392 learned_addr learned my addr 172.21.15.64:0/1206902392 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:55.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.756+0000 7ff1c3fff640 1 --2- 172.21.15.64:0/1206902392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f93798e390 0x55f937d636b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:55.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.756+0000 7ff1d1282740 1 --2- 172.21.15.64:0/1206902392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9379af070 0x55f937d6d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:55.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.757+0000 7ff1d1282740 1 -- 172.21.15.64:0/1206902392 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c41b0060 con 0x55f9379861b0 2024-09-21T04:04:55.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.757+0000 7ff1d1282740 1 -- 172.21.15.64:0/1206902392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c41ad930 con 0x55f93798e390 2024-09-21T04:04:55.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.757+0000 7ff1d1282740 1 -- 172.21.15.64:0/1206902392 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c41ab780 con 0x55f9379af070 2024-09-21T04:04:55.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.757+0000 7ff1c92ab640 1 --2- 172.21.15.64:0/1206902392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9379af070 0x55f937d6d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:55.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.757+0000 7ff1c3fff640 1 -- 172.21.15.64:0/1206902392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9379af070 msgr2=0x55f937d6d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:55.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.757+0000 7ff1c3fff640 1 --2- 172.21.15.64:0/1206902392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9379af070 0x55f937d6d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.757+0000 7ff1c3fff640 1 -- 172.21.15.64:0/1206902392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9379861b0 msgr2=0x55f937d57d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:55.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.757+0000 7ff1c3fff640 1 --2- 172.21.15.64:0/1206902392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9379861b0 0x55f937d57d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.757+0000 7ff1c3fff640 1 -- 172.21.15.64:0/1206902392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f937d58560 con 0x55f93798e390 2024-09-21T04:04:55.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.757+0000 7ff1c3fff640 1 --2- 172.21.15.64:0/1206902392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f93798e390 0x55f937d636b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0004f00 tx=0x7ff1b0011a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a387be4bbbabe37 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:55.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.758+0000 7ff1ae7fc640 1 -- 172.21.15.64:0/1206902392 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0009d40 con 0x55f93798e390 2024-09-21T04:04:55.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.758+0000 7ff1ae7fc640 1 -- 172.21.15.64:0/1206902392 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1b0029070 con 0x55f93798e390 2024-09-21T04:04:55.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.758+0000 7ff1ae7fc640 1 -- 172.21.15.64:0/1206902392 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0004680 con 0x55f93798e390 2024-09-21T04:04:55.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.759+0000 7ff1d1282740 1 -- 172.21.15.64:0/1206902392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f93798e390 msgr2=0x55f937d636b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:55.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.759+0000 7ff1d1282740 1 --2- 172.21.15.64:0/1206902392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f93798e390 0x55f937d636b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0004f00 tx=0x7ff1b0011a30 comp rx=0 tx=0).stop 2024-09-21T04:04:55.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.759+0000 7ff1d1282740 1 -- 172.21.15.64:0/1206902392 shutdown_connections 2024-09-21T04:04:55.762 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.759+0000 7ff1d1282740 1 --2- 172.21.15.64:0/1206902392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9379af070 0x55f937d6d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.759+0000 7ff1d1282740 1 --2- 172.21.15.64:0/1206902392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f93798e390 0x55f937d636b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.759+0000 7ff1d1282740 1 --2- 172.21.15.64:0/1206902392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f9379861b0 0x55f937d57d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.759+0000 7ff1d1282740 1 -- 172.21.15.64:0/1206902392 >> 172.21.15.64:0/1206902392 conn(0x55f9379a3460 msgr2=0x55f937d6cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:55.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.759+0000 7ff1d1282740 1 -- 172.21.15.64:0/1206902392 shutdown_connections 2024-09-21T04:04:55.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.759+0000 7ff1d1282740 1 -- 172.21.15.64:0/1206902392 wait complete. 2024-09-21T04:04:55.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.760+0000 7ff1d1282740 1 -- start start 2024-09-21T04:04:55.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.760+0000 7ff1d1282740 1 Processor -- start 2024-09-21T04:04:55.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.761+0000 7ff1d1282740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9379861b0 0x55f937d65f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:55.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.761+0000 7ff1d1282740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f93798e390 0x55f937d66450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:55.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.761+0000 7ff1c3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f93798e390 0x55f937d66450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:55.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.761+0000 7ff1c3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f93798e390 0x55f937d66450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34478/0 (socket says 172.21.15.64:34478) 2024-09-21T04:04:55.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.761+0000 7ff1d1282740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9379af070 0x55f937d66990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:55.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.761+0000 7ff1c8aaa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9379861b0 0x55f937d65f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:55.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.761+0000 7ff1c3fff640 1 -- 172.21.15.64:0/4267242608 learned_addr learned my addr 172.21.15.64:0/4267242608 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:04:55.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.762+0000 7ff1d1282740 1 -- 172.21.15.64:0/4267242608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c41abdf0 con 0x55f93798e390 2024-09-21T04:04:55.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.762+0000 7ff1d1282740 1 -- 172.21.15.64:0/4267242608 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c41ad260 con 0x55f9379861b0 2024-09-21T04:04:55.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.762+0000 7ff1d1282740 1 -- 172.21.15.64:0/4267242608 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c41ac4c0 con 0x55f9379af070 2024-09-21T04:04:55.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.762+0000 7ff1c92ab640 1 --2- 172.21.15.64:0/4267242608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9379af070 0x55f937d66990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:55.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.762+0000 7ff1c3fff640 1 -- 172.21.15.64:0/4267242608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9379af070 msgr2=0x55f937d66990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:55.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.762+0000 7ff1c3fff640 1 --2- 172.21.15.64:0/4267242608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9379af070 0x55f937d66990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.762+0000 7ff1c3fff640 1 -- 172.21.15.64:0/4267242608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9379861b0 msgr2=0x55f937d65f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:04:55.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.762+0000 7ff1c3fff640 1 --2- 172.21.15.64:0/4267242608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9379861b0 0x55f937d65f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.762+0000 7ff1c3fff640 1 -- 172.21.15.64:0/4267242608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f937ddd4e0 con 0x55f93798e390 2024-09-21T04:04:55.765 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.762+0000 7ff1c92ab640 1 --2- 172.21.15.64:0/4267242608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9379af070 0x55f937d66990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:04:55.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.763+0000 7ff1c3fff640 1 --2- 172.21.15.64:0/4267242608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f93798e390 0x55f937d66450 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0011570 tx=0x7ff1b00244e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c620edc6221257 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:04:55.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.763+0000 7ff1a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:04:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.763+0000 7ff1a3fff640 1 -- 172.21.15.64:0/4267242608 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0046060 con 0x55f93798e390 2024-09-21T04:04:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.763+0000 7ff1d1282740 10 client.0 _ll_register_callbacks cb 0x7ff1b001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:04:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.763+0000 7ff1a3fff640 1 -- 172.21.15.64:0/4267242608 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7ff1b00248f0 con 0x55f93798e390 2024-09-21T04:04:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.763+0000 7ff1d1282740 1 -- 172.21.15.64:0/4267242608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1b0029070 con 0x55f93798e390 2024-09-21T04:04:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.763+0000 7ff1a3fff640 1 -- 172.21.15.64:0/4267242608 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff1b00197a0 con 0x55f93798e390 2024-09-21T04:04:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.763+0000 7ff1d1282740 10 client.15765 fetch_fsmap learned FSMap version 25 2024-09-21T04:04:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.763+0000 7ff1d1282740 1 -- 172.21.15.64:0/4267242608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55f937dde4d0 con 0x55f93798e390 2024-09-21T04:04:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.764+0000 7ff1a3fff640 1 -- 172.21.15.64:0/4267242608 <== mon.0 v2:172.21.15.64:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff1b0024af0 con 0x55f93798e390 2024-09-21T04:04:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.764+0000 7ff1d1282740 10 client.15765 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:04:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.764+0000 7ff1d1282740 10 client.15765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:04:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.764+0000 7ff1d1282740 20 client.15765 populate_metadata read hostname 'smithi064' 2024-09-21T04:04:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.764+0000 7ff1d1282740 1 --2- 172.21.15.64:0/4267242608 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f937942da0 0x55f937de0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:04:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.764+0000 7ff1d1282740 4 client.15765 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T04:04:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.764+0000 7ff1d1282740 1 -- 172.21.15.64:0/4267242608 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7ff1b0029070 con 0x55f937942da0 2024-09-21T04:04:55.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.764+0000 7ff1c8aaa640 1 --2- 172.21.15.64:0/4267242608 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f937942da0 0x55f937de0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:04:55.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.765+0000 7ff1a3fff640 10 client.15765 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:55.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.765+0000 7ff1c8aaa640 1 --2- 172.21.15.64:0/4267242608 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f937942da0 0x55f937de0ac0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13e4bb17cbdbb2ff server_cookie=d12fc8afeb8703e in_seq=0 out_seq=0 2024-09-21T04:04:55.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.765+0000 7ff1a3fff640 1 -- 172.21.15.64:0/4267242608 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1138 (crc 0 0 0) 0x7ff1b0029070 con 0x55f937942da0 2024-09-21T04:04:55.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.765+0000 7ff1a3fff640 10 client.15765 handle_command_reply: tid=0 2024-09-21T04:04:55.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.765+0000 7ff1d1282740 1 client.15765 shutdown 2024-09-21T04:04:55.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.765+0000 7ff1d1282740 1 -- 172.21.15.64:0/4267242608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f93798e390 msgr2=0x55f937d66450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:55.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.765+0000 7ff1d1282740 1 --2- 172.21.15.64:0/4267242608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f93798e390 0x55f937d66450 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0011570 tx=0x7ff1b00244e0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.765+0000 7ff1d1282740 1 -- 172.21.15.64:0/4267242608 shutdown_connections 2024-09-21T04:04:55.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.765+0000 7ff1d1282740 1 --2- 172.21.15.64:0/4267242608 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9379af070 0x55f937d66990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.765+0000 7ff1d1282740 1 --2- 172.21.15.64:0/4267242608 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f937942da0 0x55f937de0ac0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.765+0000 7ff1a3fff640 0 client.15765 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:04:55.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.765+0000 7ff1d1282740 1 --2- 172.21.15.64:0/4267242608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f93798e390 0x55f937d66450 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.765+0000 7ff1d1282740 1 --2- 172.21.15.64:0/4267242608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f9379861b0 0x55f937d65f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.765+0000 7ff1d1282740 1 -- 172.21.15.64:0/4267242608 >> 172.21.15.64:0/4267242608 conn(0x55f9379a3460 msgr2=0x55f937d6d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:55.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.766+0000 7ff1d1282740 1 -- 172.21.15.64:0/4267242608 shutdown_connections 2024-09-21T04:04:55.770 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.766+0000 7ff1d1282740 1 -- 172.21.15.64:0/4267242608 wait complete. 2024-09-21T04:04:55.771 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.766+0000 7ff1d1282740 20 client.15765 trim_cache size 0 max 16384 2024-09-21T04:04:55.771 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:04:55.771 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T04:04:55.771 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-09-21T04:04:55.771 INFO:teuthology.orchestra.run.smithi064.stdout: "b52b950c-0f97-49ec-a37c-1a8e04348ce8": { 2024-09-21T04:04:55.771 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:04:55.771 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "b52b950c-0f97-49ec-a37c-1a8e04348ce8", 2024-09-21T04:04:55.771 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-09-21T04:04:55.771 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:04:55.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:04:55.771 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_stats": { 2024-09-21T04:04:55.771 INFO:teuthology.orchestra.run.smithi064.stdout: "b52b950c-0f97-49ec-a37c-1a8e04348ce8": { 2024-09-21T04:04:55.772 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:54.003800+0000", 2024-09-21T04:04:55.772 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:04:55.772 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:04:55.772 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:04:55.772 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:04:55.772 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:04:55.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:04:55.772 INFO:teuthology.orchestra.run.smithi064.stdout: "0ab717a7-36c6-4dca-94e7-af0bdf2aa112": { 2024-09-21T04:04:55.772 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:42.691086+0000", 2024-09-21T04:04:55.772 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:04:55.772 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:04:55.772 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:04:55.772 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:04:55.772 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:04:55.773 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:04:55.773 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:04:55.773 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:03:59.570174+0000", 2024-09-21T04:04:55.773 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:04:55.773 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:04:55.773 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:04:55.773 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:04:55.773 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:04:55.773 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:04:55.773 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:04:55.773 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:04:55.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.767+0000 7ff1cad0e640 1 -- 172.21.15.64:0/972385877 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff1b40b6710 msgr2=0x7ff1b40b8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:55.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.767+0000 7ff1cad0e640 1 --2- 172.21.15.64:0/972385877 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff1b40b6710 0x7ff1b40b8c20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc006010 tx=0x7ff1bc02a8c0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.767+0000 7ff1cad0e640 1 -- 172.21.15.64:0/972385877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1c4102ae0 msgr2=0x7ff1c419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:04:55.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.767+0000 7ff1cad0e640 1 --2- 172.21.15.64:0/972385877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1c4102ae0 0x7ff1c419a440 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800e9f0 tx=0x7ff1b800eeb0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.768+0000 7ff1cad0e640 1 -- 172.21.15.64:0/972385877 shutdown_connections 2024-09-21T04:04:55.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.768+0000 7ff1cad0e640 1 --2- 172.21.15.64:0/972385877 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff1b40b6710 0x7ff1b40b8c20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.768+0000 7ff1cad0e640 1 --2- 172.21.15.64:0/972385877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff1c4108d20 0x7ff1c419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.768+0000 7ff1cad0e640 1 --2- 172.21.15.64:0/972385877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff1c41083d0 0x7ff1c419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.768+0000 7ff1cad0e640 1 --2- 172.21.15.64:0/972385877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1c4102ae0 0x7ff1c419a440 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:04:55.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.768+0000 7ff1cad0e640 1 -- 172.21.15.64:0/972385877 >> 172.21.15.64:0/972385877 conn(0x7ff1c4079760 msgr2=0x7ff1c4105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:04:55.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.768+0000 7ff1cad0e640 1 -- 172.21.15.64:0/972385877 shutdown_connections 2024-09-21T04:04:55.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:04:55.768+0000 7ff1cad0e640 1 -- 172.21.15.64:0/972385877 wait complete. 2024-09-21T04:04:56.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:55 smithi119 ceph-mon[34811]: scrub queued for path: / 2024-09-21T04:04:56.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:55 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:04:56.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:55 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:04:56.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:55 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:04:56.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:55 smithi119 ceph-mon[34811]: pgmap v348: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T04:04:56.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:55 smithi064 ceph-mon[31180]: scrub queued for path: / 2024-09-21T04:04:56.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:55 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:04:56.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:55 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:04:56.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:55 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:04:56.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:55 smithi064 ceph-mon[31180]: pgmap v348: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T04:04:56.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:55 smithi131 ceph-mon[35193]: scrub queued for path: / 2024-09-21T04:04:56.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:55 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:04:56.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:55 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:04:56.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:55 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:04:56.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:55 smithi131 ceph-mon[35193]: pgmap v348: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T04:04:56.540 INFO:tasks.cephfs.filesystem:scrub status for tag:b52b950c-0f97-49ec-a37c-1a8e04348ce8 - {'path': '/', 'tag': 'b52b950c-0f97-49ec-a37c-1a8e04348ce8', 'options': 'recursive,force'} 2024-09-21T04:04:58.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:57 smithi119 ceph-mon[34811]: pgmap v349: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-21T04:04:58.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:57 smithi064 ceph-mon[31180]: pgmap v349: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-21T04:04:58.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:57 smithi131 ceph-mon[35193]: pgmap v349: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-21T04:04:59.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:58 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:04:59.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:58 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:04:59.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:58 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:05:00.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:04:59 smithi119 ceph-mon[34811]: pgmap v350: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-21T04:05:00.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:04:59 smithi064 ceph-mon[31180]: pgmap v350: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-21T04:05:00.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:04:59 smithi131 ceph-mon[35193]: pgmap v350: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-21T04:05:02.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:01 smithi119 ceph-mon[34811]: pgmap v351: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T04:05:02.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:01 smithi119 ceph-mon[34811]: mgrmap e26: x(active, since 9m), standbys: z, y 2024-09-21T04:05:02.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:01 smithi064 ceph-mon[31180]: pgmap v351: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T04:05:02.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:01 smithi064 ceph-mon[31180]: mgrmap e26: x(active, since 9m), standbys: z, y 2024-09-21T04:05:02.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:01 smithi131 ceph-mon[35193]: pgmap v351: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T04:05:02.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:01 smithi131 ceph-mon[35193]: mgrmap e26: x(active, since 9m), standbys: z, y 2024-09-21T04:05:04.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:03 smithi119 ceph-mon[34811]: pgmap v352: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-21T04:05:04.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:03 smithi064 ceph-mon[31180]: pgmap v352: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-21T04:05:04.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:03 smithi131 ceph-mon[35193]: pgmap v352: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-21T04:05:06.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:05 smithi119 ceph-mon[34811]: pgmap v353: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-21T04:05:06.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:05 smithi064 ceph-mon[31180]: pgmap v353: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-21T04:05:06.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:05 smithi131 ceph-mon[35193]: pgmap v353: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-21T04:05:06.260 INFO:tasks.workunit.client.0.smithi064.stderr:Updating files: 92% (11745/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-21T04:05:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.422+0000 7f6d2cf39640 1 -- 172.21.15.64:0/2697097203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d28109d80 msgr2=0x7f6d281105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.422+0000 7f6d2cf39640 1 --2- 172.21.15.64:0/2697097203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d28109d80 0x7f6d281105b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6d20009ca0 tx=0x7f6d2002a8c0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.422+0000 7f6d2cf39640 1 -- 172.21.15.64:0/2697097203 shutdown_connections 2024-09-21T04:05:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.422+0000 7f6d2cf39640 1 --2- 172.21.15.64:0/2697097203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d28109d80 0x7f6d281105b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.422+0000 7f6d2cf39640 1 --2- 172.21.15.64:0/2697097203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d28109430 0x7f6d28109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.422+0000 7f6d2cf39640 1 --2- 172.21.15.64:0/2697097203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d28102060 0x7f6d28102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.422+0000 7f6d2cf39640 1 -- 172.21.15.64:0/2697097203 >> 172.21.15.64:0/2697097203 conn(0x7f6d28079e90 msgr2=0x7f6d2807a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.423+0000 7f6d2cf39640 1 -- 172.21.15.64:0/2697097203 shutdown_connections 2024-09-21T04:05:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.423+0000 7f6d2cf39640 1 -- 172.21.15.64:0/2697097203 wait complete. 2024-09-21T04:05:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.423+0000 7f6d2cf39640 1 Processor -- start 2024-09-21T04:05:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.423+0000 7f6d2cf39640 1 -- start start 2024-09-21T04:05:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.424+0000 7f6d2cf39640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d28102060 0x7f6d2819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.424+0000 7f6d2cf39640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d28109430 0x7f6d2819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.424+0000 7f6d2cf39640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d28109d80 0x7f6d2819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.424+0000 7f6d2cf39640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6d28113a40 con 0x7f6d28109d80 2024-09-21T04:05:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.424+0000 7f6d2cf39640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281138c0 con 0x7f6d28109430 2024-09-21T04:05:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.424+0000 7f6d2cf39640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6d28113bc0 con 0x7f6d28102060 2024-09-21T04:05:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.424+0000 7f6d26d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d28109d80 0x7f6d2819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.424+0000 7f6d26d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d28109d80 0x7f6d2819fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37212/0 (socket says 172.21.15.64:37212) 2024-09-21T04:05:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.424+0000 7f6d26d9d640 1 -- 172.21.15.64:0/3190437920 learned_addr learned my addr 172.21.15.64:0/3190437920 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.424+0000 7f6d2659c640 1 --2- 172.21.15.64:0/3190437920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d28102060 0x7f6d2819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.424+0000 7f6d25d9b640 1 --2- 172.21.15.64:0/3190437920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d28109430 0x7f6d2819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.424+0000 7f6d26d9d640 1 -- 172.21.15.64:0/3190437920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d28102060 msgr2=0x7f6d2819a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.424+0000 7f6d26d9d640 1 --2- 172.21.15.64:0/3190437920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d28102060 0x7f6d2819a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.424+0000 7f6d26d9d640 1 -- 172.21.15.64:0/3190437920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d28109430 msgr2=0x7f6d2819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.424+0000 7f6d26d9d640 1 --2- 172.21.15.64:0/3190437920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d28109430 0x7f6d2819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.424+0000 7f6d26d9d640 1 -- 172.21.15.64:0/3190437920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d281a0350 con 0x7f6d28109d80 2024-09-21T04:05:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.424+0000 7f6d26d9d640 1 --2- 172.21.15.64:0/3190437920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d28109d80 0x7f6d2819fc50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6d20005cc0 tx=0x7f6d2002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.425+0000 7f6d177fe640 1 -- 172.21.15.64:0/3190437920 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d20003bf0 con 0x7f6d28109d80 2024-09-21T04:05:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.425+0000 7f6d2cf39640 1 -- 172.21.15.64:0/3190437920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d281a05e0 con 0x7f6d28109d80 2024-09-21T04:05:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.425+0000 7f6d177fe640 1 -- 172.21.15.64:0/3190437920 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d20003d90 con 0x7f6d28109d80 2024-09-21T04:05:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.425+0000 7f6d177fe640 1 -- 172.21.15.64:0/3190437920 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2002fbc0 con 0x7f6d28109d80 2024-09-21T04:05:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.425+0000 7f6d2cf39640 1 -- 172.21.15.64:0/3190437920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d281a6dd0 con 0x7f6d28109d80 2024-09-21T04:05:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.427+0000 7f6d33eaa740 1 Processor -- start 2024-09-21T04:05:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.427+0000 7f6d177fe640 1 -- 172.21.15.64:0/3190437920 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f6d2002fd60 con 0x7f6d28109d80 2024-09-21T04:05:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.428+0000 7f6d33eaa740 1 -- start start 2024-09-21T04:05:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.428+0000 7f6d33eaa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55926ffba1b0 0x55927035f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.428+0000 7f6d33eaa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55926ff912f0 0x55927035fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.428+0000 7f6d33eaa740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55926ff994d0 0x559270360430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.428+0000 7f6d33eaa740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5592702d46f0 con 0x55926ffba1b0 2024-09-21T04:05:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.428+0000 7f6d33eaa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5592702c3b20 con 0x55926ff912f0 2024-09-21T04:05:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.428+0000 7f6d33eaa740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5592702c7930 con 0x55926ff994d0 2024-09-21T04:05:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.428+0000 7f6d25d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55926ffba1b0 0x55927035f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.428+0000 7f6d25d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55926ffba1b0 0x55927035f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37218/0 (socket says 172.21.15.64:37218) 2024-09-21T04:05:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.428+0000 7f6d25d9b640 1 -- 172.21.15.64:0/790551981 learned_addr learned my addr 172.21.15.64:0/790551981 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.428+0000 7f6d2659c640 1 --2- 172.21.15.64:0/790551981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55926ff912f0 0x55927035fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.429+0000 7f6d25d9b640 1 --2- 172.21.15.64:0/790551981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55926ff994d0 0x559270360430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.429+0000 7f6d177fe640 1 --2- 172.21.15.64:0/3190437920 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6d100b6680 0x7f6d100b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.429+0000 7f6d25d9b640 1 -- 172.21.15.64:0/790551981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55926ff994d0 msgr2=0x559270360430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.429+0000 7f6d25d9b640 1 --2- 172.21.15.64:0/790551981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55926ff994d0 0x559270360430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.429+0000 7f6d25d9b640 1 -- 172.21.15.64:0/790551981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55926ff912f0 msgr2=0x55927035fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.429+0000 7f6d25d9b640 1 --2- 172.21.15.64:0/790551981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55926ff912f0 0x55927035fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.429+0000 7f6d25d9b640 1 -- 172.21.15.64:0/790551981 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559270289530 con 0x55926ffba1b0 2024-09-21T04:05:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.429+0000 7f6d26d9d640 1 --2- 172.21.15.64:0/3190437920 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6d100b6680 0x7f6d100b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.429+0000 7f6d177fe640 1 -- 172.21.15.64:0/3190437920 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f6d20108c90 con 0x7f6d28109d80 2024-09-21T04:05:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.429+0000 7f6d25d9b640 1 --2- 172.21.15.64:0/790551981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55926ffba1b0 0x55927035f9b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00bfa0 tx=0x7f6d1c00b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8cda088a4cd1a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.429+0000 7f6d14ff9640 1 -- 172.21.15.64:0/790551981 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c019030 con 0x55926ffba1b0 2024-09-21T04:05:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.429+0000 7f6d14ff9640 1 -- 172.21.15.64:0/790551981 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d1c0118f0 con 0x55926ffba1b0 2024-09-21T04:05:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.429+0000 7f6d14ff9640 1 -- 172.21.15.64:0/790551981 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c00f6b0 con 0x55926ffba1b0 2024-09-21T04:05:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.430+0000 7f6d33eaa740 1 -- 172.21.15.64:0/790551981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55926ffba1b0 msgr2=0x55927035f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.430+0000 7f6d26d9d640 1 --2- 172.21.15.64:0/3190437920 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6d100b6680 0x7f6d100b8b90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6d20106460 tx=0x7f6d2009b870 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.430+0000 7f6d33eaa740 1 --2- 172.21.15.64:0/790551981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55926ffba1b0 0x55927035f9b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00bfa0 tx=0x7f6d1c00b650 comp rx=0 tx=0).stop 2024-09-21T04:05:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.430+0000 7f6d33eaa740 1 -- 172.21.15.64:0/790551981 shutdown_connections 2024-09-21T04:05:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.430+0000 7f6d33eaa740 1 --2- 172.21.15.64:0/790551981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55926ff994d0 0x559270360430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.430+0000 7f6d33eaa740 1 --2- 172.21.15.64:0/790551981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55926ff912f0 0x55927035fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.430+0000 7f6d33eaa740 1 --2- 172.21.15.64:0/790551981 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55926ffba1b0 0x55927035f9b0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.430+0000 7f6d33eaa740 1 -- 172.21.15.64:0/790551981 >> 172.21.15.64:0/790551981 conn(0x55926ffae300 msgr2=0x55927035d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.430+0000 7f6d33eaa740 1 -- 172.21.15.64:0/790551981 shutdown_connections 2024-09-21T04:05:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.430+0000 7f6d33eaa740 1 -- 172.21.15.64:0/790551981 wait complete. 2024-09-21T04:05:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.431+0000 7f6d33eaa740 1 -- start start 2024-09-21T04:05:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.431+0000 7f6d33eaa740 1 Processor -- start 2024-09-21T04:05:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.431+0000 7f6d33eaa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55926ff912f0 0x55927036b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.431+0000 7f6d33eaa740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55926ff994d0 0x55927036b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.431+0000 7f6d33eaa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55926ffba1b0 0x559270377420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.431+0000 7f6d33eaa740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5592701fbab0 con 0x55926ffba1b0 2024-09-21T04:05:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.431+0000 7f6d33eaa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559270269730 con 0x55926ff912f0 2024-09-21T04:05:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.431+0000 7f6d33eaa740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55927036e130 con 0x55926ff994d0 2024-09-21T04:05:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.431+0000 7f6d33eaa740 10 client.0 _ll_register_callbacks cb 0x5592702bd6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:05:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.432+0000 7f6d25d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55926ffba1b0 0x559270377420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.432+0000 7f6d25d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55926ffba1b0 0x559270377420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37232/0 (socket says 172.21.15.64:37232) 2024-09-21T04:05:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.432+0000 7f6d25d9b640 1 -- 172.21.15.64:0/3730076618 learned_addr learned my addr 172.21.15.64:0/3730076618 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.432+0000 7f6d25d9b640 1 --2- 172.21.15.64:0/3730076618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55926ff912f0 0x55927036b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.432+0000 7f6d2659c640 1 --2- 172.21.15.64:0/3730076618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55926ff994d0 0x55927036b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.432+0000 7f6d25d9b640 1 -- 172.21.15.64:0/3730076618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55926ff994d0 msgr2=0x55927036b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.432+0000 7f6d25d9b640 1 --2- 172.21.15.64:0/3730076618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55926ff994d0 0x55927036b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.432+0000 7f6d25d9b640 1 -- 172.21.15.64:0/3730076618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55926ff912f0 msgr2=0x55927036b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.432+0000 7f6d25d9b640 1 --2- 172.21.15.64:0/3730076618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55926ff912f0 0x55927036b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.432+0000 7f6d25d9b640 1 -- 172.21.15.64:0/3730076618 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559270025790 con 0x55926ffba1b0 2024-09-21T04:05:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.432+0000 7f6d25d9b640 1 --2- 172.21.15.64:0/3730076618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55926ffba1b0 0x559270377420 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x559270362080 tx=0x7f6d1c005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2622f312687d6544 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.432+0000 7f6d0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:05:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.432+0000 7f6d0dffb640 1 -- 172.21.15.64:0/3730076618 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c019040 con 0x55926ffba1b0 2024-09-21T04:05:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.432+0000 7f6d0dffb640 1 -- 172.21.15.64:0/3730076618 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f6d1c011c50 con 0x55926ffba1b0 2024-09-21T04:05:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.432+0000 7f6d33eaa740 1 -- 172.21.15.64:0/3730076618 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d1c0118f0 con 0x55926ffba1b0 2024-09-21T04:05:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.433+0000 7f6d0dffb640 1 -- 172.21.15.64:0/3730076618 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6d1c011e50 con 0x55926ffba1b0 2024-09-21T04:05:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.433+0000 7f6d33eaa740 10 client.15777 fetch_fsmap learned FSMap version 25 2024-09-21T04:05:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.433+0000 7f6d33eaa740 1 -- 172.21.15.64:0/3730076618 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55927009e620 con 0x55926ffba1b0 2024-09-21T04:05:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.433+0000 7f6d0dffb640 1 -- 172.21.15.64:0/3730076618 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d1c0118f0 con 0x55926ffba1b0 2024-09-21T04:05:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.433+0000 7f6d0dffb640 1 -- 172.21.15.64:0/3730076618 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6d1c02b020 con 0x55926ffba1b0 2024-09-21T04:05:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.433+0000 7f6d33eaa740 10 client.15777 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:05:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.433+0000 7f6d33eaa740 10 client.15777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:05:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.433+0000 7f6d33eaa740 20 client.15777 populate_metadata read hostname 'smithi064' 2024-09-21T04:05:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.433+0000 7f6d33eaa740 1 --2- 172.21.15.64:0/3730076618 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55926ff78d80 0x55927037d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.433+0000 7f6d33eaa740 4 client.15777 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:05:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.433+0000 7f6d33eaa740 1 -- 172.21.15.64:0/3730076618 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592702d6640 con 0x55926ff78d80 2024-09-21T04:05:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.434+0000 7f6d2659c640 1 --2- 172.21.15.64:0/3730076618 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55926ff78d80 0x55927037d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.434+0000 7f6d0dffb640 10 client.15777 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.434+0000 7f6d2659c640 1 --2- 172.21.15.64:0/3730076618 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55926ff78d80 0x55927037d0e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c16ae7139d3e65e server_cookie=5a9b32bf9bd962e in_seq=0 out_seq=0 2024-09-21T04:05:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.435+0000 7f6d0dffb640 1 -- 172.21.15.64:0/3730076618 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5592702d6640 con 0x55926ff78d80 2024-09-21T04:05:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.435+0000 7f6d0dffb640 10 client.15777 handle_command_reply: tid=0 2024-09-21T04:05:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.435+0000 7f6d33eaa740 1 client.15777 shutdown 2024-09-21T04:05:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.435+0000 7f6d33eaa740 1 -- 172.21.15.64:0/3730076618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55926ffba1b0 msgr2=0x559270377420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.435+0000 7f6d33eaa740 1 --2- 172.21.15.64:0/3730076618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55926ffba1b0 0x559270377420 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x559270362080 tx=0x7f6d1c005c90 comp rx=0 tx=0).stop 2024-09-21T04:05:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.435+0000 7f6d33eaa740 1 -- 172.21.15.64:0/3730076618 shutdown_connections 2024-09-21T04:05:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.435+0000 7f6d33eaa740 1 --2- 172.21.15.64:0/3730076618 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55926ff78d80 0x55927037d0e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.435+0000 7f6d33eaa740 1 --2- 172.21.15.64:0/3730076618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55926ffba1b0 0x559270377420 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.435+0000 7f6d33eaa740 1 --2- 172.21.15.64:0/3730076618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55926ff994d0 0x55927036b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.435+0000 7f6d0dffb640 0 client.15777 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.435+0000 7f6d33eaa740 1 --2- 172.21.15.64:0/3730076618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55926ff912f0 0x55927036b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.435+0000 7f6d33eaa740 1 -- 172.21.15.64:0/3730076618 >> 172.21.15.64:0/3730076618 conn(0x55926ffae300 msgr2=0x5592703609e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.435+0000 7f6d33eaa740 1 -- 172.21.15.64:0/3730076618 shutdown_connections 2024-09-21T04:05:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.435+0000 7f6d33eaa740 1 -- 172.21.15.64:0/3730076618 wait complete. 2024-09-21T04:05:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.435+0000 7f6d33eaa740 20 client.15777 trim_cache size 0 max 16384 2024-09-21T04:05:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.463+0000 7f6d33eaa740 1 Processor -- start 2024-09-21T04:05:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.463+0000 7f6d33eaa740 1 -- start start 2024-09-21T04:05:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.463+0000 7f6d33eaa740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55926ff912f0 0x55927035c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.463+0000 7f6d33eaa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55926ff994d0 0x5592703610f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.463+0000 7f6d33eaa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55926ffba1b0 0x559270368220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.463+0000 7f6d33eaa740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281b2040 con 0x55926ff994d0 2024-09-21T04:05:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.463+0000 7f6d33eaa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281af910 con 0x55926ffba1b0 2024-09-21T04:05:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.463+0000 7f6d33eaa740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281ad760 con 0x55926ff912f0 2024-09-21T04:05:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.463+0000 7f6d2659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55926ff994d0 0x5592703610f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.464+0000 7f6d2659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55926ff994d0 0x5592703610f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37246/0 (socket says 172.21.15.64:37246) 2024-09-21T04:05:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.464+0000 7f6d2659c640 1 -- 172.21.15.64:0/3101806782 learned_addr learned my addr 172.21.15.64:0/3101806782 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.464+0000 7f6d25d9b640 1 --2- 172.21.15.64:0/3101806782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55926ff912f0 0x55927035c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.464+0000 7f6d25d9b640 1 --2- 172.21.15.64:0/3101806782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55926ffba1b0 0x559270368220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.464+0000 7f6d2659c640 1 -- 172.21.15.64:0/3101806782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55926ff912f0 msgr2=0x55927035c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.464+0000 7f6d2659c640 1 --2- 172.21.15.64:0/3101806782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55926ff912f0 0x55927035c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.464+0000 7f6d2659c640 1 -- 172.21.15.64:0/3101806782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55926ffba1b0 msgr2=0x559270368220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.464+0000 7f6d2659c640 1 --2- 172.21.15.64:0/3101806782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55926ffba1b0 0x559270368220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.464+0000 7f6d2659c640 1 -- 172.21.15.64:0/3101806782 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559270358f50 con 0x55926ff994d0 2024-09-21T04:05:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.464+0000 7f6d2659c640 1 --2- 172.21.15.64:0/3101806782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55926ff994d0 0x5592703610f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6d1800b880 tx=0x7f6d1800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c378ab0d406c71c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:06.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.464+0000 7f6d0ffff640 1 -- 172.21.15.64:0/3101806782 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18003b20 con 0x55926ff994d0 2024-09-21T04:05:06.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.464+0000 7f6d0ffff640 1 -- 172.21.15.64:0/3101806782 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d180054d0 con 0x55926ff994d0 2024-09-21T04:05:06.469 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.464+0000 7f6d0ffff640 1 -- 172.21.15.64:0/3101806782 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18004e00 con 0x55926ff994d0 2024-09-21T04:05:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.465+0000 7f6d33eaa740 1 -- 172.21.15.64:0/3101806782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55926ff994d0 msgr2=0x5592703610f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.465+0000 7f6d33eaa740 1 --2- 172.21.15.64:0/3101806782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55926ff994d0 0x5592703610f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6d1800b880 tx=0x7f6d1800bd60 comp rx=0 tx=0).stop 2024-09-21T04:05:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.465+0000 7f6d33eaa740 1 -- 172.21.15.64:0/3101806782 shutdown_connections 2024-09-21T04:05:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.465+0000 7f6d33eaa740 1 --2- 172.21.15.64:0/3101806782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55926ffba1b0 0x559270368220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.465+0000 7f6d33eaa740 1 --2- 172.21.15.64:0/3101806782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55926ff994d0 0x5592703610f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.465+0000 7f6d33eaa740 1 --2- 172.21.15.64:0/3101806782 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55926ff912f0 0x55927035c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.465+0000 7f6d33eaa740 1 -- 172.21.15.64:0/3101806782 >> 172.21.15.64:0/3101806782 conn(0x55926ffae300 msgr2=0x5592703609e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.465+0000 7f6d33eaa740 1 -- 172.21.15.64:0/3101806782 shutdown_connections 2024-09-21T04:05:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.465+0000 7f6d33eaa740 1 -- 172.21.15.64:0/3101806782 wait complete. 2024-09-21T04:05:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.466+0000 7f6d33eaa740 1 -- start start 2024-09-21T04:05:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.466+0000 7f6d33eaa740 1 Processor -- start 2024-09-21T04:05:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.466+0000 7f6d33eaa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55926ff912f0 0x5592703707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.466+0000 7f6d33eaa740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55926ff994d0 0x559270370d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.466+0000 7f6d33eaa740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55926ffba1b0 0x5592703e5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.466+0000 7f6d33eaa740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281addd0 con 0x55926ffba1b0 2024-09-21T04:05:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.466+0000 7f6d33eaa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281af240 con 0x55926ff912f0 2024-09-21T04:05:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.466+0000 7f6d33eaa740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281ae4a0 con 0x55926ff994d0 2024-09-21T04:05:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.466+0000 7f6d33eaa740 10 client.0 _ll_register_callbacks cb 0x7f6d1c013400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:05:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.466+0000 7f6d25d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55926ffba1b0 0x5592703e5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.466+0000 7f6d25d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55926ff912f0 0x5592703707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.466+0000 7f6d25d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55926ff912f0 0x5592703707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:49980/0 (socket says 172.21.15.64:49980) 2024-09-21T04:05:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.466+0000 7f6d25d9b640 1 -- 172.21.15.64:0/3887297681 learned_addr learned my addr 172.21.15.64:0/3887297681 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.466+0000 7f6d2659c640 1 --2- 172.21.15.64:0/3887297681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55926ff994d0 0x559270370d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.467+0000 7f6d25d9b640 1 -- 172.21.15.64:0/3887297681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55926ff994d0 msgr2=0x559270370d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.467+0000 7f6d25d9b640 1 --2- 172.21.15.64:0/3887297681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55926ff994d0 0x559270370d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.467+0000 7f6d25d9b640 1 -- 172.21.15.64:0/3887297681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55926ff912f0 msgr2=0x5592703707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.467+0000 7f6d25d9b640 1 --2- 172.21.15.64:0/3887297681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55926ff912f0 0x5592703707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.467+0000 7f6d25d9b640 1 -- 172.21.15.64:0/3887297681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592703e5e50 con 0x55926ffba1b0 2024-09-21T04:05:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.467+0000 7f6d25d9b640 1 --2- 172.21.15.64:0/3887297681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55926ffba1b0 0x5592703e5910 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x5592703742d0 tx=0x7f6d1c005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d450a156a3b137 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.467+0000 7f6d0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:05:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.467+0000 7f6d0dffb640 1 -- 172.21.15.64:0/3887297681 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c019070 con 0x55926ffba1b0 2024-09-21T04:05:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.467+0000 7f6d0dffb640 1 -- 172.21.15.64:0/3887297681 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f6d1c004890 con 0x55926ffba1b0 2024-09-21T04:05:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.467+0000 7f6d33eaa740 1 -- 172.21.15.64:0/3887297681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d180054d0 con 0x55926ffba1b0 2024-09-21T04:05:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.467+0000 7f6d0dffb640 1 -- 172.21.15.64:0/3887297681 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6d1c00b760 con 0x55926ffba1b0 2024-09-21T04:05:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.467+0000 7f6d33eaa740 10 client.15783 fetch_fsmap learned FSMap version 25 2024-09-21T04:05:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.467+0000 7f6d33eaa740 1 -- 172.21.15.64:0/3887297681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5592703e6ff0 con 0x55926ffba1b0 2024-09-21T04:05:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.467+0000 7f6d0dffb640 1 -- 172.21.15.64:0/3887297681 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d180054d0 con 0x55926ffba1b0 2024-09-21T04:05:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.468+0000 7f6d0dffb640 1 -- 172.21.15.64:0/3887297681 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6d1c02b040 con 0x55926ffba1b0 2024-09-21T04:05:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.468+0000 7f6d33eaa740 10 client.15783 fetch_fsmap finished waiting for FSMap version 25 2024-09-21T04:05:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.468+0000 7f6d33eaa740 10 client.15783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:05:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.468+0000 7f6d33eaa740 20 client.15783 populate_metadata read hostname 'smithi064' 2024-09-21T04:05:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.468+0000 7f6d33eaa740 1 --2- 172.21.15.64:0/3887297681 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55926ff4e1c0 0x5592703e9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.468+0000 7f6d33eaa740 4 client.15783 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:05:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.468+0000 7f6d33eaa740 1 -- 172.21.15.64:0/3887297681 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5592702d20f0 con 0x55926ff4e1c0 2024-09-21T04:05:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.468+0000 7f6d2659c640 1 --2- 172.21.15.64:0/3887297681 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55926ff4e1c0 0x5592703e9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.469+0000 7f6d2659c640 1 --2- 172.21.15.64:0/3887297681 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55926ff4e1c0 0x5592703e9700 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c78a35eae78e3419 server_cookie=5a51ad5ae753108c in_seq=0 out_seq=0 2024-09-21T04:05:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.469+0000 7f6d0dffb640 10 client.15783 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:06.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.475+0000 7f6d0dffb640 1 -- 172.21.15.64:0/3887297681 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5592702d20f0 con 0x55926ff4e1c0 2024-09-21T04:05:06.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.475+0000 7f6d0dffb640 10 client.15783 handle_command_reply: tid=0 2024-09-21T04:05:06.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.475+0000 7f6d33eaa740 1 client.15783 shutdown 2024-09-21T04:05:06.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.475+0000 7f6d33eaa740 1 -- 172.21.15.64:0/3887297681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55926ffba1b0 msgr2=0x5592703e5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:06.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.475+0000 7f6d33eaa740 1 --2- 172.21.15.64:0/3887297681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55926ffba1b0 0x5592703e5910 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x5592703742d0 tx=0x7f6d1c005c90 comp rx=0 tx=0).stop 2024-09-21T04:05:06.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.475+0000 7f6d33eaa740 1 -- 172.21.15.64:0/3887297681 shutdown_connections 2024-09-21T04:05:06.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.475+0000 7f6d33eaa740 1 --2- 172.21.15.64:0/3887297681 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55926ff4e1c0 0x5592703e9700 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.475+0000 7f6d33eaa740 1 --2- 172.21.15.64:0/3887297681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55926ffba1b0 0x5592703e5910 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.475+0000 7f6d33eaa740 1 --2- 172.21.15.64:0/3887297681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55926ff994d0 0x559270370d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.478 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.475+0000 7f6d0dffb640 0 client.15783 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:06.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.475+0000 7f6d33eaa740 1 --2- 172.21.15.64:0/3887297681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55926ff912f0 0x5592703707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.475+0000 7f6d33eaa740 1 -- 172.21.15.64:0/3887297681 >> 172.21.15.64:0/3887297681 conn(0x55926ffae300 msgr2=0x55927035c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:06.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.475+0000 7f6d33eaa740 1 -- 172.21.15.64:0/3887297681 shutdown_connections 2024-09-21T04:05:06.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.475+0000 7f6d33eaa740 1 -- 172.21.15.64:0/3887297681 wait complete. 2024-09-21T04:05:06.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.475+0000 7f6d33eaa740 20 client.15783 trim_cache size 0 max 16384 2024-09-21T04:05:06.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:05:06.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:05:06.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:05:06.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:05:06.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.476+0000 7f6d2cf39640 1 -- 172.21.15.64:0/3190437920 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6d100b6680 msgr2=0x7f6d100b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:06.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.476+0000 7f6d2cf39640 1 --2- 172.21.15.64:0/3190437920 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6d100b6680 0x7f6d100b8b90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6d20106460 tx=0x7f6d2009b870 comp rx=0 tx=0).stop 2024-09-21T04:05:06.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.476+0000 7f6d2cf39640 1 -- 172.21.15.64:0/3190437920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d28109d80 msgr2=0x7f6d2819fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:06.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.476+0000 7f6d2cf39640 1 --2- 172.21.15.64:0/3190437920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d28109d80 0x7f6d2819fc50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6d20005cc0 tx=0x7f6d2002f380 comp rx=0 tx=0).stop 2024-09-21T04:05:06.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.476+0000 7f6d2cf39640 1 -- 172.21.15.64:0/3190437920 shutdown_connections 2024-09-21T04:05:06.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.476+0000 7f6d2cf39640 1 --2- 172.21.15.64:0/3190437920 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6d100b6680 0x7f6d100b8b90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.476+0000 7f6d2cf39640 1 --2- 172.21.15.64:0/3190437920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d28109d80 0x7f6d2819fc50 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.480 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.476+0000 7f6d2cf39640 1 --2- 172.21.15.64:0/3190437920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d28109430 0x7f6d2819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.476+0000 7f6d2cf39640 1 --2- 172.21.15.64:0/3190437920 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d28102060 0x7f6d2819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:06.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.476+0000 7f6d2cf39640 1 -- 172.21.15.64:0/3190437920 >> 172.21.15.64:0/3190437920 conn(0x7f6d28079e90 msgr2=0x7f6d28107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:06.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.476+0000 7f6d2cf39640 1 -- 172.21.15.64:0/3190437920 shutdown_connections 2024-09-21T04:05:06.481 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:06.476+0000 7f6d2cf39640 1 -- 172.21.15.64:0/3190437920 wait complete. 2024-09-21T04:05:06.875 INFO:tasks.workunit.client.0.smithi064.stderr:Note: switching to '1843221895adccfb3b692060251dd6a0183bc410'. 2024-09-21T04:05:06.875 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-09-21T04:05:06.875 INFO:tasks.workunit.client.0.smithi064.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-21T04:05:06.875 INFO:tasks.workunit.client.0.smithi064.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-21T04:05:06.875 INFO:tasks.workunit.client.0.smithi064.stderr:state without impacting any branches by switching back to a branch. 2024-09-21T04:05:06.876 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-09-21T04:05:06.876 INFO:tasks.workunit.client.0.smithi064.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-21T04:05:06.876 INFO:tasks.workunit.client.0.smithi064.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-21T04:05:06.876 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-09-21T04:05:06.876 INFO:tasks.workunit.client.0.smithi064.stderr: git switch -c 2024-09-21T04:05:06.876 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-09-21T04:05:06.876 INFO:tasks.workunit.client.0.smithi064.stderr:Or undo this operation with: 2024-09-21T04:05:06.876 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-09-21T04:05:06.876 INFO:tasks.workunit.client.0.smithi064.stderr: git switch - 2024-09-21T04:05:06.876 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-09-21T04:05:06.876 INFO:tasks.workunit.client.0.smithi064.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-21T04:05:06.876 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-09-21T04:05:06.876 INFO:tasks.workunit.client.0.smithi064.stderr:HEAD is now at 1843221895a Merge PR #56776 into wip-pdonnell-testing-20240920.212106-debug 2024-09-21T04:05:06.880 DEBUG:teuthology.orchestra.run.smithi064:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-21T04:05:06.945 INFO:tasks.workunit.client.0.smithi064.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-21T04:05:06.947 INFO:tasks.workunit.client.0.smithi064.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T04:05:06.947 INFO:tasks.workunit.client.0.smithi064.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-21T04:05:07.668 INFO:tasks.workunit.client.0.smithi064.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-21T04:05:07.745 INFO:tasks.workunit.client.0.smithi064.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-21T04:05:07.789 INFO:tasks.workunit.client.0.smithi064.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T04:05:07.790 INFO:tasks.workunit.client.0.smithi064.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T04:05:07.791 INFO:tasks.workunit.client.0.smithi064.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-21T04:05:07.831 INFO:tasks.workunit.client.0.smithi064.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T04:05:07.835 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-21T04:05:07.835 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-21T04:05:07.892 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2024-09-21T04:05:07.893 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2024-09-21T04:05:07.893 DEBUG:teuthology.orchestra.run.smithi064:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1843221895adccfb3b692060251dd6a0183bc410 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2024-09-21T04:05:07.955 INFO:tasks.workunit.client.0.smithi064.stdout:getting blogbench 2024-09-21T04:05:07.955 INFO:tasks.workunit.client.0.smithi064.stderr:+ echo 'getting blogbench' 2024-09-21T04:05:07.956 INFO:tasks.workunit.client.0.smithi064.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-09-21T04:05:07.979 INFO:tasks.workunit.client.0.smithi064.stderr:--2024-09-21 04:05:07-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-09-21T04:05:07.980 INFO:tasks.workunit.client.0.smithi064.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-09-21T04:05:08.001 INFO:tasks.workunit.client.0.smithi064.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-09-21T04:05:08.008 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:07 smithi119 ceph-mon[34811]: pgmap v354: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-21T04:05:08.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:07 smithi064 ceph-mon[31180]: pgmap v354: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-21T04:05:08.022 INFO:tasks.workunit.client.0.smithi064.stderr:HTTP request sent, awaiting response... 200 OK 2024-09-21T04:05:08.022 INFO:tasks.workunit.client.0.smithi064.stderr:Length: 103355 (101K) [application/octet-stream] 2024-09-21T04:05:08.024 INFO:tasks.workunit.client.0.smithi064.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2024-09-21T04:05:08.026 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-09-21T04:05:08.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:07 smithi131 ceph-mon[35193]: pgmap v354: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-21T04:05:08.064 INFO:tasks.workunit.client.0.smithi064.stderr: 0K .......... .......... .......... .......... .......... 49% 1.22M 0s 2024-09-21T04:05:08.084 INFO:tasks.workunit.client.0.smithi064.stderr: 50K .......... .......... .......... .......... .......... 99% 2.39M 0s 2024-09-21T04:05:08.085 INFO:tasks.workunit.client.0.smithi064.stderr: 100K 100% 1.74T=0.06s 2024-09-21T04:05:08.085 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-09-21T04:05:08.085 INFO:tasks.workunit.client.0.smithi064.stderr:2024-09-21 04:05:08 (1.63 MB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2024-09-21T04:05:08.085 INFO:tasks.workunit.client.0.smithi064.stderr: 2024-09-21T04:05:08.085 INFO:tasks.workunit.client.0.smithi064.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2024-09-21T04:05:08.134 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/ 2024-09-21T04:05:08.135 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/m4/ 2024-09-21T04:05:08.136 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/m4/acx_pthread.m4 2024-09-21T04:05:08.137 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/man/ 2024-09-21T04:05:08.138 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/man/blogbench.8 2024-09-21T04:05:08.139 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/man/Makefile.am 2024-09-21T04:05:08.140 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/man/Makefile.in 2024-09-21T04:05:08.141 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/src/ 2024-09-21T04:05:08.142 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/src/mysnprintf.c 2024-09-21T04:05:08.143 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/src/mysnprintf.h 2024-09-21T04:05:08.145 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/src/blogbench.c 2024-09-21T04:05:08.146 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/src/blogbench.h 2024-09-21T04:05:08.146 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/src/fakesnprintf.c 2024-09-21T04:05:08.147 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/src/fakesnprintf.h 2024-09-21T04:05:08.148 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/src/blogbench_p.h 2024-09-21T04:05:08.149 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/src/bsd-getopt_long.c 2024-09-21T04:05:08.150 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/src/bsd-getopt_long.h 2024-09-21T04:05:08.151 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/src/Makefile.am 2024-09-21T04:05:08.152 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/src/Makefile.in 2024-09-21T04:05:08.154 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/src/process.c 2024-09-21T04:05:08.155 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/src/globals.h 2024-09-21T04:05:08.156 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/src/commenter.c 2024-09-21T04:05:08.157 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/src/rewriter.c 2024-09-21T04:05:08.158 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/src/reader.c 2024-09-21T04:05:08.158 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/src/writer.c 2024-09-21T04:05:08.159 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/src/helpers.c 2024-09-21T04:05:08.160 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/NEWS 2024-09-21T04:05:08.161 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/compile 2024-09-21T04:05:08.162 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/depcomp 2024-09-21T04:05:08.164 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/aclocal.m4 2024-09-21T04:05:08.165 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/README 2024-09-21T04:05:08.166 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/configure 2024-09-21T04:05:08.170 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/configure.ac 2024-09-21T04:05:08.171 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/config.guess 2024-09-21T04:05:08.172 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/install-sh 2024-09-21T04:05:08.172 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/config.sub 2024-09-21T04:05:08.173 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/missing 2024-09-21T04:05:08.174 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/mkinstalldirs 2024-09-21T04:05:08.175 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/Makefile.am 2024-09-21T04:05:08.176 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/Makefile.in 2024-09-21T04:05:08.177 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/config.h.in 2024-09-21T04:05:08.178 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/AUTHORS 2024-09-21T04:05:08.179 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/INSTALL 2024-09-21T04:05:08.180 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/ChangeLog 2024-09-21T04:05:08.182 INFO:tasks.workunit.client.0.smithi064.stdout:blogbench-1.0/COPYING 2024-09-21T04:05:08.183 INFO:tasks.workunit.client.0.smithi064.stderr:+ cd blogbench-1.0/ 2024-09-21T04:05:08.183 INFO:tasks.workunit.client.0.smithi064.stdout:making blogbench 2024-09-21T04:05:08.183 INFO:tasks.workunit.client.0.smithi064.stderr:+ echo 'making blogbench' 2024-09-21T04:05:08.183 INFO:tasks.workunit.client.0.smithi064.stderr:+ ./configure 2024-09-21T04:05:08.283 INFO:tasks.workunit.client.0.smithi064.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-21T04:05:09.292 INFO:tasks.workunit.client.0.smithi064.stdout:checking whether build environment is sane... yes 2024-09-21T04:05:09.302 INFO:tasks.workunit.client.0.smithi064.stdout:checking for gawk... gawk 2024-09-21T04:05:09.319 INFO:tasks.workunit.client.0.smithi064.stdout:checking whether make sets $(MAKE)... yes 2024-09-21T04:05:09.339 INFO:tasks.workunit.client.0.smithi064.stdout:checking for gcc... gcc 2024-09-21T04:05:09.393 INFO:tasks.workunit.client.0.smithi064.stdout:checking for C compiler default output file name... a.out 2024-09-21T04:05:09.394 INFO:tasks.workunit.client.0.smithi064.stdout:checking whether the C compiler works... yes 2024-09-21T04:05:09.397 INFO:tasks.workunit.client.0.smithi064.stdout:checking whether we are cross compiling... no 2024-09-21T04:05:09.435 INFO:tasks.workunit.client.0.smithi064.stdout:checking for suffix of executables... 2024-09-21T04:05:09.471 INFO:tasks.workunit.client.0.smithi064.stdout:checking for suffix of object files... o 2024-09-21T04:05:09.504 INFO:tasks.workunit.client.0.smithi064.stdout:checking whether we are using the GNU C compiler... yes 2024-09-21T04:05:09.539 INFO:tasks.workunit.client.0.smithi064.stdout:checking whether gcc accepts -g... yes 2024-09-21T04:05:09.594 INFO:tasks.workunit.client.0.smithi064.stdout:checking for gcc option to accept ANSI C... none needed 2024-09-21T04:05:09.646 INFO:tasks.workunit.client.0.smithi064.stdout:checking for style of include used by make... GNU 2024-09-21T04:05:09.743 INFO:tasks.workunit.client.0.smithi064.stdout:checking dependency style of gcc... gcc3 2024-09-21T04:05:09.744 INFO:tasks.workunit.client.0.smithi064.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-21T04:05:09.805 INFO:tasks.workunit.client.0.smithi064.stdout:checking how to run the C preprocessor... gcc -E 2024-09-21T04:05:09.850 INFO:tasks.workunit.client.0.smithi064.stdout:checking for egrep... grep -E 2024-09-21T04:05:09.859 INFO:tasks.workunit.client.0.smithi064.stdout:checking for AIX... no 2024-09-21T04:05:09.924 INFO:tasks.workunit.client.0.smithi064.stdout:checking for library containing strerror... none required 2024-09-21T04:05:10.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:09 smithi119 ceph-mon[34811]: pgmap v355: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-21T04:05:10.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:09 smithi064 ceph-mon[31180]: pgmap v355: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-21T04:05:10.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:09 smithi131 ceph-mon[35193]: pgmap v355: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-21T04:05:10.156 INFO:tasks.workunit.client.0.smithi064.stdout:checking for ANSI C header files... yes 2024-09-21T04:05:10.193 INFO:tasks.workunit.client.0.smithi064.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-21T04:05:10.241 INFO:tasks.workunit.client.0.smithi064.stdout:checking for sys/types.h... yes 2024-09-21T04:05:10.289 INFO:tasks.workunit.client.0.smithi064.stdout:checking for sys/stat.h... yes 2024-09-21T04:05:10.336 INFO:tasks.workunit.client.0.smithi064.stdout:checking for stdlib.h... yes 2024-09-21T04:05:10.386 INFO:tasks.workunit.client.0.smithi064.stdout:checking for string.h... yes 2024-09-21T04:05:10.435 INFO:tasks.workunit.client.0.smithi064.stdout:checking for memory.h... yes 2024-09-21T04:05:10.485 INFO:tasks.workunit.client.0.smithi064.stdout:checking for strings.h... yes 2024-09-21T04:05:10.534 INFO:tasks.workunit.client.0.smithi064.stdout:checking for inttypes.h... yes 2024-09-21T04:05:10.583 INFO:tasks.workunit.client.0.smithi064.stdout:checking for stdint.h... yes 2024-09-21T04:05:10.633 INFO:tasks.workunit.client.0.smithi064.stdout:checking for unistd.h... yes 2024-09-21T04:05:10.645 INFO:tasks.workunit.client.0.smithi064.stdout:checking for memory.h... (cached) yes 2024-09-21T04:05:10.650 INFO:tasks.workunit.client.0.smithi064.stdout:checking for strings.h... (cached) yes 2024-09-21T04:05:10.655 INFO:tasks.workunit.client.0.smithi064.stdout:checking for unistd.h... (cached) yes 2024-09-21T04:05:10.701 INFO:tasks.workunit.client.0.smithi064.stdout:checking locale.h usability... yes 2024-09-21T04:05:10.720 INFO:tasks.workunit.client.0.smithi064.stdout:checking locale.h presence... yes 2024-09-21T04:05:10.721 INFO:tasks.workunit.client.0.smithi064.stdout:checking for locale.h... yes 2024-09-21T04:05:10.736 INFO:tasks.workunit.client.0.smithi064.stdout:checking for stdlib.h... (cached) yes 2024-09-21T04:05:10.741 INFO:tasks.workunit.client.0.smithi064.stdout:checking for string.h... (cached) yes 2024-09-21T04:05:10.788 INFO:tasks.workunit.client.0.smithi064.stdout:checking sys/socket.h usability... yes 2024-09-21T04:05:10.809 INFO:tasks.workunit.client.0.smithi064.stdout:checking sys/socket.h presence... yes 2024-09-21T04:05:10.810 INFO:tasks.workunit.client.0.smithi064.stdout:checking for sys/socket.h... yes 2024-09-21T04:05:10.863 INFO:tasks.workunit.client.0.smithi064.stdout:checking sys/time.h usability... yes 2024-09-21T04:05:10.882 INFO:tasks.workunit.client.0.smithi064.stdout:checking sys/time.h presence... yes 2024-09-21T04:05:10.883 INFO:tasks.workunit.client.0.smithi064.stdout:checking for sys/time.h... yes 2024-09-21T04:05:10.941 INFO:tasks.workunit.client.0.smithi064.stdout:checking fcntl.h usability... yes 2024-09-21T04:05:10.961 INFO:tasks.workunit.client.0.smithi064.stdout:checking fcntl.h presence... yes 2024-09-21T04:05:10.961 INFO:tasks.workunit.client.0.smithi064.stdout:checking for fcntl.h... yes 2024-09-21T04:05:11.015 INFO:tasks.workunit.client.0.smithi064.stdout:checking sys/fcntl.h usability... yes 2024-09-21T04:05:11.034 INFO:tasks.workunit.client.0.smithi064.stdout:checking sys/fcntl.h presence... yes 2024-09-21T04:05:11.035 INFO:tasks.workunit.client.0.smithi064.stdout:checking for sys/fcntl.h... yes 2024-09-21T04:05:11.112 INFO:tasks.workunit.client.0.smithi064.stdout:checking sys/param.h usability... yes 2024-09-21T04:05:11.140 INFO:tasks.workunit.client.0.smithi064.stdout:checking sys/param.h presence... yes 2024-09-21T04:05:11.141 INFO:tasks.workunit.client.0.smithi064.stdout:checking for sys/param.h... yes 2024-09-21T04:05:11.187 INFO:tasks.workunit.client.0.smithi064.stdout:checking ioctl.h usability... no 2024-09-21T04:05:11.207 INFO:tasks.workunit.client.0.smithi064.stdout:checking ioctl.h presence... no 2024-09-21T04:05:11.207 INFO:tasks.workunit.client.0.smithi064.stdout:checking for ioctl.h... no 2024-09-21T04:05:11.257 INFO:tasks.workunit.client.0.smithi064.stdout:checking sys/ioctl.h usability... yes 2024-09-21T04:05:11.277 INFO:tasks.workunit.client.0.smithi064.stdout:checking sys/ioctl.h presence... yes 2024-09-21T04:05:11.277 INFO:tasks.workunit.client.0.smithi064.stdout:checking for sys/ioctl.h... yes 2024-09-21T04:05:11.327 INFO:tasks.workunit.client.0.smithi064.stdout:checking return type of signal handlers... void 2024-09-21T04:05:11.376 INFO:tasks.workunit.client.0.smithi064.stdout:checking for size_t... yes 2024-09-21T04:05:11.413 INFO:tasks.workunit.client.0.smithi064.stdout:checking for dev_t... yes 2024-09-21T04:05:11.457 INFO:tasks.workunit.client.0.smithi064.stdout:checking for short... yes 2024-09-21T04:05:11.536 INFO:tasks.workunit.client.0.smithi064.stdout:checking size of short... 2 2024-09-21T04:05:11.580 INFO:tasks.workunit.client.0.smithi064.stdout:checking for int... yes 2024-09-21T04:05:11.648 INFO:tasks.workunit.client.0.smithi064.stdout:checking size of int... 4 2024-09-21T04:05:11.694 INFO:tasks.workunit.client.0.smithi064.stdout:checking for long... yes 2024-09-21T04:05:11.764 INFO:tasks.workunit.client.0.smithi064.stdout:checking size of long... 8 2024-09-21T04:05:11.809 INFO:tasks.workunit.client.0.smithi064.stdout:checking for long long... yes 2024-09-21T04:05:11.888 INFO:tasks.workunit.client.0.smithi064.stdout:checking size of long long... 8 2024-09-21T04:05:11.929 INFO:tasks.workunit.client.0.smithi064.stdout:checking for ssize_t... yes 2024-09-21T04:05:11.970 INFO:tasks.workunit.client.0.smithi064.stdout:checking for sig_atomic_t... yes 2024-09-21T04:05:12.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:11 smithi119 ceph-mon[34811]: pgmap v356: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-21T04:05:12.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:05:12.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:05:12.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:05:12.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:05:12.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:05:12.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:05:12.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:05:12.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:05:12.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:05:12.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:11 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:05:12.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:11 smithi064 ceph-mon[31180]: pgmap v356: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-21T04:05:12.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:05:12.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:05:12.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:05:12.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:05:12.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:05:12.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:05:12.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:05:12.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:05:12.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:05:12.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:11 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:05:12.025 INFO:tasks.workunit.client.0.smithi064.stdout:checking whether gcc needs -traditional... no 2024-09-21T04:05:12.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:11 smithi131 ceph-mon[35193]: pgmap v356: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-21T04:05:12.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:05:12.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:05:12.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:05:12.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:05:12.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:05:12.055 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:05:12.055 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:05:12.055 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:05:12.055 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:05:12.055 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:11 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:05:12.059 INFO:tasks.workunit.client.0.smithi064.stdout:checking for an ANSI C-conforming const... yes 2024-09-21T04:05:12.083 INFO:tasks.workunit.client.0.smithi064.stdout:checking for inline... inline 2024-09-21T04:05:12.142 INFO:tasks.workunit.client.0.smithi064.stdout:checking for working alloca.h... yes 2024-09-21T04:05:12.189 INFO:tasks.workunit.client.0.smithi064.stdout:checking for alloca... yes 2024-09-21T04:05:12.256 INFO:tasks.workunit.client.0.smithi064.stdout:checking for working memcmp... yes 2024-09-21T04:05:12.312 INFO:tasks.workunit.client.0.smithi064.stdout:checking for strftime... yes 2024-09-21T04:05:12.384 INFO:tasks.workunit.client.0.smithi064.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2024-09-21T04:05:12.443 INFO:tasks.workunit.client.0.smithi064.stdout:checking whether stat accepts an empty string... no 2024-09-21T04:05:12.498 INFO:tasks.workunit.client.0.smithi064.stdout:checking for vprintf... yes 2024-09-21T04:05:12.556 INFO:tasks.workunit.client.0.smithi064.stdout:checking for _doprnt... no 2024-09-21T04:05:12.578 INFO:tasks.workunit.client.0.smithi064.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-21T04:05:12.590 INFO:tasks.workunit.client.0.smithi064.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-21T04:05:12.653 INFO:tasks.workunit.client.0.smithi064.stdout:checking for the pthreads library -lpthreads... no 2024-09-21T04:05:12.707 INFO:tasks.workunit.client.0.smithi064.stdout:checking whether pthreads work without any flags... yes 2024-09-21T04:05:12.758 INFO:tasks.workunit.client.0.smithi064.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2024-09-21T04:05:12.759 INFO:tasks.workunit.client.0.smithi064.stdout:checking if more special flags are required for pthreads... no 2024-09-21T04:05:12.759 INFO:tasks.workunit.client.0.smithi064.stdout:checking for cc_r... gcc 2024-09-21T04:05:12.822 INFO:tasks.workunit.client.0.smithi064.stdout:checking for setrlimit... yes 2024-09-21T04:05:12.885 INFO:tasks.workunit.client.0.smithi064.stdout:checking for setproctitle... no 2024-09-21T04:05:12.945 INFO:tasks.workunit.client.0.smithi064.stdout:checking for getopt_long... yes 2024-09-21T04:05:13.007 INFO:tasks.workunit.client.0.smithi064.stdout:checking for strtoull... yes 2024-09-21T04:05:13.068 INFO:tasks.workunit.client.0.smithi064.stdout:checking for strtoq... yes 2024-09-21T04:05:13.133 INFO:tasks.workunit.client.0.smithi064.stdout:checking for snprintf... yes 2024-09-21T04:05:13.246 INFO:tasks.workunit.client.0.smithi064.stdout:checking for vsnprintf... yes 2024-09-21T04:05:13.320 INFO:tasks.workunit.client.0.smithi064.stdout:checking for vfprintf... yes 2024-09-21T04:05:13.385 INFO:tasks.workunit.client.0.smithi064.stdout:checking for gethostname... yes 2024-09-21T04:05:13.450 INFO:tasks.workunit.client.0.smithi064.stdout:checking for setlocale... yes 2024-09-21T04:05:13.513 INFO:tasks.workunit.client.0.smithi064.stdout:checking for sync... yes 2024-09-21T04:05:13.583 INFO:tasks.workunit.client.0.smithi064.stdout:checking whether snprintf is C99 conformant... done 2024-09-21T04:05:13.593 INFO:tasks.workunit.client.0.smithi064.stdout:configure: creating ./config.status 2024-09-21T04:05:13.754 INFO:tasks.workunit.client.0.smithi064.stdout:config.status: creating Makefile 2024-09-21T04:05:13.773 INFO:tasks.workunit.client.0.smithi064.stdout:config.status: creating src/Makefile 2024-09-21T04:05:14.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:13 smithi119 ceph-mon[34811]: pgmap v357: 97 pgs: 97 active+clean; 3.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 168 KiB/s wr, 27 op/s 2024-09-21T04:05:14.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:13 smithi119 ceph-mon[34811]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:05:14.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:13 smithi064 ceph-mon[31180]: pgmap v357: 97 pgs: 97 active+clean; 3.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 168 KiB/s wr, 27 op/s 2024-09-21T04:05:14.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:13 smithi064 ceph-mon[31180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:05:14.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:13 smithi131 ceph-mon[35193]: pgmap v357: 97 pgs: 97 active+clean; 3.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 168 KiB/s wr, 27 op/s 2024-09-21T04:05:14.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:13 smithi131 ceph-mon[35193]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:05:16.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:15 smithi119 ceph-mon[34811]: pgmap v358: 97 pgs: 97 active+clean; 3.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 145 KiB/s wr, 9 op/s 2024-09-21T04:05:16.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:15 smithi064 ceph-mon[31180]: pgmap v358: 97 pgs: 97 active+clean; 3.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 145 KiB/s wr, 9 op/s 2024-09-21T04:05:16.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:15 smithi131 ceph-mon[35193]: pgmap v358: 97 pgs: 97 active+clean; 3.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 145 KiB/s wr, 9 op/s 2024-09-21T04:05:18.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:17 smithi119 ceph-mon[34811]: pgmap v359: 97 pgs: 97 active+clean; 3.8 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 252 KiB/s wr, 36 op/s 2024-09-21T04:05:18.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:17 smithi064 ceph-mon[31180]: pgmap v359: 97 pgs: 97 active+clean; 3.8 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 252 KiB/s wr, 36 op/s 2024-09-21T04:05:18.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:17 smithi131 ceph-mon[35193]: pgmap v359: 97 pgs: 97 active+clean; 3.8 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 252 KiB/s wr, 36 op/s 2024-09-21T04:05:19.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:18 smithi119 ceph-mon[34811]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:05:19.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:18 smithi064 ceph-mon[31180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:05:19.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:18 smithi131 ceph-mon[35193]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:05:19.199 INFO:tasks.workunit.client.0.smithi064.stdout:config.status: creating man/Makefile 2024-09-21T04:05:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.588+0000 7f4d453ad640 1 -- 172.21.15.64:0/2380651202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d40104960 msgr2=0x7f4d40104d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.588+0000 7f4d453ad640 1 --2- 172.21.15.64:0/2380651202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d40104960 0x7f4d40104d70 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4d30009ca0 tx=0x7f4d3002a8c0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.588+0000 7f4d453ad640 1 -- 172.21.15.64:0/2380651202 shutdown_connections 2024-09-21T04:05:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.588+0000 7f4d453ad640 1 --2- 172.21.15.64:0/2380651202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d4010ddc0 0x7f4d401102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.588+0000 7f4d453ad640 1 --2- 172.21.15.64:0/2380651202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d401052b0 0x7f4d4010d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.588+0000 7f4d453ad640 1 --2- 172.21.15.64:0/2380651202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d40104960 0x7f4d40104d70 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.588+0000 7f4d453ad640 1 -- 172.21.15.64:0/2380651202 >> 172.21.15.64:0/2380651202 conn(0x7f4d400fe410 msgr2=0x7f4d40100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.588+0000 7f4d453ad640 1 -- 172.21.15.64:0/2380651202 shutdown_connections 2024-09-21T04:05:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.588+0000 7f4d453ad640 1 -- 172.21.15.64:0/2380651202 wait complete. 2024-09-21T04:05:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.589+0000 7f4d453ad640 1 Processor -- start 2024-09-21T04:05:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.589+0000 7f4d453ad640 1 -- start start 2024-09-21T04:05:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.589+0000 7f4d453ad640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d40104960 0x7f4d4019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.589+0000 7f4d453ad640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d401052b0 0x7f4d4019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.589+0000 7f4d453ad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d4010ddc0 0x7f4d4019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.589+0000 7f4d453ad640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4d40113a40 con 0x7f4d4010ddc0 2024-09-21T04:05:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.589+0000 7f4d453ad640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4d401138c0 con 0x7f4d401052b0 2024-09-21T04:05:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.589+0000 7f4d453ad640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4d40113bc0 con 0x7f4d40104960 2024-09-21T04:05:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.590+0000 7f4d3f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d4010ddc0 0x7f4d4019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:19.594 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.590+0000 7f4d3f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d4010ddc0 0x7f4d4019fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33282/0 (socket says 172.21.15.64:33282) 2024-09-21T04:05:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.590+0000 7f4d3f7fe640 1 -- 172.21.15.64:0/1444738318 learned_addr learned my addr 172.21.15.64:0/1444738318 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.590+0000 7f4d3e7fc640 1 --2- 172.21.15.64:0/1444738318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d401052b0 0x7f4d4019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.590+0000 7f4d3effd640 1 --2- 172.21.15.64:0/1444738318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d40104960 0x7f4d4019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.590+0000 7f4d3f7fe640 1 -- 172.21.15.64:0/1444738318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d40104960 msgr2=0x7f4d4019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.590+0000 7f4d3f7fe640 1 --2- 172.21.15.64:0/1444738318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d40104960 0x7f4d4019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.590+0000 7f4d3f7fe640 1 -- 172.21.15.64:0/1444738318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d401052b0 msgr2=0x7f4d4019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.590+0000 7f4d3f7fe640 1 --2- 172.21.15.64:0/1444738318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d401052b0 0x7f4d4019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.590+0000 7f4d3f7fe640 1 -- 172.21.15.64:0/1444738318 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d401a0410 con 0x7f4d4010ddc0 2024-09-21T04:05:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.590+0000 7f4d3e7fc640 1 --2- 172.21.15.64:0/1444738318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d401052b0 0x7f4d4019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:05:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.590+0000 7f4d3f7fe640 1 --2- 172.21.15.64:0/1444738318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d4010ddc0 0x7f4d4019fd10 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800e970 tx=0x7f4d3800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.590+0000 7f4d37fff640 1 -- 172.21.15.64:0/1444738318 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3800ccc0 con 0x7f4d4010ddc0 2024-09-21T04:05:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.590+0000 7f4d37fff640 1 -- 172.21.15.64:0/1444738318 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d3800ce60 con 0x7f4d4010ddc0 2024-09-21T04:05:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.590+0000 7f4d37fff640 1 -- 172.21.15.64:0/1444738318 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d38010430 con 0x7f4d4010ddc0 2024-09-21T04:05:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.590+0000 7f4d453ad640 1 -- 172.21.15.64:0/1444738318 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d401a0700 con 0x7f4d4010ddc0 2024-09-21T04:05:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.590+0000 7f4d453ad640 1 -- 172.21.15.64:0/1444738318 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d401a6e90 con 0x7f4d4010ddc0 2024-09-21T04:05:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.593+0000 7f4d4c3fb740 1 Processor -- start 2024-09-21T04:05:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.593+0000 7f4d37fff640 1 -- 172.21.15.64:0/1444738318 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f4d38004f90 con 0x7f4d4010ddc0 2024-09-21T04:05:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.593+0000 7f4d4c3fb740 1 -- start start 2024-09-21T04:05:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.594+0000 7f4d4c3fb740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590a99781b0 0x5590a9d1d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.594+0000 7f4d4c3fb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a994f2f0 0x5590a9d1def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.594+0000 7f4d4c3fb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590a99574d0 0x5590a9d1e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.594+0000 7f4d4c3fb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5590a9c926f0 con 0x5590a994f2f0 2024-09-21T04:05:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.594+0000 7f4d4c3fb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5590a9c81b20 con 0x5590a99574d0 2024-09-21T04:05:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.594+0000 7f4d4c3fb740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5590a9c85930 con 0x5590a99781b0 2024-09-21T04:05:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.594+0000 7f4d3effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a994f2f0 0x5590a9d1def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.594+0000 7f4d3effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a994f2f0 0x5590a9d1def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33294/0 (socket says 172.21.15.64:33294) 2024-09-21T04:05:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.594+0000 7f4d3effd640 1 -- 172.21.15.64:0/1035895318 learned_addr learned my addr 172.21.15.64:0/1035895318 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.594+0000 7f4d3e7fc640 1 --2- 172.21.15.64:0/1035895318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590a99781b0 0x5590a9d1d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.594+0000 7f4d3e7fc640 1 --2- 172.21.15.64:0/1035895318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590a99574d0 0x5590a9d1e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.594+0000 7f4d3effd640 1 -- 172.21.15.64:0/1035895318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590a99781b0 msgr2=0x5590a9d1d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.594+0000 7f4d3effd640 1 --2- 172.21.15.64:0/1035895318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590a99781b0 0x5590a9d1d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.594+0000 7f4d3effd640 1 -- 172.21.15.64:0/1035895318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590a99574d0 msgr2=0x5590a9d1e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.594+0000 7f4d3effd640 1 --2- 172.21.15.64:0/1035895318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590a99574d0 0x5590a9d1e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.594+0000 7f4d3effd640 1 -- 172.21.15.64:0/1035895318 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590a9c47530 con 0x5590a994f2f0 2024-09-21T04:05:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.594+0000 7f4d3e7fc640 1 --2- 172.21.15.64:0/1035895318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590a99781b0 0x5590a9d1d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:05:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.594+0000 7f4d3effd640 1 --2- 172.21.15.64:0/1035895318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a994f2f0 0x5590a9d1def0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4d30035000 tx=0x7f4d3002f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e17909141ccac44 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.594+0000 7f4d37fff640 1 --2- 172.21.15.64:0/1444738318 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4d2c0b6680 0x7f4d2c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.595+0000 7f4d3e7fc640 1 --2- 172.21.15.64:0/1444738318 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4d2c0b6680 0x7f4d2c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.595+0000 7f4d357fa640 1 -- 172.21.15.64:0/1035895318 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d30031cf0 con 0x5590a994f2f0 2024-09-21T04:05:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.595+0000 7f4d37fff640 1 -- 172.21.15.64:0/1444738318 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f4d380e7bf0 con 0x7f4d4010ddc0 2024-09-21T04:05:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.595+0000 7f4d357fa640 1 -- 172.21.15.64:0/1035895318 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d30036040 con 0x5590a994f2f0 2024-09-21T04:05:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.595+0000 7f4d357fa640 1 -- 172.21.15.64:0/1035895318 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d30004280 con 0x5590a994f2f0 2024-09-21T04:05:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.596+0000 7f4d4c3fb740 1 -- 172.21.15.64:0/1035895318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a994f2f0 msgr2=0x5590a9d1def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.596+0000 7f4d4c3fb740 1 --2- 172.21.15.64:0/1035895318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a994f2f0 0x5590a9d1def0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4d30035000 tx=0x7f4d3002f410 comp rx=0 tx=0).stop 2024-09-21T04:05:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.596+0000 7f4d3e7fc640 1 --2- 172.21.15.64:0/1444738318 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4d2c0b6680 0x7f4d2c0b8b90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4d401102d0 tx=0x7f4d28005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.596+0000 7f4d4c3fb740 1 -- 172.21.15.64:0/1035895318 shutdown_connections 2024-09-21T04:05:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.596+0000 7f4d4c3fb740 1 --2- 172.21.15.64:0/1035895318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590a99574d0 0x5590a9d1e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.596+0000 7f4d4c3fb740 1 --2- 172.21.15.64:0/1035895318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a994f2f0 0x5590a9d1def0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.596+0000 7f4d4c3fb740 1 --2- 172.21.15.64:0/1035895318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590a99781b0 0x5590a9d1d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.596+0000 7f4d4c3fb740 1 -- 172.21.15.64:0/1035895318 >> 172.21.15.64:0/1035895318 conn(0x5590a996c300 msgr2=0x5590a9d1b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.596+0000 7f4d4c3fb740 1 -- 172.21.15.64:0/1035895318 shutdown_connections 2024-09-21T04:05:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.596+0000 7f4d4c3fb740 1 -- 172.21.15.64:0/1035895318 wait complete. 2024-09-21T04:05:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.596+0000 7f4d4c3fb740 1 -- start start 2024-09-21T04:05:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.596+0000 7f4d4c3fb740 1 Processor -- start 2024-09-21T04:05:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.597+0000 7f4d4c3fb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590a994f2f0 0x5590a9d29040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.597+0000 7f4d4c3fb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a99574d0 0x5590a9d29580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.597+0000 7f4d4c3fb740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590a99781b0 0x5590a9d353c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.597+0000 7f4d4c3fb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5590a9bb9ab0 con 0x5590a99574d0 2024-09-21T04:05:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.597+0000 7f4d4c3fb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5590a9c27730 con 0x5590a994f2f0 2024-09-21T04:05:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.597+0000 7f4d4c3fb740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5590a9d2c0d0 con 0x5590a99781b0 2024-09-21T04:05:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.597+0000 7f4d4c3fb740 10 client.0 _ll_register_callbacks cb 0x5590a9c7b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:05:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.597+0000 7f4d3e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a99574d0 0x5590a9d29580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.597+0000 7f4d3e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a99574d0 0x5590a9d29580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33308/0 (socket says 172.21.15.64:33308) 2024-09-21T04:05:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.597+0000 7f4d3e7fc640 1 -- 172.21.15.64:0/4212639614 learned_addr learned my addr 172.21.15.64:0/4212639614 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.597+0000 7f4d3effd640 1 --2- 172.21.15.64:0/4212639614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590a994f2f0 0x5590a9d29040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.597+0000 7f4d3f7fe640 1 --2- 172.21.15.64:0/4212639614 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590a99781b0 0x5590a9d353c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.597+0000 7f4d3e7fc640 1 -- 172.21.15.64:0/4212639614 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590a99781b0 msgr2=0x5590a9d353c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.597+0000 7f4d3e7fc640 1 --2- 172.21.15.64:0/4212639614 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590a99781b0 0x5590a9d353c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.597+0000 7f4d3e7fc640 1 -- 172.21.15.64:0/4212639614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590a994f2f0 msgr2=0x5590a9d29040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.597+0000 7f4d3e7fc640 1 --2- 172.21.15.64:0/4212639614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590a994f2f0 0x5590a9d29040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.597+0000 7f4d3e7fc640 1 -- 172.21.15.64:0/4212639614 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590a99e3790 con 0x5590a99574d0 2024-09-21T04:05:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.597+0000 7f4d3e7fc640 1 --2- 172.21.15.64:0/4212639614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a99574d0 0x5590a9d29580 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4d280043c0 tx=0x7f4d28009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53b21842541033e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.597+0000 7f4d267fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:05:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.597+0000 7f4d267fc640 1 -- 172.21.15.64:0/4212639614 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28018030 con 0x5590a99574d0 2024-09-21T04:05:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.598+0000 7f4d267fc640 1 -- 172.21.15.64:0/4212639614 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f4d28021650 con 0x5590a99574d0 2024-09-21T04:05:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.598+0000 7f4d4c3fb740 1 -- 172.21.15.64:0/4212639614 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d30036040 con 0x5590a99574d0 2024-09-21T04:05:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.598+0000 7f4d267fc640 1 -- 172.21.15.64:0/4212639614 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4d28021850 con 0x5590a99574d0 2024-09-21T04:05:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.598+0000 7f4d4c3fb740 10 client.15795 fetch_fsmap learned FSMap version 27 2024-09-21T04:05:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.598+0000 7f4d4c3fb740 1 -- 172.21.15.64:0/4212639614 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5590a9a5c620 con 0x5590a99574d0 2024-09-21T04:05:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.598+0000 7f4d267fc640 1 -- 172.21.15.64:0/4212639614 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d30036040 con 0x5590a99574d0 2024-09-21T04:05:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.598+0000 7f4d267fc640 1 -- 172.21.15.64:0/4212639614 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4d28030070 con 0x5590a99574d0 2024-09-21T04:05:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.598+0000 7f4d4c3fb740 10 client.15795 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T04:05:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.598+0000 7f4d4c3fb740 10 client.15795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:05:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.598+0000 7f4d4c3fb740 20 client.15795 populate_metadata read hostname 'smithi064' 2024-09-21T04:05:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.598+0000 7f4d4c3fb740 1 --2- 172.21.15.64:0/4212639614 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5590a9936d80 0x5590a9d3b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.598+0000 7f4d4c3fb740 4 client.15795 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:05:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.598+0000 7f4d4c3fb740 1 -- 172.21.15.64:0/4212639614 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590a9c94640 con 0x5590a9936d80 2024-09-21T04:05:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.598+0000 7f4d3effd640 1 --2- 172.21.15.64:0/4212639614 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5590a9936d80 0x5590a9d3b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.599+0000 7f4d267fc640 10 client.15795 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.599+0000 7f4d3effd640 1 --2- 172.21.15.64:0/4212639614 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5590a9936d80 0x5590a9d3b1b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e1268775cc9229b server_cookie=41704c95e1a7ed6f in_seq=0 out_seq=0 2024-09-21T04:05:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.600+0000 7f4d267fc640 1 -- 172.21.15.64:0/4212639614 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5590a9c94640 con 0x5590a9936d80 2024-09-21T04:05:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.600+0000 7f4d267fc640 10 client.15795 handle_command_reply: tid=0 2024-09-21T04:05:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.600+0000 7f4d4c3fb740 1 client.15795 shutdown 2024-09-21T04:05:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.600+0000 7f4d4c3fb740 1 -- 172.21.15.64:0/4212639614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a99574d0 msgr2=0x5590a9d29580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.600+0000 7f4d4c3fb740 1 --2- 172.21.15.64:0/4212639614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a99574d0 0x5590a9d29580 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4d280043c0 tx=0x7f4d28009680 comp rx=0 tx=0).stop 2024-09-21T04:05:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.600+0000 7f4d4c3fb740 1 -- 172.21.15.64:0/4212639614 shutdown_connections 2024-09-21T04:05:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.600+0000 7f4d4c3fb740 1 --2- 172.21.15.64:0/4212639614 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590a99781b0 0x5590a9d353c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.600+0000 7f4d4c3fb740 1 --2- 172.21.15.64:0/4212639614 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5590a9936d80 0x5590a9d3b1b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.600+0000 7f4d4c3fb740 1 --2- 172.21.15.64:0/4212639614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a99574d0 0x5590a9d29580 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.600+0000 7f4d267fc640 0 client.15795 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.600+0000 7f4d4c3fb740 1 --2- 172.21.15.64:0/4212639614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590a994f2f0 0x5590a9d29040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.600+0000 7f4d4c3fb740 1 -- 172.21.15.64:0/4212639614 >> 172.21.15.64:0/4212639614 conn(0x5590a996c300 msgr2=0x5590a9d1e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.600+0000 7f4d4c3fb740 1 -- 172.21.15.64:0/4212639614 shutdown_connections 2024-09-21T04:05:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.600+0000 7f4d4c3fb740 1 -- 172.21.15.64:0/4212639614 wait complete. 2024-09-21T04:05:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.600+0000 7f4d4c3fb740 20 client.15795 trim_cache size 0 max 16384 2024-09-21T04:05:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.615+0000 7f4d4c3fb740 1 Processor -- start 2024-09-21T04:05:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.615+0000 7f4d4c3fb740 1 -- start start 2024-09-21T04:05:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.615+0000 7f4d4c3fb740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590a994f2f0 0x5590a9d1a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.615+0000 7f4d4c3fb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590a99574d0 0x5590a9d1f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.616+0000 7f4d4c3fb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a99781b0 0x5590a9d26210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.616+0000 7f4d4c3fb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4d401b21d0 con 0x5590a99781b0 2024-09-21T04:05:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.616+0000 7f4d4c3fb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4d401afaa0 con 0x5590a99574d0 2024-09-21T04:05:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.616+0000 7f4d4c3fb740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4d401ad8f0 con 0x5590a994f2f0 2024-09-21T04:05:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.616+0000 7f4d3f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a99781b0 0x5590a9d26210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.616+0000 7f4d3f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a99781b0 0x5590a9d26210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33320/0 (socket says 172.21.15.64:33320) 2024-09-21T04:05:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.616+0000 7f4d3f7fe640 1 -- 172.21.15.64:0/1633733711 learned_addr learned my addr 172.21.15.64:0/1633733711 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.616+0000 7f4d3effd640 1 --2- 172.21.15.64:0/1633733711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590a994f2f0 0x5590a9d1a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.616+0000 7f4d3e7fc640 1 --2- 172.21.15.64:0/1633733711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590a99574d0 0x5590a9d1f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.616+0000 7f4d3f7fe640 1 -- 172.21.15.64:0/1633733711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590a994f2f0 msgr2=0x5590a9d1a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.616+0000 7f4d3f7fe640 1 --2- 172.21.15.64:0/1633733711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590a994f2f0 0x5590a9d1a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.616+0000 7f4d3f7fe640 1 -- 172.21.15.64:0/1633733711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590a99574d0 msgr2=0x5590a9d1f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.616+0000 7f4d3f7fe640 1 --2- 172.21.15.64:0/1633733711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590a99574d0 0x5590a9d1f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.616+0000 7f4d3f7fe640 1 -- 172.21.15.64:0/1633733711 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590a9d1fcb0 con 0x5590a99781b0 2024-09-21T04:05:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.616+0000 7f4d3e7fc640 1 --2- 172.21.15.64:0/1633733711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590a99574d0 0x5590a9d1f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:05:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.616+0000 7f4d3f7fe640 1 --2- 172.21.15.64:0/1633733711 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a99781b0 0x5590a9d26210 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800c6a0 tx=0x7f4d380ad420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4e7328119d201a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.616+0000 7f4d34ff9640 1 -- 172.21.15.64:0/1633733711 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d380d10d0 con 0x5590a99781b0 2024-09-21T04:05:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.616+0000 7f4d34ff9640 1 -- 172.21.15.64:0/1633733711 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d380055f0 con 0x5590a99781b0 2024-09-21T04:05:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.616+0000 7f4d34ff9640 1 -- 172.21.15.64:0/1633733711 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d380b24f0 con 0x5590a99781b0 2024-09-21T04:05:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.617+0000 7f4d4c3fb740 1 -- 172.21.15.64:0/1633733711 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a99781b0 msgr2=0x5590a9d26210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.617+0000 7f4d4c3fb740 1 --2- 172.21.15.64:0/1633733711 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a99781b0 0x5590a9d26210 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800c6a0 tx=0x7f4d380ad420 comp rx=0 tx=0).stop 2024-09-21T04:05:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.617+0000 7f4d4c3fb740 1 -- 172.21.15.64:0/1633733711 shutdown_connections 2024-09-21T04:05:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.617+0000 7f4d4c3fb740 1 --2- 172.21.15.64:0/1633733711 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a99781b0 0x5590a9d26210 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.617+0000 7f4d4c3fb740 1 --2- 172.21.15.64:0/1633733711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590a99574d0 0x5590a9d1f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.617+0000 7f4d4c3fb740 1 --2- 172.21.15.64:0/1633733711 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590a994f2f0 0x5590a9d1a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.617+0000 7f4d4c3fb740 1 -- 172.21.15.64:0/1633733711 >> 172.21.15.64:0/1633733711 conn(0x5590a996c300 msgr2=0x5590a9d1e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.617+0000 7f4d4c3fb740 1 -- 172.21.15.64:0/1633733711 shutdown_connections 2024-09-21T04:05:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.617+0000 7f4d4c3fb740 1 -- 172.21.15.64:0/1633733711 wait complete. 2024-09-21T04:05:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.618+0000 7f4d4c3fb740 1 -- start start 2024-09-21T04:05:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.618+0000 7f4d4c3fb740 1 Processor -- start 2024-09-21T04:05:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.618+0000 7f4d4c3fb740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a994f2f0 0x5590a9d2ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.618+0000 7f4d4c3fb740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590a99574d0 0x5590a9d2f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.618+0000 7f4d4c3fb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590a99781b0 0x5590a9d9cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.618+0000 7f4d4c3fb740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4d401adf60 con 0x5590a994f2f0 2024-09-21T04:05:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.618+0000 7f4d4c3fb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4d401af3d0 con 0x5590a99781b0 2024-09-21T04:05:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.618+0000 7f4d4c3fb740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4d401ae630 con 0x5590a99574d0 2024-09-21T04:05:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.618+0000 7f4d4c3fb740 10 client.0 _ll_register_callbacks cb 0x7f4d28009c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:05:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.618+0000 7f4d3effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a994f2f0 0x5590a9d2ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.618+0000 7f4d3effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a994f2f0 0x5590a9d2ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:33326/0 (socket says 172.21.15.64:33326) 2024-09-21T04:05:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.618+0000 7f4d3effd640 1 -- 172.21.15.64:0/3323038465 learned_addr learned my addr 172.21.15.64:0/3323038465 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.618+0000 7f4d3e7fc640 1 --2- 172.21.15.64:0/3323038465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590a99574d0 0x5590a9d2f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.618+0000 7f4d3f7fe640 1 --2- 172.21.15.64:0/3323038465 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590a99781b0 0x5590a9d9cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.619+0000 7f4d3effd640 1 -- 172.21.15.64:0/3323038465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590a99574d0 msgr2=0x5590a9d2f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.619+0000 7f4d3effd640 1 --2- 172.21.15.64:0/3323038465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590a99574d0 0x5590a9d2f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.619+0000 7f4d3effd640 1 -- 172.21.15.64:0/3323038465 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590a99781b0 msgr2=0x5590a9d9cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.619+0000 7f4d3effd640 1 --2- 172.21.15.64:0/3323038465 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590a99781b0 0x5590a9d9cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.619+0000 7f4d3effd640 1 -- 172.21.15.64:0/3323038465 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590a9d9d260 con 0x5590a994f2f0 2024-09-21T04:05:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.619+0000 7f4d3f7fe640 1 --2- 172.21.15.64:0/3323038465 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590a99781b0 0x5590a9d9cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:05:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.619+0000 7f4d3e7fc640 1 --2- 172.21.15.64:0/3323038465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590a99574d0 0x5590a9d2f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:05:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.619+0000 7f4d3effd640 1 --2- 172.21.15.64:0/3323038465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a994f2f0 0x5590a9d2ee70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4d30006010 tx=0x7f4d30009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d71a19ea88d74d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.619+0000 7f4d267fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:05:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.619+0000 7f4d267fc640 1 -- 172.21.15.64:0/3323038465 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3005b020 con 0x5590a994f2f0 2024-09-21T04:05:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.619+0000 7f4d267fc640 1 -- 172.21.15.64:0/3323038465 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f4d30031cf0 con 0x5590a994f2f0 2024-09-21T04:05:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.619+0000 7f4d4c3fb740 1 -- 172.21.15.64:0/3323038465 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5590a9c900f0 con 0x5590a994f2f0 2024-09-21T04:05:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.619+0000 7f4d267fc640 1 -- 172.21.15.64:0/3323038465 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4d30066050 con 0x5590a994f2f0 2024-09-21T04:05:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.619+0000 7f4d4c3fb740 10 client.15801 fetch_fsmap learned FSMap version 27 2024-09-21T04:05:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.619+0000 7f4d4c3fb740 1 -- 172.21.15.64:0/3323038465 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5590a9da5720 con 0x5590a994f2f0 2024-09-21T04:05:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.619+0000 7f4d267fc640 1 -- 172.21.15.64:0/3323038465 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5590a9c900f0 con 0x5590a994f2f0 2024-09-21T04:05:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.619+0000 7f4d267fc640 1 -- 172.21.15.64:0/3323038465 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4d3002fb80 con 0x5590a994f2f0 2024-09-21T04:05:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.620+0000 7f4d4c3fb740 10 client.15801 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T04:05:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.620+0000 7f4d4c3fb740 10 client.15801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:05:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.620+0000 7f4d4c3fb740 20 client.15801 populate_metadata read hostname 'smithi064' 2024-09-21T04:05:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.620+0000 7f4d4c3fb740 1 --2- 172.21.15.64:0/3323038465 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5590a990c1c0 0x5590a9da7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.620+0000 7f4d4c3fb740 4 client.15801 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:05:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.620+0000 7f4d4c3fb740 1 -- 172.21.15.64:0/3323038465 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5590a9c8d520 con 0x5590a990c1c0 2024-09-21T04:05:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.620+0000 7f4d3e7fc640 1 --2- 172.21.15.64:0/3323038465 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5590a990c1c0 0x5590a9da7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.620+0000 7f4d3e7fc640 1 --2- 172.21.15.64:0/3323038465 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5590a990c1c0 0x5590a9da7b80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9acf34aec7dd31ed server_cookie=6d6556c57ab3de0c in_seq=0 out_seq=0 2024-09-21T04:05:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.620+0000 7f4d267fc640 10 client.15801 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.645+0000 7f4d267fc640 1 -- 172.21.15.64:0/3323038465 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5590a9c8d520 con 0x5590a990c1c0 2024-09-21T04:05:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.645+0000 7f4d267fc640 10 client.15801 handle_command_reply: tid=0 2024-09-21T04:05:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.645+0000 7f4d4c3fb740 1 client.15801 shutdown 2024-09-21T04:05:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.645+0000 7f4d4c3fb740 1 -- 172.21.15.64:0/3323038465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a994f2f0 msgr2=0x5590a9d2ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.645+0000 7f4d4c3fb740 1 --2- 172.21.15.64:0/3323038465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a994f2f0 0x5590a9d2ee70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4d30006010 tx=0x7f4d30009540 comp rx=0 tx=0).stop 2024-09-21T04:05:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.645+0000 7f4d4c3fb740 1 -- 172.21.15.64:0/3323038465 shutdown_connections 2024-09-21T04:05:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.645+0000 7f4d4c3fb740 1 --2- 172.21.15.64:0/3323038465 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5590a99781b0 0x5590a9d9cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.645+0000 7f4d4c3fb740 1 --2- 172.21.15.64:0/3323038465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5590a99574d0 0x5590a9d2f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.645+0000 7f4d4c3fb740 1 --2- 172.21.15.64:0/3323038465 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5590a990c1c0 0x5590a9da7b80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.645+0000 7f4d4c3fb740 1 --2- 172.21.15.64:0/3323038465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5590a994f2f0 0x5590a9d2ee70 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.645+0000 7f4d267fc640 0 client.15801 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.645+0000 7f4d4c3fb740 1 -- 172.21.15.64:0/3323038465 >> 172.21.15.64:0/3323038465 conn(0x5590a996c300 msgr2=0x5590a9d1a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.645+0000 7f4d4c3fb740 1 -- 172.21.15.64:0/3323038465 shutdown_connections 2024-09-21T04:05:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.645+0000 7f4d4c3fb740 1 -- 172.21.15.64:0/3323038465 wait complete. 2024-09-21T04:05:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.645+0000 7f4d4c3fb740 20 client.15801 trim_cache size 0 max 16384 2024-09-21T04:05:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:05:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:05:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:05:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:05:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.646+0000 7f4d453ad640 1 -- 172.21.15.64:0/1444738318 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4d2c0b6680 msgr2=0x7f4d2c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.646+0000 7f4d453ad640 1 --2- 172.21.15.64:0/1444738318 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4d2c0b6680 0x7f4d2c0b8b90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4d401102d0 tx=0x7f4d28005f90 comp rx=0 tx=0).stop 2024-09-21T04:05:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.646+0000 7f4d453ad640 1 -- 172.21.15.64:0/1444738318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d4010ddc0 msgr2=0x7f4d4019fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.646+0000 7f4d453ad640 1 --2- 172.21.15.64:0/1444738318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d4010ddc0 0x7f4d4019fd10 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800e970 tx=0x7f4d3800ed70 comp rx=0 tx=0).stop 2024-09-21T04:05:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.646+0000 7f4d453ad640 1 -- 172.21.15.64:0/1444738318 shutdown_connections 2024-09-21T04:05:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.646+0000 7f4d453ad640 1 --2- 172.21.15.64:0/1444738318 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4d2c0b6680 0x7f4d2c0b8b90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.646+0000 7f4d453ad640 1 --2- 172.21.15.64:0/1444738318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4d4010ddc0 0x7f4d4019fd10 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.646+0000 7f4d453ad640 1 --2- 172.21.15.64:0/1444738318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d401052b0 0x7f4d4019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.646+0000 7f4d453ad640 1 --2- 172.21.15.64:0/1444738318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d40104960 0x7f4d4019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.646+0000 7f4d453ad640 1 -- 172.21.15.64:0/1444738318 >> 172.21.15.64:0/1444738318 conn(0x7f4d400fe410 msgr2=0x7f4d40100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.646+0000 7f4d453ad640 1 -- 172.21.15.64:0/1444738318 shutdown_connections 2024-09-21T04:05:19.651 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:19.646+0000 7f4d453ad640 1 -- 172.21.15.64:0/1444738318 wait complete. 2024-09-21T04:05:20.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:19 smithi119 ceph-mon[34811]: pgmap v360: 97 pgs: 97 active+clean; 4.7 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 357 KiB/s wr, 49 op/s 2024-09-21T04:05:20.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:19 smithi064 ceph-mon[31180]: pgmap v360: 97 pgs: 97 active+clean; 4.7 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 357 KiB/s wr, 49 op/s 2024-09-21T04:05:20.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:19 smithi131 ceph-mon[35193]: pgmap v360: 97 pgs: 97 active+clean; 4.7 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 357 KiB/s wr, 49 op/s 2024-09-21T04:05:21.687 INFO:tasks.workunit.client.0.smithi064.stdout:config.status: creating config.h 2024-09-21T04:05:22.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:21 smithi119 ceph-mon[34811]: pgmap v361: 97 pgs: 97 active+clean; 4.7 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 357 KiB/s wr, 48 op/s 2024-09-21T04:05:22.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:21 smithi064 ceph-mon[31180]: pgmap v361: 97 pgs: 97 active+clean; 4.7 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 357 KiB/s wr, 48 op/s 2024-09-21T04:05:22.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:21 smithi131 ceph-mon[35193]: pgmap v361: 97 pgs: 97 active+clean; 4.7 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 357 KiB/s wr, 48 op/s 2024-09-21T04:05:24.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:23 smithi064 ceph-mon[31180]: pgmap v362: 97 pgs: 97 active+clean; 4.9 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 381 KiB/s wr, 65 op/s 2024-09-21T04:05:24.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:23 smithi131 ceph-mon[35193]: pgmap v362: 97 pgs: 97 active+clean; 4.9 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 381 KiB/s wr, 65 op/s 2024-09-21T04:05:24.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:23 smithi119 ceph-mon[34811]: pgmap v362: 97 pgs: 97 active+clean; 4.9 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 381 KiB/s wr, 65 op/s 2024-09-21T04:05:26.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:25 smithi064 ceph-mon[31180]: pgmap v363: 97 pgs: 97 active+clean; 4.9 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 236 KiB/s wr, 60 op/s 2024-09-21T04:05:26.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:25 smithi131 ceph-mon[35193]: pgmap v363: 97 pgs: 97 active+clean; 4.9 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 236 KiB/s wr, 60 op/s 2024-09-21T04:05:26.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:25 smithi119 ceph-mon[34811]: pgmap v363: 97 pgs: 97 active+clean; 4.9 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 236 KiB/s wr, 60 op/s 2024-09-21T04:05:26.541 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T04:05:26.695 INFO:tasks.workunit.client.0.smithi064.stdout:config.status: executing depfiles commands 2024-09-21T04:05:26.817 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:05:26.838 INFO:tasks.workunit.client.0.smithi064.stderr:+ make 2024-09-21T04:05:26.849 INFO:tasks.workunit.client.0.smithi064.stdout:make all-recursive 2024-09-21T04:05:26.852 INFO:tasks.workunit.client.0.smithi064.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-21T04:05:26.855 INFO:tasks.workunit.client.0.smithi064.stdout:Making all in src 2024-09-21T04:05:26.868 INFO:tasks.workunit.client.0.smithi064.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-09-21T04:05:26.868 INFO:tasks.workunit.client.0.smithi064.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2024-09-21T04:05:26.868 INFO:tasks.workunit.client.0.smithi064.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2024-09-21T04:05:27.011 INFO:tasks.workunit.client.0.smithi064.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2024-09-21T04:05:27.012 INFO:tasks.workunit.client.0.smithi064.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2024-09-21T04:05:27.114 INFO:tasks.workunit.client.0.smithi064.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2024-09-21T04:05:27.114 INFO:tasks.workunit.client.0.smithi064.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2024-09-21T04:05:27.221 INFO:tasks.workunit.client.0.smithi064.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2024-09-21T04:05:27.221 INFO:tasks.workunit.client.0.smithi064.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2024-09-21T04:05:27.308 INFO:tasks.workunit.client.0.smithi064.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2024-09-21T04:05:27.308 INFO:tasks.workunit.client.0.smithi064.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2024-09-21T04:05:27.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.328+0000 7f9207542640 1 -- 172.21.15.64:0/3929727306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92001049f0 msgr2=0x7f9200104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:27.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.328+0000 7f9207542640 1 --2- 172.21.15.64:0/3929727306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92001049f0 0x7f9200104e00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f91fc009900 tx=0x7f91fc02a410 comp rx=0 tx=0).stop 2024-09-21T04:05:27.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.328+0000 7f9207542640 1 -- 172.21.15.64:0/3929727306 shutdown_connections 2024-09-21T04:05:27.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.328+0000 7f9207542640 1 --2- 172.21.15.64:0/3929727306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f920010de50 0x7f9200110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.328+0000 7f9207542640 1 --2- 172.21.15.64:0/3929727306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9200105340 0x7f920010d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.328+0000 7f9207542640 1 --2- 172.21.15.64:0/3929727306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92001049f0 0x7f9200104e00 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.328+0000 7f9207542640 1 -- 172.21.15.64:0/3929727306 >> 172.21.15.64:0/3929727306 conn(0x7f92000797e0 msgr2=0x7f9200079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:27.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.328+0000 7f9207542640 1 -- 172.21.15.64:0/3929727306 shutdown_connections 2024-09-21T04:05:27.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.328+0000 7f9207542640 1 -- 172.21.15.64:0/3929727306 wait complete. 2024-09-21T04:05:27.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.329+0000 7f9207542640 1 Processor -- start 2024-09-21T04:05:27.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.329+0000 7f9207542640 1 -- start start 2024-09-21T04:05:27.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.329+0000 7f9207542640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92001049f0 0x7f920019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:27.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.329+0000 7f9207542640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9200105340 0x7f920019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:27.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.329+0000 7f9207542640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f920010de50 0x7f920019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:27.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.330+0000 7f9207542640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9200113ad0 con 0x7f9200105340 2024-09-21T04:05:27.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.330+0000 7f9207542640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9200113950 con 0x7f92001049f0 2024-09-21T04:05:27.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.330+0000 7f9207542640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9200113c50 con 0x7f920010de50 2024-09-21T04:05:27.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.330+0000 7f9204add640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9200105340 0x7f920019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:27.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.330+0000 7f9204add640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9200105340 0x7f920019aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55732/0 (socket says 172.21.15.64:55732) 2024-09-21T04:05:27.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.330+0000 7f9204add640 1 -- 172.21.15.64:0/1835390072 learned_addr learned my addr 172.21.15.64:0/1835390072 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:27.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.330+0000 7f9205adf640 1 --2- 172.21.15.64:0/1835390072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f920010de50 0x7f920019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:27.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.330+0000 7f92052de640 1 --2- 172.21.15.64:0/1835390072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92001049f0 0x7f920019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:27.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.330+0000 7f9204add640 1 -- 172.21.15.64:0/1835390072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f920010de50 msgr2=0x7f920019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:27.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.330+0000 7f9204add640 1 --2- 172.21.15.64:0/1835390072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f920010de50 0x7f920019fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.330+0000 7f9204add640 1 -- 172.21.15.64:0/1835390072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92001049f0 msgr2=0x7f920019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:27.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.330+0000 7f9204add640 1 --2- 172.21.15.64:0/1835390072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92001049f0 0x7f920019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.330+0000 7f9204add640 1 -- 172.21.15.64:0/1835390072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92001a0450 con 0x7f9200105340 2024-09-21T04:05:27.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.330+0000 7f9205adf640 1 --2- 172.21.15.64:0/1835390072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f920010de50 0x7f920019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:05:27.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.330+0000 7f9204add640 1 --2- 172.21.15.64:0/1835390072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9200105340 0x7f920019aa80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f91f0009bb0 tx=0x7f91f000b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:27.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.330+0000 7f91f67fc640 1 -- 172.21.15.64:0/1835390072 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f0005020 con 0x7f9200105340 2024-09-21T04:05:27.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.331+0000 7f9207542640 1 -- 172.21.15.64:0/1835390072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92001a0740 con 0x7f9200105340 2024-09-21T04:05:27.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.331+0000 7f91f67fc640 1 -- 172.21.15.64:0/1835390072 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91f0005670 con 0x7f9200105340 2024-09-21T04:05:27.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.331+0000 7f91f67fc640 1 -- 172.21.15.64:0/1835390072 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f0019670 con 0x7f9200105340 2024-09-21T04:05:27.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.331+0000 7f9207542640 1 -- 172.21.15.64:0/1835390072 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9200102610 con 0x7f9200105340 2024-09-21T04:05:27.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.334+0000 7f91f67fc640 1 -- 172.21.15.64:0/1835390072 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f91f0019810 con 0x7f9200105340 2024-09-21T04:05:27.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.334+0000 7f920dab6740 1 Processor -- start 2024-09-21T04:05:27.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.334+0000 7f920dab6740 1 -- start start 2024-09-21T04:05:27.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.334+0000 7f920dab6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564a63b04070 0x564a63eab270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:27.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.334+0000 7f920dab6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a63adb1b0 0x564a63eab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:27.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.334+0000 7f920dab6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a63ae3390 0x564a63eabcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:27.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.334+0000 7f920dab6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564a63e21e50 con 0x564a63ae3390 2024-09-21T04:05:27.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.334+0000 7f920dab6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564a63e11280 con 0x564a63b04070 2024-09-21T04:05:27.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.334+0000 7f920dab6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564a63e15090 con 0x564a63adb1b0 2024-09-21T04:05:27.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.334+0000 7f9204add640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a63ae3390 0x564a63eabcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:27.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.334+0000 7f92052de640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a63adb1b0 0x564a63eab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:27.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.334+0000 7f92052de640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a63adb1b0 0x564a63eab7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:46976/0 (socket says 172.21.15.64:46976) 2024-09-21T04:05:27.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.334+0000 7f92052de640 1 -- 172.21.15.64:0/1189763692 learned_addr learned my addr 172.21.15.64:0/1189763692 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:27.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.334+0000 7f9205adf640 1 --2- 172.21.15.64:0/1189763692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564a63b04070 0x564a63eab270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:27.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.335+0000 7f92052de640 1 -- 172.21.15.64:0/1189763692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564a63b04070 msgr2=0x564a63eab270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:27.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.335+0000 7f92052de640 1 --2- 172.21.15.64:0/1189763692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564a63b04070 0x564a63eab270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.335+0000 7f92052de640 1 -- 172.21.15.64:0/1189763692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a63ae3390 msgr2=0x564a63eabcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:27.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.335+0000 7f92052de640 1 --2- 172.21.15.64:0/1189763692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a63ae3390 0x564a63eabcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.335+0000 7f92052de640 1 -- 172.21.15.64:0/1189763692 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a63b6f650 con 0x564a63adb1b0 2024-09-21T04:05:27.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.335+0000 7f91f67fc640 1 --2- 172.21.15.64:0/1835390072 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f91ec0b6680 0x7f91ec0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:27.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.335+0000 7f9205adf640 1 --2- 172.21.15.64:0/1189763692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564a63b04070 0x564a63eab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:05:27.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.335+0000 7f92052de640 1 --2- 172.21.15.64:0/1189763692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a63adb1b0 0x564a63eab7b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f920006c620 tx=0x7f91fc032b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ed173f7f6e4c522 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:27.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.335+0000 7f91f67fc640 1 -- 172.21.15.64:0/1835390072 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f91f00eb110 con 0x7f9200105340 2024-09-21T04:05:27.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.335+0000 7f9205adf640 1 --2- 172.21.15.64:0/1835390072 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f91ec0b6680 0x7f91ec0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:27.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.335+0000 7f91eb7fe640 1 -- 172.21.15.64:0/1189763692 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc040070 con 0x564a63adb1b0 2024-09-21T04:05:27.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.335+0000 7f91eb7fe640 1 -- 172.21.15.64:0/1189763692 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91fc004030 con 0x564a63adb1b0 2024-09-21T04:05:27.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.336+0000 7f91eb7fe640 1 -- 172.21.15.64:0/1189763692 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc035040 con 0x564a63adb1b0 2024-09-21T04:05:27.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.336+0000 7f920dab6740 1 -- 172.21.15.64:0/1189763692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a63adb1b0 msgr2=0x564a63eab7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:27.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.336+0000 7f9205adf640 1 --2- 172.21.15.64:0/1835390072 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f91ec0b6680 0x7f91ec0b8b90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f91f8005fd0 tx=0x7f91f8009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:27.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.336+0000 7f920dab6740 1 --2- 172.21.15.64:0/1189763692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a63adb1b0 0x564a63eab7b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f920006c620 tx=0x7f91fc032b20 comp rx=0 tx=0).stop 2024-09-21T04:05:27.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.336+0000 7f920dab6740 1 -- 172.21.15.64:0/1189763692 shutdown_connections 2024-09-21T04:05:27.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.337+0000 7f920dab6740 1 --2- 172.21.15.64:0/1189763692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a63ae3390 0x564a63eabcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.337+0000 7f920dab6740 1 --2- 172.21.15.64:0/1189763692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a63adb1b0 0x564a63eab7b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.337+0000 7f920dab6740 1 --2- 172.21.15.64:0/1189763692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564a63b04070 0x564a63eab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.337+0000 7f920dab6740 1 -- 172.21.15.64:0/1189763692 >> 172.21.15.64:0/1189763692 conn(0x564a63af8460 msgr2=0x564a63ea9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:27.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.337+0000 7f920dab6740 1 -- 172.21.15.64:0/1189763692 shutdown_connections 2024-09-21T04:05:27.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.337+0000 7f920dab6740 1 -- 172.21.15.64:0/1189763692 wait complete. 2024-09-21T04:05:27.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.337+0000 7f920dab6740 1 -- start start 2024-09-21T04:05:27.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.337+0000 7f920dab6740 1 Processor -- start 2024-09-21T04:05:27.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.338+0000 7f920dab6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a63adb1b0 0x564a63eb68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:27.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.338+0000 7f920dab6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564a63ae3390 0x564a63eb6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:27.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.338+0000 7f920dab6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a63b04070 0x564a63ec2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:27.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.338+0000 7f920dab6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564a63c72700 con 0x564a63b04070 2024-09-21T04:05:27.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.338+0000 7f920dab6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564a63dac110 con 0x564a63ae3390 2024-09-21T04:05:27.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.338+0000 7f920dab6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564a63eb99f0 con 0x564a63adb1b0 2024-09-21T04:05:27.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.338+0000 7f920dab6740 10 client.0 _ll_register_callbacks cb 0x564a63dc5990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:05:27.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.338+0000 7f9205adf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a63b04070 0x564a63ec2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:27.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.338+0000 7f92052de640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a63adb1b0 0x564a63eb68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:27.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.338+0000 7f92052de640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a63adb1b0 0x564a63eb68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:46992/0 (socket says 172.21.15.64:46992) 2024-09-21T04:05:27.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.338+0000 7f9205adf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a63b04070 0x564a63ec2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55742/0 (socket says 172.21.15.64:55742) 2024-09-21T04:05:27.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.338+0000 7f92052de640 1 -- 172.21.15.64:0/14480684 learned_addr learned my addr 172.21.15.64:0/14480684 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:27.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.338+0000 7f9204add640 1 --2- 172.21.15.64:0/14480684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564a63ae3390 0x564a63eb6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:27.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.338+0000 7f92052de640 1 -- 172.21.15.64:0/14480684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564a63ae3390 msgr2=0x564a63eb6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:27.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.338+0000 7f92052de640 1 --2- 172.21.15.64:0/14480684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564a63ae3390 0x564a63eb6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.338+0000 7f92052de640 1 -- 172.21.15.64:0/14480684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a63b04070 msgr2=0x564a63ec2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:27.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.338+0000 7f92052de640 1 --2- 172.21.15.64:0/14480684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a63b04070 0x564a63ec2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.338+0000 7f92052de640 1 -- 172.21.15.64:0/14480684 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a63dd9c40 con 0x564a63adb1b0 2024-09-21T04:05:27.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.338+0000 7f92052de640 1 --2- 172.21.15.64:0/14480684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a63adb1b0 0x564a63eb68d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f91fc006010 tx=0x7f91fc02a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=118389375f3d6429 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:27.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.338+0000 7f91e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-21T04:05:27.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.339+0000 7f91e8ff9640 1 -- 172.21.15.64:0/14480684 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc040070 con 0x564a63adb1b0 2024-09-21T04:05:27.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.339+0000 7f920dab6740 1 -- 172.21.15.64:0/14480684 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91fc004030 con 0x564a63adb1b0 2024-09-21T04:05:27.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.339+0000 7f91e8ff9640 1 -- 172.21.15.64:0/14480684 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f91fc0314a0 con 0x564a63adb1b0 2024-09-21T04:05:27.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.339+0000 7f91e8ff9640 1 -- 172.21.15.64:0/14480684 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f91fc056050 con 0x564a63adb1b0 2024-09-21T04:05:27.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.339+0000 7f920dab6740 10 client.25450 fetch_fsmap learned FSMap version 27 2024-09-21T04:05:27.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.339+0000 7f920dab6740 1 -- 172.21.15.64:0/14480684 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564a63ec56a0 con 0x564a63adb1b0 2024-09-21T04:05:27.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.339+0000 7f91e8ff9640 1 -- 172.21.15.64:0/14480684 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91fc004030 con 0x564a63adb1b0 2024-09-21T04:05:27.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.339+0000 7f91e8ff9640 1 -- 172.21.15.64:0/14480684 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f91fc004b70 con 0x564a63adb1b0 2024-09-21T04:05:27.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.340+0000 7f920dab6740 10 client.25450 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T04:05:27.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.340+0000 7f920dab6740 10 client.25450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:05:27.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.340+0000 7f920dab6740 20 client.25450 populate_metadata read hostname 'smithi064' 2024-09-21T04:05:27.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.340+0000 7f920dab6740 1 --2- 172.21.15.64:0/14480684 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564a63ac29f0 0x564a63ec8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:27.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.340+0000 7f920dab6740 4 client.25450 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:05:27.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.340+0000 7f920dab6740 1 -- 172.21.15.64:0/14480684 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a63e23da0 con 0x564a63ac29f0 2024-09-21T04:05:27.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.340+0000 7f9204add640 1 --2- 172.21.15.64:0/14480684 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564a63ac29f0 0x564a63ec8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:27.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.340+0000 7f91e8ff9640 10 client.25450 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:27.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.340+0000 7f9204add640 1 --2- 172.21.15.64:0/14480684 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564a63ac29f0 0x564a63ec8a00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4b9c281ec3dd836 server_cookie=2cd5eb38ed34dac4 in_seq=0 out_seq=0 2024-09-21T04:05:27.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.341+0000 7f91e8ff9640 1 -- 172.21.15.64:0/14480684 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564a63e23da0 con 0x564a63ac29f0 2024-09-21T04:05:27.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.341+0000 7f91e8ff9640 10 client.25450 handle_command_reply: tid=0 2024-09-21T04:05:27.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.341+0000 7f920dab6740 1 client.25450 shutdown 2024-09-21T04:05:27.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.341+0000 7f920dab6740 1 -- 172.21.15.64:0/14480684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a63adb1b0 msgr2=0x564a63eb68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:27.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.341+0000 7f920dab6740 1 --2- 172.21.15.64:0/14480684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a63adb1b0 0x564a63eb68d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f91fc006010 tx=0x7f91fc02a410 comp rx=0 tx=0).stop 2024-09-21T04:05:27.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.341+0000 7f920dab6740 1 -- 172.21.15.64:0/14480684 shutdown_connections 2024-09-21T04:05:27.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.341+0000 7f920dab6740 1 --2- 172.21.15.64:0/14480684 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564a63ac29f0 0x564a63ec8a00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.341+0000 7f920dab6740 1 --2- 172.21.15.64:0/14480684 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a63b04070 0x564a63ec2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.341+0000 7f920dab6740 1 --2- 172.21.15.64:0/14480684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564a63ae3390 0x564a63eb6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.341+0000 7f91e8ff9640 0 client.25450 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:27.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.341+0000 7f920dab6740 1 --2- 172.21.15.64:0/14480684 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a63adb1b0 0x564a63eb68d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.341+0000 7f920dab6740 1 -- 172.21.15.64:0/14480684 >> 172.21.15.64:0/14480684 conn(0x564a63af8460 msgr2=0x564a63eae450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:27.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.342+0000 7f920dab6740 1 -- 172.21.15.64:0/14480684 shutdown_connections 2024-09-21T04:05:27.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.342+0000 7f920dab6740 1 -- 172.21.15.64:0/14480684 wait complete. 2024-09-21T04:05:27.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.342+0000 7f920dab6740 20 client.25450 trim_cache size 0 max 16384 2024-09-21T04:05:27.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.359+0000 7f920dab6740 1 Processor -- start 2024-09-21T04:05:27.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.359+0000 7f920dab6740 1 -- start start 2024-09-21T04:05:27.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.360+0000 7f920dab6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a63ac29f0 0x564a63eacc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:27.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.360+0000 7f920dab6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564a63adb1b0 0x564a63eb8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:27.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.360+0000 7f920dab6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a63ae3390 0x564a63ea8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:27.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.360+0000 7f920dab6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f92001aff80 con 0x564a63ac29f0 2024-09-21T04:05:27.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.360+0000 7f920dab6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f92001ad850 con 0x564a63adb1b0 2024-09-21T04:05:27.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.360+0000 7f920dab6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f92001ab6a0 con 0x564a63ae3390 2024-09-21T04:05:27.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.360+0000 7f92052de640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a63ac29f0 0x564a63eacc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:27.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.360+0000 7f9205adf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a63ae3390 0x564a63ea8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:27.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.360+0000 7f92052de640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a63ac29f0 0x564a63eacc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55744/0 (socket says 172.21.15.64:55744) 2024-09-21T04:05:27.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.360+0000 7f9205adf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a63ae3390 0x564a63ea8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:47006/0 (socket says 172.21.15.64:47006) 2024-09-21T04:05:27.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.360+0000 7f92052de640 1 -- 172.21.15.64:0/2311884860 learned_addr learned my addr 172.21.15.64:0/2311884860 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:27.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.360+0000 7f9204add640 1 --2- 172.21.15.64:0/2311884860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564a63adb1b0 0x564a63eb8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:27.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.360+0000 7f9205adf640 1 -- 172.21.15.64:0/2311884860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564a63adb1b0 msgr2=0x564a63eb8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:27.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.360+0000 7f9205adf640 1 --2- 172.21.15.64:0/2311884860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564a63adb1b0 0x564a63eb8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.360+0000 7f9205adf640 1 -- 172.21.15.64:0/2311884860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a63ac29f0 msgr2=0x564a63eacc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:27.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.360+0000 7f9205adf640 1 --2- 172.21.15.64:0/2311884860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a63ac29f0 0x564a63eacc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.360+0000 7f9205adf640 1 -- 172.21.15.64:0/2311884860 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a63ea9a50 con 0x564a63ae3390 2024-09-21T04:05:27.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.360+0000 7f92052de640 1 --2- 172.21.15.64:0/2311884860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a63ac29f0 0x564a63eacc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:05:27.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.361+0000 7f9205adf640 1 --2- 172.21.15.64:0/2311884860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a63ae3390 0x564a63ea8710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f91f800f790 tx=0x7f91f801b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6f1b71df29707b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:27.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.361+0000 7f91eaffd640 1 -- 172.21.15.64:0/2311884860 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f800fd60 con 0x564a63ae3390 2024-09-21T04:05:27.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.361+0000 7f91eaffd640 1 -- 172.21.15.64:0/2311884860 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91f80045c0 con 0x564a63ae3390 2024-09-21T04:05:27.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.361+0000 7f91eaffd640 1 -- 172.21.15.64:0/2311884860 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f801f430 con 0x564a63ae3390 2024-09-21T04:05:27.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.362+0000 7f920dab6740 1 -- 172.21.15.64:0/2311884860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a63ae3390 msgr2=0x564a63ea8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:27.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.362+0000 7f920dab6740 1 --2- 172.21.15.64:0/2311884860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a63ae3390 0x564a63ea8710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f91f800f790 tx=0x7f91f801b000 comp rx=0 tx=0).stop 2024-09-21T04:05:27.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.362+0000 7f920dab6740 1 -- 172.21.15.64:0/2311884860 shutdown_connections 2024-09-21T04:05:27.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.362+0000 7f920dab6740 1 --2- 172.21.15.64:0/2311884860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a63ae3390 0x564a63ea8710 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.362+0000 7f920dab6740 1 --2- 172.21.15.64:0/2311884860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564a63adb1b0 0x564a63eb8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.362+0000 7f920dab6740 1 --2- 172.21.15.64:0/2311884860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a63ac29f0 0x564a63eacc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.362+0000 7f920dab6740 1 -- 172.21.15.64:0/2311884860 >> 172.21.15.64:0/2311884860 conn(0x564a63af8460 msgr2=0x564a63eb7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:27.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.362+0000 7f920dab6740 1 -- 172.21.15.64:0/2311884860 shutdown_connections 2024-09-21T04:05:27.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.362+0000 7f920dab6740 1 -- 172.21.15.64:0/2311884860 wait complete. 2024-09-21T04:05:27.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.362+0000 7f920dab6740 1 -- start start 2024-09-21T04:05:27.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.362+0000 7f920dab6740 1 Processor -- start 2024-09-21T04:05:27.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.363+0000 7f920dab6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564a63ac29f0 0x564a63ecc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:27.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.363+0000 7f920dab6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a63adb1b0 0x564a63ec96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:27.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.363+0000 7f920dab6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a63ae3390 0x564a63ec9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:27.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.363+0000 7f920dab6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f92001abd10 con 0x564a63ae3390 2024-09-21T04:05:27.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.363+0000 7f920dab6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f92001ad180 con 0x564a63ac29f0 2024-09-21T04:05:27.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.363+0000 7f920dab6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f92001ac3e0 con 0x564a63adb1b0 2024-09-21T04:05:27.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.363+0000 7f920dab6740 10 client.0 _ll_register_callbacks cb 0x7f91f00eeb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:05:27.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.363+0000 7f92052de640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564a63ac29f0 0x564a63ecc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:27.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.363+0000 7f92052de640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564a63ac29f0 0x564a63ecc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:35528/0 (socket says 172.21.15.64:35528) 2024-09-21T04:05:27.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.363+0000 7f92052de640 1 -- 172.21.15.64:0/1328171992 learned_addr learned my addr 172.21.15.64:0/1328171992 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:27.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.363+0000 7f9205adf640 1 --2- 172.21.15.64:0/1328171992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a63ae3390 0x564a63ec9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:27.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.363+0000 7f9204add640 1 --2- 172.21.15.64:0/1328171992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a63adb1b0 0x564a63ec96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:27.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.364+0000 7f92052de640 1 -- 172.21.15.64:0/1328171992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a63adb1b0 msgr2=0x564a63ec96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:27.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.364+0000 7f92052de640 1 --2- 172.21.15.64:0/1328171992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a63adb1b0 0x564a63ec96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.364+0000 7f92052de640 1 -- 172.21.15.64:0/1328171992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a63ae3390 msgr2=0x564a63ec9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:27.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.364+0000 7f92052de640 1 --2- 172.21.15.64:0/1328171992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a63ae3390 0x564a63ec9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.364+0000 7f92052de640 1 -- 172.21.15.64:0/1328171992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a63eca320 con 0x564a63ac29f0 2024-09-21T04:05:27.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.364+0000 7f9204add640 1 --2- 172.21.15.64:0/1328171992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a63adb1b0 0x564a63ec96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:05:27.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.364+0000 7f92052de640 1 --2- 172.21.15.64:0/1328171992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564a63ac29f0 0x564a63ecc1d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f91fc003640 tx=0x7f91fc032b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97abd802c73860fd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:27.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.364+0000 7f91e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:05:27.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.364+0000 7f91e8ff9640 1 -- 172.21.15.64:0/1328171992 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc040040 con 0x564a63ac29f0 2024-09-21T04:05:27.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.364+0000 7f91e8ff9640 1 -- 172.21.15.64:0/1328171992 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f91fc0316a0 con 0x564a63ac29f0 2024-09-21T04:05:27.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.364+0000 7f920dab6740 1 -- 172.21.15.64:0/1328171992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91f80045c0 con 0x564a63ac29f0 2024-09-21T04:05:27.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.364+0000 7f91e8ff9640 1 -- 172.21.15.64:0/1328171992 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f91fc035040 con 0x564a63ac29f0 2024-09-21T04:05:27.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.365+0000 7f920dab6740 10 client.25532 fetch_fsmap learned FSMap version 27 2024-09-21T04:05:27.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.365+0000 7f920dab6740 1 -- 172.21.15.64:0/1328171992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564a63f33080 con 0x564a63ac29f0 2024-09-21T04:05:27.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.365+0000 7f91e8ff9640 1 -- 172.21.15.64:0/1328171992 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91f80045c0 con 0x564a63ac29f0 2024-09-21T04:05:27.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.365+0000 7f91e8ff9640 1 -- 172.21.15.64:0/1328171992 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f91fc004b50 con 0x564a63ac29f0 2024-09-21T04:05:27.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.365+0000 7f920dab6740 10 client.25532 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T04:05:27.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.365+0000 7f920dab6740 10 client.25532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:05:27.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.365+0000 7f920dab6740 20 client.25532 populate_metadata read hostname 'smithi064' 2024-09-21T04:05:27.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.365+0000 7f920dab6740 1 --2- 172.21.15.64:0/1328171992 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564a63b04070 0x564a63f35750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:27.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.365+0000 7f920dab6740 4 client.25532 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T04:05:27.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.365+0000 7f920dab6740 1 -- 172.21.15.64:0/1328171992 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564a63e1f850 con 0x564a63b04070 2024-09-21T04:05:27.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.365+0000 7f9204add640 1 --2- 172.21.15.64:0/1328171992 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564a63b04070 0x564a63f35750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:27.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.366+0000 7f91e8ff9640 10 client.25532 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:27.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.366+0000 7f9204add640 1 --2- 172.21.15.64:0/1328171992 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564a63b04070 0x564a63f35750 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5100ad93c3064278 server_cookie=f5c4f88ed624d492 in_seq=0 out_seq=0 2024-09-21T04:05:27.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.366+0000 7f91e8ff9640 1 -- 172.21.15.64:0/1328171992 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+942 (crc 0 0 0) 0x564a63e1f850 con 0x564a63b04070 2024-09-21T04:05:27.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.366+0000 7f91e8ff9640 10 client.25532 handle_command_reply: tid=0 2024-09-21T04:05:27.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.366+0000 7f920dab6740 1 client.25532 shutdown 2024-09-21T04:05:27.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.366+0000 7f920dab6740 1 -- 172.21.15.64:0/1328171992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564a63ac29f0 msgr2=0x564a63ecc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:27.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.366+0000 7f920dab6740 1 --2- 172.21.15.64:0/1328171992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564a63ac29f0 0x564a63ecc1d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f91fc003640 tx=0x7f91fc032b20 comp rx=0 tx=0).stop 2024-09-21T04:05:27.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.366+0000 7f920dab6740 1 -- 172.21.15.64:0/1328171992 shutdown_connections 2024-09-21T04:05:27.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.366+0000 7f920dab6740 1 --2- 172.21.15.64:0/1328171992 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564a63b04070 0x564a63f35750 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.366+0000 7f920dab6740 1 --2- 172.21.15.64:0/1328171992 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a63ae3390 0x564a63ec9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.366+0000 7f920dab6740 1 --2- 172.21.15.64:0/1328171992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564a63adb1b0 0x564a63ec96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.366+0000 7f920dab6740 1 --2- 172.21.15.64:0/1328171992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564a63ac29f0 0x564a63ecc1d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.366+0000 7f920dab6740 1 -- 172.21.15.64:0/1328171992 >> 172.21.15.64:0/1328171992 conn(0x564a63af8460 msgr2=0x564a63eaca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:27.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.366+0000 7f91e8ff9640 0 client.25532 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:27.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.366+0000 7f920dab6740 1 -- 172.21.15.64:0/1328171992 shutdown_connections 2024-09-21T04:05:27.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.366+0000 7f920dab6740 1 -- 172.21.15.64:0/1328171992 wait complete. 2024-09-21T04:05:27.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.367+0000 7f920dab6740 20 client.25532 trim_cache size 0 max 16384 2024-09-21T04:05:27.373 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:05:27.373 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-21T04:05:27.373 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {}, 2024-09-21T04:05:27.373 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_stats": { 2024-09-21T04:05:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: "b52b950c-0f97-49ec-a37c-1a8e04348ce8": { 2024-09-21T04:05:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:54.003800+0000", 2024-09-21T04:05:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:05:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:05:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:05:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:05:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:05:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:05:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: "0ab717a7-36c6-4dca-94e7-af0bdf2aa112": { 2024-09-21T04:05:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:42.691086+0000", 2024-09-21T04:05:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:05:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:05:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:05:27.374 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:05:27.375 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:05:27.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:05:27.375 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:05:27.375 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:03:59.570174+0000", 2024-09-21T04:05:27.375 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:05:27.375 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:05:27.375 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:05:27.375 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:05:27.375 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:05:27.375 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:05:27.375 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:05:27.375 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:05:27.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.367+0000 7f9207542640 1 -- 172.21.15.64:0/1835390072 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f91ec0b6680 msgr2=0x7f91ec0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:27.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.367+0000 7f9207542640 1 --2- 172.21.15.64:0/1835390072 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f91ec0b6680 0x7f91ec0b8b90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f91f8005fd0 tx=0x7f91f8009500 comp rx=0 tx=0).stop 2024-09-21T04:05:27.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.367+0000 7f9207542640 1 -- 172.21.15.64:0/1835390072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9200105340 msgr2=0x7f920019aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:27.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.367+0000 7f9207542640 1 --2- 172.21.15.64:0/1835390072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9200105340 0x7f920019aa80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f91f0009bb0 tx=0x7f91f000b690 comp rx=0 tx=0).stop 2024-09-21T04:05:27.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.367+0000 7f9207542640 1 -- 172.21.15.64:0/1835390072 shutdown_connections 2024-09-21T04:05:27.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.367+0000 7f9207542640 1 --2- 172.21.15.64:0/1835390072 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f91ec0b6680 0x7f91ec0b8b90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.367+0000 7f9207542640 1 --2- 172.21.15.64:0/1835390072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f920010de50 0x7f920019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.367+0000 7f9207542640 1 --2- 172.21.15.64:0/1835390072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9200105340 0x7f920019aa80 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.367+0000 7f9207542640 1 --2- 172.21.15.64:0/1835390072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92001049f0 0x7f920019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:27.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.367+0000 7f9207542640 1 -- 172.21.15.64:0/1835390072 >> 172.21.15.64:0/1835390072 conn(0x7f92000797e0 msgr2=0x7f92001008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:27.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.368+0000 7f9207542640 1 -- 172.21.15.64:0/1835390072 shutdown_connections 2024-09-21T04:05:27.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:27.368+0000 7f9207542640 1 -- 172.21.15.64:0/1835390072 wait complete. 2024-09-21T04:05:27.397 INFO:tasks.workunit.client.0.smithi064.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2024-09-21T04:05:27.397 INFO:tasks.workunit.client.0.smithi064.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2024-09-21T04:05:27.488 INFO:tasks.workunit.client.0.smithi064.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2024-09-21T04:05:27.488 INFO:tasks.workunit.client.0.smithi064.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2024-09-21T04:05:27.648 INFO:tasks.workunit.client.0.smithi064.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2024-09-21T04:05:27.648 INFO:tasks.workunit.client.0.smithi064.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2024-09-21T04:05:27.718 INFO:tasks.workunit.client.0.smithi064.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2024-09-21T04:05:27.718 INFO:tasks.workunit.client.0.smithi064.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2024-09-21T04:05:27.788 INFO:tasks.workunit.client.0.smithi064.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2024-09-21T04:05:27.788 INFO:tasks.workunit.client.0.smithi064.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2024-09-21T04:05:27.849 INFO:tasks.workunit.client.0.smithi064.stdout:gcc -g -O2 -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2024-09-21T04:05:27.870 INFO:tasks.workunit.client.0.smithi064.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-09-21T04:05:27.870 INFO:tasks.workunit.client.0.smithi064.stdout:Making all in man 2024-09-21T04:05:27.875 INFO:tasks.workunit.client.0.smithi064.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-09-21T04:05:27.875 INFO:tasks.workunit.client.0.smithi064.stdout:make[2]: Nothing to be done for 'all'. 2024-09-21T04:05:27.875 INFO:tasks.workunit.client.0.smithi064.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-09-21T04:05:27.876 INFO:tasks.workunit.client.0.smithi064.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-21T04:05:27.877 INFO:tasks.workunit.client.0.smithi064.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-21T04:05:27.878 INFO:tasks.workunit.client.0.smithi064.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-21T04:05:27.878 INFO:tasks.workunit.client.0.smithi064.stderr:+ cd src 2024-09-21T04:05:27.883 INFO:tasks.workunit.client.0.smithi064.stderr:+ mkdir blogtest_in 2024-09-21T04:05:27.887 INFO:tasks.workunit.client.0.smithi064.stdout:running blogbench 2024-09-21T04:05:27.887 INFO:tasks.workunit.client.0.smithi064.stderr:+ echo 'running blogbench' 2024-09-21T04:05:27.887 INFO:tasks.workunit.client.0.smithi064.stderr:+ ./blogbench -d blogtest_in 2024-09-21T04:05:27.887 INFO:tasks.workunit.client.0.smithi064.stdout: 2024-09-21T04:05:27.887 INFO:tasks.workunit.client.0.smithi064.stdout:Frequency = 10 secs 2024-09-21T04:05:27.887 INFO:tasks.workunit.client.0.smithi064.stdout:Scratch dir = [blogtest_in] 2024-09-21T04:05:27.950 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:27 smithi064 ceph-mon[31180]: pgmap v364: 97 pgs: 97 active+clean; 5.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 252 KiB/s wr, 65 op/s 2024-09-21T04:05:28.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:27 smithi131 ceph-mon[35193]: pgmap v364: 97 pgs: 97 active+clean; 5.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 252 KiB/s wr, 65 op/s 2024-09-21T04:05:28.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:27 smithi119 ceph-mon[34811]: pgmap v364: 97 pgs: 97 active+clean; 5.1 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 252 KiB/s wr, 65 op/s 2024-09-21T04:05:30.022 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:29 smithi064 ceph-mon[31180]: pgmap v365: 97 pgs: 97 active+clean; 5.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 163 KiB/s wr, 42 op/s 2024-09-21T04:05:30.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:29 smithi131 ceph-mon[35193]: pgmap v365: 97 pgs: 97 active+clean; 5.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 163 KiB/s wr, 42 op/s 2024-09-21T04:05:30.258 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:29 smithi119 ceph-mon[34811]: pgmap v365: 97 pgs: 97 active+clean; 5.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 163 KiB/s wr, 42 op/s 2024-09-21T04:05:31.166 INFO:tasks.workunit.client.0.smithi064.stdout:Spawning 3 writers... 2024-09-21T04:05:31.167 INFO:tasks.workunit.client.0.smithi064.stdout:Spawning 1 rewriters... 2024-09-21T04:05:31.167 INFO:tasks.workunit.client.0.smithi064.stdout:Spawning 5 commenters... 2024-09-21T04:05:31.167 INFO:tasks.workunit.client.0.smithi064.stdout:Spawning 100 readers... 2024-09-21T04:05:31.171 INFO:tasks.workunit.client.0.smithi064.stdout:Benchmarking for 30 iterations. 2024-09-21T04:05:31.171 INFO:tasks.workunit.client.0.smithi064.stdout:The test will run during 5 minutes. 2024-09-21T04:05:31.171 INFO:tasks.workunit.client.0.smithi064.stdout: 2024-09-21T04:05:31.171 INFO:tasks.workunit.client.0.smithi064.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2024-09-21T04:05:31.354 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T04:05:31.354 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:05:31.631 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:05:32.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:31 smithi064 ceph-mon[31180]: pgmap v366: 97 pgs: 97 active+clean; 5.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 60 KiB/s wr, 28 op/s 2024-09-21T04:05:32.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:31 smithi131 ceph-mon[35193]: pgmap v366: 97 pgs: 97 active+clean; 5.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 60 KiB/s wr, 28 op/s 2024-09-21T04:05:32.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.122+0000 7f55690a5640 1 -- 172.21.15.64:0/2373669830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55641083a0 msgr2=0x7f55641087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:32.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.122+0000 7f55690a5640 1 --2- 172.21.15.64:0/2373669830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55641083a0 0x7f55641087b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f554c009900 tx=0x7f554c02a410 comp rx=0 tx=0).stop 2024-09-21T04:05:32.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.122+0000 7f55690a5640 1 -- 172.21.15.64:0/2373669830 shutdown_connections 2024-09-21T04:05:32.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.122+0000 7f55690a5640 1 --2- 172.21.15.64:0/2373669830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5564108cf0 0x7f556410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.122+0000 7f55690a5640 1 --2- 172.21.15.64:0/2373669830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55641083a0 0x7f55641087b0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.122+0000 7f55690a5640 1 --2- 172.21.15.64:0/2373669830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5564102a60 0x7f5564102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.122+0000 7f55690a5640 1 -- 172.21.15.64:0/2373669830 >> 172.21.15.64:0/2373669830 conn(0x7f5564079730 msgr2=0x7f5564079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:32.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.122+0000 7f55690a5640 1 -- 172.21.15.64:0/2373669830 shutdown_connections 2024-09-21T04:05:32.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.122+0000 7f55690a5640 1 -- 172.21.15.64:0/2373669830 wait complete. 2024-09-21T04:05:32.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.123+0000 7f55690a5640 1 Processor -- start 2024-09-21T04:05:32.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.123+0000 7f55690a5640 1 -- start start 2024-09-21T04:05:32.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.123+0000 7f55690a5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5564102a60 0x7f556410ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:32.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.123+0000 7f55690a5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55641083a0 0x7f55641104f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:32.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.123+0000 7f55690a5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5564108cf0 0x7f5564110a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:32.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.123+0000 7f55690a5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5564113a80 con 0x7f55641083a0 2024-09-21T04:05:32.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.123+0000 7f55690a5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5564113900 con 0x7f5564108cf0 2024-09-21T04:05:32.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.123+0000 7f55690a5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5564113c00 con 0x7f5564102a60 2024-09-21T04:05:32.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.123+0000 7f556259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55641083a0 0x7f55641104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:32.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.123+0000 7f5562d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5564102a60 0x7f556410ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:32.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.124+0000 7f556259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55641083a0 0x7f55641104f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35870/0 (socket says 172.21.15.64:35870) 2024-09-21T04:05:32.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.124+0000 7f556259c640 1 -- 172.21.15.64:0/3467146458 learned_addr learned my addr 172.21.15.64:0/3467146458 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:32.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.124+0000 7f556359e640 1 --2- 172.21.15.64:0/3467146458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5564108cf0 0x7f5564110a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:32.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.124+0000 7f5562d9d640 1 -- 172.21.15.64:0/3467146458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5564108cf0 msgr2=0x7f5564110a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:32.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.124+0000 7f5562d9d640 1 --2- 172.21.15.64:0/3467146458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5564108cf0 0x7f5564110a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.124+0000 7f5562d9d640 1 -- 172.21.15.64:0/3467146458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55641083a0 msgr2=0x7f55641104f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:32.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.124+0000 7f5562d9d640 1 --2- 172.21.15.64:0/3467146458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55641083a0 0x7f55641104f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.124+0000 7f5562d9d640 1 -- 172.21.15.64:0/3467146458 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55641a87f0 con 0x7f5564102a60 2024-09-21T04:05:32.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.124+0000 7f556259c640 1 --2- 172.21.15.64:0/3467146458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55641083a0 0x7f55641104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:05:32.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.124+0000 7f556359e640 1 --2- 172.21.15.64:0/3467146458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5564108cf0 0x7f5564110a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:05:32.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.124+0000 7f5562d9d640 1 --2- 172.21.15.64:0/3467146458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5564102a60 0x7f556410ffb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f555400e9f0 tx=0x7f555400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:32.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.124+0000 7f555bfff640 1 -- 172.21.15.64:0/3467146458 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555400ccc0 con 0x7f5564102a60 2024-09-21T04:05:32.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.124+0000 7f55690a5640 1 -- 172.21.15.64:0/3467146458 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55641a8ae0 con 0x7f5564102a60 2024-09-21T04:05:32.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.124+0000 7f555bfff640 1 -- 172.21.15.64:0/3467146458 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5554005020 con 0x7f5564102a60 2024-09-21T04:05:32.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.124+0000 7f55690a5640 1 -- 172.21.15.64:0/3467146458 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55641a9020 con 0x7f5564102a60 2024-09-21T04:05:32.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.124+0000 7f555bfff640 1 -- 172.21.15.64:0/3467146458 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5554004a00 con 0x7f5564102a60 2024-09-21T04:05:32.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.125+0000 7f55690a5640 1 -- 172.21.15.64:0/3467146458 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5564102e70 con 0x7f5564102a60 2024-09-21T04:05:32.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.127+0000 7f555bfff640 1 -- 172.21.15.64:0/3467146458 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f5554004ba0 con 0x7f5564102a60 2024-09-21T04:05:32.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.128+0000 7f555bfff640 1 --2- 172.21.15.64:0/3467146458 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f55500b6700 0x7f55500b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:32.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.128+0000 7f556259c640 1 --2- 172.21.15.64:0/3467146458 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f55500b6700 0x7f55500b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:32.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.128+0000 7f555bfff640 1 -- 172.21.15.64:0/3467146458 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f55540e77e0 con 0x7f5564102a60 2024-09-21T04:05:32.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.129+0000 7f556259c640 1 --2- 172.21.15.64:0/3467146458 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f55500b6700 0x7f55500b8c10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f554c0096c0 tx=0x7f554c032ed0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:32.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.135+0000 7f555bfff640 1 -- 172.21.15.64:0/3467146458 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f55540add90 con 0x7f5564102a60 2024-09-21T04:05:32.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:31 smithi119 ceph-mon[34811]: pgmap v366: 97 pgs: 97 active+clean; 5.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 60 KiB/s wr, 28 op/s 2024-09-21T04:05:32.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.369+0000 7f55690a5640 1 -- 172.21.15.64:0/3467146458 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55641112b0 con 0x7f5564102a60 2024-09-21T04:05:32.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.369+0000 7f555bfff640 1 -- 172.21.15.64:0/3467146458 <== mon.1 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11915 (secure 0 0 0) 0x7f55641112b0 con 0x7f5564102a60 2024-09-21T04:05:32.373 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:05:32.373 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":28,"btime":"2024-09-21T04:05:31:766361+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14805,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/97995119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":97995119},{"type":"v1","addr":"172.21.15.64:6839","nonce":97995119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/390891068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":390891068},{"type":"v1","addr":"172.21.15.64:6841","nonce":390891068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:02:03.354744+0000","modified":"2024-09-21T04:05:31.675250+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24710,"mds_1":24680,"mds_2":24661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24634":{"gid":24634,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6839/782190424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782190424},{"type":"v1","addr":"172.21.15.131:6839","nonce":782190424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24680":{"gid":24680,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24710":{"gid":24710,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6839/1319793636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1319793636},{"type":"v1","addr":"172.21.15.119:6839","nonce":1319793636}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24710,"qdb_cluster":[24661,24680,24710]},"id":1}]} 2024-09-21T04:05:32.373 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 28 2024-09-21T04:05:32.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.372+0000 7f55690a5640 1 -- 172.21.15.64:0/3467146458 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f55500b6700 msgr2=0x7f55500b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:32.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.372+0000 7f55690a5640 1 --2- 172.21.15.64:0/3467146458 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f55500b6700 0x7f55500b8c10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f554c0096c0 tx=0x7f554c032ed0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.372+0000 7f55690a5640 1 -- 172.21.15.64:0/3467146458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5564102a60 msgr2=0x7f556410ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:32.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.372+0000 7f55690a5640 1 --2- 172.21.15.64:0/3467146458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5564102a60 0x7f556410ffb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f555400e9f0 tx=0x7f555400eeb0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.372+0000 7f55690a5640 1 -- 172.21.15.64:0/3467146458 shutdown_connections 2024-09-21T04:05:32.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.372+0000 7f55690a5640 1 --2- 172.21.15.64:0/3467146458 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f55500b6700 0x7f55500b8c10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.372+0000 7f55690a5640 1 --2- 172.21.15.64:0/3467146458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5564108cf0 0x7f5564110a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.372+0000 7f55690a5640 1 --2- 172.21.15.64:0/3467146458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55641083a0 0x7f55641104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.372+0000 7f55690a5640 1 --2- 172.21.15.64:0/3467146458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5564102a60 0x7f556410ffb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.372+0000 7f55690a5640 1 -- 172.21.15.64:0/3467146458 >> 172.21.15.64:0/3467146458 conn(0x7f5564079730 msgr2=0x7f5564104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:32.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.372+0000 7f55690a5640 1 -- 172.21.15.64:0/3467146458 shutdown_connections 2024-09-21T04:05:32.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:32.372+0000 7f55690a5640 1 -- 172.21.15.64:0/3467146458 wait complete. 2024-09-21T04:05:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.747+0000 7f15b85da640 1 -- 172.21.15.64:0/2959980022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15b0108c30 msgr2=0x7f15b010f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.747+0000 7f15b85da640 1 --2- 172.21.15.64:0/2959980022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15b0108c30 0x7f15b010f460 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f15a8009ca0 tx=0x7f15a802a8c0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.747+0000 7f15b85da640 1 -- 172.21.15.64:0/2959980022 shutdown_connections 2024-09-21T04:05:32.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.747+0000 7f15b85da640 1 --2- 172.21.15.64:0/2959980022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15b0108c30 0x7f15b010f460 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.747+0000 7f15b85da640 1 --2- 172.21.15.64:0/2959980022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15b01082e0 0x7f15b01086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.747+0000 7f15b85da640 1 --2- 172.21.15.64:0/2959980022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15b01029b0 0x7f15b0102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.747+0000 7f15b85da640 1 -- 172.21.15.64:0/2959980022 >> 172.21.15.64:0/2959980022 conn(0x7f15b007a740 msgr2=0x7f15b007ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.748+0000 7f15b85da640 1 -- 172.21.15.64:0/2959980022 shutdown_connections 2024-09-21T04:05:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.748+0000 7f15b85da640 1 -- 172.21.15.64:0/2959980022 wait complete. 2024-09-21T04:05:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.748+0000 7f15b85da640 1 Processor -- start 2024-09-21T04:05:32.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.748+0000 7f15b85da640 1 -- start start 2024-09-21T04:05:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.748+0000 7f15b85da640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15b01029b0 0x7f15b019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.748+0000 7f15b85da640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15b01082e0 0x7f15b019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.748+0000 7f15b85da640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15b0108c30 0x7f15b019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.748+0000 7f15b85da640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f15b01128f0 con 0x7f15b0108c30 2024-09-21T04:05:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.748+0000 7f15b85da640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f15b0112770 con 0x7f15b01029b0 2024-09-21T04:05:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.748+0000 7f15b85da640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f15b0112a70 con 0x7f15b01082e0 2024-09-21T04:05:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.749+0000 7f15b6b77640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15b0108c30 0x7f15b019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.749+0000 7f15b6b77640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15b0108c30 0x7f15b019fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35900/0 (socket says 172.21.15.64:35900) 2024-09-21T04:05:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.749+0000 7f15b6b77640 1 -- 172.21.15.64:0/4085615551 learned_addr learned my addr 172.21.15.64:0/4085615551 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.749+0000 7f15b5b75640 1 --2- 172.21.15.64:0/4085615551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15b01082e0 0x7f15b019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.749+0000 7f15b6376640 1 --2- 172.21.15.64:0/4085615551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15b01029b0 0x7f15b019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.749+0000 7f15b6b77640 1 -- 172.21.15.64:0/4085615551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15b01082e0 msgr2=0x7f15b019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:32.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.749+0000 7f15b6b77640 1 --2- 172.21.15.64:0/4085615551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15b01082e0 0x7f15b019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.749+0000 7f15b6b77640 1 -- 172.21.15.64:0/4085615551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15b01029b0 msgr2=0x7f15b019a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.749+0000 7f15b6b77640 1 --2- 172.21.15.64:0/4085615551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15b01029b0 0x7f15b019a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.749+0000 7f15b6b77640 1 -- 172.21.15.64:0/4085615551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b01a0370 con 0x7f15b0108c30 2024-09-21T04:05:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.749+0000 7f15b6b77640 1 --2- 172.21.15.64:0/4085615551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15b0108c30 0x7f15b019fc70 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f15a80098d0 tx=0x7f15a80317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.749+0000 7f15a77fe640 1 -- 172.21.15.64:0/4085615551 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a8003b70 con 0x7f15b0108c30 2024-09-21T04:05:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.749+0000 7f15b85da640 1 -- 172.21.15.64:0/4085615551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b01a0600 con 0x7f15b0108c30 2024-09-21T04:05:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.749+0000 7f15a77fe640 1 -- 172.21.15.64:0/4085615551 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15a8003d10 con 0x7f15b0108c30 2024-09-21T04:05:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.749+0000 7f15a77fe640 1 -- 172.21.15.64:0/4085615551 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a802fa20 con 0x7f15b0108c30 2024-09-21T04:05:32.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.749+0000 7f15b85da640 1 -- 172.21.15.64:0/4085615551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b01a6df0 con 0x7f15b0108c30 2024-09-21T04:05:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.752+0000 7f15bf622740 1 Processor -- start 2024-09-21T04:05:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.752+0000 7f15a77fe640 1 -- 172.21.15.64:0/4085615551 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f15a802fc40 con 0x7f15b0108c30 2024-09-21T04:05:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.752+0000 7f15bf622740 1 -- start start 2024-09-21T04:05:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.753+0000 7f15bf622740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b31d3711b0 0x55b31d7169b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:32.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.753+0000 7f15bf622740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b31d3482f0 0x55b31d716ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.753+0000 7f15bf622740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b31d3504d0 0x55b31d717430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.753+0000 7f15bf622740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b31d68b6f0 con 0x55b31d3482f0 2024-09-21T04:05:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.753+0000 7f15bf622740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b31d67ab20 con 0x55b31d3711b0 2024-09-21T04:05:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.753+0000 7f15bf622740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b31d67e930 con 0x55b31d3504d0 2024-09-21T04:05:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.753+0000 7f15b6376640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b31d3482f0 0x55b31d716ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.753+0000 7f15b5b75640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b31d3504d0 0x55b31d717430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.753+0000 7f15b6376640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b31d3482f0 0x55b31d716ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35914/0 (socket says 172.21.15.64:35914) 2024-09-21T04:05:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.753+0000 7f15b6376640 1 -- 172.21.15.64:0/3134248133 learned_addr learned my addr 172.21.15.64:0/3134248133 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.753+0000 7f15b5b75640 1 --2- 172.21.15.64:0/3134248133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b31d3711b0 0x55b31d7169b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.753+0000 7f15b5b75640 1 -- 172.21.15.64:0/3134248133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b31d3711b0 msgr2=0x55b31d7169b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.753+0000 7f15b5b75640 1 --2- 172.21.15.64:0/3134248133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b31d3711b0 0x55b31d7169b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.753+0000 7f15b5b75640 1 -- 172.21.15.64:0/3134248133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b31d3482f0 msgr2=0x55b31d716ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.753+0000 7f15b5b75640 1 --2- 172.21.15.64:0/3134248133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b31d3482f0 0x55b31d716ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.753+0000 7f15a77fe640 1 --2- 172.21.15.64:0/4085615551 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f159c0b6680 0x7f159c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.753+0000 7f15b5b75640 1 -- 172.21.15.64:0/3134248133 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b31d640530 con 0x55b31d3504d0 2024-09-21T04:05:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.753+0000 7f15b6b77640 1 --2- 172.21.15.64:0/4085615551 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f159c0b6680 0x7f159c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.754+0000 7f15b5b75640 1 --2- 172.21.15.64:0/3134248133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b31d3504d0 0x55b31d717430 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f15b006c9c0 tx=0x7f15a00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be2bb7e3d7bfa42a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.754+0000 7f15a77fe640 1 -- 172.21.15.64:0/4085615551 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f15a8108a00 con 0x7f15b0108c30 2024-09-21T04:05:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.754+0000 7f15a4bfe640 1 -- 172.21.15.64:0/3134248133 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a0005020 con 0x55b31d3504d0 2024-09-21T04:05:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.754+0000 7f15a4bfe640 1 -- 172.21.15.64:0/3134248133 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15a00047a0 con 0x55b31d3504d0 2024-09-21T04:05:32.759 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.754+0000 7f15a4bfe640 1 -- 172.21.15.64:0/3134248133 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a000f6e0 con 0x55b31d3504d0 2024-09-21T04:05:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.755+0000 7f15bf622740 1 -- 172.21.15.64:0/3134248133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b31d3504d0 msgr2=0x55b31d717430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.755+0000 7f15bf622740 1 --2- 172.21.15.64:0/3134248133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b31d3504d0 0x55b31d717430 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f15b006c9c0 tx=0x7f15a00046c0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.763 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.755+0000 7f15b6b77640 1 --2- 172.21.15.64:0/4085615551 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f159c0b6680 0x7f159c0b8b90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f15a80cba30 tx=0x7f15a80d1040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.755+0000 7f15bf622740 1 -- 172.21.15.64:0/3134248133 shutdown_connections 2024-09-21T04:05:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.755+0000 7f15bf622740 1 --2- 172.21.15.64:0/3134248133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b31d3504d0 0x55b31d717430 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.755+0000 7f15bf622740 1 --2- 172.21.15.64:0/3134248133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b31d3482f0 0x55b31d716ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.755+0000 7f15bf622740 1 --2- 172.21.15.64:0/3134248133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b31d3711b0 0x55b31d7169b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.755+0000 7f15bf622740 1 -- 172.21.15.64:0/3134248133 >> 172.21.15.64:0/3134248133 conn(0x55b31d365300 msgr2=0x55b31d7148a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.755+0000 7f15bf622740 1 -- 172.21.15.64:0/3134248133 shutdown_connections 2024-09-21T04:05:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.755+0000 7f15bf622740 1 -- 172.21.15.64:0/3134248133 wait complete. 2024-09-21T04:05:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.755+0000 7f15bf622740 1 -- start start 2024-09-21T04:05:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.755+0000 7f15bf622740 1 Processor -- start 2024-09-21T04:05:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.756+0000 7f15bf622740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b31d3482f0 0x55b31d722110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.756+0000 7f15bf622740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b31d3504d0 0x55b31d722650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.756+0000 7f15bf622740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b31d3711b0 0x55b31d72e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.756+0000 7f15bf622740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b31d5b2ab0 con 0x55b31d3504d0 2024-09-21T04:05:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.756+0000 7f15bf622740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b31d620730 con 0x55b31d3711b0 2024-09-21T04:05:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.756+0000 7f15bf622740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b31d725130 con 0x55b31d3482f0 2024-09-21T04:05:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.756+0000 7f15bf622740 10 client.0 _ll_register_callbacks cb 0x55b31d6746b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:05:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.756+0000 7f15b5b75640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b31d3482f0 0x55b31d722110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.756+0000 7f15b6376640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b31d3504d0 0x55b31d722650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.756+0000 7f15b5b75640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b31d3482f0 0x55b31d722110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:58700/0 (socket says 172.21.15.64:58700) 2024-09-21T04:05:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.756+0000 7f15b6376640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b31d3504d0 0x55b31d722650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35920/0 (socket says 172.21.15.64:35920) 2024-09-21T04:05:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.756+0000 7f15b5b75640 1 -- 172.21.15.64:0/1709766016 learned_addr learned my addr 172.21.15.64:0/1709766016 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.756+0000 7f15b5b75640 1 --2- 172.21.15.64:0/1709766016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b31d3711b0 0x55b31d72e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.756+0000 7f15b6376640 1 -- 172.21.15.64:0/1709766016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b31d3482f0 msgr2=0x55b31d722110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.756+0000 7f15b6376640 1 --2- 172.21.15.64:0/1709766016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b31d3482f0 0x55b31d722110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.756+0000 7f15b6376640 1 -- 172.21.15.64:0/1709766016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b31d3711b0 msgr2=0x55b31d72e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.756+0000 7f15b6376640 1 --2- 172.21.15.64:0/1709766016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b31d3711b0 0x55b31d72e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.756+0000 7f15b6376640 1 -- 172.21.15.64:0/1709766016 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b31d3dc790 con 0x55b31d3504d0 2024-09-21T04:05:32.765 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.756+0000 7f15b6376640 1 --2- 172.21.15.64:0/1709766016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b31d3504d0 0x55b31d722650 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x55b31d715630 tx=0x7f15ac011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191b0d23198b1085 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.756+0000 7f1599ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:05:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.756+0000 7f1599ffb640 1 -- 172.21.15.64:0/1709766016 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac004020 con 0x55b31d3504d0 2024-09-21T04:05:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.757+0000 7f1599ffb640 1 -- 172.21.15.64:0/1709766016 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f15ac0041c0 con 0x55b31d3504d0 2024-09-21T04:05:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.757+0000 7f15bf622740 1 -- 172.21.15.64:0/1709766016 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15a00047a0 con 0x55b31d3504d0 2024-09-21T04:05:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.757+0000 7f1599ffb640 1 -- 172.21.15.64:0/1709766016 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f15ac0043c0 con 0x55b31d3504d0 2024-09-21T04:05:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.757+0000 7f15bf622740 10 client.15837 fetch_fsmap learned FSMap version 28 2024-09-21T04:05:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.757+0000 7f15bf622740 1 -- 172.21.15.64:0/1709766016 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b31d455620 con 0x55b31d3504d0 2024-09-21T04:05:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.757+0000 7f1599ffb640 1 -- 172.21.15.64:0/1709766016 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15a00047a0 con 0x55b31d3504d0 2024-09-21T04:05:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.757+0000 7f1599ffb640 1 -- 172.21.15.64:0/1709766016 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f15ac020030 con 0x55b31d3504d0 2024-09-21T04:05:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.757+0000 7f15bf622740 10 client.15837 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T04:05:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.757+0000 7f15bf622740 10 client.15837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:05:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.757+0000 7f15bf622740 20 client.15837 populate_metadata read hostname 'smithi064' 2024-09-21T04:05:32.766 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.757+0000 7f15bf622740 1 --2- 172.21.15.64:0/1709766016 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b31d32fd80 0x55b31d734340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.757+0000 7f15bf622740 4 client.15837 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:05:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.757+0000 7f15bf622740 1 -- 172.21.15.64:0/1709766016 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b31d68d640 con 0x55b31d32fd80 2024-09-21T04:05:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.757+0000 7f15b5b75640 1 --2- 172.21.15.64:0/1709766016 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b31d32fd80 0x55b31d734340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.758+0000 7f15b5b75640 1 --2- 172.21.15.64:0/1709766016 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b31d32fd80 0x55b31d734340 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce9b66793e5eee64 server_cookie=a514fcb3f563de8e in_seq=0 out_seq=0 2024-09-21T04:05:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.758+0000 7f1599ffb640 10 client.15837 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.759+0000 7f1599ffb640 1 -- 172.21.15.64:0/1709766016 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b31d68d640 con 0x55b31d32fd80 2024-09-21T04:05:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.759+0000 7f1599ffb640 10 client.15837 handle_command_reply: tid=0 2024-09-21T04:05:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.759+0000 7f15bf622740 1 client.15837 shutdown 2024-09-21T04:05:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.759+0000 7f15bf622740 1 -- 172.21.15.64:0/1709766016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b31d3504d0 msgr2=0x55b31d722650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.759+0000 7f15bf622740 1 --2- 172.21.15.64:0/1709766016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b31d3504d0 0x55b31d722650 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x55b31d715630 tx=0x7f15ac011810 comp rx=0 tx=0).stop 2024-09-21T04:05:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.759+0000 7f15bf622740 1 -- 172.21.15.64:0/1709766016 shutdown_connections 2024-09-21T04:05:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.759+0000 7f15bf622740 1 --2- 172.21.15.64:0/1709766016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b31d3711b0 0x55b31d72e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.759+0000 7f15bf622740 1 --2- 172.21.15.64:0/1709766016 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b31d32fd80 0x55b31d734340 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.759+0000 7f15bf622740 1 --2- 172.21.15.64:0/1709766016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b31d3504d0 0x55b31d722650 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.759+0000 7f15bf622740 1 --2- 172.21.15.64:0/1709766016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b31d3482f0 0x55b31d722110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.759+0000 7f1599ffb640 0 client.15837 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:32.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.759+0000 7f15bf622740 1 -- 172.21.15.64:0/1709766016 >> 172.21.15.64:0/1709766016 conn(0x55b31d365300 msgr2=0x55b31d7179e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:32.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.759+0000 7f15bf622740 1 -- 172.21.15.64:0/1709766016 shutdown_connections 2024-09-21T04:05:32.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.759+0000 7f15bf622740 1 -- 172.21.15.64:0/1709766016 wait complete. 2024-09-21T04:05:32.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.759+0000 7f15bf622740 20 client.15837 trim_cache size 0 max 16384 2024-09-21T04:05:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.776+0000 7f15bf622740 1 Processor -- start 2024-09-21T04:05:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.776+0000 7f15bf622740 1 -- start start 2024-09-21T04:05:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.776+0000 7f15bf622740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b31d3482f0 0x55b31d720fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:32.780 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.776+0000 7f15bf622740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b31d3504d0 0x55b31d7225f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.776+0000 7f15bf622740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b31d3711b0 0x55b31d723d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.776+0000 7f15bf622740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f15b01b2190 con 0x55b31d3482f0 2024-09-21T04:05:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.776+0000 7f15bf622740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f15b01afa30 con 0x55b31d3711b0 2024-09-21T04:05:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.776+0000 7f15bf622740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f15b01ad880 con 0x55b31d3504d0 2024-09-21T04:05:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.776+0000 7f15b5b75640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b31d3482f0 0x55b31d720fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.776+0000 7f15b5b75640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b31d3482f0 0x55b31d720fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35930/0 (socket says 172.21.15.64:35930) 2024-09-21T04:05:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.776+0000 7f15b5b75640 1 -- 172.21.15.64:0/4080210877 learned_addr learned my addr 172.21.15.64:0/4080210877 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.776+0000 7f15b6376640 1 --2- 172.21.15.64:0/4080210877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b31d3504d0 0x55b31d7225f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.776+0000 7f15b5b75640 1 --2- 172.21.15.64:0/4080210877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b31d3711b0 0x55b31d723d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.777+0000 7f15b5b75640 1 -- 172.21.15.64:0/4080210877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b31d3504d0 msgr2=0x55b31d7225f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.777+0000 7f15b5b75640 1 --2- 172.21.15.64:0/4080210877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b31d3504d0 0x55b31d7225f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.777+0000 7f15b5b75640 1 -- 172.21.15.64:0/4080210877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b31d3711b0 msgr2=0x55b31d723d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.777+0000 7f15b5b75640 1 --2- 172.21.15.64:0/4080210877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b31d3711b0 0x55b31d723d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.777+0000 7f15b5b75640 1 -- 172.21.15.64:0/4080210877 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b31d724c50 con 0x55b31d3482f0 2024-09-21T04:05:32.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.777+0000 7f15b6376640 1 --2- 172.21.15.64:0/4080210877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b31d3504d0 0x55b31d7225f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:05:32.782 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.777+0000 7f15b5b75640 1 --2- 172.21.15.64:0/4080210877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b31d3482f0 0x55b31d720fe0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f15a00058c0 tx=0x7f15a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46fe5250a9a71e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.780+0000 7f159bfff640 1 -- 172.21.15.64:0/4080210877 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a0004220 con 0x55b31d3482f0 2024-09-21T04:05:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.781+0000 7f159bfff640 1 -- 172.21.15.64:0/4080210877 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15a00043c0 con 0x55b31d3482f0 2024-09-21T04:05:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.781+0000 7f159bfff640 1 -- 172.21.15.64:0/4080210877 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a0003850 con 0x55b31d3482f0 2024-09-21T04:05:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.781+0000 7f15bf622740 1 -- 172.21.15.64:0/4080210877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b31d3482f0 msgr2=0x55b31d720fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.781+0000 7f15bf622740 1 --2- 172.21.15.64:0/4080210877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b31d3482f0 0x55b31d720fe0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f15a00058c0 tx=0x7f15a00046c0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.781+0000 7f15bf622740 1 -- 172.21.15.64:0/4080210877 shutdown_connections 2024-09-21T04:05:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.781+0000 7f15bf622740 1 --2- 172.21.15.64:0/4080210877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b31d3711b0 0x55b31d723d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.781+0000 7f15bf622740 1 --2- 172.21.15.64:0/4080210877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b31d3504d0 0x55b31d7225f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.781+0000 7f15bf622740 1 --2- 172.21.15.64:0/4080210877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b31d3482f0 0x55b31d720fe0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.781+0000 7f15bf622740 1 -- 172.21.15.64:0/4080210877 >> 172.21.15.64:0/4080210877 conn(0x55b31d365300 msgr2=0x55b31d723630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.782+0000 7f15bf622740 1 -- 172.21.15.64:0/4080210877 shutdown_connections 2024-09-21T04:05:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.782+0000 7f15bf622740 1 -- 172.21.15.64:0/4080210877 wait complete. 2024-09-21T04:05:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.782+0000 7f15bf622740 1 -- start start 2024-09-21T04:05:32.786 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.782+0000 7f15bf622740 1 Processor -- start 2024-09-21T04:05:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.782+0000 7f15bf622740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b31d3482f0 0x55b31d725780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.782+0000 7f15bf622740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b31d3504d0 0x55b31d725cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.782+0000 7f15bf622740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b31d3711b0 0x55b31d79ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.782+0000 7f15bf622740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f15b01adef0 con 0x55b31d3482f0 2024-09-21T04:05:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.782+0000 7f15bf622740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f15b01af360 con 0x55b31d3504d0 2024-09-21T04:05:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.782+0000 7f15bf622740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f15b01ae5c0 con 0x55b31d3711b0 2024-09-21T04:05:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.782+0000 7f15bf622740 10 client.0 _ll_register_callbacks cb 0x7f15ac00bb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:05:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.782+0000 7f15b5b75640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b31d3482f0 0x55b31d725780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.783+0000 7f15b5b75640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b31d3482f0 0x55b31d725780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35940/0 (socket says 172.21.15.64:35940) 2024-09-21T04:05:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.783+0000 7f15b5b75640 1 -- 172.21.15.64:0/11133114 learned_addr learned my addr 172.21.15.64:0/11133114 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.783+0000 7f15b5b75640 1 --2- 172.21.15.64:0/11133114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b31d3711b0 0x55b31d79ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.783+0000 7f15b6376640 1 --2- 172.21.15.64:0/11133114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b31d3504d0 0x55b31d725cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.783+0000 7f15b5b75640 1 -- 172.21.15.64:0/11133114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b31d3711b0 msgr2=0x55b31d79ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.783+0000 7f15b5b75640 1 --2- 172.21.15.64:0/11133114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b31d3711b0 0x55b31d79ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.783+0000 7f15b5b75640 1 -- 172.21.15.64:0/11133114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b31d3504d0 msgr2=0x55b31d725cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.783+0000 7f15b5b75640 1 --2- 172.21.15.64:0/11133114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b31d3504d0 0x55b31d725cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.783+0000 7f15b5b75640 1 -- 172.21.15.64:0/11133114 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b31d79d210 con 0x55b31d3482f0 2024-09-21T04:05:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.783+0000 7f15b6376640 1 --2- 172.21.15.64:0/11133114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b31d3504d0 0x55b31d725cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:05:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.783+0000 7f15b5b75640 1 --2- 172.21.15.64:0/11133114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b31d3482f0 0x55b31d725780 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f15a0004830 tx=0x7f15a0005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f6f74f5bf4945e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.783+0000 7f1599ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:05:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.783+0000 7f1599ffb640 1 -- 172.21.15.64:0/11133114 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a00112f0 con 0x55b31d3482f0 2024-09-21T04:05:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.783+0000 7f1599ffb640 1 -- 172.21.15.64:0/11133114 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f15a0004e50 con 0x55b31d3482f0 2024-09-21T04:05:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.783+0000 7f15bf622740 1 -- 172.21.15.64:0/11133114 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15a00043c0 con 0x55b31d3482f0 2024-09-21T04:05:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.783+0000 7f1599ffb640 1 -- 172.21.15.64:0/11133114 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f15a0005050 con 0x55b31d3482f0 2024-09-21T04:05:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.783+0000 7f15bf622740 10 client.15843 fetch_fsmap learned FSMap version 28 2024-09-21T04:05:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.783+0000 7f15bf622740 1 -- 172.21.15.64:0/11133114 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b31d79e310 con 0x55b31d3482f0 2024-09-21T04:05:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.784+0000 7f1599ffb640 1 -- 172.21.15.64:0/11133114 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15a00043c0 con 0x55b31d3482f0 2024-09-21T04:05:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.784+0000 7f1599ffb640 1 -- 172.21.15.64:0/11133114 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f15a001d030 con 0x55b31d3482f0 2024-09-21T04:05:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.784+0000 7f15bf622740 10 client.15843 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T04:05:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.784+0000 7f15bf622740 10 client.15843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:05:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.784+0000 7f15bf622740 20 client.15843 populate_metadata read hostname 'smithi064' 2024-09-21T04:05:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.784+0000 7f15bf622740 1 --2- 172.21.15.64:0/11133114 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b31d3051c0 0x55b31d7a0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.784+0000 7f15bf622740 4 client.15843 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:05:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.784+0000 7f15bf622740 1 -- 172.21.15.64:0/11133114 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b31d6890f0 con 0x55b31d3051c0 2024-09-21T04:05:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.784+0000 7f15b6376640 1 --2- 172.21.15.64:0/11133114 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b31d3051c0 0x55b31d7a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.785+0000 7f15b6376640 1 --2- 172.21.15.64:0/11133114 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b31d3051c0 0x55b31d7a0700 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b19e1e7141520df server_cookie=2cda786f42c84e64 in_seq=0 out_seq=0 2024-09-21T04:05:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.785+0000 7f1599ffb640 10 client.15843 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.803+0000 7f1599ffb640 1 -- 172.21.15.64:0/11133114 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b31d6890f0 con 0x55b31d3051c0 2024-09-21T04:05:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.803+0000 7f1599ffb640 10 client.15843 handle_command_reply: tid=0 2024-09-21T04:05:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.803+0000 7f15bf622740 1 client.15843 shutdown 2024-09-21T04:05:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.803+0000 7f15bf622740 1 -- 172.21.15.64:0/11133114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b31d3482f0 msgr2=0x55b31d725780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.803+0000 7f15bf622740 1 --2- 172.21.15.64:0/11133114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b31d3482f0 0x55b31d725780 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f15a0004830 tx=0x7f15a0005720 comp rx=0 tx=0).stop 2024-09-21T04:05:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.803+0000 7f15bf622740 1 -- 172.21.15.64:0/11133114 shutdown_connections 2024-09-21T04:05:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.803+0000 7f15bf622740 1 --2- 172.21.15.64:0/11133114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b31d3711b0 0x55b31d79ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.803+0000 7f15bf622740 1 --2- 172.21.15.64:0/11133114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b31d3504d0 0x55b31d725cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.803+0000 7f15bf622740 1 --2- 172.21.15.64:0/11133114 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b31d3051c0 0x55b31d7a0700 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.803+0000 7f15bf622740 1 --2- 172.21.15.64:0/11133114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b31d3482f0 0x55b31d725780 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.803+0000 7f1599ffb640 0 client.15843 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.803+0000 7f15bf622740 1 -- 172.21.15.64:0/11133114 >> 172.21.15.64:0/11133114 conn(0x55b31d365300 msgr2=0x55b31d718170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.803+0000 7f15bf622740 1 -- 172.21.15.64:0/11133114 shutdown_connections 2024-09-21T04:05:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.803+0000 7f15bf622740 1 -- 172.21.15.64:0/11133114 wait complete. 2024-09-21T04:05:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.803+0000 7f15bf622740 20 client.15843 trim_cache size 0 max 16384 2024-09-21T04:05:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:05:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:05:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:05:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:05:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.803+0000 7f15b85da640 1 -- 172.21.15.64:0/4085615551 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f159c0b6680 msgr2=0x7f159c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.803+0000 7f15b85da640 1 --2- 172.21.15.64:0/4085615551 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f159c0b6680 0x7f159c0b8b90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f15a80cba30 tx=0x7f15a80d1040 comp rx=0 tx=0).stop 2024-09-21T04:05:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.804+0000 7f15b85da640 1 -- 172.21.15.64:0/4085615551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15b0108c30 msgr2=0x7f15b019fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.804+0000 7f15b85da640 1 --2- 172.21.15.64:0/4085615551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15b0108c30 0x7f15b019fc70 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f15a80098d0 tx=0x7f15a80317b0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.804+0000 7f15b85da640 1 -- 172.21.15.64:0/4085615551 shutdown_connections 2024-09-21T04:05:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.804+0000 7f15b85da640 1 --2- 172.21.15.64:0/4085615551 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f159c0b6680 0x7f159c0b8b90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.804+0000 7f15b85da640 1 --2- 172.21.15.64:0/4085615551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15b0108c30 0x7f15b019fc70 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.804+0000 7f15b85da640 1 --2- 172.21.15.64:0/4085615551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15b01082e0 0x7f15b019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.804+0000 7f15b85da640 1 --2- 172.21.15.64:0/4085615551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15b01029b0 0x7f15b019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.804+0000 7f15b85da640 1 -- 172.21.15.64:0/4085615551 >> 172.21.15.64:0/4085615551 conn(0x7f15b007a740 msgr2=0x7f15b01019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.804+0000 7f15b85da640 1 -- 172.21.15.64:0/4085615551 shutdown_connections 2024-09-21T04:05:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:32.804+0000 7f15b85da640 1 -- 172.21.15.64:0/4085615551 wait complete. 2024-09-21T04:05:33.027 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T04:05:33.051 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:32 smithi064 ceph-mon[31180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:05:33.051 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:32 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3467146458' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:05:33.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:32 smithi131 ceph-mon[35193]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:05:33.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:32 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3467146458' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:05:33.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:32 smithi119 ceph-mon[34811]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:05:33.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:32 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3467146458' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:05:33.298 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:05:33.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.811+0000 7f6eccc1b640 1 -- 172.21.15.64:0/3198646098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ec8109df0 msgr2=0x7f6ec8110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:33.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.811+0000 7f6eccc1b640 1 --2- 172.21.15.64:0/3198646098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ec8109df0 0x7f6ec8110620 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0009900 tx=0x7f6ec002a3f0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.811+0000 7f6eccc1b640 1 -- 172.21.15.64:0/3198646098 shutdown_connections 2024-09-21T04:05:33.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.811+0000 7f6eccc1b640 1 --2- 172.21.15.64:0/3198646098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ec8109df0 0x7f6ec8110620 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.811+0000 7f6eccc1b640 1 --2- 172.21.15.64:0/3198646098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ec81094a0 0x7f6ec81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.811+0000 7f6eccc1b640 1 --2- 172.21.15.64:0/3198646098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ec8103b60 0x7f6ec8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.811+0000 7f6eccc1b640 1 -- 172.21.15.64:0/3198646098 >> 172.21.15.64:0/3198646098 conn(0x7f6ec807b850 msgr2=0x7f6ec807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:33.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.811+0000 7f6eccc1b640 1 -- 172.21.15.64:0/3198646098 shutdown_connections 2024-09-21T04:05:33.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.811+0000 7f6eccc1b640 1 -- 172.21.15.64:0/3198646098 wait complete. 2024-09-21T04:05:33.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.812+0000 7f6eccc1b640 1 Processor -- start 2024-09-21T04:05:33.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.812+0000 7f6eccc1b640 1 -- start start 2024-09-21T04:05:33.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.812+0000 7f6eccc1b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ec8103b60 0x7f6ec8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:33.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.812+0000 7f6eccc1b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ec81094a0 0x7f6ec8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:33.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.812+0000 7f6eccc1b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ec8109df0 0x7f6ec8076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:33.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.812+0000 7f6eccc1b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec8113ab0 con 0x7f6ec81094a0 2024-09-21T04:05:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.812+0000 7f6eccc1b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec8113930 con 0x7f6ec8103b60 2024-09-21T04:05:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.812+0000 7f6eccc1b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec8113c30 con 0x7f6ec8109df0 2024-09-21T04:05:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.813+0000 7f6ec5d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ec81094a0 0x7f6ec8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.813+0000 7f6ec5d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ec81094a0 0x7f6ec8075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35956/0 (socket says 172.21.15.64:35956) 2024-09-21T04:05:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.813+0000 7f6ec5d9b640 1 -- 172.21.15.64:0/1357066596 learned_addr learned my addr 172.21.15.64:0/1357066596 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.813+0000 7f6ec6d9d640 1 --2- 172.21.15.64:0/1357066596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ec8109df0 0x7f6ec8076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.813+0000 7f6ec659c640 1 --2- 172.21.15.64:0/1357066596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ec8103b60 0x7f6ec8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.813+0000 7f6ec5d9b640 1 -- 172.21.15.64:0/1357066596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ec8109df0 msgr2=0x7f6ec8076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.813+0000 7f6ec5d9b640 1 --2- 172.21.15.64:0/1357066596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ec8109df0 0x7f6ec8076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.813+0000 7f6ec5d9b640 1 -- 172.21.15.64:0/1357066596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ec8103b60 msgr2=0x7f6ec8075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.813+0000 7f6ec5d9b640 1 --2- 172.21.15.64:0/1357066596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ec8103b60 0x7f6ec8075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.813+0000 7f6ec5d9b640 1 -- 172.21.15.64:0/1357066596 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec81a8670 con 0x7f6ec81094a0 2024-09-21T04:05:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.813+0000 7f6ec6d9d640 1 --2- 172.21.15.64:0/1357066596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ec8109df0 0x7f6ec8076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:05:33.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.813+0000 7f6ec5d9b640 1 --2- 172.21.15.64:0/1357066596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ec81094a0 0x7f6ec8075c50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000e9c0 tx=0x7f6eb000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:33.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.813+0000 7f6ebf7fe640 1 -- 172.21.15.64:0/1357066596 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb000ccc0 con 0x7f6ec81094a0 2024-09-21T04:05:33.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.813+0000 7f6ebf7fe640 1 -- 172.21.15.64:0/1357066596 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eb00054d0 con 0x7f6ec81094a0 2024-09-21T04:05:33.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.813+0000 7f6eccc1b640 1 -- 172.21.15.64:0/1357066596 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ec81a8960 con 0x7f6ec81094a0 2024-09-21T04:05:33.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.813+0000 7f6ebf7fe640 1 -- 172.21.15.64:0/1357066596 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0010430 con 0x7f6ec81094a0 2024-09-21T04:05:33.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.814+0000 7f6eccc1b640 1 -- 172.21.15.64:0/1357066596 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ec81a8ea0 con 0x7f6ec81094a0 2024-09-21T04:05:33.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.816+0000 7f6ebf7fe640 1 -- 172.21.15.64:0/1357066596 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f6eb0005020 con 0x7f6ec81094a0 2024-09-21T04:05:33.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.816+0000 7f6ed318f740 1 Processor -- start 2024-09-21T04:05:33.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.817+0000 7f6ed318f740 1 -- start start 2024-09-21T04:05:33.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.817+0000 7f6ed318f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562672429070 0x5626727d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:33.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.817+0000 7f6ed318f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626724001b0 0x5626727d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:33.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.817+0000 7f6ed318f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562672408390 0x5626727d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:33.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.817+0000 7f6ed318f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562672746ea0 con 0x562672429070 2024-09-21T04:05:33.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.817+0000 7f6ed318f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5626727362d0 con 0x5626724001b0 2024-09-21T04:05:33.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.817+0000 7f6ed318f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56267273a0e0 con 0x562672408390 2024-09-21T04:05:33.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.817+0000 7f6ec6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562672429070 0x5626727d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:33.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.817+0000 7f6ec6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562672429070 0x5626727d06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35968/0 (socket says 172.21.15.64:35968) 2024-09-21T04:05:33.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.817+0000 7f6ec6d9d640 1 -- 172.21.15.64:0/2295710171 learned_addr learned my addr 172.21.15.64:0/2295710171 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:33.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.817+0000 7f6ec5d9b640 1 --2- 172.21.15.64:0/2295710171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562672408390 0x5626727d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:33.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.817+0000 7f6ec659c640 1 --2- 172.21.15.64:0/2295710171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626724001b0 0x5626727d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:33.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.818+0000 7f6ec6d9d640 1 -- 172.21.15.64:0/2295710171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562672408390 msgr2=0x5626727d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:33.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.818+0000 7f6ec6d9d640 1 --2- 172.21.15.64:0/2295710171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562672408390 0x5626727d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.818+0000 7f6ec6d9d640 1 -- 172.21.15.64:0/2295710171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626724001b0 msgr2=0x5626727d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:33.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.818+0000 7f6ec6d9d640 1 --2- 172.21.15.64:0/2295710171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626724001b0 0x5626727d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.818+0000 7f6ec6d9d640 1 -- 172.21.15.64:0/2295710171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562672494650 con 0x562672429070 2024-09-21T04:05:33.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.818+0000 7f6ebf7fe640 1 --2- 172.21.15.64:0/1357066596 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6eb40b6700 0x7f6eb40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:33.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.818+0000 7f6ec6d9d640 1 --2- 172.21.15.64:0/2295710171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562672429070 0x5626727d06a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6ec002a910 tx=0x7f6ec0031520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb079e5f7665dd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:33.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.818+0000 7f6ec659c640 1 --2- 172.21.15.64:0/1357066596 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6eb40b6700 0x7f6eb40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:33.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.818+0000 7f6ebcff9640 1 -- 172.21.15.64:0/2295710171 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0040070 con 0x562672429070 2024-09-21T04:05:33.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.818+0000 7f6ebf7fe640 1 -- 172.21.15.64:0/1357066596 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f6eb00e7a30 con 0x7f6ec81094a0 2024-09-21T04:05:33.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.818+0000 7f6ebcff9640 1 -- 172.21.15.64:0/2295710171 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec0031910 con 0x562672429070 2024-09-21T04:05:33.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.818+0000 7f6ebcff9640 1 -- 172.21.15.64:0/2295710171 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0035040 con 0x562672429070 2024-09-21T04:05:33.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.819+0000 7f6ed318f740 1 -- 172.21.15.64:0/2295710171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562672429070 msgr2=0x5626727d06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:33.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.819+0000 7f6ec659c640 1 --2- 172.21.15.64:0/1357066596 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6eb40b6700 0x7f6eb40b8c10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x5626727cf320 tx=0x7f6eb80075b0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:33.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.819+0000 7f6ed318f740 1 --2- 172.21.15.64:0/2295710171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562672429070 0x5626727d06a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6ec002a910 tx=0x7f6ec0031520 comp rx=0 tx=0).stop 2024-09-21T04:05:33.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.819+0000 7f6ed318f740 1 -- 172.21.15.64:0/2295710171 shutdown_connections 2024-09-21T04:05:33.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.819+0000 7f6ed318f740 1 --2- 172.21.15.64:0/2295710171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562672408390 0x5626727d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.819+0000 7f6ed318f740 1 --2- 172.21.15.64:0/2295710171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5626724001b0 0x5626727d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.819+0000 7f6ed318f740 1 --2- 172.21.15.64:0/2295710171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562672429070 0x5626727d06a0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.819+0000 7f6ed318f740 1 -- 172.21.15.64:0/2295710171 >> 172.21.15.64:0/2295710171 conn(0x56267241d460 msgr2=0x5626727ce590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:33.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.819+0000 7f6ed318f740 1 -- 172.21.15.64:0/2295710171 shutdown_connections 2024-09-21T04:05:33.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.819+0000 7f6ed318f740 1 -- 172.21.15.64:0/2295710171 wait complete. 2024-09-21T04:05:33.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.820+0000 7f6ed318f740 1 -- start start 2024-09-21T04:05:33.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.820+0000 7f6ed318f740 1 Processor -- start 2024-09-21T04:05:33.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.820+0000 7f6ed318f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626724001b0 0x5626727dbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:33.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.820+0000 7f6ed318f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562672408390 0x5626727dc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:33.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.820+0000 7f6ed318f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562672429070 0x5626727e8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:33.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.820+0000 7f6ed318f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562672597700 con 0x5626724001b0 2024-09-21T04:05:33.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.820+0000 7f6ed318f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5626726d1160 con 0x562672429070 2024-09-21T04:05:33.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.820+0000 7f6ed318f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5626727dee70 con 0x562672408390 2024-09-21T04:05:33.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.820+0000 7f6ed318f740 10 client.0 _ll_register_callbacks cb 0x5626726ea9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:05:33.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.820+0000 7f6ec659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626724001b0 0x5626727dbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:33.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.820+0000 7f6ec659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626724001b0 0x5626727dbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35984/0 (socket says 172.21.15.64:35984) 2024-09-21T04:05:33.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.820+0000 7f6ec659c640 1 -- 172.21.15.64:0/2486901727 learned_addr learned my addr 172.21.15.64:0/2486901727 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:33.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.820+0000 7f6ec5d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562672408390 0x5626727dc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:33.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.821+0000 7f6ec6d9d640 1 --2- 172.21.15.64:0/2486901727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562672429070 0x5626727e8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:33.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.821+0000 7f6ec659c640 1 -- 172.21.15.64:0/2486901727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562672408390 msgr2=0x5626727dc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:33.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.821+0000 7f6ec659c640 1 --2- 172.21.15.64:0/2486901727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562672408390 0x5626727dc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.821+0000 7f6ec659c640 1 -- 172.21.15.64:0/2486901727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562672429070 msgr2=0x5626727e8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:33.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.821+0000 7f6ec659c640 1 --2- 172.21.15.64:0/2486901727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562672429070 0x5626727e8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.821+0000 7f6ec659c640 1 -- 172.21.15.64:0/2486901727 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626726fec90 con 0x5626724001b0 2024-09-21T04:05:33.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.821+0000 7f6ec5d9b640 1 --2- 172.21.15.64:0/2486901727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562672408390 0x5626727dc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:05:33.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.821+0000 7f6ec659c640 1 --2- 172.21.15.64:0/2486901727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626724001b0 0x5626727dbd80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6eb801ec30 tx=0x7f6eb8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b6e0e373f42fbe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:33.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.821+0000 7f6eadffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:05:33.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.821+0000 7f6eadffb640 1 -- 172.21.15.64:0/2486901727 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb800fe70 con 0x5626724001b0 2024-09-21T04:05:33.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.821+0000 7f6eadffb640 1 -- 172.21.15.64:0/2486901727 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f6eb8016030 con 0x5626724001b0 2024-09-21T04:05:33.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.821+0000 7f6ed318f740 1 -- 172.21.15.64:0/2486901727 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ec0031910 con 0x5626724001b0 2024-09-21T04:05:33.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.821+0000 7f6eadffb640 1 -- 172.21.15.64:0/2486901727 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6eb801e020 con 0x5626724001b0 2024-09-21T04:05:33.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.822+0000 7f6ed318f740 10 client.15855 fetch_fsmap learned FSMap version 28 2024-09-21T04:05:33.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.822+0000 7f6ed318f740 1 -- 172.21.15.64:0/2486901727 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5626727eabd0 con 0x5626724001b0 2024-09-21T04:05:33.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.822+0000 7f6eadffb640 1 -- 172.21.15.64:0/2486901727 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec0031910 con 0x5626724001b0 2024-09-21T04:05:33.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.822+0000 7f6eadffb640 1 -- 172.21.15.64:0/2486901727 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f6eb801d420 con 0x5626724001b0 2024-09-21T04:05:33.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.822+0000 7f6ed318f740 10 client.15855 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T04:05:33.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.822+0000 7f6ed318f740 10 client.15855 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:05:33.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.822+0000 7f6ed318f740 20 client.15855 populate_metadata read hostname 'smithi064' 2024-09-21T04:05:33.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.822+0000 7f6ed318f740 1 --2- 172.21.15.64:0/2486901727 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x5626723e79f0 0x5626727ee130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:33.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.822+0000 7f6ed318f740 4 client.15855 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:05:33.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.822+0000 7f6ed318f740 1 -- 172.21.15.64:0/2486901727 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562672748df0 con 0x5626723e79f0 2024-09-21T04:05:33.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.822+0000 7f6ec5d9b640 1 --2- 172.21.15.64:0/2486901727 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x5626723e79f0 0x5626727ee130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:33.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.823+0000 7f6ec5d9b640 1 --2- 172.21.15.64:0/2486901727 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x5626723e79f0 0x5626727ee130 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7eff00c7331e1aae server_cookie=d92d04e3d087852a in_seq=0 out_seq=0 2024-09-21T04:05:33.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.823+0000 7f6eadffb640 10 client.15855 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:05:33.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.825+0000 7f6eadffb640 1 -- 172.21.15.64:0/2486901727 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562672748df0 con 0x5626723e79f0 2024-09-21T04:05:33.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.825+0000 7f6eadffb640 10 client.15855 handle_command_reply: tid=0 2024-09-21T04:05:33.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.825+0000 7f6ed318f740 1 client.15855 shutdown 2024-09-21T04:05:33.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.825+0000 7f6ed318f740 1 -- 172.21.15.64:0/2486901727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626724001b0 msgr2=0x5626727dbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:33.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.825+0000 7f6ed318f740 1 --2- 172.21.15.64:0/2486901727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626724001b0 0x5626727dbd80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6eb801ec30 tx=0x7f6eb8004ab0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.825+0000 7f6ed318f740 1 -- 172.21.15.64:0/2486901727 shutdown_connections 2024-09-21T04:05:33.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.825+0000 7f6ed318f740 1 --2- 172.21.15.64:0/2486901727 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x5626723e79f0 0x5626727ee130 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.826+0000 7f6ed318f740 1 --2- 172.21.15.64:0/2486901727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562672429070 0x5626727e8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.826+0000 7f6eadffb640 0 client.15855 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:05:33.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.826+0000 7f6ed318f740 1 --2- 172.21.15.64:0/2486901727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562672408390 0x5626727dc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.826+0000 7f6ed318f740 1 --2- 172.21.15.64:0/2486901727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626724001b0 0x5626727dbd80 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.826+0000 7f6ed318f740 1 -- 172.21.15.64:0/2486901727 >> 172.21.15.64:0/2486901727 conn(0x56267241d460 msgr2=0x5626727d16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:33.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.826+0000 7f6ed318f740 1 -- 172.21.15.64:0/2486901727 shutdown_connections 2024-09-21T04:05:33.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.826+0000 7f6ed318f740 1 -- 172.21.15.64:0/2486901727 wait complete. 2024-09-21T04:05:33.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.826+0000 7f6ed318f740 20 client.15855 trim_cache size 0 max 16384 2024-09-21T04:05:33.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.840+0000 7f6ed318f740 1 Processor -- start 2024-09-21T04:05:33.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.840+0000 7f6ed318f740 1 -- start start 2024-09-21T04:05:33.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.840+0000 7f6ed318f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626723e79f0 0x5626727d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:33.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.840+0000 7f6ed318f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626724001b0 0x5626727dc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:33.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.840+0000 7f6ed318f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562672408390 0x5626727e8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:33.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.840+0000 7f6ed318f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec81b42c0 con 0x5626724001b0 2024-09-21T04:05:33.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.840+0000 7f6ed318f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec81b1b60 con 0x562672408390 2024-09-21T04:05:33.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.840+0000 7f6ed318f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec81af9b0 con 0x5626723e79f0 2024-09-21T04:05:33.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.840+0000 7f6ec5d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626724001b0 0x5626727dc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:33.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.840+0000 7f6ec5d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626724001b0 0x5626727dc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35986/0 (socket says 172.21.15.64:35986) 2024-09-21T04:05:33.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.840+0000 7f6ec5d9b640 1 -- 172.21.15.64:0/3071768348 learned_addr learned my addr 172.21.15.64:0/3071768348 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:33.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.840+0000 7f6ec659c640 1 --2- 172.21.15.64:0/3071768348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626723e79f0 0x5626727d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:33.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.840+0000 7f6ec6d9d640 1 --2- 172.21.15.64:0/3071768348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562672408390 0x5626727e8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:33.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.841+0000 7f6ec5d9b640 1 -- 172.21.15.64:0/3071768348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626723e79f0 msgr2=0x5626727d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:33.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.841+0000 7f6ec5d9b640 1 --2- 172.21.15.64:0/3071768348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626723e79f0 0x5626727d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.841+0000 7f6ec5d9b640 1 -- 172.21.15.64:0/3071768348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562672408390 msgr2=0x5626727e8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:33.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.841+0000 7f6ec5d9b640 1 --2- 172.21.15.64:0/3071768348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562672408390 0x5626727e8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.841+0000 7f6ec5d9b640 1 -- 172.21.15.64:0/3071768348 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626727dccd0 con 0x5626724001b0 2024-09-21T04:05:33.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.841+0000 7f6ec5d9b640 1 --2- 172.21.15.64:0/3071768348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626724001b0 0x5626727dc240 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0009250 tx=0x7f6eb00d1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566f36fe654b8053 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:33.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.841+0000 7f6eaffff640 1 -- 172.21.15.64:0/3071768348 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb00ad620 con 0x5626724001b0 2024-09-21T04:05:33.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.841+0000 7f6eaffff640 1 -- 172.21.15.64:0/3071768348 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eb0005310 con 0x5626724001b0 2024-09-21T04:05:33.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.841+0000 7f6eaffff640 1 -- 172.21.15.64:0/3071768348 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb000ce60 con 0x5626724001b0 2024-09-21T04:05:33.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.842+0000 7f6ed318f740 1 -- 172.21.15.64:0/3071768348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626724001b0 msgr2=0x5626727dc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:33.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.842+0000 7f6ed318f740 1 --2- 172.21.15.64:0/3071768348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626724001b0 0x5626727dc240 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0009250 tx=0x7f6eb00d1040 comp rx=0 tx=0).stop 2024-09-21T04:05:33.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.842+0000 7f6ed318f740 1 -- 172.21.15.64:0/3071768348 shutdown_connections 2024-09-21T04:05:33.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.842+0000 7f6ed318f740 1 --2- 172.21.15.64:0/3071768348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562672408390 0x5626727e8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.842+0000 7f6ed318f740 1 --2- 172.21.15.64:0/3071768348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626724001b0 0x5626727dc240 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.842+0000 7f6ed318f740 1 --2- 172.21.15.64:0/3071768348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626723e79f0 0x5626727d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.842+0000 7f6ed318f740 1 -- 172.21.15.64:0/3071768348 >> 172.21.15.64:0/3071768348 conn(0x56267241d460 msgr2=0x5626727dbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:33.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.842+0000 7f6ed318f740 1 -- 172.21.15.64:0/3071768348 shutdown_connections 2024-09-21T04:05:33.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.842+0000 7f6ed318f740 1 -- 172.21.15.64:0/3071768348 wait complete. 2024-09-21T04:05:33.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.842+0000 7f6ed318f740 1 -- start start 2024-09-21T04:05:33.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.842+0000 7f6ed318f740 1 Processor -- start 2024-09-21T04:05:33.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.843+0000 7f6ed318f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626723e79f0 0x5626727e0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:33.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.843+0000 7f6ed318f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626724001b0 0x5626727e05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:33.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.843+0000 7f6ed318f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562672408390 0x5626727e0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:33.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.843+0000 7f6ed318f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec81b0020 con 0x5626724001b0 2024-09-21T04:05:33.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.843+0000 7f6ed318f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec81b1490 con 0x562672408390 2024-09-21T04:05:33.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.843+0000 7f6ed318f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec81b06f0 con 0x5626723e79f0 2024-09-21T04:05:33.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.843+0000 7f6ed318f740 10 client.0 _ll_register_callbacks cb 0x7f6eb8007d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:05:33.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.843+0000 7f6ec5d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626724001b0 0x5626727e05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:33.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.843+0000 7f6ec659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626723e79f0 0x5626727e0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:33.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.843+0000 7f6ec5d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626724001b0 0x5626727e05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35994/0 (socket says 172.21.15.64:35994) 2024-09-21T04:05:33.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.843+0000 7f6ec5d9b640 1 -- 172.21.15.64:0/2072246823 learned_addr learned my addr 172.21.15.64:0/2072246823 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:33.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.843+0000 7f6ec6d9d640 1 --2- 172.21.15.64:0/2072246823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562672408390 0x5626727e0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:33.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.843+0000 7f6ec5d9b640 1 -- 172.21.15.64:0/2072246823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626723e79f0 msgr2=0x5626727e0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:33.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.843+0000 7f6ec5d9b640 1 --2- 172.21.15.64:0/2072246823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626723e79f0 0x5626727e0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.843+0000 7f6ec5d9b640 1 -- 172.21.15.64:0/2072246823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562672408390 msgr2=0x5626727e0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:33.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.843+0000 7f6ec5d9b640 1 --2- 172.21.15.64:0/2072246823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562672408390 0x5626727e0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.843+0000 7f6ec5d9b640 1 -- 172.21.15.64:0/2072246823 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562672857b30 con 0x5626724001b0 2024-09-21T04:05:33.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.843+0000 7f6ec6d9d640 1 --2- 172.21.15.64:0/2072246823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562672408390 0x5626727e0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:05:33.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.843+0000 7f6ec5d9b640 1 --2- 172.21.15.64:0/2072246823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626724001b0 0x5626727e05d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00ad550 tx=0x7f6eb00a9cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ebb3ab0914c451 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:33.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.843+0000 7f6eadffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:05:33.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.844+0000 7f6eadffb640 1 -- 172.21.15.64:0/2072246823 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb00acfc0 con 0x5626724001b0 2024-09-21T04:05:33.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.844+0000 7f6eadffb640 1 -- 172.21.15.64:0/2072246823 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f6eb00046c0 con 0x5626724001b0 2024-09-21T04:05:33.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.844+0000 7f6ed318f740 1 -- 172.21.15.64:0/2072246823 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5626727c99e0 con 0x5626724001b0 2024-09-21T04:05:33.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.844+0000 7f6eadffb640 1 -- 172.21.15.64:0/2072246823 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6eb00ad1a0 con 0x5626724001b0 2024-09-21T04:05:33.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.844+0000 7f6ed318f740 10 client.15861 fetch_fsmap learned FSMap version 28 2024-09-21T04:05:33.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.844+0000 7f6ed318f740 1 -- 172.21.15.64:0/2072246823 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562672858d80 con 0x5626724001b0 2024-09-21T04:05:33.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.844+0000 7f6eadffb640 1 -- 172.21.15.64:0/2072246823 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5626727c99e0 con 0x5626724001b0 2024-09-21T04:05:33.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.844+0000 7f6eadffb640 1 -- 172.21.15.64:0/2072246823 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7f6eb00aeb80 con 0x5626724001b0 2024-09-21T04:05:33.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.844+0000 7f6ed318f740 10 client.15861 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T04:05:33.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.844+0000 7f6ed318f740 10 client.15861 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:05:33.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.844+0000 7f6ed318f740 20 client.15861 populate_metadata read hostname 'smithi064' 2024-09-21T04:05:33.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.844+0000 7f6ed318f740 1 --2- 172.21.15.64:0/2072246823 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562672429070 0x56267285b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:33.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.844+0000 7f6ed318f740 4 client.15861 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:05:33.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.844+0000 7f6ed318f740 1 -- 172.21.15.64:0/2072246823 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5626727448a0 con 0x562672429070 2024-09-21T04:05:33.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.845+0000 7f6ec659c640 1 --2- 172.21.15.64:0/2072246823 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562672429070 0x56267285b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:33.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.845+0000 7f6ec659c640 1 --2- 172.21.15.64:0/2072246823 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562672429070 0x56267285b4b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf9e5ef98e648613 server_cookie=dd1e1ccbd3742a45 in_seq=0 out_seq=0 2024-09-21T04:05:33.852 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.845+0000 7f6eadffb640 10 client.15861 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:05:33.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.849+0000 7f6eadffb640 1 -- 172.21.15.64:0/2072246823 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5626727448a0 con 0x562672429070 2024-09-21T04:05:33.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.849+0000 7f6eadffb640 10 client.15861 handle_command_reply: tid=0 2024-09-21T04:05:33.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.849+0000 7f6ed318f740 1 client.15861 shutdown 2024-09-21T04:05:33.853 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.850+0000 7f6ed318f740 1 -- 172.21.15.64:0/2072246823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626724001b0 msgr2=0x5626727e05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:33.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.850+0000 7f6ed318f740 1 --2- 172.21.15.64:0/2072246823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626724001b0 0x5626727e05d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00ad550 tx=0x7f6eb00a9cd0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.850+0000 7f6ed318f740 1 -- 172.21.15.64:0/2072246823 shutdown_connections 2024-09-21T04:05:33.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.850+0000 7f6ed318f740 1 --2- 172.21.15.64:0/2072246823 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562672429070 0x56267285b4b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.850+0000 7f6ed318f740 1 --2- 172.21.15.64:0/2072246823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562672408390 0x5626727e0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.850+0000 7f6ed318f740 1 --2- 172.21.15.64:0/2072246823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5626724001b0 0x5626727e05d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.850+0000 7f6eadffb640 0 client.15861 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:05:33.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.850+0000 7f6ed318f740 1 --2- 172.21.15.64:0/2072246823 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626723e79f0 0x5626727e0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.850+0000 7f6ed318f740 1 -- 172.21.15.64:0/2072246823 >> 172.21.15.64:0/2072246823 conn(0x56267241d460 msgr2=0x5626727cdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:33.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.850+0000 7f6ed318f740 1 -- 172.21.15.64:0/2072246823 shutdown_connections 2024-09-21T04:05:33.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.850+0000 7f6ed318f740 1 -- 172.21.15.64:0/2072246823 wait complete. 2024-09-21T04:05:33.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.850+0000 7f6ed318f740 20 client.15861 trim_cache size 0 max 16384 2024-09-21T04:05:33.855 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:05:33.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.850+0000 7f6eccc1b640 1 -- 172.21.15.64:0/1357066596 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6eb40b6700 msgr2=0x7f6eb40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:33.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.850+0000 7f6eccc1b640 1 --2- 172.21.15.64:0/1357066596 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6eb40b6700 0x7f6eb40b8c10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x5626727cf320 tx=0x7f6eb80075b0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.850+0000 7f6eccc1b640 1 -- 172.21.15.64:0/1357066596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ec81094a0 msgr2=0x7f6ec8075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:33.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.850+0000 7f6eccc1b640 1 --2- 172.21.15.64:0/1357066596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ec81094a0 0x7f6ec8075c50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000e9c0 tx=0x7f6eb000ee80 comp rx=0 tx=0).stop 2024-09-21T04:05:33.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.851+0000 7f6eccc1b640 1 -- 172.21.15.64:0/1357066596 shutdown_connections 2024-09-21T04:05:33.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.851+0000 7f6eccc1b640 1 --2- 172.21.15.64:0/1357066596 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6eb40b6700 0x7f6eb40b8c10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.851+0000 7f6eccc1b640 1 --2- 172.21.15.64:0/1357066596 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ec8109df0 0x7f6ec8076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.851+0000 7f6eccc1b640 1 --2- 172.21.15.64:0/1357066596 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6ec81094a0 0x7f6ec8075c50 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.851+0000 7f6eccc1b640 1 --2- 172.21.15.64:0/1357066596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ec8103b60 0x7f6ec8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:33.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.851+0000 7f6eccc1b640 1 -- 172.21.15.64:0/1357066596 >> 172.21.15.64:0/1357066596 conn(0x7f6ec807b850 msgr2=0x7f6ec81074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:33.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.851+0000 7f6eccc1b640 1 -- 172.21.15.64:0/1357066596 shutdown_connections 2024-09-21T04:05:33.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:33.851+0000 7f6eccc1b640 1 -- 172.21.15.64:0/1357066596 wait complete. 2024-09-21T04:05:34.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:33 smithi064 ceph-mon[31180]: pgmap v367: 97 pgs: 97 active+clean; 5.6 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 104 KiB/s wr, 41 op/s 2024-09-21T04:05:34.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:33 smithi131 ceph-mon[35193]: pgmap v367: 97 pgs: 97 active+clean; 5.6 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 104 KiB/s wr, 41 op/s 2024-09-21T04:05:34.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:33 smithi119 ceph-mon[34811]: pgmap v367: 97 pgs: 97 active+clean; 5.6 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 104 KiB/s wr, 41 op/s 2024-09-21T04:05:34.579 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T04:05:34.890 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:05:35.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.362+0000 7f342897d640 1 -- 172.21.15.64:0/720475416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3424108cf0 msgr2=0x7f342410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:35.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.362+0000 7f342897d640 1 --2- 172.21.15.64:0/720475416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3424108cf0 0x7f342410f520 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3414009ca0 tx=0x7f341402a8c0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.362+0000 7f342897d640 1 -- 172.21.15.64:0/720475416 shutdown_connections 2024-09-21T04:05:35.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.362+0000 7f342897d640 1 --2- 172.21.15.64:0/720475416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3424108cf0 0x7f342410f520 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.362+0000 7f342897d640 1 --2- 172.21.15.64:0/720475416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34241083a0 0x7f34241087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.362+0000 7f342897d640 1 --2- 172.21.15.64:0/720475416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3424102a60 0x7f3424102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.362+0000 7f342897d640 1 -- 172.21.15.64:0/720475416 >> 172.21.15.64:0/720475416 conn(0x7f3424079730 msgr2=0x7f3424079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:35.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.362+0000 7f342897d640 1 -- 172.21.15.64:0/720475416 shutdown_connections 2024-09-21T04:05:35.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.362+0000 7f342897d640 1 -- 172.21.15.64:0/720475416 wait complete. 2024-09-21T04:05:35.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.363+0000 7f342897d640 1 Processor -- start 2024-09-21T04:05:35.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.363+0000 7f342897d640 1 -- start start 2024-09-21T04:05:35.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.364+0000 7f342897d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3424102a60 0x7f342419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:35.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.364+0000 7f342897d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34241083a0 0x7f342419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:35.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.364+0000 7f342897d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3424108cf0 0x7f342419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:35.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.364+0000 7f342897d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3424112960 con 0x7f3424102a60 2024-09-21T04:05:35.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.364+0000 7f342897d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f34241127e0 con 0x7f34241083a0 2024-09-21T04:05:35.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.364+0000 7f342897d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3424112ae0 con 0x7f3424108cf0 2024-09-21T04:05:35.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.364+0000 7f342259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3424102a60 0x7f342419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:35.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.364+0000 7f342259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3424102a60 0x7f342419a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36020/0 (socket says 172.21.15.64:36020) 2024-09-21T04:05:35.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.364+0000 7f3422d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3424108cf0 0x7f342419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:35.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.364+0000 7f342259c640 1 -- 172.21.15.64:0/610373651 learned_addr learned my addr 172.21.15.64:0/610373651 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:35.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.364+0000 7f3421d9b640 1 --2- 172.21.15.64:0/610373651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34241083a0 0x7f342419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:35.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.364+0000 7f342259c640 1 -- 172.21.15.64:0/610373651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3424108cf0 msgr2=0x7f342419fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:35.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.364+0000 7f342259c640 1 --2- 172.21.15.64:0/610373651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3424108cf0 0x7f342419fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.364+0000 7f342259c640 1 -- 172.21.15.64:0/610373651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34241083a0 msgr2=0x7f342419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:35.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.364+0000 7f342259c640 1 --2- 172.21.15.64:0/610373651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34241083a0 0x7f342419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.364+0000 7f342259c640 1 -- 172.21.15.64:0/610373651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34241a0410 con 0x7f3424102a60 2024-09-21T04:05:35.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.364+0000 7f3421d9b640 1 --2- 172.21.15.64:0/610373651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34241083a0 0x7f342419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:05:35.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.364+0000 7f342259c640 1 --2- 172.21.15.64:0/610373651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3424102a60 0x7f342419a4a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f341c00d950 tx=0x7f341c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:35.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.365+0000 7f341b7fe640 1 -- 172.21.15.64:0/610373651 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c017070 con 0x7f3424102a60 2024-09-21T04:05:35.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.365+0000 7f342897d640 1 -- 172.21.15.64:0/610373651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34241a0700 con 0x7f3424102a60 2024-09-21T04:05:35.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.365+0000 7f341b7fe640 1 -- 172.21.15.64:0/610373651 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f341c009c30 con 0x7f3424102a60 2024-09-21T04:05:35.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.365+0000 7f341b7fe640 1 -- 172.21.15.64:0/610373651 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c00f620 con 0x7f3424102a60 2024-09-21T04:05:35.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.365+0000 7f342897d640 1 -- 172.21.15.64:0/610373651 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34241a6e60 con 0x7f3424102a60 2024-09-21T04:05:35.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.367+0000 7f342eef1740 1 Processor -- start 2024-09-21T04:05:35.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.368+0000 7f342eef1740 1 -- start start 2024-09-21T04:05:35.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.368+0000 7f341b7fe640 1 -- 172.21.15.64:0/610373651 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f341c010460 con 0x7f3424102a60 2024-09-21T04:05:35.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.368+0000 7f342eef1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c37204070 0x561c375ab2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:35.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.368+0000 7f342eef1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561c371db1b0 0x561c375ab800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:35.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.368+0000 7f342eef1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c371e3390 0x561c375abd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:35.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.368+0000 7f342eef1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561c37521ea0 con 0x561c371e3390 2024-09-21T04:05:35.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.368+0000 7f342eef1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561c375112d0 con 0x561c37204070 2024-09-21T04:05:35.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.368+0000 7f342eef1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x561c375150e0 con 0x561c371db1b0 2024-09-21T04:05:35.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.368+0000 7f342259c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561c371db1b0 0x561c375ab800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:35.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.368+0000 7f3421d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c371e3390 0x561c375abd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:35.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.368+0000 7f3421d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c371e3390 0x561c375abd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36036/0 (socket says 172.21.15.64:36036) 2024-09-21T04:05:35.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.368+0000 7f342259c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561c371db1b0 0x561c375ab800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:58812/0 (socket says 172.21.15.64:58812) 2024-09-21T04:05:35.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.368+0000 7f3421d9b640 1 -- 172.21.15.64:0/2280799172 learned_addr learned my addr 172.21.15.64:0/2280799172 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:35.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.368+0000 7f3422d9d640 1 --2- 172.21.15.64:0/2280799172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c37204070 0x561c375ab2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:35.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.369+0000 7f3422d9d640 1 -- 172.21.15.64:0/2280799172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561c371db1b0 msgr2=0x561c375ab800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:35.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.369+0000 7f3422d9d640 1 --2- 172.21.15.64:0/2280799172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561c371db1b0 0x561c375ab800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.369+0000 7f3422d9d640 1 -- 172.21.15.64:0/2280799172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c371e3390 msgr2=0x561c375abd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:35.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.369+0000 7f3422d9d640 1 --2- 172.21.15.64:0/2280799172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c371e3390 0x561c375abd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.369+0000 7f3422d9d640 1 -- 172.21.15.64:0/2280799172 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c3726f650 con 0x561c37204070 2024-09-21T04:05:35.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.369+0000 7f342259c640 1 --2- 172.21.15.64:0/2280799172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561c371db1b0 0x561c375ab800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:05:35.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.369+0000 7f3421d9b640 1 --2- 172.21.15.64:0/2280799172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c371e3390 0x561c375abd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:05:35.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.369+0000 7f341b7fe640 1 --2- 172.21.15.64:0/610373651 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f34100b6680 0x7f34100b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:35.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.369+0000 7f3422d9d640 1 --2- 172.21.15.64:0/2280799172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c37204070 0x561c375ab2c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3414005e30 tx=0x7f341402f4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35fb5d083b44f8a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:35.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.369+0000 7f342259c640 1 --2- 172.21.15.64:0/610373651 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f34100b6680 0x7f34100b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:35.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.369+0000 7f3418ff9640 1 -- 172.21.15.64:0/2280799172 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3414031cf0 con 0x561c37204070 2024-09-21T04:05:35.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.369+0000 7f341b7fe640 1 -- 172.21.15.64:0/610373651 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f341c0e73c0 con 0x7f3424102a60 2024-09-21T04:05:35.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.370+0000 7f3418ff9640 1 -- 172.21.15.64:0/2280799172 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3414036030 con 0x561c37204070 2024-09-21T04:05:35.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.370+0000 7f3418ff9640 1 -- 172.21.15.64:0/2280799172 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3414003e60 con 0x561c37204070 2024-09-21T04:05:35.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.370+0000 7f342eef1740 1 -- 172.21.15.64:0/2280799172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c37204070 msgr2=0x561c375ab2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:35.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.370+0000 7f342eef1740 1 --2- 172.21.15.64:0/2280799172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c37204070 0x561c375ab2c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3414005e30 tx=0x7f341402f4c0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.370+0000 7f342259c640 1 --2- 172.21.15.64:0/610373651 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f34100b6680 0x7f34100b8b90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f341c0e4b90 tx=0x7f341c094040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:35.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.371+0000 7f342eef1740 1 -- 172.21.15.64:0/2280799172 shutdown_connections 2024-09-21T04:05:35.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.371+0000 7f342eef1740 1 --2- 172.21.15.64:0/2280799172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c371e3390 0x561c375abd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.371+0000 7f342eef1740 1 --2- 172.21.15.64:0/2280799172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561c371db1b0 0x561c375ab800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.371+0000 7f342eef1740 1 --2- 172.21.15.64:0/2280799172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c37204070 0x561c375ab2c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.371+0000 7f342eef1740 1 -- 172.21.15.64:0/2280799172 >> 172.21.15.64:0/2280799172 conn(0x561c371f8460 msgr2=0x561c375a91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:35.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.371+0000 7f342eef1740 1 -- 172.21.15.64:0/2280799172 shutdown_connections 2024-09-21T04:05:35.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.371+0000 7f342eef1740 1 -- 172.21.15.64:0/2280799172 wait complete. 2024-09-21T04:05:35.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.371+0000 7f342eef1740 1 -- start start 2024-09-21T04:05:35.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.371+0000 7f342eef1740 1 Processor -- start 2024-09-21T04:05:35.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.372+0000 7f342eef1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561c371db1b0 0x561c375bad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:35.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.372+0000 7f342eef1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c371e3390 0x561c375bb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:35.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.372+0000 7f342eef1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c37204070 0x561c375bb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:35.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.372+0000 7f342eef1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561c37372700 con 0x561c37204070 2024-09-21T04:05:35.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.372+0000 7f342eef1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561c374ac160 con 0x561c371e3390 2024-09-21T04:05:35.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.372+0000 7f342eef1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x561c375bdbc0 con 0x561c371db1b0 2024-09-21T04:05:35.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.372+0000 7f342eef1740 10 client.0 _ll_register_callbacks cb 0x561c374c59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:05:35.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.372+0000 7f3422d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c37204070 0x561c375bb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:35.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.372+0000 7f3422d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561c371db1b0 0x561c375bad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:35.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.372+0000 7f3422d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561c371db1b0 0x561c375bad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:58820/0 (socket says 172.21.15.64:58820) 2024-09-21T04:05:35.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.372+0000 7f3422d9d640 1 -- 172.21.15.64:0/1428445532 learned_addr learned my addr 172.21.15.64:0/1428445532 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:35.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.372+0000 7f3421d9b640 1 --2- 172.21.15.64:0/1428445532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c371e3390 0x561c375bb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:35.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.372+0000 7f3422d9d640 1 -- 172.21.15.64:0/1428445532 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561c371db1b0 msgr2=0x561c375bad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:35.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.372+0000 7f3422d9d640 1 --2- 172.21.15.64:0/1428445532 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561c371db1b0 0x561c375bad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.372+0000 7f3422d9d640 1 -- 172.21.15.64:0/1428445532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c371e3390 msgr2=0x561c375bb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:35.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.372+0000 7f3422d9d640 1 --2- 172.21.15.64:0/1428445532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c371e3390 0x561c375bb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.372+0000 7f3422d9d640 1 -- 172.21.15.64:0/1428445532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c374d9c90 con 0x561c37204070 2024-09-21T04:05:35.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.373+0000 7f3422d9d640 1 --2- 172.21.15.64:0/1428445532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c37204070 0x561c375bb800 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3414006010 tx=0x7f3414003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b181a2cb25c9b2f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:35.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.373+0000 7f3409ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:05:35.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.373+0000 7f3409ffb640 1 -- 172.21.15.64:0/1428445532 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3414059060 con 0x561c37204070 2024-09-21T04:05:35.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.373+0000 7f3409ffb640 1 -- 172.21.15.64:0/1428445532 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f3414031710 con 0x561c37204070 2024-09-21T04:05:35.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.373+0000 7f342eef1740 1 -- 172.21.15.64:0/1428445532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3414036030 con 0x561c37204070 2024-09-21T04:05:35.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.373+0000 7f3409ffb640 1 -- 172.21.15.64:0/1428445532 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f341404a040 con 0x561c37204070 2024-09-21T04:05:35.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.373+0000 7f342eef1740 10 client.15873 fetch_fsmap learned FSMap version 29 2024-09-21T04:05:35.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.373+0000 7f342eef1740 1 -- 172.21.15.64:0/1428445532 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561c375c98a0 con 0x561c37204070 2024-09-21T04:05:35.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.373+0000 7f3409ffb640 1 -- 172.21.15.64:0/1428445532 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3414036030 con 0x561c37204070 2024-09-21T04:05:35.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.373+0000 7f3409ffb640 1 -- 172.21.15.64:0/1428445532 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3414043070 con 0x561c37204070 2024-09-21T04:05:35.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.374+0000 7f342eef1740 10 client.15873 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T04:05:35.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.374+0000 7f342eef1740 10 client.15873 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:05:35.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.374+0000 7f342eef1740 20 client.15873 populate_metadata read hostname 'smithi064' 2024-09-21T04:05:35.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.374+0000 7f342eef1740 1 --2- 172.21.15.64:0/1428445532 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x561c371c29f0 0x561c375ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:35.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.374+0000 7f342eef1740 4 client.15873 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:05:35.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.374+0000 7f342eef1740 1 -- 172.21.15.64:0/1428445532 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c37523df0 con 0x561c371c29f0 2024-09-21T04:05:35.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.374+0000 7f3421d9b640 1 --2- 172.21.15.64:0/1428445532 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x561c371c29f0 0x561c375ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:35.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.374+0000 7f3421d9b640 1 --2- 172.21.15.64:0/1428445532 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x561c371c29f0 0x561c375ccc00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a823e0f2f67e9c4 server_cookie=94b38f90de6116fd in_seq=0 out_seq=0 2024-09-21T04:05:35.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.374+0000 7f3409ffb640 10 client.15873 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:05:35.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.375+0000 7f3409ffb640 1 -- 172.21.15.64:0/1428445532 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561c37523df0 con 0x561c371c29f0 2024-09-21T04:05:35.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.375+0000 7f3409ffb640 10 client.15873 handle_command_reply: tid=0 2024-09-21T04:05:35.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.375+0000 7f342eef1740 1 client.15873 shutdown 2024-09-21T04:05:35.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.375+0000 7f342eef1740 1 -- 172.21.15.64:0/1428445532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c37204070 msgr2=0x561c375bb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:35.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.375+0000 7f342eef1740 1 --2- 172.21.15.64:0/1428445532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c37204070 0x561c375bb800 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3414006010 tx=0x7f3414003c60 comp rx=0 tx=0).stop 2024-09-21T04:05:35.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.375+0000 7f342eef1740 1 -- 172.21.15.64:0/1428445532 shutdown_connections 2024-09-21T04:05:35.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.375+0000 7f342eef1740 1 --2- 172.21.15.64:0/1428445532 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x561c371c29f0 0x561c375ccc00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.375+0000 7f342eef1740 1 --2- 172.21.15.64:0/1428445532 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c37204070 0x561c375bb800 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.375+0000 7f342eef1740 1 --2- 172.21.15.64:0/1428445532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c371e3390 0x561c375bb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.375+0000 7f342eef1740 1 --2- 172.21.15.64:0/1428445532 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561c371db1b0 0x561c375bad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.375+0000 7f342eef1740 1 -- 172.21.15.64:0/1428445532 >> 172.21.15.64:0/1428445532 conn(0x561c371f8460 msgr2=0x561c375accb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:35.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.375+0000 7f3409ffb640 0 client.15873 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:05:35.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.375+0000 7f342eef1740 1 -- 172.21.15.64:0/1428445532 shutdown_connections 2024-09-21T04:05:35.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.375+0000 7f342eef1740 1 -- 172.21.15.64:0/1428445532 wait complete. 2024-09-21T04:05:35.387 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.376+0000 7f342eef1740 20 client.15873 trim_cache size 0 max 16384 2024-09-21T04:05:35.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.389+0000 7f342eef1740 1 Processor -- start 2024-09-21T04:05:35.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.389+0000 7f342eef1740 1 -- start start 2024-09-21T04:05:35.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.389+0000 7f342eef1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561c371db1b0 0x561c375bcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:35.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.389+0000 7f342eef1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c371e3390 0x561c375aca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:35.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.389+0000 7f342eef1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c37204070 0x561c375b7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:35.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.389+0000 7f342eef1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f34241aff80 con 0x561c37204070 2024-09-21T04:05:35.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.389+0000 7f342eef1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f34241ad7f0 con 0x561c371e3390 2024-09-21T04:05:35.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.389+0000 7f342eef1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f34241ab640 con 0x561c371db1b0 2024-09-21T04:05:35.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.389+0000 7f3422d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c37204070 0x561c375b7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:35.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.389+0000 7f3422d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561c371db1b0 0x561c375bcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:35.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.389+0000 7f3422d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561c371db1b0 0x561c375bcc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:58822/0 (socket says 172.21.15.64:58822) 2024-09-21T04:05:35.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.389+0000 7f3422d9d640 1 -- 172.21.15.64:0/3627766306 learned_addr learned my addr 172.21.15.64:0/3627766306 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:35.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.389+0000 7f3421d9b640 1 --2- 172.21.15.64:0/3627766306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c371e3390 0x561c375aca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:35.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.389+0000 7f3422d9d640 1 -- 172.21.15.64:0/3627766306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c371e3390 msgr2=0x561c375aca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:35.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.389+0000 7f3422d9d640 1 --2- 172.21.15.64:0/3627766306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c371e3390 0x561c375aca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.389+0000 7f3422d9d640 1 -- 172.21.15.64:0/3627766306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c37204070 msgr2=0x561c375b7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:35.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.389+0000 7f3422d9d640 1 --2- 172.21.15.64:0/3627766306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c37204070 0x561c375b7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.389+0000 7f3422d9d640 1 -- 172.21.15.64:0/3627766306 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c375adb00 con 0x561c371db1b0 2024-09-21T04:05:35.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.390+0000 7f3422d9d640 1 --2- 172.21.15.64:0/3627766306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561c371db1b0 0x561c375bcc20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f341402fa50 tx=0x7f3414000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ee2f433f0928a8c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:35.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.390+0000 7f340bfff640 1 -- 172.21.15.64:0/3627766306 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3414059070 con 0x561c371db1b0 2024-09-21T04:05:35.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.390+0000 7f340bfff640 1 -- 172.21.15.64:0/3627766306 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3414036070 con 0x561c371db1b0 2024-09-21T04:05:35.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.390+0000 7f340bfff640 1 -- 172.21.15.64:0/3627766306 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34140044b0 con 0x561c371db1b0 2024-09-21T04:05:35.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.391+0000 7f342eef1740 1 -- 172.21.15.64:0/3627766306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561c371db1b0 msgr2=0x561c375bcc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:35.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.391+0000 7f342eef1740 1 --2- 172.21.15.64:0/3627766306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561c371db1b0 0x561c375bcc20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f341402fa50 tx=0x7f3414000f40 comp rx=0 tx=0).stop 2024-09-21T04:05:35.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.391+0000 7f342eef1740 1 -- 172.21.15.64:0/3627766306 shutdown_connections 2024-09-21T04:05:35.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.391+0000 7f342eef1740 1 --2- 172.21.15.64:0/3627766306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c37204070 0x561c375b7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.391+0000 7f342eef1740 1 --2- 172.21.15.64:0/3627766306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c371e3390 0x561c375aca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.391+0000 7f342eef1740 1 --2- 172.21.15.64:0/3627766306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561c371db1b0 0x561c375bcc20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.391+0000 7f342eef1740 1 -- 172.21.15.64:0/3627766306 >> 172.21.15.64:0/3627766306 conn(0x561c371f8460 msgr2=0x561c375ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:35.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.391+0000 7f342eef1740 1 -- 172.21.15.64:0/3627766306 shutdown_connections 2024-09-21T04:05:35.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.391+0000 7f342eef1740 1 -- 172.21.15.64:0/3627766306 wait complete. 2024-09-21T04:05:35.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.391+0000 7f342eef1740 1 -- start start 2024-09-21T04:05:35.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.391+0000 7f342eef1740 1 Processor -- start 2024-09-21T04:05:35.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.392+0000 7f342eef1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561c371db1b0 0x561c375c0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:35.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.392+0000 7f342eef1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c371e3390 0x561c375c1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:35.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.392+0000 7f342eef1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c37204070 0x561c375c1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:35.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.392+0000 7f342eef1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f34241abcb0 con 0x561c37204070 2024-09-21T04:05:35.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.392+0000 7f342eef1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f34241ad120 con 0x561c371e3390 2024-09-21T04:05:35.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.392+0000 7f342eef1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f34241ac380 con 0x561c371db1b0 2024-09-21T04:05:35.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.392+0000 7f342eef1740 10 client.0 _ll_register_callbacks cb 0x7f341c0eade0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:05:35.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.392+0000 7f3422d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c37204070 0x561c375c1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:35.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.392+0000 7f3422d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c37204070 0x561c375c1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36058/0 (socket says 172.21.15.64:36058) 2024-09-21T04:05:35.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.392+0000 7f3422d9d640 1 -- 172.21.15.64:0/4217938190 learned_addr learned my addr 172.21.15.64:0/4217938190 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:35.398 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.392+0000 7f3421d9b640 1 --2- 172.21.15.64:0/4217938190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c371e3390 0x561c375c1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:35.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.392+0000 7f3422d9d640 1 --2- 172.21.15.64:0/4217938190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561c371db1b0 0x561c375c0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:35.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.392+0000 7f3422d9d640 1 -- 172.21.15.64:0/4217938190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561c371db1b0 msgr2=0x561c375c0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:35.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.392+0000 7f3422d9d640 1 --2- 172.21.15.64:0/4217938190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561c371db1b0 0x561c375c0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.392+0000 7f3422d9d640 1 -- 172.21.15.64:0/4217938190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c371e3390 msgr2=0x561c375c1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:35.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.392+0000 7f3422d9d640 1 --2- 172.21.15.64:0/4217938190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c371e3390 0x561c375c1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.392+0000 7f3422d9d640 1 -- 172.21.15.64:0/4217938190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c375ad7e0 con 0x561c37204070 2024-09-21T04:05:35.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.392+0000 7f3422d9d640 1 --2- 172.21.15.64:0/4217938190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c37204070 0x561c375c1970 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3414003ca0 tx=0x7f341402fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7f21798ae4aec1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:35.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.392+0000 7f3409ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:05:35.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.393+0000 7f3409ffb640 1 -- 172.21.15.64:0/4217938190 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3414059040 con 0x561c37204070 2024-09-21T04:05:35.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.393+0000 7f3409ffb640 1 -- 172.21.15.64:0/4217938190 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f3414003d00 con 0x561c37204070 2024-09-21T04:05:35.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.393+0000 7f342eef1740 1 -- 172.21.15.64:0/4217938190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3414036070 con 0x561c37204070 2024-09-21T04:05:35.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.393+0000 7f3409ffb640 1 -- 172.21.15.64:0/4217938190 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3414009360 con 0x561c37204070 2024-09-21T04:05:35.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.393+0000 7f342eef1740 10 client.15879 fetch_fsmap learned FSMap version 29 2024-09-21T04:05:35.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.393+0000 7f342eef1740 1 -- 172.21.15.64:0/4217938190 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561c37637510 con 0x561c37204070 2024-09-21T04:05:35.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.393+0000 7f3409ffb640 1 -- 172.21.15.64:0/4217938190 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3414036070 con 0x561c37204070 2024-09-21T04:05:35.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.393+0000 7f3409ffb640 1 -- 172.21.15.64:0/4217938190 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f341406f020 con 0x561c37204070 2024-09-21T04:05:35.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.393+0000 7f342eef1740 10 client.15879 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T04:05:35.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.393+0000 7f342eef1740 10 client.15879 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:05:35.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.393+0000 7f342eef1740 20 client.15879 populate_metadata read hostname 'smithi064' 2024-09-21T04:05:35.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.393+0000 7f342eef1740 1 --2- 172.21.15.64:0/4217938190 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x561c37197da0 0x561c37639a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:35.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.393+0000 7f342eef1740 4 client.15879 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:05:35.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.393+0000 7f342eef1740 1 -- 172.21.15.64:0/4217938190 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c3751f8a0 con 0x561c37197da0 2024-09-21T04:05:35.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.394+0000 7f3421d9b640 1 --2- 172.21.15.64:0/4217938190 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x561c37197da0 0x561c37639a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:35.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.394+0000 7f3421d9b640 1 --2- 172.21.15.64:0/4217938190 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x561c37197da0 0x561c37639a50 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e30ac3d8a358211b server_cookie=743008122a054aa0 in_seq=0 out_seq=0 2024-09-21T04:05:35.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.394+0000 7f3409ffb640 10 client.15879 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:05:35.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.409+0000 7f3409ffb640 1 -- 172.21.15.64:0/4217938190 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c3751f8a0 con 0x561c37197da0 2024-09-21T04:05:35.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.409+0000 7f3409ffb640 10 client.15879 handle_command_reply: tid=0 2024-09-21T04:05:35.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.409+0000 7f342eef1740 1 client.15879 shutdown 2024-09-21T04:05:35.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.409+0000 7f342eef1740 1 -- 172.21.15.64:0/4217938190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c37204070 msgr2=0x561c375c1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:35.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.409+0000 7f342eef1740 1 --2- 172.21.15.64:0/4217938190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c37204070 0x561c375c1970 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3414003ca0 tx=0x7f341402fc80 comp rx=0 tx=0).stop 2024-09-21T04:05:35.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.409+0000 7f342eef1740 1 -- 172.21.15.64:0/4217938190 shutdown_connections 2024-09-21T04:05:35.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.409+0000 7f342eef1740 1 --2- 172.21.15.64:0/4217938190 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x561c37197da0 0x561c37639a50 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.409+0000 7f342eef1740 1 --2- 172.21.15.64:0/4217938190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561c37204070 0x561c375c1970 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.409+0000 7f3409ffb640 0 client.15879 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:05:35.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.409+0000 7f342eef1740 1 --2- 172.21.15.64:0/4217938190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c371e3390 0x561c375c1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.409+0000 7f342eef1740 1 --2- 172.21.15.64:0/4217938190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561c371db1b0 0x561c375c0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.409+0000 7f342eef1740 1 -- 172.21.15.64:0/4217938190 >> 172.21.15.64:0/4217938190 conn(0x561c371f8460 msgr2=0x561c375c6570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:35.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.409+0000 7f342eef1740 1 -- 172.21.15.64:0/4217938190 shutdown_connections 2024-09-21T04:05:35.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.409+0000 7f342eef1740 1 -- 172.21.15.64:0/4217938190 wait complete. 2024-09-21T04:05:35.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.409+0000 7f342eef1740 20 client.15879 trim_cache size 0 max 16384 2024-09-21T04:05:35.414 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:05:35.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.410+0000 7f342897d640 1 -- 172.21.15.64:0/610373651 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f34100b6680 msgr2=0x7f34100b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:35.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.410+0000 7f342897d640 1 --2- 172.21.15.64:0/610373651 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f34100b6680 0x7f34100b8b90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f341c0e4b90 tx=0x7f341c094040 comp rx=0 tx=0).stop 2024-09-21T04:05:35.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.410+0000 7f342897d640 1 -- 172.21.15.64:0/610373651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3424102a60 msgr2=0x7f342419a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:35.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.410+0000 7f342897d640 1 --2- 172.21.15.64:0/610373651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3424102a60 0x7f342419a4a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f341c00d950 tx=0x7f341c00de10 comp rx=0 tx=0).stop 2024-09-21T04:05:35.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.410+0000 7f342897d640 1 -- 172.21.15.64:0/610373651 shutdown_connections 2024-09-21T04:05:35.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.410+0000 7f342897d640 1 --2- 172.21.15.64:0/610373651 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f34100b6680 0x7f34100b8b90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.410+0000 7f342897d640 1 --2- 172.21.15.64:0/610373651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3424108cf0 0x7f342419fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.410+0000 7f342897d640 1 --2- 172.21.15.64:0/610373651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34241083a0 0x7f342419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.410+0000 7f342897d640 1 --2- 172.21.15.64:0/610373651 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3424102a60 0x7f342419a4a0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:35.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.410+0000 7f342897d640 1 -- 172.21.15.64:0/610373651 >> 172.21.15.64:0/610373651 conn(0x7f3424079730 msgr2=0x7f342406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:35.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.410+0000 7f342897d640 1 -- 172.21.15.64:0/610373651 shutdown_connections 2024-09-21T04:05:35.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:35.410+0000 7f342897d640 1 -- 172.21.15.64:0/610373651 wait complete. 2024-09-21T04:05:36.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:35 smithi131 ceph-mon[35193]: pgmap v368: 97 pgs: 97 active+clean; 5.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 82 KiB/s wr, 25 op/s 2024-09-21T04:05:36.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:35 smithi131 ceph-mon[35193]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:05:36.142 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:35 smithi064 ceph-mon[31180]: pgmap v368: 97 pgs: 97 active+clean; 5.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 82 KiB/s wr, 25 op/s 2024-09-21T04:05:36.142 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:35 smithi064 ceph-mon[31180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:05:36.208 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T04:05:36.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:35 smithi119 ceph-mon[34811]: pgmap v368: 97 pgs: 97 active+clean; 5.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 82 KiB/s wr, 25 op/s 2024-09-21T04:05:36.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:35 smithi119 ceph-mon[34811]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:05:36.610 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:05:37.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.141+0000 7f4531a8e640 1 -- 172.21.15.64:0/1069495314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f452c108d00 msgr2=0x7f452c10f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:37.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.141+0000 7f4531a8e640 1 --2- 172.21.15.64:0/1069495314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f452c108d00 0x7f452c10f530 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4524009ca0 tx=0x7f452402a8c0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.141+0000 7f4531a8e640 1 -- 172.21.15.64:0/1069495314 shutdown_connections 2024-09-21T04:05:37.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.141+0000 7f4531a8e640 1 --2- 172.21.15.64:0/1069495314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f452c108d00 0x7f452c10f530 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.141+0000 7f4531a8e640 1 --2- 172.21.15.64:0/1069495314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f452c1083b0 0x7f452c1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.141+0000 7f4531a8e640 1 --2- 172.21.15.64:0/1069495314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f452c102ac0 0x7f452c102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.141+0000 7f4531a8e640 1 -- 172.21.15.64:0/1069495314 >> 172.21.15.64:0/1069495314 conn(0x7f452c079790 msgr2=0x7f452c079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:37.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.141+0000 7f4531a8e640 1 -- 172.21.15.64:0/1069495314 shutdown_connections 2024-09-21T04:05:37.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.141+0000 7f4531a8e640 1 -- 172.21.15.64:0/1069495314 wait complete. 2024-09-21T04:05:37.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.142+0000 7f4531a8e640 1 Processor -- start 2024-09-21T04:05:37.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.142+0000 7f4531a8e640 1 -- start start 2024-09-21T04:05:37.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.143+0000 7f4531a8e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f452c102ac0 0x7f452c1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:37.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.143+0000 7f4531a8e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f452c1083b0 0x7f452c199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:37.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.143+0000 7f4531a8e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f452c108d00 0x7f452c19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:37.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.143+0000 7f4531a8e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f452c1129c0 con 0x7f452c108d00 2024-09-21T04:05:37.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.143+0000 7f4531a8e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f452c112840 con 0x7f452c1083b0 2024-09-21T04:05:37.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.143+0000 7f4531a8e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f452c112b40 con 0x7f452c102ac0 2024-09-21T04:05:37.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.143+0000 7f452bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f452c108d00 0x7f452c19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:37.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.143+0000 7f452bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f452c108d00 0x7f452c19ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36074/0 (socket says 172.21.15.64:36074) 2024-09-21T04:05:37.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.143+0000 7f452affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f452c1083b0 0x7f452c199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:37.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.143+0000 7f452bfff640 1 -- 172.21.15.64:0/3313613703 learned_addr learned my addr 172.21.15.64:0/3313613703 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:37.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.143+0000 7f452b7fe640 1 --2- 172.21.15.64:0/3313613703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f452c102ac0 0x7f452c1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:37.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.143+0000 7f452affd640 1 -- 172.21.15.64:0/3313613703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f452c102ac0 msgr2=0x7f452c1993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:37.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.143+0000 7f452affd640 1 --2- 172.21.15.64:0/3313613703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f452c102ac0 0x7f452c1993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.143+0000 7f452affd640 1 -- 172.21.15.64:0/3313613703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f452c108d00 msgr2=0x7f452c19ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:37.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.143+0000 7f452affd640 1 --2- 172.21.15.64:0/3313613703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f452c108d00 0x7f452c19ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.143+0000 7f452affd640 1 -- 172.21.15.64:0/3313613703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452c19f360 con 0x7f452c1083b0 2024-09-21T04:05:37.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.143+0000 7f452bfff640 1 --2- 172.21.15.64:0/3313613703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f452c108d00 0x7f452c19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:05:37.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.143+0000 7f452affd640 1 --2- 172.21.15.64:0/3313613703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f452c1083b0 0x7f452c199990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f452c06ab20 tx=0x7f452000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:37.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.143+0000 7f4528ff9640 1 -- 172.21.15.64:0/3313613703 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452000cc40 con 0x7f452c1083b0 2024-09-21T04:05:37.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.144+0000 7f4528ff9640 1 -- 172.21.15.64:0/3313613703 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f452000cde0 con 0x7f452c1083b0 2024-09-21T04:05:37.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.144+0000 7f4531a8e640 1 -- 172.21.15.64:0/3313613703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452c19f650 con 0x7f452c1083b0 2024-09-21T04:05:37.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.144+0000 7f4528ff9640 1 -- 172.21.15.64:0/3313613703 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520004a00 con 0x7f452c1083b0 2024-09-21T04:05:37.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.144+0000 7f4531a8e640 1 -- 172.21.15.64:0/3313613703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f452c1a5db0 con 0x7f452c1083b0 2024-09-21T04:05:37.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.146+0000 7f4538002740 1 Processor -- start 2024-09-21T04:05:37.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.146+0000 7f4538002740 1 -- start start 2024-09-21T04:05:37.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.146+0000 7f4528ff9640 1 -- 172.21.15.64:0/3313613703 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f4520004ba0 con 0x7f452c1083b0 2024-09-21T04:05:37.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.147+0000 7f4538002740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9568f0070 0x55c956c976a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:37.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.147+0000 7f4538002740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568c71b0 0x55c956c97be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:37.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.147+0000 7f4538002740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c9568cf390 0x55c956c98120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:37.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.147+0000 7f4538002740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c956c0dea0 con 0x55c9568c71b0 2024-09-21T04:05:37.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.147+0000 7f4538002740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c956bfd2d0 con 0x55c9568f0070 2024-09-21T04:05:37.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.147+0000 7f4538002740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c956c010e0 con 0x55c9568cf390 2024-09-21T04:05:37.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.147+0000 7f452b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568c71b0 0x55c956c97be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:37.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.147+0000 7f452b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568c71b0 0x55c956c97be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36084/0 (socket says 172.21.15.64:36084) 2024-09-21T04:05:37.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.147+0000 7f452bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9568f0070 0x55c956c976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:37.156 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.147+0000 7f452b7fe640 1 -- 172.21.15.64:0/1939894161 learned_addr learned my addr 172.21.15.64:0/1939894161 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:37.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.147+0000 7f452affd640 1 --2- 172.21.15.64:0/1939894161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c9568cf390 0x55c956c98120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:37.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.147+0000 7f452b7fe640 1 -- 172.21.15.64:0/1939894161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c9568cf390 msgr2=0x55c956c98120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:37.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.147+0000 7f452b7fe640 1 --2- 172.21.15.64:0/1939894161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c9568cf390 0x55c956c98120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.147+0000 7f452b7fe640 1 -- 172.21.15.64:0/1939894161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9568f0070 msgr2=0x55c956c976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:37.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.147+0000 7f452b7fe640 1 --2- 172.21.15.64:0/1939894161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9568f0070 0x55c956c976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.147+0000 7f452b7fe640 1 -- 172.21.15.64:0/1939894161 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c95695b650 con 0x55c9568c71b0 2024-09-21T04:05:37.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.147+0000 7f452b7fe640 1 --2- 172.21.15.64:0/1939894161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568c71b0 0x55c956c97be0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f451c009cf0 tx=0x7f451c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3722f406d2fadae1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:37.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.147+0000 7f4528ff9640 1 --2- 172.21.15.64:0/3313613703 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f45140b6680 0x7f45140b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:37.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.148+0000 7f4511ffb640 1 -- 172.21.15.64:0/1939894161 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c004230 con 0x55c9568c71b0 2024-09-21T04:05:37.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.148+0000 7f452bfff640 1 --2- 172.21.15.64:0/3313613703 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f45140b6680 0x7f45140b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:37.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.148+0000 7f4528ff9640 1 -- 172.21.15.64:0/3313613703 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f45200e7650 con 0x7f452c1083b0 2024-09-21T04:05:37.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.148+0000 7f4511ffb640 1 -- 172.21.15.64:0/1939894161 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f451c0043d0 con 0x55c9568c71b0 2024-09-21T04:05:37.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.148+0000 7f4511ffb640 1 -- 172.21.15.64:0/1939894161 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c019880 con 0x55c9568c71b0 2024-09-21T04:05:37.157 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.149+0000 7f4538002740 1 -- 172.21.15.64:0/1939894161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568c71b0 msgr2=0x55c956c97be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:37.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.149+0000 7f452bfff640 1 --2- 172.21.15.64:0/3313613703 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f45140b6680 0x7f45140b8b90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4524005e30 tx=0x7f4524005c50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:37.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.149+0000 7f4538002740 1 --2- 172.21.15.64:0/1939894161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568c71b0 0x55c956c97be0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f451c009cf0 tx=0x7f451c011810 comp rx=0 tx=0).stop 2024-09-21T04:05:37.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.149+0000 7f4538002740 1 -- 172.21.15.64:0/1939894161 shutdown_connections 2024-09-21T04:05:37.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.149+0000 7f4538002740 1 --2- 172.21.15.64:0/1939894161 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c9568cf390 0x55c956c98120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.149+0000 7f4538002740 1 --2- 172.21.15.64:0/1939894161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568c71b0 0x55c956c97be0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.149+0000 7f4538002740 1 --2- 172.21.15.64:0/1939894161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9568f0070 0x55c956c976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.149+0000 7f4538002740 1 -- 172.21.15.64:0/1939894161 >> 172.21.15.64:0/1939894161 conn(0x55c9568e4460 msgr2=0x55c956c95590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:37.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.149+0000 7f4538002740 1 -- 172.21.15.64:0/1939894161 shutdown_connections 2024-09-21T04:05:37.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.149+0000 7f4538002740 1 -- 172.21.15.64:0/1939894161 wait complete. 2024-09-21T04:05:37.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.149+0000 7f4538002740 1 -- start start 2024-09-21T04:05:37.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.150+0000 7f4538002740 1 Processor -- start 2024-09-21T04:05:37.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.150+0000 7f4538002740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9568c71b0 0x55c956ca2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:37.158 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.150+0000 7f4538002740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568cf390 0x55c956ca33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.150+0000 7f4538002740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c9568f0070 0x55c956cb12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.150+0000 7f4538002740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c956a5e700 con 0x55c9568cf390 2024-09-21T04:05:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.150+0000 7f4538002740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c956b98160 con 0x55c9568c71b0 2024-09-21T04:05:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.150+0000 7f4538002740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c956ca8030 con 0x55c9568f0070 2024-09-21T04:05:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.150+0000 7f4538002740 10 client.0 _ll_register_callbacks cb 0x55c956bb19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:05:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.150+0000 7f452affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568cf390 0x55c956ca33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.150+0000 7f452b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9568c71b0 0x55c956ca2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.150+0000 7f452affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568cf390 0x55c956ca33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36094/0 (socket says 172.21.15.64:36094) 2024-09-21T04:05:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.150+0000 7f452affd640 1 -- 172.21.15.64:0/3932825109 learned_addr learned my addr 172.21.15.64:0/3932825109 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.150+0000 7f452bfff640 1 --2- 172.21.15.64:0/3932825109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c9568f0070 0x55c956cb12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.150+0000 7f452b7fe640 1 -- 172.21.15.64:0/3932825109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c9568f0070 msgr2=0x55c956cb12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.150+0000 7f452b7fe640 1 --2- 172.21.15.64:0/3932825109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c9568f0070 0x55c956cb12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.150+0000 7f452b7fe640 1 -- 172.21.15.64:0/3932825109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568cf390 msgr2=0x55c956ca33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.150+0000 7f452b7fe640 1 --2- 172.21.15.64:0/3932825109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568cf390 0x55c956ca33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.150+0000 7f452b7fe640 1 -- 172.21.15.64:0/3932825109 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c956bc5c90 con 0x55c9568c71b0 2024-09-21T04:05:37.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.150+0000 7f452affd640 1 --2- 172.21.15.64:0/3932825109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568cf390 0x55c956ca33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:05:37.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.151+0000 7f452b7fe640 1 --2- 172.21.15.64:0/3932825109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9568c71b0 0x55c956ca2ea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55c956c96320 tx=0x7f451c0045f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb600f7b51b4802f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:37.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.151+0000 7f450affd640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:05:37.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.151+0000 7f450affd640 1 -- 172.21.15.64:0/3932825109 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c027070 con 0x55c9568c71b0 2024-09-21T04:05:37.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.151+0000 7f4538002740 1 -- 172.21.15.64:0/3932825109 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f451c0043d0 con 0x55c9568c71b0 2024-09-21T04:05:37.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.151+0000 7f450affd640 1 -- 172.21.15.64:0/3932825109 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f451c005240 con 0x55c9568c71b0 2024-09-21T04:05:37.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.151+0000 7f450affd640 1 -- 172.21.15.64:0/3932825109 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f451c033050 con 0x55c9568c71b0 2024-09-21T04:05:37.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.151+0000 7f4538002740 10 client.25601 fetch_fsmap learned FSMap version 29 2024-09-21T04:05:37.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.151+0000 7f4538002740 1 -- 172.21.15.64:0/3932825109 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c956cb3df0 con 0x55c9568c71b0 2024-09-21T04:05:37.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.151+0000 7f450affd640 1 -- 172.21.15.64:0/3932825109 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f451c0043d0 con 0x55c9568c71b0 2024-09-21T04:05:37.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.151+0000 7f450affd640 1 -- 172.21.15.64:0/3932825109 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f451c004b70 con 0x55c9568c71b0 2024-09-21T04:05:37.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.152+0000 7f4538002740 10 client.25601 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T04:05:37.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.152+0000 7f4538002740 10 client.25601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:05:37.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.152+0000 7f4538002740 20 client.25601 populate_metadata read hostname 'smithi064' 2024-09-21T04:05:37.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.152+0000 7f4538002740 1 --2- 172.21.15.64:0/3932825109 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55c9568ae9f0 0x55c956cb71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:37.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.152+0000 7f4538002740 4 client.25601 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:05:37.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.152+0000 7f4538002740 1 -- 172.21.15.64:0/3932825109 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c956c0fdf0 con 0x55c9568ae9f0 2024-09-21T04:05:37.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.152+0000 7f452affd640 1 --2- 172.21.15.64:0/3932825109 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55c9568ae9f0 0x55c956cb71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:37.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.152+0000 7f450affd640 10 client.25601 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:37.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.152+0000 7f452affd640 1 --2- 172.21.15.64:0/3932825109 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55c9568ae9f0 0x55c956cb71b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1939bad4554b1a74 server_cookie=c80f8e27ca40ab9e in_seq=0 out_seq=0 2024-09-21T04:05:37.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.153+0000 7f450affd640 1 -- 172.21.15.64:0/3932825109 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c956c0fdf0 con 0x55c9568ae9f0 2024-09-21T04:05:37.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.153+0000 7f450affd640 10 client.25601 handle_command_reply: tid=0 2024-09-21T04:05:37.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.153+0000 7f4538002740 1 client.25601 shutdown 2024-09-21T04:05:37.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.153+0000 7f4538002740 1 -- 172.21.15.64:0/3932825109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9568c71b0 msgr2=0x55c956ca2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:37.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.153+0000 7f4538002740 1 --2- 172.21.15.64:0/3932825109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9568c71b0 0x55c956ca2ea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55c956c96320 tx=0x7f451c0045f0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.153+0000 7f4538002740 1 -- 172.21.15.64:0/3932825109 shutdown_connections 2024-09-21T04:05:37.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.153+0000 7f4538002740 1 --2- 172.21.15.64:0/3932825109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c9568f0070 0x55c956cb12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.153+0000 7f4538002740 1 --2- 172.21.15.64:0/3932825109 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55c9568ae9f0 0x55c956cb71b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.153+0000 7f4538002740 1 --2- 172.21.15.64:0/3932825109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568cf390 0x55c956ca33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.153+0000 7f4538002740 1 --2- 172.21.15.64:0/3932825109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9568c71b0 0x55c956ca2ea0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.153+0000 7f4538002740 1 -- 172.21.15.64:0/3932825109 >> 172.21.15.64:0/3932825109 conn(0x55c9568e4460 msgr2=0x55c956c9a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:37.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.153+0000 7f450affd640 0 client.25601 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:37.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.154+0000 7f4538002740 1 -- 172.21.15.64:0/3932825109 shutdown_connections 2024-09-21T04:05:37.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.154+0000 7f4538002740 1 -- 172.21.15.64:0/3932825109 wait complete. 2024-09-21T04:05:37.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.154+0000 7f4538002740 20 client.25601 trim_cache size 0 max 16384 2024-09-21T04:05:37.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.167+0000 7f4538002740 1 Processor -- start 2024-09-21T04:05:37.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.167+0000 7f4538002740 1 -- start start 2024-09-21T04:05:37.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.167+0000 7f4538002740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c9568c71b0 0x55c956c9b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:37.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.167+0000 7f4538002740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568cf390 0x55c956ca4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:37.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.167+0000 7f4538002740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9568f0070 0x55c956ca0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:37.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.167+0000 7f4538002740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f452c1b0030 con 0x55c9568cf390 2024-09-21T04:05:37.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.167+0000 7f4538002740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f452c1ad8a0 con 0x55c9568f0070 2024-09-21T04:05:37.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.167+0000 7f4538002740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f452c1ab6f0 con 0x55c9568c71b0 2024-09-21T04:05:37.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.167+0000 7f452affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568cf390 0x55c956ca4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:37.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.167+0000 7f452affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568cf390 0x55c956ca4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36100/0 (socket says 172.21.15.64:36100) 2024-09-21T04:05:37.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.167+0000 7f452affd640 1 -- 172.21.15.64:0/3340700158 learned_addr learned my addr 172.21.15.64:0/3340700158 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:37.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.167+0000 7f452bfff640 1 --2- 172.21.15.64:0/3340700158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9568f0070 0x55c956ca0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:37.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.167+0000 7f452b7fe640 1 --2- 172.21.15.64:0/3340700158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c9568c71b0 0x55c956c9b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:37.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.168+0000 7f452affd640 1 -- 172.21.15.64:0/3340700158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c9568c71b0 msgr2=0x55c956c9b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:37.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.168+0000 7f452affd640 1 --2- 172.21.15.64:0/3340700158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c9568c71b0 0x55c956c9b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.168+0000 7f452affd640 1 -- 172.21.15.64:0/3340700158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9568f0070 msgr2=0x55c956ca0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:37.172 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.168+0000 7f452affd640 1 --2- 172.21.15.64:0/3340700158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9568f0070 0x55c956ca0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.168+0000 7f452affd640 1 -- 172.21.15.64:0/3340700158 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c956ca4160 con 0x55c9568cf390 2024-09-21T04:05:37.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.168+0000 7f452b7fe640 1 --2- 172.21.15.64:0/3340700158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c9568c71b0 0x55c956c9b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:05:37.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.168+0000 7f452affd640 1 --2- 172.21.15.64:0/3340700158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568cf390 0x55c956ca4ad0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f45200a9a40 tx=0x7f45200a94d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c850848d1b1dd5b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:37.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.168+0000 7f45117fa640 1 -- 172.21.15.64:0/3340700158 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452001c040 con 0x55c9568cf390 2024-09-21T04:05:37.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.168+0000 7f45117fa640 1 -- 172.21.15.64:0/3340700158 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45200acf80 con 0x55c9568cf390 2024-09-21T04:05:37.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.168+0000 7f45117fa640 1 -- 172.21.15.64:0/3340700158 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45200b1f50 con 0x55c9568cf390 2024-09-21T04:05:37.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.169+0000 7f4538002740 1 -- 172.21.15.64:0/3340700158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568cf390 msgr2=0x55c956ca4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:37.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.169+0000 7f4538002740 1 --2- 172.21.15.64:0/3340700158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568cf390 0x55c956ca4ad0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f45200a9a40 tx=0x7f45200a94d0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.169+0000 7f4538002740 1 -- 172.21.15.64:0/3340700158 shutdown_connections 2024-09-21T04:05:37.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.169+0000 7f4538002740 1 --2- 172.21.15.64:0/3340700158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9568f0070 0x55c956ca0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.169+0000 7f4538002740 1 --2- 172.21.15.64:0/3340700158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568cf390 0x55c956ca4ad0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.169+0000 7f4538002740 1 --2- 172.21.15.64:0/3340700158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c9568c71b0 0x55c956c9b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.169+0000 7f4538002740 1 -- 172.21.15.64:0/3340700158 >> 172.21.15.64:0/3340700158 conn(0x55c9568e4460 msgr2=0x55c956ca43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:37.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.169+0000 7f4538002740 1 -- 172.21.15.64:0/3340700158 shutdown_connections 2024-09-21T04:05:37.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.169+0000 7f4538002740 1 -- 172.21.15.64:0/3340700158 wait complete. 2024-09-21T04:05:37.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.169+0000 7f4538002740 1 -- start start 2024-09-21T04:05:37.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.169+0000 7f4538002740 1 Processor -- start 2024-09-21T04:05:37.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.170+0000 7f4538002740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c9568c71b0 0x55c956caab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:37.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.170+0000 7f4538002740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568cf390 0x55c956cab0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:37.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.170+0000 7f4538002740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9568f0070 0x55c956d1fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:37.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.170+0000 7f4538002740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f452c1abd60 con 0x55c9568cf390 2024-09-21T04:05:37.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.170+0000 7f4538002740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f452c1ad1d0 con 0x55c9568f0070 2024-09-21T04:05:37.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.170+0000 7f4538002740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f452c1ac430 con 0x55c9568c71b0 2024-09-21T04:05:37.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.170+0000 7f4538002740 10 client.0 _ll_register_callbacks cb 0x7f4520005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:05:37.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.170+0000 7f452affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568cf390 0x55c956cab0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:37.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.170+0000 7f452affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568cf390 0x55c956cab0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36112/0 (socket says 172.21.15.64:36112) 2024-09-21T04:05:37.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.170+0000 7f452affd640 1 -- 172.21.15.64:0/2876806931 learned_addr learned my addr 172.21.15.64:0/2876806931 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:37.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.170+0000 7f452bfff640 1 --2- 172.21.15.64:0/2876806931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9568f0070 0x55c956d1fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:37.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.170+0000 7f452b7fe640 1 --2- 172.21.15.64:0/2876806931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c9568c71b0 0x55c956caab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:37.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.170+0000 7f452affd640 1 -- 172.21.15.64:0/2876806931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c9568c71b0 msgr2=0x55c956caab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:37.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.170+0000 7f452affd640 1 --2- 172.21.15.64:0/2876806931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c9568c71b0 0x55c956caab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.170+0000 7f452affd640 1 -- 172.21.15.64:0/2876806931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9568f0070 msgr2=0x55c956d1fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:37.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.170+0000 7f452affd640 1 --2- 172.21.15.64:0/2876806931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9568f0070 0x55c956d1fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.170+0000 7f452affd640 1 -- 172.21.15.64:0/2876806931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c956d200f0 con 0x55c9568cf390 2024-09-21T04:05:37.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.170+0000 7f452affd640 1 --2- 172.21.15.64:0/2876806931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568cf390 0x55c956cab0c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f45200e5370 tx=0x7f45200a9390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f143d34372dab56 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:37.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.170+0000 7f450affd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:05:37.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.171+0000 7f450affd640 1 -- 172.21.15.64:0/2876806931 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45200d4020 con 0x55c9568cf390 2024-09-21T04:05:37.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.171+0000 7f450affd640 1 -- 172.21.15.64:0/2876806931 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f45200e9b80 con 0x55c9568cf390 2024-09-21T04:05:37.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.171+0000 7f4538002740 1 -- 172.21.15.64:0/2876806931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45200acf80 con 0x55c9568cf390 2024-09-21T04:05:37.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.171+0000 7f450affd640 1 -- 172.21.15.64:0/2876806931 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f452001c070 con 0x55c9568cf390 2024-09-21T04:05:37.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.171+0000 7f4538002740 10 client.15897 fetch_fsmap learned FSMap version 29 2024-09-21T04:05:37.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.171+0000 7f4538002740 1 -- 172.21.15.64:0/2876806931 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c956ca5c40 con 0x55c9568cf390 2024-09-21T04:05:37.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.171+0000 7f450affd640 1 -- 172.21.15.64:0/2876806931 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45200acf80 con 0x55c9568cf390 2024-09-21T04:05:37.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.171+0000 7f450affd640 1 -- 172.21.15.64:0/2876806931 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4520094bb0 con 0x55c9568cf390 2024-09-21T04:05:37.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.171+0000 7f4538002740 10 client.15897 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T04:05:37.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.171+0000 7f4538002740 10 client.15897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:05:37.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.171+0000 7f4538002740 20 client.15897 populate_metadata read hostname 'smithi064' 2024-09-21T04:05:37.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.171+0000 7f4538002740 1 --2- 172.21.15.64:0/2876806931 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55c956883da0 0x55c956ca6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:37.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.171+0000 7f4538002740 4 client.15897 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:05:37.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.171+0000 7f4538002740 1 -- 172.21.15.64:0/2876806931 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c956c0b8a0 con 0x55c956883da0 2024-09-21T04:05:37.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.171+0000 7f452b7fe640 1 --2- 172.21.15.64:0/2876806931 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55c956883da0 0x55c956ca6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:37.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.172+0000 7f450affd640 10 client.15897 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:37.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.172+0000 7f452b7fe640 1 --2- 172.21.15.64:0/2876806931 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55c956883da0 0x55c956ca6470 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51ff92c1fbfb0cc9 server_cookie=71afa8bcc0213a7e in_seq=0 out_seq=0 2024-09-21T04:05:37.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.172+0000 7f450affd640 1 -- 172.21.15.64:0/2876806931 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c956c0b8a0 con 0x55c956883da0 2024-09-21T04:05:37.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.172+0000 7f450affd640 10 client.15897 handle_command_reply: tid=0 2024-09-21T04:05:37.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.172+0000 7f4538002740 1 client.15897 shutdown 2024-09-21T04:05:37.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.172+0000 7f4538002740 1 -- 172.21.15.64:0/2876806931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568cf390 msgr2=0x55c956cab0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:37.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.172+0000 7f4538002740 1 --2- 172.21.15.64:0/2876806931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568cf390 0x55c956cab0c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f45200e5370 tx=0x7f45200a9390 comp rx=0 tx=0).stop 2024-09-21T04:05:37.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.172+0000 7f4538002740 1 -- 172.21.15.64:0/2876806931 shutdown_connections 2024-09-21T04:05:37.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.172+0000 7f4538002740 1 --2- 172.21.15.64:0/2876806931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c9568f0070 0x55c956d1fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.172+0000 7f4538002740 1 --2- 172.21.15.64:0/2876806931 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55c956883da0 0x55c956ca6470 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.172+0000 7f4538002740 1 --2- 172.21.15.64:0/2876806931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c9568cf390 0x55c956cab0c0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.172+0000 7f4538002740 1 --2- 172.21.15.64:0/2876806931 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c9568c71b0 0x55c956caab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.172+0000 7f450affd640 0 client.15897 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:37.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.172+0000 7f4538002740 1 -- 172.21.15.64:0/2876806931 >> 172.21.15.64:0/2876806931 conn(0x55c9568e4460 msgr2=0x55c956c9af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:37.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.172+0000 7f4538002740 1 -- 172.21.15.64:0/2876806931 shutdown_connections 2024-09-21T04:05:37.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.172+0000 7f4538002740 1 -- 172.21.15.64:0/2876806931 wait complete. 2024-09-21T04:05:37.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.172+0000 7f4538002740 20 client.15897 trim_cache size 0 max 16384 2024-09-21T04:05:37.181 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:05:37.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.173+0000 7f4531a8e640 1 -- 172.21.15.64:0/3313613703 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f45140b6680 msgr2=0x7f45140b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:37.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.173+0000 7f4531a8e640 1 --2- 172.21.15.64:0/3313613703 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f45140b6680 0x7f45140b8b90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4524005e30 tx=0x7f4524005c50 comp rx=0 tx=0).stop 2024-09-21T04:05:37.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.173+0000 7f4531a8e640 1 -- 172.21.15.64:0/3313613703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f452c1083b0 msgr2=0x7f452c199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:37.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.173+0000 7f4531a8e640 1 --2- 172.21.15.64:0/3313613703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f452c1083b0 0x7f452c199990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f452c06ab20 tx=0x7f452000ee50 comp rx=0 tx=0).stop 2024-09-21T04:05:37.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.173+0000 7f4531a8e640 1 -- 172.21.15.64:0/3313613703 shutdown_connections 2024-09-21T04:05:37.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.173+0000 7f4531a8e640 1 --2- 172.21.15.64:0/3313613703 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f45140b6680 0x7f45140b8b90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.173+0000 7f4531a8e640 1 --2- 172.21.15.64:0/3313613703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f452c108d00 0x7f452c19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.173+0000 7f4531a8e640 1 --2- 172.21.15.64:0/3313613703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f452c1083b0 0x7f452c199990 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.173+0000 7f4531a8e640 1 --2- 172.21.15.64:0/3313613703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f452c102ac0 0x7f452c1993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:37.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.173+0000 7f4531a8e640 1 -- 172.21.15.64:0/3313613703 >> 172.21.15.64:0/3313613703 conn(0x7f452c079790 msgr2=0x7f452c106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:37.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.173+0000 7f4531a8e640 1 -- 172.21.15.64:0/3313613703 shutdown_connections 2024-09-21T04:05:37.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:37.173+0000 7f4531a8e640 1 -- 172.21.15.64:0/3313613703 wait complete. 2024-09-21T04:05:38.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:37 smithi131 ceph-mon[35193]: pgmap v369: 97 pgs: 97 active+clean; 15 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 874 KiB/s wr, 37 op/s 2024-09-21T04:05:38.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:37 smithi119 ceph-mon[34811]: pgmap v369: 97 pgs: 97 active+clean; 15 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 874 KiB/s wr, 37 op/s 2024-09-21T04:05:38.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:37 smithi064 ceph-mon[31180]: pgmap v369: 97 pgs: 97 active+clean; 15 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 874 KiB/s wr, 37 op/s 2024-09-21T04:05:38.852 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T04:05:38.852 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T04:05:39.180 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:05:39.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.707+0000 7f31b8475640 1 -- 172.21.15.64:0/3405342157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31b01083c0 msgr2=0x7f31b01087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:39.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.707+0000 7f31b8475640 1 --2- 172.21.15.64:0/3405342157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31b01083c0 0x7f31b01087d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f31a0009900 tx=0x7f31a002a410 comp rx=0 tx=0).stop 2024-09-21T04:05:39.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.707+0000 7f31b8475640 1 -- 172.21.15.64:0/3405342157 shutdown_connections 2024-09-21T04:05:39.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.707+0000 7f31b8475640 1 --2- 172.21.15.64:0/3405342157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31b0108d10 0x7f31b010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.707+0000 7f31b8475640 1 --2- 172.21.15.64:0/3405342157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31b01083c0 0x7f31b01087d0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.707+0000 7f31b8475640 1 --2- 172.21.15.64:0/3405342157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31b0102a80 0x7f31b0102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.707+0000 7f31b8475640 1 -- 172.21.15.64:0/3405342157 >> 172.21.15.64:0/3405342157 conn(0x7f31b0079740 msgr2=0x7f31b0079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:39.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.707+0000 7f31b8475640 1 -- 172.21.15.64:0/3405342157 shutdown_connections 2024-09-21T04:05:39.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.707+0000 7f31b8475640 1 -- 172.21.15.64:0/3405342157 wait complete. 2024-09-21T04:05:39.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.708+0000 7f31b8475640 1 Processor -- start 2024-09-21T04:05:39.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.708+0000 7f31b8475640 1 -- start start 2024-09-21T04:05:39.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.708+0000 7f31b8475640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31b0102a80 0x7f31b019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:39.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.708+0000 7f31b8475640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31b01083c0 0x7f31b019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:39.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.708+0000 7f31b8475640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31b0108d10 0x7f31b019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:39.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.708+0000 7f31b8475640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f31b0112980 con 0x7f31b01083c0 2024-09-21T04:05:39.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.708+0000 7f31b8475640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f31b0112800 con 0x7f31b0108d10 2024-09-21T04:05:39.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.708+0000 7f31b8475640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f31b0112b00 con 0x7f31b0102a80 2024-09-21T04:05:39.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.708+0000 7f31b5a10640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31b01083c0 0x7f31b019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:39.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.708+0000 7f31b5a10640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31b01083c0 0x7f31b019aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36126/0 (socket says 172.21.15.64:36126) 2024-09-21T04:05:39.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.708+0000 7f31b5a10640 1 -- 172.21.15.64:0/1411005340 learned_addr learned my addr 172.21.15.64:0/1411005340 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:39.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.708+0000 7f31b6a12640 1 --2- 172.21.15.64:0/1411005340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31b0108d10 0x7f31b019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:39.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.708+0000 7f31b6211640 1 --2- 172.21.15.64:0/1411005340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31b0102a80 0x7f31b019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:39.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.708+0000 7f31b5a10640 1 -- 172.21.15.64:0/1411005340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31b0102a80 msgr2=0x7f31b019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:39.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.708+0000 7f31b5a10640 1 --2- 172.21.15.64:0/1411005340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31b0102a80 0x7f31b019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.708+0000 7f31b5a10640 1 -- 172.21.15.64:0/1411005340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31b0108d10 msgr2=0x7f31b019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:39.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.708+0000 7f31b5a10640 1 --2- 172.21.15.64:0/1411005340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31b0108d10 0x7f31b019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.708+0000 7f31b5a10640 1 -- 172.21.15.64:0/1411005340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b01a0430 con 0x7f31b01083c0 2024-09-21T04:05:39.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.709+0000 7f31b5a10640 1 --2- 172.21.15.64:0/1411005340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31b01083c0 0x7f31b019aa60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f31a0032fd0 tx=0x7f31a0031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:39.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.709+0000 7f31a77fe640 1 -- 172.21.15.64:0/1411005340 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a003f030 con 0x7f31b01083c0 2024-09-21T04:05:39.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.709+0000 7f31b8475640 1 -- 172.21.15.64:0/1411005340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31b01a06c0 con 0x7f31b01083c0 2024-09-21T04:05:39.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.709+0000 7f31a77fe640 1 -- 172.21.15.64:0/1411005340 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31a0004490 con 0x7f31b01083c0 2024-09-21T04:05:39.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.709+0000 7f31a77fe640 1 -- 172.21.15.64:0/1411005340 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0035070 con 0x7f31b01083c0 2024-09-21T04:05:39.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.709+0000 7f31b8475640 1 -- 172.21.15.64:0/1411005340 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31b01a6e80 con 0x7f31b01083c0 2024-09-21T04:05:39.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.712+0000 7f31a77fe640 1 -- 172.21.15.64:0/1411005340 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f31a0042030 con 0x7f31b01083c0 2024-09-21T04:05:39.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.712+0000 7f31be9e9740 1 Processor -- start 2024-09-21T04:05:39.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.712+0000 7f31be9e9740 1 -- start start 2024-09-21T04:05:39.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.712+0000 7f31be9e9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56418bf2d350 0x56418c2d4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:39.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.712+0000 7f31be9e9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56418bf04490 0x56418c2d4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:39.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.712+0000 7f31be9e9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56418bf0c670 0x56418c2d4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:39.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.712+0000 7f31be9e9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56418c24ae60 con 0x56418bf0c670 2024-09-21T04:05:39.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.712+0000 7f31be9e9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56418c23a290 con 0x56418bf04490 2024-09-21T04:05:39.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.712+0000 7f31be9e9740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56418c23e0a0 con 0x56418bf2d350 2024-09-21T04:05:39.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.713+0000 7f31b6211640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56418bf04490 0x56418c2d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:39.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.713+0000 7f31b5a10640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56418bf0c670 0x56418c2d4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:39.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.713+0000 7f31b6211640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56418bf04490 0x56418c2d4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:39648/0 (socket says 172.21.15.64:39648) 2024-09-21T04:05:39.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.713+0000 7f31b6211640 1 -- 172.21.15.64:0/1231475077 learned_addr learned my addr 172.21.15.64:0/1231475077 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:39.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.713+0000 7f31b6a12640 1 --2- 172.21.15.64:0/1231475077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56418bf2d350 0x56418c2d4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:39.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.713+0000 7f31a77fe640 1 --2- 172.21.15.64:0/1411005340 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f319c0b6680 0x7f319c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:39.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.713+0000 7f31b6211640 1 -- 172.21.15.64:0/1231475077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56418bf2d350 msgr2=0x56418c2d4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:39.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.713+0000 7f31b6211640 1 --2- 172.21.15.64:0/1231475077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56418bf2d350 0x56418c2d4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.713+0000 7f31b6211640 1 -- 172.21.15.64:0/1231475077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56418bf0c670 msgr2=0x56418c2d4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:39.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.713+0000 7f31b6211640 1 --2- 172.21.15.64:0/1231475077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56418bf0c670 0x56418c2d4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.713+0000 7f31b6211640 1 -- 172.21.15.64:0/1231475077 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56418bf98930 con 0x56418bf04490 2024-09-21T04:05:39.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.713+0000 7f31b6a12640 1 --2- 172.21.15.64:0/1411005340 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f319c0b6680 0x7f319c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:39.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.713+0000 7f31a77fe640 1 -- 172.21.15.64:0/1411005340 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f31a0107a70 con 0x7f31b01083c0 2024-09-21T04:05:39.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.713+0000 7f31b6211640 1 --2- 172.21.15.64:0/1231475077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56418bf04490 0x56418c2d4970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f31ac009cf0 tx=0x7f31ac011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73ec1cbb54f5ffda server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:39.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.713+0000 7f31a4d9a640 1 -- 172.21.15.64:0/1231475077 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac004230 con 0x56418bf04490 2024-09-21T04:05:39.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.714+0000 7f31a4d9a640 1 -- 172.21.15.64:0/1231475077 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31ac0043d0 con 0x56418bf04490 2024-09-21T04:05:39.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.714+0000 7f31a4d9a640 1 -- 172.21.15.64:0/1231475077 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac019880 con 0x56418bf04490 2024-09-21T04:05:39.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.714+0000 7f31be9e9740 1 -- 172.21.15.64:0/1231475077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56418bf04490 msgr2=0x56418c2d4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:39.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.714+0000 7f31b6a12640 1 --2- 172.21.15.64:0/1411005340 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f319c0b6680 0x7f319c0b8b90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f31a8009fd0 tx=0x7f31a8004a40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:39.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.714+0000 7f31be9e9740 1 --2- 172.21.15.64:0/1231475077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56418bf04490 0x56418c2d4970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f31ac009cf0 tx=0x7f31ac011810 comp rx=0 tx=0).stop 2024-09-21T04:05:39.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.714+0000 7f31be9e9740 1 -- 172.21.15.64:0/1231475077 shutdown_connections 2024-09-21T04:05:39.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.715+0000 7f31be9e9740 1 --2- 172.21.15.64:0/1231475077 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56418bf0c670 0x56418c2d4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.715+0000 7f31be9e9740 1 --2- 172.21.15.64:0/1231475077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56418bf04490 0x56418c2d4970 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.715+0000 7f31be9e9740 1 --2- 172.21.15.64:0/1231475077 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56418bf2d350 0x56418c2d4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.715+0000 7f31be9e9740 1 -- 172.21.15.64:0/1231475077 >> 172.21.15.64:0/1231475077 conn(0x56418bf21740 msgr2=0x56418c2d2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:39.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.715+0000 7f31be9e9740 1 -- 172.21.15.64:0/1231475077 shutdown_connections 2024-09-21T04:05:39.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.715+0000 7f31be9e9740 1 -- 172.21.15.64:0/1231475077 wait complete. 2024-09-21T04:05:39.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.715+0000 7f31be9e9740 1 -- start start 2024-09-21T04:05:39.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.715+0000 7f31be9e9740 1 Processor -- start 2024-09-21T04:05:39.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.715+0000 7f31be9e9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56418bf04490 0x56418c2dffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:39.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.715+0000 7f31be9e9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56418bf0c670 0x56418c2e0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:39.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.715+0000 7f31be9e9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56418bf2d350 0x56418c2ebf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:39.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.715+0000 7f31be9e9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56418c09b890 con 0x56418bf2d350 2024-09-21T04:05:39.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.715+0000 7f31be9e9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56418c1d4d40 con 0x56418bf04490 2024-09-21T04:05:39.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.715+0000 7f31be9e9740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56418c2e2c10 con 0x56418bf0c670 2024-09-21T04:05:39.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.716+0000 7f31be9e9740 10 client.0 _ll_register_callbacks cb 0x56418c238c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:05:39.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.716+0000 7f31b5a10640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56418bf0c670 0x56418c2e0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:39.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.716+0000 7f31b5a10640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56418bf0c670 0x56418c2e0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:58888/0 (socket says 172.21.15.64:58888) 2024-09-21T04:05:39.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.716+0000 7f31b5a10640 1 -- 172.21.15.64:0/1237236267 learned_addr learned my addr 172.21.15.64:0/1237236267 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:39.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.716+0000 7f31b6211640 1 --2- 172.21.15.64:0/1237236267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56418bf04490 0x56418c2dffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:39.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.716+0000 7f31b6a12640 1 --2- 172.21.15.64:0/1237236267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56418bf2d350 0x56418c2ebf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:39.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.716+0000 7f31b5a10640 1 -- 172.21.15.64:0/1237236267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56418bf04490 msgr2=0x56418c2dffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:39.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.716+0000 7f31b5a10640 1 --2- 172.21.15.64:0/1237236267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56418bf04490 0x56418c2dffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.716+0000 7f31b5a10640 1 -- 172.21.15.64:0/1237236267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56418bf2d350 msgr2=0x56418c2ebf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:39.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.716+0000 7f31b5a10640 1 --2- 172.21.15.64:0/1237236267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56418bf2d350 0x56418c2ebf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.716+0000 7f31b5a10640 1 -- 172.21.15.64:0/1237236267 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56418c2ec4f0 con 0x56418bf0c670 2024-09-21T04:05:39.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.716+0000 7f31b6a12640 1 --2- 172.21.15.64:0/1237236267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56418bf2d350 0x56418c2ebf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:05:39.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.716+0000 7f31b5a10640 1 --2- 172.21.15.64:0/1237236267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56418bf0c670 0x56418c2e0510 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f31a0031440 tx=0x7f31a003b560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e926a519abab8729 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:39.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.716+0000 7f3199ffb640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-21T04:05:39.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.716+0000 7f3199ffb640 1 -- 172.21.15.64:0/1237236267 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0049070 con 0x56418bf0c670 2024-09-21T04:05:39.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.716+0000 7f3199ffb640 1 -- 172.21.15.64:0/1237236267 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f31a00c9bb0 con 0x56418bf0c670 2024-09-21T04:05:39.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.716+0000 7f31be9e9740 1 -- 172.21.15.64:0/1237236267 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31ac0043d0 con 0x56418bf0c670 2024-09-21T04:05:39.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.717+0000 7f3199ffb640 1 -- 172.21.15.64:0/1237236267 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f31a00c97e0 con 0x56418bf0c670 2024-09-21T04:05:39.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.717+0000 7f31be9e9740 10 client.25543 fetch_fsmap learned FSMap version 29 2024-09-21T04:05:39.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.717+0000 7f31be9e9740 1 -- 172.21.15.64:0/1237236267 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56418c2eee20 con 0x56418bf0c670 2024-09-21T04:05:39.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.717+0000 7f3199ffb640 1 -- 172.21.15.64:0/1237236267 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31ac0043d0 con 0x56418bf0c670 2024-09-21T04:05:39.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.717+0000 7f3199ffb640 1 -- 172.21.15.64:0/1237236267 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f31a00d6e20 con 0x56418bf0c670 2024-09-21T04:05:39.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.717+0000 7f31be9e9740 10 client.25543 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T04:05:39.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.717+0000 7f31be9e9740 10 client.25543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:05:39.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.717+0000 7f31be9e9740 20 client.25543 populate_metadata read hostname 'smithi064' 2024-09-21T04:05:39.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.717+0000 7f31be9e9740 1 --2- 172.21.15.64:0/1237236267 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56418beebcd0 0x56418c2f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:39.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.717+0000 7f31be9e9740 4 client.25543 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:05:39.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.717+0000 7f31be9e9740 1 -- 172.21.15.64:0/1237236267 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56418c24cdb0 con 0x56418beebcd0 2024-09-21T04:05:39.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.717+0000 7f31b6211640 1 --2- 172.21.15.64:0/1237236267 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56418beebcd0 0x56418c2f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:39.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.718+0000 7f3199ffb640 10 client.25543 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:39.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.718+0000 7f31b6211640 1 --2- 172.21.15.64:0/1237236267 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56418beebcd0 0x56418c2f22d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aec61be8e58f0cd1 server_cookie=ea1215846a40c809 in_seq=0 out_seq=0 2024-09-21T04:05:39.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.718+0000 7f3199ffb640 1 -- 172.21.15.64:0/1237236267 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56418c24cdb0 con 0x56418beebcd0 2024-09-21T04:05:39.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.719+0000 7f3199ffb640 10 client.25543 handle_command_reply: tid=0 2024-09-21T04:05:39.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.719+0000 7f31be9e9740 1 client.25543 shutdown 2024-09-21T04:05:39.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.719+0000 7f31be9e9740 1 -- 172.21.15.64:0/1237236267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56418bf0c670 msgr2=0x56418c2e0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:39.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.719+0000 7f31be9e9740 1 --2- 172.21.15.64:0/1237236267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56418bf0c670 0x56418c2e0510 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f31a0031440 tx=0x7f31a003b560 comp rx=0 tx=0).stop 2024-09-21T04:05:39.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.719+0000 7f31be9e9740 1 -- 172.21.15.64:0/1237236267 shutdown_connections 2024-09-21T04:05:39.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.719+0000 7f31be9e9740 1 --2- 172.21.15.64:0/1237236267 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56418beebcd0 0x56418c2f22d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.719+0000 7f31be9e9740 1 --2- 172.21.15.64:0/1237236267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56418bf2d350 0x56418c2ebf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.719+0000 7f31be9e9740 1 --2- 172.21.15.64:0/1237236267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56418bf0c670 0x56418c2e0510 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.719+0000 7f3199ffb640 0 client.25543 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:39.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.719+0000 7f31be9e9740 1 --2- 172.21.15.64:0/1237236267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56418bf04490 0x56418c2dffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.719+0000 7f31be9e9740 1 -- 172.21.15.64:0/1237236267 >> 172.21.15.64:0/1237236267 conn(0x56418bf21740 msgr2=0x56418c2d75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:39.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.719+0000 7f31be9e9740 1 -- 172.21.15.64:0/1237236267 shutdown_connections 2024-09-21T04:05:39.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.719+0000 7f31be9e9740 1 -- 172.21.15.64:0/1237236267 wait complete. 2024-09-21T04:05:39.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.719+0000 7f31be9e9740 20 client.25543 trim_cache size 0 max 16384 2024-09-21T04:05:39.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.739+0000 7f31be9e9740 1 Processor -- start 2024-09-21T04:05:39.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.739+0000 7f31be9e9740 1 -- start start 2024-09-21T04:05:39.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.739+0000 7f31be9e9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56418beebcd0 0x56418c2dcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:39.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.739+0000 7f31be9e9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56418bf04490 0x56418c2dd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:39.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.739+0000 7f31be9e9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56418bf0c670 0x56418c2f7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:39.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.739+0000 7f31be9e9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f31b01b2140 con 0x56418beebcd0 2024-09-21T04:05:39.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.739+0000 7f31be9e9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f31b01af9b0 con 0x56418bf04490 2024-09-21T04:05:39.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.739+0000 7f31be9e9740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f31b01ad800 con 0x56418bf0c670 2024-09-21T04:05:39.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.739+0000 7f31b6211640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56418beebcd0 0x56418c2dcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:39.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.739+0000 7f31b6211640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56418beebcd0 0x56418c2dcfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36150/0 (socket says 172.21.15.64:36150) 2024-09-21T04:05:39.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.739+0000 7f31b6211640 1 -- 172.21.15.64:0/2287909303 learned_addr learned my addr 172.21.15.64:0/2287909303 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:39.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.739+0000 7f31b5a10640 1 --2- 172.21.15.64:0/2287909303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56418bf04490 0x56418c2dd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:39.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.739+0000 7f31b6a12640 1 --2- 172.21.15.64:0/2287909303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56418bf0c670 0x56418c2f7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:39.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.739+0000 7f31b6211640 1 -- 172.21.15.64:0/2287909303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56418bf0c670 msgr2=0x56418c2f7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:39.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.739+0000 7f31b6211640 1 --2- 172.21.15.64:0/2287909303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56418bf0c670 0x56418c2f7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.739+0000 7f31b6211640 1 -- 172.21.15.64:0/2287909303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56418bf04490 msgr2=0x56418c2dd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:39.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.739+0000 7f31b6211640 1 --2- 172.21.15.64:0/2287909303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56418bf04490 0x56418c2dd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.740+0000 7f31b6211640 1 -- 172.21.15.64:0/2287909303 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56418c2ded30 con 0x56418beebcd0 2024-09-21T04:05:39.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.740+0000 7f31b6211640 1 --2- 172.21.15.64:0/2287909303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56418beebcd0 0x56418c2dcfc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f31ac011d10 tx=0x7f31ac011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38605546977c7701 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:39.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.740+0000 7f319bfff640 1 -- 172.21.15.64:0/2287909303 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac02f020 con 0x56418beebcd0 2024-09-21T04:05:39.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.740+0000 7f319bfff640 1 -- 172.21.15.64:0/2287909303 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31ac005040 con 0x56418beebcd0 2024-09-21T04:05:39.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.740+0000 7f319bfff640 1 -- 172.21.15.64:0/2287909303 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac019740 con 0x56418beebcd0 2024-09-21T04:05:39.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.741+0000 7f31be9e9740 1 -- 172.21.15.64:0/2287909303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56418beebcd0 msgr2=0x56418c2dcfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:39.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.741+0000 7f31be9e9740 1 --2- 172.21.15.64:0/2287909303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56418beebcd0 0x56418c2dcfc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f31ac011d10 tx=0x7f31ac011810 comp rx=0 tx=0).stop 2024-09-21T04:05:39.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.741+0000 7f31be9e9740 1 -- 172.21.15.64:0/2287909303 shutdown_connections 2024-09-21T04:05:39.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.741+0000 7f31be9e9740 1 --2- 172.21.15.64:0/2287909303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56418bf0c670 0x56418c2f7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.741+0000 7f31be9e9740 1 --2- 172.21.15.64:0/2287909303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56418bf04490 0x56418c2dd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.741+0000 7f31be9e9740 1 --2- 172.21.15.64:0/2287909303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56418beebcd0 0x56418c2dcfc0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.741+0000 7f31be9e9740 1 -- 172.21.15.64:0/2287909303 >> 172.21.15.64:0/2287909303 conn(0x56418bf21740 msgr2=0x56418c2daf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:39.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.741+0000 7f31be9e9740 1 -- 172.21.15.64:0/2287909303 shutdown_connections 2024-09-21T04:05:39.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.741+0000 7f31be9e9740 1 -- 172.21.15.64:0/2287909303 wait complete. 2024-09-21T04:05:39.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.741+0000 7f31be9e9740 1 -- start start 2024-09-21T04:05:39.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.741+0000 7f31be9e9740 1 Processor -- start 2024-09-21T04:05:39.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.742+0000 7f31be9e9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56418beebcd0 0x56418c2e8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:39.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.742+0000 7f31be9e9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56418bf04490 0x56418c2e4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:39.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.742+0000 7f31be9e9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56418bf0c670 0x56418c2e5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:39.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.742+0000 7f31be9e9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f31b01ade70 con 0x56418beebcd0 2024-09-21T04:05:39.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.742+0000 7f31be9e9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f31b01af2e0 con 0x56418bf0c670 2024-09-21T04:05:39.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.742+0000 7f31be9e9740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f31b01ae540 con 0x56418bf04490 2024-09-21T04:05:39.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.742+0000 7f31be9e9740 10 client.0 _ll_register_callbacks cb 0x7f31a0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:05:39.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.742+0000 7f31b6211640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56418beebcd0 0x56418c2e8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:39.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.742+0000 7f31b6211640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56418beebcd0 0x56418c2e8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36154/0 (socket says 172.21.15.64:36154) 2024-09-21T04:05:39.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.742+0000 7f31b6211640 1 -- 172.21.15.64:0/3773325586 learned_addr learned my addr 172.21.15.64:0/3773325586 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:39.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.742+0000 7f31b6a12640 1 --2- 172.21.15.64:0/3773325586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56418bf0c670 0x56418c2e5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:39.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.742+0000 7f31b5a10640 1 --2- 172.21.15.64:0/3773325586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56418bf04490 0x56418c2e4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:39.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.742+0000 7f31b6211640 1 -- 172.21.15.64:0/3773325586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56418bf04490 msgr2=0x56418c2e4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:39.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.742+0000 7f31b6211640 1 --2- 172.21.15.64:0/3773325586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56418bf04490 0x56418c2e4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.742+0000 7f31b6211640 1 -- 172.21.15.64:0/3773325586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56418bf0c670 msgr2=0x56418c2e5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:39.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.742+0000 7f31b6211640 1 --2- 172.21.15.64:0/3773325586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56418bf0c670 0x56418c2e5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.742+0000 7f31b6211640 1 -- 172.21.15.64:0/3773325586 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56418c2e5770 con 0x56418beebcd0 2024-09-21T04:05:39.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.742+0000 7f31b6211640 1 --2- 172.21.15.64:0/3773325586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56418beebcd0 0x56418c2e8630 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f31ac005e00 tx=0x7f31ac0096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e69a5c6da9be81 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:39.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.742+0000 7f3199ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:05:39.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.743+0000 7f3199ffb640 1 -- 172.21.15.64:0/3773325586 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac02f030 con 0x56418beebcd0 2024-09-21T04:05:39.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.743+0000 7f3199ffb640 1 -- 172.21.15.64:0/3773325586 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f31ac0054b0 con 0x56418beebcd0 2024-09-21T04:05:39.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.743+0000 7f31be9e9740 1 -- 172.21.15.64:0/3773325586 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31ac005040 con 0x56418beebcd0 2024-09-21T04:05:39.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.743+0000 7f3199ffb640 1 -- 172.21.15.64:0/3773325586 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f31ac0035f0 con 0x56418beebcd0 2024-09-21T04:05:39.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.743+0000 7f31be9e9740 10 client.15915 fetch_fsmap learned FSMap version 29 2024-09-21T04:05:39.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.743+0000 7f31be9e9740 1 -- 172.21.15.64:0/3773325586 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56418c362020 con 0x56418beebcd0 2024-09-21T04:05:39.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.743+0000 7f3199ffb640 1 -- 172.21.15.64:0/3773325586 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31ac005040 con 0x56418beebcd0 2024-09-21T04:05:39.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.743+0000 7f3199ffb640 1 -- 172.21.15.64:0/3773325586 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f31ac045070 con 0x56418beebcd0 2024-09-21T04:05:39.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.743+0000 7f31be9e9740 10 client.15915 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T04:05:39.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.743+0000 7f31be9e9740 10 client.15915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:05:39.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.743+0000 7f31be9e9740 20 client.15915 populate_metadata read hostname 'smithi064' 2024-09-21T04:05:39.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.743+0000 7f31be9e9740 1 --2- 172.21.15.64:0/3773325586 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56418bf2d350 0x56418c364610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:39.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.743+0000 7f31be9e9740 4 client.15915 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T04:05:39.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.743+0000 7f31be9e9740 1 -- 172.21.15.64:0/3773325586 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56418c248860 con 0x56418bf2d350 2024-09-21T04:05:39.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.744+0000 7f31b5a10640 1 --2- 172.21.15.64:0/3773325586 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56418bf2d350 0x56418c364610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:39.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.744+0000 7f31b5a10640 1 --2- 172.21.15.64:0/3773325586 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56418bf2d350 0x56418c364610 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e23bd47b830a5865 server_cookie=ceb31bff671d3ade in_seq=0 out_seq=0 2024-09-21T04:05:39.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.744+0000 7f3199ffb640 10 client.15915 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:39.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.746+0000 7f3199ffb640 1 -- 172.21.15.64:0/3773325586 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56418c248860 con 0x56418bf2d350 2024-09-21T04:05:39.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.746+0000 7f3199ffb640 10 client.15915 handle_command_reply: tid=0 2024-09-21T04:05:39.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.746+0000 7f31be9e9740 1 client.15915 shutdown 2024-09-21T04:05:39.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.746+0000 7f31be9e9740 1 -- 172.21.15.64:0/3773325586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56418beebcd0 msgr2=0x56418c2e8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:39.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.746+0000 7f31be9e9740 1 --2- 172.21.15.64:0/3773325586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56418beebcd0 0x56418c2e8630 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f31ac005e00 tx=0x7f31ac0096d0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.746+0000 7f31be9e9740 1 -- 172.21.15.64:0/3773325586 shutdown_connections 2024-09-21T04:05:39.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.746+0000 7f31be9e9740 1 --2- 172.21.15.64:0/3773325586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56418bf0c670 0x56418c2e5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.746+0000 7f31be9e9740 1 --2- 172.21.15.64:0/3773325586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56418bf04490 0x56418c2e4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.746+0000 7f31be9e9740 1 --2- 172.21.15.64:0/3773325586 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56418bf2d350 0x56418c364610 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.746+0000 7f31be9e9740 1 --2- 172.21.15.64:0/3773325586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56418beebcd0 0x56418c2e8630 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.746+0000 7f31be9e9740 1 -- 172.21.15.64:0/3773325586 >> 172.21.15.64:0/3773325586 conn(0x56418bf21740 msgr2=0x56418c2daf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:39.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.746+0000 7f3199ffb640 0 client.15915 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:39.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.746+0000 7f31be9e9740 1 -- 172.21.15.64:0/3773325586 shutdown_connections 2024-09-21T04:05:39.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.746+0000 7f31be9e9740 1 -- 172.21.15.64:0/3773325586 wait complete. 2024-09-21T04:05:39.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.746+0000 7f31be9e9740 20 client.15915 trim_cache size 0 max 16384 2024-09-21T04:05:39.752 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:05:39.752 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-21T04:05:39.752 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "e9242f88-6599-40eb-adf1-bdb3e946a1e0", 2024-09-21T04:05:39.752 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-21T04:05:39.752 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:05:39.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.747+0000 7f31b8475640 1 -- 172.21.15.64:0/1411005340 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f319c0b6680 msgr2=0x7f319c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:39.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.747+0000 7f31b8475640 1 --2- 172.21.15.64:0/1411005340 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f319c0b6680 0x7f319c0b8b90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f31a8009fd0 tx=0x7f31a8004a40 comp rx=0 tx=0).stop 2024-09-21T04:05:39.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.747+0000 7f31b8475640 1 -- 172.21.15.64:0/1411005340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31b01083c0 msgr2=0x7f31b019aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:39.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.747+0000 7f31b8475640 1 --2- 172.21.15.64:0/1411005340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31b01083c0 0x7f31b019aa60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f31a0032fd0 tx=0x7f31a0031a20 comp rx=0 tx=0).stop 2024-09-21T04:05:39.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.747+0000 7f31b8475640 1 -- 172.21.15.64:0/1411005340 shutdown_connections 2024-09-21T04:05:39.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.747+0000 7f31b8475640 1 --2- 172.21.15.64:0/1411005340 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f319c0b6680 0x7f319c0b8b90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.747+0000 7f31b8475640 1 --2- 172.21.15.64:0/1411005340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31b0108d10 0x7f31b019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.747+0000 7f31b8475640 1 --2- 172.21.15.64:0/1411005340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f31b01083c0 0x7f31b019aa60 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.747+0000 7f31b8475640 1 --2- 172.21.15.64:0/1411005340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31b0102a80 0x7f31b019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:39.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.747+0000 7f31b8475640 1 -- 172.21.15.64:0/1411005340 >> 172.21.15.64:0/1411005340 conn(0x7f31b0079740 msgr2=0x7f31b006bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:39.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.747+0000 7f31b8475640 1 -- 172.21.15.64:0/1411005340 shutdown_connections 2024-09-21T04:05:39.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:39.748+0000 7f31b8475640 1 -- 172.21.15.64:0/1411005340 wait complete. 2024-09-21T04:05:40.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:39 smithi064 ceph-mon[31180]: pgmap v370: 97 pgs: 97 active+clean; 26 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 1.9 MiB/s wr, 82 op/s 2024-09-21T04:05:40.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:39 smithi131 ceph-mon[35193]: pgmap v370: 97 pgs: 97 active+clean; 26 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 1.9 MiB/s wr, 82 op/s 2024-09-21T04:05:40.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:39 smithi119 ceph-mon[34811]: pgmap v370: 97 pgs: 97 active+clean; 26 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 1.9 MiB/s wr, 82 op/s 2024-09-21T04:05:40.511 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T04:05:40.782 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:05:41.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:40 smithi131 ceph-mon[35193]: scrub queued for path: / 2024-09-21T04:05:41.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:40 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:05:41.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:40 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:05:41.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:40 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:05:41.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:40 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:05:41.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:40 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:05:41.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:40 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:05:41.067 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:40 smithi064 ceph-mon[31180]: scrub queued for path: / 2024-09-21T04:05:41.067 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:40 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:05:41.067 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:40 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:05:41.068 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:40 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:05:41.068 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:40 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:05:41.068 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:40 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:05:41.068 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:40 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:05:41.171 INFO:tasks.workunit.client.0.smithi064.stdout: 16 93382 1005 67667 898 15671 501 2024-09-21T04:05:41.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:40 smithi119 ceph-mon[34811]: scrub queued for path: / 2024-09-21T04:05:41.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:40 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:05:41.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:40 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:05:41.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:40 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:05:41.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:40 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:05:41.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:40 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:05:41.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:40 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:05:41.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.281+0000 7f259e75f640 1 -- 172.21.15.64:0/3941174610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2598102a60 msgr2=0x7f2598102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:41.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.281+0000 7f259e75f640 1 --2- 172.21.15.64:0/3941174610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2598102a60 0x7f2598102e70 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2588009900 tx=0x7f258802a410 comp rx=0 tx=0).stop 2024-09-21T04:05:41.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.281+0000 7f259e75f640 1 -- 172.21.15.64:0/3941174610 shutdown_connections 2024-09-21T04:05:41.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.281+0000 7f259e75f640 1 --2- 172.21.15.64:0/3941174610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2598108cf0 0x7f259810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.281+0000 7f259e75f640 1 --2- 172.21.15.64:0/3941174610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25981083a0 0x7f25981087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.281+0000 7f259e75f640 1 --2- 172.21.15.64:0/3941174610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2598102a60 0x7f2598102e70 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.281+0000 7f259e75f640 1 -- 172.21.15.64:0/3941174610 >> 172.21.15.64:0/3941174610 conn(0x7f2598079730 msgr2=0x7f2598079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:41.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.281+0000 7f259e75f640 1 -- 172.21.15.64:0/3941174610 shutdown_connections 2024-09-21T04:05:41.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.281+0000 7f259e75f640 1 -- 172.21.15.64:0/3941174610 wait complete. 2024-09-21T04:05:41.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.282+0000 7f259e75f640 1 Processor -- start 2024-09-21T04:05:41.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.282+0000 7f259e75f640 1 -- start start 2024-09-21T04:05:41.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.282+0000 7f259e75f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2598102a60 0x7f259819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:41.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.282+0000 7f259e75f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25981083a0 0x7f259819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:41.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.282+0000 7f259e75f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2598108cf0 0x7f259819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:41.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.282+0000 7f259e75f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2598112960 con 0x7f25981083a0 2024-09-21T04:05:41.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.282+0000 7f259e75f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f25981127e0 con 0x7f2598108cf0 2024-09-21T04:05:41.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.282+0000 7f259e75f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2598112ae0 con 0x7f2598102a60 2024-09-21T04:05:41.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.282+0000 7f258ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25981083a0 0x7f259819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:41.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.283+0000 7f258ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25981083a0 0x7f259819a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36170/0 (socket says 172.21.15.64:36170) 2024-09-21T04:05:41.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.283+0000 7f2597fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2598102a60 0x7f259819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:41.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.283+0000 7f258ffff640 1 -- 172.21.15.64:0/2133553224 learned_addr learned my addr 172.21.15.64:0/2133553224 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:41.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.283+0000 7f259ccfc640 1 --2- 172.21.15.64:0/2133553224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2598108cf0 0x7f259819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:41.286 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.283+0000 7f258ffff640 1 -- 172.21.15.64:0/2133553224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2598102a60 msgr2=0x7f259819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:41.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.283+0000 7f258ffff640 1 --2- 172.21.15.64:0/2133553224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2598102a60 0x7f259819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.283+0000 7f258ffff640 1 -- 172.21.15.64:0/2133553224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2598108cf0 msgr2=0x7f259819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:41.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.283+0000 7f258ffff640 1 --2- 172.21.15.64:0/2133553224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2598108cf0 0x7f259819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.283+0000 7f258ffff640 1 -- 172.21.15.64:0/2133553224 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25981a03b0 con 0x7f25981083a0 2024-09-21T04:05:41.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.283+0000 7f2597fff640 1 --2- 172.21.15.64:0/2133553224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2598102a60 0x7f259819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:05:41.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.283+0000 7f258ffff640 1 --2- 172.21.15.64:0/2133553224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25981083a0 0x7f259819a9e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f258000e9c0 tx=0x7f258000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:41.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.283+0000 7f2595ffb640 1 -- 172.21.15.64:0/2133553224 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258000ccc0 con 0x7f25981083a0 2024-09-21T04:05:41.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.284+0000 7f2595ffb640 1 -- 172.21.15.64:0/2133553224 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25800054d0 con 0x7f25981083a0 2024-09-21T04:05:41.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.284+0000 7f259e75f640 1 -- 172.21.15.64:0/2133553224 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25981a06a0 con 0x7f25981083a0 2024-09-21T04:05:41.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.284+0000 7f2595ffb640 1 -- 172.21.15.64:0/2133553224 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580010430 con 0x7f25981083a0 2024-09-21T04:05:41.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.284+0000 7f259e75f640 1 -- 172.21.15.64:0/2133553224 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25981a6e00 con 0x7f25981083a0 2024-09-21T04:05:41.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.286+0000 7f25a4cd3740 1 Processor -- start 2024-09-21T04:05:41.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.287+0000 7f25a4cd3740 1 -- start start 2024-09-21T04:05:41.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.287+0000 7f25a4cd3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d067ce070 0x563d06b75270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:41.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.287+0000 7f2595ffb640 1 -- 172.21.15.64:0/2133553224 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f2580004f20 con 0x7f25981083a0 2024-09-21T04:05:41.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.287+0000 7f25a4cd3740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d067a51b0 0x563d06b757b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:41.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.287+0000 7f25a4cd3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d067ad390 0x563d06b75cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:41.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.287+0000 7f25a4cd3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563d06aebe50 con 0x563d067ce070 2024-09-21T04:05:41.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.287+0000 7f25a4cd3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563d06adb280 con 0x563d067ad390 2024-09-21T04:05:41.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.287+0000 7f25a4cd3740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x563d06adf090 con 0x563d067a51b0 2024-09-21T04:05:41.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.287+0000 7f258ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d067ad390 0x563d06b75cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:41.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.287+0000 7f258ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d067ad390 0x563d06b75cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:39700/0 (socket says 172.21.15.64:39700) 2024-09-21T04:05:41.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.287+0000 7f2597fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d067a51b0 0x563d06b757b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:41.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.287+0000 7f258ffff640 1 -- 172.21.15.64:0/2980844852 learned_addr learned my addr 172.21.15.64:0/2980844852 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:41.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.287+0000 7f259ccfc640 1 --2- 172.21.15.64:0/2980844852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d067ce070 0x563d06b75270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:41.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.288+0000 7f2597fff640 1 -- 172.21.15.64:0/2980844852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d067ad390 msgr2=0x563d06b75cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:41.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.288+0000 7f2597fff640 1 --2- 172.21.15.64:0/2980844852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d067ad390 0x563d06b75cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.288+0000 7f2597fff640 1 -- 172.21.15.64:0/2980844852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d067ce070 msgr2=0x563d06b75270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:41.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.288+0000 7f2597fff640 1 --2- 172.21.15.64:0/2980844852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d067ce070 0x563d06b75270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.288+0000 7f2597fff640 1 -- 172.21.15.64:0/2980844852 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d06839650 con 0x563d067a51b0 2024-09-21T04:05:41.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.288+0000 7f259ccfc640 1 --2- 172.21.15.64:0/2980844852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d067ce070 0x563d06b75270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:05:41.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.288+0000 7f2597fff640 1 --2- 172.21.15.64:0/2980844852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d067a51b0 0x563d06b757b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2588009230 tx=0x7f2588005e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60c99961a3c4e635 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:41.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.288+0000 7f258e7fc640 1 -- 172.21.15.64:0/2980844852 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2588040070 con 0x563d067a51b0 2024-09-21T04:05:41.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.288+0000 7f258e7fc640 1 -- 172.21.15.64:0/2980844852 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2588031720 con 0x563d067a51b0 2024-09-21T04:05:41.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.288+0000 7f258e7fc640 1 -- 172.21.15.64:0/2980844852 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2588035040 con 0x563d067a51b0 2024-09-21T04:05:41.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.289+0000 7f2595ffb640 1 --2- 172.21.15.64:0/2133553224 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f25840b6700 0x7f25840b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:41.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.289+0000 7f259ccfc640 1 --2- 172.21.15.64:0/2133553224 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f25840b6700 0x7f25840b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:41.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.289+0000 7f2595ffb640 1 -- 172.21.15.64:0/2133553224 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f25800e7cb0 con 0x7f25981083a0 2024-09-21T04:05:41.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.289+0000 7f25a4cd3740 1 -- 172.21.15.64:0/2980844852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d067a51b0 msgr2=0x563d06b757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:41.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.289+0000 7f25a4cd3740 1 --2- 172.21.15.64:0/2980844852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d067a51b0 0x563d06b757b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2588009230 tx=0x7f2588005e20 comp rx=0 tx=0).stop 2024-09-21T04:05:41.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.289+0000 7f25a4cd3740 1 -- 172.21.15.64:0/2980844852 shutdown_connections 2024-09-21T04:05:41.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.289+0000 7f25a4cd3740 1 --2- 172.21.15.64:0/2980844852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d067ad390 0x563d06b75cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.289+0000 7f25a4cd3740 1 --2- 172.21.15.64:0/2980844852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d067a51b0 0x563d06b757b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.289+0000 7f25a4cd3740 1 --2- 172.21.15.64:0/2980844852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d067ce070 0x563d06b75270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.289+0000 7f25a4cd3740 1 -- 172.21.15.64:0/2980844852 >> 172.21.15.64:0/2980844852 conn(0x563d067c2460 msgr2=0x563d06b73160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:41.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.290+0000 7f259ccfc640 1 --2- 172.21.15.64:0/2133553224 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f25840b6700 0x7f25840b8c10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2590005fd0 tx=0x7f2590009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:41.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.290+0000 7f25a4cd3740 1 -- 172.21.15.64:0/2980844852 shutdown_connections 2024-09-21T04:05:41.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.290+0000 7f25a4cd3740 1 -- 172.21.15.64:0/2980844852 wait complete. 2024-09-21T04:05:41.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.290+0000 7f25a4cd3740 1 -- start start 2024-09-21T04:05:41.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.290+0000 7f25a4cd3740 1 Processor -- start 2024-09-21T04:05:41.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.291+0000 7f25a4cd3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d067a51b0 0x563d06b80890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:41.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.291+0000 7f25a4cd3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d067ad390 0x563d06b80dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:41.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.291+0000 7f25a4cd3740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d067ce070 0x563d06b81330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:41.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.291+0000 7f25a4cd3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563d0693c700 con 0x563d067a51b0 2024-09-21T04:05:41.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.291+0000 7f25a4cd3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563d06a76110 con 0x563d067ad390 2024-09-21T04:05:41.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.291+0000 7f25a4cd3740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x563d06b839f0 con 0x563d067ce070 2024-09-21T04:05:41.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.291+0000 7f25a4cd3740 10 client.0 _ll_register_callbacks cb 0x563d06a8f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:05:41.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.291+0000 7f2597fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d067a51b0 0x563d06b80890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:41.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.291+0000 7f2597fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d067a51b0 0x563d06b80890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36178/0 (socket says 172.21.15.64:36178) 2024-09-21T04:05:41.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.291+0000 7f2597fff640 1 -- 172.21.15.64:0/1019595826 learned_addr learned my addr 172.21.15.64:0/1019595826 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:41.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.291+0000 7f258ffff640 1 --2- 172.21.15.64:0/1019595826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d067ad390 0x563d06b80dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:41.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.291+0000 7f259ccfc640 1 --2- 172.21.15.64:0/1019595826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d067ce070 0x563d06b81330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:41.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.291+0000 7f258ffff640 1 -- 172.21.15.64:0/1019595826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d067ce070 msgr2=0x563d06b81330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:41.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.291+0000 7f258ffff640 1 --2- 172.21.15.64:0/1019595826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d067ce070 0x563d06b81330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.291+0000 7f258ffff640 1 -- 172.21.15.64:0/1019595826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d067a51b0 msgr2=0x563d06b80890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:41.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.291+0000 7f258ffff640 1 --2- 172.21.15.64:0/1019595826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d067a51b0 0x563d06b80890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.291+0000 7f258ffff640 1 -- 172.21.15.64:0/1019595826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d06aa3c40 con 0x563d067ad390 2024-09-21T04:05:41.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.291+0000 7f259ccfc640 1 --2- 172.21.15.64:0/1019595826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d067ce070 0x563d06b81330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:05:41.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.292+0000 7f258ffff640 1 --2- 172.21.15.64:0/1019595826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d067ad390 0x563d06b80dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f25800aad40 tx=0x7f2580005440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f598825808e45cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:41.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.292+0000 7f25777fe640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:05:41.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.292+0000 7f25777fe640 1 -- 172.21.15.64:0/1019595826 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25800aa120 con 0x563d067ad390 2024-09-21T04:05:41.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.292+0000 7f25777fe640 1 -- 172.21.15.64:0/1019595826 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f25800acd30 con 0x563d067ad390 2024-09-21T04:05:41.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.292+0000 7f25a4cd3740 1 -- 172.21.15.64:0/1019595826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2588031720 con 0x563d067ad390 2024-09-21T04:05:41.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.292+0000 7f25777fe640 1 -- 172.21.15.64:0/1019595826 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f25800a9240 con 0x563d067ad390 2024-09-21T04:05:41.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.292+0000 7f25a4cd3740 10 client.25637 fetch_fsmap learned FSMap version 29 2024-09-21T04:05:41.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.292+0000 7f25a4cd3740 1 -- 172.21.15.64:0/1019595826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563d06b8f6d0 con 0x563d067ad390 2024-09-21T04:05:41.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.292+0000 7f25777fe640 1 -- 172.21.15.64:0/1019595826 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2588031720 con 0x563d067ad390 2024-09-21T04:05:41.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.292+0000 7f25777fe640 1 -- 172.21.15.64:0/1019595826 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f25800b20c0 con 0x563d067ad390 2024-09-21T04:05:41.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.293+0000 7f25a4cd3740 10 client.25637 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T04:05:41.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.293+0000 7f25a4cd3740 10 client.25637 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:05:41.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.293+0000 7f25a4cd3740 20 client.25637 populate_metadata read hostname 'smithi064' 2024-09-21T04:05:41.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.293+0000 7f25a4cd3740 1 --2- 172.21.15.64:0/1019595826 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x563d0678c9f0 0x563d06b92c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:41.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.293+0000 7f25a4cd3740 4 client.25637 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:05:41.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.293+0000 7f25a4cd3740 1 -- 172.21.15.64:0/1019595826 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d06aedda0 con 0x563d0678c9f0 2024-09-21T04:05:41.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.293+0000 7f2597fff640 1 --2- 172.21.15.64:0/1019595826 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x563d0678c9f0 0x563d06b92c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:41.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.293+0000 7f25777fe640 10 client.25637 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:41.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.293+0000 7f2597fff640 1 --2- 172.21.15.64:0/1019595826 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x563d0678c9f0 0x563d06b92c00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7a8468a92bffcb3 server_cookie=3cf7619136706a65 in_seq=0 out_seq=0 2024-09-21T04:05:41.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.294+0000 7f25777fe640 1 -- 172.21.15.64:0/1019595826 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563d06aedda0 con 0x563d0678c9f0 2024-09-21T04:05:41.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.294+0000 7f25777fe640 10 client.25637 handle_command_reply: tid=0 2024-09-21T04:05:41.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.294+0000 7f25a4cd3740 1 client.25637 shutdown 2024-09-21T04:05:41.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.294+0000 7f25a4cd3740 1 -- 172.21.15.64:0/1019595826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d067ad390 msgr2=0x563d06b80dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:41.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.294+0000 7f25a4cd3740 1 --2- 172.21.15.64:0/1019595826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d067ad390 0x563d06b80dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f25800aad40 tx=0x7f2580005440 comp rx=0 tx=0).stop 2024-09-21T04:05:41.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.294+0000 7f25a4cd3740 1 -- 172.21.15.64:0/1019595826 shutdown_connections 2024-09-21T04:05:41.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.294+0000 7f25a4cd3740 1 --2- 172.21.15.64:0/1019595826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d067ce070 0x563d06b81330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.294+0000 7f25a4cd3740 1 --2- 172.21.15.64:0/1019595826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d067ad390 0x563d06b80dd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.294+0000 7f25a4cd3740 1 --2- 172.21.15.64:0/1019595826 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x563d0678c9f0 0x563d06b92c00 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.294+0000 7f25a4cd3740 1 --2- 172.21.15.64:0/1019595826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d067a51b0 0x563d06b80890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.294+0000 7f25777fe640 0 client.25637 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:41.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.294+0000 7f25a4cd3740 1 -- 172.21.15.64:0/1019595826 >> 172.21.15.64:0/1019595826 conn(0x563d067c2460 msgr2=0x563d06b78450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:41.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.295+0000 7f25a4cd3740 1 -- 172.21.15.64:0/1019595826 shutdown_connections 2024-09-21T04:05:41.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.295+0000 7f25a4cd3740 1 -- 172.21.15.64:0/1019595826 wait complete. 2024-09-21T04:05:41.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.295+0000 7f25a4cd3740 20 client.25637 trim_cache size 0 max 16384 2024-09-21T04:05:41.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.313+0000 7f25a4cd3740 1 Processor -- start 2024-09-21T04:05:41.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.314+0000 7f25a4cd3740 1 -- start start 2024-09-21T04:05:41.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.314+0000 7f25a4cd3740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d067a51b0 0x563d06b769b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:41.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.314+0000 7f25a4cd3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d067ad390 0x563d06b7da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:41.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.314+0000 7f25a4cd3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d067ce070 0x563d06b7dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:41.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.314+0000 7f25a4cd3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f25981aff10 con 0x563d067ad390 2024-09-21T04:05:41.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.314+0000 7f25a4cd3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f25981ad780 con 0x563d067ce070 2024-09-21T04:05:41.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.314+0000 7f25a4cd3740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f25981ab5d0 con 0x563d067a51b0 2024-09-21T04:05:41.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.314+0000 7f258ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d067ad390 0x563d06b7da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:41.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.314+0000 7f258ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d067ad390 0x563d06b7da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36190/0 (socket says 172.21.15.64:36190) 2024-09-21T04:05:41.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.314+0000 7f258ffff640 1 -- 172.21.15.64:0/171388745 learned_addr learned my addr 172.21.15.64:0/171388745 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:41.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.314+0000 7f259ccfc640 1 --2- 172.21.15.64:0/171388745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d067ce070 0x563d06b7dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:41.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.314+0000 7f2597fff640 1 --2- 172.21.15.64:0/171388745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d067a51b0 0x563d06b769b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:41.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.314+0000 7f258ffff640 1 -- 172.21.15.64:0/171388745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d067a51b0 msgr2=0x563d06b769b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:41.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.314+0000 7f258ffff640 1 --2- 172.21.15.64:0/171388745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d067a51b0 0x563d06b769b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.314+0000 7f258ffff640 1 -- 172.21.15.64:0/171388745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d067ce070 msgr2=0x563d06b7dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:41.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.314+0000 7f258ffff640 1 --2- 172.21.15.64:0/171388745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d067ce070 0x563d06b7dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.314+0000 7f258ffff640 1 -- 172.21.15.64:0/171388745 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d06b7f7f0 con 0x563d067ad390 2024-09-21T04:05:41.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.315+0000 7f258ffff640 1 --2- 172.21.15.64:0/171388745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d067ad390 0x563d06b7da80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f25800e59d0 tx=0x7f2580004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc27f0f213f0fc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:41.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.315+0000 7f258dffb640 1 -- 172.21.15.64:0/171388745 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258000ce60 con 0x563d067ad390 2024-09-21T04:05:41.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.315+0000 7f258dffb640 1 -- 172.21.15.64:0/171388745 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25800d5080 con 0x563d067ad390 2024-09-21T04:05:41.319 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.315+0000 7f258dffb640 1 -- 172.21.15.64:0/171388745 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25800ad240 con 0x563d067ad390 2024-09-21T04:05:41.321 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.316+0000 7f25a4cd3740 1 -- 172.21.15.64:0/171388745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d067ad390 msgr2=0x563d06b7da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:41.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.316+0000 7f25a4cd3740 1 --2- 172.21.15.64:0/171388745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d067ad390 0x563d06b7da80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f25800e59d0 tx=0x7f2580004780 comp rx=0 tx=0).stop 2024-09-21T04:05:41.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.316+0000 7f25a4cd3740 1 -- 172.21.15.64:0/171388745 shutdown_connections 2024-09-21T04:05:41.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.316+0000 7f25a4cd3740 1 --2- 172.21.15.64:0/171388745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d067ce070 0x563d06b7dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.316+0000 7f25a4cd3740 1 --2- 172.21.15.64:0/171388745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d067ad390 0x563d06b7da80 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.316+0000 7f25a4cd3740 1 --2- 172.21.15.64:0/171388745 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d067a51b0 0x563d06b769b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.316+0000 7f25a4cd3740 1 -- 172.21.15.64:0/171388745 >> 172.21.15.64:0/171388745 conn(0x563d067c2460 msgr2=0x563d06b762a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:41.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.316+0000 7f25a4cd3740 1 -- 172.21.15.64:0/171388745 shutdown_connections 2024-09-21T04:05:41.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.316+0000 7f25a4cd3740 1 -- 172.21.15.64:0/171388745 wait complete. 2024-09-21T04:05:41.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.316+0000 7f25a4cd3740 1 -- start start 2024-09-21T04:05:41.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.317+0000 7f25a4cd3740 1 Processor -- start 2024-09-21T04:05:41.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.317+0000 7f25a4cd3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d067a51b0 0x563d06b84cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:41.322 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.317+0000 7f25a4cd3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d067ad390 0x563d06b85210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:41.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.317+0000 7f25a4cd3740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d067ce070 0x563d06b85750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:41.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.317+0000 7f25a4cd3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f25981abc40 con 0x563d067ad390 2024-09-21T04:05:41.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.317+0000 7f25a4cd3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f25981ad0b0 con 0x563d067a51b0 2024-09-21T04:05:41.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.317+0000 7f25a4cd3740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f25981ac310 con 0x563d067ce070 2024-09-21T04:05:41.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.317+0000 7f25a4cd3740 10 client.0 _ll_register_callbacks cb 0x7f2580004840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:05:41.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.317+0000 7f2597fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d067a51b0 0x563d06b84cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:41.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.317+0000 7f2597fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d067a51b0 0x563d06b84cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:39726/0 (socket says 172.21.15.64:39726) 2024-09-21T04:05:41.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.317+0000 7f2597fff640 1 -- 172.21.15.64:0/280723509 learned_addr learned my addr 172.21.15.64:0/280723509 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:41.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.317+0000 7f259ccfc640 1 --2- 172.21.15.64:0/280723509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d067ce070 0x563d06b85750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:41.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.317+0000 7f258ffff640 1 --2- 172.21.15.64:0/280723509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d067ad390 0x563d06b85210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:41.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.317+0000 7f2597fff640 1 -- 172.21.15.64:0/280723509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d067ce070 msgr2=0x563d06b85750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:41.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.317+0000 7f2597fff640 1 --2- 172.21.15.64:0/280723509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d067ce070 0x563d06b85750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.317+0000 7f2597fff640 1 -- 172.21.15.64:0/280723509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d067ad390 msgr2=0x563d06b85210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:41.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.317+0000 7f2597fff640 1 --2- 172.21.15.64:0/280723509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d067ad390 0x563d06b85210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.318+0000 7f2597fff640 1 -- 172.21.15.64:0/280723509 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d06bfc670 con 0x563d067a51b0 2024-09-21T04:05:41.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.318+0000 7f2597fff640 1 --2- 172.21.15.64:0/280723509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d067a51b0 0x563d06b84cd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2588033000 tx=0x7f2588005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d025b78af7d6952d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:41.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.318+0000 7f25777fe640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:05:41.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.318+0000 7f25777fe640 1 -- 172.21.15.64:0/280723509 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258805e060 con 0x563d067a51b0 2024-09-21T04:05:41.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.318+0000 7f25a4cd3740 1 -- 172.21.15.64:0/280723509 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563d06ae9850 con 0x563d067a51b0 2024-09-21T04:05:41.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.318+0000 7f25777fe640 1 -- 172.21.15.64:0/280723509 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f2588004040 con 0x563d067a51b0 2024-09-21T04:05:41.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.318+0000 7f25777fe640 1 -- 172.21.15.64:0/280723509 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2588035040 con 0x563d067a51b0 2024-09-21T04:05:41.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.318+0000 7f25a4cd3740 10 client.25643 fetch_fsmap learned FSMap version 29 2024-09-21T04:05:41.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.318+0000 7f25a4cd3740 1 -- 172.21.15.64:0/280723509 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563d06bfd960 con 0x563d067a51b0 2024-09-21T04:05:41.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.319+0000 7f25777fe640 1 -- 172.21.15.64:0/280723509 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563d06ae9850 con 0x563d067a51b0 2024-09-21T04:05:41.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.319+0000 7f25777fe640 1 -- 172.21.15.64:0/280723509 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f25880035f0 con 0x563d067a51b0 2024-09-21T04:05:41.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.319+0000 7f25a4cd3740 10 client.25643 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T04:05:41.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.319+0000 7f25a4cd3740 10 client.25643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:05:41.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.319+0000 7f25a4cd3740 20 client.25643 populate_metadata read hostname 'smithi064' 2024-09-21T04:05:41.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.319+0000 7f25a4cd3740 1 --2- 172.21.15.64:0/280723509 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x563d06761da0 0x563d06bffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:41.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.319+0000 7f25a4cd3740 4 client.25643 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T04:05:41.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.319+0000 7f25a4cd3740 1 -- 172.21.15.64:0/280723509 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563d06ae6c80 con 0x563d06761da0 2024-09-21T04:05:41.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.319+0000 7f258ffff640 1 --2- 172.21.15.64:0/280723509 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x563d06761da0 0x563d06bffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:41.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.320+0000 7f258ffff640 1 --2- 172.21.15.64:0/280723509 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x563d06761da0 0x563d06bffcd0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5918bdd4ea4c408f server_cookie=c10aa857c13cc64b in_seq=0 out_seq=0 2024-09-21T04:05:41.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.320+0000 7f25777fe640 10 client.25643 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:41.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.320+0000 7f25777fe640 1 -- 172.21.15.64:0/280723509 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1422 (crc 0 0 0) 0x563d06ae6c80 con 0x563d06761da0 2024-09-21T04:05:41.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.320+0000 7f25777fe640 10 client.25643 handle_command_reply: tid=0 2024-09-21T04:05:41.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.320+0000 7f25a4cd3740 1 client.25643 shutdown 2024-09-21T04:05:41.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.320+0000 7f25a4cd3740 1 -- 172.21.15.64:0/280723509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d067a51b0 msgr2=0x563d06b84cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:41.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.320+0000 7f25a4cd3740 1 --2- 172.21.15.64:0/280723509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d067a51b0 0x563d06b84cd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2588033000 tx=0x7f2588005fd0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.320+0000 7f25a4cd3740 1 -- 172.21.15.64:0/280723509 shutdown_connections 2024-09-21T04:05:41.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.320+0000 7f25a4cd3740 1 --2- 172.21.15.64:0/280723509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563d067ce070 0x563d06b85750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.320+0000 7f25a4cd3740 1 --2- 172.21.15.64:0/280723509 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x563d06761da0 0x563d06bffcd0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.320+0000 7f25a4cd3740 1 --2- 172.21.15.64:0/280723509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563d067ad390 0x563d06b85210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.320+0000 7f25a4cd3740 1 --2- 172.21.15.64:0/280723509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d067a51b0 0x563d06b84cd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.320+0000 7f25a4cd3740 1 -- 172.21.15.64:0/280723509 >> 172.21.15.64:0/280723509 conn(0x563d067c2460 msgr2=0x563d06b762a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:41.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.320+0000 7f25777fe640 0 client.25643 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:41.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.320+0000 7f25a4cd3740 1 -- 172.21.15.64:0/280723509 shutdown_connections 2024-09-21T04:05:41.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.320+0000 7f25a4cd3740 1 -- 172.21.15.64:0/280723509 wait complete. 2024-09-21T04:05:41.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.320+0000 7f25a4cd3740 20 client.25643 trim_cache size 0 max 16384 2024-09-21T04:05:41.327 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:05:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T04:05:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-09-21T04:05:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "e9242f88-6599-40eb-adf1-bdb3e946a1e0": { 2024-09-21T04:05:41.327 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:05:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "e9242f88-6599-40eb-adf1-bdb3e946a1e0", 2024-09-21T04:05:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-09-21T04:05:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:05:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:05:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_stats": { 2024-09-21T04:05:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "e9242f88-6599-40eb-adf1-bdb3e946a1e0": { 2024-09-21T04:05:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:05:39.744646+0000", 2024-09-21T04:05:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:05:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:05:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:05:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:05:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:05:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:05:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "b52b950c-0f97-49ec-a37c-1a8e04348ce8": { 2024-09-21T04:05:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:54.003800+0000", 2024-09-21T04:05:41.328 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:05:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:05:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:05:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:05:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:05:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:05:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: "0ab717a7-36c6-4dca-94e7-af0bdf2aa112": { 2024-09-21T04:05:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:42.691086+0000", 2024-09-21T04:05:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:05:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:05:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:05:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:05:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:05:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:05:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:05:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:03:59.570174+0000", 2024-09-21T04:05:41.329 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:05:41.330 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:05:41.330 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:05:41.330 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:05:41.330 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:05:41.330 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:05:41.330 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:05:41.330 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:05:41.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.321+0000 7f259e75f640 1 -- 172.21.15.64:0/2133553224 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f25840b6700 msgr2=0x7f25840b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:41.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.321+0000 7f259e75f640 1 --2- 172.21.15.64:0/2133553224 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f25840b6700 0x7f25840b8c10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2590005fd0 tx=0x7f2590009500 comp rx=0 tx=0).stop 2024-09-21T04:05:41.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.321+0000 7f259e75f640 1 -- 172.21.15.64:0/2133553224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25981083a0 msgr2=0x7f259819a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:41.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.321+0000 7f259e75f640 1 --2- 172.21.15.64:0/2133553224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25981083a0 0x7f259819a9e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f258000e9c0 tx=0x7f258000edc0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.321+0000 7f259e75f640 1 -- 172.21.15.64:0/2133553224 shutdown_connections 2024-09-21T04:05:41.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.321+0000 7f259e75f640 1 --2- 172.21.15.64:0/2133553224 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f25840b6700 0x7f25840b8c10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.321+0000 7f259e75f640 1 --2- 172.21.15.64:0/2133553224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2598108cf0 0x7f259819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.321+0000 7f259e75f640 1 --2- 172.21.15.64:0/2133553224 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25981083a0 0x7f259819a9e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.321+0000 7f259e75f640 1 --2- 172.21.15.64:0/2133553224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2598102a60 0x7f259819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:41.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.321+0000 7f259e75f640 1 -- 172.21.15.64:0/2133553224 >> 172.21.15.64:0/2133553224 conn(0x7f2598079730 msgr2=0x7f2598101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:41.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.321+0000 7f259e75f640 1 -- 172.21.15.64:0/2133553224 shutdown_connections 2024-09-21T04:05:41.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:05:41.321+0000 7f259e75f640 1 -- 172.21.15.64:0/2133553224 wait complete. 2024-09-21T04:05:41.993 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:41 smithi064 ceph-mon[31180]: pgmap v371: 97 pgs: 97 active+clean; 29 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 2.1 MiB/s wr, 88 op/s 2024-09-21T04:05:42.023 INFO:tasks.cephfs.filesystem:scrub status for tag:e9242f88-6599-40eb-adf1-bdb3e946a1e0 - {'path': '/', 'tag': 'e9242f88-6599-40eb-adf1-bdb3e946a1e0', 'options': 'recursive,force'} 2024-09-21T04:05:42.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:41 smithi119 ceph-mon[34811]: pgmap v371: 97 pgs: 97 active+clean; 29 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 2.1 MiB/s wr, 88 op/s 2024-09-21T04:05:42.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:41 smithi131 ceph-mon[35193]: pgmap v371: 97 pgs: 97 active+clean; 29 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 2.1 MiB/s wr, 88 op/s 2024-09-21T04:05:43.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:42 smithi119 ceph-mon[34811]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:05:43.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:42 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:05:43.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:42 smithi064 ceph-mon[31180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:05:43.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:42 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:05:43.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:42 smithi131 ceph-mon[35193]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:05:43.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:42 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:05:44.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:43 smithi119 ceph-mon[34811]: pgmap v372: 97 pgs: 97 active+clean; 65 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 259 op/s 2024-09-21T04:05:44.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:43 smithi064 ceph-mon[31180]: pgmap v372: 97 pgs: 97 active+clean; 65 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 259 op/s 2024-09-21T04:05:44.309 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:43 smithi131 ceph-mon[35193]: pgmap v372: 97 pgs: 97 active+clean; 65 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 259 op/s 2024-09-21T04:05:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.915+0000 7f007f720640 1 -- 172.21.15.64:0/2199726688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0078109430 msgr2=0x7f0078109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.915+0000 7f007f720640 1 --2- 172.21.15.64:0/2199726688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0078109430 0x7f0078109840 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0068009900 tx=0x7f006802a410 comp rx=0 tx=0).stop 2024-09-21T04:05:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.915+0000 7f007f720640 1 -- 172.21.15.64:0/2199726688 shutdown_connections 2024-09-21T04:05:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.915+0000 7f007f720640 1 --2- 172.21.15.64:0/2199726688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0078109d80 0x7f00781105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.915+0000 7f007f720640 1 --2- 172.21.15.64:0/2199726688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0078109430 0x7f0078109840 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.915+0000 7f007f720640 1 --2- 172.21.15.64:0/2199726688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0078102060 0x7f0078102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.915+0000 7f007f720640 1 -- 172.21.15.64:0/2199726688 >> 172.21.15.64:0/2199726688 conn(0x7f0078079e90 msgr2=0x7f007807a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.915+0000 7f007f720640 1 -- 172.21.15.64:0/2199726688 shutdown_connections 2024-09-21T04:05:45.919 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.915+0000 7f007f720640 1 -- 172.21.15.64:0/2199726688 wait complete. 2024-09-21T04:05:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.916+0000 7f007f720640 1 Processor -- start 2024-09-21T04:05:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.916+0000 7f007f720640 1 -- start start 2024-09-21T04:05:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.916+0000 7f007f720640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0078102060 0x7f007819c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.916+0000 7f007f720640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0078109430 0x7f007819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.916+0000 7f007f720640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0078109d80 0x7f00781a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.916+0000 7f007f720640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0078113a40 con 0x7f0078109d80 2024-09-21T04:05:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.916+0000 7f007f720640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f00781138c0 con 0x7f0078109430 2024-09-21T04:05:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.916+0000 7f007f720640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0078113bc0 con 0x7f0078102060 2024-09-21T04:05:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.916+0000 7f007dcbd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0078109d80 0x7f00781a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.916+0000 7f007dcbd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0078109d80 0x7f00781a1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54596/0 (socket says 172.21.15.64:54596) 2024-09-21T04:05:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.916+0000 7f007dcbd640 1 -- 172.21.15.64:0/678696502 learned_addr learned my addr 172.21.15.64:0/678696502 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.916+0000 7f007ccbb640 1 --2- 172.21.15.64:0/678696502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0078109430 0x7f007819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.916+0000 7f007d4bc640 1 --2- 172.21.15.64:0/678696502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0078102060 0x7f007819c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.917+0000 7f007dcbd640 1 -- 172.21.15.64:0/678696502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0078102060 msgr2=0x7f007819c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.917+0000 7f007dcbd640 1 --2- 172.21.15.64:0/678696502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0078102060 0x7f007819c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.917+0000 7f007dcbd640 1 -- 172.21.15.64:0/678696502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0078109430 msgr2=0x7f007819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.917+0000 7f007dcbd640 1 --2- 172.21.15.64:0/678696502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0078109430 0x7f007819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.917+0000 7f007dcbd640 1 -- 172.21.15.64:0/678696502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00781a2610 con 0x7f0078109d80 2024-09-21T04:05:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.917+0000 7f007dcbd640 1 --2- 172.21.15.64:0/678696502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0078109d80 0x7f00781a1f10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f007000eb10 tx=0x7f007000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.917+0000 7f006e7fc640 1 -- 172.21.15.64:0/678696502 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0070004b20 con 0x7f0078109d80 2024-09-21T04:05:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.917+0000 7f006e7fc640 1 -- 172.21.15.64:0/678696502 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0070004cc0 con 0x7f0078109d80 2024-09-21T04:05:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.917+0000 7f007f720640 1 -- 172.21.15.64:0/678696502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00781a2900 con 0x7f0078109d80 2024-09-21T04:05:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.917+0000 7f006e7fc640 1 -- 172.21.15.64:0/678696502 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0070010670 con 0x7f0078109d80 2024-09-21T04:05:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.917+0000 7f007f720640 1 -- 172.21.15.64:0/678696502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00781a9090 con 0x7f0078109d80 2024-09-21T04:05:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.920+0000 7f008668c740 1 Processor -- start 2024-09-21T04:05:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.921+0000 7f006e7fc640 1 -- 172.21.15.64:0/678696502 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f0070005020 con 0x7f0078109d80 2024-09-21T04:05:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.921+0000 7f008668c740 1 -- start start 2024-09-21T04:05:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.921+0000 7f008668c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863a8121b0 0x55863abb79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.921+0000 7f008668c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7e92f0 0x55863abb7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.921+0000 7f008668c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55863a7f14d0 0x55863abb8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.921+0000 7f008668c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55863ab2c6f0 con 0x55863a7e92f0 2024-09-21T04:05:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.921+0000 7f008668c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55863ab1bb20 con 0x55863a7f14d0 2024-09-21T04:05:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.921+0000 7f008668c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55863ab1f930 con 0x55863a8121b0 2024-09-21T04:05:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.921+0000 7f007d4bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7e92f0 0x55863abb7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.921+0000 7f007ccbb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863a8121b0 0x55863abb79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.921+0000 7f007d4bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7e92f0 0x55863abb7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54610/0 (socket says 172.21.15.64:54610) 2024-09-21T04:05:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.921+0000 7f007d4bc640 1 -- 172.21.15.64:0/1963923076 learned_addr learned my addr 172.21.15.64:0/1963923076 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.921+0000 7f007ccbb640 1 --2- 172.21.15.64:0/1963923076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55863a7f14d0 0x55863abb8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.922+0000 7f006e7fc640 1 --2- 172.21.15.64:0/678696502 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f00640b6680 0x7f00640b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.922+0000 7f007ccbb640 1 -- 172.21.15.64:0/1963923076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55863a7f14d0 msgr2=0x55863abb8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.922+0000 7f007ccbb640 1 --2- 172.21.15.64:0/1963923076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55863a7f14d0 0x55863abb8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.922+0000 7f007ccbb640 1 -- 172.21.15.64:0/1963923076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7e92f0 msgr2=0x55863abb7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.922+0000 7f007ccbb640 1 --2- 172.21.15.64:0/1963923076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7e92f0 0x55863abb7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.922+0000 7f007ccbb640 1 -- 172.21.15.64:0/1963923076 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55863aae1530 con 0x55863a8121b0 2024-09-21T04:05:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.922+0000 7f007d4bc640 1 --2- 172.21.15.64:0/1963923076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7e92f0 0x55863abb7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:05:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.922+0000 7f007dcbd640 1 --2- 172.21.15.64:0/678696502 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f00640b6680 0x7f00640b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.922+0000 7f007ccbb640 1 --2- 172.21.15.64:0/1963923076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863a8121b0 0x55863abb79b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f00780756a0 tx=0x7f0068000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13d9dbdd387c3c8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.922+0000 7f006e7fc640 1 -- 172.21.15.64:0/678696502 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f00700e7cd0 con 0x7f0078109d80 2024-09-21T04:05:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.922+0000 7f00637fe640 1 -- 172.21.15.64:0/1963923076 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0068035030 con 0x55863a8121b0 2024-09-21T04:05:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.922+0000 7f00637fe640 1 -- 172.21.15.64:0/1963923076 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00680318a0 con 0x55863a8121b0 2024-09-21T04:05:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.922+0000 7f00637fe640 1 -- 172.21.15.64:0/1963923076 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0068004750 con 0x55863a8121b0 2024-09-21T04:05:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.923+0000 7f008668c740 1 -- 172.21.15.64:0/1963923076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863a8121b0 msgr2=0x55863abb79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.923+0000 7f008668c740 1 --2- 172.21.15.64:0/1963923076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863a8121b0 0x55863abb79b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f00780756a0 tx=0x7f0068000bc0 comp rx=0 tx=0).stop 2024-09-21T04:05:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.923+0000 7f007dcbd640 1 --2- 172.21.15.64:0/678696502 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f00640b6680 0x7f00640b8b90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f00700ad7f0 tx=0x7f00700d1040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.923+0000 7f008668c740 1 -- 172.21.15.64:0/1963923076 shutdown_connections 2024-09-21T04:05:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.923+0000 7f008668c740 1 --2- 172.21.15.64:0/1963923076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55863a7f14d0 0x55863abb8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.923+0000 7f008668c740 1 --2- 172.21.15.64:0/1963923076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7e92f0 0x55863abb7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.923+0000 7f008668c740 1 --2- 172.21.15.64:0/1963923076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863a8121b0 0x55863abb79b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.923+0000 7f008668c740 1 -- 172.21.15.64:0/1963923076 >> 172.21.15.64:0/1963923076 conn(0x55863a806300 msgr2=0x55863abb58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.923+0000 7f008668c740 1 -- 172.21.15.64:0/1963923076 shutdown_connections 2024-09-21T04:05:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.923+0000 7f008668c740 1 -- 172.21.15.64:0/1963923076 wait complete. 2024-09-21T04:05:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.924+0000 7f008668c740 1 -- start start 2024-09-21T04:05:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.924+0000 7f008668c740 1 Processor -- start 2024-09-21T04:05:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.924+0000 7f008668c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863a7e92f0 0x55863abc3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.924+0000 7f008668c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7f14d0 0x55863abc3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.924+0000 7f008668c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55863a8121b0 0x55863abcf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.924+0000 7f008668c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55863aa53ab0 con 0x55863a7f14d0 2024-09-21T04:05:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.924+0000 7f008668c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55863aac1730 con 0x55863a8121b0 2024-09-21T04:05:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.924+0000 7f008668c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55863abc6070 con 0x55863a7e92f0 2024-09-21T04:05:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.924+0000 7f008668c740 10 client.0 _ll_register_callbacks cb 0x55863ab156b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:05:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.924+0000 7f007d4bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7f14d0 0x55863abc3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.924+0000 7f007ccbb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863a7e92f0 0x55863abc3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.924+0000 7f007d4bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7f14d0 0x55863abc3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54620/0 (socket says 172.21.15.64:54620) 2024-09-21T04:05:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.924+0000 7f007ccbb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863a7e92f0 0x55863abc3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:54036/0 (socket says 172.21.15.64:54036) 2024-09-21T04:05:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.924+0000 7f007d4bc640 1 -- 172.21.15.64:0/1501136383 learned_addr learned my addr 172.21.15.64:0/1501136383 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.924+0000 7f007ccbb640 1 --2- 172.21.15.64:0/1501136383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55863a8121b0 0x55863abcf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.925+0000 7f007d4bc640 1 -- 172.21.15.64:0/1501136383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863a7e92f0 msgr2=0x55863abc3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.925+0000 7f007d4bc640 1 --2- 172.21.15.64:0/1501136383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863a7e92f0 0x55863abc3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.925+0000 7f007d4bc640 1 -- 172.21.15.64:0/1501136383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55863a8121b0 msgr2=0x55863abcf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.925+0000 7f007d4bc640 1 --2- 172.21.15.64:0/1501136383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55863a8121b0 0x55863abcf360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.925+0000 7f007d4bc640 1 -- 172.21.15.64:0/1501136383 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55863a87d790 con 0x55863a7f14d0 2024-09-21T04:05:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.925+0000 7f007d4bc640 1 --2- 172.21.15.64:0/1501136383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7f14d0 0x55863abc3550 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0074012840 tx=0x7f0074012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73cf65929b313d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.925+0000 7f0060ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:05:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.925+0000 7f0060ff9640 1 -- 172.21.15.64:0/1501136383 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0074004020 con 0x55863a7f14d0 2024-09-21T04:05:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.925+0000 7f008668c740 1 -- 172.21.15.64:0/1501136383 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00680318a0 con 0x55863a7f14d0 2024-09-21T04:05:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.925+0000 7f0060ff9640 1 -- 172.21.15.64:0/1501136383 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f00740041c0 con 0x55863a7f14d0 2024-09-21T04:05:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.925+0000 7f0060ff9640 1 -- 172.21.15.64:0/1501136383 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f00740043c0 con 0x55863a7f14d0 2024-09-21T04:05:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.925+0000 7f008668c740 10 client.15945 fetch_fsmap learned FSMap version 30 2024-09-21T04:05:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.925+0000 7f008668c740 1 -- 172.21.15.64:0/1501136383 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55863a8f6620 con 0x55863a7f14d0 2024-09-21T04:05:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.926+0000 7f0060ff9640 1 -- 172.21.15.64:0/1501136383 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00680318a0 con 0x55863a7f14d0 2024-09-21T04:05:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.926+0000 7f0060ff9640 1 -- 172.21.15.64:0/1501136383 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f0074023070 con 0x55863a7f14d0 2024-09-21T04:05:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.926+0000 7f008668c740 10 client.15945 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:05:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.926+0000 7f008668c740 10 client.15945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:05:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.926+0000 7f008668c740 20 client.15945 populate_metadata read hostname 'smithi064' 2024-09-21T04:05:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.926+0000 7f008668c740 1 --2- 172.21.15.64:0/1501136383 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55863a7d0d80 0x55863abd5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.926+0000 7f008668c740 4 client.15945 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:05:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.926+0000 7f008668c740 1 -- 172.21.15.64:0/1501136383 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55863ab2e640 con 0x55863a7d0d80 2024-09-21T04:05:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.926+0000 7f007ccbb640 1 --2- 172.21.15.64:0/1501136383 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55863a7d0d80 0x55863abd5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.926+0000 7f0060ff9640 10 client.15945 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.926+0000 7f007ccbb640 1 --2- 172.21.15.64:0/1501136383 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55863a7d0d80 0x55863abd5110 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8beb29f9c8a9f02 server_cookie=27235c97e89b3762 in_seq=0 out_seq=0 2024-09-21T04:05:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.927+0000 7f0060ff9640 1 -- 172.21.15.64:0/1501136383 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55863ab2e640 con 0x55863a7d0d80 2024-09-21T04:05:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.927+0000 7f0060ff9640 10 client.15945 handle_command_reply: tid=0 2024-09-21T04:05:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.927+0000 7f008668c740 1 client.15945 shutdown 2024-09-21T04:05:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.927+0000 7f008668c740 1 -- 172.21.15.64:0/1501136383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7f14d0 msgr2=0x55863abc3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.927+0000 7f008668c740 1 --2- 172.21.15.64:0/1501136383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7f14d0 0x55863abc3550 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0074012840 tx=0x7f0074012d00 comp rx=0 tx=0).stop 2024-09-21T04:05:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.928+0000 7f008668c740 1 -- 172.21.15.64:0/1501136383 shutdown_connections 2024-09-21T04:05:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.928+0000 7f008668c740 1 --2- 172.21.15.64:0/1501136383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55863a8121b0 0x55863abcf360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.928+0000 7f008668c740 1 --2- 172.21.15.64:0/1501136383 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55863a7d0d80 0x55863abd5110 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.928+0000 7f008668c740 1 --2- 172.21.15.64:0/1501136383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7f14d0 0x55863abc3550 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.928+0000 7f008668c740 1 --2- 172.21.15.64:0/1501136383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863a7e92f0 0x55863abc3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.928+0000 7f0060ff9640 0 client.15945 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.928+0000 7f008668c740 1 -- 172.21.15.64:0/1501136383 >> 172.21.15.64:0/1501136383 conn(0x55863a806300 msgr2=0x55863abb89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.928+0000 7f008668c740 1 -- 172.21.15.64:0/1501136383 shutdown_connections 2024-09-21T04:05:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.928+0000 7f008668c740 1 -- 172.21.15.64:0/1501136383 wait complete. 2024-09-21T04:05:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.928+0000 7f008668c740 20 client.15945 trim_cache size 0 max 16384 2024-09-21T04:05:45.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.944+0000 7f008668c740 1 Processor -- start 2024-09-21T04:05:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.944+0000 7f008668c740 1 -- start start 2024-09-21T04:05:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.944+0000 7f008668c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7e92f0 0x55863abb90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.944+0000 7f008668c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863a7f14d0 0x55863abcf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.944+0000 7f008668c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55863a8121b0 0x55863abc01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.944+0000 7f008668c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f00781b44d0 con 0x55863a7e92f0 2024-09-21T04:05:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.944+0000 7f008668c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f00781b1d10 con 0x55863a8121b0 2024-09-21T04:05:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.944+0000 7f008668c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f00781afb60 con 0x55863a7f14d0 2024-09-21T04:05:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.944+0000 7f007ccbb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7e92f0 0x55863abb90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.944+0000 7f007ccbb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7e92f0 0x55863abb90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54636/0 (socket says 172.21.15.64:54636) 2024-09-21T04:05:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.944+0000 7f007ccbb640 1 -- 172.21.15.64:0/3772972625 learned_addr learned my addr 172.21.15.64:0/3772972625 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.944+0000 7f007d4bc640 1 --2- 172.21.15.64:0/3772972625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863a7f14d0 0x55863abcf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.944+0000 7f007ccbb640 1 --2- 172.21.15.64:0/3772972625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55863a8121b0 0x55863abc01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.945+0000 7f007ccbb640 1 -- 172.21.15.64:0/3772972625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863a7f14d0 msgr2=0x55863abcf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:45.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.945+0000 7f007ccbb640 1 --2- 172.21.15.64:0/3772972625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863a7f14d0 0x55863abcf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.945+0000 7f007ccbb640 1 -- 172.21.15.64:0/3772972625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55863a8121b0 msgr2=0x55863abc01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.945+0000 7f007ccbb640 1 --2- 172.21.15.64:0/3772972625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55863a8121b0 0x55863abc01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.945+0000 7f007ccbb640 1 -- 172.21.15.64:0/3772972625 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55863abb9730 con 0x55863a7e92f0 2024-09-21T04:05:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.945+0000 7f007d4bc640 1 --2- 172.21.15.64:0/3772972625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863a7f14d0 0x55863abcf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:05:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.945+0000 7f007ccbb640 1 --2- 172.21.15.64:0/3772972625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7e92f0 0x55863abb90f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0068009900 tx=0x7f0068009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb2e26a8995b95 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.945+0000 7f0062ffd640 1 -- 172.21.15.64:0/3772972625 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0068035040 con 0x55863a7e92f0 2024-09-21T04:05:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.945+0000 7f0062ffd640 1 -- 172.21.15.64:0/3772972625 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00680317c0 con 0x55863a7e92f0 2024-09-21T04:05:45.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.945+0000 7f0062ffd640 1 -- 172.21.15.64:0/3772972625 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0068003c40 con 0x55863a7e92f0 2024-09-21T04:05:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.946+0000 7f008668c740 1 -- 172.21.15.64:0/3772972625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7e92f0 msgr2=0x55863abb90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.946+0000 7f008668c740 1 --2- 172.21.15.64:0/3772972625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7e92f0 0x55863abb90f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0068009900 tx=0x7f0068009640 comp rx=0 tx=0).stop 2024-09-21T04:05:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.946+0000 7f008668c740 1 -- 172.21.15.64:0/3772972625 shutdown_connections 2024-09-21T04:05:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.946+0000 7f008668c740 1 --2- 172.21.15.64:0/3772972625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55863a8121b0 0x55863abc01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.946+0000 7f008668c740 1 --2- 172.21.15.64:0/3772972625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863a7f14d0 0x55863abcf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.946+0000 7f008668c740 1 --2- 172.21.15.64:0/3772972625 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7e92f0 0x55863abb90f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.946+0000 7f008668c740 1 -- 172.21.15.64:0/3772972625 >> 172.21.15.64:0/3772972625 conn(0x55863a806300 msgr2=0x55863abb89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.946+0000 7f008668c740 1 -- 172.21.15.64:0/3772972625 shutdown_connections 2024-09-21T04:05:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.946+0000 7f008668c740 1 -- 172.21.15.64:0/3772972625 wait complete. 2024-09-21T04:05:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.946+0000 7f008668c740 1 -- start start 2024-09-21T04:05:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.946+0000 7f008668c740 1 Processor -- start 2024-09-21T04:05:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.947+0000 7f008668c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863a7e92f0 0x55863abcb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.947+0000 7f008668c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7f14d0 0x55863abc8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.947+0000 7f008668c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55863a8121b0 0x55863abc9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.947+0000 7f008668c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f00781b01d0 con 0x55863a7f14d0 2024-09-21T04:05:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.947+0000 7f008668c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f00781b1640 con 0x55863a8121b0 2024-09-21T04:05:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.947+0000 7f008668c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f00781b08a0 con 0x55863a7e92f0 2024-09-21T04:05:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.947+0000 7f008668c740 10 client.0 _ll_register_callbacks cb 0x7f0074009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:05:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.947+0000 7f007d4bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7f14d0 0x55863abc8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:45.953 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.947+0000 7f007d4bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7f14d0 0x55863abc8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54638/0 (socket says 172.21.15.64:54638) 2024-09-21T04:05:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.947+0000 7f007d4bc640 1 -- 172.21.15.64:0/1983725124 learned_addr learned my addr 172.21.15.64:0/1983725124 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.947+0000 7f007ccbb640 1 --2- 172.21.15.64:0/1983725124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55863a8121b0 0x55863abc9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.947+0000 7f007ccbb640 1 --2- 172.21.15.64:0/1983725124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863a7e92f0 0x55863abcb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.947+0000 7f007d4bc640 1 -- 172.21.15.64:0/1983725124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863a7e92f0 msgr2=0x55863abcb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.947+0000 7f007d4bc640 1 --2- 172.21.15.64:0/1983725124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863a7e92f0 0x55863abcb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.947+0000 7f007d4bc640 1 -- 172.21.15.64:0/1983725124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55863a8121b0 msgr2=0x55863abc9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.947+0000 7f007d4bc640 1 --2- 172.21.15.64:0/1983725124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55863a8121b0 0x55863abc9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.947+0000 7f007d4bc640 1 -- 172.21.15.64:0/1983725124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55863abc97e0 con 0x55863a7f14d0 2024-09-21T04:05:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.947+0000 7f007ccbb640 1 --2- 172.21.15.64:0/1983725124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55863a8121b0 0x55863abc9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:05:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.948+0000 7f007d4bc640 1 --2- 172.21.15.64:0/1983725124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7f14d0 0x55863abc8d30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0074009ea0 tx=0x7f0074003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bfc931f764c5606 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.948+0000 7f0060ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:05:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.948+0000 7f0060ff9640 1 -- 172.21.15.64:0/1983725124 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0074029040 con 0x55863a7f14d0 2024-09-21T04:05:45.954 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.948+0000 7f0060ff9640 1 -- 172.21.15.64:0/1983725124 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f0074003d80 con 0x55863a7f14d0 2024-09-21T04:05:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.948+0000 7f008668c740 1 -- 172.21.15.64:0/1983725124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00680317c0 con 0x55863a7f14d0 2024-09-21T04:05:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.948+0000 7f0060ff9640 1 -- 172.21.15.64:0/1983725124 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f00740123a0 con 0x55863a7f14d0 2024-09-21T04:05:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.948+0000 7f008668c740 10 client.15951 fetch_fsmap learned FSMap version 30 2024-09-21T04:05:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.948+0000 7f008668c740 1 -- 172.21.15.64:0/1983725124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55863ac3f850 con 0x55863a7f14d0 2024-09-21T04:05:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.948+0000 7f0060ff9640 1 -- 172.21.15.64:0/1983725124 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00680317c0 con 0x55863a7f14d0 2024-09-21T04:05:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.948+0000 7f0060ff9640 1 -- 172.21.15.64:0/1983725124 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f007402c070 con 0x55863a7f14d0 2024-09-21T04:05:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.948+0000 7f008668c740 10 client.15951 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:05:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.948+0000 7f008668c740 10 client.15951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:05:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.948+0000 7f008668c740 20 client.15951 populate_metadata read hostname 'smithi064' 2024-09-21T04:05:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.948+0000 7f008668c740 1 --2- 172.21.15.64:0/1983725124 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55863a7a61c0 0x55863ac42090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.948+0000 7f008668c740 4 client.15951 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:05:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.949+0000 7f008668c740 1 -- 172.21.15.64:0/1983725124 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55863ab2a0f0 con 0x55863a7a61c0 2024-09-21T04:05:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.949+0000 7f007ccbb640 1 --2- 172.21.15.64:0/1983725124 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55863a7a61c0 0x55863ac42090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.949+0000 7f0060ff9640 10 client.15951 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:45.956 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:45.949+0000 7f007ccbb640 1 --2- 172.21.15.64:0/1983725124 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55863a7a61c0 0x55863ac42090 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f31d39003aa18728 server_cookie=5e21f88918019b67 in_seq=0 out_seq=0 2024-09-21T04:05:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:46.078+0000 7f0060ff9640 1 -- 172.21.15.64:0/1983725124 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55863ab2a0f0 con 0x55863a7a61c0 2024-09-21T04:05:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:46.078+0000 7f0060ff9640 10 client.15951 handle_command_reply: tid=0 2024-09-21T04:05:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:46.078+0000 7f008668c740 1 client.15951 shutdown 2024-09-21T04:05:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:46.079+0000 7f008668c740 1 -- 172.21.15.64:0/1983725124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7f14d0 msgr2=0x55863abc8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:46.079+0000 7f008668c740 1 --2- 172.21.15.64:0/1983725124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7f14d0 0x55863abc8d30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0074009ea0 tx=0x7f0074003820 comp rx=0 tx=0).stop 2024-09-21T04:05:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:46.079+0000 7f008668c740 1 -- 172.21.15.64:0/1983725124 shutdown_connections 2024-09-21T04:05:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:46.079+0000 7f008668c740 1 --2- 172.21.15.64:0/1983725124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55863a8121b0 0x55863abc9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:46.079+0000 7f008668c740 1 --2- 172.21.15.64:0/1983725124 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55863a7a61c0 0x55863ac42090 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:46.079+0000 7f008668c740 1 --2- 172.21.15.64:0/1983725124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55863a7f14d0 0x55863abc8d30 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:46.079+0000 7f008668c740 1 --2- 172.21.15.64:0/1983725124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55863a7e92f0 0x55863abcb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:46.079+0000 7f0060ff9640 0 client.15951 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:46.079+0000 7f008668c740 1 -- 172.21.15.64:0/1983725124 >> 172.21.15.64:0/1983725124 conn(0x55863a806300 msgr2=0x55863abb3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:46.079+0000 7f008668c740 1 -- 172.21.15.64:0/1983725124 shutdown_connections 2024-09-21T04:05:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:46.079+0000 7f008668c740 1 -- 172.21.15.64:0/1983725124 wait complete. 2024-09-21T04:05:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:46.079+0000 7f008668c740 20 client.15951 trim_cache size 0 max 16384 2024-09-21T04:05:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:05:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:05:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:05:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:05:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:46.080+0000 7f007f720640 1 -- 172.21.15.64:0/678696502 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f00640b6680 msgr2=0x7f00640b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:46.080+0000 7f007f720640 1 --2- 172.21.15.64:0/678696502 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f00640b6680 0x7f00640b8b90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f00700ad7f0 tx=0x7f00700d1040 comp rx=0 tx=0).stop 2024-09-21T04:05:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:46.080+0000 7f007f720640 1 -- 172.21.15.64:0/678696502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0078109d80 msgr2=0x7f00781a1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:46.080+0000 7f007f720640 1 --2- 172.21.15.64:0/678696502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0078109d80 0x7f00781a1f10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f007000eb10 tx=0x7f007000c2d0 comp rx=0 tx=0).stop 2024-09-21T04:05:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:46.080+0000 7f007f720640 1 -- 172.21.15.64:0/678696502 shutdown_connections 2024-09-21T04:05:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:46.080+0000 7f007f720640 1 --2- 172.21.15.64:0/678696502 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f00640b6680 0x7f00640b8b90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:46.080+0000 7f007f720640 1 --2- 172.21.15.64:0/678696502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0078109d80 0x7f00781a1f10 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:46.080+0000 7f007f720640 1 --2- 172.21.15.64:0/678696502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0078109430 0x7f007819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:46.080+0000 7f007f720640 1 --2- 172.21.15.64:0/678696502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0078102060 0x7f007819c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:46.080+0000 7f007f720640 1 -- 172.21.15.64:0/678696502 >> 172.21.15.64:0/678696502 conn(0x7f0078079e90 msgr2=0x7f00781051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:46.080+0000 7f007f720640 1 -- 172.21.15.64:0/678696502 shutdown_connections 2024-09-21T04:05:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:46.080+0000 7f007f720640 1 -- 172.21.15.64:0/678696502 wait complete. 2024-09-21T04:05:46.105 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:45 smithi131 ceph-mon[35193]: pgmap v373: 97 pgs: 97 active+clean; 65 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 250 op/s 2024-09-21T04:05:46.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:45 smithi119 ceph-mon[34811]: pgmap v373: 97 pgs: 97 active+clean; 65 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 250 op/s 2024-09-21T04:05:46.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:45 smithi064 ceph-mon[31180]: pgmap v373: 97 pgs: 97 active+clean; 65 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 250 op/s 2024-09-21T04:05:48.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:48 smithi119 ceph-mon[34811]: pgmap v374: 97 pgs: 97 active+clean; 72 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 416 op/s 2024-09-21T04:05:48.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:48 smithi064 ceph-mon[31180]: pgmap v374: 97 pgs: 97 active+clean; 72 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 416 op/s 2024-09-21T04:05:48.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:48 smithi131 ceph-mon[35193]: pgmap v374: 97 pgs: 97 active+clean; 72 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.9 MiB/s wr, 416 op/s 2024-09-21T04:05:50.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:50 smithi119 ceph-mon[34811]: pgmap v375: 97 pgs: 97 active+clean; 72 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 421 op/s 2024-09-21T04:05:50.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:50 smithi064 ceph-mon[31180]: pgmap v375: 97 pgs: 97 active+clean; 72 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 421 op/s 2024-09-21T04:05:50.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:50 smithi131 ceph-mon[35193]: pgmap v375: 97 pgs: 97 active+clean; 72 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 421 op/s 2024-09-21T04:05:51.171 INFO:tasks.workunit.client.0.smithi064.stdout: 25 432892 560 270407 374 104240 760 2024-09-21T04:05:52.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:52 smithi119 ceph-mon[34811]: pgmap v376: 97 pgs: 97 active+clean; 72 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 374 op/s 2024-09-21T04:05:52.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:52 smithi064 ceph-mon[31180]: pgmap v376: 97 pgs: 97 active+clean; 72 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 374 op/s 2024-09-21T04:05:52.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:52 smithi131 ceph-mon[35193]: pgmap v376: 97 pgs: 97 active+clean; 72 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 374 op/s 2024-09-21T04:05:53.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:53 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:05:53.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:53 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:05:53.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:53 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:05:54.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:54 smithi119 ceph-mon[34811]: pgmap v377: 97 pgs: 97 active+clean; 80 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 407 op/s 2024-09-21T04:05:54.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:54 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:05:54.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:54 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:05:54.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:54 smithi064 ceph-mon[31180]: pgmap v377: 97 pgs: 97 active+clean; 80 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 407 op/s 2024-09-21T04:05:54.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:54 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:05:54.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:54 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:05:54.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:54 smithi131 ceph-mon[35193]: pgmap v377: 97 pgs: 97 active+clean; 80 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 407 op/s 2024-09-21T04:05:54.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:54 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:05:54.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:54 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:05:56.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:56 smithi119 ceph-mon[34811]: pgmap v378: 97 pgs: 97 active+clean; 80 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 239 op/s 2024-09-21T04:05:56.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:56 smithi064 ceph-mon[31180]: pgmap v378: 97 pgs: 97 active+clean; 80 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 239 op/s 2024-09-21T04:05:56.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:56 smithi131 ceph-mon[35193]: pgmap v378: 97 pgs: 97 active+clean; 80 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 239 op/s 2024-09-21T04:05:58.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:05:58 smithi119 ceph-mon[34811]: pgmap v379: 97 pgs: 97 active+clean; 85 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 371 op/s 2024-09-21T04:05:58.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:05:58 smithi064 ceph-mon[31180]: pgmap v379: 97 pgs: 97 active+clean; 85 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 371 op/s 2024-09-21T04:05:58.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:05:58 smithi131 ceph-mon[35193]: pgmap v379: 97 pgs: 97 active+clean; 85 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 371 op/s 2024-09-21T04:05:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.183+0000 7f4366c25640 1 -- 172.21.15.64:0/411174134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43601082e0 msgr2=0x7f43601086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.183+0000 7f4366c25640 1 --2- 172.21.15.64:0/411174134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43601082e0 0x7f43601086f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f434c009900 tx=0x7f434c02a410 comp rx=0 tx=0).stop 2024-09-21T04:05:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.183+0000 7f4366c25640 1 -- 172.21.15.64:0/411174134 shutdown_connections 2024-09-21T04:05:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.183+0000 7f4366c25640 1 --2- 172.21.15.64:0/411174134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4360108c30 0x7f436010f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.183+0000 7f4366c25640 1 --2- 172.21.15.64:0/411174134 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43601082e0 0x7f43601086f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.183+0000 7f4366c25640 1 --2- 172.21.15.64:0/411174134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f43601029b0 0x7f4360102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.183+0000 7f4366c25640 1 -- 172.21.15.64:0/411174134 >> 172.21.15.64:0/411174134 conn(0x7f436007a740 msgr2=0x7f436007ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.183+0000 7f4366c25640 1 -- 172.21.15.64:0/411174134 shutdown_connections 2024-09-21T04:05:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.183+0000 7f4366c25640 1 -- 172.21.15.64:0/411174134 wait complete. 2024-09-21T04:05:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.184+0000 7f4366c25640 1 Processor -- start 2024-09-21T04:05:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.184+0000 7f4366c25640 1 -- start start 2024-09-21T04:05:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.184+0000 7f4366c25640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43601029b0 0x7f4360199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.184+0000 7f4366c25640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43601082e0 0x7f4360199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.184+0000 7f4366c25640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4360108c30 0x7f4360199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.184+0000 7f4366c25640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f43601128f0 con 0x7f43601082e0 2024-09-21T04:05:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.184+0000 7f4366c25640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4360112770 con 0x7f43601029b0 2024-09-21T04:05:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.184+0000 7f4366c25640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4360112a70 con 0x7f4360108c30 2024-09-21T04:05:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.184+0000 7f435ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43601082e0 0x7f4360199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.184+0000 7f435ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43601082e0 0x7f4360199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56464/0 (socket says 172.21.15.64:56464) 2024-09-21T04:05:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.184+0000 7f435ffff640 1 -- 172.21.15.64:0/3051272031 learned_addr learned my addr 172.21.15.64:0/3051272031 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.185+0000 7f43649c1640 1 --2- 172.21.15.64:0/3051272031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43601029b0 0x7f4360199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.185+0000 7f43651c2640 1 --2- 172.21.15.64:0/3051272031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4360108c30 0x7f4360199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.185+0000 7f435ffff640 1 -- 172.21.15.64:0/3051272031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4360108c30 msgr2=0x7f4360199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.185+0000 7f435ffff640 1 --2- 172.21.15.64:0/3051272031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4360108c30 0x7f4360199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.185+0000 7f435ffff640 1 -- 172.21.15.64:0/3051272031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43601029b0 msgr2=0x7f4360199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.185+0000 7f435ffff640 1 --2- 172.21.15.64:0/3051272031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43601029b0 0x7f4360199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.185+0000 7f435ffff640 1 -- 172.21.15.64:0/3051272031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43601a87e0 con 0x7f43601082e0 2024-09-21T04:05:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.185+0000 7f435ffff640 1 --2- 172.21.15.64:0/3051272031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43601082e0 0x7f4360199670 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f434c006010 tx=0x7f434c004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.185+0000 7f435dffb640 1 -- 172.21.15.64:0/3051272031 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c03f030 con 0x7f43601082e0 2024-09-21T04:05:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.185+0000 7f435dffb640 1 -- 172.21.15.64:0/3051272031 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f434c003aa0 con 0x7f43601082e0 2024-09-21T04:05:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.185+0000 7f4366c25640 1 -- 172.21.15.64:0/3051272031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43601a8a70 con 0x7f43601082e0 2024-09-21T04:05:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.185+0000 7f435dffb640 1 -- 172.21.15.64:0/3051272031 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c035070 con 0x7f43601082e0 2024-09-21T04:05:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.185+0000 7f4366c25640 1 -- 172.21.15.64:0/3051272031 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43601a8fb0 con 0x7f43601082e0 2024-09-21T04:05:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.188+0000 7f435dffb640 1 -- 172.21.15.64:0/3051272031 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f434c042030 con 0x7f43601082e0 2024-09-21T04:05:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.188+0000 7f436dc71740 1 Processor -- start 2024-09-21T04:05:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.188+0000 7f436dc71740 1 -- start start 2024-09-21T04:05:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.189+0000 7f436dc71740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636c4afb1b0 0x5636c4ea09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.189+0000 7f436dc71740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636c4ad22f0 0x5636c4ea0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.189+0000 7f436dc71740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4ada4d0 0x5636c4ea1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.189+0000 7f436dc71740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5636c4e156f0 con 0x5636c4ada4d0 2024-09-21T04:05:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.189+0000 7f436dc71740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5636c4e04b20 con 0x5636c4afb1b0 2024-09-21T04:05:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.189+0000 7f436dc71740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5636c4e08930 con 0x5636c4ad22f0 2024-09-21T04:05:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.189+0000 7f435dffb640 1 --2- 172.21.15.64:0/3051272031 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f43500b6700 0x7f43500b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.189+0000 7f435ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4ada4d0 0x5636c4ea1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.189+0000 7f43651c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636c4afb1b0 0x5636c4ea09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.189+0000 7f43651c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636c4afb1b0 0x5636c4ea09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:39244/0 (socket says 172.21.15.64:39244) 2024-09-21T04:05:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.189+0000 7f43651c2640 1 -- 172.21.15.64:0/3908962372 learned_addr learned my addr 172.21.15.64:0/3908962372 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.189+0000 7f435ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4ada4d0 0x5636c4ea1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56470/0 (socket says 172.21.15.64:56470) 2024-09-21T04:05:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.189+0000 7f43649c1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636c4ad22f0 0x5636c4ea0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.189+0000 7f43651c2640 1 --2- 172.21.15.64:0/3051272031 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f43500b6700 0x7f43500b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.189+0000 7f435dffb640 1 -- 172.21.15.64:0/3051272031 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f434c107c80 con 0x7f43601082e0 2024-09-21T04:05:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.189+0000 7f435ffff640 1 -- 172.21.15.64:0/3908962372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636c4ad22f0 msgr2=0x5636c4ea0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.189+0000 7f435ffff640 1 --2- 172.21.15.64:0/3908962372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636c4ad22f0 0x5636c4ea0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.189+0000 7f435ffff640 1 -- 172.21.15.64:0/3908962372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636c4afb1b0 msgr2=0x5636c4ea09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.189+0000 7f435ffff640 1 --2- 172.21.15.64:0/3908962372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636c4afb1b0 0x5636c4ea09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.189+0000 7f435ffff640 1 -- 172.21.15.64:0/3908962372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636c4dca530 con 0x5636c4ada4d0 2024-09-21T04:05:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.190+0000 7f43651c2640 1 --2- 172.21.15.64:0/3051272031 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f43500b6700 0x7f43500b8c10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4358005c40 tx=0x7f4358009be0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.190+0000 7f435ffff640 1 --2- 172.21.15.64:0/3908962372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4ada4d0 0x5636c4ea1430 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f434c105340 tx=0x7f434c09a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03619aadec9527e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.190+0000 7f434affd640 1 -- 172.21.15.64:0/3908962372 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c049070 con 0x5636c4ada4d0 2024-09-21T04:05:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.190+0000 7f434affd640 1 -- 172.21.15.64:0/3908962372 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f434c0cd550 con 0x5636c4ada4d0 2024-09-21T04:05:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.190+0000 7f434affd640 1 -- 172.21.15.64:0/3908962372 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c0ce140 con 0x5636c4ada4d0 2024-09-21T04:05:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.191+0000 7f436dc71740 1 -- 172.21.15.64:0/3908962372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4ada4d0 msgr2=0x5636c4ea1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.191+0000 7f436dc71740 1 --2- 172.21.15.64:0/3908962372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4ada4d0 0x5636c4ea1430 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f434c105340 tx=0x7f434c09a6f0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.191+0000 7f436dc71740 1 -- 172.21.15.64:0/3908962372 shutdown_connections 2024-09-21T04:05:59.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.191+0000 7f436dc71740 1 --2- 172.21.15.64:0/3908962372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4ada4d0 0x5636c4ea1430 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.191+0000 7f436dc71740 1 --2- 172.21.15.64:0/3908962372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636c4ad22f0 0x5636c4ea0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.191+0000 7f436dc71740 1 --2- 172.21.15.64:0/3908962372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636c4afb1b0 0x5636c4ea09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.191+0000 7f436dc71740 1 -- 172.21.15.64:0/3908962372 >> 172.21.15.64:0/3908962372 conn(0x5636c4aef300 msgr2=0x5636c4e9e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.191+0000 7f436dc71740 1 -- 172.21.15.64:0/3908962372 shutdown_connections 2024-09-21T04:05:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.191+0000 7f436dc71740 1 -- 172.21.15.64:0/3908962372 wait complete. 2024-09-21T04:05:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.191+0000 7f436dc71740 1 -- start start 2024-09-21T04:05:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.191+0000 7f436dc71740 1 Processor -- start 2024-09-21T04:05:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.192+0000 7f436dc71740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636c4ad22f0 0x5636c4eac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.192+0000 7f436dc71740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4ada4d0 0x5636c4eac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.192+0000 7f436dc71740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636c4afb1b0 0x5636c4eb8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.192+0000 7f436dc71740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5636c4d3cab0 con 0x5636c4ada4d0 2024-09-21T04:05:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.192+0000 7f436dc71740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5636c4daa730 con 0x5636c4afb1b0 2024-09-21T04:05:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.192+0000 7f436dc71740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5636c4e156f0 con 0x5636c4ad22f0 2024-09-21T04:05:59.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.192+0000 7f436dc71740 10 client.0 _ll_register_callbacks cb 0x5636c4dfe6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:05:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.192+0000 7f435ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4ada4d0 0x5636c4eac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.192+0000 7f435ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4ada4d0 0x5636c4eac600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56482/0 (socket says 172.21.15.64:56482) 2024-09-21T04:05:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.192+0000 7f435ffff640 1 -- 172.21.15.64:0/2693125054 learned_addr learned my addr 172.21.15.64:0/2693125054 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.192+0000 7f43651c2640 1 --2- 172.21.15.64:0/2693125054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636c4afb1b0 0x5636c4eb8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.192+0000 7f43649c1640 1 --2- 172.21.15.64:0/2693125054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636c4ad22f0 0x5636c4eac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.192+0000 7f435ffff640 1 -- 172.21.15.64:0/2693125054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636c4ad22f0 msgr2=0x5636c4eac0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.192+0000 7f435ffff640 1 --2- 172.21.15.64:0/2693125054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636c4ad22f0 0x5636c4eac0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.192+0000 7f435ffff640 1 -- 172.21.15.64:0/2693125054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636c4afb1b0 msgr2=0x5636c4eb8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.192+0000 7f435ffff640 1 --2- 172.21.15.64:0/2693125054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636c4afb1b0 0x5636c4eb8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.192+0000 7f435ffff640 1 -- 172.21.15.64:0/2693125054 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636c4b66790 con 0x5636c4ada4d0 2024-09-21T04:05:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.193+0000 7f435ffff640 1 --2- 172.21.15.64:0/2693125054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4ada4d0 0x5636c4eac600 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f434c0cacb0 tx=0x7f434c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3daed8924d60a35 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.193+0000 7f433ffff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:05:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.193+0000 7f433ffff640 1 -- 172.21.15.64:0/2693125054 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c049040 con 0x5636c4ada4d0 2024-09-21T04:05:59.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.193+0000 7f433ffff640 1 -- 172.21.15.64:0/2693125054 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f434c0d6fd0 con 0x5636c4ada4d0 2024-09-21T04:05:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.193+0000 7f436dc71740 1 -- 172.21.15.64:0/2693125054 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f434c0cd550 con 0x5636c4ada4d0 2024-09-21T04:05:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.193+0000 7f433ffff640 1 -- 172.21.15.64:0/2693125054 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f434c0c91b0 con 0x5636c4ada4d0 2024-09-21T04:05:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.193+0000 7f436dc71740 10 client.15963 fetch_fsmap learned FSMap version 30 2024-09-21T04:05:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.193+0000 7f436dc71740 1 -- 172.21.15.64:0/2693125054 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5636c4bdf620 con 0x5636c4ada4d0 2024-09-21T04:05:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.193+0000 7f433ffff640 1 -- 172.21.15.64:0/2693125054 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f434c0cd550 con 0x5636c4ada4d0 2024-09-21T04:05:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.193+0000 7f433ffff640 1 -- 172.21.15.64:0/2693125054 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f434c0eecf0 con 0x5636c4ada4d0 2024-09-21T04:05:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.193+0000 7f436dc71740 10 client.15963 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:05:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.193+0000 7f436dc71740 10 client.15963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:05:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.193+0000 7f436dc71740 20 client.15963 populate_metadata read hostname 'smithi064' 2024-09-21T04:05:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.193+0000 7f436dc71740 1 --2- 172.21.15.64:0/2693125054 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5636c4ab9d80 0x5636c4ebdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.193+0000 7f436dc71740 4 client.15963 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:05:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.193+0000 7f436dc71740 1 -- 172.21.15.64:0/2693125054 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636c4e17640 con 0x5636c4ab9d80 2024-09-21T04:05:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.194+0000 7f43649c1640 1 --2- 172.21.15.64:0/2693125054 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5636c4ab9d80 0x5636c4ebdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.194+0000 7f43649c1640 1 --2- 172.21.15.64:0/2693125054 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5636c4ab9d80 0x5636c4ebdd10 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecf6e1b8d4e2afae server_cookie=60a276541184dba6 in_seq=0 out_seq=0 2024-09-21T04:05:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.194+0000 7f433ffff640 10 client.15963 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.195+0000 7f433ffff640 1 -- 172.21.15.64:0/2693125054 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5636c4e17640 con 0x5636c4ab9d80 2024-09-21T04:05:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.195+0000 7f433ffff640 10 client.15963 handle_command_reply: tid=0 2024-09-21T04:05:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.195+0000 7f436dc71740 1 client.15963 shutdown 2024-09-21T04:05:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.195+0000 7f436dc71740 1 -- 172.21.15.64:0/2693125054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4ada4d0 msgr2=0x5636c4eac600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.195+0000 7f436dc71740 1 --2- 172.21.15.64:0/2693125054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4ada4d0 0x5636c4eac600 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f434c0cacb0 tx=0x7f434c02a410 comp rx=0 tx=0).stop 2024-09-21T04:05:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.195+0000 7f436dc71740 1 -- 172.21.15.64:0/2693125054 shutdown_connections 2024-09-21T04:05:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.195+0000 7f436dc71740 1 --2- 172.21.15.64:0/2693125054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636c4afb1b0 0x5636c4eb8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.195+0000 7f436dc71740 1 --2- 172.21.15.64:0/2693125054 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5636c4ab9d80 0x5636c4ebdd10 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.195+0000 7f436dc71740 1 --2- 172.21.15.64:0/2693125054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4ada4d0 0x5636c4eac600 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.195+0000 7f436dc71740 1 --2- 172.21.15.64:0/2693125054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636c4ad22f0 0x5636c4eac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.195+0000 7f433ffff640 0 client.15963 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.195+0000 7f436dc71740 1 -- 172.21.15.64:0/2693125054 >> 172.21.15.64:0/2693125054 conn(0x5636c4aef300 msgr2=0x5636c4ea19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.196+0000 7f436dc71740 1 -- 172.21.15.64:0/2693125054 shutdown_connections 2024-09-21T04:05:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.196+0000 7f436dc71740 1 -- 172.21.15.64:0/2693125054 wait complete. 2024-09-21T04:05:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.196+0000 7f436dc71740 20 client.15963 trim_cache size 0 max 16384 2024-09-21T04:05:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.210+0000 7f436dc71740 1 Processor -- start 2024-09-21T04:05:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.210+0000 7f436dc71740 1 -- start start 2024-09-21T04:05:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.211+0000 7f436dc71740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636c4ad22f0 0x5636c4eac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.211+0000 7f436dc71740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636c4ada4d0 0x5636c4ea20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.211+0000 7f436dc71740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4afb1b0 0x5636c4eb7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.211+0000 7f436dc71740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f43601b4320 con 0x5636c4afb1b0 2024-09-21T04:05:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.211+0000 7f436dc71740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f43601b1b60 con 0x5636c4ad22f0 2024-09-21T04:05:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.211+0000 7f436dc71740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f43601af9b0 con 0x5636c4ada4d0 2024-09-21T04:05:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.211+0000 7f43651c2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4afb1b0 0x5636c4eb7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.211+0000 7f43649c1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636c4ad22f0 0x5636c4eac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.211+0000 7f43651c2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4afb1b0 0x5636c4eb7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56494/0 (socket says 172.21.15.64:56494) 2024-09-21T04:05:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.211+0000 7f43651c2640 1 -- 172.21.15.64:0/3924585892 learned_addr learned my addr 172.21.15.64:0/3924585892 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.211+0000 7f435ffff640 1 --2- 172.21.15.64:0/3924585892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636c4ada4d0 0x5636c4ea20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.211+0000 7f43651c2640 1 -- 172.21.15.64:0/3924585892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636c4ada4d0 msgr2=0x5636c4ea20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.211+0000 7f43651c2640 1 --2- 172.21.15.64:0/3924585892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636c4ada4d0 0x5636c4ea20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.211+0000 7f43651c2640 1 -- 172.21.15.64:0/3924585892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636c4ad22f0 msgr2=0x5636c4eac5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.211+0000 7f43651c2640 1 --2- 172.21.15.64:0/3924585892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636c4ad22f0 0x5636c4eac5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.211+0000 7f43651c2640 1 -- 172.21.15.64:0/3924585892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636c4ea65b0 con 0x5636c4afb1b0 2024-09-21T04:05:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.211+0000 7f43649c1640 1 --2- 172.21.15.64:0/3924585892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636c4ad22f0 0x5636c4eac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:05:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.211+0000 7f43651c2640 1 --2- 172.21.15.64:0/3924585892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4afb1b0 0x5636c4eb7f60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4358009f10 tx=0x7f435801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317e1d765788ddd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.212+0000 7f434a7fc640 1 -- 172.21.15.64:0/3924585892 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4358004b20 con 0x5636c4afb1b0 2024-09-21T04:05:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.212+0000 7f434a7fc640 1 -- 172.21.15.64:0/3924585892 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4358004cc0 con 0x5636c4afb1b0 2024-09-21T04:05:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.212+0000 7f434a7fc640 1 -- 172.21.15.64:0/3924585892 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4358005700 con 0x5636c4afb1b0 2024-09-21T04:05:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.212+0000 7f436dc71740 1 -- 172.21.15.64:0/3924585892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4afb1b0 msgr2=0x5636c4eb7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.212+0000 7f436dc71740 1 --2- 172.21.15.64:0/3924585892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4afb1b0 0x5636c4eb7f60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4358009f10 tx=0x7f435801d000 comp rx=0 tx=0).stop 2024-09-21T04:05:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.212+0000 7f436dc71740 1 -- 172.21.15.64:0/3924585892 shutdown_connections 2024-09-21T04:05:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.212+0000 7f436dc71740 1 --2- 172.21.15.64:0/3924585892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4afb1b0 0x5636c4eb7f60 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.212+0000 7f436dc71740 1 --2- 172.21.15.64:0/3924585892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636c4ada4d0 0x5636c4ea20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.212+0000 7f436dc71740 1 --2- 172.21.15.64:0/3924585892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636c4ad22f0 0x5636c4eac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.212+0000 7f436dc71740 1 -- 172.21.15.64:0/3924585892 >> 172.21.15.64:0/3924585892 conn(0x5636c4aef300 msgr2=0x5636c4ea19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.212+0000 7f436dc71740 1 -- 172.21.15.64:0/3924585892 shutdown_connections 2024-09-21T04:05:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.213+0000 7f436dc71740 1 -- 172.21.15.64:0/3924585892 wait complete. 2024-09-21T04:05:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.213+0000 7f436dc71740 1 -- start start 2024-09-21T04:05:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.213+0000 7f436dc71740 1 Processor -- start 2024-09-21T04:05:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.213+0000 7f436dc71740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4ad22f0 0x5636c4eb4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.213+0000 7f436dc71740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636c4ada4d0 0x5636c4eaf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.213+0000 7f436dc71740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636c4afb1b0 0x5636c4eafec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.213+0000 7f436dc71740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f43601b0020 con 0x5636c4ad22f0 2024-09-21T04:05:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.213+0000 7f436dc71740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f43601b1490 con 0x5636c4ada4d0 2024-09-21T04:05:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.213+0000 7f436dc71740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f43601b06f0 con 0x5636c4afb1b0 2024-09-21T04:05:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.213+0000 7f436dc71740 10 client.0 _ll_register_callbacks cb 0x7f4358011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:05:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.214+0000 7f43649c1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4ad22f0 0x5636c4eb4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.214+0000 7f435ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636c4ada4d0 0x5636c4eaf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.214+0000 7f43649c1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4ad22f0 0x5636c4eb4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56506/0 (socket says 172.21.15.64:56506) 2024-09-21T04:05:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.214+0000 7f43649c1640 1 -- 172.21.15.64:0/3880877273 learned_addr learned my addr 172.21.15.64:0/3880877273 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:05:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.214+0000 7f43651c2640 1 --2- 172.21.15.64:0/3880877273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636c4afb1b0 0x5636c4eafec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.214+0000 7f43649c1640 1 -- 172.21.15.64:0/3880877273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636c4afb1b0 msgr2=0x5636c4eafec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.214+0000 7f43649c1640 1 --2- 172.21.15.64:0/3880877273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636c4afb1b0 0x5636c4eafec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.214+0000 7f43649c1640 1 -- 172.21.15.64:0/3880877273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636c4ada4d0 msgr2=0x5636c4eaf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:05:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.214+0000 7f43649c1640 1 --2- 172.21.15.64:0/3880877273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636c4ada4d0 0x5636c4eaf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.214+0000 7f43649c1640 1 -- 172.21.15.64:0/3880877273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636c4eb0400 con 0x5636c4ad22f0 2024-09-21T04:05:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.214+0000 7f435ffff640 1 --2- 172.21.15.64:0/3880877273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636c4ada4d0 0x5636c4eaf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:05:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.214+0000 7f43649c1640 1 --2- 172.21.15.64:0/3880877273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4ad22f0 0x5636c4eb4490 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4354005fd0 tx=0x7f4354010af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05e7c6ef4ffd940 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:05:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.214+0000 7f433ffff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:05:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.214+0000 7f433ffff640 1 -- 172.21.15.64:0/3880877273 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4354004220 con 0x5636c4ad22f0 2024-09-21T04:05:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.214+0000 7f433ffff640 1 -- 172.21.15.64:0/3880877273 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f43540043c0 con 0x5636c4ad22f0 2024-09-21T04:05:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.214+0000 7f436dc71740 1 -- 172.21.15.64:0/3880877273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4358004cc0 con 0x5636c4ad22f0 2024-09-21T04:05:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.214+0000 7f433ffff640 1 -- 172.21.15.64:0/3880877273 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4354005020 con 0x5636c4ad22f0 2024-09-21T04:05:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.214+0000 7f436dc71740 10 client.15969 fetch_fsmap learned FSMap version 30 2024-09-21T04:05:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.214+0000 7f436dc71740 1 -- 172.21.15.64:0/3880877273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5636c4f28390 con 0x5636c4ad22f0 2024-09-21T04:05:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.215+0000 7f433ffff640 1 -- 172.21.15.64:0/3880877273 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4358004cc0 con 0x5636c4ad22f0 2024-09-21T04:05:59.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.215+0000 7f433ffff640 1 -- 172.21.15.64:0/3880877273 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f435402c020 con 0x5636c4ad22f0 2024-09-21T04:05:59.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.215+0000 7f436dc71740 10 client.15969 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:05:59.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.215+0000 7f436dc71740 10 client.15969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:05:59.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.215+0000 7f436dc71740 20 client.15969 populate_metadata read hostname 'smithi064' 2024-09-21T04:05:59.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.215+0000 7f436dc71740 1 --2- 172.21.15.64:0/3880877273 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5636c4a8f1c0 0x5636c4f2ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:05:59.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.215+0000 7f436dc71740 4 client.15969 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:05:59.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.215+0000 7f436dc71740 1 -- 172.21.15.64:0/3880877273 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5636c4e130f0 con 0x5636c4a8f1c0 2024-09-21T04:05:59.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.215+0000 7f435ffff640 1 --2- 172.21.15.64:0/3880877273 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5636c4a8f1c0 0x5636c4f2ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:05:59.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.216+0000 7f433ffff640 10 client.15969 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:59.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.216+0000 7f435ffff640 1 --2- 172.21.15.64:0/3880877273 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5636c4a8f1c0 0x5636c4f2ab90 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=752f017bed5aa24a server_cookie=e21270d04912c627 in_seq=0 out_seq=0 2024-09-21T04:05:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.425+0000 7f433ffff640 1 -- 172.21.15.64:0/3880877273 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5636c4e130f0 con 0x5636c4a8f1c0 2024-09-21T04:05:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.425+0000 7f433ffff640 10 client.15969 handle_command_reply: tid=0 2024-09-21T04:05:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.438+0000 7f436dc71740 1 client.15969 shutdown 2024-09-21T04:05:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.458+0000 7f436dc71740 1 -- 172.21.15.64:0/3880877273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4ad22f0 msgr2=0x5636c4eb4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.458+0000 7f436dc71740 1 --2- 172.21.15.64:0/3880877273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4ad22f0 0x5636c4eb4490 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4354005fd0 tx=0x7f4354010af0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.459+0000 7f436dc71740 1 -- 172.21.15.64:0/3880877273 shutdown_connections 2024-09-21T04:05:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.459+0000 7f436dc71740 1 --2- 172.21.15.64:0/3880877273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636c4afb1b0 0x5636c4eafec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.459+0000 7f436dc71740 1 --2- 172.21.15.64:0/3880877273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636c4ada4d0 0x5636c4eaf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.459+0000 7f436dc71740 1 --2- 172.21.15.64:0/3880877273 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5636c4a8f1c0 0x5636c4f2ab90 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.459+0000 7f433ffff640 0 client.15969 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:05:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.459+0000 7f436dc71740 1 --2- 172.21.15.64:0/3880877273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636c4ad22f0 0x5636c4eb4490 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.459+0000 7f436dc71740 1 -- 172.21.15.64:0/3880877273 >> 172.21.15.64:0/3880877273 conn(0x5636c4aef300 msgr2=0x5636c4e9cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:59.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.460+0000 7f436dc71740 1 -- 172.21.15.64:0/3880877273 shutdown_connections 2024-09-21T04:05:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.461+0000 7f436dc71740 1 -- 172.21.15.64:0/3880877273 wait complete. 2024-09-21T04:05:59.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.461+0000 7f436dc71740 20 client.15969 trim_cache size 0 max 16384 2024-09-21T04:05:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:05:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:05:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:05:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:05:59.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.493+0000 7f4366c25640 1 -- 172.21.15.64:0/3051272031 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f43500b6700 msgr2=0x7f43500b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.493+0000 7f4366c25640 1 --2- 172.21.15.64:0/3051272031 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f43500b6700 0x7f43500b8c10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4358005c40 tx=0x7f4358009be0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.493+0000 7f4366c25640 1 -- 172.21.15.64:0/3051272031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43601082e0 msgr2=0x7f4360199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:05:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.493+0000 7f4366c25640 1 --2- 172.21.15.64:0/3051272031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43601082e0 0x7f4360199670 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f434c006010 tx=0x7f434c004060 comp rx=0 tx=0).stop 2024-09-21T04:05:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.493+0000 7f4366c25640 1 -- 172.21.15.64:0/3051272031 shutdown_connections 2024-09-21T04:05:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.493+0000 7f4366c25640 1 --2- 172.21.15.64:0/3051272031 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f43500b6700 0x7f43500b8c10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.493+0000 7f4366c25640 1 --2- 172.21.15.64:0/3051272031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4360108c30 0x7f4360199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.493+0000 7f4366c25640 1 --2- 172.21.15.64:0/3051272031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43601082e0 0x7f4360199670 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.493+0000 7f4366c25640 1 --2- 172.21.15.64:0/3051272031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43601029b0 0x7f4360199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:05:59.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.493+0000 7f4366c25640 1 -- 172.21.15.64:0/3051272031 >> 172.21.15.64:0/3051272031 conn(0x7f436007a740 msgr2=0x7f4360106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:05:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.494+0000 7f4366c25640 1 -- 172.21.15.64:0/3051272031 shutdown_connections 2024-09-21T04:05:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:05:59.494+0000 7f4366c25640 1 -- 172.21.15.64:0/3051272031 wait complete. 2024-09-21T04:06:00.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:00 smithi119 ceph-mon[34811]: pgmap v380: 97 pgs: 97 active+clean; 89 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.7 MiB/s wr, 240 op/s 2024-09-21T04:06:00.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:00 smithi064 ceph-mon[31180]: pgmap v380: 97 pgs: 97 active+clean; 89 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.7 MiB/s wr, 240 op/s 2024-09-21T04:06:00.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:00 smithi131 ceph-mon[35193]: pgmap v380: 97 pgs: 97 active+clean; 89 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.7 MiB/s wr, 240 op/s 2024-09-21T04:06:01.171 INFO:tasks.workunit.client.0.smithi064.stdout: 36 1020830 587 677481 634 305546 1642 2024-09-21T04:06:02.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:02 smithi119 ceph-mon[34811]: pgmap v381: 97 pgs: 97 active+clean; 89 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.7 MiB/s wr, 231 op/s 2024-09-21T04:06:02.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:02 smithi064 ceph-mon[31180]: pgmap v381: 97 pgs: 97 active+clean; 89 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.7 MiB/s wr, 231 op/s 2024-09-21T04:06:02.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:02 smithi131 ceph-mon[35193]: pgmap v381: 97 pgs: 97 active+clean; 89 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.7 MiB/s wr, 231 op/s 2024-09-21T04:06:04.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:04 smithi119 ceph-mon[34811]: pgmap v382: 97 pgs: 97 active+clean; 97 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.9 MiB/s wr, 443 op/s 2024-09-21T04:06:04.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:04 smithi064 ceph-mon[31180]: pgmap v382: 97 pgs: 97 active+clean; 97 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.9 MiB/s wr, 443 op/s 2024-09-21T04:06:04.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:04 smithi131 ceph-mon[35193]: pgmap v382: 97 pgs: 97 active+clean; 97 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.9 MiB/s wr, 443 op/s 2024-09-21T04:06:06.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:06 smithi119 ceph-mon[34811]: pgmap v383: 97 pgs: 97 active+clean; 105 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.9 MiB/s wr, 454 op/s 2024-09-21T04:06:06.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:06 smithi064 ceph-mon[31180]: pgmap v383: 97 pgs: 97 active+clean; 105 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.9 MiB/s wr, 454 op/s 2024-09-21T04:06:06.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:06 smithi131 ceph-mon[35193]: pgmap v383: 97 pgs: 97 active+clean; 105 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.9 MiB/s wr, 454 op/s 2024-09-21T04:06:08.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:08 smithi119 ceph-mon[34811]: pgmap v384: 97 pgs: 97 active+clean; 173 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 8.9 MiB/s wr, 802 op/s 2024-09-21T04:06:08.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:08 smithi064 ceph-mon[31180]: pgmap v384: 97 pgs: 97 active+clean; 173 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 8.9 MiB/s wr, 802 op/s 2024-09-21T04:06:08.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:08 smithi131 ceph-mon[35193]: pgmap v384: 97 pgs: 97 active+clean; 173 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 8.9 MiB/s wr, 802 op/s 2024-09-21T04:06:10.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:10 smithi119 ceph-mon[34811]: pgmap v385: 97 pgs: 97 active+clean; 186 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.7 MiB/s wr, 743 op/s 2024-09-21T04:06:10.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:10 smithi064 ceph-mon[31180]: pgmap v385: 97 pgs: 97 active+clean; 186 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.7 MiB/s wr, 743 op/s 2024-09-21T04:06:10.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:10 smithi131 ceph-mon[35193]: pgmap v385: 97 pgs: 97 active+clean; 186 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 9.7 MiB/s wr, 743 op/s 2024-09-21T04:06:11.171 INFO:tasks.workunit.client.0.smithi064.stdout: 39 423503 305 294669 63 152211 486 2024-09-21T04:06:12.023 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T04:06:12.348 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:06:12.470 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:12 smithi064 ceph-mon[31180]: pgmap v386: 97 pgs: 97 active+clean; 186 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.3 MiB/s wr, 714 op/s 2024-09-21T04:06:12.470 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:12 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:06:12.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:12 smithi119 ceph-mon[34811]: pgmap v386: 97 pgs: 97 active+clean; 186 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.3 MiB/s wr, 714 op/s 2024-09-21T04:06:12.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:12 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:06:12.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:12 smithi131 ceph-mon[35193]: pgmap v386: 97 pgs: 97 active+clean; 186 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.3 MiB/s wr, 714 op/s 2024-09-21T04:06:12.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:12 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:06:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.621+0000 7f39e24ec640 1 -- 172.21.15.64:0/120184475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39dc104920 msgr2=0x7f39dc104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.621+0000 7f39e24ec640 1 --2- 172.21.15.64:0/120184475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39dc104920 0x7f39dc104d30 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f39cc009900 tx=0x7f39cc02a3f0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.622+0000 7f39e24ec640 1 -- 172.21.15.64:0/120184475 shutdown_connections 2024-09-21T04:06:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.622+0000 7f39e24ec640 1 --2- 172.21.15.64:0/120184475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39dc10dd80 0x7f39dc110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.622+0000 7f39e24ec640 1 --2- 172.21.15.64:0/120184475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39dc105270 0x7f39dc10d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.622+0000 7f39e24ec640 1 --2- 172.21.15.64:0/120184475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39dc104920 0x7f39dc104d30 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.622+0000 7f39e24ec640 1 -- 172.21.15.64:0/120184475 >> 172.21.15.64:0/120184475 conn(0x7f39dc0fe3f0 msgr2=0x7f39dc100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.622+0000 7f39e24ec640 1 -- 172.21.15.64:0/120184475 shutdown_connections 2024-09-21T04:06:12.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.622+0000 7f39e24ec640 1 -- 172.21.15.64:0/120184475 wait complete. 2024-09-21T04:06:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.622+0000 7f39e24ec640 1 Processor -- start 2024-09-21T04:06:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.623+0000 7f39e24ec640 1 -- start start 2024-09-21T04:06:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.623+0000 7f39e24ec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39dc104920 0x7f39dc19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.623+0000 7f39e24ec640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39dc105270 0x7f39dc19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.623+0000 7f39e24ec640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39dc10dd80 0x7f39dc19fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.623+0000 7f39e24ec640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1139e0 con 0x7f39dc105270 2024-09-21T04:06:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.623+0000 7f39e24ec640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc113860 con 0x7f39dc104920 2024-09-21T04:06:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.623+0000 7f39e24ec640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc113b60 con 0x7f39dc10dd80 2024-09-21T04:06:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.623+0000 7f39db7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39dc105270 0x7f39dc19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.623+0000 7f39db7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39dc105270 0x7f39dc19aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41702/0 (socket says 172.21.15.64:41702) 2024-09-21T04:06:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.623+0000 7f39dbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39dc104920 0x7f39dc19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.623+0000 7f39db7fe640 1 -- 172.21.15.64:0/874509610 learned_addr learned my addr 172.21.15.64:0/874509610 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.623+0000 7f39e0a89640 1 --2- 172.21.15.64:0/874509610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39dc10dd80 0x7f39dc19fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.624+0000 7f39db7fe640 1 -- 172.21.15.64:0/874509610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39dc10dd80 msgr2=0x7f39dc19fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.624+0000 7f39db7fe640 1 --2- 172.21.15.64:0/874509610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39dc10dd80 0x7f39dc19fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.624+0000 7f39db7fe640 1 -- 172.21.15.64:0/874509610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39dc104920 msgr2=0x7f39dc19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.624+0000 7f39db7fe640 1 --2- 172.21.15.64:0/874509610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39dc104920 0x7f39dc19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.624+0000 7f39db7fe640 1 -- 172.21.15.64:0/874509610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39dc1a03a0 con 0x7f39dc105270 2024-09-21T04:06:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.624+0000 7f39e0a89640 1 --2- 172.21.15.64:0/874509610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39dc10dd80 0x7f39dc19fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:06:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.624+0000 7f39dbfff640 1 --2- 172.21.15.64:0/874509610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39dc104920 0x7f39dc19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:06:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.624+0000 7f39db7fe640 1 --2- 172.21.15.64:0/874509610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39dc105270 0x7f39dc19aaa0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f39d000e9f0 tx=0x7f39d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.624+0000 7f39d97fa640 1 -- 172.21.15.64:0/874509610 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d000ccc0 con 0x7f39dc105270 2024-09-21T04:06:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.624+0000 7f39d97fa640 1 -- 172.21.15.64:0/874509610 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39d00108e0 con 0x7f39dc105270 2024-09-21T04:06:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.624+0000 7f39e24ec640 1 -- 172.21.15.64:0/874509610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39dc1a0690 con 0x7f39dc105270 2024-09-21T04:06:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.624+0000 7f39d97fa640 1 -- 172.21.15.64:0/874509610 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0018860 con 0x7f39dc105270 2024-09-21T04:06:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.624+0000 7f39e24ec640 1 -- 172.21.15.64:0/874509610 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39dc1a6eb0 con 0x7f39dc105270 2024-09-21T04:06:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.627+0000 7f39e945a740 1 Processor -- start 2024-09-21T04:06:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.627+0000 7f39e945a740 1 -- start start 2024-09-21T04:06:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.628+0000 7f39d97fa640 1 -- 172.21.15.64:0/874509610 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f39d0010430 con 0x7f39dc105270 2024-09-21T04:06:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.628+0000 7f39e945a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc29a3b1b0 0x55bc29de09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.628+0000 7f39e945a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc29a122f0 0x55bc29de0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.628+0000 7f39e945a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc29a1a4d0 0x55bc29de1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.628+0000 7f39e945a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bc29d556f0 con 0x55bc29a1a4d0 2024-09-21T04:06:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.628+0000 7f39e945a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bc29d44b20 con 0x55bc29a122f0 2024-09-21T04:06:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.628+0000 7f39e945a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bc29d48930 con 0x55bc29a3b1b0 2024-09-21T04:06:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.628+0000 7f39db7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc29a1a4d0 0x55bc29de1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.628+0000 7f39db7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc29a1a4d0 0x55bc29de1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41718/0 (socket says 172.21.15.64:41718) 2024-09-21T04:06:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.628+0000 7f39dbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc29a122f0 0x55bc29de0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.628+0000 7f39db7fe640 1 -- 172.21.15.64:0/671584873 learned_addr learned my addr 172.21.15.64:0/671584873 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.628+0000 7f39e0a89640 1 --2- 172.21.15.64:0/671584873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc29a3b1b0 0x55bc29de09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.628+0000 7f39dbfff640 1 -- 172.21.15.64:0/671584873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc29a3b1b0 msgr2=0x55bc29de09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.628+0000 7f39dbfff640 1 --2- 172.21.15.64:0/671584873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc29a3b1b0 0x55bc29de09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.628+0000 7f39dbfff640 1 -- 172.21.15.64:0/671584873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc29a1a4d0 msgr2=0x55bc29de1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.628+0000 7f39dbfff640 1 --2- 172.21.15.64:0/671584873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc29a1a4d0 0x55bc29de1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.628+0000 7f39dbfff640 1 -- 172.21.15.64:0/671584873 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc29d0a530 con 0x55bc29a122f0 2024-09-21T04:06:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.629+0000 7f39e0a89640 1 --2- 172.21.15.64:0/671584873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc29a3b1b0 0x55bc29de09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:06:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.629+0000 7f39db7fe640 1 --2- 172.21.15.64:0/671584873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc29a1a4d0 0x55bc29de1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:06:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.629+0000 7f39dbfff640 1 --2- 172.21.15.64:0/671584873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc29a122f0 0x55bc29de0ef0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f39cc006010 tx=0x7f39cc031550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a1e3e40269a7398 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.629+0000 7f39d97fa640 1 --2- 172.21.15.64:0/874509610 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f39c40b6680 0x7f39c40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.629+0000 7f39c27fc640 1 -- 172.21.15.64:0/671584873 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc035040 con 0x55bc29a122f0 2024-09-21T04:06:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.629+0000 7f39e0a89640 1 --2- 172.21.15.64:0/874509610 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f39c40b6680 0x7f39c40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.629+0000 7f39c27fc640 1 -- 172.21.15.64:0/671584873 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39cc005050 con 0x55bc29a122f0 2024-09-21T04:06:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.629+0000 7f39d97fa640 1 -- 172.21.15.64:0/874509610 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f39d00e7aa0 con 0x7f39dc105270 2024-09-21T04:06:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.629+0000 7f39c27fc640 1 -- 172.21.15.64:0/671584873 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc0049d0 con 0x55bc29a122f0 2024-09-21T04:06:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.630+0000 7f39e945a740 1 -- 172.21.15.64:0/671584873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc29a122f0 msgr2=0x55bc29de0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.630+0000 7f39e945a740 1 --2- 172.21.15.64:0/671584873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc29a122f0 0x55bc29de0ef0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f39cc006010 tx=0x7f39cc031550 comp rx=0 tx=0).stop 2024-09-21T04:06:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.630+0000 7f39e0a89640 1 --2- 172.21.15.64:0/874509610 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f39c40b6680 0x7f39c40b8b90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f39d4005cc0 tx=0x7f39d40084f0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.630+0000 7f39e945a740 1 -- 172.21.15.64:0/671584873 shutdown_connections 2024-09-21T04:06:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.630+0000 7f39e945a740 1 --2- 172.21.15.64:0/671584873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc29a1a4d0 0x55bc29de1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.630+0000 7f39e945a740 1 --2- 172.21.15.64:0/671584873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc29a122f0 0x55bc29de0ef0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.630+0000 7f39e945a740 1 --2- 172.21.15.64:0/671584873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc29a3b1b0 0x55bc29de09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.630+0000 7f39e945a740 1 -- 172.21.15.64:0/671584873 >> 172.21.15.64:0/671584873 conn(0x55bc29a2f300 msgr2=0x55bc29dde8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.630+0000 7f39e945a740 1 -- 172.21.15.64:0/671584873 shutdown_connections 2024-09-21T04:06:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.630+0000 7f39e945a740 1 -- 172.21.15.64:0/671584873 wait complete. 2024-09-21T04:06:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.631+0000 7f39e945a740 1 -- start start 2024-09-21T04:06:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.631+0000 7f39e945a740 1 Processor -- start 2024-09-21T04:06:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.631+0000 7f39e945a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc29a122f0 0x55bc29dec0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.631+0000 7f39e945a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc29a1a4d0 0x55bc29dec5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.631+0000 7f39e945a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc29a3b1b0 0x55bc29df83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.631+0000 7f39e945a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bc29c7cab0 con 0x55bc29a3b1b0 2024-09-21T04:06:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.631+0000 7f39e945a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bc29cea730 con 0x55bc29a1a4d0 2024-09-21T04:06:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.631+0000 7f39e945a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bc29def0d0 con 0x55bc29a122f0 2024-09-21T04:06:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.631+0000 7f39e945a740 10 client.0 _ll_register_callbacks cb 0x55bc29d3e6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:06:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.632+0000 7f39db7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc29a1a4d0 0x55bc29dec5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.632+0000 7f39dbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc29a122f0 0x55bc29dec0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.632+0000 7f39e0a89640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc29a3b1b0 0x55bc29df83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.632+0000 7f39dbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc29a122f0 0x55bc29dec0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:42206/0 (socket says 172.21.15.64:42206) 2024-09-21T04:06:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.632+0000 7f39db7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc29a1a4d0 0x55bc29dec5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:48636/0 (socket says 172.21.15.64:48636) 2024-09-21T04:06:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.632+0000 7f39dbfff640 1 -- 172.21.15.64:0/2266106824 learned_addr learned my addr 172.21.15.64:0/2266106824 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.632+0000 7f39db7fe640 1 -- 172.21.15.64:0/2266106824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc29a122f0 msgr2=0x55bc29dec0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.632+0000 7f39db7fe640 1 --2- 172.21.15.64:0/2266106824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc29a122f0 0x55bc29dec0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.632+0000 7f39db7fe640 1 -- 172.21.15.64:0/2266106824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc29a3b1b0 msgr2=0x55bc29df83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.632+0000 7f39db7fe640 1 --2- 172.21.15.64:0/2266106824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc29a3b1b0 0x55bc29df83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.632+0000 7f39db7fe640 1 -- 172.21.15.64:0/2266106824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc29aa6790 con 0x55bc29a1a4d0 2024-09-21T04:06:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.632+0000 7f39dbfff640 1 --2- 172.21.15.64:0/2266106824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc29a122f0 0x55bc29dec0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:06:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.632+0000 7f39e0a89640 1 --2- 172.21.15.64:0/2266106824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc29a3b1b0 0x55bc29df83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:06:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.632+0000 7f39db7fe640 1 --2- 172.21.15.64:0/2266106824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc29a1a4d0 0x55bc29dec5f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f39d000e9c0 tx=0x7f39d0009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11800442912f9653 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.632+0000 7f39bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:06:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.632+0000 7f39bb7fe640 1 -- 172.21.15.64:0/2266106824 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d00a9f10 con 0x55bc29a1a4d0 2024-09-21T04:06:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.632+0000 7f39e945a740 1 -- 172.21.15.64:0/2266106824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39cc005050 con 0x55bc29a1a4d0 2024-09-21T04:06:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.632+0000 7f39bb7fe640 1 -- 172.21.15.64:0/2266106824 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f39d00ac590 con 0x55bc29a1a4d0 2024-09-21T04:06:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.633+0000 7f39bb7fe640 1 -- 172.21.15.64:0/2266106824 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f39d00ad030 con 0x55bc29a1a4d0 2024-09-21T04:06:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.633+0000 7f39e945a740 10 client.25691 fetch_fsmap learned FSMap version 30 2024-09-21T04:06:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.633+0000 7f39e945a740 1 -- 172.21.15.64:0/2266106824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bc29b1f620 con 0x55bc29a1a4d0 2024-09-21T04:06:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.633+0000 7f39bb7fe640 1 -- 172.21.15.64:0/2266106824 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39cc005050 con 0x55bc29a1a4d0 2024-09-21T04:06:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.633+0000 7f39bb7fe640 1 -- 172.21.15.64:0/2266106824 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f39d00ac790 con 0x55bc29a1a4d0 2024-09-21T04:06:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.633+0000 7f39e945a740 10 client.25691 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:06:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.633+0000 7f39e945a740 10 client.25691 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:06:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.633+0000 7f39e945a740 20 client.25691 populate_metadata read hostname 'smithi064' 2024-09-21T04:06:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.633+0000 7f39e945a740 1 --2- 172.21.15.64:0/2266106824 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55bc299f9d80 0x55bc29dfe1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.633+0000 7f39e945a740 4 client.25691 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:06:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.633+0000 7f39e945a740 1 -- 172.21.15.64:0/2266106824 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc29d57640 con 0x55bc299f9d80 2024-09-21T04:06:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.633+0000 7f39dbfff640 1 --2- 172.21.15.64:0/2266106824 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55bc299f9d80 0x55bc29dfe1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.634+0000 7f39dbfff640 1 --2- 172.21.15.64:0/2266106824 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55bc299f9d80 0x55bc29dfe1d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b70915b48ed9dd0 server_cookie=28e1d33ee060fa72 in_seq=0 out_seq=0 2024-09-21T04:06:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.634+0000 7f39bb7fe640 10 client.25691 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.635+0000 7f39bb7fe640 1 -- 172.21.15.64:0/2266106824 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bc29d57640 con 0x55bc299f9d80 2024-09-21T04:06:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.635+0000 7f39bb7fe640 10 client.25691 handle_command_reply: tid=0 2024-09-21T04:06:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.635+0000 7f39e945a740 1 client.25691 shutdown 2024-09-21T04:06:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.635+0000 7f39e945a740 1 -- 172.21.15.64:0/2266106824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc29a1a4d0 msgr2=0x55bc29dec5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.635+0000 7f39e945a740 1 --2- 172.21.15.64:0/2266106824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc29a1a4d0 0x55bc29dec5f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f39d000e9c0 tx=0x7f39d0009740 comp rx=0 tx=0).stop 2024-09-21T04:06:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.635+0000 7f39e945a740 1 -- 172.21.15.64:0/2266106824 shutdown_connections 2024-09-21T04:06:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.635+0000 7f39e945a740 1 --2- 172.21.15.64:0/2266106824 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55bc299f9d80 0x55bc29dfe1d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.635+0000 7f39e945a740 1 --2- 172.21.15.64:0/2266106824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc29a3b1b0 0x55bc29df83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.635+0000 7f39e945a740 1 --2- 172.21.15.64:0/2266106824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc29a1a4d0 0x55bc29dec5f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.635+0000 7f39e945a740 1 --2- 172.21.15.64:0/2266106824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc29a122f0 0x55bc29dec0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.635+0000 7f39e945a740 1 -- 172.21.15.64:0/2266106824 >> 172.21.15.64:0/2266106824 conn(0x55bc29a2f300 msgr2=0x55bc29de19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.635+0000 7f39bb7fe640 0 client.25691 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.635+0000 7f39e945a740 1 -- 172.21.15.64:0/2266106824 shutdown_connections 2024-09-21T04:06:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.635+0000 7f39e945a740 1 -- 172.21.15.64:0/2266106824 wait complete. 2024-09-21T04:06:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.635+0000 7f39e945a740 20 client.25691 trim_cache size 0 max 16384 2024-09-21T04:06:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.650+0000 7f39e945a740 1 Processor -- start 2024-09-21T04:06:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.650+0000 7f39e945a740 1 -- start start 2024-09-21T04:06:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.650+0000 7f39e945a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc29a122f0 0x55bc29de60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.650+0000 7f39e945a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc29a1a4d0 0x55bc29dec590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.650+0000 7f39e945a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc29a3b1b0 0x55bc29de20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.650+0000 7f39e945a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1b2210 con 0x55bc29a1a4d0 2024-09-21T04:06:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.650+0000 7f39e945a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1afa50 con 0x55bc29a122f0 2024-09-21T04:06:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.650+0000 7f39e945a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1ad8a0 con 0x55bc29a3b1b0 2024-09-21T04:06:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.651+0000 7f39db7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc29a1a4d0 0x55bc29dec590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.651+0000 7f39dbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc29a122f0 0x55bc29de60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.651+0000 7f39db7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc29a1a4d0 0x55bc29dec590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41738/0 (socket says 172.21.15.64:41738) 2024-09-21T04:06:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.651+0000 7f39db7fe640 1 -- 172.21.15.64:0/3886128486 learned_addr learned my addr 172.21.15.64:0/3886128486 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.651+0000 7f39e0a89640 1 --2- 172.21.15.64:0/3886128486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc29a3b1b0 0x55bc29de20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.651+0000 7f39db7fe640 1 -- 172.21.15.64:0/3886128486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc29a3b1b0 msgr2=0x55bc29de20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.651+0000 7f39db7fe640 1 --2- 172.21.15.64:0/3886128486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc29a3b1b0 0x55bc29de20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.651+0000 7f39db7fe640 1 -- 172.21.15.64:0/3886128486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc29a122f0 msgr2=0x55bc29de60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.651+0000 7f39db7fe640 1 --2- 172.21.15.64:0/3886128486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc29a122f0 0x55bc29de60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.651+0000 7f39db7fe640 1 -- 172.21.15.64:0/3886128486 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc29dd8b40 con 0x55bc29a1a4d0 2024-09-21T04:06:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.651+0000 7f39db7fe640 1 --2- 172.21.15.64:0/3886128486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc29a1a4d0 0x55bc29dec590 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f39d00e7f20 tx=0x7f39d000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fafae8b4f19fb6e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.651+0000 7f39c1ffb640 1 -- 172.21.15.64:0/3886128486 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d00ed020 con 0x55bc29a1a4d0 2024-09-21T04:06:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.651+0000 7f39c1ffb640 1 -- 172.21.15.64:0/3886128486 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39d00b2190 con 0x55bc29a1a4d0 2024-09-21T04:06:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.651+0000 7f39c1ffb640 1 -- 172.21.15.64:0/3886128486 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d00ad210 con 0x55bc29a1a4d0 2024-09-21T04:06:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.652+0000 7f39e945a740 1 -- 172.21.15.64:0/3886128486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc29a1a4d0 msgr2=0x55bc29dec590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.652+0000 7f39e945a740 1 --2- 172.21.15.64:0/3886128486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc29a1a4d0 0x55bc29dec590 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f39d00e7f20 tx=0x7f39d000ce60 comp rx=0 tx=0).stop 2024-09-21T04:06:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.652+0000 7f39e945a740 1 -- 172.21.15.64:0/3886128486 shutdown_connections 2024-09-21T04:06:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.652+0000 7f39e945a740 1 --2- 172.21.15.64:0/3886128486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc29a3b1b0 0x55bc29de20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.652+0000 7f39e945a740 1 --2- 172.21.15.64:0/3886128486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc29a1a4d0 0x55bc29dec590 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.652+0000 7f39e945a740 1 --2- 172.21.15.64:0/3886128486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc29a122f0 0x55bc29de60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.652+0000 7f39e945a740 1 -- 172.21.15.64:0/3886128486 >> 172.21.15.64:0/3886128486 conn(0x55bc29a2f300 msgr2=0x55bc29de19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.652+0000 7f39e945a740 1 -- 172.21.15.64:0/3886128486 shutdown_connections 2024-09-21T04:06:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.652+0000 7f39e945a740 1 -- 172.21.15.64:0/3886128486 wait complete. 2024-09-21T04:06:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.653+0000 7f39e945a740 1 -- start start 2024-09-21T04:06:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.653+0000 7f39e945a740 1 Processor -- start 2024-09-21T04:06:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.653+0000 7f39e945a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc29a122f0 0x55bc29df4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.653+0000 7f39e945a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc29a1a4d0 0x55bc29def2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.653+0000 7f39e945a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc29a3b1b0 0x55bc29def9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.653+0000 7f39e945a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1adf10 con 0x55bc29a3b1b0 2024-09-21T04:06:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.653+0000 7f39e945a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1af380 con 0x55bc29a122f0 2024-09-21T04:06:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.653+0000 7f39e945a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1ae5e0 con 0x55bc29a1a4d0 2024-09-21T04:06:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.653+0000 7f39e945a740 10 client.0 _ll_register_callbacks cb 0x7f39d00056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:06:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.653+0000 7f39e0a89640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc29a3b1b0 0x55bc29def9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.653+0000 7f39e0a89640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc29a3b1b0 0x55bc29def9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41750/0 (socket says 172.21.15.64:41750) 2024-09-21T04:06:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.653+0000 7f39dbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc29a122f0 0x55bc29df4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.653+0000 7f39e0a89640 1 -- 172.21.15.64:0/1477603632 learned_addr learned my addr 172.21.15.64:0/1477603632 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.654+0000 7f39db7fe640 1 --2- 172.21.15.64:0/1477603632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc29a1a4d0 0x55bc29def2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.654+0000 7f39e0a89640 1 -- 172.21.15.64:0/1477603632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc29a1a4d0 msgr2=0x55bc29def2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.654+0000 7f39e0a89640 1 --2- 172.21.15.64:0/1477603632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc29a1a4d0 0x55bc29def2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.654+0000 7f39e0a89640 1 -- 172.21.15.64:0/1477603632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc29a122f0 msgr2=0x55bc29df4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.654+0000 7f39e0a89640 1 --2- 172.21.15.64:0/1477603632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc29a122f0 0x55bc29df4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.654+0000 7f39e0a89640 1 -- 172.21.15.64:0/1477603632 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc29e66bd0 con 0x55bc29a3b1b0 2024-09-21T04:06:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.654+0000 7f39db7fe640 1 --2- 172.21.15.64:0/1477603632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc29a1a4d0 0x55bc29def2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:06:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.654+0000 7f39e0a89640 1 --2- 172.21.15.64:0/1477603632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc29a3b1b0 0x55bc29def9d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f39d401f6f0 tx=0x7f39d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb08aad81ff2d52 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.654+0000 7f39bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:06:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.654+0000 7f39bb7fe640 1 -- 172.21.15.64:0/1477603632 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d401da10 con 0x55bc29a3b1b0 2024-09-21T04:06:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.654+0000 7f39bb7fe640 1 -- 172.21.15.64:0/1477603632 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f39d401dbb0 con 0x55bc29a3b1b0 2024-09-21T04:06:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.654+0000 7f39e945a740 1 -- 172.21.15.64:0/1477603632 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39d00b2190 con 0x55bc29a3b1b0 2024-09-21T04:06:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.654+0000 7f39bb7fe640 1 -- 172.21.15.64:0/1477603632 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f39d401ddb0 con 0x55bc29a3b1b0 2024-09-21T04:06:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.654+0000 7f39e945a740 10 client.15987 fetch_fsmap learned FSMap version 30 2024-09-21T04:06:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.654+0000 7f39e945a740 1 -- 172.21.15.64:0/1477603632 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bc29e67d10 con 0x55bc29a3b1b0 2024-09-21T04:06:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.655+0000 7f39bb7fe640 1 -- 172.21.15.64:0/1477603632 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39d00b2190 con 0x55bc29a3b1b0 2024-09-21T04:06:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.655+0000 7f39bb7fe640 1 -- 172.21.15.64:0/1477603632 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f39d40295a0 con 0x55bc29a3b1b0 2024-09-21T04:06:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.655+0000 7f39e945a740 10 client.15987 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:06:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.655+0000 7f39e945a740 10 client.15987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:06:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.655+0000 7f39e945a740 20 client.15987 populate_metadata read hostname 'smithi064' 2024-09-21T04:06:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.655+0000 7f39e945a740 1 --2- 172.21.15.64:0/1477603632 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55bc299cf1c0 0x55bc29e6a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.655+0000 7f39e945a740 4 client.15987 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:06:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.655+0000 7f39e945a740 1 -- 172.21.15.64:0/1477603632 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bc29d530f0 con 0x55bc299cf1c0 2024-09-21T04:06:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.655+0000 7f39dbfff640 1 --2- 172.21.15.64:0/1477603632 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55bc299cf1c0 0x55bc29e6a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.655+0000 7f39bb7fe640 10 client.15987 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:12.655+0000 7f39dbfff640 1 --2- 172.21.15.64:0/1477603632 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55bc299cf1c0 0x55bc29e6a560 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc9101f0d5ec44fd server_cookie=2ff19c0e7eb37de in_seq=0 out_seq=0 2024-09-21T04:06:12.965 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.963+0000 7fcc5befd640 1 -- 172.21.15.64:0/2724980984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc54105ad0 msgr2=0x7fcc5410f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:12.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.963+0000 7fcc5befd640 1 --2- 172.21.15.64:0/2724980984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc54105ad0 0x7fcc5410f560 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c009c10 tx=0x7fcc4c02a6e0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.963+0000 7fcc5befd640 1 -- 172.21.15.64:0/2724980984 shutdown_connections 2024-09-21T04:06:12.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.963+0000 7fcc5befd640 1 --2- 172.21.15.64:0/2724980984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc54105ad0 0x7fcc5410f560 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.963+0000 7fcc5befd640 1 --2- 172.21.15.64:0/2724980984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc54105180 0x7fcc54105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.963+0000 7fcc5befd640 1 --2- 172.21.15.64:0/2724980984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc54103100 0x7fcc54103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.963+0000 7fcc5befd640 1 -- 172.21.15.64:0/2724980984 >> 172.21.15.64:0/2724980984 conn(0x7fcc54077c60 msgr2=0x7fcc540780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:12.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.963+0000 7fcc5befd640 1 -- 172.21.15.64:0/2724980984 shutdown_connections 2024-09-21T04:06:12.966 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.963+0000 7fcc5befd640 1 -- 172.21.15.64:0/2724980984 wait complete. 2024-09-21T04:06:12.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.964+0000 7fcc5befd640 1 Processor -- start 2024-09-21T04:06:12.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.964+0000 7fcc5befd640 1 -- start start 2024-09-21T04:06:12.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.964+0000 7fcc5befd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc54103100 0x7fcc5419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.964+0000 7fcc5befd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc54105180 0x7fcc5419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.964+0000 7fcc5befd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc54105ad0 0x7fcc5419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.964+0000 7fcc5befd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcc541129a0 con 0x7fcc54105180 2024-09-21T04:06:12.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.965+0000 7fcc5befd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcc54112820 con 0x7fcc54103100 2024-09-21T04:06:12.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.965+0000 7fcc59c99640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc54103100 0x7fcc5419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.965+0000 7fcc59c99640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc54103100 0x7fcc5419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:48658/0 (socket says 172.21.15.64:48658) 2024-09-21T04:06:12.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.965+0000 7fcc59c99640 1 -- 172.21.15.64:0/265950215 learned_addr learned my addr 172.21.15.64:0/265950215 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:12.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.965+0000 7fcc5a49a640 1 --2- 172.21.15.64:0/265950215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc54105ad0 0x7fcc5419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.965+0000 7fcc5befd640 1 -- 172.21.15.64:0/265950215 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcc54112b20 con 0x7fcc54105ad0 2024-09-21T04:06:12.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.965+0000 7fcc59498640 1 --2- 172.21.15.64:0/265950215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc54105180 0x7fcc5419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.965+0000 7fcc59498640 1 -- 172.21.15.64:0/265950215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc54105ad0 msgr2=0x7fcc5419fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:12.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.965+0000 7fcc59498640 1 --2- 172.21.15.64:0/265950215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc54105ad0 0x7fcc5419fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.965+0000 7fcc59498640 1 -- 172.21.15.64:0/265950215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc54103100 msgr2=0x7fcc5419a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:12.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.965+0000 7fcc59498640 1 --2- 172.21.15.64:0/265950215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc54103100 0x7fcc5419a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.965+0000 7fcc59498640 1 -- 172.21.15.64:0/265950215 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc541a03b0 con 0x7fcc54105180 2024-09-21T04:06:12.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.966+0000 7fcc59498640 1 --2- 172.21.15.64:0/265950215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc54105180 0x7fcc5419a9e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fcc54075820 tx=0x7fcc4400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:12.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.966+0000 7fcc4affd640 1 -- 172.21.15.64:0/265950215 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4400ccd0 con 0x7fcc54105180 2024-09-21T04:06:12.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.966+0000 7fcc4affd640 1 -- 172.21.15.64:0/265950215 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc440035f0 con 0x7fcc54105180 2024-09-21T04:06:12.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.966+0000 7fcc5befd640 1 -- 172.21.15.64:0/265950215 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc541a06a0 con 0x7fcc54105180 2024-09-21T04:06:12.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.966+0000 7fcc4affd640 1 -- 172.21.15.64:0/265950215 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44010690 con 0x7fcc54105180 2024-09-21T04:06:12.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.966+0000 7fcc5befd640 1 -- 172.21.15.64:0/265950215 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc541a6e00 con 0x7fcc54105180 2024-09-21T04:06:12.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.969+0000 7fcc62471740 1 Processor -- start 2024-09-21T04:06:12.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.969+0000 7fcc62471740 1 -- start start 2024-09-21T04:06:12.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.969+0000 7fcc62471740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a001a58070 0x55a001dff650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.969+0000 7fcc62471740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a001a2f1b0 0x55a001dffb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.969+0000 7fcc62471740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a001a37390 0x55a001e000d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.969+0000 7fcc62471740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a001d75e50 con 0x55a001a2f1b0 2024-09-21T04:06:12.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.969+0000 7fcc62471740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a001d65280 con 0x55a001a58070 2024-09-21T04:06:12.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.969+0000 7fcc62471740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a001d69090 con 0x55a001a37390 2024-09-21T04:06:12.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.969+0000 7fcc5a49a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a001a58070 0x55a001dff650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.970+0000 7fcc59c99640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a001a2f1b0 0x55a001dffb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.970+0000 7fcc5a49a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a001a58070 0x55a001dff650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:48672/0 (socket says 172.21.15.64:48672) 2024-09-21T04:06:12.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.970+0000 7fcc5a49a640 1 -- 172.21.15.64:0/2285679249 learned_addr learned my addr 172.21.15.64:0/2285679249 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:12.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.970+0000 7fcc59c99640 1 -- 172.21.15.64:0/2285679249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a001a37390 msgr2=0x55a001e000d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:06:12.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.970+0000 7fcc59c99640 1 --2- 172.21.15.64:0/2285679249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a001a37390 0x55a001e000d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.970+0000 7fcc59c99640 1 -- 172.21.15.64:0/2285679249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a001a58070 msgr2=0x55a001dff650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:12.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.970+0000 7fcc59c99640 1 --2- 172.21.15.64:0/2285679249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a001a58070 0x55a001dff650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.970+0000 7fcc59c99640 1 -- 172.21.15.64:0/2285679249 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a001ac3650 con 0x55a001a2f1b0 2024-09-21T04:06:12.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.970+0000 7fcc59c99640 1 --2- 172.21.15.64:0/2285679249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a001a2f1b0 0x55a001dffb90 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fcc50009c50 tx=0x7fcc50013ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516181f0ebaf3cbd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:12.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.970+0000 7fcc4affd640 1 -- 172.21.15.64:0/265950215 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fcc44003aa0 con 0x7fcc54105180 2024-09-21T04:06:12.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.971+0000 7fcc3ffff640 1 -- 172.21.15.64:0/2285679249 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5000e990 con 0x55a001a2f1b0 2024-09-21T04:06:12.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.971+0000 7fcc3ffff640 1 -- 172.21.15.64:0/2285679249 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc50005020 con 0x55a001a2f1b0 2024-09-21T04:06:12.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.971+0000 7fcc3ffff640 1 -- 172.21.15.64:0/2285679249 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc500049e0 con 0x55a001a2f1b0 2024-09-21T04:06:12.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.971+0000 7fcc4affd640 1 --2- 172.21.15.64:0/265950215 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcc400b6700 0x7fcc400b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.971+0000 7fcc5a49a640 1 --2- 172.21.15.64:0/265950215 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcc400b6700 0x7fcc400b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.972+0000 7fcc4affd640 1 -- 172.21.15.64:0/265950215 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fcc440e78d0 con 0x7fcc54105180 2024-09-21T04:06:12.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.972+0000 7fcc5a49a640 1 --2- 172.21.15.64:0/265950215 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcc400b6700 0x7fcc400b8c10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c0099a0 tx=0x7fcc4c004a40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:12.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.972+0000 7fcc62471740 1 -- 172.21.15.64:0/2285679249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a001a2f1b0 msgr2=0x55a001dffb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:12.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.972+0000 7fcc62471740 1 --2- 172.21.15.64:0/2285679249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a001a2f1b0 0x55a001dffb90 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fcc50009c50 tx=0x7fcc50013ba0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.972+0000 7fcc62471740 1 -- 172.21.15.64:0/2285679249 shutdown_connections 2024-09-21T04:06:12.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.972+0000 7fcc62471740 1 --2- 172.21.15.64:0/2285679249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a001a37390 0x55a001e000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.972+0000 7fcc62471740 1 --2- 172.21.15.64:0/2285679249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a001a2f1b0 0x55a001dffb90 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.972+0000 7fcc62471740 1 --2- 172.21.15.64:0/2285679249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a001a58070 0x55a001dff650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.972+0000 7fcc62471740 1 -- 172.21.15.64:0/2285679249 >> 172.21.15.64:0/2285679249 conn(0x55a001a4c460 msgr2=0x55a001dfd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:12.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.972+0000 7fcc62471740 1 -- 172.21.15.64:0/2285679249 shutdown_connections 2024-09-21T04:06:12.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.972+0000 7fcc62471740 1 -- 172.21.15.64:0/2285679249 wait complete. 2024-09-21T04:06:12.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.973+0000 7fcc62471740 1 -- start start 2024-09-21T04:06:12.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.973+0000 7fcc62471740 1 Processor -- start 2024-09-21T04:06:12.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.974+0000 7fcc62471740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a001a2f1b0 0x55a001e0ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.974+0000 7fcc62471740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a001a37390 0x55a001e0b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.974+0000 7fcc62471740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a001a58070 0x55a001e171a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.974+0000 7fcc62471740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a001bc6700 con 0x55a001a2f1b0 2024-09-21T04:06:12.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.974+0000 7fcc62471740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a001d00110 con 0x55a001a58070 2024-09-21T04:06:12.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.974+0000 7fcc62471740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a001e0de80 con 0x55a001a37390 2024-09-21T04:06:12.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.974+0000 7fcc62471740 10 client.0 _ll_register_callbacks cb 0x55a001d19990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:06:12.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.974+0000 7fcc5a49a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a001a58070 0x55a001e171a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.974+0000 7fcc59c99640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a001a2f1b0 0x55a001e0ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.974+0000 7fcc5a49a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a001a58070 0x55a001e171a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:48680/0 (socket says 172.21.15.64:48680) 2024-09-21T04:06:12.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.974+0000 7fcc5a49a640 1 -- 172.21.15.64:0/232638219 learned_addr learned my addr 172.21.15.64:0/232638219 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:12.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.974+0000 7fcc59498640 1 --2- 172.21.15.64:0/232638219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a001a37390 0x55a001e0b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.975+0000 7fcc59c99640 1 -- 172.21.15.64:0/232638219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a001a37390 msgr2=0x55a001e0b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:12.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.975+0000 7fcc59c99640 1 --2- 172.21.15.64:0/232638219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a001a37390 0x55a001e0b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.975+0000 7fcc59c99640 1 -- 172.21.15.64:0/232638219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a001a58070 msgr2=0x55a001e171a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:12.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.975+0000 7fcc59c99640 1 --2- 172.21.15.64:0/232638219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a001a58070 0x55a001e171a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.975+0000 7fcc59c99640 1 -- 172.21.15.64:0/232638219 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a001d2dc40 con 0x55a001a2f1b0 2024-09-21T04:06:12.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.975+0000 7fcc5a49a640 1 --2- 172.21.15.64:0/232638219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a001a58070 0x55a001e171a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:06:12.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.975+0000 7fcc59c99640 1 --2- 172.21.15.64:0/232638219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a001a2f1b0 0x55a001e0ad20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fcc5001e040 tx=0x7fcc5000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5a23ec467283c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:12.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.975+0000 7fcc3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:06:12.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.975+0000 7fcc3d7fa640 1 -- 172.21.15.64:0/232638219 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5002a030 con 0x55a001a2f1b0 2024-09-21T04:06:12.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.975+0000 7fcc62471740 1 -- 172.21.15.64:0/232638219 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc50005020 con 0x55a001a2f1b0 2024-09-21T04:06:12.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.975+0000 7fcc3d7fa640 1 -- 172.21.15.64:0/232638219 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fcc50019040 con 0x55a001a2f1b0 2024-09-21T04:06:12.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.976+0000 7fcc3d7fa640 1 -- 172.21.15.64:0/232638219 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcc50034050 con 0x55a001a2f1b0 2024-09-21T04:06:12.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.976+0000 7fcc62471740 10 client.15999 fetch_fsmap learned FSMap version 30 2024-09-21T04:06:12.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.976+0000 7fcc62471740 1 -- 172.21.15.64:0/232638219 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a001e19b80 con 0x55a001a2f1b0 2024-09-21T04:06:12.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.976+0000 7fcc3d7fa640 1 -- 172.21.15.64:0/232638219 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc50005020 con 0x55a001a2f1b0 2024-09-21T04:06:12.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.976+0000 7fcc3d7fa640 1 -- 172.21.15.64:0/232638219 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fcc50004c20 con 0x55a001a2f1b0 2024-09-21T04:06:12.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.976+0000 7fcc62471740 10 client.15999 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:06:12.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.976+0000 7fcc62471740 10 client.15999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:06:12.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.976+0000 7fcc62471740 20 client.15999 populate_metadata read hostname 'smithi064' 2024-09-21T04:06:12.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.976+0000 7fcc62471740 1 --2- 172.21.15.64:0/232638219 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a001a169f0 0x55a001e1d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:12.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.976+0000 7fcc62471740 4 client.15999 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:06:12.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.976+0000 7fcc62471740 1 -- 172.21.15.64:0/232638219 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a001d77da0 con 0x55a001a169f0 2024-09-21T04:06:12.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.977+0000 7fcc59498640 1 --2- 172.21.15.64:0/232638219 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a001a169f0 0x55a001e1d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:12.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.977+0000 7fcc59498640 1 --2- 172.21.15.64:0/232638219 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a001a169f0 0x55a001e1d3d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eeb1b70bf012308 server_cookie=f85f4c6d8fabf10e in_seq=0 out_seq=0 2024-09-21T04:06:12.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.977+0000 7fcc3d7fa640 10 client.15999 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:12.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.978+0000 7fcc3d7fa640 1 -- 172.21.15.64:0/232638219 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a001d77da0 con 0x55a001a169f0 2024-09-21T04:06:12.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.978+0000 7fcc3d7fa640 10 client.15999 handle_command_reply: tid=0 2024-09-21T04:06:12.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.978+0000 7fcc62471740 1 client.15999 shutdown 2024-09-21T04:06:12.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.978+0000 7fcc62471740 1 -- 172.21.15.64:0/232638219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a001a2f1b0 msgr2=0x55a001e0ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:12.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.978+0000 7fcc62471740 1 --2- 172.21.15.64:0/232638219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a001a2f1b0 0x55a001e0ad20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fcc5001e040 tx=0x7fcc5000ec40 comp rx=0 tx=0).stop 2024-09-21T04:06:12.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.978+0000 7fcc62471740 1 -- 172.21.15.64:0/232638219 shutdown_connections 2024-09-21T04:06:12.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.978+0000 7fcc62471740 1 --2- 172.21.15.64:0/232638219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a001a58070 0x55a001e171a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.978+0000 7fcc62471740 1 --2- 172.21.15.64:0/232638219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a001a37390 0x55a001e0b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.978+0000 7fcc62471740 1 --2- 172.21.15.64:0/232638219 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a001a169f0 0x55a001e1d3d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.978+0000 7fcc62471740 1 --2- 172.21.15.64:0/232638219 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a001a2f1b0 0x55a001e0ad20 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:12.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.978+0000 7fcc62471740 1 -- 172.21.15.64:0/232638219 >> 172.21.15.64:0/232638219 conn(0x55a001a4c460 msgr2=0x55a001e01040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:12.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.978+0000 7fcc3d7fa640 0 client.15999 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:12.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.978+0000 7fcc62471740 1 -- 172.21.15.64:0/232638219 shutdown_connections 2024-09-21T04:06:12.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.978+0000 7fcc62471740 1 -- 172.21.15.64:0/232638219 wait complete. 2024-09-21T04:06:12.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.979+0000 7fcc62471740 20 client.15999 trim_cache size 0 max 16384 2024-09-21T04:06:13.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.998+0000 7fcc62471740 1 Processor -- start 2024-09-21T04:06:13.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.998+0000 7fcc62471740 1 -- start start 2024-09-21T04:06:13.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.998+0000 7fcc62471740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a001a2f1b0 0x55a001e00d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:13.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.998+0000 7fcc62471740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a001a37390 0x55a001e07f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:13.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.998+0000 7fcc62471740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a001a58070 0x55a001e08450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:13.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.998+0000 7fcc62471740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcc541b0090 con 0x55a001a2f1b0 2024-09-21T04:06:13.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.998+0000 7fcc62471740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcc541ad8d0 con 0x55a001a58070 2024-09-21T04:06:13.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.998+0000 7fcc62471740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcc541ab720 con 0x55a001a37390 2024-09-21T04:06:13.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.998+0000 7fcc59c99640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a001a2f1b0 0x55a001e00d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:13.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.998+0000 7fcc59c99640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a001a2f1b0 0x55a001e00d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41794/0 (socket says 172.21.15.64:41794) 2024-09-21T04:06:13.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.998+0000 7fcc59c99640 1 -- 172.21.15.64:0/2550314524 learned_addr learned my addr 172.21.15.64:0/2550314524 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:13.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.998+0000 7fcc59498640 1 --2- 172.21.15.64:0/2550314524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a001a37390 0x55a001e07f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:13.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.998+0000 7fcc5a49a640 1 --2- 172.21.15.64:0/2550314524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a001a58070 0x55a001e08450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:13.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.999+0000 7fcc59498640 1 -- 172.21.15.64:0/2550314524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a001a58070 msgr2=0x55a001e08450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:13.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.999+0000 7fcc59498640 1 --2- 172.21.15.64:0/2550314524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a001a58070 0x55a001e08450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:13.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.999+0000 7fcc59498640 1 -- 172.21.15.64:0/2550314524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a001a2f1b0 msgr2=0x55a001e00d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:13.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.999+0000 7fcc59498640 1 --2- 172.21.15.64:0/2550314524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a001a2f1b0 0x55a001e00d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:13.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.999+0000 7fcc59498640 1 -- 172.21.15.64:0/2550314524 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a001e0bfe0 con 0x55a001a37390 2024-09-21T04:06:13.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.999+0000 7fcc5a49a640 1 --2- 172.21.15.64:0/2550314524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a001a58070 0x55a001e08450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:06:13.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.999+0000 7fcc59498640 1 --2- 172.21.15.64:0/2550314524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a001a37390 0x55a001e07f10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcc440e4f90 tx=0x7fcc44003e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d4138eaf4fb66a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:13.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.999+0000 7fcc3f7fe640 1 -- 172.21.15.64:0/2550314524 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc440add90 con 0x55a001a37390 2024-09-21T04:06:13.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.999+0000 7fcc3f7fe640 1 -- 172.21.15.64:0/2550314524 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc440b2170 con 0x55a001a37390 2024-09-21T04:06:13.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:12.999+0000 7fcc3f7fe640 1 -- 172.21.15.64:0/2550314524 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc440ac360 con 0x55a001a37390 2024-09-21T04:06:13.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.000+0000 7fcc62471740 1 -- 172.21.15.64:0/2550314524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a001a37390 msgr2=0x55a001e07f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:13.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.000+0000 7fcc62471740 1 --2- 172.21.15.64:0/2550314524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a001a37390 0x55a001e07f10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcc440e4f90 tx=0x7fcc44003e40 comp rx=0 tx=0).stop 2024-09-21T04:06:13.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.000+0000 7fcc62471740 1 -- 172.21.15.64:0/2550314524 shutdown_connections 2024-09-21T04:06:13.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.000+0000 7fcc62471740 1 --2- 172.21.15.64:0/2550314524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a001a58070 0x55a001e08450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:13.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.000+0000 7fcc62471740 1 --2- 172.21.15.64:0/2550314524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a001a37390 0x55a001e07f10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:13.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.000+0000 7fcc62471740 1 --2- 172.21.15.64:0/2550314524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a001a2f1b0 0x55a001e00d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:13.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.000+0000 7fcc62471740 1 -- 172.21.15.64:0/2550314524 >> 172.21.15.64:0/2550314524 conn(0x55a001a4c460 msgr2=0x55a001e00680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:13.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.000+0000 7fcc62471740 1 -- 172.21.15.64:0/2550314524 shutdown_connections 2024-09-21T04:06:13.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.000+0000 7fcc62471740 1 -- 172.21.15.64:0/2550314524 wait complete. 2024-09-21T04:06:13.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.001+0000 7fcc62471740 1 -- start start 2024-09-21T04:06:13.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.001+0000 7fcc62471740 1 Processor -- start 2024-09-21T04:06:13.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.002+0000 7fcc62471740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a001a2f1b0 0x55a001e10a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:13.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.002+0000 7fcc62471740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a001a37390 0x55a001e10fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:13.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.002+0000 7fcc62471740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a001a58070 0x55a001e7fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:13.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.002+0000 7fcc62471740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcc541abd90 con 0x55a001a58070 2024-09-21T04:06:13.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.002+0000 7fcc62471740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcc541ad200 con 0x55a001a2f1b0 2024-09-21T04:06:13.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.002+0000 7fcc62471740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcc541ac460 con 0x55a001a37390 2024-09-21T04:06:13.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.002+0000 7fcc62471740 10 client.0 _ll_register_callbacks cb 0x7fcc50005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:06:13.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.002+0000 7fcc59c99640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a001a2f1b0 0x55a001e10a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:13.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.002+0000 7fcc5a49a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a001a58070 0x55a001e7fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:13.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.002+0000 7fcc59c99640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a001a2f1b0 0x55a001e10a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:48700/0 (socket says 172.21.15.64:48700) 2024-09-21T04:06:13.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.002+0000 7fcc59c99640 1 -- 172.21.15.64:0/679583120 learned_addr learned my addr 172.21.15.64:0/679583120 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:13.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.002+0000 7fcc59c99640 1 -- 172.21.15.64:0/679583120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a001a37390 msgr2=0x55a001e10fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:13.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.002+0000 7fcc59498640 1 --2- 172.21.15.64:0/679583120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a001a37390 0x55a001e10fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:13.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.002+0000 7fcc59c99640 1 --2- 172.21.15.64:0/679583120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a001a37390 0x55a001e10fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:13.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.002+0000 7fcc59c99640 1 -- 172.21.15.64:0/679583120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a001a58070 msgr2=0x55a001e7fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:13.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.002+0000 7fcc59c99640 1 --2- 172.21.15.64:0/679583120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a001a58070 0x55a001e7fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:13.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.002+0000 7fcc59c99640 1 -- 172.21.15.64:0/679583120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a001e80410 con 0x55a001a2f1b0 2024-09-21T04:06:13.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.003+0000 7fcc59498640 1 --2- 172.21.15.64:0/679583120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a001a37390 0x55a001e10fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:06:13.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.003+0000 7fcc59c99640 1 --2- 172.21.15.64:0/679583120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a001a2f1b0 0x55a001e10a00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcc50013be0 tx=0x7fcc50005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8039e3ffb069f56d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:13.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.003+0000 7fcc3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:06:13.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.003+0000 7fcc3d7fa640 1 -- 172.21.15.64:0/679583120 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5003d060 con 0x55a001a2f1b0 2024-09-21T04:06:13.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.003+0000 7fcc62471740 1 -- 172.21.15.64:0/679583120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc440b2170 con 0x55a001a2f1b0 2024-09-21T04:06:13.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.003+0000 7fcc3d7fa640 1 -- 172.21.15.64:0/679583120 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fcc500054e0 con 0x55a001a2f1b0 2024-09-21T04:06:13.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.003+0000 7fcc3d7fa640 1 -- 172.21.15.64:0/679583120 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcc5002a040 con 0x55a001a2f1b0 2024-09-21T04:06:13.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.003+0000 7fcc62471740 10 client.25715 fetch_fsmap learned FSMap version 30 2024-09-21T04:06:13.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.003+0000 7fcc62471740 1 -- 172.21.15.64:0/679583120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a001e87720 con 0x55a001a2f1b0 2024-09-21T04:06:13.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.004+0000 7fcc3d7fa640 1 -- 172.21.15.64:0/679583120 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc440b2170 con 0x55a001a2f1b0 2024-09-21T04:06:13.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.004+0000 7fcc3d7fa640 1 -- 172.21.15.64:0/679583120 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fcc50003900 con 0x55a001a2f1b0 2024-09-21T04:06:13.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.004+0000 7fcc62471740 10 client.25715 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:06:13.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.004+0000 7fcc62471740 10 client.25715 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:06:13.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.004+0000 7fcc62471740 20 client.25715 populate_metadata read hostname 'smithi064' 2024-09-21T04:06:13.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.004+0000 7fcc62471740 1 --2- 172.21.15.64:0/679583120 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a0019ebda0 0x55a001e89f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:13.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.004+0000 7fcc62471740 4 client.25715 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T04:06:13.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.004+0000 7fcc62471740 1 -- 172.21.15.64:0/679583120 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a001d73850 con 0x55a0019ebda0 2024-09-21T04:06:13.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.004+0000 7fcc59498640 1 --2- 172.21.15.64:0/679583120 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a0019ebda0 0x55a001e89f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:13.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.005+0000 7fcc3d7fa640 10 client.25715 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:13.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.005+0000 7fcc59498640 1 --2- 172.21.15.64:0/679583120 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a0019ebda0 0x55a001e89f10 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=817d5ee095ceac0 server_cookie=9205f27240cb0ded in_seq=0 out_seq=0 2024-09-21T04:06:13.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.005+0000 7fcc3d7fa640 1 -- 172.21.15.64:0/679583120 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1226 (crc 0 0 0) 0x55a001d73850 con 0x55a0019ebda0 2024-09-21T04:06:13.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.006+0000 7fcc3d7fa640 10 client.25715 handle_command_reply: tid=0 2024-09-21T04:06:13.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.006+0000 7fcc62471740 1 client.25715 shutdown 2024-09-21T04:06:13.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.006+0000 7fcc62471740 1 -- 172.21.15.64:0/679583120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a001a2f1b0 msgr2=0x55a001e10a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:13.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.006+0000 7fcc62471740 1 --2- 172.21.15.64:0/679583120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a001a2f1b0 0x55a001e10a00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcc50013be0 tx=0x7fcc50005f10 comp rx=0 tx=0).stop 2024-09-21T04:06:13.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.006+0000 7fcc62471740 1 -- 172.21.15.64:0/679583120 shutdown_connections 2024-09-21T04:06:13.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.006+0000 7fcc62471740 1 --2- 172.21.15.64:0/679583120 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a0019ebda0 0x55a001e89f10 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:13.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.006+0000 7fcc62471740 1 --2- 172.21.15.64:0/679583120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a001a58070 0x55a001e7fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:13.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.006+0000 7fcc62471740 1 --2- 172.21.15.64:0/679583120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a001a37390 0x55a001e10fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:13.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.006+0000 7fcc62471740 1 --2- 172.21.15.64:0/679583120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a001a2f1b0 0x55a001e10a00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:13.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.006+0000 7fcc62471740 1 -- 172.21.15.64:0/679583120 >> 172.21.15.64:0/679583120 conn(0x55a001a4c460 msgr2=0x55a001e0c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:13.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.006+0000 7fcc3d7fa640 0 client.25715 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:13.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.006+0000 7fcc62471740 1 -- 172.21.15.64:0/679583120 shutdown_connections 2024-09-21T04:06:13.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.006+0000 7fcc62471740 1 -- 172.21.15.64:0/679583120 wait complete. 2024-09-21T04:06:13.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.006+0000 7fcc62471740 20 client.25715 trim_cache size 0 max 16384 2024-09-21T04:06:13.012 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:06:13.012 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-21T04:06:13.012 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {}, 2024-09-21T04:06:13.012 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_stats": { 2024-09-21T04:06:13.012 INFO:teuthology.orchestra.run.smithi064.stdout: "e9242f88-6599-40eb-adf1-bdb3e946a1e0": { 2024-09-21T04:06:13.012 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:05:39.744646+0000", 2024-09-21T04:06:13.012 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:06:13.013 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:06:13.013 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:06:13.013 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:06:13.013 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:06:13.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:06:13.013 INFO:teuthology.orchestra.run.smithi064.stdout: "b52b950c-0f97-49ec-a37c-1a8e04348ce8": { 2024-09-21T04:06:13.013 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:54.003800+0000", 2024-09-21T04:06:13.013 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:06:13.013 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:06:13.013 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:06:13.013 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:06:13.013 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:06:13.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:06:13.013 INFO:teuthology.orchestra.run.smithi064.stdout: "0ab717a7-36c6-4dca-94e7-af0bdf2aa112": { 2024-09-21T04:06:13.013 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:42.691086+0000", 2024-09-21T04:06:13.014 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:06:13.014 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:06:13.014 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:06:13.014 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:06:13.014 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:06:13.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:06:13.014 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:06:13.014 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:03:59.570174+0000", 2024-09-21T04:06:13.014 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:06:13.014 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:06:13.014 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:06:13.014 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:06:13.014 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:06:13.014 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:06:13.015 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:06:13.015 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:06:13.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.007+0000 7fcc5befd640 1 -- 172.21.15.64:0/265950215 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcc400b6700 msgr2=0x7fcc400b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:13.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.007+0000 7fcc5befd640 1 --2- 172.21.15.64:0/265950215 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcc400b6700 0x7fcc400b8c10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c0099a0 tx=0x7fcc4c004a40 comp rx=0 tx=0).stop 2024-09-21T04:06:13.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.007+0000 7fcc5befd640 1 -- 172.21.15.64:0/265950215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc54105180 msgr2=0x7fcc5419a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:13.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.007+0000 7fcc5befd640 1 --2- 172.21.15.64:0/265950215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc54105180 0x7fcc5419a9e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fcc54075820 tx=0x7fcc4400ee50 comp rx=0 tx=0).stop 2024-09-21T04:06:13.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.008+0000 7fcc5befd640 1 -- 172.21.15.64:0/265950215 shutdown_connections 2024-09-21T04:06:13.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.008+0000 7fcc5befd640 1 --2- 172.21.15.64:0/265950215 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcc400b6700 0x7fcc400b8c10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:13.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.008+0000 7fcc5befd640 1 --2- 172.21.15.64:0/265950215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc54105ad0 0x7fcc5419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:13.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.008+0000 7fcc5befd640 1 --2- 172.21.15.64:0/265950215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcc54105180 0x7fcc5419a9e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:13.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.008+0000 7fcc5befd640 1 --2- 172.21.15.64:0/265950215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc54103100 0x7fcc5419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:13.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.008+0000 7fcc5befd640 1 -- 172.21.15.64:0/265950215 >> 172.21.15.64:0/265950215 conn(0x7fcc54077c60 msgr2=0x7fcc541086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:13.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.008+0000 7fcc5befd640 1 -- 172.21.15.64:0/265950215 shutdown_connections 2024-09-21T04:06:13.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:13.008+0000 7fcc5befd640 1 -- 172.21.15.64:0/265950215 wait complete. 2024-09-21T04:06:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:13.340+0000 7f39bb7fe640 1 -- 172.21.15.64:0/1477603632 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bc29d530f0 con 0x55bc299cf1c0 2024-09-21T04:06:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:13.340+0000 7f39bb7fe640 10 client.15987 handle_command_reply: tid=0 2024-09-21T04:06:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:13.340+0000 7f39e945a740 1 client.15987 shutdown 2024-09-21T04:06:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:13.342+0000 7f39e945a740 1 -- 172.21.15.64:0/1477603632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc29a3b1b0 msgr2=0x55bc29def9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:13.342+0000 7f39e945a740 1 --2- 172.21.15.64:0/1477603632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc29a3b1b0 0x55bc29def9d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f39d401f6f0 tx=0x7f39d4004a40 comp rx=0 tx=0).stop 2024-09-21T04:06:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:06:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:06:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:06:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:06:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:13.345+0000 7f39e945a740 1 -- 172.21.15.64:0/1477603632 shutdown_connections 2024-09-21T04:06:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:13.345+0000 7f39e945a740 1 --2- 172.21.15.64:0/1477603632 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55bc299cf1c0 0x55bc29e6a560 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:13.345+0000 7f39e945a740 1 --2- 172.21.15.64:0/1477603632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bc29a3b1b0 0x55bc29def9d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:13.345+0000 7f39e945a740 1 --2- 172.21.15.64:0/1477603632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bc29a1a4d0 0x55bc29def2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:13.345+0000 7f39e945a740 1 --2- 172.21.15.64:0/1477603632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bc29a122f0 0x55bc29df4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:13.346+0000 7f39e945a740 1 -- 172.21.15.64:0/1477603632 >> 172.21.15.64:0/1477603632 conn(0x55bc29a2f300 msgr2=0x55bc29deac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:13.346+0000 7f39bb7fe640 0 client.15987 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:13.346+0000 7f39e945a740 1 -- 172.21.15.64:0/1477603632 shutdown_connections 2024-09-21T04:06:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:13.346+0000 7f39e945a740 1 -- 172.21.15.64:0/1477603632 wait complete. 2024-09-21T04:06:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:13.346+0000 7f39e945a740 20 client.15987 trim_cache size 0 max 16384 2024-09-21T04:06:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:13.367+0000 7f39e24ec640 1 -- 172.21.15.64:0/874509610 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f39c40b6680 msgr2=0x7f39c40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:13.367+0000 7f39e24ec640 1 --2- 172.21.15.64:0/874509610 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f39c40b6680 0x7f39c40b8b90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f39d4005cc0 tx=0x7f39d40084f0 comp rx=0 tx=0).stop 2024-09-21T04:06:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:13.367+0000 7f39e24ec640 1 -- 172.21.15.64:0/874509610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39dc105270 msgr2=0x7f39dc19aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:13.367+0000 7f39e24ec640 1 --2- 172.21.15.64:0/874509610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39dc105270 0x7f39dc19aaa0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f39d000e9f0 tx=0x7f39d000eeb0 comp rx=0 tx=0).stop 2024-09-21T04:06:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:13.368+0000 7f39e24ec640 1 -- 172.21.15.64:0/874509610 shutdown_connections 2024-09-21T04:06:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:13.368+0000 7f39e24ec640 1 --2- 172.21.15.64:0/874509610 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f39c40b6680 0x7f39c40b8b90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:13.368+0000 7f39e24ec640 1 --2- 172.21.15.64:0/874509610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39dc10dd80 0x7f39dc19fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:13.368+0000 7f39e24ec640 1 --2- 172.21.15.64:0/874509610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39dc105270 0x7f39dc19aaa0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:13.368+0000 7f39e24ec640 1 --2- 172.21.15.64:0/874509610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39dc104920 0x7f39dc19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:13.368+0000 7f39e24ec640 1 -- 172.21.15.64:0/874509610 >> 172.21.15.64:0/874509610 conn(0x7f39dc0fe3f0 msgr2=0x7f39dc107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:13.368+0000 7f39e24ec640 1 -- 172.21.15.64:0/874509610 shutdown_connections 2024-09-21T04:06:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:13.368+0000 7f39e24ec640 1 -- 172.21.15.64:0/874509610 wait complete. 2024-09-21T04:06:13.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:13 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:06:13.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:13 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:06:13.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:13 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:06:13.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:13 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:06:13.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:13 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:06:13.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:13 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:06:13.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:13 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:06:13.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:13 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:06:13.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:13 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:06:13.952 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T04:06:13.952 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:06:14.286 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:06:14.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:14 smithi119 ceph-mon[34811]: pgmap v387: 97 pgs: 97 active+clean; 190 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 10 MiB/s wr, 819 op/s 2024-09-21T04:06:14.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:14 smithi064 ceph-mon[31180]: pgmap v387: 97 pgs: 97 active+clean; 190 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 10 MiB/s wr, 819 op/s 2024-09-21T04:06:14.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:14 smithi131 ceph-mon[35193]: pgmap v387: 97 pgs: 97 active+clean; 190 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 10 MiB/s wr, 819 op/s 2024-09-21T04:06:14.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.819+0000 7f7c1930b640 1 -- 172.21.15.64:0/295559612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c14109e00 msgr2=0x7f7c14110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:14.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.819+0000 7f7c1930b640 1 --2- 172.21.15.64:0/295559612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c14109e00 0x7f7c14110630 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009ca0 tx=0x7f7c0c02a8c0 comp rx=0 tx=0).stop 2024-09-21T04:06:14.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.820+0000 7f7c1930b640 1 -- 172.21.15.64:0/295559612 shutdown_connections 2024-09-21T04:06:14.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.820+0000 7f7c1930b640 1 --2- 172.21.15.64:0/295559612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c14109e00 0x7f7c14110630 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:14.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.820+0000 7f7c1930b640 1 --2- 172.21.15.64:0/295559612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c141094b0 0x7f7c141098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:14.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.820+0000 7f7c1930b640 1 --2- 172.21.15.64:0/295559612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c14103bc0 0x7f7c14103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:14.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.820+0000 7f7c1930b640 1 -- 172.21.15.64:0/295559612 >> 172.21.15.64:0/295559612 conn(0x7f7c1407b910 msgr2=0x7f7c1407bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:14.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.820+0000 7f7c1930b640 1 -- 172.21.15.64:0/295559612 shutdown_connections 2024-09-21T04:06:14.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.820+0000 7f7c1930b640 1 -- 172.21.15.64:0/295559612 wait complete. 2024-09-21T04:06:14.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.821+0000 7f7c1930b640 1 Processor -- start 2024-09-21T04:06:14.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.821+0000 7f7c1930b640 1 -- start start 2024-09-21T04:06:14.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.822+0000 7f7c1930b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c14103bc0 0x7f7c1419c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:14.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.822+0000 7f7c1930b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c141094b0 0x7f7c1419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:14.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.822+0000 7f7c1930b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c14109e00 0x7f7c141a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:14.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.822+0000 7f7c1930b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7c14113ac0 con 0x7f7c14109e00 2024-09-21T04:06:14.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.822+0000 7f7c1930b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7c14113940 con 0x7f7c14103bc0 2024-09-21T04:06:14.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.822+0000 7f7c1930b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7c14113c40 con 0x7f7c141094b0 2024-09-21T04:06:14.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.822+0000 7f7c12ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c14103bc0 0x7f7c1419c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:14.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.822+0000 7f7c12ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c14103bc0 0x7f7c1419c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:48718/0 (socket says 172.21.15.64:48718) 2024-09-21T04:06:14.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.822+0000 7f7c12ffd640 1 -- 172.21.15.64:0/2356849756 learned_addr learned my addr 172.21.15.64:0/2356849756 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:14.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.822+0000 7f7c127fc640 1 --2- 172.21.15.64:0/2356849756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c141094b0 0x7f7c1419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:14.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.822+0000 7f7c137fe640 1 --2- 172.21.15.64:0/2356849756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c14109e00 0x7f7c141a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:14.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.822+0000 7f7c12ffd640 1 -- 172.21.15.64:0/2356849756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c141094b0 msgr2=0x7f7c1419cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:14.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.822+0000 7f7c12ffd640 1 --2- 172.21.15.64:0/2356849756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c141094b0 0x7f7c1419cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:14.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.822+0000 7f7c12ffd640 1 -- 172.21.15.64:0/2356849756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c14109e00 msgr2=0x7f7c141a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:14.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.822+0000 7f7c12ffd640 1 --2- 172.21.15.64:0/2356849756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c14109e00 0x7f7c141a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:14.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.822+0000 7f7c12ffd640 1 -- 172.21.15.64:0/2356849756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c141a2640 con 0x7f7c14103bc0 2024-09-21T04:06:14.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.823+0000 7f7c12ffd640 1 --2- 172.21.15.64:0/2356849756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c14103bc0 0x7f7c1419c6d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400e9f0 tx=0x7f7c0400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.823+0000 7f7bfbfff640 1 -- 172.21.15.64:0/2356849756 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0400cd80 con 0x7f7c14103bc0 2024-09-21T04:06:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.823+0000 7f7c1930b640 1 -- 172.21.15.64:0/2356849756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c141a2930 con 0x7f7c14103bc0 2024-09-21T04:06:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.823+0000 7f7bfbfff640 1 -- 172.21.15.64:0/2356849756 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c040108e0 con 0x7f7c14103bc0 2024-09-21T04:06:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.823+0000 7f7bfbfff640 1 -- 172.21.15.64:0/2356849756 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04018980 con 0x7f7c14103bc0 2024-09-21T04:06:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.823+0000 7f7c1930b640 1 -- 172.21.15.64:0/2356849756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c141a9090 con 0x7f7c14103bc0 2024-09-21T04:06:14.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.824+0000 7f7c1930b640 1 -- 172.21.15.64:0/2356849756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c14103fd0 con 0x7f7c14103bc0 2024-09-21T04:06:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.827+0000 7f7bfbfff640 1 -- 172.21.15.64:0/2356849756 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f7c04010430 con 0x7f7c14103bc0 2024-09-21T04:06:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.828+0000 7f7bfbfff640 1 --2- 172.21.15.64:0/2356849756 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7bfc0b6680 0x7f7bfc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.828+0000 7f7c127fc640 1 --2- 172.21.15.64:0/2356849756 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7bfc0b6680 0x7f7bfc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.828+0000 7f7bfbfff640 1 -- 172.21.15.64:0/2356849756 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f7c040e7e80 con 0x7f7c14103bc0 2024-09-21T04:06:14.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.829+0000 7f7c127fc640 1 --2- 172.21.15.64:0/2356849756 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7bfc0b6680 0x7f7bfc0b8b90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7c14108fd0 tx=0x7f7c08005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:14.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:14.834+0000 7f7bfbfff640 1 -- 172.21.15.64:0/2356849756 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7c040b2780 con 0x7f7c14103bc0 2024-09-21T04:06:15.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:15.082+0000 7f7c1930b640 1 -- 172.21.15.64:0/2356849756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c1419d800 con 0x7f7c14103bc0 2024-09-21T04:06:15.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:15.083+0000 7f7bfbfff640 1 -- 172.21.15.64:0/2356849756 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11918 (secure 0 0 0) 0x7f7c040b76c0 con 0x7f7c14103bc0 2024-09-21T04:06:15.086 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:06:15.086 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":30,"btime":"2024-09-21T04:05:41:799101+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14805,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/97995119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":97995119},{"type":"v1","addr":"172.21.15.64:6839","nonce":97995119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/390891068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":390891068},{"type":"v1","addr":"172.21.15.64:6841","nonce":390891068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:02:03.354744+0000","modified":"2024-09-21T04:05:41.678788+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24710,"mds_1":24680,"mds_2":24661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24634":{"gid":24634,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6839/782190424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782190424},{"type":"v1","addr":"172.21.15.131:6839","nonce":782190424}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24680":{"gid":24680,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24710":{"gid":24710,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6839/1319793636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1319793636},{"type":"v1","addr":"172.21.15.119:6839","nonce":1319793636}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24710,"qdb_cluster":[24661,24680,24710]},"id":1}]} 2024-09-21T04:06:15.086 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 30 2024-09-21T04:06:15.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:15.085+0000 7f7c1930b640 1 -- 172.21.15.64:0/2356849756 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7bfc0b6680 msgr2=0x7f7bfc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:15.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:15.086+0000 7f7c1930b640 1 --2- 172.21.15.64:0/2356849756 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7bfc0b6680 0x7f7bfc0b8b90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7c14108fd0 tx=0x7f7c08005f90 comp rx=0 tx=0).stop 2024-09-21T04:06:15.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:15.086+0000 7f7c1930b640 1 -- 172.21.15.64:0/2356849756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c14103bc0 msgr2=0x7f7c1419c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:15.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:15.086+0000 7f7c1930b640 1 --2- 172.21.15.64:0/2356849756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c14103bc0 0x7f7c1419c6d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400e9f0 tx=0x7f7c0400eeb0 comp rx=0 tx=0).stop 2024-09-21T04:06:15.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:15.086+0000 7f7c1930b640 1 -- 172.21.15.64:0/2356849756 shutdown_connections 2024-09-21T04:06:15.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:15.086+0000 7f7c1930b640 1 --2- 172.21.15.64:0/2356849756 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7bfc0b6680 0x7f7bfc0b8b90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:15.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:15.086+0000 7f7c1930b640 1 --2- 172.21.15.64:0/2356849756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c14109e00 0x7f7c141a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:15.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:15.086+0000 7f7c1930b640 1 --2- 172.21.15.64:0/2356849756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7c141094b0 0x7f7c1419cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:15.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:15.086+0000 7f7c1930b640 1 --2- 172.21.15.64:0/2356849756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c14103bc0 0x7f7c1419c6d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:15.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:15.086+0000 7f7c1930b640 1 -- 172.21.15.64:0/2356849756 >> 172.21.15.64:0/2356849756 conn(0x7f7c1407b910 msgr2=0x7f7c14107400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:15.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:15.086+0000 7f7c1930b640 1 -- 172.21.15.64:0/2356849756 shutdown_connections 2024-09-21T04:06:15.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:15.086+0000 7f7c1930b640 1 -- 172.21.15.64:0/2356849756 wait complete. 2024-09-21T04:06:15.464 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:15 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2356849756' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:06:15.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:15 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2356849756' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:06:15.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:15 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2356849756' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:06:15.889 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T04:06:16.170 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:06:16.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:16 smithi119 ceph-mon[34811]: pgmap v388: 97 pgs: 97 active+clean; 190 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.9 MiB/s wr, 609 op/s 2024-09-21T04:06:16.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:16 smithi064 ceph-mon[31180]: pgmap v388: 97 pgs: 97 active+clean; 190 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.9 MiB/s wr, 609 op/s 2024-09-21T04:06:16.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:16 smithi131 ceph-mon[35193]: pgmap v388: 97 pgs: 97 active+clean; 190 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.9 MiB/s wr, 609 op/s 2024-09-21T04:06:16.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.904+0000 7f54e7a52640 1 -- 172.21.15.64:0/1997686876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e01022a0 msgr2=0x7f54e01026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:16.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.904+0000 7f54e7a52640 1 --2- 172.21.15.64:0/1997686876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e01022a0 0x7f54e01026b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009900 tx=0x7f54dc02a410 comp rx=0 tx=0).stop 2024-09-21T04:06:16.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.904+0000 7f54e7a52640 1 -- 172.21.15.64:0/1997686876 shutdown_connections 2024-09-21T04:06:16.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.904+0000 7f54e7a52640 1 --2- 172.21.15.64:0/1997686876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54e0108530 0x7f54e010ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.904+0000 7f54e7a52640 1 --2- 172.21.15.64:0/1997686876 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54e0107be0 0x7f54e0107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.904+0000 7f54e7a52640 1 --2- 172.21.15.64:0/1997686876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e01022a0 0x7f54e01026b0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.904+0000 7f54e7a52640 1 -- 172.21.15.64:0/1997686876 >> 172.21.15.64:0/1997686876 conn(0x7f54e0079ec0 msgr2=0x7f54e007a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:16.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.904+0000 7f54e7a52640 1 -- 172.21.15.64:0/1997686876 shutdown_connections 2024-09-21T04:06:16.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.904+0000 7f54e7a52640 1 -- 172.21.15.64:0/1997686876 wait complete. 2024-09-21T04:06:16.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.905+0000 7f54e7a52640 1 Processor -- start 2024-09-21T04:06:16.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.905+0000 7f54e7a52640 1 -- start start 2024-09-21T04:06:16.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.905+0000 7f54e7a52640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e01022a0 0x7f54e019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:16.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.905+0000 7f54e7a52640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54e0107be0 0x7f54e019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:16.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.905+0000 7f54e7a52640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54e0108530 0x7f54e019fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:16.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.905+0000 7f54e7a52640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f54e0111970 con 0x7f54e01022a0 2024-09-21T04:06:16.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.905+0000 7f54e7a52640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01117f0 con 0x7f54e0107be0 2024-09-21T04:06:16.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.905+0000 7f54e7a52640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f54e0111af0 con 0x7f54e0108530 2024-09-21T04:06:16.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.905+0000 7f54e57ee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e01022a0 0x7f54e019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:16.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.905+0000 7f54e57ee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e01022a0 0x7f54e019a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41838/0 (socket says 172.21.15.64:41838) 2024-09-21T04:06:16.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.905+0000 7f54e57ee640 1 -- 172.21.15.64:0/2161001808 learned_addr learned my addr 172.21.15.64:0/2161001808 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:16.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.905+0000 7f54e4fed640 1 --2- 172.21.15.64:0/2161001808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54e0107be0 0x7f54e019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:16.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.906+0000 7f54e5fef640 1 --2- 172.21.15.64:0/2161001808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54e0108530 0x7f54e019fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:16.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.906+0000 7f54e4fed640 1 -- 172.21.15.64:0/2161001808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54e0108530 msgr2=0x7f54e019fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:16.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.906+0000 7f54e4fed640 1 --2- 172.21.15.64:0/2161001808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54e0108530 0x7f54e019fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.906+0000 7f54e4fed640 1 -- 172.21.15.64:0/2161001808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e01022a0 msgr2=0x7f54e019a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:16.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.906+0000 7f54e4fed640 1 --2- 172.21.15.64:0/2161001808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e01022a0 0x7f54e019a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.906+0000 7f54e4fed640 1 -- 172.21.15.64:0/2161001808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e01a0520 con 0x7f54e0107be0 2024-09-21T04:06:16.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.906+0000 7f54e57ee640 1 --2- 172.21.15.64:0/2161001808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e01022a0 0x7f54e019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:06:16.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.906+0000 7f54e4fed640 1 --2- 172.21.15.64:0/2161001808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54e0107be0 0x7f54e019ab50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f54e0075a60 tx=0x7f54d000eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:16.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.906+0000 7f54d67fc640 1 -- 172.21.15.64:0/2161001808 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0017440 con 0x7f54e0107be0 2024-09-21T04:06:16.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.906+0000 7f54e7a52640 1 -- 172.21.15.64:0/2161001808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e01a0810 con 0x7f54e0107be0 2024-09-21T04:06:16.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.906+0000 7f54d67fc640 1 -- 172.21.15.64:0/2161001808 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54d0016890 con 0x7f54e0107be0 2024-09-21T04:06:16.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.906+0000 7f54d67fc640 1 -- 172.21.15.64:0/2161001808 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d00157d0 con 0x7f54e0107be0 2024-09-21T04:06:16.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.906+0000 7f54e7a52640 1 -- 172.21.15.64:0/2161001808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e01a6f70 con 0x7f54e0107be0 2024-09-21T04:06:16.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.909+0000 7f54edfc6740 1 Processor -- start 2024-09-21T04:06:16.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.909+0000 7f54edfc6740 1 -- start start 2024-09-21T04:06:16.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.909+0000 7f54edfc6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636308bb070 0x563630c622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:16.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.909+0000 7f54edfc6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636308921b0 0x563630c62800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:16.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.909+0000 7f54edfc6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56363089a390 0x563630c62d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:16.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.909+0000 7f54edfc6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563630bd8ea0 con 0x5636308bb070 2024-09-21T04:06:16.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.909+0000 7f54edfc6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563630bc82d0 con 0x5636308921b0 2024-09-21T04:06:16.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.909+0000 7f54edfc6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x563630bcc0e0 con 0x56363089a390 2024-09-21T04:06:16.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.910+0000 7f54d67fc640 1 -- 172.21.15.64:0/2161001808 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f54d00163e0 con 0x7f54e0107be0 2024-09-21T04:06:16.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.910+0000 7f54e57ee640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636308921b0 0x563630c62800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:16.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.910+0000 7f54e5fef640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636308bb070 0x563630c622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:16.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.910+0000 7f54e57ee640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636308921b0 0x563630c62800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:48742/0 (socket says 172.21.15.64:48742) 2024-09-21T04:06:16.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.910+0000 7f54e57ee640 1 -- 172.21.15.64:0/3093595852 learned_addr learned my addr 172.21.15.64:0/3093595852 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:16.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.910+0000 7f54e57ee640 1 -- 172.21.15.64:0/3093595852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56363089a390 msgr2=0x563630c62d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:16.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.910+0000 7f54e57ee640 1 --2- 172.21.15.64:0/3093595852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56363089a390 0x563630c62d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.910+0000 7f54e57ee640 1 -- 172.21.15.64:0/3093595852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636308bb070 msgr2=0x563630c622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:16.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.910+0000 7f54e57ee640 1 --2- 172.21.15.64:0/3093595852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636308bb070 0x563630c622c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.910+0000 7f54e57ee640 1 -- 172.21.15.64:0/3093595852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563630926650 con 0x5636308921b0 2024-09-21T04:06:16.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.910+0000 7f54e57ee640 1 --2- 172.21.15.64:0/3093595852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636308921b0 0x563630c62800 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f54dc000c00 tx=0x7f54dc0314a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a457644a39979127 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:16.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.911+0000 7f54cb7fe640 1 -- 172.21.15.64:0/3093595852 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc035040 con 0x5636308921b0 2024-09-21T04:06:16.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.911+0000 7f54cb7fe640 1 -- 172.21.15.64:0/3093595852 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54dc031890 con 0x5636308921b0 2024-09-21T04:06:16.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.911+0000 7f54cb7fe640 1 -- 172.21.15.64:0/3093595852 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc0049e0 con 0x5636308921b0 2024-09-21T04:06:16.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.911+0000 7f54d67fc640 1 --2- 172.21.15.64:0/2161001808 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f54cc0b6680 0x7f54cc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:16.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.911+0000 7f54e5fef640 1 --2- 172.21.15.64:0/2161001808 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f54cc0b6680 0x7f54cc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:16.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.911+0000 7f54d67fc640 1 -- 172.21.15.64:0/2161001808 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f54d00e7b40 con 0x7f54e0107be0 2024-09-21T04:06:16.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.912+0000 7f54edfc6740 1 -- 172.21.15.64:0/3093595852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636308921b0 msgr2=0x563630c62800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:16.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.912+0000 7f54edfc6740 1 --2- 172.21.15.64:0/3093595852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636308921b0 0x563630c62800 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f54dc000c00 tx=0x7f54dc0314a0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.912+0000 7f54e5fef640 1 --2- 172.21.15.64:0/2161001808 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f54cc0b6680 0x7f54cc0b8b90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f54d8005fd0 tx=0x7f54d8009600 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:16.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.912+0000 7f54edfc6740 1 -- 172.21.15.64:0/3093595852 shutdown_connections 2024-09-21T04:06:16.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.912+0000 7f54edfc6740 1 --2- 172.21.15.64:0/3093595852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56363089a390 0x563630c62d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.912+0000 7f54edfc6740 1 --2- 172.21.15.64:0/3093595852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636308921b0 0x563630c62800 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.912+0000 7f54edfc6740 1 --2- 172.21.15.64:0/3093595852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636308bb070 0x563630c622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.912+0000 7f54edfc6740 1 -- 172.21.15.64:0/3093595852 >> 172.21.15.64:0/3093595852 conn(0x5636308af460 msgr2=0x563630c601b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:16.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.912+0000 7f54edfc6740 1 -- 172.21.15.64:0/3093595852 shutdown_connections 2024-09-21T04:06:16.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.912+0000 7f54edfc6740 1 -- 172.21.15.64:0/3093595852 wait complete. 2024-09-21T04:06:16.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.913+0000 7f54edfc6740 1 -- start start 2024-09-21T04:06:16.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.913+0000 7f54edfc6740 1 Processor -- start 2024-09-21T04:06:16.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.913+0000 7f54edfc6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636308921b0 0x563630c6d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:16.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.913+0000 7f54edfc6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56363089a390 0x563630c6df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:16.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.913+0000 7f54edfc6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636308bb070 0x563630c79d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:16.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.913+0000 7f54edfc6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563630a29700 con 0x56363089a390 2024-09-21T04:06:16.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.913+0000 7f54edfc6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563630b63160 con 0x5636308921b0 2024-09-21T04:06:16.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.913+0000 7f54edfc6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x563630c70a40 con 0x5636308bb070 2024-09-21T04:06:16.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.913+0000 7f54edfc6740 10 client.0 _ll_register_callbacks cb 0x563630b7c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:06:16.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.913+0000 7f54e57ee640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636308921b0 0x563630c6d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:16.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.913+0000 7f54e4fed640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56363089a390 0x563630c6df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:16.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.913+0000 7f54e4fed640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56363089a390 0x563630c6df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41846/0 (socket says 172.21.15.64:41846) 2024-09-21T04:06:16.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.913+0000 7f54e57ee640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636308921b0 0x563630c6d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:48748/0 (socket says 172.21.15.64:48748) 2024-09-21T04:06:16.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.914+0000 7f54e4fed640 1 -- 172.21.15.64:0/338440193 learned_addr learned my addr 172.21.15.64:0/338440193 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:16.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.914+0000 7f54e5fef640 1 --2- 172.21.15.64:0/338440193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636308bb070 0x563630c79d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:16.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.914+0000 7f54e57ee640 1 -- 172.21.15.64:0/338440193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636308bb070 msgr2=0x563630c79d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:16.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.914+0000 7f54e57ee640 1 --2- 172.21.15.64:0/338440193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636308bb070 0x563630c79d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.922 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.914+0000 7f54e57ee640 1 -- 172.21.15.64:0/338440193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56363089a390 msgr2=0x563630c6df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:16.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.914+0000 7f54e57ee640 1 --2- 172.21.15.64:0/338440193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56363089a390 0x563630c6df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.914+0000 7f54e57ee640 1 -- 172.21.15.64:0/338440193 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563630b90c90 con 0x5636308921b0 2024-09-21T04:06:16.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.914+0000 7f54e4fed640 1 --2- 172.21.15.64:0/338440193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56363089a390 0x563630c6df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:06:16.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.914+0000 7f54e57ee640 1 --2- 172.21.15.64:0/338440193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636308921b0 0x563630c6d9f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f54dc005940 tx=0x7f54dc005550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d580b3d9133f7337 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:16.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.914+0000 7f54c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:06:16.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.914+0000 7f54c8ff9640 1 -- 172.21.15.64:0/338440193 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc035040 con 0x5636308921b0 2024-09-21T04:06:16.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.914+0000 7f54c8ff9640 1 -- 172.21.15.64:0/338440193 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f54dc0035f0 con 0x5636308921b0 2024-09-21T04:06:16.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.914+0000 7f54edfc6740 1 -- 172.21.15.64:0/338440193 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54dc031890 con 0x5636308921b0 2024-09-21T04:06:16.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.915+0000 7f54c8ff9640 1 -- 172.21.15.64:0/338440193 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f54dc057050 con 0x5636308921b0 2024-09-21T04:06:16.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.915+0000 7f54edfc6740 10 client.25733 fetch_fsmap learned FSMap version 30 2024-09-21T04:06:16.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.915+0000 7f54edfc6740 1 -- 172.21.15.64:0/338440193 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563630c7c760 con 0x5636308921b0 2024-09-21T04:06:16.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.915+0000 7f54c8ff9640 1 -- 172.21.15.64:0/338440193 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54dc031890 con 0x5636308921b0 2024-09-21T04:06:16.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.915+0000 7f54c8ff9640 1 -- 172.21.15.64:0/338440193 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f54dc052030 con 0x5636308921b0 2024-09-21T04:06:16.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.915+0000 7f54edfc6740 10 client.25733 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:06:16.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.915+0000 7f54edfc6740 10 client.25733 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:06:16.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.915+0000 7f54edfc6740 20 client.25733 populate_metadata read hostname 'smithi064' 2024-09-21T04:06:16.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.915+0000 7f54edfc6740 1 --2- 172.21.15.64:0/338440193 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x5636308799f0 0x563630c7fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:16.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.915+0000 7f54edfc6740 4 client.25733 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:06:16.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.915+0000 7f54edfc6740 1 -- 172.21.15.64:0/338440193 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563630bdadf0 con 0x5636308799f0 2024-09-21T04:06:16.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.915+0000 7f54e4fed640 1 --2- 172.21.15.64:0/338440193 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x5636308799f0 0x563630c7fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:16.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.916+0000 7f54c8ff9640 10 client.25733 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:06:16.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.916+0000 7f54e4fed640 1 --2- 172.21.15.64:0/338440193 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x5636308799f0 0x563630c7fac0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a8769c7b646ab26 server_cookie=4b1990c84a7b8571 in_seq=0 out_seq=0 2024-09-21T04:06:16.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.916+0000 7f54c8ff9640 1 -- 172.21.15.64:0/338440193 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563630bdadf0 con 0x5636308799f0 2024-09-21T04:06:16.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.917+0000 7f54c8ff9640 10 client.25733 handle_command_reply: tid=0 2024-09-21T04:06:16.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.917+0000 7f54edfc6740 1 client.25733 shutdown 2024-09-21T04:06:16.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.917+0000 7f54edfc6740 1 -- 172.21.15.64:0/338440193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636308921b0 msgr2=0x563630c6d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:16.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.917+0000 7f54edfc6740 1 --2- 172.21.15.64:0/338440193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636308921b0 0x563630c6d9f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f54dc005940 tx=0x7f54dc005550 comp rx=0 tx=0).stop 2024-09-21T04:06:16.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.917+0000 7f54edfc6740 1 -- 172.21.15.64:0/338440193 shutdown_connections 2024-09-21T04:06:16.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.917+0000 7f54edfc6740 1 --2- 172.21.15.64:0/338440193 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x5636308799f0 0x563630c7fac0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.917+0000 7f54edfc6740 1 --2- 172.21.15.64:0/338440193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636308bb070 0x563630c79d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.917+0000 7f54edfc6740 1 --2- 172.21.15.64:0/338440193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56363089a390 0x563630c6df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.917+0000 7f54c8ff9640 0 client.25733 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:06:16.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.917+0000 7f54edfc6740 1 --2- 172.21.15.64:0/338440193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5636308921b0 0x563630c6d9f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.917+0000 7f54edfc6740 1 -- 172.21.15.64:0/338440193 >> 172.21.15.64:0/338440193 conn(0x5636308af460 msgr2=0x563630c654a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:16.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.917+0000 7f54edfc6740 1 -- 172.21.15.64:0/338440193 shutdown_connections 2024-09-21T04:06:16.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.917+0000 7f54edfc6740 1 -- 172.21.15.64:0/338440193 wait complete. 2024-09-21T04:06:16.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.917+0000 7f54edfc6740 20 client.25733 trim_cache size 0 max 16384 2024-09-21T04:06:16.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.932+0000 7f54edfc6740 1 Processor -- start 2024-09-21T04:06:16.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.932+0000 7f54edfc6740 1 -- start start 2024-09-21T04:06:16.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.932+0000 7f54edfc6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636308921b0 0x563630c63a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:16.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.932+0000 7f54edfc6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56363089a390 0x563630c5f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:16.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.932+0000 7f54edfc6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636308bb070 0x563630c5fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:16.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.932+0000 7f54edfc6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01b0050 con 0x5636308921b0 2024-09-21T04:06:16.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.932+0000 7f54edfc6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01ad890 con 0x56363089a390 2024-09-21T04:06:16.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.932+0000 7f54edfc6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01ab6e0 con 0x5636308bb070 2024-09-21T04:06:16.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.932+0000 7f54e57ee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636308921b0 0x563630c63a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:16.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.932+0000 7f54e57ee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636308921b0 0x563630c63a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41860/0 (socket says 172.21.15.64:41860) 2024-09-21T04:06:16.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.932+0000 7f54e57ee640 1 -- 172.21.15.64:0/2126041429 learned_addr learned my addr 172.21.15.64:0/2126041429 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:16.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.932+0000 7f54e4fed640 1 --2- 172.21.15.64:0/2126041429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56363089a390 0x563630c5f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:16.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.932+0000 7f54e5fef640 1 --2- 172.21.15.64:0/2126041429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636308bb070 0x563630c5fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:16.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.932+0000 7f54e57ee640 1 -- 172.21.15.64:0/2126041429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636308bb070 msgr2=0x563630c5fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:16.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.932+0000 7f54e57ee640 1 --2- 172.21.15.64:0/2126041429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636308bb070 0x563630c5fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.932+0000 7f54e57ee640 1 -- 172.21.15.64:0/2126041429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56363089a390 msgr2=0x563630c5f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:16.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.932+0000 7f54e57ee640 1 --2- 172.21.15.64:0/2126041429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56363089a390 0x563630c5f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.932+0000 7f54e57ee640 1 -- 172.21.15.64:0/2126041429 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563630c64b10 con 0x5636308921b0 2024-09-21T04:06:16.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.932+0000 7f54e4fed640 1 --2- 172.21.15.64:0/2126041429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56363089a390 0x563630c5f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:06:16.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.933+0000 7f54e57ee640 1 --2- 172.21.15.64:0/2126041429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636308921b0 0x563630c63a00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f54dc0466a0 tx=0x7f54dc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268bc609aabec615 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:16.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.933+0000 7f54caffd640 1 -- 172.21.15.64:0/2126041429 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc060070 con 0x5636308921b0 2024-09-21T04:06:16.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.933+0000 7f54caffd640 1 -- 172.21.15.64:0/2126041429 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54dc046a90 con 0x5636308921b0 2024-09-21T04:06:16.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.933+0000 7f54caffd640 1 -- 172.21.15.64:0/2126041429 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc035070 con 0x5636308921b0 2024-09-21T04:06:16.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.934+0000 7f54edfc6740 1 -- 172.21.15.64:0/2126041429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636308921b0 msgr2=0x563630c63a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:16.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.934+0000 7f54edfc6740 1 --2- 172.21.15.64:0/2126041429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636308921b0 0x563630c63a00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f54dc0466a0 tx=0x7f54dc02a410 comp rx=0 tx=0).stop 2024-09-21T04:06:16.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.934+0000 7f54edfc6740 1 -- 172.21.15.64:0/2126041429 shutdown_connections 2024-09-21T04:06:16.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.934+0000 7f54edfc6740 1 --2- 172.21.15.64:0/2126041429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636308bb070 0x563630c5fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.934+0000 7f54edfc6740 1 --2- 172.21.15.64:0/2126041429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56363089a390 0x563630c5f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.934+0000 7f54edfc6740 1 --2- 172.21.15.64:0/2126041429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636308921b0 0x563630c63a00 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.934+0000 7f54edfc6740 1 -- 172.21.15.64:0/2126041429 >> 172.21.15.64:0/2126041429 conn(0x5636308af460 msgr2=0x563630c632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:16.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.934+0000 7f54edfc6740 1 -- 172.21.15.64:0/2126041429 shutdown_connections 2024-09-21T04:06:16.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.934+0000 7f54edfc6740 1 -- 172.21.15.64:0/2126041429 wait complete. 2024-09-21T04:06:16.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.935+0000 7f54edfc6740 1 -- start start 2024-09-21T04:06:16.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.935+0000 7f54edfc6740 1 Processor -- start 2024-09-21T04:06:16.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.935+0000 7f54edfc6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636308921b0 0x563630c83190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:16.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.935+0000 7f54edfc6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56363089a390 0x563630c80660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:16.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.935+0000 7f54edfc6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636308bb070 0x563630c80ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:16.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.935+0000 7f54edfc6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01abd50 con 0x5636308921b0 2024-09-21T04:06:16.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.935+0000 7f54edfc6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01ad1c0 con 0x56363089a390 2024-09-21T04:06:16.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.935+0000 7f54edfc6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01ac420 con 0x5636308bb070 2024-09-21T04:06:16.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.935+0000 7f54edfc6740 10 client.0 _ll_register_callbacks cb 0x7f54dc040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:06:16.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.935+0000 7f54e57ee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636308921b0 0x563630c83190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:16.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.935+0000 7f54e57ee640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636308921b0 0x563630c83190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41870/0 (socket says 172.21.15.64:41870) 2024-09-21T04:06:16.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.935+0000 7f54e4fed640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56363089a390 0x563630c80660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:16.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.935+0000 7f54e57ee640 1 -- 172.21.15.64:0/3013951724 learned_addr learned my addr 172.21.15.64:0/3013951724 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:16.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.936+0000 7f54e5fef640 1 --2- 172.21.15.64:0/3013951724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636308bb070 0x563630c80ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:16.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.936+0000 7f54e57ee640 1 -- 172.21.15.64:0/3013951724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636308bb070 msgr2=0x563630c80ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:16.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.936+0000 7f54e57ee640 1 --2- 172.21.15.64:0/3013951724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636308bb070 0x563630c80ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.936+0000 7f54e57ee640 1 -- 172.21.15.64:0/3013951724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56363089a390 msgr2=0x563630c80660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:16.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.936+0000 7f54e57ee640 1 --2- 172.21.15.64:0/3013951724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56363089a390 0x563630c80660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.936+0000 7f54e57ee640 1 -- 172.21.15.64:0/3013951724 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563630c811e0 con 0x5636308921b0 2024-09-21T04:06:16.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.936+0000 7f54e4fed640 1 --2- 172.21.15.64:0/3013951724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56363089a390 0x563630c80660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:06:16.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.936+0000 7f54e57ee640 1 --2- 172.21.15.64:0/3013951724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636308921b0 0x563630c83190 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f54dc005ed0 tx=0x7f54dc005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36874cf47f7e1768 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:16.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.936+0000 7f54c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:06:16.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.936+0000 7f54c8ff9640 1 -- 172.21.15.64:0/3013951724 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc06a060 con 0x5636308921b0 2024-09-21T04:06:16.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.936+0000 7f54c8ff9640 1 -- 172.21.15.64:0/3013951724 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f54dc031890 con 0x5636308921b0 2024-09-21T04:06:16.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.936+0000 7f54edfc6740 1 -- 172.21.15.64:0/3013951724 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54dc046a90 con 0x5636308921b0 2024-09-21T04:06:16.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.937+0000 7f54c8ff9640 1 -- 172.21.15.64:0/3013951724 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f54dc035070 con 0x5636308921b0 2024-09-21T04:06:16.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.937+0000 7f54edfc6740 10 client.16029 fetch_fsmap learned FSMap version 30 2024-09-21T04:06:16.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.937+0000 7f54edfc6740 1 -- 172.21.15.64:0/3013951724 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563630cea030 con 0x5636308921b0 2024-09-21T04:06:16.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.937+0000 7f54c8ff9640 1 -- 172.21.15.64:0/3013951724 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54dc046a90 con 0x5636308921b0 2024-09-21T04:06:16.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.937+0000 7f54c8ff9640 1 -- 172.21.15.64:0/3013951724 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f54dc052040 con 0x5636308921b0 2024-09-21T04:06:16.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.937+0000 7f54edfc6740 10 client.16029 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:06:16.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.937+0000 7f54edfc6740 10 client.16029 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:06:16.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.937+0000 7f54edfc6740 20 client.16029 populate_metadata read hostname 'smithi064' 2024-09-21T04:06:16.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.937+0000 7f54edfc6740 1 --2- 172.21.15.64:0/3013951724 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x56363084eda0 0x563630cec7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:16.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.937+0000 7f54edfc6740 4 client.16029 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:06:16.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.937+0000 7f54edfc6740 1 -- 172.21.15.64:0/3013951724 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563630bd68a0 con 0x56363084eda0 2024-09-21T04:06:16.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.937+0000 7f54e4fed640 1 --2- 172.21.15.64:0/3013951724 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x56363084eda0 0x563630cec7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:16.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.938+0000 7f54c8ff9640 10 client.16029 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:06:16.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.938+0000 7f54e4fed640 1 --2- 172.21.15.64:0/3013951724 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x56363084eda0 0x563630cec7f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8569c459300ea11e server_cookie=cf1863c1d81c80ff in_seq=0 out_seq=0 2024-09-21T04:06:16.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.938+0000 7f54c8ff9640 1 -- 172.21.15.64:0/3013951724 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563630bd68a0 con 0x56363084eda0 2024-09-21T04:06:16.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.938+0000 7f54c8ff9640 10 client.16029 handle_command_reply: tid=0 2024-09-21T04:06:16.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.938+0000 7f54edfc6740 1 client.16029 shutdown 2024-09-21T04:06:16.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.938+0000 7f54edfc6740 1 -- 172.21.15.64:0/3013951724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636308921b0 msgr2=0x563630c83190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:16.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.938+0000 7f54edfc6740 1 --2- 172.21.15.64:0/3013951724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636308921b0 0x563630c83190 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f54dc005ed0 tx=0x7f54dc005780 comp rx=0 tx=0).stop 2024-09-21T04:06:16.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.938+0000 7f54edfc6740 1 -- 172.21.15.64:0/3013951724 shutdown_connections 2024-09-21T04:06:16.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.938+0000 7f54edfc6740 1 --2- 172.21.15.64:0/3013951724 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x56363084eda0 0x563630cec7f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.938+0000 7f54edfc6740 1 --2- 172.21.15.64:0/3013951724 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5636308bb070 0x563630c80ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.938+0000 7f54edfc6740 1 --2- 172.21.15.64:0/3013951724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56363089a390 0x563630c80660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.938+0000 7f54c8ff9640 0 client.16029 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:06:16.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.938+0000 7f54edfc6740 1 --2- 172.21.15.64:0/3013951724 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5636308921b0 0x563630c83190 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.938+0000 7f54edfc6740 1 -- 172.21.15.64:0/3013951724 >> 172.21.15.64:0/3013951724 conn(0x5636308af460 msgr2=0x563630c632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:16.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.938+0000 7f54edfc6740 1 -- 172.21.15.64:0/3013951724 shutdown_connections 2024-09-21T04:06:16.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.938+0000 7f54edfc6740 1 -- 172.21.15.64:0/3013951724 wait complete. 2024-09-21T04:06:16.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.939+0000 7f54edfc6740 20 client.16029 trim_cache size 0 max 16384 2024-09-21T04:06:16.945 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:06:16.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.939+0000 7f54e7a52640 1 -- 172.21.15.64:0/2161001808 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f54cc0b6680 msgr2=0x7f54cc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:16.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.939+0000 7f54e7a52640 1 --2- 172.21.15.64:0/2161001808 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f54cc0b6680 0x7f54cc0b8b90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f54d8005fd0 tx=0x7f54d8009600 comp rx=0 tx=0).stop 2024-09-21T04:06:16.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.939+0000 7f54e7a52640 1 -- 172.21.15.64:0/2161001808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54e0107be0 msgr2=0x7f54e019ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:16.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.939+0000 7f54e7a52640 1 --2- 172.21.15.64:0/2161001808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54e0107be0 0x7f54e019ab50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f54e0075a60 tx=0x7f54d000eef0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.939+0000 7f54e7a52640 1 -- 172.21.15.64:0/2161001808 shutdown_connections 2024-09-21T04:06:16.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.939+0000 7f54e7a52640 1 --2- 172.21.15.64:0/2161001808 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f54cc0b6680 0x7f54cc0b8b90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.939+0000 7f54e7a52640 1 --2- 172.21.15.64:0/2161001808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54e0108530 0x7f54e019fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.939+0000 7f54e7a52640 1 --2- 172.21.15.64:0/2161001808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54e0107be0 0x7f54e019ab50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.939+0000 7f54e7a52640 1 --2- 172.21.15.64:0/2161001808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54e01022a0 0x7f54e019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:16.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.939+0000 7f54e7a52640 1 -- 172.21.15.64:0/2161001808 >> 172.21.15.64:0/2161001808 conn(0x7f54e0079ec0 msgr2=0x7f54e0105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:16.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.939+0000 7f54e7a52640 1 -- 172.21.15.64:0/2161001808 shutdown_connections 2024-09-21T04:06:16.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:16.940+0000 7f54e7a52640 1 -- 172.21.15.64:0/2161001808 wait complete. 2024-09-21T04:06:17.826 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T04:06:18.111 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:06:18.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:18 smithi119 ceph-mon[34811]: pgmap v389: 97 pgs: 97 active+clean; 199 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.0 MiB/s wr, 706 op/s 2024-09-21T04:06:18.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:18 smithi064 ceph-mon[31180]: pgmap v389: 97 pgs: 97 active+clean; 199 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.0 MiB/s wr, 706 op/s 2024-09-21T04:06:18.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:18 smithi131 ceph-mon[35193]: pgmap v389: 97 pgs: 97 active+clean; 199 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.0 MiB/s wr, 706 op/s 2024-09-21T04:06:18.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.633+0000 7f52e1542640 1 -- 172.21.15.64:0/1625225504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52dc103c30 msgr2=0x7f52dc104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:18.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.633+0000 7f52e1542640 1 --2- 172.21.15.64:0/1625225504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52dc103c30 0x7f52dc104040 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f52cc009900 tx=0x7f52cc02a410 comp rx=0 tx=0).stop 2024-09-21T04:06:18.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.633+0000 7f52e1542640 1 -- 172.21.15.64:0/1625225504 shutdown_connections 2024-09-21T04:06:18.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.633+0000 7f52e1542640 1 --2- 172.21.15.64:0/1625225504 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52dc109ec0 0x7f52dc1106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.633+0000 7f52e1542640 1 --2- 172.21.15.64:0/1625225504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52dc109570 0x7f52dc109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.633+0000 7f52e1542640 1 --2- 172.21.15.64:0/1625225504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52dc103c30 0x7f52dc104040 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.633+0000 7f52e1542640 1 -- 172.21.15.64:0/1625225504 >> 172.21.15.64:0/1625225504 conn(0x7f52dc07b8d0 msgr2=0x7f52dc07bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:18.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.633+0000 7f52e1542640 1 -- 172.21.15.64:0/1625225504 shutdown_connections 2024-09-21T04:06:18.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.633+0000 7f52e1542640 1 -- 172.21.15.64:0/1625225504 wait complete. 2024-09-21T04:06:18.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.634+0000 7f52e1542640 1 Processor -- start 2024-09-21T04:06:18.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.634+0000 7f52e1542640 1 -- start start 2024-09-21T04:06:18.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.635+0000 7f52e1542640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52dc103c30 0x7f52dc19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:18.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.635+0000 7f52e1542640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52dc109570 0x7f52dc19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:18.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.635+0000 7f52e1542640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52dc109ec0 0x7f52dc19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:18.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.635+0000 7f52e1542640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc113ae0 con 0x7f52dc103c30 2024-09-21T04:06:18.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.635+0000 7f52e1542640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc113960 con 0x7f52dc109570 2024-09-21T04:06:18.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.635+0000 7f52e1542640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc113c60 con 0x7f52dc109ec0 2024-09-21T04:06:18.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.635+0000 7f52daffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52dc103c30 0x7f52dc19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:18.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.635+0000 7f52daffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52dc103c30 0x7f52dc19a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41880/0 (socket says 172.21.15.64:41880) 2024-09-21T04:06:18.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.635+0000 7f52daffd640 1 -- 172.21.15.64:0/644820638 learned_addr learned my addr 172.21.15.64:0/644820638 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:18.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.635+0000 7f52da7fc640 1 --2- 172.21.15.64:0/644820638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52dc109570 0x7f52dc19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:18.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.635+0000 7f52db7fe640 1 --2- 172.21.15.64:0/644820638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52dc109ec0 0x7f52dc19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:18.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.635+0000 7f52daffd640 1 -- 172.21.15.64:0/644820638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52dc109ec0 msgr2=0x7f52dc19fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:18.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.635+0000 7f52daffd640 1 --2- 172.21.15.64:0/644820638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52dc109ec0 0x7f52dc19fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.635+0000 7f52daffd640 1 -- 172.21.15.64:0/644820638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52dc109570 msgr2=0x7f52dc19aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:18.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.635+0000 7f52daffd640 1 --2- 172.21.15.64:0/644820638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52dc109570 0x7f52dc19aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.635+0000 7f52daffd640 1 -- 172.21.15.64:0/644820638 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52dc1a0420 con 0x7f52dc103c30 2024-09-21T04:06:18.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.635+0000 7f52db7fe640 1 --2- 172.21.15.64:0/644820638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52dc109ec0 0x7f52dc19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:06:18.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.636+0000 7f52daffd640 1 --2- 172.21.15.64:0/644820638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52dc103c30 0x7f52dc19a4b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f52cc005940 tx=0x7f52cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:18.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.636+0000 7f52c3fff640 1 -- 172.21.15.64:0/644820638 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc03f030 con 0x7f52dc103c30 2024-09-21T04:06:18.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.636+0000 7f52c3fff640 1 -- 172.21.15.64:0/644820638 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52cc003cb0 con 0x7f52dc103c30 2024-09-21T04:06:18.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.636+0000 7f52c3fff640 1 -- 172.21.15.64:0/644820638 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc035070 con 0x7f52dc103c30 2024-09-21T04:06:18.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.636+0000 7f52e1542640 1 -- 172.21.15.64:0/644820638 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52dc1a06b0 con 0x7f52dc103c30 2024-09-21T04:06:18.640 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.636+0000 7f52e1542640 1 -- 172.21.15.64:0/644820638 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52dc1a6e70 con 0x7f52dc103c30 2024-09-21T04:06:18.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.639+0000 7f52c3fff640 1 -- 172.21.15.64:0/644820638 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f52cc042030 con 0x7f52dc103c30 2024-09-21T04:06:18.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.639+0000 7f52e7ab6740 1 Processor -- start 2024-09-21T04:06:18.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.639+0000 7f52e7ab6740 1 -- start start 2024-09-21T04:06:18.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.640+0000 7f52e7ab6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56394ab58070 0x56394af036d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:18.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.640+0000 7f52e7ab6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56394ab2f1b0 0x56394af03c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:18.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.640+0000 7f52e7ab6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56394ab37390 0x56394af04150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:18.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.640+0000 7f52e7ab6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56394ae75ea0 con 0x56394ab58070 2024-09-21T04:06:18.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.640+0000 7f52e7ab6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56394ae652d0 con 0x56394ab37390 2024-09-21T04:06:18.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.640+0000 7f52e7ab6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56394ae690e0 con 0x56394ab2f1b0 2024-09-21T04:06:18.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.640+0000 7f52daffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56394ab2f1b0 0x56394af03c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:18.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.640+0000 7f52db7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56394ab58070 0x56394af036d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:18.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.640+0000 7f52db7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56394ab58070 0x56394af036d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41892/0 (socket says 172.21.15.64:41892) 2024-09-21T04:06:18.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.640+0000 7f52daffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56394ab2f1b0 0x56394af03c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:42392/0 (socket says 172.21.15.64:42392) 2024-09-21T04:06:18.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.640+0000 7f52db7fe640 1 -- 172.21.15.64:0/3899674485 learned_addr learned my addr 172.21.15.64:0/3899674485 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:18.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.640+0000 7f52da7fc640 1 --2- 172.21.15.64:0/3899674485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56394ab37390 0x56394af04150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:18.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.640+0000 7f52c3fff640 1 --2- 172.21.15.64:0/644820638 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f52c40b6700 0x7f52c40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:18.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.640+0000 7f52db7fe640 1 -- 172.21.15.64:0/3899674485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56394ab2f1b0 msgr2=0x56394af03c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.640+0000 7f52db7fe640 1 --2- 172.21.15.64:0/3899674485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56394ab2f1b0 0x56394af03c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.640+0000 7f52db7fe640 1 -- 172.21.15.64:0/3899674485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56394ab37390 msgr2=0x56394af04150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.640+0000 7f52db7fe640 1 --2- 172.21.15.64:0/3899674485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56394ab37390 0x56394af04150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.640+0000 7f52db7fe640 1 -- 172.21.15.64:0/3899674485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56394abc3650 con 0x56394ab58070 2024-09-21T04:06:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.640+0000 7f52da7fc640 1 --2- 172.21.15.64:0/3899674485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56394ab37390 0x56394af04150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:06:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.640+0000 7f52daffd640 1 --2- 172.21.15.64:0/3899674485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56394ab2f1b0 0x56394af03c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:06:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.640+0000 7f52db7fe640 1 --2- 172.21.15.64:0/644820638 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f52c40b6700 0x7f52c40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.640+0000 7f52c3fff640 1 -- 172.21.15.64:0/644820638 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f52cc107f80 con 0x7f52dc103c30 2024-09-21T04:06:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.641+0000 7f52db7fe640 1 --2- 172.21.15.64:0/3899674485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56394ab58070 0x56394af036d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f52d4010420 tx=0x7f52d40108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41792cb66ee9585d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.641+0000 7f52c17fa640 1 -- 172.21.15.64:0/3899674485 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d4023070 con 0x56394ab58070 2024-09-21T04:06:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.641+0000 7f52c17fa640 1 -- 172.21.15.64:0/3899674485 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52d401c470 con 0x56394ab58070 2024-09-21T04:06:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.641+0000 7f52c17fa640 1 -- 172.21.15.64:0/3899674485 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d401d6a0 con 0x56394ab58070 2024-09-21T04:06:18.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.641+0000 7f52db7fe640 1 --2- 172.21.15.64:0/644820638 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f52c40b6700 0x7f52c40b8c10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f52d4020000 tx=0x7f52d401a000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:18.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.642+0000 7f52e7ab6740 1 -- 172.21.15.64:0/3899674485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56394ab58070 msgr2=0x56394af036d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:18.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.642+0000 7f52e7ab6740 1 --2- 172.21.15.64:0/3899674485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56394ab58070 0x56394af036d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f52d4010420 tx=0x7f52d40108e0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.642+0000 7f52e7ab6740 1 -- 172.21.15.64:0/3899674485 shutdown_connections 2024-09-21T04:06:18.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.642+0000 7f52e7ab6740 1 --2- 172.21.15.64:0/3899674485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56394ab37390 0x56394af04150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.642+0000 7f52e7ab6740 1 --2- 172.21.15.64:0/3899674485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56394ab2f1b0 0x56394af03c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.642+0000 7f52e7ab6740 1 --2- 172.21.15.64:0/3899674485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56394ab58070 0x56394af036d0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.642+0000 7f52e7ab6740 1 -- 172.21.15.64:0/3899674485 >> 172.21.15.64:0/3899674485 conn(0x56394ab4c460 msgr2=0x56394af015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:18.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.642+0000 7f52e7ab6740 1 -- 172.21.15.64:0/3899674485 shutdown_connections 2024-09-21T04:06:18.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.642+0000 7f52e7ab6740 1 -- 172.21.15.64:0/3899674485 wait complete. 2024-09-21T04:06:18.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.643+0000 7f52e7ab6740 1 -- start start 2024-09-21T04:06:18.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.643+0000 7f52e7ab6740 1 Processor -- start 2024-09-21T04:06:18.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.643+0000 7f52e7ab6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56394ab2f1b0 0x56394af0ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:18.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.643+0000 7f52e7ab6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56394ab37390 0x56394af0f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:18.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.643+0000 7f52e7ab6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56394ab58070 0x56394af1b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:18.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.643+0000 7f52e7ab6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56394acc6700 con 0x56394ab58070 2024-09-21T04:06:18.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.643+0000 7f52e7ab6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56394ae00160 con 0x56394ab2f1b0 2024-09-21T04:06:18.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.643+0000 7f52e7ab6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56394af11eb0 con 0x56394ab37390 2024-09-21T04:06:18.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.643+0000 7f52e7ab6740 10 client.0 _ll_register_callbacks cb 0x56394ae199e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:06:18.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.643+0000 7f52daffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56394ab2f1b0 0x56394af0ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:18.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.643+0000 7f52daffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56394ab2f1b0 0x56394af0ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:48808/0 (socket says 172.21.15.64:48808) 2024-09-21T04:06:18.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.643+0000 7f52daffd640 1 -- 172.21.15.64:0/1249560670 learned_addr learned my addr 172.21.15.64:0/1249560670 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:18.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.643+0000 7f52da7fc640 1 --2- 172.21.15.64:0/1249560670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56394ab37390 0x56394af0f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:18.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.644+0000 7f52db7fe640 1 --2- 172.21.15.64:0/1249560670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56394ab58070 0x56394af1b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:18.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.644+0000 7f52daffd640 1 -- 172.21.15.64:0/1249560670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56394ab37390 msgr2=0x56394af0f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:18.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.644+0000 7f52daffd640 1 --2- 172.21.15.64:0/1249560670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56394ab37390 0x56394af0f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.644+0000 7f52daffd640 1 -- 172.21.15.64:0/1249560670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56394ab58070 msgr2=0x56394af1b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:18.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.644+0000 7f52daffd640 1 --2- 172.21.15.64:0/1249560670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56394ab58070 0x56394af1b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.644+0000 7f52daffd640 1 -- 172.21.15.64:0/1249560670 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56394ae2dc90 con 0x56394ab2f1b0 2024-09-21T04:06:18.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.644+0000 7f52db7fe640 1 --2- 172.21.15.64:0/1249560670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56394ab58070 0x56394af1b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:06:18.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.644+0000 7f52daffd640 1 --2- 172.21.15.64:0/1249560670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56394ab2f1b0 0x56394af0ede0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f52cc003fd0 tx=0x7f52cc02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6b93034753584e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:18.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.644+0000 7f52beff5640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:06:18.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.644+0000 7f52beff5640 1 -- 172.21.15.64:0/1249560670 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc049070 con 0x56394ab2f1b0 2024-09-21T04:06:18.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.644+0000 7f52beff5640 1 -- 172.21.15.64:0/1249560670 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f52cc0ca010 con 0x56394ab2f1b0 2024-09-21T04:06:18.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.644+0000 7f52e7ab6740 1 -- 172.21.15.64:0/1249560670 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52d401c470 con 0x56394ab2f1b0 2024-09-21T04:06:18.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.644+0000 7f52beff5640 1 -- 172.21.15.64:0/1249560670 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f52cc0cdb70 con 0x56394ab2f1b0 2024-09-21T04:06:18.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.644+0000 7f52e7ab6740 10 client.25751 fetch_fsmap learned FSMap version 30 2024-09-21T04:06:18.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.645+0000 7f52e7ab6740 1 -- 172.21.15.64:0/1249560670 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56394af1dbf0 con 0x56394ab2f1b0 2024-09-21T04:06:18.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.645+0000 7f52beff5640 1 -- 172.21.15.64:0/1249560670 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52d401c470 con 0x56394ab2f1b0 2024-09-21T04:06:18.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.645+0000 7f52beff5640 1 -- 172.21.15.64:0/1249560670 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f52cc10bc80 con 0x56394ab2f1b0 2024-09-21T04:06:18.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.645+0000 7f52e7ab6740 10 client.25751 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:06:18.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.645+0000 7f52e7ab6740 10 client.25751 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:06:18.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.645+0000 7f52e7ab6740 20 client.25751 populate_metadata read hostname 'smithi064' 2024-09-21T04:06:18.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.645+0000 7f52e7ab6740 1 --2- 172.21.15.64:0/1249560670 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x56394ab169f0 0x56394af21250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:18.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.645+0000 7f52e7ab6740 4 client.25751 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:06:18.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.645+0000 7f52e7ab6740 1 -- 172.21.15.64:0/1249560670 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56394ae77df0 con 0x56394ab169f0 2024-09-21T04:06:18.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.645+0000 7f52da7fc640 1 --2- 172.21.15.64:0/1249560670 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x56394ab169f0 0x56394af21250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:18.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.646+0000 7f52beff5640 10 client.25751 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:06:18.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.646+0000 7f52da7fc640 1 --2- 172.21.15.64:0/1249560670 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x56394ab169f0 0x56394af21250 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4a4377dcfdd80fa8 server_cookie=682bbf0aff698d47 in_seq=0 out_seq=0 2024-09-21T04:06:18.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.648+0000 7f52beff5640 1 -- 172.21.15.64:0/1249560670 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56394ae77df0 con 0x56394ab169f0 2024-09-21T04:06:18.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.648+0000 7f52beff5640 10 client.25751 handle_command_reply: tid=0 2024-09-21T04:06:18.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.648+0000 7f52e7ab6740 1 client.25751 shutdown 2024-09-21T04:06:18.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.648+0000 7f52e7ab6740 1 -- 172.21.15.64:0/1249560670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56394ab2f1b0 msgr2=0x56394af0ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:18.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.648+0000 7f52e7ab6740 1 --2- 172.21.15.64:0/1249560670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56394ab2f1b0 0x56394af0ede0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f52cc003fd0 tx=0x7f52cc02a410 comp rx=0 tx=0).stop 2024-09-21T04:06:18.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.648+0000 7f52e7ab6740 1 -- 172.21.15.64:0/1249560670 shutdown_connections 2024-09-21T04:06:18.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.648+0000 7f52e7ab6740 1 --2- 172.21.15.64:0/1249560670 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x56394ab169f0 0x56394af21250 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.648+0000 7f52e7ab6740 1 --2- 172.21.15.64:0/1249560670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56394ab58070 0x56394af1b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.648+0000 7f52e7ab6740 1 --2- 172.21.15.64:0/1249560670 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56394ab37390 0x56394af0f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.648+0000 7f52e7ab6740 1 --2- 172.21.15.64:0/1249560670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56394ab2f1b0 0x56394af0ede0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.648+0000 7f52beff5640 0 client.25751 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:06:18.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.648+0000 7f52e7ab6740 1 -- 172.21.15.64:0/1249560670 >> 172.21.15.64:0/1249560670 conn(0x56394ab4c460 msgr2=0x56394af068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:18.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.648+0000 7f52e7ab6740 1 -- 172.21.15.64:0/1249560670 shutdown_connections 2024-09-21T04:06:18.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.648+0000 7f52e7ab6740 1 -- 172.21.15.64:0/1249560670 wait complete. 2024-09-21T04:06:18.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.648+0000 7f52e7ab6740 20 client.25751 trim_cache size 0 max 16384 2024-09-21T04:06:18.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.677+0000 7f52e7ab6740 1 Processor -- start 2024-09-21T04:06:18.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.677+0000 7f52e7ab6740 1 -- start start 2024-09-21T04:06:18.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.677+0000 7f52e7ab6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56394ab2f1b0 0x56394af0f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:18.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.677+0000 7f52e7ab6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56394ab37390 0x56394af1b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:18.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.677+0000 7f52e7ab6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56394ab58070 0x56394af0bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:18.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.678+0000 7f52e7ab6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc1aff50 con 0x56394ab37390 2024-09-21T04:06:18.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.678+0000 7f52e7ab6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc1ad790 con 0x56394ab2f1b0 2024-09-21T04:06:18.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.678+0000 7f52e7ab6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc1ab5e0 con 0x56394ab58070 2024-09-21T04:06:18.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.678+0000 7f52daffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56394ab2f1b0 0x56394af0f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:18.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.678+0000 7f52daffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56394ab2f1b0 0x56394af0f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:48818/0 (socket says 172.21.15.64:48818) 2024-09-21T04:06:18.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.678+0000 7f52daffd640 1 -- 172.21.15.64:0/2365535635 learned_addr learned my addr 172.21.15.64:0/2365535635 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:18.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.678+0000 7f52da7fc640 1 --2- 172.21.15.64:0/2365535635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56394ab37390 0x56394af1b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:18.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.678+0000 7f52db7fe640 1 --2- 172.21.15.64:0/2365535635 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56394ab58070 0x56394af0bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:18.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.678+0000 7f52daffd640 1 -- 172.21.15.64:0/2365535635 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56394ab58070 msgr2=0x56394af0bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:18.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.678+0000 7f52daffd640 1 --2- 172.21.15.64:0/2365535635 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56394ab58070 0x56394af0bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.678+0000 7f52daffd640 1 -- 172.21.15.64:0/2365535635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56394ab37390 msgr2=0x56394af1b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:18.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.678+0000 7f52daffd640 1 --2- 172.21.15.64:0/2365535635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56394ab37390 0x56394af1b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.678+0000 7f52daffd640 1 -- 172.21.15.64:0/2365535635 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56394aef7440 con 0x56394ab2f1b0 2024-09-21T04:06:18.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.678+0000 7f52da7fc640 1 --2- 172.21.15.64:0/2365535635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56394ab37390 0x56394af1b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:06:18.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.679+0000 7f52daffd640 1 --2- 172.21.15.64:0/2365535635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56394ab2f1b0 0x56394af0f2a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f52cc006010 tx=0x7f52cc03b6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7923951c39bff22 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:18.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.679+0000 7f52c0ff9640 1 -- 172.21.15.64:0/2365535635 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc049040 con 0x56394ab2f1b0 2024-09-21T04:06:18.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.679+0000 7f52c0ff9640 1 -- 172.21.15.64:0/2365535635 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52cc10b440 con 0x56394ab2f1b0 2024-09-21T04:06:18.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.679+0000 7f52c0ff9640 1 -- 172.21.15.64:0/2365535635 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc0cd510 con 0x56394ab2f1b0 2024-09-21T04:06:18.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.680+0000 7f52e7ab6740 1 -- 172.21.15.64:0/2365535635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56394ab2f1b0 msgr2=0x56394af0f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:18.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.680+0000 7f52e7ab6740 1 --2- 172.21.15.64:0/2365535635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56394ab2f1b0 0x56394af0f2a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f52cc006010 tx=0x7f52cc03b6b0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.680+0000 7f52e7ab6740 1 -- 172.21.15.64:0/2365535635 shutdown_connections 2024-09-21T04:06:18.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.680+0000 7f52e7ab6740 1 --2- 172.21.15.64:0/2365535635 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56394ab58070 0x56394af0bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.680+0000 7f52e7ab6740 1 --2- 172.21.15.64:0/2365535635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56394ab37390 0x56394af1b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.680+0000 7f52e7ab6740 1 --2- 172.21.15.64:0/2365535635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56394ab2f1b0 0x56394af0f2a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.680+0000 7f52e7ab6740 1 -- 172.21.15.64:0/2365535635 >> 172.21.15.64:0/2365535635 conn(0x56394ab4c460 msgr2=0x56394af0eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:18.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.681+0000 7f52e7ab6740 1 -- 172.21.15.64:0/2365535635 shutdown_connections 2024-09-21T04:06:18.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.681+0000 7f52e7ab6740 1 -- 172.21.15.64:0/2365535635 wait complete. 2024-09-21T04:06:18.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.681+0000 7f52e7ab6740 1 -- start start 2024-09-21T04:06:18.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.681+0000 7f52e7ab6740 1 Processor -- start 2024-09-21T04:06:18.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.682+0000 7f52e7ab6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56394ab2f1b0 0x56394af13280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:18.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.682+0000 7f52e7ab6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56394ab37390 0x56394af137c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:18.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.682+0000 7f52e7ab6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56394ab58070 0x56394af13d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:18.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.682+0000 7f52e7ab6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc1abc50 con 0x56394ab58070 2024-09-21T04:06:18.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.682+0000 7f52e7ab6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc1ad0c0 con 0x56394ab2f1b0 2024-09-21T04:06:18.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.682+0000 7f52e7ab6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc1ac320 con 0x56394ab37390 2024-09-21T04:06:18.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.682+0000 7f52e7ab6740 10 client.0 _ll_register_callbacks cb 0x7f52cc10b7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:06:18.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.682+0000 7f52daffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56394ab2f1b0 0x56394af13280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:18.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.682+0000 7f52daffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56394ab2f1b0 0x56394af13280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:48826/0 (socket says 172.21.15.64:48826) 2024-09-21T04:06:18.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.682+0000 7f52daffd640 1 -- 172.21.15.64:0/818602636 learned_addr learned my addr 172.21.15.64:0/818602636 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:18.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.682+0000 7f52db7fe640 1 --2- 172.21.15.64:0/818602636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56394ab58070 0x56394af13d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:18.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.682+0000 7f52daffd640 1 -- 172.21.15.64:0/818602636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56394ab37390 msgr2=0x56394af137c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:06:18.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.682+0000 7f52daffd640 1 --2- 172.21.15.64:0/818602636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56394ab37390 0x56394af137c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.682+0000 7f52daffd640 1 -- 172.21.15.64:0/818602636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56394ab58070 msgr2=0x56394af13d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:18.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.682+0000 7f52daffd640 1 --2- 172.21.15.64:0/818602636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56394ab58070 0x56394af13d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.682+0000 7f52daffd640 1 -- 172.21.15.64:0/818602636 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56394af8acd0 con 0x56394ab2f1b0 2024-09-21T04:06:18.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.683+0000 7f52daffd640 1 --2- 172.21.15.64:0/818602636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56394ab2f1b0 0x56394af13280 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f52cc031440 tx=0x7f52cc02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db567b83a837b1a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:18.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.683+0000 7f52beff5640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:06:18.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.683+0000 7f52beff5640 1 -- 172.21.15.64:0/818602636 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc049040 con 0x56394ab2f1b0 2024-09-21T04:06:18.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.683+0000 7f52beff5640 1 -- 172.21.15.64:0/818602636 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f52cc0ca010 con 0x56394ab2f1b0 2024-09-21T04:06:18.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.683+0000 7f52e7ab6740 1 -- 172.21.15.64:0/818602636 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56394ae738a0 con 0x56394ab2f1b0 2024-09-21T04:06:18.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.683+0000 7f52beff5640 1 -- 172.21.15.64:0/818602636 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f52cc0cd6f0 con 0x56394ab2f1b0 2024-09-21T04:06:18.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.683+0000 7f52e7ab6740 10 client.25757 fetch_fsmap learned FSMap version 30 2024-09-21T04:06:18.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.683+0000 7f52e7ab6740 1 -- 172.21.15.64:0/818602636 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56394af8acd0 con 0x56394ab2f1b0 2024-09-21T04:06:18.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.683+0000 7f52beff5640 1 -- 172.21.15.64:0/818602636 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52cc0ce4a0 con 0x56394ab2f1b0 2024-09-21T04:06:18.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.684+0000 7f52beff5640 1 -- 172.21.15.64:0/818602636 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f52cc03bbf0 con 0x56394ab2f1b0 2024-09-21T04:06:18.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.684+0000 7f52e7ab6740 10 client.25757 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:06:18.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.684+0000 7f52e7ab6740 10 client.25757 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:06:18.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.684+0000 7f52e7ab6740 20 client.25757 populate_metadata read hostname 'smithi064' 2024-09-21T04:06:18.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.684+0000 7f52e7ab6740 1 --2- 172.21.15.64:0/818602636 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x56394aaebda0 0x56394af8e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:18.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.684+0000 7f52e7ab6740 4 client.25757 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:06:18.688 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.684+0000 7f52e7ab6740 1 -- 172.21.15.64:0/818602636 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56394ae738a0 con 0x56394aaebda0 2024-09-21T04:06:18.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.684+0000 7f52da7fc640 1 --2- 172.21.15.64:0/818602636 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x56394aaebda0 0x56394af8e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:18.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.685+0000 7f52beff5640 10 client.25757 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:06:18.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.685+0000 7f52da7fc640 1 --2- 172.21.15.64:0/818602636 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x56394aaebda0 0x56394af8e200 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3c464887e36f5070 server_cookie=62ceceefe3ae0d53 in_seq=0 out_seq=0 2024-09-21T04:06:18.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.685+0000 7f52beff5640 1 -- 172.21.15.64:0/818602636 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56394ae738a0 con 0x56394aaebda0 2024-09-21T04:06:18.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.685+0000 7f52beff5640 10 client.25757 handle_command_reply: tid=0 2024-09-21T04:06:18.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.685+0000 7f52e7ab6740 1 client.25757 shutdown 2024-09-21T04:06:18.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.686+0000 7f52e7ab6740 1 -- 172.21.15.64:0/818602636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56394ab2f1b0 msgr2=0x56394af13280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:18.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.686+0000 7f52e7ab6740 1 --2- 172.21.15.64:0/818602636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56394ab2f1b0 0x56394af13280 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f52cc031440 tx=0x7f52cc02a410 comp rx=0 tx=0).stop 2024-09-21T04:06:18.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.686+0000 7f52e7ab6740 1 -- 172.21.15.64:0/818602636 shutdown_connections 2024-09-21T04:06:18.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.686+0000 7f52e7ab6740 1 --2- 172.21.15.64:0/818602636 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x56394aaebda0 0x56394af8e200 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.686+0000 7f52e7ab6740 1 --2- 172.21.15.64:0/818602636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56394ab58070 0x56394af13d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.686+0000 7f52e7ab6740 1 --2- 172.21.15.64:0/818602636 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56394ab37390 0x56394af137c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.686+0000 7f52e7ab6740 1 --2- 172.21.15.64:0/818602636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56394ab2f1b0 0x56394af13280 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.686+0000 7f52beff5640 0 client.25757 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:06:18.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.686+0000 7f52e7ab6740 1 -- 172.21.15.64:0/818602636 >> 172.21.15.64:0/818602636 conn(0x56394ab4c460 msgr2=0x56394af10220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:18.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.686+0000 7f52e7ab6740 1 -- 172.21.15.64:0/818602636 shutdown_connections 2024-09-21T04:06:18.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.686+0000 7f52e7ab6740 1 -- 172.21.15.64:0/818602636 wait complete. 2024-09-21T04:06:18.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.686+0000 7f52e7ab6740 20 client.25757 trim_cache size 0 max 16384 2024-09-21T04:06:18.691 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:06:18.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.687+0000 7f52e1542640 1 -- 172.21.15.64:0/644820638 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f52c40b6700 msgr2=0x7f52c40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:18.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.687+0000 7f52e1542640 1 --2- 172.21.15.64:0/644820638 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f52c40b6700 0x7f52c40b8c10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f52d4020000 tx=0x7f52d401a000 comp rx=0 tx=0).stop 2024-09-21T04:06:18.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.687+0000 7f52e1542640 1 -- 172.21.15.64:0/644820638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52dc103c30 msgr2=0x7f52dc19a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:18.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.687+0000 7f52e1542640 1 --2- 172.21.15.64:0/644820638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52dc103c30 0x7f52dc19a4b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f52cc005940 tx=0x7f52cc0042f0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.687+0000 7f52e1542640 1 -- 172.21.15.64:0/644820638 shutdown_connections 2024-09-21T04:06:18.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.687+0000 7f52e1542640 1 --2- 172.21.15.64:0/644820638 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f52c40b6700 0x7f52c40b8c10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.687+0000 7f52e1542640 1 --2- 172.21.15.64:0/644820638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52dc109ec0 0x7f52dc19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.687+0000 7f52e1542640 1 --2- 172.21.15.64:0/644820638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52dc109570 0x7f52dc19aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.687+0000 7f52e1542640 1 --2- 172.21.15.64:0/644820638 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52dc103c30 0x7f52dc19a4b0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:18.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.687+0000 7f52e1542640 1 -- 172.21.15.64:0/644820638 >> 172.21.15.64:0/644820638 conn(0x7f52dc07b8d0 msgr2=0x7f52dc107660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:18.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.688+0000 7f52e1542640 1 -- 172.21.15.64:0/644820638 shutdown_connections 2024-09-21T04:06:18.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:18.688+0000 7f52e1542640 1 -- 172.21.15.64:0/644820638 wait complete. 2024-09-21T04:06:19.462 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T04:06:19.753 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:06:20.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.241+0000 7f92f65cf640 1 -- 172.21.15.64:0/2244709839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f0103bb0 msgr2=0x7f92f0103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:20.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.241+0000 7f92f65cf640 1 --2- 172.21.15.64:0/2244709839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f0103bb0 0x7f92f0103fc0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009ca0 tx=0x7f92e802a8c0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.241+0000 7f92f65cf640 1 -- 172.21.15.64:0/2244709839 shutdown_connections 2024-09-21T04:06:20.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.241+0000 7f92f65cf640 1 --2- 172.21.15.64:0/2244709839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92f0109e40 0x7f92f0110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.241+0000 7f92f65cf640 1 --2- 172.21.15.64:0/2244709839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92f01094f0 0x7f92f0109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.241+0000 7f92f65cf640 1 --2- 172.21.15.64:0/2244709839 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f0103bb0 0x7f92f0103fc0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.241+0000 7f92f65cf640 1 -- 172.21.15.64:0/2244709839 >> 172.21.15.64:0/2244709839 conn(0x7f92f00ff600 msgr2=0x7f92f0101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:20.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.241+0000 7f92f65cf640 1 -- 172.21.15.64:0/2244709839 shutdown_connections 2024-09-21T04:06:20.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.241+0000 7f92f65cf640 1 -- 172.21.15.64:0/2244709839 wait complete. 2024-09-21T04:06:20.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.242+0000 7f92f65cf640 1 Processor -- start 2024-09-21T04:06:20.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.242+0000 7f92f65cf640 1 -- start start 2024-09-21T04:06:20.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.242+0000 7f92f65cf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f0103bb0 0x7f92f019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:20.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.242+0000 7f92f65cf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92f01094f0 0x7f92f019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:20.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.242+0000 7f92f65cf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92f0109e40 0x7f92f01a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:20.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.242+0000 7f92f65cf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0113b00 con 0x7f92f0103bb0 2024-09-21T04:06:20.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.242+0000 7f92f65cf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0113980 con 0x7f92f0109e40 2024-09-21T04:06:20.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.242+0000 7f92f65cf640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0113c80 con 0x7f92f01094f0 2024-09-21T04:06:20.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.242+0000 7f92effff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f0103bb0 0x7f92f019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:20.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.242+0000 7f92effff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f0103bb0 0x7f92f019c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41924/0 (socket says 172.21.15.64:41924) 2024-09-21T04:06:20.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.242+0000 7f92effff640 1 -- 172.21.15.64:0/1857874423 learned_addr learned my addr 172.21.15.64:0/1857874423 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:20.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.242+0000 7f92f4b6c640 1 --2- 172.21.15.64:0/1857874423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92f0109e40 0x7f92f01a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:20.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.242+0000 7f92ef7fe640 1 --2- 172.21.15.64:0/1857874423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92f01094f0 0x7f92f019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:20.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.243+0000 7f92effff640 1 -- 172.21.15.64:0/1857874423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92f01094f0 msgr2=0x7f92f019ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:20.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.243+0000 7f92effff640 1 --2- 172.21.15.64:0/1857874423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92f01094f0 0x7f92f019ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.243+0000 7f92effff640 1 -- 172.21.15.64:0/1857874423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92f0109e40 msgr2=0x7f92f01a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:20.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.243+0000 7f92effff640 1 --2- 172.21.15.64:0/1857874423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92f0109e40 0x7f92f01a1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.243+0000 7f92effff640 1 -- 172.21.15.64:0/1857874423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f01a2680 con 0x7f92f0103bb0 2024-09-21T04:06:20.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.243+0000 7f92effff640 1 --2- 172.21.15.64:0/1857874423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f0103bb0 0x7f92f019c710 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f92e802ade0 tx=0x7f92e8031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:20.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.243+0000 7f92ed7fa640 1 -- 172.21.15.64:0/1857874423 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e8003b70 con 0x7f92f0103bb0 2024-09-21T04:06:20.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.243+0000 7f92f65cf640 1 -- 172.21.15.64:0/1857874423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f01a2910 con 0x7f92f0103bb0 2024-09-21T04:06:20.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.243+0000 7f92ed7fa640 1 -- 172.21.15.64:0/1857874423 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92e8003d10 con 0x7f92f0103bb0 2024-09-21T04:06:20.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.243+0000 7f92f65cf640 1 -- 172.21.15.64:0/1857874423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f01a90d0 con 0x7f92f0103bb0 2024-09-21T04:06:20.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.243+0000 7f92ed7fa640 1 -- 172.21.15.64:0/1857874423 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e802fa50 con 0x7f92f0103bb0 2024-09-21T04:06:20.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.246+0000 7f92fcb43740 1 Processor -- start 2024-09-21T04:06:20.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.246+0000 7f92fcb43740 1 -- start start 2024-09-21T04:06:20.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.246+0000 7f92ed7fa640 1 -- 172.21.15.64:0/1857874423 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f92e802fc10 con 0x7f92f0103bb0 2024-09-21T04:06:20.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.246+0000 7f92fcb43740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ee72cf070 0x564ee76762c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:20.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.246+0000 7f92fcb43740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564ee72a61b0 0x564ee7676800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:20.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.246+0000 7f92fcb43740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ee72ae390 0x564ee7676d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:20.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.246+0000 7f92fcb43740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564ee75ecea0 con 0x564ee72ae390 2024-09-21T04:06:20.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.246+0000 7f92fcb43740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564ee75dc2d0 con 0x564ee72cf070 2024-09-21T04:06:20.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.246+0000 7f92fcb43740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564ee75e00e0 con 0x564ee72a61b0 2024-09-21T04:06:20.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.246+0000 7f92f4b6c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ee72cf070 0x564ee76762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:20.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.246+0000 7f92ef7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ee72ae390 0x564ee7676d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:20.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.246+0000 7f92f4b6c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ee72cf070 0x564ee76762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:48844/0 (socket says 172.21.15.64:48844) 2024-09-21T04:06:20.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.246+0000 7f92effff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564ee72a61b0 0x564ee7676800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:20.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.246+0000 7f92f4b6c640 1 -- 172.21.15.64:0/939053821 learned_addr learned my addr 172.21.15.64:0/939053821 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:20.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.246+0000 7f92f4b6c640 1 -- 172.21.15.64:0/939053821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564ee72a61b0 msgr2=0x564ee7676800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:20.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.247+0000 7f92f4b6c640 1 --2- 172.21.15.64:0/939053821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564ee72a61b0 0x564ee7676800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.247+0000 7f92f4b6c640 1 -- 172.21.15.64:0/939053821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ee72ae390 msgr2=0x564ee7676d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:20.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.247+0000 7f92f4b6c640 1 --2- 172.21.15.64:0/939053821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ee72ae390 0x564ee7676d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.247+0000 7f92f4b6c640 1 -- 172.21.15.64:0/939053821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ee733a650 con 0x564ee72cf070 2024-09-21T04:06:20.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.247+0000 7f92ef7fe640 1 --2- 172.21.15.64:0/939053821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ee72ae390 0x564ee7676d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:06:20.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.247+0000 7f92ed7fa640 1 --2- 172.21.15.64:0/1857874423 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f92dc0b6680 0x7f92dc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:20.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.247+0000 7f92f4b6c640 1 --2- 172.21.15.64:0/939053821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ee72cf070 0x564ee76762c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f92d800b700 tx=0x7f92d800bbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f15529a9d6f3d2d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:20.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.247+0000 7f92effff640 1 --2- 172.21.15.64:0/1857874423 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f92dc0b6680 0x7f92dc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:20.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.247+0000 7f92e67fc640 1 -- 172.21.15.64:0/939053821 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8003dd0 con 0x564ee72cf070 2024-09-21T04:06:20.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.247+0000 7f92ed7fa640 1 -- 172.21.15.64:0/1857874423 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f92e8108b80 con 0x7f92f0103bb0 2024-09-21T04:06:20.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.247+0000 7f92e67fc640 1 -- 172.21.15.64:0/939053821 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92d8015070 con 0x564ee72cf070 2024-09-21T04:06:20.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.247+0000 7f92e67fc640 1 -- 172.21.15.64:0/939053821 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8010640 con 0x564ee72cf070 2024-09-21T04:06:20.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.248+0000 7f92fcb43740 1 -- 172.21.15.64:0/939053821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ee72cf070 msgr2=0x564ee76762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:20.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.248+0000 7f92fcb43740 1 --2- 172.21.15.64:0/939053821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ee72cf070 0x564ee76762c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f92d800b700 tx=0x7f92d800bbc0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.248+0000 7f92effff640 1 --2- 172.21.15.64:0/1857874423 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f92dc0b6680 0x7f92dc0b8b90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f92e8106350 tx=0x7f92e810c6d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:20.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.248+0000 7f92fcb43740 1 -- 172.21.15.64:0/939053821 shutdown_connections 2024-09-21T04:06:20.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.248+0000 7f92fcb43740 1 --2- 172.21.15.64:0/939053821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ee72ae390 0x564ee7676d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.248+0000 7f92fcb43740 1 --2- 172.21.15.64:0/939053821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564ee72a61b0 0x564ee7676800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.248+0000 7f92fcb43740 1 --2- 172.21.15.64:0/939053821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ee72cf070 0x564ee76762c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.248+0000 7f92fcb43740 1 -- 172.21.15.64:0/939053821 >> 172.21.15.64:0/939053821 conn(0x564ee72c3460 msgr2=0x564ee76741b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:20.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.248+0000 7f92fcb43740 1 -- 172.21.15.64:0/939053821 shutdown_connections 2024-09-21T04:06:20.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.248+0000 7f92fcb43740 1 -- 172.21.15.64:0/939053821 wait complete. 2024-09-21T04:06:20.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.249+0000 7f92fcb43740 1 -- start start 2024-09-21T04:06:20.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.249+0000 7f92fcb43740 1 Processor -- start 2024-09-21T04:06:20.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.250+0000 7f92fcb43740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ee72a61b0 0x564ee76819b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:20.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.250+0000 7f92fcb43740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ee72ae390 0x564ee7681ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:20.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.250+0000 7f92fcb43740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564ee72cf070 0x564ee768dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:20.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.250+0000 7f92fcb43740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564ee743d700 con 0x564ee72ae390 2024-09-21T04:06:20.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.250+0000 7f92fcb43740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564ee7577160 con 0x564ee72a61b0 2024-09-21T04:06:20.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.250+0000 7f92fcb43740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564ee7684aa0 con 0x564ee72cf070 2024-09-21T04:06:20.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.250+0000 7f92fcb43740 10 client.0 _ll_register_callbacks cb 0x564ee75909e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:06:20.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.250+0000 7f92f4b6c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ee72a61b0 0x564ee76819b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:20.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.250+0000 7f92ef7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ee72ae390 0x564ee7681ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:20.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.250+0000 7f92f4b6c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ee72a61b0 0x564ee76819b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:48854/0 (socket says 172.21.15.64:48854) 2024-09-21T04:06:20.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.250+0000 7f92ef7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ee72ae390 0x564ee7681ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41938/0 (socket says 172.21.15.64:41938) 2024-09-21T04:06:20.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.250+0000 7f92f4b6c640 1 -- 172.21.15.64:0/327089400 learned_addr learned my addr 172.21.15.64:0/327089400 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:20.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.250+0000 7f92f4b6c640 1 --2- 172.21.15.64:0/327089400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564ee72cf070 0x564ee768dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:20.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.250+0000 7f92f4b6c640 1 -- 172.21.15.64:0/327089400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564ee72cf070 msgr2=0x564ee768dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:20.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.250+0000 7f92f4b6c640 1 --2- 172.21.15.64:0/327089400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564ee72cf070 0x564ee768dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.250+0000 7f92f4b6c640 1 -- 172.21.15.64:0/327089400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ee72ae390 msgr2=0x564ee7681ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:20.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.250+0000 7f92f4b6c640 1 --2- 172.21.15.64:0/327089400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ee72ae390 0x564ee7681ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.250+0000 7f92f4b6c640 1 -- 172.21.15.64:0/327089400 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ee75a4c90 con 0x564ee72a61b0 2024-09-21T04:06:20.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.250+0000 7f92ef7fe640 1 --2- 172.21.15.64:0/327089400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ee72ae390 0x564ee7681ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:06:20.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.251+0000 7f92f4b6c640 1 --2- 172.21.15.64:0/327089400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ee72a61b0 0x564ee76819b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f92d80044d0 tx=0x7f92d8011dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76775a36d1abfaa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:20.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.251+0000 7f92cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:06:20.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.251+0000 7f92cf7fe640 1 -- 172.21.15.64:0/327089400 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8023040 con 0x564ee72a61b0 2024-09-21T04:06:20.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.251+0000 7f92fcb43740 1 -- 172.21.15.64:0/327089400 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92d8015070 con 0x564ee72a61b0 2024-09-21T04:06:20.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.251+0000 7f92cf7fe640 1 -- 172.21.15.64:0/327089400 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f92d80107d0 con 0x564ee72a61b0 2024-09-21T04:06:20.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.251+0000 7f92cf7fe640 1 -- 172.21.15.64:0/327089400 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f92d80109d0 con 0x564ee72a61b0 2024-09-21T04:06:20.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.251+0000 7f92fcb43740 10 client.25769 fetch_fsmap learned FSMap version 30 2024-09-21T04:06:20.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.251+0000 7f92fcb43740 1 -- 172.21.15.64:0/327089400 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564ee76907b0 con 0x564ee72a61b0 2024-09-21T04:06:20.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.251+0000 7f92cf7fe640 1 -- 172.21.15.64:0/327089400 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92d8015070 con 0x564ee72a61b0 2024-09-21T04:06:20.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.251+0000 7f92cf7fe640 1 -- 172.21.15.64:0/327089400 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f92d802c4f0 con 0x564ee72a61b0 2024-09-21T04:06:20.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.252+0000 7f92fcb43740 10 client.25769 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:06:20.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.252+0000 7f92fcb43740 10 client.25769 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:06:20.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.252+0000 7f92fcb43740 20 client.25769 populate_metadata read hostname 'smithi064' 2024-09-21T04:06:20.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.252+0000 7f92fcb43740 1 --2- 172.21.15.64:0/327089400 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564ee728d9f0 0x564ee7694060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:20.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.252+0000 7f92fcb43740 4 client.25769 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:06:20.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.252+0000 7f92fcb43740 1 -- 172.21.15.64:0/327089400 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ee75eedf0 con 0x564ee728d9f0 2024-09-21T04:06:20.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.252+0000 7f92ef7fe640 1 --2- 172.21.15.64:0/327089400 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564ee728d9f0 0x564ee7694060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:20.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.252+0000 7f92cf7fe640 10 client.25769 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:20.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.252+0000 7f92ef7fe640 1 --2- 172.21.15.64:0/327089400 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564ee728d9f0 0x564ee7694060 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=637eccd2a5789aa6 server_cookie=947dba9f42d9f477 in_seq=0 out_seq=0 2024-09-21T04:06:20.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.253+0000 7f92cf7fe640 1 -- 172.21.15.64:0/327089400 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564ee75eedf0 con 0x564ee728d9f0 2024-09-21T04:06:20.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.253+0000 7f92cf7fe640 10 client.25769 handle_command_reply: tid=0 2024-09-21T04:06:20.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.253+0000 7f92fcb43740 1 client.25769 shutdown 2024-09-21T04:06:20.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.253+0000 7f92fcb43740 1 -- 172.21.15.64:0/327089400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ee72a61b0 msgr2=0x564ee76819b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:20.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.253+0000 7f92fcb43740 1 --2- 172.21.15.64:0/327089400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ee72a61b0 0x564ee76819b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f92d80044d0 tx=0x7f92d8011dd0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.253+0000 7f92fcb43740 1 -- 172.21.15.64:0/327089400 shutdown_connections 2024-09-21T04:06:20.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.253+0000 7f92fcb43740 1 --2- 172.21.15.64:0/327089400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564ee72cf070 0x564ee768dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.253+0000 7f92fcb43740 1 --2- 172.21.15.64:0/327089400 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564ee728d9f0 0x564ee7694060 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.253+0000 7f92fcb43740 1 --2- 172.21.15.64:0/327089400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ee72ae390 0x564ee7681ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.253+0000 7f92fcb43740 1 --2- 172.21.15.64:0/327089400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ee72a61b0 0x564ee76819b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.253+0000 7f92fcb43740 1 -- 172.21.15.64:0/327089400 >> 172.21.15.64:0/327089400 conn(0x564ee72c3460 msgr2=0x564ee76794a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:20.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.253+0000 7f92cf7fe640 0 client.25769 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:20.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.254+0000 7f92fcb43740 1 -- 172.21.15.64:0/327089400 shutdown_connections 2024-09-21T04:06:20.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.254+0000 7f92fcb43740 1 -- 172.21.15.64:0/327089400 wait complete. 2024-09-21T04:06:20.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.254+0000 7f92fcb43740 20 client.25769 trim_cache size 0 max 16384 2024-09-21T04:06:20.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.268+0000 7f92fcb43740 1 Processor -- start 2024-09-21T04:06:20.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.268+0000 7f92fcb43740 1 -- start start 2024-09-21T04:06:20.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.268+0000 7f92fcb43740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564ee72a61b0 0x564ee7677cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:20.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.268+0000 7f92fcb43740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ee72ae390 0x564ee76835e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:20.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.268+0000 7f92fcb43740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ee72cf070 0x564ee7673760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:20.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.269+0000 7f92fcb43740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01b4450 con 0x564ee72cf070 2024-09-21T04:06:20.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.269+0000 7f92fcb43740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01b1c90 con 0x564ee72ae390 2024-09-21T04:06:20.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.269+0000 7f92fcb43740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01afae0 con 0x564ee72a61b0 2024-09-21T04:06:20.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.269+0000 7f92ef7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ee72ae390 0x564ee76835e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:20.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.269+0000 7f92ef7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ee72ae390 0x564ee76835e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:48860/0 (socket says 172.21.15.64:48860) 2024-09-21T04:06:20.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.269+0000 7f92ef7fe640 1 -- 172.21.15.64:0/3689322747 learned_addr learned my addr 172.21.15.64:0/3689322747 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:20.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.269+0000 7f92f4b6c640 1 --2- 172.21.15.64:0/3689322747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ee72cf070 0x564ee7673760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:20.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.269+0000 7f92f4b6c640 1 --2- 172.21.15.64:0/3689322747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564ee72a61b0 0x564ee7677cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:20.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.269+0000 7f92ef7fe640 1 -- 172.21.15.64:0/3689322747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564ee72a61b0 msgr2=0x564ee7677cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:20.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.269+0000 7f92ef7fe640 1 --2- 172.21.15.64:0/3689322747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564ee72a61b0 0x564ee7677cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.269+0000 7f92ef7fe640 1 -- 172.21.15.64:0/3689322747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ee72cf070 msgr2=0x564ee7673760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:20.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.269+0000 7f92ef7fe640 1 --2- 172.21.15.64:0/3689322747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ee72cf070 0x564ee7673760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.269+0000 7f92ef7fe640 1 -- 172.21.15.64:0/3689322747 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ee7682c70 con 0x564ee72ae390 2024-09-21T04:06:20.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.269+0000 7f92f4b6c640 1 --2- 172.21.15.64:0/3689322747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564ee72a61b0 0x564ee7677cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:06:20.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.269+0000 7f92ef7fe640 1 --2- 172.21.15.64:0/3689322747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ee72ae390 0x564ee76835e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f92f0107e60 tx=0x7f92e0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3916f292bec4155a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:20.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.269+0000 7f92e5ffb640 1 -- 172.21.15.64:0/3689322747 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e00122d0 con 0x564ee72ae390 2024-09-21T04:06:20.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.270+0000 7f92e5ffb640 1 -- 172.21.15.64:0/3689322747 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92e000f7f0 con 0x564ee72ae390 2024-09-21T04:06:20.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.270+0000 7f92e5ffb640 1 -- 172.21.15.64:0/3689322747 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e00038e0 con 0x564ee72ae390 2024-09-21T04:06:20.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.270+0000 7f92fcb43740 1 -- 172.21.15.64:0/3689322747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ee72ae390 msgr2=0x564ee76835e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:20.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.271+0000 7f92fcb43740 1 --2- 172.21.15.64:0/3689322747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ee72ae390 0x564ee76835e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f92f0107e60 tx=0x7f92e0005f90 comp rx=0 tx=0).stop 2024-09-21T04:06:20.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.271+0000 7f92fcb43740 1 -- 172.21.15.64:0/3689322747 shutdown_connections 2024-09-21T04:06:20.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.271+0000 7f92fcb43740 1 --2- 172.21.15.64:0/3689322747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ee72cf070 0x564ee7673760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.271+0000 7f92fcb43740 1 --2- 172.21.15.64:0/3689322747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ee72ae390 0x564ee76835e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.271+0000 7f92fcb43740 1 --2- 172.21.15.64:0/3689322747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564ee72a61b0 0x564ee7677cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.271+0000 7f92fcb43740 1 -- 172.21.15.64:0/3689322747 >> 172.21.15.64:0/3689322747 conn(0x564ee72c3460 msgr2=0x564ee7682ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:20.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.271+0000 7f92fcb43740 1 -- 172.21.15.64:0/3689322747 shutdown_connections 2024-09-21T04:06:20.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.271+0000 7f92fcb43740 1 -- 172.21.15.64:0/3689322747 wait complete. 2024-09-21T04:06:20.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.271+0000 7f92fcb43740 1 -- start start 2024-09-21T04:06:20.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.271+0000 7f92fcb43740 1 Processor -- start 2024-09-21T04:06:20.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.272+0000 7f92fcb43740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ee72a61b0 0x564ee76854c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:20.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.272+0000 7f92fcb43740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564ee72ae390 0x564ee7685a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:20.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.272+0000 7f92fcb43740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ee72cf070 0x564ee76fca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:20.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.272+0000 7f92fcb43740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01b0150 con 0x564ee72a61b0 2024-09-21T04:06:20.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.272+0000 7f92fcb43740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01b15c0 con 0x564ee72cf070 2024-09-21T04:06:20.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.272+0000 7f92fcb43740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01b0820 con 0x564ee72ae390 2024-09-21T04:06:20.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.272+0000 7f92fcb43740 10 client.0 _ll_register_callbacks cb 0x7f92e00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:06:20.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.272+0000 7f92f4b6c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ee72a61b0 0x564ee76854c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:20.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.272+0000 7f92ef7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564ee72ae390 0x564ee7685a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:20.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.272+0000 7f92f4b6c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ee72a61b0 0x564ee76854c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41962/0 (socket says 172.21.15.64:41962) 2024-09-21T04:06:20.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.272+0000 7f92ef7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564ee72ae390 0x564ee7685a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:42476/0 (socket says 172.21.15.64:42476) 2024-09-21T04:06:20.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.272+0000 7f92f4b6c640 1 -- 172.21.15.64:0/2668814531 learned_addr learned my addr 172.21.15.64:0/2668814531 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:20.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.272+0000 7f92f4b6c640 1 --2- 172.21.15.64:0/2668814531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ee72cf070 0x564ee76fca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:20.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.273+0000 7f92f4b6c640 1 -- 172.21.15.64:0/2668814531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564ee72ae390 msgr2=0x564ee7685a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:20.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.273+0000 7f92f4b6c640 1 --2- 172.21.15.64:0/2668814531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564ee72ae390 0x564ee7685a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.273+0000 7f92f4b6c640 1 -- 172.21.15.64:0/2668814531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ee72cf070 msgr2=0x564ee76fca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:20.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.273+0000 7f92f4b6c640 1 --2- 172.21.15.64:0/2668814531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ee72cf070 0x564ee76fca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.273+0000 7f92f4b6c640 1 -- 172.21.15.64:0/2668814531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ee76fcf50 con 0x564ee72a61b0 2024-09-21T04:06:20.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.273+0000 7f92f4b6c640 1 --2- 172.21.15.64:0/2668814531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ee72a61b0 0x564ee76854c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f92d8011da0 tx=0x7f92d8011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9328f50eff3fe5d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:20.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.273+0000 7f92cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:06:20.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.273+0000 7f92cf7fe640 1 -- 172.21.15.64:0/2668814531 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8036040 con 0x564ee72a61b0 2024-09-21T04:06:20.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.273+0000 7f92cf7fe640 1 -- 172.21.15.64:0/2668814531 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f92d802cdb0 con 0x564ee72a61b0 2024-09-21T04:06:20.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.273+0000 7f92fcb43740 1 -- 172.21.15.64:0/2668814531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92e000f7f0 con 0x564ee72a61b0 2024-09-21T04:06:20.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.273+0000 7f92cf7fe640 1 -- 172.21.15.64:0/2668814531 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f92d8046050 con 0x564ee72a61b0 2024-09-21T04:06:20.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.273+0000 7f92fcb43740 10 client.16065 fetch_fsmap learned FSMap version 30 2024-09-21T04:06:20.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.273+0000 7f92fcb43740 1 -- 172.21.15.64:0/2668814531 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564ee76fe130 con 0x564ee72a61b0 2024-09-21T04:06:20.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.274+0000 7f92cf7fe640 1 -- 172.21.15.64:0/2668814531 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92e000f7f0 con 0x564ee72a61b0 2024-09-21T04:06:20.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.274+0000 7f92cf7fe640 1 -- 172.21.15.64:0/2668814531 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f92d803c070 con 0x564ee72a61b0 2024-09-21T04:06:20.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.274+0000 7f92fcb43740 10 client.16065 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:06:20.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.274+0000 7f92fcb43740 10 client.16065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:06:20.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.274+0000 7f92fcb43740 20 client.16065 populate_metadata read hostname 'smithi064' 2024-09-21T04:06:20.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.274+0000 7f92fcb43740 1 --2- 172.21.15.64:0/2668814531 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564ee7262da0 0x564ee77008d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:20.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.274+0000 7f92fcb43740 4 client.16065 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:06:20.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.274+0000 7f92fcb43740 1 -- 172.21.15.64:0/2668814531 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ee75ea8a0 con 0x564ee7262da0 2024-09-21T04:06:20.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.274+0000 7f92ef7fe640 1 --2- 172.21.15.64:0/2668814531 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564ee7262da0 0x564ee77008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:20.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.275+0000 7f92ef7fe640 1 --2- 172.21.15.64:0/2668814531 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564ee7262da0 0x564ee77008d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10779f57791e2856 server_cookie=52b1186906b024c5 in_seq=0 out_seq=0 2024-09-21T04:06:20.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.275+0000 7f92cf7fe640 10 client.16065 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:20.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.275+0000 7f92cf7fe640 1 -- 172.21.15.64:0/2668814531 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ee75ea8a0 con 0x564ee7262da0 2024-09-21T04:06:20.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.275+0000 7f92cf7fe640 10 client.16065 handle_command_reply: tid=0 2024-09-21T04:06:20.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.275+0000 7f92fcb43740 1 client.16065 shutdown 2024-09-21T04:06:20.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.275+0000 7f92fcb43740 1 -- 172.21.15.64:0/2668814531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ee72a61b0 msgr2=0x564ee76854c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:20.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.275+0000 7f92fcb43740 1 --2- 172.21.15.64:0/2668814531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ee72a61b0 0x564ee76854c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f92d8011da0 tx=0x7f92d8011dd0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.275+0000 7f92fcb43740 1 -- 172.21.15.64:0/2668814531 shutdown_connections 2024-09-21T04:06:20.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.275+0000 7f92fcb43740 1 --2- 172.21.15.64:0/2668814531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564ee72cf070 0x564ee76fca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.275+0000 7f92fcb43740 1 --2- 172.21.15.64:0/2668814531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564ee72ae390 0x564ee7685a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.275+0000 7f92fcb43740 1 --2- 172.21.15.64:0/2668814531 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564ee7262da0 0x564ee77008d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.276+0000 7f92fcb43740 1 --2- 172.21.15.64:0/2668814531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564ee72a61b0 0x564ee76854c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.276+0000 7f92fcb43740 1 -- 172.21.15.64:0/2668814531 >> 172.21.15.64:0/2668814531 conn(0x564ee72c3460 msgr2=0x564ee7677a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:20.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.276+0000 7f92cf7fe640 0 client.16065 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:20.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.276+0000 7f92fcb43740 1 -- 172.21.15.64:0/2668814531 shutdown_connections 2024-09-21T04:06:20.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.276+0000 7f92fcb43740 1 -- 172.21.15.64:0/2668814531 wait complete. 2024-09-21T04:06:20.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.276+0000 7f92fcb43740 20 client.16065 trim_cache size 0 max 16384 2024-09-21T04:06:20.282 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:06:20.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.276+0000 7f92f65cf640 1 -- 172.21.15.64:0/1857874423 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f92dc0b6680 msgr2=0x7f92dc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:20.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.277+0000 7f92f65cf640 1 --2- 172.21.15.64:0/1857874423 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f92dc0b6680 0x7f92dc0b8b90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f92e8106350 tx=0x7f92e810c6d0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.277+0000 7f92f65cf640 1 -- 172.21.15.64:0/1857874423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f0103bb0 msgr2=0x7f92f019c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:20.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.277+0000 7f92f65cf640 1 --2- 172.21.15.64:0/1857874423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f0103bb0 0x7f92f019c710 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f92e802ade0 tx=0x7f92e8031770 comp rx=0 tx=0).stop 2024-09-21T04:06:20.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.277+0000 7f92f65cf640 1 -- 172.21.15.64:0/1857874423 shutdown_connections 2024-09-21T04:06:20.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.277+0000 7f92f65cf640 1 --2- 172.21.15.64:0/1857874423 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f92dc0b6680 0x7f92dc0b8b90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.277+0000 7f92f65cf640 1 --2- 172.21.15.64:0/1857874423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92f0109e40 0x7f92f01a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.277+0000 7f92f65cf640 1 --2- 172.21.15.64:0/1857874423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92f01094f0 0x7f92f019ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.277+0000 7f92f65cf640 1 --2- 172.21.15.64:0/1857874423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f0103bb0 0x7f92f019c710 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:20.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.277+0000 7f92f65cf640 1 -- 172.21.15.64:0/1857874423 >> 172.21.15.64:0/1857874423 conn(0x7f92f00ff600 msgr2=0x7f92f0108bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:20.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.277+0000 7f92f65cf640 1 -- 172.21.15.64:0/1857874423 shutdown_connections 2024-09-21T04:06:20.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:20.277+0000 7f92f65cf640 1 -- 172.21.15.64:0/1857874423 wait complete. 2024-09-21T04:06:20.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:20 smithi119 ceph-mon[34811]: pgmap v390: 97 pgs: 97 active+clean; 199 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 397 op/s 2024-09-21T04:06:20.513 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:20 smithi064 ceph-mon[31180]: pgmap v390: 97 pgs: 97 active+clean; 199 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 397 op/s 2024-09-21T04:06:20.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:20 smithi131 ceph-mon[35193]: pgmap v390: 97 pgs: 97 active+clean; 199 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 397 op/s 2024-09-21T04:06:21.172 INFO:tasks.workunit.client.0.smithi064.stdout: 49 628518 654 412991 496 249070 1525 2024-09-21T04:06:22.096 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T04:06:22.096 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T04:06:22.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:22 smithi119 ceph-mon[34811]: pgmap v391: 97 pgs: 97 active+clean; 203 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.0 MiB/s wr, 333 op/s 2024-09-21T04:06:22.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:22 smithi064 ceph-mon[31180]: pgmap v391: 97 pgs: 97 active+clean; 203 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.0 MiB/s wr, 333 op/s 2024-09-21T04:06:22.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:22 smithi131 ceph-mon[35193]: pgmap v391: 97 pgs: 97 active+clean; 203 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.0 MiB/s wr, 333 op/s 2024-09-21T04:06:23.142 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:06:23.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.716+0000 7f26d1900640 1 -- 172.21.15.64:0/754992072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26cc101da0 msgr2=0x7f26cc1021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:23.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.716+0000 7f26d1900640 1 --2- 172.21.15.64:0/754992072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26cc101da0 0x7f26cc1021b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f26c0009860 tx=0x7f26c002a340 comp rx=0 tx=0).stop 2024-09-21T04:06:23.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.716+0000 7f26d1900640 1 -- 172.21.15.64:0/754992072 shutdown_connections 2024-09-21T04:06:23.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.716+0000 7f26d1900640 1 --2- 172.21.15.64:0/754992072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26cc102890 0x7f26cc112530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.716+0000 7f26d1900640 1 --2- 172.21.15.64:0/754992072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26cc101da0 0x7f26cc1021b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.716+0000 7f26d1900640 1 --2- 172.21.15.64:0/754992072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26cc104690 0x7f26cc104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.716+0000 7f26d1900640 1 -- 172.21.15.64:0/754992072 >> 172.21.15.64:0/754992072 conn(0x7f26cc077c60 msgr2=0x7f26cc0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:23.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.716+0000 7f26d1900640 1 -- 172.21.15.64:0/754992072 shutdown_connections 2024-09-21T04:06:23.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.716+0000 7f26d1900640 1 -- 172.21.15.64:0/754992072 wait complete. 2024-09-21T04:06:23.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.717+0000 7f26d1900640 1 Processor -- start 2024-09-21T04:06:23.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.718+0000 7f26d1900640 1 -- start start 2024-09-21T04:06:23.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.718+0000 7f26d1900640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26cc101da0 0x7f26cc19c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:23.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.718+0000 7f26d1900640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26cc102890 0x7f26cc19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:23.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.718+0000 7f26d1900640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26cc104690 0x7f26cc1a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:23.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.718+0000 7f26d1900640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc115c80 con 0x7f26cc101da0 2024-09-21T04:06:23.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.718+0000 7f26d1900640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc115b00 con 0x7f26cc104690 2024-09-21T04:06:23.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.718+0000 7f26d1900640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc115e00 con 0x7f26cc102890 2024-09-21T04:06:23.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.719+0000 7f26caffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26cc101da0 0x7f26cc19c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:23.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.719+0000 7f26caffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26cc101da0 0x7f26cc19c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59454/0 (socket says 172.21.15.64:59454) 2024-09-21T04:06:23.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.719+0000 7f26caffd640 1 -- 172.21.15.64:0/611224943 learned_addr learned my addr 172.21.15.64:0/611224943 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:23.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.719+0000 7f26ca7fc640 1 --2- 172.21.15.64:0/611224943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26cc102890 0x7f26cc19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:23.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.719+0000 7f26cb7fe640 1 --2- 172.21.15.64:0/611224943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26cc104690 0x7f26cc1a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:23.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.719+0000 7f26ca7fc640 1 -- 172.21.15.64:0/611224943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26cc104690 msgr2=0x7f26cc1a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:23.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.719+0000 7f26ca7fc640 1 --2- 172.21.15.64:0/611224943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26cc104690 0x7f26cc1a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.719+0000 7f26ca7fc640 1 -- 172.21.15.64:0/611224943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26cc101da0 msgr2=0x7f26cc19c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:23.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.719+0000 7f26ca7fc640 1 --2- 172.21.15.64:0/611224943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26cc101da0 0x7f26cc19c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.719+0000 7f26ca7fc640 1 -- 172.21.15.64:0/611224943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26cc1a25a0 con 0x7f26cc102890 2024-09-21T04:06:23.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.719+0000 7f26ca7fc640 1 --2- 172.21.15.64:0/611224943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26cc102890 0x7f26cc19cbd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f26c002a860 tx=0x7f26c0032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:23.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.720+0000 7f26d08fe640 1 -- 172.21.15.64:0/611224943 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c0040020 con 0x7f26cc102890 2024-09-21T04:06:23.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.720+0000 7f26d1900640 1 -- 172.21.15.64:0/611224943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26cc1a2830 con 0x7f26cc102890 2024-09-21T04:06:23.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.720+0000 7f26d1900640 1 -- 172.21.15.64:0/611224943 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26cc1a8ff0 con 0x7f26cc102890 2024-09-21T04:06:23.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.720+0000 7f26d08fe640 1 -- 172.21.15.64:0/611224943 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26c0030e10 con 0x7f26cc102890 2024-09-21T04:06:23.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.720+0000 7f26d08fe640 1 -- 172.21.15.64:0/611224943 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c0037040 con 0x7f26cc102890 2024-09-21T04:06:23.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.724+0000 7f26d08fe640 1 -- 172.21.15.64:0/611224943 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f26c0043020 con 0x7f26cc102890 2024-09-21T04:06:23.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.724+0000 7f26d7e74740 1 Processor -- start 2024-09-21T04:06:23.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.725+0000 7f26d7e74740 1 -- start start 2024-09-21T04:06:23.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.725+0000 7f26d7e74740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556682967350 0x556682d0e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:23.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.725+0000 7f26d7e74740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668293e490 0x556682d0e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:23.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.725+0000 7f26d7e74740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556682946670 0x556682d0eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:23.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.725+0000 7f26cb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556682967350 0x556682d0e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:23.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.725+0000 7f26caffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668293e490 0x556682d0e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:23.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.725+0000 7f26caffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668293e490 0x556682d0e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:39688/0 (socket says 172.21.15.64:39688) 2024-09-21T04:06:23.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.725+0000 7f26cb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556682967350 0x556682d0e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59460/0 (socket says 172.21.15.64:59460) 2024-09-21T04:06:23.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.725+0000 7f26cb7fe640 1 -- 172.21.15.64:0/1456625408 learned_addr learned my addr 172.21.15.64:0/1456625408 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:23.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.725+0000 7f26ca7fc640 1 --2- 172.21.15.64:0/1456625408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556682946670 0x556682d0eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:23.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.725+0000 7f26d7e74740 1 -- 172.21.15.64:0/1456625408 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556682c84e60 con 0x556682967350 2024-09-21T04:06:23.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.725+0000 7f26d7e74740 1 -- 172.21.15.64:0/1456625408 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556682c74290 con 0x55668293e490 2024-09-21T04:06:23.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.725+0000 7f26d7e74740 1 -- 172.21.15.64:0/1456625408 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556682c780a0 con 0x556682946670 2024-09-21T04:06:23.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.726+0000 7f26d08fe640 1 --2- 172.21.15.64:0/611224943 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f26b40b6680 0x7f26b40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:23.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.726+0000 7f26caffd640 1 -- 172.21.15.64:0/1456625408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556682946670 msgr2=0x556682d0eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:23.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.726+0000 7f26caffd640 1 --2- 172.21.15.64:0/1456625408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556682946670 0x556682d0eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.726+0000 7f26caffd640 1 -- 172.21.15.64:0/1456625408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556682967350 msgr2=0x556682d0e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:23.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.726+0000 7f26caffd640 1 --2- 172.21.15.64:0/1456625408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556682967350 0x556682d0e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.726+0000 7f26caffd640 1 -- 172.21.15.64:0/1456625408 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566829d2930 con 0x55668293e490 2024-09-21T04:06:23.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.726+0000 7f26cb7fe640 1 --2- 172.21.15.64:0/611224943 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f26b40b6680 0x7f26b40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:23.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.726+0000 7f26d08fe640 1 -- 172.21.15.64:0/611224943 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f26c0107360 con 0x7f26cc102890 2024-09-21T04:06:23.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.726+0000 7f26caffd640 1 --2- 172.21.15.64:0/1456625408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668293e490 0x556682d0e970 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f26bc009c80 tx=0x7f26bc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54a028016ac5b256 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:23.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.727+0000 7f26b1ffb640 1 -- 172.21.15.64:0/1456625408 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc004230 con 0x55668293e490 2024-09-21T04:06:23.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.727+0000 7f26b1ffb640 1 -- 172.21.15.64:0/1456625408 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26bc0043d0 con 0x55668293e490 2024-09-21T04:06:23.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.727+0000 7f26b1ffb640 1 -- 172.21.15.64:0/1456625408 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc004d80 con 0x55668293e490 2024-09-21T04:06:23.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.729+0000 7f26cb7fe640 1 --2- 172.21.15.64:0/611224943 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f26b40b6680 0x7f26b40b8b90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f26c400d580 tx=0x7f26c4011040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:23.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.729+0000 7f26d7e74740 1 -- 172.21.15.64:0/1456625408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668293e490 msgr2=0x556682d0e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:23.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.729+0000 7f26d7e74740 1 --2- 172.21.15.64:0/1456625408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668293e490 0x556682d0e970 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f26bc009c80 tx=0x7f26bc011810 comp rx=0 tx=0).stop 2024-09-21T04:06:23.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.729+0000 7f26d7e74740 1 -- 172.21.15.64:0/1456625408 shutdown_connections 2024-09-21T04:06:23.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.729+0000 7f26d7e74740 1 --2- 172.21.15.64:0/1456625408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556682946670 0x556682d0eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.729+0000 7f26d7e74740 1 --2- 172.21.15.64:0/1456625408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668293e490 0x556682d0e970 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.729+0000 7f26d7e74740 1 --2- 172.21.15.64:0/1456625408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556682967350 0x556682d0e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.729+0000 7f26d7e74740 1 -- 172.21.15.64:0/1456625408 >> 172.21.15.64:0/1456625408 conn(0x55668295b740 msgr2=0x556682d0c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:23.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.734+0000 7f26d7e74740 1 -- 172.21.15.64:0/1456625408 shutdown_connections 2024-09-21T04:06:23.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.736+0000 7f26d7e74740 1 -- 172.21.15.64:0/1456625408 wait complete. 2024-09-21T04:06:23.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.737+0000 7f26d7e74740 1 -- start start 2024-09-21T04:06:23.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.737+0000 7f26d7e74740 1 Processor -- start 2024-09-21T04:06:23.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.738+0000 7f26d7e74740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55668293e490 0x556682d22f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:23.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.738+0000 7f26caffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55668293e490 0x556682d22f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:23.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.738+0000 7f26caffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55668293e490 0x556682d22f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59464/0 (socket says 172.21.15.64:59464) 2024-09-21T04:06:23.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.738+0000 7f26caffd640 1 -- 172.21.15.64:0/3140685743 learned_addr learned my addr 172.21.15.64:0/3140685743 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:23.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.738+0000 7f26d7e74740 1 --2- 172.21.15.64:0/3140685743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556682946670 0x556682d23500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:23.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.739+0000 7f26d7e74740 1 --2- 172.21.15.64:0/3140685743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556682967350 0x556682d28520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:23.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.739+0000 7f26d7e74740 1 -- 172.21.15.64:0/3140685743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556682ad5890 con 0x55668293e490 2024-09-21T04:06:23.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.739+0000 7f26d7e74740 1 -- 172.21.15.64:0/3140685743 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556682c0ed40 con 0x556682967350 2024-09-21T04:06:23.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.739+0000 7f26d7e74740 1 -- 172.21.15.64:0/3140685743 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556682d1cc10 con 0x556682946670 2024-09-21T04:06:23.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.739+0000 7f26cb7fe640 1 --2- 172.21.15.64:0/3140685743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556682967350 0x556682d28520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:23.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.740+0000 7f26caffd640 1 -- 172.21.15.64:0/3140685743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556682946670 msgr2=0x556682d23500 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T04:06:23.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.740+0000 7f26caffd640 1 --2- 172.21.15.64:0/3140685743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556682946670 0x556682d23500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.740+0000 7f26caffd640 1 -- 172.21.15.64:0/3140685743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556682967350 msgr2=0x556682d28520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:23.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.740+0000 7f26caffd640 1 --2- 172.21.15.64:0/3140685743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556682967350 0x556682d28520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.740+0000 7f26caffd640 1 -- 172.21.15.64:0/3140685743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556682d28b90 con 0x55668293e490 2024-09-21T04:06:23.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.740+0000 7f26caffd640 1 --2- 172.21.15.64:0/3140685743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55668293e490 0x556682d22f20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f26bc004880 tx=0x7f26bc019a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e45afa36929340 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:23.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.741+0000 7f26d7e74740 10 client.0 _ll_register_callbacks cb 0x556682c72c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:06:23.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.741+0000 7f26d7e74740 1 -- 172.21.15.64:0/3140685743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26bc0043d0 con 0x55668293e490 2024-09-21T04:06:23.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.741+0000 7f26aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:06:23.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.743+0000 7f26aaffd640 1 -- 172.21.15.64:0/3140685743 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc028020 con 0x55668293e490 2024-09-21T04:06:23.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.743+0000 7f26aaffd640 1 -- 172.21.15.64:0/3140685743 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f26bc019df0 con 0x55668293e490 2024-09-21T04:06:23.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.743+0000 7f26aaffd640 1 -- 172.21.15.64:0/3140685743 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f26bc024420 con 0x55668293e490 2024-09-21T04:06:23.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.743+0000 7f26d7e74740 10 client.16077 fetch_fsmap learned FSMap version 30 2024-09-21T04:06:23.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.744+0000 7f26d7e74740 1 -- 172.21.15.64:0/3140685743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({fsmap=30}) -- 0x556682d16470 con 0x55668293e490 2024-09-21T04:06:23.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.744+0000 7f26aaffd640 1 -- 172.21.15.64:0/3140685743 <== mon.0 v2:172.21.15.64:3300/0 4 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f26bc018070 con 0x55668293e490 2024-09-21T04:06:23.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.745+0000 7f26d7e74740 10 client.16077 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:06:23.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.745+0000 7f26d7e74740 10 client.16077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:06:23.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.745+0000 7f26d7e74740 20 client.16077 populate_metadata read hostname 'smithi064' 2024-09-21T04:06:23.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.745+0000 7f26d7e74740 1 --2- 172.21.15.64:0/3140685743 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556682925cd0 0x556682d197f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:23.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.745+0000 7f26d7e74740 4 client.16077 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:06:23.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.745+0000 7f26d7e74740 1 -- 172.21.15.64:0/3140685743 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556682c86db0 con 0x556682925cd0 2024-09-21T04:06:23.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.746+0000 7f26ca7fc640 1 --2- 172.21.15.64:0/3140685743 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556682925cd0 0x556682d197f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:23.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.747+0000 7f26ca7fc640 1 --2- 172.21.15.64:0/3140685743 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556682925cd0 0x556682d197f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc4a77c508d33c46 server_cookie=a77e11d96d962e5e in_seq=0 out_seq=0 2024-09-21T04:06:23.750 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.747+0000 7f26aaffd640 10 client.16077 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:23.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.747+0000 7f26aaffd640 1 -- 172.21.15.64:0/3140685743 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556682c86db0 con 0x556682925cd0 2024-09-21T04:06:23.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.747+0000 7f26aaffd640 10 client.16077 handle_command_reply: tid=0 2024-09-21T04:06:23.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.747+0000 7f26d7e74740 1 client.16077 shutdown 2024-09-21T04:06:23.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.748+0000 7f26d7e74740 1 -- 172.21.15.64:0/3140685743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55668293e490 msgr2=0x556682d22f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:23.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.748+0000 7f26d7e74740 1 --2- 172.21.15.64:0/3140685743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55668293e490 0x556682d22f20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f26bc004880 tx=0x7f26bc019a50 comp rx=0 tx=0).stop 2024-09-21T04:06:23.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.748+0000 7f26d7e74740 1 -- 172.21.15.64:0/3140685743 shutdown_connections 2024-09-21T04:06:23.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.749+0000 7f26d7e74740 1 --2- 172.21.15.64:0/3140685743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556682967350 0x556682d28520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.749+0000 7f26d7e74740 1 --2- 172.21.15.64:0/3140685743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556682946670 0x556682d23500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.749+0000 7f26d7e74740 1 --2- 172.21.15.64:0/3140685743 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556682925cd0 0x556682d197f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.749+0000 7f26d7e74740 1 --2- 172.21.15.64:0/3140685743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55668293e490 0x556682d22f20 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.749+0000 7f26d7e74740 1 -- 172.21.15.64:0/3140685743 >> 172.21.15.64:0/3140685743 conn(0x55668295b740 msgr2=0x556682d11610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:23.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.749+0000 7f26aaffd640 0 client.16077 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:23.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.749+0000 7f26d7e74740 1 -- 172.21.15.64:0/3140685743 shutdown_connections 2024-09-21T04:06:23.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.749+0000 7f26d7e74740 1 -- 172.21.15.64:0/3140685743 wait complete. 2024-09-21T04:06:23.754 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.751+0000 7f26d7e74740 20 client.16077 trim_cache size 0 max 16384 2024-09-21T04:06:23.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.816+0000 7f26d7e74740 1 Processor -- start 2024-09-21T04:06:23.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.817+0000 7f26d7e74740 1 -- start start 2024-09-21T04:06:23.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.817+0000 7f26d7e74740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668293e490 0x556682d1e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:23.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.817+0000 7f26d7e74740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556682946670 0x556682d1d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:23.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.817+0000 7f26d7e74740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556682967350 0x556682d22f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:23.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.817+0000 7f26d7e74740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc1aff30 con 0x556682946670 2024-09-21T04:06:23.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.817+0000 7f26d7e74740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc1ad770 con 0x55668293e490 2024-09-21T04:06:23.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.817+0000 7f26d7e74740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc1ab5e0 con 0x556682967350 2024-09-21T04:06:23.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.817+0000 7f26caffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668293e490 0x556682d1e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:23.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.817+0000 7f26cb7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556682967350 0x556682d22f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:23.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.817+0000 7f26caffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668293e490 0x556682d1e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:39704/0 (socket says 172.21.15.64:39704) 2024-09-21T04:06:23.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.817+0000 7f26caffd640 1 -- 172.21.15.64:0/2466439573 learned_addr learned my addr 172.21.15.64:0/2466439573 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:23.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.817+0000 7f26ca7fc640 1 --2- 172.21.15.64:0/2466439573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556682946670 0x556682d1d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:23.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.818+0000 7f26caffd640 1 -- 172.21.15.64:0/2466439573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556682967350 msgr2=0x556682d22f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:23.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.818+0000 7f26caffd640 1 --2- 172.21.15.64:0/2466439573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556682967350 0x556682d22f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.818+0000 7f26caffd640 1 -- 172.21.15.64:0/2466439573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556682946670 msgr2=0x556682d1d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:23.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.818+0000 7f26caffd640 1 --2- 172.21.15.64:0/2466439573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556682946670 0x556682d1d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.818+0000 7f26caffd640 1 -- 172.21.15.64:0/2466439573 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556682d06560 con 0x55668293e490 2024-09-21T04:06:23.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.818+0000 7f26caffd640 1 --2- 172.21.15.64:0/2466439573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668293e490 0x556682d1e4d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f26bc0095b0 tx=0x7f26bc024000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a054b54db15fa91 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:23.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.823+0000 7f26b17fa640 1 -- 172.21.15.64:0/2466439573 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc028030 con 0x55668293e490 2024-09-21T04:06:23.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.823+0000 7f26b17fa640 1 -- 172.21.15.64:0/2466439573 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26bc0043d0 con 0x55668293e490 2024-09-21T04:06:23.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.823+0000 7f26b17fa640 1 -- 172.21.15.64:0/2466439573 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc036b20 con 0x55668293e490 2024-09-21T04:06:23.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.824+0000 7f26d7e74740 1 -- 172.21.15.64:0/2466439573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668293e490 msgr2=0x556682d1e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:23.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.824+0000 7f26d7e74740 1 --2- 172.21.15.64:0/2466439573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668293e490 0x556682d1e4d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f26bc0095b0 tx=0x7f26bc024000 comp rx=0 tx=0).stop 2024-09-21T04:06:23.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.825+0000 7f26d7e74740 1 -- 172.21.15.64:0/2466439573 shutdown_connections 2024-09-21T04:06:23.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.825+0000 7f26d7e74740 1 --2- 172.21.15.64:0/2466439573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556682967350 0x556682d22f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.825+0000 7f26d7e74740 1 --2- 172.21.15.64:0/2466439573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556682946670 0x556682d1d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.825+0000 7f26d7e74740 1 --2- 172.21.15.64:0/2466439573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668293e490 0x556682d1e4d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.825+0000 7f26d7e74740 1 -- 172.21.15.64:0/2466439573 >> 172.21.15.64:0/2466439573 conn(0x55668295b740 msgr2=0x556682d1ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:23.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.825+0000 7f26d7e74740 1 -- 172.21.15.64:0/2466439573 shutdown_connections 2024-09-21T04:06:23.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.825+0000 7f26d7e74740 1 -- 172.21.15.64:0/2466439573 wait complete. 2024-09-21T04:06:23.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.826+0000 7f26d7e74740 1 -- start start 2024-09-21T04:06:23.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.827+0000 7f26d7e74740 1 Processor -- start 2024-09-21T04:06:23.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.828+0000 7f26d7e74740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668293e490 0x556682d4f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:23.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.828+0000 7f26caffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668293e490 0x556682d4f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:23.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.828+0000 7f26caffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668293e490 0x556682d4f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:39716/0 (socket says 172.21.15.64:39716) 2024-09-21T04:06:23.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.828+0000 7f26d7e74740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556682946670 0x556682d4fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:23.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.828+0000 7f26caffd640 1 -- 172.21.15.64:0/2022946618 learned_addr learned my addr 172.21.15.64:0/2022946618 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:23.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.828+0000 7f26ca7fc640 1 --2- 172.21.15.64:0/2022946618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556682946670 0x556682d4fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:23.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.828+0000 7f26d7e74740 1 --2- 172.21.15.64:0/2022946618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556682967350 0x556682d4b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:23.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.829+0000 7f26cb7fe640 1 --2- 172.21.15.64:0/2022946618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556682967350 0x556682d4b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:23.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.829+0000 7f26d7e74740 1 -- 172.21.15.64:0/2022946618 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc1abc30 con 0x556682967350 2024-09-21T04:06:23.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.829+0000 7f26d7e74740 1 -- 172.21.15.64:0/2022946618 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc1ad0a0 con 0x55668293e490 2024-09-21T04:06:23.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.829+0000 7f26d7e74740 1 -- 172.21.15.64:0/2022946618 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc1ac300 con 0x556682946670 2024-09-21T04:06:23.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.830+0000 7f26ca7fc640 1 -- 172.21.15.64:0/2022946618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668293e490 msgr2=0x556682d4f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:23.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.830+0000 7f26ca7fc640 1 --2- 172.21.15.64:0/2022946618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668293e490 0x556682d4f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.830+0000 7f26ca7fc640 1 -- 172.21.15.64:0/2022946618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556682967350 msgr2=0x556682d4b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:23.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.830+0000 7f26ca7fc640 1 --2- 172.21.15.64:0/2022946618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556682967350 0x556682d4b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.830+0000 7f26ca7fc640 1 -- 172.21.15.64:0/2022946618 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556682d4bb70 con 0x556682946670 2024-09-21T04:06:23.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.830+0000 7f26cb7fe640 1 --2- 172.21.15.64:0/2022946618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556682967350 0x556682d4b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:06:23.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.831+0000 7f26d7e74740 10 client.0 _ll_register_callbacks cb 0x7f26c4009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:06:23.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.831+0000 7f26d7e74740 1 -- 172.21.15.64:0/2022946618 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26bc0043d0 con 0x556682946670 2024-09-21T04:06:23.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.831+0000 7f26ca7fc640 1 --2- 172.21.15.64:0/2022946618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556682946670 0x556682d4fbb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f26c0104a80 tx=0x7f26c0107720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad2e03f71b57493a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:23.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.831+0000 7f26aaffd640 10 client.25687 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-21T04:06:23.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.832+0000 7f26aaffd640 1 -- 172.21.15.64:0/2022946618 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c00cc8f0 con 0x556682946670 2024-09-21T04:06:23.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.832+0000 7f26aaffd640 1 -- 172.21.15.64:0/2022946618 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f26c00c93f0 con 0x556682946670 2024-09-21T04:06:23.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.832+0000 7f26caffd640 1 --2- 172.21.15.64:0/2022946618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668293e490 0x556682d4f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:06:23.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.832+0000 7f26aaffd640 1 -- 172.21.15.64:0/2022946618 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f26c00c9020 con 0x556682946670 2024-09-21T04:06:23.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.832+0000 7f26d7e74740 10 client.25687 fetch_fsmap learned FSMap version 30 2024-09-21T04:06:23.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.832+0000 7f26d7e74740 1 -- 172.21.15.64:0/2022946618 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=30}) -- 0x556682da7f10 con 0x556682946670 2024-09-21T04:06:23.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.832+0000 7f26aaffd640 1 -- 172.21.15.64:0/2022946618 <== mon.1 v2:172.21.15.131:3300/0 4 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f26c0092ce0 con 0x556682946670 2024-09-21T04:06:23.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.832+0000 7f26d7e74740 10 client.25687 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:06:23.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.832+0000 7f26d7e74740 10 client.25687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:06:23.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.833+0000 7f26d7e74740 20 client.25687 populate_metadata read hostname 'smithi064' 2024-09-21T04:06:23.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.833+0000 7f26d7e74740 1 --2- 172.21.15.64:0/2022946618 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5566828fb080 0x556682daa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:23.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.833+0000 7f26d7e74740 4 client.25687 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T04:06:23.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.833+0000 7f26d7e74740 1 -- 172.21.15.64:0/2022946618 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556682c82860 con 0x5566828fb080 2024-09-21T04:06:23.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.833+0000 7f26caffd640 1 --2- 172.21.15.64:0/2022946618 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5566828fb080 0x556682daa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:23.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.834+0000 7f26caffd640 1 --2- 172.21.15.64:0/2022946618 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5566828fb080 0x556682daa4f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38f354d4fac148f server_cookie=81c35c8efad83efc in_seq=0 out_seq=0 2024-09-21T04:06:23.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.834+0000 7f26aaffd640 10 client.25687 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:23.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.835+0000 7f26aaffd640 1 -- 172.21.15.64:0/2022946618 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556682c82860 con 0x5566828fb080 2024-09-21T04:06:23.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.835+0000 7f26aaffd640 10 client.25687 handle_command_reply: tid=0 2024-09-21T04:06:23.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.835+0000 7f26d7e74740 1 client.25687 shutdown 2024-09-21T04:06:23.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.836+0000 7f26d7e74740 1 -- 172.21.15.64:0/2022946618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556682946670 msgr2=0x556682d4fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:23.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.836+0000 7f26d7e74740 1 --2- 172.21.15.64:0/2022946618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556682946670 0x556682d4fbb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f26c0104a80 tx=0x7f26c0107720 comp rx=0 tx=0).stop 2024-09-21T04:06:23.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.837+0000 7f26d7e74740 1 -- 172.21.15.64:0/2022946618 shutdown_connections 2024-09-21T04:06:23.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.837+0000 7f26d7e74740 1 --2- 172.21.15.64:0/2022946618 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5566828fb080 0x556682daa4f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.837+0000 7f26aaffd640 0 client.25687 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:23.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.837+0000 7f26d7e74740 1 --2- 172.21.15.64:0/2022946618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556682967350 0x556682d4b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.837+0000 7f26d7e74740 1 --2- 172.21.15.64:0/2022946618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556682946670 0x556682d4fbb0 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f26c0104a80 tx=0x7f26c0107720 comp rx=0 tx=0).stop 2024-09-21T04:06:23.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.837+0000 7f26d7e74740 1 --2- 172.21.15.64:0/2022946618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55668293e490 0x556682d4f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.837+0000 7f26d7e74740 1 -- 172.21.15.64:0/2022946618 >> 172.21.15.64:0/2022946618 conn(0x55668295b740 msgr2=0x556682d20120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:23.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.839+0000 7f26d7e74740 1 -- 172.21.15.64:0/2022946618 shutdown_connections 2024-09-21T04:06:23.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.839+0000 7f26d7e74740 1 -- 172.21.15.64:0/2022946618 wait complete. 2024-09-21T04:06:23.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.839+0000 7f26d7e74740 20 client.25687 trim_cache size 0 max 16384 2024-09-21T04:06:23.843 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:06:23.843 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-21T04:06:23.843 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "6bde0fe5-f775-44b1-ad0c-f9e596574dc0", 2024-09-21T04:06:23.843 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-21T04:06:23.843 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:06:23.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.841+0000 7f26d1900640 1 -- 172.21.15.64:0/611224943 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f26b40b6680 msgr2=0x7f26b40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:23.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.841+0000 7f26d1900640 1 --2- 172.21.15.64:0/611224943 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f26b40b6680 0x7f26b40b8b90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f26c400d580 tx=0x7f26c4011040 comp rx=0 tx=0).stop 2024-09-21T04:06:23.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.841+0000 7f26d1900640 1 -- 172.21.15.64:0/611224943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26cc102890 msgr2=0x7f26cc19cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:23.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.841+0000 7f26d1900640 1 --2- 172.21.15.64:0/611224943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26cc102890 0x7f26cc19cbd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f26c002a860 tx=0x7f26c0032d50 comp rx=0 tx=0).stop 2024-09-21T04:06:23.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.841+0000 7f26d1900640 1 -- 172.21.15.64:0/611224943 shutdown_connections 2024-09-21T04:06:23.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.841+0000 7f26d1900640 1 --2- 172.21.15.64:0/611224943 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f26b40b6680 0x7f26b40b8b90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.841+0000 7f26d1900640 1 --2- 172.21.15.64:0/611224943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26cc104690 0x7f26cc1a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.841+0000 7f26d1900640 1 --2- 172.21.15.64:0/611224943 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26cc102890 0x7f26cc19cbd0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.842+0000 7f26d1900640 1 --2- 172.21.15.64:0/611224943 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26cc101da0 0x7f26cc19c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:23.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.842+0000 7f26d1900640 1 -- 172.21.15.64:0/611224943 >> 172.21.15.64:0/611224943 conn(0x7f26cc077c60 msgr2=0x7f26cc0ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:23.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.843+0000 7f26d1900640 1 -- 172.21.15.64:0/611224943 shutdown_connections 2024-09-21T04:06:23.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:23.843+0000 7f26d1900640 1 -- 172.21.15.64:0/611224943 wait complete. 2024-09-21T04:06:24.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:24 smithi119 ceph-mon[34811]: pgmap v392: 97 pgs: 97 active+clean; 210 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.6 MiB/s wr, 582 op/s 2024-09-21T04:06:24.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:24 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:06:24.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:24 smithi064 ceph-mon[31180]: pgmap v392: 97 pgs: 97 active+clean; 210 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.6 MiB/s wr, 582 op/s 2024-09-21T04:06:24.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:24 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:06:24.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:24 smithi131 ceph-mon[35193]: pgmap v392: 97 pgs: 97 active+clean; 210 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.6 MiB/s wr, 582 op/s 2024-09-21T04:06:24.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:24 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:06:24.684 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T04:06:25.045 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:06:25.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:25 smithi119 ceph-mon[34811]: scrub queued for path: / 2024-09-21T04:06:25.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:25 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:06:25.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:25 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:06:25.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:25 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:06:25.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:25 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:06:25.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:25 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:06:25.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:25 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:06:25.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:25 smithi064 ceph-mon[31180]: scrub queued for path: / 2024-09-21T04:06:25.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:25 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:06:25.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:25 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:06:25.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:25 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:06:25.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:25 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:06:25.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:25 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:06:25.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:25 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:06:25.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:25 smithi131 ceph-mon[35193]: scrub queued for path: / 2024-09-21T04:06:25.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:25 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:06:25.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:25 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:06:25.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:25 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:06:25.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:25 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:06:25.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:25 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:06:25.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:25 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:06:25.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.975+0000 7f3c1ab7d640 1 -- 172.21.15.64:0/592068123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c14109df0 msgr2=0x7f3c14110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:25.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.975+0000 7f3c1ab7d640 1 --2- 172.21.15.64:0/592068123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c14109df0 0x7f3c14110620 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009ca0 tx=0x7f3c0c02a8c0 comp rx=0 tx=0).stop 2024-09-21T04:06:25.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.976+0000 7f3c1ab7d640 1 -- 172.21.15.64:0/592068123 shutdown_connections 2024-09-21T04:06:25.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.976+0000 7f3c1ab7d640 1 --2- 172.21.15.64:0/592068123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c14109df0 0x7f3c14110620 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:25.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.976+0000 7f3c1ab7d640 1 --2- 172.21.15.64:0/592068123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c141094a0 0x7f3c141098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:25.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.976+0000 7f3c1ab7d640 1 --2- 172.21.15.64:0/592068123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c14103b60 0x7f3c14103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:25.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.976+0000 7f3c1ab7d640 1 -- 172.21.15.64:0/592068123 >> 172.21.15.64:0/592068123 conn(0x7f3c140ff5d0 msgr2=0x7f3c14101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:25.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.976+0000 7f3c1ab7d640 1 -- 172.21.15.64:0/592068123 shutdown_connections 2024-09-21T04:06:25.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.976+0000 7f3c1ab7d640 1 -- 172.21.15.64:0/592068123 wait complete. 2024-09-21T04:06:25.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.976+0000 7f3c1ab7d640 1 Processor -- start 2024-09-21T04:06:25.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.976+0000 7f3c1ab7d640 1 -- start start 2024-09-21T04:06:25.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.977+0000 7f3c1ab7d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c14103b60 0x7f3c1419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.977+0000 7f3c1ab7d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c141094a0 0x7f3c1419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.977+0000 7f3c1ab7d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c14109df0 0x7f3c1419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.977+0000 7f3c1ab7d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3c14113b00 con 0x7f3c14109df0 2024-09-21T04:06:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.977+0000 7f3c1ab7d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3c14113980 con 0x7f3c141094a0 2024-09-21T04:06:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.977+0000 7f3c1ab7d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3c14113c80 con 0x7f3c14103b60 2024-09-21T04:06:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.977+0000 7f3c18919640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c14103b60 0x7f3c1419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.977+0000 7f3c1911a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c14109df0 0x7f3c1419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.977+0000 7f3c13fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c141094a0 0x7f3c1419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.977+0000 7f3c18919640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c14103b60 0x7f3c1419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:44438/0 (socket says 172.21.15.64:44438) 2024-09-21T04:06:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.977+0000 7f3c18919640 1 -- 172.21.15.64:0/3266639915 learned_addr learned my addr 172.21.15.64:0/3266639915 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.977+0000 7f3c1911a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c14109df0 0x7f3c1419fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59488/0 (socket says 172.21.15.64:59488) 2024-09-21T04:06:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.977+0000 7f3c13fff640 1 -- 172.21.15.64:0/3266639915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c14103b60 msgr2=0x7f3c1419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.977+0000 7f3c13fff640 1 --2- 172.21.15.64:0/3266639915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c14103b60 0x7f3c1419a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:25.982 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.977+0000 7f3c13fff640 1 -- 172.21.15.64:0/3266639915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c14109df0 msgr2=0x7f3c1419fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:25.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.977+0000 7f3c13fff640 1 --2- 172.21.15.64:0/3266639915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c14109df0 0x7f3c1419fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:25.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.977+0000 7f3c13fff640 1 -- 172.21.15.64:0/3266639915 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c141a03c0 con 0x7f3c141094a0 2024-09-21T04:06:25.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.978+0000 7f3c13fff640 1 --2- 172.21.15.64:0/3266639915 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c141094a0 0x7f3c1419a9f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3c1406cc20 tx=0x7f3c0000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:25.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.978+0000 7f3c11ffb640 1 -- 172.21.15.64:0/3266639915 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0000cc40 con 0x7f3c141094a0 2024-09-21T04:06:25.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.978+0000 7f3c1ab7d640 1 -- 172.21.15.64:0/3266639915 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c141a06b0 con 0x7f3c141094a0 2024-09-21T04:06:25.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.978+0000 7f3c11ffb640 1 -- 172.21.15.64:0/3266639915 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c0000cde0 con 0x7f3c141094a0 2024-09-21T04:06:25.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.978+0000 7f3c11ffb640 1 -- 172.21.15.64:0/3266639915 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c00018780 con 0x7f3c141094a0 2024-09-21T04:06:25.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.978+0000 7f3c1ab7d640 1 -- 172.21.15.64:0/3266639915 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c140755f0 con 0x7f3c141094a0 2024-09-21T04:06:25.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.981+0000 7f3c210f1740 1 Processor -- start 2024-09-21T04:06:25.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.981+0000 7f3c210f1740 1 -- start start 2024-09-21T04:06:25.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.981+0000 7f3c11ffb640 1 -- 172.21.15.64:0/3266639915 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f3c00018920 con 0x7f3c141094a0 2024-09-21T04:06:25.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.981+0000 7f3c210f1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555babd34070 0x555bac0db650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:25.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.981+0000 7f3c210f1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555babd0b1b0 0x555bac0dbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:25.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.981+0000 7f3c210f1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555babd13390 0x555bac0dc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:25.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.981+0000 7f3c210f1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555bac051e50 con 0x555babd34070 2024-09-21T04:06:25.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.981+0000 7f3c210f1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x555bac041280 con 0x555babd0b1b0 2024-09-21T04:06:25.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.981+0000 7f3c210f1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555bac045090 con 0x555babd13390 2024-09-21T04:06:25.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.982+0000 7f3c1911a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555babd34070 0x555bac0db650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:25.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.982+0000 7f3c18919640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555babd0b1b0 0x555bac0dbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:25.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.982+0000 7f3c1911a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555babd34070 0x555bac0db650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59498/0 (socket says 172.21.15.64:59498) 2024-09-21T04:06:25.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.982+0000 7f3c1911a640 1 -- 172.21.15.64:0/990304235 learned_addr learned my addr 172.21.15.64:0/990304235 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:25.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.982+0000 7f3c13fff640 1 --2- 172.21.15.64:0/990304235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555babd13390 0x555bac0dc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:25.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.982+0000 7f3c1911a640 1 -- 172.21.15.64:0/990304235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555babd13390 msgr2=0x555bac0dc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:25.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.982+0000 7f3c1911a640 1 --2- 172.21.15.64:0/990304235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555babd13390 0x555bac0dc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:25.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.982+0000 7f3c1911a640 1 -- 172.21.15.64:0/990304235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555babd0b1b0 msgr2=0x555bac0dbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:25.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.982+0000 7f3c1911a640 1 --2- 172.21.15.64:0/990304235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555babd0b1b0 0x555bac0dbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:25.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.982+0000 7f3c1911a640 1 -- 172.21.15.64:0/990304235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555babd9f650 con 0x555babd34070 2024-09-21T04:06:25.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.983+0000 7f3c1911a640 1 --2- 172.21.15.64:0/990304235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555babd34070 0x555bac0db650 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009ca0 tx=0x7f3c0c009430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41efbacc3be3ad33 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:25.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.983+0000 7f3c11ffb640 1 --2- 172.21.15.64:0/3266639915 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3c040b6680 0x7f3c040b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:25.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.983+0000 7f3c18919640 1 --2- 172.21.15.64:0/3266639915 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3c040b6680 0x7f3c040b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:25.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.983+0000 7f3bfeffd640 1 -- 172.21.15.64:0/990304235 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c031cf0 con 0x555babd34070 2024-09-21T04:06:25.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.983+0000 7f3bfeffd640 1 -- 172.21.15.64:0/990304235 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c0c036030 con 0x555babd34070 2024-09-21T04:06:25.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.983+0000 7f3c11ffb640 1 -- 172.21.15.64:0/3266639915 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f3c000e7a60 con 0x7f3c141094a0 2024-09-21T04:06:25.988 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.983+0000 7f3bfeffd640 1 -- 172.21.15.64:0/990304235 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c0042a0 con 0x555babd34070 2024-09-21T04:06:25.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.984+0000 7f3c210f1740 1 -- 172.21.15.64:0/990304235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555babd34070 msgr2=0x555bac0db650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:25.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.984+0000 7f3c18919640 1 --2- 172.21.15.64:0/3266639915 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3c040b6680 0x7f3c040b8b90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x555bac0da2d0 tx=0x7f3c08012040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:25.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.984+0000 7f3c210f1740 1 --2- 172.21.15.64:0/990304235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555babd34070 0x555bac0db650 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009ca0 tx=0x7f3c0c009430 comp rx=0 tx=0).stop 2024-09-21T04:06:25.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.984+0000 7f3c210f1740 1 -- 172.21.15.64:0/990304235 shutdown_connections 2024-09-21T04:06:25.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.984+0000 7f3c210f1740 1 --2- 172.21.15.64:0/990304235 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555babd13390 0x555bac0dc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:25.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.984+0000 7f3c210f1740 1 --2- 172.21.15.64:0/990304235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555babd0b1b0 0x555bac0dbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:25.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.984+0000 7f3c210f1740 1 --2- 172.21.15.64:0/990304235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555babd34070 0x555bac0db650 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:25.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.984+0000 7f3c210f1740 1 -- 172.21.15.64:0/990304235 >> 172.21.15.64:0/990304235 conn(0x555babd28460 msgr2=0x555bac0d9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:25.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.985+0000 7f3c210f1740 1 -- 172.21.15.64:0/990304235 shutdown_connections 2024-09-21T04:06:25.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.985+0000 7f3c210f1740 1 -- 172.21.15.64:0/990304235 wait complete. 2024-09-21T04:06:25.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.985+0000 7f3c210f1740 1 -- start start 2024-09-21T04:06:25.989 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.985+0000 7f3c210f1740 1 Processor -- start 2024-09-21T04:06:25.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.986+0000 7f3c210f1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555babd0b1b0 0x555bac0e6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:25.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.986+0000 7f3c210f1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555babd13390 0x555bac0e7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:25.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.986+0000 7f3c210f1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555babd34070 0x555bac0f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:25.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.986+0000 7f3c210f1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555babea2700 con 0x555babd34070 2024-09-21T04:06:25.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.986+0000 7f3c210f1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x555babfdc110 con 0x555babd0b1b0 2024-09-21T04:06:25.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.986+0000 7f3c210f1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555bac0e9c50 con 0x555babd13390 2024-09-21T04:06:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.986+0000 7f3c210f1740 10 client.0 _ll_register_callbacks cb 0x555babff5990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:06:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.986+0000 7f3c18919640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555babd0b1b0 0x555bac0e6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.986+0000 7f3c1911a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555babd34070 0x555bac0f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.986+0000 7f3c18919640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555babd0b1b0 0x555bac0e6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:39764/0 (socket says 172.21.15.64:39764) 2024-09-21T04:06:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.986+0000 7f3c18919640 1 -- 172.21.15.64:0/2200416916 learned_addr learned my addr 172.21.15.64:0/2200416916 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.986+0000 7f3c13fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555babd13390 0x555bac0e7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.987+0000 7f3c18919640 1 -- 172.21.15.64:0/2200416916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555babd13390 msgr2=0x555bac0e7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.987+0000 7f3c18919640 1 --2- 172.21.15.64:0/2200416916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555babd13390 0x555bac0e7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.987+0000 7f3c18919640 1 -- 172.21.15.64:0/2200416916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555babd34070 msgr2=0x555bac0f2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.987+0000 7f3c18919640 1 --2- 172.21.15.64:0/2200416916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555babd34070 0x555bac0f2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.987+0000 7f3c18919640 1 -- 172.21.15.64:0/2200416916 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bac009c40 con 0x555babd0b1b0 2024-09-21T04:06:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.987+0000 7f3c18919640 1 --2- 172.21.15.64:0/2200416916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555babd0b1b0 0x555bac0e6e30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3c0801dbf0 tx=0x7f3c08004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d4273a7166494ff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.987+0000 7f3bf3fff640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:06:25.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.987+0000 7f3bf3fff640 1 -- 172.21.15.64:0/2200416916 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0802b030 con 0x555babd0b1b0 2024-09-21T04:06:25.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.987+0000 7f3bf3fff640 1 -- 172.21.15.64:0/2200416916 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f3c0800b930 con 0x555babd0b1b0 2024-09-21T04:06:25.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.987+0000 7f3c210f1740 1 -- 172.21.15.64:0/2200416916 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c0c036030 con 0x555babd0b1b0 2024-09-21T04:06:25.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.987+0000 7f3bf3fff640 1 -- 172.21.15.64:0/2200416916 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3c08035050 con 0x555babd0b1b0 2024-09-21T04:06:25.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.987+0000 7f3c210f1740 10 client.25805 fetch_fsmap learned FSMap version 30 2024-09-21T04:06:25.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.987+0000 7f3c210f1740 1 -- 172.21.15.64:0/2200416916 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555bac0f5a90 con 0x555babd0b1b0 2024-09-21T04:06:25.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.988+0000 7f3bf3fff640 1 -- 172.21.15.64:0/2200416916 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c0c036030 con 0x555babd0b1b0 2024-09-21T04:06:25.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.988+0000 7f3bf3fff640 1 -- 172.21.15.64:0/2200416916 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f3c08030400 con 0x555babd0b1b0 2024-09-21T04:06:25.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.988+0000 7f3c210f1740 10 client.25805 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:06:25.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.988+0000 7f3c210f1740 10 client.25805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:06:25.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.988+0000 7f3c210f1740 20 client.25805 populate_metadata read hostname 'smithi064' 2024-09-21T04:06:25.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.988+0000 7f3c210f1740 1 --2- 172.21.15.64:0/2200416916 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x555babcf29f0 0x555bac0f9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:25.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.988+0000 7f3c210f1740 4 client.25805 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:06:25.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.988+0000 7f3c210f1740 1 -- 172.21.15.64:0/2200416916 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bac053da0 con 0x555babcf29f0 2024-09-21T04:06:25.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.988+0000 7f3c13fff640 1 --2- 172.21.15.64:0/2200416916 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x555babcf29f0 0x555bac0f9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:25.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.989+0000 7f3bf3fff640 10 client.25805 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:25.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.989+0000 7f3c13fff640 1 --2- 172.21.15.64:0/2200416916 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x555babcf29f0 0x555bac0f9040 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8df079ed15e9e2a5 server_cookie=3f92202426b0bc8f in_seq=0 out_seq=0 2024-09-21T04:06:25.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.990+0000 7f3bf3fff640 1 -- 172.21.15.64:0/2200416916 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555bac053da0 con 0x555babcf29f0 2024-09-21T04:06:25.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.990+0000 7f3bf3fff640 10 client.25805 handle_command_reply: tid=0 2024-09-21T04:06:25.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.990+0000 7f3c210f1740 1 client.25805 shutdown 2024-09-21T04:06:25.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.990+0000 7f3c210f1740 1 -- 172.21.15.64:0/2200416916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555babd0b1b0 msgr2=0x555bac0e6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:25.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.990+0000 7f3c210f1740 1 --2- 172.21.15.64:0/2200416916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555babd0b1b0 0x555bac0e6e30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3c0801dbf0 tx=0x7f3c08004730 comp rx=0 tx=0).stop 2024-09-21T04:06:25.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.990+0000 7f3c210f1740 1 -- 172.21.15.64:0/2200416916 shutdown_connections 2024-09-21T04:06:25.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.990+0000 7f3c210f1740 1 --2- 172.21.15.64:0/2200416916 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x555babcf29f0 0x555bac0f9040 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:25.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.990+0000 7f3c210f1740 1 --2- 172.21.15.64:0/2200416916 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555babd34070 0x555bac0f2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:25.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.990+0000 7f3c210f1740 1 --2- 172.21.15.64:0/2200416916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555babd13390 0x555bac0e7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:25.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.990+0000 7f3c210f1740 1 --2- 172.21.15.64:0/2200416916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555babd0b1b0 0x555bac0e6e30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:25.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.990+0000 7f3c210f1740 1 -- 172.21.15.64:0/2200416916 >> 172.21.15.64:0/2200416916 conn(0x555babd28460 msgr2=0x555bac0de830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:25.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.990+0000 7f3bf3fff640 0 client.25805 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:25.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.991+0000 7f3c210f1740 1 -- 172.21.15.64:0/2200416916 shutdown_connections 2024-09-21T04:06:25.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.991+0000 7f3c210f1740 1 -- 172.21.15.64:0/2200416916 wait complete. 2024-09-21T04:06:25.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:25.991+0000 7f3c210f1740 20 client.25805 trim_cache size 0 max 16384 2024-09-21T04:06:26.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.012+0000 7f3c210f1740 1 Processor -- start 2024-09-21T04:06:26.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.012+0000 7f3c210f1740 1 -- start start 2024-09-21T04:06:26.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.012+0000 7f3c210f1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555babd0b1b0 0x555bac0d9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:26.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.012+0000 7f3c210f1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555babd13390 0x555bac0f2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:26.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.012+0000 7f3c210f1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555babd34070 0x555bac0f3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:26.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.013+0000 7f3c210f1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3c141aff90 con 0x555babd13390 2024-09-21T04:06:26.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.013+0000 7f3c210f1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3c141ad7d0 con 0x555babd34070 2024-09-21T04:06:26.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.013+0000 7f3c210f1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3c141ab620 con 0x555babd0b1b0 2024-09-21T04:06:26.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.013+0000 7f3c1911a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555babd34070 0x555bac0f3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:26.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.013+0000 7f3c1911a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555babd34070 0x555bac0f3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:39774/0 (socket says 172.21.15.64:39774) 2024-09-21T04:06:26.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.013+0000 7f3c1911a640 1 -- 172.21.15.64:0/72120867 learned_addr learned my addr 172.21.15.64:0/72120867 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:26.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.013+0000 7f3c18919640 1 --2- 172.21.15.64:0/72120867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555babd0b1b0 0x555bac0d9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:26.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.013+0000 7f3c13fff640 1 --2- 172.21.15.64:0/72120867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555babd13390 0x555bac0f2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:26.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.013+0000 7f3c1911a640 1 -- 172.21.15.64:0/72120867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555babd0b1b0 msgr2=0x555bac0d9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:26.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.013+0000 7f3c1911a640 1 --2- 172.21.15.64:0/72120867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555babd0b1b0 0x555bac0d9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.013+0000 7f3c1911a640 1 -- 172.21.15.64:0/72120867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555babd13390 msgr2=0x555bac0f2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:26.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.013+0000 7f3c1911a640 1 --2- 172.21.15.64:0/72120867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555babd13390 0x555bac0f2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.013+0000 7f3c1911a640 1 -- 172.21.15.64:0/72120867 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bac0ddc30 con 0x555babd34070 2024-09-21T04:06:26.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.013+0000 7f3c18919640 1 --2- 172.21.15.64:0/72120867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555babd0b1b0 0x555bac0d9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:06:26.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.013+0000 7f3c13fff640 1 --2- 172.21.15.64:0/72120867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555babd13390 0x555bac0f2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:06:26.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.014+0000 7f3c1911a640 1 --2- 172.21.15.64:0/72120867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555babd34070 0x555bac0f3380 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c02aeb0 tx=0x7f3c0c003ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49dd1d13ffccf625 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:26.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.014+0000 7f3bfe7fc640 1 -- 172.21.15.64:0/72120867 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c056070 con 0x555babd34070 2024-09-21T04:06:26.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.014+0000 7f3bfe7fc640 1 -- 172.21.15.64:0/72120867 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c0c036040 con 0x555babd34070 2024-09-21T04:06:26.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.014+0000 7f3bfe7fc640 1 -- 172.21.15.64:0/72120867 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c04a040 con 0x555babd34070 2024-09-21T04:06:26.022 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.015+0000 7f3c210f1740 1 -- 172.21.15.64:0/72120867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555babd34070 msgr2=0x555bac0f3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:26.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.015+0000 7f3c210f1740 1 --2- 172.21.15.64:0/72120867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555babd34070 0x555bac0f3380 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c02aeb0 tx=0x7f3c0c003ce0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.015+0000 7f3c210f1740 1 -- 172.21.15.64:0/72120867 shutdown_connections 2024-09-21T04:06:26.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.015+0000 7f3c210f1740 1 --2- 172.21.15.64:0/72120867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555babd34070 0x555bac0f3380 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.015+0000 7f3c210f1740 1 --2- 172.21.15.64:0/72120867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555babd13390 0x555bac0f2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.015+0000 7f3c210f1740 1 --2- 172.21.15.64:0/72120867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555babd0b1b0 0x555bac0d9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.015+0000 7f3c210f1740 1 -- 172.21.15.64:0/72120867 >> 172.21.15.64:0/72120867 conn(0x555babd28460 msgr2=0x555bac0d8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:26.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.015+0000 7f3c210f1740 1 -- 172.21.15.64:0/72120867 shutdown_connections 2024-09-21T04:06:26.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.015+0000 7f3c210f1740 1 -- 172.21.15.64:0/72120867 wait complete. 2024-09-21T04:06:26.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.016+0000 7f3c210f1740 1 -- start start 2024-09-21T04:06:26.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.016+0000 7f3c210f1740 1 Processor -- start 2024-09-21T04:06:26.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.016+0000 7f3c210f1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555babd0b1b0 0x555bac0ef520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:26.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.016+0000 7f3c210f1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555babd13390 0x555bac0ea9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:26.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.016+0000 7f3c210f1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555babd34070 0x555bac0eaf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:26.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.017+0000 7f3c210f1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3c141abc90 con 0x555babd34070 2024-09-21T04:06:26.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.017+0000 7f3c210f1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3c141ad100 con 0x555babd0b1b0 2024-09-21T04:06:26.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.017+0000 7f3c210f1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3c141ac360 con 0x555babd13390 2024-09-21T04:06:26.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.017+0000 7f3c210f1740 10 client.0 _ll_register_callbacks cb 0x7f3c0801d8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:06:26.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.017+0000 7f3c18919640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555babd0b1b0 0x555bac0ef520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:26.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.017+0000 7f3c18919640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555babd0b1b0 0x555bac0ef520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:39782/0 (socket says 172.21.15.64:39782) 2024-09-21T04:06:26.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.017+0000 7f3c13fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555babd13390 0x555bac0ea9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:26.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.017+0000 7f3c18919640 1 -- 172.21.15.64:0/361837099 learned_addr learned my addr 172.21.15.64:0/361837099 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:26.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.017+0000 7f3c1911a640 1 --2- 172.21.15.64:0/361837099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555babd34070 0x555bac0eaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:26.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.017+0000 7f3c18919640 1 -- 172.21.15.64:0/361837099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555babd13390 msgr2=0x555bac0ea9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:26.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.017+0000 7f3c18919640 1 --2- 172.21.15.64:0/361837099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555babd13390 0x555bac0ea9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.017+0000 7f3c18919640 1 -- 172.21.15.64:0/361837099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555babd34070 msgr2=0x555bac0eaf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:26.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.017+0000 7f3c18919640 1 --2- 172.21.15.64:0/361837099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555babd34070 0x555bac0eaf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.017+0000 7f3c18919640 1 -- 172.21.15.64:0/361837099 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bac0eb550 con 0x555babd0b1b0 2024-09-21T04:06:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.017+0000 7f3c1911a640 1 --2- 172.21.15.64:0/361837099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555babd34070 0x555bac0eaf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:06:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.017+0000 7f3c18919640 1 --2- 172.21.15.64:0/361837099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555babd0b1b0 0x555bac0ef520 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3c08012010 tx=0x7f3c08030a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22d6bf8976176148 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.017+0000 7f3bf3fff640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:06:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.018+0000 7f3bf3fff640 1 -- 172.21.15.64:0/361837099 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0802b040 con 0x555babd0b1b0 2024-09-21T04:06:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.018+0000 7f3c210f1740 1 -- 172.21.15.64:0/361837099 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c0c036040 con 0x555babd0b1b0 2024-09-21T04:06:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.018+0000 7f3bf3fff640 1 -- 172.21.15.64:0/361837099 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f3c08010030 con 0x555babd0b1b0 2024-09-21T04:06:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.018+0000 7f3bf3fff640 1 -- 172.21.15.64:0/361837099 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3c08047050 con 0x555babd0b1b0 2024-09-21T04:06:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.018+0000 7f3c210f1740 10 client.25811 fetch_fsmap learned FSMap version 30 2024-09-21T04:06:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.018+0000 7f3c210f1740 1 -- 172.21.15.64:0/361837099 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555bac163850 con 0x555babd0b1b0 2024-09-21T04:06:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.018+0000 7f3bf3fff640 1 -- 172.21.15.64:0/361837099 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c0c036040 con 0x555babd0b1b0 2024-09-21T04:06:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.019+0000 7f3bf3fff640 1 -- 172.21.15.64:0/361837099 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f3c0800b800 con 0x555babd0b1b0 2024-09-21T04:06:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.019+0000 7f3c210f1740 10 client.25811 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:06:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.019+0000 7f3c210f1740 10 client.25811 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:06:26.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.019+0000 7f3c210f1740 20 client.25811 populate_metadata read hostname 'smithi064' 2024-09-21T04:06:26.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.019+0000 7f3c210f1740 1 --2- 172.21.15.64:0/361837099 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x555babcc7da0 0x555bac166090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:26.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.019+0000 7f3c210f1740 4 client.25811 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T04:06:26.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.019+0000 7f3c210f1740 1 -- 172.21.15.64:0/361837099 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555bac04f850 con 0x555babcc7da0 2024-09-21T04:06:26.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.019+0000 7f3c13fff640 1 --2- 172.21.15.64:0/361837099 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x555babcc7da0 0x555bac166090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:26.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.020+0000 7f3bf3fff640 10 client.25811 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:26.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.020+0000 7f3c13fff640 1 --2- 172.21.15.64:0/361837099 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x555babcc7da0 0x555bac166090 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8607e55cc8da33d server_cookie=bc3af3ad13bf06ec in_seq=0 out_seq=0 2024-09-21T04:06:26.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.020+0000 7f3bf3fff640 1 -- 172.21.15.64:0/361837099 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1706 (crc 0 0 0) 0x555bac04f850 con 0x555babcc7da0 2024-09-21T04:06:26.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.020+0000 7f3bf3fff640 10 client.25811 handle_command_reply: tid=0 2024-09-21T04:06:26.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.020+0000 7f3c210f1740 1 client.25811 shutdown 2024-09-21T04:06:26.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.020+0000 7f3c210f1740 1 -- 172.21.15.64:0/361837099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555babd0b1b0 msgr2=0x555bac0ef520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:26.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.020+0000 7f3c210f1740 1 --2- 172.21.15.64:0/361837099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555babd0b1b0 0x555bac0ef520 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3c08012010 tx=0x7f3c08030a90 comp rx=0 tx=0).stop 2024-09-21T04:06:26.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.020+0000 7f3c210f1740 1 -- 172.21.15.64:0/361837099 shutdown_connections 2024-09-21T04:06:26.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.020+0000 7f3c210f1740 1 --2- 172.21.15.64:0/361837099 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x555babcc7da0 0x555bac166090 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.020+0000 7f3bf3fff640 0 client.25811 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:26.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.020+0000 7f3c210f1740 1 --2- 172.21.15.64:0/361837099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555babd34070 0x555bac0eaf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.020+0000 7f3c210f1740 1 --2- 172.21.15.64:0/361837099 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555babd13390 0x555bac0ea9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.020+0000 7f3c210f1740 1 --2- 172.21.15.64:0/361837099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555babd0b1b0 0x555bac0ef520 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.020+0000 7f3c210f1740 1 -- 172.21.15.64:0/361837099 >> 172.21.15.64:0/361837099 conn(0x555babd28460 msgr2=0x555bac0de750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:26.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.021+0000 7f3c210f1740 1 -- 172.21.15.64:0/361837099 shutdown_connections 2024-09-21T04:06:26.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.021+0000 7f3c210f1740 1 -- 172.21.15.64:0/361837099 wait complete. 2024-09-21T04:06:26.028 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.021+0000 7f3c210f1740 20 client.25811 trim_cache size 0 max 16384 2024-09-21T04:06:26.029 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:06:26.029 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T04:06:26.029 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-09-21T04:06:26.029 INFO:teuthology.orchestra.run.smithi064.stdout: "6bde0fe5-f775-44b1-ad0c-f9e596574dc0": { 2024-09-21T04:06:26.029 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:06:26.029 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "6bde0fe5-f775-44b1-ad0c-f9e596574dc0", 2024-09-21T04:06:26.029 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-09-21T04:06:26.029 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:06:26.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:06:26.029 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_stats": { 2024-09-21T04:06:26.029 INFO:teuthology.orchestra.run.smithi064.stdout: "e9242f88-6599-40eb-adf1-bdb3e946a1e0": { 2024-09-21T04:06:26.029 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:05:39.744646+0000", 2024-09-21T04:06:26.029 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:06:26.030 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:06:26.030 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:06:26.030 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:06:26.030 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:06:26.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:06:26.030 INFO:teuthology.orchestra.run.smithi064.stdout: "6bde0fe5-f775-44b1-ad0c-f9e596574dc0": { 2024-09-21T04:06:26.030 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:06:23.835536+0000", 2024-09-21T04:06:26.030 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:06:26.030 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:06:26.030 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:06:26.030 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:06:26.030 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:06:26.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:06:26.030 INFO:teuthology.orchestra.run.smithi064.stdout: "b52b950c-0f97-49ec-a37c-1a8e04348ce8": { 2024-09-21T04:06:26.031 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:54.003800+0000", 2024-09-21T04:06:26.031 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:06:26.031 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:06:26.031 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:06:26.031 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:06:26.031 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:06:26.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:06:26.031 INFO:teuthology.orchestra.run.smithi064.stdout: "0ab717a7-36c6-4dca-94e7-af0bdf2aa112": { 2024-09-21T04:06:26.031 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:42.691086+0000", 2024-09-21T04:06:26.031 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:06:26.031 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:06:26.031 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:06:26.031 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:06:26.031 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:06:26.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:06:26.032 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:06:26.032 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:03:59.570174+0000", 2024-09-21T04:06:26.032 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:06:26.032 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:06:26.032 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:06:26.032 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:06:26.032 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:06:26.032 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:06:26.032 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:06:26.032 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:06:26.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.022+0000 7f3c1ab7d640 1 -- 172.21.15.64:0/3266639915 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3c040b6680 msgr2=0x7f3c040b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:26.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.022+0000 7f3c1ab7d640 1 --2- 172.21.15.64:0/3266639915 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3c040b6680 0x7f3c040b8b90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x555bac0da2d0 tx=0x7f3c08012040 comp rx=0 tx=0).stop 2024-09-21T04:06:26.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.022+0000 7f3c1ab7d640 1 -- 172.21.15.64:0/3266639915 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c141094a0 msgr2=0x7f3c1419a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:26.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.022+0000 7f3c1ab7d640 1 --2- 172.21.15.64:0/3266639915 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c141094a0 0x7f3c1419a9f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3c1406cc20 tx=0x7f3c0000ee50 comp rx=0 tx=0).stop 2024-09-21T04:06:26.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.022+0000 7f3c1ab7d640 1 -- 172.21.15.64:0/3266639915 shutdown_connections 2024-09-21T04:06:26.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.022+0000 7f3c1ab7d640 1 --2- 172.21.15.64:0/3266639915 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3c040b6680 0x7f3c040b8b90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.022+0000 7f3c1ab7d640 1 --2- 172.21.15.64:0/3266639915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c14109df0 0x7f3c1419fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.022+0000 7f3c1ab7d640 1 --2- 172.21.15.64:0/3266639915 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c141094a0 0x7f3c1419a9f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.022+0000 7f3c1ab7d640 1 --2- 172.21.15.64:0/3266639915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c14103b60 0x7f3c1419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.022+0000 7f3c1ab7d640 1 -- 172.21.15.64:0/3266639915 >> 172.21.15.64:0/3266639915 conn(0x7f3c140ff5d0 msgr2=0x7f3c14108af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:26.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.022+0000 7f3c1ab7d640 1 -- 172.21.15.64:0/3266639915 shutdown_connections 2024-09-21T04:06:26.034 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:26.022+0000 7f3c1ab7d640 1 -- 172.21.15.64:0/3266639915 wait complete. 2024-09-21T04:06:26.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:26 smithi119 ceph-mon[34811]: pgmap v393: 97 pgs: 97 active+clean; 210 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 1.8 MiB/s wr, 470 op/s 2024-09-21T04:06:26.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:26 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:06:26.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:26 smithi064 ceph-mon[31180]: pgmap v393: 97 pgs: 97 active+clean; 210 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 1.8 MiB/s wr, 470 op/s 2024-09-21T04:06:26.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:26 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:06:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.522+0000 7fd9506ec640 1 -- 172.21.15.64:0/4218411004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9481082e0 msgr2=0x7fd9481086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.522+0000 7fd9506ec640 1 --2- 172.21.15.64:0/4218411004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9481082e0 0x7fd9481086f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd938009900 tx=0x7fd93802a410 comp rx=0 tx=0).stop 2024-09-21T04:06:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.522+0000 7fd9506ec640 1 -- 172.21.15.64:0/4218411004 shutdown_connections 2024-09-21T04:06:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.522+0000 7fd9506ec640 1 --2- 172.21.15.64:0/4218411004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd948108c30 0x7fd94810f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.522+0000 7fd9506ec640 1 --2- 172.21.15.64:0/4218411004 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9481082e0 0x7fd9481086f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.522+0000 7fd9506ec640 1 --2- 172.21.15.64:0/4218411004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9481029b0 0x7fd948102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.522+0000 7fd9506ec640 1 -- 172.21.15.64:0/4218411004 >> 172.21.15.64:0/4218411004 conn(0x7fd94807a740 msgr2=0x7fd94807ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.522+0000 7fd9506ec640 1 -- 172.21.15.64:0/4218411004 shutdown_connections 2024-09-21T04:06:26.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.522+0000 7fd9506ec640 1 -- 172.21.15.64:0/4218411004 wait complete. 2024-09-21T04:06:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.523+0000 7fd9506ec640 1 Processor -- start 2024-09-21T04:06:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.523+0000 7fd9506ec640 1 -- start start 2024-09-21T04:06:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.524+0000 7fd9506ec640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9481029b0 0x7fd948199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.524+0000 7fd9506ec640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9481082e0 0x7fd9481998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.524+0000 7fd9506ec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd948108c30 0x7fd94819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.524+0000 7fd9506ec640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd9481128f0 con 0x7fd9481029b0 2024-09-21T04:06:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.524+0000 7fd9506ec640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd948112770 con 0x7fd948108c30 2024-09-21T04:06:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.524+0000 7fd9506ec640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd948112a70 con 0x7fd9481082e0 2024-09-21T04:06:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.524+0000 7fd94e488640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9481029b0 0x7fd948199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.524+0000 7fd94e488640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9481029b0 0x7fd948199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59550/0 (socket says 172.21.15.64:59550) 2024-09-21T04:06:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.524+0000 7fd94e488640 1 -- 172.21.15.64:0/1634728414 learned_addr learned my addr 172.21.15.64:0/1634728414 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.524+0000 7fd94ec89640 1 --2- 172.21.15.64:0/1634728414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd948108c30 0x7fd94819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.524+0000 7fd94dc87640 1 --2- 172.21.15.64:0/1634728414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9481082e0 0x7fd9481998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.524+0000 7fd94e488640 1 -- 172.21.15.64:0/1634728414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9481082e0 msgr2=0x7fd9481998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.524+0000 7fd94e488640 1 --2- 172.21.15.64:0/1634728414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9481082e0 0x7fd9481998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.524+0000 7fd94e488640 1 -- 172.21.15.64:0/1634728414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd948108c30 msgr2=0x7fd94819eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.524+0000 7fd94e488640 1 --2- 172.21.15.64:0/1634728414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd948108c30 0x7fd94819eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.524+0000 7fd94e488640 1 -- 172.21.15.64:0/1634728414 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94819f280 con 0x7fd9481029b0 2024-09-21T04:06:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.524+0000 7fd94e488640 1 --2- 172.21.15.64:0/1634728414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9481029b0 0x7fd948199350 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd94400e9f0 tx=0x7fd94400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.524+0000 7fd93f7fe640 1 -- 172.21.15.64:0/1634728414 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94400cd80 con 0x7fd9481029b0 2024-09-21T04:06:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.524+0000 7fd93f7fe640 1 -- 172.21.15.64:0/1634728414 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9440108e0 con 0x7fd9481029b0 2024-09-21T04:06:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.525+0000 7fd9506ec640 1 -- 172.21.15.64:0/1634728414 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94819f570 con 0x7fd9481029b0 2024-09-21T04:06:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.524+0000 7fd93f7fe640 1 -- 172.21.15.64:0/1634728414 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd944018980 con 0x7fd9481029b0 2024-09-21T04:06:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.525+0000 7fd9506ec640 1 -- 172.21.15.64:0/1634728414 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9481a5d00 con 0x7fd9481029b0 2024-09-21T04:06:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.528+0000 7fd93f7fe640 1 -- 172.21.15.64:0/1634728414 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fd944010430 con 0x7fd9481029b0 2024-09-21T04:06:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.528+0000 7fd95765c740 1 Processor -- start 2024-09-21T04:06:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.528+0000 7fd95765c740 1 -- start start 2024-09-21T04:06:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.529+0000 7fd95765c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562ecc661b0 0x5562ed00b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.529+0000 7fd95765c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562ecc3d2f0 0x5562ed00bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.529+0000 7fd95765c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5562ecc454d0 0x5562ed00c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.529+0000 7fd95765c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5562ecf806f0 con 0x5562ecc454d0 2024-09-21T04:06:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.529+0000 7fd95765c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5562ecf6fb20 con 0x5562ecc661b0 2024-09-21T04:06:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.529+0000 7fd95765c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5562ecf73930 con 0x5562ecc3d2f0 2024-09-21T04:06:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.529+0000 7fd94e488640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562ecc3d2f0 0x5562ed00bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.529+0000 7fd94dc87640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5562ecc454d0 0x5562ed00c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.529+0000 7fd94dc87640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5562ecc454d0 0x5562ed00c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59562/0 (socket says 172.21.15.64:59562) 2024-09-21T04:06:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.529+0000 7fd94dc87640 1 -- 172.21.15.64:0/770401560 learned_addr learned my addr 172.21.15.64:0/770401560 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.529+0000 7fd94ec89640 1 --2- 172.21.15.64:0/770401560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562ecc661b0 0x5562ed00b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.529+0000 7fd94e488640 1 -- 172.21.15.64:0/770401560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562ecc661b0 msgr2=0x5562ed00b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.529+0000 7fd94e488640 1 --2- 172.21.15.64:0/770401560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562ecc661b0 0x5562ed00b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.529+0000 7fd94e488640 1 -- 172.21.15.64:0/770401560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5562ecc454d0 msgr2=0x5562ed00c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.529+0000 7fd94e488640 1 --2- 172.21.15.64:0/770401560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5562ecc454d0 0x5562ed00c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.529+0000 7fd94e488640 1 -- 172.21.15.64:0/770401560 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562ecf35530 con 0x5562ecc3d2f0 2024-09-21T04:06:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.530+0000 7fd93f7fe640 1 --2- 172.21.15.64:0/1634728414 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd9340b6700 0x7fd9340b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.530+0000 7fd94e488640 1 --2- 172.21.15.64:0/770401560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562ecc3d2f0 0x5562ed00bef0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd9440e5110 tx=0x7fd9440ebe80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e2f3d9ca62fe978 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.530+0000 7fd94ec89640 1 --2- 172.21.15.64:0/1634728414 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd9340b6700 0x7fd9340b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.530+0000 7fd93cff9640 1 -- 172.21.15.64:0/770401560 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9440a9f10 con 0x5562ecc3d2f0 2024-09-21T04:06:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.530+0000 7fd93cff9640 1 -- 172.21.15.64:0/770401560 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9440ad3d0 con 0x5562ecc3d2f0 2024-09-21T04:06:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.531+0000 7fd93f7fe640 1 -- 172.21.15.64:0/1634728414 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fd9440e79f0 con 0x7fd9481029b0 2024-09-21T04:06:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.531+0000 7fd93cff9640 1 -- 172.21.15.64:0/770401560 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9440ad030 con 0x5562ecc3d2f0 2024-09-21T04:06:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.532+0000 7fd95765c740 1 -- 172.21.15.64:0/770401560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562ecc3d2f0 msgr2=0x5562ed00bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.532+0000 7fd94ec89640 1 --2- 172.21.15.64:0/1634728414 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd9340b6700 0x7fd9340b8c10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd94000af10 tx=0x7fd94000a4d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.532+0000 7fd95765c740 1 --2- 172.21.15.64:0/770401560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562ecc3d2f0 0x5562ed00bef0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd9440e5110 tx=0x7fd9440ebe80 comp rx=0 tx=0).stop 2024-09-21T04:06:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.532+0000 7fd95765c740 1 -- 172.21.15.64:0/770401560 shutdown_connections 2024-09-21T04:06:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.532+0000 7fd95765c740 1 --2- 172.21.15.64:0/770401560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5562ecc454d0 0x5562ed00c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.532+0000 7fd95765c740 1 --2- 172.21.15.64:0/770401560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562ecc3d2f0 0x5562ed00bef0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.532+0000 7fd95765c740 1 --2- 172.21.15.64:0/770401560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562ecc661b0 0x5562ed00b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.532+0000 7fd95765c740 1 -- 172.21.15.64:0/770401560 >> 172.21.15.64:0/770401560 conn(0x5562ecc5a300 msgr2=0x5562ed0098a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.532+0000 7fd95765c740 1 -- 172.21.15.64:0/770401560 shutdown_connections 2024-09-21T04:06:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.532+0000 7fd95765c740 1 -- 172.21.15.64:0/770401560 wait complete. 2024-09-21T04:06:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.533+0000 7fd95765c740 1 -- start start 2024-09-21T04:06:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.533+0000 7fd95765c740 1 Processor -- start 2024-09-21T04:06:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.533+0000 7fd95765c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562ecc3d2f0 0x5562ed0178c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.533+0000 7fd95765c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562ecc454d0 0x5562ed017ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.533+0000 7fd95765c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5562ecc661b0 0x5562ed021360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.534+0000 7fd95765c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5562ecea7ab0 con 0x5562ecc661b0 2024-09-21T04:06:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.534+0000 7fd94e488640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562ecc3d2f0 0x5562ed0178c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.534+0000 7fd94e488640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562ecc3d2f0 0x5562ed0178c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:44506/0 (socket says 172.21.15.64:44506) 2024-09-21T04:06:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.534+0000 7fd94e488640 1 -- 172.21.15.64:0/1447659472 learned_addr learned my addr 172.21.15.64:0/1447659472 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.534+0000 7fd94dc87640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562ecc454d0 0x5562ed017ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.534+0000 7fd95765c740 1 -- 172.21.15.64:0/1447659472 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5562ecf15730 con 0x5562ecc454d0 2024-09-21T04:06:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.534+0000 7fd95765c740 1 -- 172.21.15.64:0/1447659472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5562ed012130 con 0x5562ecc3d2f0 2024-09-21T04:06:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.534+0000 7fd94ec89640 1 --2- 172.21.15.64:0/1447659472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5562ecc661b0 0x5562ed021360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.534+0000 7fd95765c740 10 client.0 _ll_register_callbacks cb 0x5562ecf696b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:06:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.534+0000 7fd94e488640 1 -- 172.21.15.64:0/1447659472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562ecc454d0 msgr2=0x5562ed017ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.534+0000 7fd94e488640 1 --2- 172.21.15.64:0/1447659472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562ecc454d0 0x5562ed017ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.534+0000 7fd94e488640 1 -- 172.21.15.64:0/1447659472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5562ecc661b0 msgr2=0x5562ed021360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.534+0000 7fd94e488640 1 --2- 172.21.15.64:0/1447659472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5562ecc661b0 0x5562ed021360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.534+0000 7fd94e488640 1 -- 172.21.15.64:0/1447659472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562eccd1790 con 0x5562ecc3d2f0 2024-09-21T04:06:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.534+0000 7fd94e488640 1 --2- 172.21.15.64:0/1447659472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562ecc3d2f0 0x5562ed0178c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd94400c6a0 tx=0x7fd944000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7896b385f96f3aac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.534+0000 7fd931ffb640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-21T04:06:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.534+0000 7fd931ffb640 1 -- 172.21.15.64:0/1447659472 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9440ad210 con 0x5562ecc3d2f0 2024-09-21T04:06:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.535+0000 7fd931ffb640 1 -- 172.21.15.64:0/1447659472 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fd9440b6fa0 con 0x5562ecc3d2f0 2024-09-21T04:06:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.535+0000 7fd95765c740 1 -- 172.21.15.64:0/1447659472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9440ad3d0 con 0x5562ecc3d2f0 2024-09-21T04:06:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.535+0000 7fd931ffb640 1 -- 172.21.15.64:0/1447659472 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd9440ad690 con 0x5562ecc3d2f0 2024-09-21T04:06:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.535+0000 7fd95765c740 10 client.25714 fetch_fsmap learned FSMap version 30 2024-09-21T04:06:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.535+0000 7fd95765c740 1 -- 172.21.15.64:0/1447659472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5562ecd4a620 con 0x5562ecc3d2f0 2024-09-21T04:06:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.535+0000 7fd931ffb640 1 -- 172.21.15.64:0/1447659472 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9440ad3d0 con 0x5562ecc3d2f0 2024-09-21T04:06:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.535+0000 7fd931ffb640 1 -- 172.21.15.64:0/1447659472 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fd9440acb20 con 0x5562ecc3d2f0 2024-09-21T04:06:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.535+0000 7fd95765c740 10 client.25714 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:06:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.535+0000 7fd95765c740 10 client.25714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:06:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.535+0000 7fd95765c740 20 client.25714 populate_metadata read hostname 'smithi064' 2024-09-21T04:06:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.535+0000 7fd95765c740 1 --2- 172.21.15.64:0/1447659472 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5562ecc24d80 0x5562ed02d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.535+0000 7fd95765c740 4 client.25714 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:06:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.535+0000 7fd95765c740 1 -- 172.21.15.64:0/1447659472 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562ecf82640 con 0x5562ecc24d80 2024-09-21T04:06:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.535+0000 7fd94dc87640 1 --2- 172.21.15.64:0/1447659472 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5562ecc24d80 0x5562ed02d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.536+0000 7fd94dc87640 1 --2- 172.21.15.64:0/1447659472 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5562ecc24d80 0x5562ed02d350 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50689658c32c59ce server_cookie=393c828b824bdcd5 in_seq=0 out_seq=0 2024-09-21T04:06:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.536+0000 7fd931ffb640 10 client.25714 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.537+0000 7fd931ffb640 1 -- 172.21.15.64:0/1447659472 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5562ecf82640 con 0x5562ecc24d80 2024-09-21T04:06:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.537+0000 7fd931ffb640 10 client.25714 handle_command_reply: tid=0 2024-09-21T04:06:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.537+0000 7fd95765c740 1 client.25714 shutdown 2024-09-21T04:06:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.537+0000 7fd95765c740 1 -- 172.21.15.64:0/1447659472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562ecc3d2f0 msgr2=0x5562ed0178c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.537+0000 7fd95765c740 1 --2- 172.21.15.64:0/1447659472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562ecc3d2f0 0x5562ed0178c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd94400c6a0 tx=0x7fd944000bc0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.537+0000 7fd95765c740 1 -- 172.21.15.64:0/1447659472 shutdown_connections 2024-09-21T04:06:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.537+0000 7fd95765c740 1 --2- 172.21.15.64:0/1447659472 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5562ecc24d80 0x5562ed02d350 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.537+0000 7fd95765c740 1 --2- 172.21.15.64:0/1447659472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5562ecc661b0 0x5562ed021360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.537+0000 7fd95765c740 1 --2- 172.21.15.64:0/1447659472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562ecc454d0 0x5562ed017ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.537+0000 7fd931ffb640 0 client.25714 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.537+0000 7fd95765c740 1 --2- 172.21.15.64:0/1447659472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562ecc3d2f0 0x5562ed0178c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.537+0000 7fd95765c740 1 -- 172.21.15.64:0/1447659472 >> 172.21.15.64:0/1447659472 conn(0x5562ecc5a300 msgr2=0x5562ed007ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.537+0000 7fd95765c740 1 -- 172.21.15.64:0/1447659472 shutdown_connections 2024-09-21T04:06:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.537+0000 7fd95765c740 1 -- 172.21.15.64:0/1447659472 wait complete. 2024-09-21T04:06:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.537+0000 7fd95765c740 20 client.25714 trim_cache size 0 max 16384 2024-09-21T04:06:26.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:26 smithi131 ceph-mon[35193]: pgmap v393: 97 pgs: 97 active+clean; 210 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 1.8 MiB/s wr, 470 op/s 2024-09-21T04:06:26.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:26 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:06:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.556+0000 7fd95765c740 1 Processor -- start 2024-09-21T04:06:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.556+0000 7fd95765c740 1 -- start start 2024-09-21T04:06:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.557+0000 7fd95765c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5562ecc24d80 0x5562ed012ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.557+0000 7fd95765c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562ecc3d2f0 0x5562ed014880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.557+0000 7fd95765c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562ecc454d0 0x5562ed014dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.557+0000 7fd95765c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd9481b0000 con 0x5562ecc24d80 2024-09-21T04:06:26.562 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.557+0000 7fd95765c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd9481ad840 con 0x5562ecc3d2f0 2024-09-21T04:06:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.557+0000 7fd95765c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd9481ab690 con 0x5562ecc454d0 2024-09-21T04:06:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.557+0000 7fd94e488640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5562ecc24d80 0x5562ed012ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.557+0000 7fd94dc87640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562ecc3d2f0 0x5562ed014880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.557+0000 7fd94e488640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5562ecc24d80 0x5562ed012ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59574/0 (socket says 172.21.15.64:59574) 2024-09-21T04:06:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.557+0000 7fd94e488640 1 -- 172.21.15.64:0/2557174507 learned_addr learned my addr 172.21.15.64:0/2557174507 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.557+0000 7fd94dc87640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562ecc3d2f0 0x5562ed014880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:39816/0 (socket says 172.21.15.64:39816) 2024-09-21T04:06:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.557+0000 7fd94ec89640 1 --2- 172.21.15.64:0/2557174507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562ecc454d0 0x5562ed014dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.557+0000 7fd94dc87640 1 -- 172.21.15.64:0/2557174507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562ecc454d0 msgr2=0x5562ed014dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.557+0000 7fd94dc87640 1 --2- 172.21.15.64:0/2557174507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562ecc454d0 0x5562ed014dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.557+0000 7fd94dc87640 1 -- 172.21.15.64:0/2557174507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5562ecc24d80 msgr2=0x5562ed012ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.557+0000 7fd94dc87640 1 --2- 172.21.15.64:0/2557174507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5562ecc24d80 0x5562ed012ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.557+0000 7fd94dc87640 1 -- 172.21.15.64:0/2557174507 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562ed00a230 con 0x5562ecc3d2f0 2024-09-21T04:06:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.557+0000 7fd94ec89640 1 --2- 172.21.15.64:0/2557174507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562ecc454d0 0x5562ed014dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:06:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.557+0000 7fd94e488640 1 --2- 172.21.15.64:0/2557174507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5562ecc24d80 0x5562ed012ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:06:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.558+0000 7fd94dc87640 1 --2- 172.21.15.64:0/2557174507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562ecc3d2f0 0x5562ed014880 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd938009230 tx=0x7fd938031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd0b87ac0bc6116a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.558+0000 7fd933fff640 1 -- 172.21.15.64:0/2557174507 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93804a020 con 0x5562ecc3d2f0 2024-09-21T04:06:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.558+0000 7fd933fff640 1 -- 172.21.15.64:0/2557174507 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd938005040 con 0x5562ecc3d2f0 2024-09-21T04:06:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.558+0000 7fd933fff640 1 -- 172.21.15.64:0/2557174507 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd938035040 con 0x5562ecc3d2f0 2024-09-21T04:06:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.559+0000 7fd95765c740 1 -- 172.21.15.64:0/2557174507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562ecc3d2f0 msgr2=0x5562ed014880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.559+0000 7fd95765c740 1 --2- 172.21.15.64:0/2557174507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562ecc3d2f0 0x5562ed014880 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd938009230 tx=0x7fd938031400 comp rx=0 tx=0).stop 2024-09-21T04:06:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.559+0000 7fd95765c740 1 -- 172.21.15.64:0/2557174507 shutdown_connections 2024-09-21T04:06:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.559+0000 7fd95765c740 1 --2- 172.21.15.64:0/2557174507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562ecc454d0 0x5562ed014dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.559+0000 7fd95765c740 1 --2- 172.21.15.64:0/2557174507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562ecc3d2f0 0x5562ed014880 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.559+0000 7fd95765c740 1 --2- 172.21.15.64:0/2557174507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5562ecc24d80 0x5562ed012ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.559+0000 7fd95765c740 1 -- 172.21.15.64:0/2557174507 >> 172.21.15.64:0/2557174507 conn(0x5562ecc5a300 msgr2=0x5562ed0123d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.559+0000 7fd95765c740 1 -- 172.21.15.64:0/2557174507 shutdown_connections 2024-09-21T04:06:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.559+0000 7fd95765c740 1 -- 172.21.15.64:0/2557174507 wait complete. 2024-09-21T04:06:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.560+0000 7fd95765c740 1 -- start start 2024-09-21T04:06:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.560+0000 7fd95765c740 1 Processor -- start 2024-09-21T04:06:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.560+0000 7fd95765c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562ecc24d80 0x5562ed025250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.560+0000 7fd95765c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5562ecc3d2f0 0x5562ed025830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.560+0000 7fd95765c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562ecc454d0 0x5562ed08eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.560+0000 7fd95765c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd9481abd00 con 0x5562ecc3d2f0 2024-09-21T04:06:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.560+0000 7fd95765c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd9481ad170 con 0x5562ecc454d0 2024-09-21T04:06:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.560+0000 7fd95765c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd9481ac3d0 con 0x5562ecc24d80 2024-09-21T04:06:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.560+0000 7fd95765c740 10 client.0 _ll_register_callbacks cb 0x7fd938009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:06:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.561+0000 7fd94e488640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562ecc24d80 0x5562ed025250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.561+0000 7fd94ec89640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562ecc454d0 0x5562ed08eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.561+0000 7fd94e488640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562ecc24d80 0x5562ed025250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:44516/0 (socket says 172.21.15.64:44516) 2024-09-21T04:06:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.561+0000 7fd94e488640 1 -- 172.21.15.64:0/8255546 learned_addr learned my addr 172.21.15.64:0/8255546 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.561+0000 7fd94dc87640 1 --2- 172.21.15.64:0/8255546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5562ecc3d2f0 0x5562ed025830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.561+0000 7fd94e488640 1 -- 172.21.15.64:0/8255546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562ecc454d0 msgr2=0x5562ed08eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.561+0000 7fd94e488640 1 --2- 172.21.15.64:0/8255546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562ecc454d0 0x5562ed08eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.561+0000 7fd94e488640 1 -- 172.21.15.64:0/8255546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5562ecc3d2f0 msgr2=0x5562ed025830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.561+0000 7fd94e488640 1 --2- 172.21.15.64:0/8255546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5562ecc3d2f0 0x5562ed025830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.561+0000 7fd94e488640 1 -- 172.21.15.64:0/8255546 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562ed08f190 con 0x5562ecc24d80 2024-09-21T04:06:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.561+0000 7fd94ec89640 1 --2- 172.21.15.64:0/8255546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562ecc454d0 0x5562ed08eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:06:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.561+0000 7fd94e488640 1 --2- 172.21.15.64:0/8255546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562ecc24d80 0x5562ed025250 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd9440aab30 tx=0x7fd9440a9d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=108d9f6ec18635b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.561+0000 7fd931ffb640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-21T04:06:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.561+0000 7fd931ffb640 1 -- 172.21.15.64:0/8255546 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9440a9030 con 0x5562ecc24d80 2024-09-21T04:06:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.562+0000 7fd931ffb640 1 -- 172.21.15.64:0/8255546 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fd9440b6fb0 con 0x5562ecc24d80 2024-09-21T04:06:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.562+0000 7fd95765c740 1 -- 172.21.15.64:0/8255546 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd938005040 con 0x5562ecc24d80 2024-09-21T04:06:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.562+0000 7fd931ffb640 1 -- 172.21.15.64:0/8255546 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd9440a9760 con 0x5562ecc24d80 2024-09-21T04:06:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.562+0000 7fd95765c740 10 client.25720 fetch_fsmap learned FSMap version 30 2024-09-21T04:06:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.562+0000 7fd95765c740 1 -- 172.21.15.64:0/8255546 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5562ed08f190 con 0x5562ecc24d80 2024-09-21T04:06:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.562+0000 7fd931ffb640 1 -- 172.21.15.64:0/8255546 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd938005040 con 0x5562ecc24d80 2024-09-21T04:06:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.562+0000 7fd931ffb640 1 -- 172.21.15.64:0/8255546 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fd9440b0c00 con 0x5562ecc24d80 2024-09-21T04:06:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.562+0000 7fd95765c740 10 client.25720 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:06:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.562+0000 7fd95765c740 10 client.25720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:06:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.562+0000 7fd95765c740 20 client.25720 populate_metadata read hostname 'smithi064' 2024-09-21T04:06:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.562+0000 7fd95765c740 1 --2- 172.21.15.64:0/8255546 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5562ecc661b0 0x5562ed099ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.562+0000 7fd95765c740 4 client.25720 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:06:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.562+0000 7fd95765c740 1 -- 172.21.15.64:0/8255546 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5562ecf7e0f0 con 0x5562ecc661b0 2024-09-21T04:06:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.563+0000 7fd94dc87640 1 --2- 172.21.15.64:0/8255546 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5562ecc661b0 0x5562ed099ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.563+0000 7fd94dc87640 1 --2- 172.21.15.64:0/8255546 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5562ecc661b0 0x5562ed099ad0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27c5e92a4586903c server_cookie=19b012ce86857731 in_seq=0 out_seq=0 2024-09-21T04:06:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.563+0000 7fd931ffb640 10 client.25720 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:26.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.622+0000 7fd931ffb640 1 -- 172.21.15.64:0/8255546 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5562ecf7e0f0 con 0x5562ecc661b0 2024-09-21T04:06:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.622+0000 7fd931ffb640 10 client.25720 handle_command_reply: tid=0 2024-09-21T04:06:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.622+0000 7fd95765c740 1 client.25720 shutdown 2024-09-21T04:06:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.622+0000 7fd95765c740 1 -- 172.21.15.64:0/8255546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562ecc24d80 msgr2=0x5562ed025250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.622+0000 7fd95765c740 1 --2- 172.21.15.64:0/8255546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562ecc24d80 0x5562ed025250 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd9440aab30 tx=0x7fd9440a9d40 comp rx=0 tx=0).stop 2024-09-21T04:06:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.622+0000 7fd95765c740 1 -- 172.21.15.64:0/8255546 shutdown_connections 2024-09-21T04:06:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.622+0000 7fd95765c740 1 --2- 172.21.15.64:0/8255546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5562ecc454d0 0x5562ed08eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.622+0000 7fd95765c740 1 --2- 172.21.15.64:0/8255546 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5562ecc661b0 0x5562ed099ad0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.622+0000 7fd95765c740 1 --2- 172.21.15.64:0/8255546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5562ecc3d2f0 0x5562ed025830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.622+0000 7fd95765c740 1 --2- 172.21.15.64:0/8255546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5562ecc24d80 0x5562ed025250 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.622+0000 7fd931ffb640 0 client.25720 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.622+0000 7fd95765c740 1 -- 172.21.15.64:0/8255546 >> 172.21.15.64:0/8255546 conn(0x5562ecc5a300 msgr2=0x5562ed012b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.622+0000 7fd95765c740 1 -- 172.21.15.64:0/8255546 shutdown_connections 2024-09-21T04:06:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.622+0000 7fd95765c740 1 -- 172.21.15.64:0/8255546 wait complete. 2024-09-21T04:06:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.622+0000 7fd95765c740 20 client.25720 trim_cache size 0 max 16384 2024-09-21T04:06:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:06:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:06:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:06:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:06:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.623+0000 7fd9506ec640 1 -- 172.21.15.64:0/1634728414 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd9340b6700 msgr2=0x7fd9340b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.623+0000 7fd9506ec640 1 --2- 172.21.15.64:0/1634728414 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd9340b6700 0x7fd9340b8c10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd94000af10 tx=0x7fd94000a4d0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.623+0000 7fd9506ec640 1 -- 172.21.15.64:0/1634728414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9481029b0 msgr2=0x7fd948199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.623+0000 7fd9506ec640 1 --2- 172.21.15.64:0/1634728414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9481029b0 0x7fd948199350 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd94400e9f0 tx=0x7fd94400eeb0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.623+0000 7fd9506ec640 1 -- 172.21.15.64:0/1634728414 shutdown_connections 2024-09-21T04:06:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.623+0000 7fd9506ec640 1 --2- 172.21.15.64:0/1634728414 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd9340b6700 0x7fd9340b8c10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.623+0000 7fd9506ec640 1 --2- 172.21.15.64:0/1634728414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd948108c30 0x7fd94819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.623+0000 7fd9506ec640 1 --2- 172.21.15.64:0/1634728414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd9481082e0 0x7fd9481998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.623+0000 7fd9506ec640 1 --2- 172.21.15.64:0/1634728414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd9481029b0 0x7fd948199350 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.623+0000 7fd9506ec640 1 -- 172.21.15.64:0/1634728414 >> 172.21.15.64:0/1634728414 conn(0x7fd94807a740 msgr2=0x7fd9481043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.623+0000 7fd9506ec640 1 -- 172.21.15.64:0/1634728414 shutdown_connections 2024-09-21T04:06:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:26.623+0000 7fd9506ec640 1 -- 172.21.15.64:0/1634728414 wait complete. 2024-09-21T04:06:26.910 INFO:tasks.cephfs.filesystem:scrub status for tag:6bde0fe5-f775-44b1-ad0c-f9e596574dc0 - {'path': '/', 'tag': '6bde0fe5-f775-44b1-ad0c-f9e596574dc0', 'options': 'recursive,force'} 2024-09-21T04:06:28.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:28 smithi119 ceph-mon[34811]: pgmap v394: 97 pgs: 97 active+clean; 214 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.2 MiB/s wr, 595 op/s 2024-09-21T04:06:28.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:28 smithi064 ceph-mon[31180]: pgmap v394: 97 pgs: 97 active+clean; 214 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.2 MiB/s wr, 595 op/s 2024-09-21T04:06:28.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:28 smithi131 ceph-mon[35193]: pgmap v394: 97 pgs: 97 active+clean; 214 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.2 MiB/s wr, 595 op/s 2024-09-21T04:06:30.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:30 smithi119 ceph-mon[34811]: pgmap v395: 97 pgs: 97 active+clean; 221 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.1 MiB/s wr, 542 op/s 2024-09-21T04:06:30.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:30 smithi064 ceph-mon[31180]: pgmap v395: 97 pgs: 97 active+clean; 221 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.1 MiB/s wr, 542 op/s 2024-09-21T04:06:30.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:30 smithi131 ceph-mon[35193]: pgmap v395: 97 pgs: 97 active+clean; 221 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.1 MiB/s wr, 542 op/s 2024-09-21T04:06:31.172 INFO:tasks.workunit.client.0.smithi064.stdout: 53 499198 185 330389 249 222433 417 2024-09-21T04:06:32.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:32 smithi064 ceph-mon[31180]: pgmap v396: 97 pgs: 97 active+clean; 221 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.1 MiB/s wr, 505 op/s 2024-09-21T04:06:32.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:32 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:06:32.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:32 smithi131 ceph-mon[35193]: pgmap v396: 97 pgs: 97 active+clean; 221 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.1 MiB/s wr, 505 op/s 2024-09-21T04:06:32.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:32 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:06:32.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:32 smithi119 ceph-mon[34811]: pgmap v396: 97 pgs: 97 active+clean; 221 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.1 MiB/s wr, 505 op/s 2024-09-21T04:06:32.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:32 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:06:34.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:34 smithi064 ceph-mon[31180]: pgmap v397: 97 pgs: 97 active+clean; 222 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.2 MiB/s wr, 605 op/s 2024-09-21T04:06:34.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:34 smithi131 ceph-mon[35193]: pgmap v397: 97 pgs: 97 active+clean; 222 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.2 MiB/s wr, 605 op/s 2024-09-21T04:06:34.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:34 smithi119 ceph-mon[34811]: pgmap v397: 97 pgs: 97 active+clean; 222 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.2 MiB/s wr, 605 op/s 2024-09-21T04:06:36.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:36 smithi131 ceph-mon[35193]: pgmap v398: 97 pgs: 97 active+clean; 222 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 350 op/s 2024-09-21T04:06:36.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:36 smithi119 ceph-mon[34811]: pgmap v398: 97 pgs: 97 active+clean; 222 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 350 op/s 2024-09-21T04:06:36.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:36 smithi064 ceph-mon[31180]: pgmap v398: 97 pgs: 97 active+clean; 222 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 350 op/s 2024-09-21T04:06:38.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:38 smithi131 ceph-mon[35193]: pgmap v399: 97 pgs: 97 active+clean; 266 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 680 op/s 2024-09-21T04:06:38.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:38 smithi119 ceph-mon[34811]: pgmap v399: 97 pgs: 97 active+clean; 266 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 680 op/s 2024-09-21T04:06:38.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:38 smithi064 ceph-mon[31180]: pgmap v399: 97 pgs: 97 active+clean; 266 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.4 MiB/s wr, 680 op/s 2024-09-21T04:06:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.738+0000 7f0289415640 1 -- 172.21.15.64:0/716381708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f028410dda0 msgr2=0x7f0284110280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.738+0000 7f0289415640 1 --2- 172.21.15.64:0/716381708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f028410dda0 0x7f0284110280 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f027c009900 tx=0x7f027c02a410 comp rx=0 tx=0).stop 2024-09-21T04:06:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.738+0000 7f0289415640 1 -- 172.21.15.64:0/716381708 shutdown_connections 2024-09-21T04:06:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.738+0000 7f0289415640 1 --2- 172.21.15.64:0/716381708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f028410dda0 0x7f0284110280 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.738+0000 7f0289415640 1 --2- 172.21.15.64:0/716381708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02841052d0 0x7f028410d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.738+0000 7f0289415640 1 --2- 172.21.15.64:0/716381708 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0284104980 0x7f0284104d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.738+0000 7f0289415640 1 -- 172.21.15.64:0/716381708 >> 172.21.15.64:0/716381708 conn(0x7f02840fe3f0 msgr2=0x7f0284100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.738+0000 7f0289415640 1 -- 172.21.15.64:0/716381708 shutdown_connections 2024-09-21T04:06:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.738+0000 7f0289415640 1 -- 172.21.15.64:0/716381708 wait complete. 2024-09-21T04:06:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.739+0000 7f0289415640 1 Processor -- start 2024-09-21T04:06:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.739+0000 7f0289415640 1 -- start start 2024-09-21T04:06:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.740+0000 7f0289415640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0284104980 0x7f028419c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.740+0000 7f0289415640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02841052d0 0x7f028419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.740+0000 7f0289415640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f028410dda0 0x7f02841a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.740+0000 7f0289415640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0284113a40 con 0x7f0284104980 2024-09-21T04:06:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.740+0000 7f0289415640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f02841138c0 con 0x7f028410dda0 2024-09-21T04:06:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.740+0000 7f0289415640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0284113bc0 con 0x7f02841052d0 2024-09-21T04:06:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.740+0000 7f0282ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0284104980 0x7f028419c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.740+0000 7f0282ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0284104980 0x7f028419c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43226/0 (socket says 172.21.15.64:43226) 2024-09-21T04:06:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.740+0000 7f0282ffd640 1 -- 172.21.15.64:0/2952369580 learned_addr learned my addr 172.21.15.64:0/2952369580 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.740+0000 7f02837fe640 1 --2- 172.21.15.64:0/2952369580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f028410dda0 0x7f02841a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.740+0000 7f02827fc640 1 --2- 172.21.15.64:0/2952369580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02841052d0 0x7f028419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.740+0000 7f0282ffd640 1 -- 172.21.15.64:0/2952369580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02841052d0 msgr2=0x7f028419cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.740+0000 7f0282ffd640 1 --2- 172.21.15.64:0/2952369580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02841052d0 0x7f028419cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.740+0000 7f0282ffd640 1 -- 172.21.15.64:0/2952369580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f028410dda0 msgr2=0x7f02841a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.740+0000 7f0282ffd640 1 --2- 172.21.15.64:0/2952369580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f028410dda0 0x7f02841a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.740+0000 7f0282ffd640 1 -- 172.21.15.64:0/2952369580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02841a2530 con 0x7f0284104980 2024-09-21T04:06:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.740+0000 7f02837fe640 1 --2- 172.21.15.64:0/2952369580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f028410dda0 0x7f02841a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:06:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.740+0000 7f0282ffd640 1 --2- 172.21.15.64:0/2952369580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0284104980 0x7f028419c600 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f027400cce0 tx=0x7f027400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.741+0000 7f027bfff640 1 -- 172.21.15.64:0/2952369580 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274004b20 con 0x7f0284104980 2024-09-21T04:06:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.741+0000 7f027bfff640 1 -- 172.21.15.64:0/2952369580 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0274004cc0 con 0x7f0284104980 2024-09-21T04:06:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.741+0000 7f0289415640 1 -- 172.21.15.64:0/2952369580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02841a2820 con 0x7f0284104980 2024-09-21T04:06:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.741+0000 7f027bfff640 1 -- 172.21.15.64:0/2952369580 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027400f6b0 con 0x7f0284104980 2024-09-21T04:06:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.741+0000 7f0289415640 1 -- 172.21.15.64:0/2952369580 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02841a8fb0 con 0x7f0284104980 2024-09-21T04:06:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.743+0000 7f029045f740 1 Processor -- start 2024-09-21T04:06:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.743+0000 7f029045f740 1 -- start start 2024-09-21T04:06:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.743+0000 7f027bfff640 1 -- 172.21.15.64:0/2952369580 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f027400f850 con 0x7f0284104980 2024-09-21T04:06:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.744+0000 7f029045f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f681741b0 0x556f685199b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.744+0000 7f029045f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f6814b2f0 0x556f68519ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.744+0000 7f029045f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f681534d0 0x556f6851a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.744+0000 7f029045f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556f6848e6f0 con 0x556f681534d0 2024-09-21T04:06:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.744+0000 7f029045f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556f6847db20 con 0x556f6814b2f0 2024-09-21T04:06:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.744+0000 7f029045f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556f68481930 con 0x556f681741b0 2024-09-21T04:06:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.744+0000 7f02827fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f681534d0 0x556f6851a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.744+0000 7f0282ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f6814b2f0 0x556f68519ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.744+0000 7f02827fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f681534d0 0x556f6851a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43234/0 (socket says 172.21.15.64:43234) 2024-09-21T04:06:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.744+0000 7f02827fc640 1 -- 172.21.15.64:0/1704249827 learned_addr learned my addr 172.21.15.64:0/1704249827 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.744+0000 7f02837fe640 1 --2- 172.21.15.64:0/1704249827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f681741b0 0x556f685199b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.744+0000 7f0282ffd640 1 -- 172.21.15.64:0/1704249827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f681741b0 msgr2=0x556f685199b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.744+0000 7f0282ffd640 1 --2- 172.21.15.64:0/1704249827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f681741b0 0x556f685199b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.744+0000 7f0282ffd640 1 -- 172.21.15.64:0/1704249827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f681534d0 msgr2=0x556f6851a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.744+0000 7f0282ffd640 1 --2- 172.21.15.64:0/1704249827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f681534d0 0x556f6851a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.745+0000 7f0282ffd640 1 -- 172.21.15.64:0/1704249827 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f68443530 con 0x556f6814b2f0 2024-09-21T04:06:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.745+0000 7f02827fc640 1 --2- 172.21.15.64:0/1704249827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f681534d0 0x556f6851a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:06:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.745+0000 7f02837fe640 1 --2- 172.21.15.64:0/1704249827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f681741b0 0x556f685199b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:06:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.745+0000 7f0282ffd640 1 --2- 172.21.15.64:0/1704249827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f6814b2f0 0x556f68519ef0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f02740e4e40 tx=0x7f02740ef690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86903f9bb6414b05 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.745+0000 7f027bfff640 1 --2- 172.21.15.64:0/2952369580 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f02700b6680 0x7f02700b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.745+0000 7f02797fa640 1 -- 172.21.15.64:0/1704249827 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02740f6070 con 0x556f6814b2f0 2024-09-21T04:06:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.745+0000 7f02797fa640 1 -- 172.21.15.64:0/1704249827 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02740d3b50 con 0x556f6814b2f0 2024-09-21T04:06:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.745+0000 7f02797fa640 1 -- 172.21.15.64:0/1704249827 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02740acf70 con 0x556f6814b2f0 2024-09-21T04:06:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.745+0000 7f02837fe640 1 --2- 172.21.15.64:0/2952369580 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f02700b6680 0x7f02700b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.745+0000 7f027bfff640 1 -- 172.21.15.64:0/2952369580 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f02740e7750 con 0x7f0284104980 2024-09-21T04:06:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.746+0000 7f029045f740 1 -- 172.21.15.64:0/1704249827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f6814b2f0 msgr2=0x556f68519ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.746+0000 7f029045f740 1 --2- 172.21.15.64:0/1704249827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f6814b2f0 0x556f68519ef0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f02740e4e40 tx=0x7f02740ef690 comp rx=0 tx=0).stop 2024-09-21T04:06:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.746+0000 7f02837fe640 1 --2- 172.21.15.64:0/2952369580 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f02700b6680 0x7f02700b8b90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f027c006010 tx=0x7f027c005a40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.746+0000 7f029045f740 1 -- 172.21.15.64:0/1704249827 shutdown_connections 2024-09-21T04:06:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.746+0000 7f029045f740 1 --2- 172.21.15.64:0/1704249827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f681534d0 0x556f6851a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.746+0000 7f029045f740 1 --2- 172.21.15.64:0/1704249827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f6814b2f0 0x556f68519ef0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.746+0000 7f029045f740 1 --2- 172.21.15.64:0/1704249827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f681741b0 0x556f685199b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.746+0000 7f029045f740 1 -- 172.21.15.64:0/1704249827 >> 172.21.15.64:0/1704249827 conn(0x556f68168300 msgr2=0x556f685178a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.746+0000 7f029045f740 1 -- 172.21.15.64:0/1704249827 shutdown_connections 2024-09-21T04:06:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.746+0000 7f029045f740 1 -- 172.21.15.64:0/1704249827 wait complete. 2024-09-21T04:06:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.747+0000 7f029045f740 1 -- start start 2024-09-21T04:06:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.747+0000 7f029045f740 1 Processor -- start 2024-09-21T04:06:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.747+0000 7f029045f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f6814b2f0 0x556f685250f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.747+0000 7f029045f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f681534d0 0x556f68525630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.747+0000 7f029045f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f681741b0 0x556f68531420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.747+0000 7f029045f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556f683b5ab0 con 0x556f681534d0 2024-09-21T04:06:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.747+0000 7f029045f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556f68423730 con 0x556f6814b2f0 2024-09-21T04:06:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.747+0000 7f029045f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556f68528130 con 0x556f681741b0 2024-09-21T04:06:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.747+0000 7f029045f740 10 client.0 _ll_register_callbacks cb 0x556f684776b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:06:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.747+0000 7f02827fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f681534d0 0x556f68525630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.747+0000 7f0282ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f6814b2f0 0x556f685250f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.747+0000 7f02827fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f681534d0 0x556f68525630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43250/0 (socket says 172.21.15.64:43250) 2024-09-21T04:06:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.747+0000 7f02827fc640 1 -- 172.21.15.64:0/1853890318 learned_addr learned my addr 172.21.15.64:0/1853890318 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.748+0000 7f02837fe640 1 --2- 172.21.15.64:0/1853890318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f681741b0 0x556f68531420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.748+0000 7f0282ffd640 1 -- 172.21.15.64:0/1853890318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f681741b0 msgr2=0x556f68531420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.748+0000 7f0282ffd640 1 --2- 172.21.15.64:0/1853890318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f681741b0 0x556f68531420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.748+0000 7f0282ffd640 1 -- 172.21.15.64:0/1853890318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f681534d0 msgr2=0x556f68525630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.748+0000 7f0282ffd640 1 --2- 172.21.15.64:0/1853890318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f681534d0 0x556f68525630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:39.755 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.748+0000 7f0282ffd640 1 -- 172.21.15.64:0/1853890318 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f681df790 con 0x556f6814b2f0 2024-09-21T04:06:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.748+0000 7f0282ffd640 1 --2- 172.21.15.64:0/1853890318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f6814b2f0 0x556f685250f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x556f68518630 tx=0x7f0274004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b60fdce03e9f8256 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.748+0000 7f026a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:06:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.748+0000 7f026a7fc640 1 -- 172.21.15.64:0/1853890318 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027407e5c0 con 0x556f6814b2f0 2024-09-21T04:06:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.748+0000 7f026a7fc640 1 -- 172.21.15.64:0/1853890318 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f02740870b0 con 0x556f6814b2f0 2024-09-21T04:06:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.748+0000 7f029045f740 1 -- 172.21.15.64:0/1853890318 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02740d3b50 con 0x556f6814b2f0 2024-09-21T04:06:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.748+0000 7f026a7fc640 1 -- 172.21.15.64:0/1853890318 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f027407e7c0 con 0x556f6814b2f0 2024-09-21T04:06:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.748+0000 7f029045f740 10 client.25841 fetch_fsmap learned FSMap version 30 2024-09-21T04:06:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.748+0000 7f029045f740 1 -- 172.21.15.64:0/1853890318 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556f68258620 con 0x556f6814b2f0 2024-09-21T04:06:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.748+0000 7f026a7fc640 1 -- 172.21.15.64:0/1853890318 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02740d3b50 con 0x556f6814b2f0 2024-09-21T04:06:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.749+0000 7f026a7fc640 1 -- 172.21.15.64:0/1853890318 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f02740b6b00 con 0x556f6814b2f0 2024-09-21T04:06:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.749+0000 7f029045f740 10 client.25841 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:06:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.749+0000 7f029045f740 10 client.25841 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:06:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.749+0000 7f029045f740 20 client.25841 populate_metadata read hostname 'smithi064' 2024-09-21T04:06:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.749+0000 7f029045f740 1 --2- 172.21.15.64:0/1853890318 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556f68132d80 0x556f68536fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.749+0000 7f029045f740 4 client.25841 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:06:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.749+0000 7f029045f740 1 -- 172.21.15.64:0/1853890318 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f68490640 con 0x556f68132d80 2024-09-21T04:06:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.749+0000 7f02827fc640 1 --2- 172.21.15.64:0/1853890318 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556f68132d80 0x556f68536fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.749+0000 7f026a7fc640 10 client.25841 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.749+0000 7f02827fc640 1 --2- 172.21.15.64:0/1853890318 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556f68132d80 0x556f68536fe0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c4c25f4e57018f8 server_cookie=903f0f1076f22bc6 in_seq=0 out_seq=0 2024-09-21T04:06:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.750+0000 7f026a7fc640 1 -- 172.21.15.64:0/1853890318 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556f68490640 con 0x556f68132d80 2024-09-21T04:06:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.750+0000 7f026a7fc640 10 client.25841 handle_command_reply: tid=0 2024-09-21T04:06:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.750+0000 7f029045f740 1 client.25841 shutdown 2024-09-21T04:06:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.750+0000 7f029045f740 1 -- 172.21.15.64:0/1853890318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f6814b2f0 msgr2=0x556f685250f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.750+0000 7f029045f740 1 --2- 172.21.15.64:0/1853890318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f6814b2f0 0x556f685250f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x556f68518630 tx=0x7f0274004a40 comp rx=0 tx=0).stop 2024-09-21T04:06:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.750+0000 7f029045f740 1 -- 172.21.15.64:0/1853890318 shutdown_connections 2024-09-21T04:06:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.750+0000 7f029045f740 1 --2- 172.21.15.64:0/1853890318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f681741b0 0x556f68531420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.750+0000 7f029045f740 1 --2- 172.21.15.64:0/1853890318 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556f68132d80 0x556f68536fe0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.750+0000 7f029045f740 1 --2- 172.21.15.64:0/1853890318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f681534d0 0x556f68525630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.750+0000 7f026a7fc640 0 client.25841 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.750+0000 7f029045f740 1 --2- 172.21.15.64:0/1853890318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f6814b2f0 0x556f685250f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.750+0000 7f029045f740 1 -- 172.21.15.64:0/1853890318 >> 172.21.15.64:0/1853890318 conn(0x556f68168300 msgr2=0x556f6851a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.750+0000 7f029045f740 1 -- 172.21.15.64:0/1853890318 shutdown_connections 2024-09-21T04:06:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.750+0000 7f029045f740 1 -- 172.21.15.64:0/1853890318 wait complete. 2024-09-21T04:06:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.751+0000 7f029045f740 20 client.25841 trim_cache size 0 max 16384 2024-09-21T04:06:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.765+0000 7f029045f740 1 Processor -- start 2024-09-21T04:06:39.768 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.765+0000 7f029045f740 1 -- start start 2024-09-21T04:06:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.766+0000 7f029045f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f6814b2f0 0x556f685207f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.766+0000 7f029045f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f681534d0 0x556f68526d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.766+0000 7f029045f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f681741b0 0x556f68531390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:39.769 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.766+0000 7f029045f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f02841b4380 con 0x556f6814b2f0 2024-09-21T04:06:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.766+0000 7f029045f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f02841b1bc0 con 0x556f681534d0 2024-09-21T04:06:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.766+0000 7f029045f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f02841afa10 con 0x556f681741b0 2024-09-21T04:06:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.766+0000 7f0282ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f6814b2f0 0x556f685207f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.766+0000 7f0282ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f6814b2f0 0x556f685207f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43254/0 (socket says 172.21.15.64:43254) 2024-09-21T04:06:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.766+0000 7f0282ffd640 1 -- 172.21.15.64:0/1053532685 learned_addr learned my addr 172.21.15.64:0/1053532685 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.766+0000 7f02827fc640 1 --2- 172.21.15.64:0/1053532685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f681534d0 0x556f68526d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.766+0000 7f02837fe640 1 --2- 172.21.15.64:0/1053532685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f681741b0 0x556f68531390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.766+0000 7f0282ffd640 1 -- 172.21.15.64:0/1053532685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f681741b0 msgr2=0x556f68531390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.766+0000 7f0282ffd640 1 --2- 172.21.15.64:0/1053532685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f681741b0 0x556f68531390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.766+0000 7f0282ffd640 1 -- 172.21.15.64:0/1053532685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f681534d0 msgr2=0x556f68526d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.766+0000 7f0282ffd640 1 --2- 172.21.15.64:0/1053532685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f681534d0 0x556f68526d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.766+0000 7f0282ffd640 1 -- 172.21.15.64:0/1053532685 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f6851bcb0 con 0x556f6814b2f0 2024-09-21T04:06:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.766+0000 7f02827fc640 1 --2- 172.21.15.64:0/1053532685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f681534d0 0x556f68526d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:06:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.766+0000 7f0282ffd640 1 --2- 172.21.15.64:0/1053532685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f6814b2f0 0x556f685207f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f02740a9bf0 tx=0x7f0274004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b75fbf75030cde server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.767+0000 7f0278ff9640 1 -- 172.21.15.64:0/1053532685 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027401d070 con 0x556f6814b2f0 2024-09-21T04:06:39.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.767+0000 7f0278ff9640 1 -- 172.21.15.64:0/1053532685 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02740ad130 con 0x556f6814b2f0 2024-09-21T04:06:39.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.767+0000 7f0278ff9640 1 -- 172.21.15.64:0/1053532685 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02740f6030 con 0x556f6814b2f0 2024-09-21T04:06:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.767+0000 7f029045f740 1 -- 172.21.15.64:0/1053532685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f6814b2f0 msgr2=0x556f685207f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.767+0000 7f029045f740 1 --2- 172.21.15.64:0/1053532685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f6814b2f0 0x556f685207f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f02740a9bf0 tx=0x7f0274004ab0 comp rx=0 tx=0).stop 2024-09-21T04:06:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.768+0000 7f029045f740 1 -- 172.21.15.64:0/1053532685 shutdown_connections 2024-09-21T04:06:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.768+0000 7f029045f740 1 --2- 172.21.15.64:0/1053532685 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f681741b0 0x556f68531390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.768+0000 7f029045f740 1 --2- 172.21.15.64:0/1053532685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f681534d0 0x556f68526d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.768+0000 7f029045f740 1 --2- 172.21.15.64:0/1053532685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f6814b2f0 0x556f685207f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.768+0000 7f029045f740 1 -- 172.21.15.64:0/1053532685 >> 172.21.15.64:0/1053532685 conn(0x556f68168300 msgr2=0x556f68526610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.768+0000 7f029045f740 1 -- 172.21.15.64:0/1053532685 shutdown_connections 2024-09-21T04:06:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.768+0000 7f029045f740 1 -- 172.21.15.64:0/1053532685 wait complete. 2024-09-21T04:06:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.768+0000 7f029045f740 1 -- start start 2024-09-21T04:06:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.768+0000 7f029045f740 1 Processor -- start 2024-09-21T04:06:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.768+0000 7f029045f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f6814b2f0 0x556f68537a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.768+0000 7f029045f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f681534d0 0x556f68538030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.768+0000 7f029045f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f681741b0 0x556f68598710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.768+0000 7f029045f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f02841b0080 con 0x556f6814b2f0 2024-09-21T04:06:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.768+0000 7f029045f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f02841b14f0 con 0x556f681534d0 2024-09-21T04:06:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.768+0000 7f029045f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f02841b0750 con 0x556f681741b0 2024-09-21T04:06:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.768+0000 7f029045f740 10 client.0 _ll_register_callbacks cb 0x7f026c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:06:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.769+0000 7f0282ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f6814b2f0 0x556f68537a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.769+0000 7f0282ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f6814b2f0 0x556f68537a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43262/0 (socket says 172.21.15.64:43262) 2024-09-21T04:06:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.769+0000 7f0282ffd640 1 -- 172.21.15.64:0/818648271 learned_addr learned my addr 172.21.15.64:0/818648271 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.769+0000 7f02827fc640 1 --2- 172.21.15.64:0/818648271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f681534d0 0x556f68538030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.769+0000 7f02837fe640 1 --2- 172.21.15.64:0/818648271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f681741b0 0x556f68598710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.769+0000 7f0282ffd640 1 -- 172.21.15.64:0/818648271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f681741b0 msgr2=0x556f68598710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.769+0000 7f0282ffd640 1 --2- 172.21.15.64:0/818648271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f681741b0 0x556f68598710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.769+0000 7f0282ffd640 1 -- 172.21.15.64:0/818648271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f681534d0 msgr2=0x556f68538030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.769+0000 7f0282ffd640 1 --2- 172.21.15.64:0/818648271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f681534d0 0x556f68538030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.769+0000 7f0282ffd640 1 -- 172.21.15.64:0/818648271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f68598db0 con 0x556f6814b2f0 2024-09-21T04:06:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.769+0000 7f0282ffd640 1 --2- 172.21.15.64:0/818648271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f6814b2f0 0x556f68537a50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f02740ed1b0 tx=0x7f02740ef690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3148c8f1fddea1ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.769+0000 7f026a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:06:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.769+0000 7f026a7fc640 1 -- 172.21.15.64:0/818648271 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274058e60 con 0x556f6814b2f0 2024-09-21T04:06:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.769+0000 7f026a7fc640 1 -- 172.21.15.64:0/818648271 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f02740872b0 con 0x556f6814b2f0 2024-09-21T04:06:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.769+0000 7f029045f740 1 -- 172.21.15.64:0/818648271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02740ad130 con 0x556f6814b2f0 2024-09-21T04:06:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.769+0000 7f026a7fc640 1 -- 172.21.15.64:0/818648271 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f027401d070 con 0x556f6814b2f0 2024-09-21T04:06:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.769+0000 7f029045f740 10 client.16137 fetch_fsmap learned FSMap version 30 2024-09-21T04:06:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.769+0000 7f029045f740 1 -- 172.21.15.64:0/818648271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556f685a1240 con 0x556f6814b2f0 2024-09-21T04:06:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.770+0000 7f026a7fc640 1 -- 172.21.15.64:0/818648271 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02740ad130 con 0x556f6814b2f0 2024-09-21T04:06:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.770+0000 7f026a7fc640 1 -- 172.21.15.64:0/818648271 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f02740ac880 con 0x556f6814b2f0 2024-09-21T04:06:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.770+0000 7f029045f740 10 client.16137 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:06:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.770+0000 7f029045f740 10 client.16137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:06:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.770+0000 7f029045f740 20 client.16137 populate_metadata read hostname 'smithi064' 2024-09-21T04:06:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.770+0000 7f029045f740 1 --2- 172.21.15.64:0/818648271 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556f681081c0 0x556f685a3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.770+0000 7f029045f740 4 client.16137 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:06:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.770+0000 7f029045f740 1 -- 172.21.15.64:0/818648271 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556f6848c0f0 con 0x556f681081c0 2024-09-21T04:06:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.770+0000 7f02827fc640 1 --2- 172.21.15.64:0/818648271 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556f681081c0 0x556f685a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.770+0000 7f026a7fc640 10 client.16137 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:39.770+0000 7f02827fc640 1 --2- 172.21.15.64:0/818648271 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556f681081c0 0x556f685a3670 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=185dd1b68d6f5ce6 server_cookie=17df8033c536052c in_seq=0 out_seq=0 2024-09-21T04:06:40.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:40.314+0000 7f026a7fc640 1 -- 172.21.15.64:0/818648271 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556f6848c0f0 con 0x556f681081c0 2024-09-21T04:06:40.317 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:40.314+0000 7f026a7fc640 10 client.16137 handle_command_reply: tid=0 2024-09-21T04:06:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:40.314+0000 7f029045f740 1 client.16137 shutdown 2024-09-21T04:06:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:40.314+0000 7f029045f740 1 -- 172.21.15.64:0/818648271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f6814b2f0 msgr2=0x556f68537a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:40.315+0000 7f029045f740 1 --2- 172.21.15.64:0/818648271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f6814b2f0 0x556f68537a50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f02740ed1b0 tx=0x7f02740ef690 comp rx=0 tx=0).stop 2024-09-21T04:06:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:40.315+0000 7f029045f740 1 -- 172.21.15.64:0/818648271 shutdown_connections 2024-09-21T04:06:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:40.315+0000 7f029045f740 1 --2- 172.21.15.64:0/818648271 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556f681741b0 0x556f68598710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:40.315+0000 7f029045f740 1 --2- 172.21.15.64:0/818648271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556f681534d0 0x556f68538030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:40.315+0000 7f029045f740 1 --2- 172.21.15.64:0/818648271 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556f681081c0 0x556f685a3670 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:40.315+0000 7f029045f740 1 --2- 172.21.15.64:0/818648271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f6814b2f0 0x556f68537a50 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:40.315+0000 7f026a7fc640 0 client.16137 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:40.315+0000 7f029045f740 1 -- 172.21.15.64:0/818648271 >> 172.21.15.64:0/818648271 conn(0x556f68168300 msgr2=0x556f6851b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:40.315+0000 7f029045f740 1 -- 172.21.15.64:0/818648271 shutdown_connections 2024-09-21T04:06:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:40.315+0000 7f029045f740 1 -- 172.21.15.64:0/818648271 wait complete. 2024-09-21T04:06:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:40.315+0000 7f029045f740 20 client.16137 trim_cache size 0 max 16384 2024-09-21T04:06:40.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:06:40.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:06:40.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:06:40.319 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:06:40.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:40.316+0000 7f0289415640 1 -- 172.21.15.64:0/2952369580 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f02700b6680 msgr2=0x7f02700b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:40.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:40.316+0000 7f0289415640 1 --2- 172.21.15.64:0/2952369580 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f02700b6680 0x7f02700b8b90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f027c006010 tx=0x7f027c005a40 comp rx=0 tx=0).stop 2024-09-21T04:06:40.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:40.316+0000 7f0289415640 1 -- 172.21.15.64:0/2952369580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0284104980 msgr2=0x7f028419c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:40.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:40.316+0000 7f0289415640 1 --2- 172.21.15.64:0/2952369580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0284104980 0x7f028419c600 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f027400cce0 tx=0x7f027400a620 comp rx=0 tx=0).stop 2024-09-21T04:06:40.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:40.316+0000 7f0289415640 1 -- 172.21.15.64:0/2952369580 shutdown_connections 2024-09-21T04:06:40.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:40.316+0000 7f0289415640 1 --2- 172.21.15.64:0/2952369580 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f02700b6680 0x7f02700b8b90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:40.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:40.316+0000 7f0289415640 1 --2- 172.21.15.64:0/2952369580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f028410dda0 0x7f02841a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:40.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:40.316+0000 7f0289415640 1 --2- 172.21.15.64:0/2952369580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f02841052d0 0x7f028419cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:40.320 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:40.316+0000 7f0289415640 1 --2- 172.21.15.64:0/2952369580 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0284104980 0x7f028419c600 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:40.316+0000 7f0289415640 1 -- 172.21.15.64:0/2952369580 >> 172.21.15.64:0/2952369580 conn(0x7f02840fe3f0 msgr2=0x7f0284102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:40.316+0000 7f0289415640 1 -- 172.21.15.64:0/2952369580 shutdown_connections 2024-09-21T04:06:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:40.316+0000 7f0289415640 1 -- 172.21.15.64:0/2952369580 wait complete. 2024-09-21T04:06:40.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:40 smithi131 ceph-mon[35193]: pgmap v400: 97 pgs: 97 active+clean; 284 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.7 MiB/s wr, 722 op/s 2024-09-21T04:06:40.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:40 smithi119 ceph-mon[34811]: pgmap v400: 97 pgs: 97 active+clean; 284 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.7 MiB/s wr, 722 op/s 2024-09-21T04:06:40.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:40 smithi064 ceph-mon[31180]: pgmap v400: 97 pgs: 97 active+clean; 284 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.7 MiB/s wr, 722 op/s 2024-09-21T04:06:41.172 INFO:tasks.workunit.client.0.smithi064.stdout: 53 686 1 529 1 361 1 2024-09-21T04:06:42.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:42 smithi131 ceph-mon[35193]: pgmap v401: 97 pgs: 97 active+clean; 284 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 6.0 MiB/s wr, 629 op/s 2024-09-21T04:06:42.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:42 smithi119 ceph-mon[34811]: pgmap v401: 97 pgs: 97 active+clean; 284 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 6.0 MiB/s wr, 629 op/s 2024-09-21T04:06:42.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:42 smithi064 ceph-mon[31180]: pgmap v401: 97 pgs: 97 active+clean; 284 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 6.0 MiB/s wr, 629 op/s 2024-09-21T04:06:44.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:44 smithi131 ceph-mon[35193]: pgmap v402: 97 pgs: 97 active+clean; 294 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.1 MiB/s wr, 700 op/s 2024-09-21T04:06:44.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:44 smithi119 ceph-mon[34811]: pgmap v402: 97 pgs: 97 active+clean; 294 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.1 MiB/s wr, 700 op/s 2024-09-21T04:06:44.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:44 smithi064 ceph-mon[31180]: pgmap v402: 97 pgs: 97 active+clean; 294 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.1 MiB/s wr, 700 op/s 2024-09-21T04:06:46.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:46 smithi131 ceph-mon[35193]: pgmap v403: 97 pgs: 97 active+clean; 294 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.6 MiB/s wr, 587 op/s 2024-09-21T04:06:46.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:46 smithi119 ceph-mon[34811]: pgmap v403: 97 pgs: 97 active+clean; 294 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.6 MiB/s wr, 587 op/s 2024-09-21T04:06:46.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:46 smithi064 ceph-mon[31180]: pgmap v403: 97 pgs: 97 active+clean; 294 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.6 MiB/s wr, 587 op/s 2024-09-21T04:06:48.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:48 smithi131 ceph-mon[35193]: pgmap v404: 97 pgs: 97 active+clean; 302 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.3 MiB/s wr, 634 op/s 2024-09-21T04:06:48.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:48 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:06:48.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:48 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:06:48.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:48 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:06:48.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:48 smithi119 ceph-mon[34811]: pgmap v404: 97 pgs: 97 active+clean; 302 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.3 MiB/s wr, 634 op/s 2024-09-21T04:06:48.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:48 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:06:48.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:48 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:06:48.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:48 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:06:48.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:48 smithi064 ceph-mon[31180]: pgmap v404: 97 pgs: 97 active+clean; 302 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.3 MiB/s wr, 634 op/s 2024-09-21T04:06:48.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:48 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:06:48.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:48 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:06:48.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:48 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:06:50.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:50 smithi131 ceph-mon[35193]: pgmap v405: 97 pgs: 97 active+clean; 302 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 313 op/s 2024-09-21T04:06:50.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:50 smithi119 ceph-mon[34811]: pgmap v405: 97 pgs: 97 active+clean; 302 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 313 op/s 2024-09-21T04:06:50.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:50 smithi064 ceph-mon[31180]: pgmap v405: 97 pgs: 97 active+clean; 302 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.6 MiB/s wr, 313 op/s 2024-09-21T04:06:51.172 INFO:tasks.workunit.client.0.smithi064.stdout: 57 345442 242 232589 266 165878 790 2024-09-21T04:06:52.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:52 smithi131 ceph-mon[35193]: pgmap v406: 97 pgs: 97 active+clean; 302 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-09-21T04:06:52.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:52 smithi119 ceph-mon[34811]: pgmap v406: 97 pgs: 97 active+clean; 302 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-09-21T04:06:52.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:52 smithi064 ceph-mon[31180]: pgmap v406: 97 pgs: 97 active+clean; 302 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-09-21T04:06:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.418+0000 7fdcdb757640 1 -- 172.21.15.64:0/3637910622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd4102060 msgr2=0x7fdcd4102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.418+0000 7fdcdb757640 1 --2- 172.21.15.64:0/3637910622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd4102060 0x7fdcd4102470 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0009900 tx=0x7fdcd002a410 comp rx=0 tx=0).stop 2024-09-21T04:06:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.418+0000 7fdcdb757640 1 -- 172.21.15.64:0/3637910622 shutdown_connections 2024-09-21T04:06:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.418+0000 7fdcdb757640 1 --2- 172.21.15.64:0/3637910622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcd4109d80 0x7fdcd41105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.418+0000 7fdcdb757640 1 --2- 172.21.15.64:0/3637910622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcd4109430 0x7fdcd4109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.418+0000 7fdcdb757640 1 --2- 172.21.15.64:0/3637910622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd4102060 0x7fdcd4102470 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.418+0000 7fdcdb757640 1 -- 172.21.15.64:0/3637910622 >> 172.21.15.64:0/3637910622 conn(0x7fdcd4079e90 msgr2=0x7fdcd407a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.418+0000 7fdcdb757640 1 -- 172.21.15.64:0/3637910622 shutdown_connections 2024-09-21T04:06:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.418+0000 7fdcdb757640 1 -- 172.21.15.64:0/3637910622 wait complete. 2024-09-21T04:06:53.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.419+0000 7fdcdb757640 1 Processor -- start 2024-09-21T04:06:53.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.419+0000 7fdcdb757640 1 -- start start 2024-09-21T04:06:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.419+0000 7fdcdb757640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcd4102060 0x7fdcd4102ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.419+0000 7fdcdb757640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd4109430 0x7fdcd4103530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.419+0000 7fdcdb757640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcd4109d80 0x7fdcd4103a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.419+0000 7fdcdb757640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd4113a40 con 0x7fdcd4109430 2024-09-21T04:06:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.419+0000 7fdcdb757640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd41138c0 con 0x7fdcd4109d80 2024-09-21T04:06:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.419+0000 7fdcdb757640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd4113bc0 con 0x7fdcd4102060 2024-09-21T04:06:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.420+0000 7fdcd8cf2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd4109430 0x7fdcd4103530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.420+0000 7fdcd9cf4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcd4109d80 0x7fdcd4103a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.420+0000 7fdcd9cf4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcd4109d80 0x7fdcd4103a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:48062/0 (socket says 172.21.15.64:48062) 2024-09-21T04:06:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.420+0000 7fdcd9cf4640 1 -- 172.21.15.64:0/684969772 learned_addr learned my addr 172.21.15.64:0/684969772 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.420+0000 7fdcd94f3640 1 --2- 172.21.15.64:0/684969772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcd4102060 0x7fdcd4102ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.420+0000 7fdcd9cf4640 1 -- 172.21.15.64:0/684969772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcd4102060 msgr2=0x7fdcd4102ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.420+0000 7fdcd9cf4640 1 --2- 172.21.15.64:0/684969772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcd4102060 0x7fdcd4102ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.420+0000 7fdcd9cf4640 1 -- 172.21.15.64:0/684969772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd4109430 msgr2=0x7fdcd4103530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.420+0000 7fdcd9cf4640 1 --2- 172.21.15.64:0/684969772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd4109430 0x7fdcd4103530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.420+0000 7fdcd9cf4640 1 -- 172.21.15.64:0/684969772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd41a8750 con 0x7fdcd4109d80 2024-09-21T04:06:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.420+0000 7fdcd9cf4640 1 --2- 172.21.15.64:0/684969772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcd4109d80 0x7fdcd4103a70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00eb10 tx=0x7fdccc00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.420+0000 7fdcca7fc640 1 -- 172.21.15.64:0/684969772 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc004b20 con 0x7fdcd4109d80 2024-09-21T04:06:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.420+0000 7fdcdb757640 1 -- 172.21.15.64:0/684969772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd41a8a40 con 0x7fdcd4109d80 2024-09-21T04:06:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.420+0000 7fdcca7fc640 1 -- 172.21.15.64:0/684969772 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdccc004cc0 con 0x7fdcd4109d80 2024-09-21T04:06:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.420+0000 7fdcca7fc640 1 -- 172.21.15.64:0/684969772 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc010670 con 0x7fdcd4109d80 2024-09-21T04:06:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.420+0000 7fdcdb757640 1 -- 172.21.15.64:0/684969772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd41a8f90 con 0x7fdcd4109d80 2024-09-21T04:06:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.423+0000 7fdce26c7740 1 Processor -- start 2024-09-21T04:06:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.425+0000 7fdcca7fc640 1 -- 172.21.15.64:0/684969772 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fdccc010810 con 0x7fdcd4109d80 2024-09-21T04:06:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.425+0000 7fdce26c7740 1 -- start start 2024-09-21T04:06:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.425+0000 7fdce26c7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56005bc281b0 0x56005bfcd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.425+0000 7fdce26c7740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56005bbff2f0 0x56005bfcdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.425+0000 7fdce26c7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005bc074d0 0x56005bfce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.425+0000 7fdce26c7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56005bf426f0 con 0x56005bc281b0 2024-09-21T04:06:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.425+0000 7fdce26c7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56005bf31b20 con 0x56005bc074d0 2024-09-21T04:06:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.425+0000 7fdce26c7740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56005bf35930 con 0x56005bbff2f0 2024-09-21T04:06:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.425+0000 7fdcd8cf2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56005bc281b0 0x56005bfcd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.425+0000 7fdcd8cf2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56005bc281b0 0x56005bfcd9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59026/0 (socket says 172.21.15.64:59026) 2024-09-21T04:06:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.425+0000 7fdcd8cf2640 1 -- 172.21.15.64:0/3066158632 learned_addr learned my addr 172.21.15.64:0/3066158632 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.425+0000 7fdcd94f3640 1 --2- 172.21.15.64:0/3066158632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56005bbff2f0 0x56005bfcdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.426+0000 7fdcd8cf2640 1 --2- 172.21.15.64:0/3066158632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005bc074d0 0x56005bfce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.426+0000 7fdcca7fc640 1 --2- 172.21.15.64:0/684969772 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdcc00b66b0 0x7fdcc00b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.426+0000 7fdcd94f3640 1 -- 172.21.15.64:0/3066158632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005bc074d0 msgr2=0x56005bfce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.426+0000 7fdcd94f3640 1 --2- 172.21.15.64:0/3066158632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005bc074d0 0x56005bfce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.426+0000 7fdcd94f3640 1 -- 172.21.15.64:0/3066158632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56005bc281b0 msgr2=0x56005bfcd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.426+0000 7fdcd94f3640 1 --2- 172.21.15.64:0/3066158632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56005bc281b0 0x56005bfcd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.426+0000 7fdcd94f3640 1 -- 172.21.15.64:0/3066158632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56005bef7530 con 0x56005bbff2f0 2024-09-21T04:06:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.426+0000 7fdcd8cf2640 1 --2- 172.21.15.64:0/3066158632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56005bc281b0 0x56005bfcd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:06:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.426+0000 7fdcd9cf4640 1 --2- 172.21.15.64:0/684969772 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdcc00b66b0 0x7fdcc00b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.426+0000 7fdcca7fc640 1 -- 172.21.15.64:0/684969772 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fdccc0e7ce0 con 0x7fdcd4109d80 2024-09-21T04:06:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.426+0000 7fdcd94f3640 1 --2- 172.21.15.64:0/3066158632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56005bbff2f0 0x56005bfcdef0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0006010 tx=0x7fdcd0032bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=150a0d2a39a4e505 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:53.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.426+0000 7fdcbf7fe640 1 -- 172.21.15.64:0/3066158632 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0035040 con 0x56005bbff2f0 2024-09-21T04:06:53.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.426+0000 7fdcbf7fe640 1 -- 172.21.15.64:0/3066158632 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcd0004030 con 0x56005bbff2f0 2024-09-21T04:06:53.431 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.426+0000 7fdcbf7fe640 1 -- 172.21.15.64:0/3066158632 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0005370 con 0x56005bbff2f0 2024-09-21T04:06:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.427+0000 7fdce26c7740 1 -- 172.21.15.64:0/3066158632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56005bbff2f0 msgr2=0x56005bfcdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.427+0000 7fdcd9cf4640 1 --2- 172.21.15.64:0/684969772 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdcc00b66b0 0x7fdcc00b8bc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdccc01b040 tx=0x7fdccc0d1040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.427+0000 7fdce26c7740 1 --2- 172.21.15.64:0/3066158632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56005bbff2f0 0x56005bfcdef0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0006010 tx=0x7fdcd0032bd0 comp rx=0 tx=0).stop 2024-09-21T04:06:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.427+0000 7fdce26c7740 1 -- 172.21.15.64:0/3066158632 shutdown_connections 2024-09-21T04:06:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.427+0000 7fdce26c7740 1 --2- 172.21.15.64:0/3066158632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005bc074d0 0x56005bfce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.427+0000 7fdce26c7740 1 --2- 172.21.15.64:0/3066158632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56005bbff2f0 0x56005bfcdef0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.427+0000 7fdce26c7740 1 --2- 172.21.15.64:0/3066158632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56005bc281b0 0x56005bfcd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.427+0000 7fdce26c7740 1 -- 172.21.15.64:0/3066158632 >> 172.21.15.64:0/3066158632 conn(0x56005bc1c300 msgr2=0x56005bfcb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.427+0000 7fdce26c7740 1 -- 172.21.15.64:0/3066158632 shutdown_connections 2024-09-21T04:06:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.427+0000 7fdce26c7740 1 -- 172.21.15.64:0/3066158632 wait complete. 2024-09-21T04:06:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.428+0000 7fdce26c7740 1 -- start start 2024-09-21T04:06:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.428+0000 7fdce26c7740 1 Processor -- start 2024-09-21T04:06:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.428+0000 7fdce26c7740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56005bbff2f0 0x56005bfd90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.428+0000 7fdce26c7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56005bc074d0 0x56005bfd95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.428+0000 7fdce26c7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005bc281b0 0x56005bfe53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.428+0000 7fdce26c7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56005be69ab0 con 0x56005bc074d0 2024-09-21T04:06:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.428+0000 7fdce26c7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56005bed7730 con 0x56005bc281b0 2024-09-21T04:06:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.428+0000 7fdce26c7740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56005bfdc0d0 con 0x56005bbff2f0 2024-09-21T04:06:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.429+0000 7fdce26c7740 10 client.0 _ll_register_callbacks cb 0x56005bf2b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:06:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.429+0000 7fdcd94f3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56005bc074d0 0x56005bfd95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.429+0000 7fdcd94f3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56005bc074d0 0x56005bfd95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59040/0 (socket says 172.21.15.64:59040) 2024-09-21T04:06:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.429+0000 7fdcd8cf2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56005bbff2f0 0x56005bfd90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.429+0000 7fdcd94f3640 1 -- 172.21.15.64:0/23667487 learned_addr learned my addr 172.21.15.64:0/23667487 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.429+0000 7fdcd8cf2640 1 --2- 172.21.15.64:0/23667487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005bc281b0 0x56005bfe53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.429+0000 7fdcd8cf2640 1 -- 172.21.15.64:0/23667487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56005bbff2f0 msgr2=0x56005bfd90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.429+0000 7fdcd8cf2640 1 --2- 172.21.15.64:0/23667487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56005bbff2f0 0x56005bfd90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.429+0000 7fdcd8cf2640 1 -- 172.21.15.64:0/23667487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56005bc074d0 msgr2=0x56005bfd95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.429+0000 7fdcd8cf2640 1 --2- 172.21.15.64:0/23667487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56005bc074d0 0x56005bfd95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.429+0000 7fdcd8cf2640 1 -- 172.21.15.64:0/23667487 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56005bc93790 con 0x56005bc281b0 2024-09-21T04:06:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.429+0000 7fdcd94f3640 1 --2- 172.21.15.64:0/23667487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56005bc074d0 0x56005bfd95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:06:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.429+0000 7fdcd8cf2640 1 --2- 172.21.15.64:0/23667487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005bc281b0 0x56005bfe53c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4005d00 tx=0x7fdcc4005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c6b62226a42c6bf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.429+0000 7fdcbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:06:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.429+0000 7fdcbcff9640 1 -- 172.21.15.64:0/23667487 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4010e40 con 0x56005bc281b0 2024-09-21T04:06:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.429+0000 7fdce26c7740 1 -- 172.21.15.64:0/23667487 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcd0004030 con 0x56005bc281b0 2024-09-21T04:06:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.429+0000 7fdcbcff9640 1 -- 172.21.15.64:0/23667487 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fdcc4004030 con 0x56005bc281b0 2024-09-21T04:06:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.430+0000 7fdcbcff9640 1 -- 172.21.15.64:0/23667487 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdcc402b050 con 0x56005bc281b0 2024-09-21T04:06:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.430+0000 7fdce26c7740 10 client.25859 fetch_fsmap learned FSMap version 30 2024-09-21T04:06:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.430+0000 7fdce26c7740 1 -- 172.21.15.64:0/23667487 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56005bd0c620 con 0x56005bc281b0 2024-09-21T04:06:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.430+0000 7fdcbcff9640 1 -- 172.21.15.64:0/23667487 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcd0004030 con 0x56005bc281b0 2024-09-21T04:06:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.430+0000 7fdcbcff9640 1 -- 172.21.15.64:0/23667487 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fdcc4026400 con 0x56005bc281b0 2024-09-21T04:06:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.430+0000 7fdce26c7740 10 client.25859 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:06:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.430+0000 7fdce26c7740 10 client.25859 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:06:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.430+0000 7fdce26c7740 20 client.25859 populate_metadata read hostname 'smithi064' 2024-09-21T04:06:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.430+0000 7fdce26c7740 1 --2- 172.21.15.64:0/23667487 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56005bbe6d80 0x56005bfeb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.430+0000 7fdce26c7740 4 client.25859 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:06:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.430+0000 7fdce26c7740 1 -- 172.21.15.64:0/23667487 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56005bf44640 con 0x56005bbe6d80 2024-09-21T04:06:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.430+0000 7fdcd94f3640 1 --2- 172.21.15.64:0/23667487 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56005bbe6d80 0x56005bfeb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.431+0000 7fdcbcff9640 10 client.25859 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.431+0000 7fdcd94f3640 1 --2- 172.21.15.64:0/23667487 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56005bbe6d80 0x56005bfeb2a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86ae3ae39681c5f4 server_cookie=4e28546bbadcc057 in_seq=0 out_seq=0 2024-09-21T04:06:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.432+0000 7fdcbcff9640 1 -- 172.21.15.64:0/23667487 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56005bf44640 con 0x56005bbe6d80 2024-09-21T04:06:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.432+0000 7fdcbcff9640 10 client.25859 handle_command_reply: tid=0 2024-09-21T04:06:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.432+0000 7fdce26c7740 1 client.25859 shutdown 2024-09-21T04:06:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.432+0000 7fdce26c7740 1 -- 172.21.15.64:0/23667487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005bc281b0 msgr2=0x56005bfe53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.432+0000 7fdce26c7740 1 --2- 172.21.15.64:0/23667487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005bc281b0 0x56005bfe53c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4005d00 tx=0x7fdcc4005c90 comp rx=0 tx=0).stop 2024-09-21T04:06:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.432+0000 7fdce26c7740 1 -- 172.21.15.64:0/23667487 shutdown_connections 2024-09-21T04:06:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.432+0000 7fdce26c7740 1 --2- 172.21.15.64:0/23667487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005bc281b0 0x56005bfe53c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.432+0000 7fdce26c7740 1 --2- 172.21.15.64:0/23667487 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56005bbe6d80 0x56005bfeb2a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.432+0000 7fdce26c7740 1 --2- 172.21.15.64:0/23667487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56005bc074d0 0x56005bfd95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.432+0000 7fdce26c7740 1 --2- 172.21.15.64:0/23667487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56005bbff2f0 0x56005bfd90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.432+0000 7fdcbcff9640 0 client.25859 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.432+0000 7fdce26c7740 1 -- 172.21.15.64:0/23667487 >> 172.21.15.64:0/23667487 conn(0x56005bc1c300 msgr2=0x56005bfce9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.432+0000 7fdce26c7740 1 -- 172.21.15.64:0/23667487 shutdown_connections 2024-09-21T04:06:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.432+0000 7fdce26c7740 1 -- 172.21.15.64:0/23667487 wait complete. 2024-09-21T04:06:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.432+0000 7fdce26c7740 20 client.25859 trim_cache size 0 max 16384 2024-09-21T04:06:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.447+0000 7fdce26c7740 1 Processor -- start 2024-09-21T04:06:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.447+0000 7fdce26c7740 1 -- start start 2024-09-21T04:06:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.447+0000 7fdce26c7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005bbff2f0 0x56005bfcf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.447+0000 7fdce26c7740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56005bc074d0 0x56005bfdacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.447+0000 7fdce26c7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56005bc281b0 0x56005bfe52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.447+0000 7fdce26c7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd41b4350 con 0x56005bc281b0 2024-09-21T04:06:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.447+0000 7fdce26c7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd41b1b90 con 0x56005bbff2f0 2024-09-21T04:06:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.447+0000 7fdce26c7740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd41af9e0 con 0x56005bc074d0 2024-09-21T04:06:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.447+0000 7fdcd8cf2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005bbff2f0 0x56005bfcf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.447+0000 7fdcd8cf2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005bbff2f0 0x56005bfcf3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:48088/0 (socket says 172.21.15.64:48088) 2024-09-21T04:06:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.447+0000 7fdcd8cf2640 1 -- 172.21.15.64:0/729572923 learned_addr learned my addr 172.21.15.64:0/729572923 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.447+0000 7fdcd8cf2640 1 --2- 172.21.15.64:0/729572923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56005bc281b0 0x56005bfe52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.447+0000 7fdcd94f3640 1 --2- 172.21.15.64:0/729572923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56005bc074d0 0x56005bfdacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.448+0000 7fdcd8cf2640 1 -- 172.21.15.64:0/729572923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56005bc074d0 msgr2=0x56005bfdacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.448+0000 7fdcd8cf2640 1 --2- 172.21.15.64:0/729572923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56005bc074d0 0x56005bfdacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.448+0000 7fdcd8cf2640 1 -- 172.21.15.64:0/729572923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56005bc281b0 msgr2=0x56005bfe52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.448+0000 7fdcd8cf2640 1 --2- 172.21.15.64:0/729572923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56005bc281b0 0x56005bfe52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.448+0000 7fdcd8cf2640 1 -- 172.21.15.64:0/729572923 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56005bfd07c0 con 0x56005bbff2f0 2024-09-21T04:06:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.448+0000 7fdcd8cf2640 1 --2- 172.21.15.64:0/729572923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005bbff2f0 0x56005bfcf3a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400bfd0 tx=0x7fdcc400b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23ba867546b964c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.448+0000 7fdcbeffd640 1 -- 172.21.15.64:0/729572923 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4012720 con 0x56005bbff2f0 2024-09-21T04:06:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.448+0000 7fdcbeffd640 1 -- 172.21.15.64:0/729572923 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcc4026400 con 0x56005bbff2f0 2024-09-21T04:06:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.448+0000 7fdcbeffd640 1 -- 172.21.15.64:0/729572923 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc40052d0 con 0x56005bbff2f0 2024-09-21T04:06:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.449+0000 7fdce26c7740 1 -- 172.21.15.64:0/729572923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005bbff2f0 msgr2=0x56005bfcf3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.449+0000 7fdce26c7740 1 --2- 172.21.15.64:0/729572923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005bbff2f0 0x56005bfcf3a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400bfd0 tx=0x7fdcc400b390 comp rx=0 tx=0).stop 2024-09-21T04:06:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.449+0000 7fdce26c7740 1 -- 172.21.15.64:0/729572923 shutdown_connections 2024-09-21T04:06:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.449+0000 7fdce26c7740 1 --2- 172.21.15.64:0/729572923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56005bc281b0 0x56005bfe52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.449+0000 7fdce26c7740 1 --2- 172.21.15.64:0/729572923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56005bc074d0 0x56005bfdacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.449+0000 7fdce26c7740 1 --2- 172.21.15.64:0/729572923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005bbff2f0 0x56005bfcf3a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.449+0000 7fdce26c7740 1 -- 172.21.15.64:0/729572923 >> 172.21.15.64:0/729572923 conn(0x56005bc1c300 msgr2=0x56005bfda5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.449+0000 7fdce26c7740 1 -- 172.21.15.64:0/729572923 shutdown_connections 2024-09-21T04:06:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.449+0000 7fdce26c7740 1 -- 172.21.15.64:0/729572923 wait complete. 2024-09-21T04:06:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.449+0000 7fdce26c7740 1 -- start start 2024-09-21T04:06:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.449+0000 7fdce26c7740 1 Processor -- start 2024-09-21T04:06:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.450+0000 7fdce26c7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005bbff2f0 0x56005bfdebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.450+0000 7fdce26c7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56005bc074d0 0x56005bfdf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.450+0000 7fdce26c7740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56005bc281b0 0x56005c04dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.450+0000 7fdce26c7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd41b0050 con 0x56005bc074d0 2024-09-21T04:06:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.450+0000 7fdce26c7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd41b14c0 con 0x56005bbff2f0 2024-09-21T04:06:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.450+0000 7fdce26c7740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd41b0720 con 0x56005bc281b0 2024-09-21T04:06:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.450+0000 7fdce26c7740 10 client.0 _ll_register_callbacks cb 0x7fdcc40124e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:06:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.450+0000 7fdcd8cf2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005bbff2f0 0x56005bfdebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.450+0000 7fdcd8cf2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005bbff2f0 0x56005bfdebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:48094/0 (socket says 172.21.15.64:48094) 2024-09-21T04:06:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.450+0000 7fdcd8cf2640 1 -- 172.21.15.64:0/1557608513 learned_addr learned my addr 172.21.15.64:0/1557608513 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.450+0000 7fdcd94f3640 1 --2- 172.21.15.64:0/1557608513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56005bc074d0 0x56005bfdf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.450+0000 7fdcd8cf2640 1 --2- 172.21.15.64:0/1557608513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56005bc281b0 0x56005c04dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.451+0000 7fdcd94f3640 1 -- 172.21.15.64:0/1557608513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56005bc281b0 msgr2=0x56005c04dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.451+0000 7fdcd94f3640 1 --2- 172.21.15.64:0/1557608513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56005bc281b0 0x56005c04dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.451+0000 7fdcd94f3640 1 -- 172.21.15.64:0/1557608513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005bbff2f0 msgr2=0x56005bfdebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.451+0000 7fdcd94f3640 1 --2- 172.21.15.64:0/1557608513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005bbff2f0 0x56005bfdebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.451+0000 7fdcd94f3640 1 -- 172.21.15.64:0/1557608513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56005c04e2a0 con 0x56005bc074d0 2024-09-21T04:06:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.451+0000 7fdcd8cf2640 1 --2- 172.21.15.64:0/1557608513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005bbff2f0 0x56005bfdebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:06:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.451+0000 7fdcd94f3640 1 --2- 172.21.15.64:0/1557608513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56005bc074d0 0x56005bfdf190 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x56005bfeb9d0 tx=0x7fdcd0005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2bc3803fb4b5f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.451+0000 7fdcbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:06:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.451+0000 7fdcbcff9640 1 -- 172.21.15.64:0/1557608513 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0035040 con 0x56005bc074d0 2024-09-21T04:06:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.451+0000 7fdcbcff9640 1 -- 172.21.15.64:0/1557608513 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fdcd0004470 con 0x56005bc074d0 2024-09-21T04:06:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.451+0000 7fdce26c7740 1 -- 172.21.15.64:0/1557608513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcc4026400 con 0x56005bc074d0 2024-09-21T04:06:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.451+0000 7fdcbcff9640 1 -- 172.21.15.64:0/1557608513 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdcd0062050 con 0x56005bc074d0 2024-09-21T04:06:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.451+0000 7fdce26c7740 10 client.16155 fetch_fsmap learned FSMap version 30 2024-09-21T04:06:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.451+0000 7fdce26c7740 1 -- 172.21.15.64:0/1557608513 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56005c055630 con 0x56005bc074d0 2024-09-21T04:06:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.452+0000 7fdcbcff9640 1 -- 172.21.15.64:0/1557608513 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcc4026400 con 0x56005bc074d0 2024-09-21T04:06:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.452+0000 7fdcbcff9640 1 -- 172.21.15.64:0/1557608513 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fdcd005d030 con 0x56005bc074d0 2024-09-21T04:06:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.452+0000 7fdce26c7740 10 client.16155 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:06:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.452+0000 7fdce26c7740 10 client.16155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:06:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.452+0000 7fdce26c7740 20 client.16155 populate_metadata read hostname 'smithi064' 2024-09-21T04:06:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.452+0000 7fdce26c7740 1 --2- 172.21.15.64:0/1557608513 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56005bbbc1c0 0x56005c0579d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.452+0000 7fdce26c7740 4 client.16155 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:06:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.452+0000 7fdce26c7740 1 -- 172.21.15.64:0/1557608513 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56005bf400f0 con 0x56005bbbc1c0 2024-09-21T04:06:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.452+0000 7fdcd8cf2640 1 --2- 172.21.15.64:0/1557608513 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56005bbbc1c0 0x56005c0579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.452+0000 7fdcbcff9640 10 client.16155 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:53.453+0000 7fdcd8cf2640 1 --2- 172.21.15.64:0/1557608513 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56005bbbc1c0 0x56005c0579d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36c80e843542bb4b server_cookie=fe3e094fbcb6780d in_seq=0 out_seq=0 2024-09-21T04:06:54.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:54 smithi131 ceph-mon[35193]: pgmap v407: 97 pgs: 97 active+clean; 308 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 168 op/s 2024-09-21T04:06:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:54.575+0000 7fdcbcff9640 1 -- 172.21.15.64:0/1557608513 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56005bf400f0 con 0x56005bbbc1c0 2024-09-21T04:06:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:54.575+0000 7fdcbcff9640 10 client.16155 handle_command_reply: tid=0 2024-09-21T04:06:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:54.575+0000 7fdce26c7740 1 client.16155 shutdown 2024-09-21T04:06:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:54.576+0000 7fdce26c7740 1 -- 172.21.15.64:0/1557608513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56005bc074d0 msgr2=0x56005bfdf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:54.576+0000 7fdce26c7740 1 --2- 172.21.15.64:0/1557608513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56005bc074d0 0x56005bfdf190 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x56005bfeb9d0 tx=0x7fdcd0005850 comp rx=0 tx=0).stop 2024-09-21T04:06:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:54.576+0000 7fdce26c7740 1 -- 172.21.15.64:0/1557608513 shutdown_connections 2024-09-21T04:06:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:54.576+0000 7fdce26c7740 1 --2- 172.21.15.64:0/1557608513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56005bc281b0 0x56005c04dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:54.576+0000 7fdce26c7740 1 --2- 172.21.15.64:0/1557608513 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56005bbbc1c0 0x56005c0579d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:54.576+0000 7fdce26c7740 1 --2- 172.21.15.64:0/1557608513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56005bc074d0 0x56005bfdf190 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:54.576+0000 7fdce26c7740 1 --2- 172.21.15.64:0/1557608513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56005bbff2f0 0x56005bfdebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:54.576+0000 7fdcbcff9640 0 client.16155 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:54.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:54.576+0000 7fdce26c7740 1 -- 172.21.15.64:0/1557608513 >> 172.21.15.64:0/1557608513 conn(0x56005bc1c300 msgr2=0x56005bfdaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:54.576+0000 7fdce26c7740 1 -- 172.21.15.64:0/1557608513 shutdown_connections 2024-09-21T04:06:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:54.576+0000 7fdce26c7740 1 -- 172.21.15.64:0/1557608513 wait complete. 2024-09-21T04:06:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:54.576+0000 7fdce26c7740 20 client.16155 trim_cache size 0 max 16384 2024-09-21T04:06:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:06:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:06:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:06:54.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:06:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:54.577+0000 7fdcdb757640 1 -- 172.21.15.64:0/684969772 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdcc00b66b0 msgr2=0x7fdcc00b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:54.577+0000 7fdcdb757640 1 --2- 172.21.15.64:0/684969772 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdcc00b66b0 0x7fdcc00b8bc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdccc01b040 tx=0x7fdccc0d1040 comp rx=0 tx=0).stop 2024-09-21T04:06:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:54.577+0000 7fdcdb757640 1 -- 172.21.15.64:0/684969772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcd4109d80 msgr2=0x7fdcd4103a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:54.577+0000 7fdcdb757640 1 --2- 172.21.15.64:0/684969772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcd4109d80 0x7fdcd4103a70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00eb10 tx=0x7fdccc00c2d0 comp rx=0 tx=0).stop 2024-09-21T04:06:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:54.577+0000 7fdcdb757640 1 -- 172.21.15.64:0/684969772 shutdown_connections 2024-09-21T04:06:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:54.577+0000 7fdcdb757640 1 --2- 172.21.15.64:0/684969772 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdcc00b66b0 0x7fdcc00b8bc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:54.577+0000 7fdcdb757640 1 --2- 172.21.15.64:0/684969772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcd4109d80 0x7fdcd4103a70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:54.577+0000 7fdcdb757640 1 --2- 172.21.15.64:0/684969772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcd4109430 0x7fdcd4103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:54.577+0000 7fdcdb757640 1 --2- 172.21.15.64:0/684969772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcd4102060 0x7fdcd4102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:54.577+0000 7fdcdb757640 1 -- 172.21.15.64:0/684969772 >> 172.21.15.64:0/684969772 conn(0x7fdcd4079e90 msgr2=0x7fdcd4107520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:54.577+0000 7fdcdb757640 1 -- 172.21.15.64:0/684969772 shutdown_connections 2024-09-21T04:06:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:06:54.577+0000 7fdcdb757640 1 -- 172.21.15.64:0/684969772 wait complete. 2024-09-21T04:06:54.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:54 smithi119 ceph-mon[34811]: pgmap v407: 97 pgs: 97 active+clean; 308 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 168 op/s 2024-09-21T04:06:54.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:54 smithi064 ceph-mon[31180]: pgmap v407: 97 pgs: 97 active+clean; 308 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 168 op/s 2024-09-21T04:06:56.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:56 smithi131 ceph-mon[35193]: pgmap v408: 97 pgs: 97 active+clean; 308 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 93 op/s 2024-09-21T04:06:56.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:56 smithi119 ceph-mon[34811]: pgmap v408: 97 pgs: 97 active+clean; 308 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 93 op/s 2024-09-21T04:06:56.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:56 smithi064 ceph-mon[31180]: pgmap v408: 97 pgs: 97 active+clean; 308 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 93 op/s 2024-09-21T04:06:56.911 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T04:06:57.768 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:06:58.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:06:58 smithi131 ceph-mon[35193]: pgmap v409: 97 pgs: 97 active+clean; 311 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 123 op/s 2024-09-21T04:06:58.636 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:06:58 smithi064 ceph-mon[31180]: pgmap v409: 97 pgs: 97 active+clean; 311 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 123 op/s 2024-09-21T04:06:58.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.736+0000 7f0e7e3ad640 1 -- 172.21.15.64:0/2475731607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e78108030 msgr2=0x7f0e7810e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:58.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.736+0000 7f0e7e3ad640 1 --2- 172.21.15.64:0/2475731607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e78108030 0x7f0e7810e860 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009370 tx=0x7f0e6c02b650 comp rx=0 tx=0).stop 2024-09-21T04:06:58.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.736+0000 7f0e7e3ad640 1 -- 172.21.15.64:0/2475731607 shutdown_connections 2024-09-21T04:06:58.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.736+0000 7f0e7e3ad640 1 --2- 172.21.15.64:0/2475731607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e78108030 0x7f0e7810e860 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.736+0000 7f0e7e3ad640 1 --2- 172.21.15.64:0/2475731607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e781076e0 0x7f0e78107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.736+0000 7f0e7e3ad640 1 --2- 172.21.15.64:0/2475731607 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e78101da0 0x7f0e781021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.736+0000 7f0e7e3ad640 1 -- 172.21.15.64:0/2475731607 >> 172.21.15.64:0/2475731607 conn(0x7f0e78077800 msgr2=0x7f0e78077c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:58.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.739+0000 7f0e7e3ad640 1 -- 172.21.15.64:0/2475731607 shutdown_connections 2024-09-21T04:06:58.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.740+0000 7f0e7e3ad640 1 -- 172.21.15.64:0/2475731607 wait complete. 2024-09-21T04:06:58.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.742+0000 7f0e7e3ad640 1 Processor -- start 2024-09-21T04:06:58.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.744+0000 7f0e7e3ad640 1 -- start start 2024-09-21T04:06:58.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.745+0000 7f0e7e3ad640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e78101da0 0x7f0e78197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:58.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.746+0000 7f0e7e3ad640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e781076e0 0x7f0e78198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:58.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.746+0000 7f0e7e3ad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e7819d660 0x7f0e7819da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:58.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.746+0000 7f0e7e3ad640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0e781118f0 con 0x7f0e7819d660 2024-09-21T04:06:58.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.746+0000 7f0e7e3ad640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0e78111770 con 0x7f0e78101da0 2024-09-21T04:06:58.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.746+0000 7f0e7e3ad640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0e78111a70 con 0x7f0e781076e0 2024-09-21T04:06:58.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.747+0000 7f0e7c94a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e7819d660 0x7f0e7819da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:58.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.747+0000 7f0e7c94a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e7819d660 0x7f0e7819da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59070/0 (socket says 172.21.15.64:59070) 2024-09-21T04:06:58.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.747+0000 7f0e7c94a640 1 -- 172.21.15.64:0/518794581 learned_addr learned my addr 172.21.15.64:0/518794581 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:58.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.747+0000 7f0e77fff640 1 --2- 172.21.15.64:0/518794581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e78101da0 0x7f0e78197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:58.753 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.747+0000 7f0e777fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e781076e0 0x7f0e78198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:58.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:06:58 smithi119 ceph-mon[34811]: pgmap v409: 97 pgs: 97 active+clean; 311 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 123 op/s 2024-09-21T04:06:58.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.752+0000 7f0e7c94a640 1 -- 172.21.15.64:0/518794581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e781076e0 msgr2=0x7f0e78198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:58.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.752+0000 7f0e7c94a640 1 --2- 172.21.15.64:0/518794581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e781076e0 0x7f0e78198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.752+0000 7f0e7c94a640 1 -- 172.21.15.64:0/518794581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e78101da0 msgr2=0x7f0e78197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:58.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.752+0000 7f0e7c94a640 1 --2- 172.21.15.64:0/518794581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e78101da0 0x7f0e78197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.752+0000 7f0e7c94a640 1 -- 172.21.15.64:0/518794581 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e7819e240 con 0x7f0e7819d660 2024-09-21T04:06:58.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.756+0000 7f0e777fe640 1 --2- 172.21.15.64:0/518794581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e781076e0 0x7f0e78198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:06:58.761 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.758+0000 7f0e7c94a640 1 --2- 172.21.15.64:0/518794581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e7819d660 0x7f0e7819da70 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c035040 tx=0x7f0e6c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:58.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.761+0000 7f0e757fa640 1 -- 172.21.15.64:0/518794581 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c03f070 con 0x7f0e7819d660 2024-09-21T04:06:58.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.761+0000 7f0e757fa640 1 -- 172.21.15.64:0/518794581 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e6c0044a0 con 0x7f0e7819d660 2024-09-21T04:06:58.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.763+0000 7f0e757fa640 1 -- 172.21.15.64:0/518794581 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c036050 con 0x7f0e7819d660 2024-09-21T04:06:58.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.765+0000 7f0e7e3ad640 1 -- 172.21.15.64:0/518794581 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e781a47e0 con 0x7f0e7819d660 2024-09-21T04:06:58.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.765+0000 7f0e7e3ad640 1 -- 172.21.15.64:0/518794581 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e781a4b80 con 0x7f0e7819d660 2024-09-21T04:06:58.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.772+0000 7f0e84921740 1 Processor -- start 2024-09-21T04:06:58.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.772+0000 7f0e84921740 1 -- start start 2024-09-21T04:06:58.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.773+0000 7f0e84921740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605aa65f070 0x5605aaa06270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:58.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.773+0000 7f0e84921740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605aa6361b0 0x5605aaa067b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:58.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.773+0000 7f0e84921740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605aa63e390 0x5605aaa06cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:58.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.773+0000 7f0e84921740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5605aa97ce50 con 0x5605aa6361b0 2024-09-21T04:06:58.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.773+0000 7f0e84921740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5605aa96c280 con 0x5605aa63e390 2024-09-21T04:06:58.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.773+0000 7f0e84921740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5605aa970090 con 0x5605aa65f070 2024-09-21T04:06:58.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.775+0000 7f0e777fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605aa65f070 0x5605aaa06270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:58.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.775+0000 7f0e777fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605aa65f070 0x5605aaa06270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:47920/0 (socket says 172.21.15.64:47920) 2024-09-21T04:06:58.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.775+0000 7f0e777fe640 1 -- 172.21.15.64:0/2839086211 learned_addr learned my addr 172.21.15.64:0/2839086211 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:58.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.776+0000 7f0e777fe640 1 --2- 172.21.15.64:0/2839086211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605aa63e390 0x5605aaa06cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:58.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.776+0000 7f0e777fe640 1 -- 172.21.15.64:0/2839086211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605aa63e390 msgr2=0x5605aaa06cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:58.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.776+0000 7f0e777fe640 1 --2- 172.21.15.64:0/2839086211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605aa63e390 0x5605aaa06cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.776+0000 7f0e777fe640 1 -- 172.21.15.64:0/2839086211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605aa6361b0 msgr2=0x5605aaa067b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:58.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.776+0000 7f0e777fe640 1 --2- 172.21.15.64:0/2839086211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605aa6361b0 0x5605aaa067b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.776+0000 7f0e777fe640 1 -- 172.21.15.64:0/2839086211 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605aa6ca650 con 0x5605aa65f070 2024-09-21T04:06:58.779 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.776+0000 7f0e777fe640 1 --2- 172.21.15.64:0/2839086211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605aa65f070 0x5605aaa06270 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0e68005e10 tx=0x7f0e6800b4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=448233a56e11eb36 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:58.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.778+0000 7f0e5e7fc640 1 -- 172.21.15.64:0/2839086211 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68004030 con 0x5605aa65f070 2024-09-21T04:06:58.780 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.778+0000 7f0e5e7fc640 1 -- 172.21.15.64:0/2839086211 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e6800ee10 con 0x5605aa65f070 2024-09-21T04:06:58.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.780+0000 7f0e84921740 1 -- 172.21.15.64:0/2839086211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605aa65f070 msgr2=0x5605aaa06270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:58.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.780+0000 7f0e84921740 1 --2- 172.21.15.64:0/2839086211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605aa65f070 0x5605aaa06270 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0e68005e10 tx=0x7f0e6800b4a0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.782+0000 7f0e757fa640 1 -- 172.21.15.64:0/518794581 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f0e6c03b480 con 0x7f0e7819d660 2024-09-21T04:06:58.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.783+0000 7f0e757fa640 1 --2- 172.21.15.64:0/518794581 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0e600b6670 0x7f0e600b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:58.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.784+0000 7f0e757fa640 1 -- 172.21.15.64:0/518794581 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f0e6c107a90 con 0x7f0e7819d660 2024-09-21T04:06:58.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.784+0000 7f0e777fe640 1 --2- 172.21.15.64:0/518794581 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0e600b6670 0x7f0e600b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:58.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.788+0000 7f0e84921740 1 -- 172.21.15.64:0/2839086211 shutdown_connections 2024-09-21T04:06:58.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.788+0000 7f0e84921740 1 --2- 172.21.15.64:0/2839086211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605aa63e390 0x5605aaa06cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.788+0000 7f0e84921740 1 --2- 172.21.15.64:0/2839086211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605aa6361b0 0x5605aaa067b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.788+0000 7f0e84921740 1 --2- 172.21.15.64:0/2839086211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605aa65f070 0x5605aaa06270 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.788+0000 7f0e84921740 1 -- 172.21.15.64:0/2839086211 >> 172.21.15.64:0/2839086211 conn(0x5605aa653460 msgr2=0x5605aaa04160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:58.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.792+0000 7f0e777fe640 1 --2- 172.21.15.64:0/518794581 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0e600b6670 0x7f0e600b8b80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0e68013430 tx=0x7f0e6800bf90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:58.798 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.796+0000 7f0e84921740 1 -- 172.21.15.64:0/2839086211 shutdown_connections 2024-09-21T04:06:58.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.796+0000 7f0e84921740 1 -- 172.21.15.64:0/2839086211 wait complete. 2024-09-21T04:06:58.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.796+0000 7f0e84921740 1 -- start start 2024-09-21T04:06:58.799 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.797+0000 7f0e84921740 1 Processor -- start 2024-09-21T04:06:58.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.797+0000 7f0e84921740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605aa6361b0 0x5605aaa228d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:58.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.797+0000 7f0e84921740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605aa63e390 0x5605aaa22e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:58.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.797+0000 7f0e84921740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605aa65f070 0x5605aaa23350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:58.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.797+0000 7f0e84921740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5605aa7cd700 con 0x5605aa65f070 2024-09-21T04:06:58.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.797+0000 7f0e84921740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5605aa907110 con 0x5605aa6361b0 2024-09-21T04:06:58.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.797+0000 7f0e84921740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5605aaa259a0 con 0x5605aa63e390 2024-09-21T04:06:58.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.797+0000 7f0e84921740 10 client.0 _ll_register_callbacks cb 0x5605aa920990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:06:58.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.798+0000 7f0e777fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605aa63e390 0x5605aaa22e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:58.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.798+0000 7f0e777fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605aa63e390 0x5605aaa22e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:47932/0 (socket says 172.21.15.64:47932) 2024-09-21T04:06:58.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.798+0000 7f0e777fe640 1 -- 172.21.15.64:0/4011609529 learned_addr learned my addr 172.21.15.64:0/4011609529 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:58.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.798+0000 7f0e7c94a640 1 --2- 172.21.15.64:0/4011609529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605aa65f070 0x5605aaa23350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:58.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.798+0000 7f0e777fe640 1 -- 172.21.15.64:0/4011609529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605aa6361b0 msgr2=0x5605aaa228d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:06:58.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.798+0000 7f0e777fe640 1 --2- 172.21.15.64:0/4011609529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605aa6361b0 0x5605aaa228d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.798+0000 7f0e777fe640 1 -- 172.21.15.64:0/4011609529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605aa65f070 msgr2=0x5605aaa23350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:58.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.799+0000 7f0e777fe640 1 --2- 172.21.15.64:0/4011609529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605aa65f070 0x5605aaa23350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.799+0000 7f0e777fe640 1 -- 172.21.15.64:0/4011609529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605aa934c40 con 0x5605aa63e390 2024-09-21T04:06:58.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.799+0000 7f0e7c94a640 1 --2- 172.21.15.64:0/4011609529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605aa65f070 0x5605aaa23350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:06:58.802 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.799+0000 7f0e777fe640 1 --2- 172.21.15.64:0/4011609529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605aa63e390 0x5605aaa22e10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800ebe0 tx=0x7f0e680043c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bca224567f79932f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:58.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.800+0000 7f0e577fe640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-21T04:06:58.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.801+0000 7f0e577fe640 1 -- 172.21.15.64:0/4011609529 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6800ec10 con 0x5605aa63e390 2024-09-21T04:06:58.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.801+0000 7f0e84921740 1 -- 172.21.15.64:0/4011609529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e6800ee10 con 0x5605aa63e390 2024-09-21T04:06:58.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.801+0000 7f0e577fe640 1 -- 172.21.15.64:0/4011609529 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f0e68003f30 con 0x5605aa63e390 2024-09-21T04:06:58.804 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.802+0000 7f0e577fe640 1 -- 172.21.15.64:0/4011609529 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0e68004130 con 0x5605aa63e390 2024-09-21T04:06:58.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.803+0000 7f0e84921740 10 client.25762 fetch_fsmap learned FSMap version 30 2024-09-21T04:06:58.805 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.803+0000 7f0e84921740 1 -- 172.21.15.64:0/4011609529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5605aaa31660 con 0x5605aa63e390 2024-09-21T04:06:58.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.806+0000 7f0e577fe640 1 -- 172.21.15.64:0/4011609529 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e6800ee10 con 0x5605aa63e390 2024-09-21T04:06:58.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.806+0000 7f0e577fe640 1 -- 172.21.15.64:0/4011609529 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f0e68034020 con 0x5605aa63e390 2024-09-21T04:06:58.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.807+0000 7f0e84921740 10 client.25762 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:06:58.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.807+0000 7f0e84921740 10 client.25762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:06:58.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.807+0000 7f0e84921740 20 client.25762 populate_metadata read hostname 'smithi064' 2024-09-21T04:06:58.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.807+0000 7f0e84921740 1 --2- 172.21.15.64:0/4011609529 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5605aa61d9f0 0x5605aaa34a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:58.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.807+0000 7f0e84921740 4 client.25762 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:06:58.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.807+0000 7f0e84921740 1 -- 172.21.15.64:0/4011609529 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605aa97eda0 con 0x5605aa61d9f0 2024-09-21T04:06:58.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.807+0000 7f0e77fff640 1 --2- 172.21.15.64:0/4011609529 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5605aa61d9f0 0x5605aaa34a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:58.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.809+0000 7f0e77fff640 1 --2- 172.21.15.64:0/4011609529 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5605aa61d9f0 0x5605aaa34a70 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f81c60bcbdb87e45 server_cookie=ed1536776304aa5e in_seq=0 out_seq=0 2024-09-21T04:06:58.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.812+0000 7f0e577fe640 10 client.25762 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:58.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.813+0000 7f0e577fe640 1 -- 172.21.15.64:0/4011609529 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5605aa97eda0 con 0x5605aa61d9f0 2024-09-21T04:06:58.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.813+0000 7f0e577fe640 10 client.25762 handle_command_reply: tid=0 2024-09-21T04:06:58.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.816+0000 7f0e84921740 1 client.25762 shutdown 2024-09-21T04:06:58.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.822+0000 7f0e84921740 1 -- 172.21.15.64:0/4011609529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605aa63e390 msgr2=0x5605aaa22e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:58.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.822+0000 7f0e84921740 1 --2- 172.21.15.64:0/4011609529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605aa63e390 0x5605aaa22e10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800ebe0 tx=0x7f0e680043c0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.829+0000 7f0e84921740 1 -- 172.21.15.64:0/4011609529 shutdown_connections 2024-09-21T04:06:58.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.829+0000 7f0e84921740 1 --2- 172.21.15.64:0/4011609529 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5605aa61d9f0 0x5605aaa34a70 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.829+0000 7f0e84921740 1 --2- 172.21.15.64:0/4011609529 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605aa65f070 0x5605aaa23350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.829+0000 7f0e84921740 1 --2- 172.21.15.64:0/4011609529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605aa63e390 0x5605aaa22e10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.829+0000 7f0e84921740 1 --2- 172.21.15.64:0/4011609529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605aa6361b0 0x5605aaa228d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.829+0000 7f0e84921740 1 -- 172.21.15.64:0/4011609529 >> 172.21.15.64:0/4011609529 conn(0x5605aa653460 msgr2=0x5605aaa02700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:58.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.831+0000 7f0e577fe640 0 client.25762 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:58.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.832+0000 7f0e84921740 1 -- 172.21.15.64:0/4011609529 shutdown_connections 2024-09-21T04:06:58.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.837+0000 7f0e84921740 1 -- 172.21.15.64:0/4011609529 wait complete. 2024-09-21T04:06:58.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.838+0000 7f0e84921740 20 client.25762 trim_cache size 0 max 16384 2024-09-21T04:06:58.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.888+0000 7f0e84921740 1 Processor -- start 2024-09-21T04:06:58.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.889+0000 7f0e84921740 1 -- start start 2024-09-21T04:06:58.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.889+0000 7f0e84921740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605aa6361b0 0x5605aaa07c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:58.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.889+0000 7f0e84921740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605aa63e390 0x5605aaa24500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:58.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.890+0000 7f0e84921740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605aa65f070 0x5605aaa1fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:58.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.890+0000 7f0e84921740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0e781a6f40 con 0x5605aa6361b0 2024-09-21T04:06:58.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.890+0000 7f0e84921740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0e781a7670 con 0x5605aa65f070 2024-09-21T04:06:58.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.890+0000 7f0e7c94a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605aa65f070 0x5605aaa1fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:58.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.890+0000 7f0e7c94a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605aa65f070 0x5605aaa1fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:48120/0 (socket says 172.21.15.64:48120) 2024-09-21T04:06:58.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.890+0000 7f0e7c94a640 1 -- 172.21.15.64:0/840682381 learned_addr learned my addr 172.21.15.64:0/840682381 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:58.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.890+0000 7f0e84921740 1 -- 172.21.15.64:0/840682381 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0e781a4eb0 con 0x5605aa63e390 2024-09-21T04:06:58.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.891+0000 7f0e777fe640 1 --2- 172.21.15.64:0/840682381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605aa63e390 0x5605aaa24500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:58.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.891+0000 7f0e7c94a640 1 -- 172.21.15.64:0/840682381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605aa63e390 msgr2=0x5605aaa24500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:58.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.891+0000 7f0e7c94a640 1 --2- 172.21.15.64:0/840682381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605aa63e390 0x5605aaa24500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.891+0000 7f0e7c94a640 1 -- 172.21.15.64:0/840682381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605aa6361b0 msgr2=0x5605aaa07c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:06:58.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.891+0000 7f0e7c94a640 1 --2- 172.21.15.64:0/840682381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605aa6361b0 0x5605aaa07c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.891+0000 7f0e7c94a640 1 -- 172.21.15.64:0/840682381 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605aaa20dd0 con 0x5605aa65f070 2024-09-21T04:06:58.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.891+0000 7f0e7c94a640 1 --2- 172.21.15.64:0/840682381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605aa65f070 0x5605aaa1fac0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c0c9e20 tx=0x7f0e6c0cd1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b61ba628ca0f7c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:58.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.892+0000 7f0e5dffb640 1 -- 172.21.15.64:0/840682381 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c032870 con 0x5605aa65f070 2024-09-21T04:06:58.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.892+0000 7f0e5dffb640 1 -- 172.21.15.64:0/840682381 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e6c0cdf50 con 0x5605aa65f070 2024-09-21T04:06:58.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.893+0000 7f0e84921740 1 -- 172.21.15.64:0/840682381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605aa65f070 msgr2=0x5605aaa1fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:58.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.893+0000 7f0e84921740 1 --2- 172.21.15.64:0/840682381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605aa65f070 0x5605aaa1fac0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c0c9e20 tx=0x7f0e6c0cd1a0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.894+0000 7f0e84921740 1 -- 172.21.15.64:0/840682381 shutdown_connections 2024-09-21T04:06:58.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.894+0000 7f0e84921740 1 --2- 172.21.15.64:0/840682381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605aa65f070 0x5605aaa1fac0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.894+0000 7f0e84921740 1 --2- 172.21.15.64:0/840682381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605aa63e390 0x5605aaa24500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.894+0000 7f0e84921740 1 --2- 172.21.15.64:0/840682381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605aa6361b0 0x5605aaa07c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.894+0000 7f0e84921740 1 -- 172.21.15.64:0/840682381 >> 172.21.15.64:0/840682381 conn(0x5605aa653460 msgr2=0x5605aaa23df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:58.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.895+0000 7f0e84921740 1 -- 172.21.15.64:0/840682381 shutdown_connections 2024-09-21T04:06:58.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.895+0000 7f0e84921740 1 -- 172.21.15.64:0/840682381 wait complete. 2024-09-21T04:06:58.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.896+0000 7f0e84921740 1 -- start start 2024-09-21T04:06:58.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.897+0000 7f0e84921740 1 Processor -- start 2024-09-21T04:06:58.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.897+0000 7f0e84921740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605aa6361b0 0x5605aaa2b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:58.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.897+0000 7f0e84921740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605aa63e390 0x5605aaa287a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:58.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.897+0000 7f0e84921740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605aa65f070 0x5605aaa28ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:58.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.897+0000 7f0e84921740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0e78073b50 con 0x5605aa6361b0 2024-09-21T04:06:58.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.897+0000 7f0e84921740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0e78074fc0 con 0x5605aa65f070 2024-09-21T04:06:58.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.897+0000 7f0e84921740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0e78074220 con 0x5605aa63e390 2024-09-21T04:06:58.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.897+0000 7f0e84921740 10 client.0 _ll_register_callbacks cb 0x7f0e6800bee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:06:58.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.898+0000 7f0e777fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605aa63e390 0x5605aaa287a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:58.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.898+0000 7f0e777fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605aa63e390 0x5605aaa287a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:47946/0 (socket says 172.21.15.64:47946) 2024-09-21T04:06:58.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.898+0000 7f0e777fe640 1 -- 172.21.15.64:0/1724728863 learned_addr learned my addr 172.21.15.64:0/1724728863 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:06:58.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.899+0000 7f0e777fe640 1 -- 172.21.15.64:0/1724728863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605aa65f070 msgr2=0x5605aaa28ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T04:06:58.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.899+0000 7f0e777fe640 1 --2- 172.21.15.64:0/1724728863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605aa65f070 0x5605aaa28ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.899+0000 7f0e777fe640 1 -- 172.21.15.64:0/1724728863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605aa6361b0 msgr2=0x5605aaa2b280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:06:58.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.899+0000 7f0e77fff640 1 --2- 172.21.15.64:0/1724728863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605aa6361b0 0x5605aaa2b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:58.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.899+0000 7f0e777fe640 1 --2- 172.21.15.64:0/1724728863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605aa6361b0 0x5605aaa2b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.899+0000 7f0e777fe640 1 -- 172.21.15.64:0/1724728863 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605aaa29250 con 0x5605aa63e390 2024-09-21T04:06:58.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.899+0000 7f0e77fff640 1 --2- 172.21.15.64:0/1724728863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605aa6361b0 0x5605aaa2b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:06:58.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.899+0000 7f0e777fe640 1 --2- 172.21.15.64:0/1724728863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605aa63e390 0x5605aaa287a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0e68013040 tx=0x7f0e6800baa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1120cd1a3c702c62 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:06:58.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.899+0000 7f0e577fe640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-21T04:06:58.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.899+0000 7f0e577fe640 1 -- 172.21.15.64:0/1724728863 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68042020 con 0x5605aa63e390 2024-09-21T04:06:58.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.900+0000 7f0e577fe640 1 -- 172.21.15.64:0/1724728863 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f0e68011d70 con 0x5605aa63e390 2024-09-21T04:06:58.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.900+0000 7f0e84921740 1 -- 172.21.15.64:0/1724728863 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e6c0cdf50 con 0x5605aa63e390 2024-09-21T04:06:58.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.900+0000 7f0e577fe640 1 -- 172.21.15.64:0/1724728863 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0e68011900 con 0x5605aa63e390 2024-09-21T04:06:58.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.902+0000 7f0e84921740 10 client.25765 fetch_fsmap learned FSMap version 30 2024-09-21T04:06:58.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.902+0000 7f0e84921740 1 -- 172.21.15.64:0/1724728863 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5605aaab00c0 con 0x5605aa63e390 2024-09-21T04:06:58.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.906+0000 7f0e577fe640 1 -- 172.21.15.64:0/1724728863 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e6c0cdf50 con 0x5605aa63e390 2024-09-21T04:06:58.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.907+0000 7f0e577fe640 1 -- 172.21.15.64:0/1724728863 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f0e68034070 con 0x5605aa63e390 2024-09-21T04:06:58.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.908+0000 7f0e84921740 10 client.25765 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:06:58.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.908+0000 7f0e84921740 10 client.25765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:06:58.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.908+0000 7f0e84921740 20 client.25765 populate_metadata read hostname 'smithi064' 2024-09-21T04:06:58.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.908+0000 7f0e84921740 1 --2- 172.21.15.64:0/1724728863 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5605aa5f2da0 0x5605aaab2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:06:58.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.908+0000 7f0e84921740 4 client.25765 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T04:06:58.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.908+0000 7f0e84921740 1 -- 172.21.15.64:0/1724728863 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5605aa97a850 con 0x5605aa5f2da0 2024-09-21T04:06:58.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.909+0000 7f0e77fff640 1 --2- 172.21.15.64:0/1724728863 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5605aa5f2da0 0x5605aaab2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:06:58.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.911+0000 7f0e577fe640 10 client.25765 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:58.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.911+0000 7f0e77fff640 1 --2- 172.21.15.64:0/1724728863 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5605aa5f2da0 0x5605aaab2940 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f79383078cb486c1 server_cookie=eef5616a23cb2de3 in_seq=0 out_seq=0 2024-09-21T04:06:58.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.914+0000 7f0e577fe640 1 -- 172.21.15.64:0/1724728863 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1510 (crc 0 0 0) 0x5605aa97a850 con 0x5605aa5f2da0 2024-09-21T04:06:58.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.914+0000 7f0e577fe640 10 client.25765 handle_command_reply: tid=0 2024-09-21T04:06:58.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.914+0000 7f0e84921740 1 client.25765 shutdown 2024-09-21T04:06:58.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.914+0000 7f0e84921740 1 -- 172.21.15.64:0/1724728863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605aa63e390 msgr2=0x5605aaa287a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:58.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.914+0000 7f0e84921740 1 --2- 172.21.15.64:0/1724728863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605aa63e390 0x5605aaa287a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0e68013040 tx=0x7f0e6800baa0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.914+0000 7f0e84921740 1 -- 172.21.15.64:0/1724728863 shutdown_connections 2024-09-21T04:06:58.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.914+0000 7f0e84921740 1 --2- 172.21.15.64:0/1724728863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605aa65f070 0x5605aaa28ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.914+0000 7f0e84921740 1 --2- 172.21.15.64:0/1724728863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5605aa63e390 0x5605aaa287a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.914+0000 7f0e84921740 1 --2- 172.21.15.64:0/1724728863 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5605aa5f2da0 0x5605aaab2940 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.914+0000 7f0e84921740 1 --2- 172.21.15.64:0/1724728863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5605aa6361b0 0x5605aaa2b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.914+0000 7f0e84921740 1 -- 172.21.15.64:0/1724728863 >> 172.21.15.64:0/1724728863 conn(0x5605aa653460 msgr2=0x5605aaa2e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:58.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.915+0000 7f0e577fe640 0 client.25765 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:06:58.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.915+0000 7f0e84921740 1 -- 172.21.15.64:0/1724728863 shutdown_connections 2024-09-21T04:06:58.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.917+0000 7f0e84921740 1 -- 172.21.15.64:0/1724728863 wait complete. 2024-09-21T04:06:58.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.918+0000 7f0e84921740 20 client.25765 trim_cache size 0 max 16384 2024-09-21T04:06:58.921 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:06:58.921 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-21T04:06:58.921 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {}, 2024-09-21T04:06:58.921 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_stats": { 2024-09-21T04:06:58.921 INFO:teuthology.orchestra.run.smithi064.stdout: "e9242f88-6599-40eb-adf1-bdb3e946a1e0": { 2024-09-21T04:06:58.921 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:05:39.744646+0000", 2024-09-21T04:06:58.921 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:06:58.921 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:06:58.921 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:06:58.921 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:06:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:06:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:06:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: "6bde0fe5-f775-44b1-ad0c-f9e596574dc0": { 2024-09-21T04:06:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:06:23.835536+0000", 2024-09-21T04:06:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:06:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:06:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:06:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:06:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:06:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:06:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: "b52b950c-0f97-49ec-a37c-1a8e04348ce8": { 2024-09-21T04:06:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:54.003800+0000", 2024-09-21T04:06:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:06:58.922 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:06:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:06:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:06:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:06:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:06:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: "0ab717a7-36c6-4dca-94e7-af0bdf2aa112": { 2024-09-21T04:06:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:42.691086+0000", 2024-09-21T04:06:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:06:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:06:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:06:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:06:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:06:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:06:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:06:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:03:59.570174+0000", 2024-09-21T04:06:58.923 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:06:58.924 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:06:58.924 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:06:58.924 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:06:58.924 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:06:58.924 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:06:58.924 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:06:58.924 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:06:58.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.921+0000 7f0e7e3ad640 1 -- 172.21.15.64:0/518794581 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0e600b6670 msgr2=0x7f0e600b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:58.924 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.921+0000 7f0e7e3ad640 1 --2- 172.21.15.64:0/518794581 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0e600b6670 0x7f0e600b8b80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0e68013430 tx=0x7f0e6800bf90 comp rx=0 tx=0).stop 2024-09-21T04:06:58.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.921+0000 7f0e7e3ad640 1 -- 172.21.15.64:0/518794581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e7819d660 msgr2=0x7f0e7819da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:06:58.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.921+0000 7f0e7e3ad640 1 --2- 172.21.15.64:0/518794581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e7819d660 0x7f0e7819da70 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c035040 tx=0x7f0e6c0043b0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.922+0000 7f0e7e3ad640 1 -- 172.21.15.64:0/518794581 shutdown_connections 2024-09-21T04:06:58.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.922+0000 7f0e7e3ad640 1 --2- 172.21.15.64:0/518794581 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0e600b6670 0x7f0e600b8b80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.922+0000 7f0e7e3ad640 1 --2- 172.21.15.64:0/518794581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e7819d660 0x7f0e7819da70 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.922+0000 7f0e7e3ad640 1 --2- 172.21.15.64:0/518794581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0e781076e0 0x7f0e78198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.922+0000 7f0e7e3ad640 1 --2- 172.21.15.64:0/518794581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e78101da0 0x7f0e78197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:06:58.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.922+0000 7f0e7e3ad640 1 -- 172.21.15.64:0/518794581 >> 172.21.15.64:0/518794581 conn(0x7f0e78077800 msgr2=0x7f0e781008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:06:58.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.922+0000 7f0e7e3ad640 1 -- 172.21.15.64:0/518794581 shutdown_connections 2024-09-21T04:06:58.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:06:58.923+0000 7f0e7e3ad640 1 -- 172.21.15.64:0/518794581 wait complete. 2024-09-21T04:06:59.852 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T04:06:59.852 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:07:00.306 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:00 smithi064 ceph-mon[31180]: pgmap v410: 97 pgs: 97 active+clean; 315 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 99 op/s 2024-09-21T04:07:00.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:00 smithi131 ceph-mon[35193]: pgmap v410: 97 pgs: 97 active+clean; 315 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 99 op/s 2024-09-21T04:07:00.566 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:07:00.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:00 smithi119 ceph-mon[34811]: pgmap v410: 97 pgs: 97 active+clean; 315 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 99 op/s 2024-09-21T04:07:01.173 INFO:tasks.workunit.client.0.smithi064.stdout: 66 1179104 442 802439 703 607469 1689 2024-09-21T04:07:01.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.203+0000 7fda31ff1640 1 -- 172.21.15.64:0/1882821586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda2c111f40 msgr2=0x7fda2c10a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:01.208 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.205+0000 7fda31ff1640 1 --2- 172.21.15.64:0/1882821586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda2c111f40 0x7fda2c10a3c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fda24009ca0 tx=0x7fda2402a8e0 comp rx=0 tx=0).stop 2024-09-21T04:07:01.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.207+0000 7fda31ff1640 1 -- 172.21.15.64:0/1882821586 shutdown_connections 2024-09-21T04:07:01.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.207+0000 7fda31ff1640 1 --2- 172.21.15.64:0/1882821586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda2c11cc30 0x7fda2c11f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:01.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.207+0000 7fda31ff1640 1 --2- 172.21.15.64:0/1882821586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda2c10a9e0 0x7fda2c10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:01.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.207+0000 7fda31ff1640 1 --2- 172.21.15.64:0/1882821586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda2c111f40 0x7fda2c10a3c0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:01.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.207+0000 7fda31ff1640 1 -- 172.21.15.64:0/1882821586 >> 172.21.15.64:0/1882821586 conn(0x7fda2c070ec0 msgr2=0x7fda2c0712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:01.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.209+0000 7fda31ff1640 1 -- 172.21.15.64:0/1882821586 shutdown_connections 2024-09-21T04:07:01.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.210+0000 7fda31ff1640 1 -- 172.21.15.64:0/1882821586 wait complete. 2024-09-21T04:07:01.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.211+0000 7fda31ff1640 1 Processor -- start 2024-09-21T04:07:01.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.215+0000 7fda31ff1640 1 -- start start 2024-09-21T04:07:01.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.227+0000 7fda31ff1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda2c10a9e0 0x7fda2c1a9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:01.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.227+0000 7fda31ff1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda2c111f40 0x7fda2c1a9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:01.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.228+0000 7fda31ff1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda2c11cc30 0x7fda2c1aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:01.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.228+0000 7fda31ff1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c122980 con 0x7fda2c10a9e0 2024-09-21T04:07:01.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.228+0000 7fda31ff1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c122800 con 0x7fda2c11cc30 2024-09-21T04:07:01.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.228+0000 7fda31ff1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c122b00 con 0x7fda2c111f40 2024-09-21T04:07:01.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.228+0000 7fda2bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda2c11cc30 0x7fda2c1aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:01.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.229+0000 7fda2bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda2c11cc30 0x7fda2c1aebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:48150/0 (socket says 172.21.15.64:48150) 2024-09-21T04:07:01.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.229+0000 7fda2bfff640 1 -- 172.21.15.64:0/3894751339 learned_addr learned my addr 172.21.15.64:0/3894751339 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:01.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.230+0000 7fda2b7fe640 1 --2- 172.21.15.64:0/3894751339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda2c10a9e0 0x7fda2c1a9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:01.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.231+0000 7fda2b7fe640 1 -- 172.21.15.64:0/3894751339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda2c111f40 msgr2=0x7fda2c1a9910 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T04:07:01.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.231+0000 7fda2b7fe640 1 --2- 172.21.15.64:0/3894751339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda2c111f40 0x7fda2c1a9910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:01.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.231+0000 7fda2b7fe640 1 -- 172.21.15.64:0/3894751339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda2c11cc30 msgr2=0x7fda2c1aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:01.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.231+0000 7fda2b7fe640 1 --2- 172.21.15.64:0/3894751339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda2c11cc30 0x7fda2c1aebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:01.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.231+0000 7fda2b7fe640 1 -- 172.21.15.64:0/3894751339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2c1af2e0 con 0x7fda2c10a9e0 2024-09-21T04:07:01.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.232+0000 7fda2b7fe640 1 --2- 172.21.15.64:0/3894751339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda2c10a9e0 0x7fda2c1a9370 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fda24009ca0 tx=0x7fda24003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:01.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.233+0000 7fda28ff9640 1 -- 172.21.15.64:0/3894751339 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda24003da0 con 0x7fda2c10a9e0 2024-09-21T04:07:01.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.233+0000 7fda28ff9640 1 -- 172.21.15.64:0/3894751339 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda240044f0 con 0x7fda2c10a9e0 2024-09-21T04:07:01.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.233+0000 7fda28ff9640 1 -- 172.21.15.64:0/3894751339 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda24030d00 con 0x7fda2c10a9e0 2024-09-21T04:07:01.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.233+0000 7fda31ff1640 1 -- 172.21.15.64:0/3894751339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda2c1af570 con 0x7fda2c10a9e0 2024-09-21T04:07:01.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.233+0000 7fda31ff1640 1 -- 172.21.15.64:0/3894751339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda2c1b5d30 con 0x7fda2c10a9e0 2024-09-21T04:07:01.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.239+0000 7fda127fc640 1 -- 172.21.15.64:0/3894751339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda18005290 con 0x7fda2c10a9e0 2024-09-21T04:07:01.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.255+0000 7fda28ff9640 1 -- 172.21.15.64:0/3894751339 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fda24004040 con 0x7fda2c10a9e0 2024-09-21T04:07:01.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.257+0000 7fda28ff9640 1 --2- 172.21.15.64:0/3894751339 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fda140b6880 0x7fda140b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:01.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.257+0000 7fda28ff9640 1 -- 172.21.15.64:0/3894751339 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fda24108b80 con 0x7fda2c10a9e0 2024-09-21T04:07:01.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.261+0000 7fda2affd640 1 --2- 172.21.15.64:0/3894751339 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fda140b6880 0x7fda140b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:01.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.270+0000 7fda2affd640 1 --2- 172.21.15.64:0/3894751339 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fda140b6880 0x7fda140b8d90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fda2c071530 tx=0x7fda1c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:01.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.270+0000 7fda28ff9640 1 -- 172.21.15.64:0/3894751339 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fda240cf040 con 0x7fda2c10a9e0 2024-09-21T04:07:01.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.750+0000 7fda127fc640 1 -- 172.21.15.64:0/3894751339 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda18005850 con 0x7fda2c10a9e0 2024-09-21T04:07:01.763 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:07:01.763 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":30,"btime":"2024-09-21T04:05:41:799101+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14805,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/97995119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":97995119},{"type":"v1","addr":"172.21.15.64:6839","nonce":97995119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/390891068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":390891068},{"type":"v1","addr":"172.21.15.64:6841","nonce":390891068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:02:03.354744+0000","modified":"2024-09-21T04:05:41.678788+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24710,"mds_1":24680,"mds_2":24661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24634":{"gid":24634,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6839/782190424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782190424},{"type":"v1","addr":"172.21.15.131:6839","nonce":782190424}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24680":{"gid":24680,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24710":{"gid":24710,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6839/1319793636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1319793636},{"type":"v1","addr":"172.21.15.119:6839","nonce":1319793636}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24710,"qdb_cluster":[24661,24680,24710]},"id":1}]} 2024-09-21T04:07:01.763 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.756+0000 7fda28ff9640 1 -- 172.21.15.64:0/3894751339 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11918 (secure 0 0 0) 0x7fda240046a0 con 0x7fda2c10a9e0 2024-09-21T04:07:01.763 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 30 2024-09-21T04:07:01.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.767+0000 7fda127fc640 1 -- 172.21.15.64:0/3894751339 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fda140b6880 msgr2=0x7fda140b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:01.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.767+0000 7fda127fc640 1 --2- 172.21.15.64:0/3894751339 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fda140b6880 0x7fda140b8d90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fda2c071530 tx=0x7fda1c005f90 comp rx=0 tx=0).stop 2024-09-21T04:07:01.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.767+0000 7fda127fc640 1 -- 172.21.15.64:0/3894751339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda2c10a9e0 msgr2=0x7fda2c1a9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:01.769 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.767+0000 7fda127fc640 1 --2- 172.21.15.64:0/3894751339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda2c10a9e0 0x7fda2c1a9370 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fda24009ca0 tx=0x7fda24003970 comp rx=0 tx=0).stop 2024-09-21T04:07:01.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.770+0000 7fda127fc640 1 -- 172.21.15.64:0/3894751339 shutdown_connections 2024-09-21T04:07:01.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.770+0000 7fda127fc640 1 --2- 172.21.15.64:0/3894751339 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fda140b6880 0x7fda140b8d90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:01.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.770+0000 7fda127fc640 1 --2- 172.21.15.64:0/3894751339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda2c11cc30 0x7fda2c1aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:01.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.770+0000 7fda127fc640 1 --2- 172.21.15.64:0/3894751339 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda2c111f40 0x7fda2c1a9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:01.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.770+0000 7fda127fc640 1 --2- 172.21.15.64:0/3894751339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda2c10a9e0 0x7fda2c1a9370 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:01.773 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.770+0000 7fda127fc640 1 -- 172.21.15.64:0/3894751339 >> 172.21.15.64:0/3894751339 conn(0x7fda2c070ec0 msgr2=0x7fda2c11d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:01.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.778+0000 7fda127fc640 1 -- 172.21.15.64:0/3894751339 shutdown_connections 2024-09-21T04:07:01.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:01.780+0000 7fda127fc640 1 -- 172.21.15.64:0/3894751339 wait complete. 2024-09-21T04:07:02.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:02 smithi119 ceph-mon[34811]: pgmap v411: 97 pgs: 97 active+clean; 318 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 100 op/s 2024-09-21T04:07:02.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:02 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3894751339' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:07:02.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:02 smithi064 ceph-mon[31180]: pgmap v411: 97 pgs: 97 active+clean; 318 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 100 op/s 2024-09-21T04:07:02.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:02 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3894751339' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:07:02.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:02 smithi131 ceph-mon[35193]: pgmap v411: 97 pgs: 97 active+clean; 318 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 100 op/s 2024-09-21T04:07:02.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:02 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3894751339' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:07:03.131 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T04:07:03.390 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:07:04.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.148+0000 7f37905c3640 1 -- 172.21.15.64:0/473026146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37881020a0 msgr2=0x7f37881024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:04.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.148+0000 7f37905c3640 1 --2- 172.21.15.64:0/473026146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37881020a0 0x7f37881024b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3784009900 tx=0x7f378402a410 comp rx=0 tx=0).stop 2024-09-21T04:07:04.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.148+0000 7f37905c3640 1 -- 172.21.15.64:0/473026146 shutdown_connections 2024-09-21T04:07:04.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.148+0000 7f37905c3640 1 --2- 172.21.15.64:0/473026146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3788109e40 0x7f3788110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.148+0000 7f37905c3640 1 --2- 172.21.15.64:0/473026146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37881094f0 0x7f3788109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.148+0000 7f37905c3640 1 --2- 172.21.15.64:0/473026146 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37881020a0 0x7f37881024b0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.148+0000 7f37905c3640 1 -- 172.21.15.64:0/473026146 >> 172.21.15.64:0/473026146 conn(0x7f3788079e40 msgr2=0x7f378807a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:04.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.148+0000 7f37905c3640 1 -- 172.21.15.64:0/473026146 shutdown_connections 2024-09-21T04:07:04.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.148+0000 7f37905c3640 1 -- 172.21.15.64:0/473026146 wait complete. 2024-09-21T04:07:04.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.149+0000 7f37905c3640 1 Processor -- start 2024-09-21T04:07:04.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.149+0000 7f37905c3640 1 -- start start 2024-09-21T04:07:04.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.149+0000 7f37905c3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37881020a0 0x7f378819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:04.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.149+0000 7f37905c3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37881094f0 0x7f378819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:04.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.149+0000 7f37905c3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3788109e40 0x7f378819fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:04.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.149+0000 7f37905c3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3788113b00 con 0x7f37881094f0 2024-09-21T04:07:04.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.149+0000 7f37905c3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3788113980 con 0x7f3788109e40 2024-09-21T04:07:04.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.149+0000 7f37905c3640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3788113c80 con 0x7f37881020a0 2024-09-21T04:07:04.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.150+0000 7f378db5e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37881094f0 0x7f378819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:04.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.150+0000 7f378e35f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37881020a0 0x7f378819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:04.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.150+0000 7f378eb60640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3788109e40 0x7f378819fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:04.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.150+0000 7f378db5e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37881094f0 0x7f378819ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58296/0 (socket says 172.21.15.64:58296) 2024-09-21T04:07:04.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.150+0000 7f378e35f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37881020a0 0x7f378819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:54584/0 (socket says 172.21.15.64:54584) 2024-09-21T04:07:04.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.150+0000 7f378db5e640 1 -- 172.21.15.64:0/1903545562 learned_addr learned my addr 172.21.15.64:0/1903545562 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:04.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.150+0000 7f378eb60640 1 -- 172.21.15.64:0/1903545562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37881020a0 msgr2=0x7f378819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:04.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.150+0000 7f378eb60640 1 --2- 172.21.15.64:0/1903545562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37881020a0 0x7f378819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.150+0000 7f378eb60640 1 -- 172.21.15.64:0/1903545562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37881094f0 msgr2=0x7f378819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:04.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.150+0000 7f378eb60640 1 --2- 172.21.15.64:0/1903545562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37881094f0 0x7f378819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.150+0000 7f378eb60640 1 -- 172.21.15.64:0/1903545562 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37881a0560 con 0x7f3788109e40 2024-09-21T04:07:04.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.150+0000 7f378db5e640 1 --2- 172.21.15.64:0/1903545562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37881094f0 0x7f378819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:07:04.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.150+0000 7f378eb60640 1 --2- 172.21.15.64:0/1903545562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3788109e40 0x7f378819fe30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f378000e940 tx=0x7f378000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:04.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.150+0000 7f377f7fe640 1 -- 172.21.15.64:0/1903545562 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3780004d20 con 0x7f3788109e40 2024-09-21T04:07:04.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.150+0000 7f37905c3640 1 -- 172.21.15.64:0/1903545562 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37881a0850 con 0x7f3788109e40 2024-09-21T04:07:04.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.150+0000 7f377f7fe640 1 -- 172.21.15.64:0/1903545562 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3780005020 con 0x7f3788109e40 2024-09-21T04:07:04.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.150+0000 7f377f7fe640 1 -- 172.21.15.64:0/1903545562 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37800116d0 con 0x7f3788109e40 2024-09-21T04:07:04.155 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.151+0000 7f37905c3640 1 -- 172.21.15.64:0/1903545562 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3788102f20 con 0x7f3788109e40 2024-09-21T04:07:04.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.153+0000 7f3796b37740 1 Processor -- start 2024-09-21T04:07:04.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.153+0000 7f377f7fe640 1 -- 172.21.15.64:0/1903545562 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f3780005460 con 0x7f3788109e40 2024-09-21T04:07:04.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.153+0000 7f3796b37740 1 -- start start 2024-09-21T04:07:04.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.154+0000 7f3796b37740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa94070 0x562a4ae3b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.154+0000 7f3796b37740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a4aa6b1b0 0x562a4ae3bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.154+0000 7f3796b37740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a4aa73390 0x562a4ae3c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.154+0000 7f3796b37740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562a4adb1ea0 con 0x562a4aa94070 2024-09-21T04:07:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.154+0000 7f3796b37740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562a4ada12d0 con 0x562a4aa6b1b0 2024-09-21T04:07:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.154+0000 7f3796b37740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562a4ada50e0 con 0x562a4aa73390 2024-09-21T04:07:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.154+0000 7f378db5e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa94070 0x562a4ae3b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.154+0000 7f378db5e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa94070 0x562a4ae3b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58298/0 (socket says 172.21.15.64:58298) 2024-09-21T04:07:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.154+0000 7f378db5e640 1 -- 172.21.15.64:0/3064360600 learned_addr learned my addr 172.21.15.64:0/3064360600 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.154+0000 7f378e35f640 1 --2- 172.21.15.64:0/3064360600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a4aa6b1b0 0x562a4ae3bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.154+0000 7f378db5e640 1 --2- 172.21.15.64:0/3064360600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a4aa73390 0x562a4ae3c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.154+0000 7f378db5e640 1 -- 172.21.15.64:0/3064360600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a4aa73390 msgr2=0x562a4ae3c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.154+0000 7f378db5e640 1 --2- 172.21.15.64:0/3064360600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a4aa73390 0x562a4ae3c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.154+0000 7f378db5e640 1 -- 172.21.15.64:0/3064360600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a4aa6b1b0 msgr2=0x562a4ae3bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:04.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.154+0000 7f378db5e640 1 --2- 172.21.15.64:0/3064360600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a4aa6b1b0 0x562a4ae3bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.154+0000 7f378db5e640 1 -- 172.21.15.64:0/3064360600 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a4aaff650 con 0x562a4aa94070 2024-09-21T04:07:04.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.154+0000 7f378e35f640 1 --2- 172.21.15.64:0/3064360600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a4aa6b1b0 0x562a4ae3bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:07:04.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.154+0000 7f377f7fe640 1 --2- 172.21.15.64:0/1903545562 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f37740b6680 0x7f37740b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:04.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.154+0000 7f378db5e640 1 --2- 172.21.15.64:0/3064360600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa94070 0x562a4ae3b6a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3778013a00 tx=0x7f3778009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc17c4a1c8bc2b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:04.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.155+0000 7f378e35f640 1 --2- 172.21.15.64:0/1903545562 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f37740b6680 0x7f37740b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:04.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.155+0000 7f377f7fe640 1 -- 172.21.15.64:0/1903545562 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f37800e7a80 con 0x7f3788109e40 2024-09-21T04:07:04.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.155+0000 7f377cff9640 1 -- 172.21.15.64:0/3064360600 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778004030 con 0x562a4aa94070 2024-09-21T04:07:04.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.155+0000 7f377cff9640 1 -- 172.21.15.64:0/3064360600 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f377800f980 con 0x562a4aa94070 2024-09-21T04:07:04.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.155+0000 7f377cff9640 1 -- 172.21.15.64:0/3064360600 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37780039a0 con 0x562a4aa94070 2024-09-21T04:07:04.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.156+0000 7f3796b37740 1 -- 172.21.15.64:0/3064360600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa94070 msgr2=0x562a4ae3b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:04.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.156+0000 7f378e35f640 1 --2- 172.21.15.64:0/1903545562 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f37740b6680 0x7f37740b8b90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3784031440 tx=0x7f3784009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:04.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.156+0000 7f3796b37740 1 --2- 172.21.15.64:0/3064360600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa94070 0x562a4ae3b6a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3778013a00 tx=0x7f3778009c60 comp rx=0 tx=0).stop 2024-09-21T04:07:04.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.156+0000 7f3796b37740 1 -- 172.21.15.64:0/3064360600 shutdown_connections 2024-09-21T04:07:04.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.156+0000 7f3796b37740 1 --2- 172.21.15.64:0/3064360600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a4aa73390 0x562a4ae3c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.156+0000 7f3796b37740 1 --2- 172.21.15.64:0/3064360600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a4aa6b1b0 0x562a4ae3bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.156+0000 7f3796b37740 1 --2- 172.21.15.64:0/3064360600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa94070 0x562a4ae3b6a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.156+0000 7f3796b37740 1 -- 172.21.15.64:0/3064360600 >> 172.21.15.64:0/3064360600 conn(0x562a4aa88460 msgr2=0x562a4ae39590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:04.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.156+0000 7f3796b37740 1 -- 172.21.15.64:0/3064360600 shutdown_connections 2024-09-21T04:07:04.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.156+0000 7f3796b37740 1 -- 172.21.15.64:0/3064360600 wait complete. 2024-09-21T04:07:04.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.156+0000 7f3796b37740 1 -- start start 2024-09-21T04:07:04.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.156+0000 7f3796b37740 1 Processor -- start 2024-09-21T04:07:04.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.157+0000 7f3796b37740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a4aa6b1b0 0x562a4ae46d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:04.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.157+0000 7f3796b37740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa73390 0x562a4ae47280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:04.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.157+0000 7f3796b37740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a4aa94070 0x562a4ae53170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:04.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.157+0000 7f3796b37740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562a4ac02700 con 0x562a4aa73390 2024-09-21T04:07:04.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.157+0000 7f3796b37740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562a4ad3c160 con 0x562a4aa6b1b0 2024-09-21T04:07:04.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.157+0000 7f3796b37740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562a4ae49e80 con 0x562a4aa94070 2024-09-21T04:07:04.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.157+0000 7f3796b37740 10 client.0 _ll_register_callbacks cb 0x562a4ad559e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:04.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.157+0000 7f378db5e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa73390 0x562a4ae47280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:04.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.157+0000 7f378e35f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a4aa6b1b0 0x562a4ae46d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:04.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.157+0000 7f378db5e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa73390 0x562a4ae47280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58308/0 (socket says 172.21.15.64:58308) 2024-09-21T04:07:04.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.157+0000 7f378db5e640 1 -- 172.21.15.64:0/1946244732 learned_addr learned my addr 172.21.15.64:0/1946244732 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:04.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.157+0000 7f378eb60640 1 --2- 172.21.15.64:0/1946244732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a4aa94070 0x562a4ae53170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:04.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.157+0000 7f378db5e640 1 -- 172.21.15.64:0/1946244732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a4aa94070 msgr2=0x562a4ae53170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:04.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.157+0000 7f378db5e640 1 --2- 172.21.15.64:0/1946244732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a4aa94070 0x562a4ae53170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.157+0000 7f378db5e640 1 -- 172.21.15.64:0/1946244732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a4aa6b1b0 msgr2=0x562a4ae46d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:04.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.157+0000 7f378db5e640 1 --2- 172.21.15.64:0/1946244732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a4aa6b1b0 0x562a4ae46d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.157+0000 7f378db5e640 1 -- 172.21.15.64:0/1946244732 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a4ad69c90 con 0x562a4aa73390 2024-09-21T04:07:04.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.158+0000 7f378db5e640 1 --2- 172.21.15.64:0/1946244732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa73390 0x562a4ae47280 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3778010010 tx=0x7f3778005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd7371bc169a25b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:04.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.158+0000 7f3771ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:07:04.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.158+0000 7f3771ffb640 1 -- 172.21.15.64:0/1946244732 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377802f060 con 0x562a4aa73390 2024-09-21T04:07:04.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.158+0000 7f3771ffb640 1 -- 172.21.15.64:0/1946244732 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f377801f640 con 0x562a4aa73390 2024-09-21T04:07:04.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.158+0000 7f3796b37740 1 -- 172.21.15.64:0/1946244732 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f377800f980 con 0x562a4aa73390 2024-09-21T04:07:04.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.158+0000 7f3771ffb640 1 -- 172.21.15.64:0/1946244732 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f377801f840 con 0x562a4aa73390 2024-09-21T04:07:04.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.158+0000 7f3796b37740 10 client.16179 fetch_fsmap learned FSMap version 30 2024-09-21T04:07:04.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.158+0000 7f3796b37740 1 -- 172.21.15.64:0/1946244732 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562a4ae55b50 con 0x562a4aa73390 2024-09-21T04:07:04.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.158+0000 7f3771ffb640 1 -- 172.21.15.64:0/1946244732 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f377800f980 con 0x562a4aa73390 2024-09-21T04:07:04.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.158+0000 7f3771ffb640 1 -- 172.21.15.64:0/1946244732 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f3778010a00 con 0x562a4aa73390 2024-09-21T04:07:04.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.159+0000 7f3796b37740 10 client.16179 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:07:04.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.159+0000 7f3796b37740 10 client.16179 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:07:04.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.159+0000 7f3796b37740 20 client.16179 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:04.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.159+0000 7f3796b37740 1 --2- 172.21.15.64:0/1946244732 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562a4aa529f0 0x562a4ae58f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:04.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.159+0000 7f3796b37740 4 client.16179 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:07:04.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.159+0000 7f3796b37740 1 -- 172.21.15.64:0/1946244732 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a4adb3df0 con 0x562a4aa529f0 2024-09-21T04:07:04.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.159+0000 7f378e35f640 1 --2- 172.21.15.64:0/1946244732 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562a4aa529f0 0x562a4ae58f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:04.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.159+0000 7f3771ffb640 10 client.16179 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:07:04.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.159+0000 7f378e35f640 1 --2- 172.21.15.64:0/1946244732 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562a4aa529f0 0x562a4ae58f60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9d4109c0bccd01a3 server_cookie=fe72bf0444ee5bf0 in_seq=0 out_seq=0 2024-09-21T04:07:04.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.160+0000 7f3771ffb640 1 -- 172.21.15.64:0/1946244732 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562a4adb3df0 con 0x562a4aa529f0 2024-09-21T04:07:04.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.160+0000 7f3771ffb640 10 client.16179 handle_command_reply: tid=0 2024-09-21T04:07:04.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.160+0000 7f3796b37740 1 client.16179 shutdown 2024-09-21T04:07:04.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.160+0000 7f3796b37740 1 -- 172.21.15.64:0/1946244732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa73390 msgr2=0x562a4ae47280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:04.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.160+0000 7f3796b37740 1 --2- 172.21.15.64:0/1946244732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa73390 0x562a4ae47280 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3778010010 tx=0x7f3778005d30 comp rx=0 tx=0).stop 2024-09-21T04:07:04.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.160+0000 7f3796b37740 1 -- 172.21.15.64:0/1946244732 shutdown_connections 2024-09-21T04:07:04.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.160+0000 7f3796b37740 1 --2- 172.21.15.64:0/1946244732 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562a4aa529f0 0x562a4ae58f60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.160+0000 7f3796b37740 1 --2- 172.21.15.64:0/1946244732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a4aa94070 0x562a4ae53170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.160+0000 7f3796b37740 1 --2- 172.21.15.64:0/1946244732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa73390 0x562a4ae47280 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.160+0000 7f3771ffb640 0 client.16179 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:07:04.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.160+0000 7f3796b37740 1 --2- 172.21.15.64:0/1946244732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a4aa6b1b0 0x562a4ae46d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.160+0000 7f3796b37740 1 -- 172.21.15.64:0/1946244732 >> 172.21.15.64:0/1946244732 conn(0x562a4aa88460 msgr2=0x562a4ae3e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:04.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.160+0000 7f3796b37740 1 -- 172.21.15.64:0/1946244732 shutdown_connections 2024-09-21T04:07:04.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.161+0000 7f3796b37740 1 -- 172.21.15.64:0/1946244732 wait complete. 2024-09-21T04:07:04.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.161+0000 7f3796b37740 20 client.16179 trim_cache size 0 max 16384 2024-09-21T04:07:04.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.174+0000 7f3796b37740 1 Processor -- start 2024-09-21T04:07:04.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.174+0000 7f3796b37740 1 -- start start 2024-09-21T04:07:04.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.174+0000 7f3796b37740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa6b1b0 0x562a4ae47200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:04.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.174+0000 7f3796b37740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a4aa73390 0x562a4ae38bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:04.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.174+0000 7f3796b37740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a4aa94070 0x562a4ae39100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:04.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.174+0000 7f3796b37740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f37881b44b0 con 0x562a4aa6b1b0 2024-09-21T04:07:04.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.174+0000 7f3796b37740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f37881b1cf0 con 0x562a4aa73390 2024-09-21T04:07:04.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.174+0000 7f3796b37740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f37881afb40 con 0x562a4aa94070 2024-09-21T04:07:04.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.174+0000 7f378e35f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa6b1b0 0x562a4ae47200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:04.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.174+0000 7f378e35f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa6b1b0 0x562a4ae47200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58324/0 (socket says 172.21.15.64:58324) 2024-09-21T04:07:04.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.174+0000 7f378e35f640 1 -- 172.21.15.64:0/1130500098 learned_addr learned my addr 172.21.15.64:0/1130500098 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:04.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.174+0000 7f378db5e640 1 --2- 172.21.15.64:0/1130500098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a4aa73390 0x562a4ae38bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:04.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.175+0000 7f378eb60640 1 --2- 172.21.15.64:0/1130500098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a4aa94070 0x562a4ae39100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:04.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.175+0000 7f378e35f640 1 -- 172.21.15.64:0/1130500098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a4aa94070 msgr2=0x562a4ae39100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:04.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.175+0000 7f378e35f640 1 --2- 172.21.15.64:0/1130500098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a4aa94070 0x562a4ae39100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.175+0000 7f378e35f640 1 -- 172.21.15.64:0/1130500098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a4aa73390 msgr2=0x562a4ae38bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:04.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.175+0000 7f378e35f640 1 --2- 172.21.15.64:0/1130500098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a4aa73390 0x562a4ae38bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.175+0000 7f378e35f640 1 -- 172.21.15.64:0/1130500098 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a4ae48000 con 0x562a4aa6b1b0 2024-09-21T04:07:04.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.175+0000 7f378eb60640 1 --2- 172.21.15.64:0/1130500098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a4aa94070 0x562a4ae39100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:07:04.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.175+0000 7f378e35f640 1 --2- 172.21.15.64:0/1130500098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa6b1b0 0x562a4ae47200 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f3784032fd0 tx=0x7f378403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a672ea62eb4756e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:04.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.175+0000 7f3773fff640 1 -- 172.21.15.64:0/1130500098 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784035040 con 0x562a4aa6b1b0 2024-09-21T04:07:04.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.175+0000 7f3773fff640 1 -- 172.21.15.64:0/1130500098 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3784056030 con 0x562a4aa6b1b0 2024-09-21T04:07:04.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.175+0000 7f3773fff640 1 -- 172.21.15.64:0/1130500098 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784004d10 con 0x562a4aa6b1b0 2024-09-21T04:07:04.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.176+0000 7f3796b37740 1 -- 172.21.15.64:0/1130500098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa6b1b0 msgr2=0x562a4ae47200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:04.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.176+0000 7f3796b37740 1 --2- 172.21.15.64:0/1130500098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa6b1b0 0x562a4ae47200 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f3784032fd0 tx=0x7f378403f000 comp rx=0 tx=0).stop 2024-09-21T04:07:04.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.176+0000 7f3796b37740 1 -- 172.21.15.64:0/1130500098 shutdown_connections 2024-09-21T04:07:04.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.176+0000 7f3796b37740 1 --2- 172.21.15.64:0/1130500098 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a4aa94070 0x562a4ae39100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.176+0000 7f3796b37740 1 --2- 172.21.15.64:0/1130500098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a4aa73390 0x562a4ae38bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.176+0000 7f3796b37740 1 --2- 172.21.15.64:0/1130500098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa6b1b0 0x562a4ae47200 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.176+0000 7f3796b37740 1 -- 172.21.15.64:0/1130500098 >> 172.21.15.64:0/1130500098 conn(0x562a4aa88460 msgr2=0x562a4ae46af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:04.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.176+0000 7f3796b37740 1 -- 172.21.15.64:0/1130500098 shutdown_connections 2024-09-21T04:07:04.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.176+0000 7f3796b37740 1 -- 172.21.15.64:0/1130500098 wait complete. 2024-09-21T04:07:04.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.176+0000 7f3796b37740 1 -- start start 2024-09-21T04:07:04.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.176+0000 7f3796b37740 1 Processor -- start 2024-09-21T04:07:04.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.177+0000 7f3796b37740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa6b1b0 0x562a4ae4f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:04.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.177+0000 7f3796b37740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a4aa73390 0x562a4ae4ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:04.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.177+0000 7f3796b37740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a4aa94070 0x562a4ae4b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:04.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.177+0000 7f3796b37740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f37881b01b0 con 0x562a4aa6b1b0 2024-09-21T04:07:04.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.177+0000 7f3796b37740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f37881b1620 con 0x562a4aa94070 2024-09-21T04:07:04.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.177+0000 7f3796b37740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f37881b0880 con 0x562a4aa73390 2024-09-21T04:07:04.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.177+0000 7f3796b37740 10 client.0 _ll_register_callbacks cb 0x7f3784009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:04.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.177+0000 7f378e35f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa6b1b0 0x562a4ae4f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:04.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.177+0000 7f378e35f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa6b1b0 0x562a4ae4f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58338/0 (socket says 172.21.15.64:58338) 2024-09-21T04:07:04.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.177+0000 7f378e35f640 1 -- 172.21.15.64:0/1952155305 learned_addr learned my addr 172.21.15.64:0/1952155305 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:04.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.177+0000 7f378db5e640 1 --2- 172.21.15.64:0/1952155305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a4aa73390 0x562a4ae4ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:04.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.177+0000 7f378eb60640 1 --2- 172.21.15.64:0/1952155305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a4aa94070 0x562a4ae4b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:04.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.177+0000 7f378e35f640 1 -- 172.21.15.64:0/1952155305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a4aa73390 msgr2=0x562a4ae4ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:04.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.177+0000 7f378e35f640 1 --2- 172.21.15.64:0/1952155305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a4aa73390 0x562a4ae4ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.177+0000 7f378e35f640 1 -- 172.21.15.64:0/1952155305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a4aa94070 msgr2=0x562a4ae4b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:04.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.177+0000 7f378e35f640 1 --2- 172.21.15.64:0/1952155305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a4aa94070 0x562a4ae4b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.177+0000 7f378e35f640 1 -- 172.21.15.64:0/1952155305 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a4ae4bb90 con 0x562a4aa6b1b0 2024-09-21T04:07:04.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.177+0000 7f378e35f640 1 --2- 172.21.15.64:0/1952155305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa6b1b0 0x562a4ae4f940 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3784031860 tx=0x7f3784009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9499628971fe2a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:04.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.177+0000 7f3771ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:07:04.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.178+0000 7f3771ffb640 1 -- 172.21.15.64:0/1952155305 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784005040 con 0x562a4aa6b1b0 2024-09-21T04:07:04.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.178+0000 7f3771ffb640 1 -- 172.21.15.64:0/1952155305 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f37840051e0 con 0x562a4aa6b1b0 2024-09-21T04:07:04.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.178+0000 7f3796b37740 1 -- 172.21.15.64:0/1952155305 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3784056030 con 0x562a4aa6b1b0 2024-09-21T04:07:04.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.178+0000 7f3771ffb640 1 -- 172.21.15.64:0/1952155305 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3784005420 con 0x562a4aa6b1b0 2024-09-21T04:07:04.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.178+0000 7f3796b37740 10 client.16185 fetch_fsmap learned FSMap version 30 2024-09-21T04:07:04.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.178+0000 7f3796b37740 1 -- 172.21.15.64:0/1952155305 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562a4aec36c0 con 0x562a4aa6b1b0 2024-09-21T04:07:04.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.178+0000 7f3771ffb640 1 -- 172.21.15.64:0/1952155305 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3784056030 con 0x562a4aa6b1b0 2024-09-21T04:07:04.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.178+0000 7f3771ffb640 1 -- 172.21.15.64:0/1952155305 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f3784003680 con 0x562a4aa6b1b0 2024-09-21T04:07:04.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.178+0000 7f3796b37740 10 client.16185 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:07:04.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.178+0000 7f3796b37740 10 client.16185 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:07:04.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.178+0000 7f3796b37740 20 client.16185 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:04.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.178+0000 7f3796b37740 1 --2- 172.21.15.64:0/1952155305 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562a4aa27da0 0x562a4aec5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:04.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.178+0000 7f3796b37740 4 client.16185 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:07:04.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.178+0000 7f3796b37740 1 -- 172.21.15.64:0/1952155305 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a4adaf8a0 con 0x562a4aa27da0 2024-09-21T04:07:04.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.179+0000 7f378db5e640 1 --2- 172.21.15.64:0/1952155305 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562a4aa27da0 0x562a4aec5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:04.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.179+0000 7f3771ffb640 10 client.16185 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:07:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.179+0000 7f378db5e640 1 --2- 172.21.15.64:0/1952155305 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562a4aa27da0 0x562a4aec5f10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cca70a9122b4a353 server_cookie=fd4ef3421f512eb1 in_seq=0 out_seq=0 2024-09-21T04:07:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.179+0000 7f3771ffb640 1 -- 172.21.15.64:0/1952155305 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a4adaf8a0 con 0x562a4aa27da0 2024-09-21T04:07:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.179+0000 7f3771ffb640 10 client.16185 handle_command_reply: tid=0 2024-09-21T04:07:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.179+0000 7f3796b37740 1 client.16185 shutdown 2024-09-21T04:07:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.179+0000 7f3796b37740 1 -- 172.21.15.64:0/1952155305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa6b1b0 msgr2=0x562a4ae4f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.179+0000 7f3796b37740 1 --2- 172.21.15.64:0/1952155305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa6b1b0 0x562a4ae4f940 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3784031860 tx=0x7f3784009230 comp rx=0 tx=0).stop 2024-09-21T04:07:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.179+0000 7f3796b37740 1 -- 172.21.15.64:0/1952155305 shutdown_connections 2024-09-21T04:07:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.179+0000 7f3796b37740 1 --2- 172.21.15.64:0/1952155305 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562a4aa27da0 0x562a4aec5f10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.179+0000 7f3796b37740 1 --2- 172.21.15.64:0/1952155305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562a4aa94070 0x562a4ae4b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.179+0000 7f3796b37740 1 --2- 172.21.15.64:0/1952155305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a4aa73390 0x562a4ae4ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.179+0000 7f3796b37740 1 --2- 172.21.15.64:0/1952155305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562a4aa6b1b0 0x562a4ae4f940 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.180+0000 7f3796b37740 1 -- 172.21.15.64:0/1952155305 >> 172.21.15.64:0/1952155305 conn(0x562a4aa88460 msgr2=0x562a4ae3e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.180+0000 7f3771ffb640 0 client.16185 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:07:04.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.180+0000 7f3796b37740 1 -- 172.21.15.64:0/1952155305 shutdown_connections 2024-09-21T04:07:04.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.180+0000 7f3796b37740 1 -- 172.21.15.64:0/1952155305 wait complete. 2024-09-21T04:07:04.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.180+0000 7f3796b37740 20 client.16185 trim_cache size 0 max 16384 2024-09-21T04:07:04.187 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:07:04.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.180+0000 7f37905c3640 1 -- 172.21.15.64:0/1903545562 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f37740b6680 msgr2=0x7f37740b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:04.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.180+0000 7f37905c3640 1 --2- 172.21.15.64:0/1903545562 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f37740b6680 0x7f37740b8b90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3784031440 tx=0x7f3784009560 comp rx=0 tx=0).stop 2024-09-21T04:07:04.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.180+0000 7f37905c3640 1 -- 172.21.15.64:0/1903545562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3788109e40 msgr2=0x7f378819fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:04.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.180+0000 7f37905c3640 1 --2- 172.21.15.64:0/1903545562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3788109e40 0x7f378819fe30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f378000e940 tx=0x7f378000ee00 comp rx=0 tx=0).stop 2024-09-21T04:07:04.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.180+0000 7f37905c3640 1 -- 172.21.15.64:0/1903545562 shutdown_connections 2024-09-21T04:07:04.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.180+0000 7f37905c3640 1 --2- 172.21.15.64:0/1903545562 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f37740b6680 0x7f37740b8b90 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.180+0000 7f37905c3640 1 --2- 172.21.15.64:0/1903545562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3788109e40 0x7f378819fe30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.180+0000 7f37905c3640 1 --2- 172.21.15.64:0/1903545562 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37881094f0 0x7f378819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.180+0000 7f37905c3640 1 --2- 172.21.15.64:0/1903545562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37881020a0 0x7f378819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:04.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.180+0000 7f37905c3640 1 -- 172.21.15.64:0/1903545562 >> 172.21.15.64:0/1903545562 conn(0x7f3788079e40 msgr2=0x7f3788107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:04.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.180+0000 7f37905c3640 1 -- 172.21.15.64:0/1903545562 shutdown_connections 2024-09-21T04:07:04.188 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:04.180+0000 7f37905c3640 1 -- 172.21.15.64:0/1903545562 wait complete. 2024-09-21T04:07:04.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:04 smithi064 ceph-mon[31180]: pgmap v412: 97 pgs: 97 active+clean; 329 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 214 op/s 2024-09-21T04:07:04.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:04 smithi119 ceph-mon[34811]: pgmap v412: 97 pgs: 97 active+clean; 329 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 214 op/s 2024-09-21T04:07:04.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:04 smithi131 ceph-mon[35193]: pgmap v412: 97 pgs: 97 active+clean; 329 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 214 op/s 2024-09-21T04:07:05.069 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T04:07:05.354 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:07:05.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.899+0000 7ff233b5f640 1 -- 172.21.15.64:0/4145817258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff22c102a60 msgr2=0x7ff22c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:05.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.899+0000 7ff233b5f640 1 --2- 172.21.15.64:0/4145817258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff22c102a60 0x7ff22c102e70 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009ca0 tx=0x7ff21c02a8c0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.899+0000 7ff233b5f640 1 -- 172.21.15.64:0/4145817258 shutdown_connections 2024-09-21T04:07:05.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.899+0000 7ff233b5f640 1 --2- 172.21.15.64:0/4145817258 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff22c108cf0 0x7ff22c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.899+0000 7ff233b5f640 1 --2- 172.21.15.64:0/4145817258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff22c1083a0 0x7ff22c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.899+0000 7ff233b5f640 1 --2- 172.21.15.64:0/4145817258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff22c102a60 0x7ff22c102e70 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.899+0000 7ff233b5f640 1 -- 172.21.15.64:0/4145817258 >> 172.21.15.64:0/4145817258 conn(0x7ff22c079730 msgr2=0x7ff22c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:05.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.899+0000 7ff233b5f640 1 -- 172.21.15.64:0/4145817258 shutdown_connections 2024-09-21T04:07:05.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.899+0000 7ff233b5f640 1 -- 172.21.15.64:0/4145817258 wait complete. 2024-09-21T04:07:05.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.900+0000 7ff233b5f640 1 Processor -- start 2024-09-21T04:07:05.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.900+0000 7ff233b5f640 1 -- start start 2024-09-21T04:07:05.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.900+0000 7ff233b5f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff22c102a60 0x7ff22c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:05.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.900+0000 7ff233b5f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff22c1083a0 0x7ff22c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:05.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.901+0000 7ff233b5f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff22c108cf0 0x7ff22c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:05.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.901+0000 7ff233b5f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c113a80 con 0x7ff22c1083a0 2024-09-21T04:07:05.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.901+0000 7ff233b5f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c113900 con 0x7ff22c102a60 2024-09-21T04:07:05.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.901+0000 7ff233b5f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c113c00 con 0x7ff22c108cf0 2024-09-21T04:07:05.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.901+0000 7ff2310fa640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff22c1083a0 0x7ff22c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:05.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.901+0000 7ff2318fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff22c102a60 0x7ff22c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:05.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.901+0000 7ff2310fa640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff22c1083a0 0x7ff22c19a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58356/0 (socket says 172.21.15.64:58356) 2024-09-21T04:07:05.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.901+0000 7ff2310fa640 1 -- 172.21.15.64:0/1510930715 learned_addr learned my addr 172.21.15.64:0/1510930715 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:05.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.901+0000 7ff2318fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff22c102a60 0x7ff22c19a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:41402/0 (socket says 172.21.15.64:41402) 2024-09-21T04:07:05.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.901+0000 7ff2320fc640 1 --2- 172.21.15.64:0/1510930715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff22c108cf0 0x7ff22c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:05.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.901+0000 7ff2310fa640 1 -- 172.21.15.64:0/1510930715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff22c108cf0 msgr2=0x7ff22c19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:05.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.901+0000 7ff2310fa640 1 --2- 172.21.15.64:0/1510930715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff22c108cf0 0x7ff22c19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.901+0000 7ff2310fa640 1 -- 172.21.15.64:0/1510930715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff22c102a60 msgr2=0x7ff22c19a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:05.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.901+0000 7ff2310fa640 1 --2- 172.21.15.64:0/1510930715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff22c102a60 0x7ff22c19a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.901+0000 7ff2310fa640 1 -- 172.21.15.64:0/1510930715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff22c1a03c0 con 0x7ff22c1083a0 2024-09-21T04:07:05.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.901+0000 7ff2318fb640 1 --2- 172.21.15.64:0/1510930715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff22c102a60 0x7ff22c19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:07:05.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.901+0000 7ff2310fa640 1 --2- 172.21.15.64:0/1510930715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff22c1083a0 0x7ff22c19a9f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff22000e9c0 tx=0x7ff22000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:05.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.901+0000 7ff22affd640 1 -- 172.21.15.64:0/1510930715 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22000cd80 con 0x7ff22c1083a0 2024-09-21T04:07:05.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.901+0000 7ff233b5f640 1 -- 172.21.15.64:0/1510930715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff22c1a06b0 con 0x7ff22c1083a0 2024-09-21T04:07:05.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.901+0000 7ff22affd640 1 -- 172.21.15.64:0/1510930715 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2200054d0 con 0x7ff22c1083a0 2024-09-21T04:07:05.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.902+0000 7ff22affd640 1 -- 172.21.15.64:0/1510930715 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220010430 con 0x7ff22c1083a0 2024-09-21T04:07:05.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.902+0000 7ff233b5f640 1 -- 172.21.15.64:0/1510930715 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff22c1a6e10 con 0x7ff22c1083a0 2024-09-21T04:07:05.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.904+0000 7ff23a0d3740 1 Processor -- start 2024-09-21T04:07:05.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.904+0000 7ff23a0d3740 1 -- start start 2024-09-21T04:07:05.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.904+0000 7ff22affd640 1 -- 172.21.15.64:0/1510930715 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7ff220004fd0 con 0x7ff22c1083a0 2024-09-21T04:07:05.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.905+0000 7ff23a0d3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562412e20070 0x5624131c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:05.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.905+0000 7ff23a0d3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562412df71b0 0x5624131c7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:05.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.905+0000 7ff23a0d3740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562412dff390 0x5624131c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:05.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.905+0000 7ff23a0d3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56241313dea0 con 0x562412df71b0 2024-09-21T04:07:05.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.905+0000 7ff23a0d3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56241312d2d0 con 0x562412e20070 2024-09-21T04:07:05.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.905+0000 7ff23a0d3740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5624131310e0 con 0x562412dff390 2024-09-21T04:07:05.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.905+0000 7ff2320fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562412e20070 0x5624131c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:05.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.905+0000 7ff2320fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562412e20070 0x5624131c72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:41414/0 (socket says 172.21.15.64:41414) 2024-09-21T04:07:05.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.905+0000 7ff2320fc640 1 -- 172.21.15.64:0/1583591231 learned_addr learned my addr 172.21.15.64:0/1583591231 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:05.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.905+0000 7ff2318fb640 1 --2- 172.21.15.64:0/1583591231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562412df71b0 0x5624131c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:05.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.905+0000 7ff2310fa640 1 --2- 172.21.15.64:0/1583591231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562412dff390 0x5624131c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:05.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.905+0000 7ff2320fc640 1 -- 172.21.15.64:0/1583591231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562412dff390 msgr2=0x5624131c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:05.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.905+0000 7ff2320fc640 1 --2- 172.21.15.64:0/1583591231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562412dff390 0x5624131c7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.905+0000 7ff2320fc640 1 -- 172.21.15.64:0/1583591231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562412df71b0 msgr2=0x5624131c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:05.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.905+0000 7ff2320fc640 1 --2- 172.21.15.64:0/1583591231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562412df71b0 0x5624131c7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.905+0000 7ff2320fc640 1 -- 172.21.15.64:0/1583591231 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562412e8b650 con 0x562412e20070 2024-09-21T04:07:05.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.905+0000 7ff2310fa640 1 --2- 172.21.15.64:0/1583591231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562412dff390 0x5624131c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:07:05.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.905+0000 7ff22affd640 1 --2- 172.21.15.64:0/1510930715 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff2140b6680 0x7ff2140b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:05.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.906+0000 7ff2320fc640 1 --2- 172.21.15.64:0/1583591231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562412e20070 0x5624131c72c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff22400bc70 tx=0x7ff22400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=671f9984fab32178 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:05.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.906+0000 7ff2318fb640 1 --2- 172.21.15.64:0/1510930715 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff2140b6680 0x7ff2140b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:05.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.906+0000 7ff22affd640 1 -- 172.21.15.64:0/1510930715 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7ff2200e7c10 con 0x7ff22c1083a0 2024-09-21T04:07:05.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.906+0000 7ff213fff640 1 -- 172.21.15.64:0/1583591231 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224015070 con 0x562412e20070 2024-09-21T04:07:05.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.906+0000 7ff213fff640 1 -- 172.21.15.64:0/1583591231 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff22401a910 con 0x562412e20070 2024-09-21T04:07:05.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.906+0000 7ff213fff640 1 -- 172.21.15.64:0/1583591231 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224011880 con 0x562412e20070 2024-09-21T04:07:05.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.907+0000 7ff23a0d3740 1 -- 172.21.15.64:0/1583591231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562412e20070 msgr2=0x5624131c72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:05.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.907+0000 7ff2318fb640 1 --2- 172.21.15.64:0/1510930715 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff2140b6680 0x7ff2140b8b90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x5624131c5f40 tx=0x7ff21c009360 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:05.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.907+0000 7ff23a0d3740 1 --2- 172.21.15.64:0/1583591231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562412e20070 0x5624131c72c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff22400bc70 tx=0x7ff22400f730 comp rx=0 tx=0).stop 2024-09-21T04:07:05.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.907+0000 7ff23a0d3740 1 -- 172.21.15.64:0/1583591231 shutdown_connections 2024-09-21T04:07:05.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.907+0000 7ff23a0d3740 1 --2- 172.21.15.64:0/1583591231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562412dff390 0x5624131c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.907+0000 7ff23a0d3740 1 --2- 172.21.15.64:0/1583591231 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562412df71b0 0x5624131c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.907+0000 7ff23a0d3740 1 --2- 172.21.15.64:0/1583591231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562412e20070 0x5624131c72c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.907+0000 7ff23a0d3740 1 -- 172.21.15.64:0/1583591231 >> 172.21.15.64:0/1583591231 conn(0x562412e14460 msgr2=0x5624131c51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:05.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.907+0000 7ff23a0d3740 1 -- 172.21.15.64:0/1583591231 shutdown_connections 2024-09-21T04:07:05.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.907+0000 7ff23a0d3740 1 -- 172.21.15.64:0/1583591231 wait complete. 2024-09-21T04:07:05.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.907+0000 7ff23a0d3740 1 -- start start 2024-09-21T04:07:05.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.908+0000 7ff23a0d3740 1 Processor -- start 2024-09-21T04:07:05.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.908+0000 7ff23a0d3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562412df71b0 0x5624131d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:05.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.908+0000 7ff23a0d3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562412dff390 0x5624131d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:05.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.908+0000 7ff23a0d3740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562412e20070 0x5624131dedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:05.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.908+0000 7ff23a0d3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562412f8e700 con 0x562412dff390 2024-09-21T04:07:05.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.908+0000 7ff23a0d3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5624130c8160 con 0x562412df71b0 2024-09-21T04:07:05.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.908+0000 7ff23a0d3740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5624131d5aa0 con 0x562412e20070 2024-09-21T04:07:05.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.908+0000 7ff23a0d3740 10 client.0 _ll_register_callbacks cb 0x5624130e19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:05.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.908+0000 7ff2310fa640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562412dff390 0x5624131d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:05.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.908+0000 7ff2310fa640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562412dff390 0x5624131d2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58368/0 (socket says 172.21.15.64:58368) 2024-09-21T04:07:05.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.908+0000 7ff2318fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562412df71b0 0x5624131d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:05.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.908+0000 7ff2310fa640 1 -- 172.21.15.64:0/2272399178 learned_addr learned my addr 172.21.15.64:0/2272399178 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:05.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.908+0000 7ff2320fc640 1 --2- 172.21.15.64:0/2272399178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562412e20070 0x5624131dedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:05.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.909+0000 7ff2310fa640 1 -- 172.21.15.64:0/2272399178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562412e20070 msgr2=0x5624131dedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:05.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.909+0000 7ff2310fa640 1 --2- 172.21.15.64:0/2272399178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562412e20070 0x5624131dedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.909+0000 7ff2310fa640 1 -- 172.21.15.64:0/2272399178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562412df71b0 msgr2=0x5624131d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:05.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.909+0000 7ff2310fa640 1 --2- 172.21.15.64:0/2272399178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562412df71b0 0x5624131d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.909+0000 7ff2310fa640 1 -- 172.21.15.64:0/2272399178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624130f5c90 con 0x562412dff390 2024-09-21T04:07:05.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.909+0000 7ff2320fc640 1 --2- 172.21.15.64:0/2272399178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562412e20070 0x5624131dedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:07:05.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.909+0000 7ff2318fb640 1 --2- 172.21.15.64:0/2272399178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562412df71b0 0x5624131d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:07:05.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.909+0000 7ff2310fa640 1 --2- 172.21.15.64:0/2272399178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562412dff390 0x5624131d2ef0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff2200ad9c0 tx=0x7ff2200a98d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41aaecea72a58398 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:05.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.909+0000 7ff2117fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:07:05.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.909+0000 7ff2117fa640 1 -- 172.21.15.64:0/2272399178 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2200ad1a0 con 0x562412dff390 2024-09-21T04:07:05.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.909+0000 7ff2117fa640 1 -- 172.21.15.64:0/2272399178 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7ff2200acc90 con 0x562412dff390 2024-09-21T04:07:05.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.909+0000 7ff23a0d3740 1 -- 172.21.15.64:0/2272399178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff22401a910 con 0x562412dff390 2024-09-21T04:07:05.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.909+0000 7ff2117fa640 1 -- 172.21.15.64:0/2272399178 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff2200a91a0 con 0x562412dff390 2024-09-21T04:07:05.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.909+0000 7ff23a0d3740 10 client.16197 fetch_fsmap learned FSMap version 30 2024-09-21T04:07:05.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.909+0000 7ff23a0d3740 1 -- 172.21.15.64:0/2272399178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5624131e17b0 con 0x562412dff390 2024-09-21T04:07:05.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.909+0000 7ff2117fa640 1 -- 172.21.15.64:0/2272399178 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff22401a910 con 0x562412dff390 2024-09-21T04:07:05.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.910+0000 7ff2117fa640 1 -- 172.21.15.64:0/2272399178 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7ff2200ac700 con 0x562412dff390 2024-09-21T04:07:05.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.910+0000 7ff23a0d3740 10 client.16197 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:07:05.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.910+0000 7ff23a0d3740 10 client.16197 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:07:05.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.910+0000 7ff23a0d3740 20 client.16197 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:05.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.910+0000 7ff23a0d3740 1 --2- 172.21.15.64:0/2272399178 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x562412dde9f0 0x5624131e4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:05.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.910+0000 7ff23a0d3740 4 client.16197 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:07:05.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.910+0000 7ff23a0d3740 1 -- 172.21.15.64:0/2272399178 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56241313fdf0 con 0x562412dde9f0 2024-09-21T04:07:05.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.910+0000 7ff2318fb640 1 --2- 172.21.15.64:0/2272399178 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x562412dde9f0 0x5624131e4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:05.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.910+0000 7ff2117fa640 10 client.16197 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:07:05.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.910+0000 7ff2318fb640 1 --2- 172.21.15.64:0/2272399178 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x562412dde9f0 0x5624131e4d10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c71345d6e85291d7 server_cookie=2dfee7703c1de8da in_seq=0 out_seq=0 2024-09-21T04:07:05.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.912+0000 7ff2117fa640 1 -- 172.21.15.64:0/2272399178 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56241313fdf0 con 0x562412dde9f0 2024-09-21T04:07:05.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.912+0000 7ff2117fa640 10 client.16197 handle_command_reply: tid=0 2024-09-21T04:07:05.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.912+0000 7ff23a0d3740 1 client.16197 shutdown 2024-09-21T04:07:05.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.913+0000 7ff23a0d3740 1 -- 172.21.15.64:0/2272399178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562412dff390 msgr2=0x5624131d2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:05.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.913+0000 7ff23a0d3740 1 --2- 172.21.15.64:0/2272399178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562412dff390 0x5624131d2ef0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff2200ad9c0 tx=0x7ff2200a98d0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.913+0000 7ff23a0d3740 1 -- 172.21.15.64:0/2272399178 shutdown_connections 2024-09-21T04:07:05.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.913+0000 7ff23a0d3740 1 --2- 172.21.15.64:0/2272399178 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x562412dde9f0 0x5624131e4d10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.913+0000 7ff23a0d3740 1 --2- 172.21.15.64:0/2272399178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562412e20070 0x5624131dedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.913+0000 7ff23a0d3740 1 --2- 172.21.15.64:0/2272399178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562412dff390 0x5624131d2ef0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.913+0000 7ff2117fa640 0 client.16197 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:07:05.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.913+0000 7ff23a0d3740 1 --2- 172.21.15.64:0/2272399178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562412df71b0 0x5624131d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.913+0000 7ff23a0d3740 1 -- 172.21.15.64:0/2272399178 >> 172.21.15.64:0/2272399178 conn(0x562412e14460 msgr2=0x5624131ca450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:05.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.913+0000 7ff23a0d3740 1 -- 172.21.15.64:0/2272399178 shutdown_connections 2024-09-21T04:07:05.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.913+0000 7ff23a0d3740 1 -- 172.21.15.64:0/2272399178 wait complete. 2024-09-21T04:07:05.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.913+0000 7ff23a0d3740 20 client.16197 trim_cache size 0 max 16384 2024-09-21T04:07:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.928+0000 7ff23a0d3740 1 Processor -- start 2024-09-21T04:07:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.928+0000 7ff23a0d3740 1 -- start start 2024-09-21T04:07:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.928+0000 7ff23a0d3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562412df71b0 0x5624131d2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.928+0000 7ff23a0d3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562412dff390 0x5624131ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.928+0000 7ff23a0d3740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562412e20070 0x5624131ca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.928+0000 7ff23a0d3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c1b2050 con 0x562412df71b0 2024-09-21T04:07:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.928+0000 7ff23a0d3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c1af890 con 0x562412dff390 2024-09-21T04:07:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.928+0000 7ff23a0d3740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c1ad6e0 con 0x562412e20070 2024-09-21T04:07:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.928+0000 7ff2318fb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562412df71b0 0x5624131d2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.928+0000 7ff2310fa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562412dff390 0x5624131ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.928+0000 7ff2320fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562412e20070 0x5624131ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:05.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.928+0000 7ff2318fb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562412df71b0 0x5624131d2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58374/0 (socket says 172.21.15.64:58374) 2024-09-21T04:07:05.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.928+0000 7ff2310fa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562412dff390 0x5624131ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:41426/0 (socket says 172.21.15.64:41426) 2024-09-21T04:07:05.932 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.928+0000 7ff2318fb640 1 -- 172.21.15.64:0/3777813303 learned_addr learned my addr 172.21.15.64:0/3777813303 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:05.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.929+0000 7ff2320fc640 1 -- 172.21.15.64:0/3777813303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562412dff390 msgr2=0x5624131ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:05.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.929+0000 7ff2320fc640 1 --2- 172.21.15.64:0/3777813303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562412dff390 0x5624131ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.929+0000 7ff2320fc640 1 -- 172.21.15.64:0/3777813303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562412df71b0 msgr2=0x5624131d2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:05.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.929+0000 7ff2320fc640 1 --2- 172.21.15.64:0/3777813303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562412df71b0 0x5624131d2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.929+0000 7ff2320fc640 1 -- 172.21.15.64:0/3777813303 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624131df460 con 0x562412e20070 2024-09-21T04:07:05.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.929+0000 7ff2318fb640 1 --2- 172.21.15.64:0/3777813303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562412df71b0 0x5624131d2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:07:05.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.929+0000 7ff2310fa640 1 --2- 172.21.15.64:0/3777813303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562412dff390 0x5624131ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:07:05.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.929+0000 7ff2320fc640 1 --2- 172.21.15.64:0/3777813303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562412e20070 0x5624131ca900 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff224011480 tx=0x7ff22400b400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b2ec42e89e940f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:05.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.929+0000 7ff2137fe640 1 -- 172.21.15.64:0/3777813303 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224015050 con 0x562412e20070 2024-09-21T04:07:05.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.929+0000 7ff2137fe640 1 -- 172.21.15.64:0/3777813303 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff224004680 con 0x562412e20070 2024-09-21T04:07:05.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.929+0000 7ff2137fe640 1 -- 172.21.15.64:0/3777813303 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224011780 con 0x562412e20070 2024-09-21T04:07:05.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.930+0000 7ff23a0d3740 1 -- 172.21.15.64:0/3777813303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562412e20070 msgr2=0x5624131ca900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:05.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.930+0000 7ff23a0d3740 1 --2- 172.21.15.64:0/3777813303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562412e20070 0x5624131ca900 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff224011480 tx=0x7ff22400b400 comp rx=0 tx=0).stop 2024-09-21T04:07:05.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.930+0000 7ff23a0d3740 1 -- 172.21.15.64:0/3777813303 shutdown_connections 2024-09-21T04:07:05.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.930+0000 7ff23a0d3740 1 --2- 172.21.15.64:0/3777813303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562412e20070 0x5624131ca900 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.930+0000 7ff23a0d3740 1 --2- 172.21.15.64:0/3777813303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562412dff390 0x5624131ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.930+0000 7ff23a0d3740 1 --2- 172.21.15.64:0/3777813303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562412df71b0 0x5624131d2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.930+0000 7ff23a0d3740 1 -- 172.21.15.64:0/3777813303 >> 172.21.15.64:0/3777813303 conn(0x562412e14460 msgr2=0x5624131d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:05.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.930+0000 7ff23a0d3740 1 -- 172.21.15.64:0/3777813303 shutdown_connections 2024-09-21T04:07:05.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.930+0000 7ff23a0d3740 1 -- 172.21.15.64:0/3777813303 wait complete. 2024-09-21T04:07:05.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.931+0000 7ff23a0d3740 1 -- start start 2024-09-21T04:07:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.931+0000 7ff23a0d3740 1 Processor -- start 2024-09-21T04:07:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.931+0000 7ff23a0d3740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562412df71b0 0x5624131dad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.931+0000 7ff23a0d3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562412dff390 0x5624131db290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.931+0000 7ff23a0d3740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562412e20070 0x5624132486c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.931+0000 7ff23a0d3740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c1add50 con 0x562412e20070 2024-09-21T04:07:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.931+0000 7ff23a0d3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c1af1c0 con 0x562412dff390 2024-09-21T04:07:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.931+0000 7ff23a0d3740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c1ae420 con 0x562412df71b0 2024-09-21T04:07:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.932+0000 7ff23a0d3740 10 client.0 _ll_register_callbacks cb 0x7ff22000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.932+0000 7ff2310fa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562412dff390 0x5624131db290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.932+0000 7ff2320fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562412e20070 0x5624132486c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.932+0000 7ff2310fa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562412dff390 0x5624131db290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:41430/0 (socket says 172.21.15.64:41430) 2024-09-21T04:07:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.932+0000 7ff2310fa640 1 -- 172.21.15.64:0/3210030739 learned_addr learned my addr 172.21.15.64:0/3210030739 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:05.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.932+0000 7ff2318fb640 1 --2- 172.21.15.64:0/3210030739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562412df71b0 0x5624131dad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:05.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.932+0000 7ff2310fa640 1 -- 172.21.15.64:0/3210030739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562412df71b0 msgr2=0x5624131dad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:05.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.932+0000 7ff2310fa640 1 --2- 172.21.15.64:0/3210030739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562412df71b0 0x5624131dad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.932+0000 7ff2310fa640 1 -- 172.21.15.64:0/3210030739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562412e20070 msgr2=0x5624132486c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:05.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.932+0000 7ff2310fa640 1 --2- 172.21.15.64:0/3210030739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562412e20070 0x5624132486c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.932+0000 7ff2310fa640 1 -- 172.21.15.64:0/3210030739 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624131db7d0 con 0x562412dff390 2024-09-21T04:07:05.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.932+0000 7ff2310fa640 1 --2- 172.21.15.64:0/3210030739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562412dff390 0x5624131db290 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff2200acb10 tx=0x7ff2200b7550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b9fd27744e86838 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:05.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.932+0000 7ff2117fa640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:07:05.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.932+0000 7ff2117fa640 1 -- 172.21.15.64:0/3210030739 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220096030 con 0x562412dff390 2024-09-21T04:07:05.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.933+0000 7ff2117fa640 1 -- 172.21.15.64:0/3210030739 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7ff2200acc90 con 0x562412dff390 2024-09-21T04:07:05.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.933+0000 7ff23a0d3740 1 -- 172.21.15.64:0/3210030739 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff224004680 con 0x562412dff390 2024-09-21T04:07:05.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.933+0000 7ff2117fa640 1 -- 172.21.15.64:0/3210030739 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff2200ad1a0 con 0x562412dff390 2024-09-21T04:07:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.933+0000 7ff23a0d3740 10 client.25913 fetch_fsmap learned FSMap version 30 2024-09-21T04:07:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.933+0000 7ff23a0d3740 1 -- 172.21.15.64:0/3210030739 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562413253d80 con 0x562412dff390 2024-09-21T04:07:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.933+0000 7ff2117fa640 1 -- 172.21.15.64:0/3210030739 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff224004680 con 0x562412dff390 2024-09-21T04:07:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.933+0000 7ff2117fa640 1 -- 172.21.15.64:0/3210030739 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7ff2200c3be0 con 0x562412dff390 2024-09-21T04:07:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.933+0000 7ff23a0d3740 10 client.25913 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:07:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.933+0000 7ff23a0d3740 10 client.25913 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:07:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.933+0000 7ff23a0d3740 20 client.25913 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.933+0000 7ff23a0d3740 1 --2- 172.21.15.64:0/3210030739 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x562412db3da0 0x562413256480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.933+0000 7ff23a0d3740 4 client.25913 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:07:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.933+0000 7ff23a0d3740 1 -- 172.21.15.64:0/3210030739 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56241313b8a0 con 0x562412db3da0 2024-09-21T04:07:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.933+0000 7ff2318fb640 1 --2- 172.21.15.64:0/3210030739 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x562412db3da0 0x562413256480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.934+0000 7ff2117fa640 10 client.25913 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:07:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.934+0000 7ff2318fb640 1 --2- 172.21.15.64:0/3210030739 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x562412db3da0 0x562413256480 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a2edd797ce158c9b server_cookie=b51a48e3cce884dd in_seq=0 out_seq=0 2024-09-21T04:07:05.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.936+0000 7ff2117fa640 1 -- 172.21.15.64:0/3210030739 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56241313b8a0 con 0x562412db3da0 2024-09-21T04:07:05.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.936+0000 7ff2117fa640 10 client.25913 handle_command_reply: tid=0 2024-09-21T04:07:05.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.936+0000 7ff23a0d3740 1 client.25913 shutdown 2024-09-21T04:07:05.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.936+0000 7ff23a0d3740 1 -- 172.21.15.64:0/3210030739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562412dff390 msgr2=0x5624131db290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:05.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.936+0000 7ff23a0d3740 1 --2- 172.21.15.64:0/3210030739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562412dff390 0x5624131db290 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff2200acb10 tx=0x7ff2200b7550 comp rx=0 tx=0).stop 2024-09-21T04:07:05.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.936+0000 7ff23a0d3740 1 -- 172.21.15.64:0/3210030739 shutdown_connections 2024-09-21T04:07:05.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.936+0000 7ff23a0d3740 1 --2- 172.21.15.64:0/3210030739 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x562412db3da0 0x562413256480 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.936+0000 7ff23a0d3740 1 --2- 172.21.15.64:0/3210030739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562412e20070 0x5624132486c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.936+0000 7ff23a0d3740 1 --2- 172.21.15.64:0/3210030739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562412dff390 0x5624131db290 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.936+0000 7ff2117fa640 0 client.25913 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:07:05.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.936+0000 7ff23a0d3740 1 --2- 172.21.15.64:0/3210030739 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562412df71b0 0x5624131dad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.936+0000 7ff23a0d3740 1 -- 172.21.15.64:0/3210030739 >> 172.21.15.64:0/3210030739 conn(0x562412e14460 msgr2=0x5624131ca3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:05.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.936+0000 7ff23a0d3740 1 -- 172.21.15.64:0/3210030739 shutdown_connections 2024-09-21T04:07:05.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.936+0000 7ff23a0d3740 1 -- 172.21.15.64:0/3210030739 wait complete. 2024-09-21T04:07:05.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.936+0000 7ff23a0d3740 20 client.25913 trim_cache size 0 max 16384 2024-09-21T04:07:05.942 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:07:05.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.937+0000 7ff233b5f640 1 -- 172.21.15.64:0/1510930715 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff2140b6680 msgr2=0x7ff2140b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:05.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.937+0000 7ff233b5f640 1 --2- 172.21.15.64:0/1510930715 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff2140b6680 0x7ff2140b8b90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x5624131c5f40 tx=0x7ff21c009360 comp rx=0 tx=0).stop 2024-09-21T04:07:05.942 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.937+0000 7ff233b5f640 1 -- 172.21.15.64:0/1510930715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff22c1083a0 msgr2=0x7ff22c19a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:05.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.937+0000 7ff233b5f640 1 --2- 172.21.15.64:0/1510930715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff22c1083a0 0x7ff22c19a9f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff22000e9c0 tx=0x7ff22000ee80 comp rx=0 tx=0).stop 2024-09-21T04:07:05.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.937+0000 7ff233b5f640 1 -- 172.21.15.64:0/1510930715 shutdown_connections 2024-09-21T04:07:05.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.937+0000 7ff233b5f640 1 --2- 172.21.15.64:0/1510930715 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff2140b6680 0x7ff2140b8b90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.937+0000 7ff233b5f640 1 --2- 172.21.15.64:0/1510930715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff22c108cf0 0x7ff22c19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.937+0000 7ff233b5f640 1 --2- 172.21.15.64:0/1510930715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff22c1083a0 0x7ff22c19a9f0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.937+0000 7ff233b5f640 1 --2- 172.21.15.64:0/1510930715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff22c102a60 0x7ff22c19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:05.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.937+0000 7ff233b5f640 1 -- 172.21.15.64:0/1510930715 >> 172.21.15.64:0/1510930715 conn(0x7ff22c079730 msgr2=0x7ff22c1013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:05.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.937+0000 7ff233b5f640 1 -- 172.21.15.64:0/1510930715 shutdown_connections 2024-09-21T04:07:05.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:05.937+0000 7ff233b5f640 1 -- 172.21.15.64:0/1510930715 wait complete. 2024-09-21T04:07:06.744 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:06 smithi064 ceph-mon[31180]: pgmap v413: 97 pgs: 97 active+clean; 336 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 222 op/s 2024-09-21T04:07:06.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:06 smithi119 ceph-mon[34811]: pgmap v413: 97 pgs: 97 active+clean; 336 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 222 op/s 2024-09-21T04:07:06.783 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T04:07:06.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:06 smithi131 ceph-mon[35193]: pgmap v413: 97 pgs: 97 active+clean; 336 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 222 op/s 2024-09-21T04:07:07.095 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:07:07.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.648+0000 7f59af22c640 1 -- 172.21.15.64:0/1009467102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59a8105340 msgr2=0x7f59a810d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:07.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.648+0000 7f59af22c640 1 --2- 172.21.15.64:0/1009467102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59a8105340 0x7f59a810d890 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5994009900 tx=0x7f599402a410 comp rx=0 tx=0).stop 2024-09-21T04:07:07.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.648+0000 7f59af22c640 1 -- 172.21.15.64:0/1009467102 shutdown_connections 2024-09-21T04:07:07.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.648+0000 7f59af22c640 1 --2- 172.21.15.64:0/1009467102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59a810de50 0x7f59a8110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.648+0000 7f59af22c640 1 --2- 172.21.15.64:0/1009467102 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59a8105340 0x7f59a810d890 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.648+0000 7f59af22c640 1 --2- 172.21.15.64:0/1009467102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59a81049f0 0x7f59a8104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.648+0000 7f59af22c640 1 -- 172.21.15.64:0/1009467102 >> 172.21.15.64:0/1009467102 conn(0x7f59a8079750 msgr2=0x7f59a8079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:07.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.648+0000 7f59af22c640 1 -- 172.21.15.64:0/1009467102 shutdown_connections 2024-09-21T04:07:07.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.648+0000 7f59af22c640 1 -- 172.21.15.64:0/1009467102 wait complete. 2024-09-21T04:07:07.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.649+0000 7f59af22c640 1 Processor -- start 2024-09-21T04:07:07.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.649+0000 7f59af22c640 1 -- start start 2024-09-21T04:07:07.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.650+0000 7f59af22c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59a81049f0 0x7f59a8102730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.650+0000 7f59af22c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59a8105340 0x7f59a8102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.650+0000 7f59af22c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59a810de50 0x7f59a81031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.650+0000 7f59af22c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f59a8113b00 con 0x7f59a81049f0 2024-09-21T04:07:07.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.650+0000 7f59af22c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f59a8113980 con 0x7f59a8105340 2024-09-21T04:07:07.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.650+0000 7f59af22c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f59a8113c80 con 0x7f59a810de50 2024-09-21T04:07:07.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.650+0000 7f59acfc8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59a81049f0 0x7f59a8102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.650+0000 7f59a7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59a8105340 0x7f59a8102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.650+0000 7f59a7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59a8105340 0x7f59a8102c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:41442/0 (socket says 172.21.15.64:41442) 2024-09-21T04:07:07.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.650+0000 7f59acfc8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59a81049f0 0x7f59a8102730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58412/0 (socket says 172.21.15.64:58412) 2024-09-21T04:07:07.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.650+0000 7f59a7fff640 1 -- 172.21.15.64:0/1112217521 learned_addr learned my addr 172.21.15.64:0/1112217521 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:07.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.650+0000 7f59ad7c9640 1 --2- 172.21.15.64:0/1112217521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59a810de50 0x7f59a81031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.651+0000 7f59a7fff640 1 -- 172.21.15.64:0/1112217521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59a810de50 msgr2=0x7f59a81031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:07.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.651+0000 7f59a7fff640 1 --2- 172.21.15.64:0/1112217521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59a810de50 0x7f59a81031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.651+0000 7f59a7fff640 1 -- 172.21.15.64:0/1112217521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59a81049f0 msgr2=0x7f59a8102730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:07.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.651+0000 7f59a7fff640 1 --2- 172.21.15.64:0/1112217521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59a81049f0 0x7f59a8102730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.651+0000 7f59a7fff640 1 -- 172.21.15.64:0/1112217521 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a81a8750 con 0x7f59a8105340 2024-09-21T04:07:07.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.651+0000 7f59a7fff640 1 --2- 172.21.15.64:0/1112217521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59a8105340 0x7f59a8102c70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5994005ab0 tx=0x7f5994003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:07.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.651+0000 7f59a5ffb640 1 -- 172.21.15.64:0/1112217521 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599403f070 con 0x7f59a8105340 2024-09-21T04:07:07.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.651+0000 7f59a5ffb640 1 -- 172.21.15.64:0/1112217521 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5994031b50 con 0x7f59a8105340 2024-09-21T04:07:07.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.651+0000 7f59af22c640 1 -- 172.21.15.64:0/1112217521 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a81a89e0 con 0x7f59a8105340 2024-09-21T04:07:07.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.651+0000 7f59a5ffb640 1 -- 172.21.15.64:0/1112217521 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5994035030 con 0x7f59a8105340 2024-09-21T04:07:07.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.652+0000 7f59af22c640 1 -- 172.21.15.64:0/1112217521 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a81a8f10 con 0x7f59a8105340 2024-09-21T04:07:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.654+0000 7f59b57a0740 1 Processor -- start 2024-09-21T04:07:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.655+0000 7f59b57a0740 1 -- start start 2024-09-21T04:07:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.655+0000 7f59b57a0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5a3672070 0x55d5a3a192c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.655+0000 7f59b57a0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5a36491b0 0x55d5a3a19800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.655+0000 7f59b57a0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a3651390 0x55d5a3a19d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.655+0000 7f59b57a0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d5a398fea0 con 0x55d5a3672070 2024-09-21T04:07:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.655+0000 7f59b57a0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d5a397f2d0 con 0x55d5a36491b0 2024-09-21T04:07:07.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.655+0000 7f59b57a0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d5a39830e0 con 0x55d5a3651390 2024-09-21T04:07:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.655+0000 7f59ad7c9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5a3672070 0x55d5a3a192c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.655+0000 7f59acfc8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5a36491b0 0x55d5a3a19800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.655+0000 7f59ad7c9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5a3672070 0x55d5a3a192c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58424/0 (socket says 172.21.15.64:58424) 2024-09-21T04:07:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.655+0000 7f59ad7c9640 1 -- 172.21.15.64:0/3120029191 learned_addr learned my addr 172.21.15.64:0/3120029191 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.656+0000 7f59acfc8640 1 -- 172.21.15.64:0/3120029191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a3651390 msgr2=0x55d5a3a19d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:07:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.656+0000 7f59acfc8640 1 --2- 172.21.15.64:0/3120029191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a3651390 0x55d5a3a19d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.656+0000 7f59acfc8640 1 -- 172.21.15.64:0/3120029191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5a3672070 msgr2=0x55d5a3a192c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.656+0000 7f59acfc8640 1 --2- 172.21.15.64:0/3120029191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5a3672070 0x55d5a3a192c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.656+0000 7f59acfc8640 1 -- 172.21.15.64:0/3120029191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5a36dd650 con 0x55d5a36491b0 2024-09-21T04:07:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.656+0000 7f59a5ffb640 1 -- 172.21.15.64:0/1112217521 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f5994042070 con 0x7f59a8105340 2024-09-21T04:07:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.656+0000 7f59acfc8640 1 --2- 172.21.15.64:0/3120029191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5a36491b0 0x55d5a3a19800 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f599c009bd0 tx=0x7f599c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a33a5f770dbc3205 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.656+0000 7f5992ffd640 1 -- 172.21.15.64:0/3120029191 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c004230 con 0x55d5a36491b0 2024-09-21T04:07:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.656+0000 7f5992ffd640 1 -- 172.21.15.64:0/3120029191 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f599c009e40 con 0x55d5a36491b0 2024-09-21T04:07:07.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.656+0000 7f5992ffd640 1 -- 172.21.15.64:0/3120029191 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c004ca0 con 0x55d5a36491b0 2024-09-21T04:07:07.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.657+0000 7f59a5ffb640 1 --2- 172.21.15.64:0/1112217521 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f59980b6700 0x7f59980b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.657+0000 7f59ad7c9640 1 --2- 172.21.15.64:0/1112217521 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f59980b6700 0x7f59980b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.657+0000 7f59a5ffb640 1 -- 172.21.15.64:0/1112217521 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f5994107cd0 con 0x7f59a8105340 2024-09-21T04:07:07.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.657+0000 7f59b57a0740 1 -- 172.21.15.64:0/3120029191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5a36491b0 msgr2=0x55d5a3a19800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:07.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.657+0000 7f59b57a0740 1 --2- 172.21.15.64:0/3120029191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5a36491b0 0x55d5a3a19800 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f599c009bd0 tx=0x7f599c011810 comp rx=0 tx=0).stop 2024-09-21T04:07:07.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.658+0000 7f59ad7c9640 1 --2- 172.21.15.64:0/1112217521 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f59980b6700 0x7f59980b8c10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f59a00116b0 tx=0x7f59a0009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:07.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.658+0000 7f59b57a0740 1 -- 172.21.15.64:0/3120029191 shutdown_connections 2024-09-21T04:07:07.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.658+0000 7f59b57a0740 1 --2- 172.21.15.64:0/3120029191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a3651390 0x55d5a3a19d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.658+0000 7f59b57a0740 1 --2- 172.21.15.64:0/3120029191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5a36491b0 0x55d5a3a19800 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.658+0000 7f59b57a0740 1 --2- 172.21.15.64:0/3120029191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5a3672070 0x55d5a3a192c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.658+0000 7f59b57a0740 1 -- 172.21.15.64:0/3120029191 >> 172.21.15.64:0/3120029191 conn(0x55d5a3666460 msgr2=0x55d5a3a171b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:07.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.658+0000 7f59b57a0740 1 -- 172.21.15.64:0/3120029191 shutdown_connections 2024-09-21T04:07:07.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.658+0000 7f59b57a0740 1 -- 172.21.15.64:0/3120029191 wait complete. 2024-09-21T04:07:07.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.659+0000 7f59b57a0740 1 -- start start 2024-09-21T04:07:07.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.659+0000 7f59b57a0740 1 Processor -- start 2024-09-21T04:07:07.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.659+0000 7f59b57a0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5a36491b0 0x55d5a3a249d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.659+0000 7f59b57a0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a3651390 0x55d5a3a24f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.659+0000 7f59b57a0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5a3672070 0x55d5a3a25490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.659+0000 7f59b57a0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d5a37e0700 con 0x55d5a3672070 2024-09-21T04:07:07.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.659+0000 7f59b57a0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d5a391a160 con 0x55d5a36491b0 2024-09-21T04:07:07.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.659+0000 7f59b57a0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d5a3a27aa0 con 0x55d5a3651390 2024-09-21T04:07:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.659+0000 7f59acfc8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5a36491b0 0x55d5a3a249d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.659+0000 7f59acfc8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5a36491b0 0x55d5a3a249d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:41468/0 (socket says 172.21.15.64:41468) 2024-09-21T04:07:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.659+0000 7f59acfc8640 1 -- 172.21.15.64:0/1266654437 learned_addr learned my addr 172.21.15.64:0/1266654437 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.660+0000 7f59ad7c9640 1 --2- 172.21.15.64:0/1266654437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5a3672070 0x55d5a3a25490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.660+0000 7f59b57a0740 10 client.0 _ll_register_callbacks cb 0x55d5a39339e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.660+0000 7f59a7fff640 1 --2- 172.21.15.64:0/1266654437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a3651390 0x55d5a3a24f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.660+0000 7f59acfc8640 1 -- 172.21.15.64:0/1266654437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a3651390 msgr2=0x55d5a3a24f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.660+0000 7f59acfc8640 1 --2- 172.21.15.64:0/1266654437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a3651390 0x55d5a3a24f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.660+0000 7f59acfc8640 1 -- 172.21.15.64:0/1266654437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5a3672070 msgr2=0x55d5a3a25490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.660+0000 7f59acfc8640 1 --2- 172.21.15.64:0/1266654437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5a3672070 0x55d5a3a25490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.660+0000 7f59acfc8640 1 -- 172.21.15.64:0/1266654437 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5a3947c90 con 0x55d5a36491b0 2024-09-21T04:07:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.660+0000 7f59acfc8640 1 --2- 172.21.15.64:0/1266654437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5a36491b0 0x55d5a3a249d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f599c005890 tx=0x7f599c019b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab1c5a58f870656f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:07.667 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.660+0000 7f5987fff640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:07:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.660+0000 7f5987fff640 1 -- 172.21.15.64:0/1266654437 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c028020 con 0x55d5a36491b0 2024-09-21T04:07:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.661+0000 7f5987fff640 1 -- 172.21.15.64:0/1266654437 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f599c024470 con 0x55d5a36491b0 2024-09-21T04:07:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.661+0000 7f59b57a0740 1 -- 172.21.15.64:0/1266654437 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d5a3991df0 con 0x55d5a36491b0 2024-09-21T04:07:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.661+0000 7f5987fff640 1 -- 172.21.15.64:0/1266654437 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f599c033050 con 0x55d5a36491b0 2024-09-21T04:07:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.661+0000 7f59b57a0740 10 client.25925 fetch_fsmap learned FSMap version 30 2024-09-21T04:07:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.661+0000 7f59b57a0740 1 -- 172.21.15.64:0/1266654437 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d5a3a33750 con 0x55d5a36491b0 2024-09-21T04:07:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.661+0000 7f5987fff640 1 -- 172.21.15.64:0/1266654437 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d5a3991df0 con 0x55d5a36491b0 2024-09-21T04:07:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.661+0000 7f5987fff640 1 -- 172.21.15.64:0/1266654437 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f599c018070 con 0x55d5a36491b0 2024-09-21T04:07:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.661+0000 7f59b57a0740 10 client.25925 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:07:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.661+0000 7f59b57a0740 10 client.25925 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:07:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.661+0000 7f59b57a0740 20 client.25925 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.662+0000 7f59b57a0740 1 --2- 172.21.15.64:0/1266654437 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d5a36309f0 0x55d5a3a36bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.662+0000 7f59b57a0740 4 client.25925 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:07:07.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.662+0000 7f59b57a0740 1 -- 172.21.15.64:0/1266654437 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5a398d8a0 con 0x55d5a36309f0 2024-09-21T04:07:07.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.662+0000 7f59a7fff640 1 --2- 172.21.15.64:0/1266654437 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d5a36309f0 0x55d5a3a36bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.662+0000 7f5987fff640 10 client.25925 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:07.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.662+0000 7f59a7fff640 1 --2- 172.21.15.64:0/1266654437 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d5a36309f0 0x55d5a3a36bc0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3e0504747ee90de server_cookie=56b9f015b79c07eb in_seq=0 out_seq=0 2024-09-21T04:07:07.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.663+0000 7f5987fff640 1 -- 172.21.15.64:0/1266654437 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d5a398d8a0 con 0x55d5a36309f0 2024-09-21T04:07:07.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.663+0000 7f5987fff640 10 client.25925 handle_command_reply: tid=0 2024-09-21T04:07:07.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.663+0000 7f59b57a0740 1 client.25925 shutdown 2024-09-21T04:07:07.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.663+0000 7f59b57a0740 1 -- 172.21.15.64:0/1266654437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5a36491b0 msgr2=0x55d5a3a249d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:07.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.663+0000 7f59b57a0740 1 --2- 172.21.15.64:0/1266654437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5a36491b0 0x55d5a3a249d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f599c005890 tx=0x7f599c019b90 comp rx=0 tx=0).stop 2024-09-21T04:07:07.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.663+0000 7f59b57a0740 1 -- 172.21.15.64:0/1266654437 shutdown_connections 2024-09-21T04:07:07.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.663+0000 7f59b57a0740 1 --2- 172.21.15.64:0/1266654437 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d5a36309f0 0x55d5a3a36bc0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.663+0000 7f59b57a0740 1 --2- 172.21.15.64:0/1266654437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5a3672070 0x55d5a3a25490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.663+0000 7f59b57a0740 1 --2- 172.21.15.64:0/1266654437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a3651390 0x55d5a3a24f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.663+0000 7f5987fff640 0 client.25925 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:07.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.663+0000 7f59b57a0740 1 --2- 172.21.15.64:0/1266654437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5a36491b0 0x55d5a3a249d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.663+0000 7f59b57a0740 1 -- 172.21.15.64:0/1266654437 >> 172.21.15.64:0/1266654437 conn(0x55d5a3666460 msgr2=0x55d5a3a1c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:07.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.663+0000 7f59b57a0740 1 -- 172.21.15.64:0/1266654437 shutdown_connections 2024-09-21T04:07:07.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.663+0000 7f59b57a0740 1 -- 172.21.15.64:0/1266654437 wait complete. 2024-09-21T04:07:07.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.663+0000 7f59b57a0740 20 client.25925 trim_cache size 0 max 16384 2024-09-21T04:07:07.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.678+0000 7f59b57a0740 1 Processor -- start 2024-09-21T04:07:07.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.678+0000 7f59b57a0740 1 -- start start 2024-09-21T04:07:07.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.678+0000 7f59b57a0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a36491b0 0x55d5a3a268b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.678+0000 7f59b57a0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5a3651390 0x55d5a3a303c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.678+0000 7f59b57a0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5a3672070 0x55d5a3a16760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.681 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.678+0000 7f59b57a0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f59a81b4300 con 0x55d5a3651390 2024-09-21T04:07:07.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.678+0000 7f59b57a0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f59a81b1b40 con 0x55d5a3672070 2024-09-21T04:07:07.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.678+0000 7f59b57a0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f59a81af990 con 0x55d5a36491b0 2024-09-21T04:07:07.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.678+0000 7f59a7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5a3651390 0x55d5a3a303c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.678+0000 7f59a7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5a3651390 0x55d5a3a303c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58444/0 (socket says 172.21.15.64:58444) 2024-09-21T04:07:07.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.678+0000 7f59a7fff640 1 -- 172.21.15.64:0/1433434326 learned_addr learned my addr 172.21.15.64:0/1433434326 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:07.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.678+0000 7f59ad7c9640 1 --2- 172.21.15.64:0/1433434326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5a3672070 0x55d5a3a16760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.678+0000 7f59acfc8640 1 --2- 172.21.15.64:0/1433434326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a36491b0 0x55d5a3a268b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.679+0000 7f59a7fff640 1 -- 172.21.15.64:0/1433434326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a36491b0 msgr2=0x55d5a3a268b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:07.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.679+0000 7f59a7fff640 1 --2- 172.21.15.64:0/1433434326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a36491b0 0x55d5a3a268b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.679+0000 7f59a7fff640 1 -- 172.21.15.64:0/1433434326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5a3672070 msgr2=0x55d5a3a16760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:07.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.679+0000 7f59a7fff640 1 --2- 172.21.15.64:0/1433434326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5a3672070 0x55d5a3a16760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.679+0000 7f59a7fff640 1 -- 172.21.15.64:0/1433434326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5a3a1b5b0 con 0x55d5a3651390 2024-09-21T04:07:07.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.679+0000 7f59a7fff640 1 --2- 172.21.15.64:0/1433434326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5a3651390 0x55d5a3a303c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f59940ca110 tx=0x7f59940b5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460865b48a7a121c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:07.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.679+0000 7f59927fc640 1 -- 172.21.15.64:0/1433434326 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5994049040 con 0x55d5a3651390 2024-09-21T04:07:07.682 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.679+0000 7f59927fc640 1 -- 172.21.15.64:0/1433434326 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59940cd650 con 0x55d5a3651390 2024-09-21T04:07:07.683 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.679+0000 7f59927fc640 1 -- 172.21.15.64:0/1433434326 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5994105610 con 0x55d5a3651390 2024-09-21T04:07:07.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.680+0000 7f59b57a0740 1 -- 172.21.15.64:0/1433434326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5a3651390 msgr2=0x55d5a3a303c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:07.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.680+0000 7f59b57a0740 1 --2- 172.21.15.64:0/1433434326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5a3651390 0x55d5a3a303c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f59940ca110 tx=0x7f59940b5040 comp rx=0 tx=0).stop 2024-09-21T04:07:07.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.680+0000 7f59b57a0740 1 -- 172.21.15.64:0/1433434326 shutdown_connections 2024-09-21T04:07:07.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.680+0000 7f59b57a0740 1 --2- 172.21.15.64:0/1433434326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5a3672070 0x55d5a3a16760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.680+0000 7f59b57a0740 1 --2- 172.21.15.64:0/1433434326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5a3651390 0x55d5a3a303c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.680+0000 7f59b57a0740 1 --2- 172.21.15.64:0/1433434326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a36491b0 0x55d5a3a268b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.680+0000 7f59b57a0740 1 -- 172.21.15.64:0/1433434326 >> 172.21.15.64:0/1433434326 conn(0x55d5a3666460 msgr2=0x55d5a3a2fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:07.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.680+0000 7f59b57a0740 1 -- 172.21.15.64:0/1433434326 shutdown_connections 2024-09-21T04:07:07.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.680+0000 7f59b57a0740 1 -- 172.21.15.64:0/1433434326 wait complete. 2024-09-21T04:07:07.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.680+0000 7f59b57a0740 1 -- start start 2024-09-21T04:07:07.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.681+0000 7f59b57a0740 1 Processor -- start 2024-09-21T04:07:07.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.681+0000 7f59b57a0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5a36491b0 0x55d5a3a2d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.681+0000 7f59b57a0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a3651390 0x55d5a3a28a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.681+0000 7f59b57a0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5a3672070 0x55d5a3a28f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.681+0000 7f59b57a0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f59a81b0000 con 0x55d5a36491b0 2024-09-21T04:07:07.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.681+0000 7f59b57a0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f59a81b1470 con 0x55d5a3672070 2024-09-21T04:07:07.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.681+0000 7f59b57a0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f59a81b06d0 con 0x55d5a3651390 2024-09-21T04:07:07.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.681+0000 7f59b57a0740 10 client.0 _ll_register_callbacks cb 0x7f59a0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:07.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.681+0000 7f59acfc8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5a36491b0 0x55d5a3a2d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.681+0000 7f59acfc8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5a36491b0 0x55d5a3a2d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58458/0 (socket says 172.21.15.64:58458) 2024-09-21T04:07:07.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.681+0000 7f59acfc8640 1 -- 172.21.15.64:0/3169330091 learned_addr learned my addr 172.21.15.64:0/3169330091 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:07.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.681+0000 7f59ad7c9640 1 --2- 172.21.15.64:0/3169330091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5a3672070 0x55d5a3a28f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.681+0000 7f59a7fff640 1 --2- 172.21.15.64:0/3169330091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a3651390 0x55d5a3a28a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.682+0000 7f59acfc8640 1 -- 172.21.15.64:0/3169330091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a3651390 msgr2=0x55d5a3a28a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:07.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.682+0000 7f59acfc8640 1 --2- 172.21.15.64:0/3169330091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a3651390 0x55d5a3a28a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.682+0000 7f59acfc8640 1 -- 172.21.15.64:0/3169330091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5a3672070 msgr2=0x55d5a3a28f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:07.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.682+0000 7f59acfc8640 1 --2- 172.21.15.64:0/3169330091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5a3672070 0x55d5a3a28f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.682+0000 7f59acfc8640 1 -- 172.21.15.64:0/3169330091 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5a3a29710 con 0x55d5a36491b0 2024-09-21T04:07:07.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.682+0000 7f59ad7c9640 1 --2- 172.21.15.64:0/3169330091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5a3672070 0x55d5a3a28f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:07:07.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.682+0000 7f59acfc8640 1 --2- 172.21.15.64:0/3169330091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5a36491b0 0x55d5a3a2d520 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f599c011850 tx=0x7f599c019860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf28381cd68ca8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:07.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.682+0000 7f5987fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:07:07.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.682+0000 7f5987fff640 1 -- 172.21.15.64:0/3169330091 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c028070 con 0x55d5a36491b0 2024-09-21T04:07:07.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.682+0000 7f5987fff640 1 -- 172.21.15.64:0/3169330091 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f599c0052b0 con 0x55d5a36491b0 2024-09-21T04:07:07.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.682+0000 7f59b57a0740 1 -- 172.21.15.64:0/3169330091 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59940cd650 con 0x55d5a36491b0 2024-09-21T04:07:07.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.682+0000 7f5987fff640 1 -- 172.21.15.64:0/3169330091 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f599c01e040 con 0x55d5a36491b0 2024-09-21T04:07:07.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.682+0000 7f59b57a0740 10 client.16221 fetch_fsmap learned FSMap version 30 2024-09-21T04:07:07.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.682+0000 7f59b57a0740 1 -- 172.21.15.64:0/3169330091 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d5a3aa14a0 con 0x55d5a36491b0 2024-09-21T04:07:07.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.683+0000 7f5987fff640 1 -- 172.21.15.64:0/3169330091 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f599c0054b0 con 0x55d5a36491b0 2024-09-21T04:07:07.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.683+0000 7f5987fff640 1 -- 172.21.15.64:0/3169330091 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f599c018040 con 0x55d5a36491b0 2024-09-21T04:07:07.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.683+0000 7f59b57a0740 10 client.16221 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:07:07.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.683+0000 7f59b57a0740 10 client.16221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:07:07.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.683+0000 7f59b57a0740 20 client.16221 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:07.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.683+0000 7f59b57a0740 1 --2- 172.21.15.64:0/3169330091 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d5a3605da0 0x55d5a3aa3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.683+0000 7f59b57a0740 4 client.16221 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:07:07.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.683+0000 7f59b57a0740 1 -- 172.21.15.64:0/3169330091 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f59940cd650 con 0x55d5a3605da0 2024-09-21T04:07:07.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.683+0000 7f59a7fff640 1 --2- 172.21.15.64:0/3169330091 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d5a3605da0 0x55d5a3aa3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.684+0000 7f5987fff640 10 client.16221 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:07.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.684+0000 7f59a7fff640 1 --2- 172.21.15.64:0/3169330091 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d5a3605da0 0x55d5a3aa3d10 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4c975e5f99420d4 server_cookie=139c8b5bc690b25d in_seq=0 out_seq=0 2024-09-21T04:07:07.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.684+0000 7f5987fff640 1 -- 172.21.15.64:0/3169330091 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f59940cd650 con 0x55d5a3605da0 2024-09-21T04:07:07.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.684+0000 7f5987fff640 10 client.16221 handle_command_reply: tid=0 2024-09-21T04:07:07.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.684+0000 7f59b57a0740 1 client.16221 shutdown 2024-09-21T04:07:07.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.684+0000 7f59b57a0740 1 -- 172.21.15.64:0/3169330091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5a36491b0 msgr2=0x55d5a3a2d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:07.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.684+0000 7f59b57a0740 1 --2- 172.21.15.64:0/3169330091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5a36491b0 0x55d5a3a2d520 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f599c011850 tx=0x7f599c019860 comp rx=0 tx=0).stop 2024-09-21T04:07:07.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.684+0000 7f59b57a0740 1 -- 172.21.15.64:0/3169330091 shutdown_connections 2024-09-21T04:07:07.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.684+0000 7f59b57a0740 1 --2- 172.21.15.64:0/3169330091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d5a3672070 0x55d5a3a28f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.684+0000 7f59b57a0740 1 --2- 172.21.15.64:0/3169330091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d5a3651390 0x55d5a3a28a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.684+0000 7f59b57a0740 1 --2- 172.21.15.64:0/3169330091 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d5a3605da0 0x55d5a3aa3d10 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.684+0000 7f59b57a0740 1 --2- 172.21.15.64:0/3169330091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d5a36491b0 0x55d5a3a2d520 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.684+0000 7f59b57a0740 1 -- 172.21.15.64:0/3169330091 >> 172.21.15.64:0/3169330091 conn(0x55d5a3666460 msgr2=0x55d5a3a30670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:07.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.684+0000 7f5987fff640 0 client.16221 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:07.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.684+0000 7f59b57a0740 1 -- 172.21.15.64:0/3169330091 shutdown_connections 2024-09-21T04:07:07.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.684+0000 7f59b57a0740 1 -- 172.21.15.64:0/3169330091 wait complete. 2024-09-21T04:07:07.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.685+0000 7f59b57a0740 20 client.16221 trim_cache size 0 max 16384 2024-09-21T04:07:07.692 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:07:07.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.685+0000 7f59af22c640 1 -- 172.21.15.64:0/1112217521 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f59980b6700 msgr2=0x7f59980b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:07.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.685+0000 7f59af22c640 1 --2- 172.21.15.64:0/1112217521 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f59980b6700 0x7f59980b8c10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f59a00116b0 tx=0x7f59a0009500 comp rx=0 tx=0).stop 2024-09-21T04:07:07.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.685+0000 7f59af22c640 1 -- 172.21.15.64:0/1112217521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59a8105340 msgr2=0x7f59a8102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:07.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.685+0000 7f59af22c640 1 --2- 172.21.15.64:0/1112217521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59a8105340 0x7f59a8102c70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5994005ab0 tx=0x7f5994003b90 comp rx=0 tx=0).stop 2024-09-21T04:07:07.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.685+0000 7f59af22c640 1 -- 172.21.15.64:0/1112217521 shutdown_connections 2024-09-21T04:07:07.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.685+0000 7f59af22c640 1 --2- 172.21.15.64:0/1112217521 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f59980b6700 0x7f59980b8c10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.685+0000 7f59af22c640 1 --2- 172.21.15.64:0/1112217521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59a810de50 0x7f59a81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.685+0000 7f59af22c640 1 --2- 172.21.15.64:0/1112217521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59a8105340 0x7f59a8102c70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.685+0000 7f59af22c640 1 --2- 172.21.15.64:0/1112217521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59a81049f0 0x7f59a8102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.685+0000 7f59af22c640 1 -- 172.21.15.64:0/1112217521 >> 172.21.15.64:0/1112217521 conn(0x7f59a8079750 msgr2=0x7f59a8100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:07.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.686+0000 7f59af22c640 1 -- 172.21.15.64:0/1112217521 shutdown_connections 2024-09-21T04:07:07.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:07.686+0000 7f59af22c640 1 -- 172.21.15.64:0/1112217521 wait complete. 2024-09-21T04:07:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.695+0000 7f379bb2b640 1 -- 172.21.15.64:0/1457374197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3794108310 msgr2=0x7f3794108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.695+0000 7f379bb2b640 1 --2- 172.21.15.64:0/1457374197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3794108310 0x7f3794108720 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3784009900 tx=0x7f378402a410 comp rx=0 tx=0).stop 2024-09-21T04:07:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.695+0000 7f379bb2b640 1 -- 172.21.15.64:0/1457374197 shutdown_connections 2024-09-21T04:07:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.695+0000 7f379bb2b640 1 --2- 172.21.15.64:0/1457374197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3794108c60 0x7f379410f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.695+0000 7f379bb2b640 1 --2- 172.21.15.64:0/1457374197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3794108310 0x7f3794108720 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.695+0000 7f379bb2b640 1 --2- 172.21.15.64:0/1457374197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f37941029e0 0x7f3794102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.695+0000 7f379bb2b640 1 -- 172.21.15.64:0/1457374197 >> 172.21.15.64:0/1457374197 conn(0x7f379407a6e0 msgr2=0x7f379407aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.695+0000 7f379bb2b640 1 -- 172.21.15.64:0/1457374197 shutdown_connections 2024-09-21T04:07:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.695+0000 7f379bb2b640 1 -- 172.21.15.64:0/1457374197 wait complete. 2024-09-21T04:07:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.696+0000 7f379bb2b640 1 Processor -- start 2024-09-21T04:07:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.696+0000 7f379bb2b640 1 -- start start 2024-09-21T04:07:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.696+0000 7f379bb2b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37941029e0 0x7f3794199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.696+0000 7f379bb2b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3794108310 0x7f37941998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.696+0000 7f379bb2b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3794108c60 0x7f379419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.697+0000 7f379bb2b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3794112920 con 0x7f3794108c60 2024-09-21T04:07:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.697+0000 7f379bb2b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f37941127a0 con 0x7f3794108310 2024-09-21T04:07:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.697+0000 7f379bb2b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3794112aa0 con 0x7f37941029e0 2024-09-21T04:07:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.697+0000 7f37990c6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3794108310 0x7f37941998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.697+0000 7f37998c7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37941029e0 0x7f3794199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.697+0000 7f379a0c8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3794108c60 0x7f379419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.697+0000 7f37990c6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3794108310 0x7f37941998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:41494/0 (socket says 172.21.15.64:41494) 2024-09-21T04:07:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.697+0000 7f37998c7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37941029e0 0x7f3794199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:54738/0 (socket says 172.21.15.64:54738) 2024-09-21T04:07:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.697+0000 7f37990c6640 1 -- 172.21.15.64:0/368727630 learned_addr learned my addr 172.21.15.64:0/368727630 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.697+0000 7f37998c7640 1 -- 172.21.15.64:0/368727630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3794108310 msgr2=0x7f37941998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.697+0000 7f37998c7640 1 --2- 172.21.15.64:0/368727630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3794108310 0x7f37941998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.697+0000 7f37998c7640 1 -- 172.21.15.64:0/368727630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3794108c60 msgr2=0x7f379419eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.697+0000 7f37998c7640 1 --2- 172.21.15.64:0/368727630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3794108c60 0x7f379419eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.697+0000 7f37998c7640 1 -- 172.21.15.64:0/368727630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379419f270 con 0x7f37941029e0 2024-09-21T04:07:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.697+0000 7f37990c6640 1 --2- 172.21.15.64:0/368727630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3794108310 0x7f37941998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:07:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.697+0000 7f37998c7640 1 --2- 172.21.15.64:0/368727630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37941029e0 0x7f3794199340 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f379000e9f0 tx=0x7f379000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.697+0000 7f378affd640 1 -- 172.21.15.64:0/368727630 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379000ccc0 con 0x7f37941029e0 2024-09-21T04:07:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.698+0000 7f379bb2b640 1 -- 172.21.15.64:0/368727630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f379419f560 con 0x7f37941029e0 2024-09-21T04:07:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.698+0000 7f378affd640 1 -- 172.21.15.64:0/368727630 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37900108e0 con 0x7f37941029e0 2024-09-21T04:07:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.698+0000 7f378affd640 1 -- 172.21.15.64:0/368727630 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3790018880 con 0x7f37941029e0 2024-09-21T04:07:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.698+0000 7f379bb2b640 1 -- 172.21.15.64:0/368727630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f379406b5a0 con 0x7f37941029e0 2024-09-21T04:07:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.700+0000 7f37a2a9b740 1 Processor -- start 2024-09-21T04:07:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.701+0000 7f37a2a9b740 1 -- start start 2024-09-21T04:07:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.701+0000 7f378affd640 1 -- 172.21.15.64:0/368727630 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f3790018a20 con 0x7f37941029e0 2024-09-21T04:07:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.701+0000 7f37a2a9b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579144b11b0 0x5579148569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.701+0000 7f37a2a9b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579144882f0 0x557914856ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.701+0000 7f37a2a9b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579144904d0 0x557914857430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.701+0000 7f37a2a9b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5579147cb6f0 con 0x5579144b11b0 2024-09-21T04:07:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.701+0000 7f37a2a9b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5579147bab20 con 0x5579144904d0 2024-09-21T04:07:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.701+0000 7f37a2a9b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5579147be930 con 0x5579144882f0 2024-09-21T04:07:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.701+0000 7f379a0c8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579144b11b0 0x5579148569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.701+0000 7f37990c6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579144904d0 0x557914857430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.701+0000 7f37998c7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579144882f0 0x557914856ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.701+0000 7f37990c6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579144904d0 0x557914857430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:41498/0 (socket says 172.21.15.64:41498) 2024-09-21T04:07:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.701+0000 7f37990c6640 1 -- 172.21.15.64:0/956341485 learned_addr learned my addr 172.21.15.64:0/956341485 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.701+0000 7f37990c6640 1 -- 172.21.15.64:0/956341485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579144882f0 msgr2=0x557914856ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.701+0000 7f37990c6640 1 --2- 172.21.15.64:0/956341485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579144882f0 0x557914856ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.701+0000 7f37990c6640 1 -- 172.21.15.64:0/956341485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579144b11b0 msgr2=0x5579148569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.701+0000 7f37990c6640 1 --2- 172.21.15.64:0/956341485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579144b11b0 0x5579148569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.702+0000 7f37990c6640 1 -- 172.21.15.64:0/956341485 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557914780530 con 0x5579144904d0 2024-09-21T04:07:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.702+0000 7f37990c6640 1 --2- 172.21.15.64:0/956341485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579144904d0 0x557914857430 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3784006010 tx=0x7f3784005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ac0d32867ceb77 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.702+0000 7f378affd640 1 --2- 172.21.15.64:0/368727630 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f37800b6680 0x7f37800b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.702+0000 7f377ffff640 1 -- 172.21.15.64:0/956341485 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784035040 con 0x5579144904d0 2024-09-21T04:07:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.702+0000 7f377ffff640 1 -- 172.21.15.64:0/956341485 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3784004030 con 0x5579144904d0 2024-09-21T04:07:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.702+0000 7f377ffff640 1 -- 172.21.15.64:0/956341485 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37840055b0 con 0x5579144904d0 2024-09-21T04:07:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.702+0000 7f379a0c8640 1 --2- 172.21.15.64:0/368727630 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f37800b6680 0x7f37800b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.702+0000 7f378affd640 1 -- 172.21.15.64:0/368727630 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f37900e7ab0 con 0x7f37941029e0 2024-09-21T04:07:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.703+0000 7f37a2a9b740 1 -- 172.21.15.64:0/956341485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579144904d0 msgr2=0x557914857430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.703+0000 7f37a2a9b740 1 --2- 172.21.15.64:0/956341485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579144904d0 0x557914857430 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3784006010 tx=0x7f3784005d90 comp rx=0 tx=0).stop 2024-09-21T04:07:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.703+0000 7f379a0c8640 1 --2- 172.21.15.64:0/368727630 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f37800b6680 0x7f37800b8b90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f378c005fd0 tx=0x7f378c009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.703+0000 7f37a2a9b740 1 -- 172.21.15.64:0/956341485 shutdown_connections 2024-09-21T04:07:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.703+0000 7f37a2a9b740 1 --2- 172.21.15.64:0/956341485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579144904d0 0x557914857430 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.703+0000 7f37a2a9b740 1 --2- 172.21.15.64:0/956341485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579144882f0 0x557914856ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.703+0000 7f37a2a9b740 1 --2- 172.21.15.64:0/956341485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579144b11b0 0x5579148569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.703+0000 7f37a2a9b740 1 -- 172.21.15.64:0/956341485 >> 172.21.15.64:0/956341485 conn(0x5579144a5300 msgr2=0x5579148548a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.703+0000 7f37a2a9b740 1 -- 172.21.15.64:0/956341485 shutdown_connections 2024-09-21T04:07:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.703+0000 7f37a2a9b740 1 -- 172.21.15.64:0/956341485 wait complete. 2024-09-21T04:07:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.704+0000 7f37a2a9b740 1 -- start start 2024-09-21T04:07:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.704+0000 7f37a2a9b740 1 Processor -- start 2024-09-21T04:07:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.705+0000 7f37a2a9b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579144882f0 0x557914866400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.705+0000 7f37a2a9b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579144904d0 0x557914866940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.705+0000 7f37a2a9b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579144b11b0 0x5579148727e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.705+0000 7f37a2a9b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5579146f2ab0 con 0x5579144b11b0 2024-09-21T04:07:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.705+0000 7f37a2a9b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557914760730 con 0x5579144882f0 2024-09-21T04:07:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.705+0000 7f37a2a9b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5579148694b0 con 0x5579144904d0 2024-09-21T04:07:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.705+0000 7f37998c7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579144882f0 0x557914866400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.705+0000 7f37990c6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579144904d0 0x557914866940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.705+0000 7f379a0c8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579144b11b0 0x5579148727e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.705+0000 7f37990c6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579144904d0 0x557914866940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:54752/0 (socket says 172.21.15.64:54752) 2024-09-21T04:07:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.705+0000 7f37998c7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579144882f0 0x557914866400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:41510/0 (socket says 172.21.15.64:41510) 2024-09-21T04:07:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.705+0000 7f37990c6640 1 -- 172.21.15.64:0/790852963 learned_addr learned my addr 172.21.15.64:0/790852963 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.705+0000 7f37a2a9b740 10 client.0 _ll_register_callbacks cb 0x5579147b46b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.705+0000 7f37990c6640 1 -- 172.21.15.64:0/790852963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579144882f0 msgr2=0x557914866400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.705+0000 7f37990c6640 1 --2- 172.21.15.64:0/790852963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579144882f0 0x557914866400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.705+0000 7f37990c6640 1 -- 172.21.15.64:0/790852963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579144b11b0 msgr2=0x5579148727e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.705+0000 7f37990c6640 1 --2- 172.21.15.64:0/790852963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579144b11b0 0x5579148727e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.705+0000 7f37990c6640 1 -- 172.21.15.64:0/790852963 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55791451c790 con 0x5579144904d0 2024-09-21T04:07:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.705+0000 7f37998c7640 1 --2- 172.21.15.64:0/790852963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579144882f0 0x557914866400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:07:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.705+0000 7f37990c6640 1 --2- 172.21.15.64:0/790852963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579144904d0 0x557914866940 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3784004550 tx=0x7f3784005130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f537779fc44341eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.705+0000 7f377d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-21T04:07:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.706+0000 7f377d7fa640 1 -- 172.21.15.64:0/790852963 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784051060 con 0x5579144904d0 2024-09-21T04:07:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.706+0000 7f37a2a9b740 1 -- 172.21.15.64:0/790852963 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3784004030 con 0x5579144904d0 2024-09-21T04:07:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.706+0000 7f377d7fa640 1 -- 172.21.15.64:0/790852963 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f3784005350 con 0x5579144904d0 2024-09-21T04:07:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.706+0000 7f377d7fa640 1 -- 172.21.15.64:0/790852963 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3784048040 con 0x5579144904d0 2024-09-21T04:07:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.706+0000 7f37a2a9b740 10 client.25825 fetch_fsmap learned FSMap version 30 2024-09-21T04:07:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.706+0000 7f37a2a9b740 1 -- 172.21.15.64:0/790852963 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557914595620 con 0x5579144904d0 2024-09-21T04:07:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.706+0000 7f377d7fa640 1 -- 172.21.15.64:0/790852963 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3784005550 con 0x5579144904d0 2024-09-21T04:07:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.706+0000 7f377d7fa640 1 -- 172.21.15.64:0/790852963 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f378404b070 con 0x5579144904d0 2024-09-21T04:07:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.706+0000 7f37a2a9b740 10 client.25825 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:07:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.706+0000 7f37a2a9b740 10 client.25825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:07:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.706+0000 7f37a2a9b740 20 client.25825 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.706+0000 7f37a2a9b740 1 --2- 172.21.15.64:0/790852963 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55791446fd80 0x557914878310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.706+0000 7f37a2a9b740 4 client.25825 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:07:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.706+0000 7f37a2a9b740 1 -- 172.21.15.64:0/790852963 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3784004030 con 0x55791446fd80 2024-09-21T04:07:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.707+0000 7f37998c7640 1 --2- 172.21.15.64:0/790852963 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55791446fd80 0x557914878310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.707+0000 7f377d7fa640 10 client.25825 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:07.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.707+0000 7f37998c7640 1 --2- 172.21.15.64:0/790852963 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55791446fd80 0x557914878310 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26e94beda14b265d server_cookie=66452c89f5cb2f0a in_seq=0 out_seq=0 2024-09-21T04:07:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.708+0000 7f377d7fa640 1 -- 172.21.15.64:0/790852963 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f3784004030 con 0x55791446fd80 2024-09-21T04:07:07.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.708+0000 7f377d7fa640 10 client.25825 handle_command_reply: tid=0 2024-09-21T04:07:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.708+0000 7f37a2a9b740 1 client.25825 shutdown 2024-09-21T04:07:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.708+0000 7f37a2a9b740 1 -- 172.21.15.64:0/790852963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579144904d0 msgr2=0x557914866940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.708+0000 7f37a2a9b740 1 --2- 172.21.15.64:0/790852963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579144904d0 0x557914866940 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3784004550 tx=0x7f3784005130 comp rx=0 tx=0).stop 2024-09-21T04:07:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.708+0000 7f37a2a9b740 1 -- 172.21.15.64:0/790852963 shutdown_connections 2024-09-21T04:07:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.708+0000 7f37a2a9b740 1 --2- 172.21.15.64:0/790852963 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55791446fd80 0x557914878310 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.708+0000 7f377d7fa640 0 client.25825 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.708+0000 7f37a2a9b740 1 --2- 172.21.15.64:0/790852963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579144b11b0 0x5579148727e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.708+0000 7f37a2a9b740 1 --2- 172.21.15.64:0/790852963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5579144904d0 0x557914866940 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.708+0000 7f37a2a9b740 1 --2- 172.21.15.64:0/790852963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579144882f0 0x557914866400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.708+0000 7f37a2a9b740 1 -- 172.21.15.64:0/790852963 >> 172.21.15.64:0/790852963 conn(0x5579144a5300 msgr2=0x557914859b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.708+0000 7f37a2a9b740 1 -- 172.21.15.64:0/790852963 shutdown_connections 2024-09-21T04:07:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.708+0000 7f37a2a9b740 1 -- 172.21.15.64:0/790852963 wait complete. 2024-09-21T04:07:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.709+0000 7f37a2a9b740 20 client.25825 trim_cache size 0 max 16384 2024-09-21T04:07:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.727+0000 7f37a2a9b740 1 Processor -- start 2024-09-21T04:07:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.727+0000 7f37a2a9b740 1 -- start start 2024-09-21T04:07:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.727+0000 7f37a2a9b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55791446fd80 0x557914868030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.727+0000 7f37a2a9b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579144882f0 0x5579148726e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.727+0000 7f37a2a9b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579144904d0 0x5579148635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.727+0000 7f37a2a9b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f37941afed0 con 0x5579144904d0 2024-09-21T04:07:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.727+0000 7f37a2a9b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f37941ad710 con 0x5579144882f0 2024-09-21T04:07:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.727+0000 7f37a2a9b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f37941ab560 con 0x55791446fd80 2024-09-21T04:07:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.728+0000 7f379a0c8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579144904d0 0x5579148635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.727+0000 7f37990c6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579144882f0 0x5579148726e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.728+0000 7f379a0c8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579144904d0 0x5579148635d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58500/0 (socket says 172.21.15.64:58500) 2024-09-21T04:07:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.728+0000 7f379a0c8640 1 -- 172.21.15.64:0/201095754 learned_addr learned my addr 172.21.15.64:0/201095754 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.728+0000 7f37998c7640 1 --2- 172.21.15.64:0/201095754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55791446fd80 0x557914868030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.728+0000 7f37990c6640 1 -- 172.21.15.64:0/201095754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55791446fd80 msgr2=0x557914868030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.728+0000 7f37990c6640 1 --2- 172.21.15.64:0/201095754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55791446fd80 0x557914868030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.728+0000 7f37990c6640 1 -- 172.21.15.64:0/201095754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579144904d0 msgr2=0x5579148635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.728+0000 7f37990c6640 1 --2- 172.21.15.64:0/201095754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579144904d0 0x5579148635d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.728+0000 7f37990c6640 1 -- 172.21.15.64:0/201095754 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557914859790 con 0x5579144882f0 2024-09-21T04:07:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.728+0000 7f379a0c8640 1 --2- 172.21.15.64:0/201095754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579144904d0 0x5579148635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:07:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.728+0000 7f37990c6640 1 --2- 172.21.15.64:0/201095754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579144882f0 0x5579148726e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3784004f10 tx=0x7f378402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ccaac32a8b44f57 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.728+0000 7f377f7fe640 1 -- 172.21.15.64:0/201095754 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784035040 con 0x5579144882f0 2024-09-21T04:07:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.728+0000 7f377f7fe640 1 -- 172.21.15.64:0/201095754 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3784004770 con 0x5579144882f0 2024-09-21T04:07:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.728+0000 7f377f7fe640 1 -- 172.21.15.64:0/201095754 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784031850 con 0x5579144882f0 2024-09-21T04:07:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.729+0000 7f37a2a9b740 1 -- 172.21.15.64:0/201095754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579144882f0 msgr2=0x5579148726e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.729+0000 7f37a2a9b740 1 --2- 172.21.15.64:0/201095754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579144882f0 0x5579148726e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3784004f10 tx=0x7f378402a410 comp rx=0 tx=0).stop 2024-09-21T04:07:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.729+0000 7f37a2a9b740 1 -- 172.21.15.64:0/201095754 shutdown_connections 2024-09-21T04:07:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.729+0000 7f37a2a9b740 1 --2- 172.21.15.64:0/201095754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579144904d0 0x5579148635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.729+0000 7f37a2a9b740 1 --2- 172.21.15.64:0/201095754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579144882f0 0x5579148726e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.729+0000 7f37a2a9b740 1 --2- 172.21.15.64:0/201095754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55791446fd80 0x557914868030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.729+0000 7f37a2a9b740 1 -- 172.21.15.64:0/201095754 >> 172.21.15.64:0/201095754 conn(0x5579144a5300 msgr2=0x557914867920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.729+0000 7f37a2a9b740 1 -- 172.21.15.64:0/201095754 shutdown_connections 2024-09-21T04:07:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.729+0000 7f37a2a9b740 1 -- 172.21.15.64:0/201095754 wait complete. 2024-09-21T04:07:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.730+0000 7f37a2a9b740 1 -- start start 2024-09-21T04:07:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.730+0000 7f37a2a9b740 1 Processor -- start 2024-09-21T04:07:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.730+0000 7f37a2a9b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55791446fd80 0x55791486eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.730+0000 7f37a2a9b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579144882f0 0x55791486f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.730+0000 7f37a2a9b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579144904d0 0x55791486a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.730+0000 7f379a0c8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579144904d0 0x55791486a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.730+0000 7f37a2a9b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f37941abbd0 con 0x5579144904d0 2024-09-21T04:07:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.730+0000 7f37a2a9b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f37941ad040 con 0x5579144882f0 2024-09-21T04:07:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.730+0000 7f37a2a9b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f37941ac2a0 con 0x55791446fd80 2024-09-21T04:07:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.730+0000 7f379a0c8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579144904d0 0x55791486a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58508/0 (socket says 172.21.15.64:58508) 2024-09-21T04:07:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.730+0000 7f379a0c8640 1 -- 172.21.15.64:0/2117901649 learned_addr learned my addr 172.21.15.64:0/2117901649 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.730+0000 7f37998c7640 1 --2- 172.21.15.64:0/2117901649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55791446fd80 0x55791486eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.730+0000 7f37990c6640 1 --2- 172.21.15.64:0/2117901649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579144882f0 0x55791486f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.731+0000 7f37a2a9b740 10 client.0 _ll_register_callbacks cb 0x7f378c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.731+0000 7f379a0c8640 1 -- 172.21.15.64:0/2117901649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55791446fd80 msgr2=0x55791486eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.731+0000 7f379a0c8640 1 --2- 172.21.15.64:0/2117901649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55791446fd80 0x55791486eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.731+0000 7f379a0c8640 1 -- 172.21.15.64:0/2117901649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579144882f0 msgr2=0x55791486f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.731+0000 7f379a0c8640 1 --2- 172.21.15.64:0/2117901649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579144882f0 0x55791486f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:07.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.731+0000 7f379a0c8640 1 -- 172.21.15.64:0/2117901649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55791486a930 con 0x5579144904d0 2024-09-21T04:07:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.731+0000 7f37998c7640 1 --2- 172.21.15.64:0/2117901649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55791446fd80 0x55791486eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:07:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.731+0000 7f379a0c8640 1 --2- 172.21.15.64:0/2117901649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579144904d0 0x55791486a1c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f378c004190 tx=0x7f378c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81116450b55e80b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.731+0000 7f377d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:07:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.731+0000 7f377d7fa640 1 -- 172.21.15.64:0/2117901649 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c02e060 con 0x5579144904d0 2024-09-21T04:07:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.731+0000 7f377d7fa640 1 -- 172.21.15.64:0/2117901649 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f378c00fa60 con 0x5579144904d0 2024-09-21T04:07:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.731+0000 7f37a2a9b740 1 -- 172.21.15.64:0/2117901649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3784004770 con 0x5579144904d0 2024-09-21T04:07:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.732+0000 7f377d7fa640 1 -- 172.21.15.64:0/2117901649 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f378c00fc60 con 0x5579144904d0 2024-09-21T04:07:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.732+0000 7f37a2a9b740 10 client.16239 fetch_fsmap learned FSMap version 30 2024-09-21T04:07:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.732+0000 7f37a2a9b740 1 -- 172.21.15.64:0/2117901649 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5579148e26e0 con 0x5579144904d0 2024-09-21T04:07:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.732+0000 7f377d7fa640 1 -- 172.21.15.64:0/2117901649 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f378c00fe00 con 0x5579144904d0 2024-09-21T04:07:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.732+0000 7f377d7fa640 1 -- 172.21.15.64:0/2117901649 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f378c01e590 con 0x5579144904d0 2024-09-21T04:07:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.732+0000 7f37a2a9b740 10 client.16239 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:07:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.732+0000 7f37a2a9b740 10 client.16239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:07:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.732+0000 7f37a2a9b740 20 client.16239 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.732+0000 7f37a2a9b740 1 --2- 172.21.15.64:0/2117901649 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5579144b11b0 0x5579148e4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.732+0000 7f37a2a9b740 4 client.16239 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:07:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.732+0000 7f37a2a9b740 1 -- 172.21.15.64:0/2117901649 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f3784004770 con 0x5579144b11b0 2024-09-21T04:07:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.733+0000 7f37998c7640 1 --2- 172.21.15.64:0/2117901649 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5579144b11b0 0x5579148e4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.733+0000 7f377d7fa640 10 client.16239 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:07.733+0000 7f37998c7640 1 --2- 172.21.15.64:0/2117901649 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5579144b11b0 0x5579148e4f30 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b85df28f5e4dabc server_cookie=474b82103f9b84 in_seq=0 out_seq=0 2024-09-21T04:07:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:08.374+0000 7f377d7fa640 1 -- 172.21.15.64:0/2117901649 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f3784004770 con 0x5579144b11b0 2024-09-21T04:07:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:08.374+0000 7f377d7fa640 10 client.16239 handle_command_reply: tid=0 2024-09-21T04:07:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:08.374+0000 7f37a2a9b740 1 client.16239 shutdown 2024-09-21T04:07:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:08.374+0000 7f37a2a9b740 1 -- 172.21.15.64:0/2117901649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579144904d0 msgr2=0x55791486a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:08.374+0000 7f37a2a9b740 1 --2- 172.21.15.64:0/2117901649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579144904d0 0x55791486a1c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f378c004190 tx=0x7f378c004a40 comp rx=0 tx=0).stop 2024-09-21T04:07:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:08.374+0000 7f37a2a9b740 1 -- 172.21.15.64:0/2117901649 shutdown_connections 2024-09-21T04:07:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:08.374+0000 7f37a2a9b740 1 --2- 172.21.15.64:0/2117901649 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5579144b11b0 0x5579148e4f30 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:08.374+0000 7f37a2a9b740 1 --2- 172.21.15.64:0/2117901649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5579144904d0 0x55791486a1c0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:08.374+0000 7f37a2a9b740 1 --2- 172.21.15.64:0/2117901649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579144882f0 0x55791486f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:08.377 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:08.374+0000 7f37a2a9b740 1 --2- 172.21.15.64:0/2117901649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55791446fd80 0x55791486eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:08.374+0000 7f37a2a9b740 1 -- 172.21.15.64:0/2117901649 >> 172.21.15.64:0/2117901649 conn(0x5579144a5300 msgr2=0x5579148579e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:08.374+0000 7f377d7fa640 0 client.16239 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:08.374+0000 7f37a2a9b740 1 -- 172.21.15.64:0/2117901649 shutdown_connections 2024-09-21T04:07:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:08.375+0000 7f37a2a9b740 1 -- 172.21.15.64:0/2117901649 wait complete. 2024-09-21T04:07:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:08.375+0000 7f37a2a9b740 20 client.16239 trim_cache size 0 max 16384 2024-09-21T04:07:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:07:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:07:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:07:08.378 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:07:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:08.375+0000 7f379bb2b640 1 -- 172.21.15.64:0/368727630 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f37800b6680 msgr2=0x7f37800b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:08.375+0000 7f379bb2b640 1 --2- 172.21.15.64:0/368727630 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f37800b6680 0x7f37800b8b90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f378c005fd0 tx=0x7f378c009500 comp rx=0 tx=0).stop 2024-09-21T04:07:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:08.375+0000 7f379bb2b640 1 -- 172.21.15.64:0/368727630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37941029e0 msgr2=0x7f3794199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:08.375+0000 7f379bb2b640 1 --2- 172.21.15.64:0/368727630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37941029e0 0x7f3794199340 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f379000e9f0 tx=0x7f379000eeb0 comp rx=0 tx=0).stop 2024-09-21T04:07:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:08.376+0000 7f379bb2b640 1 -- 172.21.15.64:0/368727630 shutdown_connections 2024-09-21T04:07:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:08.376+0000 7f379bb2b640 1 --2- 172.21.15.64:0/368727630 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f37800b6680 0x7f37800b8b90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:08.376+0000 7f379bb2b640 1 --2- 172.21.15.64:0/368727630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3794108c60 0x7f379419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:08.376+0000 7f379bb2b640 1 --2- 172.21.15.64:0/368727630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3794108310 0x7f37941998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:08.376+0000 7f379bb2b640 1 --2- 172.21.15.64:0/368727630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37941029e0 0x7f3794199340 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:08.376+0000 7f379bb2b640 1 -- 172.21.15.64:0/368727630 >> 172.21.15.64:0/368727630 conn(0x7f379407a6e0 msgr2=0x7f37941063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:08.376+0000 7f379bb2b640 1 -- 172.21.15.64:0/368727630 shutdown_connections 2024-09-21T04:07:08.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:08.376+0000 7f379bb2b640 1 -- 172.21.15.64:0/368727630 wait complete. 2024-09-21T04:07:08.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:08 smithi119 ceph-mon[34811]: pgmap v414: 97 pgs: 97 active+clean; 386 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.3 MiB/s wr, 490 op/s 2024-09-21T04:07:08.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:08 smithi064 ceph-mon[31180]: pgmap v414: 97 pgs: 97 active+clean; 386 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.3 MiB/s wr, 490 op/s 2024-09-21T04:07:08.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:08 smithi131 ceph-mon[35193]: pgmap v414: 97 pgs: 97 active+clean; 386 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.3 MiB/s wr, 490 op/s 2024-09-21T04:07:09.531 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T04:07:09.531 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T04:07:10.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:10 smithi119 ceph-mon[34811]: pgmap v415: 97 pgs: 97 active+clean; 401 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.2 MiB/s wr, 490 op/s 2024-09-21T04:07:10.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:10 smithi064 ceph-mon[31180]: pgmap v415: 97 pgs: 97 active+clean; 401 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.2 MiB/s wr, 490 op/s 2024-09-21T04:07:10.774 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:07:10.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:10 smithi131 ceph-mon[35193]: pgmap v415: 97 pgs: 97 active+clean; 401 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.2 MiB/s wr, 490 op/s 2024-09-21T04:07:11.173 INFO:tasks.workunit.client.0.smithi064.stdout: 70 659948 282 453270 145 362712 690 2024-09-21T04:07:11.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.322+0000 7f7ebd8ec640 1 -- 172.21.15.64:0/1804380713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb81083a0 msgr2=0x7f7eb81087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:11.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.322+0000 7f7ebd8ec640 1 --2- 172.21.15.64:0/1804380713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb81083a0 0x7f7eb81087b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009900 tx=0x7f7ea002a410 comp rx=0 tx=0).stop 2024-09-21T04:07:11.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.322+0000 7f7ebd8ec640 1 -- 172.21.15.64:0/1804380713 shutdown_connections 2024-09-21T04:07:11.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.322+0000 7f7ebd8ec640 1 --2- 172.21.15.64:0/1804380713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7eb8108cf0 0x7f7eb810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.322+0000 7f7ebd8ec640 1 --2- 172.21.15.64:0/1804380713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb81083a0 0x7f7eb81087b0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.322+0000 7f7ebd8ec640 1 --2- 172.21.15.64:0/1804380713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7eb8102a60 0x7f7eb8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.323+0000 7f7ebd8ec640 1 -- 172.21.15.64:0/1804380713 >> 172.21.15.64:0/1804380713 conn(0x7f7eb8079730 msgr2=0x7f7eb8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:11.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.323+0000 7f7ebd8ec640 1 -- 172.21.15.64:0/1804380713 shutdown_connections 2024-09-21T04:07:11.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.323+0000 7f7ebd8ec640 1 -- 172.21.15.64:0/1804380713 wait complete. 2024-09-21T04:07:11.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.323+0000 7f7ebd8ec640 1 Processor -- start 2024-09-21T04:07:11.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.323+0000 7f7ebd8ec640 1 -- start start 2024-09-21T04:07:11.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.324+0000 7f7ebd8ec640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb8102a60 0x7f7eb81a6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:11.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.324+0000 7f7eb6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb8102a60 0x7f7eb81a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:11.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.324+0000 7f7eb6ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb8102a60 0x7f7eb81a6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58532/0 (socket says 172.21.15.64:58532) 2024-09-21T04:07:11.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.324+0000 7f7eb6ffd640 1 -- 172.21.15.64:0/3071748311 learned_addr learned my addr 172.21.15.64:0/3071748311 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:11.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.324+0000 7f7ebd8ec640 1 --2- 172.21.15.64:0/3071748311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7eb81083a0 0x7f7eb81a6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:11.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.324+0000 7f7ebd8ec640 1 --2- 172.21.15.64:0/3071748311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7eb8108cf0 0x7f7eb810e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:11.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.324+0000 7f7ebd8ec640 1 -- 172.21.15.64:0/3071748311 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb8111860 con 0x7f7eb8102a60 2024-09-21T04:07:11.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.324+0000 7f7ebd8ec640 1 -- 172.21.15.64:0/3071748311 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb81116e0 con 0x7f7eb8108cf0 2024-09-21T04:07:11.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.324+0000 7f7ebd8ec640 1 -- 172.21.15.64:0/3071748311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb81119e0 con 0x7f7eb81083a0 2024-09-21T04:07:11.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.324+0000 7f7eb77fe640 1 --2- 172.21.15.64:0/3071748311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7eb8108cf0 0x7f7eb810e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:11.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.325+0000 7f7eaffff640 1 --2- 172.21.15.64:0/3071748311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7eb81083a0 0x7f7eb81a6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:11.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.325+0000 7f7eb6ffd640 1 -- 172.21.15.64:0/3071748311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7eb81083a0 msgr2=0x7f7eb81a6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:11.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.325+0000 7f7eb6ffd640 1 --2- 172.21.15.64:0/3071748311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7eb81083a0 0x7f7eb81a6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.325+0000 7f7eb6ffd640 1 -- 172.21.15.64:0/3071748311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7eb8108cf0 msgr2=0x7f7eb810e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:11.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.325+0000 7f7eb6ffd640 1 --2- 172.21.15.64:0/3071748311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7eb8108cf0 0x7f7eb810e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.325+0000 7f7eb6ffd640 1 -- 172.21.15.64:0/3071748311 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb810f010 con 0x7f7eb8102a60 2024-09-21T04:07:11.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.325+0000 7f7eb6ffd640 1 --2- 172.21.15.64:0/3071748311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb8102a60 0x7f7eb81a6540 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7ea800ea60 tx=0x7f7ea800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:11.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.325+0000 7f7ebc8ea640 1 -- 172.21.15.64:0/3071748311 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea800cd20 con 0x7f7eb8102a60 2024-09-21T04:07:11.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.326+0000 7f7ebc8ea640 1 -- 172.21.15.64:0/3071748311 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ea80035f0 con 0x7f7eb8102a60 2024-09-21T04:07:11.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.326+0000 7f7ebc8ea640 1 -- 172.21.15.64:0/3071748311 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8010690 con 0x7f7eb8102a60 2024-09-21T04:07:11.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.326+0000 7f7ebd8ec640 1 -- 172.21.15.64:0/3071748311 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb810f300 con 0x7f7eb8102a60 2024-09-21T04:07:11.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.326+0000 7f7ebd8ec640 1 -- 172.21.15.64:0/3071748311 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb81ad380 con 0x7f7eb8102a60 2024-09-21T04:07:11.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.329+0000 7f7ec3e60740 1 Processor -- start 2024-09-21T04:07:11.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.329+0000 7f7ec3e60740 1 -- start start 2024-09-21T04:07:11.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.329+0000 7f7ec3e60740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577be9e4350 0x5577bed8b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:11.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.329+0000 7f7ec3e60740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577be9bb490 0x5577bed8b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:11.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.329+0000 7f7ec3e60740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577be9c3670 0x5577bed8beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:11.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.329+0000 7f7ec3e60740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5577bed01e60 con 0x5577be9c3670 2024-09-21T04:07:11.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.329+0000 7f7ec3e60740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5577becf1290 con 0x5577be9e4350 2024-09-21T04:07:11.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.329+0000 7f7ec3e60740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5577becf50a0 con 0x5577be9bb490 2024-09-21T04:07:11.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.329+0000 7f7eb77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577be9e4350 0x5577bed8b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:11.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.329+0000 7f7eb77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577be9e4350 0x5577bed8b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:41568/0 (socket says 172.21.15.64:41568) 2024-09-21T04:07:11.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.329+0000 7f7eb77fe640 1 -- 172.21.15.64:0/3997396019 learned_addr learned my addr 172.21.15.64:0/3997396019 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:11.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.329+0000 7f7ebc8ea640 1 -- 172.21.15.64:0/3071748311 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f7ea80037a0 con 0x7f7eb8102a60 2024-09-21T04:07:11.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.329+0000 7f7eaffff640 1 --2- 172.21.15.64:0/3997396019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577be9c3670 0x5577bed8beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:11.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.330+0000 7f7eb6ffd640 1 --2- 172.21.15.64:0/3997396019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577be9bb490 0x5577bed8b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:11.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.330+0000 7f7eb77fe640 1 -- 172.21.15.64:0/3997396019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577be9bb490 msgr2=0x5577bed8b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:11.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.330+0000 7f7eb77fe640 1 --2- 172.21.15.64:0/3997396019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577be9bb490 0x5577bed8b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.330+0000 7f7eb77fe640 1 -- 172.21.15.64:0/3997396019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577be9c3670 msgr2=0x5577bed8beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:11.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.330+0000 7f7eb77fe640 1 --2- 172.21.15.64:0/3997396019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577be9c3670 0x5577bed8beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.330+0000 7f7eb77fe640 1 -- 172.21.15.64:0/3997396019 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577bea4f930 con 0x5577be9e4350 2024-09-21T04:07:11.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.330+0000 7f7eb77fe640 1 --2- 172.21.15.64:0/3997396019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577be9e4350 0x5577bed8b430 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0011440 tx=0x7f7eb0011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f53a6aefc7b7606 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:11.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.331+0000 7f7eadffb640 1 -- 172.21.15.64:0/3997396019 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0003c30 con 0x5577be9e4350 2024-09-21T04:07:11.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.331+0000 7f7eadffb640 1 -- 172.21.15.64:0/3997396019 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7eb0003dd0 con 0x5577be9e4350 2024-09-21T04:07:11.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.331+0000 7f7eadffb640 1 -- 172.21.15.64:0/3997396019 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb000fae0 con 0x5577be9e4350 2024-09-21T04:07:11.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.331+0000 7f7ebc8ea640 1 --2- 172.21.15.64:0/3071748311 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7ea40b6770 0x7f7ea40b8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:11.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.331+0000 7f7ebc8ea640 1 -- 172.21.15.64:0/3071748311 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f7ea80e7af0 con 0x7f7eb8102a60 2024-09-21T04:07:11.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.332+0000 7f7eb6ffd640 1 --2- 172.21.15.64:0/3071748311 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7ea40b6770 0x7f7ea40b8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:11.335 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.332+0000 7f7ec3e60740 1 -- 172.21.15.64:0/3997396019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577be9e4350 msgr2=0x5577bed8b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:11.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.332+0000 7f7ec3e60740 1 --2- 172.21.15.64:0/3997396019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577be9e4350 0x5577bed8b430 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0011440 tx=0x7f7eb0011900 comp rx=0 tx=0).stop 2024-09-21T04:07:11.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.332+0000 7f7ec3e60740 1 -- 172.21.15.64:0/3997396019 shutdown_connections 2024-09-21T04:07:11.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.332+0000 7f7ec3e60740 1 --2- 172.21.15.64:0/3997396019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577be9c3670 0x5577bed8beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.332+0000 7f7ec3e60740 1 --2- 172.21.15.64:0/3997396019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577be9bb490 0x5577bed8b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.332+0000 7f7ec3e60740 1 --2- 172.21.15.64:0/3997396019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577be9e4350 0x5577bed8b430 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.332+0000 7f7ec3e60740 1 -- 172.21.15.64:0/3997396019 >> 172.21.15.64:0/3997396019 conn(0x5577be9d8740 msgr2=0x5577bed89320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:11.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.332+0000 7f7ec3e60740 1 -- 172.21.15.64:0/3997396019 shutdown_connections 2024-09-21T04:07:11.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.332+0000 7f7ec3e60740 1 -- 172.21.15.64:0/3997396019 wait complete. 2024-09-21T04:07:11.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.332+0000 7f7eb6ffd640 1 --2- 172.21.15.64:0/3071748311 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7ea40b6770 0x7f7ea40b8c80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x5577bed8a0b0 tx=0x7f7ea8003e90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:11.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.333+0000 7f7ec3e60740 1 -- start start 2024-09-21T04:07:11.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.333+0000 7f7ec3e60740 1 Processor -- start 2024-09-21T04:07:11.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.334+0000 7f7ec3e60740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577be9bb490 0x5577bed96f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:11.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.334+0000 7f7ec3e60740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577be9c3670 0x5577bed974a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:11.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.334+0000 7f7ec3e60740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577be9e4350 0x5577beda2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:11.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.334+0000 7f7ec3e60740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5577beb52890 con 0x5577be9c3670 2024-09-21T04:07:11.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.334+0000 7f7ec3e60740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5577bec8bd40 con 0x5577be9bb490 2024-09-21T04:07:11.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.334+0000 7f7ec3e60740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5577bed99c10 con 0x5577be9e4350 2024-09-21T04:07:11.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.334+0000 7f7ec3e60740 10 client.0 _ll_register_callbacks cb 0x5577becefc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:11.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.334+0000 7f7eb77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577be9bb490 0x5577bed96f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:11.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.334+0000 7f7eaffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577be9c3670 0x5577bed974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:11.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.334+0000 7f7eb77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577be9bb490 0x5577bed96f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:41570/0 (socket says 172.21.15.64:41570) 2024-09-21T04:07:11.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.334+0000 7f7eb77fe640 1 -- 172.21.15.64:0/1329974889 learned_addr learned my addr 172.21.15.64:0/1329974889 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:11.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.334+0000 7f7eb77fe640 1 --2- 172.21.15.64:0/1329974889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577be9e4350 0x5577beda2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:11.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.334+0000 7f7eaffff640 1 -- 172.21.15.64:0/1329974889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577be9e4350 msgr2=0x5577beda2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:11.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.334+0000 7f7eaffff640 1 --2- 172.21.15.64:0/1329974889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577be9e4350 0x5577beda2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.334+0000 7f7eaffff640 1 -- 172.21.15.64:0/1329974889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577be9bb490 msgr2=0x5577bed96f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:11.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.334+0000 7f7eaffff640 1 --2- 172.21.15.64:0/1329974889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577be9bb490 0x5577bed96f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.334+0000 7f7eaffff640 1 -- 172.21.15.64:0/1329974889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577beda34a0 con 0x5577be9c3670 2024-09-21T04:07:11.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.334+0000 7f7eb77fe640 1 --2- 172.21.15.64:0/1329974889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577be9bb490 0x5577bed96f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:07:11.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.335+0000 7f7eaffff640 1 --2- 172.21.15.64:0/1329974889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577be9c3670 0x5577bed974a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0006010 tx=0x7f7ea00316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f6dea60b4cafd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:11.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.335+0000 7f7e96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:07:11.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.335+0000 7f7e96ffd640 1 -- 172.21.15.64:0/1329974889 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0035040 con 0x5577be9c3670 2024-09-21T04:07:11.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.336+0000 7f7e96ffd640 1 -- 172.21.15.64:0/1329974889 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f7ea0004040 con 0x5577be9c3670 2024-09-21T04:07:11.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.336+0000 7f7ec3e60740 1 -- 172.21.15.64:0/1329974889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7eb0003dd0 con 0x5577be9c3670 2024-09-21T04:07:11.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.336+0000 7f7e96ffd640 1 -- 172.21.15.64:0/1329974889 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7ea0004240 con 0x5577be9c3670 2024-09-21T04:07:11.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.336+0000 7f7ec3e60740 10 client.16251 fetch_fsmap learned FSMap version 30 2024-09-21T04:07:11.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.336+0000 7f7ec3e60740 1 -- 172.21.15.64:0/1329974889 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5577beda5dd0 con 0x5577be9c3670 2024-09-21T04:07:11.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.336+0000 7f7e96ffd640 1 -- 172.21.15.64:0/1329974889 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ea00043e0 con 0x5577be9c3670 2024-09-21T04:07:11.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.336+0000 7f7e96ffd640 1 -- 172.21.15.64:0/1329974889 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f7ea004c070 con 0x5577be9c3670 2024-09-21T04:07:11.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.336+0000 7f7ec3e60740 10 client.16251 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:07:11.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.336+0000 7f7ec3e60740 10 client.16251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:07:11.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.336+0000 7f7ec3e60740 20 client.16251 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:11.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.336+0000 7f7ec3e60740 1 --2- 172.21.15.64:0/1329974889 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5577be9a2cd0 0x5577beda9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:11.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.336+0000 7f7ec3e60740 4 client.16251 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:07:11.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.336+0000 7f7ec3e60740 1 -- 172.21.15.64:0/1329974889 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7eb0003dd0 con 0x5577be9a2cd0 2024-09-21T04:07:11.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.337+0000 7f7eb77fe640 1 --2- 172.21.15.64:0/1329974889 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5577be9a2cd0 0x5577beda9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:11.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.337+0000 7f7e96ffd640 10 client.16251 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:11.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.337+0000 7f7eb77fe640 1 --2- 172.21.15.64:0/1329974889 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5577be9a2cd0 0x5577beda9330 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e59b6cd6e7a2f728 server_cookie=cdaa0e87f89b2ec3 in_seq=0 out_seq=0 2024-09-21T04:07:11.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.338+0000 7f7e96ffd640 1 -- 172.21.15.64:0/1329974889 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f7eb0003dd0 con 0x5577be9a2cd0 2024-09-21T04:07:11.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.338+0000 7f7e96ffd640 10 client.16251 handle_command_reply: tid=0 2024-09-21T04:07:11.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.338+0000 7f7ec3e60740 1 client.16251 shutdown 2024-09-21T04:07:11.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.338+0000 7f7ec3e60740 1 -- 172.21.15.64:0/1329974889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577be9c3670 msgr2=0x5577bed974a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:11.342 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.338+0000 7f7ec3e60740 1 --2- 172.21.15.64:0/1329974889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577be9c3670 0x5577bed974a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0006010 tx=0x7f7ea00316d0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.338+0000 7f7ec3e60740 1 -- 172.21.15.64:0/1329974889 shutdown_connections 2024-09-21T04:07:11.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.338+0000 7f7ec3e60740 1 --2- 172.21.15.64:0/1329974889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577be9e4350 0x5577beda2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.338+0000 7f7ec3e60740 1 --2- 172.21.15.64:0/1329974889 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5577be9a2cd0 0x5577beda9330 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.338+0000 7f7ec3e60740 1 --2- 172.21.15.64:0/1329974889 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577be9c3670 0x5577bed974a0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.338+0000 7f7ec3e60740 1 --2- 172.21.15.64:0/1329974889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577be9bb490 0x5577bed96f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.338+0000 7f7ec3e60740 1 -- 172.21.15.64:0/1329974889 >> 172.21.15.64:0/1329974889 conn(0x5577be9d8740 msgr2=0x5577bed8e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:11.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.338+0000 7f7e96ffd640 0 client.16251 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:11.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.339+0000 7f7ec3e60740 1 -- 172.21.15.64:0/1329974889 shutdown_connections 2024-09-21T04:07:11.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.339+0000 7f7ec3e60740 1 -- 172.21.15.64:0/1329974889 wait complete. 2024-09-21T04:07:11.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.339+0000 7f7ec3e60740 20 client.16251 trim_cache size 0 max 16384 2024-09-21T04:07:11.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.357+0000 7f7ec3e60740 1 Processor -- start 2024-09-21T04:07:11.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.358+0000 7f7ec3e60740 1 -- start start 2024-09-21T04:07:11.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.358+0000 7f7ec3e60740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577be9bb490 0x5577bed91550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:11.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.358+0000 7f7ec3e60740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577be9c3670 0x5577beda2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:11.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.358+0000 7f7ec3e60740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577be9e4350 0x5577bed93f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:11.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.358+0000 7f7ec3e60740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb81b8850 con 0x5577be9c3670 2024-09-21T04:07:11.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.358+0000 7f7ec3e60740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb81b6090 con 0x5577be9bb490 2024-09-21T04:07:11.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.358+0000 7f7ec3e60740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb81b3ee0 con 0x5577be9e4350 2024-09-21T04:07:11.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.358+0000 7f7eaffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577be9c3670 0x5577beda2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:11.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.358+0000 7f7eb77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577be9bb490 0x5577bed91550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:11.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.358+0000 7f7eaffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577be9c3670 0x5577beda2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58550/0 (socket says 172.21.15.64:58550) 2024-09-21T04:07:11.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.358+0000 7f7eaffff640 1 -- 172.21.15.64:0/1532660483 learned_addr learned my addr 172.21.15.64:0/1532660483 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:11.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.358+0000 7f7eb77fe640 1 --2- 172.21.15.64:0/1532660483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577be9e4350 0x5577bed93f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:11.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.358+0000 7f7eaffff640 1 -- 172.21.15.64:0/1532660483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577be9e4350 msgr2=0x5577bed93f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:11.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.358+0000 7f7eaffff640 1 --2- 172.21.15.64:0/1532660483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577be9e4350 0x5577bed93f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.358+0000 7f7eaffff640 1 -- 172.21.15.64:0/1532660483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577be9bb490 msgr2=0x5577bed91550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:11.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.358+0000 7f7eaffff640 1 --2- 172.21.15.64:0/1532660483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577be9bb490 0x5577bed91550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.358+0000 7f7eaffff640 1 -- 172.21.15.64:0/1532660483 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577bed83520 con 0x5577be9c3670 2024-09-21T04:07:11.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.358+0000 7f7eb77fe640 1 --2- 172.21.15.64:0/1532660483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577be9bb490 0x5577bed91550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:07:11.362 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.359+0000 7f7eaffff640 1 --2- 172.21.15.64:0/1532660483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577be9c3670 0x5577beda2e60 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7ea002a930 tx=0x7f7ea0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e3c79382f9dbe5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:11.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.359+0000 7f7ead7fa640 1 -- 172.21.15.64:0/1532660483 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0035030 con 0x5577be9c3670 2024-09-21T04:07:11.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.359+0000 7f7ead7fa640 1 -- 172.21.15.64:0/1532660483 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ea00046c0 con 0x5577be9c3670 2024-09-21T04:07:11.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.359+0000 7f7ead7fa640 1 -- 172.21.15.64:0/1532660483 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0052070 con 0x5577be9c3670 2024-09-21T04:07:11.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.360+0000 7f7ec3e60740 1 -- 172.21.15.64:0/1532660483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577be9c3670 msgr2=0x5577beda2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:11.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.360+0000 7f7ec3e60740 1 --2- 172.21.15.64:0/1532660483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577be9c3670 0x5577beda2e60 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7ea002a930 tx=0x7f7ea0000bc0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.360+0000 7f7ec3e60740 1 -- 172.21.15.64:0/1532660483 shutdown_connections 2024-09-21T04:07:11.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.360+0000 7f7ec3e60740 1 --2- 172.21.15.64:0/1532660483 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577be9e4350 0x5577bed93f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.360+0000 7f7ec3e60740 1 --2- 172.21.15.64:0/1532660483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577be9c3670 0x5577beda2e60 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.360+0000 7f7ec3e60740 1 --2- 172.21.15.64:0/1532660483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577be9bb490 0x5577bed91550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.360+0000 7f7ec3e60740 1 -- 172.21.15.64:0/1532660483 >> 172.21.15.64:0/1532660483 conn(0x5577be9d8740 msgr2=0x5577bed955e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:11.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.360+0000 7f7ec3e60740 1 -- 172.21.15.64:0/1532660483 shutdown_connections 2024-09-21T04:07:11.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.360+0000 7f7ec3e60740 1 -- 172.21.15.64:0/1532660483 wait complete. 2024-09-21T04:07:11.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.360+0000 7f7ec3e60740 1 -- start start 2024-09-21T04:07:11.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.361+0000 7f7ec3e60740 1 Processor -- start 2024-09-21T04:07:11.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.361+0000 7f7ec3e60740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577be9bb490 0x5577bed9b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:11.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.361+0000 7f7ec3e60740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577be9c3670 0x5577bed9bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:11.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.361+0000 7f7ec3e60740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577be9e4350 0x5577bee12d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:11.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.361+0000 7f7ec3e60740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb81b4550 con 0x5577be9bb490 2024-09-21T04:07:11.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.361+0000 7f7ec3e60740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb81b59c0 con 0x5577be9e4350 2024-09-21T04:07:11.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.361+0000 7f7ec3e60740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb81b4c20 con 0x5577be9c3670 2024-09-21T04:07:11.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.361+0000 7f7ec3e60740 10 client.0 _ll_register_callbacks cb 0x7f7ea8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:11.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.361+0000 7f7eb77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577be9bb490 0x5577bed9b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:11.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.361+0000 7f7eb77fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577be9bb490 0x5577bed9b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35168/0 (socket says 172.21.15.64:35168) 2024-09-21T04:07:11.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.361+0000 7f7eb77fe640 1 -- 172.21.15.64:0/1206025972 learned_addr learned my addr 172.21.15.64:0/1206025972 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:11.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.361+0000 7f7eb77fe640 1 --2- 172.21.15.64:0/1206025972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577be9e4350 0x5577bee12d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:11.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.361+0000 7f7eaffff640 1 --2- 172.21.15.64:0/1206025972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577be9c3670 0x5577bed9bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:11.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.361+0000 7f7eb77fe640 1 -- 172.21.15.64:0/1206025972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577be9c3670 msgr2=0x5577bed9bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:11.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.361+0000 7f7eb77fe640 1 --2- 172.21.15.64:0/1206025972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577be9c3670 0x5577bed9bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.361+0000 7f7eb77fe640 1 -- 172.21.15.64:0/1206025972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577be9e4350 msgr2=0x5577bee12d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:11.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.361+0000 7f7eb77fe640 1 --2- 172.21.15.64:0/1206025972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577be9e4350 0x5577bee12d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.361+0000 7f7eb77fe640 1 -- 172.21.15.64:0/1206025972 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577bee13270 con 0x5577be9bb490 2024-09-21T04:07:11.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.362+0000 7f7eb77fe640 1 --2- 172.21.15.64:0/1206025972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577be9bb490 0x5577bed9b5a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0011940 tx=0x7f7eb000f680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad8a2c99df01f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:11.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.362+0000 7f7e96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:07:11.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.362+0000 7f7e96ffd640 1 -- 172.21.15.64:0/1206025972 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0005050 con 0x5577be9bb490 2024-09-21T04:07:11.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.362+0000 7f7e96ffd640 1 -- 172.21.15.64:0/1206025972 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f7eb00051f0 con 0x5577be9bb490 2024-09-21T04:07:11.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.362+0000 7f7ec3e60740 1 -- 172.21.15.64:0/1206025972 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ea00046c0 con 0x5577be9bb490 2024-09-21T04:07:11.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.362+0000 7f7e96ffd640 1 -- 172.21.15.64:0/1206025972 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7eb003d050 con 0x5577be9bb490 2024-09-21T04:07:11.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.362+0000 7f7ec3e60740 10 client.16257 fetch_fsmap learned FSMap version 30 2024-09-21T04:07:11.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.362+0000 7f7ec3e60740 1 -- 172.21.15.64:0/1206025972 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5577bee14500 con 0x5577be9bb490 2024-09-21T04:07:11.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.362+0000 7f7e96ffd640 1 -- 172.21.15.64:0/1206025972 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ea00046c0 con 0x5577be9bb490 2024-09-21T04:07:11.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.363+0000 7f7e96ffd640 1 -- 172.21.15.64:0/1206025972 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f7eb0028720 con 0x5577be9bb490 2024-09-21T04:07:11.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.363+0000 7f7ec3e60740 10 client.16257 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:07:11.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.363+0000 7f7ec3e60740 10 client.16257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:07:11.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.363+0000 7f7ec3e60740 20 client.16257 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:11.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.363+0000 7f7ec3e60740 1 --2- 172.21.15.64:0/1206025972 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5577be978080 0x5577bee16aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:11.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.363+0000 7f7ec3e60740 4 client.16257 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T04:07:11.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.363+0000 7f7ec3e60740 1 -- 172.21.15.64:0/1206025972 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5577bed03db0 con 0x5577be978080 2024-09-21T04:07:11.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.363+0000 7f7eaffff640 1 --2- 172.21.15.64:0/1206025972 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5577be978080 0x5577bee16aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:11.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.363+0000 7f7e96ffd640 10 client.16257 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:11.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.363+0000 7f7eaffff640 1 --2- 172.21.15.64:0/1206025972 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5577be978080 0x5577bee16aa0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dbcf1aa0849a25c server_cookie=6b5168462de5ac8f in_seq=0 out_seq=0 2024-09-21T04:07:11.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.365+0000 7f7e96ffd640 1 -- 172.21.15.64:0/1206025972 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5577bed03db0 con 0x5577be978080 2024-09-21T04:07:11.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.365+0000 7f7e96ffd640 10 client.16257 handle_command_reply: tid=0 2024-09-21T04:07:11.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.365+0000 7f7ec3e60740 1 client.16257 shutdown 2024-09-21T04:07:11.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.365+0000 7f7ec3e60740 1 -- 172.21.15.64:0/1206025972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577be9bb490 msgr2=0x5577bed9b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:11.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.365+0000 7f7ec3e60740 1 --2- 172.21.15.64:0/1206025972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577be9bb490 0x5577bed9b5a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0011940 tx=0x7f7eb000f680 comp rx=0 tx=0).stop 2024-09-21T04:07:11.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.365+0000 7f7ec3e60740 1 -- 172.21.15.64:0/1206025972 shutdown_connections 2024-09-21T04:07:11.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.365+0000 7f7ec3e60740 1 --2- 172.21.15.64:0/1206025972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577be9e4350 0x5577bee12d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.365+0000 7f7ec3e60740 1 --2- 172.21.15.64:0/1206025972 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5577be9c3670 0x5577bed9bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.365+0000 7f7ec3e60740 1 --2- 172.21.15.64:0/1206025972 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5577be978080 0x5577bee16aa0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.365+0000 7f7ec3e60740 1 --2- 172.21.15.64:0/1206025972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5577be9bb490 0x5577bed9b5a0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.365+0000 7f7ec3e60740 1 -- 172.21.15.64:0/1206025972 >> 172.21.15.64:0/1206025972 conn(0x5577be9d8740 msgr2=0x5577bed951b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:11.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.365+0000 7f7e96ffd640 0 client.16257 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:11.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.365+0000 7f7ec3e60740 1 -- 172.21.15.64:0/1206025972 shutdown_connections 2024-09-21T04:07:11.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.365+0000 7f7ec3e60740 1 -- 172.21.15.64:0/1206025972 wait complete. 2024-09-21T04:07:11.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.365+0000 7f7ec3e60740 20 client.16257 trim_cache size 0 max 16384 2024-09-21T04:07:11.371 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:07:11.371 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-21T04:07:11.372 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "bed40d88-2102-43a6-ac2d-b89d5a2264d0", 2024-09-21T04:07:11.372 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-21T04:07:11.372 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:07:11.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.366+0000 7f7ebd8ec640 1 -- 172.21.15.64:0/3071748311 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7ea40b6770 msgr2=0x7f7ea40b8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:11.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.366+0000 7f7ebd8ec640 1 --2- 172.21.15.64:0/3071748311 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7ea40b6770 0x7f7ea40b8c80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x5577bed8a0b0 tx=0x7f7ea8003e90 comp rx=0 tx=0).stop 2024-09-21T04:07:11.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.366+0000 7f7ebd8ec640 1 -- 172.21.15.64:0/3071748311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb8102a60 msgr2=0x7f7eb81a6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:11.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.366+0000 7f7ebd8ec640 1 --2- 172.21.15.64:0/3071748311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb8102a60 0x7f7eb81a6540 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7ea800ea60 tx=0x7f7ea800ee60 comp rx=0 tx=0).stop 2024-09-21T04:07:11.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.366+0000 7f7ebd8ec640 1 -- 172.21.15.64:0/3071748311 shutdown_connections 2024-09-21T04:07:11.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.366+0000 7f7ebd8ec640 1 --2- 172.21.15.64:0/3071748311 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7ea40b6770 0x7f7ea40b8c80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.366+0000 7f7ebd8ec640 1 --2- 172.21.15.64:0/3071748311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7eb8108cf0 0x7f7eb810e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.366+0000 7f7ebd8ec640 1 --2- 172.21.15.64:0/3071748311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7eb81083a0 0x7f7eb81a6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.366+0000 7f7ebd8ec640 1 --2- 172.21.15.64:0/3071748311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7eb8102a60 0x7f7eb81a6540 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:11.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.366+0000 7f7ebd8ec640 1 -- 172.21.15.64:0/3071748311 >> 172.21.15.64:0/3071748311 conn(0x7f7eb8079730 msgr2=0x7f7eb8104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:11.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.366+0000 7f7ebd8ec640 1 -- 172.21.15.64:0/3071748311 shutdown_connections 2024-09-21T04:07:11.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:11.366+0000 7f7ebd8ec640 1 -- 172.21.15.64:0/3071748311 wait complete. 2024-09-21T04:07:12.179 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T04:07:12.493 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:07:12.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:12 smithi064 ceph-mon[31180]: pgmap v416: 97 pgs: 97 active+clean; 400 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.8 MiB/s wr, 475 op/s 2024-09-21T04:07:12.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:12 smithi064 ceph-mon[31180]: scrub queued for path: / 2024-09-21T04:07:12.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:12 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:12.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:12 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:12.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:12 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:12.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:12 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:12.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:12 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:12.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:12 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:12.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:12 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:07:12.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:12 smithi119 ceph-mon[34811]: pgmap v416: 97 pgs: 97 active+clean; 400 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.8 MiB/s wr, 475 op/s 2024-09-21T04:07:12.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:12 smithi119 ceph-mon[34811]: scrub queued for path: / 2024-09-21T04:07:12.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:12 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:12.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:12 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:12.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:12 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:12.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:12 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:12.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:12 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:12.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:12 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:12.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:12 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:07:12.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:12 smithi131 ceph-mon[35193]: pgmap v416: 97 pgs: 97 active+clean; 400 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.8 MiB/s wr, 475 op/s 2024-09-21T04:07:12.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:12 smithi131 ceph-mon[35193]: scrub queued for path: / 2024-09-21T04:07:12.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:12 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:12.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:12 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:12.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:12 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:12.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:12 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:12.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:12 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:12.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:12 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:12.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:12 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:07:13.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.081+0000 7f567cd24640 1 -- 172.21.15.64:0/2786299804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f567810de60 msgr2=0x7f5678110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:13.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.081+0000 7f567cd24640 1 --2- 172.21.15.64:0/2786299804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f567810de60 0x7f5678110330 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5670009ca0 tx=0x7f567002a8c0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.081+0000 7f567cd24640 1 -- 172.21.15.64:0/2786299804 shutdown_connections 2024-09-21T04:07:13.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.081+0000 7f567cd24640 1 --2- 172.21.15.64:0/2786299804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f567810de60 0x7f5678110330 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.081+0000 7f567cd24640 1 --2- 172.21.15.64:0/2786299804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5678105380 0x7f567810d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.081+0000 7f567cd24640 1 --2- 172.21.15.64:0/2786299804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5678104a30 0x7f5678104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.081+0000 7f567cd24640 1 -- 172.21.15.64:0/2786299804 >> 172.21.15.64:0/2786299804 conn(0x7f56780797b0 msgr2=0x7f5678079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:13.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.081+0000 7f567cd24640 1 -- 172.21.15.64:0/2786299804 shutdown_connections 2024-09-21T04:07:13.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.081+0000 7f567cd24640 1 -- 172.21.15.64:0/2786299804 wait complete. 2024-09-21T04:07:13.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.083+0000 7f567cd24640 1 Processor -- start 2024-09-21T04:07:13.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.083+0000 7f567cd24640 1 -- start start 2024-09-21T04:07:13.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.083+0000 7f567cd24640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5678104a30 0x7f567819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:13.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.083+0000 7f567cd24640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5678105380 0x7f567819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:13.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.083+0000 7f567cd24640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f567810de60 0x7f567819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:13.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.083+0000 7f567cd24640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5678113ac0 con 0x7f5678104a30 2024-09-21T04:07:13.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.083+0000 7f567cd24640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5678113940 con 0x7f567810de60 2024-09-21T04:07:13.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.083+0000 7f567cd24640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5678113c40 con 0x7f5678105380 2024-09-21T04:07:13.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.083+0000 7f567659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5678104a30 0x7f567819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:13.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.083+0000 7f5676d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f567810de60 0x7f567819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:13.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.083+0000 7f5676d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f567810de60 0x7f567819fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:56228/0 (socket says 172.21.15.64:56228) 2024-09-21T04:07:13.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.083+0000 7f567659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5678104a30 0x7f567819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35186/0 (socket says 172.21.15.64:35186) 2024-09-21T04:07:13.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.083+0000 7f5676d9d640 1 -- 172.21.15.64:0/1239795380 learned_addr learned my addr 172.21.15.64:0/1239795380 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:13.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.084+0000 7f5675d9b640 1 --2- 172.21.15.64:0/1239795380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5678105380 0x7f567819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:13.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.084+0000 7f567659c640 1 -- 172.21.15.64:0/1239795380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5678105380 msgr2=0x7f567819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:13.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.084+0000 7f567659c640 1 --2- 172.21.15.64:0/1239795380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5678105380 0x7f567819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.084+0000 7f567659c640 1 -- 172.21.15.64:0/1239795380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f567810de60 msgr2=0x7f567819fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:13.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.084+0000 7f567659c640 1 --2- 172.21.15.64:0/1239795380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f567810de60 0x7f567819fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.084+0000 7f567659c640 1 -- 172.21.15.64:0/1239795380 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56781a03e0 con 0x7f5678104a30 2024-09-21T04:07:13.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.084+0000 7f5675d9b640 1 --2- 172.21.15.64:0/1239795380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5678105380 0x7f567819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:07:13.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.084+0000 7f567659c640 1 --2- 172.21.15.64:0/1239795380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5678104a30 0x7f567819a470 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f566800e9f0 tx=0x7f566800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:13.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.084+0000 7f56677fe640 1 -- 172.21.15.64:0/1239795380 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566800ccc0 con 0x7f5678104a30 2024-09-21T04:07:13.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.084+0000 7f56677fe640 1 -- 172.21.15.64:0/1239795380 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5668005020 con 0x7f5678104a30 2024-09-21T04:07:13.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.084+0000 7f56677fe640 1 -- 172.21.15.64:0/1239795380 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5668004a00 con 0x7f5678104a30 2024-09-21T04:07:13.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.085+0000 7f567cd24640 1 -- 172.21.15.64:0/1239795380 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56781a06d0 con 0x7f5678104a30 2024-09-21T04:07:13.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.085+0000 7f567cd24640 1 -- 172.21.15.64:0/1239795380 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56781a6e30 con 0x7f5678104a30 2024-09-21T04:07:13.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.088+0000 7f5683298740 1 Processor -- start 2024-09-21T04:07:13.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.088+0000 7f5683298740 1 -- start start 2024-09-21T04:07:13.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.088+0000 7f5683298740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f9fe05070 0x564fa01a9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:13.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.088+0000 7f5683298740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f9fddc1b0 0x564fa01aa250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:13.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.088+0000 7f5683298740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f9fde4390 0x564fa01bc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:13.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.088+0000 7f5683298740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564fa0122e50 con 0x564f9fe05070 2024-09-21T04:07:13.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.088+0000 7f5683298740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564fa0112280 con 0x564f9fddc1b0 2024-09-21T04:07:13.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.088+0000 7f5683298740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564fa0116090 con 0x564f9fde4390 2024-09-21T04:07:13.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.089+0000 7f56677fe640 1 -- 172.21.15.64:0/1239795380 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f5668004ba0 con 0x7f5678104a30 2024-09-21T04:07:13.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.089+0000 7f5676d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f9fe05070 0x564fa01a9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:13.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.089+0000 7f5676d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f9fe05070 0x564fa01a9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35196/0 (socket says 172.21.15.64:35196) 2024-09-21T04:07:13.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.089+0000 7f5676d9d640 1 -- 172.21.15.64:0/2034422734 learned_addr learned my addr 172.21.15.64:0/2034422734 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:13.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.089+0000 7f5675d9b640 1 --2- 172.21.15.64:0/2034422734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f9fde4390 0x564fa01bc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:13.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.089+0000 7f567659c640 1 --2- 172.21.15.64:0/2034422734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f9fddc1b0 0x564fa01aa250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:13.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.089+0000 7f5675d9b640 1 -- 172.21.15.64:0/2034422734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f9fddc1b0 msgr2=0x564fa01aa250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:13.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.089+0000 7f5675d9b640 1 --2- 172.21.15.64:0/2034422734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f9fddc1b0 0x564fa01aa250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.089+0000 7f5675d9b640 1 -- 172.21.15.64:0/2034422734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f9fe05070 msgr2=0x564fa01a9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:13.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.089+0000 7f5675d9b640 1 --2- 172.21.15.64:0/2034422734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f9fe05070 0x564fa01a9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.089+0000 7f5675d9b640 1 -- 172.21.15.64:0/2034422734 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f9fe70650 con 0x564f9fde4390 2024-09-21T04:07:13.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.089+0000 7f5676d9d640 1 --2- 172.21.15.64:0/2034422734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f9fe05070 0x564fa01a9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:07:13.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.089+0000 7f567659c640 1 --2- 172.21.15.64:0/2034422734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f9fddc1b0 0x564fa01aa250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:07:13.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.089+0000 7f5675d9b640 1 --2- 172.21.15.64:0/2034422734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f9fde4390 0x564fa01bc620 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f567806aaa0 tx=0x7f566c0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54805e401d575bc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:13.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.089+0000 7f5664ff9640 1 -- 172.21.15.64:0/2034422734 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c012450 con 0x564f9fde4390 2024-09-21T04:07:13.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.090+0000 7f5664ff9640 1 -- 172.21.15.64:0/2034422734 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f566c012aa0 con 0x564f9fde4390 2024-09-21T04:07:13.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.090+0000 7f5664ff9640 1 -- 172.21.15.64:0/2034422734 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c00f9a0 con 0x564f9fde4390 2024-09-21T04:07:13.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.090+0000 7f56677fe640 1 --2- 172.21.15.64:0/1239795380 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f56600b6700 0x7f56600b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:13.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.090+0000 7f5676d9d640 1 --2- 172.21.15.64:0/1239795380 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f56600b6700 0x7f56600b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:13.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.090+0000 7f56677fe640 1 -- 172.21.15.64:0/1239795380 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f56680e7740 con 0x7f5678104a30 2024-09-21T04:07:13.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.091+0000 7f5683298740 1 -- 172.21.15.64:0/2034422734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f9fde4390 msgr2=0x564fa01bc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:13.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.091+0000 7f5683298740 1 --2- 172.21.15.64:0/2034422734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f9fde4390 0x564fa01bc620 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f567806aaa0 tx=0x7f566c0117b0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.091+0000 7f5683298740 1 -- 172.21.15.64:0/2034422734 shutdown_connections 2024-09-21T04:07:13.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.091+0000 7f5683298740 1 --2- 172.21.15.64:0/2034422734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f9fde4390 0x564fa01bc620 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.091+0000 7f5683298740 1 --2- 172.21.15.64:0/2034422734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f9fddc1b0 0x564fa01aa250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.091+0000 7f5683298740 1 --2- 172.21.15.64:0/2034422734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f9fe05070 0x564fa01a9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.091+0000 7f5683298740 1 -- 172.21.15.64:0/2034422734 >> 172.21.15.64:0/2034422734 conn(0x564f9fdf9460 msgr2=0x564fa01ac1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:13.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.091+0000 7f5676d9d640 1 --2- 172.21.15.64:0/1239795380 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f56600b6700 0x7f56600b8c10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5670031bd0 tx=0x7f5670005c50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:13.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.091+0000 7f5683298740 1 -- 172.21.15.64:0/2034422734 shutdown_connections 2024-09-21T04:07:13.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.091+0000 7f5683298740 1 -- 172.21.15.64:0/2034422734 wait complete. 2024-09-21T04:07:13.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.091+0000 7f5683298740 1 -- start start 2024-09-21T04:07:13.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.092+0000 7f5683298740 1 Processor -- start 2024-09-21T04:07:13.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.092+0000 7f5683298740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f9fddc1b0 0x564fa01c0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:13.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.092+0000 7f5683298740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f9fde4390 0x564fa01c0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:13.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.092+0000 7f5683298740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f9fe05070 0x564fa01c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:13.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.092+0000 7f5683298740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564f9ff73700 con 0x564f9fde4390 2024-09-21T04:07:13.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.092+0000 7f5683298740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564fa00ad110 con 0x564f9fe05070 2024-09-21T04:07:13.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.092+0000 7f5683298740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564fa01bd170 con 0x564f9fddc1b0 2024-09-21T04:07:13.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.092+0000 7f5683298740 10 client.0 _ll_register_callbacks cb 0x564fa00c6990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:13.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.092+0000 7f5675d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f9fde4390 0x564fa01c0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:13.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.092+0000 7f5675d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f9fde4390 0x564fa01c0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35208/0 (socket says 172.21.15.64:35208) 2024-09-21T04:07:13.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.092+0000 7f5675d9b640 1 -- 172.21.15.64:0/475806822 learned_addr learned my addr 172.21.15.64:0/475806822 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:13.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.092+0000 7f567659c640 1 --2- 172.21.15.64:0/475806822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f9fddc1b0 0x564fa01c0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:13.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.093+0000 7f567659c640 1 -- 172.21.15.64:0/475806822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f9fe05070 msgr2=0x564fa01c5f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:07:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.093+0000 7f5676d9d640 1 --2- 172.21.15.64:0/475806822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f9fe05070 0x564fa01c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.093+0000 7f567659c640 1 --2- 172.21.15.64:0/475806822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f9fe05070 0x564fa01c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.093+0000 7f567659c640 1 -- 172.21.15.64:0/475806822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f9fde4390 msgr2=0x564fa01c0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.093+0000 7f567659c640 1 --2- 172.21.15.64:0/475806822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f9fde4390 0x564fa01c0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.093+0000 7f567659c640 1 -- 172.21.15.64:0/475806822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fa00dac40 con 0x564f9fddc1b0 2024-09-21T04:07:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.093+0000 7f5676d9d640 1 --2- 172.21.15.64:0/475806822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f9fe05070 0x564fa01c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:07:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.093+0000 7f567659c640 1 --2- 172.21.15.64:0/475806822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f9fddc1b0 0x564fa01c0930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f56680e4e60 tx=0x7f56680ad3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41f7b71bd44f4ab6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.093+0000 7f565dffb640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-21T04:07:13.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.093+0000 7f565dffb640 1 -- 172.21.15.64:0/475806822 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566801c040 con 0x564f9fddc1b0 2024-09-21T04:07:13.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.094+0000 7f5683298740 1 -- 172.21.15.64:0/475806822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f566c012aa0 con 0x564f9fddc1b0 2024-09-21T04:07:13.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.094+0000 7f565dffb640 1 -- 172.21.15.64:0/475806822 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f56680ac9c0 con 0x564f9fddc1b0 2024-09-21T04:07:13.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.094+0000 7f565dffb640 1 -- 172.21.15.64:0/475806822 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f56680adc60 con 0x564f9fddc1b0 2024-09-21T04:07:13.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.094+0000 7f5683298740 10 client.25861 fetch_fsmap learned FSMap version 30 2024-09-21T04:07:13.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.094+0000 7f5683298740 1 -- 172.21.15.64:0/475806822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564fa01cee50 con 0x564f9fddc1b0 2024-09-21T04:07:13.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.094+0000 7f565dffb640 1 -- 172.21.15.64:0/475806822 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f566c012aa0 con 0x564f9fddc1b0 2024-09-21T04:07:13.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.094+0000 7f565dffb640 1 -- 172.21.15.64:0/475806822 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f56680b31d0 con 0x564f9fddc1b0 2024-09-21T04:07:13.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.094+0000 7f5683298740 10 client.25861 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:07:13.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.094+0000 7f5683298740 10 client.25861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:07:13.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.094+0000 7f5683298740 20 client.25861 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:13.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.094+0000 7f5683298740 1 --2- 172.21.15.64:0/475806822 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564f9fdc39f0 0x564fa01d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:13.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.094+0000 7f5683298740 4 client.25861 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:07:13.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.094+0000 7f5683298740 1 -- 172.21.15.64:0/475806822 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fa0124da0 con 0x564f9fdc39f0 2024-09-21T04:07:13.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.095+0000 7f5675d9b640 1 --2- 172.21.15.64:0/475806822 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564f9fdc39f0 0x564fa01d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:13.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.095+0000 7f5675d9b640 1 --2- 172.21.15.64:0/475806822 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564f9fdc39f0 0x564fa01d2370 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d49a729afd4ddc5f server_cookie=21f036bc3b72ac74 in_seq=0 out_seq=0 2024-09-21T04:07:13.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.095+0000 7f565dffb640 10 client.25861 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:13.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.096+0000 7f565dffb640 1 -- 172.21.15.64:0/475806822 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564fa0124da0 con 0x564f9fdc39f0 2024-09-21T04:07:13.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.096+0000 7f565dffb640 10 client.25861 handle_command_reply: tid=0 2024-09-21T04:07:13.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.096+0000 7f5683298740 1 client.25861 shutdown 2024-09-21T04:07:13.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.096+0000 7f5683298740 1 -- 172.21.15.64:0/475806822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f9fddc1b0 msgr2=0x564fa01c0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:13.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.096+0000 7f5683298740 1 --2- 172.21.15.64:0/475806822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f9fddc1b0 0x564fa01c0930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f56680e4e60 tx=0x7f56680ad3f0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.096+0000 7f5683298740 1 -- 172.21.15.64:0/475806822 shutdown_connections 2024-09-21T04:07:13.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.096+0000 7f5683298740 1 --2- 172.21.15.64:0/475806822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f9fe05070 0x564fa01c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.096+0000 7f5683298740 1 --2- 172.21.15.64:0/475806822 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564f9fdc39f0 0x564fa01d2370 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.096+0000 7f5683298740 1 --2- 172.21.15.64:0/475806822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f9fde4390 0x564fa01c0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.096+0000 7f5683298740 1 --2- 172.21.15.64:0/475806822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f9fddc1b0 0x564fa01c0930 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.096+0000 7f565dffb640 0 client.25861 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:13.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.096+0000 7f5683298740 1 -- 172.21.15.64:0/475806822 >> 172.21.15.64:0/475806822 conn(0x564f9fdf9460 msgr2=0x564fa01ab1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:13.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.096+0000 7f5683298740 1 -- 172.21.15.64:0/475806822 shutdown_connections 2024-09-21T04:07:13.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.096+0000 7f5683298740 1 -- 172.21.15.64:0/475806822 wait complete. 2024-09-21T04:07:13.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.096+0000 7f5683298740 20 client.25861 trim_cache size 0 max 16384 2024-09-21T04:07:13.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.117+0000 7f5683298740 1 Processor -- start 2024-09-21T04:07:13.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.117+0000 7f5683298740 1 -- start start 2024-09-21T04:07:13.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.117+0000 7f5683298740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f9fddc1b0 0x564fa01abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:13.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.117+0000 7f5683298740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f9fde4390 0x564fa01ac230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:13.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.117+0000 7f5683298740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f9fe05070 0x564fa01bb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:13.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.117+0000 7f5683298740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f56781affe0 con 0x564f9fde4390 2024-09-21T04:07:13.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.117+0000 7f5683298740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f56781ad820 con 0x564f9fe05070 2024-09-21T04:07:13.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.117+0000 7f5683298740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f56781ab670 con 0x564f9fddc1b0 2024-09-21T04:07:13.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.117+0000 7f5675d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f9fde4390 0x564fa01ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:13.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.117+0000 7f5675d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f9fde4390 0x564fa01ac230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35222/0 (socket says 172.21.15.64:35222) 2024-09-21T04:07:13.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.117+0000 7f5675d9b640 1 -- 172.21.15.64:0/2218928863 learned_addr learned my addr 172.21.15.64:0/2218928863 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:13.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.117+0000 7f567659c640 1 --2- 172.21.15.64:0/2218928863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f9fddc1b0 0x564fa01abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:13.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.117+0000 7f5676d9d640 1 --2- 172.21.15.64:0/2218928863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f9fe05070 0x564fa01bb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:13.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.118+0000 7f5675d9b640 1 -- 172.21.15.64:0/2218928863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f9fddc1b0 msgr2=0x564fa01abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:13.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.118+0000 7f5675d9b640 1 --2- 172.21.15.64:0/2218928863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f9fddc1b0 0x564fa01abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.118+0000 7f5675d9b640 1 -- 172.21.15.64:0/2218928863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f9fe05070 msgr2=0x564fa01bb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:13.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.118+0000 7f5675d9b640 1 --2- 172.21.15.64:0/2218928863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f9fe05070 0x564fa01bb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.118+0000 7f5675d9b640 1 -- 172.21.15.64:0/2218928863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fa01acae0 con 0x564f9fde4390 2024-09-21T04:07:13.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.118+0000 7f567659c640 1 --2- 172.21.15.64:0/2218928863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f9fddc1b0 0x564fa01abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:07:13.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.118+0000 7f5675d9b640 1 --2- 172.21.15.64:0/2218928863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f9fde4390 0x564fa01ac230 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f566c012010 tx=0x7f566c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c92f8b237bffc88 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:13.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.118+0000 7f565ffff640 1 -- 172.21.15.64:0/2218928863 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c033070 con 0x564f9fde4390 2024-09-21T04:07:13.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.118+0000 7f565ffff640 1 -- 172.21.15.64:0/2218928863 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f566c009dc0 con 0x564f9fde4390 2024-09-21T04:07:13.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.118+0000 7f565ffff640 1 -- 172.21.15.64:0/2218928863 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c012630 con 0x564f9fde4390 2024-09-21T04:07:13.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.119+0000 7f5683298740 1 -- 172.21.15.64:0/2218928863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f9fde4390 msgr2=0x564fa01ac230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:13.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.119+0000 7f5683298740 1 --2- 172.21.15.64:0/2218928863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f9fde4390 0x564fa01ac230 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f566c012010 tx=0x7f566c0058b0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.120+0000 7f5683298740 1 -- 172.21.15.64:0/2218928863 shutdown_connections 2024-09-21T04:07:13.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.120+0000 7f5683298740 1 --2- 172.21.15.64:0/2218928863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f9fe05070 0x564fa01bb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.120+0000 7f5683298740 1 --2- 172.21.15.64:0/2218928863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f9fde4390 0x564fa01ac230 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.120+0000 7f5683298740 1 --2- 172.21.15.64:0/2218928863 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f9fddc1b0 0x564fa01abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.120+0000 7f5683298740 1 -- 172.21.15.64:0/2218928863 >> 172.21.15.64:0/2218928863 conn(0x564f9fdf9460 msgr2=0x564fa01a9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:13.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.120+0000 7f5683298740 1 -- 172.21.15.64:0/2218928863 shutdown_connections 2024-09-21T04:07:13.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.120+0000 7f5683298740 1 -- 172.21.15.64:0/2218928863 wait complete. 2024-09-21T04:07:13.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.120+0000 7f5683298740 1 -- start start 2024-09-21T04:07:13.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.120+0000 7f5683298740 1 Processor -- start 2024-09-21T04:07:13.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.121+0000 7f5683298740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f9fddc1b0 0x564fa01d6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:13.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.121+0000 7f5683298740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f9fde4390 0x564fa01d70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:13.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.121+0000 7f5683298740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f9fe05070 0x564fa01d7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:13.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.121+0000 7f5683298740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f56781abce0 con 0x564f9fde4390 2024-09-21T04:07:13.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.121+0000 7f5683298740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f56781ad150 con 0x564f9fddc1b0 2024-09-21T04:07:13.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.121+0000 7f5683298740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f56781ac3b0 con 0x564f9fe05070 2024-09-21T04:07:13.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.121+0000 7f5683298740 10 client.0 _ll_register_callbacks cb 0x7f56680046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:13.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.121+0000 7f5675d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f9fde4390 0x564fa01d70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:13.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.121+0000 7f5675d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f9fde4390 0x564fa01d70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35228/0 (socket says 172.21.15.64:35228) 2024-09-21T04:07:13.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.121+0000 7f5675d9b640 1 -- 172.21.15.64:0/307767060 learned_addr learned my addr 172.21.15.64:0/307767060 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:13.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.121+0000 7f5676d9d640 1 --2- 172.21.15.64:0/307767060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f9fe05070 0x564fa01d7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:13.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.121+0000 7f567659c640 1 --2- 172.21.15.64:0/307767060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f9fddc1b0 0x564fa01d6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:13.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.121+0000 7f5675d9b640 1 -- 172.21.15.64:0/307767060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f9fe05070 msgr2=0x564fa01d7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:13.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.121+0000 7f5675d9b640 1 --2- 172.21.15.64:0/307767060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f9fe05070 0x564fa01d7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.121+0000 7f5675d9b640 1 -- 172.21.15.64:0/307767060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f9fddc1b0 msgr2=0x564fa01d6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:13.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.121+0000 7f5675d9b640 1 --2- 172.21.15.64:0/307767060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f9fddc1b0 0x564fa01d6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.121+0000 7f5675d9b640 1 -- 172.21.15.64:0/307767060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fa01d7db0 con 0x564f9fde4390 2024-09-21T04:07:13.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.121+0000 7f567659c640 1 --2- 172.21.15.64:0/307767060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f9fddc1b0 0x564fa01d6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:07:13.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.121+0000 7f5675d9b640 1 --2- 172.21.15.64:0/307767060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f9fde4390 0x564fa01d70a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f566c031040 tx=0x7f566c009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8033dd88195cf05c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:13.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.121+0000 7f565dffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:07:13.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.122+0000 7f565dffb640 1 -- 172.21.15.64:0/307767060 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c033070 con 0x564f9fde4390 2024-09-21T04:07:13.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.122+0000 7f5683298740 1 -- 172.21.15.64:0/307767060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f566c009dc0 con 0x564f9fde4390 2024-09-21T04:07:13.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.122+0000 7f565dffb640 1 -- 172.21.15.64:0/307767060 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f566c0127e0 con 0x564f9fde4390 2024-09-21T04:07:13.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.122+0000 7f565dffb640 1 -- 172.21.15.64:0/307767060 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f566c0129e0 con 0x564f9fde4390 2024-09-21T04:07:13.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.122+0000 7f5683298740 10 client.16275 fetch_fsmap learned FSMap version 30 2024-09-21T04:07:13.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.122+0000 7f5683298740 1 -- 172.21.15.64:0/307767060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564fa023d2f0 con 0x564f9fde4390 2024-09-21T04:07:13.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.122+0000 7f565dffb640 1 -- 172.21.15.64:0/307767060 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f566c009dc0 con 0x564f9fde4390 2024-09-21T04:07:13.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.122+0000 7f565dffb640 1 -- 172.21.15.64:0/307767060 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f566c03e070 con 0x564f9fde4390 2024-09-21T04:07:13.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.122+0000 7f5683298740 10 client.16275 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:07:13.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.122+0000 7f5683298740 10 client.16275 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:07:13.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.122+0000 7f5683298740 20 client.16275 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:13.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.122+0000 7f5683298740 1 --2- 172.21.15.64:0/307767060 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564f9fd98da0 0x564fa023f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:13.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.122+0000 7f5683298740 4 client.16275 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T04:07:13.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.122+0000 7f5683298740 1 -- 172.21.15.64:0/307767060 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564fa0120850 con 0x564f9fd98da0 2024-09-21T04:07:13.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.123+0000 7f567659c640 1 --2- 172.21.15.64:0/307767060 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564f9fd98da0 0x564fa023f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:13.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.124+0000 7f567659c640 1 --2- 172.21.15.64:0/307767060 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564f9fd98da0 0x564fa023f940 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e98e675762215af server_cookie=f35eb2a3cecd5b2e in_seq=0 out_seq=0 2024-09-21T04:07:13.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.124+0000 7f565dffb640 10 client.16275 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:13.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.124+0000 7f565dffb640 1 -- 172.21.15.64:0/307767060 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1990 (crc 0 0 0) 0x564fa0120850 con 0x564f9fd98da0 2024-09-21T04:07:13.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.124+0000 7f565dffb640 10 client.16275 handle_command_reply: tid=0 2024-09-21T04:07:13.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.124+0000 7f5683298740 1 client.16275 shutdown 2024-09-21T04:07:13.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.124+0000 7f5683298740 1 -- 172.21.15.64:0/307767060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f9fde4390 msgr2=0x564fa01d70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:13.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.124+0000 7f5683298740 1 --2- 172.21.15.64:0/307767060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f9fde4390 0x564fa01d70a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f566c031040 tx=0x7f566c009620 comp rx=0 tx=0).stop 2024-09-21T04:07:13.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.124+0000 7f5683298740 1 -- 172.21.15.64:0/307767060 shutdown_connections 2024-09-21T04:07:13.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.124+0000 7f5683298740 1 --2- 172.21.15.64:0/307767060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f9fe05070 0x564fa01d7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.124+0000 7f5683298740 1 --2- 172.21.15.64:0/307767060 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564f9fd98da0 0x564fa023f940 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.124+0000 7f5683298740 1 --2- 172.21.15.64:0/307767060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f9fde4390 0x564fa01d70a0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.124+0000 7f565dffb640 0 client.16275 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:13.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.124+0000 7f5683298740 1 --2- 172.21.15.64:0/307767060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f9fddc1b0 0x564fa01d6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.125+0000 7f5683298740 1 -- 172.21.15.64:0/307767060 >> 172.21.15.64:0/307767060 conn(0x564f9fdf9460 msgr2=0x564fa01bb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:13.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.125+0000 7f5683298740 1 -- 172.21.15.64:0/307767060 shutdown_connections 2024-09-21T04:07:13.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.125+0000 7f5683298740 1 -- 172.21.15.64:0/307767060 wait complete. 2024-09-21T04:07:13.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.125+0000 7f5683298740 20 client.16275 trim_cache size 0 max 16384 2024-09-21T04:07:13.130 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:07:13.130 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T04:07:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-09-21T04:07:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: "bed40d88-2102-43a6-ac2d-b89d5a2264d0": { 2024-09-21T04:07:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:07:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "bed40d88-2102-43a6-ac2d-b89d5a2264d0", 2024-09-21T04:07:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-09-21T04:07:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:07:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:07:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_stats": { 2024-09-21T04:07:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: "bed40d88-2102-43a6-ac2d-b89d5a2264d0": { 2024-09-21T04:07:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:11.363340+0000", 2024-09-21T04:07:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:07:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:07:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:07:13.131 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:07:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:07:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:07:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: "e9242f88-6599-40eb-adf1-bdb3e946a1e0": { 2024-09-21T04:07:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:05:39.744646+0000", 2024-09-21T04:07:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:07:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:07:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:07:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:07:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:07:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:07:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: "6bde0fe5-f775-44b1-ad0c-f9e596574dc0": { 2024-09-21T04:07:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:06:23.835536+0000", 2024-09-21T04:07:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:07:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:07:13.132 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:07:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:07:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:07:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:07:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: "b52b950c-0f97-49ec-a37c-1a8e04348ce8": { 2024-09-21T04:07:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:54.003800+0000", 2024-09-21T04:07:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:07:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:07:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:07:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:07:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:07:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:07:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: "0ab717a7-36c6-4dca-94e7-af0bdf2aa112": { 2024-09-21T04:07:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:42.691086+0000", 2024-09-21T04:07:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:07:13.133 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:07:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:07:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:07:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:07:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:07:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:07:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:03:59.570174+0000", 2024-09-21T04:07:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:07:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:07:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:07:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:07:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:07:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:07:13.134 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:07:13.135 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:07:13.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.126+0000 7f567cd24640 1 -- 172.21.15.64:0/1239795380 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f56600b6700 msgr2=0x7f56600b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:13.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.126+0000 7f567cd24640 1 --2- 172.21.15.64:0/1239795380 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f56600b6700 0x7f56600b8c10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5670031bd0 tx=0x7f5670005c50 comp rx=0 tx=0).stop 2024-09-21T04:07:13.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.126+0000 7f567cd24640 1 -- 172.21.15.64:0/1239795380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5678104a30 msgr2=0x7f567819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:13.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.126+0000 7f567cd24640 1 --2- 172.21.15.64:0/1239795380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5678104a30 0x7f567819a470 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f566800e9f0 tx=0x7f566800eeb0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.126+0000 7f567cd24640 1 -- 172.21.15.64:0/1239795380 shutdown_connections 2024-09-21T04:07:13.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.126+0000 7f567cd24640 1 --2- 172.21.15.64:0/1239795380 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f56600b6700 0x7f56600b8c10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.126+0000 7f567cd24640 1 --2- 172.21.15.64:0/1239795380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f567810de60 0x7f567819fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.126+0000 7f567cd24640 1 --2- 172.21.15.64:0/1239795380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5678105380 0x7f567819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.126+0000 7f567cd24640 1 --2- 172.21.15.64:0/1239795380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5678104a30 0x7f567819a470 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:13.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.126+0000 7f567cd24640 1 -- 172.21.15.64:0/1239795380 >> 172.21.15.64:0/1239795380 conn(0x7f56780797b0 msgr2=0x7f5678107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:13.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.126+0000 7f567cd24640 1 -- 172.21.15.64:0/1239795380 shutdown_connections 2024-09-21T04:07:13.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:13.126+0000 7f567cd24640 1 -- 172.21.15.64:0/1239795380 wait complete. 2024-09-21T04:07:13.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:13 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:13.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:13 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:13.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:13 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:07:13.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:13 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:07:13.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:13 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:07:13.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:13 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:13.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:13 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:13.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:13 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:07:13.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:13 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:07:13.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:13 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:07:13.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:13 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:13.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:13 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:13.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:13 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:07:13.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:13 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:07:13.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:13 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:07:13.977 INFO:tasks.cephfs.filesystem:scrub status for tag:bed40d88-2102-43a6-ac2d-b89d5a2264d0 - {'path': '/', 'tag': 'bed40d88-2102-43a6-ac2d-b89d5a2264d0', 'options': 'recursive,force'} 2024-09-21T04:07:14.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: pgmap v417: 97 pgs: 97 active+clean; 415 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.9 MiB/s wr, 599 op/s 2024-09-21T04:07:14.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:14.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:14.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:14.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:14.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:14.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:14.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:14.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:14.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:14.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:14.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:14.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:14.759 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.759 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:14.759 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.759 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:14 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:14.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: pgmap v417: 97 pgs: 97 active+clean; 415 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.9 MiB/s wr, 599 op/s 2024-09-21T04:07:14.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:14.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:14.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:14.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:14.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:14.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:14.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:14.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:14.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:14.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:14.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:14.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:14.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:14.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.763 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:14 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:14.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: pgmap v417: 97 pgs: 97 active+clean; 415 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.9 MiB/s wr, 599 op/s 2024-09-21T04:07:14.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:14.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:14.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:14.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:14.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:14.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:14.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:14.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:14.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:14.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:14.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:14.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:14.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:14.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:14.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:14 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.759 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.759 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.759 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.759 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.759 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.759 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.759 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.759 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.759 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.759 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.759 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.759 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.759 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.759 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.760 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.760 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.760 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.760 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.760 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.760 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.760 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.760 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.760 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.760 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.760 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.760 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.760 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.760 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.761 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.761 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.761 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.761 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.761 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.761 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.761 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.761 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.761 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.761 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.761 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.762 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.762 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.762 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.762 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.762 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.762 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.762 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.762 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.762 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.762 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.762 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.762 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.763 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.763 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.763 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.763 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.763 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.763 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.763 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.763 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.763 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.763 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.763 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:07:15.763 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:15 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.764 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.764 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.764 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.764 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.764 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.764 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.764 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.764 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.764 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.764 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.765 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.765 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.765 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.765 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.765 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.765 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.765 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.765 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.765 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.765 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.765 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.765 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.765 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.765 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.765 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.766 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.766 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.766 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.766 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.766 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.766 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.766 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.766 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.766 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.766 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.766 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.766 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.766 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.766 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.767 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.767 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.767 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.767 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.767 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.767 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.767 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.767 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.767 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.767 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.767 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.767 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.767 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.767 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.768 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.768 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.768 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.768 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.768 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.768 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.768 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.768 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.768 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.768 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.768 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.768 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.768 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.768 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.768 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.769 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.769 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.769 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.769 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.769 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.769 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.769 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.769 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:07:15.769 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:15 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.809 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.809 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.809 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.809 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.809 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.809 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.809 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.809 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:15.809 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:07:15.809 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:15 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:16.657 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:16 smithi119 ceph-mon[34811]: pgmap v418: 97 pgs: 97 active+clean; 415 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.8 MiB/s wr, 484 op/s 2024-09-21T04:07:16.704 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:16 smithi131 ceph-mon[35193]: pgmap v418: 97 pgs: 97 active+clean; 415 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.8 MiB/s wr, 484 op/s 2024-09-21T04:07:16.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:16 smithi064 ceph-mon[31180]: pgmap v418: 97 pgs: 97 active+clean; 415 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.8 MiB/s wr, 484 op/s 2024-09-21T04:07:17.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:17 smithi119 ceph-mon[34811]: pgmap v419: 97 pgs: 97 active+clean; 418 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.5 MiB/s wr, 529 op/s 2024-09-21T04:07:17.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:17 smithi064 ceph-mon[31180]: pgmap v419: 97 pgs: 97 active+clean; 418 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.5 MiB/s wr, 529 op/s 2024-09-21T04:07:17.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:17 smithi131 ceph-mon[35193]: pgmap v419: 97 pgs: 97 active+clean; 418 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.5 MiB/s wr, 529 op/s 2024-09-21T04:07:20.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:19 smithi119 ceph-mon[34811]: pgmap v420: 97 pgs: 97 active+clean; 421 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 343 op/s 2024-09-21T04:07:20.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:19 smithi064 ceph-mon[31180]: pgmap v420: 97 pgs: 97 active+clean; 421 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 343 op/s 2024-09-21T04:07:20.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:19 smithi131 ceph-mon[35193]: pgmap v420: 97 pgs: 97 active+clean; 421 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 343 op/s 2024-09-21T04:07:21.173 INFO:tasks.workunit.client.0.smithi064.stdout: 70 109075 40 72961 30 55438 114 2024-09-21T04:07:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.480+0000 7f918ec94640 1 -- 172.21.15.64:0/3846792307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9188109d50 msgr2=0x7f9188110580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.480+0000 7f918ec94640 1 --2- 172.21.15.64:0/3846792307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9188109d50 0x7f9188110580 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9180009ca0 tx=0x7f918002a8c0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.480+0000 7f918ec94640 1 -- 172.21.15.64:0/3846792307 shutdown_connections 2024-09-21T04:07:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.480+0000 7f918ec94640 1 --2- 172.21.15.64:0/3846792307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9188109d50 0x7f9188110580 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.480+0000 7f918ec94640 1 --2- 172.21.15.64:0/3846792307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9188109400 0x7f9188109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.480+0000 7f918ec94640 1 --2- 172.21.15.64:0/3846792307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9188103ad0 0x7f9188103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.480+0000 7f918ec94640 1 -- 172.21.15.64:0/3846792307 >> 172.21.15.64:0/3846792307 conn(0x7f91880ff510 msgr2=0x7f9188101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.480+0000 7f918ec94640 1 -- 172.21.15.64:0/3846792307 shutdown_connections 2024-09-21T04:07:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.480+0000 7f918ec94640 1 -- 172.21.15.64:0/3846792307 wait complete. 2024-09-21T04:07:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.481+0000 7f918ec94640 1 Processor -- start 2024-09-21T04:07:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.481+0000 7f918ec94640 1 -- start start 2024-09-21T04:07:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.481+0000 7f918ec94640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9188103ad0 0x7f918819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.481+0000 7f918ec94640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9188109400 0x7f918819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.481+0000 7f918ec94640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9188109d50 0x7f918819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.481+0000 7f918ec94640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9188113a60 con 0x7f9188103ad0 2024-09-21T04:07:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.481+0000 7f918ec94640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f91881138e0 con 0x7f9188109d50 2024-09-21T04:07:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.481+0000 7f918ec94640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9188113be0 con 0x7f9188109400 2024-09-21T04:07:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.481+0000 7f918ca30640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9188103ad0 0x7f918819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.481+0000 7f918ca30640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9188103ad0 0x7f918819a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38272/0 (socket says 172.21.15.64:38272) 2024-09-21T04:07:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.481+0000 7f918ca30640 1 -- 172.21.15.64:0/2703409016 learned_addr learned my addr 172.21.15.64:0/2703409016 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.481+0000 7f9187fff640 1 --2- 172.21.15.64:0/2703409016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9188109400 0x7f918819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.481+0000 7f918d231640 1 --2- 172.21.15.64:0/2703409016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9188109d50 0x7f918819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.482+0000 7f918ca30640 1 -- 172.21.15.64:0/2703409016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9188109400 msgr2=0x7f918819aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.482+0000 7f918ca30640 1 --2- 172.21.15.64:0/2703409016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9188109400 0x7f918819aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.482+0000 7f918ca30640 1 -- 172.21.15.64:0/2703409016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9188109d50 msgr2=0x7f918819fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.482+0000 7f918ca30640 1 --2- 172.21.15.64:0/2703409016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9188109d50 0x7f918819fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.482+0000 7f918ca30640 1 -- 172.21.15.64:0/2703409016 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91881a0390 con 0x7f9188103ad0 2024-09-21T04:07:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.482+0000 7f918d231640 1 --2- 172.21.15.64:0/2703409016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9188109d50 0x7f918819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:07:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.482+0000 7f918ca30640 1 --2- 172.21.15.64:0/2703409016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9188103ad0 0x7f918819a460 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f917c00e9f0 tx=0x7f917c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.482+0000 7f9185ffb640 1 -- 172.21.15.64:0/2703409016 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c00cd80 con 0x7f9188103ad0 2024-09-21T04:07:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.482+0000 7f9185ffb640 1 -- 172.21.15.64:0/2703409016 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f917c0108e0 con 0x7f9188103ad0 2024-09-21T04:07:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.482+0000 7f918ec94640 1 -- 172.21.15.64:0/2703409016 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91881a0680 con 0x7f9188103ad0 2024-09-21T04:07:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.482+0000 7f9185ffb640 1 -- 172.21.15.64:0/2703409016 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c018980 con 0x7f9188103ad0 2024-09-21T04:07:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.482+0000 7f918ec94640 1 -- 172.21.15.64:0/2703409016 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91881a6e10 con 0x7f9188103ad0 2024-09-21T04:07:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.485+0000 7f9185ffb640 1 -- 172.21.15.64:0/2703409016 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f917c010430 con 0x7f9188103ad0 2024-09-21T04:07:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.485+0000 7f9195c06740 1 Processor -- start 2024-09-21T04:07:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.485+0000 7f9195c06740 1 -- start start 2024-09-21T04:07:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.485+0000 7f9195c06740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e216a31b0 0x558e21a489b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.485+0000 7f9195c06740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e2167a2f0 0x558e21a48ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.485+0000 7f9195c06740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e216824d0 0x558e21a49430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.485+0000 7f9195c06740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558e219bd6f0 con 0x558e216824d0 2024-09-21T04:07:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.485+0000 7f9195c06740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x558e219acb20 con 0x558e2167a2f0 2024-09-21T04:07:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.485+0000 7f9195c06740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x558e219b0930 con 0x558e216a31b0 2024-09-21T04:07:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.486+0000 7f9187fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e216824d0 0x558e21a49430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.486+0000 7f918ca30640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e2167a2f0 0x558e21a48ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.486+0000 7f9187fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e216824d0 0x558e21a49430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38278/0 (socket says 172.21.15.64:38278) 2024-09-21T04:07:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.486+0000 7f9187fff640 1 -- 172.21.15.64:0/951975028 learned_addr learned my addr 172.21.15.64:0/951975028 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.486+0000 7f918d231640 1 --2- 172.21.15.64:0/951975028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e216a31b0 0x558e21a489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.486+0000 7f9187fff640 1 -- 172.21.15.64:0/951975028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e216a31b0 msgr2=0x558e21a489b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.486+0000 7f9187fff640 1 --2- 172.21.15.64:0/951975028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e216a31b0 0x558e21a489b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.486+0000 7f9187fff640 1 -- 172.21.15.64:0/951975028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e2167a2f0 msgr2=0x558e21a48ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.486+0000 7f9187fff640 1 --2- 172.21.15.64:0/951975028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e2167a2f0 0x558e21a48ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.486+0000 7f9185ffb640 1 --2- 172.21.15.64:0/2703409016 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f91780b6700 0x7f91780b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.486+0000 7f9187fff640 1 -- 172.21.15.64:0/951975028 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e21972530 con 0x558e216824d0 2024-09-21T04:07:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.486+0000 7f918ca30640 1 --2- 172.21.15.64:0/951975028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e2167a2f0 0x558e21a48ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:07:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.486+0000 7f918d231640 1 --2- 172.21.15.64:0/2703409016 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f91780b6700 0x7f91780b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.486+0000 7f9187fff640 1 --2- 172.21.15.64:0/951975028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e216824d0 0x558e21a49430 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9174009ba0 tx=0x7f9174011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8e5b4c6fdf3997 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.486+0000 7f9185ffb640 1 -- 172.21.15.64:0/2703409016 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f917c0e7a00 con 0x7f9188103ad0 2024-09-21T04:07:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.486+0000 7f9172ffd640 1 -- 172.21.15.64:0/951975028 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9174004230 con 0x558e216824d0 2024-09-21T04:07:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.486+0000 7f9172ffd640 1 -- 172.21.15.64:0/951975028 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9174009e10 con 0x558e216824d0 2024-09-21T04:07:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.486+0000 7f9172ffd640 1 -- 172.21.15.64:0/951975028 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9174004d00 con 0x558e216824d0 2024-09-21T04:07:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.487+0000 7f9195c06740 1 -- 172.21.15.64:0/951975028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e216824d0 msgr2=0x558e21a49430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.487+0000 7f918d231640 1 --2- 172.21.15.64:0/2703409016 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f91780b6700 0x7f91780b8c10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9180006010 tx=0x7f9180005d80 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.487+0000 7f9195c06740 1 --2- 172.21.15.64:0/951975028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e216824d0 0x558e21a49430 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9174009ba0 tx=0x7f9174011810 comp rx=0 tx=0).stop 2024-09-21T04:07:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.487+0000 7f9195c06740 1 -- 172.21.15.64:0/951975028 shutdown_connections 2024-09-21T04:07:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.487+0000 7f9195c06740 1 --2- 172.21.15.64:0/951975028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e216824d0 0x558e21a49430 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.487+0000 7f9195c06740 1 --2- 172.21.15.64:0/951975028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e2167a2f0 0x558e21a48ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.487+0000 7f9195c06740 1 --2- 172.21.15.64:0/951975028 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e216a31b0 0x558e21a489b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.487+0000 7f9195c06740 1 -- 172.21.15.64:0/951975028 >> 172.21.15.64:0/951975028 conn(0x558e21697300 msgr2=0x558e21a468a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.487+0000 7f9195c06740 1 -- 172.21.15.64:0/951975028 shutdown_connections 2024-09-21T04:07:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.487+0000 7f9195c06740 1 -- 172.21.15.64:0/951975028 wait complete. 2024-09-21T04:07:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.488+0000 7f9195c06740 1 -- start start 2024-09-21T04:07:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.488+0000 7f9195c06740 1 Processor -- start 2024-09-21T04:07:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.488+0000 7f9195c06740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e2167a2f0 0x558e21a583b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.488+0000 7f9195c06740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e216824d0 0x558e21a588f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.488+0000 7f9195c06740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e216a31b0 0x558e21a58e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.488+0000 7f9195c06740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558e218e4ab0 con 0x558e2167a2f0 2024-09-21T04:07:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.488+0000 7f9195c06740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x558e21952730 con 0x558e216824d0 2024-09-21T04:07:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.488+0000 7f9195c06740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x558e21a5b510 con 0x558e216a31b0 2024-09-21T04:07:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.489+0000 7f918ca30640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e2167a2f0 0x558e21a583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.489+0000 7f9195c06740 10 client.0 _ll_register_callbacks cb 0x558e219a66b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.489+0000 7f9187fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e216824d0 0x558e21a588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.489+0000 7f918ca30640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e2167a2f0 0x558e21a583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38288/0 (socket says 172.21.15.64:38288) 2024-09-21T04:07:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.489+0000 7f9187fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e216824d0 0x558e21a588f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:34704/0 (socket says 172.21.15.64:34704) 2024-09-21T04:07:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.489+0000 7f918ca30640 1 -- 172.21.15.64:0/4146139356 learned_addr learned my addr 172.21.15.64:0/4146139356 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.489+0000 7f918d231640 1 --2- 172.21.15.64:0/4146139356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e216a31b0 0x558e21a58e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.489+0000 7f9187fff640 1 -- 172.21.15.64:0/4146139356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e216a31b0 msgr2=0x558e21a58e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.489+0000 7f9187fff640 1 --2- 172.21.15.64:0/4146139356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e216a31b0 0x558e21a58e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.489+0000 7f9187fff640 1 -- 172.21.15.64:0/4146139356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e2167a2f0 msgr2=0x558e21a583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.489+0000 7f9187fff640 1 --2- 172.21.15.64:0/4146139356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e2167a2f0 0x558e21a583b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.489+0000 7f9187fff640 1 -- 172.21.15.64:0/4146139356 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e2170e790 con 0x558e216824d0 2024-09-21T04:07:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.489+0000 7f9187fff640 1 --2- 172.21.15.64:0/4146139356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e216824d0 0x558e21a588f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9174009ba0 tx=0x7f9174003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2851cca2835cfb21 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.489+0000 7f9167fff640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:07:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.489+0000 7f9167fff640 1 -- 172.21.15.64:0/4146139356 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9174027020 con 0x558e216824d0 2024-09-21T04:07:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.490+0000 7f9195c06740 1 -- 172.21.15.64:0/4146139356 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9174009e10 con 0x558e216824d0 2024-09-21T04:07:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.490+0000 7f9167fff640 1 -- 172.21.15.64:0/4146139356 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f9174003c10 con 0x558e216824d0 2024-09-21T04:07:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.490+0000 7f9167fff640 1 -- 172.21.15.64:0/4146139356 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9174032050 con 0x558e216824d0 2024-09-21T04:07:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.490+0000 7f9195c06740 10 client.25988 fetch_fsmap learned FSMap version 30 2024-09-21T04:07:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.490+0000 7f9195c06740 1 -- 172.21.15.64:0/4146139356 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558e21787620 con 0x558e216824d0 2024-09-21T04:07:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.490+0000 7f9167fff640 1 -- 172.21.15.64:0/4146139356 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9174009e10 con 0x558e216824d0 2024-09-21T04:07:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.490+0000 7f9167fff640 1 -- 172.21.15.64:0/4146139356 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f9174018070 con 0x558e216824d0 2024-09-21T04:07:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.490+0000 7f9195c06740 10 client.25988 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:07:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.490+0000 7f9195c06740 10 client.25988 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:07:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.490+0000 7f9195c06740 20 client.25988 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.490+0000 7f9195c06740 1 --2- 172.21.15.64:0/4146139356 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x558e21661d80 0x558e21a6a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.490+0000 7f9195c06740 4 client.25988 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:07:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.490+0000 7f9195c06740 1 -- 172.21.15.64:0/4146139356 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e219bf640 con 0x558e21661d80 2024-09-21T04:07:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.490+0000 7f918ca30640 1 --2- 172.21.15.64:0/4146139356 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x558e21661d80 0x558e21a6a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.491+0000 7f9167fff640 10 client.25988 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.491+0000 7f918ca30640 1 --2- 172.21.15.64:0/4146139356 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x558e21661d80 0x558e21a6a440 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce5e9fed93342753 server_cookie=aa5e13afe3477cd2 in_seq=0 out_seq=0 2024-09-21T04:07:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.492+0000 7f9167fff640 1 -- 172.21.15.64:0/4146139356 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558e219bf640 con 0x558e21661d80 2024-09-21T04:07:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.492+0000 7f9167fff640 10 client.25988 handle_command_reply: tid=0 2024-09-21T04:07:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.492+0000 7f9195c06740 1 client.25988 shutdown 2024-09-21T04:07:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.492+0000 7f9195c06740 1 -- 172.21.15.64:0/4146139356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e216824d0 msgr2=0x558e21a588f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.492+0000 7f9195c06740 1 --2- 172.21.15.64:0/4146139356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e216824d0 0x558e21a588f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9174009ba0 tx=0x7f9174003820 comp rx=0 tx=0).stop 2024-09-21T04:07:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.492+0000 7f9195c06740 1 -- 172.21.15.64:0/4146139356 shutdown_connections 2024-09-21T04:07:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.492+0000 7f9195c06740 1 --2- 172.21.15.64:0/4146139356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e216a31b0 0x558e21a58e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.492+0000 7f9195c06740 1 --2- 172.21.15.64:0/4146139356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e216824d0 0x558e21a588f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.492+0000 7f9195c06740 1 --2- 172.21.15.64:0/4146139356 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x558e21661d80 0x558e21a6a440 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.492+0000 7f9195c06740 1 --2- 172.21.15.64:0/4146139356 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e2167a2f0 0x558e21a583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.492+0000 7f9167fff640 0 client.25988 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.492+0000 7f9195c06740 1 -- 172.21.15.64:0/4146139356 >> 172.21.15.64:0/4146139356 conn(0x558e21697300 msgr2=0x558e21a4a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.492+0000 7f9195c06740 1 -- 172.21.15.64:0/4146139356 shutdown_connections 2024-09-21T04:07:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.492+0000 7f9195c06740 1 -- 172.21.15.64:0/4146139356 wait complete. 2024-09-21T04:07:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.492+0000 7f9195c06740 20 client.25988 trim_cache size 0 max 16384 2024-09-21T04:07:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.508+0000 7f9195c06740 1 Processor -- start 2024-09-21T04:07:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.509+0000 7f9195c06740 1 -- start start 2024-09-21T04:07:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.509+0000 7f9195c06740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e2167a2f0 0x558e21a64740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.509+0000 7f9195c06740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e216824d0 0x558e21a58890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.509+0000 7f9195c06740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e216a31b0 0x558e21a58dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.509+0000 7f9195c06740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f91881affa0 con 0x558e2167a2f0 2024-09-21T04:07:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.509+0000 7f9195c06740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f91881ad7e0 con 0x558e216824d0 2024-09-21T04:07:21.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.509+0000 7f9195c06740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f91881ab630 con 0x558e216a31b0 2024-09-21T04:07:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.509+0000 7f918ca30640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e2167a2f0 0x558e21a64740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.509+0000 7f918ca30640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e2167a2f0 0x558e21a64740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38304/0 (socket says 172.21.15.64:38304) 2024-09-21T04:07:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.509+0000 7f918ca30640 1 -- 172.21.15.64:0/3627253660 learned_addr learned my addr 172.21.15.64:0/3627253660 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.509+0000 7f9187fff640 1 --2- 172.21.15.64:0/3627253660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e216824d0 0x558e21a58890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.509+0000 7f918d231640 1 --2- 172.21.15.64:0/3627253660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e216a31b0 0x558e21a58dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.509+0000 7f918ca30640 1 -- 172.21.15.64:0/3627253660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e216a31b0 msgr2=0x558e21a58dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.509+0000 7f918ca30640 1 --2- 172.21.15.64:0/3627253660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e216a31b0 0x558e21a58dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.509+0000 7f918ca30640 1 -- 172.21.15.64:0/3627253660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e216824d0 msgr2=0x558e21a58890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.509+0000 7f918ca30640 1 --2- 172.21.15.64:0/3627253660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e216824d0 0x558e21a58890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.509+0000 7f918ca30640 1 -- 172.21.15.64:0/3627253660 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e21a4b810 con 0x558e2167a2f0 2024-09-21T04:07:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.509+0000 7f918ca30640 1 --2- 172.21.15.64:0/3627253660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e2167a2f0 0x558e21a64740 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f917c0ad5d0 tx=0x7f917c0eb550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=810036986601dfcd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.510+0000 7f91727fc640 1 -- 172.21.15.64:0/3627253660 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c0f1040 con 0x558e2167a2f0 2024-09-21T04:07:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.510+0000 7f91727fc640 1 -- 172.21.15.64:0/3627253660 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f917c010720 con 0x558e2167a2f0 2024-09-21T04:07:21.513 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.510+0000 7f91727fc640 1 -- 172.21.15.64:0/3627253660 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c0ad040 con 0x558e2167a2f0 2024-09-21T04:07:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.510+0000 7f9195c06740 1 -- 172.21.15.64:0/3627253660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e2167a2f0 msgr2=0x558e21a64740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.510+0000 7f9195c06740 1 --2- 172.21.15.64:0/3627253660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e2167a2f0 0x558e21a64740 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f917c0ad5d0 tx=0x7f917c0eb550 comp rx=0 tx=0).stop 2024-09-21T04:07:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.511+0000 7f9195c06740 1 -- 172.21.15.64:0/3627253660 shutdown_connections 2024-09-21T04:07:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.511+0000 7f9195c06740 1 --2- 172.21.15.64:0/3627253660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e216a31b0 0x558e21a58dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.511+0000 7f9195c06740 1 --2- 172.21.15.64:0/3627253660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e216824d0 0x558e21a58890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.511+0000 7f9195c06740 1 --2- 172.21.15.64:0/3627253660 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e2167a2f0 0x558e21a64740 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.511+0000 7f9195c06740 1 -- 172.21.15.64:0/3627253660 >> 172.21.15.64:0/3627253660 conn(0x558e21697300 msgr2=0x558e21a56c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.511+0000 7f9195c06740 1 -- 172.21.15.64:0/3627253660 shutdown_connections 2024-09-21T04:07:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.511+0000 7f9195c06740 1 -- 172.21.15.64:0/3627253660 wait complete. 2024-09-21T04:07:21.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.511+0000 7f9195c06740 1 -- start start 2024-09-21T04:07:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.511+0000 7f9195c06740 1 Processor -- start 2024-09-21T04:07:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.512+0000 7f9195c06740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e2167a2f0 0x558e21a5bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.512+0000 7f9195c06740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e216824d0 0x558e21a5c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.512+0000 7f9195c06740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e216a31b0 0x558e21ad2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.512+0000 7f9195c06740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f91881abca0 con 0x558e2167a2f0 2024-09-21T04:07:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.512+0000 7f9195c06740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f91881ad110 con 0x558e216a31b0 2024-09-21T04:07:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.512+0000 7f9195c06740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f91881ac370 con 0x558e216824d0 2024-09-21T04:07:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.512+0000 7f9195c06740 10 client.0 _ll_register_callbacks cb 0x7f917c0eb420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.512+0000 7f918ca30640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e2167a2f0 0x558e21a5bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.512+0000 7f918ca30640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e2167a2f0 0x558e21a5bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38318/0 (socket says 172.21.15.64:38318) 2024-09-21T04:07:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.512+0000 7f918ca30640 1 -- 172.21.15.64:0/291531564 learned_addr learned my addr 172.21.15.64:0/291531564 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.512+0000 7f918d231640 1 --2- 172.21.15.64:0/291531564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e216a31b0 0x558e21ad2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.512+0000 7f9187fff640 1 --2- 172.21.15.64:0/291531564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e216824d0 0x558e21a5c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:21.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.512+0000 7f918ca30640 1 -- 172.21.15.64:0/291531564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e216824d0 msgr2=0x558e21a5c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.512+0000 7f918ca30640 1 --2- 172.21.15.64:0/291531564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e216824d0 0x558e21a5c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.512+0000 7f918ca30640 1 -- 172.21.15.64:0/291531564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e216a31b0 msgr2=0x558e21ad2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.512+0000 7f918ca30640 1 --2- 172.21.15.64:0/291531564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e216a31b0 0x558e21ad2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.512+0000 7f918ca30640 1 -- 172.21.15.64:0/291531564 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e21ad3280 con 0x558e2167a2f0 2024-09-21T04:07:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.512+0000 7f918ca30640 1 --2- 172.21.15.64:0/291531564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e2167a2f0 0x558e21a5bfa0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f917c0aab40 tx=0x7f917c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8f08e7f2eabdca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.512+0000 7f9167fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:07:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.512+0000 7f9167fff640 1 -- 172.21.15.64:0/291531564 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c0ee030 con 0x558e2167a2f0 2024-09-21T04:07:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.512+0000 7f9167fff640 1 -- 172.21.15.64:0/291531564 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f917c018b20 con 0x558e2167a2f0 2024-09-21T04:07:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.512+0000 7f9195c06740 1 -- 172.21.15.64:0/291531564 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558e219bb0f0 con 0x558e2167a2f0 2024-09-21T04:07:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.513+0000 7f9167fff640 1 -- 172.21.15.64:0/291531564 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f917c0ad220 con 0x558e2167a2f0 2024-09-21T04:07:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.513+0000 7f9195c06740 10 client.16293 fetch_fsmap learned FSMap version 30 2024-09-21T04:07:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.513+0000 7f9195c06740 1 -- 172.21.15.64:0/291531564 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558e21ad44f0 con 0x558e2167a2f0 2024-09-21T04:07:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.513+0000 7f9167fff640 1 -- 172.21.15.64:0/291531564 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558e219bb0f0 con 0x558e2167a2f0 2024-09-21T04:07:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.513+0000 7f9167fff640 1 -- 172.21.15.64:0/291531564 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f917c0a9220 con 0x558e2167a2f0 2024-09-21T04:07:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.513+0000 7f9195c06740 10 client.16293 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T04:07:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.513+0000 7f9195c06740 10 client.16293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:07:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.513+0000 7f9195c06740 20 client.16293 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.513+0000 7f9195c06740 1 --2- 172.21.15.64:0/291531564 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x558e216371c0 0x558e21ad6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.513+0000 7f9195c06740 4 client.16293 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:07:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.513+0000 7f9195c06740 1 -- 172.21.15.64:0/291531564 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558e219b8520 con 0x558e216371c0 2024-09-21T04:07:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.513+0000 7f9187fff640 1 --2- 172.21.15.64:0/291531564 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x558e216371c0 0x558e21ad6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.514+0000 7f9167fff640 10 client.16293 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.514+0000 7f9187fff640 1 --2- 172.21.15.64:0/291531564 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x558e216371c0 0x558e21ad6c50 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3160383882d19fe server_cookie=3562aae9ebfbdfb3 in_seq=0 out_seq=0 2024-09-21T04:07:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.564+0000 7f9167fff640 1 -- 172.21.15.64:0/291531564 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558e219b8520 con 0x558e216371c0 2024-09-21T04:07:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.564+0000 7f9167fff640 10 client.16293 handle_command_reply: tid=0 2024-09-21T04:07:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.564+0000 7f9195c06740 1 client.16293 shutdown 2024-09-21T04:07:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.565+0000 7f9195c06740 1 -- 172.21.15.64:0/291531564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e2167a2f0 msgr2=0x558e21a5bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.565+0000 7f9195c06740 1 --2- 172.21.15.64:0/291531564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e2167a2f0 0x558e21a5bfa0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f917c0aab40 tx=0x7f917c000bc0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.565+0000 7f9195c06740 1 -- 172.21.15.64:0/291531564 shutdown_connections 2024-09-21T04:07:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.565+0000 7f9195c06740 1 --2- 172.21.15.64:0/291531564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558e216a31b0 0x558e21ad2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.565+0000 7f9195c06740 1 --2- 172.21.15.64:0/291531564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558e216824d0 0x558e21a5c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.565+0000 7f9195c06740 1 --2- 172.21.15.64:0/291531564 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x558e216371c0 0x558e21ad6c50 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.565+0000 7f9195c06740 1 --2- 172.21.15.64:0/291531564 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558e2167a2f0 0x558e21a5bfa0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.565+0000 7f9167fff640 0 client.16293 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.565+0000 7f9195c06740 1 -- 172.21.15.64:0/291531564 >> 172.21.15.64:0/291531564 conn(0x558e21697300 msgr2=0x558e21a530f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.565+0000 7f9195c06740 1 -- 172.21.15.64:0/291531564 shutdown_connections 2024-09-21T04:07:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.565+0000 7f9195c06740 1 -- 172.21.15.64:0/291531564 wait complete. 2024-09-21T04:07:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.565+0000 7f9195c06740 20 client.16293 trim_cache size 0 max 16384 2024-09-21T04:07:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:07:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:07:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:07:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:07:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.566+0000 7f918ec94640 1 -- 172.21.15.64:0/2703409016 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f91780b6700 msgr2=0x7f91780b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.566+0000 7f918ec94640 1 --2- 172.21.15.64:0/2703409016 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f91780b6700 0x7f91780b8c10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9180006010 tx=0x7f9180005d80 comp rx=0 tx=0).stop 2024-09-21T04:07:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.566+0000 7f918ec94640 1 -- 172.21.15.64:0/2703409016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9188103ad0 msgr2=0x7f918819a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.566+0000 7f918ec94640 1 --2- 172.21.15.64:0/2703409016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9188103ad0 0x7f918819a460 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f917c00e9f0 tx=0x7f917c00eeb0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.566+0000 7f918ec94640 1 -- 172.21.15.64:0/2703409016 shutdown_connections 2024-09-21T04:07:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.566+0000 7f918ec94640 1 --2- 172.21.15.64:0/2703409016 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f91780b6700 0x7f91780b8c10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.566+0000 7f918ec94640 1 --2- 172.21.15.64:0/2703409016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9188109d50 0x7f918819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.566+0000 7f918ec94640 1 --2- 172.21.15.64:0/2703409016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9188109400 0x7f918819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.566+0000 7f918ec94640 1 --2- 172.21.15.64:0/2703409016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9188103ad0 0x7f918819a460 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.566+0000 7f918ec94640 1 -- 172.21.15.64:0/2703409016 >> 172.21.15.64:0/2703409016 conn(0x7f91880ff510 msgr2=0x7f91881069d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.566+0000 7f918ec94640 1 -- 172.21.15.64:0/2703409016 shutdown_connections 2024-09-21T04:07:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:21.566+0000 7f918ec94640 1 -- 172.21.15.64:0/2703409016 wait complete. 2024-09-21T04:07:22.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:21 smithi119 ceph-mon[34811]: pgmap v421: 97 pgs: 97 active+clean; 421 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 313 op/s 2024-09-21T04:07:22.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:21 smithi064 ceph-mon[31180]: pgmap v421: 97 pgs: 97 active+clean; 421 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 313 op/s 2024-09-21T04:07:22.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:21 smithi131 ceph-mon[35193]: pgmap v421: 97 pgs: 97 active+clean; 421 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 313 op/s 2024-09-21T04:07:23.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:22 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:23.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:22 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:23.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:22 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:07:24.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:23 smithi119 ceph-mon[34811]: pgmap v422: 97 pgs: 97 active+clean; 427 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 393 op/s 2024-09-21T04:07:24.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:23 smithi064 ceph-mon[31180]: pgmap v422: 97 pgs: 97 active+clean; 427 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 393 op/s 2024-09-21T04:07:24.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:23 smithi131 ceph-mon[35193]: pgmap v422: 97 pgs: 97 active+clean; 427 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 393 op/s 2024-09-21T04:07:25.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:24 smithi119 ceph-mon[34811]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:07:25.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:24 smithi064 ceph-mon[31180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:07:25.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:24 smithi131 ceph-mon[35193]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:07:26.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:25 smithi119 ceph-mon[34811]: pgmap v423: 97 pgs: 97 active+clean; 427 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 259 op/s 2024-09-21T04:07:26.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:25 smithi064 ceph-mon[31180]: pgmap v423: 97 pgs: 97 active+clean; 427 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 259 op/s 2024-09-21T04:07:26.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:25 smithi131 ceph-mon[35193]: pgmap v423: 97 pgs: 97 active+clean; 427 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 259 op/s 2024-09-21T04:07:28.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:27 smithi119 ceph-mon[34811]: pgmap v424: 97 pgs: 97 active+clean; 435 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 355 op/s 2024-09-21T04:07:28.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:27 smithi064 ceph-mon[31180]: pgmap v424: 97 pgs: 97 active+clean; 435 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 355 op/s 2024-09-21T04:07:28.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:27 smithi131 ceph-mon[35193]: pgmap v424: 97 pgs: 97 active+clean; 435 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 355 op/s 2024-09-21T04:07:30.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:29 smithi119 ceph-mon[34811]: pgmap v425: 97 pgs: 97 active+clean; 440 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 336 op/s 2024-09-21T04:07:30.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:29 smithi119 ceph-mon[34811]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:07:30.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:29 smithi064 ceph-mon[31180]: pgmap v425: 97 pgs: 97 active+clean; 440 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 336 op/s 2024-09-21T04:07:30.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:29 smithi064 ceph-mon[31180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:07:30.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:29 smithi131 ceph-mon[35193]: pgmap v425: 97 pgs: 97 active+clean; 440 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 336 op/s 2024-09-21T04:07:30.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:29 smithi131 ceph-mon[35193]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:07:31.173 INFO:tasks.workunit.client.0.smithi064.stdout: 73 3851 45 2323 138 1927 156 2024-09-21T04:07:32.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:31 smithi119 ceph-mon[34811]: pgmap v426: 97 pgs: 97 active+clean; 440 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 256 op/s 2024-09-21T04:07:32.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:31 smithi064 ceph-mon[31180]: pgmap v426: 97 pgs: 97 active+clean; 440 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 256 op/s 2024-09-21T04:07:32.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:31 smithi131 ceph-mon[35193]: pgmap v426: 97 pgs: 97 active+clean; 440 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 256 op/s 2024-09-21T04:07:34.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:33 smithi119 ceph-mon[34811]: pgmap v427: 97 pgs: 97 active+clean; 453 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 321 op/s 2024-09-21T04:07:34.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:33 smithi064 ceph-mon[31180]: pgmap v427: 97 pgs: 97 active+clean; 453 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 321 op/s 2024-09-21T04:07:34.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:33 smithi131 ceph-mon[35193]: pgmap v427: 97 pgs: 97 active+clean; 453 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 321 op/s 2024-09-21T04:07:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.696+0000 7f8e36430640 1 -- 172.21.15.64:0/4243405239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e301029b0 msgr2=0x7f8e30102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.696+0000 7f8e36430640 1 --2- 172.21.15.64:0/4243405239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e301029b0 0x7f8e30102dc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009900 tx=0x7f8e2002a410 comp rx=0 tx=0).stop 2024-09-21T04:07:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.696+0000 7f8e36430640 1 -- 172.21.15.64:0/4243405239 shutdown_connections 2024-09-21T04:07:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.696+0000 7f8e36430640 1 --2- 172.21.15.64:0/4243405239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e30108c30 0x7f8e3010f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.696+0000 7f8e36430640 1 --2- 172.21.15.64:0/4243405239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e301082e0 0x7f8e301086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.696+0000 7f8e36430640 1 --2- 172.21.15.64:0/4243405239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e301029b0 0x7f8e30102dc0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.696+0000 7f8e36430640 1 -- 172.21.15.64:0/4243405239 >> 172.21.15.64:0/4243405239 conn(0x7f8e3007a740 msgr2=0x7f8e3007ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.696+0000 7f8e36430640 1 -- 172.21.15.64:0/4243405239 shutdown_connections 2024-09-21T04:07:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.696+0000 7f8e36430640 1 -- 172.21.15.64:0/4243405239 wait complete. 2024-09-21T04:07:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.697+0000 7f8e36430640 1 Processor -- start 2024-09-21T04:07:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.697+0000 7f8e36430640 1 -- start start 2024-09-21T04:07:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.698+0000 7f8e36430640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e301029b0 0x7f8e3019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.698+0000 7f8e36430640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e301082e0 0x7f8e3019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.698+0000 7f8e36430640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e30108c30 0x7f8e3019fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.698+0000 7f8e36430640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8e301128f0 con 0x7f8e301082e0 2024-09-21T04:07:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.698+0000 7f8e36430640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8e30112770 con 0x7f8e30108c30 2024-09-21T04:07:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.698+0000 7f8e36430640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8e30112a70 con 0x7f8e301029b0 2024-09-21T04:07:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.698+0000 7f8e2f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e301082e0 0x7f8e3019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.698+0000 7f8e2f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e301082e0 0x7f8e3019ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37684/0 (socket says 172.21.15.64:37684) 2024-09-21T04:07:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.698+0000 7f8e2f7fe640 1 -- 172.21.15.64:0/3236318925 learned_addr learned my addr 172.21.15.64:0/3236318925 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.698+0000 7f8e349cd640 1 --2- 172.21.15.64:0/3236318925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e30108c30 0x7f8e3019fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.698+0000 7f8e2ffff640 1 --2- 172.21.15.64:0/3236318925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e301029b0 0x7f8e3019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.698+0000 7f8e349cd640 1 -- 172.21.15.64:0/3236318925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e301029b0 msgr2=0x7f8e3019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.698+0000 7f8e349cd640 1 --2- 172.21.15.64:0/3236318925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e301029b0 0x7f8e3019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.698+0000 7f8e349cd640 1 -- 172.21.15.64:0/3236318925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e301082e0 msgr2=0x7f8e3019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.698+0000 7f8e349cd640 1 --2- 172.21.15.64:0/3236318925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e301082e0 0x7f8e3019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.698+0000 7f8e349cd640 1 -- 172.21.15.64:0/3236318925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e301a04b0 con 0x7f8e30108c30 2024-09-21T04:07:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.698+0000 7f8e2f7fe640 1 --2- 172.21.15.64:0/3236318925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e301082e0 0x7f8e3019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:07:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.698+0000 7f8e349cd640 1 --2- 172.21.15.64:0/3236318925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e30108c30 0x7f8e3019fdb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8e28009fd0 tx=0x7f8e2800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.699+0000 7f8e2d7fa640 1 -- 172.21.15.64:0/3236318925 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2800ce20 con 0x7f8e30108c30 2024-09-21T04:07:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.699+0000 7f8e2d7fa640 1 -- 172.21.15.64:0/3236318925 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e28005020 con 0x7f8e30108c30 2024-09-21T04:07:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.699+0000 7f8e36430640 1 -- 172.21.15.64:0/3236318925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e301a07a0 con 0x7f8e30108c30 2024-09-21T04:07:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.699+0000 7f8e2d7fa640 1 -- 172.21.15.64:0/3236318925 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e28010690 con 0x7f8e30108c30 2024-09-21T04:07:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.699+0000 7f8e36430640 1 -- 172.21.15.64:0/3236318925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e301a6f30 con 0x7f8e30108c30 2024-09-21T04:07:34.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.701+0000 7f8e3d47e740 1 Processor -- start 2024-09-21T04:07:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.701+0000 7f8e2d7fa640 1 -- 172.21.15.64:0/3236318925 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f8e28010830 con 0x7f8e30108c30 2024-09-21T04:07:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.702+0000 7f8e3d47e740 1 -- start start 2024-09-21T04:07:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.702+0000 7f8e3d47e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638711dc1b0 0x5638715819b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.702+0000 7f8e3d47e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638711b32f0 0x563871581ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.702+0000 7f8e3d47e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638711bb4d0 0x563871582430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.702+0000 7f8e3d47e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5638714f66f0 con 0x5638711dc1b0 2024-09-21T04:07:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.702+0000 7f8e3d47e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5638714e5b20 con 0x5638711bb4d0 2024-09-21T04:07:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.702+0000 7f8e3d47e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5638714e9930 con 0x5638711b32f0 2024-09-21T04:07:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.702+0000 7f8e2f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638711dc1b0 0x5638715819b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.702+0000 7f8e2f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638711dc1b0 0x5638715819b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37692/0 (socket says 172.21.15.64:37692) 2024-09-21T04:07:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.702+0000 7f8e2f7fe640 1 -- 172.21.15.64:0/1331728242 learned_addr learned my addr 172.21.15.64:0/1331728242 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.702+0000 7f8e2f7fe640 1 --2- 172.21.15.64:0/1331728242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638711bb4d0 0x563871582430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.702+0000 7f8e2ffff640 1 --2- 172.21.15.64:0/1331728242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638711b32f0 0x563871581ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.702+0000 7f8e2f7fe640 1 -- 172.21.15.64:0/1331728242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638711b32f0 msgr2=0x563871581ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.702+0000 7f8e2f7fe640 1 --2- 172.21.15.64:0/1331728242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638711b32f0 0x563871581ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.702+0000 7f8e2d7fa640 1 --2- 172.21.15.64:0/3236318925 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8e180b6680 0x7f8e180b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.702+0000 7f8e2f7fe640 1 -- 172.21.15.64:0/1331728242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638711bb4d0 msgr2=0x563871582430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.702+0000 7f8e2f7fe640 1 --2- 172.21.15.64:0/1331728242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638711bb4d0 0x563871582430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.702+0000 7f8e2f7fe640 1 -- 172.21.15.64:0/1331728242 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638714ab530 con 0x5638711dc1b0 2024-09-21T04:07:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.703+0000 7f8e2f7fe640 1 --2- 172.21.15.64:0/1331728242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638711dc1b0 0x5638715819b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8e24013c90 tx=0x7f8e24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23750e69266be3f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.703+0000 7f8e349cd640 1 --2- 172.21.15.64:0/3236318925 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8e180b6680 0x7f8e180b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.703+0000 7f8e2d7fa640 1 -- 172.21.15.64:0/3236318925 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f8e280e7e40 con 0x7f8e30108c30 2024-09-21T04:07:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.703+0000 7f8e167fc640 1 -- 172.21.15.64:0/1331728242 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24013e40 con 0x5638711dc1b0 2024-09-21T04:07:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.703+0000 7f8e167fc640 1 -- 172.21.15.64:0/1331728242 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e24013170 con 0x5638711dc1b0 2024-09-21T04:07:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.703+0000 7f8e167fc640 1 -- 172.21.15.64:0/1331728242 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24010bd0 con 0x5638711dc1b0 2024-09-21T04:07:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.704+0000 7f8e3d47e740 1 -- 172.21.15.64:0/1331728242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638711dc1b0 msgr2=0x5638715819b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.704+0000 7f8e3d47e740 1 --2- 172.21.15.64:0/1331728242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638711dc1b0 0x5638715819b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8e24013c90 tx=0x7f8e24009bb0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.704+0000 7f8e349cd640 1 --2- 172.21.15.64:0/3236318925 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8e180b6680 0x7f8e180b8b90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8e280e5610 tx=0x7f8e280053b0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.704+0000 7f8e3d47e740 1 -- 172.21.15.64:0/1331728242 shutdown_connections 2024-09-21T04:07:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.704+0000 7f8e3d47e740 1 --2- 172.21.15.64:0/1331728242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638711bb4d0 0x563871582430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.704+0000 7f8e3d47e740 1 --2- 172.21.15.64:0/1331728242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638711b32f0 0x563871581ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.704+0000 7f8e3d47e740 1 --2- 172.21.15.64:0/1331728242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638711dc1b0 0x5638715819b0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.704+0000 7f8e3d47e740 1 -- 172.21.15.64:0/1331728242 >> 172.21.15.64:0/1331728242 conn(0x5638711d0300 msgr2=0x56387157f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.704+0000 7f8e3d47e740 1 -- 172.21.15.64:0/1331728242 shutdown_connections 2024-09-21T04:07:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.704+0000 7f8e3d47e740 1 -- 172.21.15.64:0/1331728242 wait complete. 2024-09-21T04:07:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.704+0000 7f8e3d47e740 1 -- start start 2024-09-21T04:07:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.704+0000 7f8e3d47e740 1 Processor -- start 2024-09-21T04:07:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.705+0000 7f8e3d47e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638711b32f0 0x56387158d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.705+0000 7f8e3d47e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638711bb4d0 0x56387158d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.705+0000 7f8e3d47e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638711dc1b0 0x563871599420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.705+0000 7f8e3d47e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56387141dab0 con 0x5638711bb4d0 2024-09-21T04:07:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.705+0000 7f8e3d47e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56387148b730 con 0x5638711b32f0 2024-09-21T04:07:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.705+0000 7f8e3d47e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x563871590130 con 0x5638711dc1b0 2024-09-21T04:07:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.705+0000 7f8e3d47e740 10 client.0 _ll_register_callbacks cb 0x5638714df6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.705+0000 7f8e2ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638711bb4d0 0x56387158d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.705+0000 7f8e2ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638711bb4d0 0x56387158d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37702/0 (socket says 172.21.15.64:37702) 2024-09-21T04:07:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.705+0000 7f8e2ffff640 1 -- 172.21.15.64:0/1801745759 learned_addr learned my addr 172.21.15.64:0/1801745759 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.705+0000 7f8e2f7fe640 1 --2- 172.21.15.64:0/1801745759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638711b32f0 0x56387158d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.705+0000 7f8e2f7fe640 1 --2- 172.21.15.64:0/1801745759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638711dc1b0 0x563871599420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.705+0000 7f8e2ffff640 1 -- 172.21.15.64:0/1801745759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638711dc1b0 msgr2=0x563871599420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.705+0000 7f8e2ffff640 1 --2- 172.21.15.64:0/1801745759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638711dc1b0 0x563871599420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.705+0000 7f8e2ffff640 1 -- 172.21.15.64:0/1801745759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638711b32f0 msgr2=0x56387158d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.705+0000 7f8e2ffff640 1 --2- 172.21.15.64:0/1801745759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638711b32f0 0x56387158d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.705+0000 7f8e2ffff640 1 -- 172.21.15.64:0/1801745759 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563871247790 con 0x5638711bb4d0 2024-09-21T04:07:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.705+0000 7f8e2f7fe640 1 --2- 172.21.15.64:0/1801745759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638711b32f0 0x56387158d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:07:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.705+0000 7f8e2ffff640 1 --2- 172.21.15.64:0/1801745759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638711bb4d0 0x56387158d5d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8e2002a930 tx=0x7f8e200316b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575c905d1ea2b217 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.705+0000 7f8e0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:07:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.706+0000 7f8e0f7fe640 1 -- 172.21.15.64:0/1801745759 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e20035040 con 0x5638711bb4d0 2024-09-21T04:07:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.706+0000 7f8e0f7fe640 1 -- 172.21.15.64:0/1801745759 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f8e20004040 con 0x5638711bb4d0 2024-09-21T04:07:34.714 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.706+0000 7f8e3d47e740 1 -- 172.21.15.64:0/1801745759 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e24013170 con 0x5638711bb4d0 2024-09-21T04:07:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.706+0000 7f8e0f7fe640 1 -- 172.21.15.64:0/1801745759 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8e20004240 con 0x5638711bb4d0 2024-09-21T04:07:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.706+0000 7f8e3d47e740 10 client.16305 fetch_fsmap learned FSMap version 32 2024-09-21T04:07:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.706+0000 7f8e3d47e740 1 -- 172.21.15.64:0/1801745759 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5638712c0620 con 0x5638711bb4d0 2024-09-21T04:07:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.706+0000 7f8e0f7fe640 1 -- 172.21.15.64:0/1801745759 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e24013170 con 0x5638711bb4d0 2024-09-21T04:07:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.706+0000 7f8e0f7fe640 1 -- 172.21.15.64:0/1801745759 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f8e20004860 con 0x5638711bb4d0 2024-09-21T04:07:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.706+0000 7f8e3d47e740 10 client.16305 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T04:07:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.706+0000 7f8e3d47e740 10 client.16305 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:07:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.706+0000 7f8e3d47e740 20 client.16305 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.706+0000 7f8e3d47e740 1 --2- 172.21.15.64:0/1801745759 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56387119ad80 0x56387159f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.706+0000 7f8e3d47e740 4 client.16305 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:07:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.706+0000 7f8e3d47e740 1 -- 172.21.15.64:0/1801745759 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638714f8640 con 0x56387119ad80 2024-09-21T04:07:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.707+0000 7f8e2f7fe640 1 --2- 172.21.15.64:0/1801745759 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56387119ad80 0x56387159f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.707+0000 7f8e0f7fe640 10 client.16305 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:34.715 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.707+0000 7f8e2f7fe640 1 --2- 172.21.15.64:0/1801745759 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56387119ad80 0x56387159f170 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67ac7720b8d1c37b server_cookie=63b4ef6ee8d46ff5 in_seq=0 out_seq=0 2024-09-21T04:07:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.708+0000 7f8e0f7fe640 1 -- 172.21.15.64:0/1801745759 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5638714f8640 con 0x56387119ad80 2024-09-21T04:07:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.708+0000 7f8e0f7fe640 10 client.16305 handle_command_reply: tid=0 2024-09-21T04:07:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.708+0000 7f8e3d47e740 1 client.16305 shutdown 2024-09-21T04:07:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.708+0000 7f8e3d47e740 1 -- 172.21.15.64:0/1801745759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638711bb4d0 msgr2=0x56387158d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.708+0000 7f8e3d47e740 1 --2- 172.21.15.64:0/1801745759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638711bb4d0 0x56387158d5d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8e2002a930 tx=0x7f8e200316b0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.708+0000 7f8e3d47e740 1 -- 172.21.15.64:0/1801745759 shutdown_connections 2024-09-21T04:07:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.708+0000 7f8e3d47e740 1 --2- 172.21.15.64:0/1801745759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638711dc1b0 0x563871599420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.708+0000 7f8e3d47e740 1 --2- 172.21.15.64:0/1801745759 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x56387119ad80 0x56387159f170 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.708+0000 7f8e0f7fe640 0 client.16305 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.708+0000 7f8e3d47e740 1 --2- 172.21.15.64:0/1801745759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638711bb4d0 0x56387158d5d0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.708+0000 7f8e3d47e740 1 --2- 172.21.15.64:0/1801745759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638711b32f0 0x56387158d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.708+0000 7f8e3d47e740 1 -- 172.21.15.64:0/1801745759 >> 172.21.15.64:0/1801745759 conn(0x5638711d0300 msgr2=0x5638715829e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.708+0000 7f8e3d47e740 1 -- 172.21.15.64:0/1801745759 shutdown_connections 2024-09-21T04:07:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.708+0000 7f8e3d47e740 1 -- 172.21.15.64:0/1801745759 wait complete. 2024-09-21T04:07:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.708+0000 7f8e3d47e740 20 client.16305 trim_cache size 0 max 16384 2024-09-21T04:07:34.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.725+0000 7f8e3d47e740 1 Processor -- start 2024-09-21T04:07:34.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.725+0000 7f8e3d47e740 1 -- start start 2024-09-21T04:07:34.731 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.726+0000 7f8e3d47e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387119ad80 0x56387158d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.726+0000 7f8e3d47e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638711b32f0 0x56387158ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.726+0000 7f8e3d47e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638711bb4d0 0x563871599320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.726+0000 7f8e3d47e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8e301b21e0 con 0x5638711b32f0 2024-09-21T04:07:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.726+0000 7f8e3d47e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8e301afa20 con 0x5638711bb4d0 2024-09-21T04:07:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.726+0000 7f8e3d47e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8e301ad870 con 0x56387119ad80 2024-09-21T04:07:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.726+0000 7f8e2ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638711b32f0 0x56387158ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.726+0000 7f8e2ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638711b32f0 0x56387158ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37716/0 (socket says 172.21.15.64:37716) 2024-09-21T04:07:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.726+0000 7f8e2ffff640 1 -- 172.21.15.64:0/4032299653 learned_addr learned my addr 172.21.15.64:0/4032299653 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.726+0000 7f8e2f7fe640 1 --2- 172.21.15.64:0/4032299653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638711bb4d0 0x563871599320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.726+0000 7f8e2f7fe640 1 --2- 172.21.15.64:0/4032299653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387119ad80 0x56387158d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.726+0000 7f8e2ffff640 1 -- 172.21.15.64:0/4032299653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387119ad80 msgr2=0x56387158d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.726+0000 7f8e2ffff640 1 --2- 172.21.15.64:0/4032299653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387119ad80 0x56387158d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.726+0000 7f8e2ffff640 1 -- 172.21.15.64:0/4032299653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638711bb4d0 msgr2=0x563871599320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.726+0000 7f8e2ffff640 1 --2- 172.21.15.64:0/4032299653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638711bb4d0 0x563871599320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.726+0000 7f8e2ffff640 1 -- 172.21.15.64:0/4032299653 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638715847c0 con 0x5638711b32f0 2024-09-21T04:07:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.726+0000 7f8e2f7fe640 1 --2- 172.21.15.64:0/4032299653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638711bb4d0 0x563871599320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:07:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.727+0000 7f8e2ffff640 1 --2- 172.21.15.64:0/4032299653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638711b32f0 0x56387158ecc0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8e20031440 tx=0x7f8e20032fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9c9d7289d8d240 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.727+0000 7f8e15ffb640 1 -- 172.21.15.64:0/4032299653 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e20051070 con 0x5638711b32f0 2024-09-21T04:07:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.727+0000 7f8e15ffb640 1 -- 172.21.15.64:0/4032299653 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e200043e0 con 0x5638711b32f0 2024-09-21T04:07:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.727+0000 7f8e15ffb640 1 -- 172.21.15.64:0/4032299653 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e20035040 con 0x5638711b32f0 2024-09-21T04:07:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.728+0000 7f8e3d47e740 1 -- 172.21.15.64:0/4032299653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638711b32f0 msgr2=0x56387158ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.728+0000 7f8e3d47e740 1 --2- 172.21.15.64:0/4032299653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638711b32f0 0x56387158ecc0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8e20031440 tx=0x7f8e20032fb0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.728+0000 7f8e3d47e740 1 -- 172.21.15.64:0/4032299653 shutdown_connections 2024-09-21T04:07:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.728+0000 7f8e3d47e740 1 --2- 172.21.15.64:0/4032299653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638711bb4d0 0x563871599320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.728+0000 7f8e3d47e740 1 --2- 172.21.15.64:0/4032299653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638711b32f0 0x56387158ecc0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.728+0000 7f8e3d47e740 1 --2- 172.21.15.64:0/4032299653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56387119ad80 0x56387158d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.733 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.728+0000 7f8e3d47e740 1 -- 172.21.15.64:0/4032299653 >> 172.21.15.64:0/4032299653 conn(0x5638711d0300 msgr2=0x56387158e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.728+0000 7f8e3d47e740 1 -- 172.21.15.64:0/4032299653 shutdown_connections 2024-09-21T04:07:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.729+0000 7f8e3d47e740 1 -- 172.21.15.64:0/4032299653 wait complete. 2024-09-21T04:07:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.729+0000 7f8e3d47e740 1 -- start start 2024-09-21T04:07:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.729+0000 7f8e3d47e740 1 Processor -- start 2024-09-21T04:07:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.730+0000 7f8e3d47e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56387119ad80 0x563871590b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.730+0000 7f8e3d47e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638711b32f0 0x563871591120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.730+0000 7f8e3d47e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638711bb4d0 0x563871607b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.730+0000 7f8e3d47e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8e301adee0 con 0x5638711bb4d0 2024-09-21T04:07:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.730+0000 7f8e3d47e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8e301af350 con 0x56387119ad80 2024-09-21T04:07:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.730+0000 7f8e3d47e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8e301ae5b0 con 0x5638711b32f0 2024-09-21T04:07:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.730+0000 7f8e3d47e740 10 client.0 _ll_register_callbacks cb 0x7f8e280093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.730+0000 7f8e2f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56387119ad80 0x563871590b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.730+0000 7f8e2f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56387119ad80 0x563871590b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:36582/0 (socket says 172.21.15.64:36582) 2024-09-21T04:07:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.730+0000 7f8e2f7fe640 1 -- 172.21.15.64:0/2486583878 learned_addr learned my addr 172.21.15.64:0/2486583878 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.730+0000 7f8e2f7fe640 1 --2- 172.21.15.64:0/2486583878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638711bb4d0 0x563871607b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.730+0000 7f8e2ffff640 1 --2- 172.21.15.64:0/2486583878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638711b32f0 0x563871591120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.730+0000 7f8e2f7fe640 1 -- 172.21.15.64:0/2486583878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638711b32f0 msgr2=0x563871591120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.730+0000 7f8e2f7fe640 1 --2- 172.21.15.64:0/2486583878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638711b32f0 0x563871591120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.730+0000 7f8e2f7fe640 1 -- 172.21.15.64:0/2486583878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638711bb4d0 msgr2=0x563871607b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.730+0000 7f8e2f7fe640 1 --2- 172.21.15.64:0/2486583878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638711bb4d0 0x563871607b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.730+0000 7f8e2f7fe640 1 -- 172.21.15.64:0/2486583878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563871608180 con 0x56387119ad80 2024-09-21T04:07:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.731+0000 7f8e2f7fe640 1 --2- 172.21.15.64:0/2486583878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56387119ad80 0x563871590b80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8e24006010 tx=0x7f8e24009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e285ad34e249a92a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.731+0000 7f8e0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:07:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.731+0000 7f8e0f7fe640 1 -- 172.21.15.64:0/2486583878 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2400fcb0 con 0x56387119ad80 2024-09-21T04:07:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.731+0000 7f8e0f7fe640 1 -- 172.21.15.64:0/2486583878 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f8e24005040 con 0x56387119ad80 2024-09-21T04:07:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.731+0000 7f8e3d47e740 1 -- 172.21.15.64:0/2486583878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e200043e0 con 0x56387119ad80 2024-09-21T04:07:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.731+0000 7f8e0f7fe640 1 -- 172.21.15.64:0/2486583878 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8e2403b080 con 0x56387119ad80 2024-09-21T04:07:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.731+0000 7f8e3d47e740 10 client.26012 fetch_fsmap learned FSMap version 32 2024-09-21T04:07:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.731+0000 7f8e3d47e740 1 -- 172.21.15.64:0/2486583878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563871609410 con 0x56387119ad80 2024-09-21T04:07:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.731+0000 7f8e0f7fe640 1 -- 172.21.15.64:0/2486583878 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e200043e0 con 0x56387119ad80 2024-09-21T04:07:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.732+0000 7f8e0f7fe640 1 -- 172.21.15.64:0/2486583878 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f8e240035f0 con 0x56387119ad80 2024-09-21T04:07:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.732+0000 7f8e3d47e740 10 client.26012 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T04:07:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.732+0000 7f8e3d47e740 10 client.26012 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:07:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.732+0000 7f8e3d47e740 20 client.26012 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.732+0000 7f8e3d47e740 1 --2- 172.21.15.64:0/2486583878 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5638711dc1b0 0x56387160b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.732+0000 7f8e3d47e740 4 client.26012 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:07:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.732+0000 7f8e3d47e740 1 -- 172.21.15.64:0/2486583878 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5638714f40f0 con 0x5638711dc1b0 2024-09-21T04:07:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.732+0000 7f8e2ffff640 1 --2- 172.21.15.64:0/2486583878 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5638711dc1b0 0x56387160b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.732+0000 7f8e0f7fe640 10 client.26012 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.733+0000 7f8e2ffff640 1 --2- 172.21.15.64:0/2486583878 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5638711dc1b0 0x56387160b9c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b651e9881e30e184 server_cookie=aa5ca9aba99e8a03 in_seq=0 out_seq=0 2024-09-21T04:07:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.922+0000 7f8e0f7fe640 1 -- 172.21.15.64:0/2486583878 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5638714f40f0 con 0x5638711dc1b0 2024-09-21T04:07:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.922+0000 7f8e0f7fe640 10 client.26012 handle_command_reply: tid=0 2024-09-21T04:07:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.922+0000 7f8e3d47e740 1 client.26012 shutdown 2024-09-21T04:07:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.922+0000 7f8e3d47e740 1 -- 172.21.15.64:0/2486583878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56387119ad80 msgr2=0x563871590b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.922+0000 7f8e3d47e740 1 --2- 172.21.15.64:0/2486583878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56387119ad80 0x563871590b80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8e24006010 tx=0x7f8e24009bb0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.922+0000 7f8e3d47e740 1 -- 172.21.15.64:0/2486583878 shutdown_connections 2024-09-21T04:07:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.922+0000 7f8e3d47e740 1 --2- 172.21.15.64:0/2486583878 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5638711dc1b0 0x56387160b9c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.922+0000 7f8e3d47e740 1 --2- 172.21.15.64:0/2486583878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638711bb4d0 0x563871607b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.922+0000 7f8e3d47e740 1 --2- 172.21.15.64:0/2486583878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5638711b32f0 0x563871591120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.922+0000 7f8e3d47e740 1 --2- 172.21.15.64:0/2486583878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56387119ad80 0x563871590b80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.922+0000 7f8e0f7fe640 0 client.26012 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.922+0000 7f8e3d47e740 1 -- 172.21.15.64:0/2486583878 >> 172.21.15.64:0/2486583878 conn(0x5638711d0300 msgr2=0x56387157f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.922+0000 7f8e3d47e740 1 -- 172.21.15.64:0/2486583878 shutdown_connections 2024-09-21T04:07:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.922+0000 7f8e3d47e740 1 -- 172.21.15.64:0/2486583878 wait complete. 2024-09-21T04:07:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.922+0000 7f8e3d47e740 20 client.26012 trim_cache size 0 max 16384 2024-09-21T04:07:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:07:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:07:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:07:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:07:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.923+0000 7f8e36430640 1 -- 172.21.15.64:0/3236318925 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8e180b6680 msgr2=0x7f8e180b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.923+0000 7f8e36430640 1 --2- 172.21.15.64:0/3236318925 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8e180b6680 0x7f8e180b8b90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8e280e5610 tx=0x7f8e280053b0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.923+0000 7f8e36430640 1 -- 172.21.15.64:0/3236318925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e30108c30 msgr2=0x7f8e3019fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.923+0000 7f8e36430640 1 --2- 172.21.15.64:0/3236318925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e30108c30 0x7f8e3019fdb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8e28009fd0 tx=0x7f8e2800ef10 comp rx=0 tx=0).stop 2024-09-21T04:07:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.923+0000 7f8e36430640 1 -- 172.21.15.64:0/3236318925 shutdown_connections 2024-09-21T04:07:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.923+0000 7f8e36430640 1 --2- 172.21.15.64:0/3236318925 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8e180b6680 0x7f8e180b8b90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.923+0000 7f8e36430640 1 --2- 172.21.15.64:0/3236318925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e30108c30 0x7f8e3019fdb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.923+0000 7f8e36430640 1 --2- 172.21.15.64:0/3236318925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e301082e0 0x7f8e3019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.923+0000 7f8e36430640 1 --2- 172.21.15.64:0/3236318925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e301029b0 0x7f8e3019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.923+0000 7f8e36430640 1 -- 172.21.15.64:0/3236318925 >> 172.21.15.64:0/3236318925 conn(0x7f8e3007a740 msgr2=0x7f8e30101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.923+0000 7f8e36430640 1 -- 172.21.15.64:0/3236318925 shutdown_connections 2024-09-21T04:07:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:34.923+0000 7f8e36430640 1 -- 172.21.15.64:0/3236318925 wait complete. 2024-09-21T04:07:36.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:35 smithi119 ceph-mon[34811]: pgmap v428: 97 pgs: 97 active+clean; 453 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 231 op/s 2024-09-21T04:07:36.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:35 smithi064 ceph-mon[31180]: pgmap v428: 97 pgs: 97 active+clean; 453 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 231 op/s 2024-09-21T04:07:36.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:35 smithi131 ceph-mon[35193]: pgmap v428: 97 pgs: 97 active+clean; 453 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 231 op/s 2024-09-21T04:07:38.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:37 smithi119 ceph-mon[34811]: pgmap v429: 97 pgs: 97 active+clean; 460 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 278 op/s 2024-09-21T04:07:38.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:37 smithi064 ceph-mon[31180]: pgmap v429: 97 pgs: 97 active+clean; 460 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 278 op/s 2024-09-21T04:07:38.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:37 smithi131 ceph-mon[35193]: pgmap v429: 97 pgs: 97 active+clean; 460 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 278 op/s 2024-09-21T04:07:39.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:38 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:07:39.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:38 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:07:39.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:38 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:07:39.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:38 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:07:39.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:38 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:07:39.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:38 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:07:39.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:38 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:07:39.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:38 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:07:39.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:38 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:07:40.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:39 smithi119 ceph-mon[34811]: pgmap v430: 97 pgs: 97 active+clean; 460 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 203 op/s 2024-09-21T04:07:40.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:39 smithi064 ceph-mon[31180]: pgmap v430: 97 pgs: 97 active+clean; 460 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 203 op/s 2024-09-21T04:07:40.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:39 smithi131 ceph-mon[35193]: pgmap v430: 97 pgs: 97 active+clean; 460 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 203 op/s 2024-09-21T04:07:41.173 INFO:tasks.workunit.client.0.smithi064.stdout: 77 629697 352 437215 312 336390 927 2024-09-21T04:07:42.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:41 smithi119 ceph-mon[34811]: pgmap v431: 97 pgs: 97 active+clean; 460 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 141 op/s 2024-09-21T04:07:42.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:41 smithi064 ceph-mon[31180]: pgmap v431: 97 pgs: 97 active+clean; 460 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 141 op/s 2024-09-21T04:07:42.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:41 smithi131 ceph-mon[35193]: pgmap v431: 97 pgs: 97 active+clean; 460 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 141 op/s 2024-09-21T04:07:43.978 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T04:07:44.003 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:43 smithi064 ceph-mon[31180]: pgmap v432: 97 pgs: 97 active+clean; 476 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 189 op/s 2024-09-21T04:07:44.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:43 smithi119 ceph-mon[34811]: pgmap v432: 97 pgs: 97 active+clean; 476 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 189 op/s 2024-09-21T04:07:44.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:43 smithi131 ceph-mon[35193]: pgmap v432: 97 pgs: 97 active+clean; 476 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 189 op/s 2024-09-21T04:07:44.258 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:07:45.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.200+0000 7f179ffff640 1 -- 172.21.15.64:0/3463900713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17a010dbf0 msgr2=0x7f17a010e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:45.206 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.200+0000 7f179ffff640 1 --2- 172.21.15.64:0/3463900713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17a010dbf0 0x7f17a010e000 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1790009900 tx=0x7f179002a420 comp rx=0 tx=0).stop 2024-09-21T04:07:45.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.207+0000 7f179ffff640 1 -- 172.21.15.64:0/3463900713 shutdown_connections 2024-09-21T04:07:45.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.207+0000 7f179ffff640 1 --2- 172.21.15.64:0/3463900713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17a011cba0 0x7f17a011efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.207+0000 7f179ffff640 1 --2- 172.21.15.64:0/3463900713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17a010dbf0 0x7f17a010e000 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.207+0000 7f179ffff640 1 --2- 172.21.15.64:0/3463900713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17a0111cd0 0x7f17a010d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.207+0000 7f179ffff640 1 -- 172.21.15.64:0/3463900713 >> 172.21.15.64:0/3463900713 conn(0x7f17a006e9f0 msgr2=0x7f17a006ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:45.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.213+0000 7f179ffff640 1 -- 172.21.15.64:0/3463900713 shutdown_connections 2024-09-21T04:07:45.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.214+0000 7f179ffff640 1 -- 172.21.15.64:0/3463900713 wait complete. 2024-09-21T04:07:45.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.215+0000 7f179ffff640 1 Processor -- start 2024-09-21T04:07:45.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.218+0000 7f179ffff640 1 -- start start 2024-09-21T04:07:45.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.218+0000 7f179ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17a010dbf0 0x7f17a011c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:45.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.218+0000 7f179ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17a0111cd0 0x7f17a0116840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:45.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.219+0000 7f179ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17a011cba0 0x7f17a0116f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:45.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.219+0000 7f179ffff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f17a0112c80 con 0x7f17a010dbf0 2024-09-21T04:07:45.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.219+0000 7f179ffff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f17a0112b00 con 0x7f17a011cba0 2024-09-21T04:07:45.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.219+0000 7f179ffff640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f17a0112e00 con 0x7f17a0111cd0 2024-09-21T04:07:45.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.219+0000 7f179e7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17a0111cd0 0x7f17a0116840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:45.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.219+0000 7f179e7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17a0111cd0 0x7f17a0116840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:42512/0 (socket says 172.21.15.64:42512) 2024-09-21T04:07:45.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.219+0000 7f179e7fc640 1 -- 172.21.15.64:0/4265572062 learned_addr learned my addr 172.21.15.64:0/4265572062 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:45.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.220+0000 7f179e7fc640 1 -- 172.21.15.64:0/4265572062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17a011cba0 msgr2=0x7f17a0116f10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T04:07:45.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.220+0000 7f179e7fc640 1 --2- 172.21.15.64:0/4265572062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17a011cba0 0x7f17a0116f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.220+0000 7f179e7fc640 1 -- 172.21.15.64:0/4265572062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17a010dbf0 msgr2=0x7f17a011c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:45.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.220+0000 7f179e7fc640 1 --2- 172.21.15.64:0/4265572062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17a010dbf0 0x7f17a011c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.220+0000 7f179e7fc640 1 -- 172.21.15.64:0/4265572062 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a01bdc40 con 0x7f17a0111cd0 2024-09-21T04:07:45.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.220+0000 7f179e7fc640 1 --2- 172.21.15.64:0/4265572062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17a0111cd0 0x7f17a0116840 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1790031440 tx=0x7f1790003d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:45.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.225+0000 7f17a4a44640 1 -- 172.21.15.64:0/4265572062 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179003f070 con 0x7f17a0111cd0 2024-09-21T04:07:45.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.225+0000 7f17a4a44640 1 -- 172.21.15.64:0/4265572062 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1790031d60 con 0x7f17a0111cd0 2024-09-21T04:07:45.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.225+0000 7f17a4a44640 1 -- 172.21.15.64:0/4265572062 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790035030 con 0x7f17a0111cd0 2024-09-21T04:07:45.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.228+0000 7f179ffff640 1 -- 172.21.15.64:0/4265572062 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a01bde10 con 0x7f17a0111cd0 2024-09-21T04:07:45.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.228+0000 7f179ffff640 1 -- 172.21.15.64:0/4265572062 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a01be1b0 con 0x7f17a0111cd0 2024-09-21T04:07:45.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.235+0000 7f17abfba740 1 Processor -- start 2024-09-21T04:07:45.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.235+0000 7f17abfba740 1 -- start start 2024-09-21T04:07:45.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.236+0000 7f17abfba740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abd947c070 0x55abd98264f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:45.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.236+0000 7f17abfba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abd94531b0 0x55abd9823b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:45.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.236+0000 7f17abfba740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abd945b390 0x55abd98240a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:45.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.236+0000 7f17abfba740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55abd9799e50 con 0x55abd945b390 2024-09-21T04:07:45.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.236+0000 7f17abfba740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55abd9789280 con 0x55abd94531b0 2024-09-21T04:07:45.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.236+0000 7f17abfba740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55abd978d090 con 0x55abd947c070 2024-09-21T04:07:45.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.239+0000 7f179effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abd94531b0 0x55abd9823b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:45.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.239+0000 7f179f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abd947c070 0x55abd98264f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:45.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.241+0000 7f17a4a44640 1 -- 172.21.15.64:0/4265572062 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f1790042070 con 0x7f17a0111cd0 2024-09-21T04:07:45.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.241+0000 7f179e7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abd945b390 0x55abd98240a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:45.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.241+0000 7f179f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abd947c070 0x55abd98264f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:42514/0 (socket says 172.21.15.64:42514) 2024-09-21T04:07:45.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.241+0000 7f179f7fe640 1 -- 172.21.15.64:0/50270493 learned_addr learned my addr 172.21.15.64:0/50270493 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:45.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.244+0000 7f17a4a44640 1 --2- 172.21.15.64:0/4265572062 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f17880b6880 0x7f17880b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:45.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.245+0000 7f17a4a44640 1 -- 172.21.15.64:0/4265572062 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f1790108380 con 0x7f17a0111cd0 2024-09-21T04:07:45.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.245+0000 7f179effd640 1 -- 172.21.15.64:0/50270493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abd947c070 msgr2=0x55abd98264f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:45.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.245+0000 7f179effd640 1 --2- 172.21.15.64:0/50270493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abd947c070 0x55abd98264f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.245+0000 7f179effd640 1 -- 172.21.15.64:0/50270493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abd945b390 msgr2=0x55abd98240a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:45.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.245+0000 7f179effd640 1 --2- 172.21.15.64:0/50270493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abd945b390 0x55abd98240a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.245+0000 7f179effd640 1 -- 172.21.15.64:0/50270493 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd94e7650 con 0x55abd94531b0 2024-09-21T04:07:45.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.245+0000 7f179f7fe640 1 --2- 172.21.15.64:0/4265572062 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f17880b6880 0x7f17880b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:45.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.245+0000 7f179e7fc640 1 --2- 172.21.15.64:0/50270493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abd945b390 0x55abd98240a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:07:45.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.246+0000 7f179effd640 1 --2- 172.21.15.64:0/50270493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abd94531b0 0x55abd9823b00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f17a0110950 tx=0x7f1794009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7dedf73d2c80f64 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:45.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.246+0000 7f179f7fe640 1 --2- 172.21.15.64:0/4265572062 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f17880b6880 0x7f17880b8d90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1798007430 tx=0x7f179800b040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:45.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.247+0000 7f179ffff640 1 -- 172.21.15.64:0/50270493 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1794015020 con 0x55abd94531b0 2024-09-21T04:07:45.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.247+0000 7f179ffff640 1 -- 172.21.15.64:0/50270493 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f179400bd70 con 0x55abd94531b0 2024-09-21T04:07:45.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.247+0000 7f179ffff640 1 -- 172.21.15.64:0/50270493 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1794010380 con 0x55abd94531b0 2024-09-21T04:07:45.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.249+0000 7f17abfba740 1 -- 172.21.15.64:0/50270493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abd94531b0 msgr2=0x55abd9823b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:45.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.249+0000 7f17abfba740 1 --2- 172.21.15.64:0/50270493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abd94531b0 0x55abd9823b00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f17a0110950 tx=0x7f1794009b80 comp rx=0 tx=0).stop 2024-09-21T04:07:45.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.250+0000 7f17abfba740 1 -- 172.21.15.64:0/50270493 shutdown_connections 2024-09-21T04:07:45.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.250+0000 7f17abfba740 1 --2- 172.21.15.64:0/50270493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abd945b390 0x55abd98240a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.250+0000 7f17abfba740 1 --2- 172.21.15.64:0/50270493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abd94531b0 0x55abd9823b00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.250+0000 7f17abfba740 1 --2- 172.21.15.64:0/50270493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abd947c070 0x55abd98264f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.250+0000 7f17abfba740 1 -- 172.21.15.64:0/50270493 >> 172.21.15.64:0/50270493 conn(0x55abd9470460 msgr2=0x55abd98219f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:45.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.251+0000 7f17abfba740 1 -- 172.21.15.64:0/50270493 shutdown_connections 2024-09-21T04:07:45.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.255+0000 7f17abfba740 1 -- 172.21.15.64:0/50270493 wait complete. 2024-09-21T04:07:45.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.255+0000 7f17abfba740 1 -- start start 2024-09-21T04:07:45.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.256+0000 7f17abfba740 1 Processor -- start 2024-09-21T04:07:45.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.256+0000 7f17abfba740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abd94531b0 0x55abd982f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:45.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.256+0000 7f17abfba740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abd945b390 0x55abd982f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:45.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.256+0000 7f17abfba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abd947c070 0x55abd982fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:45.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.256+0000 7f17abfba740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55abd95ea700 con 0x55abd94531b0 2024-09-21T04:07:45.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.256+0000 7f17abfba740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55abd9724110 con 0x55abd947c070 2024-09-21T04:07:45.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.256+0000 7f17abfba740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55abd9831fa0 con 0x55abd945b390 2024-09-21T04:07:45.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.256+0000 7f17abfba740 10 client.0 _ll_register_callbacks cb 0x55abd973d990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:45.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.258+0000 7f179effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abd94531b0 0x55abd982f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:45.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.258+0000 7f179effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abd94531b0 0x55abd982f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50928/0 (socket says 172.21.15.64:50928) 2024-09-21T04:07:45.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.258+0000 7f179effd640 1 -- 172.21.15.64:0/509595676 learned_addr learned my addr 172.21.15.64:0/509595676 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:45.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.259+0000 7f179effd640 1 -- 172.21.15.64:0/509595676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abd945b390 msgr2=0x55abd982f660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T04:07:45.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.259+0000 7f179effd640 1 --2- 172.21.15.64:0/509595676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abd945b390 0x55abd982f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.259+0000 7f179effd640 1 -- 172.21.15.64:0/509595676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abd947c070 msgr2=0x55abd982fbc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T04:07:45.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.259+0000 7f179effd640 1 --2- 172.21.15.64:0/509595676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abd947c070 0x55abd982fbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.259+0000 7f179effd640 1 -- 172.21.15.64:0/509595676 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abd9751c40 con 0x55abd94531b0 2024-09-21T04:07:45.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.259+0000 7f179effd640 1 --2- 172.21.15.64:0/509595676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abd94531b0 0x55abd982f120 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1794004800 tx=0x7f1794011650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=701a818df7f696b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:45.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.260+0000 7f177effd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:07:45.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.260+0000 7f177effd640 1 -- 172.21.15.64:0/509595676 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1794015030 con 0x55abd94531b0 2024-09-21T04:07:45.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.260+0000 7f177effd640 1 -- 172.21.15.64:0/509595676 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f179400bad0 con 0x55abd94531b0 2024-09-21T04:07:45.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.262+0000 7f17abfba740 1 -- 172.21.15.64:0/509595676 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f179400bd70 con 0x55abd94531b0 2024-09-21T04:07:45.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.263+0000 7f177effd640 1 -- 172.21.15.64:0/509595676 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1794029050 con 0x55abd94531b0 2024-09-21T04:07:45.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.263+0000 7f17abfba740 10 client.16317 fetch_fsmap learned FSMap version 32 2024-09-21T04:07:45.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.263+0000 7f17abfba740 1 -- 172.21.15.64:0/509595676 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55abd983dc90 con 0x55abd94531b0 2024-09-21T04:07:45.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.263+0000 7f177effd640 1 -- 172.21.15.64:0/509595676 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f179400bd70 con 0x55abd94531b0 2024-09-21T04:07:45.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.263+0000 7f177effd640 1 -- 172.21.15.64:0/509595676 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f1794003b60 con 0x55abd94531b0 2024-09-21T04:07:45.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.263+0000 7f17abfba740 10 client.16317 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T04:07:45.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.263+0000 7f17abfba740 10 client.16317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:07:45.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.263+0000 7f17abfba740 20 client.16317 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:45.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.264+0000 7f17abfba740 1 --2- 172.21.15.64:0/509595676 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55abd943a9f0 0x55abd9841580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:45.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.264+0000 7f17abfba740 4 client.16317 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:07:45.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.264+0000 7f17abfba740 1 -- 172.21.15.64:0/509595676 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abd979bda0 con 0x55abd943a9f0 2024-09-21T04:07:45.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.268+0000 7f179e7fc640 1 --2- 172.21.15.64:0/509595676 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55abd943a9f0 0x55abd9841580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:45.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.277+0000 7f179e7fc640 1 --2- 172.21.15.64:0/509595676 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55abd943a9f0 0x55abd9841580 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fce3c63b284b953d server_cookie=7bc358f8bd03391 in_seq=0 out_seq=0 2024-09-21T04:07:45.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.278+0000 7f177effd640 10 client.16317 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:45.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.279+0000 7f177effd640 1 -- 172.21.15.64:0/509595676 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55abd979bda0 con 0x55abd943a9f0 2024-09-21T04:07:45.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.279+0000 7f177effd640 10 client.16317 handle_command_reply: tid=0 2024-09-21T04:07:45.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.279+0000 7f17abfba740 1 client.16317 shutdown 2024-09-21T04:07:45.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.285+0000 7f17abfba740 1 -- 172.21.15.64:0/509595676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abd94531b0 msgr2=0x55abd982f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:45.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.285+0000 7f17abfba740 1 --2- 172.21.15.64:0/509595676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abd94531b0 0x55abd982f120 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1794004800 tx=0x7f1794011650 comp rx=0 tx=0).stop 2024-09-21T04:07:45.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.285+0000 7f17abfba740 1 -- 172.21.15.64:0/509595676 shutdown_connections 2024-09-21T04:07:45.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.285+0000 7f17abfba740 1 --2- 172.21.15.64:0/509595676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abd947c070 0x55abd982fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.285+0000 7f17abfba740 1 --2- 172.21.15.64:0/509595676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abd945b390 0x55abd982f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.285+0000 7f17abfba740 1 --2- 172.21.15.64:0/509595676 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55abd943a9f0 0x55abd9841580 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.285+0000 7f17abfba740 1 --2- 172.21.15.64:0/509595676 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abd94531b0 0x55abd982f120 secure :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1794004800 tx=0x7f1794011650 comp rx=0 tx=0).stop 2024-09-21T04:07:45.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.285+0000 7f17abfba740 1 -- 172.21.15.64:0/509595676 >> 172.21.15.64:0/509595676 conn(0x55abd9470460 msgr2=0x55abd98254f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:45.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.286+0000 7f177effd640 0 client.16317 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:45.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.289+0000 7f17abfba740 1 -- 172.21.15.64:0/509595676 shutdown_connections 2024-09-21T04:07:45.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.294+0000 7f17abfba740 1 -- 172.21.15.64:0/509595676 wait complete. 2024-09-21T04:07:45.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.299+0000 7f17abfba740 20 client.16317 trim_cache size 0 max 16384 2024-09-21T04:07:45.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.343+0000 7f17abfba740 1 Processor -- start 2024-09-21T04:07:45.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.345+0000 7f17abfba740 1 -- start start 2024-09-21T04:07:45.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.345+0000 7f17abfba740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abd945b390 0x55abd982e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:45.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.345+0000 7f17abfba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abd947c070 0x55abd9823600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:45.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.346+0000 7f179effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abd945b390 0x55abd982e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:45.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.346+0000 7f179effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abd945b390 0x55abd982e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:42540/0 (socket says 172.21.15.64:42540) 2024-09-21T04:07:45.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.346+0000 7f179effd640 1 -- 172.21.15.64:0/3000297810 learned_addr learned my addr 172.21.15.64:0/3000297810 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:45.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.346+0000 7f17abfba740 1 --2- 172.21.15.64:0/3000297810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abd940fda0 0x55abd9821a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:45.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.346+0000 7f179f7fe640 1 --2- 172.21.15.64:0/3000297810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abd940fda0 0x55abd9821a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:45.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.346+0000 7f17abfba740 1 -- 172.21.15.64:0/3000297810 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f17a01c9850 con 0x55abd940fda0 2024-09-21T04:07:45.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.346+0000 7f17abfba740 1 -- 172.21.15.64:0/3000297810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f17a01c7090 con 0x55abd947c070 2024-09-21T04:07:45.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.347+0000 7f17abfba740 1 -- 172.21.15.64:0/3000297810 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f17a01c4ee0 con 0x55abd945b390 2024-09-21T04:07:45.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.348+0000 7f179e7fc640 1 --2- 172.21.15.64:0/3000297810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abd947c070 0x55abd9823600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:45.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.348+0000 7f179f7fe640 1 -- 172.21.15.64:0/3000297810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abd945b390 msgr2=0x55abd982e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:45.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.348+0000 7f179f7fe640 1 --2- 172.21.15.64:0/3000297810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abd945b390 0x55abd982e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.348+0000 7f179f7fe640 1 -- 172.21.15.64:0/3000297810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abd947c070 msgr2=0x55abd9823600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:45.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.348+0000 7f179f7fe640 1 --2- 172.21.15.64:0/3000297810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abd947c070 0x55abd9823600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.348+0000 7f179f7fe640 1 -- 172.21.15.64:0/3000297810 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd983b200 con 0x55abd940fda0 2024-09-21T04:07:45.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.355+0000 7f179f7fe640 1 --2- 172.21.15.64:0/3000297810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abd940fda0 0x55abd9821a20 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x55abd9830100 tx=0x7f1798009a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d347b82ac61d79b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:45.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.356+0000 7f17857fa640 1 -- 172.21.15.64:0/3000297810 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1798019e00 con 0x55abd940fda0 2024-09-21T04:07:45.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.356+0000 7f17857fa640 1 -- 172.21.15.64:0/3000297810 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1798015900 con 0x55abd940fda0 2024-09-21T04:07:45.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.356+0000 7f17857fa640 1 -- 172.21.15.64:0/3000297810 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1798017640 con 0x55abd940fda0 2024-09-21T04:07:45.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.362+0000 7f17abfba740 1 -- 172.21.15.64:0/3000297810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abd940fda0 msgr2=0x55abd9821a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:45.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.363+0000 7f17abfba740 1 --2- 172.21.15.64:0/3000297810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abd940fda0 0x55abd9821a20 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x55abd9830100 tx=0x7f1798009a40 comp rx=0 tx=0).stop 2024-09-21T04:07:45.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.370+0000 7f17abfba740 1 -- 172.21.15.64:0/3000297810 shutdown_connections 2024-09-21T04:07:45.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.370+0000 7f17abfba740 1 --2- 172.21.15.64:0/3000297810 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abd940fda0 0x55abd9821a20 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.370+0000 7f17abfba740 1 --2- 172.21.15.64:0/3000297810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abd947c070 0x55abd9823600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.370+0000 7f17abfba740 1 --2- 172.21.15.64:0/3000297810 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abd945b390 0x55abd982e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.370+0000 7f17abfba740 1 -- 172.21.15.64:0/3000297810 >> 172.21.15.64:0/3000297810 conn(0x55abd9470460 msgr2=0x55abd9821310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:45.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.372+0000 7f17abfba740 1 -- 172.21.15.64:0/3000297810 shutdown_connections 2024-09-21T04:07:45.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.372+0000 7f17abfba740 1 -- 172.21.15.64:0/3000297810 wait complete. 2024-09-21T04:07:45.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.373+0000 7f17abfba740 1 -- start start 2024-09-21T04:07:45.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.373+0000 7f17abfba740 1 Processor -- start 2024-09-21T04:07:45.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.374+0000 7f17abfba740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abd940fda0 0x55abd9832ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:45.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.374+0000 7f17abfba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abd945b390 0x55abd9833280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:45.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.374+0000 7f17abfba740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abd947c070 0x55abd9837170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:45.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.374+0000 7f179e7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abd945b390 0x55abd9833280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:45.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.374+0000 7f179e7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abd945b390 0x55abd9833280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:52042/0 (socket says 172.21.15.64:52042) 2024-09-21T04:07:45.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.375+0000 7f179effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abd940fda0 0x55abd9832ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:45.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.375+0000 7f179effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abd940fda0 0x55abd9832ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50938/0 (socket says 172.21.15.64:50938) 2024-09-21T04:07:45.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.375+0000 7f179e7fc640 1 -- 172.21.15.64:0/823949322 learned_addr learned my addr 172.21.15.64:0/823949322 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:45.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.375+0000 7f17abfba740 1 -- 172.21.15.64:0/823949322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f17a01c5550 con 0x55abd940fda0 2024-09-21T04:07:45.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.375+0000 7f179f7fe640 1 --2- 172.21.15.64:0/823949322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abd947c070 0x55abd9837170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:45.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.376+0000 7f17abfba740 1 -- 172.21.15.64:0/823949322 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f17a01c69c0 con 0x55abd945b390 2024-09-21T04:07:45.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.376+0000 7f17abfba740 1 -- 172.21.15.64:0/823949322 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f17a01c5c20 con 0x55abd947c070 2024-09-21T04:07:45.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.376+0000 7f179e7fc640 1 -- 172.21.15.64:0/823949322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abd947c070 msgr2=0x55abd9837170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:45.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.376+0000 7f179e7fc640 1 --2- 172.21.15.64:0/823949322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abd947c070 0x55abd9837170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.376+0000 7f179e7fc640 1 -- 172.21.15.64:0/823949322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abd940fda0 msgr2=0x55abd9832ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:45.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.376+0000 7f179e7fc640 1 --2- 172.21.15.64:0/823949322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abd940fda0 0x55abd9832ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.376+0000 7f179e7fc640 1 -- 172.21.15.64:0/823949322 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abd98377e0 con 0x55abd945b390 2024-09-21T04:07:45.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.377+0000 7f179e7fc640 1 --2- 172.21.15.64:0/823949322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abd945b390 0x55abd9833280 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f17900ce0d0 tx=0x7f1790031f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5c989eb4dd87a4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:45.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.377+0000 7f17abfba740 10 client.0 _ll_register_callbacks cb 0x7f179010bd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:45.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.377+0000 7f17abfba740 1 -- 172.21.15.64:0/823949322 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1798015900 con 0x55abd945b390 2024-09-21T04:07:45.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.379+0000 7f177effd640 10 client.26021 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:07:45.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.379+0000 7f177effd640 1 -- 172.21.15.64:0/823949322 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17900cdf10 con 0x55abd945b390 2024-09-21T04:07:45.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.379+0000 7f177effd640 1 -- 172.21.15.64:0/823949322 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f17900318b0 con 0x55abd945b390 2024-09-21T04:07:45.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.379+0000 7f177effd640 1 -- 172.21.15.64:0/823949322 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f17900cd8b0 con 0x55abd945b390 2024-09-21T04:07:45.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.379+0000 7f17abfba740 10 client.26021 fetch_fsmap learned FSMap version 32 2024-09-21T04:07:45.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.379+0000 7f17abfba740 1 -- 172.21.15.64:0/823949322 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55abd98ab8e0 con 0x55abd945b390 2024-09-21T04:07:45.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.383+0000 7f177effd640 1 -- 172.21.15.64:0/823949322 <== mon.2 v2:172.21.15.119:3300/0 4 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f17900b9240 con 0x55abd945b390 2024-09-21T04:07:45.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.383+0000 7f17abfba740 10 client.26021 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T04:07:45.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.383+0000 7f17abfba740 10 client.26021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:07:45.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.383+0000 7f17abfba740 20 client.26021 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:45.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.383+0000 7f17abfba740 1 --2- 172.21.15.64:0/823949322 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55abd98abeb0 0x55abd98ae330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:45.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.383+0000 7f17abfba740 4 client.26021 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T04:07:45.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.383+0000 7f17abfba740 1 -- 172.21.15.64:0/823949322 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f1798015900 con 0x55abd98abeb0 2024-09-21T04:07:45.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.384+0000 7f179effd640 1 --2- 172.21.15.64:0/823949322 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55abd98abeb0 0x55abd98ae330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:45.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.393+0000 7f179effd640 1 --2- 172.21.15.64:0/823949322 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55abd98abeb0 0x55abd98ae330 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=508f5a58a4acc050 server_cookie=bc8c20b2353f99b1 in_seq=0 out_seq=0 2024-09-21T04:07:45.396 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.393+0000 7f177effd640 10 client.26021 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:45.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.397+0000 7f177effd640 1 -- 172.21.15.64:0/823949322 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1794 (crc 0 0 0) 0x7f1798015900 con 0x55abd98abeb0 2024-09-21T04:07:45.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.397+0000 7f177effd640 10 client.26021 handle_command_reply: tid=0 2024-09-21T04:07:45.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.397+0000 7f17abfba740 1 client.26021 shutdown 2024-09-21T04:07:45.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.397+0000 7f17abfba740 1 -- 172.21.15.64:0/823949322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abd945b390 msgr2=0x55abd9833280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:45.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.397+0000 7f17abfba740 1 --2- 172.21.15.64:0/823949322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abd945b390 0x55abd9833280 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f17900ce0d0 tx=0x7f1790031f10 comp rx=0 tx=0).stop 2024-09-21T04:07:45.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.398+0000 7f17abfba740 1 -- 172.21.15.64:0/823949322 shutdown_connections 2024-09-21T04:07:45.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.398+0000 7f17abfba740 1 --2- 172.21.15.64:0/823949322 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55abd947c070 0x55abd9837170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.398+0000 7f17abfba740 1 --2- 172.21.15.64:0/823949322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55abd945b390 0x55abd9833280 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.398+0000 7f17abfba740 1 --2- 172.21.15.64:0/823949322 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55abd98abeb0 0x55abd98ae330 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.398+0000 7f17abfba740 1 --2- 172.21.15.64:0/823949322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55abd940fda0 0x55abd9832ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.398+0000 7f17abfba740 1 -- 172.21.15.64:0/823949322 >> 172.21.15.64:0/823949322 conn(0x55abd9470460 msgr2=0x55abd983a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:45.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.398+0000 7f177effd640 0 client.26021 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:45.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.398+0000 7f17abfba740 1 -- 172.21.15.64:0/823949322 shutdown_connections 2024-09-21T04:07:45.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.398+0000 7f17abfba740 1 -- 172.21.15.64:0/823949322 wait complete. 2024-09-21T04:07:45.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.403+0000 7f17abfba740 20 client.26021 trim_cache size 0 max 16384 2024-09-21T04:07:45.407 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:07:45.407 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-21T04:07:45.407 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {}, 2024-09-21T04:07:45.407 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_stats": { 2024-09-21T04:07:45.407 INFO:teuthology.orchestra.run.smithi064.stdout: "bed40d88-2102-43a6-ac2d-b89d5a2264d0": { 2024-09-21T04:07:45.407 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:11.363340+0000", 2024-09-21T04:07:45.407 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:07:45.407 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:07:45.407 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:07:45.407 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:07:45.407 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:07:45.407 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:07:45.408 INFO:teuthology.orchestra.run.smithi064.stdout: "e9242f88-6599-40eb-adf1-bdb3e946a1e0": { 2024-09-21T04:07:45.408 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:05:39.744646+0000", 2024-09-21T04:07:45.408 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:07:45.408 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:07:45.408 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:07:45.408 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:07:45.408 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:07:45.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:07:45.408 INFO:teuthology.orchestra.run.smithi064.stdout: "6bde0fe5-f775-44b1-ad0c-f9e596574dc0": { 2024-09-21T04:07:45.408 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:06:23.835536+0000", 2024-09-21T04:07:45.408 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:07:45.408 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:07:45.408 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:07:45.409 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:07:45.409 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:07:45.409 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:07:45.409 INFO:teuthology.orchestra.run.smithi064.stdout: "b52b950c-0f97-49ec-a37c-1a8e04348ce8": { 2024-09-21T04:07:45.409 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:54.003800+0000", 2024-09-21T04:07:45.409 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:07:45.409 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:07:45.409 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:07:45.409 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:07:45.409 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:07:45.409 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:07:45.409 INFO:teuthology.orchestra.run.smithi064.stdout: "0ab717a7-36c6-4dca-94e7-af0bdf2aa112": { 2024-09-21T04:07:45.409 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:42.691086+0000", 2024-09-21T04:07:45.409 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:07:45.409 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:07:45.410 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:07:45.410 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:07:45.410 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:07:45.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:07:45.410 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:07:45.410 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:03:59.570174+0000", 2024-09-21T04:07:45.410 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:07:45.410 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:07:45.410 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:07:45.410 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:07:45.410 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:07:45.410 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:07:45.410 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:07:45.410 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:07:45.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.405+0000 7f17867fc640 1 -- 172.21.15.64:0/4265572062 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f17880b6880 msgr2=0x7f17880b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:45.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.405+0000 7f17867fc640 1 --2- 172.21.15.64:0/4265572062 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f17880b6880 0x7f17880b8d90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1798007430 tx=0x7f179800b040 comp rx=0 tx=0).stop 2024-09-21T04:07:45.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.405+0000 7f17867fc640 1 -- 172.21.15.64:0/4265572062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17a0111cd0 msgr2=0x7f17a0116840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:45.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.405+0000 7f17867fc640 1 --2- 172.21.15.64:0/4265572062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17a0111cd0 0x7f17a0116840 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1790031440 tx=0x7f1790003d60 comp rx=0 tx=0).stop 2024-09-21T04:07:45.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.405+0000 7f17867fc640 1 -- 172.21.15.64:0/4265572062 shutdown_connections 2024-09-21T04:07:45.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.405+0000 7f17867fc640 1 --2- 172.21.15.64:0/4265572062 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f17880b6880 0x7f17880b8d90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.405+0000 7f17867fc640 1 --2- 172.21.15.64:0/4265572062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17a011cba0 0x7f17a0116f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.405+0000 7f17867fc640 1 --2- 172.21.15.64:0/4265572062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17a0111cd0 0x7f17a0116840 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.405+0000 7f17867fc640 1 --2- 172.21.15.64:0/4265572062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f17a010dbf0 0x7f17a011c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:45.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.405+0000 7f17867fc640 1 -- 172.21.15.64:0/4265572062 >> 172.21.15.64:0/4265572062 conn(0x7f17a006e9f0 msgr2=0x7f17a0110980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:45.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.406+0000 7f17867fc640 1 -- 172.21.15.64:0/4265572062 shutdown_connections 2024-09-21T04:07:45.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:45.406+0000 7f17867fc640 1 -- 172.21.15.64:0/4265572062 wait complete. 2024-09-21T04:07:46.004 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:45 smithi064 ceph-mon[31180]: pgmap v433: 97 pgs: 97 active+clean; 476 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 123 op/s 2024-09-21T04:07:46.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:45 smithi119 ceph-mon[34811]: pgmap v433: 97 pgs: 97 active+clean; 476 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 123 op/s 2024-09-21T04:07:46.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:45 smithi131 ceph-mon[35193]: pgmap v433: 97 pgs: 97 active+clean; 476 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 123 op/s 2024-09-21T04:07:46.314 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T04:07:46.315 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:07:46.685 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:07:47.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.473+0000 7f295f59e640 1 -- 172.21.15.64:0/1651148369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f296011cba0 msgr2=0x7f296011efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:47.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.473+0000 7f295f59e640 1 --2- 172.21.15.64:0/1651148369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f296011cba0 0x7f296011efc0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2954009900 tx=0x7f295402a440 comp rx=0 tx=0).stop 2024-09-21T04:07:47.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.476+0000 7f295f59e640 1 -- 172.21.15.64:0/1651148369 shutdown_connections 2024-09-21T04:07:47.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.476+0000 7f295f59e640 1 --2- 172.21.15.64:0/1651148369 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f296011cba0 0x7f296011efc0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:47.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.476+0000 7f295f59e640 1 --2- 172.21.15.64:0/1651148369 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f296010dbf0 0x7f296010e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:47.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.476+0000 7f295f59e640 1 --2- 172.21.15.64:0/1651148369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2960111cd0 0x7f296010d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:47.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.476+0000 7f295f59e640 1 -- 172.21.15.64:0/1651148369 >> 172.21.15.64:0/1651148369 conn(0x7f296006e9f0 msgr2=0x7f296006ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:47.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.477+0000 7f295f59e640 1 -- 172.21.15.64:0/1651148369 shutdown_connections 2024-09-21T04:07:47.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.477+0000 7f295f59e640 1 -- 172.21.15.64:0/1651148369 wait complete. 2024-09-21T04:07:47.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.479+0000 7f295f59e640 1 Processor -- start 2024-09-21T04:07:47.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.479+0000 7f295f59e640 1 -- start start 2024-09-21T04:07:47.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.479+0000 7f295f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f296010dbf0 0x7f296011bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:47.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.479+0000 7f295f59e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2960111cd0 0x7f296011c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:47.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.479+0000 7f295f59e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f296011cba0 0x7f2960116350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:47.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.480+0000 7f295f59e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2960112850 con 0x7f296010dbf0 2024-09-21T04:07:47.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.480+0000 7f295f59e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f29601126d0 con 0x7f296011cba0 2024-09-21T04:07:47.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.480+0000 7f295f59e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f29601129d0 con 0x7f2960111cd0 2024-09-21T04:07:47.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.480+0000 7f295ed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f296011cba0 0x7f2960116350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:47.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.480+0000 7f295ed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f296011cba0 0x7f2960116350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:52058/0 (socket says 172.21.15.64:52058) 2024-09-21T04:07:47.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.480+0000 7f295ed9d640 1 -- 172.21.15.64:0/1801390827 learned_addr learned my addr 172.21.15.64:0/1801390827 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:47.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.481+0000 7f295dd9b640 1 --2- 172.21.15.64:0/1801390827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2960111cd0 0x7f296011c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:47.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.482+0000 7f295ed9d640 1 -- 172.21.15.64:0/1801390827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2960111cd0 msgr2=0x7f296011c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:47.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.482+0000 7f295ed9d640 1 --2- 172.21.15.64:0/1801390827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2960111cd0 0x7f296011c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:47.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.482+0000 7f295ed9d640 1 -- 172.21.15.64:0/1801390827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f296010dbf0 msgr2=0x7f296011bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:47.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.481+0000 7f295e59c640 1 --2- 172.21.15.64:0/1801390827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f296010dbf0 0x7f296011bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:47.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.482+0000 7f295ed9d640 1 --2- 172.21.15.64:0/1801390827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f296010dbf0 0x7f296011bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:47.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.482+0000 7f295ed9d640 1 -- 172.21.15.64:0/1801390827 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2960116b70 con 0x7f296011cba0 2024-09-21T04:07:47.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.482+0000 7f295e59c640 1 --2- 172.21.15.64:0/1801390827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f296010dbf0 0x7f296011bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:07:47.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.482+0000 7f295ed9d640 1 --2- 172.21.15.64:0/1801390827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f296011cba0 0x7f2960116350 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2954030a80 tx=0x7f2954030ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:47.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.483+0000 7f294f7fe640 1 -- 172.21.15.64:0/1801390827 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2954040070 con 0x7f296011cba0 2024-09-21T04:07:47.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.483+0000 7f295f59e640 1 -- 172.21.15.64:0/1801390827 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2960116e00 con 0x7f296011cba0 2024-09-21T04:07:47.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.483+0000 7f295f59e640 1 -- 172.21.15.64:0/1801390827 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29601be260 con 0x7f296011cba0 2024-09-21T04:07:47.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.483+0000 7f294f7fe640 1 -- 172.21.15.64:0/1801390827 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f295403c920 con 0x7f296011cba0 2024-09-21T04:07:47.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.483+0000 7f294f7fe640 1 -- 172.21.15.64:0/1801390827 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2954035030 con 0x7f296011cba0 2024-09-21T04:07:47.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.485+0000 7f295f59e640 1 -- 172.21.15.64:0/1801390827 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29400052c0 con 0x7f296011cba0 2024-09-21T04:07:47.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.506+0000 7f294f7fe640 1 -- 172.21.15.64:0/1801390827 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f295403b3d0 con 0x7f296011cba0 2024-09-21T04:07:47.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.507+0000 7f294f7fe640 1 --2- 172.21.15.64:0/1801390827 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f29480b6880 0x7f29480b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:47.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.508+0000 7f294f7fe640 1 -- 172.21.15.64:0/1801390827 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f29541082f0 con 0x7f296011cba0 2024-09-21T04:07:47.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.512+0000 7f295e59c640 1 --2- 172.21.15.64:0/1801390827 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f29480b6880 0x7f29480b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:47.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.512+0000 7f294f7fe640 1 -- 172.21.15.64:0/1801390827 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2954049030 con 0x7f296011cba0 2024-09-21T04:07:47.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:47.518+0000 7f295e59c640 1 --2- 172.21.15.64:0/1801390827 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f29480b6880 0x7f29480b8d90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f296011bcf0 tx=0x7f29580073c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:47.764 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:47 smithi064 ceph-mon[31180]: pgmap v434: 97 pgs: 97 active+clean; 503 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 320 op/s 2024-09-21T04:07:48.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:47 smithi119 ceph-mon[34811]: pgmap v434: 97 pgs: 97 active+clean; 503 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 320 op/s 2024-09-21T04:07:48.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:48.009+0000 7f295f59e640 1 -- 172.21.15.64:0/1801390827 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29400055b0 con 0x7f296011cba0 2024-09-21T04:07:48.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:48.012+0000 7f294f7fe640 1 -- 172.21.15.64:0/1801390827 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11915 (secure 0 0 0) 0x7f29540ce830 con 0x7f296011cba0 2024-09-21T04:07:48.020 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:07:48.020 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":32,"btime":"2024-09-21T04:07:28:653063+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14805,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/97995119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":97995119},{"type":"v1","addr":"172.21.15.64:6839","nonce":97995119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/390891068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":390891068},{"type":"v1","addr":"172.21.15.64:6841","nonce":390891068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:02:03.354744+0000","modified":"2024-09-21T04:07:27.709655+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24710,"mds_1":24680,"mds_2":24661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24634":{"gid":24634,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6839/782190424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782190424},{"type":"v1","addr":"172.21.15.131:6839","nonce":782190424}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24680":{"gid":24680,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24710":{"gid":24710,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6839/1319793636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1319793636},{"type":"v1","addr":"172.21.15.119:6839","nonce":1319793636}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24710,"qdb_cluster":[24661,24680,24710]},"id":1}]} 2024-09-21T04:07:48.020 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 32 2024-09-21T04:07:48.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:48.023+0000 7f294d7fa640 1 -- 172.21.15.64:0/1801390827 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f29480b6880 msgr2=0x7f29480b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:48.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:48.023+0000 7f294d7fa640 1 --2- 172.21.15.64:0/1801390827 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f29480b6880 0x7f29480b8d90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f296011bcf0 tx=0x7f29580073c0 comp rx=0 tx=0).stop 2024-09-21T04:07:48.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:48.023+0000 7f294d7fa640 1 -- 172.21.15.64:0/1801390827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f296011cba0 msgr2=0x7f2960116350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:48.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:48.023+0000 7f294d7fa640 1 --2- 172.21.15.64:0/1801390827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f296011cba0 0x7f2960116350 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2954030a80 tx=0x7f2954030ab0 comp rx=0 tx=0).stop 2024-09-21T04:07:48.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:48.026+0000 7f294d7fa640 1 -- 172.21.15.64:0/1801390827 shutdown_connections 2024-09-21T04:07:48.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:48.026+0000 7f294d7fa640 1 --2- 172.21.15.64:0/1801390827 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f29480b6880 0x7f29480b8d90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:48.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:48.026+0000 7f294d7fa640 1 --2- 172.21.15.64:0/1801390827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f296011cba0 0x7f2960116350 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:48.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:48.026+0000 7f294d7fa640 1 --2- 172.21.15.64:0/1801390827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2960111cd0 0x7f296011c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:48.029 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:48.026+0000 7f294d7fa640 1 --2- 172.21.15.64:0/1801390827 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f296010dbf0 0x7f296011bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:48.030 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:48.026+0000 7f294d7fa640 1 -- 172.21.15.64:0/1801390827 >> 172.21.15.64:0/1801390827 conn(0x7f296006e9f0 msgr2=0x7f296011cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:48.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:48.031+0000 7f294d7fa640 1 -- 172.21.15.64:0/1801390827 shutdown_connections 2024-09-21T04:07:48.035 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:48.031+0000 7f294d7fa640 1 -- 172.21.15.64:0/1801390827 wait complete. 2024-09-21T04:07:48.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:47 smithi131 ceph-mon[35193]: pgmap v434: 97 pgs: 97 active+clean; 503 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 320 op/s 2024-09-21T04:07:48.723 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:48 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1801390827' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:07:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.753+0000 7efd708bd640 1 -- 172.21.15.64:0/1703877553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd6c10a660 msgr2=0x7efd6c10aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.753+0000 7efd708bd640 1 --2- 172.21.15.64:0/1703877553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd6c10a660 0x7efd6c10aa70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009900 tx=0x7efd5c02a410 comp rx=0 tx=0).stop 2024-09-21T04:07:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.754+0000 7efd708bd640 1 -- 172.21.15.64:0/1703877553 shutdown_connections 2024-09-21T04:07:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.754+0000 7efd708bd640 1 --2- 172.21.15.64:0/1703877553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd6c110350 0x7efd6c110b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.754+0000 7efd708bd640 1 --2- 172.21.15.64:0/1703877553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd6c10fa00 0x7efd6c10fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.754+0000 7efd708bd640 1 --2- 172.21.15.64:0/1703877553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd6c10a660 0x7efd6c10aa70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.754+0000 7efd708bd640 1 -- 172.21.15.64:0/1703877553 >> 172.21.15.64:0/1703877553 conn(0x7efd6c070dd0 msgr2=0x7efd6c0711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:48.784 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.781+0000 7efd708bd640 1 -- 172.21.15.64:0/1703877553 shutdown_connections 2024-09-21T04:07:48.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.782+0000 7efd708bd640 1 -- 172.21.15.64:0/1703877553 wait complete. 2024-09-21T04:07:48.785 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.783+0000 7efd708bd640 1 Processor -- start 2024-09-21T04:07:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.783+0000 7efd708bd640 1 -- start start 2024-09-21T04:07:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.784+0000 7efd708bd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd6c10fa00 0x7efd6c120150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.784+0000 7efd708bd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd6c110350 0x7efd6c119350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.784+0000 7efd708bd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd6c119890 0x7efd6c119ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.784+0000 7efd708bd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c122840 con 0x7efd6c10fa00 2024-09-21T04:07:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.784+0000 7efd708bd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c1226c0 con 0x7efd6c119890 2024-09-21T04:07:48.787 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.784+0000 7efd708bd640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c1229c0 con 0x7efd6c110350 2024-09-21T04:07:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.784+0000 7efd6ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd6c119890 0x7efd6c119ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.785+0000 7efd6a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd6c10fa00 0x7efd6c120150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.785+0000 7efd6ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd6c119890 0x7efd6c119ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:52082/0 (socket says 172.21.15.64:52082) 2024-09-21T04:07:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.785+0000 7efd6ad9d640 1 -- 172.21.15.64:0/319188744 learned_addr learned my addr 172.21.15.64:0/319188744 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.785+0000 7efd69d9b640 1 --2- 172.21.15.64:0/319188744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd6c110350 0x7efd6c119350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.785+0000 7efd6a59c640 1 -- 172.21.15.64:0/319188744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd6c110350 msgr2=0x7efd6c119350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.785+0000 7efd6a59c640 1 --2- 172.21.15.64:0/319188744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd6c110350 0x7efd6c119350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.785+0000 7efd6a59c640 1 -- 172.21.15.64:0/319188744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd6c119890 msgr2=0x7efd6c119ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.785+0000 7efd6a59c640 1 --2- 172.21.15.64:0/319188744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd6c119890 0x7efd6c119ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.785+0000 7efd6a59c640 1 -- 172.21.15.64:0/319188744 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd6c11ecf0 con 0x7efd6c10fa00 2024-09-21T04:07:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.786+0000 7efd6a59c640 1 --2- 172.21.15.64:0/319188744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd6c10fa00 0x7efd6c120150 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7efd5c000c00 tx=0x7efd5c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:48.798 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.796+0000 7efd5b7fe640 1 -- 172.21.15.64:0/319188744 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c040070 con 0x7efd6c10fa00 2024-09-21T04:07:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.796+0000 7efd708bd640 1 -- 172.21.15.64:0/319188744 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd6c11ef80 con 0x7efd6c10fa00 2024-09-21T04:07:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.796+0000 7efd708bd640 1 -- 172.21.15.64:0/319188744 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd6c11f470 con 0x7efd6c10fa00 2024-09-21T04:07:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.800+0000 7efd5b7fe640 1 -- 172.21.15.64:0/319188744 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd5c003da0 con 0x7efd6c10fa00 2024-09-21T04:07:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.800+0000 7efd5b7fe640 1 -- 172.21.15.64:0/319188744 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c035040 con 0x7efd6c10fa00 2024-09-21T04:07:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.806+0000 7efd7782f740 1 Processor -- start 2024-09-21T04:07:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.807+0000 7efd7782f740 1 -- start start 2024-09-21T04:07:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.808+0000 7efd7782f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb441b0 0x556c8eee99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.808+0000 7efd7782f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c8eb1b2f0 0x556c8eee9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.808+0000 7efd7782f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c8eb234d0 0x556c8eeea430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.808+0000 7efd7782f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556c8ee5e6f0 con 0x556c8eb1b2f0 2024-09-21T04:07:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.808+0000 7efd6a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c8eb1b2f0 0x556c8eee9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.808+0000 7efd6ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb441b0 0x556c8eee99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.808+0000 7efd7782f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556c8ee4db20 con 0x556c8eb441b0 2024-09-21T04:07:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.808+0000 7efd5b7fe640 1 -- 172.21.15.64:0/319188744 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7efd5c043070 con 0x7efd6c10fa00 2024-09-21T04:07:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.808+0000 7efd6ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb441b0 0x556c8eee99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:52088/0 (socket says 172.21.15.64:52088) 2024-09-21T04:07:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.808+0000 7efd6ad9d640 1 -- 172.21.15.64:0/2187867547 learned_addr learned my addr 172.21.15.64:0/2187867547 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.809+0000 7efd7782f740 1 -- 172.21.15.64:0/2187867547 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556c8ee51930 con 0x556c8eb234d0 2024-09-21T04:07:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.809+0000 7efd69d9b640 1 --2- 172.21.15.64:0/2187867547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c8eb234d0 0x556c8eeea430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.810+0000 7efd6ad9d640 1 -- 172.21.15.64:0/2187867547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c8eb234d0 msgr2=0x556c8eeea430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.810+0000 7efd6ad9d640 1 --2- 172.21.15.64:0/2187867547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c8eb234d0 0x556c8eeea430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.810+0000 7efd6ad9d640 1 -- 172.21.15.64:0/2187867547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c8eb1b2f0 msgr2=0x556c8eee9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.810+0000 7efd6ad9d640 1 --2- 172.21.15.64:0/2187867547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c8eb1b2f0 0x556c8eee9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.810+0000 7efd6ad9d640 1 -- 172.21.15.64:0/2187867547 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c8ee13530 con 0x556c8eb441b0 2024-09-21T04:07:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.810+0000 7efd6a59c640 1 --2- 172.21.15.64:0/2187867547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c8eb1b2f0 0x556c8eee9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:07:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.811+0000 7efd6ad9d640 1 --2- 172.21.15.64:0/2187867547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb441b0 0x556c8eee99b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efd6000ad10 tx=0x7efd6000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aeec5d0f8408549 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.812+0000 7efd5b7fe640 1 --2- 172.21.15.64:0/319188744 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7efd540b6800 0x7efd540b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.813+0000 7efd6a59c640 1 --2- 172.21.15.64:0/319188744 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7efd540b6800 0x7efd540b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.814+0000 7efd6a59c640 1 --2- 172.21.15.64:0/319188744 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7efd540b6800 0x7efd540b8d10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x556c8eee8630 tx=0x7efd5c02a410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.814+0000 7efd5b7fe640 1 -- 172.21.15.64:0/319188744 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7efd5c1079c0 con 0x7efd6c10fa00 2024-09-21T04:07:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.833+0000 7efd58ff9640 1 -- 172.21.15.64:0/2187867547 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60005020 con 0x556c8eb441b0 2024-09-21T04:07:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.833+0000 7efd58ff9640 1 -- 172.21.15.64:0/2187867547 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd60005670 con 0x556c8eb441b0 2024-09-21T04:07:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.833+0000 7efd58ff9640 1 -- 172.21.15.64:0/2187867547 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd600108a0 con 0x556c8eb441b0 2024-09-21T04:07:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.834+0000 7efd7782f740 1 -- 172.21.15.64:0/2187867547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb441b0 msgr2=0x556c8eee99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.834+0000 7efd7782f740 1 --2- 172.21.15.64:0/2187867547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb441b0 0x556c8eee99b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efd6000ad10 tx=0x7efd6000f790 comp rx=0 tx=0).stop 2024-09-21T04:07:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.848+0000 7efd7782f740 1 -- 172.21.15.64:0/2187867547 shutdown_connections 2024-09-21T04:07:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.848+0000 7efd7782f740 1 --2- 172.21.15.64:0/2187867547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c8eb234d0 0x556c8eeea430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.848+0000 7efd7782f740 1 --2- 172.21.15.64:0/2187867547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c8eb1b2f0 0x556c8eee9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.848+0000 7efd7782f740 1 --2- 172.21.15.64:0/2187867547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb441b0 0x556c8eee99b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:48.850 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.848+0000 7efd7782f740 1 -- 172.21.15.64:0/2187867547 >> 172.21.15.64:0/2187867547 conn(0x556c8eb38300 msgr2=0x556c8eee78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.852+0000 7efd7782f740 1 -- 172.21.15.64:0/2187867547 shutdown_connections 2024-09-21T04:07:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.852+0000 7efd7782f740 1 -- 172.21.15.64:0/2187867547 wait complete. 2024-09-21T04:07:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.853+0000 7efd7782f740 1 -- start start 2024-09-21T04:07:48.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.853+0000 7efd7782f740 1 Processor -- start 2024-09-21T04:07:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.853+0000 7efd7782f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb1b2f0 0x556c8eef5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.853+0000 7efd7782f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c8eb234d0 0x556c8eef5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.853+0000 7efd7782f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c8eb441b0 0x556c8ef014b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.853+0000 7efd7782f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556c8ed85ab0 con 0x556c8eb234d0 2024-09-21T04:07:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.853+0000 7efd7782f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556c8edf3730 con 0x556c8eb1b2f0 2024-09-21T04:07:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.853+0000 7efd7782f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556c8eef8130 con 0x556c8eb441b0 2024-09-21T04:07:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.853+0000 7efd7782f740 10 client.0 _ll_register_callbacks cb 0x556c8ee476b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.854+0000 7efd6ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb1b2f0 0x556c8eef5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.854+0000 7efd6ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb1b2f0 0x556c8eef5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:52100/0 (socket says 172.21.15.64:52100) 2024-09-21T04:07:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.854+0000 7efd69d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c8eb234d0 0x556c8eef5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.854+0000 7efd6ad9d640 1 -- 172.21.15.64:0/938261748 learned_addr learned my addr 172.21.15.64:0/938261748 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.854+0000 7efd6ad9d640 1 --2- 172.21.15.64:0/938261748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c8eb441b0 0x556c8ef014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.855+0000 7efd6ad9d640 1 -- 172.21.15.64:0/938261748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c8eb441b0 msgr2=0x556c8ef014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.855+0000 7efd6ad9d640 1 --2- 172.21.15.64:0/938261748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c8eb441b0 0x556c8ef014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:48.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.855+0000 7efd6ad9d640 1 -- 172.21.15.64:0/938261748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c8eb234d0 msgr2=0x556c8eef5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:48.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.855+0000 7efd6ad9d640 1 --2- 172.21.15.64:0/938261748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c8eb234d0 0x556c8eef5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:48.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.855+0000 7efd6ad9d640 1 -- 172.21.15.64:0/938261748 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c8ebaf790 con 0x556c8eb1b2f0 2024-09-21T04:07:48.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.855+0000 7efd6ad9d640 1 --2- 172.21.15.64:0/938261748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb1b2f0 0x556c8eef5120 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efd6000aa70 tx=0x7efd60004e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bb3dfdd6d8e62c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.872+0000 7efd51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:07:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.872+0000 7efd51ffb640 1 -- 172.21.15.64:0/938261748 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6002c060 con 0x556c8eb1b2f0 2024-09-21T04:07:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.872+0000 7efd7782f740 1 -- 172.21.15.64:0/938261748 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd60005670 con 0x556c8eb1b2f0 2024-09-21T04:07:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.872+0000 7efd51ffb640 1 -- 172.21.15.64:0/938261748 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7efd60005440 con 0x556c8eb1b2f0 2024-09-21T04:07:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.872+0000 7efd51ffb640 1 -- 172.21.15.64:0/938261748 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efd60004030 con 0x556c8eb1b2f0 2024-09-21T04:07:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.873+0000 7efd7782f740 10 client.26036 fetch_fsmap learned FSMap version 32 2024-09-21T04:07:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.873+0000 7efd7782f740 1 -- 172.21.15.64:0/938261748 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556c8ec28620 con 0x556c8eb1b2f0 2024-09-21T04:07:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.878+0000 7efd51ffb640 1 -- 172.21.15.64:0/938261748 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd60005670 con 0x556c8eb1b2f0 2024-09-21T04:07:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.878+0000 7efd51ffb640 1 -- 172.21.15.64:0/938261748 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7efd600035f0 con 0x556c8eb1b2f0 2024-09-21T04:07:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.878+0000 7efd7782f740 10 client.26036 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T04:07:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.878+0000 7efd7782f740 10 client.26036 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:07:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.878+0000 7efd7782f740 20 client.26036 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.878+0000 7efd7782f740 1 --2- 172.21.15.64:0/938261748 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556c8eb02d80 0x556c8ef07670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.878+0000 7efd7782f740 4 client.26036 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:07:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.878+0000 7efd7782f740 1 -- 172.21.15.64:0/938261748 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c8ee60640 con 0x556c8eb02d80 2024-09-21T04:07:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.878+0000 7efd69d9b640 1 --2- 172.21.15.64:0/938261748 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556c8eb02d80 0x556c8ef07670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.879+0000 7efd51ffb640 10 client.26036 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.880+0000 7efd69d9b640 1 --2- 172.21.15.64:0/938261748 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556c8eb02d80 0x556c8ef07670 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bf97c8e40dd583a server_cookie=59af802c3b5334f5 in_seq=0 out_seq=0 2024-09-21T04:07:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.882+0000 7efd51ffb640 1 -- 172.21.15.64:0/938261748 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556c8ee60640 con 0x556c8eb02d80 2024-09-21T04:07:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.882+0000 7efd51ffb640 10 client.26036 handle_command_reply: tid=0 2024-09-21T04:07:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.882+0000 7efd7782f740 1 client.26036 shutdown 2024-09-21T04:07:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.886+0000 7efd7782f740 1 -- 172.21.15.64:0/938261748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb1b2f0 msgr2=0x556c8eef5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.886+0000 7efd7782f740 1 --2- 172.21.15.64:0/938261748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb1b2f0 0x556c8eef5120 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efd6000aa70 tx=0x7efd60004e00 comp rx=0 tx=0).stop 2024-09-21T04:07:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.886+0000 7efd7782f740 1 -- 172.21.15.64:0/938261748 shutdown_connections 2024-09-21T04:07:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.886+0000 7efd7782f740 1 --2- 172.21.15.64:0/938261748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c8eb441b0 0x556c8ef014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.886+0000 7efd7782f740 1 --2- 172.21.15.64:0/938261748 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556c8eb02d80 0x556c8ef07670 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.886+0000 7efd7782f740 1 --2- 172.21.15.64:0/938261748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c8eb234d0 0x556c8eef5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.886+0000 7efd7782f740 1 --2- 172.21.15.64:0/938261748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb1b2f0 0x556c8eef5120 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.886+0000 7efd7782f740 1 -- 172.21.15.64:0/938261748 >> 172.21.15.64:0/938261748 conn(0x556c8eb38300 msgr2=0x556c8eeea9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.887+0000 7efd51ffb640 0 client.26036 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.890+0000 7efd7782f740 1 -- 172.21.15.64:0/938261748 shutdown_connections 2024-09-21T04:07:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.890+0000 7efd7782f740 1 -- 172.21.15.64:0/938261748 wait complete. 2024-09-21T04:07:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.890+0000 7efd7782f740 20 client.26036 trim_cache size 0 max 16384 2024-09-21T04:07:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.989+0000 7efd7782f740 1 Processor -- start 2024-09-21T04:07:48.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.993+0000 7efd7782f740 1 -- start start 2024-09-21T04:07:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.993+0000 7efd7782f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c8eb1b2f0 0x556c8eef3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.994+0000 7efd7782f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb234d0 0x556c8eeeb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.994+0000 7efd7782f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c8eb441b0 0x556c8ef013b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.994+0000 7efd7782f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c0113a0 con 0x556c8eb1b2f0 2024-09-21T04:07:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.994+0000 7efd7782f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c00ebe0 con 0x556c8eb234d0 2024-09-21T04:07:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.994+0000 7efd7782f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c00ca30 con 0x556c8eb441b0 2024-09-21T04:07:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.994+0000 7efd69d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb234d0 0x556c8eeeb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.994+0000 7efd69d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb234d0 0x556c8eeeb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:52106/0 (socket says 172.21.15.64:52106) 2024-09-21T04:07:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.994+0000 7efd69d9b640 1 -- 172.21.15.64:0/3436762340 learned_addr learned my addr 172.21.15.64:0/3436762340 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.994+0000 7efd6ad9d640 1 --2- 172.21.15.64:0/3436762340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c8eb1b2f0 0x556c8eef3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.995+0000 7efd69d9b640 1 -- 172.21.15.64:0/3436762340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c8eb441b0 msgr2=0x556c8ef013b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:07:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.995+0000 7efd69d9b640 1 --2- 172.21.15.64:0/3436762340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c8eb441b0 0x556c8ef013b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.995+0000 7efd69d9b640 1 -- 172.21.15.64:0/3436762340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c8eb1b2f0 msgr2=0x556c8eef3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.995+0000 7efd69d9b640 1 --2- 172.21.15.64:0/3436762340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c8eb1b2f0 0x556c8eef3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.995+0000 7efd69d9b640 1 -- 172.21.15.64:0/3436762340 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c8eb02fb0 con 0x556c8eb234d0 2024-09-21T04:07:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.995+0000 7efd6ad9d640 1 --2- 172.21.15.64:0/3436762340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c8eb1b2f0 0x556c8eef3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:07:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:48.996+0000 7efd69d9b640 1 --2- 172.21.15.64:0/3436762340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb234d0 0x556c8eeeb0f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efd64000f80 tx=0x7efd64009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab67730f7bcaedfd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:49.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:48 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1801390827' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:07:49.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:48 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1801390827' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:07:49.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.083+0000 7efd53fff640 1 -- 172.21.15.64:0/3436762340 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd64020070 con 0x556c8eb234d0 2024-09-21T04:07:49.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.083+0000 7efd53fff640 1 -- 172.21.15.64:0/3436762340 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd6400fd50 con 0x556c8eb234d0 2024-09-21T04:07:49.085 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.083+0000 7efd53fff640 1 -- 172.21.15.64:0/3436762340 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6401ed70 con 0x556c8eb234d0 2024-09-21T04:07:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.084+0000 7efd7782f740 1 -- 172.21.15.64:0/3436762340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb234d0 msgr2=0x556c8eeeb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.084+0000 7efd7782f740 1 --2- 172.21.15.64:0/3436762340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb234d0 0x556c8eeeb0f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efd64000f80 tx=0x7efd64009500 comp rx=0 tx=0).stop 2024-09-21T04:07:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.084+0000 7efd7782f740 1 -- 172.21.15.64:0/3436762340 shutdown_connections 2024-09-21T04:07:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.084+0000 7efd7782f740 1 --2- 172.21.15.64:0/3436762340 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c8eb441b0 0x556c8ef013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.085+0000 7efd7782f740 1 --2- 172.21.15.64:0/3436762340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb234d0 0x556c8eeeb0f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.085+0000 7efd7782f740 1 --2- 172.21.15.64:0/3436762340 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c8eb1b2f0 0x556c8eef3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.085+0000 7efd7782f740 1 -- 172.21.15.64:0/3436762340 >> 172.21.15.64:0/3436762340 conn(0x556c8eb38300 msgr2=0x556c8eeea9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.085+0000 7efd7782f740 1 -- 172.21.15.64:0/3436762340 shutdown_connections 2024-09-21T04:07:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.085+0000 7efd7782f740 1 -- 172.21.15.64:0/3436762340 wait complete. 2024-09-21T04:07:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.086+0000 7efd7782f740 1 -- start start 2024-09-21T04:07:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.086+0000 7efd7782f740 1 Processor -- start 2024-09-21T04:07:49.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.087+0000 7efd7782f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c8eb1b2f0 0x556c8eefd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:49.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.087+0000 7efd7782f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb234d0 0x556c8eefc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.088+0000 7efd6ad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c8eb1b2f0 0x556c8eefd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.088+0000 7efd6ad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c8eb1b2f0 0x556c8eefd9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:42620/0 (socket says 172.21.15.64:42620) 2024-09-21T04:07:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.088+0000 7efd69d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb234d0 0x556c8eefc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.088+0000 7efd69d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb234d0 0x556c8eefc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:52122/0 (socket says 172.21.15.64:52122) 2024-09-21T04:07:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.088+0000 7efd7782f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c8eb441b0 0x556c8eefc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.088+0000 7efd6ad9d640 1 -- 172.21.15.64:0/4153703847 learned_addr learned my addr 172.21.15.64:0/4153703847 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.088+0000 7efd7782f740 1 -- 172.21.15.64:0/4153703847 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c00d0a0 con 0x556c8eb441b0 2024-09-21T04:07:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.088+0000 7efd7782f740 1 -- 172.21.15.64:0/4153703847 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c00e510 con 0x556c8eb234d0 2024-09-21T04:07:49.092 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.088+0000 7efd7782f740 1 -- 172.21.15.64:0/4153703847 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c00d770 con 0x556c8eb1b2f0 2024-09-21T04:07:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.089+0000 7efd6ad9d640 1 --2- 172.21.15.64:0/4153703847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c8eb441b0 0x556c8eefc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.089+0000 7efd69d9b640 1 -- 172.21.15.64:0/4153703847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c8eb1b2f0 msgr2=0x556c8eefd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.089+0000 7efd69d9b640 1 --2- 172.21.15.64:0/4153703847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c8eb1b2f0 0x556c8eefd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.089+0000 7efd69d9b640 1 -- 172.21.15.64:0/4153703847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c8eb441b0 msgr2=0x556c8eefc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.089+0000 7efd69d9b640 1 --2- 172.21.15.64:0/4153703847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c8eb441b0 0x556c8eefc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.089+0000 7efd69d9b640 1 -- 172.21.15.64:0/4153703847 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c8eefcf20 con 0x556c8eb234d0 2024-09-21T04:07:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.090+0000 7efd6ad9d640 1 --2- 172.21.15.64:0/4153703847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c8eb441b0 0x556c8eefc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:07:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.090+0000 7efd69d9b640 1 --2- 172.21.15.64:0/4153703847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb234d0 0x556c8eefc4a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efd64009370 tx=0x7efd6401eaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7acd51368fea2376 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.090+0000 7efd7782f740 10 client.0 _ll_register_callbacks cb 0x7efd6000ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:49.093 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.091+0000 7efd7782f740 1 -- 172.21.15.64:0/4153703847 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd6400fd50 con 0x556c8eb234d0 2024-09-21T04:07:49.097 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T04:07:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.135+0000 7efd51ffb640 10 client.26042 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:07:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.135+0000 7efd51ffb640 1 -- 172.21.15.64:0/4153703847 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd64020050 con 0x556c8eb234d0 2024-09-21T04:07:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.135+0000 7efd51ffb640 1 -- 172.21.15.64:0/4153703847 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7efd64004cb0 con 0x556c8eb234d0 2024-09-21T04:07:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.135+0000 7efd51ffb640 1 -- 172.21.15.64:0/4153703847 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efd6401edf0 con 0x556c8eb234d0 2024-09-21T04:07:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.135+0000 7efd7782f740 10 client.26042 fetch_fsmap learned FSMap version 32 2024-09-21T04:07:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.135+0000 7efd7782f740 1 -- 172.21.15.64:0/4153703847 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({fsmap=32}) -- 0x556c8ef70a70 con 0x556c8eb234d0 2024-09-21T04:07:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.136+0000 7efd51ffb640 1 -- 172.21.15.64:0/4153703847 <== mon.2 v2:172.21.15.119:3300/0 4 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7efd6401e440 con 0x556c8eb234d0 2024-09-21T04:07:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.136+0000 7efd7782f740 10 client.26042 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T04:07:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.136+0000 7efd7782f740 10 client.26042 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:07:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.136+0000 7efd7782f740 20 client.26042 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.136+0000 7efd7782f740 1 --2- 172.21.15.64:0/4153703847 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556c8ead81c0 0x556c8ef73310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.136+0000 7efd7782f740 4 client.26042 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:07:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.136+0000 7efd7782f740 1 -- 172.21.15.64:0/4153703847 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7efd6400fd50 con 0x556c8ead81c0 2024-09-21T04:07:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.136+0000 7efd6ad9d640 1 --2- 172.21.15.64:0/4153703847 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556c8ead81c0 0x556c8ef73310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.137+0000 7efd6ad9d640 1 --2- 172.21.15.64:0/4153703847 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556c8ead81c0 0x556c8ef73310 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2fba0613fd1a997 server_cookie=4e4b792740d4e85 in_seq=0 out_seq=0 2024-09-21T04:07:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.137+0000 7efd51ffb640 10 client.26042 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.359+0000 7efd51ffb640 1 -- 172.21.15.64:0/4153703847 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7efd6400fd50 con 0x556c8ead81c0 2024-09-21T04:07:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.359+0000 7efd51ffb640 10 client.26042 handle_command_reply: tid=0 2024-09-21T04:07:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.359+0000 7efd7782f740 1 client.26042 shutdown 2024-09-21T04:07:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.359+0000 7efd7782f740 1 -- 172.21.15.64:0/4153703847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb234d0 msgr2=0x556c8eefc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.359+0000 7efd7782f740 1 --2- 172.21.15.64:0/4153703847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb234d0 0x556c8eefc4a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efd64009370 tx=0x7efd6401eaf0 comp rx=0 tx=0).stop 2024-09-21T04:07:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.360+0000 7efd7782f740 1 -- 172.21.15.64:0/4153703847 shutdown_connections 2024-09-21T04:07:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.360+0000 7efd7782f740 1 --2- 172.21.15.64:0/4153703847 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556c8ead81c0 0x556c8ef73310 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.360+0000 7efd51ffb640 0 client.26042 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.360+0000 7efd7782f740 1 --2- 172.21.15.64:0/4153703847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556c8eb441b0 0x556c8eefc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.360+0000 7efd7782f740 1 --2- 172.21.15.64:0/4153703847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556c8eb234d0 0x556c8eefc4a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.360+0000 7efd7782f740 1 --2- 172.21.15.64:0/4153703847 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556c8eb1b2f0 0x556c8eefd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.360+0000 7efd7782f740 1 -- 172.21.15.64:0/4153703847 >> 172.21.15.64:0/4153703847 conn(0x556c8eb38300 msgr2=0x556c8eeea8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.361+0000 7efd7782f740 1 -- 172.21.15.64:0/4153703847 shutdown_connections 2024-09-21T04:07:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.361+0000 7efd7782f740 1 -- 172.21.15.64:0/4153703847 wait complete. 2024-09-21T04:07:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.361+0000 7efd7782f740 20 client.26042 trim_cache size 0 max 16384 2024-09-21T04:07:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:07:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:07:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:07:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:07:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.379+0000 7efd708bd640 1 -- 172.21.15.64:0/319188744 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7efd540b6800 msgr2=0x7efd540b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.379+0000 7efd708bd640 1 --2- 172.21.15.64:0/319188744 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7efd540b6800 0x7efd540b8d10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x556c8eee8630 tx=0x7efd5c02a410 comp rx=0 tx=0).stop 2024-09-21T04:07:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.379+0000 7efd708bd640 1 -- 172.21.15.64:0/319188744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd6c10fa00 msgr2=0x7efd6c120150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.379+0000 7efd708bd640 1 --2- 172.21.15.64:0/319188744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd6c10fa00 0x7efd6c120150 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7efd5c000c00 tx=0x7efd5c0056d0 comp rx=0 tx=0).stop 2024-09-21T04:07:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.380+0000 7efd708bd640 1 -- 172.21.15.64:0/319188744 shutdown_connections 2024-09-21T04:07:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.380+0000 7efd708bd640 1 --2- 172.21.15.64:0/319188744 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7efd540b6800 0x7efd540b8d10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.380+0000 7efd708bd640 1 --2- 172.21.15.64:0/319188744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd6c119890 0x7efd6c119ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.380+0000 7efd708bd640 1 --2- 172.21.15.64:0/319188744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd6c110350 0x7efd6c119350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.380+0000 7efd708bd640 1 --2- 172.21.15.64:0/319188744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd6c10fa00 0x7efd6c120150 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.380+0000 7efd708bd640 1 -- 172.21.15.64:0/319188744 >> 172.21.15.64:0/319188744 conn(0x7efd6c070dd0 msgr2=0x7efd6c073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.381+0000 7efd708bd640 1 -- 172.21.15.64:0/319188744 shutdown_connections 2024-09-21T04:07:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:07:49.381+0000 7efd708bd640 1 -- 172.21.15.64:0/319188744 wait complete. 2024-09-21T04:07:50.009 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:49 smithi119 ceph-mon[34811]: pgmap v435: 97 pgs: 97 active+clean; 518 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.2 MiB/s wr, 335 op/s 2024-09-21T04:07:50.015 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:49 smithi064 ceph-mon[31180]: pgmap v435: 97 pgs: 97 active+clean; 518 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.2 MiB/s wr, 335 op/s 2024-09-21T04:07:50.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:49 smithi131 ceph-mon[35193]: pgmap v435: 97 pgs: 97 active+clean; 518 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.2 MiB/s wr, 335 op/s 2024-09-21T04:07:50.786 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:07:51.173 INFO:tasks.workunit.client.0.smithi064.stdout: 84 1527650 392 1071851 491 836676 1357 2024-09-21T04:07:51.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.545+0000 7f368d66e640 1 -- 172.21.15.64:0/976393171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3688110640 msgr2=0x7f3688116e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:51.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.545+0000 7f368d66e640 1 --2- 172.21.15.64:0/976393171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3688110640 0x7f3688116e90 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3680005fd0 tx=0x7f3680032d90 comp rx=0 tx=0).stop 2024-09-21T04:07:51.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.545+0000 7f368d66e640 1 -- 172.21.15.64:0/976393171 shutdown_connections 2024-09-21T04:07:51.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.545+0000 7f368d66e640 1 --2- 172.21.15.64:0/976393171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3688110640 0x7f3688116e90 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.545+0000 7f368d66e640 1 --2- 172.21.15.64:0/976393171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f368810fcf0 0x7f3688110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.545+0000 7f368d66e640 1 --2- 172.21.15.64:0/976393171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f368810a650 0x7f368810aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.545+0000 7f368d66e640 1 -- 172.21.15.64:0/976393171 >> 172.21.15.64:0/976393171 conn(0x7f368806ee50 msgr2=0x7f368806f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:51.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.548+0000 7f368d66e640 1 -- 172.21.15.64:0/976393171 shutdown_connections 2024-09-21T04:07:51.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.548+0000 7f368d66e640 1 -- 172.21.15.64:0/976393171 wait complete. 2024-09-21T04:07:51.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.549+0000 7f368d66e640 1 Processor -- start 2024-09-21T04:07:51.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.549+0000 7f368d66e640 1 -- start start 2024-09-21T04:07:51.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.550+0000 7f368d66e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f368810a650 0x7f36881a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:51.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.550+0000 7f368d66e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f368810fcf0 0x7f36881a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:51.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.550+0000 7f3687fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f368810a650 0x7f36881a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:51.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.550+0000 7f3687fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f368810a650 0x7f36881a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:45098/0 (socket says 172.21.15.64:45098) 2024-09-21T04:07:51.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.550+0000 7f3687fff640 1 -- 172.21.15.64:0/3880244947 learned_addr learned my addr 172.21.15.64:0/3880244947 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:51.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.550+0000 7f368d66e640 1 --2- 172.21.15.64:0/3880244947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3688110640 0x7f36881a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:51.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.550+0000 7f368d66e640 1 -- 172.21.15.64:0/3880244947 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3688119f00 con 0x7f368810fcf0 2024-09-21T04:07:51.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.550+0000 7f368d66e640 1 -- 172.21.15.64:0/3880244947 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3688119d80 con 0x7f368810a650 2024-09-21T04:07:51.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.551+0000 7f368d66e640 1 -- 172.21.15.64:0/3880244947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f368811a080 con 0x7f3688110640 2024-09-21T04:07:51.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.551+0000 7f368ce6d640 1 --2- 172.21.15.64:0/3880244947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3688110640 0x7f36881a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:51.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.551+0000 7f368ce6d640 1 -- 172.21.15.64:0/3880244947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f368810a650 msgr2=0x7f36881a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:51.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.551+0000 7f368ce6d640 1 --2- 172.21.15.64:0/3880244947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f368810a650 0x7f36881a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.551+0000 7f368ce6d640 1 -- 172.21.15.64:0/3880244947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f368810fcf0 msgr2=0x7f36881a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:51.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.551+0000 7f368ce6d640 1 --2- 172.21.15.64:0/3880244947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f368810fcf0 0x7f36881a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.551+0000 7f368ce6d640 1 -- 172.21.15.64:0/3880244947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36881a6820 con 0x7f3688110640 2024-09-21T04:07:51.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.552+0000 7f368ce6d640 1 --2- 172.21.15.64:0/3880244947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3688110640 0x7f36881a60f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3680033380 tx=0x7f368003b5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:51.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.552+0000 7f36857fa640 1 -- 172.21.15.64:0/3880244947 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3680049020 con 0x7f3688110640 2024-09-21T04:07:51.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.552+0000 7f36857fa640 1 -- 172.21.15.64:0/3880244947 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3680003e20 con 0x7f3688110640 2024-09-21T04:07:51.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.552+0000 7f368d66e640 1 -- 172.21.15.64:0/3880244947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36881a6ab0 con 0x7f3688110640 2024-09-21T04:07:51.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.553+0000 7f36857fa640 1 -- 172.21.15.64:0/3880244947 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368003bc60 con 0x7f3688110640 2024-09-21T04:07:51.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.553+0000 7f368d66e640 1 -- 172.21.15.64:0/3880244947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36881ad270 con 0x7f3688110640 2024-09-21T04:07:51.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.557+0000 7f3694e44740 1 Processor -- start 2024-09-21T04:07:51.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.558+0000 7f36857fa640 1 -- 172.21.15.64:0/3880244947 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f368004c020 con 0x7f3688110640 2024-09-21T04:07:51.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.560+0000 7f36857fa640 1 --2- 172.21.15.64:0/3880244947 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f36700b6780 0x7f36700b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:51.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.561+0000 7f36857fa640 1 -- 172.21.15.64:0/3880244947 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f3680110d70 con 0x7f3688110640 2024-09-21T04:07:51.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.561+0000 7f3694e44740 1 -- start start 2024-09-21T04:07:51.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.562+0000 7f3694e44740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558806a6f070 0x558806e16860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:51.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.562+0000 7f3694e44740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558806a461b0 0x558806e16da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:51.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.562+0000 7f3694e44740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558806a4e390 0x558806e172e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:51.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.562+0000 7f3694e44740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558806d8cea0 con 0x558806a461b0 2024-09-21T04:07:51.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.562+0000 7f3694e44740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x558806d7c2d0 con 0x558806a6f070 2024-09-21T04:07:51.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.562+0000 7f3694e44740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x558806d800e0 con 0x558806a4e390 2024-09-21T04:07:51.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.563+0000 7f368ce6d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558806a4e390 0x558806e172e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:51.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.563+0000 7f368ce6d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558806a4e390 0x558806e172e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:38928/0 (socket says 172.21.15.64:38928) 2024-09-21T04:07:51.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.563+0000 7f368ce6d640 1 -- 172.21.15.64:0/2564082406 learned_addr learned my addr 172.21.15.64:0/2564082406 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:51.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.563+0000 7f3687fff640 1 --2- 172.21.15.64:0/2564082406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558806a6f070 0x558806e16860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:51.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.563+0000 7f36877fe640 1 --2- 172.21.15.64:0/3880244947 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f36700b6780 0x7f36700b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:51.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.563+0000 7f3687fff640 1 -- 172.21.15.64:0/2564082406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558806a4e390 msgr2=0x558806e172e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:51.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.563+0000 7f3687fff640 1 --2- 172.21.15.64:0/2564082406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558806a4e390 0x558806e172e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.563+0000 7f3687fff640 1 -- 172.21.15.64:0/2564082406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558806a461b0 msgr2=0x558806e16da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:07:51.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.563+0000 7f3687fff640 1 --2- 172.21.15.64:0/2564082406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558806a461b0 0x558806e16da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.563+0000 7f3687fff640 1 -- 172.21.15.64:0/2564082406 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558806ada650 con 0x558806a6f070 2024-09-21T04:07:51.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.563+0000 7f368ce6d640 1 --2- 172.21.15.64:0/2564082406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558806a4e390 0x558806e172e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:07:51.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.566+0000 7f36877fe640 1 --2- 172.21.15.64:0/3880244947 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f36700b6780 0x7f36700b8c90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x558806e154e0 tx=0x7f3678006c40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:51.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.566+0000 7f3687fff640 1 --2- 172.21.15.64:0/2564082406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558806a6f070 0x558806e16860 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f367c00dcd0 tx=0x7f367c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=685af16633d2acc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:51.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.569+0000 7f366e7fc640 1 -- 172.21.15.64:0/2564082406 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c017440 con 0x558806a6f070 2024-09-21T04:07:51.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.569+0000 7f366e7fc640 1 -- 172.21.15.64:0/2564082406 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f367c017a90 con 0x558806a6f070 2024-09-21T04:07:51.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.570+0000 7f3694e44740 1 -- 172.21.15.64:0/2564082406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558806a6f070 msgr2=0x558806e16860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:51.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.570+0000 7f3694e44740 1 --2- 172.21.15.64:0/2564082406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558806a6f070 0x558806e16860 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f367c00dcd0 tx=0x7f367c005f90 comp rx=0 tx=0).stop 2024-09-21T04:07:51.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.571+0000 7f3694e44740 1 -- 172.21.15.64:0/2564082406 shutdown_connections 2024-09-21T04:07:51.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.571+0000 7f3694e44740 1 --2- 172.21.15.64:0/2564082406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558806a4e390 0x558806e172e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.571+0000 7f3694e44740 1 --2- 172.21.15.64:0/2564082406 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558806a461b0 0x558806e16da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.571+0000 7f3694e44740 1 --2- 172.21.15.64:0/2564082406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558806a6f070 0x558806e16860 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.574 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.571+0000 7f3694e44740 1 -- 172.21.15.64:0/2564082406 >> 172.21.15.64:0/2564082406 conn(0x558806a63460 msgr2=0x558806e14750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:51.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.572+0000 7f3694e44740 1 -- 172.21.15.64:0/2564082406 shutdown_connections 2024-09-21T04:07:51.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.572+0000 7f3694e44740 1 -- 172.21.15.64:0/2564082406 wait complete. 2024-09-21T04:07:51.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.572+0000 7f3694e44740 1 -- start start 2024-09-21T04:07:51.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.573+0000 7f3694e44740 1 Processor -- start 2024-09-21T04:07:51.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.573+0000 7f3694e44740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558806a461b0 0x558806e21ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:51.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.573+0000 7f3694e44740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558806a4e390 0x558806e22430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:51.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.574+0000 7f3694e44740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558806a6f070 0x558806e229b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:51.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.574+0000 7f3694e44740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558806bdd700 con 0x558806a4e390 2024-09-21T04:07:51.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.574+0000 7f3694e44740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x558806d17160 con 0x558806a6f070 2024-09-21T04:07:51.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.574+0000 7f3694e44740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x558806e360a0 con 0x558806a461b0 2024-09-21T04:07:51.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.574+0000 7f3694e44740 10 client.0 _ll_register_callbacks cb 0x558806d309e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:51.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.575+0000 7f36877fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558806a4e390 0x558806e22430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:51.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.575+0000 7f36877fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558806a4e390 0x558806e22430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38752/0 (socket says 172.21.15.64:38752) 2024-09-21T04:07:51.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.575+0000 7f36877fe640 1 -- 172.21.15.64:0/2070588227 learned_addr learned my addr 172.21.15.64:0/2070588227 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:51.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.575+0000 7f3687fff640 1 --2- 172.21.15.64:0/2070588227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558806a461b0 0x558806e21ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:51.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.576+0000 7f36877fe640 1 -- 172.21.15.64:0/2070588227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558806a461b0 msgr2=0x558806e21ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:51.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.576+0000 7f36877fe640 1 --2- 172.21.15.64:0/2070588227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558806a461b0 0x558806e21ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.576+0000 7f36877fe640 1 -- 172.21.15.64:0/2070588227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558806a6f070 msgr2=0x558806e229b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:51.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.576+0000 7f368ce6d640 1 --2- 172.21.15.64:0/2070588227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558806a6f070 0x558806e229b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:51.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.576+0000 7f36877fe640 1 --2- 172.21.15.64:0/2070588227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558806a6f070 0x558806e229b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.576+0000 7f36877fe640 1 -- 172.21.15.64:0/2070588227 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558806d44c90 con 0x558806a4e390 2024-09-21T04:07:51.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.576+0000 7f3687fff640 1 --2- 172.21.15.64:0/2070588227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558806a461b0 0x558806e21ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:07:51.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.576+0000 7f368ce6d640 1 --2- 172.21.15.64:0/2070588227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558806a6f070 0x558806e229b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:07:51.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.577+0000 7f36877fe640 1 --2- 172.21.15.64:0/2070588227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558806a4e390 0x558806e22430 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f3678015e70 tx=0x7f3678004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e1e2aa684b7f35 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:51.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.577+0000 7f36677fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:07:51.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.577+0000 7f36677fe640 1 -- 172.21.15.64:0/2070588227 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3678023060 con 0x558806a4e390 2024-09-21T04:07:51.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.577+0000 7f3694e44740 1 -- 172.21.15.64:0/2070588227 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f367c017a90 con 0x558806a4e390 2024-09-21T04:07:51.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.578+0000 7f36677fe640 1 -- 172.21.15.64:0/2070588227 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f36780150d0 con 0x558806a4e390 2024-09-21T04:07:51.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.578+0000 7f36677fe640 1 -- 172.21.15.64:0/2070588227 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3678015330 con 0x558806a4e390 2024-09-21T04:07:51.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.578+0000 7f3694e44740 10 client.16347 fetch_fsmap learned FSMap version 32 2024-09-21T04:07:51.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.578+0000 7f3694e44740 1 -- 172.21.15.64:0/2070588227 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558806e41e40 con 0x558806a4e390 2024-09-21T04:07:51.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.580+0000 7f36677fe640 1 -- 172.21.15.64:0/2070588227 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f367c017a90 con 0x558806a4e390 2024-09-21T04:07:51.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.580+0000 7f36677fe640 1 -- 172.21.15.64:0/2070588227 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f3678012070 con 0x558806a4e390 2024-09-21T04:07:51.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.580+0000 7f3694e44740 10 client.16347 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T04:07:51.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.580+0000 7f3694e44740 10 client.16347 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:07:51.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.580+0000 7f3694e44740 20 client.16347 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:51.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.580+0000 7f3694e44740 1 --2- 172.21.15.64:0/2070588227 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x558806a2d9f0 0x558806e457a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:51.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.580+0000 7f3694e44740 4 client.16347 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:07:51.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.580+0000 7f3694e44740 1 -- 172.21.15.64:0/2070588227 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558806d8edf0 con 0x558806a2d9f0 2024-09-21T04:07:51.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.581+0000 7f3687fff640 1 --2- 172.21.15.64:0/2070588227 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x558806a2d9f0 0x558806e457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:51.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.581+0000 7f36677fe640 10 client.16347 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:07:51.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.583+0000 7f3687fff640 1 --2- 172.21.15.64:0/2070588227 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x558806a2d9f0 0x558806e457a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ef59a3360e719742 server_cookie=ab3f4b6963e585d8 in_seq=0 out_seq=0 2024-09-21T04:07:51.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.584+0000 7f36677fe640 1 -- 172.21.15.64:0/2070588227 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558806d8edf0 con 0x558806a2d9f0 2024-09-21T04:07:51.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.584+0000 7f36677fe640 10 client.16347 handle_command_reply: tid=0 2024-09-21T04:07:51.588 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.584+0000 7f3694e44740 1 client.16347 shutdown 2024-09-21T04:07:51.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.587+0000 7f3694e44740 1 -- 172.21.15.64:0/2070588227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558806a4e390 msgr2=0x558806e22430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:51.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.587+0000 7f3694e44740 1 --2- 172.21.15.64:0/2070588227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558806a4e390 0x558806e22430 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f3678015e70 tx=0x7f3678004750 comp rx=0 tx=0).stop 2024-09-21T04:07:51.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.589+0000 7f3694e44740 1 -- 172.21.15.64:0/2070588227 shutdown_connections 2024-09-21T04:07:51.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.589+0000 7f3694e44740 1 --2- 172.21.15.64:0/2070588227 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x558806a2d9f0 0x558806e457a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.589+0000 7f3694e44740 1 --2- 172.21.15.64:0/2070588227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558806a6f070 0x558806e229b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.589+0000 7f3694e44740 1 --2- 172.21.15.64:0/2070588227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558806a4e390 0x558806e22430 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.589+0000 7f3694e44740 1 --2- 172.21.15.64:0/2070588227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558806a461b0 0x558806e21ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.589+0000 7f3694e44740 1 -- 172.21.15.64:0/2070588227 >> 172.21.15.64:0/2070588227 conn(0x558806a63460 msgr2=0x558806e19a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:51.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.589+0000 7f36677fe640 0 client.16347 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:07:51.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.591+0000 7f3694e44740 1 -- 172.21.15.64:0/2070588227 shutdown_connections 2024-09-21T04:07:51.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.591+0000 7f3694e44740 1 -- 172.21.15.64:0/2070588227 wait complete. 2024-09-21T04:07:51.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.591+0000 7f3694e44740 20 client.16347 trim_cache size 0 max 16384 2024-09-21T04:07:51.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.609+0000 7f3694e44740 1 Processor -- start 2024-09-21T04:07:51.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.609+0000 7f3694e44740 1 -- start start 2024-09-21T04:07:51.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.609+0000 7f3694e44740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558806a461b0 0x558806e223b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:51.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.609+0000 7f3694e44740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558806a4e390 0x558806e13d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:51.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.610+0000 7f3694e44740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558806a6f070 0x558806e14260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:51.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.610+0000 7f3694e44740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f36881b87b0 con 0x558806a6f070 2024-09-21T04:07:51.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.610+0000 7f3694e44740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f36881b5ff0 con 0x558806a461b0 2024-09-21T04:07:51.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.610+0000 7f3694e44740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f36881b3e40 con 0x558806a4e390 2024-09-21T04:07:51.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.610+0000 7f368ce6d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558806a6f070 0x558806e14260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:51.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.610+0000 7f3687fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558806a461b0 0x558806e223b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:51.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.610+0000 7f3687fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558806a461b0 0x558806e223b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:45116/0 (socket says 172.21.15.64:45116) 2024-09-21T04:07:51.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.610+0000 7f3687fff640 1 -- 172.21.15.64:0/228364152 learned_addr learned my addr 172.21.15.64:0/228364152 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:51.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.610+0000 7f368ce6d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558806a6f070 0x558806e14260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38760/0 (socket says 172.21.15.64:38760) 2024-09-21T04:07:51.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.610+0000 7f36877fe640 1 --2- 172.21.15.64:0/228364152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558806a4e390 0x558806e13d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:51.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.610+0000 7f3687fff640 1 -- 172.21.15.64:0/228364152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558806a4e390 msgr2=0x558806e13d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:51.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.610+0000 7f3687fff640 1 --2- 172.21.15.64:0/228364152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558806a4e390 0x558806e13d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.610+0000 7f3687fff640 1 -- 172.21.15.64:0/228364152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558806a6f070 msgr2=0x558806e14260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:51.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.610+0000 7f3687fff640 1 --2- 172.21.15.64:0/228364152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558806a6f070 0x558806e14260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.610+0000 7f3687fff640 1 -- 172.21.15.64:0/228364152 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558806e1d780 con 0x558806a461b0 2024-09-21T04:07:51.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.610+0000 7f3687fff640 1 --2- 172.21.15.64:0/228364152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558806a461b0 0x558806e223b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f367c016450 tx=0x7f367c016480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcd59e78a5db2cfd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:51.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.611+0000 7f366dffb640 1 -- 172.21.15.64:0/228364152 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c016b20 con 0x558806a461b0 2024-09-21T04:07:51.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.611+0000 7f366dffb640 1 -- 172.21.15.64:0/228364152 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f367c005570 con 0x558806a461b0 2024-09-21T04:07:51.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.611+0000 7f366dffb640 1 -- 172.21.15.64:0/228364152 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c003bb0 con 0x558806a461b0 2024-09-21T04:07:51.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.612+0000 7f3694e44740 1 -- 172.21.15.64:0/228364152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558806a461b0 msgr2=0x558806e223b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:51.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.612+0000 7f3694e44740 1 --2- 172.21.15.64:0/228364152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558806a461b0 0x558806e223b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f367c016450 tx=0x7f367c016480 comp rx=0 tx=0).stop 2024-09-21T04:07:51.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.612+0000 7f3694e44740 1 -- 172.21.15.64:0/228364152 shutdown_connections 2024-09-21T04:07:51.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.612+0000 7f3694e44740 1 --2- 172.21.15.64:0/228364152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558806a6f070 0x558806e14260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.612+0000 7f3694e44740 1 --2- 172.21.15.64:0/228364152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558806a4e390 0x558806e13d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.612+0000 7f3694e44740 1 --2- 172.21.15.64:0/228364152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558806a461b0 0x558806e223b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.612+0000 7f3694e44740 1 -- 172.21.15.64:0/228364152 >> 172.21.15.64:0/228364152 conn(0x558806a63460 msgr2=0x558806e21ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:51.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.612+0000 7f3694e44740 1 -- 172.21.15.64:0/228364152 shutdown_connections 2024-09-21T04:07:51.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.612+0000 7f3694e44740 1 -- 172.21.15.64:0/228364152 wait complete. 2024-09-21T04:07:51.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.612+0000 7f3694e44740 1 -- start start 2024-09-21T04:07:51.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.613+0000 7f3694e44740 1 Processor -- start 2024-09-21T04:07:51.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.613+0000 7f3694e44740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558806a461b0 0x558806e3ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:51.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.613+0000 7f3694e44740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558806a4e390 0x558806e36f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:51.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.613+0000 7f3694e44740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558806a6f070 0x558806e374d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:51.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.613+0000 7f3694e44740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f36881b44b0 con 0x558806a461b0 2024-09-21T04:07:51.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.613+0000 7f3694e44740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f36881b5920 con 0x558806a6f070 2024-09-21T04:07:51.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.613+0000 7f3694e44740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f36881b4b80 con 0x558806a4e390 2024-09-21T04:07:51.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.613+0000 7f3694e44740 10 client.0 _ll_register_callbacks cb 0x7f36780093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:51.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.613+0000 7f3687fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558806a461b0 0x558806e3ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:51.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.613+0000 7f3687fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558806a461b0 0x558806e3ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38770/0 (socket says 172.21.15.64:38770) 2024-09-21T04:07:51.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.613+0000 7f3687fff640 1 -- 172.21.15.64:0/3696198427 learned_addr learned my addr 172.21.15.64:0/3696198427 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:51.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.613+0000 7f36877fe640 1 --2- 172.21.15.64:0/3696198427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558806a4e390 0x558806e36f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:51.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.614+0000 7f36877fe640 1 -- 172.21.15.64:0/3696198427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558806a6f070 msgr2=0x558806e374d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T04:07:51.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.614+0000 7f36877fe640 1 --2- 172.21.15.64:0/3696198427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558806a6f070 0x558806e374d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.614+0000 7f36877fe640 1 -- 172.21.15.64:0/3696198427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558806a461b0 msgr2=0x558806e3ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:51.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.614+0000 7f36877fe640 1 --2- 172.21.15.64:0/3696198427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558806a461b0 0x558806e3ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.614+0000 7f36877fe640 1 -- 172.21.15.64:0/3696198427 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558806e37bc0 con 0x558806a4e390 2024-09-21T04:07:51.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.614+0000 7f36877fe640 1 --2- 172.21.15.64:0/3696198427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558806a4e390 0x558806e36f90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f367800b100 tx=0x7f3678013a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69187ca5e8deae08 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:51.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.614+0000 7f36677fe640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-21T04:07:51.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.614+0000 7f36677fe640 1 -- 172.21.15.64:0/3696198427 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367802d080 con 0x558806a4e390 2024-09-21T04:07:51.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.614+0000 7f36677fe640 1 -- 172.21.15.64:0/3696198427 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f3678004020 con 0x558806a4e390 2024-09-21T04:07:51.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.615+0000 7f3694e44740 1 -- 172.21.15.64:0/3696198427 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f367c005570 con 0x558806a4e390 2024-09-21T04:07:51.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.615+0000 7f36677fe640 1 -- 172.21.15.64:0/3696198427 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3678015180 con 0x558806a4e390 2024-09-21T04:07:51.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.615+0000 7f3694e44740 10 client.25948 fetch_fsmap learned FSMap version 32 2024-09-21T04:07:51.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.615+0000 7f3694e44740 1 -- 172.21.15.64:0/3696198427 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558806eb0000 con 0x558806a4e390 2024-09-21T04:07:51.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.615+0000 7f36677fe640 1 -- 172.21.15.64:0/3696198427 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f367c005570 con 0x558806a4e390 2024-09-21T04:07:51.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.615+0000 7f36677fe640 1 -- 172.21.15.64:0/3696198427 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f3678038070 con 0x558806a4e390 2024-09-21T04:07:51.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.615+0000 7f3694e44740 10 client.25948 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T04:07:51.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.615+0000 7f3694e44740 10 client.25948 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:07:51.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.615+0000 7f3694e44740 20 client.25948 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:51.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.615+0000 7f3694e44740 1 --2- 172.21.15.64:0/3696198427 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x558806a02da0 0x558806eb2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:51.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.615+0000 7f3694e44740 4 client.25948 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:07:51.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.615+0000 7f3694e44740 1 -- 172.21.15.64:0/3696198427 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558806d8a8a0 con 0x558806a02da0 2024-09-21T04:07:51.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.616+0000 7f3687fff640 1 --2- 172.21.15.64:0/3696198427 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x558806a02da0 0x558806eb2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:51.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.616+0000 7f36677fe640 10 client.25948 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:07:51.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.616+0000 7f3687fff640 1 --2- 172.21.15.64:0/3696198427 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x558806a02da0 0x558806eb2820 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9f29ac8f555bbec9 server_cookie=2c00362053c2f754 in_seq=0 out_seq=0 2024-09-21T04:07:51.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.617+0000 7f36677fe640 1 -- 172.21.15.64:0/3696198427 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558806d8a8a0 con 0x558806a02da0 2024-09-21T04:07:51.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.617+0000 7f36677fe640 10 client.25948 handle_command_reply: tid=0 2024-09-21T04:07:51.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.617+0000 7f3694e44740 1 client.25948 shutdown 2024-09-21T04:07:51.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.617+0000 7f3694e44740 1 -- 172.21.15.64:0/3696198427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558806a4e390 msgr2=0x558806e36f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:51.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.617+0000 7f3694e44740 1 --2- 172.21.15.64:0/3696198427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558806a4e390 0x558806e36f90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f367800b100 tx=0x7f3678013a80 comp rx=0 tx=0).stop 2024-09-21T04:07:51.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.617+0000 7f3694e44740 1 -- 172.21.15.64:0/3696198427 shutdown_connections 2024-09-21T04:07:51.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.617+0000 7f3694e44740 1 --2- 172.21.15.64:0/3696198427 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x558806a02da0 0x558806eb2820 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.617+0000 7f36677fe640 0 client.25948 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:07:51.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.617+0000 7f3694e44740 1 --2- 172.21.15.64:0/3696198427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x558806a6f070 0x558806e374d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.617+0000 7f3694e44740 1 --2- 172.21.15.64:0/3696198427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558806a4e390 0x558806e36f90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.617+0000 7f3694e44740 1 --2- 172.21.15.64:0/3696198427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558806a461b0 0x558806e3ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.617+0000 7f3694e44740 1 -- 172.21.15.64:0/3696198427 >> 172.21.15.64:0/3696198427 conn(0x558806a63460 msgr2=0x558806e3e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:51.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.617+0000 7f3694e44740 1 -- 172.21.15.64:0/3696198427 shutdown_connections 2024-09-21T04:07:51.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.617+0000 7f3694e44740 1 -- 172.21.15.64:0/3696198427 wait complete. 2024-09-21T04:07:51.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.618+0000 7f3694e44740 20 client.25948 trim_cache size 0 max 16384 2024-09-21T04:07:51.623 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:07:51.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.618+0000 7f368d66e640 1 -- 172.21.15.64:0/3880244947 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f36700b6780 msgr2=0x7f36700b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:51.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.618+0000 7f368d66e640 1 --2- 172.21.15.64:0/3880244947 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f36700b6780 0x7f36700b8c90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x558806e154e0 tx=0x7f3678006c40 comp rx=0 tx=0).stop 2024-09-21T04:07:51.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.619+0000 7f368d66e640 1 -- 172.21.15.64:0/3880244947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3688110640 msgr2=0x7f36881a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:51.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.619+0000 7f368d66e640 1 --2- 172.21.15.64:0/3880244947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3688110640 0x7f36881a60f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3680033380 tx=0x7f368003b5a0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.619+0000 7f368d66e640 1 -- 172.21.15.64:0/3880244947 shutdown_connections 2024-09-21T04:07:51.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.619+0000 7f368d66e640 1 --2- 172.21.15.64:0/3880244947 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f36700b6780 0x7f36700b8c90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.619+0000 7f368d66e640 1 --2- 172.21.15.64:0/3880244947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3688110640 0x7f36881a60f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.619+0000 7f368d66e640 1 --2- 172.21.15.64:0/3880244947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f368810fcf0 0x7f36881a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.619+0000 7f368d66e640 1 --2- 172.21.15.64:0/3880244947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f368810a650 0x7f36881a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:51.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.619+0000 7f368d66e640 1 -- 172.21.15.64:0/3880244947 >> 172.21.15.64:0/3880244947 conn(0x7f368806ee50 msgr2=0x7f368810dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:51.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.619+0000 7f368d66e640 1 -- 172.21.15.64:0/3880244947 shutdown_connections 2024-09-21T04:07:51.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:51.619+0000 7f368d66e640 1 -- 172.21.15.64:0/3880244947 wait complete. 2024-09-21T04:07:52.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:51 smithi119 ceph-mon[34811]: pgmap v436: 97 pgs: 97 active+clean; 518 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 319 op/s 2024-09-21T04:07:52.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:51 smithi064 ceph-mon[31180]: pgmap v436: 97 pgs: 97 active+clean; 518 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 319 op/s 2024-09-21T04:07:52.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:51 smithi131 ceph-mon[35193]: pgmap v436: 97 pgs: 97 active+clean; 518 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 319 op/s 2024-09-21T04:07:52.445 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T04:07:52.772 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:07:53.323 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.320+0000 7f0622e22640 1 -- 172.21.15.64:0/2065618603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f061c105db0 msgr2=0x7f061c1061c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:53.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.320+0000 7f0622e22640 1 --2- 172.21.15.64:0/2065618603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f061c105db0 0x7f061c1061c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0614009900 tx=0x7f061402a410 comp rx=0 tx=0).stop 2024-09-21T04:07:53.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.321+0000 7f0622e22640 1 -- 172.21.15.64:0/2065618603 shutdown_connections 2024-09-21T04:07:53.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.321+0000 7f0622e22640 1 --2- 172.21.15.64:0/2065618603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f061c10c040 0x7f061c112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.321+0000 7f0622e22640 1 --2- 172.21.15.64:0/2065618603 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f061c10b6f0 0x7f061c10bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.321+0000 7f0622e22640 1 --2- 172.21.15.64:0/2065618603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f061c105db0 0x7f061c1061c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.321+0000 7f0622e22640 1 -- 172.21.15.64:0/2065618603 >> 172.21.15.64:0/2065618603 conn(0x7f061c1017e0 msgr2=0x7f061c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:53.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.321+0000 7f0622e22640 1 -- 172.21.15.64:0/2065618603 shutdown_connections 2024-09-21T04:07:53.324 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.321+0000 7f0622e22640 1 -- 172.21.15.64:0/2065618603 wait complete. 2024-09-21T04:07:53.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.322+0000 7f0622e22640 1 Processor -- start 2024-09-21T04:07:53.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.322+0000 7f0622e22640 1 -- start start 2024-09-21T04:07:53.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.322+0000 7f0622e22640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f061c105db0 0x7f061c19c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:53.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.322+0000 7f0622e22640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f061c10b6f0 0x7f061c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:53.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.322+0000 7f0622e22640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f061c10c040 0x7f061c1a1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:53.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.322+0000 7f0622e22640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f061c115d50 con 0x7f061c10b6f0 2024-09-21T04:07:53.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.322+0000 7f0622e22640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f061c115bd0 con 0x7f061c10c040 2024-09-21T04:07:53.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.322+0000 7f0622e22640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f061c115ed0 con 0x7f061c105db0 2024-09-21T04:07:53.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.322+0000 7f061bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f061c10b6f0 0x7f061c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:53.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.322+0000 7f0620bbe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f061c105db0 0x7f061c19c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:53.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.322+0000 7f061bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f061c10b6f0 0x7f061c19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38790/0 (socket says 172.21.15.64:38790) 2024-09-21T04:07:53.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.322+0000 7f0620bbe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f061c105db0 0x7f061c19c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:38976/0 (socket says 172.21.15.64:38976) 2024-09-21T04:07:53.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.322+0000 7f061bfff640 1 -- 172.21.15.64:0/3833373133 learned_addr learned my addr 172.21.15.64:0/3833373133 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:53.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.322+0000 7f06213bf640 1 --2- 172.21.15.64:0/3833373133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f061c10c040 0x7f061c1a1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:53.326 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.323+0000 7f0620bbe640 1 -- 172.21.15.64:0/3833373133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f061c10c040 msgr2=0x7f061c1a1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:53.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.323+0000 7f0620bbe640 1 --2- 172.21.15.64:0/3833373133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f061c10c040 0x7f061c1a1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.323+0000 7f0620bbe640 1 -- 172.21.15.64:0/3833373133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f061c10b6f0 msgr2=0x7f061c19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:53.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.323+0000 7f0620bbe640 1 --2- 172.21.15.64:0/3833373133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f061c10b6f0 0x7f061c19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.323+0000 7f0620bbe640 1 -- 172.21.15.64:0/3833373133 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f061c1a26f0 con 0x7f061c105db0 2024-09-21T04:07:53.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.323+0000 7f061bfff640 1 --2- 172.21.15.64:0/3833373133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f061c10b6f0 0x7f061c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:07:53.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.323+0000 7f0620bbe640 1 --2- 172.21.15.64:0/3833373133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f061c105db0 0x7f061c19c780 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f061402aa00 tx=0x7f0614003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:53.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.323+0000 7f0619ffb640 1 -- 172.21.15.64:0/3833373133 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061403f070 con 0x7f061c105db0 2024-09-21T04:07:53.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.323+0000 7f0619ffb640 1 -- 172.21.15.64:0/3833373133 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0614031b50 con 0x7f061c105db0 2024-09-21T04:07:53.327 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.323+0000 7f0619ffb640 1 -- 172.21.15.64:0/3833373133 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0614035030 con 0x7f061c105db0 2024-09-21T04:07:53.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.323+0000 7f0622e22640 1 -- 172.21.15.64:0/3833373133 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f061c1a2980 con 0x7f061c105db0 2024-09-21T04:07:53.328 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.324+0000 7f0622e22640 1 -- 172.21.15.64:0/3833373133 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f061c1a2de0 con 0x7f061c105db0 2024-09-21T04:07:53.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.326+0000 7f0629396740 1 Processor -- start 2024-09-21T04:07:53.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.326+0000 7f0629396740 1 -- start start 2024-09-21T04:07:53.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.327+0000 7f0629396740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564fdfee070 0x5564fe3956a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:53.329 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.327+0000 7f0629396740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564fdfc51b0 0x5564fe395be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:53.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.327+0000 7f0629396740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564fdfcd390 0x5564fe396120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:53.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.327+0000 7f0629396740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5564fe30bea0 con 0x5564fdfcd390 2024-09-21T04:07:53.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.327+0000 7f0629396740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5564fe2fb2d0 con 0x5564fdfee070 2024-09-21T04:07:53.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.327+0000 7f0629396740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5564fe2ff0e0 con 0x5564fdfc51b0 2024-09-21T04:07:53.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.327+0000 7f06213bf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564fdfee070 0x5564fe3956a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:53.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.327+0000 7f061bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564fdfcd390 0x5564fe396120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:53.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.327+0000 7f061bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564fdfcd390 0x5564fe396120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38804/0 (socket says 172.21.15.64:38804) 2024-09-21T04:07:53.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.327+0000 7f061bfff640 1 -- 172.21.15.64:0/1718101066 learned_addr learned my addr 172.21.15.64:0/1718101066 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:53.330 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.327+0000 7f06213bf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564fdfee070 0x5564fe3956a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:45144/0 (socket says 172.21.15.64:45144) 2024-09-21T04:07:53.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.327+0000 7f061bfff640 1 -- 172.21.15.64:0/1718101066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564fdfc51b0 msgr2=0x5564fe395be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:07:53.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.327+0000 7f061bfff640 1 --2- 172.21.15.64:0/1718101066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564fdfc51b0 0x5564fe395be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.327+0000 7f061bfff640 1 -- 172.21.15.64:0/1718101066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564fdfee070 msgr2=0x5564fe3956a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:53.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.327+0000 7f061bfff640 1 --2- 172.21.15.64:0/1718101066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564fdfee070 0x5564fe3956a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.327+0000 7f061bfff640 1 -- 172.21.15.64:0/1718101066 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564fe059650 con 0x5564fdfcd390 2024-09-21T04:07:53.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.328+0000 7f061bfff640 1 --2- 172.21.15.64:0/1718101066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564fdfcd390 0x5564fe396120 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f0604011540 tx=0x7f0604011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09b7dacfe448cf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:53.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.328+0000 7f0619ffb640 1 -- 172.21.15.64:0/3833373133 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f0614042070 con 0x7f061c105db0 2024-09-21T04:07:53.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.328+0000 7f0612ffd640 1 -- 172.21.15.64:0/1718101066 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060401e070 con 0x5564fdfcd390 2024-09-21T04:07:53.331 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.328+0000 7f0612ffd640 1 -- 172.21.15.64:0/1718101066 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f060400fa70 con 0x5564fdfcd390 2024-09-21T04:07:53.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.328+0000 7f0612ffd640 1 -- 172.21.15.64:0/1718101066 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0604019b60 con 0x5564fdfcd390 2024-09-21T04:07:53.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.329+0000 7f0619ffb640 1 --2- 172.21.15.64:0/3833373133 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f06080b6700 0x7f06080b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:53.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.329+0000 7f0629396740 1 -- 172.21.15.64:0/1718101066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564fdfcd390 msgr2=0x5564fe396120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:53.332 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.329+0000 7f0629396740 1 --2- 172.21.15.64:0/1718101066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564fdfcd390 0x5564fe396120 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f0604011540 tx=0x7f0604011940 comp rx=0 tx=0).stop 2024-09-21T04:07:53.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.329+0000 7f0619ffb640 1 -- 172.21.15.64:0/3833373133 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f0614107c20 con 0x7f061c105db0 2024-09-21T04:07:53.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.329+0000 7f06213bf640 1 --2- 172.21.15.64:0/3833373133 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f06080b6700 0x7f06080b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:53.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.330+0000 7f0629396740 1 -- 172.21.15.64:0/1718101066 shutdown_connections 2024-09-21T04:07:53.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.330+0000 7f0629396740 1 --2- 172.21.15.64:0/1718101066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564fdfcd390 0x5564fe396120 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.330+0000 7f0629396740 1 --2- 172.21.15.64:0/1718101066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564fdfc51b0 0x5564fe395be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.330+0000 7f0629396740 1 --2- 172.21.15.64:0/1718101066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564fdfee070 0x5564fe3956a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.333 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.330+0000 7f0629396740 1 -- 172.21.15.64:0/1718101066 >> 172.21.15.64:0/1718101066 conn(0x5564fdfe2460 msgr2=0x5564fe393590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:53.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.330+0000 7f0629396740 1 -- 172.21.15.64:0/1718101066 shutdown_connections 2024-09-21T04:07:53.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.330+0000 7f06213bf640 1 --2- 172.21.15.64:0/3833373133 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f06080b6700 0x7f06080b8c10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f060c005fd0 tx=0x7f060c009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:53.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.330+0000 7f0629396740 1 -- 172.21.15.64:0/1718101066 wait complete. 2024-09-21T04:07:53.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.331+0000 7f0629396740 1 -- start start 2024-09-21T04:07:53.334 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.331+0000 7f0629396740 1 Processor -- start 2024-09-21T04:07:53.336 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.333+0000 7f0629396740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564fdfc51b0 0x5564fe3a9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:53.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.333+0000 7f0629396740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564fdfcd390 0x5564fe3aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:53.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.333+0000 7f0629396740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564fdfee070 0x5564fe3af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:53.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.333+0000 7f0629396740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5564fe15c700 con 0x5564fdfcd390 2024-09-21T04:07:53.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.333+0000 7f0629396740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5564fe296160 con 0x5564fdfee070 2024-09-21T04:07:53.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.333+0000 7f0629396740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5564fe3a3e80 con 0x5564fdfc51b0 2024-09-21T04:07:53.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.333+0000 7f0629396740 10 client.0 _ll_register_callbacks cb 0x5564fe2af9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:53.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.334+0000 7f061bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564fdfcd390 0x5564fe3aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:53.337 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.334+0000 7f061bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564fdfcd390 0x5564fe3aa4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38818/0 (socket says 172.21.15.64:38818) 2024-09-21T04:07:53.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.334+0000 7f061bfff640 1 -- 172.21.15.64:0/1536934423 learned_addr learned my addr 172.21.15.64:0/1536934423 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:53.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.334+0000 7f061bfff640 1 -- 172.21.15.64:0/1536934423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564fdfc51b0 msgr2=0x5564fe3a9ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T04:07:53.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.334+0000 7f061bfff640 1 --2- 172.21.15.64:0/1536934423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564fdfc51b0 0x5564fe3a9ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.334+0000 7f061bfff640 1 -- 172.21.15.64:0/1536934423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564fdfee070 msgr2=0x5564fe3af4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T04:07:53.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.334+0000 7f061bfff640 1 --2- 172.21.15.64:0/1536934423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564fdfee070 0x5564fe3af4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.334+0000 7f061bfff640 1 -- 172.21.15.64:0/1536934423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564fe2c3c90 con 0x5564fdfcd390 2024-09-21T04:07:53.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.334+0000 7f061bfff640 1 --2- 172.21.15.64:0/1536934423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564fdfcd390 0x5564fe3aa4b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0604011510 tx=0x7f060400f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d3898ffa8cd6cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:53.338 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.334+0000 7f05fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:07:53.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.335+0000 7f05fbfff640 1 -- 172.21.15.64:0/1536934423 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0604028060 con 0x5564fdfcd390 2024-09-21T04:07:53.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.335+0000 7f05fbfff640 1 -- 172.21.15.64:0/1536934423 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f0604019920 con 0x5564fdfcd390 2024-09-21T04:07:53.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.335+0000 7f0629396740 1 -- 172.21.15.64:0/1536934423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f060400fa70 con 0x5564fdfcd390 2024-09-21T04:07:53.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.336+0000 7f05fbfff640 1 -- 172.21.15.64:0/1536934423 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f060401e070 con 0x5564fdfcd390 2024-09-21T04:07:53.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.336+0000 7f0629396740 10 client.16365 fetch_fsmap learned FSMap version 32 2024-09-21T04:07:53.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.336+0000 7f0629396740 1 -- 172.21.15.64:0/1536934423 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5564fe3b8420 con 0x5564fdfcd390 2024-09-21T04:07:53.339 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.336+0000 7f05fbfff640 1 -- 172.21.15.64:0/1536934423 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f060400fa70 con 0x5564fdfcd390 2024-09-21T04:07:53.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.337+0000 7f05fbfff640 1 -- 172.21.15.64:0/1536934423 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f06040046c0 con 0x5564fdfcd390 2024-09-21T04:07:53.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.337+0000 7f0629396740 10 client.16365 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T04:07:53.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.337+0000 7f0629396740 10 client.16365 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:07:53.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.337+0000 7f0629396740 20 client.16365 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:53.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.337+0000 7f0629396740 1 --2- 172.21.15.64:0/1536934423 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5564fdfac9f0 0x5564fe3bb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:53.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.337+0000 7f0629396740 4 client.16365 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:07:53.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.337+0000 7f0629396740 1 -- 172.21.15.64:0/1536934423 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564fe30ddf0 con 0x5564fdfac9f0 2024-09-21T04:07:53.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.337+0000 7f0620bbe640 1 --2- 172.21.15.64:0/1536934423 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5564fdfac9f0 0x5564fe3bb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:53.340 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.338+0000 7f05fbfff640 10 client.16365 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:07:53.341 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.339+0000 7f0620bbe640 1 --2- 172.21.15.64:0/1536934423 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5564fdfac9f0 0x5564fe3bb7e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b79764a78a5a50d server_cookie=b65b2c39ddff3f02 in_seq=0 out_seq=0 2024-09-21T04:07:53.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.343+0000 7f05fbfff640 1 -- 172.21.15.64:0/1536934423 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5564fe30ddf0 con 0x5564fdfac9f0 2024-09-21T04:07:53.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.343+0000 7f05fbfff640 10 client.16365 handle_command_reply: tid=0 2024-09-21T04:07:53.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.344+0000 7f0629396740 1 client.16365 shutdown 2024-09-21T04:07:53.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.344+0000 7f0629396740 1 -- 172.21.15.64:0/1536934423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564fdfcd390 msgr2=0x5564fe3aa4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:53.347 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.344+0000 7f0629396740 1 --2- 172.21.15.64:0/1536934423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564fdfcd390 0x5564fe3aa4b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0604011510 tx=0x7f060400f5c0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.345+0000 7f0629396740 1 -- 172.21.15.64:0/1536934423 shutdown_connections 2024-09-21T04:07:53.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.345+0000 7f0629396740 1 --2- 172.21.15.64:0/1536934423 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5564fdfac9f0 0x5564fe3bb7e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.345+0000 7f0629396740 1 --2- 172.21.15.64:0/1536934423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564fdfee070 0x5564fe3af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.345+0000 7f0629396740 1 --2- 172.21.15.64:0/1536934423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564fdfcd390 0x5564fe3aa4b0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.345+0000 7f0629396740 1 --2- 172.21.15.64:0/1536934423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564fdfc51b0 0x5564fe3a9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.345+0000 7f0629396740 1 -- 172.21.15.64:0/1536934423 >> 172.21.15.64:0/1536934423 conn(0x5564fdfe2460 msgr2=0x5564fe398830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:53.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.345+0000 7f05fbfff640 0 client.16365 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:07:53.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.345+0000 7f0629396740 1 -- 172.21.15.64:0/1536934423 shutdown_connections 2024-09-21T04:07:53.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.346+0000 7f0629396740 1 -- 172.21.15.64:0/1536934423 wait complete. 2024-09-21T04:07:53.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.346+0000 7f0629396740 20 client.16365 trim_cache size 0 max 16384 2024-09-21T04:07:53.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.367+0000 7f0629396740 1 Processor -- start 2024-09-21T04:07:53.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.367+0000 7f0629396740 1 -- start start 2024-09-21T04:07:53.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.367+0000 7f0629396740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564fdfc51b0 0x5564fe3af340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:53.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.367+0000 7f0629396740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564fdfcd390 0x5564fe3a31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:53.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.367+0000 7f0629396740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564fdfee070 0x5564fe392eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:53.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.367+0000 7f0629396740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f061c1b0020 con 0x5564fdfee070 2024-09-21T04:07:53.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.367+0000 7f0629396740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f061c1ad860 con 0x5564fdfcd390 2024-09-21T04:07:53.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.367+0000 7f0629396740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f061c1ab6b0 con 0x5564fdfc51b0 2024-09-21T04:07:53.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.367+0000 7f06213bf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564fdfee070 0x5564fe392eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:53.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.368+0000 7f06213bf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564fdfee070 0x5564fe392eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38830/0 (socket says 172.21.15.64:38830) 2024-09-21T04:07:53.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.368+0000 7f06213bf640 1 -- 172.21.15.64:0/3423765903 learned_addr learned my addr 172.21.15.64:0/3423765903 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:53.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.368+0000 7f061bfff640 1 --2- 172.21.15.64:0/3423765903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564fdfcd390 0x5564fe3a31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:53.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.368+0000 7f0620bbe640 1 --2- 172.21.15.64:0/3423765903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564fdfc51b0 0x5564fe3af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:53.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.368+0000 7f06213bf640 1 -- 172.21.15.64:0/3423765903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564fdfc51b0 msgr2=0x5564fe3af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:53.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.368+0000 7f06213bf640 1 --2- 172.21.15.64:0/3423765903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564fdfc51b0 0x5564fe3af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.368+0000 7f06213bf640 1 -- 172.21.15.64:0/3423765903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564fdfcd390 msgr2=0x5564fe3a31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:53.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.368+0000 7f06213bf640 1 --2- 172.21.15.64:0/3423765903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564fdfcd390 0x5564fe3a31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.368+0000 7f06213bf640 1 -- 172.21.15.64:0/3423765903 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564fe3aaf90 con 0x5564fdfee070 2024-09-21T04:07:53.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.368+0000 7f0620bbe640 1 --2- 172.21.15.64:0/3423765903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564fdfc51b0 0x5564fe3af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:07:53.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.368+0000 7f06213bf640 1 --2- 172.21.15.64:0/3423765903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564fdfee070 0x5564fe392eb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f060c003d40 tx=0x7f060c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed020eb99024f35e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:53.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.368+0000 7f06127fc640 1 -- 172.21.15.64:0/3423765903 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060c003e30 con 0x5564fdfee070 2024-09-21T04:07:53.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.368+0000 7f06127fc640 1 -- 172.21.15.64:0/3423765903 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f060c00f930 con 0x5564fdfee070 2024-09-21T04:07:53.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.368+0000 7f06127fc640 1 -- 172.21.15.64:0/3423765903 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060c01ea60 con 0x5564fdfee070 2024-09-21T04:07:53.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.369+0000 7f0629396740 1 -- 172.21.15.64:0/3423765903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564fdfee070 msgr2=0x5564fe392eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:53.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.369+0000 7f0629396740 1 --2- 172.21.15.64:0/3423765903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564fdfee070 0x5564fe392eb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f060c003d40 tx=0x7f060c016000 comp rx=0 tx=0).stop 2024-09-21T04:07:53.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.369+0000 7f0629396740 1 -- 172.21.15.64:0/3423765903 shutdown_connections 2024-09-21T04:07:53.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.369+0000 7f0629396740 1 --2- 172.21.15.64:0/3423765903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564fdfee070 0x5564fe392eb0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.370+0000 7f0629396740 1 --2- 172.21.15.64:0/3423765903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564fdfcd390 0x5564fe3a31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.370+0000 7f0629396740 1 --2- 172.21.15.64:0/3423765903 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564fdfc51b0 0x5564fe3af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.370+0000 7f0629396740 1 -- 172.21.15.64:0/3423765903 >> 172.21.15.64:0/3423765903 conn(0x5564fdfe2460 msgr2=0x5564fe3a9ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:53.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.370+0000 7f0629396740 1 -- 172.21.15.64:0/3423765903 shutdown_connections 2024-09-21T04:07:53.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.370+0000 7f0629396740 1 -- 172.21.15.64:0/3423765903 wait complete. 2024-09-21T04:07:53.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.370+0000 7f0629396740 1 -- start start 2024-09-21T04:07:53.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.370+0000 7f0629396740 1 Processor -- start 2024-09-21T04:07:53.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.371+0000 7f0629396740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564fdfc51b0 0x5564fe3b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:53.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.371+0000 7f0629396740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564fdfcd390 0x5564fe3b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:53.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.371+0000 7f0629396740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564fdfee070 0x5564fe3b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:53.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.371+0000 7f0629396740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f061c1abd20 con 0x5564fdfee070 2024-09-21T04:07:53.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.371+0000 7f0629396740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f061c1ad190 con 0x5564fdfc51b0 2024-09-21T04:07:53.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.371+0000 7f0629396740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f061c1ac3f0 con 0x5564fdfcd390 2024-09-21T04:07:53.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.371+0000 7f0620bbe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564fdfc51b0 0x5564fe3b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:53.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.371+0000 7f0629396740 10 client.0 _ll_register_callbacks cb 0x7f060401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:53.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.371+0000 7f0620bbe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564fdfc51b0 0x5564fe3b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:45166/0 (socket says 172.21.15.64:45166) 2024-09-21T04:07:53.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.371+0000 7f0620bbe640 1 -- 172.21.15.64:0/2506110690 learned_addr learned my addr 172.21.15.64:0/2506110690 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:53.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.371+0000 7f06213bf640 1 --2- 172.21.15.64:0/2506110690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564fdfee070 0x5564fe3b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:53.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.371+0000 7f061bfff640 1 --2- 172.21.15.64:0/2506110690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564fdfcd390 0x5564fe3b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:53.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.371+0000 7f0620bbe640 1 -- 172.21.15.64:0/2506110690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564fdfcd390 msgr2=0x5564fe3b09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:53.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.371+0000 7f0620bbe640 1 --2- 172.21.15.64:0/2506110690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564fdfcd390 0x5564fe3b09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.371+0000 7f0620bbe640 1 -- 172.21.15.64:0/2506110690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564fdfee070 msgr2=0x5564fe3b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:53.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.371+0000 7f0620bbe640 1 --2- 172.21.15.64:0/2506110690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564fdfee070 0x5564fe3b0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.371+0000 7f0620bbe640 1 -- 172.21.15.64:0/2506110690 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564fe3b5500 con 0x5564fdfc51b0 2024-09-21T04:07:53.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.371+0000 7f06213bf640 1 --2- 172.21.15.64:0/2506110690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564fdfee070 0x5564fe3b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:07:53.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.371+0000 7f0620bbe640 1 --2- 172.21.15.64:0/2506110690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564fdfc51b0 0x5564fe3b04b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f06140cda80 tx=0x7f06140c9ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be648170774ccc81 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:53.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.371+0000 7f05fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:07:53.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.372+0000 7f05fbfff640 1 -- 172.21.15.64:0/2506110690 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0614049030 con 0x5564fdfc51b0 2024-09-21T04:07:53.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.372+0000 7f05fbfff640 1 -- 172.21.15.64:0/2506110690 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f06140d2100 con 0x5564fdfc51b0 2024-09-21T04:07:53.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.372+0000 7f0629396740 1 -- 172.21.15.64:0/2506110690 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f060c00f930 con 0x5564fdfc51b0 2024-09-21T04:07:53.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.372+0000 7f05fbfff640 1 -- 172.21.15.64:0/2506110690 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f06140d2300 con 0x5564fdfc51b0 2024-09-21T04:07:53.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.372+0000 7f0629396740 10 client.26066 fetch_fsmap learned FSMap version 32 2024-09-21T04:07:53.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.372+0000 7f0629396740 1 -- 172.21.15.64:0/2506110690 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5564fe426290 con 0x5564fdfc51b0 2024-09-21T04:07:53.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.372+0000 7f05fbfff640 1 -- 172.21.15.64:0/2506110690 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f060c00f930 con 0x5564fdfc51b0 2024-09-21T04:07:53.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.372+0000 7f05fbfff640 1 -- 172.21.15.64:0/2506110690 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f06140d6fd0 con 0x5564fdfc51b0 2024-09-21T04:07:53.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.372+0000 7f0629396740 10 client.26066 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T04:07:53.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.372+0000 7f0629396740 10 client.26066 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:07:53.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.372+0000 7f0629396740 20 client.26066 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:53.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.372+0000 7f0629396740 1 --2- 172.21.15.64:0/2506110690 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5564fdf81da0 0x5564fe428b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:53.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.372+0000 7f0629396740 4 client.26066 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:07:53.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.372+0000 7f0629396740 1 -- 172.21.15.64:0/2506110690 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5564fe3098a0 con 0x5564fdf81da0 2024-09-21T04:07:53.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.372+0000 7f061bfff640 1 --2- 172.21.15.64:0/2506110690 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5564fdf81da0 0x5564fe428b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:53.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.373+0000 7f061bfff640 1 --2- 172.21.15.64:0/2506110690 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5564fdf81da0 0x5564fe428b00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c3e46e5884a5aa6 server_cookie=3f1e3654b489f79d in_seq=0 out_seq=0 2024-09-21T04:07:53.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.373+0000 7f05fbfff640 10 client.26066 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:07:53.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.374+0000 7f05fbfff640 1 -- 172.21.15.64:0/2506110690 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5564fe3098a0 con 0x5564fdf81da0 2024-09-21T04:07:53.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.374+0000 7f05fbfff640 10 client.26066 handle_command_reply: tid=0 2024-09-21T04:07:53.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.374+0000 7f0629396740 1 client.26066 shutdown 2024-09-21T04:07:53.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.374+0000 7f0629396740 1 -- 172.21.15.64:0/2506110690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564fdfc51b0 msgr2=0x5564fe3b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:53.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.375+0000 7f0629396740 1 --2- 172.21.15.64:0/2506110690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564fdfc51b0 0x5564fe3b04b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f06140cda80 tx=0x7f06140c9ac0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.375+0000 7f0629396740 1 -- 172.21.15.64:0/2506110690 shutdown_connections 2024-09-21T04:07:53.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.375+0000 7f0629396740 1 --2- 172.21.15.64:0/2506110690 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5564fdf81da0 0x5564fe428b00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.375+0000 7f0629396740 1 --2- 172.21.15.64:0/2506110690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564fdfee070 0x5564fe3b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.375+0000 7f0629396740 1 --2- 172.21.15.64:0/2506110690 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5564fdfcd390 0x5564fe3b09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.375+0000 7f0629396740 1 --2- 172.21.15.64:0/2506110690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5564fdfc51b0 0x5564fe3b04b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.375+0000 7f05fbfff640 0 client.26066 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:07:53.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.375+0000 7f0629396740 1 -- 172.21.15.64:0/2506110690 >> 172.21.15.64:0/2506110690 conn(0x5564fdfe2460 msgr2=0x5564fe392e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:53.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.375+0000 7f0629396740 1 -- 172.21.15.64:0/2506110690 shutdown_connections 2024-09-21T04:07:53.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.375+0000 7f0629396740 1 -- 172.21.15.64:0/2506110690 wait complete. 2024-09-21T04:07:53.381 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.375+0000 7f0629396740 20 client.26066 trim_cache size 0 max 16384 2024-09-21T04:07:53.382 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:07:53.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.376+0000 7f0622e22640 1 -- 172.21.15.64:0/3833373133 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f06080b6700 msgr2=0x7f06080b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:53.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.376+0000 7f0622e22640 1 --2- 172.21.15.64:0/3833373133 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f06080b6700 0x7f06080b8c10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f060c005fd0 tx=0x7f060c009500 comp rx=0 tx=0).stop 2024-09-21T04:07:53.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.376+0000 7f0622e22640 1 -- 172.21.15.64:0/3833373133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f061c105db0 msgr2=0x7f061c19c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:53.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.376+0000 7f0622e22640 1 --2- 172.21.15.64:0/3833373133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f061c105db0 0x7f061c19c780 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f061402aa00 tx=0x7f0614003b90 comp rx=0 tx=0).stop 2024-09-21T04:07:53.382 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.376+0000 7f0622e22640 1 -- 172.21.15.64:0/3833373133 shutdown_connections 2024-09-21T04:07:53.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.376+0000 7f0622e22640 1 --2- 172.21.15.64:0/3833373133 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f06080b6700 0x7f06080b8c10 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.376+0000 7f0622e22640 1 --2- 172.21.15.64:0/3833373133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f061c10c040 0x7f061c1a1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.376+0000 7f0622e22640 1 --2- 172.21.15.64:0/3833373133 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f061c10b6f0 0x7f061c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.376+0000 7f0622e22640 1 --2- 172.21.15.64:0/3833373133 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f061c105db0 0x7f061c19c780 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:53.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.376+0000 7f0622e22640 1 -- 172.21.15.64:0/3833373133 >> 172.21.15.64:0/3833373133 conn(0x7f061c1017e0 msgr2=0x7f061c10ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:53.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.376+0000 7f0622e22640 1 -- 172.21.15.64:0/3833373133 shutdown_connections 2024-09-21T04:07:53.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:53.376+0000 7f0622e22640 1 -- 172.21.15.64:0/3833373133 wait complete. 2024-09-21T04:07:54.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:53 smithi119 ceph-mon[34811]: pgmap v437: 97 pgs: 97 active+clean; 533 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.5 MiB/s wr, 501 op/s 2024-09-21T04:07:54.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:53 smithi064 ceph-mon[31180]: pgmap v437: 97 pgs: 97 active+clean; 533 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.5 MiB/s wr, 501 op/s 2024-09-21T04:07:54.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:53 smithi131 ceph-mon[35193]: pgmap v437: 97 pgs: 97 active+clean; 533 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.5 MiB/s wr, 501 op/s 2024-09-21T04:07:54.214 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T04:07:54.490 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:07:55.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.579+0000 7f552ac85640 1 -- 172.21.15.64:0/2709399814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f551c0aaca0 msgr2=0x7f551c0ab0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:55.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.579+0000 7f552ac85640 1 --2- 172.21.15.64:0/2709399814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f551c0aaca0 0x7f551c0ab0b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f55240524b0 tx=0x7f552408f850 comp rx=0 tx=0).stop 2024-09-21T04:07:55.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.582+0000 7f552ac85640 1 -- 172.21.15.64:0/2709399814 shutdown_connections 2024-09-21T04:07:55.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.582+0000 7f552ac85640 1 --2- 172.21.15.64:0/2709399814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f551c0ab5f0 0x7f551c0b1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.582+0000 7f552ac85640 1 --2- 172.21.15.64:0/2709399814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f551c0aaca0 0x7f551c0ab0b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.582+0000 7f552ac85640 1 --2- 172.21.15.64:0/2709399814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f551c0a5360 0x7f551c0a5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.582+0000 7f552ac85640 1 -- 172.21.15.64:0/2709399814 >> 172.21.15.64:0/2709399814 conn(0x7f551c01d090 msgr2=0x7f551c0a39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:55.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.587+0000 7f552ac85640 1 -- 172.21.15.64:0/2709399814 shutdown_connections 2024-09-21T04:07:55.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.590+0000 7f552ac85640 1 -- 172.21.15.64:0/2709399814 wait complete. 2024-09-21T04:07:55.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.591+0000 7f552ac85640 1 Processor -- start 2024-09-21T04:07:55.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.595+0000 7f552ac85640 1 -- start start 2024-09-21T04:07:55.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.596+0000 7f552ac85640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f551c0a5360 0x7f551c144010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:55.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.596+0000 7f552ac85640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f551c0aaca0 0x7f551c1445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:55.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.596+0000 7f552ac85640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f551c0ab5f0 0x7f551c149850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:55.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.596+0000 7f552ac85640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f551c0b4e10 con 0x7f551c0aaca0 2024-09-21T04:07:55.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.596+0000 7f552ac85640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f551c0b4c90 con 0x7f551c0ab5f0 2024-09-21T04:07:55.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.597+0000 7f552ac85640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f551c0b4f90 con 0x7f551c0a5360 2024-09-21T04:07:55.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.597+0000 7f5523fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f551c0aaca0 0x7f551c1445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:55.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.597+0000 7f5523fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f551c0aaca0 0x7f551c1445b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38866/0 (socket says 172.21.15.64:38866) 2024-09-21T04:07:55.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.597+0000 7f5523fff640 1 -- 172.21.15.64:0/3938825575 learned_addr learned my addr 172.21.15.64:0/3938825575 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:55.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.597+0000 7f5523fff640 1 -- 172.21.15.64:0/3938825575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f551c0a5360 msgr2=0x7f551c144010 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T04:07:55.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.598+0000 7f5523fff640 1 --2- 172.21.15.64:0/3938825575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f551c0a5360 0x7f551c144010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.598+0000 7f5523fff640 1 -- 172.21.15.64:0/3938825575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f551c0ab5f0 msgr2=0x7f551c149850 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T04:07:55.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.598+0000 7f5523fff640 1 --2- 172.21.15.64:0/3938825575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f551c0ab5f0 0x7f551c149850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.598+0000 7f5523fff640 1 -- 172.21.15.64:0/3938825575 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551c149f80 con 0x7f551c0aaca0 2024-09-21T04:07:55.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.598+0000 7f5523fff640 1 --2- 172.21.15.64:0/3938825575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f551c0aaca0 0x7f551c1445b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5524052860 tx=0x7f552406c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:55.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.600+0000 7f5521ffb640 1 -- 172.21.15.64:0/3938825575 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55240a6020 con 0x7f551c0aaca0 2024-09-21T04:07:55.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.601+0000 7f5521ffb640 1 -- 172.21.15.64:0/3938825575 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5524097620 con 0x7f551c0aaca0 2024-09-21T04:07:55.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.601+0000 7f5521ffb640 1 -- 172.21.15.64:0/3938825575 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55240a1370 con 0x7f551c0aaca0 2024-09-21T04:07:55.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.602+0000 7f552ac85640 1 -- 172.21.15.64:0/3938825575 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f551c14a210 con 0x7f551c0aaca0 2024-09-21T04:07:55.604 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.602+0000 7f552ac85640 1 -- 172.21.15.64:0/3938825575 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f551c14a670 con 0x7f551c0aaca0 2024-09-21T04:07:55.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.614+0000 7f5521ffb640 1 -- 172.21.15.64:0/3938825575 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f552409f030 con 0x7f551c0aaca0 2024-09-21T04:07:55.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.617+0000 7f5521ffb640 1 --2- 172.21.15.64:0/3938825575 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f55140b6880 0x7f55140b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:55.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.617+0000 7f5521ffb640 1 -- 172.21.15.64:0/3938825575 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f552416c220 con 0x7f551c0aaca0 2024-09-21T04:07:55.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.621+0000 7f55311f9740 1 Processor -- start 2024-09-21T04:07:55.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.621+0000 7f5528a21640 1 --2- 172.21.15.64:0/3938825575 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f55140b6880 0x7f55140b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:55.627 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.625+0000 7f55311f9740 1 -- start start 2024-09-21T04:07:55.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.626+0000 7f55311f9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a59b29e070 0x55a59b645860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:55.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.626+0000 7f55311f9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59b2751b0 0x55a59b645da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:55.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.626+0000 7f55311f9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a59b27d390 0x55a59b6462e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:55.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.626+0000 7f55311f9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a59b5bbea0 con 0x55a59b2751b0 2024-09-21T04:07:55.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.626+0000 7f55311f9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a59b5ab2d0 con 0x55a59b29e070 2024-09-21T04:07:55.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.626+0000 7f55311f9740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a59b5af0e0 con 0x55a59b27d390 2024-09-21T04:07:55.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.626+0000 7f5528a21640 1 --2- 172.21.15.64:0/3938825575 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f55140b6880 0x7f55140b8d90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f551800b990 tx=0x7f551800b370 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:55.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.626+0000 7f5528a21640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a59b29e070 0x55a59b645860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:55.629 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.626+0000 7f5529222640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a59b27d390 0x55a59b6462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:55.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.626+0000 7f5529222640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a59b27d390 0x55a59b6462e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:39048/0 (socket says 172.21.15.64:39048) 2024-09-21T04:07:55.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.626+0000 7f5529222640 1 -- 172.21.15.64:0/134341583 learned_addr learned my addr 172.21.15.64:0/134341583 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:55.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.627+0000 7f5523fff640 1 --2- 172.21.15.64:0/134341583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59b2751b0 0x55a59b645da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:55.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.627+0000 7f5523fff640 1 -- 172.21.15.64:0/134341583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a59b27d390 msgr2=0x55a59b6462e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:55.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.627+0000 7f5523fff640 1 --2- 172.21.15.64:0/134341583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a59b27d390 0x55a59b6462e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.627+0000 7f5523fff640 1 -- 172.21.15.64:0/134341583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a59b29e070 msgr2=0x55a59b645860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:55.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.627+0000 7f5523fff640 1 --2- 172.21.15.64:0/134341583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a59b29e070 0x55a59b645860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.627+0000 7f5523fff640 1 -- 172.21.15.64:0/134341583 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a59b309650 con 0x55a59b2751b0 2024-09-21T04:07:55.630 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.628+0000 7f5523fff640 1 --2- 172.21.15.64:0/134341583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59b2751b0 0x55a59b645da0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f55241698e0 tx=0x7f5524069460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364085d93325b852 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:55.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.628+0000 7f550effd640 1 -- 172.21.15.64:0/134341583 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5524169d60 con 0x55a59b2751b0 2024-09-21T04:07:55.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.628+0000 7f550effd640 1 -- 172.21.15.64:0/134341583 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55241326e0 con 0x55a59b2751b0 2024-09-21T04:07:55.631 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.628+0000 7f550effd640 1 -- 172.21.15.64:0/134341583 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552412db90 con 0x55a59b2751b0 2024-09-21T04:07:55.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.630+0000 7f55311f9740 1 -- 172.21.15.64:0/134341583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59b2751b0 msgr2=0x55a59b645da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:55.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.630+0000 7f55311f9740 1 --2- 172.21.15.64:0/134341583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59b2751b0 0x55a59b645da0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f55241698e0 tx=0x7f5524069460 comp rx=0 tx=0).stop 2024-09-21T04:07:55.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.636+0000 7f55311f9740 1 -- 172.21.15.64:0/134341583 shutdown_connections 2024-09-21T04:07:55.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.636+0000 7f55311f9740 1 --2- 172.21.15.64:0/134341583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a59b27d390 0x55a59b6462e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.636+0000 7f55311f9740 1 --2- 172.21.15.64:0/134341583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59b2751b0 0x55a59b645da0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.636+0000 7f55311f9740 1 --2- 172.21.15.64:0/134341583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a59b29e070 0x55a59b645860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.639 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.636+0000 7f55311f9740 1 -- 172.21.15.64:0/134341583 >> 172.21.15.64:0/134341583 conn(0x55a59b292460 msgr2=0x55a59b643750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:55.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.644+0000 7f55311f9740 1 -- 172.21.15.64:0/134341583 shutdown_connections 2024-09-21T04:07:55.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.645+0000 7f55311f9740 1 -- 172.21.15.64:0/134341583 wait complete. 2024-09-21T04:07:55.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.645+0000 7f55311f9740 1 -- start start 2024-09-21T04:07:55.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.645+0000 7f55311f9740 1 Processor -- start 2024-09-21T04:07:55.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.650+0000 7f55311f9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a59b2751b0 0x55a59b650f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:55.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.650+0000 7f55311f9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59b27d390 0x55a59b6514a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:55.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.650+0000 7f55311f9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a59b29e070 0x55a59b65cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:55.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.650+0000 7f55311f9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a59b40c700 con 0x55a59b27d390 2024-09-21T04:07:55.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.650+0000 7f55311f9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a59b546160 con 0x55a59b2751b0 2024-09-21T04:07:55.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.650+0000 7f55311f9740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a59b5bbea0 con 0x55a59b29e070 2024-09-21T04:07:55.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.651+0000 7f5523fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59b27d390 0x55a59b6514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:55.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.651+0000 7f5523fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59b27d390 0x55a59b6514a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38874/0 (socket says 172.21.15.64:38874) 2024-09-21T04:07:55.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.651+0000 7f5523fff640 1 -- 172.21.15.64:0/1279245612 learned_addr learned my addr 172.21.15.64:0/1279245612 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:55.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.651+0000 7f5528a21640 1 --2- 172.21.15.64:0/1279245612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a59b2751b0 0x55a59b650f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:55.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.652+0000 7f5523fff640 1 -- 172.21.15.64:0/1279245612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a59b29e070 msgr2=0x55a59b65cf70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:07:55.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.652+0000 7f5523fff640 1 --2- 172.21.15.64:0/1279245612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a59b29e070 0x55a59b65cf70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.652+0000 7f5523fff640 1 -- 172.21.15.64:0/1279245612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a59b2751b0 msgr2=0x55a59b650f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:55.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.652+0000 7f55311f9740 10 client.0 _ll_register_callbacks cb 0x55a59b55f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:55.658 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.652+0000 7f5523fff640 1 --2- 172.21.15.64:0/1279245612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a59b2751b0 0x55a59b650f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.653+0000 7f5523fff640 1 -- 172.21.15.64:0/1279245612 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a59b573c90 con 0x55a59b27d390 2024-09-21T04:07:55.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.653+0000 7f5528a21640 1 --2- 172.21.15.64:0/1279245612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a59b2751b0 0x55a59b650f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:07:55.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.653+0000 7f55311f9740 1 -- 172.21.15.64:0/1279245612 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55241326e0 con 0x55a59b27d390 2024-09-21T04:07:55.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.653+0000 7f5523fff640 1 --2- 172.21.15.64:0/1279245612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59b27d390 0x55a59b6514a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5524132890 tx=0x7f552408f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e964a8b579716681 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:55.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.654+0000 7f5503fff640 10 client.16383 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:07:55.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.654+0000 7f5503fff640 1 -- 172.21.15.64:0/1279245612 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55240e1020 con 0x55a59b27d390 2024-09-21T04:07:55.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.655+0000 7f5503fff640 1 -- 172.21.15.64:0/1279245612 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f5524131470 con 0x55a59b27d390 2024-09-21T04:07:55.659 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.655+0000 7f5503fff640 1 -- 172.21.15.64:0/1279245612 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5524136ac0 con 0x55a59b27d390 2024-09-21T04:07:55.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.657+0000 7f55311f9740 10 client.16383 fetch_fsmap learned FSMap version 32 2024-09-21T04:07:55.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.657+0000 7f55311f9740 1 -- 172.21.15.64:0/1279245612 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55a59b65f720 con 0x55a59b27d390 2024-09-21T04:07:55.662 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.659+0000 7f5503fff640 1 -- 172.21.15.64:0/1279245612 <== mon.0 v2:172.21.15.64:3300/0 4 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f5524072040 con 0x55a59b27d390 2024-09-21T04:07:55.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.660+0000 7f55311f9740 10 client.16383 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T04:07:55.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.660+0000 7f55311f9740 10 client.16383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:07:55.664 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.660+0000 7f55311f9740 20 client.16383 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:55.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.660+0000 7f55311f9740 1 --2- 172.21.15.64:0/1279245612 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a59b25c9f0 0x55a59b6628b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:55.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.660+0000 7f55311f9740 4 client.16383 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:07:55.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.660+0000 7f55311f9740 1 -- 172.21.15.64:0/1279245612 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a59b5bddf0 con 0x55a59b25c9f0 2024-09-21T04:07:55.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.660+0000 7f5528a21640 1 --2- 172.21.15.64:0/1279245612 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a59b25c9f0 0x55a59b6628b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:55.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.662+0000 7f5528a21640 1 --2- 172.21.15.64:0/1279245612 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a59b25c9f0 0x55a59b6628b0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8df9304d8dc1fc13 server_cookie=6efdeee006a566d8 in_seq=0 out_seq=0 2024-09-21T04:07:55.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.662+0000 7f5503fff640 10 client.16383 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:55.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.664+0000 7f5503fff640 1 -- 172.21.15.64:0/1279245612 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a59b5bddf0 con 0x55a59b25c9f0 2024-09-21T04:07:55.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.664+0000 7f5503fff640 10 client.16383 handle_command_reply: tid=0 2024-09-21T04:07:55.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.665+0000 7f55311f9740 1 client.16383 shutdown 2024-09-21T04:07:55.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.668+0000 7f55311f9740 1 -- 172.21.15.64:0/1279245612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59b27d390 msgr2=0x55a59b6514a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:55.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.668+0000 7f55311f9740 1 --2- 172.21.15.64:0/1279245612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59b27d390 0x55a59b6514a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5524132890 tx=0x7f552408f850 comp rx=0 tx=0).stop 2024-09-21T04:07:55.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.671+0000 7f55311f9740 1 -- 172.21.15.64:0/1279245612 shutdown_connections 2024-09-21T04:07:55.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.671+0000 7f55311f9740 1 --2- 172.21.15.64:0/1279245612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a59b29e070 0x55a59b65cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.671+0000 7f55311f9740 1 --2- 172.21.15.64:0/1279245612 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a59b25c9f0 0x55a59b6628b0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.671+0000 7f55311f9740 1 --2- 172.21.15.64:0/1279245612 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59b27d390 0x55a59b6514a0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.671+0000 7f55311f9740 1 --2- 172.21.15.64:0/1279245612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a59b2751b0 0x55a59b650f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.671+0000 7f55311f9740 1 -- 172.21.15.64:0/1279245612 >> 172.21.15.64:0/1279245612 conn(0x55a59b292460 msgr2=0x55a59b6489f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:55.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.671+0000 7f5503fff640 0 client.16383 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:55.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.675+0000 7f55311f9740 1 -- 172.21.15.64:0/1279245612 shutdown_connections 2024-09-21T04:07:55.677 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.675+0000 7f55311f9740 1 -- 172.21.15.64:0/1279245612 wait complete. 2024-09-21T04:07:55.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.675+0000 7f55311f9740 20 client.16383 trim_cache size 0 max 16384 2024-09-21T04:07:55.711 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.709+0000 7f55311f9740 1 Processor -- start 2024-09-21T04:07:55.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.709+0000 7f55311f9740 1 -- start start 2024-09-21T04:07:55.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.709+0000 7f55311f9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59b2751b0 0x55a59b646fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:55.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.709+0000 7f55311f9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a59b27d390 0x55a59b642d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:55.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.709+0000 7f55311f9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a59b29e070 0x55a59b643260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:55.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.709+0000 7f55311f9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f551c010c40 con 0x55a59b2751b0 2024-09-21T04:07:55.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.709+0000 7f55311f9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f551c00e480 con 0x55a59b29e070 2024-09-21T04:07:55.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.709+0000 7f55311f9740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f551c0a57b0 con 0x55a59b27d390 2024-09-21T04:07:55.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.710+0000 7f5529222640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a59b29e070 0x55a59b643260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:55.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.710+0000 7f5529222640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a59b29e070 0x55a59b643260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:45230/0 (socket says 172.21.15.64:45230) 2024-09-21T04:07:55.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.710+0000 7f5529222640 1 -- 172.21.15.64:0/570778062 learned_addr learned my addr 172.21.15.64:0/570778062 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:55.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.711+0000 7f5529222640 1 -- 172.21.15.64:0/570778062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a59b27d390 msgr2=0x55a59b642d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:07:55.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.711+0000 7f5529222640 1 --2- 172.21.15.64:0/570778062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a59b27d390 0x55a59b642d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.711+0000 7f5529222640 1 -- 172.21.15.64:0/570778062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59b2751b0 msgr2=0x55a59b646fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:07:55.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.711+0000 7f5529222640 1 --2- 172.21.15.64:0/570778062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59b2751b0 0x55a59b646fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.711+0000 7f5529222640 1 -- 172.21.15.64:0/570778062 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a59b64bb90 con 0x55a59b29e070 2024-09-21T04:07:55.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.712+0000 7f5529222640 1 --2- 172.21.15.64:0/570778062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a59b29e070 0x55a59b643260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f551000da90 tx=0x7f551000df50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eff1a356827242c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:55.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.716+0000 7f550e7fc640 1 -- 172.21.15.64:0/570778062 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551000bb90 con 0x55a59b29e070 2024-09-21T04:07:55.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.716+0000 7f550e7fc640 1 -- 172.21.15.64:0/570778062 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f551000bd30 con 0x55a59b29e070 2024-09-21T04:07:55.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.716+0000 7f550e7fc640 1 -- 172.21.15.64:0/570778062 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5510016400 con 0x55a59b29e070 2024-09-21T04:07:55.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.718+0000 7f55311f9740 1 -- 172.21.15.64:0/570778062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a59b29e070 msgr2=0x55a59b643260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:55.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.718+0000 7f55311f9740 1 --2- 172.21.15.64:0/570778062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a59b29e070 0x55a59b643260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f551000da90 tx=0x7f551000df50 comp rx=0 tx=0).stop 2024-09-21T04:07:55.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.727+0000 7f55311f9740 1 -- 172.21.15.64:0/570778062 shutdown_connections 2024-09-21T04:07:55.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.727+0000 7f55311f9740 1 --2- 172.21.15.64:0/570778062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a59b29e070 0x55a59b643260 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.727+0000 7f55311f9740 1 --2- 172.21.15.64:0/570778062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a59b27d390 0x55a59b642d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.727+0000 7f55311f9740 1 --2- 172.21.15.64:0/570778062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59b2751b0 0x55a59b646fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.727+0000 7f55311f9740 1 -- 172.21.15.64:0/570778062 >> 172.21.15.64:0/570778062 conn(0x55a59b292460 msgr2=0x55a59b646890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:55.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.728+0000 7f55311f9740 1 -- 172.21.15.64:0/570778062 shutdown_connections 2024-09-21T04:07:55.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.728+0000 7f55311f9740 1 -- 172.21.15.64:0/570778062 wait complete. 2024-09-21T04:07:55.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.729+0000 7f55311f9740 1 -- start start 2024-09-21T04:07:55.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.729+0000 7f55311f9740 1 Processor -- start 2024-09-21T04:07:55.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.733+0000 7f55311f9740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a59b2751b0 0x55a59b654fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:55.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.733+0000 7f55311f9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a59b27d390 0x55a59b6554f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:55.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.733+0000 7f55311f9740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59b29e070 0x55a59b655a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:55.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.733+0000 7f55311f9740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f551c00c940 con 0x55a59b29e070 2024-09-21T04:07:55.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.733+0000 7f55311f9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f551c00ddb0 con 0x55a59b27d390 2024-09-21T04:07:55.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.733+0000 7f55311f9740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f551c00d010 con 0x55a59b2751b0 2024-09-21T04:07:55.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.734+0000 7f55311f9740 10 client.0 _ll_register_callbacks cb 0x7f5518005eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:55.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.734+0000 7f5528a21640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a59b2751b0 0x55a59b654fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:55.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.734+0000 7f5528a21640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a59b2751b0 0x55a59b654fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:39060/0 (socket says 172.21.15.64:39060) 2024-09-21T04:07:55.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.734+0000 7f5528a21640 1 -- 172.21.15.64:0/2421734725 learned_addr learned my addr 172.21.15.64:0/2421734725 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:55.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.734+0000 7f5529222640 1 --2- 172.21.15.64:0/2421734725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59b29e070 0x55a59b655a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:55.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.734+0000 7f5523fff640 1 --2- 172.21.15.64:0/2421734725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a59b27d390 0x55a59b6554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:55.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.734+0000 7f5528a21640 1 -- 172.21.15.64:0/2421734725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a59b27d390 msgr2=0x55a59b6554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:55.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.734+0000 7f5528a21640 1 --2- 172.21.15.64:0/2421734725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a59b27d390 0x55a59b6554f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.734+0000 7f5528a21640 1 -- 172.21.15.64:0/2421734725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59b29e070 msgr2=0x55a59b655a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:55.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.734+0000 7f5528a21640 1 --2- 172.21.15.64:0/2421734725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59b29e070 0x55a59b655a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.734+0000 7f5528a21640 1 -- 172.21.15.64:0/2421734725 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a59b6cc300 con 0x55a59b2751b0 2024-09-21T04:07:55.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.735+0000 7f5528a21640 1 --2- 172.21.15.64:0/2421734725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a59b2751b0 0x55a59b654fb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f551800dea0 tx=0x7f55180046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aced630d765bcc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:55.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.736+0000 7f5503fff640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-21T04:07:55.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.736+0000 7f5503fff640 1 -- 172.21.15.64:0/2421734725 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551801e8c0 con 0x55a59b2751b0 2024-09-21T04:07:55.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.736+0000 7f55311f9740 1 -- 172.21.15.64:0/2421734725 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f551000bd30 con 0x55a59b2751b0 2024-09-21T04:07:55.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.736+0000 7f5503fff640 1 -- 172.21.15.64:0/2421734725 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f551801ea60 con 0x55a59b2751b0 2024-09-21T04:07:55.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.739+0000 7f5503fff640 1 -- 172.21.15.64:0/2421734725 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5518037050 con 0x55a59b2751b0 2024-09-21T04:07:55.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.743+0000 7f55311f9740 10 client.25969 fetch_fsmap learned FSMap version 32 2024-09-21T04:07:55.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.743+0000 7f55311f9740 1 -- 172.21.15.64:0/2421734725 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a59b6cd5e0 con 0x55a59b2751b0 2024-09-21T04:07:55.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.744+0000 7f5503fff640 1 -- 172.21.15.64:0/2421734725 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f551000bd30 con 0x55a59b2751b0 2024-09-21T04:07:55.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.744+0000 7f5503fff640 1 -- 172.21.15.64:0/2421734725 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f5518027340 con 0x55a59b2751b0 2024-09-21T04:07:55.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.744+0000 7f55311f9740 10 client.25969 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T04:07:55.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.744+0000 7f55311f9740 10 client.25969 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:07:55.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.744+0000 7f55311f9740 20 client.25969 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:55.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.744+0000 7f55311f9740 1 --2- 172.21.15.64:0/2421734725 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a59b231da0 0x55a59b6cfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:55.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.744+0000 7f55311f9740 4 client.25969 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:07:55.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.744+0000 7f55311f9740 1 -- 172.21.15.64:0/2421734725 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a59b5b98a0 con 0x55a59b231da0 2024-09-21T04:07:55.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.746+0000 7f5523fff640 1 --2- 172.21.15.64:0/2421734725 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a59b231da0 0x55a59b6cfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:55.751 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.749+0000 7f5523fff640 1 --2- 172.21.15.64:0/2421734725 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a59b231da0 0x55a59b6cfd00 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9cf949e4f1191a3 server_cookie=f986e40e27b529c8 in_seq=0 out_seq=0 2024-09-21T04:07:55.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.749+0000 7f5503fff640 10 client.25969 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:55.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.749+0000 7f5503fff640 1 -- 172.21.15.64:0/2421734725 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a59b5b98a0 con 0x55a59b231da0 2024-09-21T04:07:55.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.749+0000 7f5503fff640 10 client.25969 handle_command_reply: tid=0 2024-09-21T04:07:55.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.749+0000 7f55311f9740 1 client.25969 shutdown 2024-09-21T04:07:55.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.757+0000 7f55311f9740 1 -- 172.21.15.64:0/2421734725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a59b2751b0 msgr2=0x55a59b654fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:55.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.757+0000 7f55311f9740 1 --2- 172.21.15.64:0/2421734725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a59b2751b0 0x55a59b654fb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f551800dea0 tx=0x7f55180046c0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.757+0000 7f55311f9740 1 -- 172.21.15.64:0/2421734725 shutdown_connections 2024-09-21T04:07:55.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.757+0000 7f55311f9740 1 --2- 172.21.15.64:0/2421734725 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a59b231da0 0x55a59b6cfd00 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.757+0000 7f55311f9740 1 --2- 172.21.15.64:0/2421734725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a59b29e070 0x55a59b655a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.757+0000 7f55311f9740 1 --2- 172.21.15.64:0/2421734725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a59b27d390 0x55a59b6554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.757+0000 7f55311f9740 1 --2- 172.21.15.64:0/2421734725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a59b2751b0 0x55a59b654fb0 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f551800dea0 tx=0x7f55180046c0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.757+0000 7f55311f9740 1 -- 172.21.15.64:0/2421734725 >> 172.21.15.64:0/2421734725 conn(0x55a59b292460 msgr2=0x55a59b646870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:55.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.757+0000 7f5503fff640 0 client.25969 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:55.764 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.758+0000 7f55311f9740 1 -- 172.21.15.64:0/2421734725 shutdown_connections 2024-09-21T04:07:55.768 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:07:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.761+0000 7f55311f9740 1 -- 172.21.15.64:0/2421734725 wait complete. 2024-09-21T04:07:55.768 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.762+0000 7f55311f9740 20 client.25969 trim_cache size 0 max 16384 2024-09-21T04:07:55.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.770+0000 7f552ac85640 1 -- 172.21.15.64:0/3938825575 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f55140b6880 msgr2=0x7f55140b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:55.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.770+0000 7f552ac85640 1 --2- 172.21.15.64:0/3938825575 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f55140b6880 0x7f55140b8d90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f551800b990 tx=0x7f551800b370 comp rx=0 tx=0).stop 2024-09-21T04:07:55.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.770+0000 7f552ac85640 1 -- 172.21.15.64:0/3938825575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f551c0aaca0 msgr2=0x7f551c1445b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:55.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.770+0000 7f552ac85640 1 --2- 172.21.15.64:0/3938825575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f551c0aaca0 0x7f551c1445b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5524052860 tx=0x7f552406c520 comp rx=0 tx=0).stop 2024-09-21T04:07:55.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.770+0000 7f552ac85640 1 -- 172.21.15.64:0/3938825575 shutdown_connections 2024-09-21T04:07:55.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.770+0000 7f552ac85640 1 --2- 172.21.15.64:0/3938825575 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f55140b6880 0x7f55140b8d90 secure :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f551800b990 tx=0x7f551800b370 comp rx=0 tx=0).stop 2024-09-21T04:07:55.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.770+0000 7f552ac85640 1 --2- 172.21.15.64:0/3938825575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f551c0ab5f0 0x7f551c149850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.770+0000 7f552ac85640 1 --2- 172.21.15.64:0/3938825575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f551c0aaca0 0x7f551c1445b0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.770+0000 7f552ac85640 1 --2- 172.21.15.64:0/3938825575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f551c0a5360 0x7f551c144010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:55.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.770+0000 7f552ac85640 1 -- 172.21.15.64:0/3938825575 >> 172.21.15.64:0/3938825575 conn(0x7f551c01d090 msgr2=0x7f551c0aa250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:55.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.772+0000 7f552ac85640 1 -- 172.21.15.64:0/3938825575 shutdown_connections 2024-09-21T04:07:55.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:55.772+0000 7f552ac85640 1 -- 172.21.15.64:0/3938825575 wait complete. 2024-09-21T04:07:56.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:55 smithi119 ceph-mon[34811]: pgmap v438: 97 pgs: 97 active+clean; 533 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.2 MiB/s wr, 450 op/s 2024-09-21T04:07:56.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:55 smithi064 ceph-mon[31180]: pgmap v438: 97 pgs: 97 active+clean; 533 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.2 MiB/s wr, 450 op/s 2024-09-21T04:07:56.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:55 smithi131 ceph-mon[35193]: pgmap v438: 97 pgs: 97 active+clean; 533 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.2 MiB/s wr, 450 op/s 2024-09-21T04:07:58.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:57 smithi119 ceph-mon[34811]: pgmap v439: 97 pgs: 97 active+clean; 533 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.4 MiB/s wr, 663 op/s 2024-09-21T04:07:58.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:57 smithi064 ceph-mon[31180]: pgmap v439: 97 pgs: 97 active+clean; 533 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.4 MiB/s wr, 663 op/s 2024-09-21T04:07:58.012 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T04:07:58.012 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T04:07:58.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:57 smithi131 ceph-mon[35193]: pgmap v439: 97 pgs: 97 active+clean; 533 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.4 MiB/s wr, 663 op/s 2024-09-21T04:07:58.360 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:07:59.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.005+0000 7feede5b6640 1 -- 172.21.15.64:0/877877277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feed00956b0 msgr2=0x7feed0099be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:59.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.005+0000 7feede5b6640 1 --2- 172.21.15.64:0/877877277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feed00956b0 0x7feed0099be0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7feecc009890 tx=0x7feecc02a370 comp rx=0 tx=0).stop 2024-09-21T04:07:59.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.008+0000 7feede5b6640 1 -- 172.21.15.64:0/877877277 shutdown_connections 2024-09-21T04:07:59.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.008+0000 7feede5b6640 1 --2- 172.21.15.64:0/877877277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feed009a1a0 0x7feed009a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.008+0000 7feede5b6640 1 --2- 172.21.15.64:0/877877277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feed00956b0 0x7feed0099be0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.008+0000 7feede5b6640 1 --2- 172.21.15.64:0/877877277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feed0094d60 0x7feed0095170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.008+0000 7feede5b6640 1 -- 172.21.15.64:0/877877277 >> 172.21.15.64:0/877877277 conn(0x7feed000cd70 msgr2=0x7feed000d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:59.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.008+0000 7feede5b6640 1 -- 172.21.15.64:0/877877277 shutdown_connections 2024-09-21T04:07:59.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.008+0000 7feede5b6640 1 -- 172.21.15.64:0/877877277 wait complete. 2024-09-21T04:07:59.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.009+0000 7feede5b6640 1 Processor -- start 2024-09-21T04:07:59.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.009+0000 7feede5b6640 1 -- start start 2024-09-21T04:07:59.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.010+0000 7feede5b6640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feed0094d60 0x7feed00a92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:59.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.010+0000 7feede5b6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feed00956b0 0x7feed00a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:59.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.010+0000 7feede5b6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feed009a1a0 0x7feed00a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:59.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.010+0000 7feede5b6640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7feed009da40 con 0x7feed0094d60 2024-09-21T04:07:59.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.010+0000 7feede5b6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feed009d8c0 con 0x7feed009a1a0 2024-09-21T04:07:59.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.010+0000 7feede5b6640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feed009dbc0 con 0x7feed00956b0 2024-09-21T04:07:59.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.010+0000 7feed77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feed00956b0 0x7feed00a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:59.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.010+0000 7feed77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feed00956b0 0x7feed00a55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:39076/0 (socket says 172.21.15.64:39076) 2024-09-21T04:07:59.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.010+0000 7feed77fe640 1 -- 172.21.15.64:0/2580526947 learned_addr learned my addr 172.21.15.64:0/2580526947 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:59.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.011+0000 7feedcb53640 1 --2- 172.21.15.64:0/2580526947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feed009a1a0 0x7feed00a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:59.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.011+0000 7feed77fe640 1 -- 172.21.15.64:0/2580526947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feed009a1a0 msgr2=0x7feed00a5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:59.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.011+0000 7feed77fe640 1 --2- 172.21.15.64:0/2580526947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feed009a1a0 0x7feed00a5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.011+0000 7feed77fe640 1 -- 172.21.15.64:0/2580526947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feed0094d60 msgr2=0x7feed00a92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:59.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.011+0000 7feed7fff640 1 --2- 172.21.15.64:0/2580526947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feed0094d60 0x7feed00a92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:59.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.011+0000 7feed77fe640 1 --2- 172.21.15.64:0/2580526947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feed0094d60 0x7feed00a92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.011+0000 7feed77fe640 1 -- 172.21.15.64:0/2580526947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed00a63e0 con 0x7feed00956b0 2024-09-21T04:07:59.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.012+0000 7feed7fff640 1 --2- 172.21.15.64:0/2580526947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feed0094d60 0x7feed00a92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:07:59.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.012+0000 7feed77fe640 1 --2- 172.21.15.64:0/2580526947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feed00956b0 0x7feed00a55d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feecc035040 tx=0x7feecc032ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:59.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.013+0000 7feed57fa640 1 -- 172.21.15.64:0/2580526947 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc037070 con 0x7feed00956b0 2024-09-21T04:07:59.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.014+0000 7feede5b6640 1 -- 172.21.15.64:0/2580526947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed0134b50 con 0x7feed00956b0 2024-09-21T04:07:59.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.014+0000 7feede5b6640 1 -- 172.21.15.64:0/2580526947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed0135000 con 0x7feed00956b0 2024-09-21T04:07:59.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.017+0000 7feed57fa640 1 -- 172.21.15.64:0/2580526947 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feecc063070 con 0x7feed00956b0 2024-09-21T04:07:59.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.017+0000 7feed57fa640 1 -- 172.21.15.64:0/2580526947 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc0046c0 con 0x7feed00956b0 2024-09-21T04:07:59.026 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.024+0000 7feee4b2a740 1 Processor -- start 2024-09-21T04:07:59.027 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.024+0000 7feee4b2a740 1 -- start start 2024-09-21T04:07:59.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.029+0000 7feee4b2a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e2674e350 0x556e26af5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:59.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.029+0000 7feee4b2a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e26725490 0x556e26af5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:59.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.029+0000 7feee4b2a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e2672d670 0x556e26af6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:59.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.029+0000 7feee4b2a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556e26a6be60 con 0x556e26725490 2024-09-21T04:07:59.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.029+0000 7feee4b2a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556e26a5b290 con 0x556e2672d670 2024-09-21T04:07:59.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.029+0000 7feee4b2a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556e26a5f0a0 con 0x556e2674e350 2024-09-21T04:07:59.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.029+0000 7feedcb53640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e2674e350 0x556e26af5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:59.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.029+0000 7feedcb53640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e2674e350 0x556e26af5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:39086/0 (socket says 172.21.15.64:39086) 2024-09-21T04:07:59.033 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.029+0000 7feedcb53640 1 -- 172.21.15.64:0/2701777780 learned_addr learned my addr 172.21.15.64:0/2701777780 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:59.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.030+0000 7feed7fff640 1 --2- 172.21.15.64:0/2701777780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e26725490 0x556e26af5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:59.036 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.031+0000 7feed57fa640 1 -- 172.21.15.64:0/2580526947 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7feecc004920 con 0x7feed00956b0 2024-09-21T04:07:59.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.032+0000 7feed57fa640 1 --2- 172.21.15.64:0/2580526947 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7feec80b6880 0x7feec80b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:59.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.033+0000 7feed7fff640 1 -- 172.21.15.64:0/2701777780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e2674e350 msgr2=0x556e26af5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:59.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.033+0000 7feed7fff640 1 --2- 172.21.15.64:0/2701777780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e2674e350 0x556e26af5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.033+0000 7feed7fff640 1 -- 172.21.15.64:0/2701777780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e2672d670 msgr2=0x556e26af6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:59.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.033+0000 7feedcb53640 1 --2- 172.21.15.64:0/2580526947 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7feec80b6880 0x7feec80b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:59.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.034+0000 7feed77fe640 1 --2- 172.21.15.64:0/2701777780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e2672d670 0x556e26af6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:59.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.034+0000 7feed7fff640 1 --2- 172.21.15.64:0/2701777780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e2672d670 0x556e26af6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.037 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.034+0000 7feed7fff640 1 -- 172.21.15.64:0/2701777780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e267b9930 con 0x556e26725490 2024-09-21T04:07:59.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.035+0000 7feed7fff640 1 --2- 172.21.15.64:0/2701777780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e26725490 0x556e26af5b40 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7feec4005fd0 tx=0x7feec4021750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb277b33cd790cca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:59.041 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.036+0000 7feed57fa640 1 -- 172.21.15.64:0/2580526947 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7feecc129c70 con 0x7feed00956b0 2024-09-21T04:07:59.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.036+0000 7feedcb53640 1 --2- 172.21.15.64:0/2580526947 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7feec80b6880 0x7feec80b8d90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feed8000b60 tx=0x7feed8069950 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:59.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.039+0000 7feec27fc640 1 -- 172.21.15.64:0/2701777780 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec401ae60 con 0x556e26725490 2024-09-21T04:07:59.042 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.039+0000 7feec27fc640 1 -- 172.21.15.64:0/2701777780 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feec40035f0 con 0x556e26725490 2024-09-21T04:07:59.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.040+0000 7feee4b2a740 1 -- 172.21.15.64:0/2701777780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e26725490 msgr2=0x556e26af5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:59.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.040+0000 7feee4b2a740 1 --2- 172.21.15.64:0/2701777780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e26725490 0x556e26af5b40 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7feec4005fd0 tx=0x7feec4021750 comp rx=0 tx=0).stop 2024-09-21T04:07:59.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.044+0000 7feee4b2a740 1 -- 172.21.15.64:0/2701777780 shutdown_connections 2024-09-21T04:07:59.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.044+0000 7feee4b2a740 1 --2- 172.21.15.64:0/2701777780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e2672d670 0x556e26af6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.044+0000 7feee4b2a740 1 --2- 172.21.15.64:0/2701777780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e26725490 0x556e26af5b40 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.044+0000 7feee4b2a740 1 --2- 172.21.15.64:0/2701777780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e2674e350 0x556e26af5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.044+0000 7feee4b2a740 1 -- 172.21.15.64:0/2701777780 >> 172.21.15.64:0/2701777780 conn(0x556e26742740 msgr2=0x556e26af34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:59.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.048+0000 7feee4b2a740 1 -- 172.21.15.64:0/2701777780 shutdown_connections 2024-09-21T04:07:59.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.050+0000 7feee4b2a740 1 -- 172.21.15.64:0/2701777780 wait complete. 2024-09-21T04:07:59.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.050+0000 7feee4b2a740 1 -- start start 2024-09-21T04:07:59.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.050+0000 7feee4b2a740 1 Processor -- start 2024-09-21T04:07:59.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.051+0000 7feee4b2a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e26725490 0x556e26b01090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:59.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.051+0000 7feee4b2a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e2672d670 0x556e26b015d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:59.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.051+0000 7feee4b2a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e2674e350 0x556e26b0d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:59.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.051+0000 7feee4b2a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556e268bc890 con 0x556e26725490 2024-09-21T04:07:59.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.051+0000 7feee4b2a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556e269f5d40 con 0x556e2674e350 2024-09-21T04:07:59.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.051+0000 7feee4b2a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556e26b03e30 con 0x556e2672d670 2024-09-21T04:07:59.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.051+0000 7feee4b2a740 10 client.0 _ll_register_callbacks cb 0x556e26a59c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:59.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.052+0000 7feed7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e26725490 0x556e26b01090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:59.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.052+0000 7feed7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e26725490 0x556e26b01090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38916/0 (socket says 172.21.15.64:38916) 2024-09-21T04:07:59.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.052+0000 7feed7fff640 1 -- 172.21.15.64:0/3103266255 learned_addr learned my addr 172.21.15.64:0/3103266255 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:59.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.052+0000 7feed77fe640 1 --2- 172.21.15.64:0/3103266255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e2672d670 0x556e26b015d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:59.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.052+0000 7feedcb53640 1 --2- 172.21.15.64:0/3103266255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e2674e350 0x556e26b0d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:59.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.053+0000 7feed7fff640 1 -- 172.21.15.64:0/3103266255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e2672d670 msgr2=0x556e26b015d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:59.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.053+0000 7feed7fff640 1 --2- 172.21.15.64:0/3103266255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e2672d670 0x556e26b015d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.053+0000 7feed7fff640 1 -- 172.21.15.64:0/3103266255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e2674e350 msgr2=0x556e26b0d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:59.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.053+0000 7feed7fff640 1 --2- 172.21.15.64:0/3103266255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e2674e350 0x556e26b0d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.053+0000 7feed7fff640 1 -- 172.21.15.64:0/3103266255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e26b0d710 con 0x556e26725490 2024-09-21T04:07:59.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.053+0000 7feed7fff640 1 --2- 172.21.15.64:0/3103266255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e26725490 0x556e26b01090 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7feec4022620 tx=0x7feec401a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1e51fcd0baf63c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:59.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.053+0000 7feeb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:07:59.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.054+0000 7feeb77fe640 1 -- 172.21.15.64:0/3103266255 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4012020 con 0x556e26725490 2024-09-21T04:07:59.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.054+0000 7feee4b2a740 1 -- 172.21.15.64:0/3103266255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feec40035f0 con 0x556e26725490 2024-09-21T04:07:59.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.054+0000 7feeb77fe640 1 -- 172.21.15.64:0/3103266255 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7feec4004210 con 0x556e26725490 2024-09-21T04:07:59.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.054+0000 7feeb77fe640 1 -- 172.21.15.64:0/3103266255 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feec401ae60 con 0x556e26725490 2024-09-21T04:07:59.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.054+0000 7feee4b2a740 10 client.16395 fetch_fsmap learned FSMap version 32 2024-09-21T04:07:59.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.054+0000 7feee4b2a740 1 -- 172.21.15.64:0/3103266255 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556e26b31fb0 con 0x556e26725490 2024-09-21T04:07:59.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.055+0000 7feeb77fe640 1 -- 172.21.15.64:0/3103266255 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feec40035f0 con 0x556e26725490 2024-09-21T04:07:59.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.055+0000 7feeb77fe640 1 -- 172.21.15.64:0/3103266255 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7feec40194b0 con 0x556e26725490 2024-09-21T04:07:59.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.055+0000 7feee4b2a740 10 client.16395 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T04:07:59.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.055+0000 7feee4b2a740 10 client.16395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:07:59.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.055+0000 7feee4b2a740 20 client.16395 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:59.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.055+0000 7feee4b2a740 1 --2- 172.21.15.64:0/3103266255 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556e2670ccd0 0x556e26b358a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:59.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.055+0000 7feee4b2a740 4 client.16395 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:07:59.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.055+0000 7feee4b2a740 1 -- 172.21.15.64:0/3103266255 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e26a6ddb0 con 0x556e2670ccd0 2024-09-21T04:07:59.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.055+0000 7feed77fe640 1 --2- 172.21.15.64:0/3103266255 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556e2670ccd0 0x556e26b358a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:59.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.056+0000 7feeb77fe640 10 client.16395 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:59.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.057+0000 7feed77fe640 1 --2- 172.21.15.64:0/3103266255 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556e2670ccd0 0x556e26b358a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23163aec8494b527 server_cookie=2b6bffd1d120d6e5 in_seq=0 out_seq=0 2024-09-21T04:07:59.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.059+0000 7feeb77fe640 1 -- 172.21.15.64:0/3103266255 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556e26a6ddb0 con 0x556e2670ccd0 2024-09-21T04:07:59.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.059+0000 7feeb77fe640 10 client.16395 handle_command_reply: tid=0 2024-09-21T04:07:59.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.059+0000 7feee4b2a740 1 client.16395 shutdown 2024-09-21T04:07:59.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.062+0000 7feee4b2a740 1 -- 172.21.15.64:0/3103266255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e26725490 msgr2=0x556e26b01090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:59.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.062+0000 7feee4b2a740 1 --2- 172.21.15.64:0/3103266255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e26725490 0x556e26b01090 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7feec4022620 tx=0x7feec401a860 comp rx=0 tx=0).stop 2024-09-21T04:07:59.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.062+0000 7feee4b2a740 1 -- 172.21.15.64:0/3103266255 shutdown_connections 2024-09-21T04:07:59.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.062+0000 7feee4b2a740 1 --2- 172.21.15.64:0/3103266255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e2674e350 0x556e26b0d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.062+0000 7feee4b2a740 1 --2- 172.21.15.64:0/3103266255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e2672d670 0x556e26b015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.062+0000 7feee4b2a740 1 --2- 172.21.15.64:0/3103266255 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556e2670ccd0 0x556e26b358a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.062+0000 7feee4b2a740 1 --2- 172.21.15.64:0/3103266255 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e26725490 0x556e26b01090 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.062+0000 7feee4b2a740 1 -- 172.21.15.64:0/3103266255 >> 172.21.15.64:0/3103266255 conn(0x556e26742740 msgr2=0x556e26af1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:59.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.062+0000 7feeb77fe640 0 client.16395 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:59.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.062+0000 7feee4b2a740 1 -- 172.21.15.64:0/3103266255 shutdown_connections 2024-09-21T04:07:59.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.062+0000 7feee4b2a740 1 -- 172.21.15.64:0/3103266255 wait complete. 2024-09-21T04:07:59.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.062+0000 7feee4b2a740 20 client.16395 trim_cache size 0 max 16384 2024-09-21T04:07:59.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.105+0000 7feee4b2a740 1 Processor -- start 2024-09-21T04:07:59.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.105+0000 7feee4b2a740 1 -- start start 2024-09-21T04:07:59.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.105+0000 7feee4b2a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e26725490 0x556e26afc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:59.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.105+0000 7feee4b2a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e2672d670 0x556e26b0d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:59.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.105+0000 7feee4b2a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e2674e350 0x556e26b014d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:59.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.105+0000 7feee4b2a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7feed01406a0 con 0x556e2672d670 2024-09-21T04:07:59.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.105+0000 7feee4b2a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feed013df60 con 0x556e26725490 2024-09-21T04:07:59.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.105+0000 7feee4b2a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feed00a68d0 con 0x556e2674e350 2024-09-21T04:07:59.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.106+0000 7feedcb53640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e2674e350 0x556e26b014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:59.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.106+0000 7feedcb53640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e2674e350 0x556e26b014d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:39102/0 (socket says 172.21.15.64:39102) 2024-09-21T04:07:59.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.106+0000 7feedcb53640 1 -- 172.21.15.64:0/2798355653 learned_addr learned my addr 172.21.15.64:0/2798355653 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:59.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.106+0000 7feed77fe640 1 --2- 172.21.15.64:0/2798355653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e2672d670 0x556e26b0d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:59.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.106+0000 7feed7fff640 1 --2- 172.21.15.64:0/2798355653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e26725490 0x556e26afc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:59.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.106+0000 7feedcb53640 1 -- 172.21.15.64:0/2798355653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e26725490 msgr2=0x556e26afc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:59.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.106+0000 7feedcb53640 1 --2- 172.21.15.64:0/2798355653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e26725490 0x556e26afc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.106+0000 7feedcb53640 1 -- 172.21.15.64:0/2798355653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e2672d670 msgr2=0x556e26b0d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:59.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.106+0000 7feedcb53640 1 --2- 172.21.15.64:0/2798355653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e2672d670 0x556e26b0d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.106+0000 7feedcb53640 1 -- 172.21.15.64:0/2798355653 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e26af7cf0 con 0x556e2674e350 2024-09-21T04:07:59.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.110+0000 7feedcb53640 1 --2- 172.21.15.64:0/2798355653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e2674e350 0x556e26b014d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7feed8052020 tx=0x7feed8075000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=833df1eea2b74c9b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:59.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.111+0000 7feec1ffb640 1 -- 172.21.15.64:0/2798355653 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8091070 con 0x556e2674e350 2024-09-21T04:07:59.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.112+0000 7feec1ffb640 1 -- 172.21.15.64:0/2798355653 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed8081dc0 con 0x556e2674e350 2024-09-21T04:07:59.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.113+0000 7feee4b2a740 1 -- 172.21.15.64:0/2798355653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e2674e350 msgr2=0x556e26b014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:59.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.113+0000 7feee4b2a740 1 --2- 172.21.15.64:0/2798355653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e2674e350 0x556e26b014d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7feed8052020 tx=0x7feed8075000 comp rx=0 tx=0).stop 2024-09-21T04:07:59.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.114+0000 7feee4b2a740 1 -- 172.21.15.64:0/2798355653 shutdown_connections 2024-09-21T04:07:59.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.114+0000 7feee4b2a740 1 --2- 172.21.15.64:0/2798355653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e2674e350 0x556e26b014d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.114+0000 7feee4b2a740 1 --2- 172.21.15.64:0/2798355653 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e2672d670 0x556e26b0d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.114+0000 7feee4b2a740 1 --2- 172.21.15.64:0/2798355653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e26725490 0x556e26afc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.114+0000 7feee4b2a740 1 -- 172.21.15.64:0/2798355653 >> 172.21.15.64:0/2798355653 conn(0x556e26742740 msgr2=0x556e26af1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:59.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.114+0000 7feee4b2a740 1 -- 172.21.15.64:0/2798355653 shutdown_connections 2024-09-21T04:07:59.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.114+0000 7feee4b2a740 1 -- 172.21.15.64:0/2798355653 wait complete. 2024-09-21T04:07:59.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.115+0000 7feee4b2a740 1 -- start start 2024-09-21T04:07:59.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.115+0000 7feee4b2a740 1 Processor -- start 2024-09-21T04:07:59.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.116+0000 7feee4b2a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e26725490 0x556e26b040f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:59.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.116+0000 7feee4b2a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e2672d670 0x556e26b04630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:59.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.116+0000 7feee4b2a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e2674e350 0x556e26b04b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:59.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.116+0000 7feee4b2a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7feed013c420 con 0x556e2672d670 2024-09-21T04:07:59.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.116+0000 7feee4b2a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feed013d890 con 0x556e26725490 2024-09-21T04:07:59.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.116+0000 7feee4b2a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feed013caf0 con 0x556e2674e350 2024-09-21T04:07:59.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.116+0000 7feee4b2a740 10 client.0 _ll_register_callbacks cb 0x7feed80515b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:07:59.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.116+0000 7feedcb53640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e2674e350 0x556e26b04b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:59.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.117+0000 7feedcb53640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e2674e350 0x556e26b04b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:39112/0 (socket says 172.21.15.64:39112) 2024-09-21T04:07:59.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.117+0000 7feedcb53640 1 -- 172.21.15.64:0/1672752996 learned_addr learned my addr 172.21.15.64:0/1672752996 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:07:59.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.117+0000 7feed7fff640 1 --2- 172.21.15.64:0/1672752996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e26725490 0x556e26b040f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:59.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.117+0000 7feed7fff640 1 -- 172.21.15.64:0/1672752996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e2674e350 msgr2=0x556e26b04b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:59.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.117+0000 7feed7fff640 1 --2- 172.21.15.64:0/1672752996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e2674e350 0x556e26b04b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.117+0000 7feed7fff640 1 -- 172.21.15.64:0/1672752996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e2672d670 msgr2=0x556e26b04630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:07:59.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.117+0000 7feed77fe640 1 --2- 172.21.15.64:0/1672752996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e2672d670 0x556e26b04630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:59.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.117+0000 7feed7fff640 1 --2- 172.21.15.64:0/1672752996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e2672d670 0x556e26b04630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.117+0000 7feed7fff640 1 -- 172.21.15.64:0/1672752996 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e26b9e290 con 0x556e26725490 2024-09-21T04:07:59.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.117+0000 7feed77fe640 1 --2- 172.21.15.64:0/1672752996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e2672d670 0x556e26b04630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:07:59.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.118+0000 7feed7fff640 1 --2- 172.21.15.64:0/1672752996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e26725490 0x556e26b040f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7feec40225f0 tx=0x7feec4005e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a4b84415e826e1e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:07:59.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.118+0000 7feeb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:07:59.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.118+0000 7feeb77fe640 1 -- 172.21.15.64:0/1672752996 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4012070 con 0x556e26725490 2024-09-21T04:07:59.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.118+0000 7feeb77fe640 1 -- 172.21.15.64:0/1672752996 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7feec4019f40 con 0x556e26725490 2024-09-21T04:07:59.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.118+0000 7feee4b2a740 1 -- 172.21.15.64:0/1672752996 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feed8081dc0 con 0x556e26725490 2024-09-21T04:07:59.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.120+0000 7feeb77fe640 1 -- 172.21.15.64:0/1672752996 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feec40190d0 con 0x556e26725490 2024-09-21T04:07:59.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.120+0000 7feee4b2a740 10 client.26096 fetch_fsmap learned FSMap version 32 2024-09-21T04:07:59.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.120+0000 7feee4b2a740 1 -- 172.21.15.64:0/1672752996 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556e26b9f520 con 0x556e26725490 2024-09-21T04:07:59.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.121+0000 7feeb77fe640 1 -- 172.21.15.64:0/1672752996 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed8081dc0 con 0x556e26725490 2024-09-21T04:07:59.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.121+0000 7feeb77fe640 1 -- 172.21.15.64:0/1672752996 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7feec40227f0 con 0x556e26725490 2024-09-21T04:07:59.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.121+0000 7feee4b2a740 10 client.26096 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T04:07:59.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.121+0000 7feee4b2a740 10 client.26096 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:07:59.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.121+0000 7feee4b2a740 20 client.26096 populate_metadata read hostname 'smithi064' 2024-09-21T04:07:59.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.121+0000 7feee4b2a740 1 --2- 172.21.15.64:0/1672752996 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556e266e2080 0x556e26ba1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:07:59.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.121+0000 7feee4b2a740 4 client.26096 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T04:07:59.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.121+0000 7feee4b2a740 1 -- 172.21.15.64:0/1672752996 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556e26a69860 con 0x556e266e2080 2024-09-21T04:07:59.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.122+0000 7feed77fe640 1 --2- 172.21.15.64:0/1672752996 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556e266e2080 0x556e26ba1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:07:59.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.123+0000 7feed77fe640 1 --2- 172.21.15.64:0/1672752996 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556e266e2080 0x556e26ba1c00 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bacc066de646f298 server_cookie=8eb616526e110947 in_seq=0 out_seq=0 2024-09-21T04:07:59.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.123+0000 7feeb77fe640 10 client.26096 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:59.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.128+0000 7feeb77fe640 1 -- 172.21.15.64:0/1672752996 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556e26a69860 con 0x556e266e2080 2024-09-21T04:07:59.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.128+0000 7feeb77fe640 10 client.26096 handle_command_reply: tid=0 2024-09-21T04:07:59.130 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.128+0000 7feee4b2a740 1 client.26096 shutdown 2024-09-21T04:07:59.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.128+0000 7feee4b2a740 1 -- 172.21.15.64:0/1672752996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e26725490 msgr2=0x556e26b040f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:59.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.128+0000 7feee4b2a740 1 --2- 172.21.15.64:0/1672752996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e26725490 0x556e26b040f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7feec40225f0 tx=0x7feec4005e00 comp rx=0 tx=0).stop 2024-09-21T04:07:59.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.128+0000 7feee4b2a740 1 -- 172.21.15.64:0/1672752996 shutdown_connections 2024-09-21T04:07:59.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.128+0000 7feee4b2a740 1 --2- 172.21.15.64:0/1672752996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556e2674e350 0x556e26b04b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.128+0000 7feee4b2a740 1 --2- 172.21.15.64:0/1672752996 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556e266e2080 0x556e26ba1c00 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.129+0000 7feee4b2a740 1 --2- 172.21.15.64:0/1672752996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556e2672d670 0x556e26b04630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.129+0000 7feee4b2a740 1 --2- 172.21.15.64:0/1672752996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556e26725490 0x556e26b040f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.129+0000 7feee4b2a740 1 -- 172.21.15.64:0/1672752996 >> 172.21.15.64:0/1672752996 conn(0x556e26742740 msgr2=0x556e26b02bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:59.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.129+0000 7feeb77fe640 0 client.26096 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:07:59.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.129+0000 7feee4b2a740 1 -- 172.21.15.64:0/1672752996 shutdown_connections 2024-09-21T04:07:59.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.129+0000 7feee4b2a740 1 -- 172.21.15.64:0/1672752996 wait complete. 2024-09-21T04:07:59.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.131+0000 7feee4b2a740 20 client.26096 trim_cache size 0 max 16384 2024-09-21T04:07:59.133 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:07:59.133 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-21T04:07:59.133 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "75db1bc5-612e-45aa-bad2-cc90feeaa840", 2024-09-21T04:07:59.133 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-21T04:07:59.133 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:07:59.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.132+0000 7feede5b6640 1 -- 172.21.15.64:0/2580526947 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7feec80b6880 msgr2=0x7feec80b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:59.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.132+0000 7feede5b6640 1 --2- 172.21.15.64:0/2580526947 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7feec80b6880 0x7feec80b8d90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feed8000b60 tx=0x7feed8069950 comp rx=0 tx=0).stop 2024-09-21T04:07:59.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.132+0000 7feede5b6640 1 -- 172.21.15.64:0/2580526947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feed00956b0 msgr2=0x7feed00a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:07:59.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.132+0000 7feede5b6640 1 --2- 172.21.15.64:0/2580526947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feed00956b0 0x7feed00a55d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feecc035040 tx=0x7feecc032ec0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.132+0000 7feede5b6640 1 -- 172.21.15.64:0/2580526947 shutdown_connections 2024-09-21T04:07:59.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.132+0000 7feede5b6640 1 --2- 172.21.15.64:0/2580526947 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7feec80b6880 0x7feec80b8d90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.132+0000 7feede5b6640 1 --2- 172.21.15.64:0/2580526947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feed009a1a0 0x7feed00a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.132+0000 7feede5b6640 1 --2- 172.21.15.64:0/2580526947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feed00956b0 0x7feed00a55d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.132+0000 7feede5b6640 1 --2- 172.21.15.64:0/2580526947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feed0094d60 0x7feed00a92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:07:59.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.133+0000 7feede5b6640 1 -- 172.21.15.64:0/2580526947 >> 172.21.15.64:0/2580526947 conn(0x7feed000cd70 msgr2=0x7feed0097bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:07:59.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.133+0000 7feede5b6640 1 -- 172.21.15.64:0/2580526947 shutdown_connections 2024-09-21T04:07:59.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:07:59.133+0000 7feede5b6640 1 -- 172.21.15.64:0/2580526947 wait complete. 2024-09-21T04:08:00.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:59 smithi119 ceph-mon[34811]: pgmap v440: 97 pgs: 97 active+clean; 537 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 510 op/s 2024-09-21T04:08:00.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:07:59 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:08:00.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:59 smithi064 ceph-mon[31180]: pgmap v440: 97 pgs: 97 active+clean; 537 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 510 op/s 2024-09-21T04:08:00.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:07:59 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:08:00.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:59 smithi131 ceph-mon[35193]: pgmap v440: 97 pgs: 97 active+clean; 537 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 510 op/s 2024-09-21T04:08:00.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:07:59 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:08:00.321 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T04:08:00.652 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:08:00.996 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:00 smithi064 ceph-mon[31180]: scrub queued for path: / 2024-09-21T04:08:00.996 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:00 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:00.996 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:00 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:08:00.996 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:00 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:08:00.996 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:00 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:00.996 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:00 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:00.996 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:00 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:08:01.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:00 smithi119 ceph-mon[34811]: scrub queued for path: / 2024-09-21T04:08:01.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:00 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:01.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:00 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:08:01.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:00 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:08:01.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:00 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:01.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:00 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:01.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:00 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:08:01.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:00 smithi131 ceph-mon[35193]: scrub queued for path: / 2024-09-21T04:08:01.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:00 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:01.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:00 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:08:01.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:00 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:08:01.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:00 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:01.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:00 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:01.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:00 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:08:01.173 INFO:tasks.workunit.client.0.smithi064.stdout: 92 1083933 374 766094 355 598420 1097 2024-09-21T04:08:01.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.227+0000 7f8fd570f640 1 -- 172.21.15.64:0/2361325922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fd01083a0 msgr2=0x7f8fd01087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:01.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.227+0000 7f8fd570f640 1 --2- 172.21.15.64:0/2361325922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fd01083a0 0x7f8fd01087b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009900 tx=0x7f8fb802a410 comp rx=0 tx=0).stop 2024-09-21T04:08:01.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.227+0000 7f8fd570f640 1 -- 172.21.15.64:0/2361325922 shutdown_connections 2024-09-21T04:08:01.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.227+0000 7f8fd570f640 1 --2- 172.21.15.64:0/2361325922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd0108cf0 0x7f8fd010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.227+0000 7f8fd570f640 1 --2- 172.21.15.64:0/2361325922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fd01083a0 0x7f8fd01087b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.227+0000 7f8fd570f640 1 --2- 172.21.15.64:0/2361325922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd0102a60 0x7f8fd0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.227+0000 7f8fd570f640 1 -- 172.21.15.64:0/2361325922 >> 172.21.15.64:0/2361325922 conn(0x7f8fd0079730 msgr2=0x7f8fd0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:01.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.227+0000 7f8fd570f640 1 -- 172.21.15.64:0/2361325922 shutdown_connections 2024-09-21T04:08:01.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.227+0000 7f8fd570f640 1 -- 172.21.15.64:0/2361325922 wait complete. 2024-09-21T04:08:01.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.228+0000 7f8fd570f640 1 Processor -- start 2024-09-21T04:08:01.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.228+0000 7f8fd570f640 1 -- start start 2024-09-21T04:08:01.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.228+0000 7f8fd570f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fd0102a60 0x7f8fd00ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:01.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.228+0000 7f8fd570f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd01083a0 0x7f8fd00ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:01.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.228+0000 7f8fd570f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd0108cf0 0x7f8fd01000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:01.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.228+0000 7f8fd570f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd0112960 con 0x7f8fd0108cf0 2024-09-21T04:08:01.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.228+0000 7f8fd570f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd01127e0 con 0x7f8fd01083a0 2024-09-21T04:08:01.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.228+0000 7f8fd570f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd0112ae0 con 0x7f8fd0102a60 2024-09-21T04:08:01.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.228+0000 7f8fceffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fd0102a60 0x7f8fd00ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:01.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.228+0000 7f8fc7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd01083a0 0x7f8fd00ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:01.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.229+0000 7f8fc7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd01083a0 0x7f8fd00ffbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:45300/0 (socket says 172.21.15.64:45300) 2024-09-21T04:08:01.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.229+0000 7f8fceffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fd0102a60 0x7f8fd00ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:39134/0 (socket says 172.21.15.64:39134) 2024-09-21T04:08:01.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.229+0000 7f8fc7fff640 1 -- 172.21.15.64:0/1168747957 learned_addr learned my addr 172.21.15.64:0/1168747957 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:01.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.229+0000 7f8fcf7fe640 1 --2- 172.21.15.64:0/1168747957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd0108cf0 0x7f8fd01000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:01.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.229+0000 7f8fceffd640 1 -- 172.21.15.64:0/1168747957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd01083a0 msgr2=0x7f8fd00ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:01.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.229+0000 7f8fceffd640 1 --2- 172.21.15.64:0/1168747957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd01083a0 0x7f8fd00ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.229+0000 7f8fceffd640 1 -- 172.21.15.64:0/1168747957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd0108cf0 msgr2=0x7f8fd01000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:01.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.229+0000 7f8fceffd640 1 --2- 172.21.15.64:0/1168747957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd0108cf0 0x7f8fd01000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.229+0000 7f8fceffd640 1 -- 172.21.15.64:0/1168747957 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd01a8720 con 0x7f8fd0102a60 2024-09-21T04:08:01.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.229+0000 7f8fceffd640 1 --2- 172.21.15.64:0/1168747957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fd0102a60 0x7f8fd00ff670 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000e9f0 tx=0x7f8fc000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:01.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.230+0000 7f8fccff9640 1 -- 172.21.15.64:0/1168747957 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc000ccc0 con 0x7f8fd0102a60 2024-09-21T04:08:01.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.230+0000 7f8fccff9640 1 -- 172.21.15.64:0/1168747957 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fc00108e0 con 0x7f8fd0102a60 2024-09-21T04:08:01.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.230+0000 7f8fd570f640 1 -- 172.21.15.64:0/1168747957 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd01a8a10 con 0x7f8fd0102a60 2024-09-21T04:08:01.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.230+0000 7f8fd570f640 1 -- 172.21.15.64:0/1168747957 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd01a8ee0 con 0x7f8fd0102a60 2024-09-21T04:08:01.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.230+0000 7f8fccff9640 1 -- 172.21.15.64:0/1168747957 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0018880 con 0x7f8fd0102a60 2024-09-21T04:08:01.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.233+0000 7f8fdbc83740 1 Processor -- start 2024-09-21T04:08:01.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.233+0000 7f8fdbc83740 1 -- start start 2024-09-21T04:08:01.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.234+0000 7f8fdbc83740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563bb1fff070 0x563bb23a6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:01.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.234+0000 7f8fdbc83740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb1fd61b0 0x563bb23a6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:01.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.234+0000 7f8fdbc83740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bb1fde390 0x563bb23a70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:01.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.234+0000 7f8fdbc83740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563bb231ce50 con 0x563bb1fd61b0 2024-09-21T04:08:01.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.234+0000 7f8fdbc83740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563bb230c280 con 0x563bb1fde390 2024-09-21T04:08:01.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.234+0000 7f8fdbc83740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x563bb2310090 con 0x563bb1fff070 2024-09-21T04:08:01.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.234+0000 7f8fccff9640 1 -- 172.21.15.64:0/1168747957 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f8fc0018a20 con 0x7f8fd0102a60 2024-09-21T04:08:01.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.234+0000 7f8fcf7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563bb1fff070 0x563bb23a6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:01.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.234+0000 7f8fcf7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563bb1fff070 0x563bb23a6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:39144/0 (socket says 172.21.15.64:39144) 2024-09-21T04:08:01.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.234+0000 7f8fcf7fe640 1 -- 172.21.15.64:0/1205022956 learned_addr learned my addr 172.21.15.64:0/1205022956 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:01.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.234+0000 7f8fc7fff640 1 --2- 172.21.15.64:0/1205022956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bb1fde390 0x563bb23a70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:01.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.234+0000 7f8fceffd640 1 --2- 172.21.15.64:0/1205022956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb1fd61b0 0x563bb23a6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:01.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.235+0000 7f8fcf7fe640 1 -- 172.21.15.64:0/1205022956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bb1fde390 msgr2=0x563bb23a70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:01.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.235+0000 7f8fcf7fe640 1 --2- 172.21.15.64:0/1205022956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bb1fde390 0x563bb23a70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.235+0000 7f8fcf7fe640 1 -- 172.21.15.64:0/1205022956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb1fd61b0 msgr2=0x563bb23a6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:01.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.235+0000 7f8fcf7fe640 1 --2- 172.21.15.64:0/1205022956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb1fd61b0 0x563bb23a6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.235+0000 7f8fcf7fe640 1 -- 172.21.15.64:0/1205022956 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bb206a650 con 0x563bb1fff070 2024-09-21T04:08:01.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.235+0000 7f8fcf7fe640 1 --2- 172.21.15.64:0/1205022956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563bb1fff070 0x563bb23a6650 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009fd0 tx=0x7f8fc80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2186179d4f113ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:01.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.235+0000 7f8fc57fa640 1 -- 172.21.15.64:0/1205022956 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8004d20 con 0x563bb1fff070 2024-09-21T04:08:01.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.235+0000 7f8fc57fa640 1 -- 172.21.15.64:0/1205022956 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fc8004050 con 0x563bb1fff070 2024-09-21T04:08:01.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.235+0000 7f8fc57fa640 1 -- 172.21.15.64:0/1205022956 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8005220 con 0x563bb1fff070 2024-09-21T04:08:01.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.235+0000 7f8fccff9640 1 --2- 172.21.15.64:0/1168747957 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8fbc0b6700 0x7f8fbc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:01.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.236+0000 7f8fceffd640 1 --2- 172.21.15.64:0/1168747957 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8fbc0b6700 0x7f8fbc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:01.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.236+0000 7f8fccff9640 1 -- 172.21.15.64:0/1168747957 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f8fc00e7a20 con 0x7f8fd0102a60 2024-09-21T04:08:01.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.236+0000 7f8fceffd640 1 --2- 172.21.15.64:0/1168747957 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8fbc0b6700 0x7f8fbc0b8c10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00e51f0 tx=0x7f8fc001f000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:01.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.236+0000 7f8fdbc83740 1 -- 172.21.15.64:0/1205022956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563bb1fff070 msgr2=0x563bb23a6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:01.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.236+0000 7f8fdbc83740 1 --2- 172.21.15.64:0/1205022956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563bb1fff070 0x563bb23a6650 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009fd0 tx=0x7f8fc80118d0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.237+0000 7f8fdbc83740 1 -- 172.21.15.64:0/1205022956 shutdown_connections 2024-09-21T04:08:01.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.237+0000 7f8fdbc83740 1 --2- 172.21.15.64:0/1205022956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bb1fde390 0x563bb23a70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.237+0000 7f8fdbc83740 1 --2- 172.21.15.64:0/1205022956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb1fd61b0 0x563bb23a6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.237+0000 7f8fdbc83740 1 --2- 172.21.15.64:0/1205022956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563bb1fff070 0x563bb23a6650 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.237+0000 7f8fdbc83740 1 -- 172.21.15.64:0/1205022956 >> 172.21.15.64:0/1205022956 conn(0x563bb1ff3460 msgr2=0x563bb23a4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:01.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.237+0000 7f8fdbc83740 1 -- 172.21.15.64:0/1205022956 shutdown_connections 2024-09-21T04:08:01.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.237+0000 7f8fdbc83740 1 -- 172.21.15.64:0/1205022956 wait complete. 2024-09-21T04:08:01.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.237+0000 7f8fdbc83740 1 -- start start 2024-09-21T04:08:01.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.238+0000 7f8fdbc83740 1 Processor -- start 2024-09-21T04:08:01.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.238+0000 7f8fdbc83740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bb1fd61b0 0x563bb23b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:01.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.238+0000 7f8fdbc83740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb1fde390 0x563bb23b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:01.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.238+0000 7f8fdbc83740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563bb1fff070 0x563bb23be120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:01.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.238+0000 7f8fcf7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bb1fd61b0 0x563bb23b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:01.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.238+0000 7f8fcf7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bb1fd61b0 0x563bb23b1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:45330/0 (socket says 172.21.15.64:45330) 2024-09-21T04:08:01.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.238+0000 7f8fc7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb1fde390 0x563bb23b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:01.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.238+0000 7f8fc7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb1fde390 0x563bb23b21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38970/0 (socket says 172.21.15.64:38970) 2024-09-21T04:08:01.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.238+0000 7f8fc7fff640 1 -- 172.21.15.64:0/1308102748 learned_addr learned my addr 172.21.15.64:0/1308102748 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:01.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.238+0000 7f8fdbc83740 1 -- 172.21.15.64:0/1308102748 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563bb216d700 con 0x563bb1fde390 2024-09-21T04:08:01.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.238+0000 7f8fdbc83740 1 -- 172.21.15.64:0/1308102748 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563bb22a7110 con 0x563bb1fd61b0 2024-09-21T04:08:01.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.238+0000 7f8fdbc83740 1 -- 172.21.15.64:0/1308102748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x563bb23b4e30 con 0x563bb1fff070 2024-09-21T04:08:01.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.238+0000 7f8fcf7fe640 1 --2- 172.21.15.64:0/1308102748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563bb1fff070 0x563bb23be120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:01.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.239+0000 7f8fc7fff640 1 -- 172.21.15.64:0/1308102748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563bb1fff070 msgr2=0x563bb23be120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:01.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.239+0000 7f8fc7fff640 1 --2- 172.21.15.64:0/1308102748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563bb1fff070 0x563bb23be120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.239+0000 7f8fc7fff640 1 -- 172.21.15.64:0/1308102748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bb1fd61b0 msgr2=0x563bb23b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:01.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.239+0000 7f8fc7fff640 1 --2- 172.21.15.64:0/1308102748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bb1fd61b0 0x563bb23b1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.239+0000 7f8fc7fff640 1 -- 172.21.15.64:0/1308102748 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bb22d4c40 con 0x563bb1fde390 2024-09-21T04:08:01.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.239+0000 7f8fcf7fe640 1 --2- 172.21.15.64:0/1308102748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563bb1fff070 0x563bb23be120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:08:01.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.239+0000 7f8fdbc83740 10 client.0 _ll_register_callbacks cb 0x563bb22c0990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:08:01.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.239+0000 7f8fdbc83740 1 -- 172.21.15.64:0/1308102748 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fc8004050 con 0x563bb1fde390 2024-09-21T04:08:01.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.239+0000 7f8fc7fff640 1 --2- 172.21.15.64:0/1308102748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb1fde390 0x563bb23b21e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009230 tx=0x7f8fb8031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9c82101bd062e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:01.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.239+0000 7f8fb67fc640 10 client.16410 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:08:01.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.239+0000 7f8fb67fc640 1 -- 172.21.15.64:0/1308102748 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8035040 con 0x563bb1fde390 2024-09-21T04:08:01.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.239+0000 7f8fb67fc640 1 -- 172.21.15.64:0/1308102748 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f8fb8031760 con 0x563bb1fde390 2024-09-21T04:08:01.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.239+0000 7f8fb67fc640 1 -- 172.21.15.64:0/1308102748 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8fb8004040 con 0x563bb1fde390 2024-09-21T04:08:01.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.239+0000 7f8fdbc83740 10 client.16410 fetch_fsmap learned FSMap version 32 2024-09-21T04:08:01.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.239+0000 7f8fdbc83740 1 -- 172.21.15.64:0/1308102748 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({fsmap=32}) -- 0x563bb23c0940 con 0x563bb1fde390 2024-09-21T04:08:01.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.240+0000 7f8fb67fc640 1 -- 172.21.15.64:0/1308102748 <== mon.0 v2:172.21.15.64:3300/0 4 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f8fb80041e0 con 0x563bb1fde390 2024-09-21T04:08:01.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.240+0000 7f8fdbc83740 10 client.16410 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T04:08:01.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.240+0000 7f8fdbc83740 10 client.16410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:08:01.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.240+0000 7f8fdbc83740 20 client.16410 populate_metadata read hostname 'smithi064' 2024-09-21T04:08:01.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.240+0000 7f8fdbc83740 1 --2- 172.21.15.64:0/1308102748 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x563bb1fbd9f0 0x563bb23c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:01.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.240+0000 7f8fdbc83740 4 client.16410 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:08:01.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.240+0000 7f8fdbc83740 1 -- 172.21.15.64:0/1308102748 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bb231eda0 con 0x563bb1fbd9f0 2024-09-21T04:08:01.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.240+0000 7f8fcf7fe640 1 --2- 172.21.15.64:0/1308102748 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x563bb1fbd9f0 0x563bb23c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:01.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.241+0000 7f8fb67fc640 10 client.16410 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:01.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.241+0000 7f8fcf7fe640 1 --2- 172.21.15.64:0/1308102748 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x563bb1fbd9f0 0x563bb23c3e50 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=848dc21f5b524d78 server_cookie=a46b47f6ecd8a698 in_seq=0 out_seq=0 2024-09-21T04:08:01.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.241+0000 7f8fb67fc640 1 -- 172.21.15.64:0/1308102748 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563bb231eda0 con 0x563bb1fbd9f0 2024-09-21T04:08:01.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.241+0000 7f8fb67fc640 10 client.16410 handle_command_reply: tid=0 2024-09-21T04:08:01.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.242+0000 7f8fdbc83740 1 client.16410 shutdown 2024-09-21T04:08:01.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.242+0000 7f8fdbc83740 1 -- 172.21.15.64:0/1308102748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb1fde390 msgr2=0x563bb23b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:01.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.242+0000 7f8fdbc83740 1 --2- 172.21.15.64:0/1308102748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb1fde390 0x563bb23b21e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009230 tx=0x7f8fb8031e60 comp rx=0 tx=0).stop 2024-09-21T04:08:01.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.242+0000 7f8fdbc83740 1 -- 172.21.15.64:0/1308102748 shutdown_connections 2024-09-21T04:08:01.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.242+0000 7f8fdbc83740 1 --2- 172.21.15.64:0/1308102748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563bb1fff070 0x563bb23be120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.242+0000 7f8fdbc83740 1 --2- 172.21.15.64:0/1308102748 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x563bb1fbd9f0 0x563bb23c3e50 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.242+0000 7f8fb67fc640 0 client.16410 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:01.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.242+0000 7f8fdbc83740 1 --2- 172.21.15.64:0/1308102748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb1fde390 0x563bb23b21e0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.242+0000 7f8fdbc83740 1 --2- 172.21.15.64:0/1308102748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bb1fd61b0 0x563bb23b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.242+0000 7f8fdbc83740 1 -- 172.21.15.64:0/1308102748 >> 172.21.15.64:0/1308102748 conn(0x563bb1ff3460 msgr2=0x563bb23a9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:01.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.242+0000 7f8fdbc83740 1 -- 172.21.15.64:0/1308102748 shutdown_connections 2024-09-21T04:08:01.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.242+0000 7f8fdbc83740 1 -- 172.21.15.64:0/1308102748 wait complete. 2024-09-21T04:08:01.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.242+0000 7f8fdbc83740 20 client.16410 trim_cache size 0 max 16384 2024-09-21T04:08:01.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.262+0000 7f8fdbc83740 1 Processor -- start 2024-09-21T04:08:01.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.262+0000 7f8fdbc83740 1 -- start start 2024-09-21T04:08:01.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.263+0000 7f8fdbc83740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bb1fbd9f0 0x563bb23b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:01.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.263+0000 7f8fdbc83740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb1fd61b0 0x563bb23be020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:01.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.263+0000 7f8fdbc83740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563bb1fde390 0x563bb23aee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:01.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.263+0000 7f8fdbc83740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd01affd0 con 0x563bb1fd61b0 2024-09-21T04:08:01.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.263+0000 7f8fdbc83740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd01ad810 con 0x563bb1fbd9f0 2024-09-21T04:08:01.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.263+0000 7f8fdbc83740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd01ab660 con 0x563bb1fde390 2024-09-21T04:08:01.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.263+0000 7f8fc7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb1fd61b0 0x563bb23be020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:01.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.263+0000 7f8fc7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb1fd61b0 0x563bb23be020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38978/0 (socket says 172.21.15.64:38978) 2024-09-21T04:08:01.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.263+0000 7f8fc7fff640 1 -- 172.21.15.64:0/1167971626 learned_addr learned my addr 172.21.15.64:0/1167971626 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:01.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.263+0000 7f8fcf7fe640 1 --2- 172.21.15.64:0/1167971626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bb1fbd9f0 0x563bb23b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:01.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.263+0000 7f8fcf7fe640 1 --2- 172.21.15.64:0/1167971626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563bb1fde390 0x563bb23aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:01.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.263+0000 7f8fc7fff640 1 -- 172.21.15.64:0/1167971626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563bb1fde390 msgr2=0x563bb23aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:01.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.263+0000 7f8fc7fff640 1 --2- 172.21.15.64:0/1167971626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563bb1fde390 0x563bb23aee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.263+0000 7f8fc7fff640 1 -- 172.21.15.64:0/1167971626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bb1fbd9f0 msgr2=0x563bb23b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:01.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.263+0000 7f8fc7fff640 1 --2- 172.21.15.64:0/1167971626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bb1fbd9f0 0x563bb23b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.263+0000 7f8fc7fff640 1 -- 172.21.15.64:0/1167971626 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bb23a94b0 con 0x563bb1fd61b0 2024-09-21T04:08:01.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.264+0000 7f8fc7fff640 1 --2- 172.21.15.64:0/1167971626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb1fd61b0 0x563bb23be020 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8000c00 tx=0x7f8fb8047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68e6fb24857bb9c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:01.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.264+0000 7f8fc4ff9640 1 -- 172.21.15.64:0/1167971626 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8035030 con 0x563bb1fd61b0 2024-09-21T04:08:01.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.264+0000 7f8fc4ff9640 1 -- 172.21.15.64:0/1167971626 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fc8004050 con 0x563bb1fd61b0 2024-09-21T04:08:01.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.264+0000 7f8fc4ff9640 1 -- 172.21.15.64:0/1167971626 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb80317c0 con 0x563bb1fd61b0 2024-09-21T04:08:01.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.265+0000 7f8fdbc83740 1 -- 172.21.15.64:0/1167971626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb1fd61b0 msgr2=0x563bb23be020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:01.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.265+0000 7f8fdbc83740 1 --2- 172.21.15.64:0/1167971626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb1fd61b0 0x563bb23be020 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8000c00 tx=0x7f8fb8047000 comp rx=0 tx=0).stop 2024-09-21T04:08:01.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.265+0000 7f8fdbc83740 1 -- 172.21.15.64:0/1167971626 shutdown_connections 2024-09-21T04:08:01.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.265+0000 7f8fdbc83740 1 --2- 172.21.15.64:0/1167971626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563bb1fde390 0x563bb23aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.265+0000 7f8fdbc83740 1 --2- 172.21.15.64:0/1167971626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb1fd61b0 0x563bb23be020 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.265+0000 7f8fdbc83740 1 --2- 172.21.15.64:0/1167971626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bb1fbd9f0 0x563bb23b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.265+0000 7f8fdbc83740 1 -- 172.21.15.64:0/1167971626 >> 172.21.15.64:0/1167971626 conn(0x563bb1ff3460 msgr2=0x563bb23b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:01.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.265+0000 7f8fdbc83740 1 -- 172.21.15.64:0/1167971626 shutdown_connections 2024-09-21T04:08:01.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.265+0000 7f8fdbc83740 1 -- 172.21.15.64:0/1167971626 wait complete. 2024-09-21T04:08:01.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.265+0000 7f8fdbc83740 1 -- start start 2024-09-21T04:08:01.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.266+0000 7f8fdbc83740 1 Processor -- start 2024-09-21T04:08:01.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.266+0000 7f8fdbc83740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb1fbd9f0 0x563bb23b6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:01.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.266+0000 7f8fdbc83740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bb1fd61b0 0x563bb23b67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:01.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.266+0000 7f8fdbc83740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563bb1fde390 0x563bb23b6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:01.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.266+0000 7f8fdbc83740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd01abcd0 con 0x563bb1fbd9f0 2024-09-21T04:08:01.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.266+0000 7f8fdbc83740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd01ad140 con 0x563bb1fd61b0 2024-09-21T04:08:01.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.266+0000 7f8fdbc83740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd01ac3a0 con 0x563bb1fde390 2024-09-21T04:08:01.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.266+0000 7f8fdbc83740 10 client.0 _ll_register_callbacks cb 0x7f8fb8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:08:01.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.266+0000 7f8fcf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb1fbd9f0 0x563bb23b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:01.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.266+0000 7f8fcf7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb1fbd9f0 0x563bb23b6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38992/0 (socket says 172.21.15.64:38992) 2024-09-21T04:08:01.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.266+0000 7f8fcf7fe640 1 -- 172.21.15.64:0/3197924412 learned_addr learned my addr 172.21.15.64:0/3197924412 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:01.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.266+0000 7f8fc7fff640 1 --2- 172.21.15.64:0/3197924412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bb1fd61b0 0x563bb23b67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:01.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.266+0000 7f8fcf7fe640 1 --2- 172.21.15.64:0/3197924412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563bb1fde390 0x563bb23b6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:01.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.266+0000 7f8fcf7fe640 1 -- 172.21.15.64:0/3197924412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563bb1fde390 msgr2=0x563bb23b6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:01.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.266+0000 7f8fcf7fe640 1 --2- 172.21.15.64:0/3197924412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563bb1fde390 0x563bb23b6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.266+0000 7f8fcf7fe640 1 -- 172.21.15.64:0/3197924412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bb1fd61b0 msgr2=0x563bb23b67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:01.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.266+0000 7f8fcf7fe640 1 --2- 172.21.15.64:0/3197924412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bb1fd61b0 0x563bb23b67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.266+0000 7f8fcf7fe640 1 -- 172.21.15.64:0/3197924412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bb242d9c0 con 0x563bb1fbd9f0 2024-09-21T04:08:01.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.267+0000 7f8fcf7fe640 1 --2- 172.21.15.64:0/3197924412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb1fbd9f0 0x563bb23b6260 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009560 tx=0x7f8fc80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999507f52829568c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:01.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.267+0000 7f8fb67fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:08:01.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.267+0000 7f8fb67fc640 1 -- 172.21.15.64:0/3197924412 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc800f4e0 con 0x563bb1fbd9f0 2024-09-21T04:08:01.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.267+0000 7f8fb67fc640 1 -- 172.21.15.64:0/3197924412 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f8fc800f680 con 0x563bb1fbd9f0 2024-09-21T04:08:01.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.267+0000 7f8fdbc83740 1 -- 172.21.15.64:0/3197924412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fc8004050 con 0x563bb1fbd9f0 2024-09-21T04:08:01.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.267+0000 7f8fb67fc640 1 -- 172.21.15.64:0/3197924412 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8fc8043050 con 0x563bb1fbd9f0 2024-09-21T04:08:01.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.267+0000 7f8fdbc83740 10 client.16416 fetch_fsmap learned FSMap version 32 2024-09-21T04:08:01.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.267+0000 7f8fdbc83740 1 -- 172.21.15.64:0/3197924412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563bb242ec50 con 0x563bb1fbd9f0 2024-09-21T04:08:01.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.267+0000 7f8fb67fc640 1 -- 172.21.15.64:0/3197924412 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fc8004050 con 0x563bb1fbd9f0 2024-09-21T04:08:01.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.268+0000 7f8fb67fc640 1 -- 172.21.15.64:0/3197924412 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f8fc8025070 con 0x563bb1fbd9f0 2024-09-21T04:08:01.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.268+0000 7f8fdbc83740 10 client.16416 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T04:08:01.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.268+0000 7f8fdbc83740 10 client.16416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:08:01.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.268+0000 7f8fdbc83740 20 client.16416 populate_metadata read hostname 'smithi064' 2024-09-21T04:08:01.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.268+0000 7f8fdbc83740 1 --2- 172.21.15.64:0/3197924412 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x563bb1fff070 0x563bb2431240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:01.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.268+0000 7f8fdbc83740 4 client.16416 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T04:08:01.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.268+0000 7f8fdbc83740 1 -- 172.21.15.64:0/3197924412 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563bb231a850 con 0x563bb1fff070 2024-09-21T04:08:01.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.268+0000 7f8fc7fff640 1 --2- 172.21.15.64:0/3197924412 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x563bb1fff070 0x563bb2431240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:01.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.268+0000 7f8fb67fc640 10 client.16416 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:01.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.268+0000 7f8fc7fff640 1 --2- 172.21.15.64:0/3197924412 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x563bb1fff070 0x563bb2431240 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=782ce772b76c83c2 server_cookie=139a36ba9495972b in_seq=0 out_seq=0 2024-09-21T04:08:01.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.269+0000 7f8fb67fc640 1 -- 172.21.15.64:0/3197924412 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2274 (crc 0 0 0) 0x563bb231a850 con 0x563bb1fff070 2024-09-21T04:08:01.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.269+0000 7f8fb67fc640 10 client.16416 handle_command_reply: tid=0 2024-09-21T04:08:01.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.269+0000 7f8fdbc83740 1 client.16416 shutdown 2024-09-21T04:08:01.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.269+0000 7f8fdbc83740 1 -- 172.21.15.64:0/3197924412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb1fbd9f0 msgr2=0x563bb23b6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:01.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.269+0000 7f8fdbc83740 1 --2- 172.21.15.64:0/3197924412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb1fbd9f0 0x563bb23b6260 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009560 tx=0x7f8fc80118d0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.269+0000 7f8fdbc83740 1 -- 172.21.15.64:0/3197924412 shutdown_connections 2024-09-21T04:08:01.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.269+0000 7f8fdbc83740 1 --2- 172.21.15.64:0/3197924412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563bb1fde390 0x563bb23b6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.269+0000 7f8fdbc83740 1 --2- 172.21.15.64:0/3197924412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563bb1fd61b0 0x563bb23b67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.269+0000 7f8fdbc83740 1 --2- 172.21.15.64:0/3197924412 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x563bb1fff070 0x563bb2431240 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.269+0000 7f8fdbc83740 1 --2- 172.21.15.64:0/3197924412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563bb1fbd9f0 0x563bb23b6260 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.269+0000 7f8fdbc83740 1 -- 172.21.15.64:0/3197924412 >> 172.21.15.64:0/3197924412 conn(0x563bb1ff3460 msgr2=0x563bb23a7640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:01.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.269+0000 7f8fb67fc640 0 client.16416 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:01.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.269+0000 7f8fdbc83740 1 -- 172.21.15.64:0/3197924412 shutdown_connections 2024-09-21T04:08:01.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.269+0000 7f8fdbc83740 1 -- 172.21.15.64:0/3197924412 wait complete. 2024-09-21T04:08:01.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.269+0000 7f8fdbc83740 20 client.16416 trim_cache size 0 max 16384 2024-09-21T04:08:01.276 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:08:01.276 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T04:08:01.276 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-09-21T04:08:01.276 INFO:teuthology.orchestra.run.smithi064.stdout: "75db1bc5-612e-45aa-bad2-cc90feeaa840": { 2024-09-21T04:08:01.276 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:08:01.276 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "75db1bc5-612e-45aa-bad2-cc90feeaa840", 2024-09-21T04:08:01.276 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-09-21T04:08:01.277 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:08:01.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:08:01.277 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_stats": { 2024-09-21T04:08:01.277 INFO:teuthology.orchestra.run.smithi064.stdout: "bed40d88-2102-43a6-ac2d-b89d5a2264d0": { 2024-09-21T04:08:01.277 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:11.363340+0000", 2024-09-21T04:08:01.277 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:08:01.277 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:08:01.277 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:08:01.277 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:08:01.277 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:08:01.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:08:01.277 INFO:teuthology.orchestra.run.smithi064.stdout: "e9242f88-6599-40eb-adf1-bdb3e946a1e0": { 2024-09-21T04:08:01.277 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:05:39.744646+0000", 2024-09-21T04:08:01.278 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:08:01.278 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:08:01.278 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:08:01.278 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:08:01.278 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:08:01.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:08:01.278 INFO:teuthology.orchestra.run.smithi064.stdout: "6bde0fe5-f775-44b1-ad0c-f9e596574dc0": { 2024-09-21T04:08:01.278 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:06:23.835536+0000", 2024-09-21T04:08:01.278 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:08:01.278 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:08:01.278 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:08:01.278 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:08:01.278 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:08:01.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:08:01.279 INFO:teuthology.orchestra.run.smithi064.stdout: "b52b950c-0f97-49ec-a37c-1a8e04348ce8": { 2024-09-21T04:08:01.279 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:54.003800+0000", 2024-09-21T04:08:01.279 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:08:01.279 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:08:01.279 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:08:01.279 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:08:01.279 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:08:01.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:08:01.279 INFO:teuthology.orchestra.run.smithi064.stdout: "75db1bc5-612e-45aa-bad2-cc90feeaa840": { 2024-09-21T04:08:01.279 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:59.123138+0000", 2024-09-21T04:08:01.279 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:08:01.279 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:08:01.279 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:08:01.279 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:08:01.280 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:08:01.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:08:01.280 INFO:teuthology.orchestra.run.smithi064.stdout: "0ab717a7-36c6-4dca-94e7-af0bdf2aa112": { 2024-09-21T04:08:01.280 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:42.691086+0000", 2024-09-21T04:08:01.280 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:08:01.280 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:08:01.280 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:08:01.280 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:08:01.280 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:08:01.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:08:01.280 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:08:01.280 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:03:59.570174+0000", 2024-09-21T04:08:01.280 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:08:01.280 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:08:01.281 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:08:01.281 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:08:01.281 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:08:01.281 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:08:01.281 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:08:01.281 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:08:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.270+0000 7f8fd570f640 1 -- 172.21.15.64:0/1168747957 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8fbc0b6700 msgr2=0x7f8fbc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.270+0000 7f8fd570f640 1 --2- 172.21.15.64:0/1168747957 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8fbc0b6700 0x7f8fbc0b8c10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00e51f0 tx=0x7f8fc001f000 comp rx=0 tx=0).stop 2024-09-21T04:08:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.270+0000 7f8fd570f640 1 -- 172.21.15.64:0/1168747957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fd0102a60 msgr2=0x7f8fd00ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.270+0000 7f8fd570f640 1 --2- 172.21.15.64:0/1168747957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fd0102a60 0x7f8fd00ff670 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000e9f0 tx=0x7f8fc000eeb0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.270+0000 7f8fd570f640 1 -- 172.21.15.64:0/1168747957 shutdown_connections 2024-09-21T04:08:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.270+0000 7f8fd570f640 1 --2- 172.21.15.64:0/1168747957 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8fbc0b6700 0x7f8fbc0b8c10 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.270+0000 7f8fd570f640 1 --2- 172.21.15.64:0/1168747957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fd0108cf0 0x7f8fd01000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.270+0000 7f8fd570f640 1 --2- 172.21.15.64:0/1168747957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fd01083a0 0x7f8fd00ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.270+0000 7f8fd570f640 1 --2- 172.21.15.64:0/1168747957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8fd0102a60 0x7f8fd00ff670 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.270+0000 7f8fd570f640 1 -- 172.21.15.64:0/1168747957 >> 172.21.15.64:0/1168747957 conn(0x7f8fd0079730 msgr2=0x7f8fd006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.270+0000 7f8fd570f640 1 -- 172.21.15.64:0/1168747957 shutdown_connections 2024-09-21T04:08:01.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:01.270+0000 7f8fd570f640 1 -- 172.21.15.64:0/1168747957 wait complete. 2024-09-21T04:08:01.848 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:01 smithi064 ceph-mon[31180]: pgmap v441: 97 pgs: 97 active+clean; 537 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 460 op/s 2024-09-21T04:08:01.848 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:01 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:01.849 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:01 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:08:02.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:01 smithi119 ceph-mon[34811]: pgmap v441: 97 pgs: 97 active+clean; 537 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 460 op/s 2024-09-21T04:08:02.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:01 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:02.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:01 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:08:02.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:01 smithi131 ceph-mon[35193]: pgmap v441: 97 pgs: 97 active+clean; 537 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 460 op/s 2024-09-21T04:08:02.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:01 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:02.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:01 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:08:02.431 INFO:tasks.cephfs.filesystem:scrub status for tag:75db1bc5-612e-45aa-bad2-cc90feeaa840 - {'path': '/', 'tag': '75db1bc5-612e-45aa-bad2-cc90feeaa840', 'options': 'recursive,force'} 2024-09-21T04:08:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.594+0000 7f443fbda640 1 -- 172.21.15.64:0/4215717383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4438108c50 msgr2=0x7f443810f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.594+0000 7f443fbda640 1 --2- 172.21.15.64:0/4215717383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4438108c50 0x7f443810f480 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4430009ca0 tx=0x7f443002a8c0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.594+0000 7f443fbda640 1 -- 172.21.15.64:0/4215717383 shutdown_connections 2024-09-21T04:08:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.594+0000 7f443fbda640 1 --2- 172.21.15.64:0/4215717383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4438108c50 0x7f443810f480 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.594+0000 7f443fbda640 1 --2- 172.21.15.64:0/4215717383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4438108300 0x7f4438108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.594+0000 7f443fbda640 1 --2- 172.21.15.64:0/4215717383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44381029d0 0x7f4438102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.594+0000 7f443fbda640 1 -- 172.21.15.64:0/4215717383 >> 172.21.15.64:0/4215717383 conn(0x7f443807a6f0 msgr2=0x7f443807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.594+0000 7f443fbda640 1 -- 172.21.15.64:0/4215717383 shutdown_connections 2024-09-21T04:08:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.594+0000 7f443fbda640 1 -- 172.21.15.64:0/4215717383 wait complete. 2024-09-21T04:08:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.595+0000 7f443fbda640 1 Processor -- start 2024-09-21T04:08:02.598 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.595+0000 7f443fbda640 1 -- start start 2024-09-21T04:08:02.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.595+0000 7f443fbda640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44381029d0 0x7f44381990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:02.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.595+0000 7f443fbda640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4438108300 0x7f44381995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:02.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.595+0000 7f443fbda640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4438108c50 0x7f4438199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:02.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.595+0000 7f443fbda640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4438112910 con 0x7f4438108c50 2024-09-21T04:08:02.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.595+0000 7f443fbda640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4438112790 con 0x7f4438108300 2024-09-21T04:08:02.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.595+0000 7f443fbda640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4438112a90 con 0x7f44381029d0 2024-09-21T04:08:02.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.596+0000 7f443e177640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4438108c50 0x7f4438199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:02.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.596+0000 7f443d976640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44381029d0 0x7f44381990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:02.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.596+0000 7f443e177640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4438108c50 0x7f4438199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46112/0 (socket says 172.21.15.64:46112) 2024-09-21T04:08:02.600 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.596+0000 7f443e177640 1 -- 172.21.15.64:0/3412849355 learned_addr learned my addr 172.21.15.64:0/3412849355 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.596+0000 7f443d976640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44381029d0 0x7f44381990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:35012/0 (socket says 172.21.15.64:35012) 2024-09-21T04:08:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.596+0000 7f443d175640 1 --2- 172.21.15.64:0/3412849355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4438108300 0x7f44381995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.596+0000 7f443e177640 1 -- 172.21.15.64:0/3412849355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44381029d0 msgr2=0x7f44381990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.596+0000 7f443e177640 1 --2- 172.21.15.64:0/3412849355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44381029d0 0x7f44381990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.596+0000 7f443e177640 1 -- 172.21.15.64:0/3412849355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4438108300 msgr2=0x7f44381995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.596+0000 7f443e177640 1 --2- 172.21.15.64:0/3412849355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4438108300 0x7f44381995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.596+0000 7f443e177640 1 -- 172.21.15.64:0/3412849355 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44381a8760 con 0x7f4438108c50 2024-09-21T04:08:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.596+0000 7f443e177640 1 --2- 172.21.15.64:0/3412849355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4438108c50 0x7f4438199b30 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4430005cc0 tx=0x7f4430031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.596+0000 7f442effd640 1 -- 172.21.15.64:0/3412849355 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4430003b70 con 0x7f4438108c50 2024-09-21T04:08:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.596+0000 7f442effd640 1 -- 172.21.15.64:0/3412849355 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44300318f0 con 0x7f4438108c50 2024-09-21T04:08:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.596+0000 7f443fbda640 1 -- 172.21.15.64:0/3412849355 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44381a89f0 con 0x7f4438108c50 2024-09-21T04:08:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.596+0000 7f442effd640 1 -- 172.21.15.64:0/3412849355 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443002f950 con 0x7f4438108c50 2024-09-21T04:08:02.601 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.596+0000 7f443fbda640 1 -- 172.21.15.64:0/3412849355 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44381a8f30 con 0x7f4438108c50 2024-09-21T04:08:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.600+0000 7f4446c21740 1 Processor -- start 2024-09-21T04:08:02.602 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.600+0000 7f442effd640 1 -- 172.21.15.64:0/3412849355 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f443002fb50 con 0x7f4438108c50 2024-09-21T04:08:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.600+0000 7f4446c21740 1 -- start start 2024-09-21T04:08:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.600+0000 7f4446c21740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9bbb951b0 0x55d9bbf3a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.600+0000 7f4446c21740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb6c2f0 0x55d9bbf3aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.600+0000 7f4446c21740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9bbb744d0 0x55d9bbf3b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:02.604 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.600+0000 7f4446c21740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d9bbeaf6f0 con 0x55d9bbb6c2f0 2024-09-21T04:08:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.600+0000 7f4446c21740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d9bbe9eb20 con 0x55d9bbb744d0 2024-09-21T04:08:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.600+0000 7f4446c21740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d9bbea2930 con 0x55d9bbb951b0 2024-09-21T04:08:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.601+0000 7f443d976640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb6c2f0 0x55d9bbf3aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.601+0000 7f443d175640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9bbb951b0 0x55d9bbf3a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.601+0000 7f443d976640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb6c2f0 0x55d9bbf3aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46116/0 (socket says 172.21.15.64:46116) 2024-09-21T04:08:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.601+0000 7f443d976640 1 -- 172.21.15.64:0/2249405097 learned_addr learned my addr 172.21.15.64:0/2249405097 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.601+0000 7f443d175640 1 --2- 172.21.15.64:0/2249405097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9bbb744d0 0x55d9bbf3b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.601+0000 7f442effd640 1 --2- 172.21.15.64:0/3412849355 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f44240b6700 0x7f44240b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.601+0000 7f443d976640 1 -- 172.21.15.64:0/2249405097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9bbb951b0 msgr2=0x55d9bbf3a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.601+0000 7f443d976640 1 --2- 172.21.15.64:0/2249405097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9bbb951b0 0x55d9bbf3a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.601+0000 7f443d976640 1 -- 172.21.15.64:0/2249405097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9bbb744d0 msgr2=0x55d9bbf3b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.601+0000 7f443d976640 1 --2- 172.21.15.64:0/2249405097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9bbb744d0 0x55d9bbf3b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.601+0000 7f443d976640 1 -- 172.21.15.64:0/2249405097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9bbe64530 con 0x55d9bbb6c2f0 2024-09-21T04:08:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.601+0000 7f443e177640 1 --2- 172.21.15.64:0/3412849355 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f44240b6700 0x7f44240b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.601+0000 7f443d976640 1 --2- 172.21.15.64:0/2249405097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb6c2f0 0x55d9bbf3aef0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4434009ca0 tx=0x7f4434011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21820553388919f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.601+0000 7f442effd640 1 -- 172.21.15.64:0/3412849355 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f4430108860 con 0x7f4438108c50 2024-09-21T04:08:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.601+0000 7f4423fff640 1 -- 172.21.15.64:0/2249405097 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4434009e50 con 0x55d9bbb6c2f0 2024-09-21T04:08:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.602+0000 7f4423fff640 1 -- 172.21.15.64:0/2249405097 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44340054d0 con 0x55d9bbb6c2f0 2024-09-21T04:08:02.606 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.602+0000 7f4423fff640 1 -- 172.21.15.64:0/2249405097 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4434019670 con 0x55d9bbb6c2f0 2024-09-21T04:08:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.602+0000 7f4446c21740 1 -- 172.21.15.64:0/2249405097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb6c2f0 msgr2=0x55d9bbf3aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.602+0000 7f443e177640 1 --2- 172.21.15.64:0/3412849355 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f44240b6700 0x7f44240b8c10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f44300cac50 tx=0x7f44300cadb0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.602+0000 7f4446c21740 1 --2- 172.21.15.64:0/2249405097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb6c2f0 0x55d9bbf3aef0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4434009ca0 tx=0x7f4434011810 comp rx=0 tx=0).stop 2024-09-21T04:08:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.602+0000 7f4446c21740 1 -- 172.21.15.64:0/2249405097 shutdown_connections 2024-09-21T04:08:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.602+0000 7f4446c21740 1 --2- 172.21.15.64:0/2249405097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9bbb744d0 0x55d9bbf3b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.610 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.602+0000 7f4446c21740 1 --2- 172.21.15.64:0/2249405097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb6c2f0 0x55d9bbf3aef0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.602+0000 7f4446c21740 1 --2- 172.21.15.64:0/2249405097 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9bbb951b0 0x55d9bbf3a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.602+0000 7f4446c21740 1 -- 172.21.15.64:0/2249405097 >> 172.21.15.64:0/2249405097 conn(0x55d9bbb89300 msgr2=0x55d9bbf388a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.603+0000 7f4446c21740 1 -- 172.21.15.64:0/2249405097 shutdown_connections 2024-09-21T04:08:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.603+0000 7f4446c21740 1 -- 172.21.15.64:0/2249405097 wait complete. 2024-09-21T04:08:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.603+0000 7f4446c21740 1 -- start start 2024-09-21T04:08:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.603+0000 7f4446c21740 1 Processor -- start 2024-09-21T04:08:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.604+0000 7f4446c21740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9bbb6c2f0 0x55d9bbf46020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.604+0000 7f4446c21740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb744d0 0x55d9bbf46560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.604+0000 7f4446c21740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9bbb951b0 0x55d9bbf52420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.604+0000 7f4446c21740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d9bbdd6ab0 con 0x55d9bbb744d0 2024-09-21T04:08:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.604+0000 7f4446c21740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d9bbe44730 con 0x55d9bbb6c2f0 2024-09-21T04:08:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.604+0000 7f4446c21740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d9bbf49130 con 0x55d9bbb951b0 2024-09-21T04:08:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.604+0000 7f4446c21740 10 client.0 _ll_register_callbacks cb 0x55d9bbe986b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:08:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.604+0000 7f443d976640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb744d0 0x55d9bbf46560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.604+0000 7f443d976640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb744d0 0x55d9bbf46560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46120/0 (socket says 172.21.15.64:46120) 2024-09-21T04:08:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.604+0000 7f443d175640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9bbb6c2f0 0x55d9bbf46020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.604+0000 7f443d976640 1 -- 172.21.15.64:0/1571432203 learned_addr learned my addr 172.21.15.64:0/1571432203 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.604+0000 7f443d175640 1 --2- 172.21.15.64:0/1571432203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9bbb951b0 0x55d9bbf52420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.604+0000 7f443d976640 1 -- 172.21.15.64:0/1571432203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9bbb951b0 msgr2=0x55d9bbf52420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.604+0000 7f443d976640 1 --2- 172.21.15.64:0/1571432203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9bbb951b0 0x55d9bbf52420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.604+0000 7f443d976640 1 -- 172.21.15.64:0/1571432203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9bbb6c2f0 msgr2=0x55d9bbf46020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.604+0000 7f443d976640 1 --2- 172.21.15.64:0/1571432203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9bbb6c2f0 0x55d9bbf46020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.604+0000 7f443d976640 1 -- 172.21.15.64:0/1571432203 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9bbc00790 con 0x55d9bbb744d0 2024-09-21T04:08:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.604+0000 7f443d175640 1 --2- 172.21.15.64:0/1571432203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9bbb6c2f0 0x55d9bbf46020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.604+0000 7f443d976640 1 --2- 172.21.15.64:0/1571432203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb744d0 0x55d9bbf46560 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4434012fd0 tx=0x7f4434005010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806ba5bc34efad5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.604+0000 7f44217fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:08:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.605+0000 7f44217fa640 1 -- 172.21.15.64:0/1571432203 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4434028070 con 0x55d9bbb744d0 2024-09-21T04:08:02.612 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.605+0000 7f44217fa640 1 -- 172.21.15.64:0/1571432203 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f4434024440 con 0x55d9bbb744d0 2024-09-21T04:08:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.605+0000 7f4446c21740 1 -- 172.21.15.64:0/1571432203 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44340054d0 con 0x55d9bbb744d0 2024-09-21T04:08:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.605+0000 7f44217fa640 1 -- 172.21.15.64:0/1571432203 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4434034050 con 0x55d9bbb744d0 2024-09-21T04:08:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.605+0000 7f4446c21740 10 client.16428 fetch_fsmap learned FSMap version 32 2024-09-21T04:08:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.605+0000 7f4446c21740 1 -- 172.21.15.64:0/1571432203 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d9bbc79620 con 0x55d9bbb744d0 2024-09-21T04:08:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.605+0000 7f44217fa640 1 -- 172.21.15.64:0/1571432203 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44340054d0 con 0x55d9bbb744d0 2024-09-21T04:08:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.605+0000 7f44217fa640 1 -- 172.21.15.64:0/1571432203 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f4434022410 con 0x55d9bbb744d0 2024-09-21T04:08:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.605+0000 7f4446c21740 10 client.16428 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T04:08:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.605+0000 7f4446c21740 10 client.16428 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:08:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.605+0000 7f4446c21740 20 client.16428 populate_metadata read hostname 'smithi064' 2024-09-21T04:08:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.605+0000 7f4446c21740 1 --2- 172.21.15.64:0/1571432203 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d9bbb53d80 0x55d9bbf58050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.605+0000 7f4446c21740 4 client.16428 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:08:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.605+0000 7f4446c21740 1 -- 172.21.15.64:0/1571432203 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9bbeb1640 con 0x55d9bbb53d80 2024-09-21T04:08:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.606+0000 7f443d175640 1 --2- 172.21.15.64:0/1571432203 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d9bbb53d80 0x55d9bbf58050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:02.613 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.606+0000 7f443d175640 1 --2- 172.21.15.64:0/1571432203 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d9bbb53d80 0x55d9bbf58050 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=126be151077be8f6 server_cookie=48fad27f723968d0 in_seq=0 out_seq=0 2024-09-21T04:08:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.606+0000 7f44217fa640 10 client.16428 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.607+0000 7f44217fa640 1 -- 172.21.15.64:0/1571432203 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d9bbeb1640 con 0x55d9bbb53d80 2024-09-21T04:08:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.607+0000 7f44217fa640 10 client.16428 handle_command_reply: tid=0 2024-09-21T04:08:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.607+0000 7f4446c21740 1 client.16428 shutdown 2024-09-21T04:08:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.607+0000 7f4446c21740 1 -- 172.21.15.64:0/1571432203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb744d0 msgr2=0x55d9bbf46560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.607+0000 7f4446c21740 1 --2- 172.21.15.64:0/1571432203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb744d0 0x55d9bbf46560 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4434012fd0 tx=0x7f4434005010 comp rx=0 tx=0).stop 2024-09-21T04:08:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.607+0000 7f4446c21740 1 -- 172.21.15.64:0/1571432203 shutdown_connections 2024-09-21T04:08:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.607+0000 7f4446c21740 1 --2- 172.21.15.64:0/1571432203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9bbb951b0 0x55d9bbf52420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.607+0000 7f4446c21740 1 --2- 172.21.15.64:0/1571432203 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d9bbb53d80 0x55d9bbf58050 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.607+0000 7f4446c21740 1 --2- 172.21.15.64:0/1571432203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb744d0 0x55d9bbf46560 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.607+0000 7f4446c21740 1 --2- 172.21.15.64:0/1571432203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9bbb6c2f0 0x55d9bbf46020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.607+0000 7f44217fa640 0 client.16428 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.607+0000 7f4446c21740 1 -- 172.21.15.64:0/1571432203 >> 172.21.15.64:0/1571432203 conn(0x55d9bbb89300 msgr2=0x55d9bbf3b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.607+0000 7f4446c21740 1 -- 172.21.15.64:0/1571432203 shutdown_connections 2024-09-21T04:08:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.607+0000 7f4446c21740 1 -- 172.21.15.64:0/1571432203 wait complete. 2024-09-21T04:08:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.607+0000 7f4446c21740 20 client.16428 trim_cache size 0 max 16384 2024-09-21T04:08:02.627 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.624+0000 7f4446c21740 1 Processor -- start 2024-09-21T04:08:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.624+0000 7f4446c21740 1 -- start start 2024-09-21T04:08:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.624+0000 7f4446c21740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9bbb6c2f0 0x55d9bbf3c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.624+0000 7f4446c21740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb744d0 0x55d9bbf52320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.624+0000 7f4446c21740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9bbb951b0 0x55d9bbf479f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.624+0000 7f4446c21740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f44381b4390 con 0x55d9bbb744d0 2024-09-21T04:08:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.624+0000 7f4446c21740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f44381b1bd0 con 0x55d9bbb6c2f0 2024-09-21T04:08:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.624+0000 7f4446c21740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f44381afa20 con 0x55d9bbb951b0 2024-09-21T04:08:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.624+0000 7f443d976640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb744d0 0x55d9bbf52320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.624+0000 7f443d976640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb744d0 0x55d9bbf52320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46128/0 (socket says 172.21.15.64:46128) 2024-09-21T04:08:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.624+0000 7f443d976640 1 -- 172.21.15.64:0/1024331132 learned_addr learned my addr 172.21.15.64:0/1024331132 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.624+0000 7f443d175640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9bbb951b0 0x55d9bbf479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.624+0000 7f443d175640 1 --2- 172.21.15.64:0/1024331132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9bbb6c2f0 0x55d9bbf3c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.624+0000 7f443d976640 1 -- 172.21.15.64:0/1024331132 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9bbb951b0 msgr2=0x55d9bbf479f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.624+0000 7f443d976640 1 --2- 172.21.15.64:0/1024331132 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9bbb951b0 0x55d9bbf479f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.624+0000 7f443d976640 1 -- 172.21.15.64:0/1024331132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9bbb6c2f0 msgr2=0x55d9bbf3c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.624+0000 7f443d976640 1 --2- 172.21.15.64:0/1024331132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9bbb6c2f0 0x55d9bbf3c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.624+0000 7f443d976640 1 -- 172.21.15.64:0/1024331132 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9bbf3d7c0 con 0x55d9bbb744d0 2024-09-21T04:08:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.625+0000 7f443d976640 1 --2- 172.21.15.64:0/1024331132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb744d0 0x55d9bbf52320 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f44340059f0 tx=0x7f4434005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fddd6ab5afa8479 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.625+0000 7f44237fe640 1 -- 172.21.15.64:0/1024331132 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4434031070 con 0x55d9bbb744d0 2024-09-21T04:08:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.625+0000 7f44237fe640 1 -- 172.21.15.64:0/1024331132 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4434009e50 con 0x55d9bbb744d0 2024-09-21T04:08:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.625+0000 7f44237fe640 1 -- 172.21.15.64:0/1024331132 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4434028040 con 0x55d9bbb744d0 2024-09-21T04:08:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.626+0000 7f4446c21740 1 -- 172.21.15.64:0/1024331132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb744d0 msgr2=0x55d9bbf52320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.626+0000 7f4446c21740 1 --2- 172.21.15.64:0/1024331132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb744d0 0x55d9bbf52320 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f44340059f0 tx=0x7f4434005b90 comp rx=0 tx=0).stop 2024-09-21T04:08:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.626+0000 7f4446c21740 1 -- 172.21.15.64:0/1024331132 shutdown_connections 2024-09-21T04:08:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.626+0000 7f4446c21740 1 --2- 172.21.15.64:0/1024331132 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9bbb951b0 0x55d9bbf479f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.626+0000 7f4446c21740 1 --2- 172.21.15.64:0/1024331132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb744d0 0x55d9bbf52320 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.626+0000 7f4446c21740 1 --2- 172.21.15.64:0/1024331132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9bbb6c2f0 0x55d9bbf3c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.626+0000 7f4446c21740 1 -- 172.21.15.64:0/1024331132 >> 172.21.15.64:0/1024331132 conn(0x55d9bbb89300 msgr2=0x55d9bbf472e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.626+0000 7f4446c21740 1 -- 172.21.15.64:0/1024331132 shutdown_connections 2024-09-21T04:08:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.626+0000 7f4446c21740 1 -- 172.21.15.64:0/1024331132 wait complete. 2024-09-21T04:08:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.626+0000 7f4446c21740 1 -- start start 2024-09-21T04:08:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.626+0000 7f4446c21740 1 Processor -- start 2024-09-21T04:08:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.627+0000 7f4446c21740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9bbb6c2f0 0x55d9bbf4b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.627+0000 7f4446c21740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb744d0 0x55d9bbf4bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.627+0000 7f4446c21740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9bbb951b0 0x55d9bbfc0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.627+0000 7f4446c21740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f44381b0090 con 0x55d9bbb744d0 2024-09-21T04:08:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.627+0000 7f4446c21740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f44381b1500 con 0x55d9bbb951b0 2024-09-21T04:08:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.627+0000 7f4446c21740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f44381b0760 con 0x55d9bbb6c2f0 2024-09-21T04:08:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.627+0000 7f4446c21740 10 client.0 _ll_register_callbacks cb 0x7f4428009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:08:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.627+0000 7f443d976640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb744d0 0x55d9bbf4bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.627+0000 7f443d976640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb744d0 0x55d9bbf4bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46132/0 (socket says 172.21.15.64:46132) 2024-09-21T04:08:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.627+0000 7f443d175640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9bbb6c2f0 0x55d9bbf4b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.627+0000 7f443d976640 1 -- 172.21.15.64:0/135823355 learned_addr learned my addr 172.21.15.64:0/135823355 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.627+0000 7f443d175640 1 --2- 172.21.15.64:0/135823355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9bbb951b0 0x55d9bbfc0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.627+0000 7f443d976640 1 -- 172.21.15.64:0/135823355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9bbb6c2f0 msgr2=0x55d9bbf4b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.627+0000 7f443d976640 1 --2- 172.21.15.64:0/135823355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9bbb6c2f0 0x55d9bbf4b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.627+0000 7f443d976640 1 -- 172.21.15.64:0/135823355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9bbb951b0 msgr2=0x55d9bbfc0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.627+0000 7f443d976640 1 --2- 172.21.15.64:0/135823355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9bbb951b0 0x55d9bbfc0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.627+0000 7f443d976640 1 -- 172.21.15.64:0/135823355 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9bbfc0e70 con 0x55d9bbb744d0 2024-09-21T04:08:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.627+0000 7f443d175640 1 --2- 172.21.15.64:0/135823355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9bbb6c2f0 0x55d9bbf4b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.628+0000 7f443d976640 1 --2- 172.21.15.64:0/135823355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb744d0 0x55d9bbf4bd60 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4434005050 tx=0x7f4434005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea4b2eae32cc9ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.628+0000 7f44217fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:08:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.628+0000 7f44217fa640 1 -- 172.21.15.64:0/135823355 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4434031070 con 0x55d9bbb744d0 2024-09-21T04:08:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.628+0000 7f4446c21740 1 -- 172.21.15.64:0/135823355 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4434009e50 con 0x55d9bbb744d0 2024-09-21T04:08:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.628+0000 7f44217fa640 1 -- 172.21.15.64:0/135823355 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f4434022950 con 0x55d9bbb744d0 2024-09-21T04:08:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.628+0000 7f44217fa640 1 -- 172.21.15.64:0/135823355 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4434028070 con 0x55d9bbb744d0 2024-09-21T04:08:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.628+0000 7f4446c21740 10 client.16434 fetch_fsmap learned FSMap version 32 2024-09-21T04:08:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.628+0000 7f4446c21740 1 -- 172.21.15.64:0/135823355 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d9bbfc2010 con 0x55d9bbb744d0 2024-09-21T04:08:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.628+0000 7f44217fa640 1 -- 172.21.15.64:0/135823355 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4434009e50 con 0x55d9bbb744d0 2024-09-21T04:08:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.629+0000 7f44217fa640 1 -- 172.21.15.64:0/135823355 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f4434046020 con 0x55d9bbb744d0 2024-09-21T04:08:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.629+0000 7f4446c21740 10 client.16434 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T04:08:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.629+0000 7f4446c21740 10 client.16434 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:08:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.629+0000 7f4446c21740 20 client.16434 populate_metadata read hostname 'smithi064' 2024-09-21T04:08:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.629+0000 7f4446c21740 1 --2- 172.21.15.64:0/135823355 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d9bbb291c0 0x55d9bbfc4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.629+0000 7f4446c21740 4 client.16434 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:08:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.629+0000 7f4446c21740 1 -- 172.21.15.64:0/135823355 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d9bbead0f0 con 0x55d9bbb291c0 2024-09-21T04:08:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.629+0000 7f443d175640 1 --2- 172.21.15.64:0/135823355 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d9bbb291c0 0x55d9bbfc4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.629+0000 7f44217fa640 10 client.16434 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:02.629+0000 7f443d175640 1 --2- 172.21.15.64:0/135823355 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d9bbb291c0 0x55d9bbfc4660 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6202ac25b2dea01e server_cookie=a3c2c8c7830ed556 in_seq=0 out_seq=0 2024-09-21T04:08:03.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:02 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:03.049+0000 7f44217fa640 1 -- 172.21.15.64:0/135823355 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d9bbead0f0 con 0x55d9bbb291c0 2024-09-21T04:08:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:03.049+0000 7f44217fa640 10 client.16434 handle_command_reply: tid=0 2024-09-21T04:08:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:03.049+0000 7f4446c21740 1 client.16434 shutdown 2024-09-21T04:08:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:03.049+0000 7f4446c21740 1 -- 172.21.15.64:0/135823355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb744d0 msgr2=0x55d9bbf4bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:03.050+0000 7f4446c21740 1 --2- 172.21.15.64:0/135823355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb744d0 0x55d9bbf4bd60 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4434005050 tx=0x7f4434005c00 comp rx=0 tx=0).stop 2024-09-21T04:08:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:03.050+0000 7f4446c21740 1 -- 172.21.15.64:0/135823355 shutdown_connections 2024-09-21T04:08:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:03.050+0000 7f4446c21740 1 --2- 172.21.15.64:0/135823355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9bbb951b0 0x55d9bbfc0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:03.050+0000 7f4446c21740 1 --2- 172.21.15.64:0/135823355 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d9bbb291c0 0x55d9bbfc4660 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:03.050+0000 7f44217fa640 0 client.16434 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:03.050+0000 7f4446c21740 1 --2- 172.21.15.64:0/135823355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d9bbb744d0 0x55d9bbf4bd60 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:03.050+0000 7f4446c21740 1 --2- 172.21.15.64:0/135823355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9bbb6c2f0 0x55d9bbf4b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:03.050+0000 7f4446c21740 1 -- 172.21.15.64:0/135823355 >> 172.21.15.64:0/135823355 conn(0x55d9bbb89300 msgr2=0x55d9bbf3c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:03.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:02 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:03.050+0000 7f4446c21740 1 -- 172.21.15.64:0/135823355 shutdown_connections 2024-09-21T04:08:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:03.051+0000 7f4446c21740 1 -- 172.21.15.64:0/135823355 wait complete. 2024-09-21T04:08:03.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:03.051+0000 7f4446c21740 20 client.16434 trim_cache size 0 max 16384 2024-09-21T04:08:03.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:08:03.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:08:03.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:08:03.055 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:08:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:03.052+0000 7f443fbda640 1 -- 172.21.15.64:0/3412849355 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f44240b6700 msgr2=0x7f44240b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:03.052+0000 7f443fbda640 1 --2- 172.21.15.64:0/3412849355 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f44240b6700 0x7f44240b8c10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f44300cac50 tx=0x7f44300cadb0 comp rx=0 tx=0).stop 2024-09-21T04:08:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:03.052+0000 7f443fbda640 1 -- 172.21.15.64:0/3412849355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4438108c50 msgr2=0x7f4438199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:03.052+0000 7f443fbda640 1 --2- 172.21.15.64:0/3412849355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4438108c50 0x7f4438199b30 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4430005cc0 tx=0x7f4430031700 comp rx=0 tx=0).stop 2024-09-21T04:08:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:03.052+0000 7f443fbda640 1 -- 172.21.15.64:0/3412849355 shutdown_connections 2024-09-21T04:08:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:03.052+0000 7f443fbda640 1 --2- 172.21.15.64:0/3412849355 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f44240b6700 0x7f44240b8c10 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:03.052+0000 7f443fbda640 1 --2- 172.21.15.64:0/3412849355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4438108c50 0x7f4438199b30 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:03.052+0000 7f443fbda640 1 --2- 172.21.15.64:0/3412849355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4438108300 0x7f44381995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:03.052+0000 7f443fbda640 1 --2- 172.21.15.64:0/3412849355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f44381029d0 0x7f44381990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:03.052+0000 7f443fbda640 1 -- 172.21.15.64:0/3412849355 >> 172.21.15.64:0/3412849355 conn(0x7f443807a6f0 msgr2=0x7f44381063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:03.052+0000 7f443fbda640 1 -- 172.21.15.64:0/3412849355 shutdown_connections 2024-09-21T04:08:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:03.052+0000 7f443fbda640 1 -- 172.21.15.64:0/3412849355 wait complete. 2024-09-21T04:08:03.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:02 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:04.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:03 smithi064 ceph-mon[31180]: pgmap v442: 97 pgs: 97 active+clean; 547 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 712 op/s 2024-09-21T04:08:04.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:03 smithi131 ceph-mon[35193]: pgmap v442: 97 pgs: 97 active+clean; 547 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 712 op/s 2024-09-21T04:08:04.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:03 smithi119 ceph-mon[34811]: pgmap v442: 97 pgs: 97 active+clean; 547 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 712 op/s 2024-09-21T04:08:06.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:05 smithi131 ceph-mon[35193]: pgmap v443: 97 pgs: 97 active+clean; 547 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 526 op/s 2024-09-21T04:08:06.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:05 smithi119 ceph-mon[34811]: pgmap v443: 97 pgs: 97 active+clean; 547 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 526 op/s 2024-09-21T04:08:06.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:05 smithi064 ceph-mon[31180]: pgmap v443: 97 pgs: 97 active+clean; 547 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 526 op/s 2024-09-21T04:08:08.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:07 smithi131 ceph-mon[35193]: pgmap v444: 97 pgs: 97 active+clean; 591 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.6 MiB/s wr, 955 op/s 2024-09-21T04:08:08.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:07 smithi119 ceph-mon[34811]: pgmap v444: 97 pgs: 97 active+clean; 591 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.6 MiB/s wr, 955 op/s 2024-09-21T04:08:08.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:07 smithi064 ceph-mon[31180]: pgmap v444: 97 pgs: 97 active+clean; 591 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.6 MiB/s wr, 955 op/s 2024-09-21T04:08:10.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:09 smithi131 ceph-mon[35193]: pgmap v445: 97 pgs: 97 active+clean; 610 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.1 MiB/s wr, 925 op/s 2024-09-21T04:08:10.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:09 smithi119 ceph-mon[34811]: pgmap v445: 97 pgs: 97 active+clean; 610 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.1 MiB/s wr, 925 op/s 2024-09-21T04:08:10.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:09 smithi064 ceph-mon[31180]: pgmap v445: 97 pgs: 97 active+clean; 610 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.1 MiB/s wr, 925 op/s 2024-09-21T04:08:11.173 INFO:tasks.workunit.client.0.smithi064.stdout: 92 48354 3 32843 1 25019 6 2024-09-21T04:08:12.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:11 smithi131 ceph-mon[35193]: pgmap v446: 97 pgs: 97 active+clean; 610 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.8 MiB/s wr, 898 op/s 2024-09-21T04:08:12.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:11 smithi119 ceph-mon[34811]: pgmap v446: 97 pgs: 97 active+clean; 610 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.8 MiB/s wr, 898 op/s 2024-09-21T04:08:12.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:11 smithi064 ceph-mon[31180]: pgmap v446: 97 pgs: 97 active+clean; 610 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.8 MiB/s wr, 898 op/s 2024-09-21T04:08:14.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:13 smithi131 ceph-mon[35193]: pgmap v447: 97 pgs: 97 active+clean; 596 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.2 MiB/s wr, 1.14k op/s 2024-09-21T04:08:14.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:13 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:08:14.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:13 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:08:14.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:13 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:08:14.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:13 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:08:14.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:13 smithi119 ceph-mon[34811]: pgmap v447: 97 pgs: 97 active+clean; 596 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.2 MiB/s wr, 1.14k op/s 2024-09-21T04:08:14.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:13 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:08:14.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:13 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:08:14.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:13 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:08:14.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:13 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:08:14.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:13 smithi064 ceph-mon[31180]: pgmap v447: 97 pgs: 97 active+clean; 596 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 8.2 MiB/s wr, 1.14k op/s 2024-09-21T04:08:14.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:13 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:08:14.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:13 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:08:14.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:13 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:08:14.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:13 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:08:16.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:15 smithi131 ceph-mon[35193]: pgmap v448: 97 pgs: 97 active+clean; 596 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.0 MiB/s wr, 878 op/s 2024-09-21T04:08:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.163+0000 7f3354121640 1 -- 172.21.15.64:0/2624261734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f334c108c30 msgr2=0x7f334c10f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.163+0000 7f3354121640 1 --2- 172.21.15.64:0/2624261734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f334c108c30 0x7f334c10f460 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3344009be0 tx=0x7f334402a6c0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.163+0000 7f3354121640 1 -- 172.21.15.64:0/2624261734 shutdown_connections 2024-09-21T04:08:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.163+0000 7f3354121640 1 --2- 172.21.15.64:0/2624261734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f334c108c30 0x7f334c10f460 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.163+0000 7f3354121640 1 --2- 172.21.15.64:0/2624261734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f334c1082e0 0x7f334c1086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.163+0000 7f3354121640 1 --2- 172.21.15.64:0/2624261734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f334c1029b0 0x7f334c102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.163+0000 7f3354121640 1 -- 172.21.15.64:0/2624261734 >> 172.21.15.64:0/2624261734 conn(0x7f334c07a740 msgr2=0x7f334c07ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.163+0000 7f3354121640 1 -- 172.21.15.64:0/2624261734 shutdown_connections 2024-09-21T04:08:16.166 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.163+0000 7f3354121640 1 -- 172.21.15.64:0/2624261734 wait complete. 2024-09-21T04:08:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.163+0000 7f3354121640 1 Processor -- start 2024-09-21T04:08:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.163+0000 7f3354121640 1 -- start start 2024-09-21T04:08:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.164+0000 7f3354121640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f334c1029b0 0x7f334c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.164+0000 7f3354121640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f334c1082e0 0x7f334c199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.164+0000 7f3354121640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f334c108c30 0x7f334c19eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.164+0000 7f3354121640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1128f0 con 0x7f334c1082e0 2024-09-21T04:08:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.164+0000 7f3354121640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f334c112770 con 0x7f334c1029b0 2024-09-21T04:08:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.164+0000 7f3354121640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f334c112a70 con 0x7f334c108c30 2024-09-21T04:08:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.164+0000 7f33516bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f334c1082e0 0x7f334c199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.164+0000 7f33516bc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f334c1082e0 0x7f334c199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48636/0 (socket says 172.21.15.64:48636) 2024-09-21T04:08:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.164+0000 7f33516bc640 1 -- 172.21.15.64:0/3670235850 learned_addr learned my addr 172.21.15.64:0/3670235850 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.164+0000 7f3351ebd640 1 --2- 172.21.15.64:0/3670235850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f334c1029b0 0x7f334c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.164+0000 7f33526be640 1 --2- 172.21.15.64:0/3670235850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f334c108c30 0x7f334c19eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.164+0000 7f33516bc640 1 -- 172.21.15.64:0/3670235850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f334c108c30 msgr2=0x7f334c19eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.164+0000 7f33516bc640 1 --2- 172.21.15.64:0/3670235850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f334c108c30 0x7f334c19eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.164+0000 7f33516bc640 1 -- 172.21.15.64:0/3670235850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f334c1029b0 msgr2=0x7f334c1992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.164+0000 7f33516bc640 1 --2- 172.21.15.64:0/3670235850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f334c1029b0 0x7f334c1992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.164+0000 7f33516bc640 1 -- 172.21.15.64:0/3670235850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f334c19f1e0 con 0x7f334c1082e0 2024-09-21T04:08:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.164+0000 7f3351ebd640 1 --2- 172.21.15.64:0/3670235850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f334c1029b0 0x7f334c1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.165+0000 7f33516bc640 1 --2- 172.21.15.64:0/3670235850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f334c1082e0 0x7f334c199850 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f334000e970 tx=0x7f334000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.165+0000 7f334affd640 1 -- 172.21.15.64:0/3670235850 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334000ccc0 con 0x7f334c1082e0 2024-09-21T04:08:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.165+0000 7f334affd640 1 -- 172.21.15.64:0/3670235850 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33400035f0 con 0x7f334c1082e0 2024-09-21T04:08:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.165+0000 7f3354121640 1 -- 172.21.15.64:0/3670235850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f334c19f4d0 con 0x7f334c1082e0 2024-09-21T04:08:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.165+0000 7f334affd640 1 -- 172.21.15.64:0/3670235850 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334000ce60 con 0x7f334c1082e0 2024-09-21T04:08:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.165+0000 7f3354121640 1 -- 172.21.15.64:0/3670235850 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f334c1a5c60 con 0x7f334c1082e0 2024-09-21T04:08:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.168+0000 7f335b088740 1 Processor -- start 2024-09-21T04:08:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.168+0000 7f335b088740 1 -- start start 2024-09-21T04:08:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.168+0000 7f335b088740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7d4d9c1b0 0x55a7d51419b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:16.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.168+0000 7f335b088740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7d4d732f0 0x55a7d5141ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:16.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.169+0000 7f335b088740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7d4d7b4d0 0x55a7d5142430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:16.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.169+0000 7f335b088740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a7d50b66f0 con 0x55a7d4d7b4d0 2024-09-21T04:08:16.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.169+0000 7f335b088740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a7d50a5b20 con 0x55a7d4d9c1b0 2024-09-21T04:08:16.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.169+0000 7f335b088740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a7d50a9930 con 0x55a7d4d732f0 2024-09-21T04:08:16.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.169+0000 7f33526be640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7d4d9c1b0 0x55a7d51419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:16.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.169+0000 7f3351ebd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7d4d732f0 0x55a7d5141ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:16.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.169+0000 7f33526be640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7d4d9c1b0 0x55a7d51419b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:57302/0 (socket says 172.21.15.64:57302) 2024-09-21T04:08:16.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.169+0000 7f33526be640 1 -- 172.21.15.64:0/4069467110 learned_addr learned my addr 172.21.15.64:0/4069467110 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:16.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.169+0000 7f3351ebd640 1 --2- 172.21.15.64:0/4069467110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7d4d732f0 0x55a7d5141ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:52360/0 (socket says 172.21.15.64:52360) 2024-09-21T04:08:16.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.169+0000 7f334affd640 1 -- 172.21.15.64:0/3670235850 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f3340010690 con 0x7f334c1082e0 2024-09-21T04:08:16.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.169+0000 7f3351ebd640 1 -- 172.21.15.64:0/4069467110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7d4d9c1b0 msgr2=0x55a7d51419b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:16.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.169+0000 7f3351ebd640 1 --2- 172.21.15.64:0/4069467110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7d4d9c1b0 0x55a7d51419b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.169+0000 7f3351ebd640 1 -- 172.21.15.64:0/4069467110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7d4d7b4d0 msgr2=0x55a7d5142430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.169+0000 7f33516bc640 1 --2- 172.21.15.64:0/4069467110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7d4d7b4d0 0x55a7d5142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.169+0000 7f3351ebd640 1 --2- 172.21.15.64:0/4069467110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7d4d7b4d0 0x55a7d5142430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.169+0000 7f3351ebd640 1 -- 172.21.15.64:0/4069467110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7d506b530 con 0x55a7d4d732f0 2024-09-21T04:08:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.169+0000 7f33516bc640 1 --2- 172.21.15.64:0/4069467110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7d4d7b4d0 0x55a7d5142430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:08:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.170+0000 7f3351ebd640 1 --2- 172.21.15.64:0/4069467110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7d4d732f0 0x55a7d5141ef0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f333c011470 tx=0x7f333c011930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79b86c508ebe45de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.170+0000 7f3333fff640 1 -- 172.21.15.64:0/4069467110 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c01e070 con 0x55a7d4d732f0 2024-09-21T04:08:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.170+0000 7f3333fff640 1 -- 172.21.15.64:0/4069467110 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f333c0035f0 con 0x55a7d4d732f0 2024-09-21T04:08:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.170+0000 7f3333fff640 1 -- 172.21.15.64:0/4069467110 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c00f840 con 0x55a7d4d732f0 2024-09-21T04:08:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.170+0000 7f334affd640 1 --2- 172.21.15.64:0/3670235850 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f33340b6680 0x7f33340b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.170+0000 7f33526be640 1 --2- 172.21.15.64:0/3670235850 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f33340b6680 0x7f33340b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.170+0000 7f334affd640 1 -- 172.21.15.64:0/3670235850 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f33400e7a00 con 0x7f334c1082e0 2024-09-21T04:08:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.171+0000 7f335b088740 1 -- 172.21.15.64:0/4069467110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7d4d732f0 msgr2=0x55a7d5141ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.171+0000 7f335b088740 1 --2- 172.21.15.64:0/4069467110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7d4d732f0 0x55a7d5141ef0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f333c011470 tx=0x7f333c011930 comp rx=0 tx=0).stop 2024-09-21T04:08:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.171+0000 7f33526be640 1 --2- 172.21.15.64:0/3670235850 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f33340b6680 0x7f33340b8b90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3344006010 tx=0x7f3344004a40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.171+0000 7f335b088740 1 -- 172.21.15.64:0/4069467110 shutdown_connections 2024-09-21T04:08:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.171+0000 7f335b088740 1 --2- 172.21.15.64:0/4069467110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7d4d7b4d0 0x55a7d5142430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.171+0000 7f335b088740 1 --2- 172.21.15.64:0/4069467110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7d4d732f0 0x55a7d5141ef0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.171+0000 7f335b088740 1 --2- 172.21.15.64:0/4069467110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7d4d9c1b0 0x55a7d51419b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.171+0000 7f335b088740 1 -- 172.21.15.64:0/4069467110 >> 172.21.15.64:0/4069467110 conn(0x55a7d4d90300 msgr2=0x55a7d513f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.171+0000 7f335b088740 1 -- 172.21.15.64:0/4069467110 shutdown_connections 2024-09-21T04:08:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.171+0000 7f335b088740 1 -- 172.21.15.64:0/4069467110 wait complete. 2024-09-21T04:08:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.172+0000 7f335b088740 1 -- start start 2024-09-21T04:08:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.172+0000 7f335b088740 1 Processor -- start 2024-09-21T04:08:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.173+0000 7f335b088740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7d4d732f0 0x55a7d5152810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.173+0000 7f335b088740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7d4d7b4d0 0x55a7d5152d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.173+0000 7f335b088740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7d4d9c1b0 0x55a7d515d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.173+0000 7f335b088740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a7d4fddab0 con 0x55a7d4d7b4d0 2024-09-21T04:08:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.173+0000 7f335b088740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a7d504b730 con 0x55a7d4d9c1b0 2024-09-21T04:08:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.173+0000 7f335b088740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a7d5152340 con 0x55a7d4d732f0 2024-09-21T04:08:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.173+0000 7f335b088740 10 client.0 _ll_register_callbacks cb 0x55a7d509f6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:08:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.173+0000 7f3351ebd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7d4d732f0 0x55a7d5152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.173+0000 7f3351ebd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7d4d732f0 0x55a7d5152810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:52366/0 (socket says 172.21.15.64:52366) 2024-09-21T04:08:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.173+0000 7f3351ebd640 1 -- 172.21.15.64:0/1699130799 learned_addr learned my addr 172.21.15.64:0/1699130799 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.173+0000 7f33526be640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7d4d9c1b0 0x55a7d515d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.173+0000 7f33516bc640 1 --2- 172.21.15.64:0/1699130799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7d4d7b4d0 0x55a7d5152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.173+0000 7f3351ebd640 1 -- 172.21.15.64:0/1699130799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7d4d9c1b0 msgr2=0x55a7d515d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.173+0000 7f3351ebd640 1 --2- 172.21.15.64:0/1699130799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7d4d9c1b0 0x55a7d515d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.173+0000 7f3351ebd640 1 -- 172.21.15.64:0/1699130799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7d4d7b4d0 msgr2=0x55a7d5152d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.173+0000 7f3351ebd640 1 --2- 172.21.15.64:0/1699130799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7d4d7b4d0 0x55a7d5152d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.173+0000 7f3351ebd640 1 -- 172.21.15.64:0/1699130799 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7d4e07790 con 0x55a7d4d732f0 2024-09-21T04:08:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.174+0000 7f3351ebd640 1 --2- 172.21.15.64:0/1699130799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7d4d732f0 0x55a7d5152810 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f333c011e30 tx=0x7f333c005df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcec2b34b6640f6c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.174+0000 7f33317fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-21T04:08:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.174+0000 7f33317fa640 1 -- 172.21.15.64:0/1699130799 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c027060 con 0x55a7d4d732f0 2024-09-21T04:08:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.174+0000 7f33317fa640 1 -- 172.21.15.64:0/1699130799 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f333c003860 con 0x55a7d4d732f0 2024-09-21T04:08:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.174+0000 7f335b088740 1 -- 172.21.15.64:0/1699130799 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f333c0035f0 con 0x55a7d4d732f0 2024-09-21T04:08:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.174+0000 7f33317fa640 1 -- 172.21.15.64:0/1699130799 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f333c01e040 con 0x55a7d4d732f0 2024-09-21T04:08:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.174+0000 7f335b088740 10 client.26035 fetch_fsmap learned FSMap version 32 2024-09-21T04:08:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.174+0000 7f335b088740 1 -- 172.21.15.64:0/1699130799 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a7d4e80620 con 0x55a7d4d732f0 2024-09-21T04:08:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.175+0000 7f33317fa640 1 -- 172.21.15.64:0/1699130799 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f333c0054e0 con 0x55a7d4d732f0 2024-09-21T04:08:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.175+0000 7f33317fa640 1 -- 172.21.15.64:0/1699130799 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f333c021030 con 0x55a7d4d732f0 2024-09-21T04:08:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.175+0000 7f335b088740 10 client.26035 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T04:08:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.175+0000 7f335b088740 10 client.26035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:08:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.175+0000 7f335b088740 20 client.26035 populate_metadata read hostname 'smithi064' 2024-09-21T04:08:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.175+0000 7f335b088740 1 --2- 172.21.15.64:0/1699130799 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a7d4d5ad80 0x55a7d5163400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.175+0000 7f335b088740 4 client.26035 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:08:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.175+0000 7f335b088740 1 -- 172.21.15.64:0/1699130799 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f333c0035f0 con 0x55a7d4d5ad80 2024-09-21T04:08:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.175+0000 7f33516bc640 1 --2- 172.21.15.64:0/1699130799 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a7d4d5ad80 0x55a7d5163400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.176+0000 7f33317fa640 10 client.26035 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.176+0000 7f33516bc640 1 --2- 172.21.15.64:0/1699130799 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a7d4d5ad80 0x55a7d5163400 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=952d0a25ab3883e6 server_cookie=baa605d5e8fcfe81 in_seq=0 out_seq=0 2024-09-21T04:08:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.177+0000 7f33317fa640 1 -- 172.21.15.64:0/1699130799 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f333c0035f0 con 0x55a7d4d5ad80 2024-09-21T04:08:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.177+0000 7f33317fa640 10 client.26035 handle_command_reply: tid=0 2024-09-21T04:08:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.177+0000 7f335b088740 1 client.26035 shutdown 2024-09-21T04:08:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.177+0000 7f335b088740 1 -- 172.21.15.64:0/1699130799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7d4d732f0 msgr2=0x55a7d5152810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.177+0000 7f335b088740 1 --2- 172.21.15.64:0/1699130799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7d4d732f0 0x55a7d5152810 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f333c011e30 tx=0x7f333c005df0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.177+0000 7f335b088740 1 -- 172.21.15.64:0/1699130799 shutdown_connections 2024-09-21T04:08:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.177+0000 7f335b088740 1 --2- 172.21.15.64:0/1699130799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7d4d9c1b0 0x55a7d515d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.177+0000 7f335b088740 1 --2- 172.21.15.64:0/1699130799 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a7d4d5ad80 0x55a7d5163400 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.177+0000 7f335b088740 1 --2- 172.21.15.64:0/1699130799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7d4d7b4d0 0x55a7d5152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.177+0000 7f33317fa640 0 client.26035 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.177+0000 7f335b088740 1 --2- 172.21.15.64:0/1699130799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7d4d732f0 0x55a7d5152810 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.177+0000 7f335b088740 1 -- 172.21.15.64:0/1699130799 >> 172.21.15.64:0/1699130799 conn(0x55a7d4d90300 msgr2=0x55a7d51429e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.177+0000 7f335b088740 1 -- 172.21.15.64:0/1699130799 shutdown_connections 2024-09-21T04:08:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.177+0000 7f335b088740 1 -- 172.21.15.64:0/1699130799 wait complete. 2024-09-21T04:08:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.178+0000 7f335b088740 20 client.26035 trim_cache size 0 max 16384 2024-09-21T04:08:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.194+0000 7f335b088740 1 Processor -- start 2024-09-21T04:08:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.194+0000 7f335b088740 1 -- start start 2024-09-21T04:08:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.194+0000 7f335b088740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7d4d732f0 0x55a7d515dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.194+0000 7f335b088740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7d4d7b4d0 0x55a7d5152580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.194+0000 7f335b088740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7d4d9c1b0 0x55a7d5152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.194+0000 7f335b088740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1b0000 con 0x55a7d4d732f0 2024-09-21T04:08:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.194+0000 7f335b088740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1ad840 con 0x55a7d4d7b4d0 2024-09-21T04:08:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.194+0000 7f335b088740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1ab690 con 0x55a7d4d9c1b0 2024-09-21T04:08:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.195+0000 7f3351ebd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7d4d732f0 0x55a7d515dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.195+0000 7f3351ebd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7d4d732f0 0x55a7d515dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48656/0 (socket says 172.21.15.64:48656) 2024-09-21T04:08:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.195+0000 7f3351ebd640 1 -- 172.21.15.64:0/2831862199 learned_addr learned my addr 172.21.15.64:0/2831862199 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.195+0000 7f33516bc640 1 --2- 172.21.15.64:0/2831862199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7d4d7b4d0 0x55a7d5152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.195+0000 7f33526be640 1 --2- 172.21.15.64:0/2831862199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7d4d9c1b0 0x55a7d5152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.195+0000 7f3351ebd640 1 -- 172.21.15.64:0/2831862199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7d4d9c1b0 msgr2=0x55a7d5152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.195+0000 7f3351ebd640 1 --2- 172.21.15.64:0/2831862199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7d4d9c1b0 0x55a7d5152ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.195+0000 7f3351ebd640 1 -- 172.21.15.64:0/2831862199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7d4d7b4d0 msgr2=0x55a7d5152580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.195+0000 7f3351ebd640 1 --2- 172.21.15.64:0/2831862199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7d4d7b4d0 0x55a7d5152580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.195+0000 7f3351ebd640 1 -- 172.21.15.64:0/2831862199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7d514f970 con 0x55a7d4d732f0 2024-09-21T04:08:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.195+0000 7f3351ebd640 1 --2- 172.21.15.64:0/2831862199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7d4d732f0 0x55a7d515dbb0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f333c025040 tx=0x7f333c004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c10ca76c4757ffba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.195+0000 7f33337fe640 1 -- 172.21.15.64:0/2831862199 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c027040 con 0x55a7d4d732f0 2024-09-21T04:08:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.195+0000 7f33337fe640 1 -- 172.21.15.64:0/2831862199 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f333c0035f0 con 0x55a7d4d732f0 2024-09-21T04:08:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.195+0000 7f33337fe640 1 -- 172.21.15.64:0/2831862199 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333c031080 con 0x55a7d4d732f0 2024-09-21T04:08:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.196+0000 7f335b088740 1 -- 172.21.15.64:0/2831862199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7d4d732f0 msgr2=0x55a7d515dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.196+0000 7f335b088740 1 --2- 172.21.15.64:0/2831862199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7d4d732f0 0x55a7d515dbb0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f333c025040 tx=0x7f333c004380 comp rx=0 tx=0).stop 2024-09-21T04:08:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.196+0000 7f335b088740 1 -- 172.21.15.64:0/2831862199 shutdown_connections 2024-09-21T04:08:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.196+0000 7f335b088740 1 --2- 172.21.15.64:0/2831862199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7d4d9c1b0 0x55a7d5152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.196+0000 7f335b088740 1 --2- 172.21.15.64:0/2831862199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7d4d7b4d0 0x55a7d5152580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.196+0000 7f335b088740 1 --2- 172.21.15.64:0/2831862199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7d4d732f0 0x55a7d515dbb0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.196+0000 7f335b088740 1 -- 172.21.15.64:0/2831862199 >> 172.21.15.64:0/2831862199 conn(0x55a7d4d90300 msgr2=0x55a7d515d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.196+0000 7f335b088740 1 -- 172.21.15.64:0/2831862199 shutdown_connections 2024-09-21T04:08:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.196+0000 7f335b088740 1 -- 172.21.15.64:0/2831862199 wait complete. 2024-09-21T04:08:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.197+0000 7f335b088740 1 -- start start 2024-09-21T04:08:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.197+0000 7f335b088740 1 Processor -- start 2024-09-21T04:08:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.197+0000 7f335b088740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7d4d732f0 0x55a7d51558e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.197+0000 7f335b088740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7d4d7b4d0 0x55a7d5155e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.197+0000 7f335b088740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7d4d9c1b0 0x55a7d51cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.197+0000 7f335b088740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1abd00 con 0x55a7d4d9c1b0 2024-09-21T04:08:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.197+0000 7f335b088740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1ad170 con 0x55a7d4d7b4d0 2024-09-21T04:08:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.197+0000 7f335b088740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f334c1ac3d0 con 0x55a7d4d732f0 2024-09-21T04:08:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.197+0000 7f335b088740 10 client.0 _ll_register_callbacks cb 0x7f333c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:08:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.197+0000 7f33526be640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7d4d9c1b0 0x55a7d51cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.197+0000 7f33516bc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7d4d7b4d0 0x55a7d5155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.198+0000 7f33516bc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7d4d7b4d0 0x55a7d5155e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:57346/0 (socket says 172.21.15.64:57346) 2024-09-21T04:08:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.198+0000 7f3351ebd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7d4d732f0 0x55a7d51558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.198+0000 7f33516bc640 1 -- 172.21.15.64:0/3814187926 learned_addr learned my addr 172.21.15.64:0/3814187926 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.198+0000 7f33516bc640 1 -- 172.21.15.64:0/3814187926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7d4d732f0 msgr2=0x55a7d51558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.198+0000 7f33516bc640 1 --2- 172.21.15.64:0/3814187926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7d4d732f0 0x55a7d51558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.198+0000 7f33516bc640 1 -- 172.21.15.64:0/3814187926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7d4d9c1b0 msgr2=0x55a7d51cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.198+0000 7f33516bc640 1 --2- 172.21.15.64:0/3814187926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7d4d9c1b0 0x55a7d51cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.198+0000 7f33516bc640 1 -- 172.21.15.64:0/3814187926 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7d51cc140 con 0x55a7d4d7b4d0 2024-09-21T04:08:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.198+0000 7f33516bc640 1 --2- 172.21.15.64:0/3814187926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7d4d7b4d0 0x55a7d5155e60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f334000c880 tx=0x7f334000e8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fe7dd205f2ffc76 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.198+0000 7f33317fa640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:08:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.198+0000 7f33317fa640 1 -- 172.21.15.64:0/3814187926 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33400adf20 con 0x55a7d4d7b4d0 2024-09-21T04:08:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.198+0000 7f33317fa640 1 -- 172.21.15.64:0/3814187926 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f33400a9a90 con 0x55a7d4d7b4d0 2024-09-21T04:08:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.198+0000 7f335b088740 1 -- 172.21.15.64:0/3814187926 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f333c0035f0 con 0x55a7d4d7b4d0 2024-09-21T04:08:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.198+0000 7f33317fa640 1 -- 172.21.15.64:0/3814187926 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f33400b2300 con 0x55a7d4d7b4d0 2024-09-21T04:08:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.198+0000 7f335b088740 10 client.26150 fetch_fsmap learned FSMap version 32 2024-09-21T04:08:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.198+0000 7f335b088740 1 -- 172.21.15.64:0/3814187926 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a7d51cd300 con 0x55a7d4d7b4d0 2024-09-21T04:08:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.199+0000 7f33317fa640 1 -- 172.21.15.64:0/3814187926 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f333c0035f0 con 0x55a7d4d7b4d0 2024-09-21T04:08:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.199+0000 7f33317fa640 1 -- 172.21.15.64:0/3814187926 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f33400aca80 con 0x55a7d4d7b4d0 2024-09-21T04:08:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.199+0000 7f335b088740 10 client.26150 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T04:08:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.199+0000 7f335b088740 10 client.26150 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:08:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.199+0000 7f335b088740 20 client.26150 populate_metadata read hostname 'smithi064' 2024-09-21T04:08:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.199+0000 7f335b088740 1 --2- 172.21.15.64:0/3814187926 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a7d4d301c0 0x55a7d51cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.199+0000 7f335b088740 4 client.26150 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:08:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.199+0000 7f335b088740 1 -- 172.21.15.64:0/3814187926 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a7d50b8640 con 0x55a7d4d301c0 2024-09-21T04:08:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.199+0000 7f3351ebd640 1 --2- 172.21.15.64:0/3814187926 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a7d4d301c0 0x55a7d51cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.199+0000 7f3351ebd640 1 --2- 172.21.15.64:0/3814187926 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a7d4d301c0 0x55a7d51cfb10 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd940b4e8e97817d server_cookie=fe87c78e8e61b38f in_seq=0 out_seq=0 2024-09-21T04:08:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.199+0000 7f33317fa640 10 client.26150 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.218+0000 7f33317fa640 1 -- 172.21.15.64:0/3814187926 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a7d50b8640 con 0x55a7d4d301c0 2024-09-21T04:08:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.218+0000 7f33317fa640 10 client.26150 handle_command_reply: tid=0 2024-09-21T04:08:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.218+0000 7f335b088740 1 client.26150 shutdown 2024-09-21T04:08:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.218+0000 7f335b088740 1 -- 172.21.15.64:0/3814187926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7d4d7b4d0 msgr2=0x55a7d5155e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.218+0000 7f335b088740 1 --2- 172.21.15.64:0/3814187926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7d4d7b4d0 0x55a7d5155e60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f334000c880 tx=0x7f334000e8f0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.219+0000 7f335b088740 1 -- 172.21.15.64:0/3814187926 shutdown_connections 2024-09-21T04:08:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.219+0000 7f335b088740 1 --2- 172.21.15.64:0/3814187926 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a7d4d301c0 0x55a7d51cfb10 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.219+0000 7f335b088740 1 --2- 172.21.15.64:0/3814187926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a7d4d9c1b0 0x55a7d51cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.219+0000 7f335b088740 1 --2- 172.21.15.64:0/3814187926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a7d4d7b4d0 0x55a7d5155e60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.219+0000 7f33317fa640 0 client.26150 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.219+0000 7f335b088740 1 --2- 172.21.15.64:0/3814187926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a7d4d732f0 0x55a7d51558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.219+0000 7f335b088740 1 -- 172.21.15.64:0/3814187926 >> 172.21.15.64:0/3814187926 conn(0x55a7d4d90300 msgr2=0x55a7d51429e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.219+0000 7f335b088740 1 -- 172.21.15.64:0/3814187926 shutdown_connections 2024-09-21T04:08:16.222 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.219+0000 7f335b088740 1 -- 172.21.15.64:0/3814187926 wait complete. 2024-09-21T04:08:16.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.219+0000 7f335b088740 20 client.26150 trim_cache size 0 max 16384 2024-09-21T04:08:16.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:08:16.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:08:16.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:08:16.223 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:08:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.219+0000 7f3354121640 1 -- 172.21.15.64:0/3670235850 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f33340b6680 msgr2=0x7f33340b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.219+0000 7f3354121640 1 --2- 172.21.15.64:0/3670235850 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f33340b6680 0x7f33340b8b90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3344006010 tx=0x7f3344004a40 comp rx=0 tx=0).stop 2024-09-21T04:08:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.219+0000 7f3354121640 1 -- 172.21.15.64:0/3670235850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f334c1082e0 msgr2=0x7f334c199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.219+0000 7f3354121640 1 --2- 172.21.15.64:0/3670235850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f334c1082e0 0x7f334c199850 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f334000e970 tx=0x7f334000ed70 comp rx=0 tx=0).stop 2024-09-21T04:08:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.220+0000 7f3354121640 1 -- 172.21.15.64:0/3670235850 shutdown_connections 2024-09-21T04:08:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.220+0000 7f3354121640 1 --2- 172.21.15.64:0/3670235850 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f33340b6680 0x7f33340b8b90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.220+0000 7f3354121640 1 --2- 172.21.15.64:0/3670235850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f334c108c30 0x7f334c19eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.220+0000 7f3354121640 1 --2- 172.21.15.64:0/3670235850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f334c1082e0 0x7f334c199850 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.220+0000 7f3354121640 1 --2- 172.21.15.64:0/3670235850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f334c1029b0 0x7f334c1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:16.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.220+0000 7f3354121640 1 -- 172.21.15.64:0/3670235850 >> 172.21.15.64:0/3670235850 conn(0x7f334c07a740 msgr2=0x7f334c1042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:16.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.220+0000 7f3354121640 1 -- 172.21.15.64:0/3670235850 shutdown_connections 2024-09-21T04:08:16.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:16.220+0000 7f3354121640 1 -- 172.21.15.64:0/3670235850 wait complete. 2024-09-21T04:08:16.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:15 smithi119 ceph-mon[34811]: pgmap v448: 97 pgs: 97 active+clean; 596 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.0 MiB/s wr, 878 op/s 2024-09-21T04:08:16.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:15 smithi064 ceph-mon[31180]: pgmap v448: 97 pgs: 97 active+clean; 596 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.0 MiB/s wr, 878 op/s 2024-09-21T04:08:18.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:17 smithi131 ceph-mon[35193]: pgmap v449: 97 pgs: 97 active+clean; 600 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.6 MiB/s wr, 968 op/s 2024-09-21T04:08:18.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:17 smithi119 ceph-mon[34811]: pgmap v449: 97 pgs: 97 active+clean; 600 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.6 MiB/s wr, 968 op/s 2024-09-21T04:08:18.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:17 smithi064 ceph-mon[31180]: pgmap v449: 97 pgs: 97 active+clean; 600 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.6 MiB/s wr, 968 op/s 2024-09-21T04:08:20.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:19 smithi131 ceph-mon[35193]: pgmap v450: 97 pgs: 97 active+clean; 601 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.2 MiB/s wr, 585 op/s 2024-09-21T04:08:20.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:19 smithi119 ceph-mon[34811]: pgmap v450: 97 pgs: 97 active+clean; 601 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.2 MiB/s wr, 585 op/s 2024-09-21T04:08:20.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:19 smithi064 ceph-mon[31180]: pgmap v450: 97 pgs: 97 active+clean; 601 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.2 MiB/s wr, 585 op/s 2024-09-21T04:08:21.173 INFO:tasks.workunit.client.0.smithi064.stdout: 92 1766 1 1627 0 1012 2 2024-09-21T04:08:22.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:21 smithi131 ceph-mon[35193]: pgmap v451: 97 pgs: 97 active+clean; 601 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 410 op/s 2024-09-21T04:08:22.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:21 smithi119 ceph-mon[34811]: pgmap v451: 97 pgs: 97 active+clean; 601 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 410 op/s 2024-09-21T04:08:22.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:21 smithi064 ceph-mon[31180]: pgmap v451: 97 pgs: 97 active+clean; 601 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 410 op/s 2024-09-21T04:08:23.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:22 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:23.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:22 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:23.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:22 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:24.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:23 smithi131 ceph-mon[35193]: pgmap v452: 97 pgs: 97 active+clean; 607 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 608 op/s 2024-09-21T04:08:24.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:23 smithi131 ceph-mon[35193]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:08:24.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:23 smithi119 ceph-mon[34811]: pgmap v452: 97 pgs: 97 active+clean; 607 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 608 op/s 2024-09-21T04:08:24.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:23 smithi119 ceph-mon[34811]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:08:24.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:23 smithi064 ceph-mon[31180]: pgmap v452: 97 pgs: 97 active+clean; 607 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 608 op/s 2024-09-21T04:08:24.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:23 smithi064 ceph-mon[31180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:08:26.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:25 smithi131 ceph-mon[35193]: pgmap v453: 97 pgs: 97 active+clean; 607 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 359 op/s 2024-09-21T04:08:26.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:25 smithi119 ceph-mon[34811]: pgmap v453: 97 pgs: 97 active+clean; 607 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 359 op/s 2024-09-21T04:08:26.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:25 smithi064 ceph-mon[31180]: pgmap v453: 97 pgs: 97 active+clean; 607 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 359 op/s 2024-09-21T04:08:28.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:27 smithi131 ceph-mon[35193]: pgmap v454: 97 pgs: 97 active+clean; 590 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 407 op/s 2024-09-21T04:08:28.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:27 smithi119 ceph-mon[34811]: pgmap v454: 97 pgs: 97 active+clean; 590 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 407 op/s 2024-09-21T04:08:28.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:27 smithi064 ceph-mon[31180]: pgmap v454: 97 pgs: 97 active+clean; 590 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 407 op/s 2024-09-21T04:08:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.322+0000 7f4fbf0de640 1 -- 172.21.15.64:0/888018018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fb810dd60 msgr2=0x7f4fb8110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.322+0000 7f4fbf0de640 1 --2- 172.21.15.64:0/888018018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fb810dd60 0x7f4fb8110250 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009ca0 tx=0x7f4fb002a8c0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.322+0000 7f4fbf0de640 1 -- 172.21.15.64:0/888018018 shutdown_connections 2024-09-21T04:08:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.322+0000 7f4fbf0de640 1 --2- 172.21.15.64:0/888018018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fb810dd60 0x7f4fb8110250 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.322+0000 7f4fbf0de640 1 --2- 172.21.15.64:0/888018018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fb8105250 0x7f4fb810d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.322+0000 7f4fbf0de640 1 --2- 172.21.15.64:0/888018018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fb8104900 0x7f4fb8104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.322+0000 7f4fbf0de640 1 -- 172.21.15.64:0/888018018 >> 172.21.15.64:0/888018018 conn(0x7f4fb807a740 msgr2=0x7f4fb807ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.322+0000 7f4fbf0de640 1 -- 172.21.15.64:0/888018018 shutdown_connections 2024-09-21T04:08:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.322+0000 7f4fbf0de640 1 -- 172.21.15.64:0/888018018 wait complete. 2024-09-21T04:08:29.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.323+0000 7f4fbf0de640 1 Processor -- start 2024-09-21T04:08:29.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.323+0000 7f4fbf0de640 1 -- start start 2024-09-21T04:08:29.327 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.323+0000 7f4fbf0de640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fb8104900 0x7f4fb819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:29.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.323+0000 7f4fbf0de640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fb8105250 0x7f4fb819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:29.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.323+0000 7f4fbf0de640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fb810dd60 0x7f4fb819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:29.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.323+0000 7f4fbf0de640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb81139c0 con 0x7f4fb8104900 2024-09-21T04:08:29.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.323+0000 7f4fbf0de640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb8113840 con 0x7f4fb8105250 2024-09-21T04:08:29.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.323+0000 7f4fbf0de640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb8113b40 con 0x7f4fb810dd60 2024-09-21T04:08:29.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.323+0000 7f4fbce7a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fb8104900 0x7f4fb819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:29.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.323+0000 7f4fbce7a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fb8104900 0x7f4fb819a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52734/0 (socket says 172.21.15.64:52734) 2024-09-21T04:08:29.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.323+0000 7f4fbce7a640 1 -- 172.21.15.64:0/2959044512 learned_addr learned my addr 172.21.15.64:0/2959044512 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:29.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.323+0000 7f4fb7fff640 1 --2- 172.21.15.64:0/2959044512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fb8105250 0x7f4fb819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:29.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.324+0000 7f4fbd67b640 1 --2- 172.21.15.64:0/2959044512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fb810dd60 0x7f4fb819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:29.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.324+0000 7f4fbce7a640 1 -- 172.21.15.64:0/2959044512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fb810dd60 msgr2=0x7f4fb819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:29.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.324+0000 7f4fbce7a640 1 --2- 172.21.15.64:0/2959044512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fb810dd60 0x7f4fb819fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.328 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.324+0000 7f4fbce7a640 1 -- 172.21.15.64:0/2959044512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fb8105250 msgr2=0x7f4fb819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:29.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.324+0000 7f4fbce7a640 1 --2- 172.21.15.64:0/2959044512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fb8105250 0x7f4fb819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.324+0000 7f4fbce7a640 1 -- 172.21.15.64:0/2959044512 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb81a0370 con 0x7f4fb8104900 2024-09-21T04:08:29.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.324+0000 7f4fbce7a640 1 --2- 172.21.15.64:0/2959044512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fb8104900 0x7f4fb819a440 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4fac00e9f0 tx=0x7f4fac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:29.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.324+0000 7f4fb5ffb640 1 -- 172.21.15.64:0/2959044512 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac00cd80 con 0x7f4fb8104900 2024-09-21T04:08:29.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.324+0000 7f4fb5ffb640 1 -- 172.21.15.64:0/2959044512 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fac0108e0 con 0x7f4fb8104900 2024-09-21T04:08:29.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.324+0000 7f4fbf0de640 1 -- 172.21.15.64:0/2959044512 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fb81a0660 con 0x7f4fb8104900 2024-09-21T04:08:29.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.324+0000 7f4fb5ffb640 1 -- 172.21.15.64:0/2959044512 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac018980 con 0x7f4fb8104900 2024-09-21T04:08:29.329 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.324+0000 7f4fbf0de640 1 -- 172.21.15.64:0/2959044512 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fb81a6df0 con 0x7f4fb8104900 2024-09-21T04:08:29.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.327+0000 7f4fb5ffb640 1 -- 172.21.15.64:0/2959044512 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f4fac010430 con 0x7f4fb8104900 2024-09-21T04:08:29.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.327+0000 7f4fc604e740 1 Processor -- start 2024-09-21T04:08:29.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.327+0000 7f4fc604e740 1 -- start start 2024-09-21T04:08:29.331 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.327+0000 7f4fc604e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b0fdb21b0 0x556b101579b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.327+0000 7f4fc604e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b0fd892f0 0x556b10157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.327+0000 7f4fc604e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b0fd914d0 0x556b10158430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.328+0000 7f4fc604e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556b100cc6f0 con 0x556b0fdb21b0 2024-09-21T04:08:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.328+0000 7f4fc604e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556b100bbb20 con 0x556b0fd892f0 2024-09-21T04:08:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.328+0000 7f4fc604e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556b100bf930 con 0x556b0fd914d0 2024-09-21T04:08:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.328+0000 7f4fbd67b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b0fdb21b0 0x556b101579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.328+0000 7f4fbce7a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b0fd892f0 0x556b10157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.328+0000 7f4fbd67b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b0fdb21b0 0x556b101579b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52748/0 (socket says 172.21.15.64:52748) 2024-09-21T04:08:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.328+0000 7f4fbd67b640 1 -- 172.21.15.64:0/2505231792 learned_addr learned my addr 172.21.15.64:0/2505231792 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.328+0000 7f4fb7fff640 1 --2- 172.21.15.64:0/2505231792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b0fd914d0 0x556b10158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.328+0000 7f4fbce7a640 1 -- 172.21.15.64:0/2505231792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b0fd914d0 msgr2=0x556b10158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.328+0000 7f4fbce7a640 1 --2- 172.21.15.64:0/2505231792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b0fd914d0 0x556b10158430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.328+0000 7f4fbce7a640 1 -- 172.21.15.64:0/2505231792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b0fdb21b0 msgr2=0x556b101579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.328+0000 7f4fbce7a640 1 --2- 172.21.15.64:0/2505231792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b0fdb21b0 0x556b101579b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.328+0000 7f4fbce7a640 1 -- 172.21.15.64:0/2505231792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b10081530 con 0x556b0fd892f0 2024-09-21T04:08:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.328+0000 7f4fbd67b640 1 --2- 172.21.15.64:0/2505231792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b0fdb21b0 0x556b101579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.328+0000 7f4fb5ffb640 1 --2- 172.21.15.64:0/2959044512 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4fa80b6680 0x7f4fa80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.328+0000 7f4fbce7a640 1 --2- 172.21.15.64:0/2505231792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b0fd892f0 0x556b10157ef0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0e5220 tx=0x7f4fac0f0670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=269628b0fe3b010b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.328+0000 7f4fbd67b640 1 --2- 172.21.15.64:0/2959044512 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4fa80b6680 0x7f4fa80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.329+0000 7f4fb5ffb640 1 -- 172.21.15.64:0/2959044512 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f4fac0e7b30 con 0x7f4fb8104900 2024-09-21T04:08:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.329+0000 7f4fa2ffd640 1 -- 172.21.15.64:0/2505231792 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac0a9fa0 con 0x556b0fd892f0 2024-09-21T04:08:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.329+0000 7f4fa2ffd640 1 -- 172.21.15.64:0/2505231792 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fac010720 con 0x556b0fd892f0 2024-09-21T04:08:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.329+0000 7f4fa2ffd640 1 -- 172.21.15.64:0/2505231792 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac0ad0c0 con 0x556b0fd892f0 2024-09-21T04:08:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.330+0000 7f4fc604e740 1 -- 172.21.15.64:0/2505231792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b0fd892f0 msgr2=0x556b10157ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.330+0000 7f4fbd67b640 1 --2- 172.21.15.64:0/2959044512 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4fa80b6680 0x7f4fa80b8b90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4fb002f500 tx=0x7f4fb0005c50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.330+0000 7f4fc604e740 1 --2- 172.21.15.64:0/2505231792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b0fd892f0 0x556b10157ef0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0e5220 tx=0x7f4fac0f0670 comp rx=0 tx=0).stop 2024-09-21T04:08:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.330+0000 7f4fc604e740 1 -- 172.21.15.64:0/2505231792 shutdown_connections 2024-09-21T04:08:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.330+0000 7f4fc604e740 1 --2- 172.21.15.64:0/2505231792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b0fd914d0 0x556b10158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.330+0000 7f4fc604e740 1 --2- 172.21.15.64:0/2505231792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b0fd892f0 0x556b10157ef0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.330+0000 7f4fc604e740 1 --2- 172.21.15.64:0/2505231792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b0fdb21b0 0x556b101579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.330+0000 7f4fc604e740 1 -- 172.21.15.64:0/2505231792 >> 172.21.15.64:0/2505231792 conn(0x556b0fda6300 msgr2=0x556b101558a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.330+0000 7f4fc604e740 1 -- 172.21.15.64:0/2505231792 shutdown_connections 2024-09-21T04:08:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.330+0000 7f4fc604e740 1 -- 172.21.15.64:0/2505231792 wait complete. 2024-09-21T04:08:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.330+0000 7f4fc604e740 1 -- start start 2024-09-21T04:08:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.330+0000 7f4fc604e740 1 Processor -- start 2024-09-21T04:08:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.331+0000 7f4fc604e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b0fd892f0 0x556b10163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.331+0000 7f4fc604e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b0fd914d0 0x556b10163640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.331+0000 7f4fc604e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b0fdb21b0 0x556b1016f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.331+0000 7f4fc604e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556b0fff3ab0 con 0x556b0fdb21b0 2024-09-21T04:08:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.331+0000 7f4fc604e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556b10061730 con 0x556b0fd892f0 2024-09-21T04:08:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.331+0000 7f4fc604e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556b10166130 con 0x556b0fd914d0 2024-09-21T04:08:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.331+0000 7f4fc604e740 10 client.0 _ll_register_callbacks cb 0x556b100b56b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:08:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.331+0000 7f4fbce7a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b0fd892f0 0x556b10163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.331+0000 7f4fbd67b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b0fdb21b0 0x556b1016f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.331+0000 7f4fbce7a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b0fd892f0 0x556b10163100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:58198/0 (socket says 172.21.15.64:58198) 2024-09-21T04:08:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.331+0000 7f4fbce7a640 1 -- 172.21.15.64:0/71555523 learned_addr learned my addr 172.21.15.64:0/71555523 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.331+0000 7f4fb7fff640 1 --2- 172.21.15.64:0/71555523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b0fd914d0 0x556b10163640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.331+0000 7f4fbce7a640 1 -- 172.21.15.64:0/71555523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b0fd914d0 msgr2=0x556b10163640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.331+0000 7f4fbce7a640 1 --2- 172.21.15.64:0/71555523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b0fd914d0 0x556b10163640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.331+0000 7f4fbce7a640 1 -- 172.21.15.64:0/71555523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b0fdb21b0 msgr2=0x556b1016f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.331+0000 7f4fbce7a640 1 --2- 172.21.15.64:0/71555523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b0fdb21b0 0x556b1016f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.331+0000 7f4fbce7a640 1 -- 172.21.15.64:0/71555523 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b0fe1d790 con 0x556b0fd892f0 2024-09-21T04:08:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.331+0000 7f4fbd67b640 1 --2- 172.21.15.64:0/71555523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b0fdb21b0 0x556b1016f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.331+0000 7f4fbce7a640 1 --2- 172.21.15.64:0/71555523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b0fd892f0 0x556b10163100 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0e5850 tx=0x7f4fac0f0f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=789648f142c66e3f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.331+0000 7f4f97fff640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:08:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.332+0000 7f4f97fff640 1 -- 172.21.15.64:0/71555523 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac0ad2a0 con 0x556b0fd892f0 2024-09-21T04:08:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.332+0000 7f4f97fff640 1 -- 172.21.15.64:0/71555523 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f4fac0b70e0 con 0x556b0fd892f0 2024-09-21T04:08:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.332+0000 7f4fc604e740 1 -- 172.21.15.64:0/71555523 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556b100ce640 con 0x556b0fd892f0 2024-09-21T04:08:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.332+0000 7f4f97fff640 1 -- 172.21.15.64:0/71555523 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4fac0ad720 con 0x556b0fd892f0 2024-09-21T04:08:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.332+0000 7f4fc604e740 10 client.26162 fetch_fsmap learned FSMap version 33 2024-09-21T04:08:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.332+0000 7f4fc604e740 1 -- 172.21.15.64:0/71555523 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556b0fe96620 con 0x556b0fd892f0 2024-09-21T04:08:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.332+0000 7f4f97fff640 1 -- 172.21.15.64:0/71555523 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556b100ce640 con 0x556b0fd892f0 2024-09-21T04:08:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.332+0000 7f4f97fff640 1 -- 172.21.15.64:0/71555523 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f4fac0b2020 con 0x556b0fd892f0 2024-09-21T04:08:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.332+0000 7f4fc604e740 10 client.26162 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T04:08:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.332+0000 7f4fc604e740 10 client.26162 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:08:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.332+0000 7f4fc604e740 20 client.26162 populate_metadata read hostname 'smithi064' 2024-09-21T04:08:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.332+0000 7f4fc604e740 1 --2- 172.21.15.64:0/71555523 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556b0fd70d80 0x556b10175070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.332+0000 7f4fc604e740 4 client.26162 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:08:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.332+0000 7f4fc604e740 1 -- 172.21.15.64:0/71555523 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b100ca0f0 con 0x556b0fd70d80 2024-09-21T04:08:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.332+0000 7f4fb7fff640 1 --2- 172.21.15.64:0/71555523 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556b0fd70d80 0x556b10175070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.333+0000 7f4f97fff640 10 client.26162 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.333+0000 7f4fb7fff640 1 --2- 172.21.15.64:0/71555523 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556b0fd70d80 0x556b10175070 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f31a02aa14e58ea3 server_cookie=96aa8661d011646b in_seq=0 out_seq=0 2024-09-21T04:08:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.334+0000 7f4f97fff640 1 -- 172.21.15.64:0/71555523 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556b100ca0f0 con 0x556b0fd70d80 2024-09-21T04:08:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.334+0000 7f4f97fff640 10 client.26162 handle_command_reply: tid=0 2024-09-21T04:08:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.334+0000 7f4fc604e740 1 client.26162 shutdown 2024-09-21T04:08:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.334+0000 7f4fc604e740 1 -- 172.21.15.64:0/71555523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b0fd892f0 msgr2=0x556b10163100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.334+0000 7f4fc604e740 1 --2- 172.21.15.64:0/71555523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b0fd892f0 0x556b10163100 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0e5850 tx=0x7f4fac0f0f60 comp rx=0 tx=0).stop 2024-09-21T04:08:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.334+0000 7f4fc604e740 1 -- 172.21.15.64:0/71555523 shutdown_connections 2024-09-21T04:08:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.334+0000 7f4fc604e740 1 --2- 172.21.15.64:0/71555523 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556b0fd70d80 0x556b10175070 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.334+0000 7f4fc604e740 1 --2- 172.21.15.64:0/71555523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b0fdb21b0 0x556b1016f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.334+0000 7f4fc604e740 1 --2- 172.21.15.64:0/71555523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b0fd914d0 0x556b10163640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.334+0000 7f4fc604e740 1 --2- 172.21.15.64:0/71555523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b0fd892f0 0x556b10163100 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.334+0000 7f4f97fff640 0 client.26162 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.334+0000 7f4fc604e740 1 -- 172.21.15.64:0/71555523 >> 172.21.15.64:0/71555523 conn(0x556b0fda6300 msgr2=0x556b101589e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.334+0000 7f4fc604e740 1 -- 172.21.15.64:0/71555523 shutdown_connections 2024-09-21T04:08:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.334+0000 7f4fc604e740 1 -- 172.21.15.64:0/71555523 wait complete. 2024-09-21T04:08:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.334+0000 7f4fc604e740 20 client.26162 trim_cache size 0 max 16384 2024-09-21T04:08:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.349+0000 7f4fc604e740 1 Processor -- start 2024-09-21T04:08:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.349+0000 7f4fc604e740 1 -- start start 2024-09-21T04:08:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.349+0000 7f4fc604e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b0fd892f0 0x556b101635e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.349+0000 7f4fc604e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b0fd914d0 0x556b10164d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.349+0000 7f4fc604e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b0fdb21b0 0x556b1016f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.349+0000 7f4fc604e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb81b2200 con 0x556b0fdb21b0 2024-09-21T04:08:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.349+0000 7f4fc604e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb81afa70 con 0x556b0fd914d0 2024-09-21T04:08:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.349+0000 7f4fc604e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb81ad8c0 con 0x556b0fd892f0 2024-09-21T04:08:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.350+0000 7f4fbd67b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b0fdb21b0 0x556b1016f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.350+0000 7f4fbd67b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b0fdb21b0 0x556b1016f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52770/0 (socket says 172.21.15.64:52770) 2024-09-21T04:08:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.350+0000 7f4fbd67b640 1 -- 172.21.15.64:0/1715184419 learned_addr learned my addr 172.21.15.64:0/1715184419 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.350+0000 7f4fb7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b0fd914d0 0x556b10164d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.350+0000 7f4fbce7a640 1 --2- 172.21.15.64:0/1715184419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b0fd892f0 0x556b101635e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.350+0000 7f4fbd67b640 1 -- 172.21.15.64:0/1715184419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b0fd892f0 msgr2=0x556b101635e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.350+0000 7f4fbd67b640 1 --2- 172.21.15.64:0/1715184419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b0fd892f0 0x556b101635e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.350+0000 7f4fbd67b640 1 -- 172.21.15.64:0/1715184419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b0fd914d0 msgr2=0x556b10164d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.350+0000 7f4fbd67b640 1 --2- 172.21.15.64:0/1715184419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b0fd914d0 0x556b10164d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.350+0000 7f4fbd67b640 1 -- 172.21.15.64:0/1715184419 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b101656e0 con 0x556b0fdb21b0 2024-09-21T04:08:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.350+0000 7f4fb7fff640 1 --2- 172.21.15.64:0/1715184419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b0fd914d0 0x556b10164d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.350+0000 7f4fbce7a640 1 --2- 172.21.15.64:0/1715184419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b0fd892f0 0x556b101635e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:08:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.350+0000 7f4fbd67b640 1 --2- 172.21.15.64:0/1715184419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b0fdb21b0 0x556b1016f390 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0005cc0 tx=0x7f4fb0043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b52755cf7e5d7be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.350+0000 7f4fa27fc640 1 -- 172.21.15.64:0/1715184419 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0004040 con 0x556b0fdb21b0 2024-09-21T04:08:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.351+0000 7f4fa27fc640 1 -- 172.21.15.64:0/1715184419 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fb0036050 con 0x556b0fdb21b0 2024-09-21T04:08:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.351+0000 7f4fa27fc640 1 -- 172.21.15.64:0/1715184419 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb002fd40 con 0x556b0fdb21b0 2024-09-21T04:08:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.351+0000 7f4fc604e740 1 -- 172.21.15.64:0/1715184419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b0fdb21b0 msgr2=0x556b1016f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.351+0000 7f4fc604e740 1 --2- 172.21.15.64:0/1715184419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b0fdb21b0 0x556b1016f390 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0005cc0 tx=0x7f4fb0043000 comp rx=0 tx=0).stop 2024-09-21T04:08:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.351+0000 7f4fc604e740 1 -- 172.21.15.64:0/1715184419 shutdown_connections 2024-09-21T04:08:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.351+0000 7f4fc604e740 1 --2- 172.21.15.64:0/1715184419 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b0fdb21b0 0x556b1016f390 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.351+0000 7f4fc604e740 1 --2- 172.21.15.64:0/1715184419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b0fd914d0 0x556b10164d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.351+0000 7f4fc604e740 1 --2- 172.21.15.64:0/1715184419 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b0fd892f0 0x556b101635e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.351+0000 7f4fc604e740 1 -- 172.21.15.64:0/1715184419 >> 172.21.15.64:0/1715184419 conn(0x556b0fda6300 msgr2=0x556b10164620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.352+0000 7f4fc604e740 1 -- 172.21.15.64:0/1715184419 shutdown_connections 2024-09-21T04:08:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.352+0000 7f4fc604e740 1 -- 172.21.15.64:0/1715184419 wait complete. 2024-09-21T04:08:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.352+0000 7f4fc604e740 1 -- start start 2024-09-21T04:08:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.352+0000 7f4fc604e740 1 Processor -- start 2024-09-21T04:08:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.352+0000 7f4fc604e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b0fd892f0 0x556b10175730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.352+0000 7f4fc604e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b0fd914d0 0x556b10175c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.352+0000 7f4fc604e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b0fdb21b0 0x556b101dd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.352+0000 7f4fc604e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb81adf30 con 0x556b0fd914d0 2024-09-21T04:08:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.352+0000 7f4fc604e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb81af3a0 con 0x556b0fdb21b0 2024-09-21T04:08:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.353+0000 7f4fc604e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb81ae600 con 0x556b0fd892f0 2024-09-21T04:08:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.353+0000 7f4fc604e740 10 client.0 _ll_register_callbacks cb 0x7f4fac005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:08:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.353+0000 7f4fb7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b0fd914d0 0x556b10175c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.353+0000 7f4fb7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b0fd914d0 0x556b10175c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52782/0 (socket says 172.21.15.64:52782) 2024-09-21T04:08:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.353+0000 7f4fb7fff640 1 -- 172.21.15.64:0/4217489012 learned_addr learned my addr 172.21.15.64:0/4217489012 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.353+0000 7f4fbce7a640 1 --2- 172.21.15.64:0/4217489012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b0fd892f0 0x556b10175730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.353+0000 7f4fbd67b640 1 --2- 172.21.15.64:0/4217489012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b0fdb21b0 0x556b101dd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.353+0000 7f4fb7fff640 1 -- 172.21.15.64:0/4217489012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b0fd892f0 msgr2=0x556b10175730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.353+0000 7f4fb7fff640 1 --2- 172.21.15.64:0/4217489012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b0fd892f0 0x556b10175730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.353+0000 7f4fb7fff640 1 -- 172.21.15.64:0/4217489012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b0fdb21b0 msgr2=0x556b101dd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.353+0000 7f4fb7fff640 1 --2- 172.21.15.64:0/4217489012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b0fdb21b0 0x556b101dd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.353+0000 7f4fb7fff640 1 -- 172.21.15.64:0/4217489012 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b101ddd20 con 0x556b0fd914d0 2024-09-21T04:08:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.353+0000 7f4fb7fff640 1 --2- 172.21.15.64:0/4217489012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b0fd914d0 0x556b10175c70 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x556b100bbc10 tx=0x7f4fa400bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d06c35d39ef6c15 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.353+0000 7f4f97fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:08:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.353+0000 7f4f97fff640 1 -- 172.21.15.64:0/4217489012 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4005220 con 0x556b0fd914d0 2024-09-21T04:08:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.354+0000 7f4f97fff640 1 -- 172.21.15.64:0/4217489012 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f4fa40053c0 con 0x556b0fd914d0 2024-09-21T04:08:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.354+0000 7f4fc604e740 1 -- 172.21.15.64:0/4217489012 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fb0036050 con 0x556b0fd914d0 2024-09-21T04:08:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.354+0000 7f4f97fff640 1 -- 172.21.15.64:0/4217489012 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4fa40055c0 con 0x556b0fd914d0 2024-09-21T04:08:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.354+0000 7f4fc604e740 10 client.16467 fetch_fsmap learned FSMap version 33 2024-09-21T04:08:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.354+0000 7f4fc604e740 1 -- 172.21.15.64:0/4217489012 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556b101def10 con 0x556b0fd914d0 2024-09-21T04:08:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.354+0000 7f4f97fff640 1 -- 172.21.15.64:0/4217489012 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fb0036050 con 0x556b0fd914d0 2024-09-21T04:08:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.354+0000 7f4f97fff640 1 -- 172.21.15.64:0/4217489012 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f4fa402c770 con 0x556b0fd914d0 2024-09-21T04:08:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.354+0000 7f4fc604e740 10 client.16467 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T04:08:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.354+0000 7f4fc604e740 10 client.16467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:08:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.354+0000 7f4fc604e740 20 client.16467 populate_metadata read hostname 'smithi064' 2024-09-21T04:08:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.354+0000 7f4fc604e740 1 --2- 172.21.15.64:0/4217489012 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556b0fd461c0 0x556b101e1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.354+0000 7f4fc604e740 4 client.16467 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:08:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.354+0000 7f4fc604e740 1 -- 172.21.15.64:0/4217489012 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556b100c7520 con 0x556b0fd461c0 2024-09-21T04:08:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.355+0000 7f4fbce7a640 1 --2- 172.21.15.64:0/4217489012 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556b0fd461c0 0x556b101e1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.355+0000 7f4fbce7a640 1 --2- 172.21.15.64:0/4217489012 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556b0fd461c0 0x556b101e1700 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efa81bb5ccc9ec93 server_cookie=285d6ab8789def89 in_seq=0 out_seq=0 2024-09-21T04:08:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.355+0000 7f4f97fff640 10 client.16467 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.365+0000 7f4f97fff640 1 -- 172.21.15.64:0/4217489012 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556b100c7520 con 0x556b0fd461c0 2024-09-21T04:08:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.365+0000 7f4f97fff640 10 client.16467 handle_command_reply: tid=0 2024-09-21T04:08:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.365+0000 7f4fc604e740 1 client.16467 shutdown 2024-09-21T04:08:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.366+0000 7f4fc604e740 1 -- 172.21.15.64:0/4217489012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b0fd914d0 msgr2=0x556b10175c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.366+0000 7f4fc604e740 1 --2- 172.21.15.64:0/4217489012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b0fd914d0 0x556b10175c70 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x556b100bbc10 tx=0x7f4fa400bee0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.366+0000 7f4fc604e740 1 -- 172.21.15.64:0/4217489012 shutdown_connections 2024-09-21T04:08:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.366+0000 7f4fc604e740 1 --2- 172.21.15.64:0/4217489012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556b0fdb21b0 0x556b101dd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.366+0000 7f4fc604e740 1 --2- 172.21.15.64:0/4217489012 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556b0fd461c0 0x556b101e1700 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.366+0000 7f4fc604e740 1 --2- 172.21.15.64:0/4217489012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556b0fd914d0 0x556b10175c70 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.366+0000 7f4fc604e740 1 --2- 172.21.15.64:0/4217489012 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b0fd892f0 0x556b10175730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.366+0000 7f4f97fff640 0 client.16467 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.366+0000 7f4fc604e740 1 -- 172.21.15.64:0/4217489012 >> 172.21.15.64:0/4217489012 conn(0x556b0fda6300 msgr2=0x556b10155880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.366+0000 7f4fc604e740 1 -- 172.21.15.64:0/4217489012 shutdown_connections 2024-09-21T04:08:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.366+0000 7f4fc604e740 1 -- 172.21.15.64:0/4217489012 wait complete. 2024-09-21T04:08:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.366+0000 7f4fc604e740 20 client.16467 trim_cache size 0 max 16384 2024-09-21T04:08:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:08:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:08:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:08:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:08:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.367+0000 7f4fbf0de640 1 -- 172.21.15.64:0/2959044512 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4fa80b6680 msgr2=0x7f4fa80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.367+0000 7f4fbf0de640 1 --2- 172.21.15.64:0/2959044512 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4fa80b6680 0x7f4fa80b8b90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4fb002f500 tx=0x7f4fb0005c50 comp rx=0 tx=0).stop 2024-09-21T04:08:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.367+0000 7f4fbf0de640 1 -- 172.21.15.64:0/2959044512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fb8104900 msgr2=0x7f4fb819a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.367+0000 7f4fbf0de640 1 --2- 172.21.15.64:0/2959044512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fb8104900 0x7f4fb819a440 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4fac00e9f0 tx=0x7f4fac00eeb0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.367+0000 7f4fbf0de640 1 -- 172.21.15.64:0/2959044512 shutdown_connections 2024-09-21T04:08:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.367+0000 7f4fbf0de640 1 --2- 172.21.15.64:0/2959044512 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4fa80b6680 0x7f4fa80b8b90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.367+0000 7f4fbf0de640 1 --2- 172.21.15.64:0/2959044512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fb810dd60 0x7f4fb819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.367+0000 7f4fbf0de640 1 --2- 172.21.15.64:0/2959044512 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fb8105250 0x7f4fb819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.367+0000 7f4fbf0de640 1 --2- 172.21.15.64:0/2959044512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fb8104900 0x7f4fb819a440 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.367+0000 7f4fbf0de640 1 -- 172.21.15.64:0/2959044512 >> 172.21.15.64:0/2959044512 conn(0x7f4fb807a740 msgr2=0x7f4fb80ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.367+0000 7f4fbf0de640 1 -- 172.21.15.64:0/2959044512 shutdown_connections 2024-09-21T04:08:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:29.367+0000 7f4fbf0de640 1 -- 172.21.15.64:0/2959044512 wait complete. 2024-09-21T04:08:30.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:29 smithi131 ceph-mon[35193]: pgmap v455: 97 pgs: 97 active+clean; 580 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 343 op/s 2024-09-21T04:08:30.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:29 smithi119 ceph-mon[34811]: pgmap v455: 97 pgs: 97 active+clean; 580 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 343 op/s 2024-09-21T04:08:30.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:29 smithi064 ceph-mon[31180]: pgmap v455: 97 pgs: 97 active+clean; 580 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 343 op/s 2024-09-21T04:08:31.173 INFO:tasks.workunit.client.0.smithi064.stdout: 93 9450 188 7034 38 5948 272 2024-09-21T04:08:32.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:31 smithi131 ceph-mon[35193]: pgmap v456: 97 pgs: 97 active+clean; 580 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 301 op/s 2024-09-21T04:08:32.062 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:31 smithi064 ceph-mon[31180]: pgmap v456: 97 pgs: 97 active+clean; 580 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 301 op/s 2024-09-21T04:08:32.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:31 smithi119 ceph-mon[34811]: pgmap v456: 97 pgs: 97 active+clean; 580 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 301 op/s 2024-09-21T04:08:32.433 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T04:08:32.704 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:08:33.124 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:32 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:08:33.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:32 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:08:33.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.219+0000 7fca78849640 1 -- 172.21.15.64:0/27719376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca74102a60 msgr2=0x7fca74102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:33.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.219+0000 7fca78849640 1 --2- 172.21.15.64:0/27719376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca74102a60 0x7fca74102e70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fca64009900 tx=0x7fca6402a3f0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.220+0000 7fca78849640 1 -- 172.21.15.64:0/27719376 shutdown_connections 2024-09-21T04:08:33.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.220+0000 7fca78849640 1 --2- 172.21.15.64:0/27719376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca74108cf0 0x7fca7410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.220+0000 7fca78849640 1 --2- 172.21.15.64:0/27719376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca741083a0 0x7fca741087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.220+0000 7fca78849640 1 --2- 172.21.15.64:0/27719376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca74102a60 0x7fca74102e70 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.220+0000 7fca78849640 1 -- 172.21.15.64:0/27719376 >> 172.21.15.64:0/27719376 conn(0x7fca74079730 msgr2=0x7fca74079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:33.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.220+0000 7fca78849640 1 -- 172.21.15.64:0/27719376 shutdown_connections 2024-09-21T04:08:33.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.220+0000 7fca78849640 1 -- 172.21.15.64:0/27719376 wait complete. 2024-09-21T04:08:33.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.221+0000 7fca78849640 1 Processor -- start 2024-09-21T04:08:33.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.221+0000 7fca78849640 1 -- start start 2024-09-21T04:08:33.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.221+0000 7fca78849640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca74102a60 0x7fca7419b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:33.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.221+0000 7fca78849640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca741083a0 0x7fca7419b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:33.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.221+0000 7fca78849640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca74108cf0 0x7fca7419bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:33.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.221+0000 7fca78849640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fca74112960 con 0x7fca74108cf0 2024-09-21T04:08:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.221+0000 7fca78849640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fca741127e0 con 0x7fca741083a0 2024-09-21T04:08:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.221+0000 7fca78849640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fca74112ae0 con 0x7fca74102a60 2024-09-21T04:08:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.221+0000 7fca72d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca74108cf0 0x7fca7419bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.221+0000 7fca6bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca741083a0 0x7fca7419b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.221+0000 7fca6bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca741083a0 0x7fca7419b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:35440/0 (socket says 172.21.15.64:35440) 2024-09-21T04:08:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.221+0000 7fca72d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca74108cf0 0x7fca7419bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41946/0 (socket says 172.21.15.64:41946) 2024-09-21T04:08:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.221+0000 7fca7259c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca74102a60 0x7fca7419b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.221+0000 7fca6bfff640 1 -- 172.21.15.64:0/3220996427 learned_addr learned my addr 172.21.15.64:0/3220996427 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.222+0000 7fca6bfff640 1 -- 172.21.15.64:0/3220996427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca74102a60 msgr2=0x7fca7419b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.222+0000 7fca6bfff640 1 --2- 172.21.15.64:0/3220996427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca74102a60 0x7fca7419b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.222+0000 7fca6bfff640 1 -- 172.21.15.64:0/3220996427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca74108cf0 msgr2=0x7fca7419bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.222+0000 7fca6bfff640 1 --2- 172.21.15.64:0/3220996427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca74108cf0 0x7fca7419bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.222+0000 7fca6bfff640 1 -- 172.21.15.64:0/3220996427 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca741a64e0 con 0x7fca741083a0 2024-09-21T04:08:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.222+0000 7fca72d9d640 1 --2- 172.21.15.64:0/3220996427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca74108cf0 0x7fca7419bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.222+0000 7fca6bfff640 1 --2- 172.21.15.64:0/3220996427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca741083a0 0x7fca7419b710 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00e9c0 tx=0x7fca5c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.222+0000 7fca6b7fe640 1 -- 172.21.15.64:0/3220996427 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c00cd80 con 0x7fca741083a0 2024-09-21T04:08:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.222+0000 7fca78849640 1 -- 172.21.15.64:0/3220996427 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca741a67d0 con 0x7fca741083a0 2024-09-21T04:08:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.222+0000 7fca6b7fe640 1 -- 172.21.15.64:0/3220996427 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca5c0054d0 con 0x7fca741083a0 2024-09-21T04:08:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.222+0000 7fca6b7fe640 1 -- 172.21.15.64:0/3220996427 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c010430 con 0x7fca741083a0 2024-09-21T04:08:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.222+0000 7fca78849640 1 -- 172.21.15.64:0/3220996427 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca741a6d60 con 0x7fca741083a0 2024-09-21T04:08:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.225+0000 7fca7edbd740 1 Processor -- start 2024-09-21T04:08:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.225+0000 7fca7edbd740 1 -- start start 2024-09-21T04:08:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.226+0000 7fca6b7fe640 1 -- 172.21.15.64:0/3220996427 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fca5c0105d0 con 0x7fca741083a0 2024-09-21T04:08:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.226+0000 7fca7edbd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d2098b070 0x555d20d32650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.226+0000 7fca7edbd740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d209621b0 0x555d20d32b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.226+0000 7fca7edbd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d2096a390 0x555d20d330d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:33.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.226+0000 7fca7edbd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555d20ca8e50 con 0x555d2098b070 2024-09-21T04:08:33.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.226+0000 7fca7edbd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x555d20c98280 con 0x555d2096a390 2024-09-21T04:08:33.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.226+0000 7fca7edbd740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555d20c9c090 con 0x555d209621b0 2024-09-21T04:08:33.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.226+0000 7fca72d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d2098b070 0x555d20d32650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:33.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.226+0000 7fca72d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d2098b070 0x555d20d32650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41960/0 (socket says 172.21.15.64:41960) 2024-09-21T04:08:33.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.226+0000 7fca72d9d640 1 -- 172.21.15.64:0/2935265573 learned_addr learned my addr 172.21.15.64:0/2935265573 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:33.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.226+0000 7fca6bfff640 1 --2- 172.21.15.64:0/2935265573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d2096a390 0x555d20d330d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:33.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.226+0000 7fca7259c640 1 --2- 172.21.15.64:0/2935265573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d209621b0 0x555d20d32b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:33.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.227+0000 7fca72d9d640 1 -- 172.21.15.64:0/2935265573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d209621b0 msgr2=0x555d20d32b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:33.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.227+0000 7fca72d9d640 1 --2- 172.21.15.64:0/2935265573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d209621b0 0x555d20d32b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.227+0000 7fca72d9d640 1 -- 172.21.15.64:0/2935265573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d2096a390 msgr2=0x555d20d330d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:33.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.227+0000 7fca72d9d640 1 --2- 172.21.15.64:0/2935265573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d2096a390 0x555d20d330d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.227+0000 7fca72d9d640 1 -- 172.21.15.64:0/2935265573 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d209f6650 con 0x555d2098b070 2024-09-21T04:08:33.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.227+0000 7fca72d9d640 1 --2- 172.21.15.64:0/2935265573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d2098b070 0x555d20d32650 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fca6c011730 tx=0x7fca6c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7139ce0ccf27f325 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:33.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.227+0000 7fca68ff9640 1 -- 172.21.15.64:0/2935265573 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6c00f8e0 con 0x555d2098b070 2024-09-21T04:08:33.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.227+0000 7fca6b7fe640 1 --2- 172.21.15.64:0/3220996427 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fca600b6680 0x7fca600b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:33.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.227+0000 7fca68ff9640 1 -- 172.21.15.64:0/2935265573 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca6c012450 con 0x555d2098b070 2024-09-21T04:08:33.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.227+0000 7fca68ff9640 1 -- 172.21.15.64:0/2935265573 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6c01a630 con 0x555d2098b070 2024-09-21T04:08:33.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.227+0000 7fca7259c640 1 --2- 172.21.15.64:0/3220996427 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fca600b6680 0x7fca600b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:33.241 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.228+0000 7fca6b7fe640 1 -- 172.21.15.64:0/3220996427 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fca5c0e7c20 con 0x7fca741083a0 2024-09-21T04:08:33.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.228+0000 7fca7edbd740 1 -- 172.21.15.64:0/2935265573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d2098b070 msgr2=0x555d20d32650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:33.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.228+0000 7fca7edbd740 1 --2- 172.21.15.64:0/2935265573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d2098b070 0x555d20d32650 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fca6c011730 tx=0x7fca6c011bf0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.228+0000 7fca7259c640 1 --2- 172.21.15.64:0/3220996427 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fca600b6680 0x7fca600b8b90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fca64000c00 tx=0x7fca64031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:33.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.228+0000 7fca7edbd740 1 -- 172.21.15.64:0/2935265573 shutdown_connections 2024-09-21T04:08:33.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.228+0000 7fca7edbd740 1 --2- 172.21.15.64:0/2935265573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d2096a390 0x555d20d330d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.228+0000 7fca7edbd740 1 --2- 172.21.15.64:0/2935265573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d209621b0 0x555d20d32b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.228+0000 7fca7edbd740 1 --2- 172.21.15.64:0/2935265573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d2098b070 0x555d20d32650 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.228+0000 7fca7edbd740 1 -- 172.21.15.64:0/2935265573 >> 172.21.15.64:0/2935265573 conn(0x555d2097f460 msgr2=0x555d20d30540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:33.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.229+0000 7fca7edbd740 1 -- 172.21.15.64:0/2935265573 shutdown_connections 2024-09-21T04:08:33.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.229+0000 7fca7edbd740 1 -- 172.21.15.64:0/2935265573 wait complete. 2024-09-21T04:08:33.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.229+0000 7fca7edbd740 1 -- start start 2024-09-21T04:08:33.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.229+0000 7fca7edbd740 1 Processor -- start 2024-09-21T04:08:33.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.229+0000 7fca7edbd740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d209621b0 0x555d20d420b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:33.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.229+0000 7fca7edbd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d2096a390 0x555d20d425f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:33.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.229+0000 7fca7edbd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d2098b070 0x555d20d42b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:33.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.229+0000 7fca7edbd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555d20af9700 con 0x555d2096a390 2024-09-21T04:08:33.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.229+0000 7fca7edbd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x555d20c33110 con 0x555d2098b070 2024-09-21T04:08:33.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.229+0000 7fca7edbd740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555d20d45190 con 0x555d209621b0 2024-09-21T04:08:33.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.230+0000 7fca7edbd740 10 client.0 _ll_register_callbacks cb 0x555d20c4c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:08:33.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.230+0000 7fca6bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d2096a390 0x555d20d425f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:33.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.230+0000 7fca6bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d2096a390 0x555d20d425f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41976/0 (socket says 172.21.15.64:41976) 2024-09-21T04:08:33.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.230+0000 7fca6bfff640 1 -- 172.21.15.64:0/2439799186 learned_addr learned my addr 172.21.15.64:0/2439799186 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:33.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.230+0000 7fca72d9d640 1 --2- 172.21.15.64:0/2439799186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d2098b070 0x555d20d42b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:33.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.230+0000 7fca7259c640 1 --2- 172.21.15.64:0/2439799186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d209621b0 0x555d20d420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:33.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.230+0000 7fca72d9d640 1 -- 172.21.15.64:0/2439799186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d209621b0 msgr2=0x555d20d420b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.230+0000 7fca72d9d640 1 --2- 172.21.15.64:0/2439799186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d209621b0 0x555d20d420b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.230+0000 7fca72d9d640 1 -- 172.21.15.64:0/2439799186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d2096a390 msgr2=0x555d20d425f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.230+0000 7fca72d9d640 1 --2- 172.21.15.64:0/2439799186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d2096a390 0x555d20d425f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.230+0000 7fca72d9d640 1 -- 172.21.15.64:0/2439799186 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d20c60c40 con 0x555d2098b070 2024-09-21T04:08:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.230+0000 7fca7259c640 1 --2- 172.21.15.64:0/2439799186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d209621b0 0x555d20d420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:08:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.230+0000 7fca6bfff640 1 --2- 172.21.15.64:0/2439799186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d2096a390 0x555d20d425f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.230+0000 7fca72d9d640 1 --2- 172.21.15.64:0/2439799186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d2098b070 0x555d20d42b50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fca6c01c040 tx=0x7fca6c0128b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3553ef93dddcb6c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.230+0000 7fca59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:08:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.230+0000 7fca59ffb640 1 -- 172.21.15.64:0/2439799186 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6c027020 con 0x555d2098b070 2024-09-21T04:08:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.230+0000 7fca59ffb640 1 -- 172.21.15.64:0/2439799186 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fca6c011e40 con 0x555d2098b070 2024-09-21T04:08:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.231+0000 7fca7edbd740 1 -- 172.21.15.64:0/2439799186 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca6c012450 con 0x555d2098b070 2024-09-21T04:08:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.231+0000 7fca59ffb640 1 -- 172.21.15.64:0/2439799186 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fca6c032050 con 0x555d2098b070 2024-09-21T04:08:33.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.231+0000 7fca7edbd740 10 client.26180 fetch_fsmap learned FSMap version 33 2024-09-21T04:08:33.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.231+0000 7fca7edbd740 1 -- 172.21.15.64:0/2439799186 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555d20d50e50 con 0x555d2098b070 2024-09-21T04:08:33.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.231+0000 7fca59ffb640 1 -- 172.21.15.64:0/2439799186 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca6c012450 con 0x555d2098b070 2024-09-21T04:08:33.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.231+0000 7fca59ffb640 1 -- 172.21.15.64:0/2439799186 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fca6c00f040 con 0x555d2098b070 2024-09-21T04:08:33.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.231+0000 7fca7edbd740 10 client.26180 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T04:08:33.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.231+0000 7fca7edbd740 10 client.26180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:08:33.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.231+0000 7fca7edbd740 20 client.26180 populate_metadata read hostname 'smithi064' 2024-09-21T04:08:33.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.231+0000 7fca7edbd740 1 --2- 172.21.15.64:0/2439799186 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x555d209499f0 0x555d20d542c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:33.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.231+0000 7fca7edbd740 4 client.26180 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:08:33.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.231+0000 7fca7edbd740 1 -- 172.21.15.64:0/2439799186 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d20caada0 con 0x555d209499f0 2024-09-21T04:08:33.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.231+0000 7fca7259c640 1 --2- 172.21.15.64:0/2439799186 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x555d209499f0 0x555d20d542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:33.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.232+0000 7fca59ffb640 10 client.26180 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:33.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.232+0000 7fca7259c640 1 --2- 172.21.15.64:0/2439799186 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x555d209499f0 0x555d20d542c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0eba4301924b821 server_cookie=4fbd9b0493237531 in_seq=0 out_seq=0 2024-09-21T04:08:33.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.233+0000 7fca59ffb640 1 -- 172.21.15.64:0/2439799186 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555d20caada0 con 0x555d209499f0 2024-09-21T04:08:33.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.233+0000 7fca59ffb640 10 client.26180 handle_command_reply: tid=0 2024-09-21T04:08:33.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.233+0000 7fca7edbd740 1 client.26180 shutdown 2024-09-21T04:08:33.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.233+0000 7fca7edbd740 1 -- 172.21.15.64:0/2439799186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d2098b070 msgr2=0x555d20d42b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:33.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.233+0000 7fca7edbd740 1 --2- 172.21.15.64:0/2439799186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d2098b070 0x555d20d42b50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fca6c01c040 tx=0x7fca6c0128b0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.233+0000 7fca7edbd740 1 -- 172.21.15.64:0/2439799186 shutdown_connections 2024-09-21T04:08:33.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.233+0000 7fca7edbd740 1 --2- 172.21.15.64:0/2439799186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d2098b070 0x555d20d42b50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.233+0000 7fca7edbd740 1 --2- 172.21.15.64:0/2439799186 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x555d209499f0 0x555d20d542c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.233+0000 7fca7edbd740 1 --2- 172.21.15.64:0/2439799186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d2096a390 0x555d20d425f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.233+0000 7fca7edbd740 1 --2- 172.21.15.64:0/2439799186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d209621b0 0x555d20d420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.233+0000 7fca59ffb640 0 client.26180 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:33.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.233+0000 7fca7edbd740 1 -- 172.21.15.64:0/2439799186 >> 172.21.15.64:0/2439799186 conn(0x555d2097f460 msgr2=0x555d20d34040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:33.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.233+0000 7fca7edbd740 1 -- 172.21.15.64:0/2439799186 shutdown_connections 2024-09-21T04:08:33.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.233+0000 7fca7edbd740 1 -- 172.21.15.64:0/2439799186 wait complete. 2024-09-21T04:08:33.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.233+0000 7fca7edbd740 20 client.26180 trim_cache size 0 max 16384 2024-09-21T04:08:33.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.253+0000 7fca7edbd740 1 Processor -- start 2024-09-21T04:08:33.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.253+0000 7fca7edbd740 1 -- start start 2024-09-21T04:08:33.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.253+0000 7fca7edbd740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d209621b0 0x555d20d43f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:33.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.253+0000 7fca7edbd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d2096a390 0x555d20d33d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:33.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.253+0000 7fca7edbd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d2098b070 0x555d20d3f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:33.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.253+0000 7fca7edbd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fca741aff70 con 0x555d2098b070 2024-09-21T04:08:33.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.253+0000 7fca7edbd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fca741ad7e0 con 0x555d2096a390 2024-09-21T04:08:33.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.253+0000 7fca7edbd740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fca741ab630 con 0x555d209621b0 2024-09-21T04:08:33.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.253+0000 7fca72d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d2098b070 0x555d20d3f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:33.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.253+0000 7fca72d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d2098b070 0x555d20d3f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41992/0 (socket says 172.21.15.64:41992) 2024-09-21T04:08:33.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.253+0000 7fca72d9d640 1 -- 172.21.15.64:0/3687339497 learned_addr learned my addr 172.21.15.64:0/3687339497 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:33.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.253+0000 7fca6bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d2096a390 0x555d20d33d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:33.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.254+0000 7fca7259c640 1 --2- 172.21.15.64:0/3687339497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d209621b0 0x555d20d43f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:33.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.254+0000 7fca72d9d640 1 -- 172.21.15.64:0/3687339497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d209621b0 msgr2=0x555d20d43f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:33.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.254+0000 7fca72d9d640 1 --2- 172.21.15.64:0/3687339497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d209621b0 0x555d20d43f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.254+0000 7fca72d9d640 1 -- 172.21.15.64:0/3687339497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d2096a390 msgr2=0x555d20d33d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:33.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.254+0000 7fca72d9d640 1 --2- 172.21.15.64:0/3687339497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d2096a390 0x555d20d33d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.254+0000 7fca72d9d640 1 -- 172.21.15.64:0/3687339497 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d20d2a8a0 con 0x555d2098b070 2024-09-21T04:08:33.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.254+0000 7fca7259c640 1 --2- 172.21.15.64:0/3687339497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d209621b0 0x555d20d43f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:08:33.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.254+0000 7fca72d9d640 1 --2- 172.21.15.64:0/3687339497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d2098b070 0x555d20d3f2a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fca6c0050b0 tx=0x7fca6c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c345e19536ffec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:33.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.254+0000 7fca5bfff640 1 -- 172.21.15.64:0/3687339497 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6c03a030 con 0x555d2098b070 2024-09-21T04:08:33.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.254+0000 7fca5bfff640 1 -- 172.21.15.64:0/3687339497 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca6c012450 con 0x555d2098b070 2024-09-21T04:08:33.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.254+0000 7fca5bfff640 1 -- 172.21.15.64:0/3687339497 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6c027040 con 0x555d2098b070 2024-09-21T04:08:33.259 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:32 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:08:33.259 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:32 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:08:33.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.255+0000 7fca7edbd740 1 -- 172.21.15.64:0/3687339497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d2098b070 msgr2=0x555d20d3f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:33.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.255+0000 7fca7edbd740 1 --2- 172.21.15.64:0/3687339497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d2098b070 0x555d20d3f2a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fca6c0050b0 tx=0x7fca6c011bf0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.255+0000 7fca7edbd740 1 -- 172.21.15.64:0/3687339497 shutdown_connections 2024-09-21T04:08:33.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.255+0000 7fca7edbd740 1 --2- 172.21.15.64:0/3687339497 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d2098b070 0x555d20d3f2a0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.255+0000 7fca7edbd740 1 --2- 172.21.15.64:0/3687339497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d2096a390 0x555d20d33d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.255+0000 7fca7edbd740 1 --2- 172.21.15.64:0/3687339497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d209621b0 0x555d20d43f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.255+0000 7fca7edbd740 1 -- 172.21.15.64:0/3687339497 >> 172.21.15.64:0/3687339497 conn(0x555d2097f460 msgr2=0x555d20d33680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:33.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.255+0000 7fca7edbd740 1 -- 172.21.15.64:0/3687339497 shutdown_connections 2024-09-21T04:08:33.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.255+0000 7fca7edbd740 1 -- 172.21.15.64:0/3687339497 wait complete. 2024-09-21T04:08:33.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.256+0000 7fca7edbd740 1 -- start start 2024-09-21T04:08:33.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.256+0000 7fca7edbd740 1 Processor -- start 2024-09-21T04:08:33.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.256+0000 7fca7edbd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d209621b0 0x555d20d54f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:33.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.256+0000 7fca7edbd740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d2096a390 0x555d20d554f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:33.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.256+0000 7fca7edbd740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d2098b070 0x555d20db6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:33.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.256+0000 7fca7edbd740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fca741abca0 con 0x555d2098b070 2024-09-21T04:08:33.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.256+0000 7fca7edbd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fca741ad110 con 0x555d209621b0 2024-09-21T04:08:33.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.256+0000 7fca7edbd740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fca741ac370 con 0x555d2096a390 2024-09-21T04:08:33.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.256+0000 7fca7edbd740 10 client.0 _ll_register_callbacks cb 0x7fca5c0eb5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:08:33.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.256+0000 7fca7259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d209621b0 0x555d20d54f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:33.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.256+0000 7fca72d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d2098b070 0x555d20db6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:33.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.256+0000 7fca72d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d2098b070 0x555d20db6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42000/0 (socket says 172.21.15.64:42000) 2024-09-21T04:08:33.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.256+0000 7fca72d9d640 1 -- 172.21.15.64:0/347776632 learned_addr learned my addr 172.21.15.64:0/347776632 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:33.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.256+0000 7fca7259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d209621b0 0x555d20d54f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:35482/0 (socket says 172.21.15.64:35482) 2024-09-21T04:08:33.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.256+0000 7fca6bfff640 1 --2- 172.21.15.64:0/347776632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d2096a390 0x555d20d554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:33.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.257+0000 7fca7259c640 1 -- 172.21.15.64:0/347776632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d2096a390 msgr2=0x555d20d554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:33.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.257+0000 7fca7259c640 1 --2- 172.21.15.64:0/347776632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d2096a390 0x555d20d554f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.257+0000 7fca7259c640 1 -- 172.21.15.64:0/347776632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d2098b070 msgr2=0x555d20db6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:33.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.257+0000 7fca7259c640 1 --2- 172.21.15.64:0/347776632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d2098b070 0x555d20db6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.257+0000 7fca7259c640 1 -- 172.21.15.64:0/347776632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d20db7260 con 0x555d209621b0 2024-09-21T04:08:33.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.257+0000 7fca6bfff640 1 --2- 172.21.15.64:0/347776632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d2096a390 0x555d20d554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:08:33.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.257+0000 7fca7259c640 1 --2- 172.21.15.64:0/347776632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d209621b0 0x555d20d54f30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fca7406ba20 tx=0x7fca64009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa550407dda7153a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:33.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.257+0000 7fca59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:08:33.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.257+0000 7fca59ffb640 1 -- 172.21.15.64:0/347776632 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca64035040 con 0x555d209621b0 2024-09-21T04:08:33.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.257+0000 7fca7edbd740 1 -- 172.21.15.64:0/347776632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca6c012450 con 0x555d209621b0 2024-09-21T04:08:33.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.257+0000 7fca59ffb640 1 -- 172.21.15.64:0/347776632 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fca64005030 con 0x555d209621b0 2024-09-21T04:08:33.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.257+0000 7fca59ffb640 1 -- 172.21.15.64:0/347776632 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fca64005250 con 0x555d209621b0 2024-09-21T04:08:33.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.257+0000 7fca7edbd740 10 client.26186 fetch_fsmap learned FSMap version 33 2024-09-21T04:08:33.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.257+0000 7fca7edbd740 1 -- 172.21.15.64:0/347776632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555d20dbe5d0 con 0x555d209621b0 2024-09-21T04:08:33.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.258+0000 7fca59ffb640 1 -- 172.21.15.64:0/347776632 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca6c012450 con 0x555d209621b0 2024-09-21T04:08:33.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.258+0000 7fca59ffb640 1 -- 172.21.15.64:0/347776632 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fca6405c070 con 0x555d209621b0 2024-09-21T04:08:33.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.258+0000 7fca7edbd740 10 client.26186 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T04:08:33.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.258+0000 7fca7edbd740 10 client.26186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:08:33.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.258+0000 7fca7edbd740 20 client.26186 populate_metadata read hostname 'smithi064' 2024-09-21T04:08:33.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.258+0000 7fca7edbd740 1 --2- 172.21.15.64:0/347776632 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x555d2091eda0 0x555d20dc0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:33.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.258+0000 7fca7edbd740 4 client.26186 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T04:08:33.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.258+0000 7fca7edbd740 1 -- 172.21.15.64:0/347776632 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555d20ca6850 con 0x555d2091eda0 2024-09-21T04:08:33.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.258+0000 7fca6bfff640 1 --2- 172.21.15.64:0/347776632 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x555d2091eda0 0x555d20dc0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:33.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.258+0000 7fca59ffb640 10 client.26186 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:33.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.258+0000 7fca6bfff640 1 --2- 172.21.15.64:0/347776632 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x555d2091eda0 0x555d20dc0d70 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc106cc04469c5d6 server_cookie=79c6e785fbfa43e4 in_seq=0 out_seq=0 2024-09-21T04:08:33.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.259+0000 7fca59ffb640 1 -- 172.21.15.64:0/347776632 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2078 (crc 0 0 0) 0x555d20ca6850 con 0x555d2091eda0 2024-09-21T04:08:33.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.259+0000 7fca59ffb640 10 client.26186 handle_command_reply: tid=0 2024-09-21T04:08:33.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.259+0000 7fca7edbd740 1 client.26186 shutdown 2024-09-21T04:08:33.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.259+0000 7fca7edbd740 1 -- 172.21.15.64:0/347776632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d209621b0 msgr2=0x555d20d54f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:33.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.259+0000 7fca7edbd740 1 --2- 172.21.15.64:0/347776632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d209621b0 0x555d20d54f30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fca7406ba20 tx=0x7fca64009260 comp rx=0 tx=0).stop 2024-09-21T04:08:33.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.259+0000 7fca7edbd740 1 -- 172.21.15.64:0/347776632 shutdown_connections 2024-09-21T04:08:33.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.259+0000 7fca7edbd740 1 --2- 172.21.15.64:0/347776632 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x555d2091eda0 0x555d20dc0d70 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.259+0000 7fca7edbd740 1 --2- 172.21.15.64:0/347776632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d2098b070 0x555d20db6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.259+0000 7fca7edbd740 1 --2- 172.21.15.64:0/347776632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555d2096a390 0x555d20d554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.259+0000 7fca59ffb640 0 client.26186 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:33.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.259+0000 7fca7edbd740 1 --2- 172.21.15.64:0/347776632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555d209621b0 0x555d20d54f30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.259+0000 7fca7edbd740 1 -- 172.21.15.64:0/347776632 >> 172.21.15.64:0/347776632 conn(0x555d2097f460 msgr2=0x555d20d43d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:33.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.259+0000 7fca7edbd740 1 -- 172.21.15.64:0/347776632 shutdown_connections 2024-09-21T04:08:33.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.259+0000 7fca7edbd740 1 -- 172.21.15.64:0/347776632 wait complete. 2024-09-21T04:08:33.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.259+0000 7fca7edbd740 20 client.26186 trim_cache size 0 max 16384 2024-09-21T04:08:33.267 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:08:33.267 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-21T04:08:33.267 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {}, 2024-09-21T04:08:33.267 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_stats": { 2024-09-21T04:08:33.267 INFO:teuthology.orchestra.run.smithi064.stdout: "bed40d88-2102-43a6-ac2d-b89d5a2264d0": { 2024-09-21T04:08:33.267 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:11.363340+0000", 2024-09-21T04:08:33.268 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:08:33.268 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:08:33.268 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:08:33.268 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:08:33.268 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:08:33.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:08:33.268 INFO:teuthology.orchestra.run.smithi064.stdout: "e9242f88-6599-40eb-adf1-bdb3e946a1e0": { 2024-09-21T04:08:33.268 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:05:39.744646+0000", 2024-09-21T04:08:33.268 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:08:33.268 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:08:33.268 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:08:33.268 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:08:33.268 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:08:33.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:08:33.269 INFO:teuthology.orchestra.run.smithi064.stdout: "6bde0fe5-f775-44b1-ad0c-f9e596574dc0": { 2024-09-21T04:08:33.269 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:06:23.835536+0000", 2024-09-21T04:08:33.269 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:08:33.269 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:08:33.269 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:08:33.269 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:08:33.269 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:08:33.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:08:33.269 INFO:teuthology.orchestra.run.smithi064.stdout: "b52b950c-0f97-49ec-a37c-1a8e04348ce8": { 2024-09-21T04:08:33.269 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:54.003800+0000", 2024-09-21T04:08:33.269 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:08:33.269 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:08:33.269 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:08:33.269 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:08:33.270 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:08:33.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:08:33.270 INFO:teuthology.orchestra.run.smithi064.stdout: "75db1bc5-612e-45aa-bad2-cc90feeaa840": { 2024-09-21T04:08:33.270 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:59.123138+0000", 2024-09-21T04:08:33.270 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:08:33.270 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:08:33.270 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:08:33.270 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:08:33.270 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:08:33.270 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:08:33.270 INFO:teuthology.orchestra.run.smithi064.stdout: "0ab717a7-36c6-4dca-94e7-af0bdf2aa112": { 2024-09-21T04:08:33.270 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:42.691086+0000", 2024-09-21T04:08:33.270 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:08:33.270 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:08:33.271 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:08:33.271 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:08:33.271 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:08:33.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:08:33.271 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:08:33.271 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:03:59.570174+0000", 2024-09-21T04:08:33.271 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:08:33.271 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:08:33.271 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:08:33.271 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:08:33.271 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:08:33.271 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:08:33.271 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:08:33.271 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:08:33.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.260+0000 7fca78849640 1 -- 172.21.15.64:0/3220996427 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fca600b6680 msgr2=0x7fca600b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:33.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.260+0000 7fca78849640 1 --2- 172.21.15.64:0/3220996427 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fca600b6680 0x7fca600b8b90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fca64000c00 tx=0x7fca64031400 comp rx=0 tx=0).stop 2024-09-21T04:08:33.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.260+0000 7fca78849640 1 -- 172.21.15.64:0/3220996427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca741083a0 msgr2=0x7fca7419b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:33.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.260+0000 7fca78849640 1 --2- 172.21.15.64:0/3220996427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca741083a0 0x7fca7419b710 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00e9c0 tx=0x7fca5c00ee80 comp rx=0 tx=0).stop 2024-09-21T04:08:33.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.260+0000 7fca78849640 1 -- 172.21.15.64:0/3220996427 shutdown_connections 2024-09-21T04:08:33.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.260+0000 7fca78849640 1 --2- 172.21.15.64:0/3220996427 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fca600b6680 0x7fca600b8b90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.260+0000 7fca78849640 1 --2- 172.21.15.64:0/3220996427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca74108cf0 0x7fca7419bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.260+0000 7fca78849640 1 --2- 172.21.15.64:0/3220996427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca741083a0 0x7fca7419b710 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.260+0000 7fca78849640 1 --2- 172.21.15.64:0/3220996427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca74102a60 0x7fca7419b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:33.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.260+0000 7fca78849640 1 -- 172.21.15.64:0/3220996427 >> 172.21.15.64:0/3220996427 conn(0x7fca74079730 msgr2=0x7fca74101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:33.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.260+0000 7fca78849640 1 -- 172.21.15.64:0/3220996427 shutdown_connections 2024-09-21T04:08:33.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:33.260+0000 7fca78849640 1 -- 172.21.15.64:0/3220996427 wait complete. 2024-09-21T04:08:33.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:32 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:08:33.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:32 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:08:34.051 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T04:08:34.052 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:08:34.078 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:33 smithi064 ceph-mon[31180]: pgmap v457: 97 pgs: 97 active+clean; 589 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 368 op/s 2024-09-21T04:08:34.078 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:33 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:08:34.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:33 smithi119 ceph-mon[34811]: pgmap v457: 97 pgs: 97 active+clean; 589 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 368 op/s 2024-09-21T04:08:34.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:33 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:08:34.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:33 smithi131 ceph-mon[35193]: pgmap v457: 97 pgs: 97 active+clean; 589 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 368 op/s 2024-09-21T04:08:34.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:33 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:08:34.321 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:08:34.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.842+0000 7f97badd5640 1 -- 172.21.15.64:0/1158763303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b4108cf0 msgr2=0x7f97b410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:34.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.842+0000 7f97badd5640 1 --2- 172.21.15.64:0/1158763303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b4108cf0 0x7f97b410f520 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f97ac009ca0 tx=0x7f97ac02a8c0 comp rx=0 tx=0).stop 2024-09-21T04:08:34.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.842+0000 7f97badd5640 1 -- 172.21.15.64:0/1158763303 shutdown_connections 2024-09-21T04:08:34.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.842+0000 7f97badd5640 1 --2- 172.21.15.64:0/1158763303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b4108cf0 0x7f97b410f520 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:34.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.842+0000 7f97badd5640 1 --2- 172.21.15.64:0/1158763303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97b41083a0 0x7f97b41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:34.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.842+0000 7f97badd5640 1 --2- 172.21.15.64:0/1158763303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97b4102a60 0x7f97b4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:34.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.842+0000 7f97badd5640 1 -- 172.21.15.64:0/1158763303 >> 172.21.15.64:0/1158763303 conn(0x7f97b4079730 msgr2=0x7f97b4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:34.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.842+0000 7f97badd5640 1 -- 172.21.15.64:0/1158763303 shutdown_connections 2024-09-21T04:08:34.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.842+0000 7f97badd5640 1 -- 172.21.15.64:0/1158763303 wait complete. 2024-09-21T04:08:34.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.843+0000 7f97badd5640 1 Processor -- start 2024-09-21T04:08:34.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.843+0000 7f97badd5640 1 -- start start 2024-09-21T04:08:34.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.843+0000 7f97badd5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97b4102a60 0x7f97b419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:34.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.844+0000 7f97badd5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b41083a0 0x7f97b419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:34.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.844+0000 7f97badd5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97b4108cf0 0x7f97b419fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:34.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.844+0000 7f97badd5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f97b4112960 con 0x7f97b41083a0 2024-09-21T04:08:34.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.844+0000 7f97badd5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f97b41127e0 con 0x7f97b4108cf0 2024-09-21T04:08:34.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.844+0000 7f97badd5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f97b4112ae0 con 0x7f97b4102a60 2024-09-21T04:08:34.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.844+0000 7f97b3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b41083a0 0x7f97b419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:34.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.844+0000 7f97b3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b41083a0 0x7f97b419a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42016/0 (socket says 172.21.15.64:42016) 2024-09-21T04:08:34.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.844+0000 7f97b3fff640 1 -- 172.21.15.64:0/237614803 learned_addr learned my addr 172.21.15.64:0/237614803 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:34.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.844+0000 7f97b9372640 1 --2- 172.21.15.64:0/237614803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97b4108cf0 0x7f97b419fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:34.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.844+0000 7f97b8b71640 1 --2- 172.21.15.64:0/237614803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97b4102a60 0x7f97b419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:34.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.844+0000 7f97b3fff640 1 -- 172.21.15.64:0/237614803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97b4102a60 msgr2=0x7f97b419a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:34.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.844+0000 7f97b3fff640 1 --2- 172.21.15.64:0/237614803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97b4102a60 0x7f97b419a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:34.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.844+0000 7f97b3fff640 1 -- 172.21.15.64:0/237614803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97b4108cf0 msgr2=0x7f97b419fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:34.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.844+0000 7f97b3fff640 1 --2- 172.21.15.64:0/237614803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97b4108cf0 0x7f97b419fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:34.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.844+0000 7f97b3fff640 1 -- 172.21.15.64:0/237614803 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b41a0350 con 0x7f97b41083a0 2024-09-21T04:08:34.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.844+0000 7f97b9372640 1 --2- 172.21.15.64:0/237614803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97b4108cf0 0x7f97b419fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:34.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.844+0000 7f97b3fff640 1 --2- 172.21.15.64:0/237614803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b41083a0 0x7f97b419a980 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f97a000e9c0 tx=0x7f97a000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:34.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.845+0000 7f97b1ffb640 1 -- 172.21.15.64:0/237614803 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a000cd80 con 0x7f97b41083a0 2024-09-21T04:08:34.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.845+0000 7f97b1ffb640 1 -- 172.21.15.64:0/237614803 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97a00054d0 con 0x7f97b41083a0 2024-09-21T04:08:34.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.845+0000 7f97badd5640 1 -- 172.21.15.64:0/237614803 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97b41a0640 con 0x7f97b41083a0 2024-09-21T04:08:34.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.845+0000 7f97b1ffb640 1 -- 172.21.15.64:0/237614803 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a0010430 con 0x7f97b41083a0 2024-09-21T04:08:34.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.845+0000 7f97badd5640 1 -- 172.21.15.64:0/237614803 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97b41a6da0 con 0x7f97b41083a0 2024-09-21T04:08:34.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.846+0000 7f97badd5640 1 -- 172.21.15.64:0/237614803 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97b4102e70 con 0x7f97b41083a0 2024-09-21T04:08:34.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.848+0000 7f97b1ffb640 1 -- 172.21.15.64:0/237614803 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f97a0004fd0 con 0x7f97b41083a0 2024-09-21T04:08:34.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.848+0000 7f97b1ffb640 1 --2- 172.21.15.64:0/237614803 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f97a40b6700 0x7f97a40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:34.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.849+0000 7f97b8b71640 1 --2- 172.21.15.64:0/237614803 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f97a40b6700 0x7f97a40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:34.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.849+0000 7f97b1ffb640 1 -- 172.21.15.64:0/237614803 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f97a00e7b10 con 0x7f97b41083a0 2024-09-21T04:08:34.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.849+0000 7f97b8b71640 1 --2- 172.21.15.64:0/237614803 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f97a40b6700 0x7f97a40b8c10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f97a8005fd0 tx=0x7f97a800b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:34.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:34.853+0000 7f97b1ffb640 1 -- 172.21.15.64:0/237614803 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f97a00ae030 con 0x7f97b41083a0 2024-09-21T04:08:35.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:35.095+0000 7f97badd5640 1 -- 172.21.15.64:0/237614803 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97b410b540 con 0x7f97b41083a0 2024-09-21T04:08:35.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:35.096+0000 7f97b1ffb640 1 -- 172.21.15.64:0/237614803 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11913 (secure 0 0 0) 0x7f97a00b2410 con 0x7f97b41083a0 2024-09-21T04:08:35.099 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:08:35.099 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":33,"btime":"2024-09-21T04:08:22:783154+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14805,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/97995119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":97995119},{"type":"v1","addr":"172.21.15.64:6839","nonce":97995119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/390891068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":390891068},{"type":"v1","addr":"172.21.15.64:6841","nonce":390891068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:02:03.354744+0000","modified":"2024-09-21T04:08:22.525096+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24710,"mds_1":24680,"mds_2":24661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24634":{"gid":24634,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6839/782190424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782190424},{"type":"v1","addr":"172.21.15.131:6839","nonce":782190424}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24680":{"gid":24680,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24710":{"gid":24710,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6839/1319793636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1319793636},{"type":"v1","addr":"172.21.15.119:6839","nonce":1319793636}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24710,"qdb_cluster":[24710,24680,24661]},"id":1}]} 2024-09-21T04:08:35.099 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 33 2024-09-21T04:08:35.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:35.098+0000 7f97badd5640 1 -- 172.21.15.64:0/237614803 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f97a40b6700 msgr2=0x7f97a40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:35.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:35.098+0000 7f97badd5640 1 --2- 172.21.15.64:0/237614803 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f97a40b6700 0x7f97a40b8c10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f97a8005fd0 tx=0x7f97a800b390 comp rx=0 tx=0).stop 2024-09-21T04:08:35.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:35.098+0000 7f97badd5640 1 -- 172.21.15.64:0/237614803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b41083a0 msgr2=0x7f97b419a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:35.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:35.098+0000 7f97badd5640 1 --2- 172.21.15.64:0/237614803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b41083a0 0x7f97b419a980 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f97a000e9c0 tx=0x7f97a000ee80 comp rx=0 tx=0).stop 2024-09-21T04:08:35.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:35.098+0000 7f97badd5640 1 -- 172.21.15.64:0/237614803 shutdown_connections 2024-09-21T04:08:35.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:35.098+0000 7f97badd5640 1 --2- 172.21.15.64:0/237614803 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f97a40b6700 0x7f97a40b8c10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:35.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:35.098+0000 7f97badd5640 1 --2- 172.21.15.64:0/237614803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97b4108cf0 0x7f97b419fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:35.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:35.098+0000 7f97badd5640 1 --2- 172.21.15.64:0/237614803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97b41083a0 0x7f97b419a980 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:35.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:35.098+0000 7f97badd5640 1 --2- 172.21.15.64:0/237614803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f97b4102a60 0x7f97b419a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:35.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:35.098+0000 7f97badd5640 1 -- 172.21.15.64:0/237614803 >> 172.21.15.64:0/237614803 conn(0x7f97b4079730 msgr2=0x7f97b406b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:35.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:35.099+0000 7f97badd5640 1 -- 172.21.15.64:0/237614803 shutdown_connections 2024-09-21T04:08:35.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:35.099+0000 7f97badd5640 1 -- 172.21.15.64:0/237614803 wait complete. 2024-09-21T04:08:35.886 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T04:08:36.203 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:08:36.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:35 smithi119 ceph-mon[34811]: pgmap v458: 97 pgs: 97 active+clean; 589 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-09-21T04:08:36.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:35 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/237614803' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:08:36.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:35 smithi064 ceph-mon[31180]: pgmap v458: 97 pgs: 97 active+clean; 589 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-09-21T04:08:36.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:35 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/237614803' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:08:36.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:35 smithi131 ceph-mon[35193]: pgmap v458: 97 pgs: 97 active+clean; 589 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-09-21T04:08:36.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:35 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/237614803' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:08:36.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.920+0000 7f350c616640 1 -- 172.21.15.64:0/960984272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3504103b80 msgr2=0x7f3504103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:36.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.920+0000 7f350c616640 1 --2- 172.21.15.64:0/960984272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3504103b80 0x7f3504103f90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f3500009900 tx=0x7f350002a410 comp rx=0 tx=0).stop 2024-09-21T04:08:36.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.920+0000 7f350c616640 1 -- 172.21.15.64:0/960984272 shutdown_connections 2024-09-21T04:08:36.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.920+0000 7f350c616640 1 --2- 172.21.15.64:0/960984272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3504109e10 0x7f3504110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.920+0000 7f350c616640 1 --2- 172.21.15.64:0/960984272 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35041094c0 0x7f35041098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.920+0000 7f350c616640 1 --2- 172.21.15.64:0/960984272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3504103b80 0x7f3504103f90 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.920+0000 7f350c616640 1 -- 172.21.15.64:0/960984272 >> 172.21.15.64:0/960984272 conn(0x7f35040ff610 msgr2=0x7f3504101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:36.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.920+0000 7f350c616640 1 -- 172.21.15.64:0/960984272 shutdown_connections 2024-09-21T04:08:36.923 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.920+0000 7f350c616640 1 -- 172.21.15.64:0/960984272 wait complete. 2024-09-21T04:08:36.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.921+0000 7f350c616640 1 Processor -- start 2024-09-21T04:08:36.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.921+0000 7f350c616640 1 -- start start 2024-09-21T04:08:36.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.921+0000 7f350c616640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3504103b80 0x7f3504075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:36.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.921+0000 7f350c616640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35041094c0 0x7f3504075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:36.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.921+0000 7f350c616640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3504109e10 0x7f3504076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:36.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.921+0000 7f350c616640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3504113ad0 con 0x7f3504109e10 2024-09-21T04:08:36.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.921+0000 7f350c616640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3504113950 con 0x7f3504103b80 2024-09-21T04:08:36.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.921+0000 7f350c616640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3504113c50 con 0x7f35041094c0 2024-09-21T04:08:36.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.921+0000 7f350a3b2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3504103b80 0x7f3504075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:36.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.921+0000 7f350abb3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3504109e10 0x7f3504076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:36.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.921+0000 7f350a3b2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3504103b80 0x7f3504075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:35524/0 (socket says 172.21.15.64:35524) 2024-09-21T04:08:36.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.921+0000 7f350a3b2640 1 -- 172.21.15.64:0/3631164731 learned_addr learned my addr 172.21.15.64:0/3631164731 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:36.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.922+0000 7f3509bb1640 1 --2- 172.21.15.64:0/3631164731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35041094c0 0x7f3504075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:36.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.922+0000 7f3509bb1640 1 -- 172.21.15.64:0/3631164731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3504103b80 msgr2=0x7f3504075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:36.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.922+0000 7f3509bb1640 1 --2- 172.21.15.64:0/3631164731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3504103b80 0x7f3504075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.922+0000 7f3509bb1640 1 -- 172.21.15.64:0/3631164731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3504109e10 msgr2=0x7f3504076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:36.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.922+0000 7f3509bb1640 1 --2- 172.21.15.64:0/3631164731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3504109e10 0x7f3504076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.922+0000 7f3509bb1640 1 -- 172.21.15.64:0/3631164731 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35041a8780 con 0x7f35041094c0 2024-09-21T04:08:36.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.922+0000 7f350a3b2640 1 --2- 172.21.15.64:0/3631164731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3504103b80 0x7f3504075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:36.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.922+0000 7f350abb3640 1 --2- 172.21.15.64:0/3631164731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3504109e10 0x7f3504076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:36.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.922+0000 7f3509bb1640 1 --2- 172.21.15.64:0/3631164731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35041094c0 0x7f3504075c50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f350406a800 tx=0x7f34f400dbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:36.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.923+0000 7f34fb7fe640 1 -- 172.21.15.64:0/3631164731 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f4017070 con 0x7f35041094c0 2024-09-21T04:08:36.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.923+0000 7f34fb7fe640 1 -- 172.21.15.64:0/3631164731 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34f40099a0 con 0x7f35041094c0 2024-09-21T04:08:36.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.923+0000 7f350c616640 1 -- 172.21.15.64:0/3631164731 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35041a8a70 con 0x7f35041094c0 2024-09-21T04:08:36.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.923+0000 7f34fb7fe640 1 -- 172.21.15.64:0/3631164731 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f40047a0 con 0x7f35041094c0 2024-09-21T04:08:36.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.923+0000 7f350c616640 1 -- 172.21.15.64:0/3631164731 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35041a8f60 con 0x7f35041094c0 2024-09-21T04:08:36.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.926+0000 7f34fb7fe640 1 -- 172.21.15.64:0/3631164731 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f34f4004940 con 0x7f35041094c0 2024-09-21T04:08:36.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.926+0000 7f3512b8a740 1 Processor -- start 2024-09-21T04:08:36.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.926+0000 7f3512b8a740 1 -- start start 2024-09-21T04:08:36.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.927+0000 7f3512b8a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55738a6e0070 0x55738aa872c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:36.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.927+0000 7f3512b8a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55738a6b71b0 0x55738aa87800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:36.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.927+0000 7f3512b8a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55738a6bf390 0x55738aa87d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:36.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.927+0000 7f34fb7fe640 1 --2- 172.21.15.64:0/3631164731 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f34f00b6680 0x7f34f00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:36.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.927+0000 7f3512b8a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55738a9fdea0 con 0x55738a6b71b0 2024-09-21T04:08:36.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.927+0000 7f3512b8a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55738a9ed2d0 con 0x55738a6bf390 2024-09-21T04:08:36.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.927+0000 7f3512b8a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55738a9f10e0 con 0x55738a6e0070 2024-09-21T04:08:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.927+0000 7f350abb3640 1 --2- 172.21.15.64:0/3631164731 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f34f00b6680 0x7f34f00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.927+0000 7f350a3b2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55738a6b71b0 0x55738aa87800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.927+0000 7f3509bb1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55738a6bf390 0x55738aa87d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.927+0000 7f350abb3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55738a6e0070 0x55738aa872c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.927+0000 7f3509bb1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55738a6bf390 0x55738aa87d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:35536/0 (socket says 172.21.15.64:35536) 2024-09-21T04:08:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.927+0000 7f350abb3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55738a6e0070 0x55738aa872c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:57302/0 (socket says 172.21.15.64:57302) 2024-09-21T04:08:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.927+0000 7f3509bb1640 1 -- 172.21.15.64:0/1725433963 learned_addr learned my addr 172.21.15.64:0/1725433963 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:36.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.928+0000 7f34fb7fe640 1 -- 172.21.15.64:0/3631164731 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f34f40e72f0 con 0x7f35041094c0 2024-09-21T04:08:36.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.928+0000 7f3509bb1640 1 -- 172.21.15.64:0/1725433963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55738a6e0070 msgr2=0x55738aa872c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:36.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.928+0000 7f3509bb1640 1 --2- 172.21.15.64:0/1725433963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55738a6e0070 0x55738aa872c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.928+0000 7f3509bb1640 1 -- 172.21.15.64:0/1725433963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55738a6b71b0 msgr2=0x55738aa87800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:36.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.928+0000 7f3509bb1640 1 --2- 172.21.15.64:0/1725433963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55738a6b71b0 0x55738aa87800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.928+0000 7f3509bb1640 1 -- 172.21.15.64:0/1725433963 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55738a74b650 con 0x55738a6bf390 2024-09-21T04:08:36.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.928+0000 7f350abb3640 1 --2- 172.21.15.64:0/1725433963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55738a6e0070 0x55738aa872c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:36.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.928+0000 7f350abb3640 1 --2- 172.21.15.64:0/3631164731 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f34f00b6680 0x7f34f00b8b90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f34fc005fd0 tx=0x7f34fc0125b0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:36.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.928+0000 7f3509bb1640 1 --2- 172.21.15.64:0/1725433963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55738a6bf390 0x55738aa87d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f34f40e5010 tx=0x7f34f40ac880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51ac66e854801315 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:36.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.928+0000 7f34f8ff9640 1 -- 172.21.15.64:0/1725433963 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f40a9760 con 0x55738a6bf390 2024-09-21T04:08:36.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.928+0000 7f34f8ff9640 1 -- 172.21.15.64:0/1725433963 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34f4009e50 con 0x55738a6bf390 2024-09-21T04:08:36.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.928+0000 7f34f8ff9640 1 -- 172.21.15.64:0/1725433963 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f40b1bf0 con 0x55738a6bf390 2024-09-21T04:08:36.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.929+0000 7f3512b8a740 1 -- 172.21.15.64:0/1725433963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55738a6bf390 msgr2=0x55738aa87d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:36.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.929+0000 7f3512b8a740 1 --2- 172.21.15.64:0/1725433963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55738a6bf390 0x55738aa87d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f34f40e5010 tx=0x7f34f40ac880 comp rx=0 tx=0).stop 2024-09-21T04:08:36.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.929+0000 7f3512b8a740 1 -- 172.21.15.64:0/1725433963 shutdown_connections 2024-09-21T04:08:36.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.929+0000 7f3512b8a740 1 --2- 172.21.15.64:0/1725433963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55738a6bf390 0x55738aa87d40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.929+0000 7f3512b8a740 1 --2- 172.21.15.64:0/1725433963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55738a6b71b0 0x55738aa87800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.929+0000 7f3512b8a740 1 --2- 172.21.15.64:0/1725433963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55738a6e0070 0x55738aa872c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.929+0000 7f3512b8a740 1 -- 172.21.15.64:0/1725433963 >> 172.21.15.64:0/1725433963 conn(0x55738a6d4460 msgr2=0x55738aa851b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:36.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.929+0000 7f3512b8a740 1 -- 172.21.15.64:0/1725433963 shutdown_connections 2024-09-21T04:08:36.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.930+0000 7f3512b8a740 1 -- 172.21.15.64:0/1725433963 wait complete. 2024-09-21T04:08:36.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.930+0000 7f3512b8a740 1 -- start start 2024-09-21T04:08:36.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.930+0000 7f3512b8a740 1 Processor -- start 2024-09-21T04:08:36.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.930+0000 7f3512b8a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55738a6b71b0 0x55738aa92910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:36.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.930+0000 7f3512b8a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55738a6bf390 0x55738aa92e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:36.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.931+0000 7f3512b8a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55738a6e0070 0x55738aaa7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:36.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.931+0000 7f3512b8a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55738a84e700 con 0x55738a6bf390 2024-09-21T04:08:36.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.931+0000 7f3512b8a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55738a988160 con 0x55738a6b71b0 2024-09-21T04:08:36.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.931+0000 7f3512b8a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55738aa9e250 con 0x55738a6e0070 2024-09-21T04:08:36.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.931+0000 7f350a3b2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55738a6b71b0 0x55738aa92910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:36.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.931+0000 7f3509bb1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55738a6bf390 0x55738aa92e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:36.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.931+0000 7f3512b8a740 10 client.0 _ll_register_callbacks cb 0x55738a9a19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:08:36.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.931+0000 7f350abb3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55738a6e0070 0x55738aaa7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:36.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.931+0000 7f3509bb1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55738a6bf390 0x55738aa92e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42036/0 (socket says 172.21.15.64:42036) 2024-09-21T04:08:36.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.931+0000 7f350a3b2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55738a6b71b0 0x55738aa92910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:35548/0 (socket says 172.21.15.64:35548) 2024-09-21T04:08:36.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.931+0000 7f3509bb1640 1 -- 172.21.15.64:0/2158728108 learned_addr learned my addr 172.21.15.64:0/2158728108 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:36.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.931+0000 7f350a3b2640 1 -- 172.21.15.64:0/2158728108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55738a6e0070 msgr2=0x55738aaa7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:36.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.931+0000 7f350a3b2640 1 --2- 172.21.15.64:0/2158728108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55738a6e0070 0x55738aaa7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.931+0000 7f350a3b2640 1 -- 172.21.15.64:0/2158728108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55738a6bf390 msgr2=0x55738aa92e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:36.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.931+0000 7f350a3b2640 1 --2- 172.21.15.64:0/2158728108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55738a6bf390 0x55738aa92e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.931+0000 7f350a3b2640 1 -- 172.21.15.64:0/2158728108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55738a9b5c90 con 0x55738a6b71b0 2024-09-21T04:08:36.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.931+0000 7f3509bb1640 1 --2- 172.21.15.64:0/2158728108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55738a6bf390 0x55738aa92e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:36.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.932+0000 7f350a3b2640 1 --2- 172.21.15.64:0/2158728108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55738a6b71b0 0x55738aa92910 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3500004770 tx=0x7f3500031810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3a3b144f10af686 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:36.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.932+0000 7f34edffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:08:36.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.932+0000 7f34edffb640 1 -- 172.21.15.64:0/2158728108 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3500035040 con 0x55738a6b71b0 2024-09-21T04:08:36.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.932+0000 7f34edffb640 1 -- 172.21.15.64:0/2158728108 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f3500004450 con 0x55738a6b71b0 2024-09-21T04:08:36.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.932+0000 7f3512b8a740 1 -- 172.21.15.64:0/2158728108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34f4009e50 con 0x55738a6b71b0 2024-09-21T04:08:36.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.932+0000 7f34edffb640 1 -- 172.21.15.64:0/2158728108 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3500005020 con 0x55738a6b71b0 2024-09-21T04:08:36.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.932+0000 7f3512b8a740 10 client.26204 fetch_fsmap learned FSMap version 33 2024-09-21T04:08:36.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.932+0000 7f3512b8a740 1 -- 172.21.15.64:0/2158728108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55738aaa9f00 con 0x55738a6b71b0 2024-09-21T04:08:36.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.932+0000 7f34edffb640 1 -- 172.21.15.64:0/2158728108 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34f4009e50 con 0x55738a6b71b0 2024-09-21T04:08:36.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.933+0000 7f34edffb640 1 -- 172.21.15.64:0/2158728108 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f35000035f0 con 0x55738a6b71b0 2024-09-21T04:08:36.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.933+0000 7f3512b8a740 10 client.26204 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T04:08:36.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.933+0000 7f3512b8a740 10 client.26204 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:08:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.933+0000 7f3512b8a740 20 client.26204 populate_metadata read hostname 'smithi064' 2024-09-21T04:08:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.933+0000 7f3512b8a740 1 --2- 172.21.15.64:0/2158728108 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55738a69e9f0 0x55738aaad490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.933+0000 7f3512b8a740 4 client.26204 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:08:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.933+0000 7f3512b8a740 1 -- 172.21.15.64:0/2158728108 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55738a9ffdf0 con 0x55738a69e9f0 2024-09-21T04:08:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.933+0000 7f3509bb1640 1 --2- 172.21.15.64:0/2158728108 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55738a69e9f0 0x55738aaad490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.934+0000 7f34edffb640 10 client.26204 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:08:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.934+0000 7f3509bb1640 1 --2- 172.21.15.64:0/2158728108 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55738a69e9f0 0x55738aaad490 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bfb52e5fa11d5996 server_cookie=53be30b487e7ce2c in_seq=0 out_seq=0 2024-09-21T04:08:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.934+0000 7f34edffb640 1 -- 172.21.15.64:0/2158728108 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55738a9ffdf0 con 0x55738a69e9f0 2024-09-21T04:08:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.934+0000 7f34edffb640 10 client.26204 handle_command_reply: tid=0 2024-09-21T04:08:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.934+0000 7f3512b8a740 1 client.26204 shutdown 2024-09-21T04:08:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.935+0000 7f3512b8a740 1 -- 172.21.15.64:0/2158728108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55738a6b71b0 msgr2=0x55738aa92910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.935+0000 7f3512b8a740 1 --2- 172.21.15.64:0/2158728108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55738a6b71b0 0x55738aa92910 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3500004770 tx=0x7f3500031810 comp rx=0 tx=0).stop 2024-09-21T04:08:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.935+0000 7f3512b8a740 1 -- 172.21.15.64:0/2158728108 shutdown_connections 2024-09-21T04:08:36.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.935+0000 7f3512b8a740 1 --2- 172.21.15.64:0/2158728108 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55738a69e9f0 0x55738aaad490 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.935+0000 7f3512b8a740 1 --2- 172.21.15.64:0/2158728108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55738a6e0070 0x55738aaa7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.935+0000 7f3512b8a740 1 --2- 172.21.15.64:0/2158728108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55738a6bf390 0x55738aa92e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.935+0000 7f3512b8a740 1 --2- 172.21.15.64:0/2158728108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55738a6b71b0 0x55738aa92910 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.935+0000 7f3512b8a740 1 -- 172.21.15.64:0/2158728108 >> 172.21.15.64:0/2158728108 conn(0x55738a6d4460 msgr2=0x55738aa882f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:36.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.935+0000 7f34edffb640 0 client.26204 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:08:36.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.935+0000 7f3512b8a740 1 -- 172.21.15.64:0/2158728108 shutdown_connections 2024-09-21T04:08:36.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.935+0000 7f3512b8a740 1 -- 172.21.15.64:0/2158728108 wait complete. 2024-09-21T04:08:36.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.935+0000 7f3512b8a740 20 client.26204 trim_cache size 0 max 16384 2024-09-21T04:08:36.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.949+0000 7f3512b8a740 1 Processor -- start 2024-09-21T04:08:36.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.949+0000 7f3512b8a740 1 -- start start 2024-09-21T04:08:36.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.949+0000 7f3512b8a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55738a6b71b0 0x55738aa947f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:36.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.949+0000 7f3512b8a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55738a6bf390 0x55738aa88a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:36.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.949+0000 7f3512b8a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55738a6e0070 0x55738aa84760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:36.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.949+0000 7f3512b8a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f35041affc0 con 0x55738a6b71b0 2024-09-21T04:08:36.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.949+0000 7f3512b8a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f35041ad830 con 0x55738a6bf390 2024-09-21T04:08:36.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.949+0000 7f3512b8a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f35041ab680 con 0x55738a6e0070 2024-09-21T04:08:36.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.949+0000 7f3509bb1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55738a6bf390 0x55738aa88a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:36.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.949+0000 7f350a3b2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55738a6b71b0 0x55738aa947f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:36.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.949+0000 7f3509bb1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55738a6bf390 0x55738aa88a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:35562/0 (socket says 172.21.15.64:35562) 2024-09-21T04:08:36.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.949+0000 7f350a3b2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55738a6b71b0 0x55738aa947f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42046/0 (socket says 172.21.15.64:42046) 2024-09-21T04:08:36.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.949+0000 7f3509bb1640 1 -- 172.21.15.64:0/2765051057 learned_addr learned my addr 172.21.15.64:0/2765051057 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:36.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.949+0000 7f350abb3640 1 --2- 172.21.15.64:0/2765051057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55738a6e0070 0x55738aa84760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:36.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.949+0000 7f3509bb1640 1 -- 172.21.15.64:0/2765051057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55738a6e0070 msgr2=0x55738aa84760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:36.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.949+0000 7f3509bb1640 1 --2- 172.21.15.64:0/2765051057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55738a6e0070 0x55738aa84760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.950+0000 7f3509bb1640 1 -- 172.21.15.64:0/2765051057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55738a6b71b0 msgr2=0x55738aa947f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:36.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.950+0000 7f3509bb1640 1 --2- 172.21.15.64:0/2765051057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55738a6b71b0 0x55738aa947f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.950+0000 7f3509bb1640 1 -- 172.21.15.64:0/2765051057 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55738aa93bd0 con 0x55738a6bf390 2024-09-21T04:08:36.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.950+0000 7f3509bb1640 1 --2- 172.21.15.64:0/2765051057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55738a6bf390 0x55738aa88a00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f34f4005c80 tx=0x7f34f4005c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dfe4f7eee31aa30 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:36.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.950+0000 7f34effff640 1 -- 172.21.15.64:0/2765051057 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f40a9760 con 0x55738a6bf390 2024-09-21T04:08:36.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.950+0000 7f34effff640 1 -- 172.21.15.64:0/2765051057 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34f4009e50 con 0x55738a6bf390 2024-09-21T04:08:36.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.950+0000 7f34effff640 1 -- 172.21.15.64:0/2765051057 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f40b1bf0 con 0x55738a6bf390 2024-09-21T04:08:36.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.951+0000 7f3512b8a740 1 -- 172.21.15.64:0/2765051057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55738a6bf390 msgr2=0x55738aa88a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:36.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.951+0000 7f3512b8a740 1 --2- 172.21.15.64:0/2765051057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55738a6bf390 0x55738aa88a00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f34f4005c80 tx=0x7f34f4005c10 comp rx=0 tx=0).stop 2024-09-21T04:08:36.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.951+0000 7f3512b8a740 1 -- 172.21.15.64:0/2765051057 shutdown_connections 2024-09-21T04:08:36.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.951+0000 7f3512b8a740 1 --2- 172.21.15.64:0/2765051057 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55738a6e0070 0x55738aa84760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.951+0000 7f3512b8a740 1 --2- 172.21.15.64:0/2765051057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55738a6bf390 0x55738aa88a00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.951+0000 7f3512b8a740 1 --2- 172.21.15.64:0/2765051057 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55738a6b71b0 0x55738aa947f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.951+0000 7f3512b8a740 1 -- 172.21.15.64:0/2765051057 >> 172.21.15.64:0/2765051057 conn(0x55738a6d4460 msgr2=0x55738aa882f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:36.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.951+0000 7f3512b8a740 1 -- 172.21.15.64:0/2765051057 shutdown_connections 2024-09-21T04:08:36.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.951+0000 7f3512b8a740 1 -- 172.21.15.64:0/2765051057 wait complete. 2024-09-21T04:08:36.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.951+0000 7f3512b8a740 1 -- start start 2024-09-21T04:08:36.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.951+0000 7f3512b8a740 1 Processor -- start 2024-09-21T04:08:36.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.952+0000 7f3512b8a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55738a6b71b0 0x55738aa9ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:36.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.952+0000 7f3512b8a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55738a6bf390 0x55738aa9f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:36.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.952+0000 7f3512b8a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55738a6e0070 0x55738ab0fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:36.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.952+0000 7f3512b8a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f35041abcf0 con 0x55738a6e0070 2024-09-21T04:08:36.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.952+0000 7f3512b8a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f35041ad160 con 0x55738a6b71b0 2024-09-21T04:08:36.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.952+0000 7f3512b8a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f35041ac3c0 con 0x55738a6bf390 2024-09-21T04:08:36.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.952+0000 7f3512b8a740 10 client.0 _ll_register_callbacks cb 0x7f3500005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:08:36.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.952+0000 7f350abb3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55738a6e0070 0x55738ab0fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:36.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.952+0000 7f350a3b2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55738a6b71b0 0x55738aa9ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:36.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.952+0000 7f350abb3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55738a6e0070 0x55738ab0fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42060/0 (socket says 172.21.15.64:42060) 2024-09-21T04:08:36.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.952+0000 7f350abb3640 1 -- 172.21.15.64:0/1070614327 learned_addr learned my addr 172.21.15.64:0/1070614327 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:36.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.952+0000 7f3509bb1640 1 --2- 172.21.15.64:0/1070614327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55738a6bf390 0x55738aa9f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:36.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.952+0000 7f350abb3640 1 -- 172.21.15.64:0/1070614327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55738a6bf390 msgr2=0x55738aa9f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:36.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.952+0000 7f350abb3640 1 --2- 172.21.15.64:0/1070614327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55738a6bf390 0x55738aa9f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.952+0000 7f350abb3640 1 -- 172.21.15.64:0/1070614327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55738a6b71b0 msgr2=0x55738aa9ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:36.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.952+0000 7f350abb3640 1 --2- 172.21.15.64:0/1070614327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55738a6b71b0 0x55738aa9ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.952+0000 7f350abb3640 1 -- 172.21.15.64:0/1070614327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55738ab10530 con 0x55738a6e0070 2024-09-21T04:08:36.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.952+0000 7f3509bb1640 1 --2- 172.21.15.64:0/1070614327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55738a6bf390 0x55738aa9f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:36.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.953+0000 7f350a3b2640 1 --2- 172.21.15.64:0/1070614327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55738a6b71b0 0x55738aa9ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:36.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.953+0000 7f350abb3640 1 --2- 172.21.15.64:0/1070614327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55738a6e0070 0x55738ab0fec0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f34fc010bd0 tx=0x7f34fc01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea03813ff02adead server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:36.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.953+0000 7f34edffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:08:36.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.953+0000 7f34edffb640 1 -- 172.21.15.64:0/1070614327 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc020790 con 0x55738a6e0070 2024-09-21T04:08:36.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.953+0000 7f34edffb640 1 -- 172.21.15.64:0/1070614327 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f34fc020930 con 0x55738a6e0070 2024-09-21T04:08:36.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.953+0000 7f3512b8a740 1 -- 172.21.15.64:0/1070614327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34f4009e50 con 0x55738a6e0070 2024-09-21T04:08:36.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.953+0000 7f34edffb640 1 -- 172.21.15.64:0/1070614327 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f34fc038050 con 0x55738a6e0070 2024-09-21T04:08:36.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.953+0000 7f3512b8a740 10 client.16506 fetch_fsmap learned FSMap version 33 2024-09-21T04:08:36.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.953+0000 7f3512b8a740 1 -- 172.21.15.64:0/1070614327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55738ab17920 con 0x55738a6e0070 2024-09-21T04:08:36.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.954+0000 7f34edffb640 1 -- 172.21.15.64:0/1070614327 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34f4009e50 con 0x55738a6e0070 2024-09-21T04:08:36.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.954+0000 7f34edffb640 1 -- 172.21.15.64:0/1070614327 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f34fc01f070 con 0x55738a6e0070 2024-09-21T04:08:36.959 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.954+0000 7f3512b8a740 10 client.16506 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T04:08:36.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.954+0000 7f3512b8a740 10 client.16506 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:08:36.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.954+0000 7f3512b8a740 20 client.16506 populate_metadata read hostname 'smithi064' 2024-09-21T04:08:36.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.954+0000 7f3512b8a740 1 --2- 172.21.15.64:0/1070614327 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55738a673da0 0x55738ab1a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:36.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.954+0000 7f3512b8a740 4 client.16506 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:08:36.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.954+0000 7f3512b8a740 1 -- 172.21.15.64:0/1070614327 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55738a9fb8a0 con 0x55738a673da0 2024-09-21T04:08:36.960 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.954+0000 7f350a3b2640 1 --2- 172.21.15.64:0/1070614327 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55738a673da0 0x55738ab1a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:36.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.955+0000 7f34edffb640 10 client.16506 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:08:36.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.955+0000 7f350a3b2640 1 --2- 172.21.15.64:0/1070614327 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55738a673da0 0x55738ab1a190 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=26660e6352dd7c4d server_cookie=46ae3838952278ee in_seq=0 out_seq=0 2024-09-21T04:08:36.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.955+0000 7f34edffb640 1 -- 172.21.15.64:0/1070614327 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55738a9fb8a0 con 0x55738a673da0 2024-09-21T04:08:36.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.955+0000 7f34edffb640 10 client.16506 handle_command_reply: tid=0 2024-09-21T04:08:36.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.955+0000 7f3512b8a740 1 client.16506 shutdown 2024-09-21T04:08:36.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.955+0000 7f3512b8a740 1 -- 172.21.15.64:0/1070614327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55738a6e0070 msgr2=0x55738ab0fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:36.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.955+0000 7f3512b8a740 1 --2- 172.21.15.64:0/1070614327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55738a6e0070 0x55738ab0fec0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f34fc010bd0 tx=0x7f34fc01c000 comp rx=0 tx=0).stop 2024-09-21T04:08:36.961 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.955+0000 7f3512b8a740 1 -- 172.21.15.64:0/1070614327 shutdown_connections 2024-09-21T04:08:36.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.955+0000 7f3512b8a740 1 --2- 172.21.15.64:0/1070614327 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55738a673da0 0x55738ab1a190 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.955+0000 7f3512b8a740 1 --2- 172.21.15.64:0/1070614327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55738a6e0070 0x55738ab0fec0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.955+0000 7f3512b8a740 1 --2- 172.21.15.64:0/1070614327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55738a6bf390 0x55738aa9f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.955+0000 7f3512b8a740 1 --2- 172.21.15.64:0/1070614327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55738a6b71b0 0x55738aa9ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.955+0000 7f3512b8a740 1 -- 172.21.15.64:0/1070614327 >> 172.21.15.64:0/1070614327 conn(0x55738a6d4460 msgr2=0x55738aa88cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:36.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.955+0000 7f34edffb640 0 client.16506 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:08:36.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.955+0000 7f3512b8a740 1 -- 172.21.15.64:0/1070614327 shutdown_connections 2024-09-21T04:08:36.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.955+0000 7f3512b8a740 1 -- 172.21.15.64:0/1070614327 wait complete. 2024-09-21T04:08:36.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.955+0000 7f3512b8a740 20 client.16506 trim_cache size 0 max 16384 2024-09-21T04:08:36.962 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:08:36.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.956+0000 7f350c616640 1 -- 172.21.15.64:0/3631164731 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f34f00b6680 msgr2=0x7f34f00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:36.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.956+0000 7f350c616640 1 --2- 172.21.15.64:0/3631164731 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f34f00b6680 0x7f34f00b8b90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f34fc005fd0 tx=0x7f34fc0125b0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.956+0000 7f350c616640 1 -- 172.21.15.64:0/3631164731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35041094c0 msgr2=0x7f3504075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:36.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.956+0000 7f350c616640 1 --2- 172.21.15.64:0/3631164731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35041094c0 0x7f3504075c50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f350406a800 tx=0x7f34f400dbb0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.956+0000 7f350c616640 1 -- 172.21.15.64:0/3631164731 shutdown_connections 2024-09-21T04:08:36.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.956+0000 7f350c616640 1 --2- 172.21.15.64:0/3631164731 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f34f00b6680 0x7f34f00b8b90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.963 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.956+0000 7f350c616640 1 --2- 172.21.15.64:0/3631164731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3504109e10 0x7f3504076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.956+0000 7f350c616640 1 --2- 172.21.15.64:0/3631164731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f35041094c0 0x7f3504075c50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.956+0000 7f350c616640 1 --2- 172.21.15.64:0/3631164731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3504103b80 0x7f3504075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:36.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.956+0000 7f350c616640 1 -- 172.21.15.64:0/3631164731 >> 172.21.15.64:0/3631164731 conn(0x7f35040ff610 msgr2=0x7f3504101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:36.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.956+0000 7f350c616640 1 -- 172.21.15.64:0/3631164731 shutdown_connections 2024-09-21T04:08:36.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:36.956+0000 7f350c616640 1 -- 172.21.15.64:0/3631164731 wait complete. 2024-09-21T04:08:37.888 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T04:08:38.161 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:08:38.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:37 smithi119 ceph-mon[34811]: pgmap v459: 97 pgs: 97 active+clean; 594 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 250 op/s 2024-09-21T04:08:38.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:37 smithi064 ceph-mon[31180]: pgmap v459: 97 pgs: 97 active+clean; 594 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 250 op/s 2024-09-21T04:08:38.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:37 smithi131 ceph-mon[35193]: pgmap v459: 97 pgs: 97 active+clean; 594 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 250 op/s 2024-09-21T04:08:38.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.866+0000 7f54b398b640 1 -- 172.21.15.64:0/394759469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54ac103b90 msgr2=0x7f54ac103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:38.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.866+0000 7f54b398b640 1 --2- 172.21.15.64:0/394759469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54ac103b90 0x7f54ac103fa0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009900 tx=0x7f54a802a410 comp rx=0 tx=0).stop 2024-09-21T04:08:38.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.866+0000 7f54b398b640 1 -- 172.21.15.64:0/394759469 shutdown_connections 2024-09-21T04:08:38.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.866+0000 7f54b398b640 1 --2- 172.21.15.64:0/394759469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54ac109e20 0x7f54ac110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.866+0000 7f54b398b640 1 --2- 172.21.15.64:0/394759469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54ac1094d0 0x7f54ac1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.866+0000 7f54b398b640 1 --2- 172.21.15.64:0/394759469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54ac103b90 0x7f54ac103fa0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.866+0000 7f54b398b640 1 -- 172.21.15.64:0/394759469 >> 172.21.15.64:0/394759469 conn(0x7f54ac0ff600 msgr2=0x7f54ac101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:38.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.866+0000 7f54b398b640 1 -- 172.21.15.64:0/394759469 shutdown_connections 2024-09-21T04:08:38.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.866+0000 7f54b398b640 1 -- 172.21.15.64:0/394759469 wait complete. 2024-09-21T04:08:38.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.867+0000 7f54b398b640 1 Processor -- start 2024-09-21T04:08:38.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.867+0000 7f54b398b640 1 -- start start 2024-09-21T04:08:38.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.867+0000 7f54b398b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54ac103b90 0x7f54ac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:38.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.867+0000 7f54b398b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54ac1094d0 0x7f54ac075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:38.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.867+0000 7f54b398b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54ac109e20 0x7f54ac076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:38.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.867+0000 7f54b398b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac113ae0 con 0x7f54ac103b90 2024-09-21T04:08:38.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.867+0000 7f54b398b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac113960 con 0x7f54ac109e20 2024-09-21T04:08:38.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.867+0000 7f54b398b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac113c60 con 0x7f54ac1094d0 2024-09-21T04:08:38.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.868+0000 7f54b1727640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54ac103b90 0x7f54ac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:38.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.868+0000 7f54b1727640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54ac103b90 0x7f54ac075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42070/0 (socket says 172.21.15.64:42070) 2024-09-21T04:08:38.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.868+0000 7f54b1727640 1 -- 172.21.15.64:0/3898650843 learned_addr learned my addr 172.21.15.64:0/3898650843 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:38.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.868+0000 7f54b1f28640 1 --2- 172.21.15.64:0/3898650843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54ac109e20 0x7f54ac076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:38.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.868+0000 7f54b0f26640 1 --2- 172.21.15.64:0/3898650843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54ac1094d0 0x7f54ac075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:38.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.868+0000 7f54b1727640 1 -- 172.21.15.64:0/3898650843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54ac1094d0 msgr2=0x7f54ac075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:38.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.868+0000 7f54b1727640 1 --2- 172.21.15.64:0/3898650843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54ac1094d0 0x7f54ac075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.868+0000 7f54b1727640 1 -- 172.21.15.64:0/3898650843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54ac109e20 msgr2=0x7f54ac076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:38.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.868+0000 7f54b1727640 1 --2- 172.21.15.64:0/3898650843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54ac109e20 0x7f54ac076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.868+0000 7f54b1727640 1 -- 172.21.15.64:0/3898650843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54ac1a8770 con 0x7f54ac103b90 2024-09-21T04:08:38.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.868+0000 7f54b1f28640 1 --2- 172.21.15.64:0/3898650843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54ac109e20 0x7f54ac076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:38.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.868+0000 7f54b1727640 1 --2- 172.21.15.64:0/3898650843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54ac103b90 0x7f54ac075710 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f54a8005940 tx=0x7f54a8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:38.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.868+0000 7f54a27fc640 1 -- 172.21.15.64:0/3898650843 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a803f030 con 0x7f54ac103b90 2024-09-21T04:08:38.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.868+0000 7f54a27fc640 1 -- 172.21.15.64:0/3898650843 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54a8003aa0 con 0x7f54ac103b90 2024-09-21T04:08:38.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.868+0000 7f54a27fc640 1 -- 172.21.15.64:0/3898650843 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a8035070 con 0x7f54ac103b90 2024-09-21T04:08:38.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.868+0000 7f54b398b640 1 -- 172.21.15.64:0/3898650843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54ac1a8a00 con 0x7f54ac103b90 2024-09-21T04:08:38.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.868+0000 7f54b398b640 1 -- 172.21.15.64:0/3898650843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54ac1a8ee0 con 0x7f54ac103b90 2024-09-21T04:08:38.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.872+0000 7f54b9eff740 1 Processor -- start 2024-09-21T04:08:38.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.872+0000 7f54b9eff740 1 -- start start 2024-09-21T04:08:38.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.872+0000 7f54a27fc640 1 -- 172.21.15.64:0/3898650843 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f54a8042030 con 0x7f54ac103b90 2024-09-21T04:08:38.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.872+0000 7f54b9eff740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f03593070 0x560f0393a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:38.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.873+0000 7f54b9eff740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f0356a1b0 0x560f0393abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:38.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.873+0000 7f54b9eff740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f03572390 0x560f0393b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:38.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.873+0000 7f54b9eff740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560f038b0ea0 con 0x560f0356a1b0 2024-09-21T04:08:38.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.873+0000 7f54b9eff740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560f038a02d0 con 0x560f03572390 2024-09-21T04:08:38.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.873+0000 7f54b9eff740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560f038a40e0 con 0x560f03593070 2024-09-21T04:08:38.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.873+0000 7f54b1727640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f0356a1b0 0x560f0393abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:38.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.873+0000 7f54b0f26640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f03572390 0x560f0393b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:38.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.873+0000 7f54b1727640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f0356a1b0 0x560f0393abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42084/0 (socket says 172.21.15.64:42084) 2024-09-21T04:08:38.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.873+0000 7f54b1727640 1 -- 172.21.15.64:0/427790712 learned_addr learned my addr 172.21.15.64:0/427790712 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:38.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.873+0000 7f54b1f28640 1 --2- 172.21.15.64:0/427790712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f03593070 0x560f0393a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:38.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.873+0000 7f54b0f26640 1 -- 172.21.15.64:0/427790712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f03593070 msgr2=0x560f0393a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:38.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.873+0000 7f54b0f26640 1 --2- 172.21.15.64:0/427790712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f03593070 0x560f0393a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.873+0000 7f54b0f26640 1 -- 172.21.15.64:0/427790712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f0356a1b0 msgr2=0x560f0393abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:38.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.873+0000 7f54b0f26640 1 --2- 172.21.15.64:0/427790712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f0356a1b0 0x560f0393abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.873+0000 7f54b0f26640 1 -- 172.21.15.64:0/427790712 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f035fe650 con 0x560f03572390 2024-09-21T04:08:38.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.873+0000 7f54b1727640 1 --2- 172.21.15.64:0/427790712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f0356a1b0 0x560f0393abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:38.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.873+0000 7f54a27fc640 1 --2- 172.21.15.64:0/3898650843 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f54980b6680 0x7f54980b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:38.886 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.873+0000 7f54b1f28640 1 --2- 172.21.15.64:0/3898650843 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f54980b6680 0x7f54980b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:38.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.873+0000 7f54b0f26640 1 --2- 172.21.15.64:0/427790712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f03572390 0x560f0393b120 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f54ac06bab0 tx=0x7f549c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d9fd42b59c8044d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:38.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.873+0000 7f54a27fc640 1 -- 172.21.15.64:0/3898650843 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f54a8107db0 con 0x7f54ac103b90 2024-09-21T04:08:38.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.874+0000 7f54977fe640 1 -- 172.21.15.64:0/427790712 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c004050 con 0x560f03572390 2024-09-21T04:08:38.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.874+0000 7f54977fe640 1 -- 172.21.15.64:0/427790712 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f549c0054d0 con 0x560f03572390 2024-09-21T04:08:38.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.874+0000 7f54977fe640 1 -- 172.21.15.64:0/427790712 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c004c80 con 0x560f03572390 2024-09-21T04:08:38.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.875+0000 7f54b9eff740 1 -- 172.21.15.64:0/427790712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f03572390 msgr2=0x560f0393b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:38.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.875+0000 7f54b1f28640 1 --2- 172.21.15.64:0/3898650843 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f54980b6680 0x7f54980b8b90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f54a4005fd0 tx=0x7f54a4009600 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:38.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.875+0000 7f54b9eff740 1 --2- 172.21.15.64:0/427790712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f03572390 0x560f0393b120 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f54ac06bab0 tx=0x7f549c011810 comp rx=0 tx=0).stop 2024-09-21T04:08:38.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.875+0000 7f54b9eff740 1 -- 172.21.15.64:0/427790712 shutdown_connections 2024-09-21T04:08:38.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.875+0000 7f54b9eff740 1 --2- 172.21.15.64:0/427790712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f03572390 0x560f0393b120 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.875+0000 7f54b9eff740 1 --2- 172.21.15.64:0/427790712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f0356a1b0 0x560f0393abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.875+0000 7f54b9eff740 1 --2- 172.21.15.64:0/427790712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f03593070 0x560f0393a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.875+0000 7f54b9eff740 1 -- 172.21.15.64:0/427790712 >> 172.21.15.64:0/427790712 conn(0x560f03587460 msgr2=0x560f03938590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:38.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.875+0000 7f54b9eff740 1 -- 172.21.15.64:0/427790712 shutdown_connections 2024-09-21T04:08:38.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.875+0000 7f54b9eff740 1 -- 172.21.15.64:0/427790712 wait complete. 2024-09-21T04:08:38.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.875+0000 7f54b9eff740 1 -- start start 2024-09-21T04:08:38.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.875+0000 7f54b9eff740 1 Processor -- start 2024-09-21T04:08:38.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.876+0000 7f54b9eff740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f0356a1b0 0x560f03945cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:38.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.876+0000 7f54b9eff740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f03572390 0x560f03946230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:38.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.876+0000 7f54b9eff740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f03593070 0x560f03946770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:38.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.876+0000 7f54b9eff740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560f03701700 con 0x560f03572390 2024-09-21T04:08:38.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.876+0000 7f54b9eff740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560f0383b160 con 0x560f0356a1b0 2024-09-21T04:08:38.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.876+0000 7f54b9eff740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560f03948e80 con 0x560f03593070 2024-09-21T04:08:38.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.876+0000 7f54b9eff740 10 client.0 _ll_register_callbacks cb 0x560f038549e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:08:38.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.876+0000 7f54b0f26640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f03572390 0x560f03946230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:38.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.876+0000 7f54b1727640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f0356a1b0 0x560f03945cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:38.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.876+0000 7f54b0f26640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f03572390 0x560f03946230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42100/0 (socket says 172.21.15.64:42100) 2024-09-21T04:08:38.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.876+0000 7f54b0f26640 1 -- 172.21.15.64:0/1063331973 learned_addr learned my addr 172.21.15.64:0/1063331973 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:38.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.876+0000 7f54b1f28640 1 --2- 172.21.15.64:0/1063331973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f03593070 0x560f03946770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:38.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.876+0000 7f54b1727640 1 -- 172.21.15.64:0/1063331973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f03593070 msgr2=0x560f03946770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:38.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.876+0000 7f54b1727640 1 --2- 172.21.15.64:0/1063331973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f03593070 0x560f03946770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.876+0000 7f54b1727640 1 -- 172.21.15.64:0/1063331973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f03572390 msgr2=0x560f03946230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:38.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.876+0000 7f54b1727640 1 --2- 172.21.15.64:0/1063331973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f03572390 0x560f03946230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.876+0000 7f54b1727640 1 -- 172.21.15.64:0/1063331973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f03868c90 con 0x560f0356a1b0 2024-09-21T04:08:38.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.877+0000 7f54b1727640 1 --2- 172.21.15.64:0/1063331973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f0356a1b0 0x560f03945cf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x560f03939320 tx=0x7f54a81059d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a968d6531dfd27f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:38.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.877+0000 7f5494ff9640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:08:38.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.877+0000 7f5494ff9640 1 -- 172.21.15.64:0/1063331973 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a8049030 con 0x560f0356a1b0 2024-09-21T04:08:38.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.877+0000 7f5494ff9640 1 -- 172.21.15.64:0/1063331973 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f54a80f10b0 con 0x560f0356a1b0 2024-09-21T04:08:38.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.877+0000 7f54b9eff740 1 -- 172.21.15.64:0/1063331973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f549c0054d0 con 0x560f0356a1b0 2024-09-21T04:08:38.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.877+0000 7f5494ff9640 1 -- 172.21.15.64:0/1063331973 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f54a80ca2d0 con 0x560f0356a1b0 2024-09-21T04:08:38.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.877+0000 7f54b9eff740 10 client.26222 fetch_fsmap learned FSMap version 33 2024-09-21T04:08:38.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.877+0000 7f54b9eff740 1 -- 172.21.15.64:0/1063331973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560f03954b30 con 0x560f0356a1b0 2024-09-21T04:08:38.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.877+0000 7f5494ff9640 1 -- 172.21.15.64:0/1063331973 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f549c0054d0 con 0x560f0356a1b0 2024-09-21T04:08:38.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.877+0000 7f5494ff9640 1 -- 172.21.15.64:0/1063331973 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f54a80efbe0 con 0x560f0356a1b0 2024-09-21T04:08:38.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.877+0000 7f54b9eff740 10 client.26222 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T04:08:38.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.877+0000 7f54b9eff740 10 client.26222 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:08:38.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.877+0000 7f54b9eff740 20 client.26222 populate_metadata read hostname 'smithi064' 2024-09-21T04:08:38.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.877+0000 7f54b9eff740 1 --2- 172.21.15.64:0/1063331973 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x560f035519f0 0x560f03958120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:38.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.877+0000 7f54b9eff740 4 client.26222 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:08:38.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.877+0000 7f54b9eff740 1 -- 172.21.15.64:0/1063331973 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f038b2df0 con 0x560f035519f0 2024-09-21T04:08:38.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.878+0000 7f54b0f26640 1 --2- 172.21.15.64:0/1063331973 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x560f035519f0 0x560f03958120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:38.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.878+0000 7f5494ff9640 10 client.26222 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:08:38.891 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.878+0000 7f54b0f26640 1 --2- 172.21.15.64:0/1063331973 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x560f035519f0 0x560f03958120 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c6169c6bd4979540 server_cookie=54491211415f8a9d in_seq=0 out_seq=0 2024-09-21T04:08:38.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.882+0000 7f5494ff9640 1 -- 172.21.15.64:0/1063331973 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560f038b2df0 con 0x560f035519f0 2024-09-21T04:08:38.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.882+0000 7f5494ff9640 10 client.26222 handle_command_reply: tid=0 2024-09-21T04:08:38.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.882+0000 7f54b9eff740 1 client.26222 shutdown 2024-09-21T04:08:38.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.882+0000 7f54b9eff740 1 -- 172.21.15.64:0/1063331973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f0356a1b0 msgr2=0x560f03945cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:38.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.882+0000 7f54b9eff740 1 --2- 172.21.15.64:0/1063331973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f0356a1b0 0x560f03945cf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x560f03939320 tx=0x7f54a81059d0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.882+0000 7f54b9eff740 1 -- 172.21.15.64:0/1063331973 shutdown_connections 2024-09-21T04:08:38.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.882+0000 7f54b9eff740 1 --2- 172.21.15.64:0/1063331973 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x560f035519f0 0x560f03958120 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.892 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.882+0000 7f54b9eff740 1 --2- 172.21.15.64:0/1063331973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f03593070 0x560f03946770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.882+0000 7f54b9eff740 1 --2- 172.21.15.64:0/1063331973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f03572390 0x560f03946230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.882+0000 7f5494ff9640 0 client.26222 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:08:38.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.882+0000 7f54b9eff740 1 --2- 172.21.15.64:0/1063331973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f0356a1b0 0x560f03945cf0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.882+0000 7f54b9eff740 1 -- 172.21.15.64:0/1063331973 >> 172.21.15.64:0/1063331973 conn(0x560f03587460 msgr2=0x560f0393d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:38.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.882+0000 7f54b9eff740 1 -- 172.21.15.64:0/1063331973 shutdown_connections 2024-09-21T04:08:38.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.882+0000 7f54b9eff740 1 -- 172.21.15.64:0/1063331973 wait complete. 2024-09-21T04:08:38.893 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.882+0000 7f54b9eff740 20 client.26222 trim_cache size 0 max 16384 2024-09-21T04:08:38.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.896+0000 7f54b9eff740 1 Processor -- start 2024-09-21T04:08:38.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.896+0000 7f54b9eff740 1 -- start start 2024-09-21T04:08:38.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.896+0000 7f54b9eff740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f0356a1b0 0x560f03947210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:38.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.896+0000 7f54b9eff740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f03572390 0x560f03937bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:38.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.896+0000 7f54b9eff740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f03593070 0x560f03938100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:38.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.896+0000 7f54b9eff740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac1b42a0 con 0x560f03572390 2024-09-21T04:08:38.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.896+0000 7f54b9eff740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac1b1b10 con 0x560f03593070 2024-09-21T04:08:38.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.896+0000 7f54b9eff740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac1af960 con 0x560f0356a1b0 2024-09-21T04:08:38.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.896+0000 7f54b0f26640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f03572390 0x560f03937bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:38.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.896+0000 7f54b0f26640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f03572390 0x560f03937bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42114/0 (socket says 172.21.15.64:42114) 2024-09-21T04:08:38.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.896+0000 7f54b0f26640 1 -- 172.21.15.64:0/3086456870 learned_addr learned my addr 172.21.15.64:0/3086456870 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:38.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.896+0000 7f54b1f28640 1 --2- 172.21.15.64:0/3086456870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f03593070 0x560f03938100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:38.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.896+0000 7f54b1727640 1 --2- 172.21.15.64:0/3086456870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f0356a1b0 0x560f03947210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:38.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.897+0000 7f54b0f26640 1 -- 172.21.15.64:0/3086456870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f0356a1b0 msgr2=0x560f03947210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:38.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.897+0000 7f54b0f26640 1 --2- 172.21.15.64:0/3086456870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f0356a1b0 0x560f03947210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.897+0000 7f54b0f26640 1 -- 172.21.15.64:0/3086456870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f03593070 msgr2=0x560f03938100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:38.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.897+0000 7f54b0f26640 1 --2- 172.21.15.64:0/3086456870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f03593070 0x560f03938100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.897+0000 7f54b0f26640 1 -- 172.21.15.64:0/3086456870 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f039480c0 con 0x560f03572390 2024-09-21T04:08:38.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.897+0000 7f54b1f28640 1 --2- 172.21.15.64:0/3086456870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f03593070 0x560f03938100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:38.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.897+0000 7f54b0f26640 1 --2- 172.21.15.64:0/3086456870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f03572390 0x560f03937bc0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f549c009b40 tx=0x7f549c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80b511f3dcbffb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:38.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.897+0000 7f5496ffd640 1 -- 172.21.15.64:0/3086456870 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c030070 con 0x560f03572390 2024-09-21T04:08:38.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.897+0000 7f5496ffd640 1 -- 172.21.15.64:0/3086456870 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f549c0198e0 con 0x560f03572390 2024-09-21T04:08:38.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.897+0000 7f5496ffd640 1 -- 172.21.15.64:0/3086456870 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c01d040 con 0x560f03572390 2024-09-21T04:08:38.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.898+0000 7f54b9eff740 1 -- 172.21.15.64:0/3086456870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f03572390 msgr2=0x560f03937bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:38.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.898+0000 7f54b9eff740 1 --2- 172.21.15.64:0/3086456870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f03572390 0x560f03937bc0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f549c009b40 tx=0x7f549c011810 comp rx=0 tx=0).stop 2024-09-21T04:08:38.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.898+0000 7f54b9eff740 1 -- 172.21.15.64:0/3086456870 shutdown_connections 2024-09-21T04:08:38.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.898+0000 7f54b9eff740 1 --2- 172.21.15.64:0/3086456870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f03593070 0x560f03938100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.898+0000 7f54b9eff740 1 --2- 172.21.15.64:0/3086456870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f03572390 0x560f03937bc0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.898+0000 7f54b9eff740 1 --2- 172.21.15.64:0/3086456870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f0356a1b0 0x560f03947210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.898+0000 7f54b9eff740 1 -- 172.21.15.64:0/3086456870 >> 172.21.15.64:0/3086456870 conn(0x560f03587460 msgr2=0x560f0393c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:38.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.898+0000 7f54b9eff740 1 -- 172.21.15.64:0/3086456870 shutdown_connections 2024-09-21T04:08:38.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.898+0000 7f54b9eff740 1 -- 172.21.15.64:0/3086456870 wait complete. 2024-09-21T04:08:38.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.898+0000 7f54b9eff740 1 -- start start 2024-09-21T04:08:38.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.899+0000 7f54b9eff740 1 Processor -- start 2024-09-21T04:08:38.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.899+0000 7f54b9eff740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f0356a1b0 0x560f0394e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:38.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.899+0000 7f54b9eff740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f03572390 0x560f03949dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:38.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.899+0000 7f54b9eff740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f03593070 0x560f0394a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:38.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.899+0000 7f54b9eff740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac1affd0 con 0x560f03572390 2024-09-21T04:08:38.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.899+0000 7f54b9eff740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac1b1440 con 0x560f0356a1b0 2024-09-21T04:08:38.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.899+0000 7f54b9eff740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac1b06a0 con 0x560f03593070 2024-09-21T04:08:38.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.899+0000 7f54b9eff740 10 client.0 _ll_register_callbacks cb 0x7f54a8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:08:38.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.899+0000 7f54b0f26640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f03572390 0x560f03949dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:38.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.899+0000 7f54b1727640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f0356a1b0 0x560f0394e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:38.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.899+0000 7f54b0f26640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f03572390 0x560f03949dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42128/0 (socket says 172.21.15.64:42128) 2024-09-21T04:08:38.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.899+0000 7f54b0f26640 1 -- 172.21.15.64:0/2975908207 learned_addr learned my addr 172.21.15.64:0/2975908207 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:38.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.900+0000 7f54b1f28640 1 --2- 172.21.15.64:0/2975908207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f03593070 0x560f0394a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:38.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.900+0000 7f54b0f26640 1 -- 172.21.15.64:0/2975908207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f03593070 msgr2=0x560f0394a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:38.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.900+0000 7f54b0f26640 1 --2- 172.21.15.64:0/2975908207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f03593070 0x560f0394a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.900+0000 7f54b0f26640 1 -- 172.21.15.64:0/2975908207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f0356a1b0 msgr2=0x560f0394e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:38.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.900+0000 7f54b0f26640 1 --2- 172.21.15.64:0/2975908207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f0356a1b0 0x560f0394e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.900+0000 7f54b0f26640 1 -- 172.21.15.64:0/2975908207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f0394aa50 con 0x560f03572390 2024-09-21T04:08:38.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.900+0000 7f54b1727640 1 --2- 172.21.15.64:0/2975908207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f0356a1b0 0x560f0394e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:38.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.900+0000 7f54b0f26640 1 --2- 172.21.15.64:0/2975908207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f03572390 0x560f03949dd0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f549c005b90 tx=0x7f549c004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c657a5842f11646 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:38.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.900+0000 7f5494ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:08:38.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.900+0000 7f5494ff9640 1 -- 172.21.15.64:0/2975908207 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c030040 con 0x560f03572390 2024-09-21T04:08:38.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.900+0000 7f5494ff9640 1 -- 172.21.15.64:0/2975908207 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f549c019430 con 0x560f03572390 2024-09-21T04:08:38.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.900+0000 7f54b9eff740 1 -- 172.21.15.64:0/2975908207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f549c0198e0 con 0x560f03572390 2024-09-21T04:08:38.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.900+0000 7f5494ff9640 1 -- 172.21.15.64:0/2975908207 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f549c041050 con 0x560f03572390 2024-09-21T04:08:38.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.900+0000 7f54b9eff740 10 client.16524 fetch_fsmap learned FSMap version 33 2024-09-21T04:08:38.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.900+0000 7f54b9eff740 1 -- 172.21.15.64:0/2975908207 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560f039c29c0 con 0x560f03572390 2024-09-21T04:08:38.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.901+0000 7f5494ff9640 1 -- 172.21.15.64:0/2975908207 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f549c0198e0 con 0x560f03572390 2024-09-21T04:08:38.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.901+0000 7f5494ff9640 1 -- 172.21.15.64:0/2975908207 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f549c033070 con 0x560f03572390 2024-09-21T04:08:38.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.901+0000 7f54b9eff740 10 client.16524 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T04:08:38.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.901+0000 7f54b9eff740 10 client.16524 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:08:38.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.901+0000 7f54b9eff740 20 client.16524 populate_metadata read hostname 'smithi064' 2024-09-21T04:08:38.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.901+0000 7f54b9eff740 1 --2- 172.21.15.64:0/2975908207 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x560f03526da0 0x560f039c4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:38.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.901+0000 7f54b9eff740 4 client.16524 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:08:38.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.901+0000 7f54b9eff740 1 -- 172.21.15.64:0/2975908207 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f038ae8a0 con 0x560f03526da0 2024-09-21T04:08:38.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.901+0000 7f54b1727640 1 --2- 172.21.15.64:0/2975908207 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x560f03526da0 0x560f039c4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:38.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.902+0000 7f5494ff9640 10 client.16524 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:08:38.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.902+0000 7f54b1727640 1 --2- 172.21.15.64:0/2975908207 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x560f03526da0 0x560f039c4f50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6e624c1b355d8a46 server_cookie=5b5ac5f1db8b7f8 in_seq=0 out_seq=0 2024-09-21T04:08:38.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.905+0000 7f5494ff9640 1 -- 172.21.15.64:0/2975908207 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f038ae8a0 con 0x560f03526da0 2024-09-21T04:08:38.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.905+0000 7f5494ff9640 10 client.16524 handle_command_reply: tid=0 2024-09-21T04:08:38.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.905+0000 7f54b9eff740 1 client.16524 shutdown 2024-09-21T04:08:38.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.905+0000 7f54b9eff740 1 -- 172.21.15.64:0/2975908207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f03572390 msgr2=0x560f03949dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:38.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.905+0000 7f54b9eff740 1 --2- 172.21.15.64:0/2975908207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f03572390 0x560f03949dd0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f549c005b90 tx=0x7f549c004680 comp rx=0 tx=0).stop 2024-09-21T04:08:38.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.905+0000 7f54b9eff740 1 -- 172.21.15.64:0/2975908207 shutdown_connections 2024-09-21T04:08:38.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.905+0000 7f54b9eff740 1 --2- 172.21.15.64:0/2975908207 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x560f03526da0 0x560f039c4f50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.905+0000 7f54b9eff740 1 --2- 172.21.15.64:0/2975908207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f03593070 0x560f0394a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.905+0000 7f54b9eff740 1 --2- 172.21.15.64:0/2975908207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560f03572390 0x560f03949dd0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.905+0000 7f5494ff9640 0 client.16524 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:08:38.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.905+0000 7f54b9eff740 1 --2- 172.21.15.64:0/2975908207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f0356a1b0 0x560f0394e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.905+0000 7f54b9eff740 1 -- 172.21.15.64:0/2975908207 >> 172.21.15.64:0/2975908207 conn(0x560f03587460 msgr2=0x560f03945aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:38.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.905+0000 7f54b9eff740 1 -- 172.21.15.64:0/2975908207 shutdown_connections 2024-09-21T04:08:38.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.905+0000 7f54b9eff740 1 -- 172.21.15.64:0/2975908207 wait complete. 2024-09-21T04:08:38.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.905+0000 7f54b9eff740 20 client.16524 trim_cache size 0 max 16384 2024-09-21T04:08:38.910 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:08:38.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.906+0000 7f54b398b640 1 -- 172.21.15.64:0/3898650843 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f54980b6680 msgr2=0x7f54980b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:38.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.906+0000 7f54b398b640 1 --2- 172.21.15.64:0/3898650843 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f54980b6680 0x7f54980b8b90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f54a4005fd0 tx=0x7f54a4009600 comp rx=0 tx=0).stop 2024-09-21T04:08:38.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.906+0000 7f54b398b640 1 -- 172.21.15.64:0/3898650843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54ac103b90 msgr2=0x7f54ac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:38.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.906+0000 7f54b398b640 1 --2- 172.21.15.64:0/3898650843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54ac103b90 0x7f54ac075710 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f54a8005940 tx=0x7f54a8004270 comp rx=0 tx=0).stop 2024-09-21T04:08:38.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.906+0000 7f54b398b640 1 -- 172.21.15.64:0/3898650843 shutdown_connections 2024-09-21T04:08:38.911 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.906+0000 7f54b398b640 1 --2- 172.21.15.64:0/3898650843 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f54980b6680 0x7f54980b8b90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.906+0000 7f54b398b640 1 --2- 172.21.15.64:0/3898650843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54ac109e20 0x7f54ac076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.906+0000 7f54b398b640 1 --2- 172.21.15.64:0/3898650843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54ac1094d0 0x7f54ac075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.906+0000 7f54b398b640 1 --2- 172.21.15.64:0/3898650843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54ac103b90 0x7f54ac075710 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:38.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.906+0000 7f54b398b640 1 -- 172.21.15.64:0/3898650843 >> 172.21.15.64:0/3898650843 conn(0x7f54ac0ff600 msgr2=0x7f54ac101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:38.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.906+0000 7f54b398b640 1 -- 172.21.15.64:0/3898650843 shutdown_connections 2024-09-21T04:08:38.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:38.906+0000 7f54b398b640 1 -- 172.21.15.64:0/3898650843 wait complete. 2024-09-21T04:08:39.840 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T04:08:39.960 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:39 smithi064 ceph-mon[31180]: pgmap v460: 97 pgs: 97 active+clean; 596 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 1.6 MiB/s wr, 233 op/s 2024-09-21T04:08:40.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:39 smithi119 ceph-mon[34811]: pgmap v460: 97 pgs: 97 active+clean; 596 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 1.6 MiB/s wr, 233 op/s 2024-09-21T04:08:40.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:39 smithi131 ceph-mon[35193]: pgmap v460: 97 pgs: 97 active+clean; 596 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 1.6 MiB/s wr, 233 op/s 2024-09-21T04:08:40.516 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:08:41.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.124+0000 7f27d4616640 1 -- 172.21.15.64:0/396628333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27cc112dc0 msgr2=0x7f27cc1151e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:41.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.124+0000 7f27d4616640 1 --2- 172.21.15.64:0/396628333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27cc112dc0 0x7f27cc1151e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009860 tx=0x7f27c802a340 comp rx=0 tx=0).stop 2024-09-21T04:08:41.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.126+0000 7f27d4616640 1 -- 172.21.15.64:0/396628333 shutdown_connections 2024-09-21T04:08:41.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.126+0000 7f27d4616640 1 --2- 172.21.15.64:0/396628333 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27cc10ac50 0x7f27cc10b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.126+0000 7f27d4616640 1 --2- 172.21.15.64:0/396628333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27cc112dc0 0x7f27cc1151e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.126+0000 7f27d4616640 1 --2- 172.21.15.64:0/396628333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27cc10bff0 0x7f27cc10c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.126+0000 7f27d4616640 1 -- 172.21.15.64:0/396628333 >> 172.21.15.64:0/396628333 conn(0x7f27cc070ec0 msgr2=0x7f27cc0712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:41.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.130+0000 7f27d4616640 1 -- 172.21.15.64:0/396628333 shutdown_connections 2024-09-21T04:08:41.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.130+0000 7f27d4616640 1 -- 172.21.15.64:0/396628333 wait complete. 2024-09-21T04:08:41.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.132+0000 7f27d4616640 1 Processor -- start 2024-09-21T04:08:41.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.132+0000 7f27d4616640 1 -- start start 2024-09-21T04:08:41.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.132+0000 7f27d4616640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27cc10ac50 0x7f27cc1a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:41.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.133+0000 7f27d4616640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27cc10bff0 0x7f27cc1a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:41.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.133+0000 7f27d23b2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27cc10ac50 0x7f27cc1a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:41.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.133+0000 7f27d23b2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27cc10ac50 0x7f27cc1a4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:35634/0 (socket says 172.21.15.64:35634) 2024-09-21T04:08:41.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.133+0000 7f27d1bb1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27cc10bff0 0x7f27cc1a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:41.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.133+0000 7f27d1bb1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27cc10bff0 0x7f27cc1a54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:57418/0 (socket says 172.21.15.64:57418) 2024-09-21T04:08:41.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.133+0000 7f27d1bb1640 1 -- 172.21.15.64:0/2547494225 learned_addr learned my addr 172.21.15.64:0/2547494225 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:41.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.133+0000 7f27d4616640 1 --2- 172.21.15.64:0/2547494225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27cc112dc0 0x7f27cc1aa780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:41.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.135+0000 7f27d2bb3640 1 --2- 172.21.15.64:0/2547494225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27cc112dc0 0x7f27cc1aa780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:41.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.135+0000 7f27d4616640 1 -- 172.21.15.64:0/2547494225 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc11e510 con 0x7f27cc112dc0 2024-09-21T04:08:41.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.135+0000 7f27d4616640 1 -- 172.21.15.64:0/2547494225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc11e390 con 0x7f27cc10ac50 2024-09-21T04:08:41.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.135+0000 7f27d4616640 1 -- 172.21.15.64:0/2547494225 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc11e690 con 0x7f27cc10bff0 2024-09-21T04:08:41.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.139+0000 7f27d23b2640 1 -- 172.21.15.64:0/2547494225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27cc10bff0 msgr2=0x7f27cc1a54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:41.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.140+0000 7f27d23b2640 1 --2- 172.21.15.64:0/2547494225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27cc10bff0 0x7f27cc1a54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.140+0000 7f27d23b2640 1 -- 172.21.15.64:0/2547494225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27cc112dc0 msgr2=0x7f27cc1aa780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:41.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.140+0000 7f27d23b2640 1 --2- 172.21.15.64:0/2547494225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27cc112dc0 0x7f27cc1aa780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.140+0000 7f27d23b2640 1 -- 172.21.15.64:0/2547494225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27cc1aaeb0 con 0x7f27cc10ac50 2024-09-21T04:08:41.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.140+0000 7f27d1bb1640 1 --2- 172.21.15.64:0/2547494225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27cc10bff0 0x7f27cc1a54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:41.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.144+0000 7f27d23b2640 1 --2- 172.21.15.64:0/2547494225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27cc10ac50 0x7f27cc1a4f40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f27c400eb00 tx=0x7f27c400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:41.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.145+0000 7f27c37fe640 1 -- 172.21.15.64:0/2547494225 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c4010430 con 0x7f27cc10ac50 2024-09-21T04:08:41.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.145+0000 7f27c37fe640 1 -- 172.21.15.64:0/2547494225 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27c4010a80 con 0x7f27cc10ac50 2024-09-21T04:08:41.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.146+0000 7f27c37fe640 1 -- 172.21.15.64:0/2547494225 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c4017a90 con 0x7f27cc10ac50 2024-09-21T04:08:41.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.147+0000 7f27d4616640 1 -- 172.21.15.64:0/2547494225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27cc1ab1a0 con 0x7f27cc10ac50 2024-09-21T04:08:41.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.147+0000 7f27d4616640 1 -- 172.21.15.64:0/2547494225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27cc1b1900 con 0x7f27cc10ac50 2024-09-21T04:08:41.159 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.155+0000 7f27dab8a740 1 Processor -- start 2024-09-21T04:08:41.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.159+0000 7f27dab8a740 1 -- start start 2024-09-21T04:08:41.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.159+0000 7f27dab8a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c0a592070 0x560c0a9396a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:41.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.160+0000 7f27dab8a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c0a5691b0 0x560c0a939be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:41.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.160+0000 7f27dab8a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560c0a571390 0x560c0a93a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:41.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.160+0000 7f27dab8a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560c0a8afea0 con 0x560c0a5691b0 2024-09-21T04:08:41.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.160+0000 7f27dab8a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560c0a89f2d0 con 0x560c0a571390 2024-09-21T04:08:41.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.160+0000 7f27dab8a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560c0a8a30e0 con 0x560c0a592070 2024-09-21T04:08:41.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.160+0000 7f27d1bb1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560c0a571390 0x560c0a93a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:41.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.161+0000 7f27d1bb1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560c0a571390 0x560c0a93a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:35640/0 (socket says 172.21.15.64:35640) 2024-09-21T04:08:41.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.161+0000 7f27d1bb1640 1 -- 172.21.15.64:0/4035180970 learned_addr learned my addr 172.21.15.64:0/4035180970 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:41.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.161+0000 7f27d1bb1640 1 -- 172.21.15.64:0/4035180970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c0a592070 msgr2=0x560c0a9396a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T04:08:41.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.161+0000 7f27d1bb1640 1 --2- 172.21.15.64:0/4035180970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c0a592070 0x560c0a9396a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.161+0000 7f27d1bb1640 1 -- 172.21.15.64:0/4035180970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c0a5691b0 msgr2=0x560c0a939be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:08:41.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.161+0000 7f27d1bb1640 1 --2- 172.21.15.64:0/4035180970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c0a5691b0 0x560c0a939be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.161+0000 7f27d1bb1640 1 -- 172.21.15.64:0/4035180970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c0a5fd650 con 0x560c0a571390 2024-09-21T04:08:41.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.162+0000 7f27d1bb1640 1 --2- 172.21.15.64:0/4035180970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560c0a571390 0x560c0a93a120 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f27c8035040 tx=0x7f27c80095a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87e290f237ac12e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:41.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.164+0000 7f27c0ff9640 1 -- 172.21.15.64:0/4035180970 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8037040 con 0x560c0a571390 2024-09-21T04:08:41.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.166+0000 7f27c0ff9640 1 -- 172.21.15.64:0/4035180970 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27c8042070 con 0x560c0a571390 2024-09-21T04:08:41.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.167+0000 7f27dab8a740 1 -- 172.21.15.64:0/4035180970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560c0a571390 msgr2=0x560c0a93a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:41.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.167+0000 7f27dab8a740 1 --2- 172.21.15.64:0/4035180970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560c0a571390 0x560c0a93a120 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f27c8035040 tx=0x7f27c80095a0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.174 INFO:tasks.workunit.client.0.smithi064.stdout: 95 374924 259 264355 183 213817 713 2024-09-21T04:08:41.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.172+0000 7f27dab8a740 1 -- 172.21.15.64:0/4035180970 shutdown_connections 2024-09-21T04:08:41.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.172+0000 7f27dab8a740 1 --2- 172.21.15.64:0/4035180970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560c0a571390 0x560c0a93a120 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.172+0000 7f27dab8a740 1 --2- 172.21.15.64:0/4035180970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c0a5691b0 0x560c0a939be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.172+0000 7f27dab8a740 1 --2- 172.21.15.64:0/4035180970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c0a592070 0x560c0a9396a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.172+0000 7f27dab8a740 1 -- 172.21.15.64:0/4035180970 >> 172.21.15.64:0/4035180970 conn(0x560c0a586460 msgr2=0x560c0a937590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:41.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.172+0000 7f27c37fe640 1 -- 172.21.15.64:0/2547494225 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f27c4010c30 con 0x7f27cc10ac50 2024-09-21T04:08:41.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.172+0000 7f27dab8a740 1 -- 172.21.15.64:0/4035180970 shutdown_connections 2024-09-21T04:08:41.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.172+0000 7f27dab8a740 1 -- 172.21.15.64:0/4035180970 wait complete. 2024-09-21T04:08:41.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.173+0000 7f27dab8a740 1 -- start start 2024-09-21T04:08:41.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.173+0000 7f27dab8a740 1 Processor -- start 2024-09-21T04:08:41.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.174+0000 7f27c37fe640 1 --2- 172.21.15.64:0/2547494225 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f27b80b64b0 0x7f27b80b89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:41.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.175+0000 7f27c37fe640 1 -- 172.21.15.64:0/2547494225 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f27c40e8060 con 0x7f27cc10ac50 2024-09-21T04:08:41.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.176+0000 7f27dab8a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c0a5691b0 0x560c0a944de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:41.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.176+0000 7f27dab8a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560c0a571390 0x560c0a945320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:41.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.176+0000 7f27dab8a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c0a592070 0x560c0a9511e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:41.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.176+0000 7f27dab8a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560c0a700700 con 0x560c0a5691b0 2024-09-21T04:08:41.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.176+0000 7f27dab8a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560c0a83a160 con 0x560c0a571390 2024-09-21T04:08:41.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.176+0000 7f27dab8a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560c0a947e70 con 0x560c0a592070 2024-09-21T04:08:41.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.176+0000 7f27dab8a740 10 client.0 _ll_register_callbacks cb 0x560c0a8539e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:08:41.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.177+0000 7f27d1bb1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560c0a571390 0x560c0a945320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:41.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.177+0000 7f27d1bb1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560c0a571390 0x560c0a945320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:35654/0 (socket says 172.21.15.64:35654) 2024-09-21T04:08:41.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.177+0000 7f27d1bb1640 1 -- 172.21.15.64:0/4143989357 learned_addr learned my addr 172.21.15.64:0/4143989357 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:41.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.177+0000 7f27d2bb3640 1 --2- 172.21.15.64:0/2547494225 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f27b80b64b0 0x7f27b80b89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:41.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.177+0000 7f27d2bb3640 1 --2- 172.21.15.64:0/4143989357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c0a592070 0x560c0a9511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:41.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.178+0000 7f27d23b2640 1 --2- 172.21.15.64:0/4143989357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c0a5691b0 0x560c0a944de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:41.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.178+0000 7f27d1bb1640 1 -- 172.21.15.64:0/4143989357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c0a592070 msgr2=0x560c0a9511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:41.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.178+0000 7f27d1bb1640 1 --2- 172.21.15.64:0/4143989357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c0a592070 0x560c0a9511e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.178+0000 7f27d1bb1640 1 -- 172.21.15.64:0/4143989357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c0a5691b0 msgr2=0x560c0a944de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:41.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.178+0000 7f27d1bb1640 1 --2- 172.21.15.64:0/4143989357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c0a5691b0 0x560c0a944de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.178+0000 7f27d1bb1640 1 -- 172.21.15.64:0/4143989357 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c0a867c90 con 0x560c0a571390 2024-09-21T04:08:41.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.181+0000 7f27d1bb1640 1 --2- 172.21.15.64:0/4143989357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560c0a571390 0x560c0a945320 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009830 tx=0x7f27c8009450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cabf546703827b3b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:41.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.182+0000 7f27b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:08:41.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.182+0000 7f27b5ffb640 1 -- 172.21.15.64:0/4143989357 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8037050 con 0x560c0a571390 2024-09-21T04:08:41.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.182+0000 7f27d2bb3640 1 --2- 172.21.15.64:0/2547494225 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f27b80b64b0 0x7f27b80b89c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f27bc005ab0 tx=0x7f27bc009660 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:41.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.183+0000 7f27dab8a740 1 -- 172.21.15.64:0/4143989357 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27c8042070 con 0x560c0a571390 2024-09-21T04:08:41.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.183+0000 7f27b5ffb640 1 -- 172.21.15.64:0/4143989357 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f27c8004230 con 0x560c0a571390 2024-09-21T04:08:41.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.186+0000 7f27b5ffb640 1 -- 172.21.15.64:0/4143989357 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f27c8004430 con 0x560c0a571390 2024-09-21T04:08:41.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.190+0000 7f27dab8a740 10 client.26240 fetch_fsmap learned FSMap version 33 2024-09-21T04:08:41.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.190+0000 7f27dab8a740 1 -- 172.21.15.64:0/4143989357 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560c0a953ba0 con 0x560c0a571390 2024-09-21T04:08:41.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.196+0000 7f27b5ffb640 1 -- 172.21.15.64:0/4143989357 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27c8042070 con 0x560c0a571390 2024-09-21T04:08:41.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.196+0000 7f27b5ffb640 1 -- 172.21.15.64:0/4143989357 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f27c8004a70 con 0x560c0a571390 2024-09-21T04:08:41.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.196+0000 7f27dab8a740 10 client.26240 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T04:08:41.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.196+0000 7f27dab8a740 10 client.26240 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:08:41.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.196+0000 7f27dab8a740 20 client.26240 populate_metadata read hostname 'smithi064' 2024-09-21T04:08:41.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.196+0000 7f27dab8a740 1 --2- 172.21.15.64:0/4143989357 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x560c0a5509f0 0x560c0a9570e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:41.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.196+0000 7f27dab8a740 4 client.26240 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:08:41.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.196+0000 7f27dab8a740 1 -- 172.21.15.64:0/4143989357 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c0a8b1df0 con 0x560c0a5509f0 2024-09-21T04:08:41.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.197+0000 7f27d23b2640 1 --2- 172.21.15.64:0/4143989357 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x560c0a5509f0 0x560c0a9570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:41.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.202+0000 7f27d23b2640 1 --2- 172.21.15.64:0/4143989357 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x560c0a5509f0 0x560c0a9570e0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deb803200acdfdd7 server_cookie=6bc165291cacefa0 in_seq=0 out_seq=0 2024-09-21T04:08:41.207 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.203+0000 7f27b5ffb640 10 client.26240 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:41.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.208+0000 7f27b5ffb640 1 -- 172.21.15.64:0/4143989357 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560c0a8b1df0 con 0x560c0a5509f0 2024-09-21T04:08:41.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.208+0000 7f27b5ffb640 10 client.26240 handle_command_reply: tid=0 2024-09-21T04:08:41.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.208+0000 7f27dab8a740 1 client.26240 shutdown 2024-09-21T04:08:41.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.211+0000 7f27dab8a740 1 -- 172.21.15.64:0/4143989357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560c0a571390 msgr2=0x560c0a945320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:41.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.211+0000 7f27dab8a740 1 --2- 172.21.15.64:0/4143989357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560c0a571390 0x560c0a945320 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009830 tx=0x7f27c8009450 comp rx=0 tx=0).stop 2024-09-21T04:08:41.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.214+0000 7f27dab8a740 1 -- 172.21.15.64:0/4143989357 shutdown_connections 2024-09-21T04:08:41.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.214+0000 7f27dab8a740 1 --2- 172.21.15.64:0/4143989357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c0a592070 0x560c0a9511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.214+0000 7f27dab8a740 1 --2- 172.21.15.64:0/4143989357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560c0a571390 0x560c0a945320 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.214+0000 7f27dab8a740 1 --2- 172.21.15.64:0/4143989357 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x560c0a5509f0 0x560c0a9570e0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.214+0000 7f27dab8a740 1 --2- 172.21.15.64:0/4143989357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c0a5691b0 0x560c0a944de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.214+0000 7f27dab8a740 1 -- 172.21.15.64:0/4143989357 >> 172.21.15.64:0/4143989357 conn(0x560c0a586460 msgr2=0x560c0a93c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:41.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.214+0000 7f27b5ffb640 0 client.26240 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:41.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.217+0000 7f27dab8a740 1 -- 172.21.15.64:0/4143989357 shutdown_connections 2024-09-21T04:08:41.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.218+0000 7f27dab8a740 1 -- 172.21.15.64:0/4143989357 wait complete. 2024-09-21T04:08:41.222 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.218+0000 7f27dab8a740 20 client.26240 trim_cache size 0 max 16384 2024-09-21T04:08:41.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.253+0000 7f27dab8a740 1 Processor -- start 2024-09-21T04:08:41.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.253+0000 7f27dab8a740 1 -- start start 2024-09-21T04:08:41.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.254+0000 7f27dab8a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c0a5691b0 0x560c0a946a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:41.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.254+0000 7f27dab8a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560c0a571390 0x560c0a936bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:41.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.254+0000 7f27dab8a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c0a592070 0x560c0a937100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:41.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.254+0000 7f27dab8a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc1bc970 con 0x560c0a5691b0 2024-09-21T04:08:41.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.254+0000 7f27dab8a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc1ba1e0 con 0x560c0a571390 2024-09-21T04:08:41.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.254+0000 7f27dab8a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc1b8030 con 0x560c0a592070 2024-09-21T04:08:41.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.255+0000 7f27d2bb3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c0a592070 0x560c0a937100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:41.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.255+0000 7f27d2bb3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c0a592070 0x560c0a937100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:57440/0 (socket says 172.21.15.64:57440) 2024-09-21T04:08:41.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.255+0000 7f27d2bb3640 1 -- 172.21.15.64:0/4203935 learned_addr learned my addr 172.21.15.64:0/4203935 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:41.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.258+0000 7f27d1bb1640 1 --2- 172.21.15.64:0/4203935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560c0a571390 0x560c0a936bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:41.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.258+0000 7f27d23b2640 1 --2- 172.21.15.64:0/4203935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c0a5691b0 0x560c0a946a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:41.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.258+0000 7f27d1bb1640 1 -- 172.21.15.64:0/4203935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c0a592070 msgr2=0x560c0a937100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:41.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.258+0000 7f27d1bb1640 1 --2- 172.21.15.64:0/4203935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c0a592070 0x560c0a937100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.258+0000 7f27d1bb1640 1 -- 172.21.15.64:0/4203935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c0a5691b0 msgr2=0x560c0a946a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:41.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.258+0000 7f27d1bb1640 1 --2- 172.21.15.64:0/4203935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c0a5691b0 0x560c0a946a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.259+0000 7f27d1bb1640 1 -- 172.21.15.64:0/4203935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c0a93c500 con 0x560c0a571390 2024-09-21T04:08:41.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.259+0000 7f27d1bb1640 1 --2- 172.21.15.64:0/4203935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560c0a571390 0x560c0a936bc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f27c80308e0 tx=0x7f27c8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa0ce5fae16dbe2e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:41.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.263+0000 7f27b7fff640 1 -- 172.21.15.64:0/4203935 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c805a070 con 0x560c0a571390 2024-09-21T04:08:41.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.263+0000 7f27b7fff640 1 -- 172.21.15.64:0/4203935 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27c8042040 con 0x560c0a571390 2024-09-21T04:08:41.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.264+0000 7f27dab8a740 1 -- 172.21.15.64:0/4203935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560c0a571390 msgr2=0x560c0a936bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:41.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.264+0000 7f27dab8a740 1 --2- 172.21.15.64:0/4203935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560c0a571390 0x560c0a936bc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f27c80308e0 tx=0x7f27c8000bc0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.269+0000 7f27dab8a740 1 -- 172.21.15.64:0/4203935 shutdown_connections 2024-09-21T04:08:41.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.269+0000 7f27dab8a740 1 --2- 172.21.15.64:0/4203935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c0a592070 0x560c0a937100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.269+0000 7f27dab8a740 1 --2- 172.21.15.64:0/4203935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560c0a571390 0x560c0a936bc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.269+0000 7f27dab8a740 1 --2- 172.21.15.64:0/4203935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c0a5691b0 0x560c0a946a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.269+0000 7f27dab8a740 1 -- 172.21.15.64:0/4203935 >> 172.21.15.64:0/4203935 conn(0x560c0a586460 msgr2=0x560c0a946300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:41.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.273+0000 7f27dab8a740 1 -- 172.21.15.64:0/4203935 shutdown_connections 2024-09-21T04:08:41.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.273+0000 7f27dab8a740 1 -- 172.21.15.64:0/4203935 wait complete. 2024-09-21T04:08:41.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.274+0000 7f27dab8a740 1 -- start start 2024-09-21T04:08:41.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.274+0000 7f27dab8a740 1 Processor -- start 2024-09-21T04:08:41.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.275+0000 7f27dab8a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c0a5691b0 0x560c0a94d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:41.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.275+0000 7f27dab8a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c0a571390 0x560c0a941d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:41.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.275+0000 7f27dab8a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560c0a592070 0x560c0a942390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:41.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.275+0000 7f27dab8a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc1b86a0 con 0x560c0a571390 2024-09-21T04:08:41.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.275+0000 7f27dab8a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc1b9b10 con 0x560c0a592070 2024-09-21T04:08:41.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.275+0000 7f27dab8a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc1b8d70 con 0x560c0a5691b0 2024-09-21T04:08:41.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.275+0000 7f27dab8a740 10 client.0 _ll_register_callbacks cb 0x7f27c40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:08:41.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.275+0000 7f27d1bb1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c0a571390 0x560c0a941d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:41.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.276+0000 7f27d1bb1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c0a571390 0x560c0a941d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42160/0 (socket says 172.21.15.64:42160) 2024-09-21T04:08:41.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.276+0000 7f27d1bb1640 1 -- 172.21.15.64:0/2371055502 learned_addr learned my addr 172.21.15.64:0/2371055502 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:41.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.276+0000 7f27d23b2640 1 --2- 172.21.15.64:0/2371055502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c0a5691b0 0x560c0a94d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:41.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.277+0000 7f27d2bb3640 1 --2- 172.21.15.64:0/2371055502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560c0a592070 0x560c0a942390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:41.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.277+0000 7f27d1bb1640 1 -- 172.21.15.64:0/2371055502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c0a5691b0 msgr2=0x560c0a94d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:41.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.277+0000 7f27d1bb1640 1 --2- 172.21.15.64:0/2371055502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c0a5691b0 0x560c0a94d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.277+0000 7f27d1bb1640 1 -- 172.21.15.64:0/2371055502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560c0a592070 msgr2=0x560c0a942390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:41.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.277+0000 7f27d1bb1640 1 --2- 172.21.15.64:0/2371055502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560c0a592070 0x560c0a942390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.277+0000 7f27d1bb1640 1 -- 172.21.15.64:0/2371055502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c0a942b70 con 0x560c0a571390 2024-09-21T04:08:41.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.279+0000 7f27d1bb1640 1 --2- 172.21.15.64:0/2371055502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c0a571390 0x560c0a941d90 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f27c8005eb0 tx=0x7f27c802a840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e014d21a6d8bc180 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:41.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.279+0000 7f27b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:08:41.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.279+0000 7f27b5ffb640 1 -- 172.21.15.64:0/2371055502 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8062060 con 0x560c0a571390 2024-09-21T04:08:41.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.279+0000 7f27dab8a740 1 -- 172.21.15.64:0/2371055502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27c8042040 con 0x560c0a571390 2024-09-21T04:08:41.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.280+0000 7f27b5ffb640 1 -- 172.21.15.64:0/2371055502 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f27c803d680 con 0x560c0a571390 2024-09-21T04:08:41.284 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.281+0000 7f27b5ffb640 1 -- 172.21.15.64:0/2371055502 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f27c805a070 con 0x560c0a571390 2024-09-21T04:08:41.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.285+0000 7f27dab8a740 10 client.16539 fetch_fsmap learned FSMap version 33 2024-09-21T04:08:41.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.285+0000 7f27dab8a740 1 -- 172.21.15.64:0/2371055502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560c0a9c1a70 con 0x560c0a571390 2024-09-21T04:08:41.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.285+0000 7f27b5ffb640 1 -- 172.21.15.64:0/2371055502 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27c8042040 con 0x560c0a571390 2024-09-21T04:08:41.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.286+0000 7f27b5ffb640 1 -- 172.21.15.64:0/2371055502 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f27c8056590 con 0x560c0a571390 2024-09-21T04:08:41.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.286+0000 7f27dab8a740 10 client.16539 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T04:08:41.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.286+0000 7f27dab8a740 10 client.16539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:08:41.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.286+0000 7f27dab8a740 20 client.16539 populate_metadata read hostname 'smithi064' 2024-09-21T04:08:41.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.286+0000 7f27dab8a740 1 --2- 172.21.15.64:0/2371055502 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x560c0a525da0 0x560c0a9c4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:41.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.286+0000 7f27dab8a740 4 client.16539 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:08:41.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.286+0000 7f27dab8a740 1 -- 172.21.15.64:0/2371055502 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560c0a8ad8a0 con 0x560c0a525da0 2024-09-21T04:08:41.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.286+0000 7f27d23b2640 1 --2- 172.21.15.64:0/2371055502 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x560c0a525da0 0x560c0a9c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:41.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.288+0000 7f27d23b2640 1 --2- 172.21.15.64:0/2371055502 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x560c0a525da0 0x560c0a9c4210 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0159243b01837f1 server_cookie=ba6c27bc3b9bbe67 in_seq=0 out_seq=0 2024-09-21T04:08:41.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.291+0000 7f27b5ffb640 10 client.16539 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:41.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.291+0000 7f27b5ffb640 1 -- 172.21.15.64:0/2371055502 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560c0a8ad8a0 con 0x560c0a525da0 2024-09-21T04:08:41.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.291+0000 7f27b5ffb640 10 client.16539 handle_command_reply: tid=0 2024-09-21T04:08:41.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.291+0000 7f27dab8a740 1 client.16539 shutdown 2024-09-21T04:08:41.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.293+0000 7f27dab8a740 1 -- 172.21.15.64:0/2371055502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c0a571390 msgr2=0x560c0a941d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:41.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.293+0000 7f27dab8a740 1 --2- 172.21.15.64:0/2371055502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c0a571390 0x560c0a941d90 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f27c8005eb0 tx=0x7f27c802a840 comp rx=0 tx=0).stop 2024-09-21T04:08:41.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.293+0000 7f27dab8a740 1 -- 172.21.15.64:0/2371055502 shutdown_connections 2024-09-21T04:08:41.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.293+0000 7f27dab8a740 1 --2- 172.21.15.64:0/2371055502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560c0a592070 0x560c0a942390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.293+0000 7f27dab8a740 1 --2- 172.21.15.64:0/2371055502 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x560c0a525da0 0x560c0a9c4210 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.293+0000 7f27dab8a740 1 --2- 172.21.15.64:0/2371055502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560c0a571390 0x560c0a941d90 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.293+0000 7f27dab8a740 1 --2- 172.21.15.64:0/2371055502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560c0a5691b0 0x560c0a94d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.293+0000 7f27dab8a740 1 -- 172.21.15.64:0/2371055502 >> 172.21.15.64:0/2371055502 conn(0x560c0a586460 msgr2=0x560c0a946160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:41.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.295+0000 7f27b5ffb640 0 client.16539 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:41.302 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.300+0000 7f27dab8a740 1 -- 172.21.15.64:0/2371055502 shutdown_connections 2024-09-21T04:08:41.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.306+0000 7f27dab8a740 1 -- 172.21.15.64:0/2371055502 wait complete. 2024-09-21T04:08:41.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.309+0000 7f27dab8a740 20 client.16539 trim_cache size 0 max 16384 2024-09-21T04:08:41.316 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:08:41.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.314+0000 7f27d4616640 1 -- 172.21.15.64:0/2547494225 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f27b80b64b0 msgr2=0x7f27b80b89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:41.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.314+0000 7f27d4616640 1 --2- 172.21.15.64:0/2547494225 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f27b80b64b0 0x7f27b80b89c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f27bc005ab0 tx=0x7f27bc009660 comp rx=0 tx=0).stop 2024-09-21T04:08:41.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.314+0000 7f27d4616640 1 -- 172.21.15.64:0/2547494225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27cc10ac50 msgr2=0x7f27cc1a4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:41.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.314+0000 7f27d4616640 1 --2- 172.21.15.64:0/2547494225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27cc10ac50 0x7f27cc1a4f40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f27c400eb00 tx=0x7f27c400c2d0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.314+0000 7f27d4616640 1 -- 172.21.15.64:0/2547494225 shutdown_connections 2024-09-21T04:08:41.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.314+0000 7f27d4616640 1 --2- 172.21.15.64:0/2547494225 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f27b80b64b0 0x7f27b80b89c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.314+0000 7f27d4616640 1 --2- 172.21.15.64:0/2547494225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27cc112dc0 0x7f27cc1aa780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.314+0000 7f27d4616640 1 --2- 172.21.15.64:0/2547494225 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27cc10bff0 0x7f27cc1a54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.314+0000 7f27d4616640 1 --2- 172.21.15.64:0/2547494225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27cc10ac50 0x7f27cc1a4f40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:41.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.314+0000 7f27d4616640 1 -- 172.21.15.64:0/2547494225 >> 172.21.15.64:0/2547494225 conn(0x7f27cc070ec0 msgr2=0x7f27cc114250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:41.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.321+0000 7f27d4616640 1 -- 172.21.15.64:0/2547494225 shutdown_connections 2024-09-21T04:08:41.325 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:41.321+0000 7f27d4616640 1 -- 172.21.15.64:0/2547494225 wait complete. 2024-09-21T04:08:42.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:41 smithi064 ceph-mon[31180]: pgmap v461: 97 pgs: 97 active+clean; 596 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 1.5 MiB/s wr, 212 op/s 2024-09-21T04:08:42.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:41 smithi119 ceph-mon[34811]: pgmap v461: 97 pgs: 97 active+clean; 596 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 1.5 MiB/s wr, 212 op/s 2024-09-21T04:08:42.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:41 smithi131 ceph-mon[35193]: pgmap v461: 97 pgs: 97 active+clean; 596 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 1.5 MiB/s wr, 212 op/s 2024-09-21T04:08:43.517 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T04:08:43.517 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T04:08:43.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.576+0000 7f5d4cf8f640 1 -- 172.21.15.64:0/3478797323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d48111ca0 msgr2=0x7f5d4810a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:43.579 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.576+0000 7f5d4cf8f640 1 --2- 172.21.15.64:0/3478797323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d48111ca0 0x7f5d4810a3c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5d40009f00 tx=0x7f5d4002ab60 comp rx=0 tx=0).stop 2024-09-21T04:08:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.577+0000 7f5d4cf8f640 1 -- 172.21.15.64:0/3478797323 shutdown_connections 2024-09-21T04:08:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.577+0000 7f5d4cf8f640 1 --2- 172.21.15.64:0/3478797323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4811cba0 0x7f5d4811efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.577+0000 7f5d4cf8f640 1 --2- 172.21.15.64:0/3478797323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d4810a900 0x7f5d4810ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.577+0000 7f5d4cf8f640 1 --2- 172.21.15.64:0/3478797323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d48111ca0 0x7f5d4810a3c0 secure :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5d40009f00 tx=0x7f5d4002ab60 comp rx=0 tx=0).stop 2024-09-21T04:08:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.577+0000 7f5d4cf8f640 1 -- 172.21.15.64:0/3478797323 >> 172.21.15.64:0/3478797323 conn(0x7f5d48070c00 msgr2=0x7f5d48071010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:43.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.580+0000 7f5d4cf8f640 1 -- 172.21.15.64:0/3478797323 shutdown_connections 2024-09-21T04:08:43.582 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.580+0000 7f5d4cf8f640 1 -- 172.21.15.64:0/3478797323 wait complete. 2024-09-21T04:08:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.581+0000 7f5d4cf8f640 1 Processor -- start 2024-09-21T04:08:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.581+0000 7f5d4cf8f640 1 -- start start 2024-09-21T04:08:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.582+0000 7f5d4cf8f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d4810a900 0x7f5d481a92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.582+0000 7f5d4cf8f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4811cba0 0x7f5d481a9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.582+0000 7f5d4cf8f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d481aea40 0x7f5d481aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.582+0000 7f5d4cf8f640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5d48122790 con 0x7f5d481aea40 2024-09-21T04:08:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.582+0000 7f5d4cf8f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5d48122610 con 0x7f5d4811cba0 2024-09-21T04:08:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.582+0000 7f5d4cf8f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5d48122910 con 0x7f5d4810a900 2024-09-21T04:08:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.582+0000 7f5d477fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d4810a900 0x7f5d481a92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.582+0000 7f5d477fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d4810a900 0x7f5d481a92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:38580/0 (socket says 172.21.15.64:38580) 2024-09-21T04:08:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.582+0000 7f5d46ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4811cba0 0x7f5d481a9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.582+0000 7f5d477fe640 1 -- 172.21.15.64:0/3881355121 learned_addr learned my addr 172.21.15.64:0/3881355121 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.583+0000 7f5d477fe640 1 -- 172.21.15.64:0/3881355121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4811cba0 msgr2=0x7f5d481a9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.583+0000 7f5d477fe640 1 --2- 172.21.15.64:0/3881355121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4811cba0 0x7f5d481a9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.583+0000 7f5d477fe640 1 -- 172.21.15.64:0/3881355121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d481aea40 msgr2=0x7f5d481aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.583+0000 7f5d477fe640 1 --2- 172.21.15.64:0/3881355121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d481aea40 0x7f5d481aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.583+0000 7f5d477fe640 1 -- 172.21.15.64:0/3881355121 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d481af6b0 con 0x7f5d4810a900 2024-09-21T04:08:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.583+0000 7f5d477fe640 1 --2- 172.21.15.64:0/3881355121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d4810a900 0x7f5d481a92d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5d4002b0d0 tx=0x7f5d40034b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.609+0000 7f5d44ff9640 1 -- 172.21.15.64:0/3881355121 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4002b7c0 con 0x7f5d4810a900 2024-09-21T04:08:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.609+0000 7f5d44ff9640 1 -- 172.21.15.64:0/3881355121 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d4002be10 con 0x7f5d4810a900 2024-09-21T04:08:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.609+0000 7f5d44ff9640 1 -- 172.21.15.64:0/3881355121 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4003e460 con 0x7f5d4810a900 2024-09-21T04:08:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.612+0000 7f5d4cf8f640 1 -- 172.21.15.64:0/3881355121 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d4807b620 con 0x7f5d4810a900 2024-09-21T04:08:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.612+0000 7f5d4cf8f640 1 -- 172.21.15.64:0/3881355121 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d4807bb40 con 0x7f5d4810a900 2024-09-21T04:08:43.618 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.616+0000 7f5d5523e740 1 Processor -- start 2024-09-21T04:08:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.617+0000 7f5d5523e740 1 -- start start 2024-09-21T04:08:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.617+0000 7f5d5523e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b66f6b1b0 0x562b673138c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.617+0000 7f5d5523e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562b66f422f0 0x562b67310db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.617+0000 7f5d5523e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b66f4a4d0 0x562b673114f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.617+0000 7f5d5523e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562b672856f0 con 0x562b66f422f0 2024-09-21T04:08:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.617+0000 7f5d5523e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562b67274b20 con 0x562b66f6b1b0 2024-09-21T04:08:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.617+0000 7f5d5523e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562b67278930 con 0x562b66f4a4d0 2024-09-21T04:08:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.617+0000 7f5d47fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b66f6b1b0 0x562b673138c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.617+0000 7f5d47fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b66f6b1b0 0x562b673138c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:52904/0 (socket says 172.21.15.64:52904) 2024-09-21T04:08:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.617+0000 7f5d47fff640 1 -- 172.21.15.64:0/3956357557 learned_addr learned my addr 172.21.15.64:0/3956357557 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.617+0000 7f5d46ffd640 1 --2- 172.21.15.64:0/3956357557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b66f4a4d0 0x562b673114f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.617+0000 7f5d477fe640 1 --2- 172.21.15.64:0/3956357557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562b66f422f0 0x562b67310db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.618+0000 7f5d46ffd640 1 -- 172.21.15.64:0/3956357557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b66f6b1b0 msgr2=0x562b673138c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.618+0000 7f5d46ffd640 1 --2- 172.21.15.64:0/3956357557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b66f6b1b0 0x562b673138c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.618+0000 7f5d46ffd640 1 -- 172.21.15.64:0/3956357557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562b66f422f0 msgr2=0x562b67310db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.618+0000 7f5d46ffd640 1 --2- 172.21.15.64:0/3956357557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562b66f422f0 0x562b67310db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.618+0000 7f5d46ffd640 1 -- 172.21.15.64:0/3956357557 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b6723a530 con 0x562b66f4a4d0 2024-09-21T04:08:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.618+0000 7f5d477fe640 1 --2- 172.21.15.64:0/3956357557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562b66f422f0 0x562b67310db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:08:43.621 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.618+0000 7f5d46ffd640 1 --2- 172.21.15.64:0/3956357557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b66f4a4d0 0x562b673114f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5d4810f810 tx=0x7f5d3c00bbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a0cffecceb9b4e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:43.623 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.621+0000 7f5d44ff9640 1 -- 172.21.15.64:0/3881355121 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f5d40045020 con 0x7f5d4810a900 2024-09-21T04:08:43.625 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.623+0000 7f5d44ff9640 1 --2- 172.21.15.64:0/3881355121 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5d300b6800 0x7f5d300b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:43.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.623+0000 7f5d47fff640 1 --2- 172.21.15.64:0/3881355121 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5d300b6800 0x7f5d300b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:43.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.623+0000 7f5d44ff9640 1 -- 172.21.15.64:0/3881355121 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f5d40109780 con 0x7f5d4810a900 2024-09-21T04:08:43.626 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.624+0000 7f5d47fff640 1 --2- 172.21.15.64:0/3881355121 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5d300b6800 0x7f5d300b8d10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5d38006fd0 tx=0x7f5d380090e0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.632+0000 7f5d4cf8f640 1 -- 172.21.15.64:0/3956357557 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c00be30 con 0x562b66f4a4d0 2024-09-21T04:08:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.632+0000 7f5d4cf8f640 1 -- 172.21.15.64:0/3956357557 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d3c019070 con 0x562b66f4a4d0 2024-09-21T04:08:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.633+0000 7f5d5523e740 1 -- 172.21.15.64:0/3956357557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b66f4a4d0 msgr2=0x562b673114f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:43.636 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.634+0000 7f5d5523e740 1 --2- 172.21.15.64:0/3956357557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b66f4a4d0 0x562b673114f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5d4810f810 tx=0x7f5d3c00bbe0 comp rx=0 tx=0).stop 2024-09-21T04:08:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.639+0000 7f5d5523e740 1 -- 172.21.15.64:0/3956357557 shutdown_connections 2024-09-21T04:08:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.639+0000 7f5d5523e740 1 --2- 172.21.15.64:0/3956357557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b66f4a4d0 0x562b673114f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.639+0000 7f5d5523e740 1 --2- 172.21.15.64:0/3956357557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562b66f422f0 0x562b67310db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.639+0000 7f5d5523e740 1 --2- 172.21.15.64:0/3956357557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b66f6b1b0 0x562b673138c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.639+0000 7f5d5523e740 1 -- 172.21.15.64:0/3956357557 >> 172.21.15.64:0/3956357557 conn(0x562b66f5f300 msgr2=0x562b6730eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.643+0000 7f5d5523e740 1 -- 172.21.15.64:0/3956357557 shutdown_connections 2024-09-21T04:08:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.643+0000 7f5d5523e740 1 -- 172.21.15.64:0/3956357557 wait complete. 2024-09-21T04:08:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.643+0000 7f5d5523e740 1 -- start start 2024-09-21T04:08:43.646 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.643+0000 7f5d5523e740 1 Processor -- start 2024-09-21T04:08:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.644+0000 7f5d5523e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562b66f422f0 0x562b6731c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.644+0000 7f5d5523e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b66f4a4d0 0x562b6731c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.645+0000 7f5d5523e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b66f6b1b0 0x562b6731cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.645+0000 7f5d5523e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562b671acab0 con 0x562b66f422f0 2024-09-21T04:08:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.645+0000 7f5d5523e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562b6721a730 con 0x562b66f6b1b0 2024-09-21T04:08:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.645+0000 7f5d46ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b66f4a4d0 0x562b6731c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.645+0000 7f5d47fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b66f6b1b0 0x562b6731cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.645+0000 7f5d477fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562b66f422f0 0x562b6731c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.645+0000 7f5d47fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b66f6b1b0 0x562b6731cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:52906/0 (socket says 172.21.15.64:52906) 2024-09-21T04:08:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.645+0000 7f5d46ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b66f4a4d0 0x562b6731c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:38596/0 (socket says 172.21.15.64:38596) 2024-09-21T04:08:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.645+0000 7f5d47fff640 1 -- 172.21.15.64:0/4289093997 learned_addr learned my addr 172.21.15.64:0/4289093997 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.645+0000 7f5d5523e740 1 -- 172.21.15.64:0/4289093997 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562b6731f3c0 con 0x562b66f4a4d0 2024-09-21T04:08:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.646+0000 7f5d5523e740 10 client.0 _ll_register_callbacks cb 0x562b6726e6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:08:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.646+0000 7f5d47fff640 1 -- 172.21.15.64:0/4289093997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b66f4a4d0 msgr2=0x562b6731c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.646+0000 7f5d47fff640 1 --2- 172.21.15.64:0/4289093997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b66f4a4d0 0x562b6731c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.646+0000 7f5d47fff640 1 -- 172.21.15.64:0/4289093997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562b66f422f0 msgr2=0x562b6731c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.646+0000 7f5d47fff640 1 --2- 172.21.15.64:0/4289093997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562b66f422f0 0x562b6731c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.646+0000 7f5d47fff640 1 -- 172.21.15.64:0/4289093997 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b66fd6790 con 0x562b66f6b1b0 2024-09-21T04:08:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.646+0000 7f5d477fe640 1 --2- 172.21.15.64:0/4289093997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562b66f422f0 0x562b6731c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.646+0000 7f5d47fff640 1 --2- 172.21.15.64:0/4289093997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b66f6b1b0 0x562b6731cd00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5d38015700 tx=0x7f5d380047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51fbf92d235968b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.661+0000 7f5d2b7f6640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:08:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.661+0000 7f5d2b7f6640 1 -- 172.21.15.64:0/4289093997 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38023020 con 0x562b66f6b1b0 2024-09-21T04:08:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.661+0000 7f5d2b7f6640 1 -- 172.21.15.64:0/4289093997 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f5d38013510 con 0x562b66f6b1b0 2024-09-21T04:08:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.661+0000 7f5d5523e740 1 -- 172.21.15.64:0/4289093997 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d3c019070 con 0x562b66f6b1b0 2024-09-21T04:08:43.672 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.670+0000 7f5d2b7f6640 1 -- 172.21.15.64:0/4289093997 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5d3801b040 con 0x562b66f6b1b0 2024-09-21T04:08:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.671+0000 7f5d5523e740 10 client.26252 fetch_fsmap learned FSMap version 33 2024-09-21T04:08:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.671+0000 7f5d5523e740 1 -- 172.21.15.64:0/4289093997 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562b6704f620 con 0x562b66f6b1b0 2024-09-21T04:08:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.671+0000 7f5d2b7f6640 1 -- 172.21.15.64:0/4289093997 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d3c019070 con 0x562b66f6b1b0 2024-09-21T04:08:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.672+0000 7f5d2b7f6640 1 -- 172.21.15.64:0/4289093997 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f5d3801e030 con 0x562b66f6b1b0 2024-09-21T04:08:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.672+0000 7f5d5523e740 10 client.26252 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T04:08:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.672+0000 7f5d5523e740 10 client.26252 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:08:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.672+0000 7f5d5523e740 20 client.26252 populate_metadata read hostname 'smithi064' 2024-09-21T04:08:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.672+0000 7f5d5523e740 1 --2- 172.21.15.64:0/4289093997 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x562b66f29d80 0x562b6732e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.672+0000 7f5d5523e740 4 client.26252 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:08:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.672+0000 7f5d5523e740 1 -- 172.21.15.64:0/4289093997 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b67287640 con 0x562b66f29d80 2024-09-21T04:08:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.672+0000 7f5d477fe640 1 --2- 172.21.15.64:0/4289093997 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x562b66f29d80 0x562b6732e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.673+0000 7f5d2b7f6640 10 client.26252 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.676+0000 7f5d477fe640 1 --2- 172.21.15.64:0/4289093997 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x562b66f29d80 0x562b6732e7f0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e66bd3d8489682ed server_cookie=97db28578b00d71a in_seq=0 out_seq=0 2024-09-21T04:08:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.677+0000 7f5d2b7f6640 1 -- 172.21.15.64:0/4289093997 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562b67287640 con 0x562b66f29d80 2024-09-21T04:08:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.677+0000 7f5d2b7f6640 10 client.26252 handle_command_reply: tid=0 2024-09-21T04:08:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.677+0000 7f5d5523e740 1 client.26252 shutdown 2024-09-21T04:08:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.688+0000 7f5d5523e740 1 -- 172.21.15.64:0/4289093997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b66f6b1b0 msgr2=0x562b6731cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.688+0000 7f5d5523e740 1 --2- 172.21.15.64:0/4289093997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b66f6b1b0 0x562b6731cd00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5d38015700 tx=0x7f5d380047c0 comp rx=0 tx=0).stop 2024-09-21T04:08:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.688+0000 7f5d5523e740 1 -- 172.21.15.64:0/4289093997 shutdown_connections 2024-09-21T04:08:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.688+0000 7f5d5523e740 1 --2- 172.21.15.64:0/4289093997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b66f6b1b0 0x562b6731cd00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.688+0000 7f5d5523e740 1 --2- 172.21.15.64:0/4289093997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b66f4a4d0 0x562b6731c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.688+0000 7f5d5523e740 1 --2- 172.21.15.64:0/4289093997 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x562b66f29d80 0x562b6732e7f0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.688+0000 7f5d5523e740 1 --2- 172.21.15.64:0/4289093997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562b66f422f0 0x562b6731c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.688+0000 7f5d5523e740 1 -- 172.21.15.64:0/4289093997 >> 172.21.15.64:0/4289093997 conn(0x562b66f5f300 msgr2=0x562b673127a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.688+0000 7f5d2b7f6640 0 client.26252 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.692+0000 7f5d5523e740 1 -- 172.21.15.64:0/4289093997 shutdown_connections 2024-09-21T04:08:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.692+0000 7f5d5523e740 1 -- 172.21.15.64:0/4289093997 wait complete. 2024-09-21T04:08:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.692+0000 7f5d5523e740 20 client.26252 trim_cache size 0 max 16384 2024-09-21T04:08:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.804+0000 7f5d5523e740 1 Processor -- start 2024-09-21T04:08:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.820+0000 7f5d5523e740 1 -- start start 2024-09-21T04:08:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.820+0000 7f5d5523e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b66f422f0 0x562b6730ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.821+0000 7f5d5523e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562b66f4a4d0 0x562b6731c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.821+0000 7f5d477fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b66f422f0 0x562b6730ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.821+0000 7f5d477fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b66f422f0 0x562b6730ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:52908/0 (socket says 172.21.15.64:52908) 2024-09-21T04:08:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.821+0000 7f5d46ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562b66f4a4d0 0x562b6731c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.821+0000 7f5d46ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562b66f4a4d0 0x562b6731c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50988/0 (socket says 172.21.15.64:50988) 2024-09-21T04:08:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.822+0000 7f5d5523e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b66f6b1b0 0x562b6731cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.822+0000 7f5d46ffd640 1 -- 172.21.15.64:0/941204989 learned_addr learned my addr 172.21.15.64:0/941204989 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.822+0000 7f5d5523e740 1 -- 172.21.15.64:0/941204989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5d34011360 con 0x562b66f4a4d0 2024-09-21T04:08:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.822+0000 7f5d5523e740 1 -- 172.21.15.64:0/941204989 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3400ebd0 con 0x562b66f422f0 2024-09-21T04:08:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.822+0000 7f5d5523e740 1 -- 172.21.15.64:0/941204989 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3400ca20 con 0x562b66f6b1b0 2024-09-21T04:08:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.822+0000 7f5d47fff640 1 --2- 172.21.15.64:0/941204989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b66f6b1b0 0x562b6731cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.822+0000 7f5d477fe640 1 -- 172.21.15.64:0/941204989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b66f6b1b0 msgr2=0x562b6731cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.822+0000 7f5d477fe640 1 --2- 172.21.15.64:0/941204989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b66f6b1b0 0x562b6731cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.822+0000 7f5d477fe640 1 -- 172.21.15.64:0/941204989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562b66f4a4d0 msgr2=0x562b6731c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.823+0000 7f5d477fe640 1 --2- 172.21.15.64:0/941204989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562b66f4a4d0 0x562b6731c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.823+0000 7f5d477fe640 1 -- 172.21.15.64:0/941204989 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b673090d0 con 0x562b66f422f0 2024-09-21T04:08:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.823+0000 7f5d46ffd640 1 --2- 172.21.15.64:0/941204989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562b66f4a4d0 0x562b6731c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.823+0000 7f5d477fe640 1 --2- 172.21.15.64:0/941204989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b66f422f0 0x562b6730ffb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5d40106e40 tx=0x7f5d40009890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3ff407fa5cca92d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.832+0000 7f5d2d7fa640 1 -- 172.21.15.64:0/941204989 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d400cfc40 con 0x562b66f422f0 2024-09-21T04:08:43.834 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.832+0000 7f5d2d7fa640 1 -- 172.21.15.64:0/941204989 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d400d4020 con 0x562b66f422f0 2024-09-21T04:08:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.833+0000 7f5d5523e740 1 -- 172.21.15.64:0/941204989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b66f422f0 msgr2=0x562b6730ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:43.836 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.834+0000 7f5d5523e740 1 --2- 172.21.15.64:0/941204989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b66f422f0 0x562b6730ffb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5d40106e40 tx=0x7f5d40009890 comp rx=0 tx=0).stop 2024-09-21T04:08:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.846+0000 7f5d5523e740 1 -- 172.21.15.64:0/941204989 shutdown_connections 2024-09-21T04:08:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.846+0000 7f5d5523e740 1 --2- 172.21.15.64:0/941204989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b66f6b1b0 0x562b6731cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.846+0000 7f5d5523e740 1 --2- 172.21.15.64:0/941204989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562b66f4a4d0 0x562b6731c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.846+0000 7f5d5523e740 1 --2- 172.21.15.64:0/941204989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b66f422f0 0x562b6730ffb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.846+0000 7f5d5523e740 1 -- 172.21.15.64:0/941204989 >> 172.21.15.64:0/941204989 conn(0x562b66f5f300 msgr2=0x562b6730f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.846+0000 7f5d5523e740 1 -- 172.21.15.64:0/941204989 shutdown_connections 2024-09-21T04:08:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.846+0000 7f5d5523e740 1 -- 172.21.15.64:0/941204989 wait complete. 2024-09-21T04:08:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.847+0000 7f5d5523e740 1 -- start start 2024-09-21T04:08:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.847+0000 7f5d5523e740 1 Processor -- start 2024-09-21T04:08:43.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.848+0000 7f5d5523e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562b66f422f0 0x562b67325000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:43.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.848+0000 7f5d5523e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b66f4a4d0 0x562b6731f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.848+0000 7f5d5523e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b66f6b1b0 0x562b6731fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.848+0000 7f5d5523e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3400d090 con 0x562b66f422f0 2024-09-21T04:08:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.848+0000 7f5d5523e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3400e500 con 0x562b66f4a4d0 2024-09-21T04:08:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.848+0000 7f5d5523e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3400d760 con 0x562b66f6b1b0 2024-09-21T04:08:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.848+0000 7f5d5523e740 10 client.0 _ll_register_callbacks cb 0x7f5d38006b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:08:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.848+0000 7f5d477fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562b66f422f0 0x562b67325000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.848+0000 7f5d477fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562b66f422f0 0x562b67325000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50996/0 (socket says 172.21.15.64:50996) 2024-09-21T04:08:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.848+0000 7f5d477fe640 1 -- 172.21.15.64:0/2500679870 learned_addr learned my addr 172.21.15.64:0/2500679870 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.849+0000 7f5d46ffd640 1 --2- 172.21.15.64:0/2500679870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b66f4a4d0 0x562b6731f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.849+0000 7f5d47fff640 1 --2- 172.21.15.64:0/2500679870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b66f6b1b0 0x562b6731fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.849+0000 7f5d477fe640 1 -- 172.21.15.64:0/2500679870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b66f6b1b0 msgr2=0x562b6731fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.849+0000 7f5d477fe640 1 --2- 172.21.15.64:0/2500679870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b66f6b1b0 0x562b6731fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:43.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.849+0000 7f5d477fe640 1 -- 172.21.15.64:0/2500679870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b66f4a4d0 msgr2=0x562b6731f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.849+0000 7f5d477fe640 1 --2- 172.21.15.64:0/2500679870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b66f4a4d0 0x562b6731f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.849+0000 7f5d477fe640 1 -- 172.21.15.64:0/2500679870 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b673204b0 con 0x562b66f422f0 2024-09-21T04:08:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.849+0000 7f5d47fff640 1 --2- 172.21.15.64:0/2500679870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b66f6b1b0 0x562b6731fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:08:43.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.849+0000 7f5d477fe640 1 --2- 172.21.15.64:0/2500679870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562b66f422f0 0x562b67325000 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f5d40046000 tx=0x7f5d401072f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8db7786fde1c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.928+0000 7f5d2b7f6640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:08:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.928+0000 7f5d2b7f6640 1 -- 172.21.15.64:0/2500679870 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d400f6020 con 0x562b66f422f0 2024-09-21T04:08:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.928+0000 7f5d2b7f6640 1 -- 172.21.15.64:0/2500679870 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f5d4002b380 con 0x562b66f422f0 2024-09-21T04:08:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.928+0000 7f5d5523e740 1 -- 172.21.15.64:0/2500679870 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d400d4020 con 0x562b66f422f0 2024-09-21T04:08:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.945+0000 7f5d2b7f6640 1 -- 172.21.15.64:0/2500679870 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5d400d8f60 con 0x562b66f422f0 2024-09-21T04:08:43.947 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.945+0000 7f5d5523e740 10 client.16554 fetch_fsmap learned FSMap version 33 2024-09-21T04:08:43.948 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.945+0000 7f5d5523e740 1 -- 172.21.15.64:0/2500679870 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562b673982a0 con 0x562b66f422f0 2024-09-21T04:08:43.949 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.946+0000 7f5d2b7f6640 1 -- 172.21.15.64:0/2500679870 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d400d4020 con 0x562b66f422f0 2024-09-21T04:08:43.950 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.948+0000 7f5d2b7f6640 1 -- 172.21.15.64:0/2500679870 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f5d4000b040 con 0x562b66f422f0 2024-09-21T04:08:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.948+0000 7f5d5523e740 10 client.16554 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T04:08:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.948+0000 7f5d5523e740 10 client.16554 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:08:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.948+0000 7f5d5523e740 20 client.16554 populate_metadata read hostname 'smithi064' 2024-09-21T04:08:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.948+0000 7f5d5523e740 1 --2- 172.21.15.64:0/2500679870 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x562b66eff1c0 0x562b6739aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.948+0000 7f5d5523e740 4 client.16554 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:08:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.948+0000 7f5d5523e740 1 -- 172.21.15.64:0/2500679870 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562b672830f0 con 0x562b66eff1c0 2024-09-21T04:08:43.951 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.949+0000 7f5d46ffd640 1 --2- 172.21.15.64:0/2500679870 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x562b66eff1c0 0x562b6739aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.949+0000 7f5d46ffd640 1 --2- 172.21.15.64:0/2500679870 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x562b66eff1c0 0x562b6739aa80 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6e9ff1b9d329fdb server_cookie=a3ad9fc6a8f577f6 in_seq=0 out_seq=0 2024-09-21T04:08:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:43.949+0000 7f5d2b7f6640 10 client.16554 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:44.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:43 smithi119 ceph-mon[34811]: pgmap v462: 97 pgs: 97 active+clean; 603 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 321 op/s 2024-09-21T04:08:44.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:43 smithi064 ceph-mon[31180]: pgmap v462: 97 pgs: 97 active+clean; 603 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 321 op/s 2024-09-21T04:08:44.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:43 smithi131 ceph-mon[35193]: pgmap v462: 97 pgs: 97 active+clean; 603 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 321 op/s 2024-09-21T04:08:44.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:44.992+0000 7f5d2b7f6640 1 -- 172.21.15.64:0/2500679870 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562b672830f0 con 0x562b66eff1c0 2024-09-21T04:08:44.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:44.992+0000 7f5d2b7f6640 10 client.16554 handle_command_reply: tid=0 2024-09-21T04:08:44.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:44.992+0000 7f5d5523e740 1 client.16554 shutdown 2024-09-21T04:08:44.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:44.992+0000 7f5d5523e740 1 -- 172.21.15.64:0/2500679870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562b66f422f0 msgr2=0x562b67325000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:44.995 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:44.992+0000 7f5d5523e740 1 --2- 172.21.15.64:0/2500679870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562b66f422f0 0x562b67325000 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f5d40046000 tx=0x7f5d401072f0 comp rx=0 tx=0).stop 2024-09-21T04:08:44.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:44.993+0000 7f5d5523e740 1 -- 172.21.15.64:0/2500679870 shutdown_connections 2024-09-21T04:08:44.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:44.993+0000 7f5d5523e740 1 --2- 172.21.15.64:0/2500679870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b66f6b1b0 0x562b6731fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:44.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:44.993+0000 7f5d5523e740 1 --2- 172.21.15.64:0/2500679870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b66f4a4d0 0x562b6731f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:44.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:44.993+0000 7f5d5523e740 1 --2- 172.21.15.64:0/2500679870 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x562b66eff1c0 0x562b6739aa80 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:44.996 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:44.993+0000 7f5d5523e740 1 --2- 172.21.15.64:0/2500679870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562b66f422f0 0x562b67325000 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:44.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:44.993+0000 7f5d5523e740 1 -- 172.21.15.64:0/2500679870 >> 172.21.15.64:0/2500679870 conn(0x562b66f5f300 msgr2=0x562b6731d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:44.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:44.993+0000 7f5d2b7f6640 0 client.16554 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:44.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:44.993+0000 7f5d5523e740 1 -- 172.21.15.64:0/2500679870 shutdown_connections 2024-09-21T04:08:44.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:44.993+0000 7f5d5523e740 1 -- 172.21.15.64:0/2500679870 wait complete. 2024-09-21T04:08:44.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:44.993+0000 7f5d5523e740 20 client.16554 trim_cache size 0 max 16384 2024-09-21T04:08:44.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:08:44.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:08:44.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:08:44.997 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:08:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:45.008+0000 7f5d2e7fc640 1 -- 172.21.15.64:0/3881355121 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5d300b6800 msgr2=0x7f5d300b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:45.008+0000 7f5d2e7fc640 1 --2- 172.21.15.64:0/3881355121 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5d300b6800 0x7f5d300b8d10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5d38006fd0 tx=0x7f5d380090e0 comp rx=0 tx=0).stop 2024-09-21T04:08:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:45.008+0000 7f5d2e7fc640 1 -- 172.21.15.64:0/3881355121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d4810a900 msgr2=0x7f5d481a92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:45.008+0000 7f5d2e7fc640 1 --2- 172.21.15.64:0/3881355121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d4810a900 0x7f5d481a92d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5d4002b0d0 tx=0x7f5d40034b90 comp rx=0 tx=0).stop 2024-09-21T04:08:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:45.010+0000 7f5d2e7fc640 1 -- 172.21.15.64:0/3881355121 shutdown_connections 2024-09-21T04:08:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:45.010+0000 7f5d2e7fc640 1 --2- 172.21.15.64:0/3881355121 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5d300b6800 0x7f5d300b8d10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:45.010+0000 7f5d2e7fc640 1 --2- 172.21.15.64:0/3881355121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5d481aea40 0x7f5d481aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:45.010+0000 7f5d2e7fc640 1 --2- 172.21.15.64:0/3881355121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4811cba0 0x7f5d481a9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:45.010+0000 7f5d2e7fc640 1 --2- 172.21.15.64:0/3881355121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d4810a900 0x7f5d481a92d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:45.010+0000 7f5d2e7fc640 1 -- 172.21.15.64:0/3881355121 >> 172.21.15.64:0/3881355121 conn(0x7f5d48070c00 msgr2=0x7f5d48069a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:45.011+0000 7f5d2e7fc640 1 -- 172.21.15.64:0/3881355121 shutdown_connections 2024-09-21T04:08:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:45.011+0000 7f5d2e7fc640 1 -- 172.21.15.64:0/3881355121 wait complete. 2024-09-21T04:08:46.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:45 smithi119 ceph-mon[34811]: pgmap v463: 97 pgs: 97 active+clean; 603 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 246 op/s 2024-09-21T04:08:46.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:45 smithi119 ceph-mon[34811]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:08:46.263 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:08:46.264 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:45 smithi064 ceph-mon[31180]: pgmap v463: 97 pgs: 97 active+clean; 603 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 246 op/s 2024-09-21T04:08:46.265 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:45 smithi064 ceph-mon[31180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:08:46.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:45 smithi131 ceph-mon[35193]: pgmap v463: 97 pgs: 97 active+clean; 603 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 246 op/s 2024-09-21T04:08:46.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:45 smithi131 ceph-mon[35193]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:08:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.001+0000 7f3f08f4c640 1 -- 172.21.15.64:0/3523844912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f04102a60 msgr2=0x7f3f04102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.001+0000 7f3f08f4c640 1 --2- 172.21.15.64:0/3523844912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f04102a60 0x7f3f04102e70 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009900 tx=0x7f3ef402a3f0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.001+0000 7f3f08f4c640 1 -- 172.21.15.64:0/3523844912 shutdown_connections 2024-09-21T04:08:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.001+0000 7f3f08f4c640 1 --2- 172.21.15.64:0/3523844912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f04108cf0 0x7f3f0410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.001+0000 7f3f08f4c640 1 --2- 172.21.15.64:0/3523844912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f041083a0 0x7f3f041087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.001+0000 7f3f08f4c640 1 --2- 172.21.15.64:0/3523844912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f04102a60 0x7f3f04102e70 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.001+0000 7f3f08f4c640 1 -- 172.21.15.64:0/3523844912 >> 172.21.15.64:0/3523844912 conn(0x7f3f04079730 msgr2=0x7f3f04079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.001+0000 7f3f08f4c640 1 -- 172.21.15.64:0/3523844912 shutdown_connections 2024-09-21T04:08:47.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.001+0000 7f3f08f4c640 1 -- 172.21.15.64:0/3523844912 wait complete. 2024-09-21T04:08:47.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.002+0000 7f3f08f4c640 1 Processor -- start 2024-09-21T04:08:47.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.002+0000 7f3f08f4c640 1 -- start start 2024-09-21T04:08:47.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.003+0000 7f3f08f4c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f04102a60 0x7f3f041990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:47.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.003+0000 7f3f08f4c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f041083a0 0x7f3f041995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:47.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.003+0000 7f3f08f4c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f04108cf0 0x7f3f04199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:47.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.003+0000 7f3f0259c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f04102a60 0x7f3f041990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:47.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.003+0000 7f3f0259c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f04102a60 0x7f3f041990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:38618/0 (socket says 172.21.15.64:38618) 2024-09-21T04:08:47.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.003+0000 7f3f0259c640 1 -- 172.21.15.64:0/3930480366 learned_addr learned my addr 172.21.15.64:0/3930480366 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:47.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.003+0000 7f3f08f4c640 1 -- 172.21.15.64:0/3930480366 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3f041129b0 con 0x7f3f04108cf0 2024-09-21T04:08:47.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.003+0000 7f3f08f4c640 1 -- 172.21.15.64:0/3930480366 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3f04112830 con 0x7f3f041083a0 2024-09-21T04:08:47.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.003+0000 7f3f08f4c640 1 -- 172.21.15.64:0/3930480366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3f04112b30 con 0x7f3f04102a60 2024-09-21T04:08:47.006 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.003+0000 7f3f02d9d640 1 --2- 172.21.15.64:0/3930480366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f04108cf0 0x7f3f04199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:47.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.004+0000 7f3f01d9b640 1 --2- 172.21.15.64:0/3930480366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f041083a0 0x7f3f041995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:47.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.004+0000 7f3f0259c640 1 -- 172.21.15.64:0/3930480366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f041083a0 msgr2=0x7f3f041995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:47.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.004+0000 7f3f0259c640 1 --2- 172.21.15.64:0/3930480366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f041083a0 0x7f3f041995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.004+0000 7f3f0259c640 1 -- 172.21.15.64:0/3930480366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f04108cf0 msgr2=0x7f3f04199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:47.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.004+0000 7f3f0259c640 1 --2- 172.21.15.64:0/3930480366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f04108cf0 0x7f3f04199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.004+0000 7f3f0259c640 1 -- 172.21.15.64:0/3930480366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f041a8760 con 0x7f3f04102a60 2024-09-21T04:08:47.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.004+0000 7f3f01d9b640 1 --2- 172.21.15.64:0/3930480366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f041083a0 0x7f3f041995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:08:47.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.004+0000 7f3f02d9d640 1 --2- 172.21.15.64:0/3930480366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f04108cf0 0x7f3f04199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:08:47.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.004+0000 7f3f0259c640 1 --2- 172.21.15.64:0/3930480366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f04102a60 0x7f3f041990b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4005ab0 tx=0x7f3ef4003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:47.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.004+0000 7f3ef37fe640 1 -- 172.21.15.64:0/3930480366 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef403f070 con 0x7f3f04102a60 2024-09-21T04:08:47.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.004+0000 7f3ef37fe640 1 -- 172.21.15.64:0/3930480366 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ef4031d60 con 0x7f3f04102a60 2024-09-21T04:08:47.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.004+0000 7f3ef37fe640 1 -- 172.21.15.64:0/3930480366 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4035030 con 0x7f3f04102a60 2024-09-21T04:08:47.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.004+0000 7f3f08f4c640 1 -- 172.21.15.64:0/3930480366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f041a89f0 con 0x7f3f04102a60 2024-09-21T04:08:47.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.004+0000 7f3f08f4c640 1 -- 172.21.15.64:0/3930480366 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f041a8f20 con 0x7f3f04102a60 2024-09-21T04:08:47.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.008+0000 7f3f0f4c0740 1 Processor -- start 2024-09-21T04:08:47.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.009+0000 7f3f0f4c0740 1 -- start start 2024-09-21T04:08:47.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.009+0000 7f3ef37fe640 1 -- 172.21.15.64:0/3930480366 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f3ef4042070 con 0x7f3f04102a60 2024-09-21T04:08:47.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.009+0000 7f3f0f4c0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d0021e3350 0x55d00258a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:47.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.009+0000 7f3f0f4c0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0021ba490 0x55d00258a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:47.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.009+0000 7f3f0f4c0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0021c2670 0x55d00258aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:47.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.009+0000 7f3f0f4c0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d002500e60 con 0x55d0021ba490 2024-09-21T04:08:47.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.009+0000 7f3f0f4c0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d0024f0290 con 0x55d0021c2670 2024-09-21T04:08:47.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.009+0000 7f3f0f4c0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d0024f40a0 con 0x55d0021e3350 2024-09-21T04:08:47.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.009+0000 7f3f0259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0021ba490 0x55d00258a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:47.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.009+0000 7f3f02d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d0021e3350 0x55d00258a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:47.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.009+0000 7f3f0259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0021ba490 0x55d00258a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51016/0 (socket says 172.21.15.64:51016) 2024-09-21T04:08:47.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.009+0000 7f3f0259c640 1 -- 172.21.15.64:0/1547018169 learned_addr learned my addr 172.21.15.64:0/1547018169 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:47.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.009+0000 7f3f01d9b640 1 --2- 172.21.15.64:0/1547018169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0021c2670 0x55d00258aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:47.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.010+0000 7f3f0259c640 1 -- 172.21.15.64:0/1547018169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d0021e3350 msgr2=0x55d00258a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:47.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.010+0000 7f3f0259c640 1 --2- 172.21.15.64:0/1547018169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d0021e3350 0x55d00258a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.010+0000 7f3f0259c640 1 -- 172.21.15.64:0/1547018169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0021c2670 msgr2=0x55d00258aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:47.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.010+0000 7f3f0259c640 1 --2- 172.21.15.64:0/1547018169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0021c2670 0x55d00258aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.010+0000 7f3f0259c640 1 -- 172.21.15.64:0/1547018169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d00224e930 con 0x55d0021ba490 2024-09-21T04:08:47.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.010+0000 7f3f01d9b640 1 --2- 172.21.15.64:0/1547018169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0021c2670 0x55d00258aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:47.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.010+0000 7f3f0259c640 1 --2- 172.21.15.64:0/1547018169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0021ba490 0x55d00258a970 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4105510 tx=0x7f3ef41101a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252e511772e320bb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:47.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.011+0000 7f3ef0ff9640 1 -- 172.21.15.64:0/1547018169 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4110db0 con 0x55d0021ba490 2024-09-21T04:08:47.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.011+0000 7f3ef0ff9640 1 -- 172.21.15.64:0/1547018169 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ef4115070 con 0x55d0021ba490 2024-09-21T04:08:47.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.011+0000 7f3ef0ff9640 1 -- 172.21.15.64:0/1547018169 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4118700 con 0x55d0021ba490 2024-09-21T04:08:47.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.011+0000 7f3ef37fe640 1 --2- 172.21.15.64:0/3930480366 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3eec0b6700 0x7f3eec0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:47.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.011+0000 7f3f02d9d640 1 --2- 172.21.15.64:0/3930480366 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3eec0b6700 0x7f3eec0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:47.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.012+0000 7f3ef37fe640 1 -- 172.21.15.64:0/3930480366 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f3ef4107e80 con 0x7f3f04102a60 2024-09-21T04:08:47.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.012+0000 7f3f0f4c0740 1 -- 172.21.15.64:0/1547018169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0021ba490 msgr2=0x55d00258a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:47.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.012+0000 7f3f0f4c0740 1 --2- 172.21.15.64:0/1547018169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0021ba490 0x55d00258a970 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4105510 tx=0x7f3ef41101a0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.012+0000 7f3f02d9d640 1 --2- 172.21.15.64:0/3930480366 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3eec0b6700 0x7f3eec0b8c10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3efc005fd0 tx=0x7f3efc0098d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:47.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.012+0000 7f3f0f4c0740 1 -- 172.21.15.64:0/1547018169 shutdown_connections 2024-09-21T04:08:47.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.012+0000 7f3f0f4c0740 1 --2- 172.21.15.64:0/1547018169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0021c2670 0x55d00258aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.012+0000 7f3f0f4c0740 1 --2- 172.21.15.64:0/1547018169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0021ba490 0x55d00258a970 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.012+0000 7f3f0f4c0740 1 --2- 172.21.15.64:0/1547018169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d0021e3350 0x55d00258a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.012+0000 7f3f0f4c0740 1 -- 172.21.15.64:0/1547018169 >> 172.21.15.64:0/1547018169 conn(0x55d0021d7740 msgr2=0x55d002588320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:47.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.013+0000 7f3f0f4c0740 1 -- 172.21.15.64:0/1547018169 shutdown_connections 2024-09-21T04:08:47.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.013+0000 7f3f0f4c0740 1 -- 172.21.15.64:0/1547018169 wait complete. 2024-09-21T04:08:47.016 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.013+0000 7f3f0f4c0740 1 -- start start 2024-09-21T04:08:47.017 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.013+0000 7f3f0f4c0740 1 Processor -- start 2024-09-21T04:08:47.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.014+0000 7f3f0f4c0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0021ba490 0x55d002596030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:47.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.014+0000 7f3f0f4c0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d0021c2670 0x55d002596570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:47.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.014+0000 7f3f0f4c0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0021e3350 0x55d0025a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:47.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.014+0000 7f3f0f4c0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d002351890 con 0x55d0021ba490 2024-09-21T04:08:47.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.014+0000 7f3f0f4c0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d00248ad40 con 0x55d0021e3350 2024-09-21T04:08:47.018 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.014+0000 7f3f0f4c0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d0025989d0 con 0x55d0021c2670 2024-09-21T04:08:47.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.014+0000 7f3f0f4c0740 10 client.0 _ll_register_callbacks cb 0x55d0024eec80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:08:47.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.014+0000 7f3f02d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0021e3350 0x55d0025a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:47.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.014+0000 7f3f01d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d0021c2670 0x55d002596570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:47.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.015+0000 7f3f0259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0021ba490 0x55d002596030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:47.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.015+0000 7f3f02d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0021e3350 0x55d0025a1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:52950/0 (socket says 172.21.15.64:52950) 2024-09-21T04:08:47.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.015+0000 7f3f01d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d0021c2670 0x55d002596570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:38642/0 (socket says 172.21.15.64:38642) 2024-09-21T04:08:47.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.015+0000 7f3f02d9d640 1 -- 172.21.15.64:0/3639416850 learned_addr learned my addr 172.21.15.64:0/3639416850 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:47.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.015+0000 7f3f0259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0021ba490 0x55d002596030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51030/0 (socket says 172.21.15.64:51030) 2024-09-21T04:08:47.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.015+0000 7f3f02d9d640 1 -- 172.21.15.64:0/3639416850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d0021c2670 msgr2=0x55d002596570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:47.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.015+0000 7f3f02d9d640 1 --2- 172.21.15.64:0/3639416850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d0021c2670 0x55d002596570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.015+0000 7f3f02d9d640 1 -- 172.21.15.64:0/3639416850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0021ba490 msgr2=0x55d002596030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:47.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.015+0000 7f3f02d9d640 1 --2- 172.21.15.64:0/3639416850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0021ba490 0x55d002596030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.019 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.015+0000 7f3f02d9d640 1 -- 172.21.15.64:0/3639416850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0025a2280 con 0x55d0021e3350 2024-09-21T04:08:47.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.015+0000 7f3f02d9d640 1 --2- 172.21.15.64:0/3639416850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0021e3350 0x55d0025a1cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3efc009b30 tx=0x7f3efc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6460acb0fc522d5f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:47.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.015+0000 7f3ee9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:08:47.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.016+0000 7f3ee9ffb640 1 -- 172.21.15.64:0/3639416850 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3efc01e820 con 0x55d0021e3350 2024-09-21T04:08:47.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.016+0000 7f3f0f4c0740 1 -- 172.21.15.64:0/3639416850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ef4115070 con 0x55d0021e3350 2024-09-21T04:08:47.020 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.016+0000 7f3ee9ffb640 1 -- 172.21.15.64:0/3639416850 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f3efc01e9c0 con 0x55d0021e3350 2024-09-21T04:08:47.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.016+0000 7f3ee9ffb640 1 -- 172.21.15.64:0/3639416850 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3efc01ebc0 con 0x55d0021e3350 2024-09-21T04:08:47.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.016+0000 7f3f0f4c0740 10 client.26267 fetch_fsmap learned FSMap version 34 2024-09-21T04:08:47.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.016+0000 7f3f0f4c0740 1 -- 172.21.15.64:0/3639416850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d0025a4c60 con 0x55d0021e3350 2024-09-21T04:08:47.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.016+0000 7f3ee9ffb640 1 -- 172.21.15.64:0/3639416850 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3efc01ed60 con 0x55d0021e3350 2024-09-21T04:08:47.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.017+0000 7f3ee9ffb640 1 -- 172.21.15.64:0/3639416850 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f3efc00f040 con 0x55d0021e3350 2024-09-21T04:08:47.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.017+0000 7f3f0f4c0740 10 client.26267 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T04:08:47.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.017+0000 7f3f0f4c0740 10 client.26267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:08:47.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.017+0000 7f3f0f4c0740 20 client.26267 populate_metadata read hostname 'smithi064' 2024-09-21T04:08:47.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.017+0000 7f3f0f4c0740 1 --2- 172.21.15.64:0/3639416850 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d0021a1cd0 0x55d0025a8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:47.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.017+0000 7f3f0f4c0740 4 client.26267 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:08:47.023 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.017+0000 7f3f0f4c0740 1 -- 172.21.15.64:0/3639416850 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3ef4115070 con 0x55d0021a1cd0 2024-09-21T04:08:47.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.017+0000 7f3f0259c640 1 --2- 172.21.15.64:0/3639416850 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d0021a1cd0 0x55d0025a8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:47.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.018+0000 7f3ee9ffb640 10 client.26267 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:47.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.018+0000 7f3f0259c640 1 --2- 172.21.15.64:0/3639416850 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d0021a1cd0 0x55d0025a8580 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59f557978cbf3b2a server_cookie=b25b36533b2d32c7 in_seq=0 out_seq=0 2024-09-21T04:08:47.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.019+0000 7f3ee9ffb640 1 -- 172.21.15.64:0/3639416850 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f3ef4115070 con 0x55d0021a1cd0 2024-09-21T04:08:47.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.019+0000 7f3ee9ffb640 10 client.26267 handle_command_reply: tid=0 2024-09-21T04:08:47.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.019+0000 7f3f0f4c0740 1 client.26267 shutdown 2024-09-21T04:08:47.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.019+0000 7f3f0f4c0740 1 -- 172.21.15.64:0/3639416850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0021e3350 msgr2=0x55d0025a1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:47.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.019+0000 7f3f0f4c0740 1 --2- 172.21.15.64:0/3639416850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0021e3350 0x55d0025a1cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3efc009b30 tx=0x7f3efc004a40 comp rx=0 tx=0).stop 2024-09-21T04:08:47.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.020+0000 7f3f0f4c0740 1 -- 172.21.15.64:0/3639416850 shutdown_connections 2024-09-21T04:08:47.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.020+0000 7f3f0f4c0740 1 --2- 172.21.15.64:0/3639416850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0021e3350 0x55d0025a1cc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.020+0000 7f3f0f4c0740 1 --2- 172.21.15.64:0/3639416850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d0021c2670 0x55d002596570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.020+0000 7f3f0f4c0740 1 --2- 172.21.15.64:0/3639416850 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d0021a1cd0 0x55d0025a8580 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.020+0000 7f3f0f4c0740 1 --2- 172.21.15.64:0/3639416850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0021ba490 0x55d002596030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.024 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.020+0000 7f3ee9ffb640 0 client.26267 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:47.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.020+0000 7f3f0f4c0740 1 -- 172.21.15.64:0/3639416850 >> 172.21.15.64:0/3639416850 conn(0x55d0021d7740 msgr2=0x55d00258d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:47.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.020+0000 7f3f0f4c0740 1 -- 172.21.15.64:0/3639416850 shutdown_connections 2024-09-21T04:08:47.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.020+0000 7f3f0f4c0740 1 -- 172.21.15.64:0/3639416850 wait complete. 2024-09-21T04:08:47.025 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.020+0000 7f3f0f4c0740 20 client.26267 trim_cache size 0 max 16384 2024-09-21T04:08:47.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.060+0000 7f3f0f4c0740 1 Processor -- start 2024-09-21T04:08:47.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.060+0000 7f3f0f4c0740 1 -- start start 2024-09-21T04:08:47.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.060+0000 7f3f0f4c0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0021ba490 0x55d00258f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:47.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.060+0000 7f3f0f4c0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d0021c2670 0x55d00258fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:47.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.060+0000 7f3f0f4c0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0021e3350 0x55d002593020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:47.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.060+0000 7f3f0f4c0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee4011210 con 0x55d0021ba490 2024-09-21T04:08:47.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.060+0000 7f3f0f4c0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee400eab0 con 0x55d0021e3350 2024-09-21T04:08:47.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.060+0000 7f3f0f4c0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee400c900 con 0x55d0021c2670 2024-09-21T04:08:47.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.060+0000 7f3f0259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0021ba490 0x55d00258f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:47.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.060+0000 7f3f0259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0021ba490 0x55d00258f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51032/0 (socket says 172.21.15.64:51032) 2024-09-21T04:08:47.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.060+0000 7f3f0259c640 1 -- 172.21.15.64:0/503851998 learned_addr learned my addr 172.21.15.64:0/503851998 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:47.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.060+0000 7f3f02d9d640 1 --2- 172.21.15.64:0/503851998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0021e3350 0x55d002593020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:47.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.061+0000 7f3f01d9b640 1 --2- 172.21.15.64:0/503851998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d0021c2670 0x55d00258fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:47.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.061+0000 7f3f0259c640 1 -- 172.21.15.64:0/503851998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d0021c2670 msgr2=0x55d00258fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.061+0000 7f3f0259c640 1 --2- 172.21.15.64:0/503851998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d0021c2670 0x55d00258fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.061+0000 7f3f0259c640 1 -- 172.21.15.64:0/503851998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0021e3350 msgr2=0x55d002593020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.061+0000 7f3f0259c640 1 --2- 172.21.15.64:0/503851998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0021e3350 0x55d002593020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.061+0000 7f3f0259c640 1 -- 172.21.15.64:0/503851998 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d002598570 con 0x55d0021ba490 2024-09-21T04:08:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.061+0000 7f3f02d9d640 1 --2- 172.21.15.64:0/503851998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0021e3350 0x55d002593020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.061+0000 7f3f0259c640 1 --2- 172.21.15.64:0/503851998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0021ba490 0x55d00258f540 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40caf10 tx=0x7f3ef402a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f90a35723655fd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.061+0000 7f3eebfff640 1 -- 172.21.15.64:0/503851998 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef40c9b40 con 0x55d0021ba490 2024-09-21T04:08:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.061+0000 7f3eebfff640 1 -- 172.21.15.64:0/503851998 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ef4115040 con 0x55d0021ba490 2024-09-21T04:08:47.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.061+0000 7f3eebfff640 1 -- 172.21.15.64:0/503851998 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef40c9410 con 0x55d0021ba490 2024-09-21T04:08:47.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.062+0000 7f3f0f4c0740 1 -- 172.21.15.64:0/503851998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0021ba490 msgr2=0x55d00258f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:47.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.062+0000 7f3f0f4c0740 1 --2- 172.21.15.64:0/503851998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0021ba490 0x55d00258f540 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40caf10 tx=0x7f3ef402a3f0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.062+0000 7f3f0f4c0740 1 -- 172.21.15.64:0/503851998 shutdown_connections 2024-09-21T04:08:47.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.062+0000 7f3f0f4c0740 1 --2- 172.21.15.64:0/503851998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0021e3350 0x55d002593020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.062+0000 7f3f0f4c0740 1 --2- 172.21.15.64:0/503851998 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d0021c2670 0x55d00258fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.062+0000 7f3f0f4c0740 1 --2- 172.21.15.64:0/503851998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0021ba490 0x55d00258f540 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.062+0000 7f3f0f4c0740 1 -- 172.21.15.64:0/503851998 >> 172.21.15.64:0/503851998 conn(0x55d0021d7740 msgr2=0x55d0025946b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:47.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.062+0000 7f3f0f4c0740 1 -- 172.21.15.64:0/503851998 shutdown_connections 2024-09-21T04:08:47.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.063+0000 7f3f0f4c0740 1 -- 172.21.15.64:0/503851998 wait complete. 2024-09-21T04:08:47.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.063+0000 7f3f0f4c0740 1 -- start start 2024-09-21T04:08:47.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.063+0000 7f3f0f4c0740 1 Processor -- start 2024-09-21T04:08:47.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.063+0000 7f3f0f4c0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d0021ba490 0x55d002599c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:47.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.063+0000 7f3f0f4c0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0021c2670 0x55d00259a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:47.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.063+0000 7f3f0f4c0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0021e3350 0x55d00259a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:47.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.063+0000 7f3f0f4c0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee400cf70 con 0x55d0021e3350 2024-09-21T04:08:47.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.063+0000 7f3f0f4c0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee400e3e0 con 0x55d0021c2670 2024-09-21T04:08:47.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.063+0000 7f3f0f4c0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee400d640 con 0x55d0021ba490 2024-09-21T04:08:47.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.064+0000 7f3f0f4c0740 10 client.0 _ll_register_callbacks cb 0x7f3efc009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:08:47.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.064+0000 7f3f01d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0021c2670 0x55d00259a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:47.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.064+0000 7f3f0259c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d0021ba490 0x55d002599c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.064+0000 7f3f02d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0021e3350 0x55d00259a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.064+0000 7f3f0259c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d0021ba490 0x55d002599c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:38650/0 (socket says 172.21.15.64:38650) 2024-09-21T04:08:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.064+0000 7f3f0259c640 1 -- 172.21.15.64:0/1666856619 learned_addr learned my addr 172.21.15.64:0/1666856619 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.064+0000 7f3f0259c640 1 -- 172.21.15.64:0/1666856619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0021c2670 msgr2=0x55d00259a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.064+0000 7f3f0259c640 1 --2- 172.21.15.64:0/1666856619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0021c2670 0x55d00259a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.064+0000 7f3f0259c640 1 -- 172.21.15.64:0/1666856619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0021e3350 msgr2=0x55d00259a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.064+0000 7f3f0259c640 1 --2- 172.21.15.64:0/1666856619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0021e3350 0x55d00259a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.064+0000 7f3f0259c640 1 -- 172.21.15.64:0/1666856619 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d002611f40 con 0x55d0021ba490 2024-09-21T04:08:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.064+0000 7f3f02d9d640 1 --2- 172.21.15.64:0/1666856619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0021e3350 0x55d00259a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.065+0000 7f3f0259c640 1 --2- 172.21.15.64:0/1666856619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d0021ba490 0x55d002599c50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40ca270 tx=0x7f3ef4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f11e132016a0cc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.065+0000 7f3ee9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-21T04:08:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.065+0000 7f3ee9ffb640 1 -- 172.21.15.64:0/1666856619 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef40cd410 con 0x55d0021ba490 2024-09-21T04:08:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.065+0000 7f3ee9ffb640 1 -- 172.21.15.64:0/1666856619 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f3ef410b7a0 con 0x55d0021ba490 2024-09-21T04:08:47.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.065+0000 7f3f0f4c0740 1 -- 172.21.15.64:0/1666856619 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ef4115040 con 0x55d0021ba490 2024-09-21T04:08:47.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.065+0000 7f3ee9ffb640 1 -- 172.21.15.64:0/1666856619 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3ef40cd5f0 con 0x55d0021ba490 2024-09-21T04:08:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.065+0000 7f3f0f4c0740 10 client.26155 fetch_fsmap learned FSMap version 34 2024-09-21T04:08:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.065+0000 7f3f0f4c0740 1 -- 172.21.15.64:0/1666856619 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d002613200 con 0x55d0021ba490 2024-09-21T04:08:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.065+0000 7f3ee9ffb640 1 -- 172.21.15.64:0/1666856619 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ef40cd7b0 con 0x55d0021ba490 2024-09-21T04:08:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.066+0000 7f3ee9ffb640 1 -- 172.21.15.64:0/1666856619 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f3ef4105820 con 0x55d0021ba490 2024-09-21T04:08:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.066+0000 7f3f0f4c0740 10 client.26155 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T04:08:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.066+0000 7f3f0f4c0740 10 client.26155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:08:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.066+0000 7f3f0f4c0740 20 client.26155 populate_metadata read hostname 'smithi064' 2024-09-21T04:08:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.066+0000 7f3f0f4c0740 1 --2- 172.21.15.64:0/1666856619 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d002177080 0x55d002615440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.066+0000 7f3f0f4c0740 4 client.26155 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T04:08:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.066+0000 7f3f0f4c0740 1 -- 172.21.15.64:0/1666856619 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f3ef4115040 con 0x55d002177080 2024-09-21T04:08:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.066+0000 7f3f01d9b640 1 --2- 172.21.15.64:0/1666856619 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d002177080 0x55d002615440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.066+0000 7f3ee9ffb640 10 client.26155 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.067+0000 7f3f01d9b640 1 --2- 172.21.15.64:0/1666856619 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d002177080 0x55d002615440 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8eff086ed70b9b4f server_cookie=6724b22b97c9751e in_seq=0 out_seq=0 2024-09-21T04:08:47.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.068+0000 7f3ee9ffb640 1 -- 172.21.15.64:0/1666856619 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f3ef4115040 con 0x55d002177080 2024-09-21T04:08:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.068+0000 7f3ee9ffb640 10 client.26155 handle_command_reply: tid=0 2024-09-21T04:08:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.068+0000 7f3f0f4c0740 1 client.26155 shutdown 2024-09-21T04:08:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.068+0000 7f3f0f4c0740 1 -- 172.21.15.64:0/1666856619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d0021ba490 msgr2=0x55d002599c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.068+0000 7f3f0f4c0740 1 --2- 172.21.15.64:0/1666856619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d0021ba490 0x55d002599c50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40ca270 tx=0x7f3ef4009260 comp rx=0 tx=0).stop 2024-09-21T04:08:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.069+0000 7f3f0f4c0740 1 -- 172.21.15.64:0/1666856619 shutdown_connections 2024-09-21T04:08:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.069+0000 7f3f0f4c0740 1 --2- 172.21.15.64:0/1666856619 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d002177080 0x55d002615440 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.069+0000 7f3f0f4c0740 1 --2- 172.21.15.64:0/1666856619 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d0021e3350 0x55d00259a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.069+0000 7f3f0f4c0740 1 --2- 172.21.15.64:0/1666856619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d0021c2670 0x55d00259a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.069+0000 7f3f0f4c0740 1 --2- 172.21.15.64:0/1666856619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d0021ba490 0x55d002599c50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.069+0000 7f3f0f4c0740 1 -- 172.21.15.64:0/1666856619 >> 172.21.15.64:0/1666856619 conn(0x55d0021d7740 msgr2=0x55d00258b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.069+0000 7f3ee9ffb640 0 client.26155 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.069+0000 7f3f0f4c0740 1 -- 172.21.15.64:0/1666856619 shutdown_connections 2024-09-21T04:08:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.069+0000 7f3f0f4c0740 1 -- 172.21.15.64:0/1666856619 wait complete. 2024-09-21T04:08:47.073 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.069+0000 7f3f0f4c0740 20 client.26155 trim_cache size 0 max 16384 2024-09-21T04:08:47.074 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:08:47.074 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-21T04:08:47.074 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "e2c2eed8-9292-4bf6-8a39-13ad27da5acb", 2024-09-21T04:08:47.074 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-21T04:08:47.074 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:08:47.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.070+0000 7f3f08f4c640 1 -- 172.21.15.64:0/3930480366 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3eec0b6700 msgr2=0x7f3eec0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:47.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.070+0000 7f3f08f4c640 1 --2- 172.21.15.64:0/3930480366 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3eec0b6700 0x7f3eec0b8c10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3efc005fd0 tx=0x7f3efc0098d0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.070+0000 7f3f08f4c640 1 -- 172.21.15.64:0/3930480366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f04102a60 msgr2=0x7f3f041990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:47.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.070+0000 7f3f08f4c640 1 --2- 172.21.15.64:0/3930480366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f04102a60 0x7f3f041990b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4005ab0 tx=0x7f3ef4003c80 comp rx=0 tx=0).stop 2024-09-21T04:08:47.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.070+0000 7f3f08f4c640 1 -- 172.21.15.64:0/3930480366 shutdown_connections 2024-09-21T04:08:47.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.070+0000 7f3f08f4c640 1 --2- 172.21.15.64:0/3930480366 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3eec0b6700 0x7f3eec0b8c10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.070+0000 7f3f08f4c640 1 --2- 172.21.15.64:0/3930480366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f04108cf0 0x7f3f04199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.070+0000 7f3f08f4c640 1 --2- 172.21.15.64:0/3930480366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f041083a0 0x7f3f041995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.070+0000 7f3f08f4c640 1 --2- 172.21.15.64:0/3930480366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3f04102a60 0x7f3f041990b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:47.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.070+0000 7f3f08f4c640 1 -- 172.21.15.64:0/3930480366 >> 172.21.15.64:0/3930480366 conn(0x7f3f04079730 msgr2=0x7f3f041013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:47.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.070+0000 7f3f08f4c640 1 -- 172.21.15.64:0/3930480366 shutdown_connections 2024-09-21T04:08:47.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:47.070+0000 7f3f08f4c640 1 -- 172.21.15.64:0/3930480366 wait complete. 2024-09-21T04:08:48.185 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:47 smithi064 ceph-mon[31180]: pgmap v464: 97 pgs: 97 active+clean; 611 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 359 op/s 2024-09-21T04:08:48.185 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:47 smithi064 ceph-mon[31180]: scrub queued for path: / 2024-09-21T04:08:48.186 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:47 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:48.186 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:47 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:08:48.186 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:47 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:08:48.186 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:47 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:08:48.186 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:47 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:48.212 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T04:08:48.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:47 smithi119 ceph-mon[34811]: pgmap v464: 97 pgs: 97 active+clean; 611 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 359 op/s 2024-09-21T04:08:48.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:47 smithi119 ceph-mon[34811]: scrub queued for path: / 2024-09-21T04:08:48.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:47 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:48.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:47 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:08:48.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:47 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:08:48.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:47 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:08:48.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:47 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:48.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:47 smithi131 ceph-mon[35193]: pgmap v464: 97 pgs: 97 active+clean; 611 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 359 op/s 2024-09-21T04:08:48.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:47 smithi131 ceph-mon[35193]: scrub queued for path: / 2024-09-21T04:08:48.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:47 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:48.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:47 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:08:48.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:47 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:08:48.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:47 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:08:48.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:47 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:48.543 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:08:48.957 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:48 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:49.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.081+0000 7f10ecccf640 1 -- 172.21.15.64:0/46933019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10e81083a0 msgr2=0x7f10e81087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:49.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.081+0000 7f10ecccf640 1 --2- 172.21.15.64:0/46933019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10e81083a0 0x7f10e81087b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f10d0009900 tx=0x7f10d002a410 comp rx=0 tx=0).stop 2024-09-21T04:08:49.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.081+0000 7f10ecccf640 1 -- 172.21.15.64:0/46933019 shutdown_connections 2024-09-21T04:08:49.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.081+0000 7f10ecccf640 1 --2- 172.21.15.64:0/46933019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10e8108cf0 0x7f10e810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.081+0000 7f10ecccf640 1 --2- 172.21.15.64:0/46933019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10e81083a0 0x7f10e81087b0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.081+0000 7f10ecccf640 1 --2- 172.21.15.64:0/46933019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10e8102a60 0x7f10e8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.081+0000 7f10ecccf640 1 -- 172.21.15.64:0/46933019 >> 172.21.15.64:0/46933019 conn(0x7f10e8079730 msgr2=0x7f10e8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:49.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.082+0000 7f10ecccf640 1 -- 172.21.15.64:0/46933019 shutdown_connections 2024-09-21T04:08:49.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.082+0000 7f10ecccf640 1 -- 172.21.15.64:0/46933019 wait complete. 2024-09-21T04:08:49.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.082+0000 7f10ecccf640 1 Processor -- start 2024-09-21T04:08:49.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.082+0000 7f10ecccf640 1 -- start start 2024-09-21T04:08:49.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.083+0000 7f10ecccf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10e8102a60 0x7f10e80ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:49.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.083+0000 7f10ecccf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10e81083a0 0x7f10e80ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:49.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.083+0000 7f10ecccf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10e8108cf0 0x7f10e81000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:49.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.083+0000 7f10ecccf640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f10e8112960 con 0x7f10e8108cf0 2024-09-21T04:08:49.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.083+0000 7f10ecccf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f10e81127e0 con 0x7f10e81083a0 2024-09-21T04:08:49.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.083+0000 7f10ecccf640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f10e8112ae0 con 0x7f10e8102a60 2024-09-21T04:08:49.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.083+0000 7f10e6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10e8108cf0 0x7f10e81000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:49.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.083+0000 7f10e6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10e8108cf0 0x7f10e81000f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51054/0 (socket says 172.21.15.64:51054) 2024-09-21T04:08:49.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.083+0000 7f10e6d9d640 1 -- 172.21.15.64:0/1459085068 learned_addr learned my addr 172.21.15.64:0/1459085068 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:49.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.083+0000 7f10e659c640 1 --2- 172.21.15.64:0/1459085068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10e8102a60 0x7f10e80ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:49.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.083+0000 7f10e5d9b640 1 --2- 172.21.15.64:0/1459085068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10e81083a0 0x7f10e80ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:49.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.083+0000 7f10e659c640 1 -- 172.21.15.64:0/1459085068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10e81083a0 msgr2=0x7f10e80ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:49.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.083+0000 7f10e659c640 1 --2- 172.21.15.64:0/1459085068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10e81083a0 0x7f10e80ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.083+0000 7f10e659c640 1 -- 172.21.15.64:0/1459085068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10e8108cf0 msgr2=0x7f10e81000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:49.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.083+0000 7f10e659c640 1 --2- 172.21.15.64:0/1459085068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10e8108cf0 0x7f10e81000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.083+0000 7f10e659c640 1 -- 172.21.15.64:0/1459085068 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e81a87a0 con 0x7f10e8102a60 2024-09-21T04:08:49.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.083+0000 7f10e6d9d640 1 --2- 172.21.15.64:0/1459085068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10e8108cf0 0x7f10e81000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:49.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.084+0000 7f10e659c640 1 --2- 172.21.15.64:0/1459085068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10e8102a60 0x7f10e80ff670 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f10d800dab0 tx=0x7f10d800df70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:49.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.084+0000 7f10df7fe640 1 -- 172.21.15.64:0/1459085068 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d800be30 con 0x7f10e8102a60 2024-09-21T04:08:49.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.084+0000 7f10df7fe640 1 -- 172.21.15.64:0/1459085068 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10d8003e20 con 0x7f10e8102a60 2024-09-21T04:08:49.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.084+0000 7f10ecccf640 1 -- 172.21.15.64:0/1459085068 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e81a8a90 con 0x7f10e8102a60 2024-09-21T04:08:49.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.084+0000 7f10df7fe640 1 -- 172.21.15.64:0/1459085068 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d80106a0 con 0x7f10e8102a60 2024-09-21T04:08:49.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.084+0000 7f10ecccf640 1 -- 172.21.15.64:0/1459085068 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e81a8fa0 con 0x7f10e8102a60 2024-09-21T04:08:49.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.086+0000 7f10f3243740 1 Processor -- start 2024-09-21T04:08:49.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.087+0000 7f10f3243740 1 -- start start 2024-09-21T04:08:49.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.087+0000 7f10df7fe640 1 -- 172.21.15.64:0/1459085068 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f10d8004000 con 0x7f10e8102a60 2024-09-21T04:08:49.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.087+0000 7f10f3243740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f64fdfd070 0x55f6501a4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:49.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.087+0000 7f10f3243740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f64fdd41b0 0x55f6501a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:49.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.087+0000 7f10f3243740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f64fddc390 0x55f6501a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:49.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.087+0000 7f10f3243740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f65011ae50 con 0x55f64fddc390 2024-09-21T04:08:49.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.087+0000 7f10f3243740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f65010a280 con 0x55f64fdfd070 2024-09-21T04:08:49.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.087+0000 7f10f3243740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f65010e090 con 0x55f64fdd41b0 2024-09-21T04:08:49.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.087+0000 7f10e659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f64fdd41b0 0x55f6501a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:49.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.087+0000 7f10e5d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f64fddc390 0x55f6501a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:49.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.087+0000 7f10e659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f64fdd41b0 0x55f6501a47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:38670/0 (socket says 172.21.15.64:38670) 2024-09-21T04:08:49.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.087+0000 7f10e659c640 1 -- 172.21.15.64:0/389423554 learned_addr learned my addr 172.21.15.64:0/389423554 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:49.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.087+0000 7f10e6d9d640 1 --2- 172.21.15.64:0/389423554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f64fdfd070 0x55f6501a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:49.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.088+0000 7f10e659c640 1 -- 172.21.15.64:0/389423554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f64fdfd070 msgr2=0x55f6501a4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:49.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.088+0000 7f10e659c640 1 --2- 172.21.15.64:0/389423554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f64fdfd070 0x55f6501a4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.088+0000 7f10e659c640 1 -- 172.21.15.64:0/389423554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f64fddc390 msgr2=0x55f6501a4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:49.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.088+0000 7f10e659c640 1 --2- 172.21.15.64:0/389423554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f64fddc390 0x55f6501a4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.088+0000 7f10e659c640 1 -- 172.21.15.64:0/389423554 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f64fe68650 con 0x55f64fdd41b0 2024-09-21T04:08:49.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.088+0000 7f10e6d9d640 1 --2- 172.21.15.64:0/389423554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f64fdfd070 0x55f6501a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:08:49.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.088+0000 7f10e659c640 1 --2- 172.21.15.64:0/389423554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f64fdd41b0 0x55f6501a47b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f10d80e5100 tx=0x7f10d80ebba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0f43ce721c69279 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:49.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.088+0000 7f10df7fe640 1 --2- 172.21.15.64:0/1459085068 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f10d40b6680 0x7f10d40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:49.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.088+0000 7f10e6d9d640 1 --2- 172.21.15.64:0/1459085068 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f10d40b6680 0x7f10d40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:49.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.088+0000 7f10dcff9640 1 -- 172.21.15.64:0/389423554 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d80ef040 con 0x55f64fdd41b0 2024-09-21T04:08:49.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.088+0000 7f10dcff9640 1 -- 172.21.15.64:0/389423554 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10d80ad3d0 con 0x55f64fdd41b0 2024-09-21T04:08:49.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.088+0000 7f10df7fe640 1 -- 172.21.15.64:0/1459085068 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f10d80e7ac0 con 0x7f10e8102a60 2024-09-21T04:08:49.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.088+0000 7f10dcff9640 1 -- 172.21.15.64:0/389423554 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d80a9760 con 0x55f64fdd41b0 2024-09-21T04:08:49.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.089+0000 7f10f3243740 1 -- 172.21.15.64:0/389423554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f64fdd41b0 msgr2=0x55f6501a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:49.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.089+0000 7f10e6d9d640 1 --2- 172.21.15.64:0/1459085068 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f10d40b6680 0x7f10d40b8b90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f10e00125f0 tx=0x7f10e0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:49.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.089+0000 7f10f3243740 1 --2- 172.21.15.64:0/389423554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f64fdd41b0 0x55f6501a47b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f10d80e5100 tx=0x7f10d80ebba0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.089+0000 7f10f3243740 1 -- 172.21.15.64:0/389423554 shutdown_connections 2024-09-21T04:08:49.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.089+0000 7f10f3243740 1 --2- 172.21.15.64:0/389423554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f64fddc390 0x55f6501a4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.089+0000 7f10f3243740 1 --2- 172.21.15.64:0/389423554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f64fdd41b0 0x55f6501a47b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.089+0000 7f10f3243740 1 --2- 172.21.15.64:0/389423554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f64fdfd070 0x55f6501a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.089+0000 7f10f3243740 1 -- 172.21.15.64:0/389423554 >> 172.21.15.64:0/389423554 conn(0x55f64fdf1460 msgr2=0x55f6501a2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.089+0000 7f10f3243740 1 -- 172.21.15.64:0/389423554 shutdown_connections 2024-09-21T04:08:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.089+0000 7f10f3243740 1 -- 172.21.15.64:0/389423554 wait complete. 2024-09-21T04:08:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.090+0000 7f10f3243740 1 -- start start 2024-09-21T04:08:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.090+0000 7f10f3243740 1 Processor -- start 2024-09-21T04:08:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.091+0000 7f10f3243740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f64fdd41b0 0x55f6501af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.091+0000 7f10f3243740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f64fddc390 0x55f6501afde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.091+0000 7f10f3243740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f64fdfd070 0x55f6501b0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.091+0000 7f10e659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f64fdd41b0 0x55f6501af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.091+0000 7f10e659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f64fdd41b0 0x55f6501af8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51068/0 (socket says 172.21.15.64:51068) 2024-09-21T04:08:49.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.091+0000 7f10e659c640 1 -- 172.21.15.64:0/408144349 learned_addr learned my addr 172.21.15.64:0/408144349 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:49.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.091+0000 7f10f3243740 1 -- 172.21.15.64:0/408144349 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f64ff6b700 con 0x55f64fdd41b0 2024-09-21T04:08:49.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.091+0000 7f10f3243740 1 -- 172.21.15.64:0/408144349 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f6500a5110 con 0x55f64fddc390 2024-09-21T04:08:49.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.091+0000 7f10f3243740 1 -- 172.21.15.64:0/408144349 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f6501b2810 con 0x55f64fdfd070 2024-09-21T04:08:49.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.091+0000 7f10e5d9b640 1 --2- 172.21.15.64:0/408144349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f64fddc390 0x55f6501afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:49.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.091+0000 7f10e6d9d640 1 --2- 172.21.15.64:0/408144349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f64fdfd070 0x55f6501b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:49.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.091+0000 7f10f3243740 10 client.0 _ll_register_callbacks cb 0x55f6500be990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:08:49.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.091+0000 7f10e659c640 1 -- 172.21.15.64:0/408144349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f64fdfd070 msgr2=0x55f6501b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:49.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.091+0000 7f10e659c640 1 --2- 172.21.15.64:0/408144349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f64fdfd070 0x55f6501b0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.091+0000 7f10e659c640 1 -- 172.21.15.64:0/408144349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f64fddc390 msgr2=0x55f6501afde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:49.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.091+0000 7f10e659c640 1 --2- 172.21.15.64:0/408144349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f64fddc390 0x55f6501afde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.091+0000 7f10e659c640 1 -- 172.21.15.64:0/408144349 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6500d2c40 con 0x55f64fdd41b0 2024-09-21T04:08:49.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.091+0000 7f10f3243740 1 -- 172.21.15.64:0/408144349 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10d80ad3d0 con 0x55f64fdd41b0 2024-09-21T04:08:49.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.091+0000 7f10e659c640 1 --2- 172.21.15.64:0/408144349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f64fdd41b0 0x55f6501af8a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f10d80ad850 tx=0x7f10d80a9d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f09ccc79cd5110 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:49.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.091+0000 7f10cdffb640 10 client.16584 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:08:49.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.092+0000 7f10cdffb640 1 -- 172.21.15.64:0/408144349 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d804f0f0 con 0x55f64fdd41b0 2024-09-21T04:08:49.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.092+0000 7f10cdffb640 1 -- 172.21.15.64:0/408144349 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f10d80e5440 con 0x55f64fdd41b0 2024-09-21T04:08:49.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.092+0000 7f10cdffb640 1 -- 172.21.15.64:0/408144349 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f10d804f330 con 0x55f64fdd41b0 2024-09-21T04:08:49.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.092+0000 7f10f3243740 10 client.16584 fetch_fsmap learned FSMap version 34 2024-09-21T04:08:49.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.092+0000 7f10f3243740 1 -- 172.21.15.64:0/408144349 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({fsmap=34}) -- 0x55f6501be380 con 0x55f64fdd41b0 2024-09-21T04:08:49.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.092+0000 7f10cdffb640 1 -- 172.21.15.64:0/408144349 <== mon.0 v2:172.21.15.64:3300/0 4 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f10d8024060 con 0x55f64fdd41b0 2024-09-21T04:08:49.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.092+0000 7f10f3243740 10 client.16584 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T04:08:49.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.092+0000 7f10f3243740 10 client.16584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:08:49.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.092+0000 7f10f3243740 20 client.16584 populate_metadata read hostname 'smithi064' 2024-09-21T04:08:49.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.092+0000 7f10f3243740 1 --2- 172.21.15.64:0/408144349 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f64fdbb9f0 0x55f6501c16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:49.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.092+0000 7f10f3243740 4 client.16584 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:08:49.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.092+0000 7f10f3243740 1 -- 172.21.15.64:0/408144349 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f65011cda0 con 0x55f64fdbb9f0 2024-09-21T04:08:49.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.093+0000 7f10e5d9b640 1 --2- 172.21.15.64:0/408144349 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f64fdbb9f0 0x55f6501c16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:49.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.093+0000 7f10cdffb640 10 client.16584 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:49.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.093+0000 7f10e5d9b640 1 --2- 172.21.15.64:0/408144349 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f64fdbb9f0 0x55f6501c16e0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6a59772686016ba server_cookie=2a3785fe9d9ee76 in_seq=0 out_seq=0 2024-09-21T04:08:49.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.094+0000 7f10cdffb640 1 -- 172.21.15.64:0/408144349 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f65011cda0 con 0x55f64fdbb9f0 2024-09-21T04:08:49.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.094+0000 7f10cdffb640 10 client.16584 handle_command_reply: tid=0 2024-09-21T04:08:49.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.094+0000 7f10f3243740 1 client.16584 shutdown 2024-09-21T04:08:49.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.094+0000 7f10f3243740 1 -- 172.21.15.64:0/408144349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f64fdd41b0 msgr2=0x55f6501af8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:49.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.094+0000 7f10f3243740 1 --2- 172.21.15.64:0/408144349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f64fdd41b0 0x55f6501af8a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f10d80ad850 tx=0x7f10d80a9d60 comp rx=0 tx=0).stop 2024-09-21T04:08:49.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.094+0000 7f10f3243740 1 -- 172.21.15.64:0/408144349 shutdown_connections 2024-09-21T04:08:49.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.094+0000 7f10f3243740 1 --2- 172.21.15.64:0/408144349 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f64fdfd070 0x55f6501b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.094+0000 7f10f3243740 1 --2- 172.21.15.64:0/408144349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f64fddc390 0x55f6501afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.094+0000 7f10f3243740 1 --2- 172.21.15.64:0/408144349 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f64fdbb9f0 0x55f6501c16e0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.094+0000 7f10cdffb640 0 client.16584 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:49.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.094+0000 7f10f3243740 1 --2- 172.21.15.64:0/408144349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f64fdd41b0 0x55f6501af8a0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.094+0000 7f10f3243740 1 -- 172.21.15.64:0/408144349 >> 172.21.15.64:0/408144349 conn(0x55f64fdf1460 msgr2=0x55f6501a7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:49.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.094+0000 7f10f3243740 1 -- 172.21.15.64:0/408144349 shutdown_connections 2024-09-21T04:08:49.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.094+0000 7f10f3243740 1 -- 172.21.15.64:0/408144349 wait complete. 2024-09-21T04:08:49.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.094+0000 7f10f3243740 20 client.16584 trim_cache size 0 max 16384 2024-09-21T04:08:49.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.113+0000 7f10f3243740 1 Processor -- start 2024-09-21T04:08:49.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.113+0000 7f10f3243740 1 -- start start 2024-09-21T04:08:49.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.113+0000 7f10f3243740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f64fdbb9f0 0x55f6501a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:49.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.113+0000 7f10f3243740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f64fdd41b0 0x55f6501b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:49.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.113+0000 7f10f3243740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f64fddc390 0x55f6501a1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:49.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.113+0000 7f10f3243740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f10e81b4330 con 0x55f64fdbb9f0 2024-09-21T04:08:49.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.113+0000 7f10f3243740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f10e81b1bd0 con 0x55f64fddc390 2024-09-21T04:08:49.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.113+0000 7f10f3243740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f10e81afa20 con 0x55f64fdd41b0 2024-09-21T04:08:49.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.113+0000 7f10e659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f64fdbb9f0 0x55f6501a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:49.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.113+0000 7f10e659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f64fdbb9f0 0x55f6501a5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51084/0 (socket says 172.21.15.64:51084) 2024-09-21T04:08:49.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.113+0000 7f10e659c640 1 -- 172.21.15.64:0/1327750632 learned_addr learned my addr 172.21.15.64:0/1327750632 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:49.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.114+0000 7f10e6d9d640 1 --2- 172.21.15.64:0/1327750632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f64fddc390 0x55f6501a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:49.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.114+0000 7f10e5d9b640 1 --2- 172.21.15.64:0/1327750632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f64fdd41b0 0x55f6501b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:49.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.114+0000 7f10e659c640 1 -- 172.21.15.64:0/1327750632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f64fdd41b0 msgr2=0x55f6501b14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:49.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.114+0000 7f10e659c640 1 --2- 172.21.15.64:0/1327750632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f64fdd41b0 0x55f6501b14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.114+0000 7f10e659c640 1 -- 172.21.15.64:0/1327750632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f64fddc390 msgr2=0x55f6501a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:49.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.114+0000 7f10e659c640 1 --2- 172.21.15.64:0/1327750632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f64fddc390 0x55f6501a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.114+0000 7f10e659c640 1 -- 172.21.15.64:0/1327750632 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6501b0b60 con 0x55f64fdbb9f0 2024-09-21T04:08:49.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.114+0000 7f10e659c640 1 --2- 172.21.15.64:0/1327750632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f64fdbb9f0 0x55f6501a5c60 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f10d80a9010 tx=0x7f10d805a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731cfd709c139257 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:49.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.114+0000 7f10cffff640 1 -- 172.21.15.64:0/1327750632 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d804d900 con 0x55f64fdbb9f0 2024-09-21T04:08:49.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.114+0000 7f10cffff640 1 -- 172.21.15.64:0/1327750632 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10d80ad3d0 con 0x55f64fdbb9f0 2024-09-21T04:08:49.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.114+0000 7f10cffff640 1 -- 172.21.15.64:0/1327750632 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d804dad0 con 0x55f64fdbb9f0 2024-09-21T04:08:49.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.115+0000 7f10f3243740 1 -- 172.21.15.64:0/1327750632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f64fdbb9f0 msgr2=0x55f6501a5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:49.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.115+0000 7f10f3243740 1 --2- 172.21.15.64:0/1327750632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f64fdbb9f0 0x55f6501a5c60 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f10d80a9010 tx=0x7f10d805a000 comp rx=0 tx=0).stop 2024-09-21T04:08:49.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.115+0000 7f10f3243740 1 -- 172.21.15.64:0/1327750632 shutdown_connections 2024-09-21T04:08:49.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.115+0000 7f10f3243740 1 --2- 172.21.15.64:0/1327750632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f64fddc390 0x55f6501a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.115+0000 7f10f3243740 1 --2- 172.21.15.64:0/1327750632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f64fdd41b0 0x55f6501b14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.115+0000 7f10f3243740 1 --2- 172.21.15.64:0/1327750632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f64fdbb9f0 0x55f6501a5c60 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.115+0000 7f10f3243740 1 -- 172.21.15.64:0/1327750632 >> 172.21.15.64:0/1327750632 conn(0x55f64fdf1460 msgr2=0x55f6501b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:49.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.115+0000 7f10f3243740 1 -- 172.21.15.64:0/1327750632 shutdown_connections 2024-09-21T04:08:49.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.115+0000 7f10f3243740 1 -- 172.21.15.64:0/1327750632 wait complete. 2024-09-21T04:08:49.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.115+0000 7f10f3243740 1 -- start start 2024-09-21T04:08:49.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.116+0000 7f10f3243740 1 Processor -- start 2024-09-21T04:08:49.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.116+0000 7f10f3243740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f64fdbb9f0 0x55f6501b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:49.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.116+0000 7f10f3243740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f64fdd41b0 0x55f6501b4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:49.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.116+0000 7f10f3243740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f64fddc390 0x55f65022aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:49.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.116+0000 7f10f3243740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f10e81b0090 con 0x55f64fdbb9f0 2024-09-21T04:08:49.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.116+0000 7f10f3243740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f10e81b1500 con 0x55f64fddc390 2024-09-21T04:08:49.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.116+0000 7f10f3243740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f10e81b0760 con 0x55f64fdd41b0 2024-09-21T04:08:49.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.116+0000 7f10f3243740 10 client.0 _ll_register_callbacks cb 0x7f10d0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:08:49.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.116+0000 7f10e659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f64fdbb9f0 0x55f6501b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:49.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.116+0000 7f10e659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f64fdbb9f0 0x55f6501b3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:51096/0 (socket says 172.21.15.64:51096) 2024-09-21T04:08:49.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.116+0000 7f10e659c640 1 -- 172.21.15.64:0/3287723963 learned_addr learned my addr 172.21.15.64:0/3287723963 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:49.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.116+0000 7f10e5d9b640 1 --2- 172.21.15.64:0/3287723963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f64fdd41b0 0x55f6501b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:49.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.116+0000 7f10e6d9d640 1 --2- 172.21.15.64:0/3287723963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f64fddc390 0x55f65022aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:49.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.116+0000 7f10e659c640 1 -- 172.21.15.64:0/3287723963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f64fdd41b0 msgr2=0x55f6501b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:49.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.116+0000 7f10e659c640 1 --2- 172.21.15.64:0/3287723963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f64fdd41b0 0x55f6501b4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.116+0000 7f10e659c640 1 -- 172.21.15.64:0/3287723963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f64fddc390 msgr2=0x55f65022aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:49.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.116+0000 7f10e659c640 1 --2- 172.21.15.64:0/3287723963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f64fddc390 0x55f65022aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.116+0000 7f10e659c640 1 -- 172.21.15.64:0/3287723963 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f65022b3f0 con 0x55f64fdbb9f0 2024-09-21T04:08:49.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.116+0000 7f10e6d9d640 1 --2- 172.21.15.64:0/3287723963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f64fddc390 0x55f65022aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:08:49.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.117+0000 7f10e659c640 1 --2- 172.21.15.64:0/3287723963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f64fdbb9f0 0x55f6501b3ec0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f10d80ad850 tx=0x7f10d800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b0f838e02f5eed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:49.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.117+0000 7f10cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:08:49.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.117+0000 7f10cdffb640 1 -- 172.21.15.64:0/3287723963 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d80ad030 con 0x55f64fdbb9f0 2024-09-21T04:08:49.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.117+0000 7f10cdffb640 1 -- 172.21.15.64:0/3287723963 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f10d80b7060 con 0x55f64fdbb9f0 2024-09-21T04:08:49.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.117+0000 7f10f3243740 1 -- 172.21.15.64:0/3287723963 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10d80ad3d0 con 0x55f64fdbb9f0 2024-09-21T04:08:49.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.117+0000 7f10cdffb640 1 -- 172.21.15.64:0/3287723963 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f10d80ad210 con 0x55f64fdbb9f0 2024-09-21T04:08:49.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.117+0000 7f10f3243740 10 client.16590 fetch_fsmap learned FSMap version 34 2024-09-21T04:08:49.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.117+0000 7f10f3243740 1 -- 172.21.15.64:0/3287723963 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f65022b3f0 con 0x55f64fdbb9f0 2024-09-21T04:08:49.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.117+0000 7f10cdffb640 1 -- 172.21.15.64:0/3287723963 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10d80ad3d0 con 0x55f64fdbb9f0 2024-09-21T04:08:49.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.117+0000 7f10cdffb640 1 -- 172.21.15.64:0/3287723963 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f10d80acb20 con 0x55f64fdbb9f0 2024-09-21T04:08:49.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.117+0000 7f10f3243740 10 client.16590 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T04:08:49.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.117+0000 7f10f3243740 10 client.16590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:08:49.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.117+0000 7f10f3243740 20 client.16590 populate_metadata read hostname 'smithi064' 2024-09-21T04:08:49.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.117+0000 7f10f3243740 1 --2- 172.21.15.64:0/3287723963 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f64fdfd070 0x55f65022e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:49.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.117+0000 7f10f3243740 4 client.16590 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T04:08:49.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.117+0000 7f10f3243740 1 -- 172.21.15.64:0/3287723963 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f650118850 con 0x55f64fdfd070 2024-09-21T04:08:49.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.118+0000 7f10e5d9b640 1 --2- 172.21.15.64:0/3287723963 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f64fdfd070 0x55f65022e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:49.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.118+0000 7f10e5d9b640 1 --2- 172.21.15.64:0/3287723963 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f64fdfd070 0x55f65022e930 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b9d634d05507fd4 server_cookie=a00fece96ad867b9 in_seq=0 out_seq=0 2024-09-21T04:08:49.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.118+0000 7f10cdffb640 10 client.16590 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:49.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.118+0000 7f10cdffb640 1 -- 172.21.15.64:0/3287723963 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2558 (crc 0 0 0) 0x55f650118850 con 0x55f64fdfd070 2024-09-21T04:08:49.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.118+0000 7f10cdffb640 10 client.16590 handle_command_reply: tid=0 2024-09-21T04:08:49.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.118+0000 7f10f3243740 1 client.16590 shutdown 2024-09-21T04:08:49.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.118+0000 7f10f3243740 1 -- 172.21.15.64:0/3287723963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f64fdbb9f0 msgr2=0x55f6501b3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:49.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.118+0000 7f10f3243740 1 --2- 172.21.15.64:0/3287723963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f64fdbb9f0 0x55f6501b3ec0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f10d80ad850 tx=0x7f10d800b8f0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.118+0000 7f10f3243740 1 -- 172.21.15.64:0/3287723963 shutdown_connections 2024-09-21T04:08:49.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.118+0000 7f10f3243740 1 --2- 172.21.15.64:0/3287723963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f64fddc390 0x55f65022aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.118+0000 7f10f3243740 1 --2- 172.21.15.64:0/3287723963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f64fdd41b0 0x55f6501b4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.118+0000 7f10f3243740 1 --2- 172.21.15.64:0/3287723963 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f64fdfd070 0x55f65022e930 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.118+0000 7f10f3243740 1 --2- 172.21.15.64:0/3287723963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f64fdbb9f0 0x55f6501b3ec0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.118+0000 7f10cdffb640 0 client.16590 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:49.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.118+0000 7f10f3243740 1 -- 172.21.15.64:0/3287723963 >> 172.21.15.64:0/3287723963 conn(0x55f64fdf1460 msgr2=0x55f6501bb3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:49.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.119+0000 7f10f3243740 1 -- 172.21.15.64:0/3287723963 shutdown_connections 2024-09-21T04:08:49.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.119+0000 7f10f3243740 1 -- 172.21.15.64:0/3287723963 wait complete. 2024-09-21T04:08:49.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.119+0000 7f10f3243740 20 client.16590 trim_cache size 0 max 16384 2024-09-21T04:08:49.129 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:08:49.129 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T04:08:49.129 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-09-21T04:08:49.129 INFO:teuthology.orchestra.run.smithi064.stdout: "e2c2eed8-9292-4bf6-8a39-13ad27da5acb": { 2024-09-21T04:08:49.129 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:08:49.129 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "e2c2eed8-9292-4bf6-8a39-13ad27da5acb", 2024-09-21T04:08:49.129 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-09-21T04:08:49.129 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:08:49.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:08:49.130 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_stats": { 2024-09-21T04:08:49.130 INFO:teuthology.orchestra.run.smithi064.stdout: "bed40d88-2102-43a6-ac2d-b89d5a2264d0": { 2024-09-21T04:08:49.130 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:11.363340+0000", 2024-09-21T04:08:49.130 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:08:49.130 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:08:49.130 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:08:49.130 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:08:49.131 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:08:49.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:08:49.131 INFO:teuthology.orchestra.run.smithi064.stdout: "e9242f88-6599-40eb-adf1-bdb3e946a1e0": { 2024-09-21T04:08:49.131 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:05:39.744646+0000", 2024-09-21T04:08:49.131 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:08:49.131 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:08:49.131 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:08:49.131 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:08:49.131 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:08:49.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:08:49.132 INFO:teuthology.orchestra.run.smithi064.stdout: "6bde0fe5-f775-44b1-ad0c-f9e596574dc0": { 2024-09-21T04:08:49.132 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:06:23.835536+0000", 2024-09-21T04:08:49.132 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:08:49.132 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:08:49.132 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:08:49.132 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:08:49.132 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:08:49.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:08:49.133 INFO:teuthology.orchestra.run.smithi064.stdout: "b52b950c-0f97-49ec-a37c-1a8e04348ce8": { 2024-09-21T04:08:49.133 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:54.003800+0000", 2024-09-21T04:08:49.133 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:08:49.133 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:08:49.133 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:08:49.133 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:08:49.133 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:08:49.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:08:49.134 INFO:teuthology.orchestra.run.smithi064.stdout: "e2c2eed8-9292-4bf6-8a39-13ad27da5acb": { 2024-09-21T04:08:49.134 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:08:47.067932+0000", 2024-09-21T04:08:49.134 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:08:49.134 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:08:49.134 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:08:49.134 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:08:49.134 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:08:49.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:08:49.134 INFO:teuthology.orchestra.run.smithi064.stdout: "75db1bc5-612e-45aa-bad2-cc90feeaa840": { 2024-09-21T04:08:49.135 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:59.123138+0000", 2024-09-21T04:08:49.135 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:08:49.135 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:08:49.135 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:08:49.135 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:08:49.135 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:08:49.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:08:49.135 INFO:teuthology.orchestra.run.smithi064.stdout: "0ab717a7-36c6-4dca-94e7-af0bdf2aa112": { 2024-09-21T04:08:49.136 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:42.691086+0000", 2024-09-21T04:08:49.136 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:08:49.136 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:08:49.136 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:08:49.136 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:08:49.136 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:08:49.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:08:49.136 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:08:49.136 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:03:59.570174+0000", 2024-09-21T04:08:49.137 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:08:49.137 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:08:49.137 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:08:49.137 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:08:49.137 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:08:49.137 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:08:49.137 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:08:49.137 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:08:49.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.119+0000 7f10ecccf640 1 -- 172.21.15.64:0/1459085068 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f10d40b6680 msgr2=0x7f10d40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:49.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.119+0000 7f10ecccf640 1 --2- 172.21.15.64:0/1459085068 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f10d40b6680 0x7f10d40b8b90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f10e00125f0 tx=0x7f10e0005f90 comp rx=0 tx=0).stop 2024-09-21T04:08:49.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.119+0000 7f10ecccf640 1 -- 172.21.15.64:0/1459085068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10e8102a60 msgr2=0x7f10e80ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:49.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.119+0000 7f10ecccf640 1 --2- 172.21.15.64:0/1459085068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10e8102a60 0x7f10e80ff670 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f10d800dab0 tx=0x7f10d800df70 comp rx=0 tx=0).stop 2024-09-21T04:08:49.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.119+0000 7f10ecccf640 1 -- 172.21.15.64:0/1459085068 shutdown_connections 2024-09-21T04:08:49.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.119+0000 7f10ecccf640 1 --2- 172.21.15.64:0/1459085068 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f10d40b6680 0x7f10d40b8b90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.119+0000 7f10ecccf640 1 --2- 172.21.15.64:0/1459085068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10e8108cf0 0x7f10e81000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.119+0000 7f10ecccf640 1 --2- 172.21.15.64:0/1459085068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10e81083a0 0x7f10e80ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.119+0000 7f10ecccf640 1 --2- 172.21.15.64:0/1459085068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10e8102a60 0x7f10e80ff670 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:49.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.119+0000 7f10ecccf640 1 -- 172.21.15.64:0/1459085068 >> 172.21.15.64:0/1459085068 conn(0x7f10e8079730 msgr2=0x7f10e806b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:49.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.119+0000 7f10ecccf640 1 -- 172.21.15.64:0/1459085068 shutdown_connections 2024-09-21T04:08:49.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:08:49.119+0000 7f10ecccf640 1 -- 172.21.15.64:0/1459085068 wait complete. 2024-09-21T04:08:49.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:48 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:49.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:48 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:50.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:49 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:08:50.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:49 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:50.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:49 smithi119 ceph-mon[34811]: pgmap v465: 97 pgs: 97 active+clean; 626 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 370 op/s 2024-09-21T04:08:50.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:49 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:08:50.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:49 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:08:50.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:49 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:50.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:49 smithi064 ceph-mon[31180]: pgmap v465: 97 pgs: 97 active+clean; 626 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 370 op/s 2024-09-21T04:08:50.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:49 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:08:50.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:49 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:08:50.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:49 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:50.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:49 smithi131 ceph-mon[35193]: pgmap v465: 97 pgs: 97 active+clean; 626 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 370 op/s 2024-09-21T04:08:50.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:49 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:08:50.369 INFO:tasks.cephfs.filesystem:scrub status for tag:e2c2eed8-9292-4bf6-8a39-13ad27da5acb - {'path': '/', 'tag': 'e2c2eed8-9292-4bf6-8a39-13ad27da5acb', 'options': 'recursive,force'} 2024-09-21T04:08:51.174 INFO:tasks.workunit.client.0.smithi064.stdout: 99 1269934 189 895640 356 717655 939 2024-09-21T04:08:51.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:50 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:08:51.258 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:50 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:51.258 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:50 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:51.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:50 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:08:51.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:50 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:51.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:50 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:51.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:50 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:08:51.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:50 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:51.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:50 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:08:52.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:51 smithi119 ceph-mon[34811]: pgmap v466: 97 pgs: 97 active+clean; 625 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 363 op/s 2024-09-21T04:08:52.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:51 smithi064 ceph-mon[31180]: pgmap v466: 97 pgs: 97 active+clean; 625 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 363 op/s 2024-09-21T04:08:52.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:51 smithi131 ceph-mon[35193]: pgmap v466: 97 pgs: 97 active+clean; 625 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 363 op/s 2024-09-21T04:08:54.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:53 smithi119 ceph-mon[34811]: pgmap v467: 97 pgs: 97 active+clean; 624 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.8 MiB/s wr, 705 op/s 2024-09-21T04:08:54.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:53 smithi064 ceph-mon[31180]: pgmap v467: 97 pgs: 97 active+clean; 624 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.8 MiB/s wr, 705 op/s 2024-09-21T04:08:54.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:53 smithi131 ceph-mon[35193]: pgmap v467: 97 pgs: 97 active+clean; 624 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.8 MiB/s wr, 705 op/s 2024-09-21T04:08:56.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:55 smithi119 ceph-mon[34811]: pgmap v468: 97 pgs: 97 active+clean; 624 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 592 op/s 2024-09-21T04:08:56.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:55 smithi064 ceph-mon[31180]: pgmap v468: 97 pgs: 97 active+clean; 624 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 592 op/s 2024-09-21T04:08:56.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:55 smithi131 ceph-mon[35193]: pgmap v468: 97 pgs: 97 active+clean; 624 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 592 op/s 2024-09-21T04:08:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.242+0000 7f949df62640 1 -- 172.21.15.64:0/3165008744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94981029b0 msgr2=0x7f9498102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.242+0000 7f949df62640 1 --2- 172.21.15.64:0/3165008744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94981029b0 0x7f9498102dc0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9488009ca0 tx=0x7f948802a8c0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.243+0000 7f949df62640 1 -- 172.21.15.64:0/3165008744 shutdown_connections 2024-09-21T04:08:58.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.243+0000 7f949df62640 1 --2- 172.21.15.64:0/3165008744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9498108c30 0x7f949810f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.243+0000 7f949df62640 1 --2- 172.21.15.64:0/3165008744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94981082e0 0x7f94981086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.243+0000 7f949df62640 1 --2- 172.21.15.64:0/3165008744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94981029b0 0x7f9498102dc0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.243+0000 7f949df62640 1 -- 172.21.15.64:0/3165008744 >> 172.21.15.64:0/3165008744 conn(0x7f949807a740 msgr2=0x7f949807ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:58.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.243+0000 7f949df62640 1 -- 172.21.15.64:0/3165008744 shutdown_connections 2024-09-21T04:08:58.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.243+0000 7f949df62640 1 -- 172.21.15.64:0/3165008744 wait complete. 2024-09-21T04:08:58.247 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.243+0000 7f949df62640 1 Processor -- start 2024-09-21T04:08:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.243+0000 7f949df62640 1 -- start start 2024-09-21T04:08:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.244+0000 7f949df62640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94981029b0 0x7f949819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.244+0000 7f949df62640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94981082e0 0x7f949819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.244+0000 7f949df62640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9498108c30 0x7f949819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.244+0000 7f949df62640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9498113a10 con 0x7f94981082e0 2024-09-21T04:08:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.244+0000 7f949df62640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9498113890 con 0x7f94981029b0 2024-09-21T04:08:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.244+0000 7f949df62640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9498113b90 con 0x7f9498108c30 2024-09-21T04:08:58.249 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.244+0000 7f9496ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94981082e0 0x7f949819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.244+0000 7f94977fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94981029b0 0x7f949819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.244+0000 7f9496ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94981082e0 0x7f949819a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60424/0 (socket says 172.21.15.64:60424) 2024-09-21T04:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.244+0000 7f9496ffd640 1 -- 172.21.15.64:0/3683739474 learned_addr learned my addr 172.21.15.64:0/3683739474 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.244+0000 7f9497fff640 1 --2- 172.21.15.64:0/3683739474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9498108c30 0x7f949819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.244+0000 7f9496ffd640 1 -- 172.21.15.64:0/3683739474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9498108c30 msgr2=0x7f949819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.244+0000 7f9496ffd640 1 --2- 172.21.15.64:0/3683739474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9498108c30 0x7f949819fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.244+0000 7f9496ffd640 1 -- 172.21.15.64:0/3683739474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94981029b0 msgr2=0x7f949819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.244+0000 7f9496ffd640 1 --2- 172.21.15.64:0/3683739474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94981029b0 0x7f949819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.244+0000 7f9496ffd640 1 -- 172.21.15.64:0/3683739474 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94981a0360 con 0x7f94981082e0 2024-09-21T04:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.244+0000 7f94977fe640 1 --2- 172.21.15.64:0/3683739474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94981029b0 0x7f949819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.244+0000 7f9496ffd640 1 --2- 172.21.15.64:0/3683739474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94981082e0 0x7f949819a9d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f948000d750 tx=0x7f948000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.245+0000 7f9494ff9640 1 -- 172.21.15.64:0/3683739474 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480017070 con 0x7f94981082e0 2024-09-21T04:08:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.245+0000 7f9494ff9640 1 -- 172.21.15.64:0/3683739474 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94800054d0 con 0x7f94981082e0 2024-09-21T04:08:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.245+0000 7f9494ff9640 1 -- 172.21.15.64:0/3683739474 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480004c30 con 0x7f94981082e0 2024-09-21T04:08:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.245+0000 7f949df62640 1 -- 172.21.15.64:0/3683739474 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94981a0650 con 0x7f94981082e0 2024-09-21T04:08:58.251 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.245+0000 7f949df62640 1 -- 172.21.15.64:0/3683739474 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94981a6de0 con 0x7f94981082e0 2024-09-21T04:08:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.247+0000 7f9494ff9640 1 -- 172.21.15.64:0/3683739474 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f9480004dd0 con 0x7f94981082e0 2024-09-21T04:08:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.247+0000 7f94a4ed0740 1 Processor -- start 2024-09-21T04:08:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.247+0000 7f94a4ed0740 1 -- start start 2024-09-21T04:08:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.248+0000 7f94a4ed0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ab1c971b0 0x557ab203c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.248+0000 7f94a4ed0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c6e2f0 0x557ab203cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.248+0000 7f94a4ed0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ab1c764d0 0x557ab203d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.248+0000 7f94a4ed0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557ab1fb16f0 con 0x557ab1c6e2f0 2024-09-21T04:08:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.248+0000 7f94a4ed0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557ab1fa0b20 con 0x557ab1c764d0 2024-09-21T04:08:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.248+0000 7f94a4ed0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557ab1fa4930 con 0x557ab1c971b0 2024-09-21T04:08:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.248+0000 7f94977fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c6e2f0 0x557ab203cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.248+0000 7f9496ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ab1c764d0 0x557ab203d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.248+0000 7f94977fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c6e2f0 0x557ab203cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60430/0 (socket says 172.21.15.64:60430) 2024-09-21T04:08:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.248+0000 7f9497fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ab1c971b0 0x557ab203c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.248+0000 7f94977fe640 1 -- 172.21.15.64:0/3820767551 learned_addr learned my addr 172.21.15.64:0/3820767551 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.248+0000 7f9496ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ab1c764d0 0x557ab203d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:44430/0 (socket says 172.21.15.64:44430) 2024-09-21T04:08:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.248+0000 7f94977fe640 1 -- 172.21.15.64:0/3820767551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ab1c971b0 msgr2=0x557ab203c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.248+0000 7f94977fe640 1 --2- 172.21.15.64:0/3820767551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ab1c971b0 0x557ab203c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.248+0000 7f94977fe640 1 -- 172.21.15.64:0/3820767551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ab1c764d0 msgr2=0x557ab203d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.248+0000 7f94977fe640 1 --2- 172.21.15.64:0/3820767551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ab1c764d0 0x557ab203d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.248+0000 7f9494ff9640 1 --2- 172.21.15.64:0/3683739474 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f94840b6700 0x7f94840b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.248+0000 7f94977fe640 1 -- 172.21.15.64:0/3820767551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ab1f66530 con 0x557ab1c6e2f0 2024-09-21T04:08:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.248+0000 7f9496ffd640 1 --2- 172.21.15.64:0/3820767551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ab1c764d0 0x557ab203d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.248+0000 7f9497fff640 1 --2- 172.21.15.64:0/3683739474 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f94840b6700 0x7f94840b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.249+0000 7f9494ff9640 1 -- 172.21.15.64:0/3683739474 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f94800e7020 con 0x7f94981082e0 2024-09-21T04:08:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.249+0000 7f94977fe640 1 --2- 172.21.15.64:0/3820767551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c6e2f0 0x557ab203cef0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f9488035000 tx=0x7f94880094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb17ff8fdcb95cba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.249+0000 7f948dffb640 1 -- 172.21.15.64:0/3820767551 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9488031650 con 0x557ab1c6e2f0 2024-09-21T04:08:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.249+0000 7f948dffb640 1 -- 172.21.15.64:0/3820767551 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9488036040 con 0x557ab1c6e2f0 2024-09-21T04:08:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.249+0000 7f948dffb640 1 -- 172.21.15.64:0/3820767551 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9488004540 con 0x557ab1c6e2f0 2024-09-21T04:08:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.250+0000 7f94a4ed0740 1 -- 172.21.15.64:0/3820767551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c6e2f0 msgr2=0x557ab203cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.250+0000 7f94a4ed0740 1 --2- 172.21.15.64:0/3820767551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c6e2f0 0x557ab203cef0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f9488035000 tx=0x7f94880094a0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.250+0000 7f9497fff640 1 --2- 172.21.15.64:0/3683739474 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f94840b6700 0x7f94840b8c10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9490009da0 tx=0x7f9490005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.250+0000 7f94a4ed0740 1 -- 172.21.15.64:0/3820767551 shutdown_connections 2024-09-21T04:08:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.250+0000 7f94a4ed0740 1 --2- 172.21.15.64:0/3820767551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ab1c764d0 0x557ab203d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.250+0000 7f94a4ed0740 1 --2- 172.21.15.64:0/3820767551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c6e2f0 0x557ab203cef0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.250+0000 7f94a4ed0740 1 --2- 172.21.15.64:0/3820767551 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ab1c971b0 0x557ab203c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.250+0000 7f94a4ed0740 1 -- 172.21.15.64:0/3820767551 >> 172.21.15.64:0/3820767551 conn(0x557ab1c8b300 msgr2=0x557ab203a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.250+0000 7f94a4ed0740 1 -- 172.21.15.64:0/3820767551 shutdown_connections 2024-09-21T04:08:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.250+0000 7f94a4ed0740 1 -- 172.21.15.64:0/3820767551 wait complete. 2024-09-21T04:08:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.250+0000 7f94a4ed0740 1 -- start start 2024-09-21T04:08:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.251+0000 7f94a4ed0740 1 Processor -- start 2024-09-21T04:08:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.251+0000 7f94a4ed0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ab1c6e2f0 0x557ab204c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.251+0000 7f94a4ed0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ab1c764d0 0x557ab204c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.251+0000 7f94a4ed0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c971b0 0x557ab2058660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.251+0000 7f94a4ed0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557ab1ed8ab0 con 0x557ab1c971b0 2024-09-21T04:08:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.251+0000 7f94a4ed0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557ab1f46730 con 0x557ab1c764d0 2024-09-21T04:08:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.251+0000 7f94a4ed0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557ab204f330 con 0x557ab1c6e2f0 2024-09-21T04:08:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.251+0000 7f94a4ed0740 10 client.0 _ll_register_callbacks cb 0x557ab1f9a6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:08:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.251+0000 7f9497fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c971b0 0x557ab2058660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.251+0000 7f9496ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ab1c764d0 0x557ab204c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.251+0000 7f9497fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c971b0 0x557ab2058660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60440/0 (socket says 172.21.15.64:60440) 2024-09-21T04:08:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.251+0000 7f9497fff640 1 -- 172.21.15.64:0/1776416819 learned_addr learned my addr 172.21.15.64:0/1776416819 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.251+0000 7f94977fe640 1 --2- 172.21.15.64:0/1776416819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ab1c6e2f0 0x557ab204c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.251+0000 7f9496ffd640 1 -- 172.21.15.64:0/1776416819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ab1c6e2f0 msgr2=0x557ab204c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.251+0000 7f9496ffd640 1 --2- 172.21.15.64:0/1776416819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ab1c6e2f0 0x557ab204c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.251+0000 7f9496ffd640 1 -- 172.21.15.64:0/1776416819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c971b0 msgr2=0x557ab2058660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.251+0000 7f9496ffd640 1 --2- 172.21.15.64:0/1776416819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c971b0 0x557ab2058660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.251+0000 7f9496ffd640 1 -- 172.21.15.64:0/1776416819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ab1d02790 con 0x557ab1c764d0 2024-09-21T04:08:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.251+0000 7f9497fff640 1 --2- 172.21.15.64:0/1776416819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c971b0 0x557ab2058660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.252+0000 7f9496ffd640 1 --2- 172.21.15.64:0/1776416819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ab1c764d0 0x557ab204c940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f94800e4740 tx=0x7f948000de60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a61591b28bcd2ca6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.252+0000 7f9476ffd640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:08:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.252+0000 7f9476ffd640 1 -- 172.21.15.64:0/1776416819 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480020070 con 0x557ab1c764d0 2024-09-21T04:08:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.252+0000 7f94a4ed0740 1 -- 172.21.15.64:0/1776416819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9488036040 con 0x557ab1c764d0 2024-09-21T04:08:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.252+0000 7f9476ffd640 1 -- 172.21.15.64:0/1776416819 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f94800a9160 con 0x557ab1c764d0 2024-09-21T04:08:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.252+0000 7f9476ffd640 1 -- 172.21.15.64:0/1776416819 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f94800af050 con 0x557ab1c764d0 2024-09-21T04:08:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.252+0000 7f94a4ed0740 10 client.26303 fetch_fsmap learned FSMap version 34 2024-09-21T04:08:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.252+0000 7f94a4ed0740 1 -- 172.21.15.64:0/1776416819 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557ab1d7b620 con 0x557ab1c764d0 2024-09-21T04:08:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.252+0000 7f9476ffd640 1 -- 172.21.15.64:0/1776416819 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9488036040 con 0x557ab1c764d0 2024-09-21T04:08:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.252+0000 7f9476ffd640 1 -- 172.21.15.64:0/1776416819 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f94800940f0 con 0x557ab1c764d0 2024-09-21T04:08:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.252+0000 7f94a4ed0740 10 client.26303 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T04:08:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.252+0000 7f94a4ed0740 10 client.26303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:08:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.252+0000 7f94a4ed0740 20 client.26303 populate_metadata read hostname 'smithi064' 2024-09-21T04:08:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.252+0000 7f94a4ed0740 1 --2- 172.21.15.64:0/1776416819 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x557ab1c55d80 0x557ab205e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.252+0000 7f94a4ed0740 4 client.26303 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:08:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.252+0000 7f94a4ed0740 1 -- 172.21.15.64:0/1776416819 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ab1fb3640 con 0x557ab1c55d80 2024-09-21T04:08:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.253+0000 7f94977fe640 1 --2- 172.21.15.64:0/1776416819 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x557ab1c55d80 0x557ab205e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.253+0000 7f9476ffd640 10 client.26303 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.253+0000 7f94977fe640 1 --2- 172.21.15.64:0/1776416819 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x557ab1c55d80 0x557ab205e3c0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77398d1569d68b41 server_cookie=d175cca3d3c7f78 in_seq=0 out_seq=0 2024-09-21T04:08:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.254+0000 7f9476ffd640 1 -- 172.21.15.64:0/1776416819 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557ab1fb3640 con 0x557ab1c55d80 2024-09-21T04:08:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.254+0000 7f9476ffd640 10 client.26303 handle_command_reply: tid=0 2024-09-21T04:08:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.254+0000 7f94a4ed0740 1 client.26303 shutdown 2024-09-21T04:08:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.254+0000 7f94a4ed0740 1 -- 172.21.15.64:0/1776416819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ab1c764d0 msgr2=0x557ab204c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.254+0000 7f94a4ed0740 1 --2- 172.21.15.64:0/1776416819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ab1c764d0 0x557ab204c940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f94800e4740 tx=0x7f948000de60 comp rx=0 tx=0).stop 2024-09-21T04:08:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.254+0000 7f94a4ed0740 1 -- 172.21.15.64:0/1776416819 shutdown_connections 2024-09-21T04:08:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.254+0000 7f94a4ed0740 1 --2- 172.21.15.64:0/1776416819 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x557ab1c55d80 0x557ab205e3c0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.254+0000 7f94a4ed0740 1 --2- 172.21.15.64:0/1776416819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c971b0 0x557ab2058660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.254+0000 7f94a4ed0740 1 --2- 172.21.15.64:0/1776416819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ab1c764d0 0x557ab204c940 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.254+0000 7f94a4ed0740 1 --2- 172.21.15.64:0/1776416819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ab1c6e2f0 0x557ab204c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.254+0000 7f9476ffd640 0 client.26303 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.254+0000 7f94a4ed0740 1 -- 172.21.15.64:0/1776416819 >> 172.21.15.64:0/1776416819 conn(0x557ab1c8b300 msgr2=0x557ab203e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.254+0000 7f94a4ed0740 1 -- 172.21.15.64:0/1776416819 shutdown_connections 2024-09-21T04:08:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.254+0000 7f94a4ed0740 1 -- 172.21.15.64:0/1776416819 wait complete. 2024-09-21T04:08:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.254+0000 7f94a4ed0740 20 client.26303 trim_cache size 0 max 16384 2024-09-21T04:08:58.264 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:57 smithi119 ceph-mon[34811]: pgmap v469: 97 pgs: 97 active+clean; 627 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.5 MiB/s wr, 758 op/s 2024-09-21T04:08:58.264 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:57 smithi064 ceph-mon[31180]: pgmap v469: 97 pgs: 97 active+clean; 627 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.5 MiB/s wr, 758 op/s 2024-09-21T04:08:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.270+0000 7f94a4ed0740 1 Processor -- start 2024-09-21T04:08:58.274 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.270+0000 7f94a4ed0740 1 -- start start 2024-09-21T04:08:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.270+0000 7f94a4ed0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ab1c6e2f0 0x557ab2058560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.270+0000 7f94a4ed0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c764d0 0x557ab20495d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.270+0000 7f94a4ed0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ab1c971b0 0x557ab2049b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.270+0000 7f94a4ed0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f94981afe30 con 0x557ab1c764d0 2024-09-21T04:08:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.270+0000 7f94a4ed0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f94981ad6d0 con 0x557ab1c6e2f0 2024-09-21T04:08:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.270+0000 7f94a4ed0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f94981ab520 con 0x557ab1c971b0 2024-09-21T04:08:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.270+0000 7f9496ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c764d0 0x557ab20495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.270+0000 7f94977fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ab1c6e2f0 0x557ab2058560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.270+0000 7f94977fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ab1c6e2f0 0x557ab2058560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:44442/0 (socket says 172.21.15.64:44442) 2024-09-21T04:08:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.270+0000 7f9496ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c764d0 0x557ab20495d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60456/0 (socket says 172.21.15.64:60456) 2024-09-21T04:08:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.270+0000 7f94977fe640 1 -- 172.21.15.64:0/108022381 learned_addr learned my addr 172.21.15.64:0/108022381 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.271+0000 7f9497fff640 1 --2- 172.21.15.64:0/108022381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ab1c971b0 0x557ab2049b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.271+0000 7f9496ffd640 1 -- 172.21.15.64:0/108022381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ab1c971b0 msgr2=0x557ab2049b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.271+0000 7f9496ffd640 1 --2- 172.21.15.64:0/108022381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ab1c971b0 0x557ab2049b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.271+0000 7f9496ffd640 1 -- 172.21.15.64:0/108022381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ab1c6e2f0 msgr2=0x557ab2058560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.271+0000 7f9496ffd640 1 --2- 172.21.15.64:0/108022381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ab1c6e2f0 0x557ab2058560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.271+0000 7f9496ffd640 1 -- 172.21.15.64:0/108022381 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ab203d7f0 con 0x557ab1c764d0 2024-09-21T04:08:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.271+0000 7f94977fe640 1 --2- 172.21.15.64:0/108022381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ab1c6e2f0 0x557ab2058560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:08:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.271+0000 7f9496ffd640 1 --2- 172.21.15.64:0/108022381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c764d0 0x557ab20495d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f94800acbf0 tx=0x7f94800b1920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23862b076ccdacc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.271+0000 7f948d7fa640 1 -- 172.21.15.64:0/108022381 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480020070 con 0x557ab1c764d0 2024-09-21T04:08:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.271+0000 7f948d7fa640 1 -- 172.21.15.64:0/108022381 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94800cee50 con 0x557ab1c764d0 2024-09-21T04:08:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.271+0000 7f948d7fa640 1 -- 172.21.15.64:0/108022381 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94800ea6f0 con 0x557ab1c764d0 2024-09-21T04:08:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.272+0000 7f94a4ed0740 1 -- 172.21.15.64:0/108022381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c764d0 msgr2=0x557ab20495d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.272+0000 7f94a4ed0740 1 --2- 172.21.15.64:0/108022381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c764d0 0x557ab20495d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f94800acbf0 tx=0x7f94800b1920 comp rx=0 tx=0).stop 2024-09-21T04:08:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.272+0000 7f94a4ed0740 1 -- 172.21.15.64:0/108022381 shutdown_connections 2024-09-21T04:08:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.272+0000 7f94a4ed0740 1 --2- 172.21.15.64:0/108022381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ab1c971b0 0x557ab2049b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.272+0000 7f94a4ed0740 1 --2- 172.21.15.64:0/108022381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c764d0 0x557ab20495d0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.272+0000 7f94a4ed0740 1 --2- 172.21.15.64:0/108022381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ab1c6e2f0 0x557ab2058560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.272+0000 7f94a4ed0740 1 -- 172.21.15.64:0/108022381 >> 172.21.15.64:0/108022381 conn(0x557ab1c8b300 msgr2=0x557ab204ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.272+0000 7f94a4ed0740 1 -- 172.21.15.64:0/108022381 shutdown_connections 2024-09-21T04:08:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.272+0000 7f94a4ed0740 1 -- 172.21.15.64:0/108022381 wait complete. 2024-09-21T04:08:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.272+0000 7f94a4ed0740 1 -- start start 2024-09-21T04:08:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.272+0000 7f94a4ed0740 1 Processor -- start 2024-09-21T04:08:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.273+0000 7f94a4ed0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ab1c6e2f0 0x557ab2050630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.273+0000 7f94a4ed0740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c764d0 0x557ab2050b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.273+0000 7f94a4ed0740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ab1c971b0 0x557ab20510b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.273+0000 7f94a4ed0740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f94981abb90 con 0x557ab1c764d0 2024-09-21T04:08:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.273+0000 7f94a4ed0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f94981ad000 con 0x557ab1c6e2f0 2024-09-21T04:08:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.273+0000 7f94a4ed0740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f94981ac260 con 0x557ab1c971b0 2024-09-21T04:08:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.273+0000 7f94a4ed0740 10 client.0 _ll_register_callbacks cb 0x7f948000d6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:08:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.273+0000 7f9496ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c764d0 0x557ab2050b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.273+0000 7f9496ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c764d0 0x557ab2050b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60470/0 (socket says 172.21.15.64:60470) 2024-09-21T04:08:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.273+0000 7f94977fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ab1c6e2f0 0x557ab2050630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.273+0000 7f9496ffd640 1 -- 172.21.15.64:0/3052229521 learned_addr learned my addr 172.21.15.64:0/3052229521 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:08:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.273+0000 7f9497fff640 1 --2- 172.21.15.64:0/3052229521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ab1c971b0 0x557ab20510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.273+0000 7f9496ffd640 1 -- 172.21.15.64:0/3052229521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ab1c971b0 msgr2=0x557ab20510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.273+0000 7f9496ffd640 1 --2- 172.21.15.64:0/3052229521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ab1c971b0 0x557ab20510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.273+0000 7f9496ffd640 1 -- 172.21.15.64:0/3052229521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ab1c6e2f0 msgr2=0x557ab2050630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:08:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.273+0000 7f9496ffd640 1 --2- 172.21.15.64:0/3052229521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ab1c6e2f0 0x557ab2050630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.273+0000 7f9496ffd640 1 -- 172.21.15.64:0/3052229521 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ab20c7dd0 con 0x557ab1c764d0 2024-09-21T04:08:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.273+0000 7f9497fff640 1 --2- 172.21.15.64:0/3052229521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ab1c971b0 0x557ab20510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:08:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.273+0000 7f9496ffd640 1 --2- 172.21.15.64:0/3052229521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c764d0 0x557ab2050b70 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f94800aa160 tx=0x7f9480009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab6daea6ae166ca9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:08:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.273+0000 7f9476ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:08:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.274+0000 7f9476ffd640 1 -- 172.21.15.64:0/3052229521 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480098070 con 0x557ab1c764d0 2024-09-21T04:08:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.274+0000 7f9476ffd640 1 -- 172.21.15.64:0/3052229521 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f94800ea6f0 con 0x557ab1c764d0 2024-09-21T04:08:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.274+0000 7f94a4ed0740 1 -- 172.21.15.64:0/3052229521 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94800cee50 con 0x557ab1c764d0 2024-09-21T04:08:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.274+0000 7f9476ffd640 1 -- 172.21.15.64:0/3052229521 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9480020050 con 0x557ab1c764d0 2024-09-21T04:08:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.274+0000 7f94a4ed0740 10 client.16608 fetch_fsmap learned FSMap version 34 2024-09-21T04:08:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.274+0000 7f94a4ed0740 1 -- 172.21.15.64:0/3052229521 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557ab20c9060 con 0x557ab1c764d0 2024-09-21T04:08:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.274+0000 7f9476ffd640 1 -- 172.21.15.64:0/3052229521 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94800cee50 con 0x557ab1c764d0 2024-09-21T04:08:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.274+0000 7f9476ffd640 1 -- 172.21.15.64:0/3052229521 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f94800d4170 con 0x557ab1c764d0 2024-09-21T04:08:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.274+0000 7f94a4ed0740 10 client.16608 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T04:08:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.274+0000 7f94a4ed0740 10 client.16608 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:08:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.274+0000 7f94a4ed0740 20 client.16608 populate_metadata read hostname 'smithi064' 2024-09-21T04:08:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.274+0000 7f94a4ed0740 1 --2- 172.21.15.64:0/3052229521 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x557ab1c2b1c0 0x557ab20cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:08:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.274+0000 7f94a4ed0740 4 client.16608 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:08:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.274+0000 7f94a4ed0740 1 -- 172.21.15.64:0/3052229521 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557ab1faf0f0 con 0x557ab1c2b1c0 2024-09-21T04:08:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.274+0000 7f94977fe640 1 --2- 172.21.15.64:0/3052229521 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x557ab1c2b1c0 0x557ab20cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:08:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.275+0000 7f94977fe640 1 --2- 172.21.15.64:0/3052229521 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x557ab1c2b1c0 0x557ab20cb4c0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3bc8355abb620e4 server_cookie=59cb844f998a91d5 in_seq=0 out_seq=0 2024-09-21T04:08:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.275+0000 7f9476ffd640 10 client.16608 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.299+0000 7f9476ffd640 1 -- 172.21.15.64:0/3052229521 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557ab1faf0f0 con 0x557ab1c2b1c0 2024-09-21T04:08:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.299+0000 7f9476ffd640 10 client.16608 handle_command_reply: tid=0 2024-09-21T04:08:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.299+0000 7f94a4ed0740 1 client.16608 shutdown 2024-09-21T04:08:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.300+0000 7f94a4ed0740 1 -- 172.21.15.64:0/3052229521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c764d0 msgr2=0x557ab2050b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.300+0000 7f94a4ed0740 1 --2- 172.21.15.64:0/3052229521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c764d0 0x557ab2050b70 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f94800aa160 tx=0x7f9480009740 comp rx=0 tx=0).stop 2024-09-21T04:08:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.300+0000 7f94a4ed0740 1 -- 172.21.15.64:0/3052229521 shutdown_connections 2024-09-21T04:08:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.300+0000 7f94a4ed0740 1 --2- 172.21.15.64:0/3052229521 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557ab1c971b0 0x557ab20510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.300+0000 7f94a4ed0740 1 --2- 172.21.15.64:0/3052229521 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x557ab1c2b1c0 0x557ab20cb4c0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.300+0000 7f94a4ed0740 1 --2- 172.21.15.64:0/3052229521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557ab1c764d0 0x557ab2050b70 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.300+0000 7f9476ffd640 0 client.16608 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:08:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.300+0000 7f94a4ed0740 1 --2- 172.21.15.64:0/3052229521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557ab1c6e2f0 0x557ab2050630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.300+0000 7f94a4ed0740 1 -- 172.21.15.64:0/3052229521 >> 172.21.15.64:0/3052229521 conn(0x557ab1c8b300 msgr2=0x557ab20470f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.300+0000 7f94a4ed0740 1 -- 172.21.15.64:0/3052229521 shutdown_connections 2024-09-21T04:08:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.300+0000 7f94a4ed0740 1 -- 172.21.15.64:0/3052229521 wait complete. 2024-09-21T04:08:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.300+0000 7f94a4ed0740 20 client.16608 trim_cache size 0 max 16384 2024-09-21T04:08:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:08:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:08:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:08:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:08:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.300+0000 7f949df62640 1 -- 172.21.15.64:0/3683739474 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f94840b6700 msgr2=0x7f94840b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.300+0000 7f949df62640 1 --2- 172.21.15.64:0/3683739474 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f94840b6700 0x7f94840b8c10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9490009da0 tx=0x7f9490005f90 comp rx=0 tx=0).stop 2024-09-21T04:08:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.300+0000 7f949df62640 1 -- 172.21.15.64:0/3683739474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94981082e0 msgr2=0x7f949819a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:08:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.301+0000 7f949df62640 1 --2- 172.21.15.64:0/3683739474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94981082e0 0x7f949819a9d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f948000d750 tx=0x7f948000dc10 comp rx=0 tx=0).stop 2024-09-21T04:08:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.301+0000 7f949df62640 1 -- 172.21.15.64:0/3683739474 shutdown_connections 2024-09-21T04:08:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.301+0000 7f949df62640 1 --2- 172.21.15.64:0/3683739474 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f94840b6700 0x7f94840b8c10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.301+0000 7f949df62640 1 --2- 172.21.15.64:0/3683739474 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9498108c30 0x7f949819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.301+0000 7f949df62640 1 --2- 172.21.15.64:0/3683739474 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94981082e0 0x7f949819a9d0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.301+0000 7f949df62640 1 --2- 172.21.15.64:0/3683739474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94981029b0 0x7f949819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:08:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.301+0000 7f949df62640 1 -- 172.21.15.64:0/3683739474 >> 172.21.15.64:0/3683739474 conn(0x7f949807a740 msgr2=0x7f9498106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:08:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.301+0000 7f949df62640 1 -- 172.21.15.64:0/3683739474 shutdown_connections 2024-09-21T04:08:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:08:58.301+0000 7f949df62640 1 -- 172.21.15.64:0/3683739474 wait complete. 2024-09-21T04:08:58.307 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:57 smithi131 ceph-mon[35193]: pgmap v469: 97 pgs: 97 active+clean; 627 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.5 MiB/s wr, 758 op/s 2024-09-21T04:09:00.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:08:59 smithi119 ceph-mon[34811]: pgmap v470: 97 pgs: 97 active+clean; 630 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 730 op/s 2024-09-21T04:09:00.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:08:59 smithi064 ceph-mon[31180]: pgmap v470: 97 pgs: 97 active+clean; 630 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 730 op/s 2024-09-21T04:09:00.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:08:59 smithi131 ceph-mon[35193]: pgmap v470: 97 pgs: 97 active+clean; 630 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 730 op/s 2024-09-21T04:09:01.174 INFO:tasks.workunit.client.0.smithi064.stdout: 99 22432 0 14895 4 13204 4 2024-09-21T04:09:01.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:00 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:01.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:00 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:01.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:00 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:02.163 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:01 smithi064 ceph-mon[31180]: pgmap v471: 97 pgs: 97 active+clean; 630 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 638 op/s 2024-09-21T04:09:02.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:01 smithi119 ceph-mon[34811]: pgmap v471: 97 pgs: 97 active+clean; 630 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 638 op/s 2024-09-21T04:09:02.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:01 smithi131 ceph-mon[35193]: pgmap v471: 97 pgs: 97 active+clean; 630 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 638 op/s 2024-09-21T04:09:04.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:03 smithi119 ceph-mon[34811]: pgmap v472: 97 pgs: 97 active+clean; 635 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 857 op/s 2024-09-21T04:09:04.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:03 smithi064 ceph-mon[31180]: pgmap v472: 97 pgs: 97 active+clean; 635 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 857 op/s 2024-09-21T04:09:04.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:03 smithi131 ceph-mon[35193]: pgmap v472: 97 pgs: 97 active+clean; 635 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 857 op/s 2024-09-21T04:09:06.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:05 smithi119 ceph-mon[34811]: pgmap v473: 97 pgs: 97 active+clean; 635 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 509 op/s 2024-09-21T04:09:06.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:05 smithi119 ceph-mon[34811]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:09:06.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:05 smithi064 ceph-mon[31180]: pgmap v473: 97 pgs: 97 active+clean; 635 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 509 op/s 2024-09-21T04:09:06.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:05 smithi064 ceph-mon[31180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:09:06.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:05 smithi131 ceph-mon[35193]: pgmap v473: 97 pgs: 97 active+clean; 635 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 509 op/s 2024-09-21T04:09:06.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:05 smithi131 ceph-mon[35193]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:09:08.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:07 smithi119 ceph-mon[34811]: pgmap v474: 97 pgs: 97 active+clean; 640 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 632 op/s 2024-09-21T04:09:08.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:07 smithi064 ceph-mon[31180]: pgmap v474: 97 pgs: 97 active+clean; 640 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 632 op/s 2024-09-21T04:09:08.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:07 smithi131 ceph-mon[35193]: pgmap v474: 97 pgs: 97 active+clean; 640 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 632 op/s 2024-09-21T04:09:10.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:09 smithi119 ceph-mon[34811]: pgmap v475: 97 pgs: 97 active+clean; 640 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 487 op/s 2024-09-21T04:09:10.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:09 smithi064 ceph-mon[31180]: pgmap v475: 97 pgs: 97 active+clean; 640 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 487 op/s 2024-09-21T04:09:10.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:09 smithi131 ceph-mon[35193]: pgmap v475: 97 pgs: 97 active+clean; 640 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 487 op/s 2024-09-21T04:09:11.174 INFO:tasks.workunit.client.0.smithi064.stdout: 100 8219 31 6187 91 5943 157 2024-09-21T04:09:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.405+0000 7f6d27627640 1 -- 172.21.15.64:0/4108397709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d201051f0 msgr2=0x7f6d2010d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.405+0000 7f6d27627640 1 --2- 172.21.15.64:0/4108397709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d201051f0 0x7f6d2010d830 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f6d10009900 tx=0x7f6d1002a410 comp rx=0 tx=0).stop 2024-09-21T04:09:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.405+0000 7f6d27627640 1 -- 172.21.15.64:0/4108397709 shutdown_connections 2024-09-21T04:09:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.405+0000 7f6d27627640 1 --2- 172.21.15.64:0/4108397709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d2010dd70 0x7f6d20110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.405+0000 7f6d27627640 1 --2- 172.21.15.64:0/4108397709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d201051f0 0x7f6d2010d830 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.405+0000 7f6d27627640 1 --2- 172.21.15.64:0/4108397709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d201047c0 0x7f6d20104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.405+0000 7f6d27627640 1 -- 172.21.15.64:0/4108397709 >> 172.21.15.64:0/4108397709 conn(0x7f6d2007a740 msgr2=0x7f6d2007ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.405+0000 7f6d27627640 1 -- 172.21.15.64:0/4108397709 shutdown_connections 2024-09-21T04:09:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.405+0000 7f6d27627640 1 -- 172.21.15.64:0/4108397709 wait complete. 2024-09-21T04:09:11.410 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.406+0000 7f6d27627640 1 Processor -- start 2024-09-21T04:09:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.406+0000 7f6d27627640 1 -- start start 2024-09-21T04:09:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.406+0000 7f6d27627640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d201047c0 0x7f6d2019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.406+0000 7f6d27627640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d201051f0 0x7f6d2019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.406+0000 7f6d27627640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d2010dd70 0x7f6d2019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.406+0000 7f6d27627640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201139d0 con 0x7f6d201047c0 2024-09-21T04:09:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.406+0000 7f6d27627640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6d20113850 con 0x7f6d201051f0 2024-09-21T04:09:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.406+0000 7f6d27627640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6d20113b50 con 0x7f6d2010dd70 2024-09-21T04:09:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.406+0000 7f6d253c3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d201047c0 0x7f6d2019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.406+0000 7f6d253c3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d201047c0 0x7f6d2019a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35286/0 (socket says 172.21.15.64:35286) 2024-09-21T04:09:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.406+0000 7f6d253c3640 1 -- 172.21.15.64:0/2749211851 learned_addr learned my addr 172.21.15.64:0/2749211851 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.406+0000 7f6d24bc2640 1 --2- 172.21.15.64:0/2749211851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d201051f0 0x7f6d2019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.406+0000 7f6d25bc4640 1 --2- 172.21.15.64:0/2749211851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d2010dd70 0x7f6d2019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:11.411 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.406+0000 7f6d253c3640 1 -- 172.21.15.64:0/2749211851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d2010dd70 msgr2=0x7f6d2019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.406+0000 7f6d253c3640 1 --2- 172.21.15.64:0/2749211851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d2010dd70 0x7f6d2019fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.406+0000 7f6d253c3640 1 -- 172.21.15.64:0/2749211851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d201051f0 msgr2=0x7f6d2019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.406+0000 7f6d253c3640 1 --2- 172.21.15.64:0/2749211851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d201051f0 0x7f6d2019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.406+0000 7f6d253c3640 1 -- 172.21.15.64:0/2749211851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d201a0420 con 0x7f6d201047c0 2024-09-21T04:09:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.407+0000 7f6d253c3640 1 --2- 172.21.15.64:0/2749211851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d201047c0 0x7f6d2019a4f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00e9f0 tx=0x7f6d1c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.407+0000 7f6d167fc640 1 -- 172.21.15.64:0/2749211851 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c00cd80 con 0x7f6d201047c0 2024-09-21T04:09:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.407+0000 7f6d167fc640 1 -- 172.21.15.64:0/2749211851 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d1c0054d0 con 0x7f6d201047c0 2024-09-21T04:09:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.407+0000 7f6d27627640 1 -- 172.21.15.64:0/2749211851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d201a0710 con 0x7f6d201047c0 2024-09-21T04:09:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.407+0000 7f6d167fc640 1 -- 172.21.15.64:0/2749211851 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c010690 con 0x7f6d201047c0 2024-09-21T04:09:11.412 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.407+0000 7f6d27627640 1 -- 172.21.15.64:0/2749211851 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d201a6ea0 con 0x7f6d201047c0 2024-09-21T04:09:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.411+0000 7f6d167fc640 1 -- 172.21.15.64:0/2749211851 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f6d1c004e90 con 0x7f6d201047c0 2024-09-21T04:09:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.411+0000 7f6d2e66f740 1 Processor -- start 2024-09-21T04:09:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.411+0000 7f6d2e66f740 1 -- start start 2024-09-21T04:09:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.411+0000 7f6d2e66f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa6cc4d1b0 0x55aa6cff29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.411+0000 7f6d2e66f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa6cc242f0 0x55aa6cff2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.411+0000 7f6d2e66f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa6cc2c4d0 0x55aa6cff3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.411+0000 7f6d2e66f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55aa6cf676f0 con 0x55aa6cc4d1b0 2024-09-21T04:09:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.411+0000 7f6d2e66f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55aa6cf56b20 con 0x55aa6cc2c4d0 2024-09-21T04:09:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.411+0000 7f6d2e66f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55aa6cf5a930 con 0x55aa6cc242f0 2024-09-21T04:09:11.415 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.411+0000 7f6d25bc4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa6cc4d1b0 0x55aa6cff29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.411+0000 7f6d25bc4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa6cc4d1b0 0x55aa6cff29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35302/0 (socket says 172.21.15.64:35302) 2024-09-21T04:09:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.411+0000 7f6d25bc4640 1 -- 172.21.15.64:0/190168396 learned_addr learned my addr 172.21.15.64:0/190168396 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.411+0000 7f6d24bc2640 1 --2- 172.21.15.64:0/190168396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa6cc2c4d0 0x55aa6cff3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.412+0000 7f6d253c3640 1 --2- 172.21.15.64:0/190168396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa6cc242f0 0x55aa6cff2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.412+0000 7f6d25bc4640 1 -- 172.21.15.64:0/190168396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa6cc242f0 msgr2=0x55aa6cff2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.412+0000 7f6d25bc4640 1 --2- 172.21.15.64:0/190168396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa6cc242f0 0x55aa6cff2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.412+0000 7f6d25bc4640 1 -- 172.21.15.64:0/190168396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa6cc2c4d0 msgr2=0x55aa6cff3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.412+0000 7f6d25bc4640 1 --2- 172.21.15.64:0/190168396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa6cc2c4d0 0x55aa6cff3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.412+0000 7f6d25bc4640 1 -- 172.21.15.64:0/190168396 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa6cf1c530 con 0x55aa6cc4d1b0 2024-09-21T04:09:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.412+0000 7f6d253c3640 1 --2- 172.21.15.64:0/190168396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa6cc242f0 0x55aa6cff2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:09:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.412+0000 7f6d24bc2640 1 --2- 172.21.15.64:0/190168396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa6cc2c4d0 0x55aa6cff3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:09:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.412+0000 7f6d167fc640 1 --2- 172.21.15.64:0/2749211851 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6d0c0b6680 0x7f6d0c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.412+0000 7f6d253c3640 1 --2- 172.21.15.64:0/2749211851 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6d0c0b6680 0x7f6d0c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:11.416 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.412+0000 7f6d25bc4640 1 --2- 172.21.15.64:0/190168396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa6cc4d1b0 0x55aa6cff29b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f6d18009f40 tx=0x7f6d18010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494049aa9248dc85 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.412+0000 7f6d167fc640 1 -- 172.21.15.64:0/2749211851 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f6d1c0e7bd0 con 0x7f6d201047c0 2024-09-21T04:09:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.412+0000 7f6d0b7fe640 1 -- 172.21.15.64:0/190168396 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18003b70 con 0x55aa6cc4d1b0 2024-09-21T04:09:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.412+0000 7f6d0b7fe640 1 -- 172.21.15.64:0/190168396 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d18003d10 con 0x55aa6cc4d1b0 2024-09-21T04:09:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.412+0000 7f6d0b7fe640 1 -- 172.21.15.64:0/190168396 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18011980 con 0x55aa6cc4d1b0 2024-09-21T04:09:11.420 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.413+0000 7f6d2e66f740 1 -- 172.21.15.64:0/190168396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa6cc4d1b0 msgr2=0x55aa6cff29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.413+0000 7f6d2e66f740 1 --2- 172.21.15.64:0/190168396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa6cc4d1b0 0x55aa6cff29b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f6d18009f40 tx=0x7f6d18010860 comp rx=0 tx=0).stop 2024-09-21T04:09:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.413+0000 7f6d253c3640 1 --2- 172.21.15.64:0/2749211851 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6d0c0b6680 0x7f6d0c0b8b90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c0e53a0 tx=0x7f6d1c004a90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.413+0000 7f6d2e66f740 1 -- 172.21.15.64:0/190168396 shutdown_connections 2024-09-21T04:09:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.413+0000 7f6d2e66f740 1 --2- 172.21.15.64:0/190168396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa6cc2c4d0 0x55aa6cff3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.413+0000 7f6d2e66f740 1 --2- 172.21.15.64:0/190168396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa6cc242f0 0x55aa6cff2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.413+0000 7f6d2e66f740 1 --2- 172.21.15.64:0/190168396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa6cc4d1b0 0x55aa6cff29b0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.413+0000 7f6d2e66f740 1 -- 172.21.15.64:0/190168396 >> 172.21.15.64:0/190168396 conn(0x55aa6cc41300 msgr2=0x55aa6cff08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.413+0000 7f6d2e66f740 1 -- 172.21.15.64:0/190168396 shutdown_connections 2024-09-21T04:09:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.413+0000 7f6d2e66f740 1 -- 172.21.15.64:0/190168396 wait complete. 2024-09-21T04:09:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.414+0000 7f6d2e66f740 1 -- start start 2024-09-21T04:09:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.414+0000 7f6d2e66f740 1 Processor -- start 2024-09-21T04:09:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.414+0000 7f6d2e66f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa6cc242f0 0x55aa6d002440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.414+0000 7f6d2e66f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa6cc2c4d0 0x55aa6d002980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.414+0000 7f6d2e66f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa6cc4d1b0 0x55aa6d00e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.414+0000 7f6d2e66f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55aa6ce8eab0 con 0x55aa6cc242f0 2024-09-21T04:09:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.414+0000 7f6d2e66f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55aa6cefc730 con 0x55aa6cc2c4d0 2024-09-21T04:09:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.414+0000 7f6d2e66f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55aa6d005510 con 0x55aa6cc4d1b0 2024-09-21T04:09:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.414+0000 7f6d2e66f740 10 client.0 _ll_register_callbacks cb 0x55aa6cf506b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:09:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.414+0000 7f6d25bc4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa6cc242f0 0x55aa6d002440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.414+0000 7f6d25bc4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa6cc242f0 0x55aa6d002440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35310/0 (socket says 172.21.15.64:35310) 2024-09-21T04:09:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.414+0000 7f6d25bc4640 1 -- 172.21.15.64:0/2905257748 learned_addr learned my addr 172.21.15.64:0/2905257748 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.414+0000 7f6d24bc2640 1 --2- 172.21.15.64:0/2905257748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa6cc2c4d0 0x55aa6d002980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.415+0000 7f6d25bc4640 1 -- 172.21.15.64:0/2905257748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa6cc4d1b0 msgr2=0x55aa6d00e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.415+0000 7f6d25bc4640 1 --2- 172.21.15.64:0/2905257748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa6cc4d1b0 0x55aa6d00e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.415+0000 7f6d25bc4640 1 -- 172.21.15.64:0/2905257748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa6cc2c4d0 msgr2=0x55aa6d002980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.415+0000 7f6d25bc4640 1 --2- 172.21.15.64:0/2905257748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa6cc2c4d0 0x55aa6d002980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.415+0000 7f6d25bc4640 1 -- 172.21.15.64:0/2905257748 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa6ccb8790 con 0x55aa6cc242f0 2024-09-21T04:09:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.415+0000 7f6d25bc4640 1 --2- 172.21.15.64:0/2905257748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa6cc242f0 0x55aa6d002440 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6d18009f10 tx=0x7f6d18004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff67c327ecf7930 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.415+0000 7f6d08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:09:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.415+0000 7f6d08ff9640 1 -- 172.21.15.64:0/2905257748 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18003ec0 con 0x55aa6cc242f0 2024-09-21T04:09:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.415+0000 7f6d08ff9640 1 -- 172.21.15.64:0/2905257748 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f6d18004060 con 0x55aa6cc242f0 2024-09-21T04:09:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.415+0000 7f6d2e66f740 1 -- 172.21.15.64:0/2905257748 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d18003d10 con 0x55aa6cc242f0 2024-09-21T04:09:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.415+0000 7f6d08ff9640 1 -- 172.21.15.64:0/2905257748 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6d18004470 con 0x55aa6cc242f0 2024-09-21T04:09:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.415+0000 7f6d2e66f740 10 client.16620 fetch_fsmap learned FSMap version 35 2024-09-21T04:09:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.415+0000 7f6d2e66f740 1 -- 172.21.15.64:0/2905257748 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55aa6cd31620 con 0x55aa6cc242f0 2024-09-21T04:09:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.415+0000 7f6d08ff9640 1 -- 172.21.15.64:0/2905257748 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d18003d10 con 0x55aa6cc242f0 2024-09-21T04:09:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.416+0000 7f6d08ff9640 1 -- 172.21.15.64:0/2905257748 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f6d180053b0 con 0x55aa6cc242f0 2024-09-21T04:09:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.416+0000 7f6d2e66f740 10 client.16620 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T04:09:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.416+0000 7f6d2e66f740 10 client.16620 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:09:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.416+0000 7f6d2e66f740 20 client.16620 populate_metadata read hostname 'smithi064' 2024-09-21T04:09:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.416+0000 7f6d2e66f740 1 --2- 172.21.15.64:0/2905257748 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55aa6cc0bd80 0x55aa6d0143c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.416+0000 7f6d2e66f740 4 client.16620 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:09:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.416+0000 7f6d2e66f740 1 -- 172.21.15.64:0/2905257748 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa6cf69640 con 0x55aa6cc0bd80 2024-09-21T04:09:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.416+0000 7f6d24bc2640 1 --2- 172.21.15.64:0/2905257748 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55aa6cc0bd80 0x55aa6d0143c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.416+0000 7f6d08ff9640 10 client.16620 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.416+0000 7f6d24bc2640 1 --2- 172.21.15.64:0/2905257748 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55aa6cc0bd80 0x55aa6d0143c0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e34345fd1e6a129 server_cookie=f4e1bef14dfa93a2 in_seq=0 out_seq=0 2024-09-21T04:09:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.417+0000 7f6d08ff9640 1 -- 172.21.15.64:0/2905257748 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55aa6cf69640 con 0x55aa6cc0bd80 2024-09-21T04:09:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.417+0000 7f6d08ff9640 10 client.16620 handle_command_reply: tid=0 2024-09-21T04:09:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.417+0000 7f6d2e66f740 1 client.16620 shutdown 2024-09-21T04:09:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.417+0000 7f6d2e66f740 1 -- 172.21.15.64:0/2905257748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa6cc242f0 msgr2=0x55aa6d002440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.417+0000 7f6d2e66f740 1 --2- 172.21.15.64:0/2905257748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa6cc242f0 0x55aa6d002440 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6d18009f10 tx=0x7f6d18004c70 comp rx=0 tx=0).stop 2024-09-21T04:09:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.417+0000 7f6d2e66f740 1 -- 172.21.15.64:0/2905257748 shutdown_connections 2024-09-21T04:09:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.417+0000 7f6d2e66f740 1 --2- 172.21.15.64:0/2905257748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa6cc4d1b0 0x55aa6d00e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.417+0000 7f6d2e66f740 1 --2- 172.21.15.64:0/2905257748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa6cc2c4d0 0x55aa6d002980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.417+0000 7f6d2e66f740 1 --2- 172.21.15.64:0/2905257748 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55aa6cc0bd80 0x55aa6d0143c0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.417+0000 7f6d2e66f740 1 --2- 172.21.15.64:0/2905257748 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa6cc242f0 0x55aa6d002440 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.417+0000 7f6d08ff9640 0 client.16620 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.417+0000 7f6d2e66f740 1 -- 172.21.15.64:0/2905257748 >> 172.21.15.64:0/2905257748 conn(0x55aa6cc41300 msgr2=0x55aa6cff43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.417+0000 7f6d2e66f740 1 -- 172.21.15.64:0/2905257748 shutdown_connections 2024-09-21T04:09:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.417+0000 7f6d2e66f740 1 -- 172.21.15.64:0/2905257748 wait complete. 2024-09-21T04:09:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.417+0000 7f6d2e66f740 20 client.16620 trim_cache size 0 max 16384 2024-09-21T04:09:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.432+0000 7f6d2e66f740 1 Processor -- start 2024-09-21T04:09:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.432+0000 7f6d2e66f740 1 -- start start 2024-09-21T04:09:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.432+0000 7f6d2e66f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa6cc0bd80 0x55aa6d00e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.432+0000 7f6d2e66f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa6cc242f0 0x55aa6cfff610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.432+0000 7f6d2e66f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa6cc2c4d0 0x55aa6cfffb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.432+0000 7f6d2e66f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201b0060 con 0x55aa6cc0bd80 2024-09-21T04:09:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.432+0000 7f6d2e66f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201ad8d0 con 0x55aa6cc242f0 2024-09-21T04:09:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.432+0000 7f6d2e66f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201ab720 con 0x55aa6cc2c4d0 2024-09-21T04:09:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.432+0000 7f6d25bc4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa6cc0bd80 0x55aa6d00e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.432+0000 7f6d25bc4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa6cc0bd80 0x55aa6d00e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35318/0 (socket says 172.21.15.64:35318) 2024-09-21T04:09:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.432+0000 7f6d25bc4640 1 -- 172.21.15.64:0/340767309 learned_addr learned my addr 172.21.15.64:0/340767309 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.432+0000 7f6d24bc2640 1 --2- 172.21.15.64:0/340767309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa6cc242f0 0x55aa6cfff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.433+0000 7f6d25bc4640 1 -- 172.21.15.64:0/340767309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa6cc2c4d0 msgr2=0x55aa6cfffb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.433+0000 7f6d25bc4640 1 --2- 172.21.15.64:0/340767309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa6cc2c4d0 0x55aa6cfffb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.433+0000 7f6d25bc4640 1 -- 172.21.15.64:0/340767309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa6cc242f0 msgr2=0x55aa6cfff610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.433+0000 7f6d25bc4640 1 --2- 172.21.15.64:0/340767309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa6cc242f0 0x55aa6cfff610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:11.436 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.433+0000 7f6d25bc4640 1 -- 172.21.15.64:0/340767309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa6d000920 con 0x55aa6cc0bd80 2024-09-21T04:09:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.433+0000 7f6d24bc2640 1 --2- 172.21.15.64:0/340767309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa6cc242f0 0x55aa6cfff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:09:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.433+0000 7f6d25bc4640 1 --2- 172.21.15.64:0/340767309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa6cc0bd80 0x55aa6d00e770 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6d18010de0 tx=0x7f6d18009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac18157b11f6e445 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.433+0000 7f6d0affd640 1 -- 172.21.15.64:0/340767309 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18004810 con 0x55aa6cc0bd80 2024-09-21T04:09:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.433+0000 7f6d0affd640 1 -- 172.21.15.64:0/340767309 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d18005000 con 0x55aa6cc0bd80 2024-09-21T04:09:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.433+0000 7f6d0affd640 1 -- 172.21.15.64:0/340767309 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1802d8a0 con 0x55aa6cc0bd80 2024-09-21T04:09:11.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.434+0000 7f6d2e66f740 1 -- 172.21.15.64:0/340767309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa6cc0bd80 msgr2=0x55aa6d00e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:11.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.434+0000 7f6d2e66f740 1 --2- 172.21.15.64:0/340767309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa6cc0bd80 0x55aa6d00e770 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6d18010de0 tx=0x7f6d18009e10 comp rx=0 tx=0).stop 2024-09-21T04:09:11.440 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.434+0000 7f6d2e66f740 1 -- 172.21.15.64:0/340767309 shutdown_connections 2024-09-21T04:09:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.434+0000 7f6d2e66f740 1 --2- 172.21.15.64:0/340767309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa6cc2c4d0 0x55aa6cfffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.434+0000 7f6d2e66f740 1 --2- 172.21.15.64:0/340767309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa6cc242f0 0x55aa6cfff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.434+0000 7f6d2e66f740 1 --2- 172.21.15.64:0/340767309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa6cc0bd80 0x55aa6d00e770 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.434+0000 7f6d2e66f740 1 -- 172.21.15.64:0/340767309 >> 172.21.15.64:0/340767309 conn(0x55aa6cc41300 msgr2=0x55aa6cff7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.434+0000 7f6d2e66f740 1 -- 172.21.15.64:0/340767309 shutdown_connections 2024-09-21T04:09:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.434+0000 7f6d2e66f740 1 -- 172.21.15.64:0/340767309 wait complete. 2024-09-21T04:09:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.434+0000 7f6d2e66f740 1 -- start start 2024-09-21T04:09:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.434+0000 7f6d2e66f740 1 Processor -- start 2024-09-21T04:09:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.435+0000 7f6d2e66f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa6cc0bd80 0x55aa6d00b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.435+0000 7f6d2e66f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa6cc242f0 0x55aa6d006720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.435+0000 7f6d2e66f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa6cc2c4d0 0x55aa6d006e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.435+0000 7f6d2e66f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201abd90 con 0x55aa6cc2c4d0 2024-09-21T04:09:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.435+0000 7f6d2e66f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201ad200 con 0x55aa6cc0bd80 2024-09-21T04:09:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.435+0000 7f6d2e66f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201ac460 con 0x55aa6cc242f0 2024-09-21T04:09:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.435+0000 7f6d2e66f740 10 client.0 _ll_register_callbacks cb 0x7f6d1c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:09:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.435+0000 7f6d25bc4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa6cc2c4d0 0x55aa6d006e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.435+0000 7f6d25bc4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa6cc0bd80 0x55aa6d00b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.435+0000 7f6d25bc4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa6cc0bd80 0x55aa6d00b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:36420/0 (socket says 172.21.15.64:36420) 2024-09-21T04:09:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.435+0000 7f6d25bc4640 1 -- 172.21.15.64:0/3242768059 learned_addr learned my addr 172.21.15.64:0/3242768059 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.435+0000 7f6d24bc2640 1 --2- 172.21.15.64:0/3242768059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa6cc242f0 0x55aa6d006720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.435+0000 7f6d25bc4640 1 -- 172.21.15.64:0/3242768059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa6cc242f0 msgr2=0x55aa6d006720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.435+0000 7f6d25bc4640 1 --2- 172.21.15.64:0/3242768059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa6cc242f0 0x55aa6d006720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.435+0000 7f6d25bc4640 1 -- 172.21.15.64:0/3242768059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa6cc2c4d0 msgr2=0x55aa6d006e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.435+0000 7f6d25bc4640 1 --2- 172.21.15.64:0/3242768059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa6cc2c4d0 0x55aa6d006e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.435+0000 7f6d25bc4640 1 -- 172.21.15.64:0/3242768059 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa6d07dca0 con 0x55aa6cc0bd80 2024-09-21T04:09:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.436+0000 7f6d25bc4640 1 --2- 172.21.15.64:0/3242768059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa6cc0bd80 0x55aa6d00b200 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6d18005970 tx=0x7f6d18009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=978083370e4d79d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.436+0000 7f6d08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:09:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.436+0000 7f6d08ff9640 1 -- 172.21.15.64:0/3242768059 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18003cc0 con 0x55aa6cc0bd80 2024-09-21T04:09:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.436+0000 7f6d08ff9640 1 -- 172.21.15.64:0/3242768059 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f6d18003e60 con 0x55aa6cc0bd80 2024-09-21T04:09:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.436+0000 7f6d2e66f740 1 -- 172.21.15.64:0/3242768059 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d18005000 con 0x55aa6cc0bd80 2024-09-21T04:09:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.436+0000 7f6d08ff9640 1 -- 172.21.15.64:0/3242768059 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6d180051b0 con 0x55aa6cc0bd80 2024-09-21T04:09:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.436+0000 7f6d2e66f740 10 client.26327 fetch_fsmap learned FSMap version 35 2024-09-21T04:09:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.436+0000 7f6d2e66f740 1 -- 172.21.15.64:0/3242768059 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55aa6d07ee40 con 0x55aa6cc0bd80 2024-09-21T04:09:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.436+0000 7f6d08ff9640 1 -- 172.21.15.64:0/3242768059 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d18005000 con 0x55aa6cc0bd80 2024-09-21T04:09:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.436+0000 7f6d08ff9640 1 -- 172.21.15.64:0/3242768059 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f6d18011500 con 0x55aa6cc0bd80 2024-09-21T04:09:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.436+0000 7f6d2e66f740 10 client.26327 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T04:09:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.436+0000 7f6d2e66f740 10 client.26327 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:09:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.436+0000 7f6d2e66f740 20 client.26327 populate_metadata read hostname 'smithi064' 2024-09-21T04:09:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.436+0000 7f6d2e66f740 1 --2- 172.21.15.64:0/3242768059 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55aa6cc4d1b0 0x55aa6d081410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.436+0000 7f6d2e66f740 4 client.26327 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:09:11.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.436+0000 7f6d2e66f740 1 -- 172.21.15.64:0/3242768059 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aa6cf650f0 con 0x55aa6cc4d1b0 2024-09-21T04:09:11.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.437+0000 7f6d24bc2640 1 --2- 172.21.15.64:0/3242768059 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55aa6cc4d1b0 0x55aa6d081410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:11.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.437+0000 7f6d24bc2640 1 --2- 172.21.15.64:0/3242768059 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55aa6cc4d1b0 0x55aa6d081410 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c06f84c8b13399a server_cookie=9b4288026cd70306 in_seq=0 out_seq=0 2024-09-21T04:09:11.444 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:11.437+0000 7f6d08ff9640 10 client.26327 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:12.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:11 smithi119 ceph-mon[34811]: pgmap v476: 97 pgs: 97 active+clean; 640 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 407 op/s 2024-09-21T04:09:12.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:11 smithi064 ceph-mon[31180]: pgmap v476: 97 pgs: 97 active+clean; 640 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 407 op/s 2024-09-21T04:09:12.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:11 smithi131 ceph-mon[35193]: pgmap v476: 97 pgs: 97 active+clean; 640 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 407 op/s 2024-09-21T04:09:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:12.714+0000 7f6d08ff9640 1 -- 172.21.15.64:0/3242768059 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aa6cf650f0 con 0x55aa6cc4d1b0 2024-09-21T04:09:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:12.714+0000 7f6d08ff9640 10 client.26327 handle_command_reply: tid=0 2024-09-21T04:09:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:12.714+0000 7f6d2e66f740 1 client.26327 shutdown 2024-09-21T04:09:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:12.714+0000 7f6d2e66f740 1 -- 172.21.15.64:0/3242768059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa6cc0bd80 msgr2=0x55aa6d00b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:12.714+0000 7f6d2e66f740 1 --2- 172.21.15.64:0/3242768059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa6cc0bd80 0x55aa6d00b200 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6d18005970 tx=0x7f6d18009fd0 comp rx=0 tx=0).stop 2024-09-21T04:09:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:12.714+0000 7f6d2e66f740 1 -- 172.21.15.64:0/3242768059 shutdown_connections 2024-09-21T04:09:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:12.714+0000 7f6d2e66f740 1 --2- 172.21.15.64:0/3242768059 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55aa6cc4d1b0 0x55aa6d081410 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:12.714+0000 7f6d2e66f740 1 --2- 172.21.15.64:0/3242768059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa6cc2c4d0 0x55aa6d006e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:12.714+0000 7f6d2e66f740 1 --2- 172.21.15.64:0/3242768059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aa6cc242f0 0x55aa6d006720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:12.714+0000 7f6d08ff9640 0 client.26327 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:12.714+0000 7f6d2e66f740 1 --2- 172.21.15.64:0/3242768059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55aa6cc0bd80 0x55aa6d00b200 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:12.714+0000 7f6d2e66f740 1 -- 172.21.15.64:0/3242768059 >> 172.21.15.64:0/3242768059 conn(0x55aa6cc41300 msgr2=0x55aa6d004280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:12.714+0000 7f6d2e66f740 1 -- 172.21.15.64:0/3242768059 shutdown_connections 2024-09-21T04:09:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:12.714+0000 7f6d2e66f740 1 -- 172.21.15.64:0/3242768059 wait complete. 2024-09-21T04:09:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:12.714+0000 7f6d2e66f740 20 client.26327 trim_cache size 0 max 16384 2024-09-21T04:09:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:09:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:09:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:09:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:09:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:12.715+0000 7f6d27627640 1 -- 172.21.15.64:0/2749211851 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6d0c0b6680 msgr2=0x7f6d0c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:12.715+0000 7f6d27627640 1 --2- 172.21.15.64:0/2749211851 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6d0c0b6680 0x7f6d0c0b8b90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c0e53a0 tx=0x7f6d1c004a90 comp rx=0 tx=0).stop 2024-09-21T04:09:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:12.715+0000 7f6d27627640 1 -- 172.21.15.64:0/2749211851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d201047c0 msgr2=0x7f6d2019a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:12.715+0000 7f6d27627640 1 --2- 172.21.15.64:0/2749211851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d201047c0 0x7f6d2019a4f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00e9f0 tx=0x7f6d1c00edf0 comp rx=0 tx=0).stop 2024-09-21T04:09:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:12.715+0000 7f6d27627640 1 -- 172.21.15.64:0/2749211851 shutdown_connections 2024-09-21T04:09:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:12.715+0000 7f6d27627640 1 --2- 172.21.15.64:0/2749211851 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6d0c0b6680 0x7f6d0c0b8b90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:12.715+0000 7f6d27627640 1 --2- 172.21.15.64:0/2749211851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6d2010dd70 0x7f6d2019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:12.715+0000 7f6d27627640 1 --2- 172.21.15.64:0/2749211851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d201051f0 0x7f6d2019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:12.715+0000 7f6d27627640 1 --2- 172.21.15.64:0/2749211851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6d201047c0 0x7f6d2019a4f0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:12.715+0000 7f6d27627640 1 -- 172.21.15.64:0/2749211851 >> 172.21.15.64:0/2749211851 conn(0x7f6d2007a740 msgr2=0x7f6d20100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:12.715+0000 7f6d27627640 1 -- 172.21.15.64:0/2749211851 shutdown_connections 2024-09-21T04:09:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:12.715+0000 7f6d27627640 1 -- 172.21.15.64:0/2749211851 wait complete. 2024-09-21T04:09:14.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:13 smithi119 ceph-mon[34811]: pgmap v477: 97 pgs: 97 active+clean; 615 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 487 op/s 2024-09-21T04:09:14.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:13 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:09:14.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:13 smithi064 ceph-mon[31180]: pgmap v477: 97 pgs: 97 active+clean; 615 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 487 op/s 2024-09-21T04:09:14.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:13 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:09:14.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:13 smithi131 ceph-mon[35193]: pgmap v477: 97 pgs: 97 active+clean; 615 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 487 op/s 2024-09-21T04:09:14.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:13 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:09:15.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:14 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:09:15.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:14 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:09:15.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:14 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:09:15.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:14 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:09:15.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:14 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:09:15.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:14 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:09:15.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:14 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:09:15.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:14 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:09:15.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:14 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:09:16.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:15 smithi119 ceph-mon[34811]: pgmap v478: 97 pgs: 97 active+clean; 615 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 1.1 MiB/s wr, 244 op/s 2024-09-21T04:09:16.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:15 smithi064 ceph-mon[31180]: pgmap v478: 97 pgs: 97 active+clean; 615 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 1.1 MiB/s wr, 244 op/s 2024-09-21T04:09:16.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:15 smithi131 ceph-mon[35193]: pgmap v478: 97 pgs: 97 active+clean; 615 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 1.1 MiB/s wr, 244 op/s 2024-09-21T04:09:18.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:17 smithi119 ceph-mon[34811]: pgmap v479: 97 pgs: 97 active+clean; 622 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 320 op/s 2024-09-21T04:09:18.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:17 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:09:18.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:17 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:09:18.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:17 smithi064 ceph-mon[31180]: pgmap v479: 97 pgs: 97 active+clean; 622 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 320 op/s 2024-09-21T04:09:18.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:17 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:09:18.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:17 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:09:18.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:17 smithi131 ceph-mon[35193]: pgmap v479: 97 pgs: 97 active+clean; 622 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 320 op/s 2024-09-21T04:09:18.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:17 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:09:18.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:17 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:09:19.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:19 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:09:19.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:19 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:09:19.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:19 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:09:20.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:20 smithi131 ceph-mon[35193]: pgmap v480: 97 pgs: 97 active+clean; 626 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 228 op/s 2024-09-21T04:09:20.370 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T04:09:20.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:20 smithi119 ceph-mon[34811]: pgmap v480: 97 pgs: 97 active+clean; 626 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 228 op/s 2024-09-21T04:09:20.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:20 smithi064 ceph-mon[31180]: pgmap v480: 97 pgs: 97 active+clean; 626 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 228 op/s 2024-09-21T04:09:21.010 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:09:21.174 INFO:tasks.workunit.client.0.smithi064.stdout: 103 510675 314 362316 165 304253 661 2024-09-21T04:09:21.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.929+0000 7fea116e3640 1 -- 172.21.15.64:0/1892796505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea0c10a9e0 msgr2=0x7fea0c10adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:21.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.929+0000 7fea116e3640 1 --2- 172.21.15.64:0/1892796505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea0c10a9e0 0x7fea0c10adf0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fea04009f00 tx=0x7fea0402ab60 comp rx=0 tx=0).stop 2024-09-21T04:09:21.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.930+0000 7fea116e3640 1 -- 172.21.15.64:0/1892796505 shutdown_connections 2024-09-21T04:09:21.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.930+0000 7fea116e3640 1 --2- 172.21.15.64:0/1892796505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea0c11cc30 0x7fea0c11f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:21.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.930+0000 7fea116e3640 1 --2- 172.21.15.64:0/1892796505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea0c10a9e0 0x7fea0c10adf0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:21.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.930+0000 7fea116e3640 1 --2- 172.21.15.64:0/1892796505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea0c111f40 0x7fea0c10a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:21.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.930+0000 7fea116e3640 1 -- 172.21.15.64:0/1892796505 >> 172.21.15.64:0/1892796505 conn(0x7fea0c070ec0 msgr2=0x7fea0c0712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:21.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.930+0000 7fea116e3640 1 -- 172.21.15.64:0/1892796505 shutdown_connections 2024-09-21T04:09:21.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.930+0000 7fea116e3640 1 -- 172.21.15.64:0/1892796505 wait complete. 2024-09-21T04:09:21.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.932+0000 7fea116e3640 1 Processor -- start 2024-09-21T04:09:21.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.932+0000 7fea116e3640 1 -- start start 2024-09-21T04:09:21.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.933+0000 7fea116e3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea0c10a9e0 0x7fea0c1a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:21.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.933+0000 7fea116e3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea0c111f40 0x7fea0c1a99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:21.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.933+0000 7fea0affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea0c10a9e0 0x7fea0c1a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:21.936 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.933+0000 7fea0affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea0c10a9e0 0x7fea0c1a9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:54322/0 (socket says 172.21.15.64:54322) 2024-09-21T04:09:21.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.933+0000 7fea116e3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea0c11cc30 0x7fea0c1aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:21.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.933+0000 7fea116e3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c122a00 con 0x7fea0c11cc30 2024-09-21T04:09:21.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.933+0000 7fea116e3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c122880 con 0x7fea0c10a9e0 2024-09-21T04:09:21.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.933+0000 7fea116e3640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c122b80 con 0x7fea0c111f40 2024-09-21T04:09:21.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.934+0000 7fea0b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea0c11cc30 0x7fea0c1aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:21.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.934+0000 7fea0b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea0c11cc30 0x7fea0c1aec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58206/0 (socket says 172.21.15.64:58206) 2024-09-21T04:09:21.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.934+0000 7fea0b7fe640 1 -- 172.21.15.64:0/3074516506 learned_addr learned my addr 172.21.15.64:0/3074516506 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:21.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.935+0000 7fea0b7fe640 1 -- 172.21.15.64:0/3074516506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea0c111f40 msgr2=0x7fea0c1a99f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T04:09:21.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.935+0000 7fea0b7fe640 1 --2- 172.21.15.64:0/3074516506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea0c111f40 0x7fea0c1a99f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:21.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.935+0000 7fea0b7fe640 1 -- 172.21.15.64:0/3074516506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea0c10a9e0 msgr2=0x7fea0c1a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:21.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.935+0000 7fea0b7fe640 1 --2- 172.21.15.64:0/3074516506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea0c10a9e0 0x7fea0c1a9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:21.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.935+0000 7fea0b7fe640 1 -- 172.21.15.64:0/3074516506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea0c1af410 con 0x7fea0c11cc30 2024-09-21T04:09:21.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.936+0000 7fea0b7fe640 1 --2- 172.21.15.64:0/3074516506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea0c11cc30 0x7fea0c1aec90 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc00b780 tx=0x7fe9fc00bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:21.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.936+0000 7fe9f3fff640 1 -- 172.21.15.64:0/3074516506 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc019070 con 0x7fea0c11cc30 2024-09-21T04:09:21.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.936+0000 7fea116e3640 1 -- 172.21.15.64:0/3074516506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea0c1af700 con 0x7fea0c11cc30 2024-09-21T04:09:21.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.936+0000 7fea116e3640 1 -- 172.21.15.64:0/3074516506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea0c1b5e10 con 0x7fea0c11cc30 2024-09-21T04:09:21.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.937+0000 7fe9f3fff640 1 -- 172.21.15.64:0/3074516506 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9fc00cbd0 con 0x7fea0c11cc30 2024-09-21T04:09:21.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.937+0000 7fe9f3fff640 1 -- 172.21.15.64:0/3074516506 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc014d30 con 0x7fea0c11cc30 2024-09-21T04:09:21.947 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.945+0000 7fe9f3fff640 1 -- 172.21.15.64:0/3074516506 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fe9fc01c070 con 0x7fea0c11cc30 2024-09-21T04:09:21.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.947+0000 7fe9f3fff640 1 --2- 172.21.15.64:0/3074516506 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe9f40b6880 0x7fe9f40b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:21.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.948+0000 7fe9f3fff640 1 -- 172.21.15.64:0/3074516506 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fe9fc0e7120 con 0x7fea0c11cc30 2024-09-21T04:09:21.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.948+0000 7fea0affd640 1 --2- 172.21.15.64:0/3074516506 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe9f40b6880 0x7fe9f40b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:21.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.948+0000 7fea17c57740 1 Processor -- start 2024-09-21T04:09:21.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.949+0000 7fea17c57740 1 -- start start 2024-09-21T04:09:21.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.949+0000 7fea17c57740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b77e38d070 0x55b77e734270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:21.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.949+0000 7fea17c57740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b77e3641b0 0x55b77e7347b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:21.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.949+0000 7fea17c57740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b77e36c390 0x55b77e734cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:21.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.950+0000 7fea0a7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b77e3641b0 0x55b77e7347b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:21.953 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.950+0000 7fea0a7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b77e3641b0 0x55b77e7347b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:54326/0 (socket says 172.21.15.64:54326) 2024-09-21T04:09:21.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.950+0000 7fea0affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b77e38d070 0x55b77e734270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:21.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.950+0000 7fea0affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b77e38d070 0x55b77e734270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58214/0 (socket says 172.21.15.64:58214) 2024-09-21T04:09:21.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.950+0000 7fea0b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b77e36c390 0x55b77e734cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:21.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.950+0000 7fea17c57740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b77e6aae50 con 0x55b77e38d070 2024-09-21T04:09:21.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.950+0000 7fea17c57740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b77e69a280 con 0x55b77e3641b0 2024-09-21T04:09:21.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.950+0000 7fea17c57740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b77e69e090 con 0x55b77e36c390 2024-09-21T04:09:21.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.950+0000 7fea0b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b77e36c390 0x55b77e734cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:44808/0 (socket says 172.21.15.64:44808) 2024-09-21T04:09:21.954 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.950+0000 7fea0b7fe640 1 -- 172.21.15.64:0/709295735 learned_addr learned my addr 172.21.15.64:0/709295735 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:21.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.952+0000 7fea0affd640 1 --2- 172.21.15.64:0/3074516506 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe9f40b6880 0x7fe9f40b8d90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fea00006fd0 tx=0x7fea000090e0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:21.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.952+0000 7fea0affd640 1 -- 172.21.15.64:0/709295735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b77e36c390 msgr2=0x55b77e734cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:21.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.952+0000 7fea0affd640 1 --2- 172.21.15.64:0/709295735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b77e36c390 0x55b77e734cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:21.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.952+0000 7fea0affd640 1 -- 172.21.15.64:0/709295735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b77e3641b0 msgr2=0x55b77e7347b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:21.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.952+0000 7fea0affd640 1 --2- 172.21.15.64:0/709295735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b77e3641b0 0x55b77e7347b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:21.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.952+0000 7fea0affd640 1 -- 172.21.15.64:0/709295735 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b77e3f8650 con 0x55b77e38d070 2024-09-21T04:09:21.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.953+0000 7fea0b7fe640 1 --2- 172.21.15.64:0/709295735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b77e36c390 0x55b77e734cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:09:21.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.953+0000 7fea0affd640 1 --2- 172.21.15.64:0/709295735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b77e38d070 0x55b77e734270 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fea0000e5a0 tx=0x7fea00019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7027f3665d1b8a6b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:21.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.954+0000 7fe9f17fa640 1 -- 172.21.15.64:0/709295735 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00015c90 con 0x55b77e38d070 2024-09-21T04:09:21.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.954+0000 7fe9f17fa640 1 -- 172.21.15.64:0/709295735 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea00024070 con 0x55b77e38d070 2024-09-21T04:09:21.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.954+0000 7fe9f17fa640 1 -- 172.21.15.64:0/709295735 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea000137c0 con 0x55b77e38d070 2024-09-21T04:09:21.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.959+0000 7fea17c57740 1 -- 172.21.15.64:0/709295735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b77e38d070 msgr2=0x55b77e734270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:21.962 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.959+0000 7fea17c57740 1 --2- 172.21.15.64:0/709295735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b77e38d070 0x55b77e734270 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fea0000e5a0 tx=0x7fea00019000 comp rx=0 tx=0).stop 2024-09-21T04:09:21.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.961+0000 7fea17c57740 1 -- 172.21.15.64:0/709295735 shutdown_connections 2024-09-21T04:09:21.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.961+0000 7fea17c57740 1 --2- 172.21.15.64:0/709295735 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b77e36c390 0x55b77e734cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:21.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.961+0000 7fea17c57740 1 --2- 172.21.15.64:0/709295735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b77e3641b0 0x55b77e7347b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:21.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.961+0000 7fea17c57740 1 --2- 172.21.15.64:0/709295735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b77e38d070 0x55b77e734270 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:21.964 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.961+0000 7fea17c57740 1 -- 172.21.15.64:0/709295735 >> 172.21.15.64:0/709295735 conn(0x55b77e381460 msgr2=0x55b77e732160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:21.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.965+0000 7fea17c57740 1 -- 172.21.15.64:0/709295735 shutdown_connections 2024-09-21T04:09:21.967 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.965+0000 7fea17c57740 1 -- 172.21.15.64:0/709295735 wait complete. 2024-09-21T04:09:21.968 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.966+0000 7fea17c57740 1 -- start start 2024-09-21T04:09:21.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.966+0000 7fea17c57740 1 Processor -- start 2024-09-21T04:09:21.969 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.967+0000 7fea17c57740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b77e3641b0 0x55b77e73f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:21.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.967+0000 7fea17c57740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b77e36c390 0x55b77e73fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:21.970 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.968+0000 7fea17c57740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b77e38d070 0x55b77e74bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:21.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.968+0000 7fea0affd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b77e3641b0 0x55b77e73f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:21.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.968+0000 7fea0a7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b77e36c390 0x55b77e73fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:21.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.968+0000 7fea0a7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b77e36c390 0x55b77e73fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:54338/0 (socket says 172.21.15.64:54338) 2024-09-21T04:09:21.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.968+0000 7fea0b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b77e38d070 0x55b77e74bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:21.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.968+0000 7fea0b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b77e38d070 0x55b77e74bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58216/0 (socket says 172.21.15.64:58216) 2024-09-21T04:09:21.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.968+0000 7fea0a7fc640 1 -- 172.21.15.64:0/1809482563 learned_addr learned my addr 172.21.15.64:0/1809482563 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:21.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.968+0000 7fea17c57740 1 -- 172.21.15.64:0/1809482563 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b77e4fb700 con 0x55b77e38d070 2024-09-21T04:09:21.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.969+0000 7fea17c57740 1 -- 172.21.15.64:0/1809482563 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b77e635110 con 0x55b77e36c390 2024-09-21T04:09:21.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.969+0000 7fea17c57740 1 -- 172.21.15.64:0/1809482563 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b77e742a50 con 0x55b77e3641b0 2024-09-21T04:09:21.972 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.969+0000 7fea17c57740 10 client.0 _ll_register_callbacks cb 0x55b77e64e990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:09:21.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.970+0000 7fea0b7fe640 1 -- 172.21.15.64:0/1809482563 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b77e3641b0 msgr2=0x55b77e73f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:21.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.970+0000 7fea0b7fe640 1 --2- 172.21.15.64:0/1809482563 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b77e3641b0 0x55b77e73f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:21.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.971+0000 7fea0b7fe640 1 -- 172.21.15.64:0/1809482563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b77e36c390 msgr2=0x55b77e73fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:21.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.971+0000 7fea0b7fe640 1 --2- 172.21.15.64:0/1809482563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b77e36c390 0x55b77e73fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:21.973 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.971+0000 7fea0b7fe640 1 -- 172.21.15.64:0/1809482563 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b77e662c40 con 0x55b77e38d070 2024-09-21T04:09:21.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.971+0000 7fea0b7fe640 1 --2- 172.21.15.64:0/1809482563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b77e38d070 0x55b77e74bda0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc013000 tx=0x7fe9fc0ac700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ae6705fb25935d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:21.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.971+0000 7fe9eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:09:21.974 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.972+0000 7fe9eeff5640 1 -- 172.21.15.64:0/1809482563 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc0e4c70 con 0x55b77e38d070 2024-09-21T04:09:21.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.972+0000 7fe9eeff5640 1 -- 172.21.15.64:0/1809482563 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fe9fc00c920 con 0x55b77e38d070 2024-09-21T04:09:21.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.972+0000 7fea17c57740 1 -- 172.21.15.64:0/1809482563 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea00024070 con 0x55b77e38d070 2024-09-21T04:09:21.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.973+0000 7fe9eeff5640 1 -- 172.21.15.64:0/1809482563 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe9fc0b6900 con 0x55b77e38d070 2024-09-21T04:09:21.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.973+0000 7fea17c57740 10 client.16638 fetch_fsmap learned FSMap version 35 2024-09-21T04:09:21.975 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.973+0000 7fea17c57740 1 -- 172.21.15.64:0/1809482563 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b77e74e780 con 0x55b77e38d070 2024-09-21T04:09:21.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.973+0000 7fe9eeff5640 1 -- 172.21.15.64:0/1809482563 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9fc0c6070 con 0x55b77e38d070 2024-09-21T04:09:21.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.974+0000 7fe9eeff5640 1 -- 172.21.15.64:0/1809482563 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fe9fc0a89f0 con 0x55b77e38d070 2024-09-21T04:09:21.976 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.974+0000 7fea17c57740 10 client.16638 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T04:09:21.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.974+0000 7fea17c57740 10 client.16638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:09:21.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.974+0000 7fea17c57740 20 client.16638 populate_metadata read hostname 'smithi064' 2024-09-21T04:09:21.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.974+0000 7fea17c57740 1 --2- 172.21.15.64:0/1809482563 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b77e34b9f0 0x55b77e751cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:21.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.974+0000 7fea17c57740 4 client.16638 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:09:21.977 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.974+0000 7fea17c57740 1 -- 172.21.15.64:0/1809482563 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b77e6acda0 con 0x55b77e34b9f0 2024-09-21T04:09:21.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.980+0000 7fea0affd640 1 --2- 172.21.15.64:0/1809482563 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b77e34b9f0 0x55b77e751cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:21.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.981+0000 7fea0affd640 1 --2- 172.21.15.64:0/1809482563 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b77e34b9f0 0x55b77e751cf0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba2373b2f276d6dd server_cookie=806405de4bec9b69 in_seq=0 out_seq=0 2024-09-21T04:09:21.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.981+0000 7fe9eeff5640 10 client.16638 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:21.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.982+0000 7fe9eeff5640 1 -- 172.21.15.64:0/1809482563 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b77e6acda0 con 0x55b77e34b9f0 2024-09-21T04:09:21.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.982+0000 7fe9eeff5640 10 client.16638 handle_command_reply: tid=0 2024-09-21T04:09:21.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.983+0000 7fea17c57740 1 client.16638 shutdown 2024-09-21T04:09:21.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.983+0000 7fea17c57740 1 -- 172.21.15.64:0/1809482563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b77e38d070 msgr2=0x55b77e74bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:21.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.983+0000 7fea17c57740 1 --2- 172.21.15.64:0/1809482563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b77e38d070 0x55b77e74bda0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc013000 tx=0x7fe9fc0ac700 comp rx=0 tx=0).stop 2024-09-21T04:09:21.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.984+0000 7fea17c57740 1 -- 172.21.15.64:0/1809482563 shutdown_connections 2024-09-21T04:09:21.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.984+0000 7fea17c57740 1 --2- 172.21.15.64:0/1809482563 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b77e34b9f0 0x55b77e751cf0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:21.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.984+0000 7fea17c57740 1 --2- 172.21.15.64:0/1809482563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b77e38d070 0x55b77e74bda0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:21.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.984+0000 7fea17c57740 1 --2- 172.21.15.64:0/1809482563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b77e36c390 0x55b77e73fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:21.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.984+0000 7fea17c57740 1 --2- 172.21.15.64:0/1809482563 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b77e3641b0 0x55b77e73f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:21.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.984+0000 7fea17c57740 1 -- 172.21.15.64:0/1809482563 >> 172.21.15.64:0/1809482563 conn(0x55b77e381460 msgr2=0x55b77e737400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:21.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.984+0000 7fe9eeff5640 0 client.16638 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:21.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.984+0000 7fea17c57740 1 -- 172.21.15.64:0/1809482563 shutdown_connections 2024-09-21T04:09:21.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.984+0000 7fea17c57740 1 -- 172.21.15.64:0/1809482563 wait complete. 2024-09-21T04:09:21.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:21.988+0000 7fea17c57740 20 client.16638 trim_cache size 0 max 16384 2024-09-21T04:09:22.016 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:22 smithi064 ceph-mon[31180]: pgmap v481: 97 pgs: 97 active+clean; 626 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 211 op/s 2024-09-21T04:09:22.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.042+0000 7fea17c57740 1 Processor -- start 2024-09-21T04:09:22.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.043+0000 7fea17c57740 1 -- start start 2024-09-21T04:09:22.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.044+0000 7fea17c57740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b77e3641b0 0x55b77e737a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:22.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.044+0000 7fea17c57740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b77e36c390 0x55b77e74bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:22.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.044+0000 7fea17c57740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b77e38d070 0x55b77e731a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:22.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.044+0000 7fea17c57740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f80112c0 con 0x55b77e3641b0 2024-09-21T04:09:22.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.044+0000 7fea17c57740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f800eb30 con 0x55b77e38d070 2024-09-21T04:09:22.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.044+0000 7fea17c57740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f800c980 con 0x55b77e36c390 2024-09-21T04:09:22.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.048+0000 7fea0b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b77e38d070 0x55b77e731a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:22.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.048+0000 7fea0a7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b77e36c390 0x55b77e74bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:22.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.049+0000 7fea0a7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b77e36c390 0x55b77e74bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:44828/0 (socket says 172.21.15.64:44828) 2024-09-21T04:09:22.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.049+0000 7fea0a7fc640 1 -- 172.21.15.64:0/522897008 learned_addr learned my addr 172.21.15.64:0/522897008 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:22.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.049+0000 7fea0b7fe640 1 -- 172.21.15.64:0/522897008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b77e36c390 msgr2=0x55b77e74bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:22.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.049+0000 7fea0b7fe640 1 --2- 172.21.15.64:0/522897008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b77e36c390 0x55b77e74bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:22.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.049+0000 7fea0b7fe640 1 -- 172.21.15.64:0/522897008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b77e3641b0 msgr2=0x55b77e737a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:09:22.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.049+0000 7fea0b7fe640 1 --2- 172.21.15.64:0/522897008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b77e3641b0 0x55b77e737a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:22.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.049+0000 7fea0b7fe640 1 -- 172.21.15.64:0/522897008 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b77e34bc10 con 0x55b77e38d070 2024-09-21T04:09:22.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.054+0000 7fea0b7fe640 1 --2- 172.21.15.64:0/522897008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b77e38d070 0x55b77e731a80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc0a8d60 tx=0x7fe9fc0b1b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31af88702f8f76a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:22.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.057+0000 7fe9f0ff9640 1 -- 172.21.15.64:0/522897008 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc0e4c70 con 0x55b77e38d070 2024-09-21T04:09:22.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.057+0000 7fe9f0ff9640 1 -- 172.21.15.64:0/522897008 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9fc0c6050 con 0x55b77e38d070 2024-09-21T04:09:22.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.057+0000 7fe9f0ff9640 1 -- 172.21.15.64:0/522897008 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc0a8e30 con 0x55b77e38d070 2024-09-21T04:09:22.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.059+0000 7fea17c57740 1 -- 172.21.15.64:0/522897008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b77e38d070 msgr2=0x55b77e731a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:22.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.059+0000 7fea17c57740 1 --2- 172.21.15.64:0/522897008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b77e38d070 0x55b77e731a80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc0a8d60 tx=0x7fe9fc0b1b40 comp rx=0 tx=0).stop 2024-09-21T04:09:22.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.060+0000 7fea17c57740 1 -- 172.21.15.64:0/522897008 shutdown_connections 2024-09-21T04:09:22.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.060+0000 7fea17c57740 1 --2- 172.21.15.64:0/522897008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b77e38d070 0x55b77e731a80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:22.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.060+0000 7fea17c57740 1 --2- 172.21.15.64:0/522897008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b77e36c390 0x55b77e74bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:22.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.060+0000 7fea17c57740 1 --2- 172.21.15.64:0/522897008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b77e3641b0 0x55b77e737a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:22.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.060+0000 7fea17c57740 1 -- 172.21.15.64:0/522897008 >> 172.21.15.64:0/522897008 conn(0x55b77e381460 msgr2=0x55b77e737370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:22.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.062+0000 7fea17c57740 1 -- 172.21.15.64:0/522897008 shutdown_connections 2024-09-21T04:09:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.062+0000 7fea17c57740 1 -- 172.21.15.64:0/522897008 wait complete. 2024-09-21T04:09:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.063+0000 7fea17c57740 1 -- start start 2024-09-21T04:09:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.063+0000 7fea17c57740 1 Processor -- start 2024-09-21T04:09:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.064+0000 7fea17c57740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b77e3641b0 0x55b77e743d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.064+0000 7fea17c57740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b77e36c390 0x55b77e7442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.064+0000 7fea0affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b77e3641b0 0x55b77e743d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.064+0000 7fea0affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b77e3641b0 0x55b77e743d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:54364/0 (socket says 172.21.15.64:54364) 2024-09-21T04:09:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.064+0000 7fea0affd640 1 -- 172.21.15.64:0/3492204327 learned_addr learned my addr 172.21.15.64:0/3492204327 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.064+0000 7fea17c57740 1 --2- 172.21.15.64:0/3492204327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b77e38d070 0x55b77e7447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.064+0000 7fea17c57740 1 -- 172.21.15.64:0/3492204327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f800cff0 con 0x55b77e38d070 2024-09-21T04:09:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.064+0000 7fea17c57740 1 -- 172.21.15.64:0/3492204327 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f800e460 con 0x55b77e3641b0 2024-09-21T04:09:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.064+0000 7fea17c57740 1 -- 172.21.15.64:0/3492204327 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f800d6c0 con 0x55b77e36c390 2024-09-21T04:09:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.065+0000 7fea0a7fc640 1 --2- 172.21.15.64:0/3492204327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b77e36c390 0x55b77e7442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.065+0000 7fea0affd640 1 -- 172.21.15.64:0/3492204327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b77e36c390 msgr2=0x55b77e7442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.065+0000 7fea0affd640 1 --2- 172.21.15.64:0/3492204327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b77e36c390 0x55b77e7442a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.065+0000 7fea0affd640 1 -- 172.21.15.64:0/3492204327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b77e38d070 msgr2=0x55b77e7447e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:09:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.065+0000 7fea0affd640 1 --2- 172.21.15.64:0/3492204327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b77e38d070 0x55b77e7447e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.065+0000 7fea0affd640 1 -- 172.21.15.64:0/3492204327 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b77e7bb770 con 0x55b77e3641b0 2024-09-21T04:09:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.065+0000 7fea17c57740 10 client.0 _ll_register_callbacks cb 0x7fe9fc0a9640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:09:22.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.065+0000 7fea17c57740 1 -- 172.21.15.64:0/3492204327 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9fc0c6050 con 0x55b77e3641b0 2024-09-21T04:09:22.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.065+0000 7fea0affd640 1 --2- 172.21.15.64:0/3492204327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b77e3641b0 0x55b77e743d60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fea0001ff90 tx=0x7fea0000e920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f47f6171dbbed81 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:22.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.066+0000 7fe9eeff5640 10 client.26342 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:09:22.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.066+0000 7fe9eeff5640 1 -- 172.21.15.64:0/3492204327 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00030070 con 0x55b77e3641b0 2024-09-21T04:09:22.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.067+0000 7fe9eeff5640 1 -- 172.21.15.64:0/3492204327 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fea00013a40 con 0x55b77e3641b0 2024-09-21T04:09:22.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.067+0000 7fe9eeff5640 1 -- 172.21.15.64:0/3492204327 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fea00013ca0 con 0x55b77e3641b0 2024-09-21T04:09:22.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.069+0000 7fea17c57740 10 client.26342 fetch_fsmap learned FSMap version 35 2024-09-21T04:09:22.071 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.069+0000 7fea17c57740 1 -- 172.21.15.64:0/3492204327 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({fsmap=35}) -- 0x55b77e7bc910 con 0x55b77e3641b0 2024-09-21T04:09:22.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.073+0000 7fe9eeff5640 1 -- 172.21.15.64:0/3492204327 <== mon.2 v2:172.21.15.119:3300/0 4 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fea00004040 con 0x55b77e3641b0 2024-09-21T04:09:22.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.077+0000 7fea17c57740 10 client.26342 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T04:09:22.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.077+0000 7fea17c57740 10 client.26342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:09:22.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.077+0000 7fea17c57740 20 client.26342 populate_metadata read hostname 'smithi064' 2024-09-21T04:09:22.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.077+0000 7fea17c57740 1 --2- 172.21.15.64:0/3492204327 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b77e320da0 0x55b77e7beb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:22.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.077+0000 7fea17c57740 4 client.26342 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T04:09:22.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.077+0000 7fea17c57740 1 -- 172.21.15.64:0/3492204327 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b77e6a8850 con 0x55b77e320da0 2024-09-21T04:09:22.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.084+0000 7fea0a7fc640 1 --2- 172.21.15.64:0/3492204327 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b77e320da0 0x55b77e7beb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:22.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.088+0000 7fea0a7fc640 1 --2- 172.21.15.64:0/3492204327 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b77e320da0 0x55b77e7beb50 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c04ed698ee9d6754 server_cookie=adf23eeabe3aeee6 in_seq=0 out_seq=0 2024-09-21T04:09:22.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.088+0000 7fe9eeff5640 10 client.26342 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:22.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.088+0000 7fe9eeff5640 1 -- 172.21.15.64:0/3492204327 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2362 (crc 0 0 0) 0x55b77e6a8850 con 0x55b77e320da0 2024-09-21T04:09:22.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.088+0000 7fe9eeff5640 10 client.26342 handle_command_reply: tid=0 2024-09-21T04:09:22.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.090+0000 7fea17c57740 1 client.26342 shutdown 2024-09-21T04:09:22.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.094+0000 7fea17c57740 1 -- 172.21.15.64:0/3492204327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b77e3641b0 msgr2=0x55b77e743d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:22.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.094+0000 7fea17c57740 1 --2- 172.21.15.64:0/3492204327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b77e3641b0 0x55b77e743d60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fea0001ff90 tx=0x7fea0000e920 comp rx=0 tx=0).stop 2024-09-21T04:09:22.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.097+0000 7fea17c57740 1 -- 172.21.15.64:0/3492204327 shutdown_connections 2024-09-21T04:09:22.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.097+0000 7fea17c57740 1 --2- 172.21.15.64:0/3492204327 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b77e320da0 0x55b77e7beb50 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:22.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.097+0000 7fea17c57740 1 --2- 172.21.15.64:0/3492204327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b77e38d070 0x55b77e7447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:22.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.097+0000 7fea17c57740 1 --2- 172.21.15.64:0/3492204327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b77e36c390 0x55b77e7442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:22.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.097+0000 7fea17c57740 1 --2- 172.21.15.64:0/3492204327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b77e3641b0 0x55b77e743d60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:22.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.097+0000 7fea17c57740 1 -- 172.21.15.64:0/3492204327 >> 172.21.15.64:0/3492204327 conn(0x55b77e381460 msgr2=0x55b77e73f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:22.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.097+0000 7fe9eeff5640 0 client.26342 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:22.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.101+0000 7fea17c57740 1 -- 172.21.15.64:0/3492204327 shutdown_connections 2024-09-21T04:09:22.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.103+0000 7fea17c57740 1 -- 172.21.15.64:0/3492204327 wait complete. 2024-09-21T04:09:22.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.103+0000 7fea17c57740 20 client.26342 trim_cache size 0 max 16384 2024-09-21T04:09:22.106 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:09:22.106 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-21T04:09:22.106 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {}, 2024-09-21T04:09:22.106 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_stats": { 2024-09-21T04:09:22.106 INFO:teuthology.orchestra.run.smithi064.stdout: "bed40d88-2102-43a6-ac2d-b89d5a2264d0": { 2024-09-21T04:09:22.106 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:11.363340+0000", 2024-09-21T04:09:22.106 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:09:22.106 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:09:22.106 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:09:22.106 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:09:22.106 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:09:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:09:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: "e9242f88-6599-40eb-adf1-bdb3e946a1e0": { 2024-09-21T04:09:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:05:39.744646+0000", 2024-09-21T04:09:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:09:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:09:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:09:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:09:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:09:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:09:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: "6bde0fe5-f775-44b1-ad0c-f9e596574dc0": { 2024-09-21T04:09:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:06:23.835536+0000", 2024-09-21T04:09:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:09:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:09:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:09:22.107 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:09:22.108 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:09:22.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:09:22.108 INFO:teuthology.orchestra.run.smithi064.stdout: "b52b950c-0f97-49ec-a37c-1a8e04348ce8": { 2024-09-21T04:09:22.108 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:54.003800+0000", 2024-09-21T04:09:22.108 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:09:22.108 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:09:22.108 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:09:22.108 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:09:22.108 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:09:22.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:09:22.108 INFO:teuthology.orchestra.run.smithi064.stdout: "e2c2eed8-9292-4bf6-8a39-13ad27da5acb": { 2024-09-21T04:09:22.108 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:08:47.067932+0000", 2024-09-21T04:09:22.108 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:09:22.108 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:09:22.109 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:09:22.109 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:09:22.109 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:09:22.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:09:22.109 INFO:teuthology.orchestra.run.smithi064.stdout: "75db1bc5-612e-45aa-bad2-cc90feeaa840": { 2024-09-21T04:09:22.109 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:59.123138+0000", 2024-09-21T04:09:22.109 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:09:22.109 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:09:22.109 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:09:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:09:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:09:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:09:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: "0ab717a7-36c6-4dca-94e7-af0bdf2aa112": { 2024-09-21T04:09:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:42.691086+0000", 2024-09-21T04:09:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:09:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:09:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:09:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:09:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:09:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:09:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:09:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:03:59.570174+0000", 2024-09-21T04:09:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:09:22.110 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:09:22.111 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:09:22.111 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:09:22.111 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:09:22.111 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:09:22.111 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:09:22.111 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:09:22.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.106+0000 7fea116e3640 1 -- 172.21.15.64:0/3074516506 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe9f40b6880 msgr2=0x7fe9f40b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:22.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.106+0000 7fea116e3640 1 --2- 172.21.15.64:0/3074516506 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe9f40b6880 0x7fe9f40b8d90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fea00006fd0 tx=0x7fea000090e0 comp rx=0 tx=0).stop 2024-09-21T04:09:22.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.106+0000 7fea116e3640 1 -- 172.21.15.64:0/3074516506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea0c11cc30 msgr2=0x7fea0c1aec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:22.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.106+0000 7fea116e3640 1 --2- 172.21.15.64:0/3074516506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea0c11cc30 0x7fea0c1aec90 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc00b780 tx=0x7fe9fc00bc40 comp rx=0 tx=0).stop 2024-09-21T04:09:22.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.106+0000 7fea116e3640 1 -- 172.21.15.64:0/3074516506 shutdown_connections 2024-09-21T04:09:22.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.106+0000 7fea116e3640 1 --2- 172.21.15.64:0/3074516506 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe9f40b6880 0x7fe9f40b8d90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:22.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.106+0000 7fea116e3640 1 --2- 172.21.15.64:0/3074516506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea0c11cc30 0x7fea0c1aec90 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:22.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.106+0000 7fea116e3640 1 --2- 172.21.15.64:0/3074516506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea0c111f40 0x7fea0c1a99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:22.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.106+0000 7fea116e3640 1 --2- 172.21.15.64:0/3074516506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea0c10a9e0 0x7fea0c1a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:22.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.106+0000 7fea116e3640 1 -- 172.21.15.64:0/3074516506 >> 172.21.15.64:0/3074516506 conn(0x7fea0c070ec0 msgr2=0x7fea0c071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:22.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.106+0000 7fea116e3640 1 -- 172.21.15.64:0/3074516506 shutdown_connections 2024-09-21T04:09:22.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:22.106+0000 7fea116e3640 1 -- 172.21.15.64:0/3074516506 wait complete. 2024-09-21T04:09:22.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:22 smithi131 ceph-mon[35193]: pgmap v481: 97 pgs: 97 active+clean; 626 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 211 op/s 2024-09-21T04:09:22.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:22 smithi119 ceph-mon[34811]: pgmap v481: 97 pgs: 97 active+clean; 626 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 211 op/s 2024-09-21T04:09:23.526 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T04:09:23.526 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:09:24.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:24 smithi131 ceph-mon[35193]: pgmap v482: 97 pgs: 97 active+clean; 634 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 287 op/s 2024-09-21T04:09:24.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:24 smithi119 ceph-mon[34811]: pgmap v482: 97 pgs: 97 active+clean; 634 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 287 op/s 2024-09-21T04:09:24.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:24 smithi064 ceph-mon[31180]: pgmap v482: 97 pgs: 97 active+clean; 634 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 287 op/s 2024-09-21T04:09:26.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:26 smithi131 ceph-mon[35193]: pgmap v483: 97 pgs: 97 active+clean; 634 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 191 op/s 2024-09-21T04:09:26.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:26 smithi119 ceph-mon[34811]: pgmap v483: 97 pgs: 97 active+clean; 634 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 191 op/s 2024-09-21T04:09:26.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:26 smithi064 ceph-mon[31180]: pgmap v483: 97 pgs: 97 active+clean; 634 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 191 op/s 2024-09-21T04:09:26.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.811+0000 7f081ea4d640 1 -- 172.21.15.64:0/3725178627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0818118900 msgr2=0x7f081811ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:26.813 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.811+0000 7f081ea4d640 1 --2- 172.21.15.64:0/3725178627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0818118900 0x7f081811ad20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0810009be0 tx=0x7f081002a6e0 comp rx=0 tx=0).stop 2024-09-21T04:09:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.813+0000 7f081ea4d640 1 -- 172.21.15.64:0/3725178627 shutdown_connections 2024-09-21T04:09:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.813+0000 7f081ea4d640 1 --2- 172.21.15.64:0/3725178627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0818118900 0x7f081811ad20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.813+0000 7f081ea4d640 1 --2- 172.21.15.64:0/3725178627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f081810ad10 0x7f08181183c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.813+0000 7f081ea4d640 1 --2- 172.21.15.64:0/3725178627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f081810a3c0 0x7f081810a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:26.815 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.813+0000 7f081ea4d640 1 -- 172.21.15.64:0/3725178627 >> 172.21.15.64:0/3725178627 conn(0x7f0818070c30 msgr2=0x7f0818071040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.814+0000 7f081ea4d640 1 -- 172.21.15.64:0/3725178627 shutdown_connections 2024-09-21T04:09:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.814+0000 7f081ea4d640 1 -- 172.21.15.64:0/3725178627 wait complete. 2024-09-21T04:09:26.818 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.816+0000 7f081ea4d640 1 Processor -- start 2024-09-21T04:09:26.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.816+0000 7f081ea4d640 1 -- start start 2024-09-21T04:09:26.819 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.817+0000 7f081ea4d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f081810a3c0 0x7f081810df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.817+0000 7f081ea4d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f081810ad10 0x7f081810e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.817+0000 7f081ea4d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0818118900 0x7f0818116fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.817+0000 7f081ea4d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f081811e2c0 con 0x7f081810ad10 2024-09-21T04:09:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.817+0000 7f081ea4d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f081811e140 con 0x7f0818118900 2024-09-21T04:09:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.817+0000 7f081ea4d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f081811e440 con 0x7f081810a3c0 2024-09-21T04:09:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.817+0000 7f0817fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f081810a3c0 0x7f081810df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.817+0000 7f081cfea640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0818118900 0x7f0818116fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.817+0000 7f0817fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f081810a3c0 0x7f081810df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:44848/0 (socket says 172.21.15.64:44848) 2024-09-21T04:09:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.817+0000 7f0817fff640 1 -- 172.21.15.64:0/3577633071 learned_addr learned my addr 172.21.15.64:0/3577633071 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:26.820 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.817+0000 7f08177fe640 1 --2- 172.21.15.64:0/3577633071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f081810ad10 0x7f081810e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:26.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.818+0000 7f0817fff640 1 -- 172.21.15.64:0/3577633071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0818118900 msgr2=0x7f0818116fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:26.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.818+0000 7f0817fff640 1 --2- 172.21.15.64:0/3577633071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0818118900 0x7f0818116fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:26.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.818+0000 7f0817fff640 1 -- 172.21.15.64:0/3577633071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f081810ad10 msgr2=0x7f081810e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:26.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.818+0000 7f0817fff640 1 --2- 172.21.15.64:0/3577633071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f081810ad10 0x7f081810e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:26.821 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.818+0000 7f0817fff640 1 -- 172.21.15.64:0/3577633071 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081810ed90 con 0x7f081810a3c0 2024-09-21T04:09:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.818+0000 7f08177fe640 1 --2- 172.21.15.64:0/3577633071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f081810ad10 0x7f081810e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:09:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.819+0000 7f0817fff640 1 --2- 172.21.15.64:0/3577633071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f081810a3c0 0x7f081810df70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f080800e9c0 tx=0x7f080800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.826+0000 7f08157fa640 1 -- 172.21.15.64:0/3577633071 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080800cde0 con 0x7f081810a3c0 2024-09-21T04:09:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.826+0000 7f08157fa640 1 -- 172.21.15.64:0/3577633071 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08080035f0 con 0x7f081810a3c0 2024-09-21T04:09:26.829 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.826+0000 7f081ea4d640 1 -- 172.21.15.64:0/3577633071 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0818117640 con 0x7f081810a3c0 2024-09-21T04:09:26.830 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.826+0000 7f081ea4d640 1 -- 172.21.15.64:0/3577633071 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0818117b90 con 0x7f081810a3c0 2024-09-21T04:09:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.837+0000 7f08259c2740 1 Processor -- start 2024-09-21T04:09:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.838+0000 7f08157fa640 1 -- 172.21.15.64:0/3577633071 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808010690 con 0x7f081810a3c0 2024-09-21T04:09:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.838+0000 7f08157fa640 1 -- 172.21.15.64:0/3577633071 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f08080108f0 con 0x7f081810a3c0 2024-09-21T04:09:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.839+0000 7f08259c2740 1 -- start start 2024-09-21T04:09:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.840+0000 7f08157fa640 1 --2- 172.21.15.64:0/3577633071 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f08000b68b0 0x7f08000b8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.840+0000 7f081cfea640 1 --2- 172.21.15.64:0/3577633071 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f08000b68b0 0x7f08000b8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.840+0000 7f08157fa640 1 -- 172.21.15.64:0/3577633071 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f08080e8920 con 0x7f081810a3c0 2024-09-21T04:09:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.841+0000 7f08259c2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567494061b0 0x5567497abba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.841+0000 7f081cfea640 1 --2- 172.21.15.64:0/3577633071 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f08000b68b0 0x7f08000b8dc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0810004af0 tx=0x7f0810009940 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.841+0000 7f08259c2740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567493dd2f0 0x5567497ac0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.841+0000 7f08259c2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567493e54d0 0x5567497ac620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.841+0000 7f08259c2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5567497206f0 con 0x5567493e54d0 2024-09-21T04:09:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.841+0000 7f0817fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567494061b0 0x5567497abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.841+0000 7f0817fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567494061b0 0x5567497abba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:54376/0 (socket says 172.21.15.64:54376) 2024-09-21T04:09:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.841+0000 7f0817fff640 1 -- 172.21.15.64:0/3032175229 learned_addr learned my addr 172.21.15.64:0/3032175229 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:26.846 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.842+0000 7f08177fe640 1 --2- 172.21.15.64:0/3032175229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567493dd2f0 0x5567497ac0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.842+0000 7f081cfea640 1 --2- 172.21.15.64:0/3032175229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567493e54d0 0x5567497ac620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.842+0000 7f08259c2740 1 -- 172.21.15.64:0/3032175229 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55674970fb20 con 0x5567494061b0 2024-09-21T04:09:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.842+0000 7f08259c2740 1 -- 172.21.15.64:0/3032175229 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556749713930 con 0x5567493dd2f0 2024-09-21T04:09:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.842+0000 7f0817fff640 1 -- 172.21.15.64:0/3032175229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567493dd2f0 msgr2=0x5567497ac0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.842+0000 7f0817fff640 1 --2- 172.21.15.64:0/3032175229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567493dd2f0 0x5567497ac0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.842+0000 7f0817fff640 1 -- 172.21.15.64:0/3032175229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567493e54d0 msgr2=0x5567497ac620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.842+0000 7f0817fff640 1 --2- 172.21.15.64:0/3032175229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567493e54d0 0x5567497ac620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.843+0000 7f0817fff640 1 -- 172.21.15.64:0/3032175229 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567496d5530 con 0x5567494061b0 2024-09-21T04:09:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.843+0000 7f081cfea640 1 --2- 172.21.15.64:0/3032175229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567493e54d0 0x5567497ac620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:09:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.843+0000 7f0817fff640 1 --2- 172.21.15.64:0/3032175229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567494061b0 0x5567497abba0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f08080ae460 tx=0x7f08080ae270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cd9d9b691635c36 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.845+0000 7f07fe7fc640 1 -- 172.21.15.64:0/3032175229 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08080a8e60 con 0x5567494061b0 2024-09-21T04:09:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.845+0000 7f07fe7fc640 1 -- 172.21.15.64:0/3032175229 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08080b8180 con 0x5567494061b0 2024-09-21T04:09:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.845+0000 7f07fe7fc640 1 -- 172.21.15.64:0/3032175229 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08080ad430 con 0x5567494061b0 2024-09-21T04:09:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.846+0000 7f08259c2740 1 -- 172.21.15.64:0/3032175229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567494061b0 msgr2=0x5567497abba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.846+0000 7f08259c2740 1 --2- 172.21.15.64:0/3032175229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567494061b0 0x5567497abba0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f08080ae460 tx=0x7f08080ae270 comp rx=0 tx=0).stop 2024-09-21T04:09:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.848+0000 7f08259c2740 1 -- 172.21.15.64:0/3032175229 shutdown_connections 2024-09-21T04:09:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.848+0000 7f08259c2740 1 --2- 172.21.15.64:0/3032175229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567493e54d0 0x5567497ac620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.849+0000 7f08259c2740 1 --2- 172.21.15.64:0/3032175229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567493dd2f0 0x5567497ac0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.849+0000 7f08259c2740 1 --2- 172.21.15.64:0/3032175229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567494061b0 0x5567497abba0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.849+0000 7f08259c2740 1 -- 172.21.15.64:0/3032175229 >> 172.21.15.64:0/3032175229 conn(0x5567493fa300 msgr2=0x5567497a9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.850+0000 7f08259c2740 1 -- 172.21.15.64:0/3032175229 shutdown_connections 2024-09-21T04:09:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.850+0000 7f08259c2740 1 -- 172.21.15.64:0/3032175229 wait complete. 2024-09-21T04:09:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.850+0000 7f08259c2740 1 -- start start 2024-09-21T04:09:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.851+0000 7f08259c2740 1 Processor -- start 2024-09-21T04:09:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.851+0000 7f08259c2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567493dd2f0 0x5567497b7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.851+0000 7f08259c2740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567493e54d0 0x5567497b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.851+0000 7f08259c2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567494061b0 0x5567497c3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.851+0000 7f08259c2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556749647ab0 con 0x5567493dd2f0 2024-09-21T04:09:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.851+0000 7f08259c2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5567496b5730 con 0x5567494061b0 2024-09-21T04:09:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.851+0000 7f08259c2740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5567497ba320 con 0x5567493e54d0 2024-09-21T04:09:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.851+0000 7f08259c2740 10 client.0 _ll_register_callbacks cb 0x5567497096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:09:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.852+0000 7f0817fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567493dd2f0 0x5567497b7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.852+0000 7f0817fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567493dd2f0 0x5567497b7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58258/0 (socket says 172.21.15.64:58258) 2024-09-21T04:09:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.852+0000 7f0817fff640 1 -- 172.21.15.64:0/2849091744 learned_addr learned my addr 172.21.15.64:0/2849091744 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.853+0000 7f08177fe640 1 --2- 172.21.15.64:0/2849091744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567493e54d0 0x5567497b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.853+0000 7f081cfea640 1 --2- 172.21.15.64:0/2849091744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567494061b0 0x5567497c3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.853+0000 7f0817fff640 1 -- 172.21.15.64:0/2849091744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567493e54d0 msgr2=0x5567497b7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.853+0000 7f0817fff640 1 --2- 172.21.15.64:0/2849091744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567493e54d0 0x5567497b7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.853+0000 7f0817fff640 1 -- 172.21.15.64:0/2849091744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567494061b0 msgr2=0x5567497c3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.853+0000 7f0817fff640 1 --2- 172.21.15.64:0/2849091744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567494061b0 0x5567497c3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.853+0000 7f0817fff640 1 -- 172.21.15.64:0/2849091744 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556749471790 con 0x5567493dd2f0 2024-09-21T04:09:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.853+0000 7f081cfea640 1 --2- 172.21.15.64:0/2849091744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567494061b0 0x5567497c3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:09:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.854+0000 7f0817fff640 1 --2- 172.21.15.64:0/2849091744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567493dd2f0 0x5567497b7310 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f08080aad30 tx=0x7f08080b80b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4967266701da81 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.860+0000 7f07fbff7640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:09:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.861+0000 7f07fbff7640 1 -- 172.21.15.64:0/2849091744 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08080a8e60 con 0x5567493dd2f0 2024-09-21T04:09:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.861+0000 7f07fbff7640 1 -- 172.21.15.64:0/2849091744 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f08080adbc0 con 0x5567493dd2f0 2024-09-21T04:09:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.861+0000 7f08259c2740 1 -- 172.21.15.64:0/2849091744 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08080b8180 con 0x5567493dd2f0 2024-09-21T04:09:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.861+0000 7f07fbff7640 1 -- 172.21.15.64:0/2849091744 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f08080aa0b0 con 0x5567493dd2f0 2024-09-21T04:09:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.861+0000 7f08259c2740 10 client.16650 fetch_fsmap learned FSMap version 35 2024-09-21T04:09:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.861+0000 7f08259c2740 1 -- 172.21.15.64:0/2849091744 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5567494ea620 con 0x5567493dd2f0 2024-09-21T04:09:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.862+0000 7f07fbff7640 1 -- 172.21.15.64:0/2849091744 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08080b8180 con 0x5567493dd2f0 2024-09-21T04:09:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.864+0000 7f07fbff7640 1 -- 172.21.15.64:0/2849091744 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f08080b43d0 con 0x5567493dd2f0 2024-09-21T04:09:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.864+0000 7f08259c2740 10 client.16650 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T04:09:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.864+0000 7f08259c2740 10 client.16650 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:09:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.864+0000 7f08259c2740 20 client.16650 populate_metadata read hostname 'smithi064' 2024-09-21T04:09:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.864+0000 7f08259c2740 1 --2- 172.21.15.64:0/2849091744 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5567493c4d80 0x5567497c9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.864+0000 7f08259c2740 4 client.16650 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:09:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.864+0000 7f08259c2740 1 -- 172.21.15.64:0/2849091744 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556749722640 con 0x5567493c4d80 2024-09-21T04:09:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.865+0000 7f08177fe640 1 --2- 172.21.15.64:0/2849091744 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5567493c4d80 0x5567497c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.865+0000 7f08177fe640 1 --2- 172.21.15.64:0/2849091744 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5567493c4d80 0x5567497c9530 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=237fa1ee12c24082 server_cookie=bd24ccbe816b79c5 in_seq=0 out_seq=0 2024-09-21T04:09:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.868+0000 7f07fbff7640 10 client.16650 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.868+0000 7f07fbff7640 1 -- 172.21.15.64:0/2849091744 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556749722640 con 0x5567493c4d80 2024-09-21T04:09:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.868+0000 7f07fbff7640 10 client.16650 handle_command_reply: tid=0 2024-09-21T04:09:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.868+0000 7f08259c2740 1 client.16650 shutdown 2024-09-21T04:09:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.868+0000 7f08259c2740 1 -- 172.21.15.64:0/2849091744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567493dd2f0 msgr2=0x5567497b7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.869+0000 7f08259c2740 1 --2- 172.21.15.64:0/2849091744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567493dd2f0 0x5567497b7310 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f08080aad30 tx=0x7f08080b80b0 comp rx=0 tx=0).stop 2024-09-21T04:09:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.869+0000 7f08259c2740 1 -- 172.21.15.64:0/2849091744 shutdown_connections 2024-09-21T04:09:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.869+0000 7f08259c2740 1 --2- 172.21.15.64:0/2849091744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567494061b0 0x5567497c3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.869+0000 7f08259c2740 1 --2- 172.21.15.64:0/2849091744 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567493e54d0 0x5567497b7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.869+0000 7f08259c2740 1 --2- 172.21.15.64:0/2849091744 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5567493c4d80 0x5567497c9530 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.869+0000 7f08259c2740 1 --2- 172.21.15.64:0/2849091744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567493dd2f0 0x5567497b7310 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.869+0000 7f08259c2740 1 -- 172.21.15.64:0/2849091744 >> 172.21.15.64:0/2849091744 conn(0x5567493fa300 msgr2=0x5567497acbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.870+0000 7f07fbff7640 0 client.16650 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.870+0000 7f08259c2740 1 -- 172.21.15.64:0/2849091744 shutdown_connections 2024-09-21T04:09:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.870+0000 7f08259c2740 1 -- 172.21.15.64:0/2849091744 wait complete. 2024-09-21T04:09:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.870+0000 7f08259c2740 20 client.16650 trim_cache size 0 max 16384 2024-09-21T04:09:26.923 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:09:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.924+0000 7f08259c2740 1 Processor -- start 2024-09-21T04:09:26.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.925+0000 7f08259c2740 1 -- start start 2024-09-21T04:09:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.925+0000 7f08259c2740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567493dd2f0 0x5567497ad290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.925+0000 7f08259c2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567493e54d0 0x5567497c3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.925+0000 7f08259c2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567494061b0 0x5567497c3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.925+0000 7f08177fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567493e54d0 0x5567497c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.925+0000 7f08177fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567493e54d0 0x5567497c3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58272/0 (socket says 172.21.15.64:58272) 2024-09-21T04:09:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.925+0000 7f08177fe640 1 -- 172.21.15.64:0/3505285814 learned_addr learned my addr 172.21.15.64:0/3505285814 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.925+0000 7f08259c2740 1 -- 172.21.15.64:0/3505285814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f08040111d0 con 0x5567493e54d0 2024-09-21T04:09:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.925+0000 7f08259c2740 1 -- 172.21.15.64:0/3505285814 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f080400ea40 con 0x5567494061b0 2024-09-21T04:09:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.925+0000 7f08259c2740 1 -- 172.21.15.64:0/3505285814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f080400c940 con 0x5567493dd2f0 2024-09-21T04:09:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.926+0000 7f081cfea640 1 --2- 172.21.15.64:0/3505285814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567494061b0 0x5567497c3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.926+0000 7f0817fff640 1 --2- 172.21.15.64:0/3505285814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567493dd2f0 0x5567497ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.926+0000 7f08177fe640 1 -- 172.21.15.64:0/3505285814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567493dd2f0 msgr2=0x5567497ad290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.926+0000 7f08177fe640 1 --2- 172.21.15.64:0/3505285814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567493dd2f0 0x5567497ad290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.926+0000 7f08177fe640 1 -- 172.21.15.64:0/3505285814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567494061b0 msgr2=0x5567497c3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.926+0000 7f08177fe640 1 --2- 172.21.15.64:0/3505285814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567494061b0 0x5567497c3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.926+0000 7f08177fe640 1 -- 172.21.15.64:0/3505285814 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567497b6250 con 0x5567493e54d0 2024-09-21T04:09:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.926+0000 7f0817fff640 1 --2- 172.21.15.64:0/3505285814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567493dd2f0 0x5567497ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:09:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.926+0000 7f081cfea640 1 --2- 172.21.15.64:0/3505285814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567494061b0 0x5567497c3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:09:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.926+0000 7f08177fe640 1 --2- 172.21.15.64:0/3505285814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567493e54d0 0x5567497c3570 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f080c009fd0 tx=0x7f080c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62fbdac97dc40886 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.927+0000 7f07fdffb640 1 -- 172.21.15.64:0/3505285814 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c0154e0 con 0x5567493e54d0 2024-09-21T04:09:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.927+0000 7f07fdffb640 1 -- 172.21.15.64:0/3505285814 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f080c015b30 con 0x5567493e54d0 2024-09-21T04:09:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.927+0000 7f07fdffb640 1 -- 172.21.15.64:0/3505285814 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c01ebe0 con 0x5567493e54d0 2024-09-21T04:09:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.928+0000 7f08259c2740 1 -- 172.21.15.64:0/3505285814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567493e54d0 msgr2=0x5567497c3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.928+0000 7f08259c2740 1 --2- 172.21.15.64:0/3505285814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567493e54d0 0x5567497c3570 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f080c009fd0 tx=0x7f080c004770 comp rx=0 tx=0).stop 2024-09-21T04:09:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.929+0000 7f08259c2740 1 -- 172.21.15.64:0/3505285814 shutdown_connections 2024-09-21T04:09:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.929+0000 7f08259c2740 1 --2- 172.21.15.64:0/3505285814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567494061b0 0x5567497c3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.929+0000 7f08259c2740 1 --2- 172.21.15.64:0/3505285814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567493e54d0 0x5567497c3570 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.929+0000 7f08259c2740 1 --2- 172.21.15.64:0/3505285814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567493dd2f0 0x5567497ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.929+0000 7f08259c2740 1 -- 172.21.15.64:0/3505285814 >> 172.21.15.64:0/3505285814 conn(0x5567493fa300 msgr2=0x5567497acb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.929+0000 7f08259c2740 1 -- 172.21.15.64:0/3505285814 shutdown_connections 2024-09-21T04:09:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.929+0000 7f08259c2740 1 -- 172.21.15.64:0/3505285814 wait complete. 2024-09-21T04:09:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.930+0000 7f08259c2740 1 -- start start 2024-09-21T04:09:26.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.930+0000 7f08259c2740 1 Processor -- start 2024-09-21T04:09:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.930+0000 7f08259c2740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567493dd2f0 0x5567497bbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.930+0000 7f08259c2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567493e54d0 0x5567497bc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.930+0000 7f08259c2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567494061b0 0x5567497bc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.931+0000 7f08259c2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f080400cf00 con 0x5567493e54d0 2024-09-21T04:09:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.931+0000 7f08259c2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f080400e370 con 0x5567494061b0 2024-09-21T04:09:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.931+0000 7f08259c2740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f080400d5d0 con 0x5567493dd2f0 2024-09-21T04:09:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.931+0000 7f08259c2740 10 client.0 _ll_register_callbacks cb 0x7f08080b3050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:09:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.931+0000 7f08177fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567493e54d0 0x5567497bc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.931+0000 7f08177fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567493e54d0 0x5567497bc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58288/0 (socket says 172.21.15.64:58288) 2024-09-21T04:09:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.931+0000 7f08177fe640 1 -- 172.21.15.64:0/1826924534 learned_addr learned my addr 172.21.15.64:0/1826924534 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.931+0000 7f0817fff640 1 --2- 172.21.15.64:0/1826924534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567493dd2f0 0x5567497bbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.931+0000 7f081cfea640 1 --2- 172.21.15.64:0/1826924534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567494061b0 0x5567497bc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.931+0000 7f08177fe640 1 -- 172.21.15.64:0/1826924534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567493dd2f0 msgr2=0x5567497bbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.931+0000 7f08177fe640 1 --2- 172.21.15.64:0/1826924534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567493dd2f0 0x5567497bbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.931+0000 7f08177fe640 1 -- 172.21.15.64:0/1826924534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567494061b0 msgr2=0x5567497bc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.931+0000 7f08177fe640 1 --2- 172.21.15.64:0/1826924534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567494061b0 0x5567497bc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.932+0000 7f08177fe640 1 -- 172.21.15.64:0/1826924534 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556749831fb0 con 0x5567493e54d0 2024-09-21T04:09:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.932+0000 7f081cfea640 1 --2- 172.21.15.64:0/1826924534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567494061b0 0x5567497bc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:09:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.932+0000 7f08177fe640 1 --2- 172.21.15.64:0/1826924534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567493e54d0 0x5567497bc120 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f080c0154e0 tx=0x7f080c004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53890c37c6df8c79 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.932+0000 7f07fbff7640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:09:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.932+0000 7f07fbff7640 1 -- 172.21.15.64:0/1826924534 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c02c070 con 0x5567493e54d0 2024-09-21T04:09:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.932+0000 7f07fbff7640 1 -- 172.21.15.64:0/1826924534 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f080c01e800 con 0x5567493e54d0 2024-09-21T04:09:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.932+0000 7f08259c2740 1 -- 172.21.15.64:0/1826924534 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f080c015b30 con 0x5567493e54d0 2024-09-21T04:09:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.933+0000 7f07fbff7640 1 -- 172.21.15.64:0/1826924534 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f080c038050 con 0x5567493e54d0 2024-09-21T04:09:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.933+0000 7f08259c2740 10 client.16656 fetch_fsmap learned FSMap version 35 2024-09-21T04:09:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.933+0000 7f08259c2740 1 -- 172.21.15.64:0/1826924534 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5567498331b0 con 0x5567493e54d0 2024-09-21T04:09:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.933+0000 7f07fbff7640 1 -- 172.21.15.64:0/1826924534 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f080c015b30 con 0x5567493e54d0 2024-09-21T04:09:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.934+0000 7f07fbff7640 1 -- 172.21.15.64:0/1826924534 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f080c015ce0 con 0x5567493e54d0 2024-09-21T04:09:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.934+0000 7f08259c2740 10 client.16656 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T04:09:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.934+0000 7f08259c2740 10 client.16656 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:09:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.934+0000 7f08259c2740 20 client.16656 populate_metadata read hostname 'smithi064' 2024-09-21T04:09:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.934+0000 7f08259c2740 1 --2- 172.21.15.64:0/1826924534 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55674939a1c0 0x556749835810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.934+0000 7f08259c2740 4 client.16656 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:09:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.934+0000 7f08259c2740 1 -- 172.21.15.64:0/1826924534 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55674971e0f0 con 0x55674939a1c0 2024-09-21T04:09:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.934+0000 7f0817fff640 1 --2- 172.21.15.64:0/1826924534 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55674939a1c0 0x556749835810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.934+0000 7f07fbff7640 10 client.16656 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:26.935+0000 7f0817fff640 1 --2- 172.21.15.64:0/1826924534 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55674939a1c0 0x556749835810 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fce11bfaca675d15 server_cookie=6bb8c2752eebcb8f in_seq=0 out_seq=0 2024-09-21T04:09:27.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.569+0000 7fb87df07640 1 -- 172.21.15.64:0/1621600941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb878111f40 msgr2=0x7fb87810a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:27.573 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.569+0000 7fb87df07640 1 --2- 172.21.15.64:0/1621600941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb878111f40 0x7fb87810a3c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb868009900 tx=0x7fb86802a440 comp rx=0 tx=0).stop 2024-09-21T04:09:27.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.576+0000 7fb87df07640 1 -- 172.21.15.64:0/1621600941 shutdown_connections 2024-09-21T04:09:27.578 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.576+0000 7fb87df07640 1 --2- 172.21.15.64:0/1621600941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb87811cc30 0x7fb87811f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:27.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.576+0000 7fb87df07640 1 --2- 172.21.15.64:0/1621600941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb87810a9e0 0x7fb87810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:27.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.576+0000 7fb87df07640 1 --2- 172.21.15.64:0/1621600941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb878111f40 0x7fb87810a3c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:27.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.576+0000 7fb87df07640 1 -- 172.21.15.64:0/1621600941 >> 172.21.15.64:0/1621600941 conn(0x7fb878070ec0 msgr2=0x7fb8780712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:27.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.576+0000 7fb87df07640 1 -- 172.21.15.64:0/1621600941 shutdown_connections 2024-09-21T04:09:27.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.576+0000 7fb87df07640 1 -- 172.21.15.64:0/1621600941 wait complete. 2024-09-21T04:09:27.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.578+0000 7fb87df07640 1 Processor -- start 2024-09-21T04:09:27.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.578+0000 7fb87df07640 1 -- start start 2024-09-21T04:09:27.582 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.578+0000 7fb87df07640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb87810a9e0 0x7fb87811a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:27.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.578+0000 7fb87df07640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb878111f40 0x7fb878113680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:27.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.578+0000 7fb87df07640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb87811cc30 0x7fb878113bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:27.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.578+0000 7fb87df07640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb878122960 con 0x7fb87811cc30 2024-09-21T04:09:27.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.578+0000 7fb87df07640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb8781227e0 con 0x7fb87810a9e0 2024-09-21T04:09:27.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.578+0000 7fb87df07640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb878122ae0 con 0x7fb878111f40 2024-09-21T04:09:27.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.579+0000 7fb876ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb878111f40 0x7fb878113680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:27.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.579+0000 7fb876ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb878111f40 0x7fb878113680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:44912/0 (socket says 172.21.15.64:44912) 2024-09-21T04:09:27.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.579+0000 7fb876ffd640 1 -- 172.21.15.64:0/347879979 learned_addr learned my addr 172.21.15.64:0/347879979 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:27.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.579+0000 7fb8777fe640 1 --2- 172.21.15.64:0/347879979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb87810a9e0 0x7fb87811a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:27.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.579+0000 7fb877fff640 1 --2- 172.21.15.64:0/347879979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb87811cc30 0x7fb878113bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:27.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.581+0000 7fb876ffd640 1 -- 172.21.15.64:0/347879979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb87810a9e0 msgr2=0x7fb87811a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:27.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.581+0000 7fb876ffd640 1 --2- 172.21.15.64:0/347879979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb87810a9e0 0x7fb87811a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:27.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.581+0000 7fb876ffd640 1 -- 172.21.15.64:0/347879979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb87811cc30 msgr2=0x7fb878113bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:27.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.581+0000 7fb876ffd640 1 --2- 172.21.15.64:0/347879979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb87811cc30 0x7fb878113bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:27.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.581+0000 7fb876ffd640 1 -- 172.21.15.64:0/347879979 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8781143e0 con 0x7fb878111f40 2024-09-21T04:09:27.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.581+0000 7fb876ffd640 1 --2- 172.21.15.64:0/347879979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb878111f40 0x7fb878113680 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00c970 tx=0x7fb86c00ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:27.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.583+0000 7fb874ff9640 1 -- 172.21.15.64:0/347879979 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c004220 con 0x7fb878111f40 2024-09-21T04:09:27.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.583+0000 7fb87df07640 1 -- 172.21.15.64:0/347879979 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb878119080 con 0x7fb878111f40 2024-09-21T04:09:27.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.583+0000 7fb87df07640 1 -- 172.21.15.64:0/347879979 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8781195c0 con 0x7fb878111f40 2024-09-21T04:09:27.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.585+0000 7fb874ff9640 1 -- 172.21.15.64:0/347879979 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb86c0043c0 con 0x7fb878111f40 2024-09-21T04:09:27.587 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.585+0000 7fb874ff9640 1 -- 172.21.15.64:0/347879979 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c00f450 con 0x7fb878111f40 2024-09-21T04:09:27.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.586+0000 7fb87df07640 1 -- 172.21.15.64:0/347879979 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb87810adf0 con 0x7fb878111f40 2024-09-21T04:09:27.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.594+0000 7fb874ff9640 1 -- 172.21.15.64:0/347879979 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fb86c0079b0 con 0x7fb878111f40 2024-09-21T04:09:27.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.595+0000 7fb874ff9640 1 --2- 172.21.15.64:0/347879979 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb8600b6880 0x7fb8600b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:27.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.598+0000 7fb8777fe640 1 --2- 172.21.15.64:0/347879979 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb8600b6880 0x7fb8600b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:27.603 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.600+0000 7fb874ff9640 1 -- 172.21.15.64:0/347879979 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fb86c0e7880 con 0x7fb878111f40 2024-09-21T04:09:27.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.601+0000 7fb874ff9640 1 -- 172.21.15.64:0/347879979 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb86c014030 con 0x7fb878111f40 2024-09-21T04:09:27.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:27.606+0000 7fb8777fe640 1 --2- 172.21.15.64:0/347879979 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb8600b6880 0x7fb8600b8d90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb86802aa30 tx=0x7fb868000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:27.803 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:27.801+0000 7f07fbff7640 1 -- 172.21.15.64:0/1826924534 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55674971e0f0 con 0x55674939a1c0 2024-09-21T04:09:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:27.801+0000 7f07fbff7640 10 client.16656 handle_command_reply: tid=0 2024-09-21T04:09:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:27.801+0000 7f08259c2740 1 client.16656 shutdown 2024-09-21T04:09:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:27.803+0000 7f08259c2740 1 -- 172.21.15.64:0/1826924534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567493e54d0 msgr2=0x5567497bc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:27.803+0000 7f08259c2740 1 --2- 172.21.15.64:0/1826924534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567493e54d0 0x5567497bc120 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f080c0154e0 tx=0x7f080c004700 comp rx=0 tx=0).stop 2024-09-21T04:09:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:27.803+0000 7f08259c2740 1 -- 172.21.15.64:0/1826924534 shutdown_connections 2024-09-21T04:09:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:27.803+0000 7f08259c2740 1 --2- 172.21.15.64:0/1826924534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5567494061b0 0x5567497bc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:27.803+0000 7f08259c2740 1 --2- 172.21.15.64:0/1826924534 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55674939a1c0 0x556749835810 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:27.803+0000 7f08259c2740 1 --2- 172.21.15.64:0/1826924534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567493e54d0 0x5567497bc120 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:27.803+0000 7f08259c2740 1 --2- 172.21.15.64:0/1826924534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5567493dd2f0 0x5567497bbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:27.803+0000 7f08259c2740 1 -- 172.21.15.64:0/1826924534 >> 172.21.15.64:0/1826924534 conn(0x5567493fa300 msgr2=0x5567497b9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:27.803+0000 7f07fbff7640 0 client.16656 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:27.803+0000 7f08259c2740 1 -- 172.21.15.64:0/1826924534 shutdown_connections 2024-09-21T04:09:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:27.803+0000 7f08259c2740 1 -- 172.21.15.64:0/1826924534 wait complete. 2024-09-21T04:09:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:27.803+0000 7f08259c2740 20 client.16656 trim_cache size 0 max 16384 2024-09-21T04:09:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:09:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:09:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:09:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:09:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:27.807+0000 7f081ea4d640 1 -- 172.21.15.64:0/3577633071 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f08000b68b0 msgr2=0x7f08000b8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:27.807+0000 7f081ea4d640 1 --2- 172.21.15.64:0/3577633071 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f08000b68b0 0x7f08000b8dc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0810004af0 tx=0x7f0810009940 comp rx=0 tx=0).stop 2024-09-21T04:09:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:27.807+0000 7f081ea4d640 1 -- 172.21.15.64:0/3577633071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f081810a3c0 msgr2=0x7f081810df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:27.807+0000 7f081ea4d640 1 --2- 172.21.15.64:0/3577633071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f081810a3c0 0x7f081810df70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f080800e9c0 tx=0x7f080800ee80 comp rx=0 tx=0).stop 2024-09-21T04:09:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:27.807+0000 7f081ea4d640 1 -- 172.21.15.64:0/3577633071 shutdown_connections 2024-09-21T04:09:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:27.807+0000 7f081ea4d640 1 --2- 172.21.15.64:0/3577633071 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f08000b68b0 0x7f08000b8dc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:27.807+0000 7f081ea4d640 1 --2- 172.21.15.64:0/3577633071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0818118900 0x7f0818116fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:27.807+0000 7f081ea4d640 1 --2- 172.21.15.64:0/3577633071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f081810ad10 0x7f081810e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:27.807+0000 7f081ea4d640 1 --2- 172.21.15.64:0/3577633071 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f081810a3c0 0x7f081810df70 secure :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f080800e9c0 tx=0x7f080800ee80 comp rx=0 tx=0).stop 2024-09-21T04:09:27.810 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:27.807+0000 7f081ea4d640 1 -- 172.21.15.64:0/3577633071 >> 172.21.15.64:0/3577633071 conn(0x7f0818070c30 msgr2=0x7f0818118e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:27.842+0000 7f081ea4d640 1 -- 172.21.15.64:0/3577633071 shutdown_connections 2024-09-21T04:09:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:27.842+0000 7f081ea4d640 1 -- 172.21.15.64:0/3577633071 wait complete. 2024-09-21T04:09:28.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:28.107+0000 7fb87df07640 1 -- 172.21.15.64:0/347879979 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8781199f0 con 0x7fb878111f40 2024-09-21T04:09:28.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:28.108+0000 7fb874ff9640 1 -- 172.21.15.64:0/347879979 <== mon.1 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11913 (secure 0 0 0) 0x7fb86c0ade40 con 0x7fb878111f40 2024-09-21T04:09:28.111 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:09:28.111 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":35,"btime":"2024-09-21T04:09:04:909285+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14805,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/97995119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":97995119},{"type":"v1","addr":"172.21.15.64:6839","nonce":97995119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/390891068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":390891068},{"type":"v1","addr":"172.21.15.64:6841","nonce":390891068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:02:03.354744+0000","modified":"2024-09-21T04:09:04.185753+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24710,"mds_1":24680,"mds_2":24661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24634":{"gid":24634,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6839/782190424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782190424},{"type":"v1","addr":"172.21.15.131:6839","nonce":782190424}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24680":{"gid":24680,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24710":{"gid":24710,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6839/1319793636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1319793636},{"type":"v1","addr":"172.21.15.119:6839","nonce":1319793636}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24710,"qdb_cluster":[24710,24680,24661]},"id":1}]} 2024-09-21T04:09:28.111 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 35 2024-09-21T04:09:28.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:28.112+0000 7fb87df07640 1 -- 172.21.15.64:0/347879979 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb8600b6880 msgr2=0x7fb8600b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:28.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:28.112+0000 7fb87df07640 1 --2- 172.21.15.64:0/347879979 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb8600b6880 0x7fb8600b8d90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb86802aa30 tx=0x7fb868000bc0 comp rx=0 tx=0).stop 2024-09-21T04:09:28.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:28.112+0000 7fb87df07640 1 -- 172.21.15.64:0/347879979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb878111f40 msgr2=0x7fb878113680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:28.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:28.112+0000 7fb87df07640 1 --2- 172.21.15.64:0/347879979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb878111f40 0x7fb878113680 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00c970 tx=0x7fb86c00ce30 comp rx=0 tx=0).stop 2024-09-21T04:09:28.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:28.113+0000 7fb87df07640 1 -- 172.21.15.64:0/347879979 shutdown_connections 2024-09-21T04:09:28.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:28.113+0000 7fb87df07640 1 --2- 172.21.15.64:0/347879979 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb8600b6880 0x7fb8600b8d90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:28.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:28.113+0000 7fb87df07640 1 --2- 172.21.15.64:0/347879979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb87811cc30 0x7fb878113bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:28.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:28.113+0000 7fb87df07640 1 --2- 172.21.15.64:0/347879979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb878111f40 0x7fb878113680 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:28.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:28.113+0000 7fb87df07640 1 --2- 172.21.15.64:0/347879979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb87810a9e0 0x7fb87811a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:28.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:28.113+0000 7fb87df07640 1 -- 172.21.15.64:0/347879979 >> 172.21.15.64:0/347879979 conn(0x7fb878070ec0 msgr2=0x7fb87811e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:28.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:28.113+0000 7fb87df07640 1 -- 172.21.15.64:0/347879979 shutdown_connections 2024-09-21T04:09:28.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:28.113+0000 7fb87df07640 1 -- 172.21.15.64:0/347879979 wait complete. 2024-09-21T04:09:28.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:28 smithi131 ceph-mon[35193]: pgmap v484: 97 pgs: 97 active+clean; 646 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 327 op/s 2024-09-21T04:09:28.442 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:28 smithi064 ceph-mon[31180]: pgmap v484: 97 pgs: 97 active+clean; 646 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 327 op/s 2024-09-21T04:09:28.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:28 smithi119 ceph-mon[34811]: pgmap v484: 97 pgs: 97 active+clean; 646 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 327 op/s 2024-09-21T04:09:28.787 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T04:09:29.132 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:09:29.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:29 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/347879979' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:09:29.375 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:29 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/347879979' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:09:29.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:29 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/347879979' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:09:29.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.606+0000 7f2a7446e640 1 -- 172.21.15.64:0/3926379971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a6c102a60 msgr2=0x7f2a6c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:29.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.606+0000 7f2a7446e640 1 --2- 172.21.15.64:0/3926379971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a6c102a60 0x7f2a6c102e70 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009900 tx=0x7f2a6802a410 comp rx=0 tx=0).stop 2024-09-21T04:09:29.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.606+0000 7f2a7446e640 1 -- 172.21.15.64:0/3926379971 shutdown_connections 2024-09-21T04:09:29.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.606+0000 7f2a7446e640 1 --2- 172.21.15.64:0/3926379971 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a6c108cf0 0x7f2a6c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.609 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.606+0000 7f2a7446e640 1 --2- 172.21.15.64:0/3926379971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a6c1083a0 0x7f2a6c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.606+0000 7f2a7446e640 1 --2- 172.21.15.64:0/3926379971 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a6c102a60 0x7f2a6c102e70 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.606+0000 7f2a7446e640 1 -- 172.21.15.64:0/3926379971 >> 172.21.15.64:0/3926379971 conn(0x7f2a6c079730 msgr2=0x7f2a6c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:29.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.606+0000 7f2a7446e640 1 -- 172.21.15.64:0/3926379971 shutdown_connections 2024-09-21T04:09:29.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.606+0000 7f2a7446e640 1 -- 172.21.15.64:0/3926379971 wait complete. 2024-09-21T04:09:29.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.607+0000 7f2a7446e640 1 Processor -- start 2024-09-21T04:09:29.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.607+0000 7f2a7446e640 1 -- start start 2024-09-21T04:09:29.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.607+0000 7f2a7446e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a6c102a60 0x7f2a6c104250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:29.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.607+0000 7f2a7446e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a6c1083a0 0x7f2a6c104790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:29.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.607+0000 7f2a7446e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a6c108cf0 0x7f2a6c104cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:29.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.607+0000 7f2a7446e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c1129c0 con 0x7f2a6c108cf0 2024-09-21T04:09:29.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.607+0000 7f2a7446e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c112840 con 0x7f2a6c1083a0 2024-09-21T04:09:29.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.607+0000 7f2a7446e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c112b40 con 0x7f2a6c102a60 2024-09-21T04:09:29.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.608+0000 7f2a72a0b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a6c108cf0 0x7f2a6c104cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:29.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.608+0000 7f2a71a09640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a6c1083a0 0x7f2a6c104790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:29.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.608+0000 7f2a7220a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a6c102a60 0x7f2a6c104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:29.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.608+0000 7f2a71a09640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a6c1083a0 0x7f2a6c104790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:54440/0 (socket says 172.21.15.64:54440) 2024-09-21T04:09:29.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.608+0000 7f2a72a0b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a6c108cf0 0x7f2a6c104cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58322/0 (socket says 172.21.15.64:58322) 2024-09-21T04:09:29.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.608+0000 7f2a71a09640 1 -- 172.21.15.64:0/3776012775 learned_addr learned my addr 172.21.15.64:0/3776012775 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:29.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.608+0000 7f2a7220a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a6c102a60 0x7f2a6c104250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:44924/0 (socket says 172.21.15.64:44924) 2024-09-21T04:09:29.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.608+0000 7f2a72a0b640 1 -- 172.21.15.64:0/3776012775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a6c102a60 msgr2=0x7f2a6c104250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:29.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.608+0000 7f2a72a0b640 1 --2- 172.21.15.64:0/3776012775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a6c102a60 0x7f2a6c104250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.608+0000 7f2a72a0b640 1 -- 172.21.15.64:0/3776012775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a6c1083a0 msgr2=0x7f2a6c104790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:29.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.608+0000 7f2a72a0b640 1 --2- 172.21.15.64:0/3776012775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a6c1083a0 0x7f2a6c104790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.608+0000 7f2a72a0b640 1 -- 172.21.15.64:0/3776012775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a6c1a8740 con 0x7f2a6c108cf0 2024-09-21T04:09:29.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.608+0000 7f2a72a0b640 1 --2- 172.21.15.64:0/3776012775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a6c108cf0 0x7f2a6c104cd0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f2a6400be20 tx=0x7f2a6400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:29.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.608+0000 7f2a5f7fe640 1 -- 172.21.15.64:0/3776012775 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64019070 con 0x7f2a6c108cf0 2024-09-21T04:09:29.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.608+0000 7f2a5f7fe640 1 -- 172.21.15.64:0/3776012775 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a640148f0 con 0x7f2a6c108cf0 2024-09-21T04:09:29.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.608+0000 7f2a7446e640 1 -- 172.21.15.64:0/3776012775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a6c1a8a30 con 0x7f2a6c108cf0 2024-09-21T04:09:29.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.608+0000 7f2a5f7fe640 1 -- 172.21.15.64:0/3776012775 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64012840 con 0x7f2a6c108cf0 2024-09-21T04:09:29.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.608+0000 7f2a7446e640 1 -- 172.21.15.64:0/3776012775 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a6c1a8ff0 con 0x7f2a6c108cf0 2024-09-21T04:09:29.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.611+0000 7f2a5f7fe640 1 -- 172.21.15.64:0/3776012775 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f2a6401c070 con 0x7f2a6c108cf0 2024-09-21T04:09:29.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.612+0000 7f2a7a9e2740 1 Processor -- start 2024-09-21T04:09:29.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.612+0000 7f2a7a9e2740 1 -- start start 2024-09-21T04:09:29.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.612+0000 7f2a7a9e2740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564931b30070 0x564931ed76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:29.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.612+0000 7f2a7a9e2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564931b071b0 0x564931ed7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:29.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.612+0000 7f2a7a9e2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564931b0f390 0x564931ed8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:29.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.612+0000 7f2a7a9e2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564931e4dea0 con 0x564931b0f390 2024-09-21T04:09:29.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.612+0000 7f2a7a9e2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564931e3d2d0 con 0x564931b071b0 2024-09-21T04:09:29.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.612+0000 7f2a7a9e2740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564931e410e0 con 0x564931b30070 2024-09-21T04:09:29.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.612+0000 7f2a7220a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564931b071b0 0x564931ed7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:29.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.612+0000 7f2a7220a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564931b071b0 0x564931ed7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:54444/0 (socket says 172.21.15.64:54444) 2024-09-21T04:09:29.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.612+0000 7f2a7220a640 1 -- 172.21.15.64:0/4031441131 learned_addr learned my addr 172.21.15.64:0/4031441131 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:29.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.612+0000 7f2a71a09640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564931b0f390 0x564931ed8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:29.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.613+0000 7f2a71a09640 1 --2- 172.21.15.64:0/4031441131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564931b30070 0x564931ed76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:29.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.613+0000 7f2a7220a640 1 -- 172.21.15.64:0/4031441131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564931b30070 msgr2=0x564931ed76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:29.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.613+0000 7f2a7220a640 1 --2- 172.21.15.64:0/4031441131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564931b30070 0x564931ed76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.613+0000 7f2a5f7fe640 1 --2- 172.21.15.64:0/3776012775 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2a580b6680 0x7f2a580b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:29.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.613+0000 7f2a7220a640 1 -- 172.21.15.64:0/4031441131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564931b0f390 msgr2=0x564931ed8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:29.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.613+0000 7f2a7220a640 1 --2- 172.21.15.64:0/4031441131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564931b0f390 0x564931ed8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.613+0000 7f2a7220a640 1 -- 172.21.15.64:0/4031441131 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564931b9b650 con 0x564931b071b0 2024-09-21T04:09:29.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.613+0000 7f2a72a0b640 1 --2- 172.21.15.64:0/3776012775 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2a580b6680 0x7f2a580b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:29.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.613+0000 7f2a7220a640 1 --2- 172.21.15.64:0/4031441131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564931b071b0 0x564931ed7be0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009900 tx=0x7f2a68032df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c80eafa858251c42 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:29.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.613+0000 7f2a5f7fe640 1 -- 172.21.15.64:0/3776012775 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f2a640e6de0 con 0x7f2a6c108cf0 2024-09-21T04:09:29.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.613+0000 7f2a5cd9a640 1 -- 172.21.15.64:0/4031441131 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68040070 con 0x564931b071b0 2024-09-21T04:09:29.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.613+0000 7f2a5cd9a640 1 -- 172.21.15.64:0/4031441131 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a68005db0 con 0x564931b071b0 2024-09-21T04:09:29.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.613+0000 7f2a5cd9a640 1 -- 172.21.15.64:0/4031441131 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68035040 con 0x564931b071b0 2024-09-21T04:09:29.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.614+0000 7f2a7a9e2740 1 -- 172.21.15.64:0/4031441131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564931b071b0 msgr2=0x564931ed7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:29.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.614+0000 7f2a72a0b640 1 --2- 172.21.15.64:0/3776012775 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2a580b6680 0x7f2a580b8b90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2a64017020 tx=0x7f2a6400cf00 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:29.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.614+0000 7f2a7a9e2740 1 --2- 172.21.15.64:0/4031441131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564931b071b0 0x564931ed7be0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009900 tx=0x7f2a68032df0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.614+0000 7f2a7a9e2740 1 -- 172.21.15.64:0/4031441131 shutdown_connections 2024-09-21T04:09:29.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.614+0000 7f2a7a9e2740 1 --2- 172.21.15.64:0/4031441131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564931b0f390 0x564931ed8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.614+0000 7f2a7a9e2740 1 --2- 172.21.15.64:0/4031441131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564931b071b0 0x564931ed7be0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.614+0000 7f2a7a9e2740 1 --2- 172.21.15.64:0/4031441131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564931b30070 0x564931ed76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.614+0000 7f2a7a9e2740 1 -- 172.21.15.64:0/4031441131 >> 172.21.15.64:0/4031441131 conn(0x564931b24460 msgr2=0x564931ed5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:29.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.614+0000 7f2a7a9e2740 1 -- 172.21.15.64:0/4031441131 shutdown_connections 2024-09-21T04:09:29.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.614+0000 7f2a7a9e2740 1 -- 172.21.15.64:0/4031441131 wait complete. 2024-09-21T04:09:29.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.615+0000 7f2a7a9e2740 1 -- start start 2024-09-21T04:09:29.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.615+0000 7f2a7a9e2740 1 Processor -- start 2024-09-21T04:09:29.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.615+0000 7f2a7a9e2740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564931b071b0 0x564931ee2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:29.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.615+0000 7f2a7a9e2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564931b0f390 0x564931ee3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:29.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.615+0000 7f2a7a9e2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564931b30070 0x564931eef140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:29.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.615+0000 7f2a7a9e2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564931c9e700 con 0x564931b30070 2024-09-21T04:09:29.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.615+0000 7f2a7a9e2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564931dd8160 con 0x564931b0f390 2024-09-21T04:09:29.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.615+0000 7f2a7a9e2740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564931ee5e20 con 0x564931b071b0 2024-09-21T04:09:29.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.615+0000 7f2a7a9e2740 10 client.0 _ll_register_callbacks cb 0x564931df19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:09:29.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.615+0000 7f2a7220a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564931b0f390 0x564931ee3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:29.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.615+0000 7f2a71a09640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564931b30070 0x564931eef140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:29.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.615+0000 7f2a7220a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564931b0f390 0x564931ee3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:54450/0 (socket says 172.21.15.64:54450) 2024-09-21T04:09:29.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.615+0000 7f2a7220a640 1 -- 172.21.15.64:0/3457188843 learned_addr learned my addr 172.21.15.64:0/3457188843 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:29.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.615+0000 7f2a71a09640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564931b30070 0x564931eef140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58354/0 (socket says 172.21.15.64:58354) 2024-09-21T04:09:29.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.616+0000 7f2a71a09640 1 --2- 172.21.15.64:0/3457188843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564931b071b0 0x564931ee2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:29.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.616+0000 7f2a7220a640 1 -- 172.21.15.64:0/3457188843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564931b071b0 msgr2=0x564931ee2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:29.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.616+0000 7f2a7220a640 1 --2- 172.21.15.64:0/3457188843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564931b071b0 0x564931ee2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.616+0000 7f2a7220a640 1 -- 172.21.15.64:0/3457188843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564931b30070 msgr2=0x564931eef140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:29.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.616+0000 7f2a7220a640 1 --2- 172.21.15.64:0/3457188843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564931b30070 0x564931eef140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.616+0000 7f2a7220a640 1 -- 172.21.15.64:0/3457188843 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564931e05c90 con 0x564931b0f390 2024-09-21T04:09:29.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.616+0000 7f2a71a09640 1 --2- 172.21.15.64:0/3457188843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564931b30070 0x564931eef140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:09:29.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.616+0000 7f2a7220a640 1 --2- 172.21.15.64:0/3457188843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564931b0f390 0x564931ee3280 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2a6802a930 tx=0x7f2a680051c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42ee7e35e3872737 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:29.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.616+0000 7f2a55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:09:29.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.616+0000 7f2a55ffb640 1 -- 172.21.15.64:0/3457188843 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68035070 con 0x564931b0f390 2024-09-21T04:09:29.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.616+0000 7f2a55ffb640 1 -- 172.21.15.64:0/3457188843 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f2a68004030 con 0x564931b0f390 2024-09-21T04:09:29.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.616+0000 7f2a7a9e2740 1 -- 172.21.15.64:0/3457188843 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a68005db0 con 0x564931b0f390 2024-09-21T04:09:29.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.617+0000 7f2a55ffb640 1 -- 172.21.15.64:0/3457188843 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2a68055050 con 0x564931b0f390 2024-09-21T04:09:29.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.617+0000 7f2a7a9e2740 10 client.26369 fetch_fsmap learned FSMap version 35 2024-09-21T04:09:29.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.617+0000 7f2a7a9e2740 1 -- 172.21.15.64:0/3457188843 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564931ef1b40 con 0x564931b0f390 2024-09-21T04:09:29.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.617+0000 7f2a55ffb640 1 -- 172.21.15.64:0/3457188843 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a68005db0 con 0x564931b0f390 2024-09-21T04:09:29.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.617+0000 7f2a55ffb640 1 -- 172.21.15.64:0/3457188843 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f2a68004750 con 0x564931b0f390 2024-09-21T04:09:29.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.617+0000 7f2a7a9e2740 10 client.26369 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T04:09:29.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.617+0000 7f2a7a9e2740 10 client.26369 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:09:29.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.617+0000 7f2a7a9e2740 20 client.26369 populate_metadata read hostname 'smithi064' 2024-09-21T04:09:29.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.617+0000 7f2a7a9e2740 1 --2- 172.21.15.64:0/3457188843 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x564931aee9f0 0x564931ef4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:29.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.617+0000 7f2a7a9e2740 4 client.26369 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:09:29.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.617+0000 7f2a7a9e2740 1 -- 172.21.15.64:0/3457188843 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564931e4fdf0 con 0x564931aee9f0 2024-09-21T04:09:29.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.617+0000 7f2a71a09640 1 --2- 172.21.15.64:0/3457188843 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x564931aee9f0 0x564931ef4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:29.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.618+0000 7f2a55ffb640 10 client.26369 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:09:29.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.618+0000 7f2a71a09640 1 --2- 172.21.15.64:0/3457188843 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x564931aee9f0 0x564931ef4ea0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6fa9b7199fc2b4a7 server_cookie=a184e115af99c395 in_seq=0 out_seq=0 2024-09-21T04:09:29.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.619+0000 7f2a55ffb640 1 -- 172.21.15.64:0/3457188843 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564931e4fdf0 con 0x564931aee9f0 2024-09-21T04:09:29.624 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.619+0000 7f2a55ffb640 10 client.26369 handle_command_reply: tid=0 2024-09-21T04:09:29.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.619+0000 7f2a7a9e2740 1 client.26369 shutdown 2024-09-21T04:09:29.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.619+0000 7f2a7a9e2740 1 -- 172.21.15.64:0/3457188843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564931b0f390 msgr2=0x564931ee3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:29.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.619+0000 7f2a7a9e2740 1 --2- 172.21.15.64:0/3457188843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564931b0f390 0x564931ee3280 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2a6802a930 tx=0x7f2a680051c0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.619+0000 7f2a7a9e2740 1 -- 172.21.15.64:0/3457188843 shutdown_connections 2024-09-21T04:09:29.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.619+0000 7f2a7a9e2740 1 --2- 172.21.15.64:0/3457188843 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x564931aee9f0 0x564931ef4ea0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.619+0000 7f2a7a9e2740 1 --2- 172.21.15.64:0/3457188843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564931b30070 0x564931eef140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.619+0000 7f2a7a9e2740 1 --2- 172.21.15.64:0/3457188843 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564931b0f390 0x564931ee3280 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.619+0000 7f2a55ffb640 0 client.26369 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:09:29.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.619+0000 7f2a7a9e2740 1 --2- 172.21.15.64:0/3457188843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564931b071b0 0x564931ee2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.619+0000 7f2a7a9e2740 1 -- 172.21.15.64:0/3457188843 >> 172.21.15.64:0/3457188843 conn(0x564931b24460 msgr2=0x564931eda830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:29.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.619+0000 7f2a7a9e2740 1 -- 172.21.15.64:0/3457188843 shutdown_connections 2024-09-21T04:09:29.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.619+0000 7f2a7a9e2740 1 -- 172.21.15.64:0/3457188843 wait complete. 2024-09-21T04:09:29.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.619+0000 7f2a7a9e2740 20 client.26369 trim_cache size 0 max 16384 2024-09-21T04:09:29.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.633+0000 7f2a7a9e2740 1 Processor -- start 2024-09-21T04:09:29.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.633+0000 7f2a7a9e2740 1 -- start start 2024-09-21T04:09:29.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.633+0000 7f2a7a9e2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564931b071b0 0x564931ee3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:29.636 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.633+0000 7f2a7a9e2740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564931b0f390 0x564931eef0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:29.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.633+0000 7f2a7a9e2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564931b30070 0x564931edff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:29.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.633+0000 7f2a7a9e2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c1b43c0 con 0x564931b30070 2024-09-21T04:09:29.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.633+0000 7f2a7a9e2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c1b1c30 con 0x564931b071b0 2024-09-21T04:09:29.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.633+0000 7f2a7a9e2740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c1afa80 con 0x564931b0f390 2024-09-21T04:09:29.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.633+0000 7f2a71a09640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564931b30070 0x564931edff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:29.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.633+0000 7f2a71a09640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564931b30070 0x564931edff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58358/0 (socket says 172.21.15.64:58358) 2024-09-21T04:09:29.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.633+0000 7f2a71a09640 1 -- 172.21.15.64:0/1276183159 learned_addr learned my addr 172.21.15.64:0/1276183159 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:29.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.633+0000 7f2a71a09640 1 --2- 172.21.15.64:0/1276183159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564931b071b0 0x564931ee3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:29.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.633+0000 7f2a7220a640 1 --2- 172.21.15.64:0/1276183159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564931b0f390 0x564931eef0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:29.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.634+0000 7f2a71a09640 1 -- 172.21.15.64:0/1276183159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564931b0f390 msgr2=0x564931eef0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:29.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.634+0000 7f2a71a09640 1 --2- 172.21.15.64:0/1276183159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564931b0f390 0x564931eef0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.634+0000 7f2a71a09640 1 -- 172.21.15.64:0/1276183159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564931b071b0 msgr2=0x564931ee3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:29.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.634+0000 7f2a71a09640 1 --2- 172.21.15.64:0/1276183159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564931b071b0 0x564931ee3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.634+0000 7f2a71a09640 1 -- 172.21.15.64:0/1276183159 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564931ed57c0 con 0x564931b30070 2024-09-21T04:09:29.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.634+0000 7f2a71a09640 1 --2- 172.21.15.64:0/1276183159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564931b30070 0x564931edff30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2a600058c0 tx=0x7f2a60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a55f8819b06c56f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:29.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.634+0000 7f2a57fff640 1 -- 172.21.15.64:0/1276183159 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a60004250 con 0x564931b30070 2024-09-21T04:09:29.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.634+0000 7f2a57fff640 1 -- 172.21.15.64:0/1276183159 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a600043f0 con 0x564931b30070 2024-09-21T04:09:29.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.634+0000 7f2a57fff640 1 -- 172.21.15.64:0/1276183159 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a60023670 con 0x564931b30070 2024-09-21T04:09:29.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.635+0000 7f2a7a9e2740 1 -- 172.21.15.64:0/1276183159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564931b30070 msgr2=0x564931edff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:29.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.635+0000 7f2a7a9e2740 1 --2- 172.21.15.64:0/1276183159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564931b30070 0x564931edff30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2a600058c0 tx=0x7f2a60004730 comp rx=0 tx=0).stop 2024-09-21T04:09:29.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.635+0000 7f2a7a9e2740 1 -- 172.21.15.64:0/1276183159 shutdown_connections 2024-09-21T04:09:29.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.635+0000 7f2a7a9e2740 1 --2- 172.21.15.64:0/1276183159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564931b30070 0x564931edff30 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.635+0000 7f2a7a9e2740 1 --2- 172.21.15.64:0/1276183159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564931b0f390 0x564931eef0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.635+0000 7f2a7a9e2740 1 --2- 172.21.15.64:0/1276183159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564931b071b0 0x564931ee3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.635+0000 7f2a7a9e2740 1 -- 172.21.15.64:0/1276183159 >> 172.21.15.64:0/1276183159 conn(0x564931b24460 msgr2=0x564931ee2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:29.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.635+0000 7f2a7a9e2740 1 -- 172.21.15.64:0/1276183159 shutdown_connections 2024-09-21T04:09:29.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.635+0000 7f2a7a9e2740 1 -- 172.21.15.64:0/1276183159 wait complete. 2024-09-21T04:09:29.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.635+0000 7f2a7a9e2740 1 -- start start 2024-09-21T04:09:29.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.636+0000 7f2a7a9e2740 1 Processor -- start 2024-09-21T04:09:29.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.636+0000 7f2a7a9e2740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564931b071b0 0x564931ef8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:29.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.636+0000 7f2a7a9e2740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564931b0f390 0x564931ef5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:29.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.636+0000 7f2a7a9e2740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564931ac3da0 0x564931ef6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:29.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.636+0000 7f2a7a9e2740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c1b00f0 con 0x564931ac3da0 2024-09-21T04:09:29.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.636+0000 7f2a7a9e2740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c1b1560 con 0x564931b0f390 2024-09-21T04:09:29.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.636+0000 7f2a7a9e2740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c1b07c0 con 0x564931b071b0 2024-09-21T04:09:29.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.636+0000 7f2a7a9e2740 10 client.0 _ll_register_callbacks cb 0x7f2a6000b860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:09:29.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.636+0000 7f2a71a09640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564931ac3da0 0x564931ef6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:29.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.636+0000 7f2a7220a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564931b0f390 0x564931ef5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:29.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.636+0000 7f2a7220a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564931b0f390 0x564931ef5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:54458/0 (socket says 172.21.15.64:54458) 2024-09-21T04:09:29.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.636+0000 7f2a7220a640 1 -- 172.21.15.64:0/982679514 learned_addr learned my addr 172.21.15.64:0/982679514 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:29.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.637+0000 7f2a71a09640 1 --2- 172.21.15.64:0/982679514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564931b071b0 0x564931ef8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:29.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.637+0000 7f2a7220a640 1 -- 172.21.15.64:0/982679514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564931b071b0 msgr2=0x564931ef8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:29.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.637+0000 7f2a7220a640 1 --2- 172.21.15.64:0/982679514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564931b071b0 0x564931ef8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.637+0000 7f2a7220a640 1 -- 172.21.15.64:0/982679514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564931ac3da0 msgr2=0x564931ef6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:29.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.637+0000 7f2a7220a640 1 --2- 172.21.15.64:0/982679514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564931ac3da0 0x564931ef6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.637+0000 7f2a7220a640 1 -- 172.21.15.64:0/982679514 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564931f5e620 con 0x564931b0f390 2024-09-21T04:09:29.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.637+0000 7f2a71a09640 1 --2- 172.21.15.64:0/982679514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564931ac3da0 0x564931ef6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:09:29.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.637+0000 7f2a7220a640 1 --2- 172.21.15.64:0/982679514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564931b0f390 0x564931ef5d60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2a68004ba0 tx=0x7f2a680048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ebd1f4cc9b02680 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:29.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.637+0000 7f2a55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:09:29.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.637+0000 7f2a55ffb640 1 -- 172.21.15.64:0/982679514 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68035070 con 0x564931b0f390 2024-09-21T04:09:29.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.637+0000 7f2a55ffb640 1 -- 172.21.15.64:0/982679514 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f2a68004fa0 con 0x564931b0f390 2024-09-21T04:09:29.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.637+0000 7f2a7a9e2740 1 -- 172.21.15.64:0/982679514 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a600043f0 con 0x564931b0f390 2024-09-21T04:09:29.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.637+0000 7f2a55ffb640 1 -- 172.21.15.64:0/982679514 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2a68067050 con 0x564931b0f390 2024-09-21T04:09:29.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.637+0000 7f2a7a9e2740 10 client.26375 fetch_fsmap learned FSMap version 35 2024-09-21T04:09:29.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.637+0000 7f2a7a9e2740 1 -- 172.21.15.64:0/982679514 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564931f5f740 con 0x564931b0f390 2024-09-21T04:09:29.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.638+0000 7f2a55ffb640 1 -- 172.21.15.64:0/982679514 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a600043f0 con 0x564931b0f390 2024-09-21T04:09:29.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.638+0000 7f2a55ffb640 1 -- 172.21.15.64:0/982679514 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f2a68032120 con 0x564931b0f390 2024-09-21T04:09:29.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.638+0000 7f2a7a9e2740 10 client.26375 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T04:09:29.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.638+0000 7f2a7a9e2740 10 client.26375 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:09:29.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.638+0000 7f2a7a9e2740 20 client.26375 populate_metadata read hostname 'smithi064' 2024-09-21T04:09:29.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.638+0000 7f2a7a9e2740 1 --2- 172.21.15.64:0/982679514 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x564931f5fe20 0x564931f62240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:29.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.638+0000 7f2a7a9e2740 4 client.26375 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:09:29.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.638+0000 7f2a7a9e2740 1 -- 172.21.15.64:0/982679514 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564931e4b8a0 con 0x564931f5fe20 2024-09-21T04:09:29.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.638+0000 7f2a71a09640 1 --2- 172.21.15.64:0/982679514 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x564931f5fe20 0x564931f62240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:29.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.639+0000 7f2a55ffb640 10 client.26375 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:09:29.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.639+0000 7f2a71a09640 1 --2- 172.21.15.64:0/982679514 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x564931f5fe20 0x564931f62240 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c1e5ccdaa4445e0a server_cookie=2425df86d45518c1 in_seq=0 out_seq=0 2024-09-21T04:09:29.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.639+0000 7f2a55ffb640 1 -- 172.21.15.64:0/982679514 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564931e4b8a0 con 0x564931f5fe20 2024-09-21T04:09:29.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.639+0000 7f2a55ffb640 10 client.26375 handle_command_reply: tid=0 2024-09-21T04:09:29.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.639+0000 7f2a7a9e2740 1 client.26375 shutdown 2024-09-21T04:09:29.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.639+0000 7f2a7a9e2740 1 -- 172.21.15.64:0/982679514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564931b0f390 msgr2=0x564931ef5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:29.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.639+0000 7f2a7a9e2740 1 --2- 172.21.15.64:0/982679514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564931b0f390 0x564931ef5d60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2a68004ba0 tx=0x7f2a680048f0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.639+0000 7f2a7a9e2740 1 -- 172.21.15.64:0/982679514 shutdown_connections 2024-09-21T04:09:29.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.639+0000 7f2a7a9e2740 1 --2- 172.21.15.64:0/982679514 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x564931f5fe20 0x564931f62240 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.639+0000 7f2a7a9e2740 1 --2- 172.21.15.64:0/982679514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564931ac3da0 0x564931ef6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.639+0000 7f2a7a9e2740 1 --2- 172.21.15.64:0/982679514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564931b0f390 0x564931ef5d60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.639+0000 7f2a55ffb640 0 client.26375 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:09:29.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.639+0000 7f2a7a9e2740 1 --2- 172.21.15.64:0/982679514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564931b071b0 0x564931ef8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.639+0000 7f2a7a9e2740 1 -- 172.21.15.64:0/982679514 >> 172.21.15.64:0/982679514 conn(0x564931b24460 msgr2=0x564931ed86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:29.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.639+0000 7f2a7a9e2740 1 -- 172.21.15.64:0/982679514 shutdown_connections 2024-09-21T04:09:29.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.639+0000 7f2a7a9e2740 1 -- 172.21.15.64:0/982679514 wait complete. 2024-09-21T04:09:29.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.639+0000 7f2a7a9e2740 20 client.26375 trim_cache size 0 max 16384 2024-09-21T04:09:29.646 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:09:29.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.640+0000 7f2a7446e640 1 -- 172.21.15.64:0/3776012775 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2a580b6680 msgr2=0x7f2a580b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:29.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.640+0000 7f2a7446e640 1 --2- 172.21.15.64:0/3776012775 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2a580b6680 0x7f2a580b8b90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2a64017020 tx=0x7f2a6400cf00 comp rx=0 tx=0).stop 2024-09-21T04:09:29.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.640+0000 7f2a7446e640 1 -- 172.21.15.64:0/3776012775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a6c108cf0 msgr2=0x7f2a6c104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:29.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.640+0000 7f2a7446e640 1 --2- 172.21.15.64:0/3776012775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a6c108cf0 0x7f2a6c104cd0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f2a6400be20 tx=0x7f2a6400bf10 comp rx=0 tx=0).stop 2024-09-21T04:09:29.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.640+0000 7f2a7446e640 1 -- 172.21.15.64:0/3776012775 shutdown_connections 2024-09-21T04:09:29.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.640+0000 7f2a7446e640 1 --2- 172.21.15.64:0/3776012775 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2a580b6680 0x7f2a580b8b90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.640+0000 7f2a7446e640 1 --2- 172.21.15.64:0/3776012775 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a6c108cf0 0x7f2a6c104cd0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.640+0000 7f2a7446e640 1 --2- 172.21.15.64:0/3776012775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a6c1083a0 0x7f2a6c104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.640+0000 7f2a7446e640 1 --2- 172.21.15.64:0/3776012775 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a6c102a60 0x7f2a6c104250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:29.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.640+0000 7f2a7446e640 1 -- 172.21.15.64:0/3776012775 >> 172.21.15.64:0/3776012775 conn(0x7f2a6c079730 msgr2=0x7f2a6c1066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:29.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.640+0000 7f2a7446e640 1 -- 172.21.15.64:0/3776012775 shutdown_connections 2024-09-21T04:09:29.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:29.640+0000 7f2a7446e640 1 -- 172.21.15.64:0/3776012775 wait complete. 2024-09-21T04:09:30.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:30 smithi131 ceph-mon[35193]: pgmap v485: 97 pgs: 97 active+clean; 658 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 347 op/s 2024-09-21T04:09:30.437 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:30 smithi064 ceph-mon[31180]: pgmap v485: 97 pgs: 97 active+clean; 658 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 347 op/s 2024-09-21T04:09:30.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:30 smithi119 ceph-mon[34811]: pgmap v485: 97 pgs: 97 active+clean; 658 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 347 op/s 2024-09-21T04:09:30.939 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T04:09:31.174 INFO:tasks.workunit.client.0.smithi064.stdout: 107 1408684 331 1023344 348 853204 1257 2024-09-21T04:09:31.676 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:09:32.118 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:32 smithi064 ceph-mon[31180]: pgmap v486: 97 pgs: 97 active+clean; 661 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 337 op/s 2024-09-21T04:09:32.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.253+0000 7f8cc84f3640 1 -- 172.21.15.64:0/1566965239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cc0105320 msgr2=0x7f8cc010d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:32.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.253+0000 7f8cc84f3640 1 --2- 172.21.15.64:0/1566965239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cc0105320 0x7f8cc010d870 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009860 tx=0x7f8cb002a340 comp rx=0 tx=0).stop 2024-09-21T04:09:32.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.253+0000 7f8cc84f3640 1 -- 172.21.15.64:0/1566965239 shutdown_connections 2024-09-21T04:09:32.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.253+0000 7f8cc84f3640 1 --2- 172.21.15.64:0/1566965239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cc010de30 0x7f8cc0110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.253+0000 7f8cc84f3640 1 --2- 172.21.15.64:0/1566965239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cc0105320 0x7f8cc010d870 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.253+0000 7f8cc84f3640 1 --2- 172.21.15.64:0/1566965239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc01049d0 0x7f8cc0104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.253+0000 7f8cc84f3640 1 -- 172.21.15.64:0/1566965239 >> 172.21.15.64:0/1566965239 conn(0x7f8cc0079740 msgr2=0x7f8cc0079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:32.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.253+0000 7f8cc84f3640 1 -- 172.21.15.64:0/1566965239 shutdown_connections 2024-09-21T04:09:32.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.253+0000 7f8cc84f3640 1 -- 172.21.15.64:0/1566965239 wait complete. 2024-09-21T04:09:32.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.254+0000 7f8cc84f3640 1 Processor -- start 2024-09-21T04:09:32.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.254+0000 7f8cc84f3640 1 -- start start 2024-09-21T04:09:32.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.255+0000 7f8cc84f3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc01049d0 0x7f8cc0102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:32.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.255+0000 7f8cc84f3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cc0105320 0x7f8cc0102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:32.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.255+0000 7f8cc84f3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cc010de30 0x7f8cc0103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:32.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.255+0000 7f8cc84f3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc0113a90 con 0x7f8cc01049d0 2024-09-21T04:09:32.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.255+0000 7f8cc84f3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc0113910 con 0x7f8cc0105320 2024-09-21T04:09:32.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.255+0000 7f8cc84f3640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc0113c10 con 0x7f8cc010de30 2024-09-21T04:09:32.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.255+0000 7f8cc5a8e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cc0105320 0x7f8cc0102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:32.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.255+0000 7f8cc5a8e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cc0105320 0x7f8cc0102c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:50320/0 (socket says 172.21.15.64:50320) 2024-09-21T04:09:32.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.255+0000 7f8cc5a8e640 1 -- 172.21.15.64:0/1512983275 learned_addr learned my addr 172.21.15.64:0/1512983275 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:32.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.255+0000 7f8cc628f640 1 --2- 172.21.15.64:0/1512983275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc01049d0 0x7f8cc0102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:32.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.255+0000 7f8cc6a90640 1 --2- 172.21.15.64:0/1512983275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cc010de30 0x7f8cc0103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:32.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.255+0000 7f8cc5a8e640 1 -- 172.21.15.64:0/1512983275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cc010de30 msgr2=0x7f8cc0103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:32.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.255+0000 7f8cc5a8e640 1 --2- 172.21.15.64:0/1512983275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cc010de30 0x7f8cc0103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.255+0000 7f8cc5a8e640 1 -- 172.21.15.64:0/1512983275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc01049d0 msgr2=0x7f8cc0102710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:32.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.255+0000 7f8cc5a8e640 1 --2- 172.21.15.64:0/1512983275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc01049d0 0x7f8cc0102710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.255+0000 7f8cc5a8e640 1 -- 172.21.15.64:0/1512983275 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc01a8780 con 0x7f8cc0105320 2024-09-21T04:09:32.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.256+0000 7f8cc5a8e640 1 --2- 172.21.15.64:0/1512983275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cc0105320 0x7f8cc0102c50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0035040 tx=0x7f8cb0032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:32.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.256+0000 7f8cb77fe640 1 -- 172.21.15.64:0/1512983275 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0040020 con 0x7f8cc0105320 2024-09-21T04:09:32.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.256+0000 7f8cb77fe640 1 -- 172.21.15.64:0/1512983275 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cb0030e10 con 0x7f8cc0105320 2024-09-21T04:09:32.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.256+0000 7f8cb77fe640 1 -- 172.21.15.64:0/1512983275 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0037040 con 0x7f8cc0105320 2024-09-21T04:09:32.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.256+0000 7f8cc84f3640 1 -- 172.21.15.64:0/1512983275 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cc01a8a10 con 0x7f8cc0105320 2024-09-21T04:09:32.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.257+0000 7f8cc84f3640 1 -- 172.21.15.64:0/1512983275 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cc01a8de0 con 0x7f8cc0105320 2024-09-21T04:09:32.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.260+0000 7f8cb77fe640 1 -- 172.21.15.64:0/1512983275 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f8cb0043020 con 0x7f8cc0105320 2024-09-21T04:09:32.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.260+0000 7f8ccea67740 1 Processor -- start 2024-09-21T04:09:32.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.262+0000 7f8cb77fe640 1 --2- 172.21.15.64:0/1512983275 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8cac0b6700 0x7f8cac0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:32.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.263+0000 7f8cc6a90640 1 --2- 172.21.15.64:0/1512983275 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8cac0b6700 0x7f8cac0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:32.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.263+0000 7f8cb77fe640 1 -- 172.21.15.64:0/1512983275 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f8cb0107560 con 0x7f8cc0105320 2024-09-21T04:09:32.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.264+0000 7f8ccea67740 1 -- start start 2024-09-21T04:09:32.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.264+0000 7f8cc6a90640 1 --2- 172.21.15.64:0/1512983275 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8cac0b6700 0x7f8cac0b8c10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8005fd0 tx=0x7f8cb8009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:32.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.265+0000 7f8ccea67740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56450acbf070 0x56450b0666a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:32.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.265+0000 7f8ccea67740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56450ac961b0 0x56450b066be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:32.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.265+0000 7f8cc628f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56450acbf070 0x56450b0666a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:32.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.265+0000 7f8cc628f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56450acbf070 0x56450b0666a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:50336/0 (socket says 172.21.15.64:50336) 2024-09-21T04:09:32.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.265+0000 7f8cc628f640 1 -- 172.21.15.64:0/746105277 learned_addr learned my addr 172.21.15.64:0/746105277 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:32.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.265+0000 7f8ccea67740 1 --2- 172.21.15.64:0/746105277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56450ac9e390 0x56450b067120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:32.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.265+0000 7f8cc5a8e640 1 --2- 172.21.15.64:0/746105277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56450ac961b0 0x56450b066be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:32.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.266+0000 7f8ccea67740 1 -- 172.21.15.64:0/746105277 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56450afdcea0 con 0x56450ac9e390 2024-09-21T04:09:32.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.266+0000 7f8cc6a90640 1 --2- 172.21.15.64:0/746105277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56450ac9e390 0x56450b067120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:32.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.266+0000 7f8ccea67740 1 -- 172.21.15.64:0/746105277 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56450afcc2d0 con 0x56450acbf070 2024-09-21T04:09:32.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.266+0000 7f8ccea67740 1 -- 172.21.15.64:0/746105277 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56450afd00e0 con 0x56450ac961b0 2024-09-21T04:09:32.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.267+0000 7f8cc628f640 1 -- 172.21.15.64:0/746105277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56450ac961b0 msgr2=0x56450b066be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:32.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.267+0000 7f8cc628f640 1 --2- 172.21.15.64:0/746105277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56450ac961b0 0x56450b066be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.267+0000 7f8cc628f640 1 -- 172.21.15.64:0/746105277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56450ac9e390 msgr2=0x56450b067120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:32.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.267+0000 7f8cc628f640 1 --2- 172.21.15.64:0/746105277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56450ac9e390 0x56450b067120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.267+0000 7f8cc628f640 1 -- 172.21.15.64:0/746105277 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56450ad2a650 con 0x56450acbf070 2024-09-21T04:09:32.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.268+0000 7f8cc628f640 1 --2- 172.21.15.64:0/746105277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56450acbf070 0x56450b0666a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009c60 tx=0x7f8cbc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dbeff6595e8ceb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:32.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.268+0000 7f8cb4ff9640 1 -- 172.21.15.64:0/746105277 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc005220 con 0x56450acbf070 2024-09-21T04:09:32.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.268+0000 7f8cb4ff9640 1 -- 172.21.15.64:0/746105277 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cbc009e10 con 0x56450acbf070 2024-09-21T04:09:32.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.268+0000 7f8cb4ff9640 1 -- 172.21.15.64:0/746105277 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc019760 con 0x56450acbf070 2024-09-21T04:09:32.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.270+0000 7f8ccea67740 1 -- 172.21.15.64:0/746105277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56450acbf070 msgr2=0x56450b0666a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:32.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.270+0000 7f8ccea67740 1 --2- 172.21.15.64:0/746105277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56450acbf070 0x56450b0666a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009c60 tx=0x7f8cbc011810 comp rx=0 tx=0).stop 2024-09-21T04:09:32.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.270+0000 7f8ccea67740 1 -- 172.21.15.64:0/746105277 shutdown_connections 2024-09-21T04:09:32.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.270+0000 7f8ccea67740 1 --2- 172.21.15.64:0/746105277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56450ac9e390 0x56450b067120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.270+0000 7f8ccea67740 1 --2- 172.21.15.64:0/746105277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56450ac961b0 0x56450b066be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.270+0000 7f8ccea67740 1 --2- 172.21.15.64:0/746105277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56450acbf070 0x56450b0666a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.270+0000 7f8ccea67740 1 -- 172.21.15.64:0/746105277 >> 172.21.15.64:0/746105277 conn(0x56450acb3460 msgr2=0x56450b064590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:32.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.271+0000 7f8ccea67740 1 -- 172.21.15.64:0/746105277 shutdown_connections 2024-09-21T04:09:32.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.271+0000 7f8ccea67740 1 -- 172.21.15.64:0/746105277 wait complete. 2024-09-21T04:09:32.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.272+0000 7f8ccea67740 1 -- start start 2024-09-21T04:09:32.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.272+0000 7f8ccea67740 1 Processor -- start 2024-09-21T04:09:32.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.273+0000 7f8ccea67740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56450ac961b0 0x56450b071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:32.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.274+0000 7f8ccea67740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56450ac9e390 0x56450b072320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:32.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.274+0000 7f8cc628f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56450ac961b0 0x56450b071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:32.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.274+0000 7f8cc628f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56450ac961b0 0x56450b071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:50338/0 (socket says 172.21.15.64:50338) 2024-09-21T04:09:32.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.274+0000 7f8cc5a8e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56450ac9e390 0x56450b072320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:32.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.275+0000 7f8cc5a8e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56450ac9e390 0x56450b072320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58142/0 (socket says 172.21.15.64:58142) 2024-09-21T04:09:32.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.275+0000 7f8cc628f640 1 -- 172.21.15.64:0/3142615112 learned_addr learned my addr 172.21.15.64:0/3142615112 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:32.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.275+0000 7f8ccea67740 1 --2- 172.21.15.64:0/3142615112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56450acbf070 0x56450b07e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:32.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.275+0000 7f8ccea67740 1 -- 172.21.15.64:0/3142615112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56450ae2d700 con 0x56450ac9e390 2024-09-21T04:09:32.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.275+0000 7f8cc6a90640 1 --2- 172.21.15.64:0/3142615112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56450acbf070 0x56450b07e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:32.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.275+0000 7f8ccea67740 1 -- 172.21.15.64:0/3142615112 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56450af67160 con 0x56450ac961b0 2024-09-21T04:09:32.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.275+0000 7f8ccea67740 1 -- 172.21.15.64:0/3142615112 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56450b074e80 con 0x56450acbf070 2024-09-21T04:09:32.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.275+0000 7f8ccea67740 10 client.0 _ll_register_callbacks cb 0x56450af809e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:09:32.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.276+0000 7f8cc6a90640 1 -- 172.21.15.64:0/3142615112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56450ac961b0 msgr2=0x56450b071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:32.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.276+0000 7f8cc6a90640 1 --2- 172.21.15.64:0/3142615112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56450ac961b0 0x56450b071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.276+0000 7f8cc6a90640 1 -- 172.21.15.64:0/3142615112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56450ac9e390 msgr2=0x56450b072320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:32.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.276+0000 7f8cc6a90640 1 --2- 172.21.15.64:0/3142615112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56450ac9e390 0x56450b072320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.276+0000 7f8cc6a90640 1 -- 172.21.15.64:0/3142615112 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56450af94c90 con 0x56450acbf070 2024-09-21T04:09:32.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.276+0000 7f8cc5a8e640 1 --2- 172.21.15.64:0/3142615112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56450ac9e390 0x56450b072320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:09:32.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.276+0000 7f8cc628f640 1 --2- 172.21.15.64:0/3142615112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56450ac961b0 0x56450b071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:09:32.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.277+0000 7f8cc6a90640 1 --2- 172.21.15.64:0/3142615112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56450acbf070 0x56450b07e170 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800eb70 tx=0x7f8cb8004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88127c2ecae6080f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:32.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.277+0000 7f8ca9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-21T04:09:32.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.277+0000 7f8ca9ffb640 1 -- 172.21.15.64:0/3142615112 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb801ed90 con 0x56450acbf070 2024-09-21T04:09:32.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.277+0000 7f8ccea67740 1 -- 172.21.15.64:0/3142615112 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cbc009e10 con 0x56450acbf070 2024-09-21T04:09:32.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.278+0000 7f8ca9ffb640 1 -- 172.21.15.64:0/3142615112 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f8cb8003970 con 0x56450acbf070 2024-09-21T04:09:32.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.278+0000 7f8ca9ffb640 1 -- 172.21.15.64:0/3142615112 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8cb8003b70 con 0x56450acbf070 2024-09-21T04:09:32.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.278+0000 7f8ccea67740 10 client.26251 fetch_fsmap learned FSMap version 35 2024-09-21T04:09:32.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.278+0000 7f8ccea67740 1 -- 172.21.15.64:0/3142615112 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56450b080ca0 con 0x56450acbf070 2024-09-21T04:09:32.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.278+0000 7f8ca9ffb640 1 -- 172.21.15.64:0/3142615112 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cbc009e10 con 0x56450acbf070 2024-09-21T04:09:32.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.279+0000 7f8ca9ffb640 1 -- 172.21.15.64:0/3142615112 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f8cb802f070 con 0x56450acbf070 2024-09-21T04:09:32.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.279+0000 7f8ccea67740 10 client.26251 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T04:09:32.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.279+0000 7f8ccea67740 10 client.26251 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:09:32.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.279+0000 7f8ccea67740 20 client.26251 populate_metadata read hostname 'smithi064' 2024-09-21T04:09:32.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.279+0000 7f8ccea67740 1 --2- 172.21.15.64:0/3142615112 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x56450ac7d9f0 0x56450b0845c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:32.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.279+0000 7f8ccea67740 4 client.26251 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:09:32.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.279+0000 7f8ccea67740 1 -- 172.21.15.64:0/3142615112 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56450afdedf0 con 0x56450ac7d9f0 2024-09-21T04:09:32.282 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.279+0000 7f8cc628f640 1 --2- 172.21.15.64:0/3142615112 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x56450ac7d9f0 0x56450b0845c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:32.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.280+0000 7f8cc628f640 1 --2- 172.21.15.64:0/3142615112 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x56450ac7d9f0 0x56450b0845c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=28b20a635ec8984e server_cookie=b36439a8247c67c3 in_seq=0 out_seq=0 2024-09-21T04:09:32.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.280+0000 7f8ca9ffb640 10 client.26251 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:09:32.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.284+0000 7f8ca9ffb640 1 -- 172.21.15.64:0/3142615112 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56450afdedf0 con 0x56450ac7d9f0 2024-09-21T04:09:32.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.284+0000 7f8ca9ffb640 10 client.26251 handle_command_reply: tid=0 2024-09-21T04:09:32.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.284+0000 7f8ccea67740 1 client.26251 shutdown 2024-09-21T04:09:32.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.285+0000 7f8ccea67740 1 -- 172.21.15.64:0/3142615112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56450acbf070 msgr2=0x56450b07e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:32.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.285+0000 7f8ccea67740 1 --2- 172.21.15.64:0/3142615112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56450acbf070 0x56450b07e170 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800eb70 tx=0x7f8cb8004ab0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.285+0000 7f8ccea67740 1 -- 172.21.15.64:0/3142615112 shutdown_connections 2024-09-21T04:09:32.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.285+0000 7f8ccea67740 1 --2- 172.21.15.64:0/3142615112 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x56450ac7d9f0 0x56450b0845c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.285+0000 7f8ccea67740 1 --2- 172.21.15.64:0/3142615112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56450acbf070 0x56450b07e170 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.285+0000 7f8ccea67740 1 --2- 172.21.15.64:0/3142615112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56450ac9e390 0x56450b072320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.285+0000 7f8ca9ffb640 0 client.26251 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:09:32.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.285+0000 7f8ccea67740 1 --2- 172.21.15.64:0/3142615112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56450ac961b0 0x56450b071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.285+0000 7f8ccea67740 1 -- 172.21.15.64:0/3142615112 >> 172.21.15.64:0/3142615112 conn(0x56450acb3460 msgr2=0x56450b069880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:32.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.285+0000 7f8ccea67740 1 -- 172.21.15.64:0/3142615112 shutdown_connections 2024-09-21T04:09:32.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.285+0000 7f8ccea67740 1 -- 172.21.15.64:0/3142615112 wait complete. 2024-09-21T04:09:32.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.285+0000 7f8ccea67740 20 client.26251 trim_cache size 0 max 16384 2024-09-21T04:09:32.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:32 smithi131 ceph-mon[35193]: pgmap v486: 97 pgs: 97 active+clean; 661 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 337 op/s 2024-09-21T04:09:32.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.326+0000 7f8ccea67740 1 Processor -- start 2024-09-21T04:09:32.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.331+0000 7f8ccea67740 1 -- start start 2024-09-21T04:09:32.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.331+0000 7f8ccea67740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56450ac961b0 0x56450b069eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:32.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.331+0000 7f8ccea67740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56450ac9e390 0x56450b071b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:32.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.331+0000 7f8ccea67740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56450acbf070 0x56450b0720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:32.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.331+0000 7f8ccea67740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc01affb0 con 0x56450ac961b0 2024-09-21T04:09:32.343 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.331+0000 7f8ccea67740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc01ad820 con 0x56450ac9e390 2024-09-21T04:09:32.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.331+0000 7f8ccea67740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc01ab6d0 con 0x56450acbf070 2024-09-21T04:09:32.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.332+0000 7f8cc628f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56450ac961b0 0x56450b069eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:32.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.332+0000 7f8cc628f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56450ac961b0 0x56450b069eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58146/0 (socket says 172.21.15.64:58146) 2024-09-21T04:09:32.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.332+0000 7f8cc628f640 1 -- 172.21.15.64:0/317946286 learned_addr learned my addr 172.21.15.64:0/317946286 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:32.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.333+0000 7f8cc628f640 1 -- 172.21.15.64:0/317946286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56450acbf070 msgr2=0x56450b0720d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:09:32.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.333+0000 7f8cc628f640 1 --2- 172.21.15.64:0/317946286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56450acbf070 0x56450b0720d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.333+0000 7f8cc628f640 1 -- 172.21.15.64:0/317946286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56450ac9e390 msgr2=0x56450b071b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:32.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.333+0000 7f8cc628f640 1 --2- 172.21.15.64:0/317946286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56450ac9e390 0x56450b071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.333+0000 7f8cc628f640 1 -- 172.21.15.64:0/317946286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56450b068dd0 con 0x56450ac961b0 2024-09-21T04:09:32.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.333+0000 7f8cc628f640 1 --2- 172.21.15.64:0/317946286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56450ac961b0 0x56450b069eb0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc005750 tx=0x7f8cbc005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8ac8dfeeec1413 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:32.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.334+0000 7f8cabfff640 1 -- 172.21.15.64:0/317946286 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc019870 con 0x56450ac961b0 2024-09-21T04:09:32.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.335+0000 7f8cabfff640 1 -- 172.21.15.64:0/317946286 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cbc035480 con 0x56450ac961b0 2024-09-21T04:09:32.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.335+0000 7f8cabfff640 1 -- 172.21.15.64:0/317946286 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc021640 con 0x56450ac961b0 2024-09-21T04:09:32.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.336+0000 7f8ccea67740 1 -- 172.21.15.64:0/317946286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56450ac961b0 msgr2=0x56450b069eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:32.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.336+0000 7f8ccea67740 1 --2- 172.21.15.64:0/317946286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56450ac961b0 0x56450b069eb0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc005750 tx=0x7f8cbc005780 comp rx=0 tx=0).stop 2024-09-21T04:09:32.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.341+0000 7f8ccea67740 1 -- 172.21.15.64:0/317946286 shutdown_connections 2024-09-21T04:09:32.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.341+0000 7f8ccea67740 1 --2- 172.21.15.64:0/317946286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56450acbf070 0x56450b0720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.341+0000 7f8ccea67740 1 --2- 172.21.15.64:0/317946286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56450ac9e390 0x56450b071b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.341+0000 7f8ccea67740 1 --2- 172.21.15.64:0/317946286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56450ac961b0 0x56450b069eb0 secure :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc005750 tx=0x7f8cbc005780 comp rx=0 tx=0).stop 2024-09-21T04:09:32.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.341+0000 7f8ccea67740 1 -- 172.21.15.64:0/317946286 >> 172.21.15.64:0/317946286 conn(0x56450acb3460 msgr2=0x56450b0697a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:32.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.346+0000 7f8ccea67740 1 -- 172.21.15.64:0/317946286 shutdown_connections 2024-09-21T04:09:32.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.348+0000 7f8ccea67740 1 -- 172.21.15.64:0/317946286 wait complete. 2024-09-21T04:09:32.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.348+0000 7f8ccea67740 1 -- start start 2024-09-21T04:09:32.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.349+0000 7f8ccea67740 1 Processor -- start 2024-09-21T04:09:32.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.353+0000 7f8ccea67740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56450ac9e390 0x56450b0ea6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:32.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.354+0000 7f8ccea67740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56450acbf070 0x56450b07a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:32.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.354+0000 7f8ccea67740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56450ac52da0 0x56450b07a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:32.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.354+0000 7f8ccea67740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc01abd20 con 0x56450acbf070 2024-09-21T04:09:32.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.354+0000 7f8ccea67740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc01ad150 con 0x56450ac52da0 2024-09-21T04:09:32.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.354+0000 7f8ccea67740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc01ac3b0 con 0x56450ac9e390 2024-09-21T04:09:32.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.354+0000 7f8cc628f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56450ac9e390 0x56450b0ea6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:32.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.354+0000 7f8cc628f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56450ac9e390 0x56450b0ea6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:59118/0 (socket says 172.21.15.64:59118) 2024-09-21T04:09:32.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.354+0000 7f8cc628f640 1 -- 172.21.15.64:0/778534845 learned_addr learned my addr 172.21.15.64:0/778534845 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:32.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.354+0000 7f8cc5a8e640 1 --2- 172.21.15.64:0/778534845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56450acbf070 0x56450b07a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:32.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.354+0000 7f8cc6a90640 1 --2- 172.21.15.64:0/778534845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56450ac52da0 0x56450b07a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:32.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.354+0000 7f8ccea67740 10 client.0 _ll_register_callbacks cb 0x7f8cb800e3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:09:32.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.355+0000 7f8cc628f640 1 -- 172.21.15.64:0/778534845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56450ac52da0 msgr2=0x56450b07a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:32.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.355+0000 7f8cc628f640 1 --2- 172.21.15.64:0/778534845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56450ac52da0 0x56450b07a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.355+0000 7f8cc628f640 1 -- 172.21.15.64:0/778534845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56450acbf070 msgr2=0x56450b07a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:32.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.355+0000 7f8cc628f640 1 --2- 172.21.15.64:0/778534845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56450acbf070 0x56450b07a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.355+0000 7f8cc628f640 1 -- 172.21.15.64:0/778534845 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56450b07ac80 con 0x56450ac9e390 2024-09-21T04:09:32.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.356+0000 7f8cc628f640 1 --2- 172.21.15.64:0/778534845 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56450ac9e390 0x56450b0ea6f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009530 tx=0x7f8cbc009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31df52ac31dca02c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:32.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.356+0000 7f8ca9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-21T04:09:32.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.357+0000 7f8ca9ffb640 1 -- 172.21.15.64:0/778534845 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc041020 con 0x56450ac9e390 2024-09-21T04:09:32.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.357+0000 7f8ca9ffb640 1 -- 172.21.15.64:0/778534845 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f8cbc005450 con 0x56450ac9e390 2024-09-21T04:09:32.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.360+0000 7f8ccea67740 1 -- 172.21.15.64:0/778534845 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cbc035480 con 0x56450ac9e390 2024-09-21T04:09:32.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.365+0000 7f8ca9ffb640 1 -- 172.21.15.64:0/778534845 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8cbc035630 con 0x56450ac9e390 2024-09-21T04:09:32.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.366+0000 7f8ccea67740 10 client.26254 fetch_fsmap learned FSMap version 35 2024-09-21T04:09:32.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.366+0000 7f8ccea67740 1 -- 172.21.15.64:0/778534845 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56450b0fffd0 con 0x56450ac9e390 2024-09-21T04:09:32.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.370+0000 7f8ca9ffb640 1 -- 172.21.15.64:0/778534845 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cbc035480 con 0x56450ac9e390 2024-09-21T04:09:32.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.370+0000 7f8ca9ffb640 1 -- 172.21.15.64:0/778534845 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f8cbc032070 con 0x56450ac9e390 2024-09-21T04:09:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.374+0000 7f8ccea67740 10 client.26254 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T04:09:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.374+0000 7f8ccea67740 10 client.26254 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:09:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.374+0000 7f8ccea67740 20 client.26254 populate_metadata read hostname 'smithi064' 2024-09-21T04:09:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.374+0000 7f8ccea67740 1 --2- 172.21.15.64:0/778534845 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x56450b1002b0 0x56450b1026d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.374+0000 7f8ccea67740 4 client.26254 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:09:32.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.374+0000 7f8ccea67740 1 -- 172.21.15.64:0/778534845 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56450b05f9e0 con 0x56450b1002b0 2024-09-21T04:09:32.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.377+0000 7f8cc5a8e640 1 --2- 172.21.15.64:0/778534845 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x56450b1002b0 0x56450b1026d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:32.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.388+0000 7f8cc5a8e640 1 --2- 172.21.15.64:0/778534845 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x56450b1002b0 0x56450b1026d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b9e6b076c8feb347 server_cookie=a8350992433d33ee in_seq=0 out_seq=0 2024-09-21T04:09:32.393 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.388+0000 7f8ca9ffb640 10 client.26254 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:09:32.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.393+0000 7f8ca9ffb640 1 -- 172.21.15.64:0/778534845 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56450b05f9e0 con 0x56450b1002b0 2024-09-21T04:09:32.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.393+0000 7f8ca9ffb640 10 client.26254 handle_command_reply: tid=0 2024-09-21T04:09:32.395 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.393+0000 7f8ccea67740 1 client.26254 shutdown 2024-09-21T04:09:32.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.395+0000 7f8ccea67740 1 -- 172.21.15.64:0/778534845 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56450ac9e390 msgr2=0x56450b0ea6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:32.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.395+0000 7f8ccea67740 1 --2- 172.21.15.64:0/778534845 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56450ac9e390 0x56450b0ea6f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009530 tx=0x7f8cbc009b60 comp rx=0 tx=0).stop 2024-09-21T04:09:32.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.396+0000 7f8ccea67740 1 -- 172.21.15.64:0/778534845 shutdown_connections 2024-09-21T04:09:32.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.396+0000 7f8ccea67740 1 --2- 172.21.15.64:0/778534845 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x56450b1002b0 0x56450b1026d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.396+0000 7f8ccea67740 1 --2- 172.21.15.64:0/778534845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56450ac52da0 0x56450b07a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.396+0000 7f8ccea67740 1 --2- 172.21.15.64:0/778534845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56450acbf070 0x56450b07a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.396+0000 7f8ccea67740 1 --2- 172.21.15.64:0/778534845 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56450ac9e390 0x56450b0ea6f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.396+0000 7f8ccea67740 1 -- 172.21.15.64:0/778534845 >> 172.21.15.64:0/778534845 conn(0x56450acb3460 msgr2=0x56450b069500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:32.399 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.396+0000 7f8ca9ffb640 0 client.26254 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:09:32.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.400+0000 7f8ccea67740 1 -- 172.21.15.64:0/778534845 shutdown_connections 2024-09-21T04:09:32.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.400+0000 7f8ccea67740 1 -- 172.21.15.64:0/778534845 wait complete. 2024-09-21T04:09:32.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.400+0000 7f8ccea67740 20 client.26254 trim_cache size 0 max 16384 2024-09-21T04:09:32.476 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:09:32.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.408+0000 7f8cc84f3640 1 -- 172.21.15.64:0/1512983275 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8cac0b6700 msgr2=0x7f8cac0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:32.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.408+0000 7f8cc84f3640 1 --2- 172.21.15.64:0/1512983275 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8cac0b6700 0x7f8cac0b8c10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8005fd0 tx=0x7f8cb8009500 comp rx=0 tx=0).stop 2024-09-21T04:09:32.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.408+0000 7f8cc84f3640 1 -- 172.21.15.64:0/1512983275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cc0105320 msgr2=0x7f8cc0102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:32.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.408+0000 7f8cc84f3640 1 --2- 172.21.15.64:0/1512983275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cc0105320 0x7f8cc0102c50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0035040 tx=0x7f8cb0032d50 comp rx=0 tx=0).stop 2024-09-21T04:09:32.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.408+0000 7f8cc84f3640 1 -- 172.21.15.64:0/1512983275 shutdown_connections 2024-09-21T04:09:32.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.408+0000 7f8cc84f3640 1 --2- 172.21.15.64:0/1512983275 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8cac0b6700 0x7f8cac0b8c10 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.409+0000 7f8cc84f3640 1 --2- 172.21.15.64:0/1512983275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cc010de30 0x7f8cc0103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.409+0000 7f8cc84f3640 1 --2- 172.21.15.64:0/1512983275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cc0105320 0x7f8cc0102c50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.409+0000 7f8cc84f3640 1 --2- 172.21.15.64:0/1512983275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc01049d0 0x7f8cc0102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:32.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.409+0000 7f8cc84f3640 1 -- 172.21.15.64:0/1512983275 >> 172.21.15.64:0/1512983275 conn(0x7f8cc0079740 msgr2=0x7f8cc00ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:32.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.409+0000 7f8cc84f3640 1 -- 172.21.15.64:0/1512983275 shutdown_connections 2024-09-21T04:09:32.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:32.409+0000 7f8cc84f3640 1 -- 172.21.15.64:0/1512983275 wait complete. 2024-09-21T04:09:32.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:32 smithi119 ceph-mon[34811]: pgmap v486: 97 pgs: 97 active+clean; 661 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 337 op/s 2024-09-21T04:09:33.669 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T04:09:34.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:34 smithi131 ceph-mon[35193]: pgmap v487: 97 pgs: 97 active+clean; 694 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.1 MiB/s wr, 582 op/s 2024-09-21T04:09:34.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:34 smithi119 ceph-mon[34811]: pgmap v487: 97 pgs: 97 active+clean; 694 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.1 MiB/s wr, 582 op/s 2024-09-21T04:09:34.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:34 smithi064 ceph-mon[31180]: pgmap v487: 97 pgs: 97 active+clean; 694 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.1 MiB/s wr, 582 op/s 2024-09-21T04:09:34.909 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:09:36.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.062+0000 7fd057fff640 1 -- 172.21.15.64:0/3516531922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd058111cd0 msgr2=0x7fd05810d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:36.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.062+0000 7fd057fff640 1 --2- 172.21.15.64:0/3516531922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd058111cd0 0x7fd05810d6b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd048009900 tx=0x7fd04802a440 comp rx=0 tx=0).stop 2024-09-21T04:09:36.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.064+0000 7fd057fff640 1 -- 172.21.15.64:0/3516531922 shutdown_connections 2024-09-21T04:09:36.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.064+0000 7fd057fff640 1 --2- 172.21.15.64:0/3516531922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd05811cba0 0x7fd05811efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.064+0000 7fd057fff640 1 --2- 172.21.15.64:0/3516531922 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd05810dbf0 0x7fd05810e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.064+0000 7fd057fff640 1 --2- 172.21.15.64:0/3516531922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd058111cd0 0x7fd05810d6b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.064+0000 7fd057fff640 1 -- 172.21.15.64:0/3516531922 >> 172.21.15.64:0/3516531922 conn(0x7fd05806e9f0 msgr2=0x7fd05806ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:36.077 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.072+0000 7fd057fff640 1 -- 172.21.15.64:0/3516531922 shutdown_connections 2024-09-21T04:09:36.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.076+0000 7fd057fff640 1 -- 172.21.15.64:0/3516531922 wait complete. 2024-09-21T04:09:36.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.077+0000 7fd057fff640 1 Processor -- start 2024-09-21T04:09:36.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.078+0000 7fd057fff640 1 -- start start 2024-09-21T04:09:36.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.078+0000 7fd057fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd05810dbf0 0x7fd05811b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:36.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.078+0000 7fd057fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd05811cba0 0x7fd05811b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:36.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.078+0000 7fd057fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0581144d0 0x7fd0581149d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:36.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.079+0000 7fd057fff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd058122770 con 0x7fd05811cba0 2024-09-21T04:09:36.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.079+0000 7fd057fff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd0581225f0 con 0x7fd0581144d0 2024-09-21T04:09:36.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.079+0000 7fd057fff640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd0581228f0 con 0x7fd05810dbf0 2024-09-21T04:09:36.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.079+0000 7fd0567fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd05811cba0 0x7fd05811b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:36.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.079+0000 7fd0567fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd05811cba0 0x7fd05811b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58166/0 (socket says 172.21.15.64:58166) 2024-09-21T04:09:36.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.079+0000 7fd0567fc640 1 -- 172.21.15.64:0/3773867556 learned_addr learned my addr 172.21.15.64:0/3773867556 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:36.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.080+0000 7fd0567fc640 1 -- 172.21.15.64:0/3773867556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd05810dbf0 msgr2=0x7fd05811b110 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T04:09:36.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.080+0000 7fd0567fc640 1 --2- 172.21.15.64:0/3773867556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd05810dbf0 0x7fd05811b110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.080+0000 7fd0567fc640 1 -- 172.21.15.64:0/3773867556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0581144d0 msgr2=0x7fd0581149d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:36.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.080+0000 7fd0567fc640 1 --2- 172.21.15.64:0/3773867556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0581144d0 0x7fd0581149d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.080+0000 7fd0567fc640 1 -- 172.21.15.64:0/3773867556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd058119cc0 con 0x7fd05811cba0 2024-09-21T04:09:36.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.080+0000 7fd0567fc640 1 --2- 172.21.15.64:0/3773867556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd05811cba0 0x7fd05811b6b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd05000b9b0 tx=0x7fd05000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:36.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.082+0000 7fd05c800640 1 -- 172.21.15.64:0/3773867556 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd050019070 con 0x7fd05811cba0 2024-09-21T04:09:36.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.083+0000 7fd05c800640 1 -- 172.21.15.64:0/3773867556 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd05000cba0 con 0x7fd05811cba0 2024-09-21T04:09:36.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.083+0000 7fd05c800640 1 -- 172.21.15.64:0/3773867556 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd050012580 con 0x7fd05811cba0 2024-09-21T04:09:36.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.084+0000 7fd057fff640 1 -- 172.21.15.64:0/3773867556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd058119fb0 con 0x7fd05811cba0 2024-09-21T04:09:36.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.084+0000 7fd057fff640 1 -- 172.21.15.64:0/3773867556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd05811a4a0 con 0x7fd05811cba0 2024-09-21T04:09:36.093 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.088+0000 7fd063d76740 1 Processor -- start 2024-09-21T04:09:36.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.088+0000 7fd063d76740 1 -- start start 2024-09-21T04:09:36.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.089+0000 7fd063d76740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619465b3070 0x56194695a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:36.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.089+0000 7fd063d76740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56194658a1b0 0x56194695ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:36.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.089+0000 7fd0577fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619465b3070 0x56194695a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:36.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.089+0000 7fd0577fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619465b3070 0x56194695a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:59150/0 (socket says 172.21.15.64:59150) 2024-09-21T04:09:36.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.089+0000 7fd063d76740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561946592390 0x56194695b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:36.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.089+0000 7fd0577fe640 1 -- 172.21.15.64:0/1362492152 learned_addr learned my addr 172.21.15.64:0/1362492152 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:36.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.089+0000 7fd063d76740 1 -- 172.21.15.64:0/1362492152 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5619468d0ea0 con 0x56194658a1b0 2024-09-21T04:09:36.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.089+0000 7fd063d76740 1 -- 172.21.15.64:0/1362492152 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5619468c02d0 con 0x561946592390 2024-09-21T04:09:36.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.089+0000 7fd063d76740 1 -- 172.21.15.64:0/1362492152 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5619468c40e0 con 0x5619465b3070 2024-09-21T04:09:36.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.093+0000 7fd056ffd640 1 --2- 172.21.15.64:0/1362492152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56194658a1b0 0x56194695ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:36.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.095+0000 7fd05c800640 1 -- 172.21.15.64:0/3773867556 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fd05001c070 con 0x7fd05811cba0 2024-09-21T04:09:36.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.097+0000 7fd0577fe640 1 -- 172.21.15.64:0/1362492152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561946592390 msgr2=0x56194695b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:36.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.099+0000 7fd0567fc640 1 --2- 172.21.15.64:0/1362492152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561946592390 0x56194695b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:36.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.099+0000 7fd0577fe640 1 --2- 172.21.15.64:0/1362492152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561946592390 0x56194695b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.100+0000 7fd0577fe640 1 -- 172.21.15.64:0/1362492152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56194658a1b0 msgr2=0x56194695ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:36.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.100+0000 7fd0577fe640 1 --2- 172.21.15.64:0/1362492152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56194658a1b0 0x56194695ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.100+0000 7fd0577fe640 1 -- 172.21.15.64:0/1362492152 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56194661e650 con 0x5619465b3070 2024-09-21T04:09:36.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.100+0000 7fd056ffd640 1 --2- 172.21.15.64:0/1362492152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56194658a1b0 0x56194695ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:09:36.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.100+0000 7fd0567fc640 1 --2- 172.21.15.64:0/1362492152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561946592390 0x56194695b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:09:36.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.100+0000 7fd0577fe640 1 --2- 172.21.15.64:0/1362492152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619465b3070 0x56194695a860 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd04c005bc0 tx=0x7fd04c02f2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fa73202d4edb8a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:36.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.102+0000 7fd03dffb640 1 -- 172.21.15.64:0/1362492152 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c038070 con 0x5619465b3070 2024-09-21T04:09:36.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.102+0000 7fd03dffb640 1 -- 172.21.15.64:0/1362492152 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd04c02fd20 con 0x5619465b3070 2024-09-21T04:09:36.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.102+0000 7fd05c800640 1 --2- 172.21.15.64:0/3773867556 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd0400b6880 0x7fd0400b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:36.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.103+0000 7fd056ffd640 1 --2- 172.21.15.64:0/3773867556 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd0400b6880 0x7fd0400b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:36.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.103+0000 7fd063d76740 1 -- 172.21.15.64:0/1362492152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619465b3070 msgr2=0x56194695a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:36.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.103+0000 7fd063d76740 1 --2- 172.21.15.64:0/1362492152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619465b3070 0x56194695a860 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd04c005bc0 tx=0x7fd04c02f2f0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.104+0000 7fd063d76740 1 -- 172.21.15.64:0/1362492152 shutdown_connections 2024-09-21T04:09:36.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.104+0000 7fd063d76740 1 --2- 172.21.15.64:0/1362492152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561946592390 0x56194695b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.104+0000 7fd063d76740 1 --2- 172.21.15.64:0/1362492152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56194658a1b0 0x56194695ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.104+0000 7fd063d76740 1 --2- 172.21.15.64:0/1362492152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619465b3070 0x56194695a860 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.104+0000 7fd063d76740 1 -- 172.21.15.64:0/1362492152 >> 172.21.15.64:0/1362492152 conn(0x5619465a7460 msgr2=0x561946958750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:36.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.104+0000 7fd056ffd640 1 --2- 172.21.15.64:0/3773867556 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd0400b6880 0x7fd0400b8d90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd048005ab0 tx=0x7fd048009350 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:36.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.104+0000 7fd05c800640 1 -- 172.21.15.64:0/3773867556 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fd0500e6bd0 con 0x7fd05811cba0 2024-09-21T04:09:36.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.104+0000 7fd063d76740 1 -- 172.21.15.64:0/1362492152 shutdown_connections 2024-09-21T04:09:36.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.104+0000 7fd063d76740 1 -- 172.21.15.64:0/1362492152 wait complete. 2024-09-21T04:09:36.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.105+0000 7fd063d76740 1 -- start start 2024-09-21T04:09:36.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.105+0000 7fd063d76740 1 Processor -- start 2024-09-21T04:09:36.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.106+0000 7fd063d76740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56194658a1b0 0x561946965bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:36.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.106+0000 7fd063d76740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561946592390 0x561946966110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:36.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.106+0000 7fd063d76740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619465719f0 0x561946966650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:36.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.106+0000 7fd063d76740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561946721700 con 0x561946592390 2024-09-21T04:09:36.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.106+0000 7fd063d76740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56194685b160 con 0x56194658a1b0 2024-09-21T04:09:36.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.106+0000 7fd063d76740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x561946969090 con 0x5619465719f0 2024-09-21T04:09:36.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.106+0000 7fd063d76740 10 client.0 _ll_register_callbacks cb 0x5619468749e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:09:36.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.106+0000 7fd0577fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619465719f0 0x561946966650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:36.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.106+0000 7fd056ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56194658a1b0 0x561946965bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:36.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.106+0000 7fd056ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56194658a1b0 0x561946965bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:50404/0 (socket says 172.21.15.64:50404) 2024-09-21T04:09:36.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.107+0000 7fd056ffd640 1 -- 172.21.15.64:0/3698262283 learned_addr learned my addr 172.21.15.64:0/3698262283 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:36.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.107+0000 7fd0577fe640 1 -- 172.21.15.64:0/3698262283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56194658a1b0 msgr2=0x561946965bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:36.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.107+0000 7fd0577fe640 1 --2- 172.21.15.64:0/3698262283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56194658a1b0 0x561946965bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.107+0000 7fd0577fe640 1 -- 172.21.15.64:0/3698262283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561946592390 msgr2=0x561946966110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:09:36.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.107+0000 7fd0577fe640 1 --2- 172.21.15.64:0/3698262283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561946592390 0x561946966110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.107+0000 7fd0577fe640 1 -- 172.21.15.64:0/3698262283 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561946888c90 con 0x5619465719f0 2024-09-21T04:09:36.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.108+0000 7fd0577fe640 1 --2- 172.21.15.64:0/3698262283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619465719f0 0x561946966650 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd04c005bc0 tx=0x7fd04c02fe90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61e57a34b9dfdd56 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:36.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.109+0000 7fd036ffd640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-21T04:09:36.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.109+0000 7fd036ffd640 1 -- 172.21.15.64:0/3698262283 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c038070 con 0x5619465719f0 2024-09-21T04:09:36.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.109+0000 7fd063d76740 1 -- 172.21.15.64:0/3698262283 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5619468d2df0 con 0x5619465719f0 2024-09-21T04:09:36.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.109+0000 7fd036ffd640 1 -- 172.21.15.64:0/3698262283 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fd04c005020 con 0x5619465719f0 2024-09-21T04:09:36.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.109+0000 7fd036ffd640 1 -- 172.21.15.64:0/3698262283 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd04c005220 con 0x5619465719f0 2024-09-21T04:09:36.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.113+0000 7fd063d76740 10 client.26260 fetch_fsmap learned FSMap version 35 2024-09-21T04:09:36.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.113+0000 7fd063d76740 1 -- 172.21.15.64:0/3698262283 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561946974aa0 con 0x5619465719f0 2024-09-21T04:09:36.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.115+0000 7fd036ffd640 1 -- 172.21.15.64:0/3698262283 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5619468d2df0 con 0x5619465719f0 2024-09-21T04:09:36.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.115+0000 7fd036ffd640 1 -- 172.21.15.64:0/3698262283 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fd04c030040 con 0x5619465719f0 2024-09-21T04:09:36.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.115+0000 7fd063d76740 10 client.26260 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T04:09:36.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.115+0000 7fd063d76740 10 client.26260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:09:36.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.115+0000 7fd063d76740 20 client.26260 populate_metadata read hostname 'smithi064' 2024-09-21T04:09:36.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.115+0000 7fd063d76740 1 --2- 172.21.15.64:0/3698262283 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x561946546da0 0x561946978360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:36.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.115+0000 7fd063d76740 4 client.26260 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:09:36.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.115+0000 7fd063d76740 1 -- 172.21.15.64:0/3698262283 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619468ce8a0 con 0x561946546da0 2024-09-21T04:09:36.119 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.116+0000 7fd056ffd640 1 --2- 172.21.15.64:0/3698262283 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x561946546da0 0x561946978360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:36.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.117+0000 7fd056ffd640 1 --2- 172.21.15.64:0/3698262283 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x561946546da0 0x561946978360 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c175b809294db5a server_cookie=9716c6bdd44b1372 in_seq=0 out_seq=0 2024-09-21T04:09:36.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.117+0000 7fd036ffd640 10 client.26260 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:36.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.117+0000 7fd036ffd640 1 -- 172.21.15.64:0/3698262283 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5619468ce8a0 con 0x561946546da0 2024-09-21T04:09:36.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.118+0000 7fd036ffd640 10 client.26260 handle_command_reply: tid=0 2024-09-21T04:09:36.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.118+0000 7fd063d76740 1 client.26260 shutdown 2024-09-21T04:09:36.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.131+0000 7fd063d76740 1 -- 172.21.15.64:0/3698262283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619465719f0 msgr2=0x561946966650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:36.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.131+0000 7fd063d76740 1 --2- 172.21.15.64:0/3698262283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619465719f0 0x561946966650 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd04c005bc0 tx=0x7fd04c02fe90 comp rx=0 tx=0).stop 2024-09-21T04:09:36.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.133+0000 7fd063d76740 1 -- 172.21.15.64:0/3698262283 shutdown_connections 2024-09-21T04:09:36.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.133+0000 7fd063d76740 1 --2- 172.21.15.64:0/3698262283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619465719f0 0x561946966650 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.133+0000 7fd063d76740 1 --2- 172.21.15.64:0/3698262283 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x561946546da0 0x561946978360 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.133+0000 7fd063d76740 1 --2- 172.21.15.64:0/3698262283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561946592390 0x561946966110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.133+0000 7fd063d76740 1 --2- 172.21.15.64:0/3698262283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56194658a1b0 0x561946965bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.133+0000 7fd063d76740 1 -- 172.21.15.64:0/3698262283 >> 172.21.15.64:0/3698262283 conn(0x5619465a7460 msgr2=0x56194695da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:36.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.133+0000 7fd036ffd640 0 client.26260 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:36.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.133+0000 7fd063d76740 1 -- 172.21.15.64:0/3698262283 shutdown_connections 2024-09-21T04:09:36.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.133+0000 7fd063d76740 1 -- 172.21.15.64:0/3698262283 wait complete. 2024-09-21T04:09:36.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.134+0000 7fd063d76740 20 client.26260 trim_cache size 0 max 16384 2024-09-21T04:09:36.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.175+0000 7fd063d76740 1 Processor -- start 2024-09-21T04:09:36.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.176+0000 7fd063d76740 1 -- start start 2024-09-21T04:09:36.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.176+0000 7fd063d76740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619465719f0 0x56194695e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:36.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.176+0000 7fd063d76740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56194658a1b0 0x561946962dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:36.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.176+0000 7fd063d76740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561946592390 0x561946963300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:36.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.176+0000 7fd063d76740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd058080270 con 0x561946592390 2024-09-21T04:09:36.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.176+0000 7fd063d76740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd05807db60 con 0x5619465719f0 2024-09-21T04:09:36.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.176+0000 7fd063d76740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd058111cd0 con 0x56194658a1b0 2024-09-21T04:09:36.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.178+0000 7fd056ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619465719f0 0x56194695e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:36.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.178+0000 7fd056ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619465719f0 0x56194695e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:50416/0 (socket says 172.21.15.64:50416) 2024-09-21T04:09:36.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.178+0000 7fd056ffd640 1 -- 172.21.15.64:0/4049150997 learned_addr learned my addr 172.21.15.64:0/4049150997 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:36.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.179+0000 7fd0577fe640 1 --2- 172.21.15.64:0/4049150997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561946592390 0x561946963300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:36.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.179+0000 7fd0567fc640 1 --2- 172.21.15.64:0/4049150997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56194658a1b0 0x561946962dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:36.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.184+0000 7fd056ffd640 1 -- 172.21.15.64:0/4049150997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56194658a1b0 msgr2=0x561946962dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:36.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.184+0000 7fd056ffd640 1 --2- 172.21.15.64:0/4049150997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56194658a1b0 0x561946962dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.184+0000 7fd056ffd640 1 -- 172.21.15.64:0/4049150997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561946592390 msgr2=0x561946963300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:36.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.184+0000 7fd056ffd640 1 --2- 172.21.15.64:0/4049150997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561946592390 0x561946963300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.184+0000 7fd056ffd640 1 -- 172.21.15.64:0/4049150997 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56194695cb50 con 0x5619465719f0 2024-09-21T04:09:36.189 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.186+0000 7fd056ffd640 1 --2- 172.21.15.64:0/4049150997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619465719f0 0x56194695e070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd048006010 tx=0x7fd04803c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b744a7f91bd773a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:36.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.189+0000 7fd03d7fa640 1 -- 172.21.15.64:0/4049150997 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd048056070 con 0x5619465719f0 2024-09-21T04:09:36.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.189+0000 7fd03d7fa640 1 -- 172.21.15.64:0/4049150997 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd048005040 con 0x5619465719f0 2024-09-21T04:09:36.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.189+0000 7fd03d7fa640 1 -- 172.21.15.64:0/4049150997 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd048035050 con 0x5619465719f0 2024-09-21T04:09:36.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.191+0000 7fd063d76740 1 -- 172.21.15.64:0/4049150997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619465719f0 msgr2=0x56194695e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:36.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.191+0000 7fd063d76740 1 --2- 172.21.15.64:0/4049150997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619465719f0 0x56194695e070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd048006010 tx=0x7fd04803c000 comp rx=0 tx=0).stop 2024-09-21T04:09:36.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.191+0000 7fd063d76740 1 -- 172.21.15.64:0/4049150997 shutdown_connections 2024-09-21T04:09:36.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.191+0000 7fd063d76740 1 --2- 172.21.15.64:0/4049150997 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561946592390 0x561946963300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.191+0000 7fd063d76740 1 --2- 172.21.15.64:0/4049150997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56194658a1b0 0x561946962dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.191+0000 7fd063d76740 1 --2- 172.21.15.64:0/4049150997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619465719f0 0x56194695e070 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.191+0000 7fd063d76740 1 -- 172.21.15.64:0/4049150997 >> 172.21.15.64:0/4049150997 conn(0x5619465a7460 msgr2=0x56194695d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:36.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.201+0000 7fd063d76740 1 -- 172.21.15.64:0/4049150997 shutdown_connections 2024-09-21T04:09:36.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.206+0000 7fd063d76740 1 -- 172.21.15.64:0/4049150997 wait complete. 2024-09-21T04:09:36.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.206+0000 7fd063d76740 1 -- start start 2024-09-21T04:09:36.209 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.207+0000 7fd063d76740 1 Processor -- start 2024-09-21T04:09:36.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.208+0000 7fd063d76740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619465719f0 0x561946969d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:36.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.208+0000 7fd063d76740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56194658a1b0 0x56194696a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:36.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.208+0000 7fd063d76740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561946592390 0x5619469dae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:36.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.208+0000 7fd063d76740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd05807c020 con 0x56194658a1b0 2024-09-21T04:09:36.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.208+0000 7fd063d76740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd05807d490 con 0x561946592390 2024-09-21T04:09:36.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.208+0000 7fd063d76740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd05807c6f0 con 0x5619465719f0 2024-09-21T04:09:36.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.208+0000 7fd063d76740 10 client.0 _ll_register_callbacks cb 0x7fd048009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:09:36.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.208+0000 7fd0577fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561946592390 0x5619469dae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:36.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.208+0000 7fd0577fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561946592390 0x5619469dae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:50424/0 (socket says 172.21.15.64:50424) 2024-09-21T04:09:36.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.208+0000 7fd0577fe640 1 -- 172.21.15.64:0/939973045 learned_addr learned my addr 172.21.15.64:0/939973045 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:36.212 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.209+0000 7fd056ffd640 1 --2- 172.21.15.64:0/939973045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619465719f0 0x561946969d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:36.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.210+0000 7fd0567fc640 1 --2- 172.21.15.64:0/939973045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56194658a1b0 0x56194696a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:36.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.210+0000 7fd0567fc640 1 -- 172.21.15.64:0/939973045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619465719f0 msgr2=0x561946969d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:36.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.210+0000 7fd0567fc640 1 --2- 172.21.15.64:0/939973045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619465719f0 0x561946969d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.210+0000 7fd0567fc640 1 -- 172.21.15.64:0/939973045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561946592390 msgr2=0x5619469dae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:36.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.210+0000 7fd0567fc640 1 --2- 172.21.15.64:0/939973045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561946592390 0x5619469dae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.210+0000 7fd0567fc640 1 -- 172.21.15.64:0/939973045 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619469db470 con 0x56194658a1b0 2024-09-21T04:09:36.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.211+0000 7fd0567fc640 1 --2- 172.21.15.64:0/939973045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56194658a1b0 0x56194696a370 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd0500e4290 tx=0x7fd0500ac350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2f9b7fec572b3e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:36.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.212+0000 7fd036ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:09:36.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.212+0000 7fd036ffd640 1 -- 172.21.15.64:0/939973045 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0500b1020 con 0x56194658a1b0 2024-09-21T04:09:36.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.212+0000 7fd036ffd640 1 -- 172.21.15.64:0/939973045 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7fd0500abe00 con 0x56194658a1b0 2024-09-21T04:09:36.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.212+0000 7fd063d76740 1 -- 172.21.15.64:0/939973045 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd048005040 con 0x56194658a1b0 2024-09-21T04:09:36.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.213+0000 7fd036ffd640 1 -- 172.21.15.64:0/939973045 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd0500ac4a0 con 0x56194658a1b0 2024-09-21T04:09:36.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.213+0000 7fd063d76740 10 client.16710 fetch_fsmap learned FSMap version 35 2024-09-21T04:09:36.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.213+0000 7fd063d76740 1 -- 172.21.15.64:0/939973045 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5619469e28c0 con 0x56194658a1b0 2024-09-21T04:09:36.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.214+0000 7fd036ffd640 1 -- 172.21.15.64:0/939973045 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd048005040 con 0x56194658a1b0 2024-09-21T04:09:36.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.215+0000 7fd036ffd640 1 -- 172.21.15.64:0/939973045 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fd0500a8440 con 0x56194658a1b0 2024-09-21T04:09:36.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.215+0000 7fd063d76740 10 client.16710 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T04:09:36.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.215+0000 7fd063d76740 10 client.16710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:09:36.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.215+0000 7fd063d76740 20 client.16710 populate_metadata read hostname 'smithi064' 2024-09-21T04:09:36.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.215+0000 7fd063d76740 1 --2- 172.21.15.64:0/939973045 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5619469e3090 0x5619469e54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:36.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.216+0000 7fd063d76740 4 client.16710 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:09:36.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.216+0000 7fd063d76740 1 -- 172.21.15.64:0/939973045 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5619468cbcd0 con 0x5619469e3090 2024-09-21T04:09:36.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.217+0000 7fd056ffd640 1 --2- 172.21.15.64:0/939973045 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5619469e3090 0x5619469e54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:36.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.226+0000 7fd036ffd640 10 client.16710 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:36.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.226+0000 7fd056ffd640 1 --2- 172.21.15.64:0/939973045 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5619469e3090 0x5619469e54b0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0d2656306b65e4e server_cookie=8f5a29109315aedb in_seq=0 out_seq=0 2024-09-21T04:09:36.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.229+0000 7fd036ffd640 1 -- 172.21.15.64:0/939973045 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5619468cbcd0 con 0x5619469e3090 2024-09-21T04:09:36.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.229+0000 7fd036ffd640 10 client.16710 handle_command_reply: tid=0 2024-09-21T04:09:36.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.229+0000 7fd063d76740 1 client.16710 shutdown 2024-09-21T04:09:36.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.236+0000 7fd063d76740 1 -- 172.21.15.64:0/939973045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56194658a1b0 msgr2=0x56194696a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:36.240 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.236+0000 7fd063d76740 1 --2- 172.21.15.64:0/939973045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56194658a1b0 0x56194696a370 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd0500e4290 tx=0x7fd0500ac350 comp rx=0 tx=0).stop 2024-09-21T04:09:36.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.239+0000 7fd063d76740 1 -- 172.21.15.64:0/939973045 shutdown_connections 2024-09-21T04:09:36.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.239+0000 7fd063d76740 1 --2- 172.21.15.64:0/939973045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561946592390 0x5619469dae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.239+0000 7fd063d76740 1 --2- 172.21.15.64:0/939973045 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5619469e3090 0x5619469e54b0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.239+0000 7fd063d76740 1 --2- 172.21.15.64:0/939973045 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56194658a1b0 0x56194696a370 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.239+0000 7fd063d76740 1 --2- 172.21.15.64:0/939973045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619465719f0 0x561946969d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.239+0000 7fd036ffd640 0 client.16710 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:36.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.239+0000 7fd063d76740 1 -- 172.21.15.64:0/939973045 >> 172.21.15.64:0/939973045 conn(0x5619465a7460 msgr2=0x561946967050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:36.242 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.239+0000 7fd063d76740 1 -- 172.21.15.64:0/939973045 shutdown_connections 2024-09-21T04:09:36.244 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:36 smithi064 ceph-mon[31180]: pgmap v488: 97 pgs: 97 active+clean; 694 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.4 MiB/s wr, 500 op/s 2024-09-21T04:09:36.247 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:09:36.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.241+0000 7fd063d76740 1 -- 172.21.15.64:0/939973045 wait complete. 2024-09-21T04:09:36.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.242+0000 7fd063d76740 20 client.16710 trim_cache size 0 max 16384 2024-09-21T04:09:36.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.243+0000 7fd057fff640 1 -- 172.21.15.64:0/3773867556 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd0400b6880 msgr2=0x7fd0400b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:36.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.243+0000 7fd057fff640 1 --2- 172.21.15.64:0/3773867556 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd0400b6880 0x7fd0400b8d90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd048005ab0 tx=0x7fd048009350 comp rx=0 tx=0).stop 2024-09-21T04:09:36.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.243+0000 7fd057fff640 1 -- 172.21.15.64:0/3773867556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd05811cba0 msgr2=0x7fd05811b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:36.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.243+0000 7fd057fff640 1 --2- 172.21.15.64:0/3773867556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd05811cba0 0x7fd05811b6b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd05000b9b0 tx=0x7fd05000be70 comp rx=0 tx=0).stop 2024-09-21T04:09:36.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.243+0000 7fd057fff640 1 -- 172.21.15.64:0/3773867556 shutdown_connections 2024-09-21T04:09:36.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.243+0000 7fd057fff640 1 --2- 172.21.15.64:0/3773867556 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd0400b6880 0x7fd0400b8d90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.243+0000 7fd057fff640 1 --2- 172.21.15.64:0/3773867556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0581144d0 0x7fd0581149d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.243+0000 7fd057fff640 1 --2- 172.21.15.64:0/3773867556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd05811cba0 0x7fd05811b6b0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.243+0000 7fd057fff640 1 --2- 172.21.15.64:0/3773867556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd05810dbf0 0x7fd05811b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:36.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.243+0000 7fd057fff640 1 -- 172.21.15.64:0/3773867556 >> 172.21.15.64:0/3773867556 conn(0x7fd05806e9f0 msgr2=0x7fd05810c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:36.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.244+0000 7fd057fff640 1 -- 172.21.15.64:0/3773867556 shutdown_connections 2024-09-21T04:09:36.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:36.244+0000 7fd057fff640 1 -- 172.21.15.64:0/3773867556 wait complete. 2024-09-21T04:09:36.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:36 smithi119 ceph-mon[34811]: pgmap v488: 97 pgs: 97 active+clean; 694 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.4 MiB/s wr, 500 op/s 2024-09-21T04:09:36.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:36 smithi131 ceph-mon[35193]: pgmap v488: 97 pgs: 97 active+clean; 694 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.4 MiB/s wr, 500 op/s 2024-09-21T04:09:38.285 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T04:09:38.285 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T04:09:38.359 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:38 smithi064 ceph-mon[31180]: pgmap v489: 97 pgs: 97 active+clean; 706 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.5 MiB/s wr, 550 op/s 2024-09-21T04:09:38.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:38 smithi119 ceph-mon[34811]: pgmap v489: 97 pgs: 97 active+clean; 706 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.5 MiB/s wr, 550 op/s 2024-09-21T04:09:38.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:38 smithi131 ceph-mon[35193]: pgmap v489: 97 pgs: 97 active+clean; 706 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.5 MiB/s wr, 550 op/s 2024-09-21T04:09:38.798 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:09:39.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.558+0000 7f52ae348640 1 -- 172.21.15.64:0/3410040730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52a81081b0 msgr2=0x7f52a81085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:39.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.558+0000 7f52ae348640 1 --2- 172.21.15.64:0/3410040730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52a81081b0 0x7f52a81085c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5298009900 tx=0x7f529802a3f0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.559+0000 7f52ae348640 1 -- 172.21.15.64:0/3410040730 shutdown_connections 2024-09-21T04:09:39.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.559+0000 7f52ae348640 1 --2- 172.21.15.64:0/3410040730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52a8108b00 0x7f52a810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.559+0000 7f52ae348640 1 --2- 172.21.15.64:0/3410040730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52a81081b0 0x7f52a81085c0 secure :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5298009900 tx=0x7f529802a3f0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.559+0000 7f52ae348640 1 --2- 172.21.15.64:0/3410040730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52a8102870 0x7f52a8102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.559+0000 7f52ae348640 1 -- 172.21.15.64:0/3410040730 >> 172.21.15.64:0/3410040730 conn(0x7f52a80fe2c0 msgr2=0x7f52a8100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:39.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.562+0000 7f52ae348640 1 -- 172.21.15.64:0/3410040730 shutdown_connections 2024-09-21T04:09:39.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.562+0000 7f52ae348640 1 -- 172.21.15.64:0/3410040730 wait complete. 2024-09-21T04:09:39.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.563+0000 7f52ae348640 1 Processor -- start 2024-09-21T04:09:39.568 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.563+0000 7f52ae348640 1 -- start start 2024-09-21T04:09:39.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.564+0000 7f52ae348640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52a8102870 0x7f52a8075020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:39.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.564+0000 7f52ae348640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52a8108b00 0x7f52a806e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:39.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.564+0000 7f52ae348640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52a806e890 0x7f52a8073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:39.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.564+0000 7f52ae348640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f52a8111700 con 0x7f52a8108b00 2024-09-21T04:09:39.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.564+0000 7f52ae348640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f52a8111580 con 0x7f52a806e890 2024-09-21T04:09:39.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.564+0000 7f52ae348640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f52a8111880 con 0x7f52a8102870 2024-09-21T04:09:39.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.564+0000 7f52ad346640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52a8102870 0x7f52a8075020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:39.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.564+0000 7f52ad346640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52a8102870 0x7f52a8075020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:59208/0 (socket says 172.21.15.64:59208) 2024-09-21T04:09:39.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.564+0000 7f52ad346640 1 -- 172.21.15.64:0/4104618207 learned_addr learned my addr 172.21.15.64:0/4104618207 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:39.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.565+0000 7f52acb45640 1 --2- 172.21.15.64:0/4104618207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52a8108b00 0x7f52a806e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:39.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.565+0000 7f52ad346640 1 -- 172.21.15.64:0/4104618207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52a806e890 msgr2=0x7f52a8073bd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T04:09:39.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.565+0000 7f52ad346640 1 --2- 172.21.15.64:0/4104618207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52a806e890 0x7f52a8073bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.565+0000 7f52ad346640 1 -- 172.21.15.64:0/4104618207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52a8108b00 msgr2=0x7f52a806e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:39.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.565+0000 7f52ad346640 1 --2- 172.21.15.64:0/4104618207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52a8108b00 0x7f52a806e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.565+0000 7f52ad346640 1 -- 172.21.15.64:0/4104618207 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a806efe0 con 0x7f52a8102870 2024-09-21T04:09:39.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.565+0000 7f52acb45640 1 --2- 172.21.15.64:0/4104618207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52a8108b00 0x7f52a806e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:09:39.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.566+0000 7f52ad346640 1 --2- 172.21.15.64:0/4104618207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52a8102870 0x7f52a8075020 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f52a000cce0 tx=0x7f52a000a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:39.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.574+0000 7f529e7fc640 1 -- 172.21.15.64:0/4104618207 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a000f650 con 0x7f52a8102870 2024-09-21T04:09:39.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.574+0000 7f52ae348640 1 -- 172.21.15.64:0/4104618207 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52a80741d0 con 0x7f52a8102870 2024-09-21T04:09:39.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.574+0000 7f52ae348640 1 -- 172.21.15.64:0/4104618207 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52a80746e0 con 0x7f52a8102870 2024-09-21T04:09:39.577 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.574+0000 7f529e7fc640 1 -- 172.21.15.64:0/4104618207 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52a000fca0 con 0x7f52a8102870 2024-09-21T04:09:39.579 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.575+0000 7f529e7fc640 1 -- 172.21.15.64:0/4104618207 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0018b60 con 0x7f52a8102870 2024-09-21T04:09:39.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.583+0000 7f529e7fc640 1 -- 172.21.15.64:0/4104618207 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f52a0018400 con 0x7f52a8102870 2024-09-21T04:09:39.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.585+0000 7f529e7fc640 1 --2- 172.21.15.64:0/4104618207 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f52940bac90 0x7f52940bd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:39.596 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.585+0000 7f52acb45640 1 --2- 172.21.15.64:0/4104618207 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f52940bac90 0x7f52940bd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:39.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.585+0000 7f529e7fc640 1 -- 172.21.15.64:0/4104618207 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f52a00e7a70 con 0x7f52a8102870 2024-09-21T04:09:39.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.610+0000 7f52b5b1e740 1 Processor -- start 2024-09-21T04:09:39.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.610+0000 7f52b5b1e740 1 -- start start 2024-09-21T04:09:39.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.610+0000 7f52b5b1e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a9812f350 0x559a984d6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:39.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.610+0000 7f52b5b1e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a98106490 0x559a984d6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:39.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.610+0000 7f52b5b1e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a9810e670 0x559a984d7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:39.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.610+0000 7f52b5b1e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559a9844ce60 con 0x559a9810e670 2024-09-21T04:09:39.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.610+0000 7f52b5b1e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559a9843c290 con 0x559a98106490 2024-09-21T04:09:39.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.610+0000 7f52b5b1e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559a984400a0 con 0x559a9812f350 2024-09-21T04:09:39.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.611+0000 7f52acb45640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a98106490 0x559a984d6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:39.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.611+0000 7f52acb45640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a98106490 0x559a984d6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:50446/0 (socket says 172.21.15.64:50446) 2024-09-21T04:09:39.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.611+0000 7f52acb45640 1 -- 172.21.15.64:0/1807449151 learned_addr learned my addr 172.21.15.64:0/1807449151 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:39.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.611+0000 7f52ad346640 1 --2- 172.21.15.64:0/1807449151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a9812f350 0x559a984d6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:39.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.613+0000 7f52acb45640 1 --2- 172.21.15.64:0/4104618207 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f52940bac90 0x7f52940bd1a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5298008000 tx=0x7f5298031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:39.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.613+0000 7f52ad346640 1 -- 172.21.15.64:0/1807449151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a98106490 msgr2=0x559a984d6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:39.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.613+0000 7f52ad346640 1 --2- 172.21.15.64:0/1807449151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a98106490 0x559a984d6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.613+0000 7f52ad346640 1 -- 172.21.15.64:0/1807449151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a9810e670 msgr2=0x559a984d7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:39.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.613+0000 7f52ad346640 1 --2- 172.21.15.64:0/1807449151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a9810e670 0x559a984d7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.613+0000 7f52ad346640 1 -- 172.21.15.64:0/1807449151 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a9819a930 con 0x559a9812f350 2024-09-21T04:09:39.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.616+0000 7f52ad346640 1 --2- 172.21.15.64:0/1807449151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a9812f350 0x559a984d6610 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f52a00e5130 tx=0x7f52a00a90d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a4f9ffb79e3a67e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:39.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.617+0000 7f52937fe640 1 -- 172.21.15.64:0/1807449151 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a00ad340 con 0x559a9812f350 2024-09-21T04:09:39.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.617+0000 7f52937fe640 1 -- 172.21.15.64:0/1807449151 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52a00adf30 con 0x559a9812f350 2024-09-21T04:09:39.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.618+0000 7f52b5b1e740 1 -- 172.21.15.64:0/1807449151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a9812f350 msgr2=0x559a984d6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:39.621 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.618+0000 7f52b5b1e740 1 --2- 172.21.15.64:0/1807449151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a9812f350 0x559a984d6610 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f52a00e5130 tx=0x7f52a00a90d0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.630+0000 7f52b5b1e740 1 -- 172.21.15.64:0/1807449151 shutdown_connections 2024-09-21T04:09:39.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.630+0000 7f52b5b1e740 1 --2- 172.21.15.64:0/1807449151 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a9810e670 0x559a984d7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.630+0000 7f52b5b1e740 1 --2- 172.21.15.64:0/1807449151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a98106490 0x559a984d6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.630+0000 7f52b5b1e740 1 --2- 172.21.15.64:0/1807449151 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a9812f350 0x559a984d6610 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.633 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.630+0000 7f52b5b1e740 1 -- 172.21.15.64:0/1807449151 >> 172.21.15.64:0/1807449151 conn(0x559a98123740 msgr2=0x559a984d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:39.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.636+0000 7f52b5b1e740 1 -- 172.21.15.64:0/1807449151 shutdown_connections 2024-09-21T04:09:39.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.636+0000 7f52b5b1e740 1 -- 172.21.15.64:0/1807449151 wait complete. 2024-09-21T04:09:39.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.637+0000 7f52b5b1e740 1 -- start start 2024-09-21T04:09:39.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.637+0000 7f52b5b1e740 1 Processor -- start 2024-09-21T04:09:39.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.639+0000 7f52b5b1e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a98106490 0x559a984eb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:39.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.639+0000 7f52b5b1e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a9810e670 0x559a984eb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:39.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.640+0000 7f52b5b1e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a9812f350 0x559a984f06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:39.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.641+0000 7f52acb45640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a9810e670 0x559a984eb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:39.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.641+0000 7f52acb45640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a9810e670 0x559a984eb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:50458/0 (socket says 172.21.15.64:50458) 2024-09-21T04:09:39.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.643+0000 7f52b5b1e740 1 -- 172.21.15.64:0/3395780723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559a9829d890 con 0x559a9812f350 2024-09-21T04:09:39.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.643+0000 7f52b5b1e740 1 -- 172.21.15.64:0/3395780723 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559a983d6d40 con 0x559a9810e670 2024-09-21T04:09:39.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.643+0000 7f52b5b1e740 1 -- 172.21.15.64:0/3395780723 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559a984e4de0 con 0x559a98106490 2024-09-21T04:09:39.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.643+0000 7f52adb47640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a9812f350 0x559a984f06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:39.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.643+0000 7f52acb45640 1 -- 172.21.15.64:0/3395780723 learned_addr learned my addr 172.21.15.64:0/3395780723 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:39.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.644+0000 7f52b5b1e740 10 client.0 _ll_register_callbacks cb 0x559a9843ac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:09:39.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.645+0000 7f52adb47640 1 -- 172.21.15.64:0/3395780723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a98106490 msgr2=0x559a984eb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:39.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.642+0000 7f52ad346640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a98106490 0x559a984eb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:39.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.649+0000 7f52adb47640 1 --2- 172.21.15.64:0/3395780723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a98106490 0x559a984eb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.649+0000 7f52adb47640 1 -- 172.21.15.64:0/3395780723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a9810e670 msgr2=0x559a984eb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:39.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.649+0000 7f52adb47640 1 --2- 172.21.15.64:0/3395780723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a9810e670 0x559a984eb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.649+0000 7f52adb47640 1 -- 172.21.15.64:0/3395780723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a984f0d10 con 0x559a9812f350 2024-09-21T04:09:39.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.649+0000 7f52b5b1e740 1 -- 172.21.15.64:0/3395780723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52a00adf30 con 0x559a9812f350 2024-09-21T04:09:39.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.649+0000 7f52acb45640 1 --2- 172.21.15.64:0/3395780723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a9810e670 0x559a984eb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:09:39.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.649+0000 7f52ad346640 1 --2- 172.21.15.64:0/3395780723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a98106490 0x559a984eb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:09:39.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.649+0000 7f52adb47640 1 --2- 172.21.15.64:0/3395780723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a9812f350 0x559a984f06a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f52a4005fd0 tx=0x7f52a400bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770903e28599c951 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:39.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.649+0000 7f5290ff9640 10 client.16719 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:09:39.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.650+0000 7f5290ff9640 1 -- 172.21.15.64:0/3395780723 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4004030 con 0x559a9812f350 2024-09-21T04:09:39.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.650+0000 7f5290ff9640 1 -- 172.21.15.64:0/3395780723 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f52a40041d0 con 0x559a9812f350 2024-09-21T04:09:39.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.650+0000 7f5290ff9640 1 -- 172.21.15.64:0/3395780723 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f52a4004430 con 0x559a9812f350 2024-09-21T04:09:39.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.655+0000 7f52b5b1e740 10 client.16719 fetch_fsmap learned FSMap version 35 2024-09-21T04:09:39.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.655+0000 7f52b5b1e740 1 -- 172.21.15.64:0/3395780723 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({fsmap=35}) -- 0x559a984f9720 con 0x559a9812f350 2024-09-21T04:09:39.660 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.657+0000 7f5290ff9640 1 -- 172.21.15.64:0/3395780723 <== mon.0 v2:172.21.15.64:3300/0 4 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f52a40035f0 con 0x559a9812f350 2024-09-21T04:09:39.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.657+0000 7f52b5b1e740 10 client.16719 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T04:09:39.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.657+0000 7f52b5b1e740 10 client.16719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:09:39.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.657+0000 7f52b5b1e740 20 client.16719 populate_metadata read hostname 'smithi064' 2024-09-21T04:09:39.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.658+0000 7f52b5b1e740 1 --2- 172.21.15.64:0/3395780723 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x559a980edcd0 0x559a984fd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:39.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.658+0000 7f52b5b1e740 4 client.16719 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:09:39.661 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.658+0000 7f52b5b1e740 1 -- 172.21.15.64:0/3395780723 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a9844edb0 con 0x559a980edcd0 2024-09-21T04:09:39.663 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.660+0000 7f52ad346640 1 --2- 172.21.15.64:0/3395780723 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x559a980edcd0 0x559a984fd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:39.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.664+0000 7f52ad346640 1 --2- 172.21.15.64:0/3395780723 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x559a980edcd0 0x559a984fd070 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be38bb2604306f2 server_cookie=20a3953da4a8410e in_seq=0 out_seq=0 2024-09-21T04:09:39.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.666+0000 7f5290ff9640 10 client.16719 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:39.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.670+0000 7f5290ff9640 1 -- 172.21.15.64:0/3395780723 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559a9844edb0 con 0x559a980edcd0 2024-09-21T04:09:39.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.670+0000 7f5290ff9640 10 client.16719 handle_command_reply: tid=0 2024-09-21T04:09:39.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.670+0000 7f52b5b1e740 1 client.16719 shutdown 2024-09-21T04:09:39.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.671+0000 7f52b5b1e740 1 -- 172.21.15.64:0/3395780723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a9812f350 msgr2=0x559a984f06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:39.674 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.671+0000 7f52b5b1e740 1 --2- 172.21.15.64:0/3395780723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a9812f350 0x559a984f06a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f52a4005fd0 tx=0x7f52a400bf80 comp rx=0 tx=0).stop 2024-09-21T04:09:39.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.676+0000 7f52b5b1e740 1 -- 172.21.15.64:0/3395780723 shutdown_connections 2024-09-21T04:09:39.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.676+0000 7f52b5b1e740 1 --2- 172.21.15.64:0/3395780723 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x559a980edcd0 0x559a984fd070 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.676+0000 7f52b5b1e740 1 --2- 172.21.15.64:0/3395780723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a9812f350 0x559a984f06a0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.676+0000 7f52b5b1e740 1 --2- 172.21.15.64:0/3395780723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a9810e670 0x559a984eb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.676+0000 7f52b5b1e740 1 --2- 172.21.15.64:0/3395780723 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a98106490 0x559a984eb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.676+0000 7f5290ff9640 0 client.16719 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:39.679 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.676+0000 7f52b5b1e740 1 -- 172.21.15.64:0/3395780723 >> 172.21.15.64:0/3395780723 conn(0x559a98123740 msgr2=0x559a984d97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:39.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.684+0000 7f52b5b1e740 1 -- 172.21.15.64:0/3395780723 shutdown_connections 2024-09-21T04:09:39.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.688+0000 7f52b5b1e740 1 -- 172.21.15.64:0/3395780723 wait complete. 2024-09-21T04:09:39.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.690+0000 7f52b5b1e740 20 client.16719 trim_cache size 0 max 16384 2024-09-21T04:09:39.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.730+0000 7f52b5b1e740 1 Processor -- start 2024-09-21T04:09:39.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.731+0000 7f52b5b1e740 1 -- start start 2024-09-21T04:09:39.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.731+0000 7f52b5b1e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a98106490 0x559a984e4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:39.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.732+0000 7f52b5b1e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a9810e670 0x559a984e5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:39.736 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.732+0000 7f52ad346640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a98106490 0x559a984e4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.732+0000 7f52ad346640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a98106490 0x559a984e4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:59236/0 (socket says 172.21.15.64:59236) 2024-09-21T04:09:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.732+0000 7f52ad346640 1 -- 172.21.15.64:0/1696016094 learned_addr learned my addr 172.21.15.64:0/1696016094 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.732+0000 7f52b5b1e740 1 --2- 172.21.15.64:0/1696016094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a9812f350 0x559a984e55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.732+0000 7f52b5b1e740 1 -- 172.21.15.64:0/1696016094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f528c011340 con 0x559a9810e670 2024-09-21T04:09:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.732+0000 7f52b5b1e740 1 -- 172.21.15.64:0/1696016094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f528c00ebb0 con 0x559a9812f350 2024-09-21T04:09:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.732+0000 7f52b5b1e740 1 -- 172.21.15.64:0/1696016094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f528c00ca00 con 0x559a98106490 2024-09-21T04:09:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.733+0000 7f52acb45640 1 --2- 172.21.15.64:0/1696016094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a9810e670 0x559a984e5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.734+0000 7f52acb45640 1 -- 172.21.15.64:0/1696016094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a98106490 msgr2=0x559a984e4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.734+0000 7f52acb45640 1 --2- 172.21.15.64:0/1696016094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a98106490 0x559a984e4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.734+0000 7f52acb45640 1 -- 172.21.15.64:0/1696016094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a9812f350 msgr2=0x559a984e55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.734+0000 7f52adb47640 1 --2- 172.21.15.64:0/1696016094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a9812f350 0x559a984e55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.734+0000 7f52acb45640 1 --2- 172.21.15.64:0/1696016094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a9812f350 0x559a984e55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.737 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.734+0000 7f52acb45640 1 -- 172.21.15.64:0/1696016094 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a984e6730 con 0x559a9810e670 2024-09-21T04:09:39.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.734+0000 7f52acb45640 1 --2- 172.21.15.64:0/1696016094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a9810e670 0x559a984e5070 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5298005f70 tx=0x7f529802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8f2f4e24999cb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:39.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.735+0000 7f5292ffd640 1 -- 172.21.15.64:0/1696016094 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298035050 con 0x559a9810e670 2024-09-21T04:09:39.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.735+0000 7f5292ffd640 1 -- 172.21.15.64:0/1696016094 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5298070070 con 0x559a9810e670 2024-09-21T04:09:39.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.735+0000 7f5292ffd640 1 -- 172.21.15.64:0/1696016094 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298003e60 con 0x559a9810e670 2024-09-21T04:09:39.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.740+0000 7f52b5b1e740 1 -- 172.21.15.64:0/1696016094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a9810e670 msgr2=0x559a984e5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:39.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.741+0000 7f52b5b1e740 1 --2- 172.21.15.64:0/1696016094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a9810e670 0x559a984e5070 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5298005f70 tx=0x7f529802a3f0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.749+0000 7f52b5b1e740 1 -- 172.21.15.64:0/1696016094 shutdown_connections 2024-09-21T04:09:39.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.749+0000 7f52b5b1e740 1 --2- 172.21.15.64:0/1696016094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a9812f350 0x559a984e55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.749+0000 7f52b5b1e740 1 --2- 172.21.15.64:0/1696016094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a9810e670 0x559a984e5070 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.749+0000 7f52b5b1e740 1 --2- 172.21.15.64:0/1696016094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a98106490 0x559a984e4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.752 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.749+0000 7f52b5b1e740 1 -- 172.21.15.64:0/1696016094 >> 172.21.15.64:0/1696016094 conn(0x559a98123740 msgr2=0x559a984e9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:39.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.757+0000 7f52b5b1e740 1 -- 172.21.15.64:0/1696016094 shutdown_connections 2024-09-21T04:09:39.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.763+0000 7f52b5b1e740 1 -- 172.21.15.64:0/1696016094 wait complete. 2024-09-21T04:09:39.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.764+0000 7f52b5b1e740 1 -- start start 2024-09-21T04:09:39.767 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.764+0000 7f52b5b1e740 1 Processor -- start 2024-09-21T04:09:39.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.769+0000 7f52b5b1e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a98106490 0x559a984ee960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:39.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.769+0000 7f52b5b1e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a9810e670 0x559a984eef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:39.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.770+0000 7f52acb45640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a9810e670 0x559a984eef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:39.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.770+0000 7f52acb45640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a9810e670 0x559a984eef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58248/0 (socket says 172.21.15.64:58248) 2024-09-21T04:09:39.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.770+0000 7f52acb45640 1 -- 172.21.15.64:0/1285867124 learned_addr learned my addr 172.21.15.64:0/1285867124 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:39.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.770+0000 7f52b5b1e740 1 --2- 172.21.15.64:0/1285867124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a9812f350 0x559a984f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:39.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.770+0000 7f52b5b1e740 1 -- 172.21.15.64:0/1285867124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f528c00d070 con 0x559a9810e670 2024-09-21T04:09:39.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.771+0000 7f52b5b1e740 1 -- 172.21.15.64:0/1285867124 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f528c00e4e0 con 0x559a9812f350 2024-09-21T04:09:39.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.771+0000 7f52ad346640 1 --2- 172.21.15.64:0/1285867124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a98106490 0x559a984ee960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:39.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.771+0000 7f52b5b1e740 1 -- 172.21.15.64:0/1285867124 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f528c00d740 con 0x559a98106490 2024-09-21T04:09:39.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.771+0000 7f52b5b1e740 10 client.0 _ll_register_callbacks cb 0x7f52a400b5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:09:39.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.772+0000 7f52adb47640 1 --2- 172.21.15.64:0/1285867124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a9812f350 0x559a984f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:39.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.773+0000 7f52acb45640 1 -- 172.21.15.64:0/1285867124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a98106490 msgr2=0x559a984ee960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:39.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.773+0000 7f52acb45640 1 --2- 172.21.15.64:0/1285867124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a98106490 0x559a984ee960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.773+0000 7f52acb45640 1 -- 172.21.15.64:0/1285867124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a9812f350 msgr2=0x559a984f48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:39.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.773+0000 7f52acb45640 1 --2- 172.21.15.64:0/1285867124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a9812f350 0x559a984f48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.773+0000 7f52acb45640 1 -- 172.21.15.64:0/1285867124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a984f4f30 con 0x559a9810e670 2024-09-21T04:09:39.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.773+0000 7f52acb45640 1 --2- 172.21.15.64:0/1285867124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a9810e670 0x559a984eef20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5298004d70 tx=0x7f5298005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ee62c76b786e77 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:39.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.773+0000 7f5290ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:09:39.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.774+0000 7f5290ff9640 1 -- 172.21.15.64:0/1285867124 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298082060 con 0x559a9810e670 2024-09-21T04:09:39.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.774+0000 7f52b5b1e740 1 -- 172.21.15.64:0/1285867124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5298070070 con 0x559a9810e670 2024-09-21T04:09:39.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.774+0000 7f5290ff9640 1 -- 172.21.15.64:0/1285867124 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f5298003a00 con 0x559a9810e670 2024-09-21T04:09:39.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.774+0000 7f5290ff9640 1 -- 172.21.15.64:0/1285867124 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5298035050 con 0x559a9810e670 2024-09-21T04:09:39.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.776+0000 7f52ad346640 1 --2- 172.21.15.64:0/1285867124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a98106490 0x559a984ee960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:09:39.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.777+0000 7f52b5b1e740 10 client.16725 fetch_fsmap learned FSMap version 35 2024-09-21T04:09:39.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.777+0000 7f52b5b1e740 1 -- 172.21.15.64:0/1285867124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559a984f6330 con 0x559a9810e670 2024-09-21T04:09:39.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.785+0000 7f5290ff9640 1 -- 172.21.15.64:0/1285867124 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5298070070 con 0x559a9810e670 2024-09-21T04:09:39.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.785+0000 7f5290ff9640 1 -- 172.21.15.64:0/1285867124 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f5298075470 con 0x559a9810e670 2024-09-21T04:09:39.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.785+0000 7f52b5b1e740 10 client.16725 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T04:09:39.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.785+0000 7f52b5b1e740 10 client.16725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:09:39.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.785+0000 7f52b5b1e740 20 client.16725 populate_metadata read hostname 'smithi064' 2024-09-21T04:09:39.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.785+0000 7f52b5b1e740 1 --2- 172.21.15.64:0/1285867124 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x559a980c3080 0x559a984f6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:39.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.785+0000 7f52b5b1e740 4 client.16725 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T04:09:39.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.785+0000 7f52b5b1e740 1 -- 172.21.15.64:0/1285867124 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559a9844a860 con 0x559a980c3080 2024-09-21T04:09:39.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.789+0000 7f52ad346640 1 --2- 172.21.15.64:0/1285867124 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x559a980c3080 0x559a984f6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:39.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.808+0000 7f52ad346640 1 --2- 172.21.15.64:0/1285867124 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x559a980c3080 0x559a984f6a20 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e06898a1e45a7b81 server_cookie=3f1b8beed692841e in_seq=0 out_seq=0 2024-09-21T04:09:39.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.808+0000 7f5290ff9640 10 client.16725 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:39.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.815+0000 7f5290ff9640 1 -- 172.21.15.64:0/1285867124 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559a9844a860 con 0x559a980c3080 2024-09-21T04:09:39.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.815+0000 7f5290ff9640 10 client.16725 handle_command_reply: tid=0 2024-09-21T04:09:39.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.816+0000 7f52b5b1e740 1 client.16725 shutdown 2024-09-21T04:09:39.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.822+0000 7f52b5b1e740 1 -- 172.21.15.64:0/1285867124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a9810e670 msgr2=0x559a984eef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:39.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.822+0000 7f52b5b1e740 1 --2- 172.21.15.64:0/1285867124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a9810e670 0x559a984eef20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5298004d70 tx=0x7f5298005fd0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.822+0000 7f52b5b1e740 1 -- 172.21.15.64:0/1285867124 shutdown_connections 2024-09-21T04:09:39.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.822+0000 7f52b5b1e740 1 --2- 172.21.15.64:0/1285867124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a9812f350 0x559a984f48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.822+0000 7f52b5b1e740 1 --2- 172.21.15.64:0/1285867124 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x559a980c3080 0x559a984f6a20 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.822+0000 7f52b5b1e740 1 --2- 172.21.15.64:0/1285867124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a9810e670 0x559a984eef20 secure :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5298004d70 tx=0x7f5298005fd0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.822+0000 7f52b5b1e740 1 --2- 172.21.15.64:0/1285867124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559a98106490 0x559a984ee960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.822+0000 7f5290ff9640 0 client.16725 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:39.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.822+0000 7f52b5b1e740 1 -- 172.21.15.64:0/1285867124 >> 172.21.15.64:0/1285867124 conn(0x559a98123740 msgr2=0x559a984e9660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:39.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.822+0000 7f52b5b1e740 1 -- 172.21.15.64:0/1285867124 shutdown_connections 2024-09-21T04:09:39.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.823+0000 7f52b5b1e740 1 -- 172.21.15.64:0/1285867124 wait complete. 2024-09-21T04:09:39.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.824+0000 7f52b5b1e740 20 client.16725 trim_cache size 0 max 16384 2024-09-21T04:09:39.828 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:09:39.828 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-21T04:09:39.828 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "2c0523fb-aae4-43fa-9568-136a9b640051", 2024-09-21T04:09:39.828 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-21T04:09:39.828 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:09:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.825+0000 7f52ae348640 1 -- 172.21.15.64:0/4104618207 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f52940bac90 msgr2=0x7f52940bd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.825+0000 7f52ae348640 1 --2- 172.21.15.64:0/4104618207 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f52940bac90 0x7f52940bd1a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5298008000 tx=0x7f5298031400 comp rx=0 tx=0).stop 2024-09-21T04:09:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.825+0000 7f52ae348640 1 -- 172.21.15.64:0/4104618207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52a8102870 msgr2=0x7f52a8075020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.825+0000 7f52ae348640 1 --2- 172.21.15.64:0/4104618207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52a8102870 0x7f52a8075020 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f52a000cce0 tx=0x7f52a000a620 comp rx=0 tx=0).stop 2024-09-21T04:09:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.826+0000 7f52ae348640 1 -- 172.21.15.64:0/4104618207 shutdown_connections 2024-09-21T04:09:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.826+0000 7f52ae348640 1 --2- 172.21.15.64:0/4104618207 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f52940bac90 0x7f52940bd1a0 secure :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5298008000 tx=0x7f5298031400 comp rx=0 tx=0).stop 2024-09-21T04:09:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.826+0000 7f52ae348640 1 --2- 172.21.15.64:0/4104618207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52a806e890 0x7f52a8073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.826+0000 7f52ae348640 1 --2- 172.21.15.64:0/4104618207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52a8108b00 0x7f52a806e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.826+0000 7f52ae348640 1 --2- 172.21.15.64:0/4104618207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f52a8102870 0x7f52a8075020 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.826+0000 7f52ae348640 1 -- 172.21.15.64:0/4104618207 >> 172.21.15.64:0/4104618207 conn(0x7f52a80fe2c0 msgr2=0x7f52a81006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.826+0000 7f52ae348640 1 -- 172.21.15.64:0/4104618207 shutdown_connections 2024-09-21T04:09:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:39.826+0000 7f52ae348640 1 -- 172.21.15.64:0/4104618207 wait complete. 2024-09-21T04:09:40.063 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:40 smithi064 ceph-mon[31180]: pgmap v490: 97 pgs: 97 active+clean; 711 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.9 MiB/s wr, 439 op/s 2024-09-21T04:09:40.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:40 smithi119 ceph-mon[34811]: pgmap v490: 97 pgs: 97 active+clean; 711 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.9 MiB/s wr, 439 op/s 2024-09-21T04:09:40.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:40 smithi131 ceph-mon[35193]: pgmap v490: 97 pgs: 97 active+clean; 711 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.9 MiB/s wr, 439 op/s 2024-09-21T04:09:41.174 INFO:tasks.workunit.client.0.smithi064.stdout: 113 1721031 501 1257344 420 1028839 1554 2024-09-21T04:09:41.296 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T04:09:41.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:41 smithi119 ceph-mon[34811]: scrub queued for path: / 2024-09-21T04:09:41.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:41 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:41.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:41 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:09:41.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:41 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:09:41.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:41 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:09:41.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:41 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:41.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:41 smithi064 ceph-mon[31180]: scrub queued for path: / 2024-09-21T04:09:41.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:41 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:41.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:41 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:09:41.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:41 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:09:41.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:41 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:09:41.513 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:41 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:41.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:41 smithi131 ceph-mon[35193]: scrub queued for path: / 2024-09-21T04:09:41.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:41 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:41.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:41 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:09:41.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:41 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:09:41.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:41 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:09:41.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:41 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:42.263 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:42 smithi064 ceph-mon[31180]: pgmap v491: 97 pgs: 97 active+clean; 704 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 351 op/s 2024-09-21T04:09:42.264 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:42 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:09:42.264 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:42 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.442+0000 7f2ea8a97640 1 -- 172.21.15.64:0/1468748575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ea4111ca0 msgr2=0x7f2ea410a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.442+0000 7f2ea8a97640 1 --2- 172.21.15.64:0/1468748575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ea4111ca0 0x7f2ea410a3c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c009be0 tx=0x7f2e9c02a6d0 comp rx=0 tx=0).stop 2024-09-21T04:09:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.443+0000 7f2ea8a97640 1 -- 172.21.15.64:0/1468748575 shutdown_connections 2024-09-21T04:09:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.443+0000 7f2ea8a97640 1 --2- 172.21.15.64:0/1468748575 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ea411cba0 0x7f2ea411efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.443+0000 7f2ea8a97640 1 --2- 172.21.15.64:0/1468748575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ea410a900 0x7f2ea410ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.443+0000 7f2ea8a97640 1 --2- 172.21.15.64:0/1468748575 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ea4111ca0 0x7f2ea410a3c0 secure :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c009be0 tx=0x7f2e9c02a6d0 comp rx=0 tx=0).stop 2024-09-21T04:09:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.443+0000 7f2ea8a97640 1 -- 172.21.15.64:0/1468748575 >> 172.21.15.64:0/1468748575 conn(0x7f2ea4070c00 msgr2=0x7f2ea4071010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.450+0000 7f2ea8a97640 1 -- 172.21.15.64:0/1468748575 shutdown_connections 2024-09-21T04:09:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.459+0000 7f2ea8a97640 1 -- 172.21.15.64:0/1468748575 wait complete. 2024-09-21T04:09:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.460+0000 7f2ea8a97640 1 Processor -- start 2024-09-21T04:09:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.460+0000 7f2ea8a97640 1 -- start start 2024-09-21T04:09:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.461+0000 7f2ea8a97640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ea410a900 0x7f2ea411c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.461+0000 7f2ea8a97640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ea411cba0 0x7f2ea4116760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.461+0000 7f2ea8a97640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ea4116d70 0x7f2ea4117230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.461+0000 7f2ea8a97640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea4112ae0 con 0x7f2ea411cba0 2024-09-21T04:09:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.461+0000 7f2ea8a97640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea4112960 con 0x7f2ea410a900 2024-09-21T04:09:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.461+0000 7f2ea8a97640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea4112c60 con 0x7f2ea4116d70 2024-09-21T04:09:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.461+0000 7f2ea37fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ea410a900 0x7f2ea411c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.461+0000 7f2ea37fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ea410a900 0x7f2ea411c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:49338/0 (socket says 172.21.15.64:49338) 2024-09-21T04:09:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.461+0000 7f2ea37fe640 1 -- 172.21.15.64:0/1509120499 learned_addr learned my addr 172.21.15.64:0/1509120499 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.461+0000 7f2ea3fff640 1 --2- 172.21.15.64:0/1509120499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ea4116d70 0x7f2ea4117230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.461+0000 7f2ea37fe640 1 -- 172.21.15.64:0/1509120499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ea4116d70 msgr2=0x7f2ea4117230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.461+0000 7f2ea37fe640 1 --2- 172.21.15.64:0/1509120499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ea4116d70 0x7f2ea4117230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.461+0000 7f2ea37fe640 1 -- 172.21.15.64:0/1509120499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ea411cba0 msgr2=0x7f2ea4116760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.461+0000 7f2ea37fe640 1 --2- 172.21.15.64:0/1509120499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ea411cba0 0x7f2ea4116760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.462+0000 7f2ea37fe640 1 -- 172.21.15.64:0/1509120499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea41bdf50 con 0x7f2ea410a900 2024-09-21T04:09:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.462+0000 7f2ea37fe640 1 --2- 172.21.15.64:0/1509120499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ea410a900 0x7f2ea411c570 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c02ac40 tx=0x7f2e9c032e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.482+0000 7f2ea0ff9640 1 -- 172.21.15.64:0/1509120499 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c041020 con 0x7f2ea410a900 2024-09-21T04:09:42.484 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.482+0000 7f2ea8a97640 1 -- 172.21.15.64:0/1509120499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ea41be1e0 con 0x7f2ea410a900 2024-09-21T04:09:42.487 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.482+0000 7f2ea0ff9640 1 -- 172.21.15.64:0/1509120499 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e9c030e20 con 0x7f2ea410a900 2024-09-21T04:09:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.482+0000 7f2ea8a97640 1 -- 172.21.15.64:0/1509120499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ea41be650 con 0x7f2ea410a900 2024-09-21T04:09:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.485+0000 7f2ea0ff9640 1 -- 172.21.15.64:0/1509120499 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c036040 con 0x7f2ea410a900 2024-09-21T04:09:42.495 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.493+0000 7f2eb0c65740 1 Processor -- start 2024-09-21T04:09:42.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.493+0000 7f2eb0c65740 1 -- start start 2024-09-21T04:09:42.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.494+0000 7f2eb0c65740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560566c191b0 0x560566fbeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:42.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.494+0000 7f2eb0c65740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560566bf02f0 0x560566fbf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:42.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.494+0000 7f2eb0c65740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560566bf84d0 0x560566fbf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:42.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.494+0000 7f2eb0c65740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560566f336f0 con 0x560566bf84d0 2024-09-21T04:09:42.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.494+0000 7f2eb0c65740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560566f22b20 con 0x560566c191b0 2024-09-21T04:09:42.496 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.494+0000 7f2eb0c65740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560566f26930 con 0x560566bf02f0 2024-09-21T04:09:42.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.494+0000 7f2ea3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560566c191b0 0x560566fbeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:42.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.494+0000 7f2ea3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560566c191b0 0x560566fbeba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:49350/0 (socket says 172.21.15.64:49350) 2024-09-21T04:09:42.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.494+0000 7f2ea3fff640 1 -- 172.21.15.64:0/138554718 learned_addr learned my addr 172.21.15.64:0/138554718 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:42.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.494+0000 7f2ea2ffd640 1 --2- 172.21.15.64:0/138554718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560566bf84d0 0x560566fbf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:42.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.495+0000 7f2ea3fff640 1 -- 172.21.15.64:0/138554718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560566bf02f0 msgr2=0x560566fbf0e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:09:42.497 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.495+0000 7f2ea3fff640 1 --2- 172.21.15.64:0/138554718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560566bf02f0 0x560566fbf0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.495+0000 7f2ea3fff640 1 -- 172.21.15.64:0/138554718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560566bf84d0 msgr2=0x560566fbf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.495+0000 7f2ea3fff640 1 --2- 172.21.15.64:0/138554718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560566bf84d0 0x560566fbf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.495+0000 7f2ea3fff640 1 -- 172.21.15.64:0/138554718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560566ee8530 con 0x560566c191b0 2024-09-21T04:09:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.495+0000 7f2ea2ffd640 1 --2- 172.21.15.64:0/138554718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560566bf84d0 0x560566fbf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:09:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.495+0000 7f2ea3fff640 1 --2- 172.21.15.64:0/138554718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560566c191b0 0x560566fbeba0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2e9400bc70 tx=0x7f2e940107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5200ba31937e8185 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.495+0000 7f2ea0ff9640 1 -- 172.21.15.64:0/1509120499 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f2e9c044020 con 0x7f2ea410a900 2024-09-21T04:09:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.497+0000 7f2ea0ff9640 1 --2- 172.21.15.64:0/1509120499 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2e8c0b6880 0x7f2e8c0b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.498+0000 7f2ea0ff9640 1 -- 172.21.15.64:0/1509120499 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f2e9c108120 con 0x7f2ea410a900 2024-09-21T04:09:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.498+0000 7f2ea37fe640 1 --2- 172.21.15.64:0/1509120499 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2e8c0b6880 0x7f2e8c0b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.499+0000 7f2ea37fe640 1 --2- 172.21.15.64:0/1509120499 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2e8c0b6880 0x7f2e8c0b8d90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x560566fbd820 tx=0x7f2e9c009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:42.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:42 smithi119 ceph-mon[34811]: pgmap v491: 97 pgs: 97 active+clean; 704 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 351 op/s 2024-09-21T04:09:42.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:42 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:09:42.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:42 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.510+0000 7f2e89ffb640 1 -- 172.21.15.64:0/138554718 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9401c480 con 0x560566c191b0 2024-09-21T04:09:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.512+0000 7f2e89ffb640 1 -- 172.21.15.64:0/138554718 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e9401cad0 con 0x560566c191b0 2024-09-21T04:09:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.513+0000 7f2eb0c65740 1 -- 172.21.15.64:0/138554718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560566c191b0 msgr2=0x560566fbeba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:42.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.513+0000 7f2eb0c65740 1 --2- 172.21.15.64:0/138554718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560566c191b0 0x560566fbeba0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2e9400bc70 tx=0x7f2e940107a0 comp rx=0 tx=0).stop 2024-09-21T04:09:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.523+0000 7f2eb0c65740 1 -- 172.21.15.64:0/138554718 shutdown_connections 2024-09-21T04:09:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.523+0000 7f2eb0c65740 1 --2- 172.21.15.64:0/138554718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560566bf84d0 0x560566fbf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.523+0000 7f2eb0c65740 1 --2- 172.21.15.64:0/138554718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560566bf02f0 0x560566fbf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:42.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.523+0000 7f2eb0c65740 1 --2- 172.21.15.64:0/138554718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560566c191b0 0x560566fbeba0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:42.526 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.523+0000 7f2eb0c65740 1 -- 172.21.15.64:0/138554718 >> 172.21.15.64:0/138554718 conn(0x560566c0d300 msgr2=0x560566fbca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.527+0000 7f2eb0c65740 1 -- 172.21.15.64:0/138554718 shutdown_connections 2024-09-21T04:09:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.527+0000 7f2eb0c65740 1 -- 172.21.15.64:0/138554718 wait complete. 2024-09-21T04:09:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.527+0000 7f2eb0c65740 1 -- start start 2024-09-21T04:09:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.528+0000 7f2eb0c65740 1 Processor -- start 2024-09-21T04:09:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.528+0000 7f2eb0c65740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560566bf02f0 0x560566fca220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.528+0000 7f2eb0c65740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560566bf84d0 0x560566fca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.528+0000 7f2eb0c65740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560566c191b0 0x560566fd6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.528+0000 7f2eb0c65740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560566e5aab0 con 0x560566c191b0 2024-09-21T04:09:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.528+0000 7f2eb0c65740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560566ec8730 con 0x560566bf02f0 2024-09-21T04:09:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.528+0000 7f2eb0c65740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560566fcd370 con 0x560566bf84d0 2024-09-21T04:09:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.528+0000 7f2eb0c65740 10 client.0 _ll_register_callbacks cb 0x560566f1c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:09:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.529+0000 7f2ea3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560566c191b0 0x560566fd6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.529+0000 7f2ea3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560566c191b0 0x560566fd6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52672/0 (socket says 172.21.15.64:52672) 2024-09-21T04:09:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.529+0000 7f2ea3fff640 1 -- 172.21.15.64:0/3774325918 learned_addr learned my addr 172.21.15.64:0/3774325918 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.530+0000 7f2ea3fff640 1 -- 172.21.15.64:0/3774325918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560566bf84d0 msgr2=0x560566fca760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:09:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.530+0000 7f2ea3fff640 1 --2- 172.21.15.64:0/3774325918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560566bf84d0 0x560566fca760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.530+0000 7f2ea3fff640 1 -- 172.21.15.64:0/3774325918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560566bf02f0 msgr2=0x560566fca220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T04:09:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.530+0000 7f2ea3fff640 1 --2- 172.21.15.64:0/3774325918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560566bf02f0 0x560566fca220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.530+0000 7f2ea3fff640 1 -- 172.21.15.64:0/3774325918 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560566c84790 con 0x560566c191b0 2024-09-21T04:09:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.531+0000 7f2ea3fff640 1 --2- 172.21.15.64:0/3774325918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560566c191b0 0x560566fd6740 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2e94011670 tx=0x7f2e9400b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04ae023665a08d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.540+0000 7f2e877f6640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:09:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.544+0000 7f2e877f6640 1 -- 172.21.15.64:0/3774325918 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9401c480 con 0x560566c191b0 2024-09-21T04:09:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.545+0000 7f2e877f6640 1 -- 172.21.15.64:0/3774325918 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f2e9401c620 con 0x560566c191b0 2024-09-21T04:09:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.545+0000 7f2eb0c65740 1 -- 172.21.15.64:0/3774325918 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e9401cad0 con 0x560566c191b0 2024-09-21T04:09:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.546+0000 7f2e877f6640 1 -- 172.21.15.64:0/3774325918 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2e9401c820 con 0x560566c191b0 2024-09-21T04:09:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.546+0000 7f2eb0c65740 10 client.16734 fetch_fsmap learned FSMap version 35 2024-09-21T04:09:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.547+0000 7f2eb0c65740 1 -- 172.21.15.64:0/3774325918 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560566cfd620 con 0x560566c191b0 2024-09-21T04:09:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.547+0000 7f2e877f6640 1 -- 172.21.15.64:0/3774325918 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e9401cad0 con 0x560566c191b0 2024-09-21T04:09:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.548+0000 7f2e877f6640 1 -- 172.21.15.64:0/3774325918 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f2e94003be0 con 0x560566c191b0 2024-09-21T04:09:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.548+0000 7f2eb0c65740 10 client.16734 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T04:09:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.548+0000 7f2eb0c65740 10 client.16734 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:09:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.548+0000 7f2eb0c65740 20 client.16734 populate_metadata read hostname 'smithi064' 2024-09-21T04:09:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.548+0000 7f2eb0c65740 1 --2- 172.21.15.64:0/3774325918 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x560566bd7d80 0x560566fdc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.548+0000 7f2eb0c65740 4 client.16734 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:09:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.548+0000 7f2eb0c65740 1 -- 172.21.15.64:0/3774325918 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560566f35640 con 0x560566bd7d80 2024-09-21T04:09:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.548+0000 7f2ea2ffd640 1 --2- 172.21.15.64:0/3774325918 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x560566bd7d80 0x560566fdc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.549+0000 7f2e877f6640 10 client.16734 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.549+0000 7f2ea2ffd640 1 --2- 172.21.15.64:0/3774325918 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x560566bd7d80 0x560566fdc7f0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eeafea5203c578ac server_cookie=f89e15f6c2f38f8e in_seq=0 out_seq=0 2024-09-21T04:09:42.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:42 smithi131 ceph-mon[35193]: pgmap v491: 97 pgs: 97 active+clean; 704 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 351 op/s 2024-09-21T04:09:42.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:42 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:09:42.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:42 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.551+0000 7f2e877f6640 1 -- 172.21.15.64:0/3774325918 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560566f35640 con 0x560566bd7d80 2024-09-21T04:09:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.552+0000 7f2e877f6640 10 client.16734 handle_command_reply: tid=0 2024-09-21T04:09:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.552+0000 7f2eb0c65740 1 client.16734 shutdown 2024-09-21T04:09:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.558+0000 7f2eb0c65740 1 -- 172.21.15.64:0/3774325918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560566c191b0 msgr2=0x560566fd6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.558+0000 7f2eb0c65740 1 --2- 172.21.15.64:0/3774325918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560566c191b0 0x560566fd6740 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2e94011670 tx=0x7f2e9400b990 comp rx=0 tx=0).stop 2024-09-21T04:09:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.558+0000 7f2eb0c65740 1 -- 172.21.15.64:0/3774325918 shutdown_connections 2024-09-21T04:09:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.558+0000 7f2eb0c65740 1 --2- 172.21.15.64:0/3774325918 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x560566bd7d80 0x560566fdc7f0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.558+0000 7f2eb0c65740 1 --2- 172.21.15.64:0/3774325918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560566c191b0 0x560566fd6740 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.558+0000 7f2eb0c65740 1 --2- 172.21.15.64:0/3774325918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560566bf84d0 0x560566fca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.558+0000 7f2eb0c65740 1 --2- 172.21.15.64:0/3774325918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560566bf02f0 0x560566fca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.559+0000 7f2eb0c65740 1 -- 172.21.15.64:0/3774325918 >> 172.21.15.64:0/3774325918 conn(0x560566c0d300 msgr2=0x560566fbfbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.559+0000 7f2e877f6640 0 client.16734 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.562+0000 7f2eb0c65740 1 -- 172.21.15.64:0/3774325918 shutdown_connections 2024-09-21T04:09:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.562+0000 7f2eb0c65740 1 -- 172.21.15.64:0/3774325918 wait complete. 2024-09-21T04:09:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.563+0000 7f2eb0c65740 20 client.16734 trim_cache size 0 max 16384 2024-09-21T04:09:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.722+0000 7f2eb0c65740 1 Processor -- start 2024-09-21T04:09:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.722+0000 7f2eb0c65740 1 -- start start 2024-09-21T04:09:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.723+0000 7f2eb0c65740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560566bf02f0 0x560566fc02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.723+0000 7f2eb0c65740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560566bf84d0 0x560566fd6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.723+0000 7f2ea3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560566bf02f0 0x560566fc02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.723+0000 7f2ea3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560566bf02f0 0x560566fc02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:45536/0 (socket says 172.21.15.64:45536) 2024-09-21T04:09:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.723+0000 7f2ea2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560566bf84d0 0x560566fd6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.723+0000 7f2ea2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560566bf84d0 0x560566fd6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52680/0 (socket says 172.21.15.64:52680) 2024-09-21T04:09:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.723+0000 7f2ea3fff640 1 -- 172.21.15.64:0/1133837382 learned_addr learned my addr 172.21.15.64:0/1133837382 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.723+0000 7f2eb0c65740 1 --2- 172.21.15.64:0/1133837382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560566c191b0 0x560566fc3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.723+0000 7f2eb0c65740 1 -- 172.21.15.64:0/1133837382 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2e900113b0 con 0x560566bf84d0 2024-09-21T04:09:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.723+0000 7f2eb0c65740 1 -- 172.21.15.64:0/1133837382 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2e9000ec20 con 0x560566c191b0 2024-09-21T04:09:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.723+0000 7f2eb0c65740 1 -- 172.21.15.64:0/1133837382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2e9000ca70 con 0x560566bf02f0 2024-09-21T04:09:42.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.724+0000 7f2ea3fff640 1 --2- 172.21.15.64:0/1133837382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560566c191b0 0x560566fc3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:42.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.724+0000 7f2ea3fff640 1 -- 172.21.15.64:0/1133837382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560566c191b0 msgr2=0x560566fc3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:42.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.724+0000 7f2ea3fff640 1 --2- 172.21.15.64:0/1133837382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560566c191b0 0x560566fc3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:42.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.724+0000 7f2ea3fff640 1 -- 172.21.15.64:0/1133837382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560566bf84d0 msgr2=0x560566fd6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:42.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.724+0000 7f2ea3fff640 1 --2- 172.21.15.64:0/1133837382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560566bf84d0 0x560566fd6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.724+0000 7f2ea3fff640 1 -- 172.21.15.64:0/1133837382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560566fc9440 con 0x560566bf02f0 2024-09-21T04:09:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.724+0000 7f2ea2ffd640 1 --2- 172.21.15.64:0/1133837382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560566bf84d0 0x560566fd6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:09:42.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.725+0000 7f2ea3fff640 1 --2- 172.21.15.64:0/1133837382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560566bf02f0 0x560566fc02e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2e94011b00 tx=0x7f2e940107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27620ed28feeac0b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:42.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.733+0000 7f2e897fa640 1 -- 172.21.15.64:0/1133837382 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9401c480 con 0x560566bf02f0 2024-09-21T04:09:42.736 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.734+0000 7f2e897fa640 1 -- 172.21.15.64:0/1133837382 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e94005050 con 0x560566bf02f0 2024-09-21T04:09:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.734+0000 7f2e897fa640 1 -- 172.21.15.64:0/1133837382 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9402ece0 con 0x560566bf02f0 2024-09-21T04:09:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.737+0000 7f2eb0c65740 1 -- 172.21.15.64:0/1133837382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560566bf02f0 msgr2=0x560566fc02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.737+0000 7f2eb0c65740 1 --2- 172.21.15.64:0/1133837382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560566bf02f0 0x560566fc02e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2e94011b00 tx=0x7f2e940107a0 comp rx=0 tx=0).stop 2024-09-21T04:09:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.741+0000 7f2eb0c65740 1 -- 172.21.15.64:0/1133837382 shutdown_connections 2024-09-21T04:09:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.741+0000 7f2eb0c65740 1 --2- 172.21.15.64:0/1133837382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560566c191b0 0x560566fc3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.741+0000 7f2eb0c65740 1 --2- 172.21.15.64:0/1133837382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560566bf84d0 0x560566fd6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.741+0000 7f2eb0c65740 1 --2- 172.21.15.64:0/1133837382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560566bf02f0 0x560566fc02e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.741+0000 7f2eb0c65740 1 -- 172.21.15.64:0/1133837382 >> 172.21.15.64:0/1133837382 conn(0x560566c0d300 msgr2=0x560566fbfbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.744+0000 7f2eb0c65740 1 -- 172.21.15.64:0/1133837382 shutdown_connections 2024-09-21T04:09:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.744+0000 7f2eb0c65740 1 -- 172.21.15.64:0/1133837382 wait complete. 2024-09-21T04:09:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.745+0000 7f2eb0c65740 1 -- start start 2024-09-21T04:09:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.746+0000 7f2eb0c65740 1 Processor -- start 2024-09-21T04:09:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.748+0000 7f2eb0c65740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560566bf02f0 0x560566fd2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.748+0000 7f2eb0c65740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560566bf84d0 0x560566fcf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.748+0000 7f2ea3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560566bf02f0 0x560566fd2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.748+0000 7f2ea3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560566bf02f0 0x560566fd2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:45548/0 (socket says 172.21.15.64:45548) 2024-09-21T04:09:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.749+0000 7f2ea2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560566bf84d0 0x560566fcf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.749+0000 7f2ea2ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560566bf84d0 0x560566fcf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52688/0 (socket says 172.21.15.64:52688) 2024-09-21T04:09:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.749+0000 7f2ea3fff640 1 -- 172.21.15.64:0/2188178939 learned_addr learned my addr 172.21.15.64:0/2188178939 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.749+0000 7f2eb0c65740 1 --2- 172.21.15.64:0/2188178939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560566c191b0 0x560566fcf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.749+0000 7f2eb0c65740 1 -- 172.21.15.64:0/2188178939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2e9000d0e0 con 0x560566bf84d0 2024-09-21T04:09:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.749+0000 7f2eb0c65740 1 -- 172.21.15.64:0/2188178939 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2e9000e550 con 0x560566c191b0 2024-09-21T04:09:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.749+0000 7f2eb0c65740 1 -- 172.21.15.64:0/2188178939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2e9000d7b0 con 0x560566bf02f0 2024-09-21T04:09:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.749+0000 7f2ea3fff640 1 --2- 172.21.15.64:0/2188178939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560566c191b0 0x560566fcf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.749+0000 7f2eb0c65740 10 client.0 _ll_register_callbacks cb 0x7f2e98005e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:09:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.749+0000 7f2ea3fff640 1 -- 172.21.15.64:0/2188178939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560566c191b0 msgr2=0x560566fcf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:42.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.749+0000 7f2ea3fff640 1 --2- 172.21.15.64:0/2188178939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560566c191b0 0x560566fcf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:42.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.749+0000 7f2ea3fff640 1 -- 172.21.15.64:0/2188178939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560566bf84d0 msgr2=0x560566fcf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:42.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.749+0000 7f2ea3fff640 1 --2- 172.21.15.64:0/2188178939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560566bf84d0 0x560566fcf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:42.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.749+0000 7f2ea3fff640 1 -- 172.21.15.64:0/2188178939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560566fcfcb0 con 0x560566bf02f0 2024-09-21T04:09:42.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.749+0000 7f2ea2ffd640 1 --2- 172.21.15.64:0/2188178939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560566bf84d0 0x560566fcf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:09:42.753 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.750+0000 7f2ea3fff640 1 --2- 172.21.15.64:0/2188178939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560566bf02f0 0x560566fd2cd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2e94011d30 tx=0x7f2e94005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8d03f5d1ca2465b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.752+0000 7f2eb0c65740 1 -- 172.21.15.64:0/2188178939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e94005050 con 0x560566bf02f0 2024-09-21T04:09:42.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.772+0000 7f2e877f6640 10 client.26284 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-21T04:09:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.772+0000 7f2e877f6640 1 -- 172.21.15.64:0/2188178939 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9400b850 con 0x560566bf02f0 2024-09-21T04:09:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.772+0000 7f2e877f6640 1 -- 172.21.15.64:0/2188178939 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f2e94011650 con 0x560566bf02f0 2024-09-21T04:09:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.772+0000 7f2e877f6640 1 -- 172.21.15.64:0/2188178939 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2e9401c9c0 con 0x560566bf02f0 2024-09-21T04:09:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.772+0000 7f2eb0c65740 10 client.26284 fetch_fsmap learned FSMap version 35 2024-09-21T04:09:42.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.772+0000 7f2eb0c65740 1 -- 172.21.15.64:0/2188178939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=35}) -- 0x560567045eb0 con 0x560566bf02f0 2024-09-21T04:09:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.774+0000 7f2e877f6640 1 -- 172.21.15.64:0/2188178939 <== mon.1 v2:172.21.15.131:3300/0 4 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f2e94037030 con 0x560566bf02f0 2024-09-21T04:09:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.774+0000 7f2eb0c65740 10 client.26284 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T04:09:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.775+0000 7f2eb0c65740 10 client.26284 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:09:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.775+0000 7f2eb0c65740 20 client.26284 populate_metadata read hostname 'smithi064' 2024-09-21T04:09:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.775+0000 7f2eb0c65740 1 --2- 172.21.15.64:0/2188178939 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x560566bad1c0 0x560567048460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.775+0000 7f2eb0c65740 4 client.26284 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:09:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.775+0000 7f2eb0c65740 1 -- 172.21.15.64:0/2188178939 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560566f310f0 con 0x560566bad1c0 2024-09-21T04:09:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.775+0000 7f2ea2ffd640 1 --2- 172.21.15.64:0/2188178939 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x560566bad1c0 0x560567048460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.776+0000 7f2ea2ffd640 1 --2- 172.21.15.64:0/2188178939 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x560566bad1c0 0x560567048460 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b4ddad49b7d52c4 server_cookie=d8211b3dea86081b in_seq=0 out_seq=0 2024-09-21T04:09:42.778 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:42.776+0000 7f2e877f6640 10 client.26284 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:43.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:43 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:43.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:43 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:43.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:43 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:43.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:43.887+0000 7f2e877f6640 1 -- 172.21.15.64:0/2188178939 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560566f310f0 con 0x560566bad1c0 2024-09-21T04:09:43.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:43.887+0000 7f2e877f6640 10 client.26284 handle_command_reply: tid=0 2024-09-21T04:09:43.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:43.887+0000 7f2eb0c65740 1 client.26284 shutdown 2024-09-21T04:09:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:43.890+0000 7f2eb0c65740 1 -- 172.21.15.64:0/2188178939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560566bf02f0 msgr2=0x560566fd2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:43.890+0000 7f2eb0c65740 1 --2- 172.21.15.64:0/2188178939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560566bf02f0 0x560566fd2cd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2e94011d30 tx=0x7f2e94005970 comp rx=0 tx=0).stop 2024-09-21T04:09:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:43.890+0000 7f2eb0c65740 1 -- 172.21.15.64:0/2188178939 shutdown_connections 2024-09-21T04:09:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:43.890+0000 7f2eb0c65740 1 --2- 172.21.15.64:0/2188178939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560566c191b0 0x560566fcf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:43.890+0000 7f2eb0c65740 1 --2- 172.21.15.64:0/2188178939 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x560566bad1c0 0x560567048460 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:43.890+0000 7f2eb0c65740 1 --2- 172.21.15.64:0/2188178939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560566bf84d0 0x560566fcf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:43.890+0000 7f2eb0c65740 1 --2- 172.21.15.64:0/2188178939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560566bf02f0 0x560566fd2cd0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:43.890+0000 7f2eb0c65740 1 -- 172.21.15.64:0/2188178939 >> 172.21.15.64:0/2188178939 conn(0x560566c0d300 msgr2=0x560566fc3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:43.890+0000 7f2e877f6640 0 client.26284 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:43.890+0000 7f2eb0c65740 1 -- 172.21.15.64:0/2188178939 shutdown_connections 2024-09-21T04:09:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:43.892+0000 7f2eb0c65740 1 -- 172.21.15.64:0/2188178939 wait complete. 2024-09-21T04:09:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:43.893+0000 7f2eb0c65740 20 client.26284 trim_cache size 0 max 16384 2024-09-21T04:09:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:09:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:09:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:09:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:09:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:43.920+0000 7f2ea8a97640 1 -- 172.21.15.64:0/1509120499 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2e8c0b6880 msgr2=0x7f2e8c0b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:43.920+0000 7f2ea8a97640 1 --2- 172.21.15.64:0/1509120499 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2e8c0b6880 0x7f2e8c0b8d90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x560566fbd820 tx=0x7f2e9c009500 comp rx=0 tx=0).stop 2024-09-21T04:09:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:43.920+0000 7f2ea8a97640 1 -- 172.21.15.64:0/1509120499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ea410a900 msgr2=0x7f2ea411c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:43.920+0000 7f2ea8a97640 1 --2- 172.21.15.64:0/1509120499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ea410a900 0x7f2ea411c570 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c02ac40 tx=0x7f2e9c032e80 comp rx=0 tx=0).stop 2024-09-21T04:09:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:43.920+0000 7f2ea8a97640 1 -- 172.21.15.64:0/1509120499 shutdown_connections 2024-09-21T04:09:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:43.920+0000 7f2ea8a97640 1 --2- 172.21.15.64:0/1509120499 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2e8c0b6880 0x7f2e8c0b8d90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:43.920+0000 7f2ea8a97640 1 --2- 172.21.15.64:0/1509120499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ea4116d70 0x7f2ea4117230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:43.923 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:43.920+0000 7f2ea8a97640 1 --2- 172.21.15.64:0/1509120499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ea411cba0 0x7f2ea4116760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:43.920+0000 7f2ea8a97640 1 --2- 172.21.15.64:0/1509120499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ea410a900 0x7f2ea411c570 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:43.920+0000 7f2ea8a97640 1 -- 172.21.15.64:0/1509120499 >> 172.21.15.64:0/1509120499 conn(0x7f2ea4070c00 msgr2=0x7f2ea4110920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:43.921+0000 7f2ea8a97640 1 -- 172.21.15.64:0/1509120499 shutdown_connections 2024-09-21T04:09:43.924 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:43.921+0000 7f2ea8a97640 1 -- 172.21.15.64:0/1509120499 wait complete. 2024-09-21T04:09:44.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:44 smithi119 ceph-mon[34811]: pgmap v492: 97 pgs: 97 active+clean; 685 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.5 MiB/s wr, 471 op/s 2024-09-21T04:09:44.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:44 smithi119 ceph-mon[34811]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:09:44.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:44 smithi064 ceph-mon[31180]: pgmap v492: 97 pgs: 97 active+clean; 685 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.5 MiB/s wr, 471 op/s 2024-09-21T04:09:44.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:44 smithi064 ceph-mon[31180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:09:44.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:44 smithi131 ceph-mon[35193]: pgmap v492: 97 pgs: 97 active+clean; 685 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.5 MiB/s wr, 471 op/s 2024-09-21T04:09:44.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:44 smithi131 ceph-mon[35193]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:09:44.687 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:09:45.132 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:45 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:09:45.133 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:45 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:45.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:45 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:09:45.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:45 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:45.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.531+0000 7f54fb8d2640 1 -- 172.21.15.64:0/2225884955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54f4111f40 msgr2=0x7f54f410a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:45.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.531+0000 7f54fb8d2640 1 --2- 172.21.15.64:0/2225884955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54f4111f40 0x7f54f410a3c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f54ec009ca0 tx=0x7f54ec02a8f0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.531+0000 7f54ebfff640 1 -- 172.21.15.64:0/2225884955 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec02f340 con 0x7f54f4111f40 2024-09-21T04:09:45.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.534+0000 7f54fb8d2640 1 -- 172.21.15.64:0/2225884955 shutdown_connections 2024-09-21T04:09:45.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.534+0000 7f54fb8d2640 1 --2- 172.21.15.64:0/2225884955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54f411cc30 0x7f54f411f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.534+0000 7f54fb8d2640 1 --2- 172.21.15.64:0/2225884955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54f410a9e0 0x7f54f410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.534+0000 7f54fb8d2640 1 --2- 172.21.15.64:0/2225884955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54f4111f40 0x7f54f410a3c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.534+0000 7f54fb8d2640 1 -- 172.21.15.64:0/2225884955 >> 172.21.15.64:0/2225884955 conn(0x7f54f4070ec0 msgr2=0x7f54f40712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:45.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.536+0000 7f54fb8d2640 1 -- 172.21.15.64:0/2225884955 shutdown_connections 2024-09-21T04:09:45.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.537+0000 7f54fb8d2640 1 -- 172.21.15.64:0/2225884955 wait complete. 2024-09-21T04:09:45.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.538+0000 7f54fb8d2640 1 Processor -- start 2024-09-21T04:09:45.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.538+0000 7f54fb8d2640 1 -- start start 2024-09-21T04:09:45.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.538+0000 7f54fb8d2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54f410a9e0 0x7f54f411c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:45.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.538+0000 7f54fb8d2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54f4111f40 0x7f54f411c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:45.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.539+0000 7f54fb8d2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54f411cc30 0x7f54f41165d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:45.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.539+0000 7f54fb8d2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f54f4112850 con 0x7f54f410a9e0 2024-09-21T04:09:45.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.539+0000 7f54fb8d2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f54f41126d0 con 0x7f54f4111f40 2024-09-21T04:09:45.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.539+0000 7f54fb8d2640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f54f41129d0 con 0x7f54f411cc30 2024-09-21T04:09:45.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.539+0000 7f54f8e6d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54f4111f40 0x7f54f411c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:45.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.539+0000 7f54f9e6f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54f411cc30 0x7f54f41165d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:45.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.539+0000 7f54f8e6d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54f4111f40 0x7f54f411c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:49372/0 (socket says 172.21.15.64:49372) 2024-09-21T04:09:45.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.539+0000 7f54f8e6d640 1 -- 172.21.15.64:0/173176072 learned_addr learned my addr 172.21.15.64:0/173176072 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:45.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.539+0000 7f54f966e640 1 --2- 172.21.15.64:0/173176072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54f410a9e0 0x7f54f411c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:45.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.539+0000 7f54f9e6f640 1 -- 172.21.15.64:0/173176072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54f4111f40 msgr2=0x7f54f411c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:45.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.539+0000 7f54f9e6f640 1 --2- 172.21.15.64:0/173176072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54f4111f40 0x7f54f411c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.539+0000 7f54f9e6f640 1 -- 172.21.15.64:0/173176072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54f410a9e0 msgr2=0x7f54f411c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:45.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.539+0000 7f54f9e6f640 1 --2- 172.21.15.64:0/173176072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54f410a9e0 0x7f54f411c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.540+0000 7f54f9e6f640 1 -- 172.21.15.64:0/173176072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f4116df0 con 0x7f54f411cc30 2024-09-21T04:09:45.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.542+0000 7f54f9e6f640 1 --2- 172.21.15.64:0/173176072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54f411cc30 0x7f54f41165d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f54f000b3e0 tx=0x7f54f000b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:45.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.543+0000 7f54ea7fc640 1 -- 172.21.15.64:0/173176072 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f0015020 con 0x7f54f411cc30 2024-09-21T04:09:45.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.543+0000 7f54ea7fc640 1 -- 172.21.15.64:0/173176072 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54f0011ce0 con 0x7f54f411cc30 2024-09-21T04:09:45.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.543+0000 7f54fb8d2640 1 -- 172.21.15.64:0/173176072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f41170e0 con 0x7f54f411cc30 2024-09-21T04:09:45.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.543+0000 7f54fb8d2640 1 -- 172.21.15.64:0/173176072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f4128860 con 0x7f54f411cc30 2024-09-21T04:09:45.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:45 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:09:45.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:45 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:45.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.553+0000 7f5501e46740 1 Processor -- start 2024-09-21T04:09:45.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.555+0000 7f5501e46740 1 -- start start 2024-09-21T04:09:45.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.556+0000 7f5501e46740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556e137b070 0x5556e1722650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:45.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.556+0000 7f5501e46740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556e13521b0 0x5556e1722b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:45.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.558+0000 7f5501e46740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556e135a390 0x5556e17230d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:45.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.558+0000 7f5501e46740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5556e1698e50 con 0x5556e137b070 2024-09-21T04:09:45.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.558+0000 7f5501e46740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5556e1688280 con 0x5556e135a390 2024-09-21T04:09:45.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.558+0000 7f5501e46740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5556e168c090 con 0x5556e13521b0 2024-09-21T04:09:45.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.558+0000 7f54f966e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556e13521b0 0x5556e1722b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:45.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.559+0000 7f54f966e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556e13521b0 0x5556e1722b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:45568/0 (socket says 172.21.15.64:45568) 2024-09-21T04:09:45.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.559+0000 7f54f966e640 1 -- 172.21.15.64:0/333495720 learned_addr learned my addr 172.21.15.64:0/333495720 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:45.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.562+0000 7f54f966e640 1 -- 172.21.15.64:0/333495720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556e135a390 msgr2=0x5556e17230d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:09:45.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.562+0000 7f54f966e640 1 --2- 172.21.15.64:0/333495720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556e135a390 0x5556e17230d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.563+0000 7f54f966e640 1 -- 172.21.15.64:0/333495720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556e137b070 msgr2=0x5556e1722650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:09:45.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.563+0000 7f54f966e640 1 --2- 172.21.15.64:0/333495720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556e137b070 0x5556e1722650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.563+0000 7f54f966e640 1 -- 172.21.15.64:0/333495720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556e13e6650 con 0x5556e13521b0 2024-09-21T04:09:45.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.564+0000 7f54ea7fc640 1 -- 172.21.15.64:0/173176072 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f0010e50 con 0x7f54f411cc30 2024-09-21T04:09:45.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.564+0000 7f54f966e640 1 --2- 172.21.15.64:0/333495720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556e13521b0 0x5556e1722b90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f54ec009bf0 tx=0x7f54ec009940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c153c8aac1ea9794 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:45.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.565+0000 7f54ea7fc640 1 -- 172.21.15.64:0/173176072 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f54f0018020 con 0x7f54f411cc30 2024-09-21T04:09:45.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.566+0000 7f54df7fe640 1 -- 172.21.15.64:0/333495720 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec031df0 con 0x5556e13521b0 2024-09-21T04:09:45.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.570+0000 7f54df7fe640 1 -- 172.21.15.64:0/333495720 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54ec036040 con 0x5556e13521b0 2024-09-21T04:09:45.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.570+0000 7f54df7fe640 1 -- 172.21.15.64:0/333495720 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec004370 con 0x5556e13521b0 2024-09-21T04:09:45.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.571+0000 7f5501e46740 1 -- 172.21.15.64:0/333495720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556e13521b0 msgr2=0x5556e1722b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:45.576 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.571+0000 7f5501e46740 1 --2- 172.21.15.64:0/333495720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556e13521b0 0x5556e1722b90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f54ec009bf0 tx=0x7f54ec009940 comp rx=0 tx=0).stop 2024-09-21T04:09:45.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.584+0000 7f54ea7fc640 1 --2- 172.21.15.64:0/173176072 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f54e00b6a20 0x7f54e00b8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.584+0000 7f54ea7fc640 1 -- 172.21.15.64:0/173176072 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f54f00e7ce0 con 0x7f54f411cc30 2024-09-21T04:09:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.585+0000 7f5501e46740 1 -- 172.21.15.64:0/333495720 shutdown_connections 2024-09-21T04:09:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.585+0000 7f5501e46740 1 --2- 172.21.15.64:0/333495720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556e135a390 0x5556e17230d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.585+0000 7f5501e46740 1 --2- 172.21.15.64:0/333495720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556e13521b0 0x5556e1722b90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.585+0000 7f5501e46740 1 --2- 172.21.15.64:0/333495720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556e137b070 0x5556e1722650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.585+0000 7f5501e46740 1 -- 172.21.15.64:0/333495720 >> 172.21.15.64:0/333495720 conn(0x5556e136f460 msgr2=0x5556e1720540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:45.590 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.586+0000 7f54f8e6d640 1 --2- 172.21.15.64:0/173176072 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f54e00b6a20 0x7f54e00b8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:45.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.589+0000 7f5501e46740 1 -- 172.21.15.64:0/333495720 shutdown_connections 2024-09-21T04:09:45.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.590+0000 7f54f8e6d640 1 --2- 172.21.15.64:0/173176072 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f54e00b6a20 0x7f54e00b8f30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f54f411e960 tx=0x7f54e4009190 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:45.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.590+0000 7f5501e46740 1 -- 172.21.15.64:0/333495720 wait complete. 2024-09-21T04:09:45.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.591+0000 7f5501e46740 1 -- start start 2024-09-21T04:09:45.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.591+0000 7f5501e46740 1 Processor -- start 2024-09-21T04:09:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.595+0000 7f5501e46740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556e13521b0 0x5556e172dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.595+0000 7f5501e46740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556e135a390 0x5556e172e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.595+0000 7f54f966e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556e13521b0 0x5556e172dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.595+0000 7f54f966e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556e13521b0 0x5556e172dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:45580/0 (socket says 172.21.15.64:45580) 2024-09-21T04:09:45.598 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.596+0000 7f54f966e640 1 -- 172.21.15.64:0/3236705017 learned_addr learned my addr 172.21.15.64:0/3236705017 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:45.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.596+0000 7f5501e46740 1 --2- 172.21.15.64:0/3236705017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556e137b070 0x5556e173a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:45.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.596+0000 7f54f8e6d640 1 --2- 172.21.15.64:0/3236705017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556e135a390 0x5556e172e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:45.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.596+0000 7f54f9e6f640 1 --2- 172.21.15.64:0/3236705017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556e137b070 0x5556e173a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:45.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.597+0000 7f5501e46740 1 -- 172.21.15.64:0/3236705017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5556e14e9700 con 0x5556e137b070 2024-09-21T04:09:45.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.597+0000 7f5501e46740 1 -- 172.21.15.64:0/3236705017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5556e1623110 con 0x5556e135a390 2024-09-21T04:09:45.599 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.597+0000 7f5501e46740 1 -- 172.21.15.64:0/3236705017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5556e1730dd0 con 0x5556e13521b0 2024-09-21T04:09:45.600 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.597+0000 7f5501e46740 10 client.0 _ll_register_callbacks cb 0x5556e163c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:09:45.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.598+0000 7f54f8e6d640 1 -- 172.21.15.64:0/3236705017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556e13521b0 msgr2=0x5556e172dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:45.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.598+0000 7f54f8e6d640 1 --2- 172.21.15.64:0/3236705017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556e13521b0 0x5556e172dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.598+0000 7f54f8e6d640 1 -- 172.21.15.64:0/3236705017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556e137b070 msgr2=0x5556e173a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:45.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.598+0000 7f54f8e6d640 1 --2- 172.21.15.64:0/3236705017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556e137b070 0x5556e173a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.598+0000 7f54f8e6d640 1 -- 172.21.15.64:0/3236705017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556e1650c40 con 0x5556e135a390 2024-09-21T04:09:45.601 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.599+0000 7f54f9e6f640 1 --2- 172.21.15.64:0/3236705017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556e137b070 0x5556e173a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:09:45.602 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.599+0000 7f54f966e640 1 --2- 172.21.15.64:0/3236705017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556e13521b0 0x5556e172dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:09:45.605 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.603+0000 7f54f8e6d640 1 --2- 172.21.15.64:0/3236705017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556e135a390 0x5556e172e2c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f54e40154d0 tx=0x7f54e4004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc340de60a2c8b51 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:45.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.605+0000 7f54dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:09:45.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.605+0000 7f54dcff9640 1 -- 172.21.15.64:0/3236705017 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4023020 con 0x5556e135a390 2024-09-21T04:09:45.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.605+0000 7f54dcff9640 1 -- 172.21.15.64:0/3236705017 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f54e4013510 con 0x5556e135a390 2024-09-21T04:09:45.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.608+0000 7f5501e46740 1 -- 172.21.15.64:0/3236705017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54ec036040 con 0x5556e135a390 2024-09-21T04:09:45.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.609+0000 7f54dcff9640 1 -- 172.21.15.64:0/3236705017 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f54e401b040 con 0x5556e135a390 2024-09-21T04:09:45.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.612+0000 7f5501e46740 10 client.26435 fetch_fsmap learned FSMap version 36 2024-09-21T04:09:45.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.612+0000 7f5501e46740 1 -- 172.21.15.64:0/3236705017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5556e173cc10 con 0x5556e135a390 2024-09-21T04:09:45.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.617+0000 7f54dcff9640 1 -- 172.21.15.64:0/3236705017 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54ec036040 con 0x5556e135a390 2024-09-21T04:09:45.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.617+0000 7f54dcff9640 1 -- 172.21.15.64:0/3236705017 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7f54e4030020 con 0x5556e135a390 2024-09-21T04:09:45.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.617+0000 7f5501e46740 10 client.26435 fetch_fsmap finished waiting for FSMap version 36 2024-09-21T04:09:45.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.617+0000 7f5501e46740 10 client.26435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:09:45.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.617+0000 7f5501e46740 20 client.26435 populate_metadata read hostname 'smithi064' 2024-09-21T04:09:45.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.617+0000 7f5501e46740 1 --2- 172.21.15.64:0/3236705017 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5556e13399f0 0x5556e1740200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:45.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.617+0000 7f5501e46740 4 client.26435 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:09:45.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.617+0000 7f5501e46740 1 -- 172.21.15.64:0/3236705017 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556e169ada0 con 0x5556e13399f0 2024-09-21T04:09:45.625 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.621+0000 7f54f966e640 1 --2- 172.21.15.64:0/3236705017 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5556e13399f0 0x5556e1740200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:45.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.631+0000 7f54dcff9640 10 client.26435 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:45.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.631+0000 7f54f966e640 1 --2- 172.21.15.64:0/3236705017 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5556e13399f0 0x5556e1740200 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4263c81f404100a8 server_cookie=7295176aea1643b5 in_seq=0 out_seq=0 2024-09-21T04:09:45.634 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.632+0000 7f54dcff9640 1 -- 172.21.15.64:0/3236705017 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5556e169ada0 con 0x5556e13399f0 2024-09-21T04:09:45.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.632+0000 7f54dcff9640 10 client.26435 handle_command_reply: tid=0 2024-09-21T04:09:45.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.632+0000 7f5501e46740 1 client.26435 shutdown 2024-09-21T04:09:45.635 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.633+0000 7f5501e46740 1 -- 172.21.15.64:0/3236705017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556e135a390 msgr2=0x5556e172e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.633+0000 7f5501e46740 1 --2- 172.21.15.64:0/3236705017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556e135a390 0x5556e172e2c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f54e40154d0 tx=0x7f54e4004770 comp rx=0 tx=0).stop 2024-09-21T04:09:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.633+0000 7f5501e46740 1 -- 172.21.15.64:0/3236705017 shutdown_connections 2024-09-21T04:09:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.633+0000 7f5501e46740 1 --2- 172.21.15.64:0/3236705017 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5556e13399f0 0x5556e1740200 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.633+0000 7f5501e46740 1 --2- 172.21.15.64:0/3236705017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556e137b070 0x5556e173a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.633+0000 7f5501e46740 1 --2- 172.21.15.64:0/3236705017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556e135a390 0x5556e172e2c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.633+0000 7f5501e46740 1 --2- 172.21.15.64:0/3236705017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556e13521b0 0x5556e172dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.633+0000 7f5501e46740 1 -- 172.21.15.64:0/3236705017 >> 172.21.15.64:0/3236705017 conn(0x5556e136f460 msgr2=0x5556e17257e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.633+0000 7f54dcff9640 0 client.26435 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.633+0000 7f5501e46740 1 -- 172.21.15.64:0/3236705017 shutdown_connections 2024-09-21T04:09:45.637 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.633+0000 7f5501e46740 1 -- 172.21.15.64:0/3236705017 wait complete. 2024-09-21T04:09:45.638 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.633+0000 7f5501e46740 20 client.26435 trim_cache size 0 max 16384 2024-09-21T04:09:45.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.677+0000 7f5501e46740 1 Processor -- start 2024-09-21T04:09:45.680 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.677+0000 7f5501e46740 1 -- start start 2024-09-21T04:09:45.685 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.678+0000 7f5501e46740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556e13521b0 0x5556e172b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:45.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.678+0000 7f54f966e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556e13521b0 0x5556e172b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:45.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.678+0000 7f54f966e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556e13521b0 0x5556e172b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52720/0 (socket says 172.21.15.64:52720) 2024-09-21T04:09:45.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.678+0000 7f54f966e640 1 -- 172.21.15.64:0/1523327956 learned_addr learned my addr 172.21.15.64:0/1523327956 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:45.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.678+0000 7f5501e46740 1 --2- 172.21.15.64:0/1523327956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556e135a390 0x5556e1725750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:45.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.678+0000 7f5501e46740 1 --2- 172.21.15.64:0/1523327956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556e137b070 0x5556e1725c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:45.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.678+0000 7f5501e46740 1 -- 172.21.15.64:0/1523327956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f54d8011330 con 0x5556e13521b0 2024-09-21T04:09:45.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.678+0000 7f5501e46740 1 -- 172.21.15.64:0/1523327956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f54d800ebd0 con 0x5556e137b070 2024-09-21T04:09:45.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.679+0000 7f5501e46740 1 -- 172.21.15.64:0/1523327956 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f54d800ca20 con 0x5556e135a390 2024-09-21T04:09:45.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.679+0000 7f54f9e6f640 1 --2- 172.21.15.64:0/1523327956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556e137b070 0x5556e1725c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:45.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.679+0000 7f54f8e6d640 1 --2- 172.21.15.64:0/1523327956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556e135a390 0x5556e1725750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:45.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.679+0000 7f54f966e640 1 -- 172.21.15.64:0/1523327956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556e135a390 msgr2=0x5556e1725750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:45.686 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.679+0000 7f54f966e640 1 --2- 172.21.15.64:0/1523327956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556e135a390 0x5556e1725750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.679+0000 7f54f966e640 1 -- 172.21.15.64:0/1523327956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556e137b070 msgr2=0x5556e1725c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:45.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.679+0000 7f54f966e640 1 --2- 172.21.15.64:0/1523327956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556e137b070 0x5556e1725c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.679+0000 7f54f966e640 1 -- 172.21.15.64:0/1523327956 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556e1724940 con 0x5556e13521b0 2024-09-21T04:09:45.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.680+0000 7f54f966e640 1 --2- 172.21.15.64:0/1523327956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556e13521b0 0x5556e172b890 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f54ec009ca0 tx=0x7f54ec009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a08fe4320cf87f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:45.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.681+0000 7f54deffd640 1 -- 172.21.15.64:0/1523327956 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec05c070 con 0x5556e13521b0 2024-09-21T04:09:45.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.681+0000 7f54deffd640 1 -- 172.21.15.64:0/1523327956 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54ec036040 con 0x5556e13521b0 2024-09-21T04:09:45.687 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.681+0000 7f54deffd640 1 -- 172.21.15.64:0/1523327956 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec04a040 con 0x5556e13521b0 2024-09-21T04:09:45.689 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.686+0000 7f5501e46740 1 -- 172.21.15.64:0/1523327956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556e13521b0 msgr2=0x5556e172b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:45.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.686+0000 7f5501e46740 1 --2- 172.21.15.64:0/1523327956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556e13521b0 0x5556e172b890 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f54ec009ca0 tx=0x7f54ec009490 comp rx=0 tx=0).stop 2024-09-21T04:09:45.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.686+0000 7f5501e46740 1 -- 172.21.15.64:0/1523327956 shutdown_connections 2024-09-21T04:09:45.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.686+0000 7f5501e46740 1 --2- 172.21.15.64:0/1523327956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556e137b070 0x5556e1725c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.686+0000 7f5501e46740 1 --2- 172.21.15.64:0/1523327956 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556e135a390 0x5556e1725750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.686+0000 7f5501e46740 1 --2- 172.21.15.64:0/1523327956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556e13521b0 0x5556e172b890 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.690 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.686+0000 7f5501e46740 1 -- 172.21.15.64:0/1523327956 >> 172.21.15.64:0/1523327956 conn(0x5556e136f460 msgr2=0x5556e172b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:45.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.690+0000 7f5501e46740 1 -- 172.21.15.64:0/1523327956 shutdown_connections 2024-09-21T04:09:45.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.690+0000 7f5501e46740 1 -- 172.21.15.64:0/1523327956 wait complete. 2024-09-21T04:09:45.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.690+0000 7f5501e46740 1 -- start start 2024-09-21T04:09:45.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.690+0000 7f5501e46740 1 Processor -- start 2024-09-21T04:09:45.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.691+0000 7f5501e46740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556e13521b0 0x5556e1734040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:45.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.691+0000 7f5501e46740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556e135a390 0x5556e1734580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:45.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.691+0000 7f5501e46740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556e137b070 0x5556e1734ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:45.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.691+0000 7f5501e46740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f54d800d090 con 0x5556e135a390 2024-09-21T04:09:45.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.691+0000 7f5501e46740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f54d800e500 con 0x5556e137b070 2024-09-21T04:09:45.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.691+0000 7f5501e46740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f54d800d760 con 0x5556e13521b0 2024-09-21T04:09:45.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.691+0000 7f5501e46740 10 client.0 _ll_register_callbacks cb 0x7f54e4009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:09:45.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.692+0000 7f54f966e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556e13521b0 0x5556e1734040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:45.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.692+0000 7f54f966e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556e13521b0 0x5556e1734040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:45594/0 (socket says 172.21.15.64:45594) 2024-09-21T04:09:45.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.692+0000 7f54f966e640 1 -- 172.21.15.64:0/1276127226 learned_addr learned my addr 172.21.15.64:0/1276127226 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:45.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.692+0000 7f54f8e6d640 1 --2- 172.21.15.64:0/1276127226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556e135a390 0x5556e1734580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:45.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.693+0000 7f54f8e6d640 1 -- 172.21.15.64:0/1276127226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556e13521b0 msgr2=0x5556e1734040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:45.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.693+0000 7f54f8e6d640 1 --2- 172.21.15.64:0/1276127226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556e13521b0 0x5556e1734040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.693+0000 7f54f8e6d640 1 -- 172.21.15.64:0/1276127226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556e137b070 msgr2=0x5556e1734ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:09:45.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.693+0000 7f54f8e6d640 1 --2- 172.21.15.64:0/1276127226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556e137b070 0x5556e1734ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.693+0000 7f54f8e6d640 1 -- 172.21.15.64:0/1276127226 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556e17a9c10 con 0x5556e135a390 2024-09-21T04:09:45.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.693+0000 7f54f8e6d640 1 --2- 172.21.15.64:0/1276127226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556e135a390 0x5556e1734580 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f54e4000c00 tx=0x7f54e40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89229a825951551 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:45.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.694+0000 7f54dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:09:45.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.694+0000 7f54dcff9640 1 -- 172.21.15.64:0/1276127226 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4023040 con 0x5556e135a390 2024-09-21T04:09:45.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.694+0000 7f5501e46740 1 -- 172.21.15.64:0/1276127226 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54ec036040 con 0x5556e135a390 2024-09-21T04:09:45.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.694+0000 7f54dcff9640 1 -- 172.21.15.64:0/1276127226 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f54e40038b0 con 0x5556e135a390 2024-09-21T04:09:45.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.694+0000 7f54dcff9640 1 -- 172.21.15.64:0/1276127226 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f54e4003af0 con 0x5556e135a390 2024-09-21T04:09:45.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.696+0000 7f5501e46740 10 client.16755 fetch_fsmap learned FSMap version 36 2024-09-21T04:09:45.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.696+0000 7f5501e46740 1 -- 172.21.15.64:0/1276127226 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5556e17aae30 con 0x5556e135a390 2024-09-21T04:09:45.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.699+0000 7f54dcff9640 1 -- 172.21.15.64:0/1276127226 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54ec036040 con 0x5556e135a390 2024-09-21T04:09:45.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.699+0000 7f54dcff9640 1 -- 172.21.15.64:0/1276127226 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 36) ==== 8350+0+0 (secure 0 0 0) 0x7f54e4030040 con 0x5556e135a390 2024-09-21T04:09:45.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.700+0000 7f5501e46740 10 client.16755 fetch_fsmap finished waiting for FSMap version 36 2024-09-21T04:09:45.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.700+0000 7f5501e46740 10 client.16755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:09:45.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.700+0000 7f5501e46740 20 client.16755 populate_metadata read hostname 'smithi064' 2024-09-21T04:09:45.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.700+0000 7f5501e46740 1 --2- 172.21.15.64:0/1276127226 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5556e130eda0 0x5556e17ad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:45.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.700+0000 7f5501e46740 4 client.16755 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T04:09:45.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.700+0000 7f5501e46740 1 -- 172.21.15.64:0/1276127226 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5556e1696850 con 0x5556e130eda0 2024-09-21T04:09:45.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.700+0000 7f54f966e640 1 --2- 172.21.15.64:0/1276127226 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5556e130eda0 0x5556e17ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:45.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.708+0000 7f54f966e640 1 --2- 172.21.15.64:0/1276127226 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5556e130eda0 0x5556e17ad520 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec827c98a37dd251 server_cookie=b1f92b40f32b8789 in_seq=0 out_seq=0 2024-09-21T04:09:45.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.708+0000 7f54dcff9640 10 client.16755 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:45.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.712+0000 7f54dcff9640 1 -- 172.21.15.64:0/1276127226 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2842 (crc 0 0 0) 0x5556e1696850 con 0x5556e130eda0 2024-09-21T04:09:45.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.712+0000 7f54dcff9640 10 client.16755 handle_command_reply: tid=0 2024-09-21T04:09:45.715 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.713+0000 7f5501e46740 1 client.16755 shutdown 2024-09-21T04:09:45.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.721+0000 7f5501e46740 1 -- 172.21.15.64:0/1276127226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556e135a390 msgr2=0x5556e1734580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:45.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.721+0000 7f5501e46740 1 --2- 172.21.15.64:0/1276127226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556e135a390 0x5556e1734580 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f54e4000c00 tx=0x7f54e40045e0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.724+0000 7f5501e46740 1 -- 172.21.15.64:0/1276127226 shutdown_connections 2024-09-21T04:09:45.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.724+0000 7f5501e46740 1 --2- 172.21.15.64:0/1276127226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556e137b070 0x5556e1734ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.724+0000 7f5501e46740 1 --2- 172.21.15.64:0/1276127226 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5556e130eda0 0x5556e17ad520 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.724+0000 7f5501e46740 1 --2- 172.21.15.64:0/1276127226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556e135a390 0x5556e1734580 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.724+0000 7f5501e46740 1 --2- 172.21.15.64:0/1276127226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556e13521b0 0x5556e1734040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.724+0000 7f54dcff9640 0 client.16755 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:45.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.724+0000 7f5501e46740 1 -- 172.21.15.64:0/1276127226 >> 172.21.15.64:0/1276127226 conn(0x5556e136f460 msgr2=0x5556e1725750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:45.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.724+0000 7f5501e46740 1 -- 172.21.15.64:0/1276127226 shutdown_connections 2024-09-21T04:09:45.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.724+0000 7f5501e46740 1 -- 172.21.15.64:0/1276127226 wait complete. 2024-09-21T04:09:45.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.724+0000 7f5501e46740 20 client.16755 trim_cache size 0 max 16384 2024-09-21T04:09:45.734 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:09:45.734 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T04:09:45.734 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-09-21T04:09:45.735 INFO:teuthology.orchestra.run.smithi064.stdout: "2c0523fb-aae4-43fa-9568-136a9b640051": { 2024-09-21T04:09:45.735 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:09:45.735 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "2c0523fb-aae4-43fa-9568-136a9b640051", 2024-09-21T04:09:45.735 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-09-21T04:09:45.735 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:09:45.735 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:09:45.735 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_stats": { 2024-09-21T04:09:45.735 INFO:teuthology.orchestra.run.smithi064.stdout: "2c0523fb-aae4-43fa-9568-136a9b640051": { 2024-09-21T04:09:45.735 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:09:39.808606+0000", 2024-09-21T04:09:45.735 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:09:45.735 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:09:45.735 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:09:45.736 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:09:45.736 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:09:45.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:09:45.736 INFO:teuthology.orchestra.run.smithi064.stdout: "bed40d88-2102-43a6-ac2d-b89d5a2264d0": { 2024-09-21T04:09:45.736 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:11.363340+0000", 2024-09-21T04:09:45.736 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:09:45.736 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:09:45.736 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:09:45.736 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:09:45.736 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:09:45.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:09:45.736 INFO:teuthology.orchestra.run.smithi064.stdout: "e9242f88-6599-40eb-adf1-bdb3e946a1e0": { 2024-09-21T04:09:45.736 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:05:39.744646+0000", 2024-09-21T04:09:45.736 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:09:45.737 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:09:45.737 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:09:45.737 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:09:45.737 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:09:45.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:09:45.737 INFO:teuthology.orchestra.run.smithi064.stdout: "6bde0fe5-f775-44b1-ad0c-f9e596574dc0": { 2024-09-21T04:09:45.737 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:06:23.835536+0000", 2024-09-21T04:09:45.737 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:09:45.737 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:09:45.737 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:09:45.737 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:09:45.737 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:09:45.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:09:45.737 INFO:teuthology.orchestra.run.smithi064.stdout: "b52b950c-0f97-49ec-a37c-1a8e04348ce8": { 2024-09-21T04:09:45.737 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:54.003800+0000", 2024-09-21T04:09:45.738 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:09:45.738 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:09:45.738 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:09:45.738 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:09:45.738 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:09:45.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:09:45.738 INFO:teuthology.orchestra.run.smithi064.stdout: "e2c2eed8-9292-4bf6-8a39-13ad27da5acb": { 2024-09-21T04:09:45.738 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:08:47.067932+0000", 2024-09-21T04:09:45.738 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:09:45.738 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:09:45.738 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:09:45.738 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:09:45.738 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:09:45.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:09:45.739 INFO:teuthology.orchestra.run.smithi064.stdout: "75db1bc5-612e-45aa-bad2-cc90feeaa840": { 2024-09-21T04:09:45.739 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:59.123138+0000", 2024-09-21T04:09:45.739 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:09:45.739 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:09:45.739 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:09:45.739 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:09:45.739 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:09:45.739 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:09:45.739 INFO:teuthology.orchestra.run.smithi064.stdout: "0ab717a7-36c6-4dca-94e7-af0bdf2aa112": { 2024-09-21T04:09:45.739 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:42.691086+0000", 2024-09-21T04:09:45.739 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:09:45.739 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:09:45.740 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:09:45.740 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:09:45.740 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:09:45.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:09:45.740 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:09:45.740 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:03:59.570174+0000", 2024-09-21T04:09:45.740 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:09:45.740 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:09:45.740 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:09:45.740 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:09:45.740 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:09:45.740 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:09:45.740 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:09:45.740 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:09:45.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.727+0000 7f54fb8d2640 1 -- 172.21.15.64:0/173176072 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f54e00b6a20 msgr2=0x7f54e00b8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:45.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.727+0000 7f54fb8d2640 1 --2- 172.21.15.64:0/173176072 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f54e00b6a20 0x7f54e00b8f30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f54f411e960 tx=0x7f54e4009190 comp rx=0 tx=0).stop 2024-09-21T04:09:45.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.727+0000 7f54fb8d2640 1 -- 172.21.15.64:0/173176072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54f411cc30 msgr2=0x7f54f41165d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:45.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.727+0000 7f54fb8d2640 1 --2- 172.21.15.64:0/173176072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54f411cc30 0x7f54f41165d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f54f000b3e0 tx=0x7f54f000b8a0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.730+0000 7f54fb8d2640 1 -- 172.21.15.64:0/173176072 shutdown_connections 2024-09-21T04:09:45.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.730+0000 7f54fb8d2640 1 --2- 172.21.15.64:0/173176072 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f54e00b6a20 0x7f54e00b8f30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.730+0000 7f54fb8d2640 1 --2- 172.21.15.64:0/173176072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54f411cc30 0x7f54f41165d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.730+0000 7f54fb8d2640 1 --2- 172.21.15.64:0/173176072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54f4111f40 0x7f54f411c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.730+0000 7f54fb8d2640 1 --2- 172.21.15.64:0/173176072 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54f410a9e0 0x7f54f411c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:45.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.730+0000 7f54fb8d2640 1 -- 172.21.15.64:0/173176072 >> 172.21.15.64:0/173176072 conn(0x7f54f4070ec0 msgr2=0x7f54f410ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:45.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.731+0000 7f54fb8d2640 1 -- 172.21.15.64:0/173176072 shutdown_connections 2024-09-21T04:09:45.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:09:45.731+0000 7f54fb8d2640 1 -- 172.21.15.64:0/173176072 wait complete. 2024-09-21T04:09:46.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:46 smithi119 ceph-mon[34811]: pgmap v493: 97 pgs: 97 active+clean; 685 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 223 op/s 2024-09-21T04:09:46.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:46 smithi064 ceph-mon[31180]: pgmap v493: 97 pgs: 97 active+clean; 685 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 223 op/s 2024-09-21T04:09:46.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:46 smithi131 ceph-mon[35193]: pgmap v493: 97 pgs: 97 active+clean; 685 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 223 op/s 2024-09-21T04:09:46.575 INFO:tasks.cephfs.filesystem:scrub status for tag:2c0523fb-aae4-43fa-9568-136a9b640051 - {'path': '/', 'tag': '2c0523fb-aae4-43fa-9568-136a9b640051', 'options': 'recursive,force'} 2024-09-21T04:09:47.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:47 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:09:47.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:47 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:47.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:47 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:09:47.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:47 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:09:47.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:47 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:47.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:47 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:09:47.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:47 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:09:47.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:47 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:47.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:47 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:09:48.454 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:48 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:48.454 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:48 smithi131 ceph-mon[35193]: pgmap v494: 97 pgs: 97 active+clean; 694 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 330 op/s 2024-09-21T04:09:48.454 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:48 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:09:48.455 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:48 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:48.455 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:48 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:09:48.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:48 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:48.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:48 smithi119 ceph-mon[34811]: pgmap v494: 97 pgs: 97 active+clean; 694 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 330 op/s 2024-09-21T04:09:48.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:48 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:09:48.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:48 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:48.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:48 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:09:48.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:48 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:48.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:48 smithi064 ceph-mon[31180]: pgmap v494: 97 pgs: 97 active+clean; 694 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 330 op/s 2024-09-21T04:09:48.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:48 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:09:48.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:48 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:48.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:48 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:09:49.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:49 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:49.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:49 smithi119 ceph-mon[34811]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:09:49.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:49 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:49.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:49 smithi064 ceph-mon[31180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:09:49.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:49 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:09:49.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:49 smithi131 ceph-mon[35193]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:09:50.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:50 smithi119 ceph-mon[34811]: pgmap v495: 97 pgs: 97 active+clean; 703 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 345 op/s 2024-09-21T04:09:50.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:50 smithi064 ceph-mon[31180]: pgmap v495: 97 pgs: 97 active+clean; 703 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 345 op/s 2024-09-21T04:09:50.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:50 smithi131 ceph-mon[35193]: pgmap v495: 97 pgs: 97 active+clean; 703 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 345 op/s 2024-09-21T04:09:51.174 INFO:tasks.workunit.client.0.smithi064.stdout: 119 1172689 340 841593 146 704987 813 2024-09-21T04:09:52.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:52 smithi119 ceph-mon[34811]: pgmap v496: 97 pgs: 97 active+clean; 706 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 348 op/s 2024-09-21T04:09:52.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:52 smithi064 ceph-mon[31180]: pgmap v496: 97 pgs: 97 active+clean; 706 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 348 op/s 2024-09-21T04:09:52.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:52 smithi131 ceph-mon[35193]: pgmap v496: 97 pgs: 97 active+clean; 706 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 348 op/s 2024-09-21T04:09:54.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:54 smithi119 ceph-mon[34811]: pgmap v497: 97 pgs: 97 active+clean; 761 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.7 MiB/s wr, 774 op/s 2024-09-21T04:09:54.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:54 smithi064 ceph-mon[31180]: pgmap v497: 97 pgs: 97 active+clean; 761 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.7 MiB/s wr, 774 op/s 2024-09-21T04:09:54.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:54 smithi131 ceph-mon[35193]: pgmap v497: 97 pgs: 97 active+clean; 761 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.7 MiB/s wr, 774 op/s 2024-09-21T04:09:56.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:56 smithi119 ceph-mon[34811]: pgmap v498: 97 pgs: 97 active+clean; 761 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.8 MiB/s wr, 633 op/s 2024-09-21T04:09:56.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:56 smithi064 ceph-mon[31180]: pgmap v498: 97 pgs: 97 active+clean; 761 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.8 MiB/s wr, 633 op/s 2024-09-21T04:09:56.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:56 smithi131 ceph-mon[35193]: pgmap v498: 97 pgs: 97 active+clean; 761 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.8 MiB/s wr, 633 op/s 2024-09-21T04:09:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.185+0000 7f3b93ec5640 1 -- 172.21.15.64:0/2113902334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b8c1029b0 msgr2=0x7f3b8c102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.185+0000 7f3b93ec5640 1 --2- 172.21.15.64:0/2113902334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b8c1029b0 0x7f3b8c102dc0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f3b88009900 tx=0x7f3b8802a410 comp rx=0 tx=0).stop 2024-09-21T04:09:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.185+0000 7f3b93ec5640 1 -- 172.21.15.64:0/2113902334 shutdown_connections 2024-09-21T04:09:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.185+0000 7f3b93ec5640 1 --2- 172.21.15.64:0/2113902334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b8c108c30 0x7f3b8c10f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.188 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.185+0000 7f3b93ec5640 1 --2- 172.21.15.64:0/2113902334 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b8c1082e0 0x7f3b8c1086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.185+0000 7f3b93ec5640 1 --2- 172.21.15.64:0/2113902334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b8c1029b0 0x7f3b8c102dc0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.185+0000 7f3b93ec5640 1 -- 172.21.15.64:0/2113902334 >> 172.21.15.64:0/2113902334 conn(0x7f3b8c07a740 msgr2=0x7f3b8c07ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.185+0000 7f3b93ec5640 1 -- 172.21.15.64:0/2113902334 shutdown_connections 2024-09-21T04:09:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.185+0000 7f3b93ec5640 1 -- 172.21.15.64:0/2113902334 wait complete. 2024-09-21T04:09:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.186+0000 7f3b93ec5640 1 Processor -- start 2024-09-21T04:09:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.186+0000 7f3b93ec5640 1 -- start start 2024-09-21T04:09:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.186+0000 7f3b93ec5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b8c1029b0 0x7f3b8c1993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.186+0000 7f3b93ec5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b8c1082e0 0x7f3b8c199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.186+0000 7f3b93ec5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b8c108c30 0x7f3b8c19ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.186+0000 7f3b93ec5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c1128f0 con 0x7f3b8c1082e0 2024-09-21T04:09:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.186+0000 7f3b93ec5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c112770 con 0x7f3b8c1029b0 2024-09-21T04:09:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.186+0000 7f3b93ec5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c112a70 con 0x7f3b8c108c30 2024-09-21T04:09:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.187+0000 7f3b91460640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b8c1082e0 0x7f3b8c199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.187+0000 7f3b91c61640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b8c1029b0 0x7f3b8c1993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.187+0000 7f3b91460640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b8c1082e0 0x7f3b8c199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58938/0 (socket says 172.21.15.64:58938) 2024-09-21T04:09:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.187+0000 7f3b91c61640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b8c1029b0 0x7f3b8c1993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:60378/0 (socket says 172.21.15.64:60378) 2024-09-21T04:09:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.187+0000 7f3b91460640 1 -- 172.21.15.64:0/1401894674 learned_addr learned my addr 172.21.15.64:0/1401894674 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.187+0000 7f3b92462640 1 --2- 172.21.15.64:0/1401894674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b8c108c30 0x7f3b8c19ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.187+0000 7f3b91460640 1 -- 172.21.15.64:0/1401894674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b8c108c30 msgr2=0x7f3b8c19ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.187+0000 7f3b91460640 1 --2- 172.21.15.64:0/1401894674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b8c108c30 0x7f3b8c19ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.187+0000 7f3b91460640 1 -- 172.21.15.64:0/1401894674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b8c1029b0 msgr2=0x7f3b8c1993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.187+0000 7f3b91460640 1 --2- 172.21.15.64:0/1401894674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b8c1029b0 0x7f3b8c1993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.187+0000 7f3b91460640 1 -- 172.21.15.64:0/1401894674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b8c19f300 con 0x7f3b8c1082e0 2024-09-21T04:09:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.187+0000 7f3b91c61640 1 --2- 172.21.15.64:0/1401894674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b8c1029b0 0x7f3b8c1993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:09:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.187+0000 7f3b91460640 1 --2- 172.21.15.64:0/1401894674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b8c1082e0 0x7f3b8c199970 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c00d740 tx=0x7f3b7c00db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.187+0000 7f3b82ffd640 1 -- 172.21.15.64:0/1401894674 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c017070 con 0x7f3b8c1082e0 2024-09-21T04:09:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.187+0000 7f3b82ffd640 1 -- 172.21.15.64:0/1401894674 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b7c010460 con 0x7f3b8c1082e0 2024-09-21T04:09:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.187+0000 7f3b93ec5640 1 -- 172.21.15.64:0/1401894674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b8c19f5f0 con 0x7f3b8c1082e0 2024-09-21T04:09:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.187+0000 7f3b82ffd640 1 -- 172.21.15.64:0/1401894674 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c00f600 con 0x7f3b8c1082e0 2024-09-21T04:09:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.187+0000 7f3b93ec5640 1 -- 172.21.15.64:0/1401894674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b8c1a5d80 con 0x7f3b8c1082e0 2024-09-21T04:09:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.190+0000 7f3b9ae32740 1 Processor -- start 2024-09-21T04:09:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.190+0000 7f3b9ae32740 1 -- start start 2024-09-21T04:09:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.191+0000 7f3b9ae32740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a893da91b0 0x55a8941536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.191+0000 7f3b9ae32740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a893d802f0 0x55a894150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.191+0000 7f3b9ae32740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a893d884d0 0x55a894151300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.191+0000 7f3b9ae32740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a8940c36f0 con 0x55a893d884d0 2024-09-21T04:09:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.191+0000 7f3b9ae32740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a8940b2b20 con 0x55a893d802f0 2024-09-21T04:09:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.191+0000 7f3b9ae32740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a8940b6930 con 0x55a893da91b0 2024-09-21T04:09:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.191+0000 7f3b82ffd640 1 -- 172.21.15.64:0/1401894674 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f3b7c01a070 con 0x7f3b8c1082e0 2024-09-21T04:09:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.191+0000 7f3b91c61640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a893d802f0 0x55a894150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.191+0000 7f3b91c61640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a893d802f0 0x55a894150bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:60386/0 (socket says 172.21.15.64:60386) 2024-09-21T04:09:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.191+0000 7f3b91c61640 1 -- 172.21.15.64:0/2050608959 learned_addr learned my addr 172.21.15.64:0/2050608959 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.191+0000 7f3b92462640 1 --2- 172.21.15.64:0/2050608959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a893da91b0 0x55a8941536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.191+0000 7f3b91460640 1 --2- 172.21.15.64:0/2050608959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a893d884d0 0x55a894151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.191+0000 7f3b91c61640 1 -- 172.21.15.64:0/2050608959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a893da91b0 msgr2=0x55a8941536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.191+0000 7f3b91c61640 1 --2- 172.21.15.64:0/2050608959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a893da91b0 0x55a8941536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.191+0000 7f3b91c61640 1 -- 172.21.15.64:0/2050608959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a893d884d0 msgr2=0x55a894151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.191+0000 7f3b91c61640 1 --2- 172.21.15.64:0/2050608959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a893d884d0 0x55a894151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.191+0000 7f3b91c61640 1 -- 172.21.15.64:0/2050608959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a894078530 con 0x55a893d802f0 2024-09-21T04:09:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.191+0000 7f3b92462640 1 --2- 172.21.15.64:0/2050608959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a893da91b0 0x55a8941536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:09:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.192+0000 7f3b91c61640 1 --2- 172.21.15.64:0/2050608959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a893d802f0 0x55a894150bc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3b88009900 tx=0x7f3b88005ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4ed7f4134c8c8bf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.192+0000 7f3b77fff640 1 -- 172.21.15.64:0/2050608959 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b88040070 con 0x55a893d802f0 2024-09-21T04:09:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.192+0000 7f3b82ffd640 1 --2- 172.21.15.64:0/1401894674 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3b780b6680 0x7f3b780b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.192+0000 7f3b77fff640 1 -- 172.21.15.64:0/2050608959 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b88031820 con 0x55a893d802f0 2024-09-21T04:09:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.192+0000 7f3b77fff640 1 -- 172.21.15.64:0/2050608959 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b88035040 con 0x55a893d802f0 2024-09-21T04:09:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.192+0000 7f3b92462640 1 --2- 172.21.15.64:0/1401894674 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3b780b6680 0x7f3b780b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.192+0000 7f3b82ffd640 1 -- 172.21.15.64:0/1401894674 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f3b7c0e70b0 con 0x7f3b8c1082e0 2024-09-21T04:09:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.193+0000 7f3b9ae32740 1 -- 172.21.15.64:0/2050608959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a893d802f0 msgr2=0x55a894150bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.193+0000 7f3b92462640 1 --2- 172.21.15.64:0/1401894674 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3b780b6680 0x7f3b780b8b90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3b84005fd0 tx=0x7f3b84009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.193+0000 7f3b9ae32740 1 --2- 172.21.15.64:0/2050608959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a893d802f0 0x55a894150bc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3b88009900 tx=0x7f3b88005ed0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.193+0000 7f3b9ae32740 1 -- 172.21.15.64:0/2050608959 shutdown_connections 2024-09-21T04:09:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.193+0000 7f3b9ae32740 1 --2- 172.21.15.64:0/2050608959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a893d884d0 0x55a894151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.193+0000 7f3b9ae32740 1 --2- 172.21.15.64:0/2050608959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a893d802f0 0x55a894150bc0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.193+0000 7f3b9ae32740 1 --2- 172.21.15.64:0/2050608959 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a893da91b0 0x55a8941536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.193+0000 7f3b9ae32740 1 -- 172.21.15.64:0/2050608959 >> 172.21.15.64:0/2050608959 conn(0x55a893d9d300 msgr2=0x55a89414eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.193+0000 7f3b9ae32740 1 -- 172.21.15.64:0/2050608959 shutdown_connections 2024-09-21T04:09:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.193+0000 7f3b9ae32740 1 -- 172.21.15.64:0/2050608959 wait complete. 2024-09-21T04:09:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.194+0000 7f3b9ae32740 1 -- start start 2024-09-21T04:09:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.194+0000 7f3b9ae32740 1 Processor -- start 2024-09-21T04:09:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.194+0000 7f3b9ae32740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a893d802f0 0x55a89415e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.194+0000 7f3b9ae32740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a893d884d0 0x55a89415eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.194+0000 7f3b9ae32740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a893da91b0 0x55a89415f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.194+0000 7f3b9ae32740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a893feaab0 con 0x55a893da91b0 2024-09-21T04:09:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.194+0000 7f3b9ae32740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a894058730 con 0x55a893d884d0 2024-09-21T04:09:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.194+0000 7f3b9ae32740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a894161510 con 0x55a893d802f0 2024-09-21T04:09:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.194+0000 7f3b9ae32740 10 client.0 _ll_register_callbacks cb 0x55a8940ac6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:09:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.194+0000 7f3b92462640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a893da91b0 0x55a89415f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.194+0000 7f3b91460640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a893d884d0 0x55a89415eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.194+0000 7f3b91c61640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a893d802f0 0x55a89415e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.194+0000 7f3b92462640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a893da91b0 0x55a89415f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58966/0 (socket says 172.21.15.64:58966) 2024-09-21T04:09:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.194+0000 7f3b92462640 1 -- 172.21.15.64:0/2172600096 learned_addr learned my addr 172.21.15.64:0/2172600096 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.195+0000 7f3b92462640 1 -- 172.21.15.64:0/2172600096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a893d802f0 msgr2=0x55a89415e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.195+0000 7f3b92462640 1 --2- 172.21.15.64:0/2172600096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a893d802f0 0x55a89415e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.195+0000 7f3b92462640 1 -- 172.21.15.64:0/2172600096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a893d884d0 msgr2=0x55a89415eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.195+0000 7f3b92462640 1 --2- 172.21.15.64:0/2172600096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a893d884d0 0x55a89415eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.195+0000 7f3b92462640 1 -- 172.21.15.64:0/2172600096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a893e14790 con 0x55a893da91b0 2024-09-21T04:09:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.195+0000 7f3b91c61640 1 --2- 172.21.15.64:0/2172600096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a893d802f0 0x55a89415e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:09:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.195+0000 7f3b91460640 1 --2- 172.21.15.64:0/2172600096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a893d884d0 0x55a89415eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:09:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.195+0000 7f3b92462640 1 --2- 172.21.15.64:0/2172600096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a893da91b0 0x55a89415f060 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3b84009b80 tx=0x7f3b84004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7239cf1719b05f0f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.195+0000 7f3b757fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:09:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.195+0000 7f3b757fa640 1 -- 172.21.15.64:0/2172600096 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b84003b80 con 0x55a893da91b0 2024-09-21T04:09:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.195+0000 7f3b9ae32740 1 -- 172.21.15.64:0/2172600096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b88031820 con 0x55a893da91b0 2024-09-21T04:09:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.195+0000 7f3b757fa640 1 -- 172.21.15.64:0/2172600096 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f3b84003d20 con 0x55a893da91b0 2024-09-21T04:09:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.195+0000 7f3b757fa640 1 -- 172.21.15.64:0/2172600096 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3b84004050 con 0x55a893da91b0 2024-09-21T04:09:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.195+0000 7f3b9ae32740 10 client.16767 fetch_fsmap learned FSMap version 37 2024-09-21T04:09:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.195+0000 7f3b9ae32740 1 -- 172.21.15.64:0/2172600096 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a893e8d620 con 0x55a893da91b0 2024-09-21T04:09:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.196+0000 7f3b757fa640 1 -- 172.21.15.64:0/2172600096 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b88031820 con 0x55a893da91b0 2024-09-21T04:09:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.196+0000 7f3b757fa640 1 -- 172.21.15.64:0/2172600096 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f3b84030070 con 0x55a893da91b0 2024-09-21T04:09:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.196+0000 7f3b9ae32740 10 client.16767 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T04:09:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.196+0000 7f3b9ae32740 10 client.16767 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:09:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.196+0000 7f3b9ae32740 20 client.16767 populate_metadata read hostname 'smithi064' 2024-09-21T04:09:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.196+0000 7f3b9ae32740 1 --2- 172.21.15.64:0/2172600096 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a893d67d80 0x55a8941705d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.196+0000 7f3b9ae32740 4 client.16767 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:09:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.196+0000 7f3b9ae32740 1 -- 172.21.15.64:0/2172600096 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8940c5640 con 0x55a893d67d80 2024-09-21T04:09:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.196+0000 7f3b91c61640 1 --2- 172.21.15.64:0/2172600096 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a893d67d80 0x55a8941705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.197+0000 7f3b91c61640 1 --2- 172.21.15.64:0/2172600096 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a893d67d80 0x55a8941705d0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=619776b9378aa8d3 server_cookie=5a154c7f704ebe5b in_seq=0 out_seq=0 2024-09-21T04:09:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.197+0000 7f3b757fa640 10 client.16767 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.197+0000 7f3b757fa640 1 -- 172.21.15.64:0/2172600096 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a8940c5640 con 0x55a893d67d80 2024-09-21T04:09:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.197+0000 7f3b757fa640 10 client.16767 handle_command_reply: tid=0 2024-09-21T04:09:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.197+0000 7f3b9ae32740 1 client.16767 shutdown 2024-09-21T04:09:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.197+0000 7f3b9ae32740 1 -- 172.21.15.64:0/2172600096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a893da91b0 msgr2=0x55a89415f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.197+0000 7f3b9ae32740 1 --2- 172.21.15.64:0/2172600096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a893da91b0 0x55a89415f060 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3b84009b80 tx=0x7f3b84004ab0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.198+0000 7f3b9ae32740 1 -- 172.21.15.64:0/2172600096 shutdown_connections 2024-09-21T04:09:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.198+0000 7f3b9ae32740 1 --2- 172.21.15.64:0/2172600096 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a893d67d80 0x55a8941705d0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.198+0000 7f3b9ae32740 1 --2- 172.21.15.64:0/2172600096 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a893da91b0 0x55a89415f060 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.198+0000 7f3b757fa640 0 client.16767 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.198+0000 7f3b9ae32740 1 --2- 172.21.15.64:0/2172600096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a893d884d0 0x55a89415eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.198+0000 7f3b9ae32740 1 --2- 172.21.15.64:0/2172600096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a893d802f0 0x55a89415e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.198+0000 7f3b9ae32740 1 -- 172.21.15.64:0/2172600096 >> 172.21.15.64:0/2172600096 conn(0x55a893d9d300 msgr2=0x55a8941525b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.198+0000 7f3b9ae32740 1 -- 172.21.15.64:0/2172600096 shutdown_connections 2024-09-21T04:09:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.198+0000 7f3b9ae32740 1 -- 172.21.15.64:0/2172600096 wait complete. 2024-09-21T04:09:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.198+0000 7f3b9ae32740 20 client.16767 trim_cache size 0 max 16384 2024-09-21T04:09:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.216+0000 7f3b9ae32740 1 Processor -- start 2024-09-21T04:09:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.216+0000 7f3b9ae32740 1 -- start start 2024-09-21T04:09:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.216+0000 7f3b9ae32740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a893d67d80 0x55a89414dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.216+0000 7f3b9ae32740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a893d802f0 0x55a89416a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.216+0000 7f3b9ae32740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a893d884d0 0x55a89415eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.216+0000 7f3b9ae32740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c1b0000 con 0x55a893d802f0 2024-09-21T04:09:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.216+0000 7f3b9ae32740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c1ad8a0 con 0x55a893d884d0 2024-09-21T04:09:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.216+0000 7f3b9ae32740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c1ab6f0 con 0x55a893d67d80 2024-09-21T04:09:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.216+0000 7f3b91460640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a893d802f0 0x55a89416a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.216+0000 7f3b91460640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a893d802f0 0x55a89416a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58982/0 (socket says 172.21.15.64:58982) 2024-09-21T04:09:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.216+0000 7f3b91460640 1 -- 172.21.15.64:0/501286159 learned_addr learned my addr 172.21.15.64:0/501286159 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.216+0000 7f3b92462640 1 --2- 172.21.15.64:0/501286159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a893d884d0 0x55a89415eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.216+0000 7f3b91c61640 1 --2- 172.21.15.64:0/501286159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a893d67d80 0x55a89414dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.216+0000 7f3b91460640 1 -- 172.21.15.64:0/501286159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a893d67d80 msgr2=0x55a89414dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.216+0000 7f3b91460640 1 --2- 172.21.15.64:0/501286159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a893d67d80 0x55a89414dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.216+0000 7f3b91460640 1 -- 172.21.15.64:0/501286159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a893d884d0 msgr2=0x55a89415eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.216+0000 7f3b91460640 1 --2- 172.21.15.64:0/501286159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a893d884d0 0x55a89415eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.216+0000 7f3b91460640 1 -- 172.21.15.64:0/501286159 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a89415d7e0 con 0x55a893d802f0 2024-09-21T04:09:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.217+0000 7f3b91460640 1 --2- 172.21.15.64:0/501286159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a893d802f0 0x55a89416a730 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0e4dd0 tx=0x7f3b7c010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1744d60e736fa536 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.217+0000 7f3b777fe640 1 -- 172.21.15.64:0/501286159 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c021070 con 0x55a893d802f0 2024-09-21T04:09:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.217+0000 7f3b777fe640 1 -- 172.21.15.64:0/501286159 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b7c0b19b0 con 0x55a893d802f0 2024-09-21T04:09:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.217+0000 7f3b777fe640 1 -- 172.21.15.64:0/501286159 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c0940b0 con 0x55a893d802f0 2024-09-21T04:09:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.218+0000 7f3b9ae32740 1 -- 172.21.15.64:0/501286159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a893d802f0 msgr2=0x55a89416a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.218+0000 7f3b9ae32740 1 --2- 172.21.15.64:0/501286159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a893d802f0 0x55a89416a730 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c0e4dd0 tx=0x7f3b7c010ce0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.218+0000 7f3b9ae32740 1 -- 172.21.15.64:0/501286159 shutdown_connections 2024-09-21T04:09:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.218+0000 7f3b9ae32740 1 --2- 172.21.15.64:0/501286159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a893d884d0 0x55a89415eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.218+0000 7f3b9ae32740 1 --2- 172.21.15.64:0/501286159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a893d802f0 0x55a89416a730 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.218+0000 7f3b9ae32740 1 --2- 172.21.15.64:0/501286159 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a893d67d80 0x55a89414dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.218+0000 7f3b9ae32740 1 -- 172.21.15.64:0/501286159 >> 172.21.15.64:0/501286159 conn(0x55a893d9d300 msgr2=0x55a89414d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.218+0000 7f3b9ae32740 1 -- 172.21.15.64:0/501286159 shutdown_connections 2024-09-21T04:09:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.218+0000 7f3b9ae32740 1 -- 172.21.15.64:0/501286159 wait complete. 2024-09-21T04:09:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.218+0000 7f3b9ae32740 1 -- start start 2024-09-21T04:09:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.218+0000 7f3b9ae32740 1 Processor -- start 2024-09-21T04:09:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.219+0000 7f3b9ae32740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a893d67d80 0x55a894166c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.219+0000 7f3b9ae32740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a893d802f0 0x55a8941671a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.219+0000 7f3b9ae32740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a893d884d0 0x55a894175840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.219+0000 7f3b9ae32740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c1abd60 con 0x55a893d884d0 2024-09-21T04:09:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.219+0000 7f3b9ae32740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c1ad1d0 con 0x55a893d802f0 2024-09-21T04:09:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.219+0000 7f3b9ae32740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c1ac430 con 0x55a893d67d80 2024-09-21T04:09:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.219+0000 7f3b9ae32740 10 client.0 _ll_register_callbacks cb 0x7f3b84009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:09:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.219+0000 7f3b92462640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a893d884d0 0x55a894175840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.219+0000 7f3b92462640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a893d884d0 0x55a894175840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:58984/0 (socket says 172.21.15.64:58984) 2024-09-21T04:09:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.219+0000 7f3b92462640 1 -- 172.21.15.64:0/224693682 learned_addr learned my addr 172.21.15.64:0/224693682 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:09:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.219+0000 7f3b91460640 1 --2- 172.21.15.64:0/224693682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a893d802f0 0x55a8941671a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.219+0000 7f3b91c61640 1 --2- 172.21.15.64:0/224693682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a893d67d80 0x55a894166c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.219+0000 7f3b92462640 1 -- 172.21.15.64:0/224693682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a893d67d80 msgr2=0x55a894166c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.219+0000 7f3b92462640 1 --2- 172.21.15.64:0/224693682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a893d67d80 0x55a894166c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.219+0000 7f3b92462640 1 -- 172.21.15.64:0/224693682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a893d802f0 msgr2=0x55a8941671a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:09:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.219+0000 7f3b92462640 1 --2- 172.21.15.64:0/224693682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a893d802f0 0x55a8941671a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.219+0000 7f3b92462640 1 -- 172.21.15.64:0/224693682 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a894175fd0 con 0x55a893d884d0 2024-09-21T04:09:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.220+0000 7f3b92462640 1 --2- 172.21.15.64:0/224693682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a893d884d0 0x55a894175840 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3b84012f60 tx=0x7f3b840117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725b7afdde6344f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:09:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.220+0000 7f3b757fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:09:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.220+0000 7f3b757fa640 1 -- 172.21.15.64:0/224693682 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b84036040 con 0x55a893d884d0 2024-09-21T04:09:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.220+0000 7f3b757fa640 1 -- 172.21.15.64:0/224693682 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f3b840041f0 con 0x55a893d884d0 2024-09-21T04:09:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.220+0000 7f3b9ae32740 1 -- 172.21.15.64:0/224693682 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b7c0b19b0 con 0x55a893d884d0 2024-09-21T04:09:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.220+0000 7f3b757fa640 1 -- 172.21.15.64:0/224693682 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3b84047050 con 0x55a893d884d0 2024-09-21T04:09:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.220+0000 7f3b9ae32740 10 client.16773 fetch_fsmap learned FSMap version 37 2024-09-21T04:09:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.220+0000 7f3b9ae32740 1 -- 172.21.15.64:0/224693682 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a8941dabf0 con 0x55a893d884d0 2024-09-21T04:09:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.220+0000 7f3b757fa640 1 -- 172.21.15.64:0/224693682 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b7c0b19b0 con 0x55a893d884d0 2024-09-21T04:09:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.220+0000 7f3b757fa640 1 -- 172.21.15.64:0/224693682 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f3b84042030 con 0x55a893d884d0 2024-09-21T04:09:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.220+0000 7f3b9ae32740 10 client.16773 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T04:09:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.220+0000 7f3b9ae32740 10 client.16773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:09:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.220+0000 7f3b9ae32740 20 client.16773 populate_metadata read hostname 'smithi064' 2024-09-21T04:09:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.220+0000 7f3b9ae32740 1 --2- 172.21.15.64:0/224693682 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a893da91b0 0x55a8941dd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:09:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.220+0000 7f3b9ae32740 4 client.16773 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:09:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.220+0000 7f3b9ae32740 1 -- 172.21.15.64:0/224693682 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a8940c10f0 con 0x55a893da91b0 2024-09-21T04:09:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.221+0000 7f3b91c61640 1 --2- 172.21.15.64:0/224693682 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a893da91b0 0x55a8941dd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:09:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.221+0000 7f3b757fa640 10 client.16773 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:57.227 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.221+0000 7f3b91c61640 1 --2- 172.21.15.64:0/224693682 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a893da91b0 0x55a8941dd400 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceb733fb8091dc2 server_cookie=b3493a2fcbf426f8 in_seq=0 out_seq=0 2024-09-21T04:09:57.259 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.256+0000 7f3b757fa640 1 -- 172.21.15.64:0/224693682 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a8940c10f0 con 0x55a893da91b0 2024-09-21T04:09:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.256+0000 7f3b757fa640 10 client.16773 handle_command_reply: tid=0 2024-09-21T04:09:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.256+0000 7f3b9ae32740 1 client.16773 shutdown 2024-09-21T04:09:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.256+0000 7f3b9ae32740 1 -- 172.21.15.64:0/224693682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a893d884d0 msgr2=0x55a894175840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.256+0000 7f3b9ae32740 1 --2- 172.21.15.64:0/224693682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a893d884d0 0x55a894175840 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3b84012f60 tx=0x7f3b840117b0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.256+0000 7f3b9ae32740 1 -- 172.21.15.64:0/224693682 shutdown_connections 2024-09-21T04:09:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.256+0000 7f3b9ae32740 1 --2- 172.21.15.64:0/224693682 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a893da91b0 0x55a8941dd400 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.256+0000 7f3b9ae32740 1 --2- 172.21.15.64:0/224693682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a893d884d0 0x55a894175840 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.256+0000 7f3b9ae32740 1 --2- 172.21.15.64:0/224693682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a893d802f0 0x55a8941671a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.256+0000 7f3b9ae32740 1 --2- 172.21.15.64:0/224693682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a893d67d80 0x55a894166c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.256+0000 7f3b757fa640 0 client.16773 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:09:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.256+0000 7f3b9ae32740 1 -- 172.21.15.64:0/224693682 >> 172.21.15.64:0/224693682 conn(0x55a893d9d300 msgr2=0x55a894151bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.257+0000 7f3b9ae32740 1 -- 172.21.15.64:0/224693682 shutdown_connections 2024-09-21T04:09:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.257+0000 7f3b9ae32740 1 -- 172.21.15.64:0/224693682 wait complete. 2024-09-21T04:09:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.257+0000 7f3b9ae32740 20 client.16773 trim_cache size 0 max 16384 2024-09-21T04:09:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:09:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:09:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:09:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:09:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.257+0000 7f3b93ec5640 1 -- 172.21.15.64:0/1401894674 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3b780b6680 msgr2=0x7f3b780b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.257+0000 7f3b93ec5640 1 --2- 172.21.15.64:0/1401894674 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3b780b6680 0x7f3b780b8b90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3b84005fd0 tx=0x7f3b84009500 comp rx=0 tx=0).stop 2024-09-21T04:09:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.257+0000 7f3b93ec5640 1 -- 172.21.15.64:0/1401894674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b8c1082e0 msgr2=0x7f3b8c199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:09:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.257+0000 7f3b93ec5640 1 --2- 172.21.15.64:0/1401894674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b8c1082e0 0x7f3b8c199970 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c00d740 tx=0x7f3b7c00db40 comp rx=0 tx=0).stop 2024-09-21T04:09:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.257+0000 7f3b93ec5640 1 -- 172.21.15.64:0/1401894674 shutdown_connections 2024-09-21T04:09:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.257+0000 7f3b93ec5640 1 --2- 172.21.15.64:0/1401894674 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f3b780b6680 0x7f3b780b8b90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.257+0000 7f3b93ec5640 1 --2- 172.21.15.64:0/1401894674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b8c108c30 0x7f3b8c19ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.257+0000 7f3b93ec5640 1 --2- 172.21.15.64:0/1401894674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3b8c1082e0 0x7f3b8c199970 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.257+0000 7f3b93ec5640 1 --2- 172.21.15.64:0/1401894674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b8c1029b0 0x7f3b8c1993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:09:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.257+0000 7f3b93ec5640 1 -- 172.21.15.64:0/1401894674 >> 172.21.15.64:0/1401894674 conn(0x7f3b8c07a740 msgr2=0x7f3b8c106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:09:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.257+0000 7f3b93ec5640 1 -- 172.21.15.64:0/1401894674 shutdown_connections 2024-09-21T04:09:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:09:57.257+0000 7f3b93ec5640 1 -- 172.21.15.64:0/1401894674 wait complete. 2024-09-21T04:09:58.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:09:58 smithi119 ceph-mon[34811]: pgmap v499: 97 pgs: 97 active+clean; 748 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.8 MiB/s wr, 879 op/s 2024-09-21T04:09:58.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:09:58 smithi064 ceph-mon[31180]: pgmap v499: 97 pgs: 97 active+clean; 748 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.8 MiB/s wr, 879 op/s 2024-09-21T04:09:58.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:09:58 smithi131 ceph-mon[35193]: pgmap v499: 97 pgs: 97 active+clean; 748 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.8 MiB/s wr, 879 op/s 2024-09-21T04:10:00.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:00 smithi119 ceph-mon[34811]: pgmap v500: 97 pgs: 97 active+clean; 743 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.3 MiB/s wr, 861 op/s 2024-09-21T04:10:00.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:00 smithi119 ceph-mon[34811]: overall HEALTH_OK 2024-09-21T04:10:00.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:00 smithi064 ceph-mon[31180]: pgmap v500: 97 pgs: 97 active+clean; 743 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.3 MiB/s wr, 861 op/s 2024-09-21T04:10:00.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:00 smithi064 ceph-mon[31180]: overall HEALTH_OK 2024-09-21T04:10:00.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:00 smithi131 ceph-mon[35193]: pgmap v500: 97 pgs: 97 active+clean; 743 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.3 MiB/s wr, 861 op/s 2024-09-21T04:10:00.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:00 smithi131 ceph-mon[35193]: overall HEALTH_OK 2024-09-21T04:10:01.174 INFO:tasks.workunit.client.0.smithi064.stdout: 119 8830 1 5491 3 3196 4 2024-09-21T04:10:02.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:02 smithi119 ceph-mon[34811]: pgmap v501: 97 pgs: 97 active+clean; 747 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.9 MiB/s wr, 832 op/s 2024-09-21T04:10:02.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:02 smithi064 ceph-mon[31180]: pgmap v501: 97 pgs: 97 active+clean; 747 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.9 MiB/s wr, 832 op/s 2024-09-21T04:10:02.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:02 smithi131 ceph-mon[35193]: pgmap v501: 97 pgs: 97 active+clean; 747 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.9 MiB/s wr, 832 op/s 2024-09-21T04:10:04.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:04 smithi119 ceph-mon[34811]: pgmap v502: 97 pgs: 97 active+clean; 749 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.7 MiB/s wr, 1.06k op/s 2024-09-21T04:10:04.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:04 smithi064 ceph-mon[31180]: pgmap v502: 97 pgs: 97 active+clean; 749 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.7 MiB/s wr, 1.06k op/s 2024-09-21T04:10:04.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:04 smithi131 ceph-mon[35193]: pgmap v502: 97 pgs: 97 active+clean; 749 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.7 MiB/s wr, 1.06k op/s 2024-09-21T04:10:06.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:06 smithi119 ceph-mon[34811]: pgmap v503: 97 pgs: 97 active+clean; 749 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 631 op/s 2024-09-21T04:10:06.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:06 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:10:06.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:06 smithi064 ceph-mon[31180]: pgmap v503: 97 pgs: 97 active+clean; 749 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 631 op/s 2024-09-21T04:10:06.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:06 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:10:06.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:06 smithi131 ceph-mon[35193]: pgmap v503: 97 pgs: 97 active+clean; 749 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 631 op/s 2024-09-21T04:10:06.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:06 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:10:08.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:08 smithi119 ceph-mon[34811]: pgmap v504: 97 pgs: 97 active+clean; 753 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 818 op/s 2024-09-21T04:10:08.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:08 smithi064 ceph-mon[31180]: pgmap v504: 97 pgs: 97 active+clean; 753 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 818 op/s 2024-09-21T04:10:08.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:08 smithi131 ceph-mon[35193]: pgmap v504: 97 pgs: 97 active+clean; 753 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 818 op/s 2024-09-21T04:10:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.363+0000 7f45b605c640 1 -- 172.21.15.64:0/2895356175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45b0078290 msgr2=0x7f45b0114160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.363+0000 7f45b605c640 1 --2- 172.21.15.64:0/2895356175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45b0078290 0x7f45b0114160 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f45a4009900 tx=0x7f45a402a410 comp rx=0 tx=0).stop 2024-09-21T04:10:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.363+0000 7f45b605c640 1 -- 172.21.15.64:0/2895356175 shutdown_connections 2024-09-21T04:10:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.363+0000 7f45b605c640 1 --2- 172.21.15.64:0/2895356175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45b01146a0 0x7f45b0116ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.363+0000 7f45b605c640 1 --2- 172.21.15.64:0/2895356175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45b0078290 0x7f45b0114160 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.363+0000 7f45b605c640 1 --2- 172.21.15.64:0/2895356175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45b0077940 0x7f45b0077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.363+0000 7f45b605c640 1 -- 172.21.15.64:0/2895356175 >> 172.21.15.64:0/2895356175 conn(0x7f45b01016f0 msgr2=0x7f45b0103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.363+0000 7f45b605c640 1 -- 172.21.15.64:0/2895356175 shutdown_connections 2024-09-21T04:10:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.363+0000 7f45b605c640 1 -- 172.21.15.64:0/2895356175 wait complete. 2024-09-21T04:10:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.363+0000 7f45b605c640 1 Processor -- start 2024-09-21T04:10:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.364+0000 7f45b605c640 1 -- start start 2024-09-21T04:10:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.364+0000 7f45b605c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45b0077940 0x7f45b01a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.364+0000 7f45b605c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45b0078290 0x7f45b01a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.364+0000 7f45b605c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45b01146a0 0x7f45b01a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.364+0000 7f45b605c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f45b0119fe0 con 0x7f45b0077940 2024-09-21T04:10:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.364+0000 7f45b605c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f45b0119e60 con 0x7f45b0078290 2024-09-21T04:10:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.364+0000 7f45b605c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f45b011a160 con 0x7f45b01146a0 2024-09-21T04:10:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.364+0000 7f45af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45b0077940 0x7f45b01a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.364+0000 7f45af7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45b0077940 0x7f45b01a0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50864/0 (socket says 172.21.15.64:50864) 2024-09-21T04:10:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.364+0000 7f45af7fe640 1 -- 172.21.15.64:0/4132189727 learned_addr learned my addr 172.21.15.64:0/4132189727 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.364+0000 7f45aeffd640 1 --2- 172.21.15.64:0/4132189727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45b0078290 0x7f45b01a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.364+0000 7f45affff640 1 --2- 172.21.15.64:0/4132189727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45b01146a0 0x7f45b01a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.364+0000 7f45af7fe640 1 -- 172.21.15.64:0/4132189727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45b01146a0 msgr2=0x7f45b01a62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.364+0000 7f45af7fe640 1 --2- 172.21.15.64:0/4132189727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45b01146a0 0x7f45b01a62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:10.369 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.364+0000 7f45af7fe640 1 -- 172.21.15.64:0/4132189727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45b0078290 msgr2=0x7f45b01a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.364+0000 7f45af7fe640 1 --2- 172.21.15.64:0/4132189727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45b0078290 0x7f45b01a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.364+0000 7f45af7fe640 1 -- 172.21.15.64:0/4132189727 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b01a69d0 con 0x7f45b0077940 2024-09-21T04:10:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.364+0000 7f45affff640 1 --2- 172.21.15.64:0/4132189727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45b01146a0 0x7f45b01a62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:10:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.364+0000 7f45aeffd640 1 --2- 172.21.15.64:0/4132189727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45b0078290 0x7f45b01a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.365+0000 7f45af7fe640 1 --2- 172.21.15.64:0/4132189727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45b0077940 0x7f45b01a0aa0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f45a000d770 tx=0x7f45a000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.365+0000 7f45acff9640 1 -- 172.21.15.64:0/4132189727 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0017070 con 0x7f45b0077940 2024-09-21T04:10:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.365+0000 7f45acff9640 1 -- 172.21.15.64:0/4132189727 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45a00054d0 con 0x7f45b0077940 2024-09-21T04:10:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.365+0000 7f45acff9640 1 -- 172.21.15.64:0/4132189727 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0004c30 con 0x7f45b0077940 2024-09-21T04:10:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.365+0000 7f45b605c640 1 -- 172.21.15.64:0/4132189727 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b01a6cc0 con 0x7f45b0077940 2024-09-21T04:10:10.370 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.365+0000 7f45b605c640 1 -- 172.21.15.64:0/4132189727 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b01ad450 con 0x7f45b0077940 2024-09-21T04:10:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.367+0000 7f45bcfca740 1 Processor -- start 2024-09-21T04:10:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.367+0000 7f45acff9640 1 -- 172.21.15.64:0/4132189727 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f45a0004e10 con 0x7f45b0077940 2024-09-21T04:10:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.367+0000 7f45bcfca740 1 -- start start 2024-09-21T04:10:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.368+0000 7f45bcfca740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5dbe121b0 0x55a5dc1b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.368+0000 7f45bcfca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5dbde92f0 0x55a5dc1b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.368+0000 7f45bcfca740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5dbdf14d0 0x55a5dc1b8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.368+0000 7f45bcfca740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a5dc12c6f0 con 0x55a5dbe121b0 2024-09-21T04:10:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.368+0000 7f45bcfca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a5dc11bb20 con 0x55a5dbde92f0 2024-09-21T04:10:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.368+0000 7f45bcfca740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a5dc11f930 con 0x55a5dbdf14d0 2024-09-21T04:10:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.368+0000 7f45affff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5dbe121b0 0x55a5dc1b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.368+0000 7f45affff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5dbe121b0 0x55a5dc1b79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50866/0 (socket says 172.21.15.64:50866) 2024-09-21T04:10:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.368+0000 7f45affff640 1 -- 172.21.15.64:0/254870327 learned_addr learned my addr 172.21.15.64:0/254870327 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.368+0000 7f45af7fe640 1 --2- 172.21.15.64:0/254870327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5dbde92f0 0x55a5dc1b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.368+0000 7f45aeffd640 1 --2- 172.21.15.64:0/254870327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5dbdf14d0 0x55a5dc1b8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.368+0000 7f45affff640 1 -- 172.21.15.64:0/254870327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5dbdf14d0 msgr2=0x55a5dc1b8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.368+0000 7f45affff640 1 --2- 172.21.15.64:0/254870327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5dbdf14d0 0x55a5dc1b8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.368+0000 7f45affff640 1 -- 172.21.15.64:0/254870327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5dbde92f0 msgr2=0x55a5dc1b7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.368+0000 7f45affff640 1 --2- 172.21.15.64:0/254870327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5dbde92f0 0x55a5dc1b7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.368+0000 7f45affff640 1 -- 172.21.15.64:0/254870327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5dc0e1530 con 0x55a5dbe121b0 2024-09-21T04:10:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.368+0000 7f45acff9640 1 --2- 172.21.15.64:0/4132189727 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f45980b6700 0x7f45980b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.368+0000 7f45af7fe640 1 --2- 172.21.15.64:0/254870327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5dbde92f0 0x55a5dc1b7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.369+0000 7f45af7fe640 1 --2- 172.21.15.64:0/4132189727 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f45980b6700 0x7f45980b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:10.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.369+0000 7f45acff9640 1 -- 172.21.15.64:0/4132189727 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f45a00e71f0 con 0x7f45b0077940 2024-09-21T04:10:10.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.369+0000 7f45affff640 1 --2- 172.21.15.64:0/254870327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5dbe121b0 0x55a5dc1b79b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009fd0 tx=0x7f45a80108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1564b8ee39bfd80a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:10.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.369+0000 7f4595ffb640 1 -- 172.21.15.64:0/254870327 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8011760 con 0x55a5dbe121b0 2024-09-21T04:10:10.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.369+0000 7f4595ffb640 1 -- 172.21.15.64:0/254870327 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45a8011db0 con 0x55a5dbe121b0 2024-09-21T04:10:10.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.369+0000 7f4595ffb640 1 -- 172.21.15.64:0/254870327 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a800e3c0 con 0x55a5dbe121b0 2024-09-21T04:10:10.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.370+0000 7f45bcfca740 1 -- 172.21.15.64:0/254870327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5dbe121b0 msgr2=0x55a5dc1b79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:10.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.370+0000 7f45af7fe640 1 --2- 172.21.15.64:0/4132189727 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f45980b6700 0x7f45980b8c10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f45a00e49c0 tx=0x7f45a00ead40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:10.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.370+0000 7f45bcfca740 1 --2- 172.21.15.64:0/254870327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5dbe121b0 0x55a5dc1b79b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009fd0 tx=0x7f45a80108e0 comp rx=0 tx=0).stop 2024-09-21T04:10:10.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.370+0000 7f45bcfca740 1 -- 172.21.15.64:0/254870327 shutdown_connections 2024-09-21T04:10:10.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.370+0000 7f45bcfca740 1 --2- 172.21.15.64:0/254870327 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5dbdf14d0 0x55a5dc1b8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:10.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.370+0000 7f45bcfca740 1 --2- 172.21.15.64:0/254870327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5dbde92f0 0x55a5dc1b7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:10.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.370+0000 7f45bcfca740 1 --2- 172.21.15.64:0/254870327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5dbe121b0 0x55a5dc1b79b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:10.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.370+0000 7f45bcfca740 1 -- 172.21.15.64:0/254870327 >> 172.21.15.64:0/254870327 conn(0x55a5dbe06300 msgr2=0x55a5dc1b58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:10.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.370+0000 7f45bcfca740 1 -- 172.21.15.64:0/254870327 shutdown_connections 2024-09-21T04:10:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.370+0000 7f45bcfca740 1 -- 172.21.15.64:0/254870327 wait complete. 2024-09-21T04:10:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.370+0000 7f45bcfca740 1 -- start start 2024-09-21T04:10:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.371+0000 7f45bcfca740 1 Processor -- start 2024-09-21T04:10:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.371+0000 7f45bcfca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5dbde92f0 0x55a5dc1c30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.371+0000 7f45bcfca740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5dbdf14d0 0x55a5dc1c3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.371+0000 7f45bcfca740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5dbe121b0 0x55a5dc1cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.371+0000 7f45bcfca740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a5dc053ab0 con 0x55a5dbdf14d0 2024-09-21T04:10:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.371+0000 7f45bcfca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a5dc0c1730 con 0x55a5dbde92f0 2024-09-21T04:10:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.371+0000 7f45bcfca740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a5dc1c6130 con 0x55a5dbe121b0 2024-09-21T04:10:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.371+0000 7f45bcfca740 10 client.0 _ll_register_callbacks cb 0x55a5dc1156b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:10:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.371+0000 7f45affff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5dbde92f0 0x55a5dc1c30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.371+0000 7f45affff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5dbde92f0 0x55a5dc1c30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:43828/0 (socket says 172.21.15.64:43828) 2024-09-21T04:10:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.371+0000 7f45affff640 1 -- 172.21.15.64:0/36843410 learned_addr learned my addr 172.21.15.64:0/36843410 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.371+0000 7f45aeffd640 1 --2- 172.21.15.64:0/36843410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5dbdf14d0 0x55a5dc1c3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.371+0000 7f45affff640 1 --2- 172.21.15.64:0/36843410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5dbe121b0 0x55a5dc1cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.371+0000 7f45affff640 1 -- 172.21.15.64:0/36843410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5dbe121b0 msgr2=0x55a5dc1cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.371+0000 7f45affff640 1 --2- 172.21.15.64:0/36843410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5dbe121b0 0x55a5dc1cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.371+0000 7f45affff640 1 -- 172.21.15.64:0/36843410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5dbdf14d0 msgr2=0x55a5dc1c3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.371+0000 7f45affff640 1 --2- 172.21.15.64:0/36843410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5dbdf14d0 0x55a5dc1c3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.371+0000 7f45affff640 1 -- 172.21.15.64:0/36843410 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5dbe7d790 con 0x55a5dbde92f0 2024-09-21T04:10:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.371+0000 7f45aeffd640 1 --2- 172.21.15.64:0/36843410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5dbdf14d0 0x55a5dc1c3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.372+0000 7f45affff640 1 --2- 172.21.15.64:0/36843410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5dbde92f0 0x55a5dc1c30f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f45a8005cc0 tx=0x7f45a801ba90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a752e745ca5ca70 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.372+0000 7f458effd640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:10:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.372+0000 7f458effd640 1 -- 172.21.15.64:0/36843410 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a801f030 con 0x55a5dbde92f0 2024-09-21T04:10:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.372+0000 7f458effd640 1 -- 172.21.15.64:0/36843410 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f45a8024070 con 0x55a5dbde92f0 2024-09-21T04:10:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.372+0000 7f45bcfca740 1 -- 172.21.15.64:0/36843410 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45a8011db0 con 0x55a5dbde92f0 2024-09-21T04:10:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.372+0000 7f458effd640 1 -- 172.21.15.64:0/36843410 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f45a8031050 con 0x55a5dbde92f0 2024-09-21T04:10:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.372+0000 7f45bcfca740 10 client.26468 fetch_fsmap learned FSMap version 37 2024-09-21T04:10:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.372+0000 7f45bcfca740 1 -- 172.21.15.64:0/36843410 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a5dbef6620 con 0x55a5dbde92f0 2024-09-21T04:10:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.372+0000 7f458effd640 1 -- 172.21.15.64:0/36843410 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45a8011db0 con 0x55a5dbde92f0 2024-09-21T04:10:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.372+0000 7f458effd640 1 -- 172.21.15.64:0/36843410 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f45a8028360 con 0x55a5dbde92f0 2024-09-21T04:10:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.373+0000 7f45bcfca740 10 client.26468 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T04:10:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.373+0000 7f45bcfca740 10 client.26468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:10:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.373+0000 7f45bcfca740 20 client.26468 populate_metadata read hostname 'smithi064' 2024-09-21T04:10:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.373+0000 7f45bcfca740 1 --2- 172.21.15.64:0/36843410 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a5dbdd0d80 0x55a5dc1d5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.373+0000 7f45bcfca740 4 client.26468 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:10:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.373+0000 7f45bcfca740 1 -- 172.21.15.64:0/36843410 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5dc12e640 con 0x55a5dbdd0d80 2024-09-21T04:10:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.373+0000 7f45aeffd640 1 --2- 172.21.15.64:0/36843410 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a5dbdd0d80 0x55a5dc1d5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.373+0000 7f45aeffd640 1 --2- 172.21.15.64:0/36843410 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a5dbdd0d80 0x55a5dc1d5180 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a24027b2bade84b7 server_cookie=8ed17682f833a8e8 in_seq=0 out_seq=0 2024-09-21T04:10:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.373+0000 7f458effd640 10 client.26468 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.374+0000 7f458effd640 1 -- 172.21.15.64:0/36843410 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a5dc12e640 con 0x55a5dbdd0d80 2024-09-21T04:10:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.374+0000 7f458effd640 10 client.26468 handle_command_reply: tid=0 2024-09-21T04:10:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.374+0000 7f45bcfca740 1 client.26468 shutdown 2024-09-21T04:10:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.374+0000 7f45bcfca740 1 -- 172.21.15.64:0/36843410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5dbde92f0 msgr2=0x55a5dc1c30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.374+0000 7f45bcfca740 1 --2- 172.21.15.64:0/36843410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5dbde92f0 0x55a5dc1c30f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f45a8005cc0 tx=0x7f45a801ba90 comp rx=0 tx=0).stop 2024-09-21T04:10:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.374+0000 7f45bcfca740 1 -- 172.21.15.64:0/36843410 shutdown_connections 2024-09-21T04:10:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.374+0000 7f45bcfca740 1 --2- 172.21.15.64:0/36843410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5dbe121b0 0x55a5dc1cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.374+0000 7f45bcfca740 1 --2- 172.21.15.64:0/36843410 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a5dbdd0d80 0x55a5dc1d5180 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.374+0000 7f45bcfca740 1 --2- 172.21.15.64:0/36843410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5dbdf14d0 0x55a5dc1c3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.374+0000 7f45bcfca740 1 --2- 172.21.15.64:0/36843410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5dbde92f0 0x55a5dc1c30f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.374+0000 7f458effd640 0 client.26468 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.374+0000 7f45bcfca740 1 -- 172.21.15.64:0/36843410 >> 172.21.15.64:0/36843410 conn(0x55a5dbe06300 msgr2=0x55a5dc1b89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.374+0000 7f45bcfca740 1 -- 172.21.15.64:0/36843410 shutdown_connections 2024-09-21T04:10:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.374+0000 7f45bcfca740 1 -- 172.21.15.64:0/36843410 wait complete. 2024-09-21T04:10:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.374+0000 7f45bcfca740 20 client.26468 trim_cache size 0 max 16384 2024-09-21T04:10:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.389+0000 7f45bcfca740 1 Processor -- start 2024-09-21T04:10:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.389+0000 7f45bcfca740 1 -- start start 2024-09-21T04:10:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.390+0000 7f45bcfca740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5dbde92f0 0x55a5dc1c35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:10.393 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.390+0000 7f45bcfca740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5dbdf14d0 0x55a5dc1c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.390+0000 7f45bcfca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5dbe121b0 0x55a5dc1cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.390+0000 7f45bcfca740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f45b01b8750 con 0x55a5dbdf14d0 2024-09-21T04:10:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.390+0000 7f45bcfca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f45b01b5ff0 con 0x55a5dbe121b0 2024-09-21T04:10:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.390+0000 7f45bcfca740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f45b01b3e40 con 0x55a5dbde92f0 2024-09-21T04:10:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.390+0000 7f45aeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5dbdf14d0 0x55a5dc1c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.390+0000 7f45aeffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5dbdf14d0 0x55a5dc1c4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50878/0 (socket says 172.21.15.64:50878) 2024-09-21T04:10:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.390+0000 7f45aeffd640 1 -- 172.21.15.64:0/2373697860 learned_addr learned my addr 172.21.15.64:0/2373697860 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.390+0000 7f45affff640 1 --2- 172.21.15.64:0/2373697860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5dbe121b0 0x55a5dc1cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.390+0000 7f45affff640 1 --2- 172.21.15.64:0/2373697860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5dbde92f0 0x55a5dc1c35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.390+0000 7f45aeffd640 1 -- 172.21.15.64:0/2373697860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5dbde92f0 msgr2=0x55a5dc1c35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.390+0000 7f45aeffd640 1 --2- 172.21.15.64:0/2373697860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5dbde92f0 0x55a5dc1c35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.390+0000 7f45aeffd640 1 -- 172.21.15.64:0/2373697860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5dbe121b0 msgr2=0x55a5dc1cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:10.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.390+0000 7f45aeffd640 1 --2- 172.21.15.64:0/2373697860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5dbe121b0 0x55a5dc1cf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.390+0000 7f45aeffd640 1 -- 172.21.15.64:0/2373697860 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5dc1ba7c0 con 0x55a5dbdf14d0 2024-09-21T04:10:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.390+0000 7f45affff640 1 --2- 172.21.15.64:0/2373697860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5dbe121b0 0x55a5dc1cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.390+0000 7f45aeffd640 1 --2- 172.21.15.64:0/2373697860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5dbdf14d0 0x55a5dc1c4d20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f45a4033000 tx=0x7f45a4031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4072bfb1d9448683 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.390+0000 7f45957fa640 1 -- 172.21.15.64:0/2373697860 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a4035040 con 0x55a5dbdf14d0 2024-09-21T04:10:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.391+0000 7f45957fa640 1 -- 172.21.15.64:0/2373697860 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45a4032890 con 0x55a5dbdf14d0 2024-09-21T04:10:10.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.391+0000 7f45957fa640 1 -- 172.21.15.64:0/2373697860 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a4005310 con 0x55a5dbdf14d0 2024-09-21T04:10:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.391+0000 7f45bcfca740 1 -- 172.21.15.64:0/2373697860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5dbdf14d0 msgr2=0x55a5dc1c4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.391+0000 7f45bcfca740 1 --2- 172.21.15.64:0/2373697860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5dbdf14d0 0x55a5dc1c4d20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f45a4033000 tx=0x7f45a4031400 comp rx=0 tx=0).stop 2024-09-21T04:10:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.392+0000 7f45bcfca740 1 -- 172.21.15.64:0/2373697860 shutdown_connections 2024-09-21T04:10:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.392+0000 7f45bcfca740 1 --2- 172.21.15.64:0/2373697860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5dbe121b0 0x55a5dc1cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.392+0000 7f45bcfca740 1 --2- 172.21.15.64:0/2373697860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5dbdf14d0 0x55a5dc1c4d20 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.392+0000 7f45bcfca740 1 --2- 172.21.15.64:0/2373697860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5dbde92f0 0x55a5dc1c35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.392+0000 7f45bcfca740 1 -- 172.21.15.64:0/2373697860 >> 172.21.15.64:0/2373697860 conn(0x55a5dbe06300 msgr2=0x55a5dc1c4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.392+0000 7f45bcfca740 1 -- 172.21.15.64:0/2373697860 shutdown_connections 2024-09-21T04:10:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.392+0000 7f45bcfca740 1 -- 172.21.15.64:0/2373697860 wait complete. 2024-09-21T04:10:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.392+0000 7f45bcfca740 1 -- start start 2024-09-21T04:10:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.392+0000 7f45bcfca740 1 Processor -- start 2024-09-21T04:10:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.393+0000 7f45bcfca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5dbde92f0 0x55a5dc1c6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.393+0000 7f45bcfca740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5dbdf14d0 0x55a5dc1c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.393+0000 7f45bcfca740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5dbe121b0 0x55a5dc1c7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.393+0000 7f45bcfca740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f45b01b44b0 con 0x55a5dbe121b0 2024-09-21T04:10:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.393+0000 7f45bcfca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f45b01b5920 con 0x55a5dbde92f0 2024-09-21T04:10:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.393+0000 7f45bcfca740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f45b01b4b80 con 0x55a5dbdf14d0 2024-09-21T04:10:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.393+0000 7f45bcfca740 10 client.0 _ll_register_callbacks cb 0x7f45a00eac10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:10:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.393+0000 7f45affff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5dbde92f0 0x55a5dc1c6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.393+0000 7f45affff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5dbde92f0 0x55a5dc1c6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:43838/0 (socket says 172.21.15.64:43838) 2024-09-21T04:10:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.393+0000 7f45affff640 1 -- 172.21.15.64:0/1275621091 learned_addr learned my addr 172.21.15.64:0/1275621091 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.393+0000 7f45affff640 1 --2- 172.21.15.64:0/1275621091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5dbe121b0 0x55a5dc1c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.393+0000 7f45aeffd640 1 --2- 172.21.15.64:0/1275621091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5dbdf14d0 0x55a5dc1c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.393+0000 7f45affff640 1 -- 172.21.15.64:0/1275621091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5dbdf14d0 msgr2=0x55a5dc1c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.393+0000 7f45affff640 1 --2- 172.21.15.64:0/1275621091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5dbdf14d0 0x55a5dc1c6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.393+0000 7f45affff640 1 -- 172.21.15.64:0/1275621091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5dbe121b0 msgr2=0x55a5dc1c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.393+0000 7f45affff640 1 --2- 172.21.15.64:0/1275621091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5dbe121b0 0x55a5dc1c7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.393+0000 7f45affff640 1 -- 172.21.15.64:0/1275621091 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5dc23dca0 con 0x55a5dbde92f0 2024-09-21T04:10:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.394+0000 7f45affff640 1 --2- 172.21.15.64:0/1275621091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5dbde92f0 0x55a5dc1c6570 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f45a8011010 tx=0x7f45a801ba90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33375a4ca135f772 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.394+0000 7f458effd640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:10:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.394+0000 7f458effd640 1 -- 172.21.15.64:0/1275621091 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a801f040 con 0x55a5dbde92f0 2024-09-21T04:10:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.394+0000 7f45bcfca740 1 -- 172.21.15.64:0/1275621091 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45a4032890 con 0x55a5dbde92f0 2024-09-21T04:10:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.394+0000 7f458effd640 1 -- 172.21.15.64:0/1275621091 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7f45a8028a80 con 0x55a5dbde92f0 2024-09-21T04:10:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.394+0000 7f458effd640 1 -- 172.21.15.64:0/1275621091 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f45a8028360 con 0x55a5dbde92f0 2024-09-21T04:10:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.394+0000 7f45bcfca740 10 client.26474 fetch_fsmap learned FSMap version 37 2024-09-21T04:10:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.394+0000 7f45bcfca740 1 -- 172.21.15.64:0/1275621091 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a5dc23ef30 con 0x55a5dbde92f0 2024-09-21T04:10:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.394+0000 7f458effd640 1 -- 172.21.15.64:0/1275621091 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45a4032890 con 0x55a5dbde92f0 2024-09-21T04:10:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.394+0000 7f458effd640 1 -- 172.21.15.64:0/1275621091 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f45a80052e0 con 0x55a5dbde92f0 2024-09-21T04:10:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.394+0000 7f45bcfca740 10 client.26474 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T04:10:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.394+0000 7f45bcfca740 10 client.26474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:10:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.394+0000 7f45bcfca740 20 client.26474 populate_metadata read hostname 'smithi064' 2024-09-21T04:10:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.394+0000 7f45bcfca740 1 --2- 172.21.15.64:0/1275621091 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a5dbda61c0 0x55a5dc241630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.394+0000 7f45bcfca740 4 client.26474 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:10:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.394+0000 7f45bcfca740 1 -- 172.21.15.64:0/1275621091 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a5dc12a0f0 con 0x55a5dbda61c0 2024-09-21T04:10:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.395+0000 7f45aeffd640 1 --2- 172.21.15.64:0/1275621091 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a5dbda61c0 0x55a5dc241630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.395+0000 7f45aeffd640 1 --2- 172.21.15.64:0/1275621091 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a5dbda61c0 0x55a5dc241630 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f362cb2fe8948e1 server_cookie=37ed95b33e48dd0e in_seq=0 out_seq=0 2024-09-21T04:10:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:10.395+0000 7f458effd640 10 client.26474 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:10.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:10 smithi119 ceph-mon[34811]: pgmap v505: 97 pgs: 97 active+clean; 753 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 613 op/s 2024-09-21T04:10:10.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:10 smithi064 ceph-mon[31180]: pgmap v505: 97 pgs: 97 active+clean; 753 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 613 op/s 2024-09-21T04:10:10.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:10 smithi131 ceph-mon[35193]: pgmap v505: 97 pgs: 97 active+clean; 753 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 613 op/s 2024-09-21T04:10:11.175 INFO:tasks.workunit.client.0.smithi064.stdout: 119 575 1 481 1 265 3 2024-09-21T04:10:12.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:12 smithi119 ceph-mon[34811]: pgmap v506: 97 pgs: 97 active+clean; 753 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 908 KiB/s wr, 528 op/s 2024-09-21T04:10:12.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:12 smithi064 ceph-mon[31180]: pgmap v506: 97 pgs: 97 active+clean; 753 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 908 KiB/s wr, 528 op/s 2024-09-21T04:10:12.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:12 smithi131 ceph-mon[35193]: pgmap v506: 97 pgs: 97 active+clean; 753 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 908 KiB/s wr, 528 op/s 2024-09-21T04:10:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:12.716+0000 7f458effd640 1 -- 172.21.15.64:0/1275621091 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a5dc12a0f0 con 0x55a5dbda61c0 2024-09-21T04:10:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:12.716+0000 7f458effd640 10 client.26474 handle_command_reply: tid=0 2024-09-21T04:10:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:12.716+0000 7f45bcfca740 1 client.26474 shutdown 2024-09-21T04:10:12.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:12.716+0000 7f45bcfca740 1 -- 172.21.15.64:0/1275621091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5dbde92f0 msgr2=0x55a5dc1c6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:12.716+0000 7f45bcfca740 1 --2- 172.21.15.64:0/1275621091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5dbde92f0 0x55a5dc1c6570 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f45a8011010 tx=0x7f45a801ba90 comp rx=0 tx=0).stop 2024-09-21T04:10:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:12.716+0000 7f45bcfca740 1 -- 172.21.15.64:0/1275621091 shutdown_connections 2024-09-21T04:10:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:12.716+0000 7f45bcfca740 1 --2- 172.21.15.64:0/1275621091 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a5dbda61c0 0x55a5dc241630 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:12.716+0000 7f45bcfca740 1 --2- 172.21.15.64:0/1275621091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5dbe121b0 0x55a5dc1c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:12.716+0000 7f45bcfca740 1 --2- 172.21.15.64:0/1275621091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a5dbdf14d0 0x55a5dc1c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:12.716+0000 7f45bcfca740 1 --2- 172.21.15.64:0/1275621091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a5dbde92f0 0x55a5dc1c6570 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:12.716+0000 7f458effd640 0 client.26474 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:12.716+0000 7f45bcfca740 1 -- 172.21.15.64:0/1275621091 >> 172.21.15.64:0/1275621091 conn(0x55a5dbe06300 msgr2=0x55a5dc1c25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:12.717+0000 7f45bcfca740 1 -- 172.21.15.64:0/1275621091 shutdown_connections 2024-09-21T04:10:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:12.717+0000 7f45bcfca740 1 -- 172.21.15.64:0/1275621091 wait complete. 2024-09-21T04:10:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:12.717+0000 7f45bcfca740 20 client.26474 trim_cache size 0 max 16384 2024-09-21T04:10:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:10:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:10:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:10:12.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:10:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:12.717+0000 7f45b605c640 1 -- 172.21.15.64:0/4132189727 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f45980b6700 msgr2=0x7f45980b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:12.717+0000 7f45b605c640 1 --2- 172.21.15.64:0/4132189727 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f45980b6700 0x7f45980b8c10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f45a00e49c0 tx=0x7f45a00ead40 comp rx=0 tx=0).stop 2024-09-21T04:10:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:12.717+0000 7f45b605c640 1 -- 172.21.15.64:0/4132189727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45b0077940 msgr2=0x7f45b01a0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:12.717+0000 7f45b605c640 1 --2- 172.21.15.64:0/4132189727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45b0077940 0x7f45b01a0aa0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f45a000d770 tx=0x7f45a000dc30 comp rx=0 tx=0).stop 2024-09-21T04:10:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:12.717+0000 7f45b605c640 1 -- 172.21.15.64:0/4132189727 shutdown_connections 2024-09-21T04:10:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:12.717+0000 7f45b605c640 1 --2- 172.21.15.64:0/4132189727 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f45980b6700 0x7f45980b8c10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:12.717+0000 7f45b605c640 1 --2- 172.21.15.64:0/4132189727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45b01146a0 0x7f45b01a62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:12.717+0000 7f45b605c640 1 --2- 172.21.15.64:0/4132189727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45b0078290 0x7f45b01a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:12.717+0000 7f45b605c640 1 --2- 172.21.15.64:0/4132189727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45b0077940 0x7f45b01a0aa0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:12.717+0000 7f45b605c640 1 -- 172.21.15.64:0/4132189727 >> 172.21.15.64:0/4132189727 conn(0x7f45b01016f0 msgr2=0x7f45b01167a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:12.718+0000 7f45b605c640 1 -- 172.21.15.64:0/4132189727 shutdown_connections 2024-09-21T04:10:12.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:12.718+0000 7f45b605c640 1 -- 172.21.15.64:0/4132189727 wait complete. 2024-09-21T04:10:14.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:14 smithi119 ceph-mon[34811]: pgmap v507: 97 pgs: 97 active+clean; 762 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 673 op/s 2024-09-21T04:10:14.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:14 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:10:14.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:14 smithi064 ceph-mon[31180]: pgmap v507: 97 pgs: 97 active+clean; 762 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 673 op/s 2024-09-21T04:10:14.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:14 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:10:14.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:14 smithi131 ceph-mon[35193]: pgmap v507: 97 pgs: 97 active+clean; 762 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 673 op/s 2024-09-21T04:10:14.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:14 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:10:16.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:16 smithi119 ceph-mon[34811]: pgmap v508: 97 pgs: 97 active+clean; 762 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 414 op/s 2024-09-21T04:10:16.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:16 smithi064 ceph-mon[31180]: pgmap v508: 97 pgs: 97 active+clean; 762 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 414 op/s 2024-09-21T04:10:16.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:16 smithi131 ceph-mon[35193]: pgmap v508: 97 pgs: 97 active+clean; 762 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 414 op/s 2024-09-21T04:10:16.576 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T04:10:16.844 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:10:17.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.471+0000 7feeea906640 1 -- 172.21.15.64:0/1270751046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feee4108d20 msgr2=0x7feee410f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:17.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.471+0000 7feeea906640 1 --2- 172.21.15.64:0/1270751046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feee4108d20 0x7feee410f550 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7feedc009ca0 tx=0x7feedc02a8c0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.471+0000 7feeea906640 1 -- 172.21.15.64:0/1270751046 shutdown_connections 2024-09-21T04:10:17.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.471+0000 7feeea906640 1 --2- 172.21.15.64:0/1270751046 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feee4108d20 0x7feee410f550 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.471+0000 7feeea906640 1 --2- 172.21.15.64:0/1270751046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feee41083d0 0x7feee41087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.471+0000 7feeea906640 1 --2- 172.21.15.64:0/1270751046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feee4102ae0 0x7feee4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.471+0000 7feeea906640 1 -- 172.21.15.64:0/1270751046 >> 172.21.15.64:0/1270751046 conn(0x7feee4079760 msgr2=0x7feee4079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:17.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.471+0000 7feeea906640 1 -- 172.21.15.64:0/1270751046 shutdown_connections 2024-09-21T04:10:17.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.471+0000 7feeea906640 1 -- 172.21.15.64:0/1270751046 wait complete. 2024-09-21T04:10:17.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.471+0000 7feeea906640 1 Processor -- start 2024-09-21T04:10:17.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.471+0000 7feeea906640 1 -- start start 2024-09-21T04:10:17.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.472+0000 7feeea906640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feee4102ae0 0x7feee4199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:17.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.472+0000 7feeea906640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feee41083d0 0x7feee41998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:17.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.472+0000 7feeea906640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feee4108d20 0x7feee419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:17.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.472+0000 7feeea906640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7feee41129e0 con 0x7feee4102ae0 2024-09-21T04:10:17.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.472+0000 7feeea906640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feee4112860 con 0x7feee41083d0 2024-09-21T04:10:17.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.472+0000 7feeea906640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feee4112b60 con 0x7feee4108d20 2024-09-21T04:10:17.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.472+0000 7feee3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feee4102ae0 0x7feee4199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:17.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.472+0000 7feee3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feee4102ae0 0x7feee4199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53682/0 (socket says 172.21.15.64:53682) 2024-09-21T04:10:17.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.472+0000 7feee3fff640 1 -- 172.21.15.64:0/2070240764 learned_addr learned my addr 172.21.15.64:0/2070240764 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:17.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.472+0000 7feee37fe640 1 --2- 172.21.15.64:0/2070240764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feee41083d0 0x7feee41998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:17.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.472+0000 7feee8ea3640 1 --2- 172.21.15.64:0/2070240764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feee4108d20 0x7feee419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:17.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.472+0000 7feee37fe640 1 -- 172.21.15.64:0/2070240764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feee4108d20 msgr2=0x7feee419eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:17.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.472+0000 7feee37fe640 1 --2- 172.21.15.64:0/2070240764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feee4108d20 0x7feee419eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.472+0000 7feee37fe640 1 -- 172.21.15.64:0/2070240764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feee4102ae0 msgr2=0x7feee4199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:17.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.472+0000 7feee37fe640 1 --2- 172.21.15.64:0/2070240764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feee4102ae0 0x7feee4199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.472+0000 7feee37fe640 1 -- 172.21.15.64:0/2070240764 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee419f2a0 con 0x7feee41083d0 2024-09-21T04:10:17.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.472+0000 7feee8ea3640 1 --2- 172.21.15.64:0/2070240764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feee4108d20 0x7feee419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:10:17.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.472+0000 7feee3fff640 1 --2- 172.21.15.64:0/2070240764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feee4102ae0 0x7feee4199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:17.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.473+0000 7feee37fe640 1 --2- 172.21.15.64:0/2070240764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feee41083d0 0x7feee41998d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feed800e9c0 tx=0x7feed800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:17.478 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.473+0000 7feee17fa640 1 -- 172.21.15.64:0/2070240764 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed800cd80 con 0x7feee41083d0 2024-09-21T04:10:17.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.473+0000 7feeea906640 1 -- 172.21.15.64:0/2070240764 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee419f590 con 0x7feee41083d0 2024-09-21T04:10:17.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.473+0000 7feee17fa640 1 -- 172.21.15.64:0/2070240764 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed80035f0 con 0x7feee41083d0 2024-09-21T04:10:17.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.473+0000 7feee17fa640 1 -- 172.21.15.64:0/2070240764 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8010690 con 0x7feee41083d0 2024-09-21T04:10:17.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.473+0000 7feeea906640 1 -- 172.21.15.64:0/2070240764 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee406bb40 con 0x7feee41083d0 2024-09-21T04:10:17.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.476+0000 7feef0e7a740 1 Processor -- start 2024-09-21T04:10:17.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.476+0000 7feef0e7a740 1 -- start start 2024-09-21T04:10:17.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.476+0000 7feef0e7a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d65eca070 0x559d66271650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:17.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.476+0000 7feef0e7a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d65ea11b0 0x559d66271b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:17.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.476+0000 7feee17fa640 1 -- 172.21.15.64:0/2070240764 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7feed80037a0 con 0x7feee41083d0 2024-09-21T04:10:17.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.476+0000 7feef0e7a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d65ea9390 0x559d662720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:17.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.476+0000 7feef0e7a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559d661e7e50 con 0x559d65ea11b0 2024-09-21T04:10:17.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.476+0000 7feef0e7a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559d661d7280 con 0x559d65eca070 2024-09-21T04:10:17.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.476+0000 7feef0e7a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559d661db090 con 0x559d65ea9390 2024-09-21T04:10:17.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.476+0000 7feee3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d65ea11b0 0x559d66271b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:17.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.476+0000 7feee3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d65ea11b0 0x559d66271b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53686/0 (socket says 172.21.15.64:53686) 2024-09-21T04:10:17.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.476+0000 7feee3fff640 1 -- 172.21.15.64:0/873705944 learned_addr learned my addr 172.21.15.64:0/873705944 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:17.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.476+0000 7feee37fe640 1 --2- 172.21.15.64:0/873705944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d65ea9390 0x559d662720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:17.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.477+0000 7feee8ea3640 1 --2- 172.21.15.64:0/873705944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d65eca070 0x559d66271650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:17.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.477+0000 7feee37fe640 1 -- 172.21.15.64:0/873705944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d65eca070 msgr2=0x559d66271650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:17.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.477+0000 7feee37fe640 1 --2- 172.21.15.64:0/873705944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d65eca070 0x559d66271650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.477+0000 7feee37fe640 1 -- 172.21.15.64:0/873705944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d65ea11b0 msgr2=0x559d66271b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:17.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.477+0000 7feee37fe640 1 --2- 172.21.15.64:0/873705944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d65ea11b0 0x559d66271b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.477+0000 7feee37fe640 1 -- 172.21.15.64:0/873705944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d65f35650 con 0x559d65ea9390 2024-09-21T04:10:17.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.477+0000 7feee3fff640 1 --2- 172.21.15.64:0/873705944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d65ea11b0 0x559d66271b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:17.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.477+0000 7feee8ea3640 1 --2- 172.21.15.64:0/873705944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d65eca070 0x559d66271650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:10:17.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.477+0000 7feee37fe640 1 --2- 172.21.15.64:0/873705944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d65ea9390 0x559d662720d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7feed80e5240 tx=0x7feed80f0270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbfd899db4b8c1c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:17.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.477+0000 7feeca7fc640 1 -- 172.21.15.64:0/873705944 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed80f7020 con 0x559d65ea9390 2024-09-21T04:10:17.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.478+0000 7feeca7fc640 1 -- 172.21.15.64:0/873705944 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed80f0c80 con 0x559d65ea9390 2024-09-21T04:10:17.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.478+0000 7feeca7fc640 1 -- 172.21.15.64:0/873705944 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed80f25b0 con 0x559d65ea9390 2024-09-21T04:10:17.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.478+0000 7feee17fa640 1 --2- 172.21.15.64:0/2070240764 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7feecc0b6680 0x7feecc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:17.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.478+0000 7feee8ea3640 1 --2- 172.21.15.64:0/2070240764 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7feecc0b6680 0x7feecc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:17.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.478+0000 7feee17fa640 1 -- 172.21.15.64:0/2070240764 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7feed80e7b80 con 0x7feee41083d0 2024-09-21T04:10:17.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.479+0000 7feef0e7a740 1 -- 172.21.15.64:0/873705944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d65ea9390 msgr2=0x559d662720d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:17.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.479+0000 7feef0e7a740 1 --2- 172.21.15.64:0/873705944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d65ea9390 0x559d662720d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7feed80e5240 tx=0x7feed80f0270 comp rx=0 tx=0).stop 2024-09-21T04:10:17.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.479+0000 7feef0e7a740 1 -- 172.21.15.64:0/873705944 shutdown_connections 2024-09-21T04:10:17.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.479+0000 7feef0e7a740 1 --2- 172.21.15.64:0/873705944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d65ea9390 0x559d662720d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.479+0000 7feef0e7a740 1 --2- 172.21.15.64:0/873705944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d65ea11b0 0x559d66271b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.479+0000 7feef0e7a740 1 --2- 172.21.15.64:0/873705944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d65eca070 0x559d66271650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.479+0000 7feef0e7a740 1 -- 172.21.15.64:0/873705944 >> 172.21.15.64:0/873705944 conn(0x559d65ebe460 msgr2=0x559d6626f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:17.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.479+0000 7feef0e7a740 1 -- 172.21.15.64:0/873705944 shutdown_connections 2024-09-21T04:10:17.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.479+0000 7feee8ea3640 1 --2- 172.21.15.64:0/2070240764 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7feecc0b6680 0x7feecc0b8b90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7feedc005e30 tx=0x7feedc02f3c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:17.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.479+0000 7feef0e7a740 1 -- 172.21.15.64:0/873705944 wait complete. 2024-09-21T04:10:17.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.479+0000 7feef0e7a740 1 -- start start 2024-09-21T04:10:17.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.480+0000 7feef0e7a740 1 Processor -- start 2024-09-21T04:10:17.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.480+0000 7feef0e7a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d65ea11b0 0x559d6627ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:17.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.480+0000 7feef0e7a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d65ea9390 0x559d6627d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:17.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.480+0000 7feef0e7a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d65eca070 0x559d6627d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:17.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.480+0000 7feef0e7a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559d66038700 con 0x559d65ea9390 2024-09-21T04:10:17.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.480+0000 7feef0e7a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559d66172110 con 0x559d65ea11b0 2024-09-21T04:10:17.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.480+0000 7feef0e7a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559d6627fe30 con 0x559d65eca070 2024-09-21T04:10:17.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.480+0000 7feef0e7a740 10 client.0 _ll_register_callbacks cb 0x559d6618b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:10:17.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.480+0000 7feee37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d65ea9390 0x559d6627d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:17.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.480+0000 7feee3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d65ea11b0 0x559d6627ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:17.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.480+0000 7feee37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d65ea9390 0x559d6627d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53692/0 (socket says 172.21.15.64:53692) 2024-09-21T04:10:17.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.480+0000 7feee37fe640 1 -- 172.21.15.64:0/2165043875 learned_addr learned my addr 172.21.15.64:0/2165043875 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:17.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.480+0000 7feee8ea3640 1 --2- 172.21.15.64:0/2165043875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d65eca070 0x559d6627d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:17.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.481+0000 7feee37fe640 1 -- 172.21.15.64:0/2165043875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d65eca070 msgr2=0x559d6627d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:17.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.481+0000 7feee37fe640 1 --2- 172.21.15.64:0/2165043875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d65eca070 0x559d6627d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.481+0000 7feee37fe640 1 -- 172.21.15.64:0/2165043875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d65ea11b0 msgr2=0x559d6627ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:17.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.481+0000 7feee37fe640 1 --2- 172.21.15.64:0/2165043875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d65ea11b0 0x559d6627ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.481+0000 7feee37fe640 1 -- 172.21.15.64:0/2165043875 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d6619fc40 con 0x559d65ea9390 2024-09-21T04:10:17.487 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.481+0000 7feee3fff640 1 --2- 172.21.15.64:0/2165043875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d65ea11b0 0x559d6627ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:17.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.481+0000 7feee37fe640 1 --2- 172.21.15.64:0/2165043875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d65ea9390 0x559d6627d200 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7feed80ad8d0 tx=0x7feed80eb470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c14a9b4b0dd9c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:17.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.481+0000 7feec37fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:10:17.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.481+0000 7feec37fe640 1 -- 172.21.15.64:0/2165043875 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed80f7030 con 0x559d65ea9390 2024-09-21T04:10:17.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.481+0000 7feec37fe640 1 -- 172.21.15.64:0/2165043875 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7feed80a9bb0 con 0x559d65ea9390 2024-09-21T04:10:17.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.481+0000 7feef0e7a740 1 -- 172.21.15.64:0/2165043875 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feed80f0c80 con 0x559d65ea9390 2024-09-21T04:10:17.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.481+0000 7feec37fe640 1 -- 172.21.15.64:0/2165043875 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7feed80a97e0 con 0x559d65ea9390 2024-09-21T04:10:17.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.481+0000 7feef0e7a740 10 client.16803 fetch_fsmap learned FSMap version 37 2024-09-21T04:10:17.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.481+0000 7feef0e7a740 1 -- 172.21.15.64:0/2165043875 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559d6628fe30 con 0x559d65ea9390 2024-09-21T04:10:17.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.482+0000 7feec37fe640 1 -- 172.21.15.64:0/2165043875 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed80f0c80 con 0x559d65ea9390 2024-09-21T04:10:17.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.482+0000 7feec37fe640 1 -- 172.21.15.64:0/2165043875 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7feed80b2020 con 0x559d65ea9390 2024-09-21T04:10:17.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.482+0000 7feef0e7a740 10 client.16803 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T04:10:17.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.482+0000 7feef0e7a740 10 client.16803 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:10:17.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.482+0000 7feef0e7a740 20 client.16803 populate_metadata read hostname 'smithi064' 2024-09-21T04:10:17.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.482+0000 7feef0e7a740 1 --2- 172.21.15.64:0/2165043875 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x559d65e889f0 0x559d66293190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:17.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.482+0000 7feef0e7a740 4 client.16803 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:10:17.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.482+0000 7feef0e7a740 1 -- 172.21.15.64:0/2165043875 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d661e9da0 con 0x559d65e889f0 2024-09-21T04:10:17.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.482+0000 7feee3fff640 1 --2- 172.21.15.64:0/2165043875 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x559d65e889f0 0x559d66293190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:17.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.483+0000 7feec37fe640 10 client.16803 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:17.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.483+0000 7feee3fff640 1 --2- 172.21.15.64:0/2165043875 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x559d65e889f0 0x559d66293190 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b57dddcd5ee50d42 server_cookie=e7431d73317c1b98 in_seq=0 out_seq=0 2024-09-21T04:10:17.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.484+0000 7feec37fe640 1 -- 172.21.15.64:0/2165043875 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559d661e9da0 con 0x559d65e889f0 2024-09-21T04:10:17.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.484+0000 7feec37fe640 10 client.16803 handle_command_reply: tid=0 2024-09-21T04:10:17.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.484+0000 7feef0e7a740 1 client.16803 shutdown 2024-09-21T04:10:17.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.484+0000 7feef0e7a740 1 -- 172.21.15.64:0/2165043875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d65ea9390 msgr2=0x559d6627d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:17.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.484+0000 7feef0e7a740 1 --2- 172.21.15.64:0/2165043875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d65ea9390 0x559d6627d200 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7feed80ad8d0 tx=0x7feed80eb470 comp rx=0 tx=0).stop 2024-09-21T04:10:17.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.484+0000 7feef0e7a740 1 -- 172.21.15.64:0/2165043875 shutdown_connections 2024-09-21T04:10:17.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.484+0000 7feef0e7a740 1 --2- 172.21.15.64:0/2165043875 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d65eca070 0x559d6627d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.484+0000 7feef0e7a740 1 --2- 172.21.15.64:0/2165043875 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x559d65e889f0 0x559d66293190 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.484+0000 7feef0e7a740 1 --2- 172.21.15.64:0/2165043875 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d65ea9390 0x559d6627d200 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.484+0000 7feef0e7a740 1 --2- 172.21.15.64:0/2165043875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d65ea11b0 0x559d6627ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.484+0000 7feef0e7a740 1 -- 172.21.15.64:0/2165043875 >> 172.21.15.64:0/2165043875 conn(0x559d65ebe460 msgr2=0x559d66274830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:17.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.484+0000 7feec37fe640 0 client.16803 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:17.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.484+0000 7feef0e7a740 1 -- 172.21.15.64:0/2165043875 shutdown_connections 2024-09-21T04:10:17.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.484+0000 7feef0e7a740 1 -- 172.21.15.64:0/2165043875 wait complete. 2024-09-21T04:10:17.491 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.484+0000 7feef0e7a740 20 client.16803 trim_cache size 0 max 16384 2024-09-21T04:10:17.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.503+0000 7feef0e7a740 1 Processor -- start 2024-09-21T04:10:17.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.503+0000 7feef0e7a740 1 -- start start 2024-09-21T04:10:17.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.503+0000 7feef0e7a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d65ea11b0 0x559d6627eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:17.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.503+0000 7feef0e7a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d65ea9390 0x559d66272680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:17.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.503+0000 7feef0e7a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d65eca070 0x559d66288790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:17.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.503+0000 7feef0e7a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7feee41aff30 con 0x559d65eca070 2024-09-21T04:10:17.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.503+0000 7feef0e7a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feee41ad7d0 con 0x559d65ea11b0 2024-09-21T04:10:17.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.503+0000 7feef0e7a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feee41ab620 con 0x559d65ea9390 2024-09-21T04:10:17.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.503+0000 7feee3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d65ea11b0 0x559d6627eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:17.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.503+0000 7feee8ea3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d65eca070 0x559d66288790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:17.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.503+0000 7feee3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d65ea11b0 0x559d6627eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:44260/0 (socket says 172.21.15.64:44260) 2024-09-21T04:10:17.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.503+0000 7feee3fff640 1 -- 172.21.15.64:0/3934652715 learned_addr learned my addr 172.21.15.64:0/3934652715 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:17.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.504+0000 7feee37fe640 1 --2- 172.21.15.64:0/3934652715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d65ea9390 0x559d66272680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:17.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.504+0000 7feee3fff640 1 -- 172.21.15.64:0/3934652715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d65ea9390 msgr2=0x559d66272680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:17.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.504+0000 7feee3fff640 1 --2- 172.21.15.64:0/3934652715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d65ea9390 0x559d66272680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.504+0000 7feee3fff640 1 -- 172.21.15.64:0/3934652715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d65eca070 msgr2=0x559d66288790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:17.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.504+0000 7feee3fff640 1 --2- 172.21.15.64:0/3934652715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d65eca070 0x559d66288790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.504+0000 7feee3fff640 1 -- 172.21.15.64:0/3934652715 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d66277770 con 0x559d65ea11b0 2024-09-21T04:10:17.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.504+0000 7feee8ea3640 1 --2- 172.21.15.64:0/3934652715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d65eca070 0x559d66288790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:17.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.504+0000 7feee3fff640 1 --2- 172.21.15.64:0/3934652715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d65ea11b0 0x559d6627eba0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7feed4004700 tx=0x7feed4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34ac919988645c33 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:17.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.504+0000 7feec9ffb640 1 -- 172.21.15.64:0/3934652715 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed4028030 con 0x559d65ea11b0 2024-09-21T04:10:17.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.504+0000 7feec9ffb640 1 -- 172.21.15.64:0/3934652715 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed40054d0 con 0x559d65ea11b0 2024-09-21T04:10:17.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.504+0000 7feec9ffb640 1 -- 172.21.15.64:0/3934652715 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed40246c0 con 0x559d65ea11b0 2024-09-21T04:10:17.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.505+0000 7feef0e7a740 1 -- 172.21.15.64:0/3934652715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d65ea11b0 msgr2=0x559d6627eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:17.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.505+0000 7feef0e7a740 1 --2- 172.21.15.64:0/3934652715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d65ea11b0 0x559d6627eba0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7feed4004700 tx=0x7feed4005f90 comp rx=0 tx=0).stop 2024-09-21T04:10:17.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.505+0000 7feef0e7a740 1 -- 172.21.15.64:0/3934652715 shutdown_connections 2024-09-21T04:10:17.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.505+0000 7feef0e7a740 1 --2- 172.21.15.64:0/3934652715 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d65eca070 0x559d66288790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.505+0000 7feef0e7a740 1 --2- 172.21.15.64:0/3934652715 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d65ea9390 0x559d66272680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.505+0000 7feef0e7a740 1 --2- 172.21.15.64:0/3934652715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d65ea11b0 0x559d6627eba0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.505+0000 7feef0e7a740 1 -- 172.21.15.64:0/3934652715 >> 172.21.15.64:0/3934652715 conn(0x559d65ebe460 msgr2=0x559d66288080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:17.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.505+0000 7feef0e7a740 1 -- 172.21.15.64:0/3934652715 shutdown_connections 2024-09-21T04:10:17.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.505+0000 7feef0e7a740 1 -- 172.21.15.64:0/3934652715 wait complete. 2024-09-21T04:10:17.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.506+0000 7feef0e7a740 1 -- start start 2024-09-21T04:10:17.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.506+0000 7feef0e7a740 1 Processor -- start 2024-09-21T04:10:17.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.506+0000 7feef0e7a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d65ea11b0 0x559d66296840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:17.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.506+0000 7feef0e7a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d65ea9390 0x559d66293d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:17.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.506+0000 7feef0e7a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d65eca070 0x559d662942a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:17.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.506+0000 7feef0e7a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7feee41abc90 con 0x559d65ea9390 2024-09-21T04:10:17.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.506+0000 7feef0e7a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feee41ad100 con 0x559d65eca070 2024-09-21T04:10:17.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.506+0000 7feef0e7a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feee41ac360 con 0x559d65ea11b0 2024-09-21T04:10:17.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.506+0000 7feef0e7a740 10 client.0 _ll_register_callbacks cb 0x7feed4009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:10:17.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.507+0000 7feee37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d65ea9390 0x559d66293d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:17.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.507+0000 7feee37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d65ea9390 0x559d66293d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53706/0 (socket says 172.21.15.64:53706) 2024-09-21T04:10:17.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.507+0000 7feee37fe640 1 -- 172.21.15.64:0/903480650 learned_addr learned my addr 172.21.15.64:0/903480650 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:17.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.507+0000 7feee3fff640 1 --2- 172.21.15.64:0/903480650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d65ea11b0 0x559d66296840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:17.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.507+0000 7feee8ea3640 1 --2- 172.21.15.64:0/903480650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d65eca070 0x559d662942a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:17.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.507+0000 7feee37fe640 1 -- 172.21.15.64:0/903480650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d65ea11b0 msgr2=0x559d66296840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:17.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.507+0000 7feee37fe640 1 --2- 172.21.15.64:0/903480650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d65ea11b0 0x559d66296840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.507+0000 7feee37fe640 1 -- 172.21.15.64:0/903480650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d65eca070 msgr2=0x559d662942a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:17.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.507+0000 7feee37fe640 1 --2- 172.21.15.64:0/903480650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d65eca070 0x559d662942a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.507+0000 7feee37fe640 1 -- 172.21.15.64:0/903480650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d662948e0 con 0x559d65ea9390 2024-09-21T04:10:17.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.507+0000 7feee3fff640 1 --2- 172.21.15.64:0/903480650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d65ea11b0 0x559d66296840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:17.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.507+0000 7feee37fe640 1 --2- 172.21.15.64:0/903480650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d65ea9390 0x559d66293d60 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7feed80b25e0 tx=0x7feed80f3c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a327e546960feb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:17.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.507+0000 7feec37fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:10:17.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.507+0000 7feec37fe640 1 -- 172.21.15.64:0/903480650 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed80f7070 con 0x559d65ea9390 2024-09-21T04:10:17.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.507+0000 7feef0e7a740 1 -- 172.21.15.64:0/903480650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feed40054d0 con 0x559d65ea9390 2024-09-21T04:10:17.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.507+0000 7feec37fe640 1 -- 172.21.15.64:0/903480650 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8571+0+0 (secure 0 0 0) 0x7feed80a9bb0 con 0x559d65ea9390 2024-09-21T04:10:17.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.508+0000 7feec37fe640 1 -- 172.21.15.64:0/903480650 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7feed80a90b0 con 0x559d65ea9390 2024-09-21T04:10:17.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.508+0000 7feef0e7a740 10 client.16809 fetch_fsmap learned FSMap version 37 2024-09-21T04:10:17.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.508+0000 7feef0e7a740 1 -- 172.21.15.64:0/903480650 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559d662948e0 con 0x559d65ea9390 2024-09-21T04:10:17.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.508+0000 7feec37fe640 1 -- 172.21.15.64:0/903480650 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed40054d0 con 0x559d65ea9390 2024-09-21T04:10:17.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.508+0000 7feec37fe640 1 -- 172.21.15.64:0/903480650 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7feed80e54c0 con 0x559d65ea9390 2024-09-21T04:10:17.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.508+0000 7feef0e7a740 10 client.16809 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T04:10:17.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.508+0000 7feef0e7a740 10 client.16809 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:10:17.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.508+0000 7feef0e7a740 20 client.16809 populate_metadata read hostname 'smithi064' 2024-09-21T04:10:17.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.508+0000 7feef0e7a740 1 --2- 172.21.15.64:0/903480650 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x559d65e5dda0 0x559d662ffb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:17.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.508+0000 7feef0e7a740 4 client.16809 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T04:10:17.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.508+0000 7feef0e7a740 1 -- 172.21.15.64:0/903480650 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559d661e5850 con 0x559d65e5dda0 2024-09-21T04:10:17.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.508+0000 7feee3fff640 1 --2- 172.21.15.64:0/903480650 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x559d65e5dda0 0x559d662ffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:17.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.509+0000 7feec37fe640 10 client.16809 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:17.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.509+0000 7feee3fff640 1 --2- 172.21.15.64:0/903480650 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x559d65e5dda0 0x559d662ffb40 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5f6e3a76fdf4cc2 server_cookie=8b6c15939c4b11b1 in_seq=0 out_seq=0 2024-09-21T04:10:17.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.509+0000 7feec37fe640 1 -- 172.21.15.64:0/903480650 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2842 (crc 0 0 0) 0x559d661e5850 con 0x559d65e5dda0 2024-09-21T04:10:17.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.509+0000 7feec37fe640 10 client.16809 handle_command_reply: tid=0 2024-09-21T04:10:17.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.509+0000 7feef0e7a740 1 client.16809 shutdown 2024-09-21T04:10:17.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.509+0000 7feef0e7a740 1 -- 172.21.15.64:0/903480650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d65ea9390 msgr2=0x559d66293d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:17.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.509+0000 7feef0e7a740 1 --2- 172.21.15.64:0/903480650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d65ea9390 0x559d66293d60 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7feed80b25e0 tx=0x7feed80f3c40 comp rx=0 tx=0).stop 2024-09-21T04:10:17.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.509+0000 7feef0e7a740 1 -- 172.21.15.64:0/903480650 shutdown_connections 2024-09-21T04:10:17.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.509+0000 7feef0e7a740 1 --2- 172.21.15.64:0/903480650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559d65eca070 0x559d662942a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.509+0000 7feef0e7a740 1 --2- 172.21.15.64:0/903480650 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x559d65e5dda0 0x559d662ffb40 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.509+0000 7feef0e7a740 1 --2- 172.21.15.64:0/903480650 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559d65ea9390 0x559d66293d60 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.509+0000 7feec37fe640 0 client.16809 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:17.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.509+0000 7feef0e7a740 1 --2- 172.21.15.64:0/903480650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559d65ea11b0 0x559d66296840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.509+0000 7feef0e7a740 1 -- 172.21.15.64:0/903480650 >> 172.21.15.64:0/903480650 conn(0x559d65ebe460 msgr2=0x559d6627eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:17.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.509+0000 7feef0e7a740 1 -- 172.21.15.64:0/903480650 shutdown_connections 2024-09-21T04:10:17.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.509+0000 7feef0e7a740 1 -- 172.21.15.64:0/903480650 wait complete. 2024-09-21T04:10:17.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.510+0000 7feef0e7a740 20 client.16809 trim_cache size 0 max 16384 2024-09-21T04:10:17.517 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:10:17.517 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T04:10:17.517 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-09-21T04:10:17.517 INFO:teuthology.orchestra.run.smithi064.stdout: "2c0523fb-aae4-43fa-9568-136a9b640051": { 2024-09-21T04:10:17.517 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:10:17.517 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "2c0523fb-aae4-43fa-9568-136a9b640051", 2024-09-21T04:10:17.517 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-09-21T04:10:17.517 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:10:17.517 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:10:17.517 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_stats": { 2024-09-21T04:10:17.518 INFO:teuthology.orchestra.run.smithi064.stdout: "2c0523fb-aae4-43fa-9568-136a9b640051": { 2024-09-21T04:10:17.518 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:09:39.808606+0000", 2024-09-21T04:10:17.518 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:10:17.518 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:10:17.518 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:10:17.518 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:10:17.518 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:10:17.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:10:17.518 INFO:teuthology.orchestra.run.smithi064.stdout: "bed40d88-2102-43a6-ac2d-b89d5a2264d0": { 2024-09-21T04:10:17.518 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:11.363340+0000", 2024-09-21T04:10:17.518 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:10:17.518 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:10:17.519 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:10:17.519 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:10:17.519 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:10:17.519 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:10:17.519 INFO:teuthology.orchestra.run.smithi064.stdout: "e9242f88-6599-40eb-adf1-bdb3e946a1e0": { 2024-09-21T04:10:17.519 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:05:39.744646+0000", 2024-09-21T04:10:17.519 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:10:17.519 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:10:17.519 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:10:17.519 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:10:17.519 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:10:17.519 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:10:17.519 INFO:teuthology.orchestra.run.smithi064.stdout: "6bde0fe5-f775-44b1-ad0c-f9e596574dc0": { 2024-09-21T04:10:17.519 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:06:23.835536+0000", 2024-09-21T04:10:17.519 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:10:17.520 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:10:17.520 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:10:17.520 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:10:17.520 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:10:17.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:10:17.520 INFO:teuthology.orchestra.run.smithi064.stdout: "b52b950c-0f97-49ec-a37c-1a8e04348ce8": { 2024-09-21T04:10:17.520 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:54.003800+0000", 2024-09-21T04:10:17.520 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:10:17.520 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:10:17.520 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:10:17.520 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:10:17.520 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:10:17.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:10:17.520 INFO:teuthology.orchestra.run.smithi064.stdout: "e2c2eed8-9292-4bf6-8a39-13ad27da5acb": { 2024-09-21T04:10:17.521 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:08:47.067932+0000", 2024-09-21T04:10:17.521 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:10:17.521 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:10:17.521 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:10:17.521 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:10:17.521 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:10:17.521 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:10:17.521 INFO:teuthology.orchestra.run.smithi064.stdout: "75db1bc5-612e-45aa-bad2-cc90feeaa840": { 2024-09-21T04:10:17.521 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:59.123138+0000", 2024-09-21T04:10:17.521 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:10:17.521 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:10:17.521 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:10:17.521 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:10:17.521 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:10:17.521 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:10:17.522 INFO:teuthology.orchestra.run.smithi064.stdout: "0ab717a7-36c6-4dca-94e7-af0bdf2aa112": { 2024-09-21T04:10:17.522 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:42.691086+0000", 2024-09-21T04:10:17.522 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:10:17.522 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:10:17.522 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:10:17.522 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:10:17.522 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:10:17.522 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:10:17.522 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:10:17.522 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:03:59.570174+0000", 2024-09-21T04:10:17.522 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:10:17.522 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:10:17.522 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:10:17.522 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:10:17.523 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:10:17.523 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:10:17.523 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:10:17.523 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:10:17.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.510+0000 7feeea906640 1 -- 172.21.15.64:0/2070240764 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7feecc0b6680 msgr2=0x7feecc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:17.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.510+0000 7feeea906640 1 --2- 172.21.15.64:0/2070240764 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7feecc0b6680 0x7feecc0b8b90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7feedc005e30 tx=0x7feedc02f3c0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.510+0000 7feeea906640 1 -- 172.21.15.64:0/2070240764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feee41083d0 msgr2=0x7feee41998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:17.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.510+0000 7feeea906640 1 --2- 172.21.15.64:0/2070240764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feee41083d0 0x7feee41998d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feed800e9c0 tx=0x7feed800ee80 comp rx=0 tx=0).stop 2024-09-21T04:10:17.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.510+0000 7feeea906640 1 -- 172.21.15.64:0/2070240764 shutdown_connections 2024-09-21T04:10:17.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.510+0000 7feeea906640 1 --2- 172.21.15.64:0/2070240764 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7feecc0b6680 0x7feecc0b8b90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.510+0000 7feeea906640 1 --2- 172.21.15.64:0/2070240764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feee4108d20 0x7feee419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.510+0000 7feeea906640 1 --2- 172.21.15.64:0/2070240764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feee41083d0 0x7feee41998d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.510+0000 7feeea906640 1 --2- 172.21.15.64:0/2070240764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feee4102ae0 0x7feee4199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:17.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.510+0000 7feeea906640 1 -- 172.21.15.64:0/2070240764 >> 172.21.15.64:0/2070240764 conn(0x7feee4079760 msgr2=0x7feee41043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:17.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.510+0000 7feeea906640 1 -- 172.21.15.64:0/2070240764 shutdown_connections 2024-09-21T04:10:17.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:17.510+0000 7feeea906640 1 -- 172.21.15.64:0/2070240764 wait complete. 2024-09-21T04:10:18.362 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:18 smithi064 ceph-mon[31180]: pgmap v509: 97 pgs: 97 active+clean; 765 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 534 op/s 2024-09-21T04:10:18.365 INFO:tasks.cephfs.filesystem:scrub status for tag:2c0523fb-aae4-43fa-9568-136a9b640051 - {'path': '/', 'tag': '2c0523fb-aae4-43fa-9568-136a9b640051', 'options': 'recursive,force'} 2024-09-21T04:10:18.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:18 smithi119 ceph-mon[34811]: pgmap v509: 97 pgs: 97 active+clean; 765 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 534 op/s 2024-09-21T04:10:18.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:18 smithi131 ceph-mon[35193]: pgmap v509: 97 pgs: 97 active+clean; 765 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 534 op/s 2024-09-21T04:10:20.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:20 smithi119 ceph-mon[34811]: pgmap v510: 97 pgs: 97 active+clean; 766 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 386 op/s 2024-09-21T04:10:20.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:20 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:10:20.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:20 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:10:20.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:20 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:10:20.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:20 smithi064 ceph-mon[31180]: pgmap v510: 97 pgs: 97 active+clean; 766 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 386 op/s 2024-09-21T04:10:20.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:20 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:10:20.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:20 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:10:20.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:20 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:10:20.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:20 smithi131 ceph-mon[35193]: pgmap v510: 97 pgs: 97 active+clean; 766 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 386 op/s 2024-09-21T04:10:20.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:20 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:10:20.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:20 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:10:20.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:20 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:10:21.175 INFO:tasks.workunit.client.0.smithi064.stdout: 120 9446 99 5351 165 5076 348 2024-09-21T04:10:22.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:22 smithi119 ceph-mon[34811]: pgmap v511: 97 pgs: 97 active+clean; 772 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 350 op/s 2024-09-21T04:10:22.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:22 smithi064 ceph-mon[31180]: pgmap v511: 97 pgs: 97 active+clean; 772 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 350 op/s 2024-09-21T04:10:22.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:22 smithi131 ceph-mon[35193]: pgmap v511: 97 pgs: 97 active+clean; 772 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 350 op/s 2024-09-21T04:10:23.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:23 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:10:23.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:23 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:10:23.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:23 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:10:23.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:23 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:10:23.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:23 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:10:23.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:23 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:10:24.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:24 smithi119 ceph-mon[34811]: pgmap v512: 97 pgs: 97 active+clean; 766 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 417 op/s 2024-09-21T04:10:24.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:24 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:10:24.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:24 smithi119 ceph-mon[34811]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T04:10:24.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:24 smithi064 ceph-mon[31180]: pgmap v512: 97 pgs: 97 active+clean; 766 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 417 op/s 2024-09-21T04:10:24.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:24 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:10:24.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:24 smithi064 ceph-mon[31180]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T04:10:24.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:24 smithi131 ceph-mon[35193]: pgmap v512: 97 pgs: 97 active+clean; 766 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 417 op/s 2024-09-21T04:10:24.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:24 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:10:24.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:24 smithi131 ceph-mon[35193]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T04:10:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.821+0000 7fd3c2cd5640 1 -- 172.21.15.64:0/1937698355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3bc108c30 msgr2=0x7fd3bc10f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.821+0000 7fd3c2cd5640 1 --2- 172.21.15.64:0/1937698355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3bc108c30 0x7fd3bc10f460 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009ca0 tx=0x7fd3b402a8c0 comp rx=0 tx=0).stop 2024-09-21T04:10:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.821+0000 7fd3c2cd5640 1 -- 172.21.15.64:0/1937698355 shutdown_connections 2024-09-21T04:10:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.821+0000 7fd3c2cd5640 1 --2- 172.21.15.64:0/1937698355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3bc108c30 0x7fd3bc10f460 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.821+0000 7fd3c2cd5640 1 --2- 172.21.15.64:0/1937698355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3bc1082e0 0x7fd3bc1086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.821+0000 7fd3c2cd5640 1 --2- 172.21.15.64:0/1937698355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3bc1029b0 0x7fd3bc102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.821+0000 7fd3c2cd5640 1 -- 172.21.15.64:0/1937698355 >> 172.21.15.64:0/1937698355 conn(0x7fd3bc07a740 msgr2=0x7fd3bc07ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.821+0000 7fd3c2cd5640 1 -- 172.21.15.64:0/1937698355 shutdown_connections 2024-09-21T04:10:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.821+0000 7fd3c2cd5640 1 -- 172.21.15.64:0/1937698355 wait complete. 2024-09-21T04:10:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.822+0000 7fd3c2cd5640 1 Processor -- start 2024-09-21T04:10:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.822+0000 7fd3c2cd5640 1 -- start start 2024-09-21T04:10:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.822+0000 7fd3c2cd5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3bc1029b0 0x7fd3bc19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.822+0000 7fd3c2cd5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3bc1082e0 0x7fd3bc19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.822+0000 7fd3c2cd5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3bc108c30 0x7fd3bc19fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.822+0000 7fd3c2cd5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc112940 con 0x7fd3bc1029b0 2024-09-21T04:10:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.822+0000 7fd3c2cd5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc1127c0 con 0x7fd3bc1082e0 2024-09-21T04:10:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.822+0000 7fd3c2cd5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc112ac0 con 0x7fd3bc108c30 2024-09-21T04:10:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.822+0000 7fd3c0a71640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3bc1029b0 0x7fd3bc19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.822+0000 7fd3c0a71640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3bc1029b0 0x7fd3bc19a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59302/0 (socket says 172.21.15.64:59302) 2024-09-21T04:10:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.822+0000 7fd3c0a71640 1 -- 172.21.15.64:0/2771344084 learned_addr learned my addr 172.21.15.64:0/2771344084 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.822+0000 7fd3bbfff640 1 --2- 172.21.15.64:0/2771344084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3bc1082e0 0x7fd3bc19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.822+0000 7fd3c1272640 1 --2- 172.21.15.64:0/2771344084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3bc108c30 0x7fd3bc19fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:25.827 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.822+0000 7fd3c0a71640 1 -- 172.21.15.64:0/2771344084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3bc108c30 msgr2=0x7fd3bc19fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.822+0000 7fd3c0a71640 1 --2- 172.21.15.64:0/2771344084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3bc108c30 0x7fd3bc19fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.822+0000 7fd3c0a71640 1 -- 172.21.15.64:0/2771344084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3bc1082e0 msgr2=0x7fd3bc19a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.822+0000 7fd3c0a71640 1 --2- 172.21.15.64:0/2771344084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3bc1082e0 0x7fd3bc19a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.822+0000 7fd3c0a71640 1 -- 172.21.15.64:0/2771344084 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3bc1a0320 con 0x7fd3bc1029b0 2024-09-21T04:10:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.822+0000 7fd3bbfff640 1 --2- 172.21.15.64:0/2771344084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3bc1082e0 0x7fd3bc19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.823+0000 7fd3c0a71640 1 --2- 172.21.15.64:0/2771344084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3bc1029b0 0x7fd3bc19a3f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd3b000e9f0 tx=0x7fd3b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.823+0000 7fd3b9ffb640 1 -- 172.21.15.64:0/2771344084 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b000cd80 con 0x7fd3bc1029b0 2024-09-21T04:10:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.823+0000 7fd3b9ffb640 1 -- 172.21.15.64:0/2771344084 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3b00108e0 con 0x7fd3bc1029b0 2024-09-21T04:10:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.823+0000 7fd3c2cd5640 1 -- 172.21.15.64:0/2771344084 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3bc1a0610 con 0x7fd3bc1029b0 2024-09-21T04:10:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.823+0000 7fd3b9ffb640 1 -- 172.21.15.64:0/2771344084 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0018980 con 0x7fd3bc1029b0 2024-09-21T04:10:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.823+0000 7fd3c2cd5640 1 -- 172.21.15.64:0/2771344084 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3bc1a6da0 con 0x7fd3bc1029b0 2024-09-21T04:10:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.826+0000 7fd3c9c3f740 1 Processor -- start 2024-09-21T04:10:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.826+0000 7fd3c9c3f740 1 -- start start 2024-09-21T04:10:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.826+0000 7fd3c9c3f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fef13fb1b0 0x55fef17a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.826+0000 7fd3c9c3f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fef13d22f0 0x55fef17a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.826+0000 7fd3c9c3f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13da4d0 0x55fef17a1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.826+0000 7fd3c9c3f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fef17156f0 con 0x55fef13da4d0 2024-09-21T04:10:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.826+0000 7fd3c9c3f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55fef1704b20 con 0x55fef13d22f0 2024-09-21T04:10:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.826+0000 7fd3c9c3f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fef1708930 con 0x55fef13fb1b0 2024-09-21T04:10:25.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.826+0000 7fd3b9ffb640 1 -- 172.21.15.64:0/2771344084 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fd3b0010430 con 0x7fd3bc1029b0 2024-09-21T04:10:25.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.826+0000 7fd3bbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13da4d0 0x55fef17a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:25.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.826+0000 7fd3bbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13da4d0 0x55fef17a1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59304/0 (socket says 172.21.15.64:59304) 2024-09-21T04:10:25.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.826+0000 7fd3bbfff640 1 -- 172.21.15.64:0/358694740 learned_addr learned my addr 172.21.15.64:0/358694740 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:25.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.826+0000 7fd3c0a71640 1 --2- 172.21.15.64:0/358694740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fef13d22f0 0x55fef17a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:25.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.826+0000 7fd3c1272640 1 --2- 172.21.15.64:0/358694740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fef13fb1b0 0x55fef17a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:25.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.827+0000 7fd3bbfff640 1 -- 172.21.15.64:0/358694740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fef13fb1b0 msgr2=0x55fef17a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:25.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.827+0000 7fd3bbfff640 1 --2- 172.21.15.64:0/358694740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fef13fb1b0 0x55fef17a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:25.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.827+0000 7fd3bbfff640 1 -- 172.21.15.64:0/358694740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fef13d22f0 msgr2=0x55fef17a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:25.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.827+0000 7fd3bbfff640 1 --2- 172.21.15.64:0/358694740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fef13d22f0 0x55fef17a0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:25.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.827+0000 7fd3bbfff640 1 -- 172.21.15.64:0/358694740 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fef16ca530 con 0x55fef13da4d0 2024-09-21T04:10:25.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.827+0000 7fd3bbfff640 1 --2- 172.21.15.64:0/358694740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13da4d0 0x55fef17a1430 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8010400 tx=0x7fd3a80108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07c4ec1203e4902 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:25.832 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.827+0000 7fd3a6ffd640 1 -- 172.21.15.64:0/358694740 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8011820 con 0x55fef13da4d0 2024-09-21T04:10:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.827+0000 7fd3a6ffd640 1 -- 172.21.15.64:0/358694740 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3a801c480 con 0x55fef13da4d0 2024-09-21T04:10:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.827+0000 7fd3a6ffd640 1 -- 172.21.15.64:0/358694740 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a801a5c0 con 0x55fef13da4d0 2024-09-21T04:10:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.827+0000 7fd3b9ffb640 1 --2- 172.21.15.64:0/2771344084 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd3ac0b6700 0x7fd3ac0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.827+0000 7fd3c1272640 1 --2- 172.21.15.64:0/2771344084 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd3ac0b6700 0x7fd3ac0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.828+0000 7fd3b9ffb640 1 -- 172.21.15.64:0/2771344084 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8659+0+0 (secure 0 0 0) 0x7fd3b00e7b10 con 0x7fd3bc1029b0 2024-09-21T04:10:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.828+0000 7fd3c9c3f740 1 -- 172.21.15.64:0/358694740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13da4d0 msgr2=0x55fef17a1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.828+0000 7fd3c9c3f740 1 --2- 172.21.15.64:0/358694740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13da4d0 0x55fef17a1430 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8010400 tx=0x7fd3a80108c0 comp rx=0 tx=0).stop 2024-09-21T04:10:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.828+0000 7fd3c1272640 1 --2- 172.21.15.64:0/2771344084 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd3ac0b6700 0x7fd3ac0b8c10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4004af0 tx=0x7fd3b4005da0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.828+0000 7fd3c9c3f740 1 -- 172.21.15.64:0/358694740 shutdown_connections 2024-09-21T04:10:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.828+0000 7fd3c9c3f740 1 --2- 172.21.15.64:0/358694740 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13da4d0 0x55fef17a1430 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.828+0000 7fd3c9c3f740 1 --2- 172.21.15.64:0/358694740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fef13d22f0 0x55fef17a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.828+0000 7fd3c9c3f740 1 --2- 172.21.15.64:0/358694740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fef13fb1b0 0x55fef17a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.828+0000 7fd3c9c3f740 1 -- 172.21.15.64:0/358694740 >> 172.21.15.64:0/358694740 conn(0x55fef13ef300 msgr2=0x55fef179e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.828+0000 7fd3c9c3f740 1 -- 172.21.15.64:0/358694740 shutdown_connections 2024-09-21T04:10:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.828+0000 7fd3c9c3f740 1 -- 172.21.15.64:0/358694740 wait complete. 2024-09-21T04:10:25.834 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.829+0000 7fd3c9c3f740 1 -- start start 2024-09-21T04:10:25.834 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.829+0000 7fd3c9c3f740 1 Processor -- start 2024-09-21T04:10:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.829+0000 7fd3c9c3f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fef13d22f0 0x55fef17b0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.829+0000 7fd3c9c3f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fef13da4d0 0x55fef17b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.829+0000 7fd3c9c3f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13fb1b0 0x55fef17bc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.829+0000 7fd3c9c3f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55fef163cab0 con 0x55fef13fb1b0 2024-09-21T04:10:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.829+0000 7fd3c9c3f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55fef16aa730 con 0x55fef13da4d0 2024-09-21T04:10:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.829+0000 7fd3c9c3f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fef17b3510 con 0x55fef13d22f0 2024-09-21T04:10:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.829+0000 7fd3c9c3f740 10 client.0 _ll_register_callbacks cb 0x55fef16fe6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:10:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.829+0000 7fd3c1272640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13fb1b0 0x55fef17bc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.829+0000 7fd3c1272640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13fb1b0 0x55fef17bc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59314/0 (socket says 172.21.15.64:59314) 2024-09-21T04:10:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.829+0000 7fd3c1272640 1 -- 172.21.15.64:0/1371258166 learned_addr learned my addr 172.21.15.64:0/1371258166 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.829+0000 7fd3bbfff640 1 --2- 172.21.15.64:0/1371258166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fef13da4d0 0x55fef17b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.829+0000 7fd3c0a71640 1 --2- 172.21.15.64:0/1371258166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fef13d22f0 0x55fef17b0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.830+0000 7fd3c1272640 1 -- 172.21.15.64:0/1371258166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fef13d22f0 msgr2=0x55fef17b0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.830+0000 7fd3c1272640 1 --2- 172.21.15.64:0/1371258166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fef13d22f0 0x55fef17b0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.830+0000 7fd3c1272640 1 -- 172.21.15.64:0/1371258166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fef13da4d0 msgr2=0x55fef17b0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.830+0000 7fd3c1272640 1 --2- 172.21.15.64:0/1371258166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fef13da4d0 0x55fef17b0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.830+0000 7fd3c1272640 1 -- 172.21.15.64:0/1371258166 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fef1466790 con 0x55fef13fb1b0 2024-09-21T04:10:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.830+0000 7fd3bbfff640 1 --2- 172.21.15.64:0/1371258166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fef13da4d0 0x55fef17b0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.830+0000 7fd3c1272640 1 --2- 172.21.15.64:0/1371258166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13fb1b0 0x55fef17bc840 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009ca0 tx=0x7fd3b402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6306fbaf330cbb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.830+0000 7fd39bfff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:10:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.830+0000 7fd39bfff640 1 -- 172.21.15.64:0/1371258166 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b402f610 con 0x55fef13fb1b0 2024-09-21T04:10:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.830+0000 7fd39bfff640 1 -- 172.21.15.64:0/1371258166 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8659+0+0 (secure 0 0 0) 0x7fd3b402f7b0 con 0x55fef13fb1b0 2024-09-21T04:10:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.830+0000 7fd3c9c3f740 1 -- 172.21.15.64:0/1371258166 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3a801c480 con 0x55fef13fb1b0 2024-09-21T04:10:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.830+0000 7fd39bfff640 1 -- 172.21.15.64:0/1371258166 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd3b402f9b0 con 0x55fef13fb1b0 2024-09-21T04:10:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.830+0000 7fd3c9c3f740 10 client.16821 fetch_fsmap learned FSMap version 37 2024-09-21T04:10:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.830+0000 7fd3c9c3f740 1 -- 172.21.15.64:0/1371258166 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fef14df620 con 0x55fef13fb1b0 2024-09-21T04:10:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.830+0000 7fd39bfff640 1 -- 172.21.15.64:0/1371258166 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3a801c480 con 0x55fef13fb1b0 2024-09-21T04:10:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.831+0000 7fd39bfff640 1 -- 172.21.15.64:0/1371258166 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fd3b4051070 con 0x55fef13fb1b0 2024-09-21T04:10:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.831+0000 7fd3c9c3f740 10 client.16821 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T04:10:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.831+0000 7fd3c9c3f740 10 client.16821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:10:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.831+0000 7fd3c9c3f740 20 client.16821 populate_metadata read hostname 'smithi064' 2024-09-21T04:10:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.831+0000 7fd3c9c3f740 1 --2- 172.21.15.64:0/1371258166 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55fef13b9d80 0x55fef17c2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.831+0000 7fd3c9c3f740 4 client.16821 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:10:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.831+0000 7fd3c9c3f740 1 -- 172.21.15.64:0/1371258166 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fef1717640 con 0x55fef13b9d80 2024-09-21T04:10:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.831+0000 7fd3c0a71640 1 --2- 172.21.15.64:0/1371258166 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55fef13b9d80 0x55fef17c2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.831+0000 7fd39bfff640 10 client.16821 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.832+0000 7fd3c0a71640 1 --2- 172.21.15.64:0/1371258166 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55fef13b9d80 0x55fef17c2610 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d44027812ec5d87e server_cookie=569e1fd3f401156 in_seq=0 out_seq=0 2024-09-21T04:10:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.832+0000 7fd39bfff640 1 -- 172.21.15.64:0/1371258166 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fef1717640 con 0x55fef13b9d80 2024-09-21T04:10:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.832+0000 7fd39bfff640 10 client.16821 handle_command_reply: tid=0 2024-09-21T04:10:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.832+0000 7fd3c9c3f740 1 client.16821 shutdown 2024-09-21T04:10:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.833+0000 7fd3c9c3f740 1 -- 172.21.15.64:0/1371258166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13fb1b0 msgr2=0x55fef17bc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.833+0000 7fd3c9c3f740 1 --2- 172.21.15.64:0/1371258166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13fb1b0 0x55fef17bc840 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009ca0 tx=0x7fd3b402a8c0 comp rx=0 tx=0).stop 2024-09-21T04:10:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.833+0000 7fd3c9c3f740 1 -- 172.21.15.64:0/1371258166 shutdown_connections 2024-09-21T04:10:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.833+0000 7fd3c9c3f740 1 --2- 172.21.15.64:0/1371258166 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55fef13b9d80 0x55fef17c2610 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.833+0000 7fd3c9c3f740 1 --2- 172.21.15.64:0/1371258166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13fb1b0 0x55fef17bc840 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.833+0000 7fd3c9c3f740 1 --2- 172.21.15.64:0/1371258166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fef13da4d0 0x55fef17b0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.833+0000 7fd39bfff640 0 client.16821 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.833+0000 7fd3c9c3f740 1 --2- 172.21.15.64:0/1371258166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fef13d22f0 0x55fef17b0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.833+0000 7fd3c9c3f740 1 -- 172.21.15.64:0/1371258166 >> 172.21.15.64:0/1371258166 conn(0x55fef13ef300 msgr2=0x55fef17a23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.833+0000 7fd3c9c3f740 1 -- 172.21.15.64:0/1371258166 shutdown_connections 2024-09-21T04:10:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.833+0000 7fd3c9c3f740 1 -- 172.21.15.64:0/1371258166 wait complete. 2024-09-21T04:10:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.833+0000 7fd3c9c3f740 20 client.16821 trim_cache size 0 max 16384 2024-09-21T04:10:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.849+0000 7fd3c9c3f740 1 Processor -- start 2024-09-21T04:10:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.849+0000 7fd3c9c3f740 1 -- start start 2024-09-21T04:10:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.849+0000 7fd3c9c3f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13d22f0 0x55fef17b0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.849+0000 7fd3c9c3f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fef13da4d0 0x55fef17b2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.849+0000 7fd3c9c3f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fef13fb1b0 0x55fef17bc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.849+0000 7fd3c9c3f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc1affb0 con 0x55fef13d22f0 2024-09-21T04:10:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.849+0000 7fd3c9c3f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc1ad850 con 0x55fef13fb1b0 2024-09-21T04:10:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.849+0000 7fd3c9c3f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc1ab6c0 con 0x55fef13da4d0 2024-09-21T04:10:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.849+0000 7fd3c0a71640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13d22f0 0x55fef17b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.849+0000 7fd3c0a71640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13d22f0 0x55fef17b0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59326/0 (socket says 172.21.15.64:59326) 2024-09-21T04:10:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.849+0000 7fd3c0a71640 1 -- 172.21.15.64:0/3767815579 learned_addr learned my addr 172.21.15.64:0/3767815579 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.849+0000 7fd3c1272640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fef13fb1b0 0x55fef17bc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.850+0000 7fd3bbfff640 1 --2- 172.21.15.64:0/3767815579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fef13da4d0 0x55fef17b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.850+0000 7fd3c0a71640 1 -- 172.21.15.64:0/3767815579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fef13da4d0 msgr2=0x55fef17b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.850+0000 7fd3c0a71640 1 --2- 172.21.15.64:0/3767815579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fef13da4d0 0x55fef17b2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.850+0000 7fd3c0a71640 1 -- 172.21.15.64:0/3767815579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fef13fb1b0 msgr2=0x55fef17bc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.850+0000 7fd3c0a71640 1 --2- 172.21.15.64:0/3767815579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fef13fb1b0 0x55fef17bc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.850+0000 7fd3c0a71640 1 -- 172.21.15.64:0/3767815579 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fef1798bd0 con 0x55fef13d22f0 2024-09-21T04:10:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.850+0000 7fd3c1272640 1 --2- 172.21.15.64:0/3767815579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fef13fb1b0 0x55fef17bc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.850+0000 7fd3c0a71640 1 --2- 172.21.15.64:0/3767815579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13d22f0 0x55fef17b0930 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd3b000c6a0 tx=0x7fd3b000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ffd2d738d94e31 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.850+0000 7fd3a67fc640 1 -- 172.21.15.64:0/3767815579 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b00ec040 con 0x55fef13d22f0 2024-09-21T04:10:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.850+0000 7fd3a67fc640 1 -- 172.21.15.64:0/3767815579 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3b0010720 con 0x55fef13d22f0 2024-09-21T04:10:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.850+0000 7fd3a67fc640 1 -- 172.21.15.64:0/3767815579 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b00b72f0 con 0x55fef13d22f0 2024-09-21T04:10:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.851+0000 7fd3c9c3f740 1 -- 172.21.15.64:0/3767815579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13d22f0 msgr2=0x55fef17b0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.851+0000 7fd3c9c3f740 1 --2- 172.21.15.64:0/3767815579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13d22f0 0x55fef17b0930 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd3b000c6a0 tx=0x7fd3b000cf20 comp rx=0 tx=0).stop 2024-09-21T04:10:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.851+0000 7fd3c9c3f740 1 -- 172.21.15.64:0/3767815579 shutdown_connections 2024-09-21T04:10:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.851+0000 7fd3c9c3f740 1 --2- 172.21.15.64:0/3767815579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fef13fb1b0 0x55fef17bc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.851+0000 7fd3c9c3f740 1 --2- 172.21.15.64:0/3767815579 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fef13da4d0 0x55fef17b2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.851+0000 7fd3c9c3f740 1 --2- 172.21.15.64:0/3767815579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13d22f0 0x55fef17b0930 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.851+0000 7fd3c9c3f740 1 -- 172.21.15.64:0/3767815579 >> 172.21.15.64:0/3767815579 conn(0x55fef13ef300 msgr2=0x55fef17b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.851+0000 7fd3c9c3f740 1 -- 172.21.15.64:0/3767815579 shutdown_connections 2024-09-21T04:10:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.851+0000 7fd3c9c3f740 1 -- 172.21.15.64:0/3767815579 wait complete. 2024-09-21T04:10:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.851+0000 7fd3c9c3f740 1 -- start start 2024-09-21T04:10:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.851+0000 7fd3c9c3f740 1 Processor -- start 2024-09-21T04:10:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.852+0000 7fd3c9c3f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13d22f0 0x55fef17b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.852+0000 7fd3c9c3f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fef13da4d0 0x55fef17b5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.852+0000 7fd3c9c3f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fef13fb1b0 0x55fef182bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.852+0000 7fd3c9c3f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc1abd10 con 0x55fef13d22f0 2024-09-21T04:10:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.852+0000 7fd3c9c3f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc1ad180 con 0x55fef13fb1b0 2024-09-21T04:10:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.852+0000 7fd3c9c3f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd3bc1ac3e0 con 0x55fef13da4d0 2024-09-21T04:10:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.852+0000 7fd3c9c3f740 10 client.0 _ll_register_callbacks cb 0x7fd3b4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:10:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.852+0000 7fd3c0a71640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13d22f0 0x55fef17b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.852+0000 7fd3c1272640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fef13fb1b0 0x55fef182bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.852+0000 7fd3c0a71640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13d22f0 0x55fef17b4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:59340/0 (socket says 172.21.15.64:59340) 2024-09-21T04:10:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.852+0000 7fd3c0a71640 1 -- 172.21.15.64:0/2593474851 learned_addr learned my addr 172.21.15.64:0/2593474851 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.852+0000 7fd3bbfff640 1 --2- 172.21.15.64:0/2593474851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fef13da4d0 0x55fef17b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.852+0000 7fd3c1272640 1 -- 172.21.15.64:0/2593474851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fef13da4d0 msgr2=0x55fef17b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.852+0000 7fd3c1272640 1 --2- 172.21.15.64:0/2593474851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fef13da4d0 0x55fef17b5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.852+0000 7fd3c1272640 1 -- 172.21.15.64:0/2593474851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13d22f0 msgr2=0x55fef17b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.852+0000 7fd3c1272640 1 --2- 172.21.15.64:0/2593474851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13d22f0 0x55fef17b4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.852+0000 7fd3c1272640 1 -- 172.21.15.64:0/2593474851 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fef182c510 con 0x55fef13fb1b0 2024-09-21T04:10:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.853+0000 7fd3c0a71640 1 --2- 172.21.15.64:0/2593474851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13d22f0 0x55fef17b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.853+0000 7fd3c1272640 1 --2- 172.21.15.64:0/2593474851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fef13fb1b0 0x55fef182bfd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4005e30 tx=0x7fd3b405a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39275cd129e0c6fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.853+0000 7fd39bfff640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:10:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.853+0000 7fd39bfff640 1 -- 172.21.15.64:0/2593474851 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4032ea0 con 0x55fef13fb1b0 2024-09-21T04:10:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.853+0000 7fd3c9c3f740 1 -- 172.21.15.64:0/2593474851 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fef17130f0 con 0x55fef13fb1b0 2024-09-21T04:10:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.853+0000 7fd39bfff640 1 -- 172.21.15.64:0/2593474851 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8659+0+0 (secure 0 0 0) 0x7fd3b40305c0 con 0x55fef13fb1b0 2024-09-21T04:10:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.853+0000 7fd39bfff640 1 -- 172.21.15.64:0/2593474851 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd3b40307c0 con 0x55fef13fb1b0 2024-09-21T04:10:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.853+0000 7fd3c9c3f740 10 client.26507 fetch_fsmap learned FSMap version 37 2024-09-21T04:10:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.853+0000 7fd3c9c3f740 1 -- 172.21.15.64:0/2593474851 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55fef182d5e0 con 0x55fef13fb1b0 2024-09-21T04:10:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.853+0000 7fd39bfff640 1 -- 172.21.15.64:0/2593474851 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55fef17130f0 con 0x55fef13fb1b0 2024-09-21T04:10:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.854+0000 7fd39bfff640 1 -- 172.21.15.64:0/2593474851 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fd3b4051050 con 0x55fef13fb1b0 2024-09-21T04:10:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.854+0000 7fd3c9c3f740 10 client.26507 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T04:10:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.854+0000 7fd3c9c3f740 10 client.26507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:10:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.854+0000 7fd3c9c3f740 20 client.26507 populate_metadata read hostname 'smithi064' 2024-09-21T04:10:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.854+0000 7fd3c9c3f740 1 --2- 172.21.15.64:0/2593474851 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55fef138f1c0 0x55fef182fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.854+0000 7fd3c9c3f740 4 client.26507 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:10:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.854+0000 7fd3c9c3f740 1 -- 172.21.15.64:0/2593474851 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fef1710520 con 0x55fef138f1c0 2024-09-21T04:10:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.854+0000 7fd3c0a71640 1 --2- 172.21.15.64:0/2593474851 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55fef138f1c0 0x55fef182fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.854+0000 7fd39bfff640 10 client.26507 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:25.855+0000 7fd3c0a71640 1 --2- 172.21.15.64:0/2593474851 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55fef138f1c0 0x55fef182fd90 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf049f3d34d5a378 server_cookie=5ae4900d3bc9f03c in_seq=0 out_seq=0 2024-09-21T04:10:26.035 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:26.032+0000 7fd39bfff640 1 -- 172.21.15.64:0/2593474851 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fef1710520 con 0x55fef138f1c0 2024-09-21T04:10:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:26.032+0000 7fd39bfff640 10 client.26507 handle_command_reply: tid=0 2024-09-21T04:10:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:26.032+0000 7fd3c9c3f740 1 client.26507 shutdown 2024-09-21T04:10:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:26.032+0000 7fd3c9c3f740 1 -- 172.21.15.64:0/2593474851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fef13fb1b0 msgr2=0x55fef182bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:26.032+0000 7fd3c9c3f740 1 --2- 172.21.15.64:0/2593474851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fef13fb1b0 0x55fef182bfd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4005e30 tx=0x7fd3b405a6f0 comp rx=0 tx=0).stop 2024-09-21T04:10:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:26.032+0000 7fd3c9c3f740 1 -- 172.21.15.64:0/2593474851 shutdown_connections 2024-09-21T04:10:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:26.032+0000 7fd3c9c3f740 1 --2- 172.21.15.64:0/2593474851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55fef13fb1b0 0x55fef182bfd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:26.032+0000 7fd3c9c3f740 1 --2- 172.21.15.64:0/2593474851 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fef13da4d0 0x55fef17b5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:26.032+0000 7fd3c9c3f740 1 --2- 172.21.15.64:0/2593474851 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55fef138f1c0 0x55fef182fd90 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:26.032+0000 7fd3c9c3f740 1 --2- 172.21.15.64:0/2593474851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55fef13d22f0 0x55fef17b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:26.032+0000 7fd39bfff640 0 client.26507 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:26.032+0000 7fd3c9c3f740 1 -- 172.21.15.64:0/2593474851 >> 172.21.15.64:0/2593474851 conn(0x55fef13ef300 msgr2=0x55fef17b1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:26.032+0000 7fd3c9c3f740 1 -- 172.21.15.64:0/2593474851 shutdown_connections 2024-09-21T04:10:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:26.032+0000 7fd3c9c3f740 1 -- 172.21.15.64:0/2593474851 wait complete. 2024-09-21T04:10:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:26.033+0000 7fd3c9c3f740 20 client.26507 trim_cache size 0 max 16384 2024-09-21T04:10:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:10:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:10:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:10:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:10:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:26.033+0000 7fd3c2cd5640 1 -- 172.21.15.64:0/2771344084 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd3ac0b6700 msgr2=0x7fd3ac0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:26.033+0000 7fd3c2cd5640 1 --2- 172.21.15.64:0/2771344084 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd3ac0b6700 0x7fd3ac0b8c10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4004af0 tx=0x7fd3b4005da0 comp rx=0 tx=0).stop 2024-09-21T04:10:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:26.033+0000 7fd3c2cd5640 1 -- 172.21.15.64:0/2771344084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3bc1029b0 msgr2=0x7fd3bc19a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:26.033+0000 7fd3c2cd5640 1 --2- 172.21.15.64:0/2771344084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3bc1029b0 0x7fd3bc19a3f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd3b000e9f0 tx=0x7fd3b000eeb0 comp rx=0 tx=0).stop 2024-09-21T04:10:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:26.033+0000 7fd3c2cd5640 1 -- 172.21.15.64:0/2771344084 shutdown_connections 2024-09-21T04:10:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:26.033+0000 7fd3c2cd5640 1 --2- 172.21.15.64:0/2771344084 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd3ac0b6700 0x7fd3ac0b8c10 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:26.033+0000 7fd3c2cd5640 1 --2- 172.21.15.64:0/2771344084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd3bc108c30 0x7fd3bc19fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:26.033+0000 7fd3c2cd5640 1 --2- 172.21.15.64:0/2771344084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3bc1082e0 0x7fd3bc19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:26.033+0000 7fd3c2cd5640 1 --2- 172.21.15.64:0/2771344084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3bc1029b0 0x7fd3bc19a3f0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:26.033+0000 7fd3c2cd5640 1 -- 172.21.15.64:0/2771344084 >> 172.21.15.64:0/2771344084 conn(0x7fd3bc07a740 msgr2=0x7fd3bc101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:26.033+0000 7fd3c2cd5640 1 -- 172.21.15.64:0/2771344084 shutdown_connections 2024-09-21T04:10:26.038 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:26.033+0000 7fd3c2cd5640 1 -- 172.21.15.64:0/2771344084 wait complete. 2024-09-21T04:10:26.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:26 smithi119 ceph-mon[34811]: pgmap v514: 97 pgs: 97 active+clean; 766 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 283 op/s 2024-09-21T04:10:26.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:26 smithi064 ceph-mon[31180]: pgmap v514: 97 pgs: 97 active+clean; 766 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 283 op/s 2024-09-21T04:10:26.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:26 smithi131 ceph-mon[35193]: pgmap v514: 97 pgs: 97 active+clean; 766 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 283 op/s 2024-09-21T04:10:28.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:28 smithi119 ceph-mon[34811]: pgmap v515: 97 pgs: 97 active+clean; 752 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 216 op/s 2024-09-21T04:10:28.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:28 smithi064 ceph-mon[31180]: pgmap v515: 97 pgs: 97 active+clean; 752 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 216 op/s 2024-09-21T04:10:28.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:28 smithi131 ceph-mon[35193]: pgmap v515: 97 pgs: 97 active+clean; 752 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 216 op/s 2024-09-21T04:10:30.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:30 smithi119 ceph-mon[34811]: pgmap v516: 97 pgs: 97 active+clean; 748 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 208 op/s 2024-09-21T04:10:30.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:30 smithi064 ceph-mon[31180]: pgmap v516: 97 pgs: 97 active+clean; 748 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 208 op/s 2024-09-21T04:10:30.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:30 smithi131 ceph-mon[35193]: pgmap v516: 97 pgs: 97 active+clean; 748 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 208 op/s 2024-09-21T04:10:31.175 INFO:tasks.workunit.client.0.smithi064.stdout: 123 177017 189 124377 207 106231 560 2024-09-21T04:10:32.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:32 smithi119 ceph-mon[34811]: pgmap v517: 97 pgs: 97 active+clean; 748 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 210 op/s 2024-09-21T04:10:32.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:32 smithi119 ceph-mon[34811]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T04:10:32.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:32 smithi064 ceph-mon[31180]: pgmap v517: 97 pgs: 97 active+clean; 748 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 210 op/s 2024-09-21T04:10:32.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:32 smithi064 ceph-mon[31180]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T04:10:32.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:32 smithi131 ceph-mon[35193]: pgmap v517: 97 pgs: 97 active+clean; 748 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 210 op/s 2024-09-21T04:10:32.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:32 smithi131 ceph-mon[35193]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T04:10:33.319 INFO:tasks.workunit.client.0.smithi064.stdout: 2024-09-21T04:10:33.319 INFO:tasks.workunit.client.0.smithi064.stdout:Final score for writes: 123 2024-09-21T04:10:33.319 INFO:tasks.workunit.client.0.smithi064.stdout:Final score for reads : 318467 2024-09-21T04:10:33.319 INFO:tasks.workunit.client.0.smithi064.stdout: 2024-09-21T04:10:33.320 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-21T04:10:33.320 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-21T04:10:34.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:34 smithi119 ceph-mon[34811]: pgmap v519: 97 pgs: 97 active+clean; 753 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 208 op/s 2024-09-21T04:10:34.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:34 smithi064 ceph-mon[31180]: pgmap v519: 97 pgs: 97 active+clean; 753 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 208 op/s 2024-09-21T04:10:34.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:34 smithi131 ceph-mon[35193]: pgmap v519: 97 pgs: 97 active+clean; 753 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 208 op/s 2024-09-21T04:10:36.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:36 smithi119 ceph-mon[34811]: pgmap v520: 97 pgs: 97 active+clean; 753 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 195 op/s 2024-09-21T04:10:36.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:36 smithi064 ceph-mon[31180]: pgmap v520: 97 pgs: 97 active+clean; 753 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 195 op/s 2024-09-21T04:10:36.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:36 smithi131 ceph-mon[35193]: pgmap v520: 97 pgs: 97 active+clean; 753 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 195 op/s 2024-09-21T04:10:38.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:38 smithi119 ceph-mon[34811]: pgmap v521: 97 pgs: 97 active+clean; 785 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.3 MiB/s wr, 388 op/s 2024-09-21T04:10:38.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:38 smithi064 ceph-mon[31180]: pgmap v521: 97 pgs: 97 active+clean; 785 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.3 MiB/s wr, 388 op/s 2024-09-21T04:10:38.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:38 smithi131 ceph-mon[35193]: pgmap v521: 97 pgs: 97 active+clean; 785 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.3 MiB/s wr, 388 op/s 2024-09-21T04:10:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.135+0000 7f7f53dc3640 1 -- 172.21.15.64:0/1040057410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f4c109d80 msgr2=0x7f7f4c1105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.135+0000 7f7f53dc3640 1 --2- 172.21.15.64:0/1040057410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f4c109d80 0x7f7f4c1105b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009ca0 tx=0x7f7f4402a8c0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.135+0000 7f7f53dc3640 1 -- 172.21.15.64:0/1040057410 shutdown_connections 2024-09-21T04:10:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.135+0000 7f7f53dc3640 1 --2- 172.21.15.64:0/1040057410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f4c109d80 0x7f7f4c1105b0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.135+0000 7f7f53dc3640 1 --2- 172.21.15.64:0/1040057410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f4c109430 0x7f7f4c109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.135+0000 7f7f53dc3640 1 --2- 172.21.15.64:0/1040057410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f4c103b00 0x7f7f4c103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.135+0000 7f7f53dc3640 1 -- 172.21.15.64:0/1040057410 >> 172.21.15.64:0/1040057410 conn(0x7f7f4c0ff500 msgr2=0x7f7f4c101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.135+0000 7f7f53dc3640 1 -- 172.21.15.64:0/1040057410 shutdown_connections 2024-09-21T04:10:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.135+0000 7f7f53dc3640 1 -- 172.21.15.64:0/1040057410 wait complete. 2024-09-21T04:10:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.136+0000 7f7f53dc3640 1 Processor -- start 2024-09-21T04:10:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.136+0000 7f7f53dc3640 1 -- start start 2024-09-21T04:10:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.136+0000 7f7f53dc3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f4c103b00 0x7f7f4c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.136+0000 7f7f53dc3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f4c109430 0x7f7f4c19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.136+0000 7f7f53dc3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f4c109d80 0x7f7f4c19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.136+0000 7f7f53dc3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c113a40 con 0x7f7f4c103b00 2024-09-21T04:10:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.136+0000 7f7f53dc3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1138c0 con 0x7f7f4c109430 2024-09-21T04:10:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.136+0000 7f7f53dc3640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c113bc0 con 0x7f7f4c109d80 2024-09-21T04:10:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.136+0000 7f7f51b5f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f4c103b00 0x7f7f4c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.136+0000 7f7f51b5f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f4c103b00 0x7f7f4c19a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55024/0 (socket says 172.21.15.64:55024) 2024-09-21T04:10:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.136+0000 7f7f51b5f640 1 -- 172.21.15.64:0/2891340124 learned_addr learned my addr 172.21.15.64:0/2891340124 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.136+0000 7f7f5135e640 1 --2- 172.21.15.64:0/2891340124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f4c109430 0x7f7f4c19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.136+0000 7f7f52360640 1 --2- 172.21.15.64:0/2891340124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f4c109d80 0x7f7f4c19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.136+0000 7f7f51b5f640 1 -- 172.21.15.64:0/2891340124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f4c109d80 msgr2=0x7f7f4c19fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.136+0000 7f7f51b5f640 1 --2- 172.21.15.64:0/2891340124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f4c109d80 0x7f7f4c19fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.136+0000 7f7f51b5f640 1 -- 172.21.15.64:0/2891340124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f4c109430 msgr2=0x7f7f4c19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.136+0000 7f7f51b5f640 1 --2- 172.21.15.64:0/2891340124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f4c109430 0x7f7f4c19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.136+0000 7f7f51b5f640 1 -- 172.21.15.64:0/2891340124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c1a0350 con 0x7f7f4c103b00 2024-09-21T04:10:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.136+0000 7f7f52360640 1 --2- 172.21.15.64:0/2891340124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f4c109d80 0x7f7f4c19fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:10:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.137+0000 7f7f51b5f640 1 --2- 172.21.15.64:0/2891340124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f4c103b00 0x7f7f4c19a420 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800e9f0 tx=0x7f7f4800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.137+0000 7f7f3effd640 1 -- 172.21.15.64:0/2891340124 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4800cd80 con 0x7f7f4c103b00 2024-09-21T04:10:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.137+0000 7f7f53dc3640 1 -- 172.21.15.64:0/2891340124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4c1a0640 con 0x7f7f4c103b00 2024-09-21T04:10:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.137+0000 7f7f3effd640 1 -- 172.21.15.64:0/2891340124 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f480108e0 con 0x7f7f4c103b00 2024-09-21T04:10:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.137+0000 7f7f53dc3640 1 -- 172.21.15.64:0/2891340124 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4c1a6dd0 con 0x7f7f4c103b00 2024-09-21T04:10:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.137+0000 7f7f3effd640 1 -- 172.21.15.64:0/2891340124 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48018920 con 0x7f7f4c103b00 2024-09-21T04:10:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.140+0000 7f7f5ae0f740 1 Processor -- start 2024-09-21T04:10:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.141+0000 7f7f5ae0f740 1 -- start start 2024-09-21T04:10:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.141+0000 7f7f3effd640 1 -- 172.21.15.64:0/2891340124 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f7f48010430 con 0x7f7f4c103b00 2024-09-21T04:10:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.141+0000 7f7f5ae0f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc938dd1b0 0x55cc93c829b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.141+0000 7f7f5ae0f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc938b42f0 0x55cc93c82ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.141+0000 7f7f5ae0f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc938bc4d0 0x55cc93c83430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.141+0000 7f7f5ae0f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55cc93bf76f0 con 0x55cc938b42f0 2024-09-21T04:10:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.141+0000 7f7f5ae0f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55cc93be6b20 con 0x55cc938bc4d0 2024-09-21T04:10:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.141+0000 7f7f5ae0f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55cc93bea930 con 0x55cc938dd1b0 2024-09-21T04:10:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.141+0000 7f7f51b5f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc938b42f0 0x55cc93c82ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.141+0000 7f7f51b5f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc938b42f0 0x55cc93c82ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55030/0 (socket says 172.21.15.64:55030) 2024-09-21T04:10:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.141+0000 7f7f51b5f640 1 -- 172.21.15.64:0/2212474187 learned_addr learned my addr 172.21.15.64:0/2212474187 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.141+0000 7f7f5135e640 1 --2- 172.21.15.64:0/2212474187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc938bc4d0 0x55cc93c83430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.141+0000 7f7f52360640 1 --2- 172.21.15.64:0/2212474187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc938dd1b0 0x55cc93c829b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.141+0000 7f7f5135e640 1 -- 172.21.15.64:0/2212474187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc938dd1b0 msgr2=0x55cc93c829b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.142+0000 7f7f5135e640 1 --2- 172.21.15.64:0/2212474187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc938dd1b0 0x55cc93c829b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.142+0000 7f7f5135e640 1 -- 172.21.15.64:0/2212474187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc938b42f0 msgr2=0x55cc93c82ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.142+0000 7f7f5135e640 1 --2- 172.21.15.64:0/2212474187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc938b42f0 0x55cc93c82ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.142+0000 7f7f5135e640 1 -- 172.21.15.64:0/2212474187 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc93bac530 con 0x55cc938bc4d0 2024-09-21T04:10:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.142+0000 7f7f5135e640 1 --2- 172.21.15.64:0/2212474187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc938bc4d0 0x55cc93c83430 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c06c720 tx=0x7f7f40011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=effe103a61c268b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.142+0000 7f7f3effd640 1 --2- 172.21.15.64:0/2891340124 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7f380b6680 0x7f7f380b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.142+0000 7f7f37fff640 1 -- 172.21.15.64:0/2212474187 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40004250 con 0x55cc938bc4d0 2024-09-21T04:10:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.142+0000 7f7f37fff640 1 -- 172.21.15.64:0/2212474187 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f40009e10 con 0x55cc938bc4d0 2024-09-21T04:10:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.142+0000 7f7f37fff640 1 -- 172.21.15.64:0/2212474187 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40004c90 con 0x55cc938bc4d0 2024-09-21T04:10:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.142+0000 7f7f52360640 1 --2- 172.21.15.64:0/2891340124 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7f380b6680 0x7f7f380b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.142+0000 7f7f3effd640 1 -- 172.21.15.64:0/2891340124 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7f7f480e7ae0 con 0x7f7f4c103b00 2024-09-21T04:10:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.143+0000 7f7f5ae0f740 1 -- 172.21.15.64:0/2212474187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc938bc4d0 msgr2=0x55cc93c83430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.143+0000 7f7f5ae0f740 1 --2- 172.21.15.64:0/2212474187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc938bc4d0 0x55cc93c83430 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c06c720 tx=0x7f7f40011810 comp rx=0 tx=0).stop 2024-09-21T04:10:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.143+0000 7f7f52360640 1 --2- 172.21.15.64:0/2891340124 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7f380b6680 0x7f7f380b8b90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7f44004af0 tx=0x7f7f44005c30 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.143+0000 7f7f5ae0f740 1 -- 172.21.15.64:0/2212474187 shutdown_connections 2024-09-21T04:10:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.143+0000 7f7f5ae0f740 1 --2- 172.21.15.64:0/2212474187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc938bc4d0 0x55cc93c83430 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.143+0000 7f7f5ae0f740 1 --2- 172.21.15.64:0/2212474187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc938b42f0 0x55cc93c82ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.143+0000 7f7f5ae0f740 1 --2- 172.21.15.64:0/2212474187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc938dd1b0 0x55cc93c829b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.143+0000 7f7f5ae0f740 1 -- 172.21.15.64:0/2212474187 >> 172.21.15.64:0/2212474187 conn(0x55cc938d1300 msgr2=0x55cc93c808a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.143+0000 7f7f5ae0f740 1 -- 172.21.15.64:0/2212474187 shutdown_connections 2024-09-21T04:10:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.143+0000 7f7f5ae0f740 1 -- 172.21.15.64:0/2212474187 wait complete. 2024-09-21T04:10:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.143+0000 7f7f5ae0f740 1 -- start start 2024-09-21T04:10:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.143+0000 7f7f5ae0f740 1 Processor -- start 2024-09-21T04:10:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.144+0000 7f7f5ae0f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc938b42f0 0x55cc93c92420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.144+0000 7f7f5ae0f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc938bc4d0 0x55cc93c92960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.144+0000 7f7f5ae0f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc938dd1b0 0x55cc93c92ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.144+0000 7f7f5ae0f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55cc93b1eab0 con 0x55cc938dd1b0 2024-09-21T04:10:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.144+0000 7f7f5ae0f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55cc93b8c730 con 0x55cc938b42f0 2024-09-21T04:10:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.144+0000 7f7f5ae0f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55cc93c95510 con 0x55cc938bc4d0 2024-09-21T04:10:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.144+0000 7f7f5ae0f740 10 client.0 _ll_register_callbacks cb 0x55cc93be06b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:10:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.144+0000 7f7f52360640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc938dd1b0 0x55cc93c92ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.144+0000 7f7f52360640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc938dd1b0 0x55cc93c92ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55046/0 (socket says 172.21.15.64:55046) 2024-09-21T04:10:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.144+0000 7f7f52360640 1 -- 172.21.15.64:0/3542302697 learned_addr learned my addr 172.21.15.64:0/3542302697 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.144+0000 7f7f51b5f640 1 --2- 172.21.15.64:0/3542302697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc938b42f0 0x55cc93c92420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.144+0000 7f7f5135e640 1 --2- 172.21.15.64:0/3542302697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc938bc4d0 0x55cc93c92960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.144+0000 7f7f52360640 1 -- 172.21.15.64:0/3542302697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc938bc4d0 msgr2=0x55cc93c92960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.144+0000 7f7f52360640 1 --2- 172.21.15.64:0/3542302697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc938bc4d0 0x55cc93c92960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.144+0000 7f7f52360640 1 -- 172.21.15.64:0/3542302697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc938b42f0 msgr2=0x55cc93c92420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.144+0000 7f7f52360640 1 --2- 172.21.15.64:0/3542302697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc938b42f0 0x55cc93c92420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.144+0000 7f7f52360640 1 -- 172.21.15.64:0/3542302697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc93948790 con 0x55cc938dd1b0 2024-09-21T04:10:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.144+0000 7f7f5135e640 1 --2- 172.21.15.64:0/3542302697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc938bc4d0 0x55cc93c92960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:10:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.144+0000 7f7f51b5f640 1 --2- 172.21.15.64:0/3542302697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc938b42f0 0x55cc93c92420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.145+0000 7f7f52360640 1 --2- 172.21.15.64:0/3542302697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc938dd1b0 0x55cc93c92ec0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7f44035000 tx=0x7f7f4402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54692af3b762a3b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.145+0000 7f7f357fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:10:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.145+0000 7f7f357fa640 1 -- 172.21.15.64:0/3542302697 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f44003c40 con 0x55cc938dd1b0 2024-09-21T04:10:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.145+0000 7f7f357fa640 1 -- 172.21.15.64:0/3542302697 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7f7f44003de0 con 0x55cc938dd1b0 2024-09-21T04:10:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.145+0000 7f7f5ae0f740 1 -- 172.21.15.64:0/3542302697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f40009e10 con 0x55cc938dd1b0 2024-09-21T04:10:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.145+0000 7f7f357fa640 1 -- 172.21.15.64:0/3542302697 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7f44004040 con 0x55cc938dd1b0 2024-09-21T04:10:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.145+0000 7f7f5ae0f740 10 client.16839 fetch_fsmap learned FSMap version 37 2024-09-21T04:10:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.145+0000 7f7f5ae0f740 1 -- 172.21.15.64:0/3542302697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55cc939c1620 con 0x55cc938dd1b0 2024-09-21T04:10:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.145+0000 7f7f357fa640 1 -- 172.21.15.64:0/3542302697 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f40009e10 con 0x55cc938dd1b0 2024-09-21T04:10:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.145+0000 7f7f357fa640 1 -- 172.21.15.64:0/3542302697 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f7f44051070 con 0x55cc938dd1b0 2024-09-21T04:10:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.146+0000 7f7f5ae0f740 10 client.16839 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T04:10:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.146+0000 7f7f5ae0f740 10 client.16839 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:10:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.146+0000 7f7f5ae0f740 20 client.16839 populate_metadata read hostname 'smithi064' 2024-09-21T04:10:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.146+0000 7f7f5ae0f740 1 --2- 172.21.15.64:0/3542302697 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55cc9389bd80 0x55cc93ca4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.146+0000 7f7f5ae0f740 4 client.16839 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:10:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.146+0000 7f7f5ae0f740 1 -- 172.21.15.64:0/3542302697 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc93bf9640 con 0x55cc9389bd80 2024-09-21T04:10:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.146+0000 7f7f51b5f640 1 --2- 172.21.15.64:0/3542302697 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55cc9389bd80 0x55cc93ca4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.146+0000 7f7f357fa640 10 client.16839 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.146+0000 7f7f51b5f640 1 --2- 172.21.15.64:0/3542302697 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55cc9389bd80 0x55cc93ca4620 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0416ce34da55ac8 server_cookie=2c4ae0b19ea0788a in_seq=0 out_seq=0 2024-09-21T04:10:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.147+0000 7f7f357fa640 1 -- 172.21.15.64:0/3542302697 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cc93bf9640 con 0x55cc9389bd80 2024-09-21T04:10:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.147+0000 7f7f357fa640 10 client.16839 handle_command_reply: tid=0 2024-09-21T04:10:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.147+0000 7f7f5ae0f740 1 client.16839 shutdown 2024-09-21T04:10:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.147+0000 7f7f5ae0f740 1 -- 172.21.15.64:0/3542302697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc938dd1b0 msgr2=0x55cc93c92ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.147+0000 7f7f5ae0f740 1 --2- 172.21.15.64:0/3542302697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc938dd1b0 0x55cc93c92ec0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7f44035000 tx=0x7f7f4402a8c0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.147+0000 7f7f5ae0f740 1 -- 172.21.15.64:0/3542302697 shutdown_connections 2024-09-21T04:10:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.147+0000 7f7f5ae0f740 1 --2- 172.21.15.64:0/3542302697 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55cc9389bd80 0x55cc93ca4620 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.148+0000 7f7f5ae0f740 1 --2- 172.21.15.64:0/3542302697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc938dd1b0 0x55cc93c92ec0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.148+0000 7f7f5ae0f740 1 --2- 172.21.15.64:0/3542302697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc938bc4d0 0x55cc93c92960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.148+0000 7f7f5ae0f740 1 --2- 172.21.15.64:0/3542302697 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc938b42f0 0x55cc93c92420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.148+0000 7f7f357fa640 0 client.16839 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.148+0000 7f7f5ae0f740 1 -- 172.21.15.64:0/3542302697 >> 172.21.15.64:0/3542302697 conn(0x55cc938d1300 msgr2=0x55cc93c843a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.148+0000 7f7f5ae0f740 1 -- 172.21.15.64:0/3542302697 shutdown_connections 2024-09-21T04:10:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.148+0000 7f7f5ae0f740 1 -- 172.21.15.64:0/3542302697 wait complete. 2024-09-21T04:10:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.148+0000 7f7f5ae0f740 20 client.16839 trim_cache size 0 max 16384 2024-09-21T04:10:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.164+0000 7f7f5ae0f740 1 Processor -- start 2024-09-21T04:10:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.164+0000 7f7f5ae0f740 1 -- start start 2024-09-21T04:10:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.164+0000 7f7f5ae0f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc938b42f0 0x55cc93c94050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.164+0000 7f7f5ae0f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc938bc4d0 0x55cc93c8f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.164+0000 7f7f5ae0f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc938dd1b0 0x55cc93c8fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.164+0000 7f7f5ae0f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1b0050 con 0x55cc938bc4d0 2024-09-21T04:10:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.164+0000 7f7f5ae0f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1ad8f0 con 0x55cc938b42f0 2024-09-21T04:10:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.164+0000 7f7f5ae0f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1ab760 con 0x55cc938dd1b0 2024-09-21T04:10:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.164+0000 7f7f5135e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc938bc4d0 0x55cc93c8f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.164+0000 7f7f5135e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc938bc4d0 0x55cc93c8f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55058/0 (socket says 172.21.15.64:55058) 2024-09-21T04:10:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.164+0000 7f7f5135e640 1 -- 172.21.15.64:0/3220420186 learned_addr learned my addr 172.21.15.64:0/3220420186 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.164+0000 7f7f51b5f640 1 --2- 172.21.15.64:0/3220420186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc938b42f0 0x55cc93c94050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.164+0000 7f7f52360640 1 --2- 172.21.15.64:0/3220420186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc938dd1b0 0x55cc93c8fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.164+0000 7f7f5135e640 1 -- 172.21.15.64:0/3220420186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc938dd1b0 msgr2=0x55cc93c8fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.164+0000 7f7f5135e640 1 --2- 172.21.15.64:0/3220420186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc938dd1b0 0x55cc93c8fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.164+0000 7f7f5135e640 1 -- 172.21.15.64:0/3220420186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc938b42f0 msgr2=0x55cc93c94050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.164+0000 7f7f5135e640 1 --2- 172.21.15.64:0/3220420186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc938b42f0 0x55cc93c94050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.164+0000 7f7f5135e640 1 -- 172.21.15.64:0/3220420186 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc93c85810 con 0x55cc938bc4d0 2024-09-21T04:10:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.164+0000 7f7f5135e640 1 --2- 172.21.15.64:0/3220420186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc938bc4d0 0x55cc93c8f5f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009780 tx=0x7f7f400097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5f9fa756589661 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.165+0000 7f7f377fe640 1 -- 172.21.15.64:0/3220420186 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40029020 con 0x55cc938bc4d0 2024-09-21T04:10:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.165+0000 7f7f377fe640 1 -- 172.21.15.64:0/3220420186 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f4001f040 con 0x55cc938bc4d0 2024-09-21T04:10:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.165+0000 7f7f377fe640 1 -- 172.21.15.64:0/3220420186 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4001e040 con 0x55cc938bc4d0 2024-09-21T04:10:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.165+0000 7f7f5ae0f740 1 -- 172.21.15.64:0/3220420186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc938bc4d0 msgr2=0x55cc93c8f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.165+0000 7f7f5ae0f740 1 --2- 172.21.15.64:0/3220420186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc938bc4d0 0x55cc93c8f5f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009780 tx=0x7f7f400097b0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.165+0000 7f7f5ae0f740 1 -- 172.21.15.64:0/3220420186 shutdown_connections 2024-09-21T04:10:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.165+0000 7f7f5ae0f740 1 --2- 172.21.15.64:0/3220420186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc938dd1b0 0x55cc93c8fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.165+0000 7f7f5ae0f740 1 --2- 172.21.15.64:0/3220420186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc938bc4d0 0x55cc93c8f5f0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.165+0000 7f7f5ae0f740 1 --2- 172.21.15.64:0/3220420186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc938b42f0 0x55cc93c94050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.165+0000 7f7f5ae0f740 1 -- 172.21.15.64:0/3220420186 >> 172.21.15.64:0/3220420186 conn(0x55cc938d1300 msgr2=0x55cc93c93940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.165+0000 7f7f5ae0f740 1 -- 172.21.15.64:0/3220420186 shutdown_connections 2024-09-21T04:10:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.165+0000 7f7f5ae0f740 1 -- 172.21.15.64:0/3220420186 wait complete. 2024-09-21T04:10:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.166+0000 7f7f5ae0f740 1 -- start start 2024-09-21T04:10:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.166+0000 7f7f5ae0f740 1 Processor -- start 2024-09-21T04:10:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.166+0000 7f7f5ae0f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc938b42f0 0x55cc93c95fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.166+0000 7f7f5ae0f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc938bc4d0 0x55cc93c96510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.166+0000 7f7f5ae0f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc938dd1b0 0x55cc93d0cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.166+0000 7f7f5ae0f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1abdb0 con 0x55cc938b42f0 2024-09-21T04:10:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.166+0000 7f7f5ae0f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1ad220 con 0x55cc938dd1b0 2024-09-21T04:10:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.166+0000 7f7f5ae0f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1ac480 con 0x55cc938bc4d0 2024-09-21T04:10:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.166+0000 7f7f5ae0f740 10 client.0 _ll_register_callbacks cb 0x7f7f40005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:10:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.166+0000 7f7f51b5f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc938b42f0 0x55cc93c95fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.166+0000 7f7f51b5f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc938b42f0 0x55cc93c95fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:55074/0 (socket says 172.21.15.64:55074) 2024-09-21T04:10:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.166+0000 7f7f51b5f640 1 -- 172.21.15.64:0/2054458047 learned_addr learned my addr 172.21.15.64:0/2054458047 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.167+0000 7f7f52360640 1 --2- 172.21.15.64:0/2054458047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc938dd1b0 0x55cc93d0cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.167+0000 7f7f5135e640 1 --2- 172.21.15.64:0/2054458047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc938bc4d0 0x55cc93c96510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.167+0000 7f7f51b5f640 1 -- 172.21.15.64:0/2054458047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc938bc4d0 msgr2=0x55cc93c96510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:39.173 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.167+0000 7f7f51b5f640 1 --2- 172.21.15.64:0/2054458047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc938bc4d0 0x55cc93c96510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.167+0000 7f7f51b5f640 1 -- 172.21.15.64:0/2054458047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc938dd1b0 msgr2=0x55cc93d0cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.167+0000 7f7f51b5f640 1 --2- 172.21.15.64:0/2054458047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc938dd1b0 0x55cc93d0cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.167+0000 7f7f51b5f640 1 -- 172.21.15.64:0/2054458047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc93d0d5c0 con 0x55cc938b42f0 2024-09-21T04:10:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.167+0000 7f7f51b5f640 1 --2- 172.21.15.64:0/2054458047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc938b42f0 0x55cc93c95fd0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7f480a7fd0 tx=0x7f7f48018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1fe584f1ab8b79a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.167+0000 7f7f357fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:10:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.167+0000 7f7f357fa640 1 -- 172.21.15.64:0/2054458047 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f480ad6a0 con 0x55cc938b42f0 2024-09-21T04:10:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.167+0000 7f7f357fa640 1 -- 172.21.15.64:0/2054458047 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7f7f480a9220 con 0x55cc938b42f0 2024-09-21T04:10:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.167+0000 7f7f5ae0f740 1 -- 172.21.15.64:0/2054458047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f4001f040 con 0x55cc938b42f0 2024-09-21T04:10:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.167+0000 7f7f357fa640 1 -- 172.21.15.64:0/2054458047 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7f480ad040 con 0x55cc938b42f0 2024-09-21T04:10:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.167+0000 7f7f5ae0f740 10 client.16845 fetch_fsmap learned FSMap version 37 2024-09-21T04:10:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.167+0000 7f7f5ae0f740 1 -- 172.21.15.64:0/2054458047 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55cc93d0e850 con 0x55cc938b42f0 2024-09-21T04:10:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.167+0000 7f7f357fa640 1 -- 172.21.15.64:0/2054458047 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f4001f040 con 0x55cc938b42f0 2024-09-21T04:10:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.168+0000 7f7f357fa640 1 -- 172.21.15.64:0/2054458047 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f7f480a9420 con 0x55cc938b42f0 2024-09-21T04:10:39.174 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.168+0000 7f7f5ae0f740 10 client.16845 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T04:10:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.168+0000 7f7f5ae0f740 10 client.16845 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:10:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.168+0000 7f7f5ae0f740 20 client.16845 populate_metadata read hostname 'smithi064' 2024-09-21T04:10:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.168+0000 7f7f5ae0f740 1 --2- 172.21.15.64:0/2054458047 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55cc938711c0 0x55cc93d11070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.168+0000 7f7f5ae0f740 4 client.16845 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:10:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.168+0000 7f7f5ae0f740 1 -- 172.21.15.64:0/2054458047 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cc93bf50f0 con 0x55cc938711c0 2024-09-21T04:10:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.168+0000 7f7f5135e640 1 --2- 172.21.15.64:0/2054458047 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55cc938711c0 0x55cc93d11070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.168+0000 7f7f357fa640 10 client.16845 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:39.175 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.168+0000 7f7f5135e640 1 --2- 172.21.15.64:0/2054458047 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55cc938711c0 0x55cc93d11070 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da0945f7c57f91d4 server_cookie=cb819cab26a62baa in_seq=0 out_seq=0 2024-09-21T04:10:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.234+0000 7f7f357fa640 1 -- 172.21.15.64:0/2054458047 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cc93bf50f0 con 0x55cc938711c0 2024-09-21T04:10:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.234+0000 7f7f357fa640 10 client.16845 handle_command_reply: tid=0 2024-09-21T04:10:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.234+0000 7f7f5ae0f740 1 client.16845 shutdown 2024-09-21T04:10:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.234+0000 7f7f5ae0f740 1 -- 172.21.15.64:0/2054458047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc938b42f0 msgr2=0x55cc93c95fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.234+0000 7f7f5ae0f740 1 --2- 172.21.15.64:0/2054458047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc938b42f0 0x55cc93c95fd0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7f480a7fd0 tx=0x7f7f48018420 comp rx=0 tx=0).stop 2024-09-21T04:10:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.235+0000 7f7f5ae0f740 1 -- 172.21.15.64:0/2054458047 shutdown_connections 2024-09-21T04:10:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.235+0000 7f7f5ae0f740 1 --2- 172.21.15.64:0/2054458047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cc938dd1b0 0x55cc93d0cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.235+0000 7f7f5ae0f740 1 --2- 172.21.15.64:0/2054458047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cc938bc4d0 0x55cc93c96510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.235+0000 7f7f5ae0f740 1 --2- 172.21.15.64:0/2054458047 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55cc938711c0 0x55cc93d11070 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.235+0000 7f7f5ae0f740 1 --2- 172.21.15.64:0/2054458047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cc938b42f0 0x55cc93c95fd0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.235+0000 7f7f357fa640 0 client.16845 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.235+0000 7f7f5ae0f740 1 -- 172.21.15.64:0/2054458047 >> 172.21.15.64:0/2054458047 conn(0x55cc938d1300 msgr2=0x55cc93c83990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.235+0000 7f7f5ae0f740 1 -- 172.21.15.64:0/2054458047 shutdown_connections 2024-09-21T04:10:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.235+0000 7f7f5ae0f740 1 -- 172.21.15.64:0/2054458047 wait complete. 2024-09-21T04:10:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.235+0000 7f7f5ae0f740 20 client.16845 trim_cache size 0 max 16384 2024-09-21T04:10:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:10:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:10:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:10:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:10:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.235+0000 7f7f53dc3640 1 -- 172.21.15.64:0/2891340124 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7f380b6680 msgr2=0x7f7f380b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.235+0000 7f7f53dc3640 1 --2- 172.21.15.64:0/2891340124 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7f380b6680 0x7f7f380b8b90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7f44004af0 tx=0x7f7f44005c30 comp rx=0 tx=0).stop 2024-09-21T04:10:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.235+0000 7f7f53dc3640 1 -- 172.21.15.64:0/2891340124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f4c103b00 msgr2=0x7f7f4c19a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.235+0000 7f7f53dc3640 1 --2- 172.21.15.64:0/2891340124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f4c103b00 0x7f7f4c19a420 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800e9f0 tx=0x7f7f4800eeb0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.236+0000 7f7f53dc3640 1 -- 172.21.15.64:0/2891340124 shutdown_connections 2024-09-21T04:10:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.236+0000 7f7f53dc3640 1 --2- 172.21.15.64:0/2891340124 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f7f380b6680 0x7f7f380b8b90 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.236+0000 7f7f53dc3640 1 --2- 172.21.15.64:0/2891340124 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7f4c109d80 0x7f7f4c19fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.236+0000 7f7f53dc3640 1 --2- 172.21.15.64:0/2891340124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f4c109430 0x7f7f4c19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.236+0000 7f7f53dc3640 1 --2- 172.21.15.64:0/2891340124 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7f4c103b00 0x7f7f4c19a420 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.236+0000 7f7f53dc3640 1 -- 172.21.15.64:0/2891340124 >> 172.21.15.64:0/2891340124 conn(0x7f7f4c0ff500 msgr2=0x7f7f4c108a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:39.240 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.236+0000 7f7f53dc3640 1 -- 172.21.15.64:0/2891340124 shutdown_connections 2024-09-21T04:10:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:39.236+0000 7f7f53dc3640 1 -- 172.21.15.64:0/2891340124 wait complete. 2024-09-21T04:10:40.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:40 smithi119 ceph-mon[34811]: pgmap v522: 97 pgs: 97 active+clean; 797 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.1 MiB/s wr, 479 op/s 2024-09-21T04:10:40.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:40 smithi064 ceph-mon[31180]: pgmap v522: 97 pgs: 97 active+clean; 797 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.1 MiB/s wr, 479 op/s 2024-09-21T04:10:40.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:40 smithi131 ceph-mon[35193]: pgmap v522: 97 pgs: 97 active+clean; 797 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.1 MiB/s wr, 479 op/s 2024-09-21T04:10:42.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:42 smithi119 ceph-mon[34811]: pgmap v523: 97 pgs: 97 active+clean; 797 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 493 op/s 2024-09-21T04:10:42.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:42 smithi064 ceph-mon[31180]: pgmap v523: 97 pgs: 97 active+clean; 797 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 493 op/s 2024-09-21T04:10:42.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:42 smithi131 ceph-mon[35193]: pgmap v523: 97 pgs: 97 active+clean; 797 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 493 op/s 2024-09-21T04:10:44.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:44 smithi119 ceph-mon[34811]: pgmap v524: 97 pgs: 97 active+clean; 816 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.0 MiB/s wr, 664 op/s 2024-09-21T04:10:44.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:44 smithi064 ceph-mon[31180]: pgmap v524: 97 pgs: 97 active+clean; 816 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.0 MiB/s wr, 664 op/s 2024-09-21T04:10:44.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:44 smithi131 ceph-mon[35193]: pgmap v524: 97 pgs: 97 active+clean; 816 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.0 MiB/s wr, 664 op/s 2024-09-21T04:10:46.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:46 smithi119 ceph-mon[34811]: pgmap v525: 97 pgs: 97 active+clean; 816 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.6 MiB/s wr, 617 op/s 2024-09-21T04:10:46.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:46 smithi064 ceph-mon[31180]: pgmap v525: 97 pgs: 97 active+clean; 816 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.6 MiB/s wr, 617 op/s 2024-09-21T04:10:46.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:46 smithi131 ceph-mon[35193]: pgmap v525: 97 pgs: 97 active+clean; 816 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.6 MiB/s wr, 617 op/s 2024-09-21T04:10:48.366 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T04:10:48.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:48 smithi119 ceph-mon[34811]: pgmap v526: 97 pgs: 97 active+clean; 799 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.3 MiB/s wr, 747 op/s 2024-09-21T04:10:48.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:48 smithi064 ceph-mon[31180]: pgmap v526: 97 pgs: 97 active+clean; 799 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.3 MiB/s wr, 747 op/s 2024-09-21T04:10:48.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:48 smithi131 ceph-mon[35193]: pgmap v526: 97 pgs: 97 active+clean; 799 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.3 MiB/s wr, 747 op/s 2024-09-21T04:10:48.661 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:10:49.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.154+0000 7ff461cf3640 1 -- 172.21.15.64:0/3493899776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff45c102a60 msgr2=0x7ff45c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:49.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.154+0000 7ff461cf3640 1 --2- 172.21.15.64:0/3493899776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff45c102a60 0x7ff45c102e70 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009900 tx=0x7ff44c02a410 comp rx=0 tx=0).stop 2024-09-21T04:10:49.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.154+0000 7ff461cf3640 1 -- 172.21.15.64:0/3493899776 shutdown_connections 2024-09-21T04:10:49.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.154+0000 7ff461cf3640 1 --2- 172.21.15.64:0/3493899776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff45c108cf0 0x7ff45c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.154+0000 7ff461cf3640 1 --2- 172.21.15.64:0/3493899776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c1083a0 0x7ff45c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.154+0000 7ff461cf3640 1 --2- 172.21.15.64:0/3493899776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff45c102a60 0x7ff45c102e70 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.154+0000 7ff461cf3640 1 -- 172.21.15.64:0/3493899776 >> 172.21.15.64:0/3493899776 conn(0x7ff45c079730 msgr2=0x7ff45c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.154+0000 7ff461cf3640 1 -- 172.21.15.64:0/3493899776 shutdown_connections 2024-09-21T04:10:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.154+0000 7ff461cf3640 1 -- 172.21.15.64:0/3493899776 wait complete. 2024-09-21T04:10:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.154+0000 7ff461cf3640 1 Processor -- start 2024-09-21T04:10:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.155+0000 7ff461cf3640 1 -- start start 2024-09-21T04:10:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.155+0000 7ff461cf3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c102a60 0x7ff45c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.155+0000 7ff461cf3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff45c1083a0 0x7ff45c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.155+0000 7ff461cf3640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff45c108cf0 0x7ff45c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.155+0000 7ff461cf3640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c113a80 con 0x7ff45c108cf0 2024-09-21T04:10:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.155+0000 7ff461cf3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c113900 con 0x7ff45c1083a0 2024-09-21T04:10:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.155+0000 7ff461cf3640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c113c00 con 0x7ff45c102a60 2024-09-21T04:10:49.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.155+0000 7ff45b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c102a60 0x7ff45c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.155+0000 7ff45bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff45c108cf0 0x7ff45c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.155+0000 7ff45b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c102a60 0x7ff45c19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:46440/0 (socket says 172.21.15.64:46440) 2024-09-21T04:10:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.155+0000 7ff45b7fe640 1 -- 172.21.15.64:0/3495146868 learned_addr learned my addr 172.21.15.64:0/3495146868 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.155+0000 7ff45affd640 1 --2- 172.21.15.64:0/3495146868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff45c1083a0 0x7ff45c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.156+0000 7ff45bfff640 1 -- 172.21.15.64:0/3495146868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c102a60 msgr2=0x7ff45c19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.156+0000 7ff45bfff640 1 --2- 172.21.15.64:0/3495146868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c102a60 0x7ff45c19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.156+0000 7ff45bfff640 1 -- 172.21.15.64:0/3495146868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff45c1083a0 msgr2=0x7ff45c19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.156+0000 7ff45bfff640 1 --2- 172.21.15.64:0/3495146868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff45c1083a0 0x7ff45c19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.156+0000 7ff45bfff640 1 -- 172.21.15.64:0/3495146868 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45c1a0400 con 0x7ff45c108cf0 2024-09-21T04:10:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.156+0000 7ff45affd640 1 --2- 172.21.15.64:0/3495146868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff45c1083a0 0x7ff45c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.156+0000 7ff45b7fe640 1 --2- 172.21.15.64:0/3495146868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c102a60 0x7ff45c19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.156+0000 7ff45bfff640 1 --2- 172.21.15.64:0/3495146868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff45c108cf0 0x7ff45c19fcd0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff45400ec50 tx=0x7ff45400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.156+0000 7ff458ff9640 1 -- 172.21.15.64:0/3495146868 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454003b70 con 0x7ff45c108cf0 2024-09-21T04:10:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.156+0000 7ff461cf3640 1 -- 172.21.15.64:0/3495146868 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff45c1a06f0 con 0x7ff45c108cf0 2024-09-21T04:10:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.156+0000 7ff458ff9640 1 -- 172.21.15.64:0/3495146868 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff454003d10 con 0x7ff45c108cf0 2024-09-21T04:10:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.156+0000 7ff458ff9640 1 -- 172.21.15.64:0/3495146868 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454010830 con 0x7ff45c108cf0 2024-09-21T04:10:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.156+0000 7ff461cf3640 1 -- 172.21.15.64:0/3495146868 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff45c1a6e50 con 0x7ff45c108cf0 2024-09-21T04:10:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.159+0000 7ff468267740 1 Processor -- start 2024-09-21T04:10:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.160+0000 7ff458ff9640 1 -- 172.21.15.64:0/3495146868 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7ff454003ec0 con 0x7ff45c108cf0 2024-09-21T04:10:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.160+0000 7ff468267740 1 -- start start 2024-09-21T04:10:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.160+0000 7ff468267740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d27d04f070 0x55d27d3f6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.160+0000 7ff468267740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d27d0261b0 0x55d27d3f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.160+0000 7ff468267740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27d02e390 0x55d27d3f70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.160+0000 7ff468267740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d27d36ce50 con 0x55d27d04f070 2024-09-21T04:10:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.160+0000 7ff468267740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d27d35c280 con 0x55d27d02e390 2024-09-21T04:10:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.160+0000 7ff468267740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d27d360090 con 0x55d27d0261b0 2024-09-21T04:10:49.298 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.160+0000 7ff45affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d27d04f070 0x55d27d3f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.160+0000 7ff45affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d27d04f070 0x55d27d3f6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56134/0 (socket says 172.21.15.64:56134) 2024-09-21T04:10:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.160+0000 7ff45affd640 1 -- 172.21.15.64:0/2648975865 learned_addr learned my addr 172.21.15.64:0/2648975865 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.160+0000 7ff45b7fe640 1 --2- 172.21.15.64:0/2648975865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d27d0261b0 0x55d27d3f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.160+0000 7ff45affd640 1 --2- 172.21.15.64:0/2648975865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27d02e390 0x55d27d3f70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.160+0000 7ff45affd640 1 -- 172.21.15.64:0/2648975865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d27d0261b0 msgr2=0x55d27d3f6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.160+0000 7ff45affd640 1 --2- 172.21.15.64:0/2648975865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d27d0261b0 0x55d27d3f6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.160+0000 7ff45affd640 1 -- 172.21.15.64:0/2648975865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27d02e390 msgr2=0x55d27d3f70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.160+0000 7ff45affd640 1 --2- 172.21.15.64:0/2648975865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27d02e390 0x55d27d3f70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.160+0000 7ff45affd640 1 -- 172.21.15.64:0/2648975865 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d27d0ba650 con 0x55d27d04f070 2024-09-21T04:10:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.160+0000 7ff458ff9640 1 --2- 172.21.15.64:0/3495146868 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff4440b6680 0x7ff4440b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.161+0000 7ff45b7fe640 1 --2- 172.21.15.64:0/3495146868 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff4440b6680 0x7ff4440b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.161+0000 7ff45affd640 1 --2- 172.21.15.64:0/2648975865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d27d04f070 0x55d27d3f6650 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ff4500139d0 tx=0x7ff450009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73bd3071c47c15eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.161+0000 7ff458ff9640 1 -- 172.21.15.64:0/3495146868 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7ff4540e7cc0 con 0x7ff45c108cf0 2024-09-21T04:10:49.299 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.161+0000 7ff441ffb640 1 -- 172.21.15.64:0/2648975865 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff450004050 con 0x55d27d04f070 2024-09-21T04:10:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.161+0000 7ff441ffb640 1 -- 172.21.15.64:0/2648975865 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff45000fe30 con 0x55d27d04f070 2024-09-21T04:10:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.161+0000 7ff441ffb640 1 -- 172.21.15.64:0/2648975865 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff450003bb0 con 0x55d27d04f070 2024-09-21T04:10:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.162+0000 7ff468267740 1 -- 172.21.15.64:0/2648975865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d27d04f070 msgr2=0x55d27d3f6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.162+0000 7ff45b7fe640 1 --2- 172.21.15.64:0/3495146868 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff4440b6680 0x7ff4440b8b90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff44c031440 tx=0x7ff44c0058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.162+0000 7ff468267740 1 --2- 172.21.15.64:0/2648975865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d27d04f070 0x55d27d3f6650 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ff4500139d0 tx=0x7ff450009be0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.162+0000 7ff468267740 1 -- 172.21.15.64:0/2648975865 shutdown_connections 2024-09-21T04:10:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.162+0000 7ff468267740 1 --2- 172.21.15.64:0/2648975865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27d02e390 0x55d27d3f70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.162+0000 7ff468267740 1 --2- 172.21.15.64:0/2648975865 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d27d0261b0 0x55d27d3f6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.162+0000 7ff468267740 1 --2- 172.21.15.64:0/2648975865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d27d04f070 0x55d27d3f6650 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.162+0000 7ff468267740 1 -- 172.21.15.64:0/2648975865 >> 172.21.15.64:0/2648975865 conn(0x55d27d043460 msgr2=0x55d27d3f4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.162+0000 7ff468267740 1 -- 172.21.15.64:0/2648975865 shutdown_connections 2024-09-21T04:10:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.162+0000 7ff468267740 1 -- 172.21.15.64:0/2648975865 wait complete. 2024-09-21T04:10:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.162+0000 7ff468267740 1 -- start start 2024-09-21T04:10:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.163+0000 7ff468267740 1 Processor -- start 2024-09-21T04:10:49.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.163+0000 7ff468267740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d27d0261b0 0x55d27d4060e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:49.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.163+0000 7ff468267740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27d02e390 0x55d27d406620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:49.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.163+0000 7ff468267740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d27d04f070 0x55d27d4124c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:49.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.163+0000 7ff468267740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d27d1bd700 con 0x55d27d0261b0 2024-09-21T04:10:49.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.163+0000 7ff468267740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d27d2f7110 con 0x55d27d02e390 2024-09-21T04:10:49.301 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.163+0000 7ff468267740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d27d409190 con 0x55d27d04f070 2024-09-21T04:10:49.301 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:10:49.301 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-21T04:10:49.301 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {}, 2024-09-21T04:10:49.301 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_stats": { 2024-09-21T04:10:49.301 INFO:teuthology.orchestra.run.smithi064.stdout: "2c0523fb-aae4-43fa-9568-136a9b640051": { 2024-09-21T04:10:49.302 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:09:39.808606+0000", 2024-09-21T04:10:49.302 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:10:49.302 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:10:49.302 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:10:49.302 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:10:49.302 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:10:49.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:10:49.302 INFO:teuthology.orchestra.run.smithi064.stdout: "bed40d88-2102-43a6-ac2d-b89d5a2264d0": { 2024-09-21T04:10:49.302 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:11.363340+0000", 2024-09-21T04:10:49.302 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:10:49.302 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:10:49.302 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:10:49.302 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:10:49.302 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:10:49.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:10:49.303 INFO:teuthology.orchestra.run.smithi064.stdout: "e9242f88-6599-40eb-adf1-bdb3e946a1e0": { 2024-09-21T04:10:49.303 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:05:39.744646+0000", 2024-09-21T04:10:49.303 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:10:49.303 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:10:49.303 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:10:49.303 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:10:49.303 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:10:49.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:10:49.303 INFO:teuthology.orchestra.run.smithi064.stdout: "6bde0fe5-f775-44b1-ad0c-f9e596574dc0": { 2024-09-21T04:10:49.303 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:06:23.835536+0000", 2024-09-21T04:10:49.303 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:10:49.303 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:10:49.303 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:10:49.303 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:10:49.304 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:10:49.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:10:49.304 INFO:teuthology.orchestra.run.smithi064.stdout: "b52b950c-0f97-49ec-a37c-1a8e04348ce8": { 2024-09-21T04:10:49.304 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:54.003800+0000", 2024-09-21T04:10:49.304 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:10:49.304 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:10:49.304 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:10:49.304 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:10:49.304 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:10:49.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:10:49.304 INFO:teuthology.orchestra.run.smithi064.stdout: "e2c2eed8-9292-4bf6-8a39-13ad27da5acb": { 2024-09-21T04:10:49.304 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:08:47.067932+0000", 2024-09-21T04:10:49.304 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:10:49.304 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:10:49.305 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:10:49.305 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:10:49.305 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:10:49.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:10:49.305 INFO:teuthology.orchestra.run.smithi064.stdout: "75db1bc5-612e-45aa-bad2-cc90feeaa840": { 2024-09-21T04:10:49.305 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:59.123138+0000", 2024-09-21T04:10:49.305 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:10:49.305 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:10:49.305 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:10:49.305 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:10:49.305 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:10:49.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:10:49.305 INFO:teuthology.orchestra.run.smithi064.stdout: "0ab717a7-36c6-4dca-94e7-af0bdf2aa112": { 2024-09-21T04:10:49.305 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:42.691086+0000", 2024-09-21T04:10:49.305 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:10:49.306 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:10:49.306 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:10:49.306 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:10:49.306 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:10:49.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:10:49.306 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:10:49.306 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:03:59.570174+0000", 2024-09-21T04:10:49.306 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:10:49.306 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:10:49.306 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:10:49.306 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:10:49.306 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:10:49.306 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:10:49.306 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:10:49.306 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:10:49.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.163+0000 7ff468267740 10 client.0 _ll_register_callbacks cb 0x55d27d310990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:10:49.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.163+0000 7ff45b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d27d0261b0 0x55d27d4060e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:49.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.163+0000 7ff45b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d27d0261b0 0x55d27d4060e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56150/0 (socket says 172.21.15.64:56150) 2024-09-21T04:10:49.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.163+0000 7ff45b7fe640 1 -- 172.21.15.64:0/2381172821 learned_addr learned my addr 172.21.15.64:0/2381172821 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:49.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.163+0000 7ff45affd640 1 --2- 172.21.15.64:0/2381172821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27d02e390 0x55d27d406620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:49.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.163+0000 7ff45bfff640 1 --2- 172.21.15.64:0/2381172821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d27d04f070 0x55d27d4124c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:49.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.164+0000 7ff45b7fe640 1 -- 172.21.15.64:0/2381172821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d27d04f070 msgr2=0x55d27d4124c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:49.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.164+0000 7ff45b7fe640 1 --2- 172.21.15.64:0/2381172821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d27d04f070 0x55d27d4124c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.307 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.164+0000 7ff45b7fe640 1 -- 172.21.15.64:0/2381172821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27d02e390 msgr2=0x55d27d406620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:49.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.164+0000 7ff45b7fe640 1 --2- 172.21.15.64:0/2381172821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27d02e390 0x55d27d406620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.164+0000 7ff45b7fe640 1 -- 172.21.15.64:0/2381172821 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d27d324c40 con 0x55d27d0261b0 2024-09-21T04:10:49.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.164+0000 7ff45affd640 1 --2- 172.21.15.64:0/2381172821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27d02e390 0x55d27d406620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:49.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.164+0000 7ff45b7fe640 1 --2- 172.21.15.64:0/2381172821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d27d0261b0 0x55d27d4060e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009a20 tx=0x7ff44c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1a356614ea63c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:49.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.164+0000 7ff43affd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:10:49.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.164+0000 7ff43affd640 1 -- 172.21.15.64:0/2381172821 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c04d070 con 0x55d27d0261b0 2024-09-21T04:10:49.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.164+0000 7ff43affd640 1 -- 172.21.15.64:0/2381172821 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7ff44c003a00 con 0x55d27d0261b0 2024-09-21T04:10:49.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.164+0000 7ff468267740 1 -- 172.21.15.64:0/2381172821 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff45000fe30 con 0x55d27d0261b0 2024-09-21T04:10:49.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.164+0000 7ff43affd640 1 -- 172.21.15.64:0/2381172821 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff44c043070 con 0x55d27d0261b0 2024-09-21T04:10:49.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.164+0000 7ff468267740 10 client.16857 fetch_fsmap learned FSMap version 38 2024-09-21T04:10:49.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.164+0000 7ff468267740 1 -- 172.21.15.64:0/2381172821 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d27d414e80 con 0x55d27d0261b0 2024-09-21T04:10:49.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.164+0000 7ff43affd640 1 -- 172.21.15.64:0/2381172821 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff45000fe30 con 0x55d27d0261b0 2024-09-21T04:10:49.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.165+0000 7ff43affd640 1 -- 172.21.15.64:0/2381172821 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7ff44c03dba0 con 0x55d27d0261b0 2024-09-21T04:10:49.308 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.165+0000 7ff468267740 10 client.16857 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T04:10:49.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.165+0000 7ff468267740 10 client.16857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:10:49.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.165+0000 7ff468267740 20 client.16857 populate_metadata read hostname 'smithi064' 2024-09-21T04:10:49.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.165+0000 7ff468267740 1 --2- 172.21.15.64:0/2381172821 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d27d00d9f0 0x55d27d418430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:49.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.165+0000 7ff468267740 4 client.16857 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:10:49.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.165+0000 7ff468267740 1 -- 172.21.15.64:0/2381172821 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d27d36eda0 con 0x55d27d00d9f0 2024-09-21T04:10:49.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.165+0000 7ff45affd640 1 --2- 172.21.15.64:0/2381172821 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d27d00d9f0 0x55d27d418430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:49.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.166+0000 7ff43affd640 10 client.16857 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:49.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.166+0000 7ff45affd640 1 --2- 172.21.15.64:0/2381172821 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d27d00d9f0 0x55d27d418430 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=632e227546f9c48 server_cookie=31b94cd131dfddb7 in_seq=0 out_seq=0 2024-09-21T04:10:49.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.166+0000 7ff43affd640 1 -- 172.21.15.64:0/2381172821 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d27d36eda0 con 0x55d27d00d9f0 2024-09-21T04:10:49.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.166+0000 7ff43affd640 10 client.16857 handle_command_reply: tid=0 2024-09-21T04:10:49.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.166+0000 7ff468267740 1 client.16857 shutdown 2024-09-21T04:10:49.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.166+0000 7ff468267740 1 -- 172.21.15.64:0/2381172821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d27d0261b0 msgr2=0x55d27d4060e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:49.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.166+0000 7ff468267740 1 --2- 172.21.15.64:0/2381172821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d27d0261b0 0x55d27d4060e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009a20 tx=0x7ff44c009260 comp rx=0 tx=0).stop 2024-09-21T04:10:49.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.166+0000 7ff468267740 1 -- 172.21.15.64:0/2381172821 shutdown_connections 2024-09-21T04:10:49.309 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.166+0000 7ff468267740 1 --2- 172.21.15.64:0/2381172821 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d27d04f070 0x55d27d4124c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.166+0000 7ff468267740 1 --2- 172.21.15.64:0/2381172821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27d02e390 0x55d27d406620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.166+0000 7ff468267740 1 --2- 172.21.15.64:0/2381172821 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d27d00d9f0 0x55d27d418430 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.166+0000 7ff468267740 1 --2- 172.21.15.64:0/2381172821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d27d0261b0 0x55d27d4060e0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.166+0000 7ff43affd640 0 client.16857 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:49.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.166+0000 7ff468267740 1 -- 172.21.15.64:0/2381172821 >> 172.21.15.64:0/2381172821 conn(0x55d27d043460 msgr2=0x55d27d3f2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:49.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.167+0000 7ff468267740 1 -- 172.21.15.64:0/2381172821 shutdown_connections 2024-09-21T04:10:49.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.167+0000 7ff468267740 1 -- 172.21.15.64:0/2381172821 wait complete. 2024-09-21T04:10:49.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.167+0000 7ff468267740 20 client.16857 trim_cache size 0 max 16384 2024-09-21T04:10:49.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.184+0000 7ff468267740 1 Processor -- start 2024-09-21T04:10:49.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.184+0000 7ff468267740 1 -- start start 2024-09-21T04:10:49.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.184+0000 7ff468267740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d27d0261b0 0x55d27d4065a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:49.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.184+0000 7ff468267740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d27d02e390 0x55d27d4123c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:49.310 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.184+0000 7ff468267740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27d04f070 0x55d27d4032d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:49.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.184+0000 7ff468267740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c1aff60 con 0x55d27d0261b0 2024-09-21T04:10:49.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.184+0000 7ff468267740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c1ad7d0 con 0x55d27d04f070 2024-09-21T04:10:49.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.184+0000 7ff468267740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c1ab640 con 0x55d27d02e390 2024-09-21T04:10:49.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.184+0000 7ff45b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d27d0261b0 0x55d27d4065a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:49.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.184+0000 7ff45b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d27d0261b0 0x55d27d4065a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56164/0 (socket says 172.21.15.64:56164) 2024-09-21T04:10:49.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.184+0000 7ff45b7fe640 1 -- 172.21.15.64:0/2319559913 learned_addr learned my addr 172.21.15.64:0/2319559913 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:49.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.184+0000 7ff45affd640 1 --2- 172.21.15.64:0/2319559913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d27d02e390 0x55d27d4123c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:49.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.184+0000 7ff45bfff640 1 --2- 172.21.15.64:0/2319559913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27d04f070 0x55d27d4032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:49.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.185+0000 7ff45b7fe640 1 -- 172.21.15.64:0/2319559913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d27d02e390 msgr2=0x55d27d4123c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:49.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.185+0000 7ff45b7fe640 1 --2- 172.21.15.64:0/2319559913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d27d02e390 0x55d27d4123c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.185+0000 7ff45b7fe640 1 -- 172.21.15.64:0/2319559913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27d04f070 msgr2=0x55d27d4032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:49.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.185+0000 7ff45b7fe640 1 --2- 172.21.15.64:0/2319559913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27d04f070 0x55d27d4032d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.185+0000 7ff45b7fe640 1 -- 172.21.15.64:0/2319559913 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d27d3f4a60 con 0x55d27d0261b0 2024-09-21T04:10:49.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.185+0000 7ff45affd640 1 --2- 172.21.15.64:0/2319559913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d27d02e390 0x55d27d4123c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:49.311 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.185+0000 7ff45b7fe640 1 --2- 172.21.15.64:0/2319559913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d27d0261b0 0x55d27d4065a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ff44c049560 tx=0x7ff44c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abbe8c03a6bd00de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:49.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.185+0000 7ff4417fa640 1 -- 172.21.15.64:0/2319559913 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c04d040 con 0x55d27d0261b0 2024-09-21T04:10:49.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.185+0000 7ff4417fa640 1 -- 172.21.15.64:0/2319559913 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff44c003cb0 con 0x55d27d0261b0 2024-09-21T04:10:49.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.185+0000 7ff4417fa640 1 -- 172.21.15.64:0/2319559913 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c035040 con 0x55d27d0261b0 2024-09-21T04:10:49.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.186+0000 7ff468267740 1 -- 172.21.15.64:0/2319559913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d27d0261b0 msgr2=0x55d27d4065a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:49.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.186+0000 7ff468267740 1 --2- 172.21.15.64:0/2319559913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d27d0261b0 0x55d27d4065a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ff44c049560 tx=0x7ff44c02a410 comp rx=0 tx=0).stop 2024-09-21T04:10:49.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.186+0000 7ff468267740 1 -- 172.21.15.64:0/2319559913 shutdown_connections 2024-09-21T04:10:49.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.186+0000 7ff468267740 1 --2- 172.21.15.64:0/2319559913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27d04f070 0x55d27d4032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.186+0000 7ff468267740 1 --2- 172.21.15.64:0/2319559913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d27d02e390 0x55d27d4123c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.186+0000 7ff468267740 1 --2- 172.21.15.64:0/2319559913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d27d0261b0 0x55d27d4065a0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.186+0000 7ff468267740 1 -- 172.21.15.64:0/2319559913 >> 172.21.15.64:0/2319559913 conn(0x55d27d043460 msgr2=0x55d27d405e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:49.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.186+0000 7ff468267740 1 -- 172.21.15.64:0/2319559913 shutdown_connections 2024-09-21T04:10:49.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.186+0000 7ff468267740 1 -- 172.21.15.64:0/2319559913 wait complete. 2024-09-21T04:10:49.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.186+0000 7ff468267740 1 -- start start 2024-09-21T04:10:49.312 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.186+0000 7ff468267740 1 Processor -- start 2024-09-21T04:10:49.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.187+0000 7ff468267740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27d0261b0 0x55d27d40a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:49.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.187+0000 7ff468267740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d27d02e390 0x55d27d40ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:49.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.187+0000 7ff468267740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d27d04f070 0x55d27d481e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:49.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.187+0000 7ff468267740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c1abc90 con 0x55d27d02e390 2024-09-21T04:10:49.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.187+0000 7ff468267740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c1ad100 con 0x55d27d0261b0 2024-09-21T04:10:49.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.187+0000 7ff468267740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c1ac360 con 0x55d27d04f070 2024-09-21T04:10:49.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.187+0000 7ff468267740 10 client.0 _ll_register_callbacks cb 0x7ff44c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:10:49.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.187+0000 7ff45b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27d0261b0 0x55d27d40a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:49.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.187+0000 7ff45affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d27d02e390 0x55d27d40ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:49.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.187+0000 7ff45b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27d0261b0 0x55d27d40a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:58414/0 (socket says 172.21.15.64:58414) 2024-09-21T04:10:49.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.187+0000 7ff45b7fe640 1 -- 172.21.15.64:0/169583371 learned_addr learned my addr 172.21.15.64:0/169583371 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:49.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.187+0000 7ff45bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d27d04f070 0x55d27d481e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:49.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.187+0000 7ff45affd640 1 -- 172.21.15.64:0/169583371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d27d04f070 msgr2=0x55d27d481e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:49.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.187+0000 7ff45affd640 1 --2- 172.21.15.64:0/169583371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d27d04f070 0x55d27d481e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.313 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.187+0000 7ff45affd640 1 -- 172.21.15.64:0/169583371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27d0261b0 msgr2=0x55d27d40a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:49.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.187+0000 7ff45affd640 1 --2- 172.21.15.64:0/169583371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27d0261b0 0x55d27d40a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.187+0000 7ff45affd640 1 -- 172.21.15.64:0/169583371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d27d40b530 con 0x55d27d02e390 2024-09-21T04:10:49.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.187+0000 7ff45b7fe640 1 --2- 172.21.15.64:0/169583371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27d0261b0 0x55d27d40a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:49.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.187+0000 7ff45affd640 1 --2- 172.21.15.64:0/169583371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d27d02e390 0x55d27d40ac60 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ff450009350 tx=0x7ff4500137a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8ae7ab2ab42864 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:49.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.187+0000 7ff43affd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:10:49.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.188+0000 7ff43affd640 1 -- 172.21.15.64:0/169583371 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4500050c0 con 0x55d27d02e390 2024-09-21T04:10:49.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.188+0000 7ff43affd640 1 -- 172.21.15.64:0/169583371 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7ff450005260 con 0x55d27d02e390 2024-09-21T04:10:49.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.188+0000 7ff468267740 1 -- 172.21.15.64:0/169583371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff44c003cb0 con 0x55d27d02e390 2024-09-21T04:10:49.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.188+0000 7ff43affd640 1 -- 172.21.15.64:0/169583371 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff450039050 con 0x55d27d02e390 2024-09-21T04:10:49.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.188+0000 7ff468267740 10 client.16863 fetch_fsmap learned FSMap version 38 2024-09-21T04:10:49.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.188+0000 7ff468267740 1 -- 172.21.15.64:0/169583371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d27d4833e0 con 0x55d27d02e390 2024-09-21T04:10:49.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.188+0000 7ff43affd640 1 -- 172.21.15.64:0/169583371 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff44c003cb0 con 0x55d27d02e390 2024-09-21T04:10:49.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.188+0000 7ff43affd640 1 -- 172.21.15.64:0/169583371 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7ff45001e070 con 0x55d27d02e390 2024-09-21T04:10:49.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.188+0000 7ff468267740 10 client.16863 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T04:10:49.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.188+0000 7ff468267740 10 client.16863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:10:49.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.188+0000 7ff468267740 20 client.16863 populate_metadata read hostname 'smithi064' 2024-09-21T04:10:49.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.188+0000 7ff468267740 1 --2- 172.21.15.64:0/169583371 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d27cfe2da0 0x55d27d485970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:49.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.188+0000 7ff468267740 4 client.16863 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T04:10:49.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.188+0000 7ff468267740 1 -- 172.21.15.64:0/169583371 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d27d36a850 con 0x55d27cfe2da0 2024-09-21T04:10:49.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.188+0000 7ff45b7fe640 1 --2- 172.21.15.64:0/169583371 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d27cfe2da0 0x55d27d485970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:49.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.189+0000 7ff43affd640 10 client.16863 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:49.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.189+0000 7ff45b7fe640 1 --2- 172.21.15.64:0/169583371 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d27cfe2da0 0x55d27d485970 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16480954cd998a14 server_cookie=f55f89a951135553 in_seq=0 out_seq=0 2024-09-21T04:10:49.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.189+0000 7ff43affd640 1 -- 172.21.15.64:0/169583371 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2646 (crc 0 0 0) 0x55d27d36a850 con 0x55d27cfe2da0 2024-09-21T04:10:49.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.189+0000 7ff43affd640 10 client.16863 handle_command_reply: tid=0 2024-09-21T04:10:49.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.189+0000 7ff468267740 1 client.16863 shutdown 2024-09-21T04:10:49.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.189+0000 7ff468267740 1 -- 172.21.15.64:0/169583371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d27d02e390 msgr2=0x55d27d40ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:49.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.189+0000 7ff468267740 1 --2- 172.21.15.64:0/169583371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d27d02e390 0x55d27d40ac60 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ff450009350 tx=0x7ff4500137a0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.189+0000 7ff468267740 1 -- 172.21.15.64:0/169583371 shutdown_connections 2024-09-21T04:10:49.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.189+0000 7ff468267740 1 --2- 172.21.15.64:0/169583371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d27d04f070 0x55d27d481e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.189+0000 7ff468267740 1 --2- 172.21.15.64:0/169583371 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55d27cfe2da0 0x55d27d485970 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.189+0000 7ff468267740 1 --2- 172.21.15.64:0/169583371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d27d02e390 0x55d27d40ac60 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.189+0000 7ff468267740 1 --2- 172.21.15.64:0/169583371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d27d0261b0 0x55d27d40a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.189+0000 7ff43affd640 0 client.16863 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:49.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.189+0000 7ff468267740 1 -- 172.21.15.64:0/169583371 >> 172.21.15.64:0/169583371 conn(0x55d27d043460 msgr2=0x55d27d3f7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:49.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.189+0000 7ff468267740 1 -- 172.21.15.64:0/169583371 shutdown_connections 2024-09-21T04:10:49.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.189+0000 7ff468267740 1 -- 172.21.15.64:0/169583371 wait complete. 2024-09-21T04:10:49.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.190+0000 7ff468267740 20 client.16863 trim_cache size 0 max 16384 2024-09-21T04:10:49.316 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.190+0000 7ff461cf3640 1 -- 172.21.15.64:0/3495146868 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff4440b6680 msgr2=0x7ff4440b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:49.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.190+0000 7ff461cf3640 1 --2- 172.21.15.64:0/3495146868 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff4440b6680 0x7ff4440b8b90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff44c031440 tx=0x7ff44c0058d0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.190+0000 7ff461cf3640 1 -- 172.21.15.64:0/3495146868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff45c108cf0 msgr2=0x7ff45c19fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:49.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.190+0000 7ff461cf3640 1 --2- 172.21.15.64:0/3495146868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff45c108cf0 0x7ff45c19fcd0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff45400ec50 tx=0x7ff45400c6a0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.190+0000 7ff461cf3640 1 -- 172.21.15.64:0/3495146868 shutdown_connections 2024-09-21T04:10:49.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.190+0000 7ff461cf3640 1 --2- 172.21.15.64:0/3495146868 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff4440b6680 0x7ff4440b8b90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.190+0000 7ff461cf3640 1 --2- 172.21.15.64:0/3495146868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff45c108cf0 0x7ff45c19fcd0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.190+0000 7ff461cf3640 1 --2- 172.21.15.64:0/3495146868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff45c1083a0 0x7ff45c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.190+0000 7ff461cf3640 1 --2- 172.21.15.64:0/3495146868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff45c102a60 0x7ff45c19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:49.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.190+0000 7ff461cf3640 1 -- 172.21.15.64:0/3495146868 >> 172.21.15.64:0/3495146868 conn(0x7ff45c079730 msgr2=0x7ff45c101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:49.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.190+0000 7ff461cf3640 1 -- 172.21.15.64:0/3495146868 shutdown_connections 2024-09-21T04:10:49.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:49.190+0000 7ff461cf3640 1 -- 172.21.15.64:0/3495146868 wait complete. 2024-09-21T04:10:49.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:49 smithi119 ceph-mon[34811]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:10:49.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:49 smithi064 ceph-mon[31180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:10:49.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:49 smithi131 ceph-mon[35193]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:10:50.119 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T04:10:50.120 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:10:50.387 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:10:50.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:50 smithi119 ceph-mon[34811]: pgmap v527: 97 pgs: 97 active+clean; 791 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.3 MiB/s wr, 574 op/s 2024-09-21T04:10:50.510 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:50 smithi064 ceph-mon[31180]: pgmap v527: 97 pgs: 97 active+clean; 791 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.3 MiB/s wr, 574 op/s 2024-09-21T04:10:50.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:50 smithi131 ceph-mon[35193]: pgmap v527: 97 pgs: 97 active+clean; 791 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.3 MiB/s wr, 574 op/s 2024-09-21T04:10:50.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.944+0000 7f698ea92640 1 -- 172.21.15.64:0/4282328624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6988109e00 msgr2=0x7f6988110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:50.955 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.944+0000 7f698ea92640 1 --2- 172.21.15.64:0/4282328624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6988109e00 0x7f6988110630 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6978009ca0 tx=0x7f697802a8c0 comp rx=0 tx=0).stop 2024-09-21T04:10:50.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.944+0000 7f698ea92640 1 -- 172.21.15.64:0/4282328624 shutdown_connections 2024-09-21T04:10:50.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.944+0000 7f698ea92640 1 --2- 172.21.15.64:0/4282328624 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6988109e00 0x7f6988110630 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:50.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.944+0000 7f698ea92640 1 --2- 172.21.15.64:0/4282328624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69881094b0 0x7f69881098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:50.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.944+0000 7f698ea92640 1 --2- 172.21.15.64:0/4282328624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6988103bc0 0x7f6988103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:50.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.944+0000 7f698ea92640 1 -- 172.21.15.64:0/4282328624 >> 172.21.15.64:0/4282328624 conn(0x7f698807b910 msgr2=0x7f698807bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:50.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.944+0000 7f698ea92640 1 -- 172.21.15.64:0/4282328624 shutdown_connections 2024-09-21T04:10:50.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.944+0000 7f698ea92640 1 -- 172.21.15.64:0/4282328624 wait complete. 2024-09-21T04:10:50.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.944+0000 7f698ea92640 1 Processor -- start 2024-09-21T04:10:50.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.945+0000 7f698ea92640 1 -- start start 2024-09-21T04:10:50.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.945+0000 7f698ea92640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6988103bc0 0x7f698819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:50.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.945+0000 7f698ea92640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69881094b0 0x7f698819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:50.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.945+0000 7f698ea92640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6988109e00 0x7f698819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:50.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.945+0000 7f698ea92640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6988113ac0 con 0x7f6988109e00 2024-09-21T04:10:50.956 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.945+0000 7f698ea92640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6988113940 con 0x7f6988103bc0 2024-09-21T04:10:50.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.945+0000 7f698ea92640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6988113c40 con 0x7f69881094b0 2024-09-21T04:10:50.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.945+0000 7f698d02f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6988109e00 0x7f698819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:50.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.945+0000 7f698c82e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6988103bc0 0x7f698819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:50.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.945+0000 7f6987fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69881094b0 0x7f698819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:50.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.945+0000 7f698d02f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6988109e00 0x7f698819fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:56178/0 (socket says 172.21.15.64:56178) 2024-09-21T04:10:50.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.945+0000 7f6987fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69881094b0 0x7f698819a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:46490/0 (socket says 172.21.15.64:46490) 2024-09-21T04:10:50.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.945+0000 7f698c82e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6988103bc0 0x7f698819a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:58442/0 (socket says 172.21.15.64:58442) 2024-09-21T04:10:50.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.945+0000 7f698d02f640 1 -- 172.21.15.64:0/625081276 learned_addr learned my addr 172.21.15.64:0/625081276 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:50.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.945+0000 7f6987fff640 1 -- 172.21.15.64:0/625081276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6988103bc0 msgr2=0x7f698819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:50.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.945+0000 7f6987fff640 1 --2- 172.21.15.64:0/625081276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6988103bc0 0x7f698819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:50.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.945+0000 7f6987fff640 1 -- 172.21.15.64:0/625081276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6988109e00 msgr2=0x7f698819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:50.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.945+0000 7f6987fff640 1 --2- 172.21.15.64:0/625081276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6988109e00 0x7f698819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:50.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.945+0000 7f6987fff640 1 -- 172.21.15.64:0/625081276 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69881a03a0 con 0x7f69881094b0 2024-09-21T04:10:50.957 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.946+0000 7f6987fff640 1 --2- 172.21.15.64:0/625081276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69881094b0 0x7f698819a9d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f698806aa20 tx=0x7f697000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:50.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.946+0000 7f6985ffb640 1 -- 172.21.15.64:0/625081276 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697000cc40 con 0x7f69881094b0 2024-09-21T04:10:50.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.946+0000 7f698ea92640 1 -- 172.21.15.64:0/625081276 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69881a0690 con 0x7f69881094b0 2024-09-21T04:10:50.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.946+0000 7f6985ffb640 1 -- 172.21.15.64:0/625081276 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f697000cde0 con 0x7f69881094b0 2024-09-21T04:10:50.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.946+0000 7f6985ffb640 1 -- 172.21.15.64:0/625081276 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970018760 con 0x7f69881094b0 2024-09-21T04:10:50.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.946+0000 7f698ea92640 1 -- 172.21.15.64:0/625081276 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69881a6df0 con 0x7f69881094b0 2024-09-21T04:10:50.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.947+0000 7f698ea92640 1 -- 172.21.15.64:0/625081276 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6988103fd0 con 0x7f69881094b0 2024-09-21T04:10:50.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.949+0000 7f6985ffb640 1 -- 172.21.15.64:0/625081276 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f6970010a90 con 0x7f69881094b0 2024-09-21T04:10:50.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.949+0000 7f6985ffb640 1 --2- 172.21.15.64:0/625081276 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f69740b6680 0x7f69740b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:50.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.950+0000 7f698c82e640 1 --2- 172.21.15.64:0/625081276 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f69740b6680 0x7f69740b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:50.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.950+0000 7f6985ffb640 1 -- 172.21.15.64:0/625081276 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7f69700e7f30 con 0x7f69881094b0 2024-09-21T04:10:50.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.950+0000 7f698c82e640 1 --2- 172.21.15.64:0/625081276 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f69740b6680 0x7f69740b8b90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6980009bd0 tx=0x7f6980005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:50.958 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:50.953+0000 7f6985ffb640 1 -- 172.21.15.64:0/625081276 <== mon.1 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f69700b27a0 con 0x7f69881094b0 2024-09-21T04:10:51.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:51.223+0000 7f698ea92640 1 -- 172.21.15.64:0/625081276 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f698819b560 con 0x7f69881094b0 2024-09-21T04:10:51.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:51.224+0000 7f6985ffb640 1 -- 172.21.15.64:0/625081276 <== mon.1 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11914 (secure 0 0 0) 0x7f69700b76e0 con 0x7f69881094b0 2024-09-21T04:10:51.227 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:10:51.227 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":38,"btime":"2024-09-21T04:10:48:196775+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14805,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/97995119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":97995119},{"type":"v1","addr":"172.21.15.64:6839","nonce":97995119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/390891068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":390891068},{"type":"v1","addr":"172.21.15.64:6841","nonce":390891068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:02:03.354744+0000","modified":"2024-09-21T04:10:47.547426+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24710,"mds_1":24680,"mds_2":24661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24634":{"gid":24634,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6839/782190424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782190424},{"type":"v1","addr":"172.21.15.131:6839","nonce":782190424}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24680":{"gid":24680,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24710":{"gid":24710,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6839/1319793636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1319793636},{"type":"v1","addr":"172.21.15.119:6839","nonce":1319793636}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24710,"qdb_cluster":[24661,24680,24710]},"id":1}]} 2024-09-21T04:10:51.227 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 38 2024-09-21T04:10:51.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:51.226+0000 7f698ea92640 1 -- 172.21.15.64:0/625081276 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f69740b6680 msgr2=0x7f69740b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:51.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:51.226+0000 7f698ea92640 1 --2- 172.21.15.64:0/625081276 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f69740b6680 0x7f69740b8b90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6980009bd0 tx=0x7f6980005f90 comp rx=0 tx=0).stop 2024-09-21T04:10:51.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:51.226+0000 7f698ea92640 1 -- 172.21.15.64:0/625081276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69881094b0 msgr2=0x7f698819a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:51.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:51.226+0000 7f698ea92640 1 --2- 172.21.15.64:0/625081276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69881094b0 0x7f698819a9d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f698806aa20 tx=0x7f697000ee50 comp rx=0 tx=0).stop 2024-09-21T04:10:51.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:51.226+0000 7f698ea92640 1 -- 172.21.15.64:0/625081276 shutdown_connections 2024-09-21T04:10:51.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:51.226+0000 7f698ea92640 1 --2- 172.21.15.64:0/625081276 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f69740b6680 0x7f69740b8b90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:51.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:51.226+0000 7f698ea92640 1 --2- 172.21.15.64:0/625081276 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6988109e00 0x7f698819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:51.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:51.226+0000 7f698ea92640 1 --2- 172.21.15.64:0/625081276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69881094b0 0x7f698819a9d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:51.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:51.226+0000 7f698ea92640 1 --2- 172.21.15.64:0/625081276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6988103bc0 0x7f698819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:51.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:51.226+0000 7f698ea92640 1 -- 172.21.15.64:0/625081276 >> 172.21.15.64:0/625081276 conn(0x7f698807b910 msgr2=0x7f6988107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:51.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:51.226+0000 7f698ea92640 1 -- 172.21.15.64:0/625081276 shutdown_connections 2024-09-21T04:10:51.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:51.226+0000 7f698ea92640 1 -- 172.21.15.64:0/625081276 wait complete. 2024-09-21T04:10:51.930 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T04:10:52.186 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:10:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.336+0000 7f794f815640 1 -- 172.21.15.64:0/2348257622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7948108c30 msgr2=0x7f794810f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.336+0000 7f794f815640 1 --2- 172.21.15.64:0/2348257622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7948108c30 0x7f794810f460 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7940009ca0 tx=0x7f794002a8c0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.336+0000 7f794f815640 1 -- 172.21.15.64:0/2348257622 shutdown_connections 2024-09-21T04:10:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.336+0000 7f794f815640 1 --2- 172.21.15.64:0/2348257622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7948108c30 0x7f794810f460 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.336+0000 7f794f815640 1 --2- 172.21.15.64:0/2348257622 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79481082e0 0x7f79481086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.336+0000 7f794f815640 1 --2- 172.21.15.64:0/2348257622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79481029b0 0x7f7948102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.336+0000 7f794f815640 1 -- 172.21.15.64:0/2348257622 >> 172.21.15.64:0/2348257622 conn(0x7f794807a740 msgr2=0x7f794807ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.337+0000 7f794f815640 1 -- 172.21.15.64:0/2348257622 shutdown_connections 2024-09-21T04:10:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.337+0000 7f794f815640 1 -- 172.21.15.64:0/2348257622 wait complete. 2024-09-21T04:10:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.337+0000 7f794f815640 1 Processor -- start 2024-09-21T04:10:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.337+0000 7f794f815640 1 -- start start 2024-09-21T04:10:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.338+0000 7f794f815640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79481029b0 0x7f79481992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.338+0000 7f794f815640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79481082e0 0x7f7948199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.338+0000 7f794f815640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7948108c30 0x7f794819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.338+0000 7f794f815640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f79481128f0 con 0x7f79481029b0 2024-09-21T04:10:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.338+0000 7f794f815640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7948112770 con 0x7f7948108c30 2024-09-21T04:10:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.338+0000 7f794f815640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7948112a70 con 0x7f79481082e0 2024-09-21T04:10:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.338+0000 7f794d5b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79481029b0 0x7f79481992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.338+0000 7f794d5b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79481029b0 0x7f79481992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35866/0 (socket says 172.21.15.64:35866) 2024-09-21T04:10:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.338+0000 7f794d5b1640 1 -- 172.21.15.64:0/1672922604 learned_addr learned my addr 172.21.15.64:0/1672922604 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.338+0000 7f794cdb0640 1 --2- 172.21.15.64:0/1672922604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79481082e0 0x7f7948199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.338+0000 7f794ddb2640 1 --2- 172.21.15.64:0/1672922604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7948108c30 0x7f794819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.338+0000 7f794d5b1640 1 -- 172.21.15.64:0/1672922604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79481082e0 msgr2=0x7f7948199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.338+0000 7f794d5b1640 1 --2- 172.21.15.64:0/1672922604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79481082e0 0x7f7948199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.338+0000 7f794d5b1640 1 -- 172.21.15.64:0/1672922604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7948108c30 msgr2=0x7f794819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.338+0000 7f794d5b1640 1 --2- 172.21.15.64:0/1672922604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7948108c30 0x7f794819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.338+0000 7f794d5b1640 1 -- 172.21.15.64:0/1672922604 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794819f200 con 0x7f79481029b0 2024-09-21T04:10:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.338+0000 7f794d5b1640 1 --2- 172.21.15.64:0/1672922604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79481029b0 0x7f79481992d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f794400e9f0 tx=0x7f794400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.338+0000 7f793e7fc640 1 -- 172.21.15.64:0/1672922604 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794400cd80 con 0x7f79481029b0 2024-09-21T04:10:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.339+0000 7f793e7fc640 1 -- 172.21.15.64:0/1672922604 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79440108e0 con 0x7f79481029b0 2024-09-21T04:10:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.339+0000 7f793e7fc640 1 -- 172.21.15.64:0/1672922604 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7944018980 con 0x7f79481029b0 2024-09-21T04:10:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.339+0000 7f794f815640 1 -- 172.21.15.64:0/1672922604 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f794819f4f0 con 0x7f79481029b0 2024-09-21T04:10:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.339+0000 7f794f815640 1 -- 172.21.15.64:0/1672922604 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79481a5c80 con 0x7f79481029b0 2024-09-21T04:10:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.342+0000 7f793e7fc640 1 -- 172.21.15.64:0/1672922604 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f7944010430 con 0x7f79481029b0 2024-09-21T04:10:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.342+0000 7f7956861740 1 Processor -- start 2024-09-21T04:10:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.342+0000 7f7956861740 1 -- start start 2024-09-21T04:10:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.342+0000 7f7956861740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599bfd6b1b0 0x5599c01134b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.342+0000 7f7956861740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599bfd422f0 0x5599c01109b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.343+0000 7f7956861740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5599bfd4a4d0 0x5599c01110f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.343+0000 7f7956861740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5599c00856f0 con 0x5599bfd422f0 2024-09-21T04:10:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.343+0000 7f7956861740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5599c0074b20 con 0x5599bfd4a4d0 2024-09-21T04:10:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.343+0000 7f7956861740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5599c0078930 con 0x5599bfd6b1b0 2024-09-21T04:10:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.343+0000 7f793e7fc640 1 --2- 172.21.15.64:0/1672922604 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f79340b6700 0x7f79340b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.343+0000 7f794d5b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599bfd422f0 0x5599c01109b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.343+0000 7f794ddb2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599bfd6b1b0 0x5599c01134b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.343+0000 7f794cdb0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5599bfd4a4d0 0x5599c01110f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.343+0000 7f794ddb2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599bfd6b1b0 0x5599c01134b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:46792/0 (socket says 172.21.15.64:46792) 2024-09-21T04:10:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.343+0000 7f794ddb2640 1 -- 172.21.15.64:0/715028413 learned_addr learned my addr 172.21.15.64:0/715028413 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.343+0000 7f794ddb2640 1 --2- 172.21.15.64:0/1672922604 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f79340b6700 0x7f79340b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.343+0000 7f793e7fc640 1 -- 172.21.15.64:0/1672922604 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7f79440e7a30 con 0x7f79481029b0 2024-09-21T04:10:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.343+0000 7f794cdb0640 1 -- 172.21.15.64:0/715028413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599bfd6b1b0 msgr2=0x5599c01134b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.343+0000 7f794cdb0640 1 --2- 172.21.15.64:0/715028413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599bfd6b1b0 0x5599c01134b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.343+0000 7f794cdb0640 1 -- 172.21.15.64:0/715028413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599bfd422f0 msgr2=0x5599c01109b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.343+0000 7f794cdb0640 1 --2- 172.21.15.64:0/715028413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599bfd422f0 0x5599c01109b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.343+0000 7f794cdb0640 1 -- 172.21.15.64:0/715028413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599c003a530 con 0x5599bfd4a4d0 2024-09-21T04:10:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.343+0000 7f794ddb2640 1 --2- 172.21.15.64:0/1672922604 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f79340b6700 0x7f79340b8c10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7940009a60 tx=0x7f79400095c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.343+0000 7f794cdb0640 1 --2- 172.21.15.64:0/715028413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5599bfd4a4d0 0x5599c01110f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f794806c730 tx=0x7f7938011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec100785f07f8303 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.344+0000 7f79337fe640 1 -- 172.21.15.64:0/715028413 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7938004050 con 0x5599bfd4a4d0 2024-09-21T04:10:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.344+0000 7f79337fe640 1 -- 172.21.15.64:0/715028413 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79380054d0 con 0x5599bfd4a4d0 2024-09-21T04:10:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.344+0000 7f79337fe640 1 -- 172.21.15.64:0/715028413 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7938004c40 con 0x5599bfd4a4d0 2024-09-21T04:10:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.344+0000 7f7956861740 1 -- 172.21.15.64:0/715028413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5599bfd4a4d0 msgr2=0x5599c01110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.344+0000 7f7956861740 1 --2- 172.21.15.64:0/715028413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5599bfd4a4d0 0x5599c01110f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f794806c730 tx=0x7f7938011810 comp rx=0 tx=0).stop 2024-09-21T04:10:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.345+0000 7f7956861740 1 -- 172.21.15.64:0/715028413 shutdown_connections 2024-09-21T04:10:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.345+0000 7f7956861740 1 --2- 172.21.15.64:0/715028413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5599bfd4a4d0 0x5599c01110f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.345+0000 7f7956861740 1 --2- 172.21.15.64:0/715028413 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599bfd422f0 0x5599c01109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.345+0000 7f7956861740 1 --2- 172.21.15.64:0/715028413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599bfd6b1b0 0x5599c01134b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.345+0000 7f7956861740 1 -- 172.21.15.64:0/715028413 >> 172.21.15.64:0/715028413 conn(0x5599bfd5f300 msgr2=0x5599c010e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.345+0000 7f7956861740 1 -- 172.21.15.64:0/715028413 shutdown_connections 2024-09-21T04:10:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.345+0000 7f7956861740 1 -- 172.21.15.64:0/715028413 wait complete. 2024-09-21T04:10:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.345+0000 7f7956861740 1 -- start start 2024-09-21T04:10:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.345+0000 7f7956861740 1 Processor -- start 2024-09-21T04:10:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.345+0000 7f7956861740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599bfd422f0 0x5599c0120590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.345+0000 7f7956861740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5599bfd4a4d0 0x5599c0120ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.345+0000 7f7956861740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599bfd6b1b0 0x5599c0121010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.346+0000 7f7956861740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5599bffacab0 con 0x5599bfd422f0 2024-09-21T04:10:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.346+0000 7f7956861740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5599c001a730 con 0x5599bfd4a4d0 2024-09-21T04:10:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.346+0000 7f7956861740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5599c0123570 con 0x5599bfd6b1b0 2024-09-21T04:10:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.346+0000 7f7956861740 10 client.0 _ll_register_callbacks cb 0x5599c006e6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:10:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.346+0000 7f794d5b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599bfd422f0 0x5599c0120590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.346+0000 7f794d5b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599bfd422f0 0x5599c0120590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35880/0 (socket says 172.21.15.64:35880) 2024-09-21T04:10:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.346+0000 7f794d5b1640 1 -- 172.21.15.64:0/2349843249 learned_addr learned my addr 172.21.15.64:0/2349843249 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.346+0000 7f794cdb0640 1 --2- 172.21.15.64:0/2349843249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5599bfd4a4d0 0x5599c0120ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.346+0000 7f794ddb2640 1 --2- 172.21.15.64:0/2349843249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599bfd6b1b0 0x5599c0121010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.346+0000 7f794d5b1640 1 -- 172.21.15.64:0/2349843249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599bfd6b1b0 msgr2=0x5599c0121010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.346+0000 7f794d5b1640 1 --2- 172.21.15.64:0/2349843249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599bfd6b1b0 0x5599c0121010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.346+0000 7f794d5b1640 1 -- 172.21.15.64:0/2349843249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5599bfd4a4d0 msgr2=0x5599c0120ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.346+0000 7f794d5b1640 1 --2- 172.21.15.64:0/2349843249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5599bfd4a4d0 0x5599c0120ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.346+0000 7f794d5b1640 1 -- 172.21.15.64:0/2349843249 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599bfdd6790 con 0x5599bfd422f0 2024-09-21T04:10:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.346+0000 7f794cdb0640 1 --2- 172.21.15.64:0/2349843249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5599bfd4a4d0 0x5599c0120ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.346+0000 7f794d5b1640 1 --2- 172.21.15.64:0/2349843249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599bfd422f0 0x5599c0120590 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f794401af50 tx=0x7f7944000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d83ea82095c627 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.346+0000 7f7930ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:10:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.346+0000 7f7930ff9640 1 -- 172.21.15.64:0/2349843249 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7944026040 con 0x5599bfd422f0 2024-09-21T04:10:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.346+0000 7f7956861740 1 -- 172.21.15.64:0/2349843249 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79380054d0 con 0x5599bfd422f0 2024-09-21T04:10:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.346+0000 7f7930ff9640 1 -- 172.21.15.64:0/2349843249 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7f79440b69b0 con 0x5599bfd422f0 2024-09-21T04:10:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.346+0000 7f7930ff9640 1 -- 172.21.15.64:0/2349843249 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f79440aaf40 con 0x5599bfd422f0 2024-09-21T04:10:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.346+0000 7f7956861740 10 client.16881 fetch_fsmap learned FSMap version 38 2024-09-21T04:10:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.347+0000 7f7956861740 1 -- 172.21.15.64:0/2349843249 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5599bfe4f620 con 0x5599bfd422f0 2024-09-21T04:10:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.347+0000 7f7930ff9640 1 -- 172.21.15.64:0/2349843249 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79380054d0 con 0x5599bfd422f0 2024-09-21T04:10:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.347+0000 7f7930ff9640 1 -- 172.21.15.64:0/2349843249 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f794407ace0 con 0x5599bfd422f0 2024-09-21T04:10:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.347+0000 7f7956861740 10 client.16881 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T04:10:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.347+0000 7f7956861740 10 client.16881 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:10:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.347+0000 7f7956861740 20 client.16881 populate_metadata read hostname 'smithi064' 2024-09-21T04:10:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.347+0000 7f7956861740 1 --2- 172.21.15.64:0/2349843249 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5599bfd29d80 0x5599c01325d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.347+0000 7f7956861740 4 client.16881 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:10:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.347+0000 7f7956861740 1 -- 172.21.15.64:0/2349843249 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599c0087640 con 0x5599bfd29d80 2024-09-21T04:10:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.347+0000 7f794cdb0640 1 --2- 172.21.15.64:0/2349843249 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5599bfd29d80 0x5599c01325d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.348+0000 7f794cdb0640 1 --2- 172.21.15.64:0/2349843249 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5599bfd29d80 0x5599c01325d0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75e7d5dbb77526c8 server_cookie=33fe0a8986d420e0 in_seq=0 out_seq=0 2024-09-21T04:10:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.348+0000 7f7930ff9640 10 client.16881 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.348+0000 7f7930ff9640 1 -- 172.21.15.64:0/2349843249 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5599c0087640 con 0x5599bfd29d80 2024-09-21T04:10:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.348+0000 7f7930ff9640 10 client.16881 handle_command_reply: tid=0 2024-09-21T04:10:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.348+0000 7f7956861740 1 client.16881 shutdown 2024-09-21T04:10:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.349+0000 7f7956861740 1 -- 172.21.15.64:0/2349843249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599bfd422f0 msgr2=0x5599c0120590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.349+0000 7f7956861740 1 --2- 172.21.15.64:0/2349843249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599bfd422f0 0x5599c0120590 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f794401af50 tx=0x7f7944000bc0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.349+0000 7f7956861740 1 -- 172.21.15.64:0/2349843249 shutdown_connections 2024-09-21T04:10:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.349+0000 7f7956861740 1 --2- 172.21.15.64:0/2349843249 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599bfd6b1b0 0x5599c0121010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.349+0000 7f7956861740 1 --2- 172.21.15.64:0/2349843249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5599bfd4a4d0 0x5599c0120ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.349+0000 7f7956861740 1 --2- 172.21.15.64:0/2349843249 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5599bfd29d80 0x5599c01325d0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.349+0000 7f7956861740 1 --2- 172.21.15.64:0/2349843249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599bfd422f0 0x5599c0120590 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.349+0000 7f7930ff9640 0 client.16881 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.349+0000 7f7956861740 1 -- 172.21.15.64:0/2349843249 >> 172.21.15.64:0/2349843249 conn(0x5599bfd5f300 msgr2=0x5599c01123a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.349+0000 7f7956861740 1 -- 172.21.15.64:0/2349843249 shutdown_connections 2024-09-21T04:10:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.349+0000 7f7956861740 1 -- 172.21.15.64:0/2349843249 wait complete. 2024-09-21T04:10:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.349+0000 7f7956861740 20 client.16881 trim_cache size 0 max 16384 2024-09-21T04:10:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.363+0000 7f7956861740 1 Processor -- start 2024-09-21T04:10:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.364+0000 7f7956861740 1 -- start start 2024-09-21T04:10:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.364+0000 7f7956861740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599bfd422f0 0x5599c0120a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.364+0000 7f7956861740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5599bfd4a4d0 0x5599c0120fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.364+0000 7f7956861740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599bfd6b1b0 0x5599c010f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.364+0000 7f7956861740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f79481afe90 con 0x5599bfd422f0 2024-09-21T04:10:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.364+0000 7f7956861740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f79481ad700 con 0x5599bfd4a4d0 2024-09-21T04:10:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.364+0000 7f7956861740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f79481ab570 con 0x5599bfd6b1b0 2024-09-21T04:10:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.364+0000 7f794d5b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599bfd422f0 0x5599c0120a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.364+0000 7f794d5b1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599bfd422f0 0x5599c0120a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35894/0 (socket says 172.21.15.64:35894) 2024-09-21T04:10:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.364+0000 7f794d5b1640 1 -- 172.21.15.64:0/4250367147 learned_addr learned my addr 172.21.15.64:0/4250367147 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.364+0000 7f794ddb2640 1 --2- 172.21.15.64:0/4250367147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599bfd6b1b0 0x5599c010f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.364+0000 7f794cdb0640 1 --2- 172.21.15.64:0/4250367147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5599bfd4a4d0 0x5599c0120fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.364+0000 7f794d5b1640 1 -- 172.21.15.64:0/4250367147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599bfd6b1b0 msgr2=0x5599c010f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.364+0000 7f794d5b1640 1 --2- 172.21.15.64:0/4250367147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599bfd6b1b0 0x5599c010f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.364+0000 7f794d5b1640 1 -- 172.21.15.64:0/4250367147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5599bfd4a4d0 msgr2=0x5599c0120fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.364+0000 7f794d5b1640 1 --2- 172.21.15.64:0/4250367147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5599bfd4a4d0 0x5599c0120fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.364+0000 7f794d5b1640 1 -- 172.21.15.64:0/4250367147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599c0122b70 con 0x5599bfd422f0 2024-09-21T04:10:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.364+0000 7f794d5b1640 1 --2- 172.21.15.64:0/4250367147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599bfd422f0 0x5599c0120a70 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f794400e9c0 tx=0x7f7944018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd8e1a0f228536f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.365+0000 7f7932ffd640 1 -- 172.21.15.64:0/4250367147 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79440aad60 con 0x5599bfd422f0 2024-09-21T04:10:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.365+0000 7f7932ffd640 1 -- 172.21.15.64:0/4250367147 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7944010720 con 0x5599bfd422f0 2024-09-21T04:10:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.365+0000 7f7932ffd640 1 -- 172.21.15.64:0/4250367147 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79440b6b40 con 0x5599bfd422f0 2024-09-21T04:10:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.365+0000 7f7956861740 1 -- 172.21.15.64:0/4250367147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599bfd422f0 msgr2=0x5599c0120a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.365+0000 7f7956861740 1 --2- 172.21.15.64:0/4250367147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599bfd422f0 0x5599c0120a70 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f794400e9c0 tx=0x7f7944018420 comp rx=0 tx=0).stop 2024-09-21T04:10:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.365+0000 7f7956861740 1 -- 172.21.15.64:0/4250367147 shutdown_connections 2024-09-21T04:10:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.365+0000 7f7956861740 1 --2- 172.21.15.64:0/4250367147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599bfd6b1b0 0x5599c010f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.365+0000 7f7956861740 1 --2- 172.21.15.64:0/4250367147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5599bfd4a4d0 0x5599c0120fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.365+0000 7f7956861740 1 --2- 172.21.15.64:0/4250367147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599bfd422f0 0x5599c0120a70 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.365+0000 7f7956861740 1 -- 172.21.15.64:0/4250367147 >> 172.21.15.64:0/4250367147 conn(0x5599bfd5f300 msgr2=0x5599c011edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.365+0000 7f7956861740 1 -- 172.21.15.64:0/4250367147 shutdown_connections 2024-09-21T04:10:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.365+0000 7f7956861740 1 -- 172.21.15.64:0/4250367147 wait complete. 2024-09-21T04:10:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.366+0000 7f7956861740 1 -- start start 2024-09-21T04:10:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.366+0000 7f7956861740 1 Processor -- start 2024-09-21T04:10:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.366+0000 7f7956861740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5599bfd422f0 0x5599c0129160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.366+0000 7f7956861740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599bfd4a4d0 0x5599c0137b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.366+0000 7f7956861740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599bfd6b1b0 0x5599c0138260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.366+0000 7f7956861740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f79481abbc0 con 0x5599bfd4a4d0 2024-09-21T04:10:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.366+0000 7f7956861740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f79481ad030 con 0x5599bfd422f0 2024-09-21T04:10:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.366+0000 7f7956861740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f79481ac290 con 0x5599bfd6b1b0 2024-09-21T04:10:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.367+0000 7f7956861740 10 client.0 _ll_register_callbacks cb 0x7f79440a9fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:10:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.367+0000 7f794cdb0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599bfd4a4d0 0x5599c0137b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.367+0000 7f794d5b1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5599bfd422f0 0x5599c0129160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.367+0000 7f794ddb2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599bfd6b1b0 0x5599c0138260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.367+0000 7f794cdb0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599bfd4a4d0 0x5599c0137b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35902/0 (socket says 172.21.15.64:35902) 2024-09-21T04:10:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.367+0000 7f794cdb0640 1 -- 172.21.15.64:0/1872526345 learned_addr learned my addr 172.21.15.64:0/1872526345 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.367+0000 7f794cdb0640 1 -- 172.21.15.64:0/1872526345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599bfd6b1b0 msgr2=0x5599c0138260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.367+0000 7f794cdb0640 1 --2- 172.21.15.64:0/1872526345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599bfd6b1b0 0x5599c0138260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.367+0000 7f794cdb0640 1 -- 172.21.15.64:0/1872526345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5599bfd422f0 msgr2=0x5599c0129160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.367+0000 7f794cdb0640 1 --2- 172.21.15.64:0/1872526345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5599bfd422f0 0x5599c0129160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.367+0000 7f794cdb0640 1 -- 172.21.15.64:0/1872526345 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599c019bfd0 con 0x5599bfd4a4d0 2024-09-21T04:10:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.367+0000 7f794d5b1640 1 --2- 172.21.15.64:0/1872526345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5599bfd422f0 0x5599c0129160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.367+0000 7f794cdb0640 1 --2- 172.21.15.64:0/1872526345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599bfd4a4d0 0x5599c0137b70 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f7938005fd0 tx=0x7f7938009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3c1814f3b2772e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.367+0000 7f7930ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:10:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.367+0000 7f7930ff9640 1 -- 172.21.15.64:0/1872526345 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793801d040 con 0x5599bfd4a4d0 2024-09-21T04:10:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.367+0000 7f7930ff9640 1 -- 172.21.15.64:0/1872526345 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7f79380035f0 con 0x5599bfd4a4d0 2024-09-21T04:10:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.367+0000 7f7956861740 1 -- 172.21.15.64:0/1872526345 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5599c00830f0 con 0x5599bfd4a4d0 2024-09-21T04:10:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.368+0000 7f7930ff9640 1 -- 172.21.15.64:0/1872526345 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f793803c050 con 0x5599bfd4a4d0 2024-09-21T04:10:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.368+0000 7f7956861740 10 client.16887 fetch_fsmap learned FSMap version 38 2024-09-21T04:10:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.368+0000 7f7956861740 1 -- 172.21.15.64:0/1872526345 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5599c019d0b0 con 0x5599bfd4a4d0 2024-09-21T04:10:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.368+0000 7f7930ff9640 1 -- 172.21.15.64:0/1872526345 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5599c00830f0 con 0x5599bfd4a4d0 2024-09-21T04:10:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.368+0000 7f7930ff9640 1 -- 172.21.15.64:0/1872526345 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f7938037030 con 0x5599bfd4a4d0 2024-09-21T04:10:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.368+0000 7f7956861740 10 client.16887 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T04:10:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.368+0000 7f7956861740 10 client.16887 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:10:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.368+0000 7f7956861740 20 client.16887 populate_metadata read hostname 'smithi064' 2024-09-21T04:10:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.368+0000 7f7956861740 1 --2- 172.21.15.64:0/1872526345 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5599bfcff1c0 0x5599c019f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.368+0000 7f7956861740 4 client.16887 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:10:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.368+0000 7f7956861740 1 -- 172.21.15.64:0/1872526345 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5599c0080520 con 0x5599bfcff1c0 2024-09-21T04:10:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.368+0000 7f794d5b1640 1 --2- 172.21.15.64:0/1872526345 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5599bfcff1c0 0x5599c019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.369+0000 7f794d5b1640 1 --2- 172.21.15.64:0/1872526345 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5599bfcff1c0 0x5599c019f760 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd3fc84657efcdc9 server_cookie=aac003339d2703b9 in_seq=0 out_seq=0 2024-09-21T04:10:52.375 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.369+0000 7f7930ff9640 10 client.16887 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.391+0000 7f7930ff9640 1 -- 172.21.15.64:0/1872526345 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5599c0080520 con 0x5599bfcff1c0 2024-09-21T04:10:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.391+0000 7f7930ff9640 10 client.16887 handle_command_reply: tid=0 2024-09-21T04:10:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.391+0000 7f7956861740 1 client.16887 shutdown 2024-09-21T04:10:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.391+0000 7f7956861740 1 -- 172.21.15.64:0/1872526345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599bfd4a4d0 msgr2=0x5599c0137b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.391+0000 7f7956861740 1 --2- 172.21.15.64:0/1872526345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599bfd4a4d0 0x5599c0137b70 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f7938005fd0 tx=0x7f7938009b80 comp rx=0 tx=0).stop 2024-09-21T04:10:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.391+0000 7f7956861740 1 -- 172.21.15.64:0/1872526345 shutdown_connections 2024-09-21T04:10:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.391+0000 7f7956861740 1 --2- 172.21.15.64:0/1872526345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5599bfd6b1b0 0x5599c0138260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.391+0000 7f7956861740 1 --2- 172.21.15.64:0/1872526345 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5599bfcff1c0 0x5599c019f760 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.391+0000 7f7956861740 1 --2- 172.21.15.64:0/1872526345 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5599bfd4a4d0 0x5599c0137b70 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.391+0000 7f7956861740 1 --2- 172.21.15.64:0/1872526345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5599bfd422f0 0x5599c0129160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.391+0000 7f7930ff9640 0 client.16887 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.391+0000 7f7956861740 1 -- 172.21.15.64:0/1872526345 >> 172.21.15.64:0/1872526345 conn(0x5599bfd5f300 msgr2=0x5599c01181c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.391+0000 7f7956861740 1 -- 172.21.15.64:0/1872526345 shutdown_connections 2024-09-21T04:10:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.391+0000 7f7956861740 1 -- 172.21.15.64:0/1872526345 wait complete. 2024-09-21T04:10:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.391+0000 7f7956861740 20 client.16887 trim_cache size 0 max 16384 2024-09-21T04:10:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:10:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:10:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:10:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:10:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.392+0000 7f794f815640 1 -- 172.21.15.64:0/1672922604 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f79340b6700 msgr2=0x7f79340b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.392+0000 7f794f815640 1 --2- 172.21.15.64:0/1672922604 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f79340b6700 0x7f79340b8c10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7940009a60 tx=0x7f79400095c0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.392+0000 7f794f815640 1 -- 172.21.15.64:0/1672922604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79481029b0 msgr2=0x7f79481992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.392+0000 7f794f815640 1 --2- 172.21.15.64:0/1672922604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79481029b0 0x7f79481992d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f794400e9f0 tx=0x7f794400eeb0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.392+0000 7f794f815640 1 -- 172.21.15.64:0/1672922604 shutdown_connections 2024-09-21T04:10:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.392+0000 7f794f815640 1 --2- 172.21.15.64:0/1672922604 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f79340b6700 0x7f79340b8c10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.392+0000 7f794f815640 1 --2- 172.21.15.64:0/1672922604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7948108c30 0x7f794819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.392+0000 7f794f815640 1 --2- 172.21.15.64:0/1672922604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79481082e0 0x7f7948199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.392+0000 7f794f815640 1 --2- 172.21.15.64:0/1672922604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79481029b0 0x7f79481992d0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.392+0000 7f794f815640 1 -- 172.21.15.64:0/1672922604 >> 172.21.15.64:0/1672922604 conn(0x7f794807a740 msgr2=0x7f7948106330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.392+0000 7f794f815640 1 -- 172.21.15.64:0/1672922604 shutdown_connections 2024-09-21T04:10:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:10:52.392+0000 7f794f815640 1 -- 172.21.15.64:0/1672922604 wait complete. 2024-09-21T04:10:52.475 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:52 smithi064 ceph-mon[31180]: pgmap v528: 97 pgs: 97 active+clean; 793 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 477 op/s 2024-09-21T04:10:52.475 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:52 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/625081276' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:10:52.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:52 smithi119 ceph-mon[34811]: pgmap v528: 97 pgs: 97 active+clean; 793 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 477 op/s 2024-09-21T04:10:52.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:52 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/625081276' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:10:52.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:52 smithi131 ceph-mon[35193]: pgmap v528: 97 pgs: 97 active+clean; 793 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 477 op/s 2024-09-21T04:10:52.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:52 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/625081276' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:10:52.691 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.682+0000 7f9f2c56a640 1 -- 172.21.15.64:0/83306009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f241049b0 msgr2=0x7f9f24104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:52.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.682+0000 7f9f2c56a640 1 --2- 172.21.15.64:0/83306009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f241049b0 0x7f9f24104dc0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f9f20009900 tx=0x7f9f2002a410 comp rx=0 tx=0).stop 2024-09-21T04:10:52.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.682+0000 7f9f2c56a640 1 -- 172.21.15.64:0/83306009 shutdown_connections 2024-09-21T04:10:52.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.682+0000 7f9f2c56a640 1 --2- 172.21.15.64:0/83306009 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f2410de10 0x7f9f24110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.682+0000 7f9f2c56a640 1 --2- 172.21.15.64:0/83306009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f24105300 0x7f9f2410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.682+0000 7f9f2c56a640 1 --2- 172.21.15.64:0/83306009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f241049b0 0x7f9f24104dc0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.682+0000 7f9f2c56a640 1 -- 172.21.15.64:0/83306009 >> 172.21.15.64:0/83306009 conn(0x7f9f24079730 msgr2=0x7f9f24079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:52.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.682+0000 7f9f2c56a640 1 -- 172.21.15.64:0/83306009 shutdown_connections 2024-09-21T04:10:52.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.682+0000 7f9f2c56a640 1 -- 172.21.15.64:0/83306009 wait complete. 2024-09-21T04:10:52.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.683+0000 7f9f2c56a640 1 Processor -- start 2024-09-21T04:10:52.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.683+0000 7f9f2c56a640 1 -- start start 2024-09-21T04:10:52.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.683+0000 7f9f2c56a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f241049b0 0x7f9f2419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.692 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.683+0000 7f9f2c56a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f24105300 0x7f9f2419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.683+0000 7f9f2c56a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f2410de10 0x7f9f2419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.683+0000 7f9f2c56a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9f24113a70 con 0x7f9f2410de10 2024-09-21T04:10:52.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.683+0000 7f9f2c56a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9f241138f0 con 0x7f9f241049b0 2024-09-21T04:10:52.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.683+0000 7f9f2c56a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9f24113bf0 con 0x7f9f24105300 2024-09-21T04:10:52.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.683+0000 7f9f2ab07640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f2410de10 0x7f9f2419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.683+0000 7f9f2a306640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f241049b0 0x7f9f2419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.684+0000 7f9f2a306640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f241049b0 0x7f9f2419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:39072/0 (socket says 172.21.15.64:39072) 2024-09-21T04:10:52.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.684+0000 7f9f2ab07640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f2410de10 0x7f9f2419fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35932/0 (socket says 172.21.15.64:35932) 2024-09-21T04:10:52.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.684+0000 7f9f2a306640 1 -- 172.21.15.64:0/2705865251 learned_addr learned my addr 172.21.15.64:0/2705865251 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:52.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.684+0000 7f9f29b05640 1 --2- 172.21.15.64:0/2705865251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f24105300 0x7f9f2419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.684+0000 7f9f2ab07640 1 -- 172.21.15.64:0/2705865251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f24105300 msgr2=0x7f9f2419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:52.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.684+0000 7f9f2ab07640 1 --2- 172.21.15.64:0/2705865251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f24105300 0x7f9f2419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.684+0000 7f9f2ab07640 1 -- 172.21.15.64:0/2705865251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f241049b0 msgr2=0x7f9f2419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:52.693 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.684+0000 7f9f2ab07640 1 --2- 172.21.15.64:0/2705865251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f241049b0 0x7f9f2419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.684+0000 7f9f2ab07640 1 -- 172.21.15.64:0/2705865251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f241a0430 con 0x7f9f2410de10 2024-09-21T04:10:52.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.684+0000 7f9f2ab07640 1 --2- 172.21.15.64:0/2705865251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f2410de10 0x7f9f2419fd00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00da10 tx=0x7f9f1c00ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:52.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.684+0000 7f9f177fe640 1 -- 172.21.15.64:0/2705865251 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c00bd70 con 0x7f9f2410de10 2024-09-21T04:10:52.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.684+0000 7f9f177fe640 1 -- 172.21.15.64:0/2705865251 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f1c003e20 con 0x7f9f2410de10 2024-09-21T04:10:52.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.684+0000 7f9f2c56a640 1 -- 172.21.15.64:0/2705865251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f241a0720 con 0x7f9f2410de10 2024-09-21T04:10:52.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.684+0000 7f9f177fe640 1 -- 172.21.15.64:0/2705865251 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c0106c0 con 0x7f9f2410de10 2024-09-21T04:10:52.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.684+0000 7f9f2c56a640 1 -- 172.21.15.64:0/2705865251 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f241a6e80 con 0x7f9f2410de10 2024-09-21T04:10:52.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.687+0000 7f9f32ade740 1 Processor -- start 2024-09-21T04:10:52.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.687+0000 7f9f177fe640 1 -- 172.21.15.64:0/2705865251 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f9f1c004270 con 0x7f9f2410de10 2024-09-21T04:10:52.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.688+0000 7f9f32ade740 1 -- start start 2024-09-21T04:10:52.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.688+0000 7f9f32ade740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562fffc2a070 0x562ffffd12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.688+0000 7f9f32ade740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fffc011b0 0x562ffffd1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.688+0000 7f9f32ade740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562fffc09390 0x562ffffd1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.688+0000 7f9f32ade740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562ffff47ea0 con 0x562fffc011b0 2024-09-21T04:10:52.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.688+0000 7f9f32ade740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562ffff372d0 con 0x562fffc09390 2024-09-21T04:10:52.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.688+0000 7f9f32ade740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562ffff3b0e0 con 0x562fffc2a070 2024-09-21T04:10:52.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.688+0000 7f9f2a306640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fffc011b0 0x562ffffd1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.688+0000 7f9f2a306640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fffc011b0 0x562ffffd1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35942/0 (socket says 172.21.15.64:35942) 2024-09-21T04:10:52.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.688+0000 7f9f2a306640 1 -- 172.21.15.64:0/754515640 learned_addr learned my addr 172.21.15.64:0/754515640 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:52.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.688+0000 7f9f29b05640 1 --2- 172.21.15.64:0/754515640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562fffc09390 0x562ffffd1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.688+0000 7f9f29b05640 1 --2- 172.21.15.64:0/754515640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562fffc2a070 0x562ffffd12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.689+0000 7f9f2a306640 1 -- 172.21.15.64:0/754515640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562fffc2a070 msgr2=0x562ffffd12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:52.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.689+0000 7f9f2a306640 1 --2- 172.21.15.64:0/754515640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562fffc2a070 0x562ffffd12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.689+0000 7f9f2a306640 1 -- 172.21.15.64:0/754515640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562fffc09390 msgr2=0x562ffffd1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:52.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.689+0000 7f9f2a306640 1 --2- 172.21.15.64:0/754515640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562fffc09390 0x562ffffd1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.689+0000 7f9f2a306640 1 -- 172.21.15.64:0/754515640 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fffc95650 con 0x562fffc011b0 2024-09-21T04:10:52.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.689+0000 7f9f177fe640 1 --2- 172.21.15.64:0/2705865251 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f9f100b6700 0x7f9f100b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.689+0000 7f9f29b05640 1 --2- 172.21.15.64:0/754515640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562fffc2a070 0x562ffffd12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:10:52.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.689+0000 7f9f2a306640 1 --2- 172.21.15.64:0/754515640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fffc011b0 0x562ffffd1800 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9f20006010 tx=0x7f9f20031bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d477e47d4497af35 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:52.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.689+0000 7f9f29b05640 1 --2- 172.21.15.64:0/2705865251 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f9f100b6700 0x7f9f100b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.689+0000 7f9f177fe640 1 -- 172.21.15.64:0/2705865251 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7f9f1c0e7ab0 con 0x7f9f2410de10 2024-09-21T04:10:52.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.689+0000 7f9f14ff9640 1 -- 172.21.15.64:0/754515640 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f20040070 con 0x562fffc011b0 2024-09-21T04:10:52.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.689+0000 7f9f14ff9640 1 -- 172.21.15.64:0/754515640 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f200316a0 con 0x562fffc011b0 2024-09-21T04:10:52.696 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.689+0000 7f9f14ff9640 1 -- 172.21.15.64:0/754515640 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f20035040 con 0x562fffc011b0 2024-09-21T04:10:52.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.690+0000 7f9f32ade740 1 -- 172.21.15.64:0/754515640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fffc011b0 msgr2=0x562ffffd1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:52.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.690+0000 7f9f29b05640 1 --2- 172.21.15.64:0/2705865251 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f9f100b6700 0x7f9f100b8c10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009730 tx=0x7f9f180096c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:52.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.690+0000 7f9f32ade740 1 --2- 172.21.15.64:0/754515640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fffc011b0 0x562ffffd1800 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9f20006010 tx=0x7f9f20031bc0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.690+0000 7f9f32ade740 1 -- 172.21.15.64:0/754515640 shutdown_connections 2024-09-21T04:10:52.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.690+0000 7f9f32ade740 1 --2- 172.21.15.64:0/754515640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562fffc09390 0x562ffffd1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.690+0000 7f9f32ade740 1 --2- 172.21.15.64:0/754515640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fffc011b0 0x562ffffd1800 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.690+0000 7f9f32ade740 1 --2- 172.21.15.64:0/754515640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562fffc2a070 0x562ffffd12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.690+0000 7f9f32ade740 1 -- 172.21.15.64:0/754515640 >> 172.21.15.64:0/754515640 conn(0x562fffc1e460 msgr2=0x562ffffcf1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:52.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.690+0000 7f9f32ade740 1 -- 172.21.15.64:0/754515640 shutdown_connections 2024-09-21T04:10:52.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.690+0000 7f9f32ade740 1 -- 172.21.15.64:0/754515640 wait complete. 2024-09-21T04:10:52.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.691+0000 7f9f32ade740 1 -- start start 2024-09-21T04:10:52.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.691+0000 7f9f32ade740 1 Processor -- start 2024-09-21T04:10:52.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.691+0000 7f9f32ade740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562fffc011b0 0x562ffffdc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.691+0000 7f9f32ade740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fffc09390 0x562ffffdcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.691+0000 7f9f32ade740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562fffc2a070 0x562ffffe8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.691+0000 7f9f32ade740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562fffd98700 con 0x562fffc09390 2024-09-21T04:10:52.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.691+0000 7f9f32ade740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562fffed2160 con 0x562fffc2a070 2024-09-21T04:10:52.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.691+0000 7f9f32ade740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562ffffdfa40 con 0x562fffc011b0 2024-09-21T04:10:52.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.691+0000 7f9f32ade740 10 client.0 _ll_register_callbacks cb 0x562fffeeb9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:10:52.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.691+0000 7f9f29b05640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fffc09390 0x562ffffdcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.691+0000 7f9f2ab07640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562fffc2a070 0x562ffffe8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.691+0000 7f9f2ab07640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562fffc2a070 0x562ffffe8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:39094/0 (socket says 172.21.15.64:39094) 2024-09-21T04:10:52.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.691+0000 7f9f2ab07640 1 -- 172.21.15.64:0/388910841 learned_addr learned my addr 172.21.15.64:0/388910841 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:52.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.692+0000 7f9f2a306640 1 --2- 172.21.15.64:0/388910841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562fffc011b0 0x562ffffdc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.692+0000 7f9f29b05640 1 -- 172.21.15.64:0/388910841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562fffc011b0 msgr2=0x562ffffdc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:52.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.692+0000 7f9f29b05640 1 --2- 172.21.15.64:0/388910841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562fffc011b0 0x562ffffdc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.692+0000 7f9f29b05640 1 -- 172.21.15.64:0/388910841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562fffc2a070 msgr2=0x562ffffe8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:52.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.692+0000 7f9f29b05640 1 --2- 172.21.15.64:0/388910841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562fffc2a070 0x562ffffe8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.692+0000 7f9f29b05640 1 -- 172.21.15.64:0/388910841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fffeffc90 con 0x562fffc09390 2024-09-21T04:10:52.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.692+0000 7f9f2ab07640 1 --2- 172.21.15.64:0/388910841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562fffc2a070 0x562ffffe8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:52.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.692+0000 7f9f2a306640 1 --2- 172.21.15.64:0/388910841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562fffc011b0 0x562ffffdc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:10:52.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.692+0000 7f9f29b05640 1 --2- 172.21.15.64:0/388910841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fffc09390 0x562ffffdcec0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9f2406aaa0 tx=0x7f9f18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aad6ce2f4dc0cd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:52.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.692+0000 7f9f0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:10:52.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.692+0000 7f9f0dffb640 1 -- 172.21.15.64:0/388910841 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18010700 con 0x562fffc09390 2024-09-21T04:10:52.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.692+0000 7f9f0dffb640 1 -- 172.21.15.64:0/388910841 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7f9f180108a0 con 0x562fffc09390 2024-09-21T04:10:52.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.692+0000 7f9f32ade740 1 -- 172.21.15.64:0/388910841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f200316a0 con 0x562fffc09390 2024-09-21T04:10:52.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.693+0000 7f9f0dffb640 1 -- 172.21.15.64:0/388910841 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9f18010aa0 con 0x562fffc09390 2024-09-21T04:10:52.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.693+0000 7f9f32ade740 10 client.16899 fetch_fsmap learned FSMap version 38 2024-09-21T04:10:52.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.693+0000 7f9f32ade740 1 -- 172.21.15.64:0/388910841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x562ffffeb780 con 0x562fffc09390 2024-09-21T04:10:52.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.693+0000 7f9f0dffb640 1 -- 172.21.15.64:0/388910841 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f200316a0 con 0x562fffc09390 2024-09-21T04:10:52.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.693+0000 7f9f0dffb640 1 -- 172.21.15.64:0/388910841 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f9f1801e030 con 0x562fffc09390 2024-09-21T04:10:52.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.693+0000 7f9f32ade740 10 client.16899 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T04:10:52.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.693+0000 7f9f32ade740 10 client.16899 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:10:52.702 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.693+0000 7f9f32ade740 20 client.16899 populate_metadata read hostname 'smithi064' 2024-09-21T04:10:52.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.693+0000 7f9f32ade740 1 --2- 172.21.15.64:0/388910841 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562fffbe89f0 0x562ffffeed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.693+0000 7f9f32ade740 4 client.16899 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:10:52.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.693+0000 7f9f32ade740 1 -- 172.21.15.64:0/388910841 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ffff49df0 con 0x562fffbe89f0 2024-09-21T04:10:52.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.693+0000 7f9f2a306640 1 --2- 172.21.15.64:0/388910841 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562fffbe89f0 0x562ffffeed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.694+0000 7f9f2a306640 1 --2- 172.21.15.64:0/388910841 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562fffbe89f0 0x562ffffeed80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=95748997d61af2c9 server_cookie=83708610932591e in_seq=0 out_seq=0 2024-09-21T04:10:52.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.694+0000 7f9f0dffb640 10 client.16899 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:10:52.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.694+0000 7f9f0dffb640 1 -- 172.21.15.64:0/388910841 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562ffff49df0 con 0x562fffbe89f0 2024-09-21T04:10:52.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.695+0000 7f9f0dffb640 10 client.16899 handle_command_reply: tid=0 2024-09-21T04:10:52.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.695+0000 7f9f32ade740 1 client.16899 shutdown 2024-09-21T04:10:52.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.695+0000 7f9f32ade740 1 -- 172.21.15.64:0/388910841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fffc09390 msgr2=0x562ffffdcec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:52.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.695+0000 7f9f32ade740 1 --2- 172.21.15.64:0/388910841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fffc09390 0x562ffffdcec0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9f2406aaa0 tx=0x7f9f18004730 comp rx=0 tx=0).stop 2024-09-21T04:10:52.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.695+0000 7f9f32ade740 1 -- 172.21.15.64:0/388910841 shutdown_connections 2024-09-21T04:10:52.703 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.695+0000 7f9f32ade740 1 --2- 172.21.15.64:0/388910841 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562fffbe89f0 0x562ffffeed80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.695+0000 7f9f32ade740 1 --2- 172.21.15.64:0/388910841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562fffc2a070 0x562ffffe8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.695+0000 7f9f0dffb640 0 client.16899 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:10:52.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.695+0000 7f9f32ade740 1 --2- 172.21.15.64:0/388910841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fffc09390 0x562ffffdcec0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.695+0000 7f9f32ade740 1 --2- 172.21.15.64:0/388910841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562fffc011b0 0x562ffffdc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.695+0000 7f9f32ade740 1 -- 172.21.15.64:0/388910841 >> 172.21.15.64:0/388910841 conn(0x562fffc1e460 msgr2=0x562ffffd4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:52.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.695+0000 7f9f32ade740 1 -- 172.21.15.64:0/388910841 shutdown_connections 2024-09-21T04:10:52.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.695+0000 7f9f32ade740 1 -- 172.21.15.64:0/388910841 wait complete. 2024-09-21T04:10:52.704 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.695+0000 7f9f32ade740 20 client.16899 trim_cache size 0 max 16384 2024-09-21T04:10:52.712 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.709+0000 7f9f32ade740 1 Processor -- start 2024-09-21T04:10:52.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.709+0000 7f9f32ade740 1 -- start start 2024-09-21T04:10:52.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.710+0000 7f9f32ade740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fffbe89f0 0x562ffffde860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.710+0000 7f9f32ade740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562fffc011b0 0x562ffffd2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.710+0000 7f9f32ade740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562fffc09390 0x562ffffce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.710+0000 7f9f32ade740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9f241b2270 con 0x562fffbe89f0 2024-09-21T04:10:52.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.710+0000 7f9f32ade740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9f241afae0 con 0x562fffc09390 2024-09-21T04:10:52.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.710+0000 7f9f32ade740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9f241ad950 con 0x562fffc011b0 2024-09-21T04:10:52.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.710+0000 7f9f2a306640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fffbe89f0 0x562ffffde860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.710+0000 7f9f2a306640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fffbe89f0 0x562ffffde860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35960/0 (socket says 172.21.15.64:35960) 2024-09-21T04:10:52.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.710+0000 7f9f2a306640 1 -- 172.21.15.64:0/1065317569 learned_addr learned my addr 172.21.15.64:0/1065317569 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:52.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.710+0000 7f9f2ab07640 1 --2- 172.21.15.64:0/1065317569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562fffc09390 0x562ffffce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.710+0000 7f9f29b05640 1 --2- 172.21.15.64:0/1065317569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562fffc011b0 0x562ffffd2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.713 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.710+0000 7f9f2a306640 1 -- 172.21.15.64:0/1065317569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562fffc011b0 msgr2=0x562ffffd2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:52.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.710+0000 7f9f2a306640 1 --2- 172.21.15.64:0/1065317569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562fffc011b0 0x562ffffd2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.710+0000 7f9f2a306640 1 -- 172.21.15.64:0/1065317569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562fffc09390 msgr2=0x562ffffce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:52.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.710+0000 7f9f2a306640 1 --2- 172.21.15.64:0/1065317569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562fffc09390 0x562ffffce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.710+0000 7f9f2a306640 1 -- 172.21.15.64:0/1065317569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ffffdd270 con 0x562fffbe89f0 2024-09-21T04:10:52.714 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.710+0000 7f9f2a306640 1 --2- 172.21.15.64:0/1065317569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fffbe89f0 0x562ffffde860 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9f20031c30 tx=0x7f9f20005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc986b79c311f208 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:52.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.710+0000 7f9f0ffff640 1 -- 172.21.15.64:0/1065317569 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f20040070 con 0x562fffbe89f0 2024-09-21T04:10:52.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.711+0000 7f9f0ffff640 1 -- 172.21.15.64:0/1065317569 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f200046c0 con 0x562fffbe89f0 2024-09-21T04:10:52.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.711+0000 7f9f0ffff640 1 -- 172.21.15.64:0/1065317569 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f20035040 con 0x562fffbe89f0 2024-09-21T04:10:52.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.711+0000 7f9f32ade740 1 -- 172.21.15.64:0/1065317569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fffbe89f0 msgr2=0x562ffffde860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:52.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.711+0000 7f9f32ade740 1 --2- 172.21.15.64:0/1065317569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fffbe89f0 0x562ffffde860 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9f20031c30 tx=0x7f9f20005900 comp rx=0 tx=0).stop 2024-09-21T04:10:52.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.711+0000 7f9f32ade740 1 -- 172.21.15.64:0/1065317569 shutdown_connections 2024-09-21T04:10:52.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.711+0000 7f9f32ade740 1 --2- 172.21.15.64:0/1065317569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562fffc09390 0x562ffffce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.711+0000 7f9f32ade740 1 --2- 172.21.15.64:0/1065317569 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562fffc011b0 0x562ffffd2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.711+0000 7f9f32ade740 1 --2- 172.21.15.64:0/1065317569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fffbe89f0 0x562ffffde860 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.711+0000 7f9f32ade740 1 -- 172.21.15.64:0/1065317569 >> 172.21.15.64:0/1065317569 conn(0x562fffc1e460 msgr2=0x562ffffd22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:52.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.711+0000 7f9f32ade740 1 -- 172.21.15.64:0/1065317569 shutdown_connections 2024-09-21T04:10:52.717 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.712+0000 7f9f32ade740 1 -- 172.21.15.64:0/1065317569 wait complete. 2024-09-21T04:10:52.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.712+0000 7f9f32ade740 1 -- start start 2024-09-21T04:10:52.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.712+0000 7f9f32ade740 1 Processor -- start 2024-09-21T04:10:52.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.712+0000 7f9f32ade740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562fffbe89f0 0x562ffffe51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.712+0000 7f9f32ade740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fffc011b0 0x562ffffe06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.712+0000 7f9f32ade740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562fffc09390 0x562ffffe0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.712+0000 7f9f32ade740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9f241adfa0 con 0x562fffc011b0 2024-09-21T04:10:52.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.712+0000 7f9f32ade740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9f241af410 con 0x562fffbe89f0 2024-09-21T04:10:52.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.712+0000 7f9f32ade740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9f241ae670 con 0x562fffc09390 2024-09-21T04:10:52.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.712+0000 7f9f29b05640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fffc011b0 0x562ffffe06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.712+0000 7f9f32ade740 10 client.0 _ll_register_callbacks cb 0x7f9f18004410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:10:52.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.713+0000 7f9f2a306640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562fffbe89f0 0x562ffffe51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.713+0000 7f9f29b05640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fffc011b0 0x562ffffe06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35970/0 (socket says 172.21.15.64:35970) 2024-09-21T04:10:52.718 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.713+0000 7f9f29b05640 1 -- 172.21.15.64:0/3120430960 learned_addr learned my addr 172.21.15.64:0/3120430960 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:52.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.713+0000 7f9f2ab07640 1 --2- 172.21.15.64:0/3120430960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562fffc09390 0x562ffffe0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.713+0000 7f9f29b05640 1 -- 172.21.15.64:0/3120430960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562fffc09390 msgr2=0x562ffffe0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:52.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.713+0000 7f9f29b05640 1 --2- 172.21.15.64:0/3120430960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562fffc09390 0x562ffffe0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.713+0000 7f9f29b05640 1 -- 172.21.15.64:0/3120430960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562fffbe89f0 msgr2=0x562ffffe51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:52.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.713+0000 7f9f29b05640 1 --2- 172.21.15.64:0/3120430960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562fffbe89f0 0x562ffffe51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.713+0000 7f9f29b05640 1 -- 172.21.15.64:0/3120430960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ffffe1170 con 0x562fffc011b0 2024-09-21T04:10:52.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.713+0000 7f9f2ab07640 1 --2- 172.21.15.64:0/3120430960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562fffc09390 0x562ffffe0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:10:52.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.713+0000 7f9f29b05640 1 --2- 172.21.15.64:0/3120430960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fffc011b0 0x562ffffe06f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9f1802bc80 tx=0x7f9f1800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=493fb5564636be97 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:52.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.713+0000 7f9f0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:10:52.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.713+0000 7f9f0dffb640 1 -- 172.21.15.64:0/3120430960 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18037070 con 0x562fffc011b0 2024-09-21T04:10:52.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.713+0000 7f9f0dffb640 1 -- 172.21.15.64:0/3120430960 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7f9f1802b470 con 0x562fffc011b0 2024-09-21T04:10:52.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.713+0000 7f9f32ade740 1 -- 172.21.15.64:0/3120430960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f200046c0 con 0x562fffc011b0 2024-09-21T04:10:52.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.714+0000 7f9f0dffb640 1 -- 172.21.15.64:0/3120430960 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9f1802b670 con 0x562fffc011b0 2024-09-21T04:10:52.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.714+0000 7f9f32ade740 10 client.16905 fetch_fsmap learned FSMap version 38 2024-09-21T04:10:52.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.714+0000 7f9f32ade740 1 -- 172.21.15.64:0/3120430960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563000059410 con 0x562fffc011b0 2024-09-21T04:10:52.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.714+0000 7f9f0dffb640 1 -- 172.21.15.64:0/3120430960 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f200046c0 con 0x562fffc011b0 2024-09-21T04:10:52.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.714+0000 7f9f0dffb640 1 -- 172.21.15.64:0/3120430960 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f9f1801e040 con 0x562fffc011b0 2024-09-21T04:10:52.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.714+0000 7f9f32ade740 10 client.16905 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T04:10:52.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.714+0000 7f9f32ade740 10 client.16905 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:10:52.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.714+0000 7f9f32ade740 20 client.16905 populate_metadata read hostname 'smithi064' 2024-09-21T04:10:52.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.714+0000 7f9f32ade740 1 --2- 172.21.15.64:0/3120430960 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562fffc2a070 0x56300005bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:52.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.714+0000 7f9f32ade740 4 client.16905 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:10:52.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.714+0000 7f9f32ade740 1 -- 172.21.15.64:0/3120430960 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ffff458a0 con 0x562fffc2a070 2024-09-21T04:10:52.720 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.714+0000 7f9f2a306640 1 --2- 172.21.15.64:0/3120430960 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562fffc2a070 0x56300005bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:52.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.715+0000 7f9f2a306640 1 --2- 172.21.15.64:0/3120430960 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562fffc2a070 0x56300005bc50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=235e4123f494e7fc server_cookie=442f2cb259834dfd in_seq=0 out_seq=0 2024-09-21T04:10:52.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.715+0000 7f9f0dffb640 10 client.16905 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:10:52.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.715+0000 7f9f0dffb640 1 -- 172.21.15.64:0/3120430960 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ffff458a0 con 0x562fffc2a070 2024-09-21T04:10:52.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.715+0000 7f9f0dffb640 10 client.16905 handle_command_reply: tid=0 2024-09-21T04:10:52.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.715+0000 7f9f32ade740 1 client.16905 shutdown 2024-09-21T04:10:52.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.715+0000 7f9f32ade740 1 -- 172.21.15.64:0/3120430960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fffc011b0 msgr2=0x562ffffe06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:52.721 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.715+0000 7f9f32ade740 1 --2- 172.21.15.64:0/3120430960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fffc011b0 0x562ffffe06f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9f1802bc80 tx=0x7f9f1800eea0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.715+0000 7f9f32ade740 1 -- 172.21.15.64:0/3120430960 shutdown_connections 2024-09-21T04:10:52.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.715+0000 7f9f32ade740 1 --2- 172.21.15.64:0/3120430960 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x562fffc2a070 0x56300005bc50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.715+0000 7f9f32ade740 1 --2- 172.21.15.64:0/3120430960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562fffc09390 0x562ffffe0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.715+0000 7f9f32ade740 1 --2- 172.21.15.64:0/3120430960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562fffc011b0 0x562ffffe06f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.715+0000 7f9f0dffb640 0 client.16905 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:10:52.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.715+0000 7f9f32ade740 1 --2- 172.21.15.64:0/3120430960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562fffbe89f0 0x562ffffe51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.715+0000 7f9f32ade740 1 -- 172.21.15.64:0/3120430960 >> 172.21.15.64:0/3120430960 conn(0x562fffc1e460 msgr2=0x562ffffde580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:52.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.715+0000 7f9f32ade740 1 -- 172.21.15.64:0/3120430960 shutdown_connections 2024-09-21T04:10:52.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.715+0000 7f9f32ade740 1 -- 172.21.15.64:0/3120430960 wait complete. 2024-09-21T04:10:52.722 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.715+0000 7f9f32ade740 20 client.16905 trim_cache size 0 max 16384 2024-09-21T04:10:52.722 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:10:52.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.716+0000 7f9f2c56a640 1 -- 172.21.15.64:0/2705865251 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f9f100b6700 msgr2=0x7f9f100b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:52.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.716+0000 7f9f2c56a640 1 --2- 172.21.15.64:0/2705865251 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f9f100b6700 0x7f9f100b8c10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009730 tx=0x7f9f180096c0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.716+0000 7f9f2c56a640 1 -- 172.21.15.64:0/2705865251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f2410de10 msgr2=0x7f9f2419fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:52.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.716+0000 7f9f2c56a640 1 --2- 172.21.15.64:0/2705865251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f2410de10 0x7f9f2419fd00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00da10 tx=0x7f9f1c00ded0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.716+0000 7f9f2c56a640 1 -- 172.21.15.64:0/2705865251 shutdown_connections 2024-09-21T04:10:52.723 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.716+0000 7f9f2c56a640 1 --2- 172.21.15.64:0/2705865251 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f9f100b6700 0x7f9f100b8c10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.716+0000 7f9f2c56a640 1 --2- 172.21.15.64:0/2705865251 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9f2410de10 0x7f9f2419fd00 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.716+0000 7f9f2c56a640 1 --2- 172.21.15.64:0/2705865251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f24105300 0x7f9f2419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.716+0000 7f9f2c56a640 1 --2- 172.21.15.64:0/2705865251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f241049b0 0x7f9f2419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:52.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.716+0000 7f9f2c56a640 1 -- 172.21.15.64:0/2705865251 >> 172.21.15.64:0/2705865251 conn(0x7f9f24079730 msgr2=0x7f9f241007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:52.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.716+0000 7f9f2c56a640 1 -- 172.21.15.64:0/2705865251 shutdown_connections 2024-09-21T04:10:52.724 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:52.716+0000 7f9f2c56a640 1 -- 172.21.15.64:0/2705865251 wait complete. 2024-09-21T04:10:53.443 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T04:10:53.744 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:10:54.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.214+0000 7f73aba6d640 1 -- 172.21.15.64:0/1445700486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73a41049d0 msgr2=0x7f73a4104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:54.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.214+0000 7f73aba6d640 1 --2- 172.21.15.64:0/1445700486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73a41049d0 0x7f73a4104de0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009900 tx=0x7f73a002a410 comp rx=0 tx=0).stop 2024-09-21T04:10:54.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.214+0000 7f73aba6d640 1 -- 172.21.15.64:0/1445700486 shutdown_connections 2024-09-21T04:10:54.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.214+0000 7f73aba6d640 1 --2- 172.21.15.64:0/1445700486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73a410de30 0x7f73a4110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.214+0000 7f73aba6d640 1 --2- 172.21.15.64:0/1445700486 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73a4105320 0x7f73a410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.214+0000 7f73aba6d640 1 --2- 172.21.15.64:0/1445700486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73a41049d0 0x7f73a4104de0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.214+0000 7f73aba6d640 1 -- 172.21.15.64:0/1445700486 >> 172.21.15.64:0/1445700486 conn(0x7f73a4079740 msgr2=0x7f73a4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:54.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.214+0000 7f73aba6d640 1 -- 172.21.15.64:0/1445700486 shutdown_connections 2024-09-21T04:10:54.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.214+0000 7f73aba6d640 1 -- 172.21.15.64:0/1445700486 wait complete. 2024-09-21T04:10:54.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.215+0000 7f73aba6d640 1 Processor -- start 2024-09-21T04:10:54.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.215+0000 7f73aba6d640 1 -- start start 2024-09-21T04:10:54.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.215+0000 7f73aba6d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73a41049d0 0x7f73a419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:54.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.215+0000 7f73aba6d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73a4105320 0x7f73a419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:54.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.215+0000 7f73aba6d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73a410de30 0x7f73a419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:54.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.215+0000 7f73aba6d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f73a4113ab0 con 0x7f73a41049d0 2024-09-21T04:10:54.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.215+0000 7f73aba6d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f73a4113930 con 0x7f73a4105320 2024-09-21T04:10:54.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.215+0000 7f73aba6d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f73a4113c30 con 0x7f73a410de30 2024-09-21T04:10:54.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.215+0000 7f73a9008640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73a4105320 0x7f73a419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:54.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.215+0000 7f73a9809640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73a41049d0 0x7f73a419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:54.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.215+0000 7f73a9008640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73a4105320 0x7f73a419aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:39124/0 (socket says 172.21.15.64:39124) 2024-09-21T04:10:54.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.215+0000 7f73a9008640 1 -- 172.21.15.64:0/1644845355 learned_addr learned my addr 172.21.15.64:0/1644845355 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:54.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.215+0000 7f73aa00a640 1 --2- 172.21.15.64:0/1644845355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73a410de30 0x7f73a419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:54.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.216+0000 7f73a9008640 1 -- 172.21.15.64:0/1644845355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73a410de30 msgr2=0x7f73a419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:54.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.216+0000 7f73a9008640 1 --2- 172.21.15.64:0/1644845355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73a410de30 0x7f73a419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.216+0000 7f73a9008640 1 -- 172.21.15.64:0/1644845355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73a41049d0 msgr2=0x7f73a419a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:54.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.216+0000 7f73a9008640 1 --2- 172.21.15.64:0/1644845355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73a41049d0 0x7f73a419a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.216+0000 7f73a9008640 1 -- 172.21.15.64:0/1644845355 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a41a0470 con 0x7f73a4105320 2024-09-21T04:10:54.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.216+0000 7f73a9008640 1 --2- 172.21.15.64:0/1644845355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73a4105320 0x7f73a419aaa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f739400e9c0 tx=0x7f739400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:54.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.216+0000 7f739affd640 1 -- 172.21.15.64:0/1644845355 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739400ccc0 con 0x7f73a4105320 2024-09-21T04:10:54.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.216+0000 7f739affd640 1 -- 172.21.15.64:0/1644845355 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73940054d0 con 0x7f73a4105320 2024-09-21T04:10:54.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.216+0000 7f73aba6d640 1 -- 172.21.15.64:0/1644845355 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a41a0760 con 0x7f73a4105320 2024-09-21T04:10:54.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.216+0000 7f739affd640 1 -- 172.21.15.64:0/1644845355 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394010430 con 0x7f73a4105320 2024-09-21T04:10:54.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.216+0000 7f73aba6d640 1 -- 172.21.15.64:0/1644845355 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a41a6ec0 con 0x7f73a4105320 2024-09-21T04:10:54.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.219+0000 7f73b1fe1740 1 Processor -- start 2024-09-21T04:10:54.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.219+0000 7f73b1fe1740 1 -- start start 2024-09-21T04:10:54.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.219+0000 7f739affd640 1 -- 172.21.15.64:0/1644845355 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f7394005020 con 0x7f73a4105320 2024-09-21T04:10:54.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.219+0000 7f73b1fe1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed99173070 0x55ed9951a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:54.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.219+0000 7f73b1fe1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed9914a1b0 0x55ed9951a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:54.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.219+0000 7f73b1fe1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed99152390 0x55ed9951ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:54.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.219+0000 7f73b1fe1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ed99490ea0 con 0x55ed99152390 2024-09-21T04:10:54.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.219+0000 7f73b1fe1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ed994802d0 con 0x55ed99173070 2024-09-21T04:10:54.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.219+0000 7f73b1fe1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ed994840e0 con 0x55ed9914a1b0 2024-09-21T04:10:54.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.219+0000 7f73a9008640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed99152390 0x55ed9951ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:54.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.219+0000 7f73aa00a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed99173070 0x55ed9951a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:54.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.219+0000 7f73a9008640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed99152390 0x55ed9951ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35990/0 (socket says 172.21.15.64:35990) 2024-09-21T04:10:54.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.219+0000 7f73aa00a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed99173070 0x55ed9951a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:39138/0 (socket says 172.21.15.64:39138) 2024-09-21T04:10:54.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.219+0000 7f73a9008640 1 -- 172.21.15.64:0/2994721646 learned_addr learned my addr 172.21.15.64:0/2994721646 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:54.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.219+0000 7f73a9809640 1 --2- 172.21.15.64:0/2994721646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed9914a1b0 0x55ed9951a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:54.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.220+0000 7f73aa00a640 1 -- 172.21.15.64:0/2994721646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed9914a1b0 msgr2=0x55ed9951a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:54.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.220+0000 7f73aa00a640 1 --2- 172.21.15.64:0/2994721646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed9914a1b0 0x55ed9951a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.220+0000 7f73aa00a640 1 -- 172.21.15.64:0/2994721646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed99152390 msgr2=0x55ed9951ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:54.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.220+0000 7f73aa00a640 1 --2- 172.21.15.64:0/2994721646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed99152390 0x55ed9951ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.220+0000 7f73aa00a640 1 -- 172.21.15.64:0/2994721646 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed991de650 con 0x55ed99173070 2024-09-21T04:10:54.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.220+0000 7f739affd640 1 --2- 172.21.15.64:0/1644845355 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f73900b6680 0x7f73900b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:54.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.220+0000 7f73aa00a640 1 --2- 172.21.15.64:0/2994721646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed99173070 0x55ed9951a2c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f739c009f00 tx=0x7f739c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c4724d3946b8d21 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:54.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.220+0000 7f73a9809640 1 --2- 172.21.15.64:0/1644845355 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f73900b6680 0x7f73900b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:54.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.220+0000 7f738ffff640 1 -- 172.21.15.64:0/2994721646 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c004d20 con 0x55ed99173070 2024-09-21T04:10:54.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.220+0000 7f738ffff640 1 -- 172.21.15.64:0/2994721646 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f739c004030 con 0x55ed99173070 2024-09-21T04:10:54.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.220+0000 7f738ffff640 1 -- 172.21.15.64:0/2994721646 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c005280 con 0x55ed99173070 2024-09-21T04:10:54.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.220+0000 7f739affd640 1 -- 172.21.15.64:0/1644845355 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7f73940e7b00 con 0x7f73a4105320 2024-09-21T04:10:54.228 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.221+0000 7f73b1fe1740 1 -- 172.21.15.64:0/2994721646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed99173070 msgr2=0x55ed9951a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:54.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.221+0000 7f73b1fe1740 1 --2- 172.21.15.64:0/2994721646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed99173070 0x55ed9951a2c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f739c009f00 tx=0x7f739c011810 comp rx=0 tx=0).stop 2024-09-21T04:10:54.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.221+0000 7f73a9809640 1 --2- 172.21.15.64:0/1644845355 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f73900b6680 0x7f73900b8b90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f73a0000c00 tx=0x7f73a0031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:54.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.221+0000 7f73b1fe1740 1 -- 172.21.15.64:0/2994721646 shutdown_connections 2024-09-21T04:10:54.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.221+0000 7f73b1fe1740 1 --2- 172.21.15.64:0/2994721646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed99152390 0x55ed9951ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.221+0000 7f73b1fe1740 1 --2- 172.21.15.64:0/2994721646 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed9914a1b0 0x55ed9951a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.221+0000 7f73b1fe1740 1 --2- 172.21.15.64:0/2994721646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed99173070 0x55ed9951a2c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.221+0000 7f73b1fe1740 1 -- 172.21.15.64:0/2994721646 >> 172.21.15.64:0/2994721646 conn(0x55ed99167460 msgr2=0x55ed995181b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:54.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.221+0000 7f73b1fe1740 1 -- 172.21.15.64:0/2994721646 shutdown_connections 2024-09-21T04:10:54.229 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.221+0000 7f73b1fe1740 1 -- 172.21.15.64:0/2994721646 wait complete. 2024-09-21T04:10:54.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.222+0000 7f73b1fe1740 1 -- start start 2024-09-21T04:10:54.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.222+0000 7f73b1fe1740 1 Processor -- start 2024-09-21T04:10:54.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.222+0000 7f73b1fe1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed9914a1b0 0x55ed99525a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:54.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.222+0000 7f73b1fe1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed99152390 0x55ed99525f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:54.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.222+0000 7f73b1fe1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed99173070 0x55ed99536160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.222+0000 7f73b1fe1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ed992e1700 con 0x55ed9914a1b0 2024-09-21T04:10:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.222+0000 7f73b1fe1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ed9941b160 con 0x55ed99173070 2024-09-21T04:10:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.222+0000 7f73b1fe1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ed9952cd60 con 0x55ed99152390 2024-09-21T04:10:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.222+0000 7f73b1fe1740 10 client.0 _ll_register_callbacks cb 0x55ed994349e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:10:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.223+0000 7f73a9809640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed9914a1b0 0x55ed99525a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.223+0000 7f73a9809640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed9914a1b0 0x55ed99525a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35996/0 (socket says 172.21.15.64:35996) 2024-09-21T04:10:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.223+0000 7f73a9809640 1 -- 172.21.15.64:0/1929753927 learned_addr learned my addr 172.21.15.64:0/1929753927 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.223+0000 7f73a9008640 1 --2- 172.21.15.64:0/1929753927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed99152390 0x55ed99525f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.223+0000 7f73aa00a640 1 --2- 172.21.15.64:0/1929753927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed99173070 0x55ed99536160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.223+0000 7f73a9809640 1 -- 172.21.15.64:0/1929753927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed99152390 msgr2=0x55ed99525f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:54.231 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.223+0000 7f73a9809640 1 --2- 172.21.15.64:0/1929753927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed99152390 0x55ed99525f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.223+0000 7f73a9809640 1 -- 172.21.15.64:0/1929753927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed99173070 msgr2=0x55ed99536160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:54.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.223+0000 7f73a9809640 1 --2- 172.21.15.64:0/1929753927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed99173070 0x55ed99536160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.223+0000 7f73a9809640 1 -- 172.21.15.64:0/1929753927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed99448c90 con 0x55ed9914a1b0 2024-09-21T04:10:54.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.223+0000 7f73a9008640 1 --2- 172.21.15.64:0/1929753927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed99152390 0x55ed99525f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:54.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.223+0000 7f73a9809640 1 --2- 172.21.15.64:0/1929753927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed9914a1b0 0x55ed99525a40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f73a0033000 tx=0x7f73a0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1f7d750a5082a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:54.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.223+0000 7f738d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:10:54.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.223+0000 7f738d7fa640 1 -- 172.21.15.64:0/1929753927 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a004d070 con 0x55ed9914a1b0 2024-09-21T04:10:54.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.223+0000 7f73b1fe1740 1 -- 172.21.15.64:0/1929753927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f739c004030 con 0x55ed9914a1b0 2024-09-21T04:10:54.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.223+0000 7f738d7fa640 1 -- 172.21.15.64:0/1929753927 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7f73a0004450 con 0x55ed9914a1b0 2024-09-21T04:10:54.232 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.224+0000 7f738d7fa640 1 -- 172.21.15.64:0/1929753927 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f73a0059050 con 0x55ed9914a1b0 2024-09-21T04:10:54.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.224+0000 7f73b1fe1740 10 client.16917 fetch_fsmap learned FSMap version 38 2024-09-21T04:10:54.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.224+0000 7f73b1fe1740 1 -- 172.21.15.64:0/1929753927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ed99538ca0 con 0x55ed9914a1b0 2024-09-21T04:10:54.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.224+0000 7f738d7fa640 1 -- 172.21.15.64:0/1929753927 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f739c004030 con 0x55ed9914a1b0 2024-09-21T04:10:54.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.224+0000 7f738d7fa640 1 -- 172.21.15.64:0/1929753927 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f73a0004b50 con 0x55ed9914a1b0 2024-09-21T04:10:54.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.224+0000 7f73b1fe1740 10 client.16917 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T04:10:54.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.224+0000 7f73b1fe1740 10 client.16917 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:10:54.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.224+0000 7f73b1fe1740 20 client.16917 populate_metadata read hostname 'smithi064' 2024-09-21T04:10:54.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.224+0000 7f73b1fe1740 1 --2- 172.21.15.64:0/1929753927 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55ed991319f0 0x55ed9953c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:54.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.224+0000 7f73b1fe1740 4 client.16917 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:10:54.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.224+0000 7f73b1fe1740 1 -- 172.21.15.64:0/1929753927 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed99492df0 con 0x55ed991319f0 2024-09-21T04:10:54.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.224+0000 7f73a9008640 1 --2- 172.21.15.64:0/1929753927 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55ed991319f0 0x55ed9953c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:54.233 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.225+0000 7f738d7fa640 10 client.16917 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:10:54.234 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.225+0000 7f73a9008640 1 --2- 172.21.15.64:0/1929753927 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55ed991319f0 0x55ed9953c170 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c15e8c61c72dcf85 server_cookie=50b90e4d07d0e2a8 in_seq=0 out_seq=0 2024-09-21T04:10:54.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.227+0000 7f738d7fa640 1 -- 172.21.15.64:0/1929753927 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ed99492df0 con 0x55ed991319f0 2024-09-21T04:10:54.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.227+0000 7f738d7fa640 10 client.16917 handle_command_reply: tid=0 2024-09-21T04:10:54.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.227+0000 7f73b1fe1740 1 client.16917 shutdown 2024-09-21T04:10:54.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.227+0000 7f73b1fe1740 1 -- 172.21.15.64:0/1929753927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed9914a1b0 msgr2=0x55ed99525a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:54.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.227+0000 7f73b1fe1740 1 --2- 172.21.15.64:0/1929753927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed9914a1b0 0x55ed99525a40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f73a0033000 tx=0x7f73a0009260 comp rx=0 tx=0).stop 2024-09-21T04:10:54.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.228+0000 7f73b1fe1740 1 -- 172.21.15.64:0/1929753927 shutdown_connections 2024-09-21T04:10:54.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.228+0000 7f73b1fe1740 1 --2- 172.21.15.64:0/1929753927 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55ed991319f0 0x55ed9953c170 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.228+0000 7f73b1fe1740 1 --2- 172.21.15.64:0/1929753927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed99173070 0x55ed99536160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.228+0000 7f73b1fe1740 1 --2- 172.21.15.64:0/1929753927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed99152390 0x55ed99525f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.235 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.228+0000 7f738d7fa640 0 client.16917 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:10:54.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.228+0000 7f73b1fe1740 1 --2- 172.21.15.64:0/1929753927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed9914a1b0 0x55ed99525a40 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.228+0000 7f73b1fe1740 1 -- 172.21.15.64:0/1929753927 >> 172.21.15.64:0/1929753927 conn(0x55ed99167460 msgr2=0x55ed9951d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:54.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.228+0000 7f73b1fe1740 1 -- 172.21.15.64:0/1929753927 shutdown_connections 2024-09-21T04:10:54.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.228+0000 7f73b1fe1740 1 -- 172.21.15.64:0/1929753927 wait complete. 2024-09-21T04:10:54.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.228+0000 7f73b1fe1740 20 client.16917 trim_cache size 0 max 16384 2024-09-21T04:10:54.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.242+0000 7f73b1fe1740 1 Processor -- start 2024-09-21T04:10:54.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.242+0000 7f73b1fe1740 1 -- start start 2024-09-21T04:10:54.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.242+0000 7f73b1fe1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9914a1b0 0x55ed99536060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:54.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.242+0000 7f73b1fe1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed99152390 0x55ed995365a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:54.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.242+0000 7f73b1fe1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed99173070 0x55ed99517af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:54.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.242+0000 7f73b1fe1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f73a41b0010 con 0x55ed99173070 2024-09-21T04:10:54.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.242+0000 7f73b1fe1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f73a41ad880 con 0x55ed9914a1b0 2024-09-21T04:10:54.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.242+0000 7f73b1fe1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f73a41ab6f0 con 0x55ed99152390 2024-09-21T04:10:54.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.242+0000 7f73a9809640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9914a1b0 0x55ed99536060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:54.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.242+0000 7f73a9809640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9914a1b0 0x55ed99536060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:39158/0 (socket says 172.21.15.64:39158) 2024-09-21T04:10:54.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.242+0000 7f73aa00a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed99173070 0x55ed99517af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:54.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.242+0000 7f73a9008640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed99152390 0x55ed995365a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:54.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.242+0000 7f73a9809640 1 -- 172.21.15.64:0/3175545014 learned_addr learned my addr 172.21.15.64:0/3175545014 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:54.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.242+0000 7f73aa00a640 1 -- 172.21.15.64:0/3175545014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed99152390 msgr2=0x55ed995365a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:54.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.242+0000 7f73aa00a640 1 --2- 172.21.15.64:0/3175545014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed99152390 0x55ed995365a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.242+0000 7f73aa00a640 1 -- 172.21.15.64:0/3175545014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9914a1b0 msgr2=0x55ed99536060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:54.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.242+0000 7f73aa00a640 1 --2- 172.21.15.64:0/3175545014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9914a1b0 0x55ed99536060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.242+0000 7f73aa00a640 1 -- 172.21.15.64:0/3175545014 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed9951d120 con 0x55ed99173070 2024-09-21T04:10:54.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.242+0000 7f73a9809640 1 --2- 172.21.15.64:0/3175545014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9914a1b0 0x55ed99536060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:54.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.243+0000 7f73aa00a640 1 --2- 172.21.15.64:0/3175545014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed99173070 0x55ed99517af0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f739c006010 tx=0x7f739c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e196a6874ce054c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:54.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.243+0000 7f738f7fe640 1 -- 172.21.15.64:0/3175545014 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c028030 con 0x55ed99173070 2024-09-21T04:10:54.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.243+0000 7f738f7fe640 1 -- 172.21.15.64:0/3175545014 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f739c005040 con 0x55ed99173070 2024-09-21T04:10:54.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.243+0000 7f738f7fe640 1 -- 172.21.15.64:0/3175545014 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c01e040 con 0x55ed99173070 2024-09-21T04:10:54.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.244+0000 7f73b1fe1740 1 -- 172.21.15.64:0/3175545014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed99173070 msgr2=0x55ed99517af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:54.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.244+0000 7f73b1fe1740 1 --2- 172.21.15.64:0/3175545014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed99173070 0x55ed99517af0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f739c006010 tx=0x7f739c009b50 comp rx=0 tx=0).stop 2024-09-21T04:10:54.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.244+0000 7f73b1fe1740 1 -- 172.21.15.64:0/3175545014 shutdown_connections 2024-09-21T04:10:54.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.244+0000 7f73b1fe1740 1 --2- 172.21.15.64:0/3175545014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed99173070 0x55ed99517af0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.244+0000 7f73b1fe1740 1 --2- 172.21.15.64:0/3175545014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed99152390 0x55ed995365a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.244+0000 7f73b1fe1740 1 --2- 172.21.15.64:0/3175545014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9914a1b0 0x55ed99536060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.244+0000 7f73b1fe1740 1 -- 172.21.15.64:0/3175545014 >> 172.21.15.64:0/3175545014 conn(0x55ed99167460 msgr2=0x55ed995242c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:54.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.244+0000 7f73b1fe1740 1 -- 172.21.15.64:0/3175545014 shutdown_connections 2024-09-21T04:10:54.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.244+0000 7f73b1fe1740 1 -- 172.21.15.64:0/3175545014 wait complete. 2024-09-21T04:10:54.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.244+0000 7f73b1fe1740 1 -- start start 2024-09-21T04:10:54.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.244+0000 7f73b1fe1740 1 Processor -- start 2024-09-21T04:10:54.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.245+0000 7f73b1fe1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9914a1b0 0x55ed995323e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:54.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.245+0000 7f73b1fe1740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed99152390 0x55ed995329a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:54.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.245+0000 7f73b1fe1740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed99173070 0x55ed9952f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:54.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.245+0000 7f73b1fe1740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f73a41abd40 con 0x55ed99152390 2024-09-21T04:10:54.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.245+0000 7f73b1fe1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f73a41ad1b0 con 0x55ed9914a1b0 2024-09-21T04:10:54.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.245+0000 7f73b1fe1740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f73a41ac410 con 0x55ed99173070 2024-09-21T04:10:54.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.245+0000 7f73b1fe1740 10 client.0 _ll_register_callbacks cb 0x7f73a0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:10:54.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.245+0000 7f73a9809640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9914a1b0 0x55ed995323e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:54.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.245+0000 7f73a9008640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed99152390 0x55ed995329a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:54.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.245+0000 7f73a9008640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed99152390 0x55ed995329a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36024/0 (socket says 172.21.15.64:36024) 2024-09-21T04:10:54.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.245+0000 7f73a9008640 1 -- 172.21.15.64:0/293438085 learned_addr learned my addr 172.21.15.64:0/293438085 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:54.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.245+0000 7f73aa00a640 1 --2- 172.21.15.64:0/293438085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed99173070 0x55ed9952f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:54.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.245+0000 7f73a9008640 1 -- 172.21.15.64:0/293438085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed99173070 msgr2=0x55ed9952f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:54.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.245+0000 7f73a9008640 1 --2- 172.21.15.64:0/293438085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed99173070 0x55ed9952f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.245+0000 7f73a9008640 1 -- 172.21.15.64:0/293438085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9914a1b0 msgr2=0x55ed995323e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.245+0000 7f73a9008640 1 --2- 172.21.15.64:0/293438085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9914a1b0 0x55ed995323e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.245+0000 7f73a9008640 1 -- 172.21.15.64:0/293438085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed9952f8a0 con 0x55ed99152390 2024-09-21T04:10:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.245+0000 7f73a9809640 1 --2- 172.21.15.64:0/293438085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9914a1b0 0x55ed995323e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.245+0000 7f73a9008640 1 --2- 172.21.15.64:0/293438085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed99152390 0x55ed995329a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f73940e5160 tx=0x7f7394004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf78d5aace7f0dd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.245+0000 7f738d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:10:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.246+0000 7f738d7fa640 1 -- 172.21.15.64:0/293438085 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73940ad240 con 0x55ed99152390 2024-09-21T04:10:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.246+0000 7f738d7fa640 1 -- 172.21.15.64:0/293438085 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7f73940b2170 con 0x55ed99152390 2024-09-21T04:10:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.246+0000 7f73b1fe1740 1 -- 172.21.15.64:0/293438085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f739c005040 con 0x55ed99152390 2024-09-21T04:10:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.246+0000 7f738d7fa640 1 -- 172.21.15.64:0/293438085 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f73940a9060 con 0x55ed99152390 2024-09-21T04:10:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.246+0000 7f73b1fe1740 10 client.16923 fetch_fsmap learned FSMap version 38 2024-09-21T04:10:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.246+0000 7f73b1fe1740 1 -- 172.21.15.64:0/293438085 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ed995a7660 con 0x55ed99152390 2024-09-21T04:10:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.246+0000 7f738d7fa640 1 -- 172.21.15.64:0/293438085 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f739c005040 con 0x55ed99152390 2024-09-21T04:10:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.246+0000 7f738d7fa640 1 -- 172.21.15.64:0/293438085 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f73940a9240 con 0x55ed99152390 2024-09-21T04:10:54.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.246+0000 7f73b1fe1740 10 client.16923 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T04:10:54.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.246+0000 7f73b1fe1740 10 client.16923 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:10:54.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.246+0000 7f73b1fe1740 20 client.16923 populate_metadata read hostname 'smithi064' 2024-09-21T04:10:54.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.246+0000 7f73b1fe1740 1 --2- 172.21.15.64:0/293438085 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55ed99106da0 0x55ed995a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:54.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.246+0000 7f73b1fe1740 4 client.16923 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:10:54.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.246+0000 7f73b1fe1740 1 -- 172.21.15.64:0/293438085 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed9948e8a0 con 0x55ed99106da0 2024-09-21T04:10:54.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.246+0000 7f73a9809640 1 --2- 172.21.15.64:0/293438085 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55ed99106da0 0x55ed995a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:54.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.247+0000 7f73a9809640 1 --2- 172.21.15.64:0/293438085 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55ed99106da0 0x55ed995a9e50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a532acd32f18a4a2 server_cookie=99f1d541c00ad488 in_seq=0 out_seq=0 2024-09-21T04:10:54.254 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.247+0000 7f738d7fa640 10 client.16923 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:10:54.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.249+0000 7f738d7fa640 1 -- 172.21.15.64:0/293438085 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed9948e8a0 con 0x55ed99106da0 2024-09-21T04:10:54.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.249+0000 7f738d7fa640 10 client.16923 handle_command_reply: tid=0 2024-09-21T04:10:54.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.249+0000 7f73b1fe1740 1 client.16923 shutdown 2024-09-21T04:10:54.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.250+0000 7f73b1fe1740 1 -- 172.21.15.64:0/293438085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed99152390 msgr2=0x55ed995329a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:54.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.250+0000 7f73b1fe1740 1 --2- 172.21.15.64:0/293438085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed99152390 0x55ed995329a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f73940e5160 tx=0x7f7394004a20 comp rx=0 tx=0).stop 2024-09-21T04:10:54.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.250+0000 7f73b1fe1740 1 -- 172.21.15.64:0/293438085 shutdown_connections 2024-09-21T04:10:54.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.250+0000 7f73b1fe1740 1 --2- 172.21.15.64:0/293438085 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55ed99106da0 0x55ed995a9e50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.255 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.250+0000 7f73b1fe1740 1 --2- 172.21.15.64:0/293438085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ed99173070 0x55ed9952f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.250+0000 7f73b1fe1740 1 --2- 172.21.15.64:0/293438085 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ed99152390 0x55ed995329a0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.250+0000 7f73b1fe1740 1 --2- 172.21.15.64:0/293438085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ed9914a1b0 0x55ed995323e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.250+0000 7f738d7fa640 0 client.16923 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:10:54.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.250+0000 7f73b1fe1740 1 -- 172.21.15.64:0/293438085 >> 172.21.15.64:0/293438085 conn(0x55ed99167460 msgr2=0x55ed995257f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:54.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.250+0000 7f73b1fe1740 1 -- 172.21.15.64:0/293438085 shutdown_connections 2024-09-21T04:10:54.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.250+0000 7f73b1fe1740 1 -- 172.21.15.64:0/293438085 wait complete. 2024-09-21T04:10:54.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.250+0000 7f73b1fe1740 20 client.16923 trim_cache size 0 max 16384 2024-09-21T04:10:54.256 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:10:54.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.251+0000 7f73aba6d640 1 -- 172.21.15.64:0/1644845355 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f73900b6680 msgr2=0x7f73900b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:54.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.251+0000 7f73aba6d640 1 --2- 172.21.15.64:0/1644845355 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f73900b6680 0x7f73900b8b90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f73a0000c00 tx=0x7f73a0031400 comp rx=0 tx=0).stop 2024-09-21T04:10:54.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.251+0000 7f73aba6d640 1 -- 172.21.15.64:0/1644845355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73a4105320 msgr2=0x7f73a419aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:54.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.251+0000 7f73aba6d640 1 --2- 172.21.15.64:0/1644845355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73a4105320 0x7f73a419aaa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f739400e9c0 tx=0x7f739400ee80 comp rx=0 tx=0).stop 2024-09-21T04:10:54.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.251+0000 7f73aba6d640 1 -- 172.21.15.64:0/1644845355 shutdown_connections 2024-09-21T04:10:54.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.251+0000 7f73aba6d640 1 --2- 172.21.15.64:0/1644845355 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f73900b6680 0x7f73900b8b90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.251+0000 7f73aba6d640 1 --2- 172.21.15.64:0/1644845355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f73a410de30 0x7f73a419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.251+0000 7f73aba6d640 1 --2- 172.21.15.64:0/1644845355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73a4105320 0x7f73a419aaa0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.251+0000 7f73aba6d640 1 --2- 172.21.15.64:0/1644845355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f73a41049d0 0x7f73a419a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:54.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.251+0000 7f73aba6d640 1 -- 172.21.15.64:0/1644845355 >> 172.21.15.64:0/1644845355 conn(0x7f73a4079740 msgr2=0x7f73a4100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:54.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.251+0000 7f73aba6d640 1 -- 172.21.15.64:0/1644845355 shutdown_connections 2024-09-21T04:10:54.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:54.251+0000 7f73aba6d640 1 -- 172.21.15.64:0/1644845355 wait complete. 2024-09-21T04:10:54.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:54 smithi119 ceph-mon[34811]: pgmap v529: 97 pgs: 97 active+clean; 804 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 637 op/s 2024-09-21T04:10:54.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:54 smithi064 ceph-mon[31180]: pgmap v529: 97 pgs: 97 active+clean; 804 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 637 op/s 2024-09-21T04:10:54.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:54 smithi131 ceph-mon[35193]: pgmap v529: 97 pgs: 97 active+clean; 804 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 637 op/s 2024-09-21T04:10:54.986 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T04:10:55.247 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:10:55.774 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.769+0000 7f6a8d80a640 1 -- 172.21.15.64:0/271064230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a88105170 msgr2=0x7f6a88105580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:55.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.769+0000 7f6a8d80a640 1 --2- 172.21.15.64:0/271064230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a88105170 0x7f6a88105580 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c009900 tx=0x7f6a7c02a410 comp rx=0 tx=0).stop 2024-09-21T04:10:55.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.769+0000 7f6a8d80a640 1 -- 172.21.15.64:0/271064230 shutdown_connections 2024-09-21T04:10:55.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.769+0000 7f6a8d80a640 1 --2- 172.21.15.64:0/271064230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a88105ac0 0x7f6a8810f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.769+0000 7f6a8d80a640 1 --2- 172.21.15.64:0/271064230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a88105170 0x7f6a88105580 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.769+0000 7f6a8d80a640 1 --2- 172.21.15.64:0/271064230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a88103160 0x7f6a88103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.769+0000 7f6a8d80a640 1 -- 172.21.15.64:0/271064230 >> 172.21.15.64:0/271064230 conn(0x7f6a88077c60 msgr2=0x7f6a880780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:55.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.769+0000 7f6a8d80a640 1 -- 172.21.15.64:0/271064230 shutdown_connections 2024-09-21T04:10:55.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.769+0000 7f6a8d80a640 1 -- 172.21.15.64:0/271064230 wait complete. 2024-09-21T04:10:55.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.770+0000 7f6a8d80a640 1 Processor -- start 2024-09-21T04:10:55.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.770+0000 7f6a8d80a640 1 -- start start 2024-09-21T04:10:55.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.771+0000 7f6a8d80a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a88103160 0x7f6a881a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:55.775 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.771+0000 7f6a8d80a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a88105170 0x7f6a881a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:55.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.771+0000 7f6a8d80a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a88105ac0 0x7f6a881a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:55.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.771+0000 7f6a8d80a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6a881129c0 con 0x7f6a88103160 2024-09-21T04:10:55.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.771+0000 7f6a8d80a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6a88112840 con 0x7f6a88105170 2024-09-21T04:10:55.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.771+0000 7f6a8d80a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6a88112b40 con 0x7f6a88105ac0 2024-09-21T04:10:55.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.771+0000 7f6a86ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a88103160 0x7f6a881a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:55.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.771+0000 7f6a86ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a88103160 0x7f6a881a0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36048/0 (socket says 172.21.15.64:36048) 2024-09-21T04:10:55.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.771+0000 7f6a867fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a88105170 0x7f6a881a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:55.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.771+0000 7f6a86ffd640 1 -- 172.21.15.64:0/3544817913 learned_addr learned my addr 172.21.15.64:0/3544817913 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:55.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.771+0000 7f6a877fe640 1 --2- 172.21.15.64:0/3544817913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a88105ac0 0x7f6a881a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:55.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.772+0000 7f6a867fc640 1 -- 172.21.15.64:0/3544817913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a88105ac0 msgr2=0x7f6a881a62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:55.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.772+0000 7f6a867fc640 1 --2- 172.21.15.64:0/3544817913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a88105ac0 0x7f6a881a62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.772+0000 7f6a867fc640 1 -- 172.21.15.64:0/3544817913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a88103160 msgr2=0x7f6a881a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:55.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.772+0000 7f6a867fc640 1 --2- 172.21.15.64:0/3544817913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a88103160 0x7f6a881a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.772+0000 7f6a867fc640 1 -- 172.21.15.64:0/3544817913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a881a69f0 con 0x7f6a88105170 2024-09-21T04:10:55.776 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.772+0000 7f6a86ffd640 1 --2- 172.21.15.64:0/3544817913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a88103160 0x7f6a881a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:10:55.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.772+0000 7f6a867fc640 1 --2- 172.21.15.64:0/3544817913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a88105170 0x7f6a881a1030 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c02a930 tx=0x7f6a7c0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:55.777 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.773+0000 7f6a8c808640 1 -- 172.21.15.64:0/3544817913 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c040070 con 0x7f6a88105170 2024-09-21T04:10:55.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.773+0000 7f6a8c808640 1 -- 172.21.15.64:0/3544817913 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a7c0316d0 con 0x7f6a88105170 2024-09-21T04:10:55.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.773+0000 7f6a8c808640 1 -- 172.21.15.64:0/3544817913 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c035020 con 0x7f6a88105170 2024-09-21T04:10:55.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.773+0000 7f6a8d80a640 1 -- 172.21.15.64:0/3544817913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a881a6c80 con 0x7f6a88105170 2024-09-21T04:10:55.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.773+0000 7f6a8d80a640 1 -- 172.21.15.64:0/3544817913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a881ad440 con 0x7f6a88105170 2024-09-21T04:10:55.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.779+0000 7f6a93d7e740 1 Processor -- start 2024-09-21T04:10:55.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.779+0000 7f6a93d7e740 1 -- start start 2024-09-21T04:10:55.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.779+0000 7f6a93d7e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a4e28ff070 0x55a4e2ca62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:55.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.779+0000 7f6a93d7e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4e28d61b0 0x55a4e2ca6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:55.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.779+0000 7f6a93d7e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a4e28de390 0x55a4e2ca6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:55.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.779+0000 7f6a93d7e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a4e2c1cea0 con 0x55a4e28de390 2024-09-21T04:10:55.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.779+0000 7f6a93d7e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a4e2c0c2d0 con 0x55a4e28d61b0 2024-09-21T04:10:55.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.779+0000 7f6a93d7e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a4e2c100e0 con 0x55a4e28ff070 2024-09-21T04:10:55.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.780+0000 7f6a877fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a4e28ff070 0x55a4e2ca62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:55.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.780+0000 7f6a877fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a4e28ff070 0x55a4e2ca62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:46994/0 (socket says 172.21.15.64:46994) 2024-09-21T04:10:55.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.780+0000 7f6a877fe640 1 -- 172.21.15.64:0/1870147183 learned_addr learned my addr 172.21.15.64:0/1870147183 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:55.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.780+0000 7f6a86ffd640 1 --2- 172.21.15.64:0/1870147183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4e28d61b0 0x55a4e2ca6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:55.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.780+0000 7f6a8c808640 1 -- 172.21.15.64:0/3544817913 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f6a7c043070 con 0x7f6a88105170 2024-09-21T04:10:55.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.781+0000 7f6a867fc640 1 --2- 172.21.15.64:0/1870147183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a4e28de390 0x55a4e2ca6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:55.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.781+0000 7f6a867fc640 1 -- 172.21.15.64:0/1870147183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a4e28ff070 msgr2=0x55a4e2ca62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:55.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.781+0000 7f6a867fc640 1 --2- 172.21.15.64:0/1870147183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a4e28ff070 0x55a4e2ca62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.781+0000 7f6a867fc640 1 -- 172.21.15.64:0/1870147183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4e28d61b0 msgr2=0x55a4e2ca6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:55.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.781+0000 7f6a867fc640 1 --2- 172.21.15.64:0/1870147183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4e28d61b0 0x55a4e2ca6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.782+0000 7f6a867fc640 1 -- 172.21.15.64:0/1870147183 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4e296a650 con 0x55a4e28de390 2024-09-21T04:10:55.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.782+0000 7f6a877fe640 1 --2- 172.21.15.64:0/1870147183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a4e28ff070 0x55a4e2ca62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:55.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.782+0000 7f6a8c808640 1 --2- 172.21.15.64:0/3544817913 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6a700b6850 0x7f6a700b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:55.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.783+0000 7f6a877fe640 1 --2- 172.21.15.64:0/3544817913 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6a700b6850 0x7f6a700b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:55.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.783+0000 7f6a8c808640 1 -- 172.21.15.64:0/3544817913 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7f6a7c108010 con 0x7f6a88105170 2024-09-21T04:10:55.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.783+0000 7f6a867fc640 1 --2- 172.21.15.64:0/1870147183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a4e28de390 0x55a4e2ca6d40 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c1056a0 tx=0x7f6a7c110220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90567f79fc77533 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:55.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.783+0000 7f6a6dffb640 1 -- 172.21.15.64:0/1870147183 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c0cb430 con 0x55a4e28de390 2024-09-21T04:10:55.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.783+0000 7f6a6dffb640 1 -- 172.21.15.64:0/1870147183 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a7c0ce8f0 con 0x55a4e28de390 2024-09-21T04:10:55.786 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.783+0000 7f6a6dffb640 1 -- 172.21.15.64:0/1870147183 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c0ce550 con 0x55a4e28de390 2024-09-21T04:10:55.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.784+0000 7f6a877fe640 1 --2- 172.21.15.64:0/3544817913 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6a700b6850 0x7f6a700b8d60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6a80011890 tx=0x7f6a80009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:55.787 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.784+0000 7f6a93d7e740 1 -- 172.21.15.64:0/1870147183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a4e28de390 msgr2=0x55a4e2ca6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:55.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.784+0000 7f6a93d7e740 1 --2- 172.21.15.64:0/1870147183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a4e28de390 0x55a4e2ca6d40 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c1056a0 tx=0x7f6a7c110220 comp rx=0 tx=0).stop 2024-09-21T04:10:55.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.784+0000 7f6a93d7e740 1 -- 172.21.15.64:0/1870147183 shutdown_connections 2024-09-21T04:10:55.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.784+0000 7f6a93d7e740 1 --2- 172.21.15.64:0/1870147183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a4e28de390 0x55a4e2ca6d40 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.784+0000 7f6a93d7e740 1 --2- 172.21.15.64:0/1870147183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4e28d61b0 0x55a4e2ca6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.784+0000 7f6a93d7e740 1 --2- 172.21.15.64:0/1870147183 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a4e28ff070 0x55a4e2ca62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.784+0000 7f6a93d7e740 1 -- 172.21.15.64:0/1870147183 >> 172.21.15.64:0/1870147183 conn(0x55a4e28f3460 msgr2=0x55a4e2ca41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:55.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.785+0000 7f6a93d7e740 1 -- 172.21.15.64:0/1870147183 shutdown_connections 2024-09-21T04:10:55.788 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.785+0000 7f6a93d7e740 1 -- 172.21.15.64:0/1870147183 wait complete. 2024-09-21T04:10:55.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.785+0000 7f6a93d7e740 1 -- start start 2024-09-21T04:10:55.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.786+0000 7f6a93d7e740 1 Processor -- start 2024-09-21T04:10:55.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.787+0000 7f6a93d7e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a4e28d61b0 0x55a4e2cb1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:55.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.787+0000 7f6a93d7e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a4e28de390 0x55a4e2cb1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:55.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.787+0000 7f6a93d7e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4e28ff070 0x55a4e2cb2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:55.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.787+0000 7f6a93d7e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a4e2a6d700 con 0x55a4e28de390 2024-09-21T04:10:55.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.787+0000 7f6a93d7e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a4e2ba7160 con 0x55a4e28ff070 2024-09-21T04:10:55.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.787+0000 7f6a93d7e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a4e2cc5870 con 0x55a4e28d61b0 2024-09-21T04:10:55.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.787+0000 7f6a867fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a4e28de390 0x55a4e2cb1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:55.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.787+0000 7f6a867fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a4e28de390 0x55a4e2cb1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36054/0 (socket says 172.21.15.64:36054) 2024-09-21T04:10:55.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.787+0000 7f6a867fc640 1 -- 172.21.15.64:0/968923626 learned_addr learned my addr 172.21.15.64:0/968923626 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:55.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.787+0000 7f6a877fe640 1 --2- 172.21.15.64:0/968923626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4e28ff070 0x55a4e2cb2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:55.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.787+0000 7f6a86ffd640 1 --2- 172.21.15.64:0/968923626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a4e28d61b0 0x55a4e2cb1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:55.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.788+0000 7f6a867fc640 1 -- 172.21.15.64:0/968923626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a4e28d61b0 msgr2=0x55a4e2cb1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:55.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.788+0000 7f6a867fc640 1 --2- 172.21.15.64:0/968923626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a4e28d61b0 0x55a4e2cb1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.788+0000 7f6a867fc640 1 -- 172.21.15.64:0/968923626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4e28ff070 msgr2=0x55a4e2cb2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:55.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.788+0000 7f6a867fc640 1 --2- 172.21.15.64:0/968923626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4e28ff070 0x55a4e2cb2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.788+0000 7f6a867fc640 1 -- 172.21.15.64:0/968923626 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4e2bd4c90 con 0x55a4e28de390 2024-09-21T04:10:55.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.787+0000 7f6a93d7e740 10 client.0 _ll_register_callbacks cb 0x55a4e2bc09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:10:55.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.788+0000 7f6a867fc640 1 --2- 172.21.15.64:0/968923626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a4e28de390 0x55a4e2cb1e40 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c0ceae0 tx=0x7f6a7c0047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34bfc7b0e2a00592 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:55.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.788+0000 7f6a93d7e740 1 -- 172.21.15.64:0/968923626 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a7c0ce8f0 con 0x55a4e28de390 2024-09-21T04:10:55.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.788+0000 7f6a66ffd640 10 client.16935 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:10:55.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.788+0000 7f6a66ffd640 1 -- 172.21.15.64:0/968923626 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c04a040 con 0x55a4e28de390 2024-09-21T04:10:55.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.788+0000 7f6a66ffd640 1 -- 172.21.15.64:0/968923626 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7f6a7c0f5160 con 0x55a4e28de390 2024-09-21T04:10:55.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.789+0000 7f6a66ffd640 1 -- 172.21.15.64:0/968923626 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f6a7c0ce730 con 0x55a4e28de390 2024-09-21T04:10:55.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.789+0000 7f6a93d7e740 10 client.16935 fetch_fsmap learned FSMap version 38 2024-09-21T04:10:55.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.789+0000 7f6a93d7e740 1 -- 172.21.15.64:0/968923626 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({fsmap=38}) -- 0x55a4e2cd1300 con 0x55a4e28de390 2024-09-21T04:10:55.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.790+0000 7f6a66ffd640 1 -- 172.21.15.64:0/968923626 <== mon.0 v2:172.21.15.64:3300/0 4 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f6a7c03a040 con 0x55a4e28de390 2024-09-21T04:10:55.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.790+0000 7f6a93d7e740 10 client.16935 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T04:10:55.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.790+0000 7f6a93d7e740 10 client.16935 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:10:55.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.790+0000 7f6a93d7e740 20 client.16935 populate_metadata read hostname 'smithi064' 2024-09-21T04:10:55.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.790+0000 7f6a93d7e740 1 --2- 172.21.15.64:0/968923626 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a4e28bd9f0 0x55a4e2cd4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:55.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.791+0000 7f6a93d7e740 4 client.16935 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:10:55.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.791+0000 7f6a93d7e740 1 -- 172.21.15.64:0/968923626 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6a7c0ce8f0 con 0x55a4e28bd9f0 2024-09-21T04:10:55.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.792+0000 7f6a86ffd640 1 --2- 172.21.15.64:0/968923626 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a4e28bd9f0 0x55a4e2cd4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:55.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.794+0000 7f6a66ffd640 10 client.16935 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:55.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.794+0000 7f6a86ffd640 1 --2- 172.21.15.64:0/968923626 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a4e28bd9f0 0x55a4e2cd4870 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5b9613a6459d201 server_cookie=ee6622ab2d474d1a in_seq=0 out_seq=0 2024-09-21T04:10:55.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.795+0000 7f6a66ffd640 1 -- 172.21.15.64:0/968923626 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f6a7c0ce8f0 con 0x55a4e28bd9f0 2024-09-21T04:10:55.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.795+0000 7f6a66ffd640 10 client.16935 handle_command_reply: tid=0 2024-09-21T04:10:55.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.795+0000 7f6a93d7e740 1 client.16935 shutdown 2024-09-21T04:10:55.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.796+0000 7f6a93d7e740 1 -- 172.21.15.64:0/968923626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a4e28de390 msgr2=0x55a4e2cb1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:55.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.796+0000 7f6a93d7e740 1 --2- 172.21.15.64:0/968923626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a4e28de390 0x55a4e2cb1e40 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c0ceae0 tx=0x7f6a7c0047a0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.796+0000 7f6a93d7e740 1 -- 172.21.15.64:0/968923626 shutdown_connections 2024-09-21T04:10:55.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.796+0000 7f6a93d7e740 1 --2- 172.21.15.64:0/968923626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4e28ff070 0x55a4e2cb2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.797+0000 7f6a93d7e740 1 --2- 172.21.15.64:0/968923626 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a4e28bd9f0 0x55a4e2cd4870 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.797+0000 7f6a93d7e740 1 --2- 172.21.15.64:0/968923626 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a4e28de390 0x55a4e2cb1e40 secure :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c0ceae0 tx=0x7f6a7c0047a0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.797+0000 7f6a93d7e740 1 --2- 172.21.15.64:0/968923626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a4e28d61b0 0x55a4e2cb1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.797+0000 7f6a93d7e740 1 -- 172.21.15.64:0/968923626 >> 172.21.15.64:0/968923626 conn(0x55a4e28f3460 msgr2=0x55a4e2ca9450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:55.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.797+0000 7f6a66ffd640 0 client.16935 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:55.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.797+0000 7f6a93d7e740 1 -- 172.21.15.64:0/968923626 shutdown_connections 2024-09-21T04:10:55.800 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.797+0000 7f6a93d7e740 1 -- 172.21.15.64:0/968923626 wait complete. 2024-09-21T04:10:55.801 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.798+0000 7f6a93d7e740 20 client.16935 trim_cache size 0 max 16384 2024-09-21T04:10:55.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.818+0000 7f6a93d7e740 1 Processor -- start 2024-09-21T04:10:55.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.818+0000 7f6a93d7e740 1 -- start start 2024-09-21T04:10:55.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.818+0000 7f6a93d7e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a4e28d61b0 0x55a4e2cb37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:55.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.818+0000 7f6a93d7e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4e28de390 0x55a4e2ca7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:55.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.818+0000 7f6a93d7e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a4e28ff070 0x55a4e2caeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:55.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.818+0000 7f6a93d7e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6a74011300 con 0x55a4e28ff070 2024-09-21T04:10:55.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.818+0000 7f6a93d7e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7400eb70 con 0x55a4e28de390 2024-09-21T04:10:55.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.818+0000 7f6a93d7e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7400c9e0 con 0x55a4e28d61b0 2024-09-21T04:10:55.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.819+0000 7f6a86ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a4e28d61b0 0x55a4e2cb37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:55.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.819+0000 7f6a877fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a4e28ff070 0x55a4e2caeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:55.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.819+0000 7f6a86ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a4e28d61b0 0x55a4e2cb37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:47022/0 (socket says 172.21.15.64:47022) 2024-09-21T04:10:55.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.819+0000 7f6a86ffd640 1 -- 172.21.15.64:0/2351278205 learned_addr learned my addr 172.21.15.64:0/2351278205 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:55.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.819+0000 7f6a86ffd640 1 -- 172.21.15.64:0/2351278205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4e28de390 msgr2=0x55a4e2ca7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:55.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.819+0000 7f6a867fc640 1 --2- 172.21.15.64:0/2351278205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4e28de390 0x55a4e2ca7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:55.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.819+0000 7f6a86ffd640 1 --2- 172.21.15.64:0/2351278205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4e28de390 0x55a4e2ca7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.819+0000 7f6a86ffd640 1 -- 172.21.15.64:0/2351278205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a4e28ff070 msgr2=0x55a4e2caeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:55.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.819+0000 7f6a86ffd640 1 --2- 172.21.15.64:0/2351278205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a4e28ff070 0x55a4e2caeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.819+0000 7f6a86ffd640 1 -- 172.21.15.64:0/2351278205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4e2ccea60 con 0x55a4e28d61b0 2024-09-21T04:10:55.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.819+0000 7f6a877fe640 1 --2- 172.21.15.64:0/2351278205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a4e28ff070 0x55a4e2caeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:55.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.819+0000 7f6a867fc640 1 --2- 172.21.15.64:0/2351278205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4e28de390 0x55a4e2ca7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:10:55.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.820+0000 7f6a86ffd640 1 --2- 172.21.15.64:0/2351278205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a4e28d61b0 0x55a4e2cb37e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6a78000c00 tx=0x7f6a7800bde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ae47a6130adaf1c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:55.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.820+0000 7f6a6d7fa640 1 -- 172.21.15.64:0/2351278205 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78003ae0 con 0x55a4e28d61b0 2024-09-21T04:10:55.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.820+0000 7f6a6d7fa640 1 -- 172.21.15.64:0/2351278205 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a78003c80 con 0x55a4e28d61b0 2024-09-21T04:10:55.823 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.820+0000 7f6a6d7fa640 1 -- 172.21.15.64:0/2351278205 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a780116a0 con 0x55a4e28d61b0 2024-09-21T04:10:55.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.821+0000 7f6a93d7e740 1 -- 172.21.15.64:0/2351278205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a4e28d61b0 msgr2=0x55a4e2cb37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:55.824 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.822+0000 7f6a93d7e740 1 --2- 172.21.15.64:0/2351278205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a4e28d61b0 0x55a4e2cb37e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6a78000c00 tx=0x7f6a7800bde0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.822+0000 7f6a93d7e740 1 -- 172.21.15.64:0/2351278205 shutdown_connections 2024-09-21T04:10:55.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.822+0000 7f6a93d7e740 1 --2- 172.21.15.64:0/2351278205 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a4e28ff070 0x55a4e2caeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.822+0000 7f6a93d7e740 1 --2- 172.21.15.64:0/2351278205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4e28de390 0x55a4e2ca7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.825 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.823+0000 7f6a93d7e740 1 --2- 172.21.15.64:0/2351278205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a4e28d61b0 0x55a4e2cb37e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.823+0000 7f6a93d7e740 1 -- 172.21.15.64:0/2351278205 >> 172.21.15.64:0/2351278205 conn(0x55a4e28f3460 msgr2=0x55a4e2ca72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:55.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.823+0000 7f6a93d7e740 1 -- 172.21.15.64:0/2351278205 shutdown_connections 2024-09-21T04:10:55.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.823+0000 7f6a93d7e740 1 -- 172.21.15.64:0/2351278205 wait complete. 2024-09-21T04:10:55.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.823+0000 7f6a93d7e740 1 -- start start 2024-09-21T04:10:55.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.824+0000 7f6a93d7e740 1 Processor -- start 2024-09-21T04:10:55.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.824+0000 7f6a93d7e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4e28d61b0 0x55a4e2ccaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:55.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.824+0000 7f6a93d7e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a4e28de390 0x55a4e2ccb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:55.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.824+0000 7f6a93d7e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a4e28ff070 0x55a4e2cc64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:55.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.825+0000 7f6a93d7e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7400d030 con 0x55a4e28ff070 2024-09-21T04:10:55.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.825+0000 7f6a93d7e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7400e4a0 con 0x55a4e28d61b0 2024-09-21T04:10:55.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.825+0000 7f6a93d7e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7400d700 con 0x55a4e28de390 2024-09-21T04:10:55.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.825+0000 7f6a93d7e740 10 client.0 _ll_register_callbacks cb 0x7f6a7800b580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:10:55.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.825+0000 7f6a877fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a4e28ff070 0x55a4e2cc64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:55.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.825+0000 7f6a877fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a4e28ff070 0x55a4e2cc64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36082/0 (socket says 172.21.15.64:36082) 2024-09-21T04:10:55.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.825+0000 7f6a877fe640 1 -- 172.21.15.64:0/2845640367 learned_addr learned my addr 172.21.15.64:0/2845640367 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:55.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.825+0000 7f6a877fe640 1 -- 172.21.15.64:0/2845640367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a4e28de390 msgr2=0x55a4e2ccb460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:10:55.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.825+0000 7f6a877fe640 1 --2- 172.21.15.64:0/2845640367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a4e28de390 0x55a4e2ccb460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.825+0000 7f6a877fe640 1 -- 172.21.15.64:0/2845640367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4e28d61b0 msgr2=0x55a4e2ccaea0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T04:10:55.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.825+0000 7f6a877fe640 1 --2- 172.21.15.64:0/2845640367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4e28d61b0 0x55a4e2ccaea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.825+0000 7f6a877fe640 1 -- 172.21.15.64:0/2845640367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4e2cc6c20 con 0x55a4e28ff070 2024-09-21T04:10:55.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.826+0000 7f6a877fe640 1 --2- 172.21.15.64:0/2845640367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a4e28ff070 0x55a4e2cc64b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f6a80003f30 tx=0x7f6a80004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344f777ac91d3383 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:55.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.826+0000 7f6a66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:10:55.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.826+0000 7f6a66ffd640 1 -- 172.21.15.64:0/2845640367 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80004250 con 0x55a4e28ff070 2024-09-21T04:10:55.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.826+0000 7f6a66ffd640 1 -- 172.21.15.64:0/2845640367 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7f6a800043f0 con 0x55a4e28ff070 2024-09-21T04:10:55.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.827+0000 7f6a93d7e740 1 -- 172.21.15.64:0/2845640367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a78003c80 con 0x55a4e28ff070 2024-09-21T04:10:55.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.827+0000 7f6a66ffd640 1 -- 172.21.15.64:0/2845640367 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f6a800045f0 con 0x55a4e28ff070 2024-09-21T04:10:55.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.828+0000 7f6a93d7e740 10 client.16941 fetch_fsmap learned FSMap version 38 2024-09-21T04:10:55.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.828+0000 7f6a93d7e740 1 -- 172.21.15.64:0/2845640367 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55a4e2d3ed40 con 0x55a4e28ff070 2024-09-21T04:10:55.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.828+0000 7f6a66ffd640 1 -- 172.21.15.64:0/2845640367 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a78003c80 con 0x55a4e28ff070 2024-09-21T04:10:55.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.828+0000 7f6a66ffd640 1 -- 172.21.15.64:0/2845640367 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f6a8001c410 con 0x55a4e28ff070 2024-09-21T04:10:55.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.828+0000 7f6a93d7e740 10 client.16941 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T04:10:55.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.828+0000 7f6a93d7e740 10 client.16941 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:10:55.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.828+0000 7f6a93d7e740 20 client.16941 populate_metadata read hostname 'smithi064' 2024-09-21T04:10:55.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.828+0000 7f6a93d7e740 1 --2- 172.21.15.64:0/2845640367 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a4e2892da0 0x55a4e2d41620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:55.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.828+0000 7f6a93d7e740 4 client.16941 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:10:55.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.828+0000 7f6a93d7e740 1 -- 172.21.15.64:0/2845640367 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a4e2c1edf0 con 0x55a4e2892da0 2024-09-21T04:10:55.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.829+0000 7f6a86ffd640 1 --2- 172.21.15.64:0/2845640367 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a4e2892da0 0x55a4e2d41620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:55.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.830+0000 7f6a86ffd640 1 --2- 172.21.15.64:0/2845640367 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a4e2892da0 0x55a4e2d41620 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21243b391ec7eec5 server_cookie=5775246e398273c0 in_seq=0 out_seq=0 2024-09-21T04:10:55.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.830+0000 7f6a66ffd640 10 client.16941 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:55.833 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.831+0000 7f6a66ffd640 1 -- 172.21.15.64:0/2845640367 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a4e2c1edf0 con 0x55a4e2892da0 2024-09-21T04:10:55.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.831+0000 7f6a66ffd640 10 client.16941 handle_command_reply: tid=0 2024-09-21T04:10:55.834 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.831+0000 7f6a93d7e740 1 client.16941 shutdown 2024-09-21T04:10:55.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.831+0000 7f6a93d7e740 1 -- 172.21.15.64:0/2845640367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a4e28ff070 msgr2=0x55a4e2cc64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:55.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.831+0000 7f6a93d7e740 1 --2- 172.21.15.64:0/2845640367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a4e28ff070 0x55a4e2cc64b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f6a80003f30 tx=0x7f6a80004ab0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.832+0000 7f6a93d7e740 1 -- 172.21.15.64:0/2845640367 shutdown_connections 2024-09-21T04:10:55.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.832+0000 7f6a93d7e740 1 --2- 172.21.15.64:0/2845640367 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a4e2892da0 0x55a4e2d41620 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.835 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.832+0000 7f6a93d7e740 1 --2- 172.21.15.64:0/2845640367 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a4e28ff070 0x55a4e2cc64b0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.832+0000 7f6a93d7e740 1 --2- 172.21.15.64:0/2845640367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a4e28de390 0x55a4e2ccb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.832+0000 7f6a93d7e740 1 --2- 172.21.15.64:0/2845640367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a4e28d61b0 0x55a4e2ccaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.832+0000 7f6a93d7e740 1 -- 172.21.15.64:0/2845640367 >> 172.21.15.64:0/2845640367 conn(0x55a4e28f3460 msgr2=0x55a4e2cce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:55.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.832+0000 7f6a66ffd640 0 client.16941 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:55.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.832+0000 7f6a93d7e740 1 -- 172.21.15.64:0/2845640367 shutdown_connections 2024-09-21T04:10:55.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.832+0000 7f6a93d7e740 1 -- 172.21.15.64:0/2845640367 wait complete. 2024-09-21T04:10:55.836 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.832+0000 7f6a93d7e740 20 client.16941 trim_cache size 0 max 16384 2024-09-21T04:10:55.836 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:10:55.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.833+0000 7f6a8d80a640 1 -- 172.21.15.64:0/3544817913 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6a700b6850 msgr2=0x7f6a700b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:55.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.833+0000 7f6a8d80a640 1 --2- 172.21.15.64:0/3544817913 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6a700b6850 0x7f6a700b8d60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6a80011890 tx=0x7f6a80009500 comp rx=0 tx=0).stop 2024-09-21T04:10:55.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.833+0000 7f6a8d80a640 1 -- 172.21.15.64:0/3544817913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a88105170 msgr2=0x7f6a881a1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:55.837 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.833+0000 7f6a8d80a640 1 --2- 172.21.15.64:0/3544817913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a88105170 0x7f6a881a1030 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c02a930 tx=0x7f6a7c0045e0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.835+0000 7f6a8d80a640 1 -- 172.21.15.64:0/3544817913 shutdown_connections 2024-09-21T04:10:55.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.835+0000 7f6a8d80a640 1 --2- 172.21.15.64:0/3544817913 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f6a700b6850 0x7f6a700b8d60 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.835+0000 7f6a8d80a640 1 --2- 172.21.15.64:0/3544817913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6a88105ac0 0x7f6a881a62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.835+0000 7f6a8d80a640 1 --2- 172.21.15.64:0/3544817913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a88105170 0x7f6a881a1030 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.835+0000 7f6a8d80a640 1 --2- 172.21.15.64:0/3544817913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a88103160 0x7f6a881a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:55.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.835+0000 7f6a8d80a640 1 -- 172.21.15.64:0/3544817913 >> 172.21.15.64:0/3544817913 conn(0x7f6a88077c60 msgr2=0x7f6a881061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:55.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.835+0000 7f6a8d80a640 1 -- 172.21.15.64:0/3544817913 shutdown_connections 2024-09-21T04:10:55.838 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:55.835+0000 7f6a8d80a640 1 -- 172.21.15.64:0/3544817913 wait complete. 2024-09-21T04:10:56.503 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:56 smithi064 ceph-mon[31180]: pgmap v530: 97 pgs: 97 active+clean; 804 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 371 op/s 2024-09-21T04:10:56.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:56 smithi119 ceph-mon[34811]: pgmap v530: 97 pgs: 97 active+clean; 804 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 371 op/s 2024-09-21T04:10:56.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:56 smithi131 ceph-mon[35193]: pgmap v530: 97 pgs: 97 active+clean; 804 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 371 op/s 2024-09-21T04:10:57.533 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T04:10:57.533 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T04:10:57.813 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:10:58.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.316+0000 7fc514703640 1 -- 172.21.15.64:0/2138367968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc50c101fd0 msgr2=0x7fc50c1023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:58.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.316+0000 7fc514703640 1 --2- 172.21.15.64:0/2138367968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc50c101fd0 0x7fc50c1023e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc508009900 tx=0x7fc50802a410 comp rx=0 tx=0).stop 2024-09-21T04:10:58.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.316+0000 7fc514703640 1 -- 172.21.15.64:0/2138367968 shutdown_connections 2024-09-21T04:10:58.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.316+0000 7fc514703640 1 --2- 172.21.15.64:0/2138367968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc50c106c00 0x7fc50c110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.316+0000 7fc514703640 1 --2- 172.21.15.64:0/2138367968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc50c1062b0 0x7fc50c1066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.316+0000 7fc514703640 1 --2- 172.21.15.64:0/2138367968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc50c101fd0 0x7fc50c1023e0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.316+0000 7fc514703640 1 -- 172.21.15.64:0/2138367968 >> 172.21.15.64:0/2138367968 conn(0x7fc50c079e10 msgr2=0x7fc50c07a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:58.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.316+0000 7fc514703640 1 -- 172.21.15.64:0/2138367968 shutdown_connections 2024-09-21T04:10:58.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.316+0000 7fc514703640 1 -- 172.21.15.64:0/2138367968 wait complete. 2024-09-21T04:10:58.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.317+0000 7fc514703640 1 Processor -- start 2024-09-21T04:10:58.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.317+0000 7fc514703640 1 -- start start 2024-09-21T04:10:58.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.317+0000 7fc514703640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc50c101fd0 0x7fc50c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:58.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.317+0000 7fc514703640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc50c1062b0 0x7fc50c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:58.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.317+0000 7fc514703640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc50c106c00 0x7fc50c19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:58.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.317+0000 7fc514703640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc50c113b00 con 0x7fc50c106c00 2024-09-21T04:10:58.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.317+0000 7fc514703640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc50c113980 con 0x7fc50c1062b0 2024-09-21T04:10:58.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.317+0000 7fc514703640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc50c113c80 con 0x7fc50c101fd0 2024-09-21T04:10:58.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.317+0000 7fc512ca0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc50c106c00 0x7fc50c19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:58.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.317+0000 7fc511c9e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc50c1062b0 0x7fc50c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:58.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.317+0000 7fc512ca0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc50c106c00 0x7fc50c19fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36092/0 (socket says 172.21.15.64:36092) 2024-09-21T04:10:58.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.317+0000 7fc511c9e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc50c1062b0 0x7fc50c19aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:39234/0 (socket says 172.21.15.64:39234) 2024-09-21T04:10:58.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.317+0000 7fc512ca0640 1 -- 172.21.15.64:0/3980799014 learned_addr learned my addr 172.21.15.64:0/3980799014 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:58.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.317+0000 7fc51249f640 1 --2- 172.21.15.64:0/3980799014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc50c101fd0 0x7fc50c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:58.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.318+0000 7fc511c9e640 1 -- 172.21.15.64:0/3980799014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc50c101fd0 msgr2=0x7fc50c19a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:58.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.318+0000 7fc511c9e640 1 --2- 172.21.15.64:0/3980799014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc50c101fd0 0x7fc50c19a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.318+0000 7fc511c9e640 1 -- 172.21.15.64:0/3980799014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc50c106c00 msgr2=0x7fc50c19fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:58.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.318+0000 7fc511c9e640 1 --2- 172.21.15.64:0/3980799014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc50c106c00 0x7fc50c19fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.318+0000 7fc511c9e640 1 -- 172.21.15.64:0/3980799014 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50c1a0440 con 0x7fc50c1062b0 2024-09-21T04:10:58.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.318+0000 7fc512ca0640 1 --2- 172.21.15.64:0/3980799014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc50c106c00 0x7fc50c19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:58.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.318+0000 7fc511c9e640 1 --2- 172.21.15.64:0/3980799014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc50c1062b0 0x7fc50c19aa70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc50c075ab0 tx=0x7fc4fc00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:58.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.318+0000 7fc5037fe640 1 -- 172.21.15.64:0/3980799014 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc00cc40 con 0x7fc50c1062b0 2024-09-21T04:10:58.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.318+0000 7fc514703640 1 -- 172.21.15.64:0/3980799014 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc50c1a0730 con 0x7fc50c1062b0 2024-09-21T04:10:58.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.318+0000 7fc5037fe640 1 -- 172.21.15.64:0/3980799014 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4fc00cde0 con 0x7fc50c1062b0 2024-09-21T04:10:58.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.318+0000 7fc5037fe640 1 -- 172.21.15.64:0/3980799014 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc018760 con 0x7fc50c1062b0 2024-09-21T04:10:58.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.318+0000 7fc514703640 1 -- 172.21.15.64:0/3980799014 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc50c1a6e90 con 0x7fc50c1062b0 2024-09-21T04:10:58.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.321+0000 7fc51ac77740 1 Processor -- start 2024-09-21T04:10:58.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.321+0000 7fc51ac77740 1 -- start start 2024-09-21T04:10:58.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.322+0000 7fc51ac77740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f27af9350 0x564f27ea0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:58.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.322+0000 7fc51ac77740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f27ad0490 0x564f27ea0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:58.352 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.322+0000 7fc51ac77740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f27ad8670 0x564f27ea0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:58.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.322+0000 7fc51ac77740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564f27e16e60 con 0x564f27ad0490 2024-09-21T04:10:58.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.322+0000 7fc51ac77740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564f27e06290 con 0x564f27af9350 2024-09-21T04:10:58.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.322+0000 7fc51ac77740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564f27e0a0a0 con 0x564f27ad8670 2024-09-21T04:10:58.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.322+0000 7fc51249f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f27ad0490 0x564f27ea0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:58.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.322+0000 7fc512ca0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f27af9350 0x564f27ea0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:58.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.322+0000 7fc512ca0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f27af9350 0x564f27ea0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:39244/0 (socket says 172.21.15.64:39244) 2024-09-21T04:10:58.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.322+0000 7fc512ca0640 1 -- 172.21.15.64:0/2555136844 learned_addr learned my addr 172.21.15.64:0/2555136844 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:58.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.322+0000 7fc512ca0640 1 -- 172.21.15.64:0/2555136844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f27ad8670 msgr2=0x564f27ea0eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:10:58.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.322+0000 7fc512ca0640 1 --2- 172.21.15.64:0/2555136844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f27ad8670 0x564f27ea0eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.322+0000 7fc512ca0640 1 -- 172.21.15.64:0/2555136844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f27ad0490 msgr2=0x564f27ea0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:58.353 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.322+0000 7fc512ca0640 1 --2- 172.21.15.64:0/2555136844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f27ad0490 0x564f27ea0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.322+0000 7fc512ca0640 1 -- 172.21.15.64:0/2555136844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f27b64930 con 0x564f27af9350 2024-09-21T04:10:58.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.322+0000 7fc5037fe640 1 -- 172.21.15.64:0/3980799014 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fc4fc010a90 con 0x7fc50c1062b0 2024-09-21T04:10:58.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.322+0000 7fc512ca0640 1 --2- 172.21.15.64:0/2555136844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f27af9350 0x564f27ea0430 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc5040117e0 tx=0x7fc504011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d708626172de113 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:58.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.323+0000 7fc500ff9640 1 -- 172.21.15.64:0/2555136844 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50400fb40 con 0x564f27af9350 2024-09-21T04:10:58.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.323+0000 7fc500ff9640 1 -- 172.21.15.64:0/2555136844 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc50400fce0 con 0x564f27af9350 2024-09-21T04:10:58.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.323+0000 7fc500ff9640 1 -- 172.21.15.64:0/2555136844 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc504010630 con 0x564f27af9350 2024-09-21T04:10:58.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.323+0000 7fc5037fe640 1 --2- 172.21.15.64:0/3980799014 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc4f80b6680 0x7fc4f80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:58.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.323+0000 7fc51249f640 1 --2- 172.21.15.64:0/3980799014 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc4f80b6680 0x7fc4f80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:58.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.323+0000 7fc5037fe640 1 -- 172.21.15.64:0/3980799014 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7fc4fc0e7a00 con 0x7fc50c1062b0 2024-09-21T04:10:58.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.324+0000 7fc51ac77740 1 -- 172.21.15.64:0/2555136844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f27af9350 msgr2=0x564f27ea0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:58.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.324+0000 7fc51ac77740 1 --2- 172.21.15.64:0/2555136844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f27af9350 0x564f27ea0430 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc5040117e0 tx=0x7fc504011ca0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.324+0000 7fc51ac77740 1 -- 172.21.15.64:0/2555136844 shutdown_connections 2024-09-21T04:10:58.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.324+0000 7fc51ac77740 1 --2- 172.21.15.64:0/2555136844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f27ad8670 0x564f27ea0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.324+0000 7fc51ac77740 1 --2- 172.21.15.64:0/2555136844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f27ad0490 0x564f27ea0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.324+0000 7fc51ac77740 1 --2- 172.21.15.64:0/2555136844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f27af9350 0x564f27ea0430 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.324+0000 7fc51ac77740 1 -- 172.21.15.64:0/2555136844 >> 172.21.15.64:0/2555136844 conn(0x564f27aed740 msgr2=0x564f27e9e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:58.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.324+0000 7fc51ac77740 1 -- 172.21.15.64:0/2555136844 shutdown_connections 2024-09-21T04:10:58.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.324+0000 7fc51ac77740 1 -- 172.21.15.64:0/2555136844 wait complete. 2024-09-21T04:10:58.355 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.324+0000 7fc51249f640 1 --2- 172.21.15.64:0/3980799014 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc4f80b6680 0x7fc4f80b8b90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc508031440 tx=0x7fc508009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:58.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.324+0000 7fc51ac77740 1 -- start start 2024-09-21T04:10:58.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.324+0000 7fc51ac77740 1 Processor -- start 2024-09-21T04:10:58.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.325+0000 7fc51ac77740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f27ad0490 0x564f27eac020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:58.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.325+0000 7fc51ac77740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f27ad8670 0x564f27eac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:58.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.325+0000 7fc51ac77740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f27af9350 0x564f27eb7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:58.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.325+0000 7fc51ac77740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564f27c67890 con 0x564f27af9350 2024-09-21T04:10:58.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.325+0000 7fc51ac77740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564f27da0d40 con 0x564f27ad0490 2024-09-21T04:10:58.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.325+0000 7fc51ac77740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564f27eaec10 con 0x564f27ad8670 2024-09-21T04:10:58.356 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.325+0000 7fc51ac77740 10 client.0 _ll_register_callbacks cb 0x564f27e04c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:10:58.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.325+0000 7fc512ca0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f27af9350 0x564f27eb7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:58.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.325+0000 7fc51249f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f27ad0490 0x564f27eac020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:58.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.325+0000 7fc511c9e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f27ad8670 0x564f27eac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:58.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.325+0000 7fc512ca0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f27af9350 0x564f27eb7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36114/0 (socket says 172.21.15.64:36114) 2024-09-21T04:10:58.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.325+0000 7fc51249f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f27ad0490 0x564f27eac020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:39258/0 (socket says 172.21.15.64:39258) 2024-09-21T04:10:58.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.325+0000 7fc512ca0640 1 -- 172.21.15.64:0/2139332631 learned_addr learned my addr 172.21.15.64:0/2139332631 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:58.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.325+0000 7fc512ca0640 1 -- 172.21.15.64:0/2139332631 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f27ad8670 msgr2=0x564f27eac5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:58.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.325+0000 7fc512ca0640 1 --2- 172.21.15.64:0/2139332631 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f27ad8670 0x564f27eac5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.325+0000 7fc512ca0640 1 -- 172.21.15.64:0/2139332631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f27ad0490 msgr2=0x564f27eac020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:58.357 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.325+0000 7fc512ca0640 1 --2- 172.21.15.64:0/2139332631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f27ad0490 0x564f27eac020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.325+0000 7fc512ca0640 1 -- 172.21.15.64:0/2139332631 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f27eb8510 con 0x564f27af9350 2024-09-21T04:10:58.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.325+0000 7fc51249f640 1 --2- 172.21.15.64:0/2139332631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f27ad0490 0x564f27eac020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:10:58.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.325+0000 7fc512ca0640 1 --2- 172.21.15.64:0/2139332631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f27af9350 0x564f27eb7f00 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fc504012df0 tx=0x7fc50400fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398feaf748d75063 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:58.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.326+0000 7fc4f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:10:58.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.326+0000 7fc4f5ffb640 1 -- 172.21.15.64:0/2139332631 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc504012860 con 0x564f27af9350 2024-09-21T04:10:58.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.326+0000 7fc51ac77740 1 -- 172.21.15.64:0/2139332631 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc50400fce0 con 0x564f27af9350 2024-09-21T04:10:58.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.326+0000 7fc4f5ffb640 1 -- 172.21.15.64:0/2139332631 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7fc504012a00 con 0x564f27af9350 2024-09-21T04:10:58.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.326+0000 7fc4f5ffb640 1 -- 172.21.15.64:0/2139332631 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc504033050 con 0x564f27af9350 2024-09-21T04:10:58.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.326+0000 7fc51ac77740 10 client.16953 fetch_fsmap learned FSMap version 38 2024-09-21T04:10:58.358 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.326+0000 7fc51ac77740 1 -- 172.21.15.64:0/2139332631 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x564f27ebae40 con 0x564f27af9350 2024-09-21T04:10:58.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.326+0000 7fc4f5ffb640 1 -- 172.21.15.64:0/2139332631 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc50400fce0 con 0x564f27af9350 2024-09-21T04:10:58.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.326+0000 7fc4f5ffb640 1 -- 172.21.15.64:0/2139332631 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7fc50400c070 con 0x564f27af9350 2024-09-21T04:10:58.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.326+0000 7fc51ac77740 10 client.16953 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T04:10:58.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.326+0000 7fc51ac77740 10 client.16953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:10:58.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.326+0000 7fc51ac77740 20 client.16953 populate_metadata read hostname 'smithi064' 2024-09-21T04:10:58.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.326+0000 7fc51ac77740 1 --2- 172.21.15.64:0/2139332631 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564f27ab7cd0 0x564f27ebe200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:58.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.326+0000 7fc51ac77740 4 client.16953 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:10:58.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.326+0000 7fc51ac77740 1 -- 172.21.15.64:0/2139332631 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f27e18db0 con 0x564f27ab7cd0 2024-09-21T04:10:58.359 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.327+0000 7fc51249f640 1 --2- 172.21.15.64:0/2139332631 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564f27ab7cd0 0x564f27ebe200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:58.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.327+0000 7fc51249f640 1 --2- 172.21.15.64:0/2139332631 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564f27ab7cd0 0x564f27ebe200 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8826eeb5c76a382a server_cookie=d548543eaa07c0d4 in_seq=0 out_seq=0 2024-09-21T04:10:58.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.327+0000 7fc4f5ffb640 10 client.16953 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:58.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.328+0000 7fc4f5ffb640 1 -- 172.21.15.64:0/2139332631 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564f27e18db0 con 0x564f27ab7cd0 2024-09-21T04:10:58.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.328+0000 7fc4f5ffb640 10 client.16953 handle_command_reply: tid=0 2024-09-21T04:10:58.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.328+0000 7fc51ac77740 1 client.16953 shutdown 2024-09-21T04:10:58.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.328+0000 7fc51ac77740 1 -- 172.21.15.64:0/2139332631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f27af9350 msgr2=0x564f27eb7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:58.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.328+0000 7fc51ac77740 1 --2- 172.21.15.64:0/2139332631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f27af9350 0x564f27eb7f00 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fc504012df0 tx=0x7fc50400fb70 comp rx=0 tx=0).stop 2024-09-21T04:10:58.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.328+0000 7fc51ac77740 1 -- 172.21.15.64:0/2139332631 shutdown_connections 2024-09-21T04:10:58.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.328+0000 7fc51ac77740 1 --2- 172.21.15.64:0/2139332631 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564f27ab7cd0 0x564f27ebe200 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.328+0000 7fc51ac77740 1 --2- 172.21.15.64:0/2139332631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f27af9350 0x564f27eb7f00 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.328+0000 7fc51ac77740 1 --2- 172.21.15.64:0/2139332631 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f27ad8670 0x564f27eac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.360 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.328+0000 7fc4f5ffb640 0 client.16953 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:58.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.328+0000 7fc51ac77740 1 --2- 172.21.15.64:0/2139332631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f27ad0490 0x564f27eac020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.328+0000 7fc51ac77740 1 -- 172.21.15.64:0/2139332631 >> 172.21.15.64:0/2139332631 conn(0x564f27aed740 msgr2=0x564f27ea3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:58.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.328+0000 7fc51ac77740 1 -- 172.21.15.64:0/2139332631 shutdown_connections 2024-09-21T04:10:58.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.328+0000 7fc51ac77740 1 -- 172.21.15.64:0/2139332631 wait complete. 2024-09-21T04:10:58.361 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.328+0000 7fc51ac77740 20 client.16953 trim_cache size 0 max 16384 2024-09-21T04:10:58.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.347+0000 7fc51ac77740 1 Processor -- start 2024-09-21T04:10:58.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.347+0000 7fc51ac77740 1 -- start start 2024-09-21T04:10:58.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.347+0000 7fc51ac77740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f27ad0490 0x564f27eb7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:58.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.348+0000 7fc51ac77740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f27ad8670 0x564f27ea56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:58.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.348+0000 7fc51ac77740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f27af9350 0x564f27ea5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:58.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.348+0000 7fc51ac77740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc50c1b2200 con 0x564f27ad0490 2024-09-21T04:10:58.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.348+0000 7fc51ac77740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc50c1afa70 con 0x564f27ad8670 2024-09-21T04:10:58.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.348+0000 7fc51ac77740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc50c1ad8e0 con 0x564f27af9350 2024-09-21T04:10:58.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.348+0000 7fc51249f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f27ad0490 0x564f27eb7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:58.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.348+0000 7fc51249f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f27ad0490 0x564f27eb7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36122/0 (socket says 172.21.15.64:36122) 2024-09-21T04:10:58.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.348+0000 7fc51249f640 1 -- 172.21.15.64:0/3569734506 learned_addr learned my addr 172.21.15.64:0/3569734506 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:58.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.348+0000 7fc512ca0640 1 --2- 172.21.15.64:0/3569734506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f27af9350 0x564f27ea5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:58.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.348+0000 7fc511c9e640 1 --2- 172.21.15.64:0/3569734506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f27ad8670 0x564f27ea56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:58.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.348+0000 7fc51249f640 1 -- 172.21.15.64:0/3569734506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f27af9350 msgr2=0x564f27ea5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:58.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.348+0000 7fc51249f640 1 --2- 172.21.15.64:0/3569734506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f27af9350 0x564f27ea5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.348+0000 7fc51249f640 1 -- 172.21.15.64:0/3569734506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f27ad8670 msgr2=0x564f27ea56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:58.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.348+0000 7fc51249f640 1 --2- 172.21.15.64:0/3569734506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f27ad8670 0x564f27ea56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.348+0000 7fc51249f640 1 -- 172.21.15.64:0/3569734506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f27ead0e0 con 0x564f27ad0490 2024-09-21T04:10:58.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.348+0000 7fc511c9e640 1 --2- 172.21.15.64:0/3569734506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f27ad8670 0x564f27ea56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:10:58.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.348+0000 7fc51249f640 1 --2- 172.21.15.64:0/3569734506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f27ad0490 0x564f27eb7e70 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc508009900 tx=0x7fc50803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d9d9479f2a2859 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:58.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.348+0000 7fc4f7fff640 1 -- 172.21.15.64:0/3569734506 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc508035040 con 0x564f27ad0490 2024-09-21T04:10:58.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.348+0000 7fc4f7fff640 1 -- 172.21.15.64:0/3569734506 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc508056030 con 0x564f27ad0490 2024-09-21T04:10:58.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.348+0000 7fc4f7fff640 1 -- 172.21.15.64:0/3569734506 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc508004d80 con 0x564f27ad0490 2024-09-21T04:10:58.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.349+0000 7fc51ac77740 1 -- 172.21.15.64:0/3569734506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f27ad0490 msgr2=0x564f27eb7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:58.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.349+0000 7fc51ac77740 1 --2- 172.21.15.64:0/3569734506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f27ad0490 0x564f27eb7e70 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc508009900 tx=0x7fc50803f000 comp rx=0 tx=0).stop 2024-09-21T04:10:58.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.349+0000 7fc51ac77740 1 -- 172.21.15.64:0/3569734506 shutdown_connections 2024-09-21T04:10:58.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.349+0000 7fc51ac77740 1 --2- 172.21.15.64:0/3569734506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f27af9350 0x564f27ea5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.349+0000 7fc51ac77740 1 --2- 172.21.15.64:0/3569734506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f27ad8670 0x564f27ea56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.349+0000 7fc51ac77740 1 --2- 172.21.15.64:0/3569734506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f27ad0490 0x564f27eb7e70 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.349+0000 7fc51ac77740 1 -- 172.21.15.64:0/3569734506 >> 172.21.15.64:0/3569734506 conn(0x564f27aed740 msgr2=0x564f27eaa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:58.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.349+0000 7fc51ac77740 1 -- 172.21.15.64:0/3569734506 shutdown_connections 2024-09-21T04:10:58.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.349+0000 7fc51ac77740 1 -- 172.21.15.64:0/3569734506 wait complete. 2024-09-21T04:10:58.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.350+0000 7fc51ac77740 1 -- start start 2024-09-21T04:10:58.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.350+0000 7fc51ac77740 1 Processor -- start 2024-09-21T04:10:58.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.350+0000 7fc51ac77740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f27ad0490 0x564f27ebd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:58.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.350+0000 7fc51ac77740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f27ad8670 0x564f27ebdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:58.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.350+0000 7fc51ac77740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f27af9350 0x564f27f2bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:58.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.350+0000 7fc51ac77740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc50c1adf30 con 0x564f27af9350 2024-09-21T04:10:58.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.350+0000 7fc51ac77740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc50c1af3a0 con 0x564f27ad0490 2024-09-21T04:10:58.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.350+0000 7fc51ac77740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc50c1ae600 con 0x564f27ad8670 2024-09-21T04:10:58.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.350+0000 7fc51ac77740 10 client.0 _ll_register_callbacks cb 0x7fc508009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:10:58.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.350+0000 7fc512ca0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f27af9350 0x564f27f2bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:58.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.350+0000 7fc51249f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f27ad0490 0x564f27ebd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:58.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.350+0000 7fc51249f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f27ad0490 0x564f27ebd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:39276/0 (socket says 172.21.15.64:39276) 2024-09-21T04:10:58.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.350+0000 7fc512ca0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f27af9350 0x564f27f2bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36134/0 (socket says 172.21.15.64:36134) 2024-09-21T04:10:58.369 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.350+0000 7fc51249f640 1 -- 172.21.15.64:0/1099544261 learned_addr learned my addr 172.21.15.64:0/1099544261 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:10:58.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.350+0000 7fc511c9e640 1 --2- 172.21.15.64:0/1099544261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f27ad8670 0x564f27ebdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:58.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.351+0000 7fc512ca0640 1 -- 172.21.15.64:0/1099544261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f27ad8670 msgr2=0x564f27ebdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:58.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.351+0000 7fc512ca0640 1 --2- 172.21.15.64:0/1099544261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f27ad8670 0x564f27ebdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.351+0000 7fc512ca0640 1 -- 172.21.15.64:0/1099544261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f27ad0490 msgr2=0x564f27ebd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:10:58.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.351+0000 7fc512ca0640 1 --2- 172.21.15.64:0/1099544261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f27ad0490 0x564f27ebd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.351+0000 7fc512ca0640 1 -- 172.21.15.64:0/1099544261 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f27f2c2c0 con 0x564f27af9350 2024-09-21T04:10:58.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.351+0000 7fc512ca0640 1 --2- 172.21.15.64:0/1099544261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f27af9350 0x564f27f2bd80 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc504012010 tx=0x7fc50400fe10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e550d74aa9690c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:10:58.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.351+0000 7fc4f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:10:58.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.351+0000 7fc4f5ffb640 1 -- 172.21.15.64:0/1099544261 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc504005bd0 con 0x564f27af9350 2024-09-21T04:10:58.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.351+0000 7fc4f5ffb640 1 -- 172.21.15.64:0/1099544261 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7fc50401ba30 con 0x564f27af9350 2024-09-21T04:10:58.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.351+0000 7fc51ac77740 1 -- 172.21.15.64:0/1099544261 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc508056030 con 0x564f27af9350 2024-09-21T04:10:58.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.351+0000 7fc4f5ffb640 1 -- 172.21.15.64:0/1099544261 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc504047050 con 0x564f27af9350 2024-09-21T04:10:58.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.351+0000 7fc51ac77740 10 client.16959 fetch_fsmap learned FSMap version 38 2024-09-21T04:10:58.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.351+0000 7fc51ac77740 1 -- 172.21.15.64:0/1099544261 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x564f27f2d3c0 con 0x564f27af9350 2024-09-21T04:10:58.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.351+0000 7fc4f5ffb640 1 -- 172.21.15.64:0/1099544261 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc508056030 con 0x564f27af9350 2024-09-21T04:10:58.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.352+0000 7fc4f5ffb640 1 -- 172.21.15.64:0/1099544261 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7fc50400c040 con 0x564f27af9350 2024-09-21T04:10:58.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.352+0000 7fc51ac77740 10 client.16959 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T04:10:58.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.352+0000 7fc51ac77740 10 client.16959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:10:58.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.352+0000 7fc51ac77740 20 client.16959 populate_metadata read hostname 'smithi064' 2024-09-21T04:10:58.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.352+0000 7fc51ac77740 1 --2- 172.21.15.64:0/1099544261 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564f27a8d080 0x564f27f2fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:10:58.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.352+0000 7fc51ac77740 4 client.16959 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T04:10:58.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.352+0000 7fc51ac77740 1 -- 172.21.15.64:0/1099544261 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564f27e14860 con 0x564f27a8d080 2024-09-21T04:10:58.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.352+0000 7fc51249f640 1 --2- 172.21.15.64:0/1099544261 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564f27a8d080 0x564f27f2fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:10:58.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.352+0000 7fc4f5ffb640 10 client.16959 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:58.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.352+0000 7fc51249f640 1 --2- 172.21.15.64:0/1099544261 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564f27a8d080 0x564f27f2fb70 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f34d865b0043827 server_cookie=d05593d003639050 in_seq=0 out_seq=0 2024-09-21T04:10:58.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.354+0000 7fc4f5ffb640 1 -- 172.21.15.64:0/1099544261 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564f27e14860 con 0x564f27a8d080 2024-09-21T04:10:58.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.354+0000 7fc4f5ffb640 10 client.16959 handle_command_reply: tid=0 2024-09-21T04:10:58.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.354+0000 7fc51ac77740 1 client.16959 shutdown 2024-09-21T04:10:58.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.354+0000 7fc51ac77740 1 -- 172.21.15.64:0/1099544261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f27af9350 msgr2=0x564f27f2bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:58.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.354+0000 7fc51ac77740 1 --2- 172.21.15.64:0/1099544261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f27af9350 0x564f27f2bd80 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc504012010 tx=0x7fc50400fe10 comp rx=0 tx=0).stop 2024-09-21T04:10:58.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.354+0000 7fc51ac77740 1 -- 172.21.15.64:0/1099544261 shutdown_connections 2024-09-21T04:10:58.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.354+0000 7fc51ac77740 1 --2- 172.21.15.64:0/1099544261 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x564f27a8d080 0x564f27f2fb70 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.354+0000 7fc51ac77740 1 --2- 172.21.15.64:0/1099544261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564f27af9350 0x564f27f2bd80 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.354+0000 7fc51ac77740 1 --2- 172.21.15.64:0/1099544261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564f27ad8670 0x564f27ebdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.354+0000 7fc51ac77740 1 --2- 172.21.15.64:0/1099544261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564f27ad0490 0x564f27ebd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.354+0000 7fc51ac77740 1 -- 172.21.15.64:0/1099544261 >> 172.21.15.64:0/1099544261 conn(0x564f27aed740 msgr2=0x564f27e9dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:58.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.354+0000 7fc4f5ffb640 0 client.16959 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:10:58.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.355+0000 7fc51ac77740 1 -- 172.21.15.64:0/1099544261 shutdown_connections 2024-09-21T04:10:58.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.355+0000 7fc51ac77740 1 -- 172.21.15.64:0/1099544261 wait complete. 2024-09-21T04:10:58.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.355+0000 7fc51ac77740 20 client.16959 trim_cache size 0 max 16384 2024-09-21T04:10:58.373 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:10:58.373 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-21T04:10:58.373 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "1b23713d-ea3a-4a86-a87f-65f6ab82671d", 2024-09-21T04:10:58.373 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-21T04:10:58.373 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:10:58.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.355+0000 7fc514703640 1 -- 172.21.15.64:0/3980799014 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc4f80b6680 msgr2=0x7fc4f80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:58.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.355+0000 7fc514703640 1 --2- 172.21.15.64:0/3980799014 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc4f80b6680 0x7fc4f80b8b90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc508031440 tx=0x7fc508009560 comp rx=0 tx=0).stop 2024-09-21T04:10:58.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.355+0000 7fc514703640 1 -- 172.21.15.64:0/3980799014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc50c1062b0 msgr2=0x7fc50c19aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:10:58.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.355+0000 7fc514703640 1 --2- 172.21.15.64:0/3980799014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc50c1062b0 0x7fc50c19aa70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc50c075ab0 tx=0x7fc4fc00ee50 comp rx=0 tx=0).stop 2024-09-21T04:10:58.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.355+0000 7fc514703640 1 -- 172.21.15.64:0/3980799014 shutdown_connections 2024-09-21T04:10:58.374 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.355+0000 7fc514703640 1 --2- 172.21.15.64:0/3980799014 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc4f80b6680 0x7fc4f80b8b90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.355+0000 7fc514703640 1 --2- 172.21.15.64:0/3980799014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc50c106c00 0x7fc50c19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.355+0000 7fc514703640 1 --2- 172.21.15.64:0/3980799014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc50c1062b0 0x7fc50c19aa70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.355+0000 7fc514703640 1 --2- 172.21.15.64:0/3980799014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc50c101fd0 0x7fc50c19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:10:58.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.355+0000 7fc514703640 1 -- 172.21.15.64:0/3980799014 >> 172.21.15.64:0/3980799014 conn(0x7fc50c079e10 msgr2=0x7fc50c108470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:10:58.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.355+0000 7fc514703640 1 -- 172.21.15.64:0/3980799014 shutdown_connections 2024-09-21T04:10:58.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:58.355+0000 7fc514703640 1 -- 172.21.15.64:0/3980799014 wait complete. 2024-09-21T04:10:58.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:58 smithi119 ceph-mon[34811]: pgmap v531: 97 pgs: 97 active+clean; 814 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 498 op/s 2024-09-21T04:10:58.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:58 smithi064 ceph-mon[31180]: pgmap v531: 97 pgs: 97 active+clean; 814 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 498 op/s 2024-09-21T04:10:58.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:58 smithi131 ceph-mon[35193]: pgmap v531: 97 pgs: 97 active+clean; 814 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 498 op/s 2024-09-21T04:10:59.049 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T04:10:59.347 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:10:59.489 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:59 smithi064 ceph-mon[31180]: scrub queued for path: / 2024-09-21T04:10:59.489 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:59 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:10:59.490 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:59 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:10:59.490 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:59 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:10:59.490 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:59 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:10:59.490 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:10:59 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:10:59.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:59 smithi119 ceph-mon[34811]: scrub queued for path: / 2024-09-21T04:10:59.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:59 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:10:59.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:59 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:10:59.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:59 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:10:59.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:59 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:10:59.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:10:59 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:10:59.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:59 smithi131 ceph-mon[35193]: scrub queued for path: / 2024-09-21T04:10:59.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:59 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:10:59.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:59 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:10:59.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:59 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:10:59.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:59 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:10:59.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:10:59 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:11:00.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.859+0000 7ff511ad2640 1 -- 172.21.15.64:0/2442970947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff50c1094a0 msgr2=0x7ff50c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:00.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.859+0000 7ff511ad2640 1 --2- 172.21.15.64:0/2442970947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff50c1094a0 0x7ff50c1098b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff500009900 tx=0x7ff50002a410 comp rx=0 tx=0).stop 2024-09-21T04:11:00.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.859+0000 7ff511ad2640 1 -- 172.21.15.64:0/2442970947 shutdown_connections 2024-09-21T04:11:00.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.859+0000 7ff511ad2640 1 --2- 172.21.15.64:0/2442970947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff50c109df0 0x7ff50c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.859+0000 7ff511ad2640 1 --2- 172.21.15.64:0/2442970947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff50c1094a0 0x7ff50c1098b0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.859+0000 7ff511ad2640 1 --2- 172.21.15.64:0/2442970947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff50c103b60 0x7ff50c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.859+0000 7ff511ad2640 1 -- 172.21.15.64:0/2442970947 >> 172.21.15.64:0/2442970947 conn(0x7ff50c0ff5d0 msgr2=0x7ff50c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:00.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.859+0000 7ff511ad2640 1 -- 172.21.15.64:0/2442970947 shutdown_connections 2024-09-21T04:11:00.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.859+0000 7ff511ad2640 1 -- 172.21.15.64:0/2442970947 wait complete. 2024-09-21T04:11:00.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.860+0000 7ff511ad2640 1 Processor -- start 2024-09-21T04:11:00.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.860+0000 7ff511ad2640 1 -- start start 2024-09-21T04:11:00.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.861+0000 7ff511ad2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff50c103b60 0x7ff50c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:00.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.861+0000 7ff511ad2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff50c1094a0 0x7ff50c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:00.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.861+0000 7ff511ad2640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff50c109df0 0x7ff50c19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:00.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.861+0000 7ff511ad2640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c113ab0 con 0x7ff50c109df0 2024-09-21T04:11:00.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.861+0000 7ff511ad2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c113930 con 0x7ff50c103b60 2024-09-21T04:11:00.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.861+0000 7ff511ad2640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c113c30 con 0x7ff50c1094a0 2024-09-21T04:11:00.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.861+0000 7ff50b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff50c103b60 0x7ff50c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:00.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.861+0000 7ff50bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff50c109df0 0x7ff50c19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:00.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.861+0000 7ff50b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff50c103b60 0x7ff50c19a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:39292/0 (socket says 172.21.15.64:39292) 2024-09-21T04:11:00.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.861+0000 7ff50b7fe640 1 -- 172.21.15.64:0/545942918 learned_addr learned my addr 172.21.15.64:0/545942918 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:00.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.861+0000 7ff50affd640 1 --2- 172.21.15.64:0/545942918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff50c1094a0 0x7ff50c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:00.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.861+0000 7ff50b7fe640 1 -- 172.21.15.64:0/545942918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff50c1094a0 msgr2=0x7ff50c19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:00.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.861+0000 7ff50b7fe640 1 --2- 172.21.15.64:0/545942918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff50c1094a0 0x7ff50c19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.861+0000 7ff50b7fe640 1 -- 172.21.15.64:0/545942918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff50c109df0 msgr2=0x7ff50c19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:00.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.861+0000 7ff50b7fe640 1 --2- 172.21.15.64:0/545942918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff50c109df0 0x7ff50c19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.861+0000 7ff50b7fe640 1 -- 172.21.15.64:0/545942918 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50c1a0410 con 0x7ff50c103b60 2024-09-21T04:11:00.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.861+0000 7ff50affd640 1 --2- 172.21.15.64:0/545942918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff50c1094a0 0x7ff50c19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:11:00.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.861+0000 7ff50bfff640 1 --2- 172.21.15.64:0/545942918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff50c109df0 0x7ff50c19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:11:00.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.861+0000 7ff50b7fe640 1 --2- 172.21.15.64:0/545942918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff50c103b60 0x7ff50c19a4a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00ea60 tx=0x7ff4fc00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:00.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.862+0000 7ff508ff9640 1 -- 172.21.15.64:0/545942918 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc00cde0 con 0x7ff50c103b60 2024-09-21T04:11:00.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.862+0000 7ff508ff9640 1 -- 172.21.15.64:0/545942918 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4fc0035f0 con 0x7ff50c103b60 2024-09-21T04:11:00.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.862+0000 7ff511ad2640 1 -- 172.21.15.64:0/545942918 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff50c1a0700 con 0x7ff50c103b60 2024-09-21T04:11:00.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.862+0000 7ff508ff9640 1 -- 172.21.15.64:0/545942918 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc010690 con 0x7ff50c103b60 2024-09-21T04:11:00.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.862+0000 7ff511ad2640 1 -- 172.21.15.64:0/545942918 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff50c1a6e60 con 0x7ff50c103b60 2024-09-21T04:11:00.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.864+0000 7ff518046740 1 Processor -- start 2024-09-21T04:11:00.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.864+0000 7ff518046740 1 -- start start 2024-09-21T04:11:00.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.864+0000 7ff508ff9640 1 -- 172.21.15.64:0/545942918 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7ff4fc010830 con 0x7ff50c103b60 2024-09-21T04:11:00.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.865+0000 7ff518046740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f192501070 0x55f1928a8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:00.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.865+0000 7ff518046740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1924d81b0 0x55f1928a8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:00.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.865+0000 7ff518046740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f1924e0390 0x55f1928a90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:00.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.865+0000 7ff518046740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f19281ee50 con 0x55f1924d81b0 2024-09-21T04:11:00.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.865+0000 7ff518046740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f19280e280 con 0x55f1924e0390 2024-09-21T04:11:00.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.865+0000 7ff518046740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f192812090 con 0x55f192501070 2024-09-21T04:11:00.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.865+0000 7ff50b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1924d81b0 0x55f1928a8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:00.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.865+0000 7ff50bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f192501070 0x55f1928a8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:00.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.865+0000 7ff50b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1924d81b0 0x55f1928a8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36154/0 (socket says 172.21.15.64:36154) 2024-09-21T04:11:00.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.865+0000 7ff50bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f192501070 0x55f1928a8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:47104/0 (socket says 172.21.15.64:47104) 2024-09-21T04:11:00.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.865+0000 7ff50affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f1924e0390 0x55f1928a90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:00.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.865+0000 7ff50bfff640 1 -- 172.21.15.64:0/822148838 learned_addr learned my addr 172.21.15.64:0/822148838 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:00.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.865+0000 7ff50bfff640 1 -- 172.21.15.64:0/822148838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f1924e0390 msgr2=0x55f1928a90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:00.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.865+0000 7ff50bfff640 1 --2- 172.21.15.64:0/822148838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f1924e0390 0x55f1928a90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.865+0000 7ff50bfff640 1 -- 172.21.15.64:0/822148838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1924d81b0 msgr2=0x55f1928a8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:00.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.865+0000 7ff50bfff640 1 --2- 172.21.15.64:0/822148838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1924d81b0 0x55f1928a8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.865+0000 7ff50bfff640 1 -- 172.21.15.64:0/822148838 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f19256c650 con 0x55f192501070 2024-09-21T04:11:00.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.865+0000 7ff50affd640 1 --2- 172.21.15.64:0/822148838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f1924e0390 0x55f1928a90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:11:00.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.865+0000 7ff50bfff640 1 --2- 172.21.15.64:0/822148838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f192501070 0x55f1928a8650 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff5040117e0 tx=0x7ff504011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8809c930d4ca509a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:00.092 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:11:00.092 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T04:11:00.092 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-09-21T04:11:00.092 INFO:teuthology.orchestra.run.smithi064.stdout: "1b23713d-ea3a-4a86-a87f-65f6ab82671d": { 2024-09-21T04:11:00.092 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:00.092 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "1b23713d-ea3a-4a86-a87f-65f6ab82671d", 2024-09-21T04:11:00.092 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-09-21T04:11:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:11:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_stats": { 2024-09-21T04:11:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: "1b23713d-ea3a-4a86-a87f-65f6ab82671d": { 2024-09-21T04:11:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:10:58.352639+0000", 2024-09-21T04:11:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: "2c0523fb-aae4-43fa-9568-136a9b640051": { 2024-09-21T04:11:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:09:39.808606+0000", 2024-09-21T04:11:00.093 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: "bed40d88-2102-43a6-ac2d-b89d5a2264d0": { 2024-09-21T04:11:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:11.363340+0000", 2024-09-21T04:11:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:00.094 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:00.095 INFO:teuthology.orchestra.run.smithi064.stdout: "e9242f88-6599-40eb-adf1-bdb3e946a1e0": { 2024-09-21T04:11:00.095 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:05:39.744646+0000", 2024-09-21T04:11:00.095 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:00.095 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:00.095 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:00.095 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:00.095 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:00.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:00.095 INFO:teuthology.orchestra.run.smithi064.stdout: "6bde0fe5-f775-44b1-ad0c-f9e596574dc0": { 2024-09-21T04:11:00.095 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:06:23.835536+0000", 2024-09-21T04:11:00.095 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:00.095 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:00.095 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: "b52b950c-0f97-49ec-a37c-1a8e04348ce8": { 2024-09-21T04:11:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:54.003800+0000", 2024-09-21T04:11:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: "e2c2eed8-9292-4bf6-8a39-13ad27da5acb": { 2024-09-21T04:11:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:08:47.067932+0000", 2024-09-21T04:11:00.096 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:00.097 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:00.097 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:00.097 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:00.097 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:00.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:00.097 INFO:teuthology.orchestra.run.smithi064.stdout: "75db1bc5-612e-45aa-bad2-cc90feeaa840": { 2024-09-21T04:11:00.097 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:59.123138+0000", 2024-09-21T04:11:00.097 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:00.097 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:00.097 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:00.097 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:00.097 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:00.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: "0ab717a7-36c6-4dca-94e7-af0bdf2aa112": { 2024-09-21T04:11:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:42.691086+0000", 2024-09-21T04:11:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:11:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:03:59.570174+0000", 2024-09-21T04:11:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:00.098 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:00.099 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:00.099 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:11:00.099 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:11:00.099 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:11:00.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.866+0000 7ff508ff9640 1 --2- 172.21.15.64:0/545942918 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff4f40b6680 0x7ff4f40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:00.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.866+0000 7ff4f1ffb640 1 -- 172.21.15.64:0/822148838 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff504011310 con 0x55f192501070 2024-09-21T04:11:00.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.866+0000 7ff4f1ffb640 1 -- 172.21.15.64:0/822148838 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff504003e20 con 0x55f192501070 2024-09-21T04:11:00.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.866+0000 7ff4f1ffb640 1 -- 172.21.15.64:0/822148838 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5040044c0 con 0x55f192501070 2024-09-21T04:11:00.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.866+0000 7ff50b7fe640 1 --2- 172.21.15.64:0/545942918 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff4f40b6680 0x7ff4f40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:00.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.866+0000 7ff508ff9640 1 -- 172.21.15.64:0/545942918 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7ff4fc0e7a30 con 0x7ff50c103b60 2024-09-21T04:11:00.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.867+0000 7ff518046740 1 -- 172.21.15.64:0/822148838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f192501070 msgr2=0x55f1928a8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:00.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.867+0000 7ff50b7fe640 1 --2- 172.21.15.64:0/545942918 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff4f40b6680 0x7ff4f40b8b90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0e51d0 tx=0x7ff4fc0eb580 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:00.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.867+0000 7ff518046740 1 --2- 172.21.15.64:0/822148838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f192501070 0x55f1928a8650 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff5040117e0 tx=0x7ff504011ca0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.867+0000 7ff518046740 1 -- 172.21.15.64:0/822148838 shutdown_connections 2024-09-21T04:11:00.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.867+0000 7ff518046740 1 --2- 172.21.15.64:0/822148838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f1924e0390 0x55f1928a90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.867+0000 7ff518046740 1 --2- 172.21.15.64:0/822148838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1924d81b0 0x55f1928a8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.867+0000 7ff518046740 1 --2- 172.21.15.64:0/822148838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f192501070 0x55f1928a8650 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.867+0000 7ff518046740 1 -- 172.21.15.64:0/822148838 >> 172.21.15.64:0/822148838 conn(0x55f1924f5460 msgr2=0x55f1928a6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:00.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.867+0000 7ff518046740 1 -- 172.21.15.64:0/822148838 shutdown_connections 2024-09-21T04:11:00.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.867+0000 7ff518046740 1 -- 172.21.15.64:0/822148838 wait complete. 2024-09-21T04:11:00.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.867+0000 7ff518046740 1 -- start start 2024-09-21T04:11:00.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.867+0000 7ff518046740 1 Processor -- start 2024-09-21T04:11:00.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.868+0000 7ff518046740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1924d81b0 0x55f1928b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:00.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.868+0000 7ff518046740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1924e0390 0x55f1928b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:00.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.868+0000 7ff518046740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f192501070 0x55f1928c0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:00.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.868+0000 7ff518046740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f19266f700 con 0x55f1924e0390 2024-09-21T04:11:00.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.868+0000 7ff518046740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f1927a9110 con 0x55f192501070 2024-09-21T04:11:00.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.868+0000 7ff518046740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f1928b6e30 con 0x55f1924d81b0 2024-09-21T04:11:00.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.868+0000 7ff518046740 10 client.0 _ll_register_callbacks cb 0x55f1927c2990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:11:00.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.868+0000 7ff50affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1924e0390 0x55f1928b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:00.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.868+0000 7ff50affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1924e0390 0x55f1928b42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36166/0 (socket says 172.21.15.64:36166) 2024-09-21T04:11:00.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.868+0000 7ff50affd640 1 -- 172.21.15.64:0/4222557648 learned_addr learned my addr 172.21.15.64:0/4222557648 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:00.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.868+0000 7ff50bfff640 1 --2- 172.21.15.64:0/4222557648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1924d81b0 0x55f1928b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:00.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.868+0000 7ff50bfff640 1 --2- 172.21.15.64:0/4222557648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f192501070 0x55f1928c0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:00.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.868+0000 7ff50affd640 1 -- 172.21.15.64:0/4222557648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1924d81b0 msgr2=0x55f1928b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:00.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.868+0000 7ff50affd640 1 --2- 172.21.15.64:0/4222557648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1924d81b0 0x55f1928b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.868+0000 7ff50affd640 1 -- 172.21.15.64:0/4222557648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f192501070 msgr2=0x55f1928c0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:00.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.868+0000 7ff50affd640 1 --2- 172.21.15.64:0/4222557648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f192501070 0x55f1928c0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.868+0000 7ff50affd640 1 -- 172.21.15.64:0/4222557648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1927d6c40 con 0x55f1924e0390 2024-09-21T04:11:00.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.868+0000 7ff50affd640 1 --2- 172.21.15.64:0/4222557648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1924e0390 0x55f1928b42d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff500009900 tx=0x7ff5000316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248b5ee761da4c8e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:00.102 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.868+0000 7ff4eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:11:00.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.869+0000 7ff4eaffd640 1 -- 172.21.15.64:0/4222557648 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff500035040 con 0x55f1924e0390 2024-09-21T04:11:00.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.869+0000 7ff4eaffd640 1 -- 172.21.15.64:0/4222557648 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7ff500004040 con 0x55f1924e0390 2024-09-21T04:11:00.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.869+0000 7ff518046740 1 -- 172.21.15.64:0/4222557648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff504003e20 con 0x55f1924e0390 2024-09-21T04:11:00.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.869+0000 7ff4eaffd640 1 -- 172.21.15.64:0/4222557648 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff500004240 con 0x55f1924e0390 2024-09-21T04:11:00.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.869+0000 7ff518046740 10 client.16971 fetch_fsmap learned FSMap version 38 2024-09-21T04:11:00.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.869+0000 7ff518046740 1 -- 172.21.15.64:0/4222557648 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f1928c2bc0 con 0x55f1924e0390 2024-09-21T04:11:00.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.869+0000 7ff4eaffd640 1 -- 172.21.15.64:0/4222557648 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff504003e20 con 0x55f1924e0390 2024-09-21T04:11:00.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.869+0000 7ff4eaffd640 1 -- 172.21.15.64:0/4222557648 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7ff5000049e0 con 0x55f1924e0390 2024-09-21T04:11:00.103 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.869+0000 7ff518046740 10 client.16971 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T04:11:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.869+0000 7ff518046740 10 client.16971 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:11:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.869+0000 7ff518046740 20 client.16971 populate_metadata read hostname 'smithi064' 2024-09-21T04:11:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.869+0000 7ff518046740 1 --2- 172.21.15.64:0/4222557648 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f1924bf9f0 0x55f1928c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.869+0000 7ff518046740 4 client.16971 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:11:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.869+0000 7ff518046740 1 -- 172.21.15.64:0/4222557648 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f192820da0 con 0x55f1924bf9f0 2024-09-21T04:11:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.870+0000 7ff50bfff640 1 --2- 172.21.15.64:0/4222557648 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f1924bf9f0 0x55f1928c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.870+0000 7ff50bfff640 1 --2- 172.21.15.64:0/4222557648 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f1924bf9f0 0x55f1928c6160 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7453c94e8068b40 server_cookie=28b8e1b260f2911 in_seq=0 out_seq=0 2024-09-21T04:11:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.870+0000 7ff4eaffd640 10 client.16971 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.871+0000 7ff4eaffd640 1 -- 172.21.15.64:0/4222557648 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f192820da0 con 0x55f1924bf9f0 2024-09-21T04:11:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.871+0000 7ff4eaffd640 10 client.16971 handle_command_reply: tid=0 2024-09-21T04:11:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.871+0000 7ff518046740 1 client.16971 shutdown 2024-09-21T04:11:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.871+0000 7ff518046740 1 -- 172.21.15.64:0/4222557648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1924e0390 msgr2=0x55f1928b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:00.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.871+0000 7ff518046740 1 --2- 172.21.15.64:0/4222557648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1924e0390 0x55f1928b42d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff500009900 tx=0x7ff5000316d0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.871+0000 7ff518046740 1 -- 172.21.15.64:0/4222557648 shutdown_connections 2024-09-21T04:11:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.871+0000 7ff518046740 1 --2- 172.21.15.64:0/4222557648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f192501070 0x55f1928c0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.871+0000 7ff518046740 1 --2- 172.21.15.64:0/4222557648 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f1924bf9f0 0x55f1928c6160 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.871+0000 7ff518046740 1 --2- 172.21.15.64:0/4222557648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1924e0390 0x55f1928b42d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.871+0000 7ff518046740 1 --2- 172.21.15.64:0/4222557648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1924d81b0 0x55f1928b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.871+0000 7ff518046740 1 -- 172.21.15.64:0/4222557648 >> 172.21.15.64:0/4222557648 conn(0x55f1924f5460 msgr2=0x55f1928ab830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.871+0000 7ff4eaffd640 0 client.16971 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.871+0000 7ff518046740 1 -- 172.21.15.64:0/4222557648 shutdown_connections 2024-09-21T04:11:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.871+0000 7ff518046740 1 -- 172.21.15.64:0/4222557648 wait complete. 2024-09-21T04:11:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.871+0000 7ff518046740 20 client.16971 trim_cache size 0 max 16384 2024-09-21T04:11:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.889+0000 7ff518046740 1 Processor -- start 2024-09-21T04:11:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.890+0000 7ff518046740 1 -- start start 2024-09-21T04:11:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.890+0000 7ff518046740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1924d81b0 0x55f1928abe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:00.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.890+0000 7ff518046740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f1924e0390 0x55f1928c0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:00.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.890+0000 7ff518046740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f192501070 0x55f1928c0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:00.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.890+0000 7ff518046740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1b2200 con 0x55f1924d81b0 2024-09-21T04:11:00.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.890+0000 7ff518046740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1afa70 con 0x55f1924e0390 2024-09-21T04:11:00.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.890+0000 7ff518046740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1ad8e0 con 0x55f192501070 2024-09-21T04:11:00.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.890+0000 7ff50bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1924d81b0 0x55f1928abe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:00.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.890+0000 7ff50bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1924d81b0 0x55f1928abe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36174/0 (socket says 172.21.15.64:36174) 2024-09-21T04:11:00.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.890+0000 7ff50bfff640 1 -- 172.21.15.64:0/1452663115 learned_addr learned my addr 172.21.15.64:0/1452663115 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:00.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.890+0000 7ff50affd640 1 --2- 172.21.15.64:0/1452663115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f1924e0390 0x55f1928c0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:00.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.890+0000 7ff50bfff640 1 --2- 172.21.15.64:0/1452663115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f192501070 0x55f1928c0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:00.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.890+0000 7ff50bfff640 1 -- 172.21.15.64:0/1452663115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f192501070 msgr2=0x55f1928c0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:00.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.890+0000 7ff50bfff640 1 --2- 172.21.15.64:0/1452663115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f192501070 0x55f1928c0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.890+0000 7ff50bfff640 1 -- 172.21.15.64:0/1452663115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f1924e0390 msgr2=0x55f1928c0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:00.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.890+0000 7ff50bfff640 1 --2- 172.21.15.64:0/1452663115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f1924e0390 0x55f1928c0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.890+0000 7ff50bfff640 1 -- 172.21.15.64:0/1452663115 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1928aa940 con 0x55f1924d81b0 2024-09-21T04:11:00.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.891+0000 7ff50bfff640 1 --2- 172.21.15.64:0/1452663115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1924d81b0 0x55f1928abe60 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff504014040 tx=0x7ff50401c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f368974161a6093f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:00.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.891+0000 7ff4f17fa640 1 -- 172.21.15.64:0/1452663115 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff504025040 con 0x55f1924d81b0 2024-09-21T04:11:00.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.891+0000 7ff4f17fa640 1 -- 172.21.15.64:0/1452663115 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff504003fb0 con 0x55f1924d81b0 2024-09-21T04:11:00.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.891+0000 7ff4f17fa640 1 -- 172.21.15.64:0/1452663115 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff504005930 con 0x55f1924d81b0 2024-09-21T04:11:00.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.892+0000 7ff518046740 1 -- 172.21.15.64:0/1452663115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1924d81b0 msgr2=0x55f1928abe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:00.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.892+0000 7ff518046740 1 --2- 172.21.15.64:0/1452663115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1924d81b0 0x55f1928abe60 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff504014040 tx=0x7ff50401c040 comp rx=0 tx=0).stop 2024-09-21T04:11:00.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.892+0000 7ff518046740 1 -- 172.21.15.64:0/1452663115 shutdown_connections 2024-09-21T04:11:00.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.892+0000 7ff518046740 1 --2- 172.21.15.64:0/1452663115 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f192501070 0x55f1928c0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.892+0000 7ff518046740 1 --2- 172.21.15.64:0/1452663115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f1924e0390 0x55f1928c0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.892+0000 7ff518046740 1 --2- 172.21.15.64:0/1452663115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f1924d81b0 0x55f1928abe60 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.892+0000 7ff518046740 1 -- 172.21.15.64:0/1452663115 >> 172.21.15.64:0/1452663115 conn(0x55f1924f5460 msgr2=0x55f1928ab750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:00.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.892+0000 7ff518046740 1 -- 172.21.15.64:0/1452663115 shutdown_connections 2024-09-21T04:11:00.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.892+0000 7ff518046740 1 -- 172.21.15.64:0/1452663115 wait complete. 2024-09-21T04:11:00.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.892+0000 7ff518046740 1 -- start start 2024-09-21T04:11:00.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.893+0000 7ff518046740 1 Processor -- start 2024-09-21T04:11:00.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.893+0000 7ff518046740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f1924d81b0 0x55f1928b7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:00.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.893+0000 7ff518046740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1924e0390 0x55f1928b8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:00.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.893+0000 7ff518046740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f192501070 0x55f1928bbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:00.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.893+0000 7ff518046740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1adf30 con 0x55f192501070 2024-09-21T04:11:00.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.893+0000 7ff518046740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1af3a0 con 0x55f1924d81b0 2024-09-21T04:11:00.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.893+0000 7ff518046740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1ae600 con 0x55f1924e0390 2024-09-21T04:11:00.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.893+0000 7ff518046740 10 client.0 _ll_register_callbacks cb 0x7ff500009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:11:00.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.893+0000 7ff50bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f192501070 0x55f1928bbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:00.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.893+0000 7ff50bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f192501070 0x55f1928bbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36190/0 (socket says 172.21.15.64:36190) 2024-09-21T04:11:00.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.893+0000 7ff50bfff640 1 -- 172.21.15.64:0/1586468718 learned_addr learned my addr 172.21.15.64:0/1586468718 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:00.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.893+0000 7ff50bfff640 1 --2- 172.21.15.64:0/1586468718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f1924d81b0 0x55f1928b7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:00.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.893+0000 7ff50affd640 1 --2- 172.21.15.64:0/1586468718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1924e0390 0x55f1928b8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:00.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.893+0000 7ff50bfff640 1 -- 172.21.15.64:0/1586468718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1924e0390 msgr2=0x55f1928b8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:00.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.893+0000 7ff50bfff640 1 --2- 172.21.15.64:0/1586468718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1924e0390 0x55f1928b8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.893+0000 7ff50bfff640 1 -- 172.21.15.64:0/1586468718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f1924d81b0 msgr2=0x55f1928b7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:00.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.893+0000 7ff50bfff640 1 --2- 172.21.15.64:0/1586468718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f1924d81b0 0x55f1928b7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.894+0000 7ff50bfff640 1 -- 172.21.15.64:0/1586468718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1928bc630 con 0x55f192501070 2024-09-21T04:11:00.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.894+0000 7ff50bfff640 1 --2- 172.21.15.64:0/1586468718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f192501070 0x55f1928bbfc0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff5040117b0 tx=0x7ff504005490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe358a74e161bc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:00.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.894+0000 7ff4eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:11:00.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.894+0000 7ff4eaffd640 1 -- 172.21.15.64:0/1586468718 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff504005210 con 0x55f192501070 2024-09-21T04:11:00.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.894+0000 7ff4eaffd640 1 -- 172.21.15.64:0/1586468718 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7ff5040056d0 con 0x55f192501070 2024-09-21T04:11:00.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.894+0000 7ff518046740 1 -- 172.21.15.64:0/1586468718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff504003fb0 con 0x55f192501070 2024-09-21T04:11:00.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.894+0000 7ff4eaffd640 1 -- 172.21.15.64:0/1586468718 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff5040058d0 con 0x55f192501070 2024-09-21T04:11:00.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.894+0000 7ff518046740 10 client.16977 fetch_fsmap learned FSMap version 38 2024-09-21T04:11:00.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.894+0000 7ff518046740 1 -- 172.21.15.64:0/1586468718 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55f192930650 con 0x55f192501070 2024-09-21T04:11:00.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.894+0000 7ff4eaffd640 1 -- 172.21.15.64:0/1586468718 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff504003fb0 con 0x55f192501070 2024-09-21T04:11:00.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.894+0000 7ff4eaffd640 1 -- 172.21.15.64:0/1586468718 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7ff504018020 con 0x55f192501070 2024-09-21T04:11:00.110 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.895+0000 7ff518046740 10 client.16977 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T04:11:00.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.895+0000 7ff518046740 10 client.16977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:11:00.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.895+0000 7ff518046740 20 client.16977 populate_metadata read hostname 'smithi064' 2024-09-21T04:11:00.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.895+0000 7ff518046740 1 --2- 172.21.15.64:0/1586468718 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f192494da0 0x55f192932bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:00.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.895+0000 7ff518046740 4 client.16977 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T04:11:00.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.895+0000 7ff518046740 1 -- 172.21.15.64:0/1586468718 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f19281c850 con 0x55f192494da0 2024-09-21T04:11:00.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.895+0000 7ff50affd640 1 --2- 172.21.15.64:0/1586468718 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f192494da0 0x55f192932bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:00.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.895+0000 7ff50affd640 1 --2- 172.21.15.64:0/1586468718 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f192494da0 0x55f192932bf0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45f1808e4c992163 server_cookie=135ac0d888eadd96 in_seq=0 out_seq=0 2024-09-21T04:11:00.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.895+0000 7ff4eaffd640 10 client.16977 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:00.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.895+0000 7ff4eaffd640 1 -- 172.21.15.64:0/1586468718 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3126 (crc 0 0 0) 0x55f19281c850 con 0x55f192494da0 2024-09-21T04:11:00.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.895+0000 7ff4eaffd640 10 client.16977 handle_command_reply: tid=0 2024-09-21T04:11:00.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.895+0000 7ff518046740 1 client.16977 shutdown 2024-09-21T04:11:00.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.896+0000 7ff518046740 1 -- 172.21.15.64:0/1586468718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f192501070 msgr2=0x55f1928bbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:00.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.896+0000 7ff518046740 1 --2- 172.21.15.64:0/1586468718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f192501070 0x55f1928bbfc0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff5040117b0 tx=0x7ff504005490 comp rx=0 tx=0).stop 2024-09-21T04:11:00.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.896+0000 7ff518046740 1 -- 172.21.15.64:0/1586468718 shutdown_connections 2024-09-21T04:11:00.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.896+0000 7ff518046740 1 --2- 172.21.15.64:0/1586468718 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f192494da0 0x55f192932bf0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.896+0000 7ff518046740 1 --2- 172.21.15.64:0/1586468718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f192501070 0x55f1928bbfc0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.896+0000 7ff518046740 1 --2- 172.21.15.64:0/1586468718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f1924e0390 0x55f1928b8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.896+0000 7ff518046740 1 --2- 172.21.15.64:0/1586468718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f1924d81b0 0x55f1928b7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.896+0000 7ff518046740 1 -- 172.21.15.64:0/1586468718 >> 172.21.15.64:0/1586468718 conn(0x55f1924f5460 msgr2=0x55f1928a9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:00.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.896+0000 7ff4eaffd640 0 client.16977 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:00.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.896+0000 7ff518046740 1 -- 172.21.15.64:0/1586468718 shutdown_connections 2024-09-21T04:11:00.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.896+0000 7ff518046740 1 -- 172.21.15.64:0/1586468718 wait complete. 2024-09-21T04:11:00.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.896+0000 7ff518046740 20 client.16977 trim_cache size 0 max 16384 2024-09-21T04:11:00.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.896+0000 7ff511ad2640 1 -- 172.21.15.64:0/545942918 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff4f40b6680 msgr2=0x7ff4f40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:00.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.896+0000 7ff511ad2640 1 --2- 172.21.15.64:0/545942918 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff4f40b6680 0x7ff4f40b8b90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0e51d0 tx=0x7ff4fc0eb580 comp rx=0 tx=0).stop 2024-09-21T04:11:00.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.896+0000 7ff511ad2640 1 -- 172.21.15.64:0/545942918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff50c103b60 msgr2=0x7ff50c19a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:00.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.896+0000 7ff511ad2640 1 --2- 172.21.15.64:0/545942918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff50c103b60 0x7ff50c19a4a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00ea60 tx=0x7ff4fc00ef20 comp rx=0 tx=0).stop 2024-09-21T04:11:00.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.897+0000 7ff511ad2640 1 -- 172.21.15.64:0/545942918 shutdown_connections 2024-09-21T04:11:00.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.897+0000 7ff511ad2640 1 --2- 172.21.15.64:0/545942918 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff4f40b6680 0x7ff4f40b8b90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.897+0000 7ff511ad2640 1 --2- 172.21.15.64:0/545942918 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff50c109df0 0x7ff50c19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.897+0000 7ff511ad2640 1 --2- 172.21.15.64:0/545942918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff50c1094a0 0x7ff50c19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.897+0000 7ff511ad2640 1 --2- 172.21.15.64:0/545942918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff50c103b60 0x7ff50c19a4a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:00.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.897+0000 7ff511ad2640 1 -- 172.21.15.64:0/545942918 >> 172.21.15.64:0/545942918 conn(0x7ff50c0ff5d0 msgr2=0x7ff50c1019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:00.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.897+0000 7ff511ad2640 1 -- 172.21.15.64:0/545942918 shutdown_connections 2024-09-21T04:11:00.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:10:59.897+0000 7ff511ad2640 1 -- 172.21.15.64:0/545942918 wait complete. 2024-09-21T04:11:00.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:00 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:11:00.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:00 smithi119 ceph-mon[34811]: pgmap v532: 97 pgs: 97 active+clean; 811 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 434 op/s 2024-09-21T04:11:00.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:00 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:11:00.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:00 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:11:00.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:00 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:11:00.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:00 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:11:00.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:00 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:11:00.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:00 smithi064 ceph-mon[31180]: pgmap v532: 97 pgs: 97 active+clean; 811 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 434 op/s 2024-09-21T04:11:00.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:00 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:11:00.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:00 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:11:00.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:00 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:11:00.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:00 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:11:00.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:00 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:11:00.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:00 smithi131 ceph-mon[35193]: pgmap v532: 97 pgs: 97 active+clean; 811 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 434 op/s 2024-09-21T04:11:00.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:00 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:11:00.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:00 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:11:00.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:00 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:11:00.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:00 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:11:00.902 INFO:tasks.cephfs.filesystem:scrub status for tag:1b23713d-ea3a-4a86-a87f-65f6ab82671d - {'path': '/', 'tag': '1b23713d-ea3a-4a86-a87f-65f6ab82671d', 'options': 'recursive,force'} 2024-09-21T04:11:02.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:02 smithi119 ceph-mon[34811]: pgmap v533: 97 pgs: 97 active+clean; 817 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 407 op/s 2024-09-21T04:11:02.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:02 smithi064 ceph-mon[31180]: pgmap v533: 97 pgs: 97 active+clean; 817 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 407 op/s 2024-09-21T04:11:02.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:02 smithi131 ceph-mon[35193]: pgmap v533: 97 pgs: 97 active+clean; 817 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 407 op/s 2024-09-21T04:11:04.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:04 smithi119 ceph-mon[34811]: pgmap v534: 97 pgs: 97 active+clean; 793 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 764 op/s 2024-09-21T04:11:04.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:04 smithi064 ceph-mon[31180]: pgmap v534: 97 pgs: 97 active+clean; 793 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 764 op/s 2024-09-21T04:11:04.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:04 smithi131 ceph-mon[35193]: pgmap v534: 97 pgs: 97 active+clean; 793 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 764 op/s 2024-09-21T04:11:05.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.499+0000 7fe00d0bd640 1 -- 172.21.15.64:0/3572644233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0081029b0 msgr2=0x7fe008102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:05.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.499+0000 7fe00d0bd640 1 --2- 172.21.15.64:0/3572644233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0081029b0 0x7fe008102dc0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fdff8009900 tx=0x7fdff802a410 comp rx=0 tx=0).stop 2024-09-21T04:11:05.502 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.499+0000 7fe00d0bd640 1 -- 172.21.15.64:0/3572644233 shutdown_connections 2024-09-21T04:11:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.499+0000 7fe00d0bd640 1 --2- 172.21.15.64:0/3572644233 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe008108c30 0x7fe00810f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.499+0000 7fe00d0bd640 1 --2- 172.21.15.64:0/3572644233 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0081082e0 0x7fe0081086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.499+0000 7fe00d0bd640 1 --2- 172.21.15.64:0/3572644233 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0081029b0 0x7fe008102dc0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.499+0000 7fe00d0bd640 1 -- 172.21.15.64:0/3572644233 >> 172.21.15.64:0/3572644233 conn(0x7fe00807a740 msgr2=0x7fe00807ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.500+0000 7fe00d0bd640 1 -- 172.21.15.64:0/3572644233 shutdown_connections 2024-09-21T04:11:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.500+0000 7fe00d0bd640 1 -- 172.21.15.64:0/3572644233 wait complete. 2024-09-21T04:11:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.500+0000 7fe00d0bd640 1 Processor -- start 2024-09-21T04:11:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.501+0000 7fe00d0bd640 1 -- start start 2024-09-21T04:11:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.501+0000 7fe00d0bd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0081029b0 0x7fe00819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.501+0000 7fe00d0bd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0081082e0 0x7fe00819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.501+0000 7fe00d0bd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe008108c30 0x7fe00819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.501+0000 7fe00d0bd640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe0081128f0 con 0x7fe0081082e0 2024-09-21T04:11:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.501+0000 7fe00d0bd640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe008112770 con 0x7fe0081029b0 2024-09-21T04:11:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.501+0000 7fe00d0bd640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe008112a70 con 0x7fe008108c30 2024-09-21T04:11:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.501+0000 7fe00659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0081082e0 0x7fe00819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.501+0000 7fe006d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0081029b0 0x7fe00819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.501+0000 7fe00659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0081082e0 0x7fe00819aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41892/0 (socket says 172.21.15.64:41892) 2024-09-21T04:11:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.501+0000 7fe006d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0081029b0 0x7fe00819a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:33456/0 (socket says 172.21.15.64:33456) 2024-09-21T04:11:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.501+0000 7fe00659c640 1 -- 172.21.15.64:0/3215591331 learned_addr learned my addr 172.21.15.64:0/3215591331 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.501+0000 7fe00759e640 1 --2- 172.21.15.64:0/3215591331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe008108c30 0x7fe00819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.501+0000 7fe00659c640 1 -- 172.21.15.64:0/3215591331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe008108c30 msgr2=0x7fe00819fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.501+0000 7fe00659c640 1 --2- 172.21.15.64:0/3215591331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe008108c30 0x7fe00819fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.501+0000 7fe00659c640 1 -- 172.21.15.64:0/3215591331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0081029b0 msgr2=0x7fe00819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.501+0000 7fe00659c640 1 --2- 172.21.15.64:0/3215591331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0081029b0 0x7fe00819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.501+0000 7fe00659c640 1 -- 172.21.15.64:0/3215591331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0081a0390 con 0x7fe0081082e0 2024-09-21T04:11:05.506 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.502+0000 7fe00659c640 1 --2- 172.21.15.64:0/3215591331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0081082e0 0x7fe00819aa00 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fdffc00e9c0 tx=0x7fdffc00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.502+0000 7fdfeffff640 1 -- 172.21.15.64:0/3215591331 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc00ccc0 con 0x7fe0081082e0 2024-09-21T04:11:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.502+0000 7fdfeffff640 1 -- 172.21.15.64:0/3215591331 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdffc0054d0 con 0x7fe0081082e0 2024-09-21T04:11:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.502+0000 7fe00d0bd640 1 -- 172.21.15.64:0/3215591331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0081a0680 con 0x7fe0081082e0 2024-09-21T04:11:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.502+0000 7fdfeffff640 1 -- 172.21.15.64:0/3215591331 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc010430 con 0x7fe0081082e0 2024-09-21T04:11:05.507 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.502+0000 7fe00d0bd640 1 -- 172.21.15.64:0/3215591331 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0081a6e10 con 0x7fe0081082e0 2024-09-21T04:11:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.505+0000 7fdfeffff640 1 -- 172.21.15.64:0/3215591331 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fdffc0105d0 con 0x7fe0081082e0 2024-09-21T04:11:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.505+0000 7fe01402e740 1 Processor -- start 2024-09-21T04:11:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.505+0000 7fe01402e740 1 -- start start 2024-09-21T04:11:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.505+0000 7fe01402e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561982c461b0 0x561982feb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.505+0000 7fe01402e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561982c1d2f0 0x561982febef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.505+0000 7fe01402e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c254d0 0x561982fec430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.505+0000 7fe01402e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561982f606f0 con 0x561982c254d0 2024-09-21T04:11:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.505+0000 7fe01402e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561982f4fb20 con 0x561982c461b0 2024-09-21T04:11:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.505+0000 7fe01402e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x561982f53930 con 0x561982c1d2f0 2024-09-21T04:11:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.505+0000 7fe00659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c254d0 0x561982fec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:05.509 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.505+0000 7fe00759e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561982c461b0 0x561982feb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.505+0000 7fe00659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c254d0 0x561982fec430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41902/0 (socket says 172.21.15.64:41902) 2024-09-21T04:11:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.505+0000 7fe00659c640 1 -- 172.21.15.64:0/1960107455 learned_addr learned my addr 172.21.15.64:0/1960107455 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.505+0000 7fe006d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561982c1d2f0 0x561982febef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.506+0000 7fe00659c640 1 -- 172.21.15.64:0/1960107455 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561982c1d2f0 msgr2=0x561982febef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.506+0000 7fe00659c640 1 --2- 172.21.15.64:0/1960107455 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561982c1d2f0 0x561982febef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.506+0000 7fe00659c640 1 -- 172.21.15.64:0/1960107455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561982c461b0 msgr2=0x561982feb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.506+0000 7fe00659c640 1 --2- 172.21.15.64:0/1960107455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561982c461b0 0x561982feb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.506+0000 7fdfeffff640 1 --2- 172.21.15.64:0/3215591331 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdff00b6680 0x7fdff00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.506+0000 7fe00659c640 1 -- 172.21.15.64:0/1960107455 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561982f15530 con 0x561982c254d0 2024-09-21T04:11:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.506+0000 7fe00759e640 1 --2- 172.21.15.64:0/3215591331 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdff00b6680 0x7fdff00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.506+0000 7fdfeffff640 1 -- 172.21.15.64:0/3215591331 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7fdffc0e7c60 con 0x7fe0081082e0 2024-09-21T04:11:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.506+0000 7fe00659c640 1 --2- 172.21.15.64:0/1960107455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c254d0 0x561982fec430 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0e52c0 tx=0x7fdffc0adc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a4b4154bc58117 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:05.510 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.506+0000 7fdfed7fa640 1 -- 172.21.15.64:0/1960107455 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc0ad820 con 0x561982c254d0 2024-09-21T04:11:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.506+0000 7fdfed7fa640 1 -- 172.21.15.64:0/1960107455 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdffc0ad560 con 0x561982c254d0 2024-09-21T04:11:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.506+0000 7fdfed7fa640 1 -- 172.21.15.64:0/1960107455 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc0ad1c0 con 0x561982c254d0 2024-09-21T04:11:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.507+0000 7fe01402e740 1 -- 172.21.15.64:0/1960107455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c254d0 msgr2=0x561982fec430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.507+0000 7fe00759e640 1 --2- 172.21.15.64:0/3215591331 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdff00b6680 0x7fdff00b8b90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe000005fd0 tx=0x7fe000009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.507+0000 7fe01402e740 1 --2- 172.21.15.64:0/1960107455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c254d0 0x561982fec430 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0e52c0 tx=0x7fdffc0adc40 comp rx=0 tx=0).stop 2024-09-21T04:11:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.507+0000 7fe01402e740 1 -- 172.21.15.64:0/1960107455 shutdown_connections 2024-09-21T04:11:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.507+0000 7fe01402e740 1 --2- 172.21.15.64:0/1960107455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c254d0 0x561982fec430 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.507+0000 7fe01402e740 1 --2- 172.21.15.64:0/1960107455 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561982c1d2f0 0x561982febef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.507+0000 7fe01402e740 1 --2- 172.21.15.64:0/1960107455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561982c461b0 0x561982feb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.507+0000 7fe01402e740 1 -- 172.21.15.64:0/1960107455 >> 172.21.15.64:0/1960107455 conn(0x561982c3a300 msgr2=0x561982fe98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:05.515 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.507+0000 7fe01402e740 1 -- 172.21.15.64:0/1960107455 shutdown_connections 2024-09-21T04:11:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.507+0000 7fe01402e740 1 -- 172.21.15.64:0/1960107455 wait complete. 2024-09-21T04:11:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.508+0000 7fe01402e740 1 -- start start 2024-09-21T04:11:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.508+0000 7fe01402e740 1 Processor -- start 2024-09-21T04:11:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.508+0000 7fe01402e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561982c1d2f0 0x561982ff7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.508+0000 7fe01402e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c254d0 0x561982ff75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.508+0000 7fe01402e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561982c461b0 0x561983003420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.508+0000 7fe01402e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561982e87ab0 con 0x561982c254d0 2024-09-21T04:11:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.508+0000 7fe01402e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561982ef5730 con 0x561982c461b0 2024-09-21T04:11:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.508+0000 7fe01402e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x561982ffa130 con 0x561982c1d2f0 2024-09-21T04:11:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.508+0000 7fe01402e740 10 client.0 _ll_register_callbacks cb 0x561982f496b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:11:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.508+0000 7fe00659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c254d0 0x561982ff75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.508+0000 7fe00659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c254d0 0x561982ff75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41916/0 (socket says 172.21.15.64:41916) 2024-09-21T04:11:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.508+0000 7fe00659c640 1 -- 172.21.15.64:0/1108334287 learned_addr learned my addr 172.21.15.64:0/1108334287 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:05.516 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.508+0000 7fe006d9d640 1 --2- 172.21.15.64:0/1108334287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561982c1d2f0 0x561982ff7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.508+0000 7fe00759e640 1 --2- 172.21.15.64:0/1108334287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561982c461b0 0x561983003420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.509+0000 7fe00659c640 1 -- 172.21.15.64:0/1108334287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561982c1d2f0 msgr2=0x561982ff7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.509+0000 7fe00659c640 1 --2- 172.21.15.64:0/1108334287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561982c1d2f0 0x561982ff7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.509+0000 7fe00659c640 1 -- 172.21.15.64:0/1108334287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561982c461b0 msgr2=0x561983003420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.509+0000 7fe00659c640 1 --2- 172.21.15.64:0/1108334287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561982c461b0 0x561983003420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.509+0000 7fe00659c640 1 -- 172.21.15.64:0/1108334287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561982cb1790 con 0x561982c254d0 2024-09-21T04:11:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.509+0000 7fe00659c640 1 --2- 172.21.15.64:0/1108334287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c254d0 0x561982ff75d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0e5400 tx=0x7fdffc0e7fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3cbaa3c9104c62 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.509+0000 7fdfeaff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:11:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.509+0000 7fdfeaff5640 1 -- 172.21.15.64:0/1108334287 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc0eb050 con 0x561982c254d0 2024-09-21T04:11:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.509+0000 7fdfeaff5640 1 -- 172.21.15.64:0/1108334287 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7fdffc0adcb0 con 0x561982c254d0 2024-09-21T04:11:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.509+0000 7fe01402e740 1 -- 172.21.15.64:0/1108334287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdffc0ad560 con 0x561982c254d0 2024-09-21T04:11:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.509+0000 7fdfeaff5640 1 -- 172.21.15.64:0/1108334287 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fdffc0ad3a0 con 0x561982c254d0 2024-09-21T04:11:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.509+0000 7fe01402e740 10 client.16989 fetch_fsmap learned FSMap version 38 2024-09-21T04:11:05.517 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.509+0000 7fe01402e740 1 -- 172.21.15.64:0/1108334287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x561982d2a620 con 0x561982c254d0 2024-09-21T04:11:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.509+0000 7fdfeaff5640 1 -- 172.21.15.64:0/1108334287 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdffc0ad560 con 0x561982c254d0 2024-09-21T04:11:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.509+0000 7fdfeaff5640 1 -- 172.21.15.64:0/1108334287 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7fdffc048ce0 con 0x561982c254d0 2024-09-21T04:11:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.510+0000 7fe01402e740 10 client.16989 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T04:11:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.510+0000 7fe01402e740 10 client.16989 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:11:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.510+0000 7fe01402e740 20 client.16989 populate_metadata read hostname 'smithi064' 2024-09-21T04:11:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.510+0000 7fe01402e740 1 --2- 172.21.15.64:0/1108334287 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x561982c04d80 0x561983009000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.510+0000 7fe01402e740 4 client.16989 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:11:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.510+0000 7fe01402e740 1 -- 172.21.15.64:0/1108334287 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561982f62640 con 0x561982c04d80 2024-09-21T04:11:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.510+0000 7fe006d9d640 1 --2- 172.21.15.64:0/1108334287 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x561982c04d80 0x561983009000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.510+0000 7fe006d9d640 1 --2- 172.21.15.64:0/1108334287 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x561982c04d80 0x561983009000 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a2702ce85b7de43 server_cookie=6b3e1a73477e347d in_seq=0 out_seq=0 2024-09-21T04:11:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.510+0000 7fdfeaff5640 10 client.16989 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.511+0000 7fdfeaff5640 1 -- 172.21.15.64:0/1108334287 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561982f62640 con 0x561982c04d80 2024-09-21T04:11:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.511+0000 7fdfeaff5640 10 client.16989 handle_command_reply: tid=0 2024-09-21T04:11:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.511+0000 7fe01402e740 1 client.16989 shutdown 2024-09-21T04:11:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.511+0000 7fe01402e740 1 -- 172.21.15.64:0/1108334287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c254d0 msgr2=0x561982ff75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.511+0000 7fe01402e740 1 --2- 172.21.15.64:0/1108334287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c254d0 0x561982ff75d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0e5400 tx=0x7fdffc0e7fd0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.511+0000 7fe01402e740 1 -- 172.21.15.64:0/1108334287 shutdown_connections 2024-09-21T04:11:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.511+0000 7fe01402e740 1 --2- 172.21.15.64:0/1108334287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561982c461b0 0x561983003420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.511+0000 7fe01402e740 1 --2- 172.21.15.64:0/1108334287 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x561982c04d80 0x561983009000 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.511+0000 7fe01402e740 1 --2- 172.21.15.64:0/1108334287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c254d0 0x561982ff75d0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.511+0000 7fdfeaff5640 0 client.16989 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.511+0000 7fe01402e740 1 --2- 172.21.15.64:0/1108334287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561982c1d2f0 0x561982ff7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.511+0000 7fe01402e740 1 -- 172.21.15.64:0/1108334287 >> 172.21.15.64:0/1108334287 conn(0x561982c3a300 msgr2=0x561982fec9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.512+0000 7fe01402e740 1 -- 172.21.15.64:0/1108334287 shutdown_connections 2024-09-21T04:11:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.512+0000 7fe01402e740 1 -- 172.21.15.64:0/1108334287 wait complete. 2024-09-21T04:11:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.512+0000 7fe01402e740 20 client.16989 trim_cache size 0 max 16384 2024-09-21T04:11:05.529 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.526+0000 7fe01402e740 1 Processor -- start 2024-09-21T04:11:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.526+0000 7fe01402e740 1 -- start start 2024-09-21T04:11:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.527+0000 7fe01402e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561982c04d80 0x561982fed0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.527+0000 7fe01402e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c1d2f0 0x561983003390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.527+0000 7fe01402e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561982c254d0 0x561982ff4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.527+0000 7fe01402e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe0081b2190 con 0x561982c1d2f0 2024-09-21T04:11:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.527+0000 7fe01402e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe0081afa00 con 0x561982c254d0 2024-09-21T04:11:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.527+0000 7fe01402e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe0081ad870 con 0x561982c04d80 2024-09-21T04:11:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.527+0000 7fe00659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c1d2f0 0x561983003390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.527+0000 7fe00659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c1d2f0 0x561983003390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41920/0 (socket says 172.21.15.64:41920) 2024-09-21T04:11:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.527+0000 7fe00659c640 1 -- 172.21.15.64:0/2733386145 learned_addr learned my addr 172.21.15.64:0/2733386145 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.527+0000 7fe006d9d640 1 --2- 172.21.15.64:0/2733386145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561982c04d80 0x561982fed0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:05.530 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.527+0000 7fe00759e640 1 --2- 172.21.15.64:0/2733386145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561982c254d0 0x561982ff4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.527+0000 7fe00659c640 1 -- 172.21.15.64:0/2733386145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561982c04d80 msgr2=0x561982fed0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.527+0000 7fe00659c640 1 --2- 172.21.15.64:0/2733386145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561982c04d80 0x561982fed0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.527+0000 7fe00659c640 1 -- 172.21.15.64:0/2733386145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561982c254d0 msgr2=0x561982ff4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.527+0000 7fe00659c640 1 --2- 172.21.15.64:0/2733386145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561982c254d0 0x561982ff4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.527+0000 7fe00659c640 1 -- 172.21.15.64:0/2733386145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561982fea220 con 0x561982c1d2f0 2024-09-21T04:11:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.527+0000 7fe00659c640 1 --2- 172.21.15.64:0/2733386145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c1d2f0 0x561983003390 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0aa020 tx=0x7fdffc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7c715d33fe0a9d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.527+0000 7fdfecff9640 1 -- 172.21.15.64:0/2733386145 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc0ad560 con 0x561982c1d2f0 2024-09-21T04:11:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.527+0000 7fdfecff9640 1 -- 172.21.15.64:0/2733386145 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdffc0d10d0 con 0x561982c1d2f0 2024-09-21T04:11:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.527+0000 7fdfecff9640 1 -- 172.21.15.64:0/2733386145 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc0eb2c0 con 0x561982c1d2f0 2024-09-21T04:11:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.528+0000 7fe01402e740 1 -- 172.21.15.64:0/2733386145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c1d2f0 msgr2=0x561983003390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.528+0000 7fe01402e740 1 --2- 172.21.15.64:0/2733386145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c1d2f0 0x561983003390 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0aa020 tx=0x7fdffc0046c0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.528+0000 7fe01402e740 1 -- 172.21.15.64:0/2733386145 shutdown_connections 2024-09-21T04:11:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.528+0000 7fe01402e740 1 --2- 172.21.15.64:0/2733386145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561982c254d0 0x561982ff4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.528+0000 7fe01402e740 1 --2- 172.21.15.64:0/2733386145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c1d2f0 0x561983003390 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.528+0000 7fe01402e740 1 --2- 172.21.15.64:0/2733386145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561982c04d80 0x561982fed0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.535 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.528+0000 7fe01402e740 1 -- 172.21.15.64:0/2733386145 >> 172.21.15.64:0/2733386145 conn(0x561982c3a300 msgr2=0x561982fec9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.528+0000 7fe01402e740 1 -- 172.21.15.64:0/2733386145 shutdown_connections 2024-09-21T04:11:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.528+0000 7fe01402e740 1 -- 172.21.15.64:0/2733386145 wait complete. 2024-09-21T04:11:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.529+0000 7fe01402e740 1 -- start start 2024-09-21T04:11:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.529+0000 7fe01402e740 1 Processor -- start 2024-09-21T04:11:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.529+0000 7fe01402e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561982c04d80 0x56198300b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.529+0000 7fe01402e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c1d2f0 0x56198300ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.529+0000 7fe01402e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561982c254d0 0x561983071780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.529+0000 7fe01402e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe0081adec0 con 0x561982c1d2f0 2024-09-21T04:11:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.529+0000 7fe01402e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe0081af330 con 0x561982c04d80 2024-09-21T04:11:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.529+0000 7fe01402e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe0081ae590 con 0x561982c254d0 2024-09-21T04:11:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.529+0000 7fe01402e740 10 client.0 _ll_register_callbacks cb 0x7fe000009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:11:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.529+0000 7fe00659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c1d2f0 0x56198300ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.529+0000 7fe00659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c1d2f0 0x56198300ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41934/0 (socket says 172.21.15.64:41934) 2024-09-21T04:11:05.536 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.529+0000 7fe00659c640 1 -- 172.21.15.64:0/949370264 learned_addr learned my addr 172.21.15.64:0/949370264 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.529+0000 7fe006d9d640 1 --2- 172.21.15.64:0/949370264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561982c04d80 0x56198300b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.530+0000 7fe00759e640 1 --2- 172.21.15.64:0/949370264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561982c254d0 0x561983071780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.530+0000 7fe00659c640 1 -- 172.21.15.64:0/949370264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561982c254d0 msgr2=0x561983071780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.530+0000 7fe00659c640 1 --2- 172.21.15.64:0/949370264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561982c254d0 0x561983071780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.530+0000 7fe00659c640 1 -- 172.21.15.64:0/949370264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561982c04d80 msgr2=0x56198300b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.530+0000 7fe00659c640 1 --2- 172.21.15.64:0/949370264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561982c04d80 0x56198300b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.530+0000 7fe00659c640 1 -- 172.21.15.64:0/949370264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561983071cc0 con 0x561982c1d2f0 2024-09-21T04:11:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.530+0000 7fe006d9d640 1 --2- 172.21.15.64:0/949370264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561982c04d80 0x56198300b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:11:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.530+0000 7fe00659c640 1 --2- 172.21.15.64:0/949370264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c1d2f0 0x56198300ba30 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0ad700 tx=0x7fdffc0ad920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce235053a8535692 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.530+0000 7fdfeaff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:11:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.530+0000 7fdfeaff5640 1 -- 172.21.15.64:0/949370264 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc0ad560 con 0x561982c1d2f0 2024-09-21T04:11:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.530+0000 7fdfeaff5640 1 -- 172.21.15.64:0/949370264 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8615+0+0 (secure 0 0 0) 0x7fdffc0eb580 con 0x561982c1d2f0 2024-09-21T04:11:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.530+0000 7fe01402e740 1 -- 172.21.15.64:0/949370264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdffc0d10d0 con 0x561982c1d2f0 2024-09-21T04:11:05.537 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.530+0000 7fdfeaff5640 1 -- 172.21.15.64:0/949370264 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fdffc0b2530 con 0x561982c1d2f0 2024-09-21T04:11:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.530+0000 7fe01402e740 10 client.16995 fetch_fsmap learned FSMap version 38 2024-09-21T04:11:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.530+0000 7fe01402e740 1 -- 172.21.15.64:0/949370264 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x561983071cc0 con 0x561982c1d2f0 2024-09-21T04:11:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.531+0000 7fdfeaff5640 1 -- 172.21.15.64:0/949370264 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdffc0d10d0 con 0x561982c1d2f0 2024-09-21T04:11:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.531+0000 7fdfeaff5640 1 -- 172.21.15.64:0/949370264 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7fdffc0a93a0 con 0x561982c1d2f0 2024-09-21T04:11:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.531+0000 7fe01402e740 10 client.16995 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T04:11:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.531+0000 7fe01402e740 10 client.16995 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:11:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.531+0000 7fe01402e740 20 client.16995 populate_metadata read hostname 'smithi064' 2024-09-21T04:11:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.531+0000 7fe01402e740 1 --2- 172.21.15.64:0/949370264 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x561982c461b0 0x5619830751c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.531+0000 7fe01402e740 4 client.16995 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:11:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.531+0000 7fe01402e740 1 -- 172.21.15.64:0/949370264 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561982f5e0f0 con 0x561982c461b0 2024-09-21T04:11:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.531+0000 7fe006d9d640 1 --2- 172.21.15.64:0/949370264 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x561982c461b0 0x5619830751c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.531+0000 7fdfeaff5640 10 client.16995 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.531+0000 7fe006d9d640 1 --2- 172.21.15.64:0/949370264 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x561982c461b0 0x5619830751c0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5e1014936b5c380 server_cookie=2c72570bdb98fe5c in_seq=0 out_seq=0 2024-09-21T04:11:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.582+0000 7fdfeaff5640 1 -- 172.21.15.64:0/949370264 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561982f5e0f0 con 0x561982c461b0 2024-09-21T04:11:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.582+0000 7fdfeaff5640 10 client.16995 handle_command_reply: tid=0 2024-09-21T04:11:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.582+0000 7fe01402e740 1 client.16995 shutdown 2024-09-21T04:11:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.582+0000 7fe01402e740 1 -- 172.21.15.64:0/949370264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c1d2f0 msgr2=0x56198300ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.582+0000 7fe01402e740 1 --2- 172.21.15.64:0/949370264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c1d2f0 0x56198300ba30 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0ad700 tx=0x7fdffc0ad920 comp rx=0 tx=0).stop 2024-09-21T04:11:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.582+0000 7fe01402e740 1 -- 172.21.15.64:0/949370264 shutdown_connections 2024-09-21T04:11:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.582+0000 7fe01402e740 1 --2- 172.21.15.64:0/949370264 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x561982c254d0 0x561983071780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.582+0000 7fe01402e740 1 --2- 172.21.15.64:0/949370264 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x561982c461b0 0x5619830751c0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.582+0000 7fe01402e740 1 --2- 172.21.15.64:0/949370264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561982c1d2f0 0x56198300ba30 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.582+0000 7fe01402e740 1 --2- 172.21.15.64:0/949370264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561982c04d80 0x56198300b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.582+0000 7fe01402e740 1 -- 172.21.15.64:0/949370264 >> 172.21.15.64:0/949370264 conn(0x561982c3a300 msgr2=0x561982fe9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.582+0000 7fdfeaff5640 0 client.16995 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.582+0000 7fe01402e740 1 -- 172.21.15.64:0/949370264 shutdown_connections 2024-09-21T04:11:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.582+0000 7fe01402e740 1 -- 172.21.15.64:0/949370264 wait complete. 2024-09-21T04:11:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.582+0000 7fe01402e740 20 client.16995 trim_cache size 0 max 16384 2024-09-21T04:11:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:11:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:11:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:11:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:11:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.583+0000 7fe00d0bd640 1 -- 172.21.15.64:0/3215591331 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdff00b6680 msgr2=0x7fdff00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.583+0000 7fe00d0bd640 1 --2- 172.21.15.64:0/3215591331 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdff00b6680 0x7fdff00b8b90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe000005fd0 tx=0x7fe000009500 comp rx=0 tx=0).stop 2024-09-21T04:11:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.583+0000 7fe00d0bd640 1 -- 172.21.15.64:0/3215591331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0081082e0 msgr2=0x7fe00819aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.583+0000 7fe00d0bd640 1 --2- 172.21.15.64:0/3215591331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0081082e0 0x7fe00819aa00 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fdffc00e9c0 tx=0x7fdffc00edc0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.583+0000 7fe00d0bd640 1 -- 172.21.15.64:0/3215591331 shutdown_connections 2024-09-21T04:11:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.583+0000 7fe00d0bd640 1 --2- 172.21.15.64:0/3215591331 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdff00b6680 0x7fdff00b8b90 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.583+0000 7fe00d0bd640 1 --2- 172.21.15.64:0/3215591331 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe008108c30 0x7fe00819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.583+0000 7fe00d0bd640 1 --2- 172.21.15.64:0/3215591331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0081082e0 0x7fe00819aa00 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.583+0000 7fe00d0bd640 1 --2- 172.21.15.64:0/3215591331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0081029b0 0x7fe00819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.583+0000 7fe00d0bd640 1 -- 172.21.15.64:0/3215591331 >> 172.21.15.64:0/3215591331 conn(0x7fe00807a740 msgr2=0x7fe008101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.583+0000 7fe00d0bd640 1 -- 172.21.15.64:0/3215591331 shutdown_connections 2024-09-21T04:11:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:05.583+0000 7fe00d0bd640 1 -- 172.21.15.64:0/3215591331 wait complete. 2024-09-21T04:11:06.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:06 smithi119 ceph-mon[34811]: pgmap v535: 97 pgs: 97 active+clean; 793 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 587 op/s 2024-09-21T04:11:06.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:06 smithi064 ceph-mon[31180]: pgmap v535: 97 pgs: 97 active+clean; 793 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 587 op/s 2024-09-21T04:11:06.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:06 smithi131 ceph-mon[35193]: pgmap v535: 97 pgs: 97 active+clean; 793 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 587 op/s 2024-09-21T04:11:08.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:08 smithi119 ceph-mon[34811]: pgmap v536: 97 pgs: 97 active+clean; 796 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 838 op/s 2024-09-21T04:11:08.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:08 smithi064 ceph-mon[31180]: pgmap v536: 97 pgs: 97 active+clean; 796 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 838 op/s 2024-09-21T04:11:08.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:08 smithi131 ceph-mon[35193]: pgmap v536: 97 pgs: 97 active+clean; 796 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 838 op/s 2024-09-21T04:11:09.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:09 smithi119 ceph-mon[34811]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T04:11:09.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:09 smithi064 ceph-mon[31180]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T04:11:09.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:09 smithi131 ceph-mon[35193]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T04:11:10.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:10 smithi119 ceph-mon[34811]: pgmap v538: 97 pgs: 97 active+clean; 792 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 849 op/s 2024-09-21T04:11:10.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:10 smithi064 ceph-mon[31180]: pgmap v538: 97 pgs: 97 active+clean; 792 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 849 op/s 2024-09-21T04:11:10.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:10 smithi131 ceph-mon[35193]: pgmap v538: 97 pgs: 97 active+clean; 792 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 849 op/s 2024-09-21T04:11:12.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:12 smithi119 ceph-mon[34811]: pgmap v539: 97 pgs: 97 active+clean; 792 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 840 op/s 2024-09-21T04:11:12.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:12 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:11:12.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:12 smithi064 ceph-mon[31180]: pgmap v539: 97 pgs: 97 active+clean; 792 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 840 op/s 2024-09-21T04:11:12.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:12 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:11:12.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:12 smithi131 ceph-mon[35193]: pgmap v539: 97 pgs: 97 active+clean; 792 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 840 op/s 2024-09-21T04:11:12.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:12 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:11:14.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:14 smithi064 ceph-mon[31180]: pgmap v540: 97 pgs: 97 active+clean; 794 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 640 KiB/s wr, 743 op/s 2024-09-21T04:11:14.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:14 smithi131 ceph-mon[35193]: pgmap v540: 97 pgs: 97 active+clean; 794 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 640 KiB/s wr, 743 op/s 2024-09-21T04:11:14.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:14 smithi119 ceph-mon[34811]: pgmap v540: 97 pgs: 97 active+clean; 794 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 640 KiB/s wr, 743 op/s 2024-09-21T04:11:16.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:16 smithi064 ceph-mon[31180]: pgmap v541: 97 pgs: 97 active+clean; 794 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 640 KiB/s wr, 743 op/s 2024-09-21T04:11:16.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:16 smithi064 ceph-mon[31180]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T04:11:16.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:16 smithi131 ceph-mon[35193]: pgmap v541: 97 pgs: 97 active+clean; 794 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 640 KiB/s wr, 743 op/s 2024-09-21T04:11:16.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:16 smithi131 ceph-mon[35193]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T04:11:16.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:16 smithi119 ceph-mon[34811]: pgmap v541: 97 pgs: 97 active+clean; 794 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 640 KiB/s wr, 743 op/s 2024-09-21T04:11:16.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:16 smithi119 ceph-mon[34811]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T04:11:18.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:18 smithi064 ceph-mon[31180]: pgmap v543: 97 pgs: 97 active+clean; 798 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 916 KiB/s wr, 628 op/s 2024-09-21T04:11:18.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:18 smithi131 ceph-mon[35193]: pgmap v543: 97 pgs: 97 active+clean; 798 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 916 KiB/s wr, 628 op/s 2024-09-21T04:11:18.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:18 smithi119 ceph-mon[34811]: pgmap v543: 97 pgs: 97 active+clean; 798 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 916 KiB/s wr, 628 op/s 2024-09-21T04:11:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.687+0000 7f5cdcc4d640 1 -- 172.21.15.64:0/539376706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cd81029e0 msgr2=0x7f5cd8102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.687+0000 7f5cdcc4d640 1 --2- 172.21.15.64:0/539376706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cd81029e0 0x7f5cd8102df0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009ca0 tx=0x7f5cc802a8c0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.687+0000 7f5cdcc4d640 1 -- 172.21.15.64:0/539376706 shutdown_connections 2024-09-21T04:11:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.687+0000 7f5cdcc4d640 1 --2- 172.21.15.64:0/539376706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cd8108c60 0x7f5cd810f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.687+0000 7f5cdcc4d640 1 --2- 172.21.15.64:0/539376706 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5cd8108310 0x7f5cd8108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.687+0000 7f5cdcc4d640 1 --2- 172.21.15.64:0/539376706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cd81029e0 0x7f5cd8102df0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.687+0000 7f5cdcc4d640 1 -- 172.21.15.64:0/539376706 >> 172.21.15.64:0/539376706 conn(0x7f5cd80fe3e0 msgr2=0x7f5cd8100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.687+0000 7f5cdcc4d640 1 -- 172.21.15.64:0/539376706 shutdown_connections 2024-09-21T04:11:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.687+0000 7f5cdcc4d640 1 -- 172.21.15.64:0/539376706 wait complete. 2024-09-21T04:11:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.688+0000 7f5cdcc4d640 1 Processor -- start 2024-09-21T04:11:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.688+0000 7f5cdcc4d640 1 -- start start 2024-09-21T04:11:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.688+0000 7f5cdcc4d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5cd81029e0 0x7f5cd80fe910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.688+0000 7f5cdcc4d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cd8108310 0x7f5cd80fee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.688+0000 7f5cdcc4d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cd8108c60 0x7f5cd80ff390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.688+0000 7f5cdcc4d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd8112920 con 0x7f5cd8108c60 2024-09-21T04:11:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.688+0000 7f5cdcc4d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd81127a0 con 0x7f5cd8108310 2024-09-21T04:11:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.688+0000 7f5cdcc4d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd8112aa0 con 0x7f5cd81029e0 2024-09-21T04:11:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.688+0000 7f5cd6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cd8108c60 0x7f5cd80ff390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.688+0000 7f5cd6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cd8108c60 0x7f5cd80ff390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52510/0 (socket says 172.21.15.64:52510) 2024-09-21T04:11:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.688+0000 7f5cd6d9d640 1 -- 172.21.15.64:0/4173847288 learned_addr learned my addr 172.21.15.64:0/4173847288 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.688+0000 7f5cd5d9b640 1 --2- 172.21.15.64:0/4173847288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cd8108310 0x7f5cd80fee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.688+0000 7f5cd659c640 1 --2- 172.21.15.64:0/4173847288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5cd81029e0 0x7f5cd80fe910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.688+0000 7f5cd6d9d640 1 -- 172.21.15.64:0/4173847288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5cd81029e0 msgr2=0x7f5cd80fe910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.688+0000 7f5cd6d9d640 1 --2- 172.21.15.64:0/4173847288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5cd81029e0 0x7f5cd80fe910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.688+0000 7f5cd6d9d640 1 -- 172.21.15.64:0/4173847288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cd8108310 msgr2=0x7f5cd80fee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.688+0000 7f5cd6d9d640 1 --2- 172.21.15.64:0/4173847288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cd8108310 0x7f5cd80fee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.688+0000 7f5cd6d9d640 1 -- 172.21.15.64:0/4173847288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd81a8720 con 0x7f5cd8108c60 2024-09-21T04:11:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.688+0000 7f5cd5d9b640 1 --2- 172.21.15.64:0/4173847288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cd8108310 0x7f5cd80fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:11:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.689+0000 7f5cd6d9d640 1 --2- 172.21.15.64:0/4173847288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cd8108c60 0x7f5cd80ff390 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000e970 tx=0x7f5cd000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.689+0000 7f5ccf7fe640 1 -- 172.21.15.64:0/4173847288 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd000ccc0 con 0x7f5cd8108c60 2024-09-21T04:11:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.689+0000 7f5ccf7fe640 1 -- 172.21.15.64:0/4173847288 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cd000ce60 con 0x7f5cd8108c60 2024-09-21T04:11:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.689+0000 7f5cdcc4d640 1 -- 172.21.15.64:0/4173847288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cd81a8a10 con 0x7f5cd8108c60 2024-09-21T04:11:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.689+0000 7f5ccf7fe640 1 -- 172.21.15.64:0/4173847288 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0010430 con 0x7f5cd8108c60 2024-09-21T04:11:18.694 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.689+0000 7f5cdcc4d640 1 -- 172.21.15.64:0/4173847288 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cd81a8f60 con 0x7f5cd8108c60 2024-09-21T04:11:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.692+0000 7f5ccf7fe640 1 -- 172.21.15.64:0/4173847288 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f5cd0004f90 con 0x7f5cd8108c60 2024-09-21T04:11:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.692+0000 7f5ce3bc6740 1 Processor -- start 2024-09-21T04:11:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.692+0000 7f5ce3bc6740 1 -- start start 2024-09-21T04:11:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.692+0000 7f5ce3bc6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac60ea11b0 0x55ac612469b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.692+0000 7f5ce3bc6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac60e782f0 0x55ac61246ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.693+0000 7f5ce3bc6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac60e804d0 0x55ac61247430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.693+0000 7f5ce3bc6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ac611bb6f0 con 0x55ac60e782f0 2024-09-21T04:11:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.693+0000 7f5ce3bc6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ac611aab20 con 0x55ac60ea11b0 2024-09-21T04:11:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.693+0000 7f5ce3bc6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ac611ae930 con 0x55ac60e804d0 2024-09-21T04:11:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.693+0000 7f5cd659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac60e782f0 0x55ac61246ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.693+0000 7f5cd5d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac60ea11b0 0x55ac612469b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.693+0000 7f5cd5d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac60ea11b0 0x55ac612469b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:43304/0 (socket says 172.21.15.64:43304) 2024-09-21T04:11:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.693+0000 7f5cd5d9b640 1 -- 172.21.15.64:0/1916025689 learned_addr learned my addr 172.21.15.64:0/1916025689 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.693+0000 7f5ccf7fe640 1 --2- 172.21.15.64:0/4173847288 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5cc40b6730 0x7f5cc40b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.693+0000 7f5cd5d9b640 1 --2- 172.21.15.64:0/1916025689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac60e804d0 0x55ac61247430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.693+0000 7f5cd6d9d640 1 --2- 172.21.15.64:0/4173847288 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5cc40b6730 0x7f5cc40b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.693+0000 7f5cd5d9b640 1 -- 172.21.15.64:0/1916025689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac60e804d0 msgr2=0x55ac61247430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.693+0000 7f5cd5d9b640 1 --2- 172.21.15.64:0/1916025689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac60e804d0 0x55ac61247430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.693+0000 7f5cd5d9b640 1 -- 172.21.15.64:0/1916025689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac60e782f0 msgr2=0x55ac61246ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.693+0000 7f5cd5d9b640 1 --2- 172.21.15.64:0/1916025689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac60e782f0 0x55ac61246ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.693+0000 7f5cd5d9b640 1 -- 172.21.15.64:0/1916025689 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac61170530 con 0x55ac60ea11b0 2024-09-21T04:11:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.693+0000 7f5cd659c640 1 --2- 172.21.15.64:0/1916025689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac60e782f0 0x55ac61246ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:11:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.693+0000 7f5ccf7fe640 1 -- 172.21.15.64:0/4173847288 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7f5cd00e7ad0 con 0x7f5cd8108c60 2024-09-21T04:11:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.693+0000 7f5cd5d9b640 1 --2- 172.21.15.64:0/1916025689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac60ea11b0 0x55ac612469b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0012b50 tx=0x7f5cc0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6776d7da38d0a563 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.693+0000 7f5cd6d9d640 1 --2- 172.21.15.64:0/4173847288 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5cc40b6730 0x7f5cc40b8c40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5cd00e5270 tx=0x7f5cd00049b0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.694+0000 7f5cccff9640 1 -- 172.21.15.64:0/1916025689 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0012c40 con 0x55ac60ea11b0 2024-09-21T04:11:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.694+0000 7f5cccff9640 1 -- 172.21.15.64:0/1916025689 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cc0012de0 con 0x55ac60ea11b0 2024-09-21T04:11:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.694+0000 7f5cccff9640 1 -- 172.21.15.64:0/1916025689 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc00049a0 con 0x55ac60ea11b0 2024-09-21T04:11:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.694+0000 7f5ce3bc6740 1 -- 172.21.15.64:0/1916025689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac60ea11b0 msgr2=0x55ac612469b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.694+0000 7f5ce3bc6740 1 --2- 172.21.15.64:0/1916025689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac60ea11b0 0x55ac612469b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0012b50 tx=0x7f5cc0004730 comp rx=0 tx=0).stop 2024-09-21T04:11:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.694+0000 7f5ce3bc6740 1 -- 172.21.15.64:0/1916025689 shutdown_connections 2024-09-21T04:11:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.694+0000 7f5ce3bc6740 1 --2- 172.21.15.64:0/1916025689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac60e804d0 0x55ac61247430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.694+0000 7f5ce3bc6740 1 --2- 172.21.15.64:0/1916025689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac60e782f0 0x55ac61246ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.694+0000 7f5ce3bc6740 1 --2- 172.21.15.64:0/1916025689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac60ea11b0 0x55ac612469b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.695+0000 7f5ce3bc6740 1 -- 172.21.15.64:0/1916025689 >> 172.21.15.64:0/1916025689 conn(0x55ac60e95300 msgr2=0x55ac612448a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.695+0000 7f5ce3bc6740 1 -- 172.21.15.64:0/1916025689 shutdown_connections 2024-09-21T04:11:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.695+0000 7f5ce3bc6740 1 -- 172.21.15.64:0/1916025689 wait complete. 2024-09-21T04:11:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.695+0000 7f5ce3bc6740 1 -- start start 2024-09-21T04:11:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.695+0000 7f5ce3bc6740 1 Processor -- start 2024-09-21T04:11:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.695+0000 7f5ce3bc6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac60e782f0 0x55ac61252070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.695+0000 7f5ce3bc6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac60e804d0 0x55ac612525b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.695+0000 7f5ce3bc6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac60ea11b0 0x55ac6125e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.695+0000 7f5ce3bc6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ac610e2ab0 con 0x55ac60e782f0 2024-09-21T04:11:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.695+0000 7f5ce3bc6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ac61150730 con 0x55ac60ea11b0 2024-09-21T04:11:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.695+0000 7f5ce3bc6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ac61255130 con 0x55ac60e804d0 2024-09-21T04:11:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.696+0000 7f5ce3bc6740 10 client.0 _ll_register_callbacks cb 0x55ac611a46b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:11:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.696+0000 7f5cd5d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac60e782f0 0x55ac61252070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.696+0000 7f5cd5d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac60e782f0 0x55ac61252070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52536/0 (socket says 172.21.15.64:52536) 2024-09-21T04:11:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.696+0000 7f5cd5d9b640 1 -- 172.21.15.64:0/1057272818 learned_addr learned my addr 172.21.15.64:0/1057272818 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.696+0000 7f5cd5d9b640 1 --2- 172.21.15.64:0/1057272818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac60ea11b0 0x55ac6125e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.696+0000 7f5cd659c640 1 --2- 172.21.15.64:0/1057272818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac60e804d0 0x55ac612525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.696+0000 7f5cd5d9b640 1 -- 172.21.15.64:0/1057272818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac60e804d0 msgr2=0x55ac612525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.696+0000 7f5cd5d9b640 1 --2- 172.21.15.64:0/1057272818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac60e804d0 0x55ac612525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.696+0000 7f5cd5d9b640 1 -- 172.21.15.64:0/1057272818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac60ea11b0 msgr2=0x55ac6125e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.696+0000 7f5cd5d9b640 1 --2- 172.21.15.64:0/1057272818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac60ea11b0 0x55ac6125e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.696+0000 7f5cd5d9b640 1 -- 172.21.15.64:0/1057272818 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac60f0c790 con 0x55ac60e782f0 2024-09-21T04:11:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.696+0000 7f5cd5d9b640 1 --2- 172.21.15.64:0/1057272818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac60e782f0 0x55ac61252070 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0010db0 tx=0x7f5cc000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2551e50eb1e34b45 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.696+0000 7f5cbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:11:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.696+0000 7f5cbdffb640 1 -- 172.21.15.64:0/1057272818 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0005520 con 0x55ac60e782f0 2024-09-21T04:11:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.696+0000 7f5ce3bc6740 1 -- 172.21.15.64:0/1057272818 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cc0012de0 con 0x55ac60e782f0 2024-09-21T04:11:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.696+0000 7f5cbdffb640 1 -- 172.21.15.64:0/1057272818 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7f5cc0004020 con 0x55ac60e782f0 2024-09-21T04:11:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.697+0000 7f5cbdffb640 1 -- 172.21.15.64:0/1057272818 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5cc0033050 con 0x55ac60e782f0 2024-09-21T04:11:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.697+0000 7f5ce3bc6740 10 client.17007 fetch_fsmap learned FSMap version 39 2024-09-21T04:11:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.697+0000 7f5ce3bc6740 1 -- 172.21.15.64:0/1057272818 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ac60f85620 con 0x55ac60e782f0 2024-09-21T04:11:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.697+0000 7f5cbdffb640 1 -- 172.21.15.64:0/1057272818 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cc0012de0 con 0x55ac60e782f0 2024-09-21T04:11:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.697+0000 7f5cbdffb640 1 -- 172.21.15.64:0/1057272818 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f5cc00035f0 con 0x55ac60e782f0 2024-09-21T04:11:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.697+0000 7f5ce3bc6740 10 client.17007 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T04:11:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.697+0000 7f5ce3bc6740 10 client.17007 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:11:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.697+0000 7f5ce3bc6740 20 client.17007 populate_metadata read hostname 'smithi064' 2024-09-21T04:11:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.697+0000 7f5ce3bc6740 1 --2- 172.21.15.64:0/1057272818 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55ac60e5fd80 0x55ac61263fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.697+0000 7f5ce3bc6740 4 client.17007 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:11:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.697+0000 7f5ce3bc6740 1 -- 172.21.15.64:0/1057272818 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac611bd640 con 0x55ac60e5fd80 2024-09-21T04:11:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.697+0000 7f5cd659c640 1 --2- 172.21.15.64:0/1057272818 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55ac60e5fd80 0x55ac61263fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.698+0000 7f5cd659c640 1 --2- 172.21.15.64:0/1057272818 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55ac60e5fd80 0x55ac61263fe0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d202f8943a127b7 server_cookie=bb490ec705b0e7d2 in_seq=0 out_seq=0 2024-09-21T04:11:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.698+0000 7f5cbdffb640 10 client.17007 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.698+0000 7f5cbdffb640 1 -- 172.21.15.64:0/1057272818 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ac611bd640 con 0x55ac60e5fd80 2024-09-21T04:11:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.698+0000 7f5cbdffb640 10 client.17007 handle_command_reply: tid=0 2024-09-21T04:11:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.699+0000 7f5ce3bc6740 1 client.17007 shutdown 2024-09-21T04:11:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.699+0000 7f5ce3bc6740 1 -- 172.21.15.64:0/1057272818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac60e782f0 msgr2=0x55ac61252070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.699+0000 7f5ce3bc6740 1 --2- 172.21.15.64:0/1057272818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac60e782f0 0x55ac61252070 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0010db0 tx=0x7f5cc000b390 comp rx=0 tx=0).stop 2024-09-21T04:11:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.699+0000 7f5ce3bc6740 1 -- 172.21.15.64:0/1057272818 shutdown_connections 2024-09-21T04:11:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.699+0000 7f5ce3bc6740 1 --2- 172.21.15.64:0/1057272818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac60ea11b0 0x55ac6125e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.699+0000 7f5ce3bc6740 1 --2- 172.21.15.64:0/1057272818 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac60e804d0 0x55ac612525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.699+0000 7f5ce3bc6740 1 --2- 172.21.15.64:0/1057272818 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55ac60e5fd80 0x55ac61263fe0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.699+0000 7f5ce3bc6740 1 --2- 172.21.15.64:0/1057272818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac60e782f0 0x55ac61252070 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.699+0000 7f5cbdffb640 0 client.17007 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.699+0000 7f5ce3bc6740 1 -- 172.21.15.64:0/1057272818 >> 172.21.15.64:0/1057272818 conn(0x55ac60e95300 msgr2=0x55ac612479e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.699+0000 7f5ce3bc6740 1 -- 172.21.15.64:0/1057272818 shutdown_connections 2024-09-21T04:11:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.699+0000 7f5ce3bc6740 1 -- 172.21.15.64:0/1057272818 wait complete. 2024-09-21T04:11:18.707 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.699+0000 7f5ce3bc6740 20 client.17007 trim_cache size 0 max 16384 2024-09-21T04:11:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.714+0000 7f5ce3bc6740 1 Processor -- start 2024-09-21T04:11:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.716+0000 7f5ce3bc6740 1 -- start start 2024-09-21T04:11:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.717+0000 7f5ce3bc6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac60e5fd80 0x55ac61253f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.717+0000 7f5ce3bc6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac60e782f0 0x55ac6125e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.717+0000 7f5ce3bc6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac60e804d0 0x55ac61247ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.717+0000 7f5ce3bc6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd81b4410 con 0x55ac60e804d0 2024-09-21T04:11:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.717+0000 7f5ce3bc6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd81b1c50 con 0x55ac60e782f0 2024-09-21T04:11:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.717+0000 7f5ce3bc6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd81afac0 con 0x55ac60e5fd80 2024-09-21T04:11:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.717+0000 7f5cd659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac60e782f0 0x55ac6125e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.717+0000 7f5cd5d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac60e5fd80 0x55ac61253f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.717+0000 7f5cd659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac60e782f0 0x55ac6125e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:43312/0 (socket says 172.21.15.64:43312) 2024-09-21T04:11:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.717+0000 7f5cd659c640 1 -- 172.21.15.64:0/1678752572 learned_addr learned my addr 172.21.15.64:0/1678752572 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.717+0000 7f5cd5d9b640 1 --2- 172.21.15.64:0/1678752572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac60e804d0 0x55ac61247ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.717+0000 7f5cd659c640 1 -- 172.21.15.64:0/1678752572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac60e5fd80 msgr2=0x55ac61253f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.717+0000 7f5cd659c640 1 --2- 172.21.15.64:0/1678752572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac60e5fd80 0x55ac61253f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.717+0000 7f5cd659c640 1 -- 172.21.15.64:0/1678752572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac60e804d0 msgr2=0x55ac61247ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.717+0000 7f5cd659c640 1 --2- 172.21.15.64:0/1678752572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac60e804d0 0x55ac61247ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.717+0000 7f5cd659c640 1 -- 172.21.15.64:0/1678752572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac612451a0 con 0x55ac60e782f0 2024-09-21T04:11:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.718+0000 7f5cd5d9b640 1 --2- 172.21.15.64:0/1678752572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac60e804d0 0x55ac61247ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:11:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.718+0000 7f5cd659c640 1 --2- 172.21.15.64:0/1678752572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac60e782f0 0x55ac6125e320 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8005e30 tx=0x7f5cc8009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67839c75c51d0ce3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.718+0000 7f5cbffff640 1 -- 172.21.15.64:0/1678752572 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8031650 con 0x55ac60e782f0 2024-09-21T04:11:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.718+0000 7f5cbffff640 1 -- 172.21.15.64:0/1678752572 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cc8036040 con 0x55ac60e782f0 2024-09-21T04:11:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.718+0000 7f5cbffff640 1 -- 172.21.15.64:0/1678752572 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8004430 con 0x55ac60e782f0 2024-09-21T04:11:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.719+0000 7f5ce3bc6740 1 -- 172.21.15.64:0/1678752572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac60e782f0 msgr2=0x55ac6125e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.719+0000 7f5ce3bc6740 1 --2- 172.21.15.64:0/1678752572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac60e782f0 0x55ac6125e320 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8005e30 tx=0x7f5cc8009360 comp rx=0 tx=0).stop 2024-09-21T04:11:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.720+0000 7f5ce3bc6740 1 -- 172.21.15.64:0/1678752572 shutdown_connections 2024-09-21T04:11:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.720+0000 7f5ce3bc6740 1 --2- 172.21.15.64:0/1678752572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac60e804d0 0x55ac61247ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.720+0000 7f5ce3bc6740 1 --2- 172.21.15.64:0/1678752572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac60e782f0 0x55ac6125e320 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.720+0000 7f5ce3bc6740 1 --2- 172.21.15.64:0/1678752572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac60e5fd80 0x55ac61253f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.720+0000 7f5ce3bc6740 1 -- 172.21.15.64:0/1678752572 >> 172.21.15.64:0/1678752572 conn(0x55ac60e95300 msgr2=0x55ac61247790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.720+0000 7f5ce3bc6740 1 -- 172.21.15.64:0/1678752572 shutdown_connections 2024-09-21T04:11:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.720+0000 7f5ce3bc6740 1 -- 172.21.15.64:0/1678752572 wait complete. 2024-09-21T04:11:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.720+0000 7f5ce3bc6740 1 -- start start 2024-09-21T04:11:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.721+0000 7f5ce3bc6740 1 Processor -- start 2024-09-21T04:11:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.721+0000 7f5ce3bc6740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac60e5fd80 0x55ac61257790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.721+0000 7f5ce3bc6740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac60e782f0 0x55ac61257d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:18.725 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.721+0000 7f5ce3bc6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac60e804d0 0x55ac612d5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.721+0000 7f5ce3bc6740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd81b0110 con 0x55ac60e5fd80 2024-09-21T04:11:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.721+0000 7f5ce3bc6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd81b1580 con 0x55ac60e804d0 2024-09-21T04:11:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.721+0000 7f5ce3bc6740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd81b07e0 con 0x55ac60e782f0 2024-09-21T04:11:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.721+0000 7f5ce3bc6740 10 client.0 _ll_register_callbacks cb 0x7f5cc8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:11:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.721+0000 7f5cd5d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac60e5fd80 0x55ac61257790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.721+0000 7f5cd5d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac60e5fd80 0x55ac61257790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:52556/0 (socket says 172.21.15.64:52556) 2024-09-21T04:11:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.721+0000 7f5cd5d9b640 1 -- 172.21.15.64:0/4051035240 learned_addr learned my addr 172.21.15.64:0/4051035240 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.722+0000 7f5cd5d9b640 1 -- 172.21.15.64:0/4051035240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac60e782f0 msgr2=0x55ac61257d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:11:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.722+0000 7f5cd5d9b640 1 --2- 172.21.15.64:0/4051035240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac60e782f0 0x55ac61257d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.722+0000 7f5cd5d9b640 1 -- 172.21.15.64:0/4051035240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac60e804d0 msgr2=0x55ac612d5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.722+0000 7f5cd5d9b640 1 --2- 172.21.15.64:0/4051035240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac60e804d0 0x55ac612d5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.722+0000 7f5cd5d9b640 1 -- 172.21.15.64:0/4051035240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac612d5560 con 0x55ac60e5fd80 2024-09-21T04:11:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.722+0000 7f5cd5d9b640 1 --2- 172.21.15.64:0/4051035240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac60e5fd80 0x55ac61257790 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000c010 tx=0x7f5cc0012470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898d38b106d74ecd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.722+0000 7f5cbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:11:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.722+0000 7f5cbdffb640 1 -- 172.21.15.64:0/4051035240 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0003800 con 0x55ac60e5fd80 2024-09-21T04:11:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.723+0000 7f5cbdffb640 1 -- 172.21.15.64:0/4051035240 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7f5cc0003be0 con 0x55ac60e5fd80 2024-09-21T04:11:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.723+0000 7f5ce3bc6740 1 -- 172.21.15.64:0/4051035240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cc8036040 con 0x55ac60e5fd80 2024-09-21T04:11:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.723+0000 7f5cbdffb640 1 -- 172.21.15.64:0/4051035240 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5cc0010bd0 con 0x55ac60e5fd80 2024-09-21T04:11:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.723+0000 7f5ce3bc6740 10 client.17013 fetch_fsmap learned FSMap version 39 2024-09-21T04:11:18.727 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.723+0000 7f5ce3bc6740 1 -- 172.21.15.64:0/4051035240 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ac612d66f0 con 0x55ac60e5fd80 2024-09-21T04:11:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.723+0000 7f5cbdffb640 1 -- 172.21.15.64:0/4051035240 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cc8036040 con 0x55ac60e5fd80 2024-09-21T04:11:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.724+0000 7f5cbdffb640 1 -- 172.21.15.64:0/4051035240 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f5cc00046f0 con 0x55ac60e5fd80 2024-09-21T04:11:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.724+0000 7f5ce3bc6740 10 client.17013 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T04:11:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.724+0000 7f5ce3bc6740 10 client.17013 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:11:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.724+0000 7f5ce3bc6740 20 client.17013 populate_metadata read hostname 'smithi064' 2024-09-21T04:11:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.724+0000 7f5ce3bc6740 1 --2- 172.21.15.64:0/4051035240 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55ac60ea11b0 0x55ac612d8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.724+0000 7f5ce3bc6740 4 client.17013 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:11:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.724+0000 7f5ce3bc6740 1 -- 172.21.15.64:0/4051035240 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac611b90f0 con 0x55ac60ea11b0 2024-09-21T04:11:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.724+0000 7f5cd659c640 1 --2- 172.21.15.64:0/4051035240 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55ac60ea11b0 0x55ac612d8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.725+0000 7f5cbdffb640 10 client.17013 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:18.729 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.725+0000 7f5cd659c640 1 --2- 172.21.15.64:0/4051035240 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55ac60ea11b0 0x55ac612d8e20 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93af80ffffc0ad08 server_cookie=177e7886823c0421 in_seq=0 out_seq=0 2024-09-21T04:11:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.767+0000 7f5cbdffb640 1 -- 172.21.15.64:0/4051035240 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ac611b90f0 con 0x55ac60ea11b0 2024-09-21T04:11:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.767+0000 7f5cbdffb640 10 client.17013 handle_command_reply: tid=0 2024-09-21T04:11:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.768+0000 7f5ce3bc6740 1 client.17013 shutdown 2024-09-21T04:11:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.768+0000 7f5ce3bc6740 1 -- 172.21.15.64:0/4051035240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac60e5fd80 msgr2=0x55ac61257790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.768+0000 7f5ce3bc6740 1 --2- 172.21.15.64:0/4051035240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac60e5fd80 0x55ac61257790 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000c010 tx=0x7f5cc0012470 comp rx=0 tx=0).stop 2024-09-21T04:11:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.768+0000 7f5ce3bc6740 1 -- 172.21.15.64:0/4051035240 shutdown_connections 2024-09-21T04:11:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.768+0000 7f5ce3bc6740 1 --2- 172.21.15.64:0/4051035240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac60e804d0 0x55ac612d5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.768+0000 7f5ce3bc6740 1 --2- 172.21.15.64:0/4051035240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac60e782f0 0x55ac61257d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.768+0000 7f5ce3bc6740 1 --2- 172.21.15.64:0/4051035240 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55ac60ea11b0 0x55ac612d8e20 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.768+0000 7f5cbdffb640 0 client.17013 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.768+0000 7f5ce3bc6740 1 --2- 172.21.15.64:0/4051035240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac60e5fd80 0x55ac61257790 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.768+0000 7f5ce3bc6740 1 -- 172.21.15.64:0/4051035240 >> 172.21.15.64:0/4051035240 conn(0x55ac60e95300 msgr2=0x55ac61248150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.768+0000 7f5ce3bc6740 1 -- 172.21.15.64:0/4051035240 shutdown_connections 2024-09-21T04:11:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.768+0000 7f5ce3bc6740 1 -- 172.21.15.64:0/4051035240 wait complete. 2024-09-21T04:11:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.768+0000 7f5ce3bc6740 20 client.17013 trim_cache size 0 max 16384 2024-09-21T04:11:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:11:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:11:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:11:18.773 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:11:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.769+0000 7f5cdcc4d640 1 -- 172.21.15.64:0/4173847288 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5cc40b6730 msgr2=0x7f5cc40b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.769+0000 7f5cdcc4d640 1 --2- 172.21.15.64:0/4173847288 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5cc40b6730 0x7f5cc40b8c40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5cd00e5270 tx=0x7f5cd00049b0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.769+0000 7f5cdcc4d640 1 -- 172.21.15.64:0/4173847288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cd8108c60 msgr2=0x7f5cd80ff390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.769+0000 7f5cdcc4d640 1 --2- 172.21.15.64:0/4173847288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cd8108c60 0x7f5cd80ff390 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000e970 tx=0x7f5cd000ed70 comp rx=0 tx=0).stop 2024-09-21T04:11:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.769+0000 7f5cdcc4d640 1 -- 172.21.15.64:0/4173847288 shutdown_connections 2024-09-21T04:11:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.769+0000 7f5cdcc4d640 1 --2- 172.21.15.64:0/4173847288 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5cc40b6730 0x7f5cc40b8c40 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.769+0000 7f5cdcc4d640 1 --2- 172.21.15.64:0/4173847288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cd8108c60 0x7f5cd80ff390 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.769+0000 7f5cdcc4d640 1 --2- 172.21.15.64:0/4173847288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cd8108310 0x7f5cd80fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.769+0000 7f5cdcc4d640 1 --2- 172.21.15.64:0/4173847288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5cd81029e0 0x7f5cd80fe910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.769+0000 7f5cdcc4d640 1 -- 172.21.15.64:0/4173847288 >> 172.21.15.64:0/4173847288 conn(0x7f5cd80fe3e0 msgr2=0x7f5cd806bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.769+0000 7f5cdcc4d640 1 -- 172.21.15.64:0/4173847288 shutdown_connections 2024-09-21T04:11:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:18.769+0000 7f5cdcc4d640 1 -- 172.21.15.64:0/4173847288 wait complete. 2024-09-21T04:11:19.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:19 smithi131 ceph-mon[35193]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:11:19.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:19 smithi119 ceph-mon[34811]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:11:19.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:19 smithi064 ceph-mon[31180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:11:20.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:20 smithi131 ceph-mon[35193]: pgmap v544: 97 pgs: 97 active+clean; 803 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 573 op/s 2024-09-21T04:11:20.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:20 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:11:20.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:20 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:20.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:20 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:20.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:20 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:20.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:20 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:20.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:20 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:20.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:20 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:20.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:20 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:20.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:20 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:20.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:20 smithi119 ceph-mon[34811]: pgmap v544: 97 pgs: 97 active+clean; 803 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 573 op/s 2024-09-21T04:11:20.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:20 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:11:20.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:20 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:20.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:20 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:20.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:20 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:20.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:20 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:20.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:20 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:20.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:20 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:20.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:20 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:20.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:20 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:20.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:20 smithi064 ceph-mon[31180]: pgmap v544: 97 pgs: 97 active+clean; 803 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 573 op/s 2024-09-21T04:11:20.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:20 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:11:20.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:20 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:20.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:20 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:20.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:20 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:20.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:20 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:20.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:20 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:20.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:20 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:20.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:20 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:20.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:20 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:21 smithi131 ceph-mon[35193]: Adjusting osd_memory_target on smithi131 to 176.0M 2024-09-21T04:11:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:21 smithi131 ceph-mon[35193]: Adjusting osd_memory_target on smithi064 to 176.0M 2024-09-21T04:11:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:21 smithi131 ceph-mon[35193]: Unable to set osd_memory_target on smithi131 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-21T04:11:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:21 smithi131 ceph-mon[35193]: Unable to set osd_memory_target on smithi064 to 184586752: error parsing value: Value '184586752' is below minimum 939524096 2024-09-21T04:11:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:21 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:21 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:21 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:21 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:21 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:11:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:21 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:11:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:21 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:11:21.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:21 smithi119 ceph-mon[34811]: Adjusting osd_memory_target on smithi131 to 176.0M 2024-09-21T04:11:21.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:21 smithi119 ceph-mon[34811]: Adjusting osd_memory_target on smithi064 to 176.0M 2024-09-21T04:11:21.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:21 smithi119 ceph-mon[34811]: Unable to set osd_memory_target on smithi131 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-21T04:11:21.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:21 smithi119 ceph-mon[34811]: Unable to set osd_memory_target on smithi064 to 184586752: error parsing value: Value '184586752' is below minimum 939524096 2024-09-21T04:11:21.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:21 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:21.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:21 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:21.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:21 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:21.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:21 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:21.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:21 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:11:21.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:21 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:11:21.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:21 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:11:21.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:21 smithi064 ceph-mon[31180]: Adjusting osd_memory_target on smithi131 to 176.0M 2024-09-21T04:11:21.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:21 smithi064 ceph-mon[31180]: Adjusting osd_memory_target on smithi064 to 176.0M 2024-09-21T04:11:21.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:21 smithi064 ceph-mon[31180]: Unable to set osd_memory_target on smithi131 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-21T04:11:21.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:21 smithi064 ceph-mon[31180]: Unable to set osd_memory_target on smithi064 to 184586752: error parsing value: Value '184586752' is below minimum 939524096 2024-09-21T04:11:21.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:21 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:21.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:21 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:21.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:21 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:21.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:21 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T04:11:21.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:21 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:11:21.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:21 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:11:21.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:21 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:11:22.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:22 smithi131 ceph-mon[35193]: Adjusting osd_memory_target on smithi119 to 176.0M 2024-09-21T04:11:22.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:22 smithi131 ceph-mon[35193]: Unable to set osd_memory_target on smithi119 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-21T04:11:22.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:22 smithi131 ceph-mon[35193]: pgmap v545: 97 pgs: 97 active+clean; 803 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 572 op/s 2024-09-21T04:11:22.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:22 smithi119 ceph-mon[34811]: Adjusting osd_memory_target on smithi119 to 176.0M 2024-09-21T04:11:22.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:22 smithi119 ceph-mon[34811]: Unable to set osd_memory_target on smithi119 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-21T04:11:22.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:22 smithi119 ceph-mon[34811]: pgmap v545: 97 pgs: 97 active+clean; 803 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 572 op/s 2024-09-21T04:11:22.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:22 smithi064 ceph-mon[31180]: Adjusting osd_memory_target on smithi119 to 176.0M 2024-09-21T04:11:22.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:22 smithi064 ceph-mon[31180]: Unable to set osd_memory_target on smithi119 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-21T04:11:22.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:22 smithi064 ceph-mon[31180]: pgmap v545: 97 pgs: 97 active+clean; 803 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 572 op/s 2024-09-21T04:11:23.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:23 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:11:23.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:23 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:11:23.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:23 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:11:23.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:23 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:11:23.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:23 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:11:23.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:23 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:11:23.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:23 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:11:23.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:23 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:11:23.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:23 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:11:24.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:24 smithi131 ceph-mon[35193]: pgmap v546: 97 pgs: 97 active+clean; 825 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 474 op/s 2024-09-21T04:11:24.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:24 smithi119 ceph-mon[34811]: pgmap v546: 97 pgs: 97 active+clean; 825 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 474 op/s 2024-09-21T04:11:24.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:24 smithi064 ceph-mon[31180]: pgmap v546: 97 pgs: 97 active+clean; 825 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 474 op/s 2024-09-21T04:11:26.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:26 smithi131 ceph-mon[35193]: pgmap v547: 97 pgs: 97 active+clean; 825 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 474 op/s 2024-09-21T04:11:26.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:26 smithi119 ceph-mon[34811]: pgmap v547: 97 pgs: 97 active+clean; 825 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 474 op/s 2024-09-21T04:11:26.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:26 smithi064 ceph-mon[31180]: pgmap v547: 97 pgs: 97 active+clean; 825 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 474 op/s 2024-09-21T04:11:28.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:28 smithi131 ceph-mon[35193]: pgmap v548: 97 pgs: 97 active+clean; 811 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 455 op/s 2024-09-21T04:11:28.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:28 smithi119 ceph-mon[34811]: pgmap v548: 97 pgs: 97 active+clean; 811 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 455 op/s 2024-09-21T04:11:28.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:28 smithi064 ceph-mon[31180]: pgmap v548: 97 pgs: 97 active+clean; 811 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 455 op/s 2024-09-21T04:11:30.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:30 smithi131 ceph-mon[35193]: pgmap v549: 97 pgs: 97 active+clean; 803 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 421 op/s 2024-09-21T04:11:30.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:30 smithi119 ceph-mon[34811]: pgmap v549: 97 pgs: 97 active+clean; 803 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 421 op/s 2024-09-21T04:11:30.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:30 smithi064 ceph-mon[31180]: pgmap v549: 97 pgs: 97 active+clean; 803 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 421 op/s 2024-09-21T04:11:30.903 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T04:11:31.219 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:11:31.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.877+0000 7f6947fa8640 1 -- 172.21.15.64:0/1865384820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f694010dd90 msgr2=0x7f6940110280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:31.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.877+0000 7f6947fa8640 1 --2- 172.21.15.64:0/1865384820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f694010dd90 0x7f6940110280 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6938009ca0 tx=0x7f693802a8c0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.877+0000 7f6947fa8640 1 -- 172.21.15.64:0/1865384820 shutdown_connections 2024-09-21T04:11:31.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.877+0000 7f6947fa8640 1 --2- 172.21.15.64:0/1865384820 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f694010dd90 0x7f6940110280 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.880 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.877+0000 7f6947fa8640 1 --2- 172.21.15.64:0/1865384820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6940105280 0x7f694010d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.877+0000 7f6947fa8640 1 --2- 172.21.15.64:0/1865384820 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6940104930 0x7f6940104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.877+0000 7f6947fa8640 1 -- 172.21.15.64:0/1865384820 >> 172.21.15.64:0/1865384820 conn(0x7f69400fe3c0 msgr2=0x7f6940100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:31.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.877+0000 7f6947fa8640 1 -- 172.21.15.64:0/1865384820 shutdown_connections 2024-09-21T04:11:31.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.877+0000 7f6947fa8640 1 -- 172.21.15.64:0/1865384820 wait complete. 2024-09-21T04:11:31.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.878+0000 7f6947fa8640 1 Processor -- start 2024-09-21T04:11:31.881 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.878+0000 7f6947fa8640 1 -- start start 2024-09-21T04:11:31.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.879+0000 7f6947fa8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6940104930 0x7f694019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.879+0000 7f6947fa8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6940105280 0x7f694019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.883 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.879+0000 7f6947fa8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f694010dd90 0x7f694019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.879+0000 7f6947fa8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f69401139f0 con 0x7f6940104930 2024-09-21T04:11:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.879+0000 7f6947fa8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6940113870 con 0x7f694010dd90 2024-09-21T04:11:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.879+0000 7f6947fa8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6940113b70 con 0x7f6940105280 2024-09-21T04:11:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.879+0000 7f6945d44640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6940104930 0x7f694019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.879+0000 7f6945d44640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6940104930 0x7f694019a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46504/0 (socket says 172.21.15.64:46504) 2024-09-21T04:11:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.879+0000 7f6945d44640 1 -- 172.21.15.64:0/3483573925 learned_addr learned my addr 172.21.15.64:0/3483573925 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.879+0000 7f6946545640 1 --2- 172.21.15.64:0/3483573925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f694010dd90 0x7f694019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.879+0000 7f6945543640 1 --2- 172.21.15.64:0/3483573925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6940105280 0x7f694019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.880+0000 7f6945d44640 1 -- 172.21.15.64:0/3483573925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6940105280 msgr2=0x7f694019a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.880+0000 7f6945d44640 1 --2- 172.21.15.64:0/3483573925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6940105280 0x7f694019a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.880+0000 7f6945d44640 1 -- 172.21.15.64:0/3483573925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f694010dd90 msgr2=0x7f694019fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.880+0000 7f6945d44640 1 --2- 172.21.15.64:0/3483573925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f694010dd90 0x7f694019fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.880+0000 7f6945d44640 1 -- 172.21.15.64:0/3483573925 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69401a02e0 con 0x7f6940104930 2024-09-21T04:11:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.880+0000 7f6946545640 1 --2- 172.21.15.64:0/3483573925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f694010dd90 0x7f694019fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:11:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.880+0000 7f6945d44640 1 --2- 172.21.15.64:0/3483573925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6940104930 0x7f694019a3b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f693c00e9f0 tx=0x7f693c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.880+0000 7f6936ffd640 1 -- 172.21.15.64:0/3483573925 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c00cd80 con 0x7f6940104930 2024-09-21T04:11:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.880+0000 7f6936ffd640 1 -- 172.21.15.64:0/3483573925 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f693c0108e0 con 0x7f6940104930 2024-09-21T04:11:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.880+0000 7f6947fa8640 1 -- 172.21.15.64:0/3483573925 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69401a05d0 con 0x7f6940104930 2024-09-21T04:11:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.880+0000 7f6936ffd640 1 -- 172.21.15.64:0/3483573925 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c018980 con 0x7f6940104930 2024-09-21T04:11:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.880+0000 7f6947fa8640 1 -- 172.21.15.64:0/3483573925 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6940102540 con 0x7f6940104930 2024-09-21T04:11:31.885 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.883+0000 7f694efee740 1 Processor -- start 2024-09-21T04:11:31.886 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.883+0000 7f694efee740 1 -- start start 2024-09-21T04:11:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.883+0000 7f694efee740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7970e91b0 0x55b79748e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.883+0000 7f694efee740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7970c02f0 0x55b79748eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.883+0000 7f694efee740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7970c84d0 0x55b79748f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.887 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.883+0000 7f694efee740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b7974036f0 con 0x55b7970c02f0 2024-09-21T04:11:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.883+0000 7f694efee740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b7973f2b20 con 0x55b7970e91b0 2024-09-21T04:11:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.883+0000 7f694efee740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b7973f6930 con 0x55b7970c84d0 2024-09-21T04:11:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.884+0000 7f6946545640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7970e91b0 0x55b79748e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.884+0000 7f6946545640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7970e91b0 0x55b79748e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:55736/0 (socket says 172.21.15.64:55736) 2024-09-21T04:11:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.884+0000 7f6946545640 1 -- 172.21.15.64:0/394504229 learned_addr learned my addr 172.21.15.64:0/394504229 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.884+0000 7f6945543640 1 --2- 172.21.15.64:0/394504229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7970c84d0 0x55b79748f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.884+0000 7f6946545640 1 -- 172.21.15.64:0/394504229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7970c84d0 msgr2=0x55b79748f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.884+0000 7f6946545640 1 --2- 172.21.15.64:0/394504229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7970c84d0 0x55b79748f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.884+0000 7f6946545640 1 -- 172.21.15.64:0/394504229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7970c02f0 msgr2=0x55b79748eef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:11:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.884+0000 7f6946545640 1 --2- 172.21.15.64:0/394504229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7970c02f0 0x55b79748eef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.884+0000 7f6946545640 1 -- 172.21.15.64:0/394504229 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7973b8530 con 0x55b7970e91b0 2024-09-21T04:11:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.884+0000 7f6946545640 1 --2- 172.21.15.64:0/394504229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7970e91b0 0x55b79748e9b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f69380094c0 tx=0x7f693802f310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41db397337c5be64 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.884+0000 7f692bfff640 1 -- 172.21.15.64:0/394504229 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6938031650 con 0x55b7970e91b0 2024-09-21T04:11:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.885+0000 7f692bfff640 1 -- 172.21.15.64:0/394504229 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6938036040 con 0x55b7970e91b0 2024-09-21T04:11:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.885+0000 7f692bfff640 1 -- 172.21.15.64:0/394504229 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6938004460 con 0x55b7970e91b0 2024-09-21T04:11:31.889 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.885+0000 7f6936ffd640 1 -- 172.21.15.64:0/3483573925 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f693c010430 con 0x7f6940104930 2024-09-21T04:11:31.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.886+0000 7f6936ffd640 1 --2- 172.21.15.64:0/3483573925 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f692c0b6700 0x7f692c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.886+0000 7f6945d44640 1 --2- 172.21.15.64:0/3483573925 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f692c0b6700 0x7f692c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:31.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.886+0000 7f694efee740 1 -- 172.21.15.64:0/394504229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7970e91b0 msgr2=0x55b79748e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:31.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.886+0000 7f694efee740 1 --2- 172.21.15.64:0/394504229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7970e91b0 0x55b79748e9b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f69380094c0 tx=0x7f693802f310 comp rx=0 tx=0).stop 2024-09-21T04:11:31.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.886+0000 7f6936ffd640 1 -- 172.21.15.64:0/3483573925 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7f693c0e7a20 con 0x7f6940104930 2024-09-21T04:11:31.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.886+0000 7f694efee740 1 -- 172.21.15.64:0/394504229 shutdown_connections 2024-09-21T04:11:31.890 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.886+0000 7f694efee740 1 --2- 172.21.15.64:0/394504229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7970c84d0 0x55b79748f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.886+0000 7f694efee740 1 --2- 172.21.15.64:0/394504229 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7970c02f0 0x55b79748eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.886+0000 7f694efee740 1 --2- 172.21.15.64:0/394504229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7970e91b0 0x55b79748e9b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.886+0000 7f694efee740 1 -- 172.21.15.64:0/394504229 >> 172.21.15.64:0/394504229 conn(0x55b7970dd300 msgr2=0x55b79748c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:31.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.886+0000 7f694efee740 1 -- 172.21.15.64:0/394504229 shutdown_connections 2024-09-21T04:11:31.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.886+0000 7f6945d44640 1 --2- 172.21.15.64:0/3483573925 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f692c0b6700 0x7f692c0b8c10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x55b79748d630 tx=0x7f693c005760 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:31.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.886+0000 7f694efee740 1 -- 172.21.15.64:0/394504229 wait complete. 2024-09-21T04:11:31.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.887+0000 7f694efee740 1 -- start start 2024-09-21T04:11:31.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.887+0000 7f694efee740 1 Processor -- start 2024-09-21T04:11:31.891 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.888+0000 7f694efee740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7970c02f0 0x55b79749a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.888+0000 7f694efee740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7970c84d0 0x55b79749a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.888+0000 7f694efee740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7970e91b0 0x55b7974a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.888+0000 7f694efee740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b79732aab0 con 0x55b7970e91b0 2024-09-21T04:11:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.888+0000 7f694efee740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b797398730 con 0x55b7970c02f0 2024-09-21T04:11:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.888+0000 7f694efee740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b79749cf50 con 0x55b7970c84d0 2024-09-21T04:11:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.888+0000 7f694efee740 10 client.0 _ll_register_callbacks cb 0x55b7973ec6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:11:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.888+0000 7f6946545640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7970c02f0 0x55b79749a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.888+0000 7f6946545640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7970c02f0 0x55b79749a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:55744/0 (socket says 172.21.15.64:55744) 2024-09-21T04:11:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.888+0000 7f6946545640 1 -- 172.21.15.64:0/1657021834 learned_addr learned my addr 172.21.15.64:0/1657021834 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.888+0000 7f6946545640 1 --2- 172.21.15.64:0/1657021834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7970e91b0 0x55b7974a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.888+0000 7f6945543640 1 --2- 172.21.15.64:0/1657021834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7970c84d0 0x55b79749a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.888+0000 7f6946545640 1 -- 172.21.15.64:0/1657021834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7970c84d0 msgr2=0x55b79749a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.888+0000 7f6946545640 1 --2- 172.21.15.64:0/1657021834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7970c84d0 0x55b79749a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.888+0000 7f6946545640 1 -- 172.21.15.64:0/1657021834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7970e91b0 msgr2=0x55b7974a62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.888+0000 7f6946545640 1 --2- 172.21.15.64:0/1657021834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7970e91b0 0x55b7974a62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.888+0000 7f6946545640 1 -- 172.21.15.64:0/1657021834 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b797154790 con 0x55b7970c02f0 2024-09-21T04:11:31.895 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.889+0000 7f6946545640 1 --2- 172.21.15.64:0/1657021834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7970c02f0 0x55b79749a040 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6938009620 tx=0x7f69380040e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0b7edb3d3d0e68e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.889+0000 7f69297fa640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:11:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.889+0000 7f69297fa640 1 -- 172.21.15.64:0/1657021834 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6938003d40 con 0x55b7970c02f0 2024-09-21T04:11:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.889+0000 7f69297fa640 1 -- 172.21.15.64:0/1657021834 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7f6938004240 con 0x55b7970c02f0 2024-09-21T04:11:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.889+0000 7f694efee740 1 -- 172.21.15.64:0/1657021834 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6938036040 con 0x55b7970c02f0 2024-09-21T04:11:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.889+0000 7f69297fa640 1 -- 172.21.15.64:0/1657021834 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6938004b40 con 0x55b7970c02f0 2024-09-21T04:11:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.889+0000 7f694efee740 10 client.26696 fetch_fsmap learned FSMap version 39 2024-09-21T04:11:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.889+0000 7f694efee740 1 -- 172.21.15.64:0/1657021834 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b7971cd620 con 0x55b7970c02f0 2024-09-21T04:11:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.889+0000 7f69297fa640 1 -- 172.21.15.64:0/1657021834 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6938036040 con 0x55b7970c02f0 2024-09-21T04:11:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.890+0000 7f69297fa640 1 -- 172.21.15.64:0/1657021834 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f6938051040 con 0x55b7970c02f0 2024-09-21T04:11:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.890+0000 7f694efee740 10 client.26696 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T04:11:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.890+0000 7f694efee740 10 client.26696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:11:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.890+0000 7f694efee740 20 client.26696 populate_metadata read hostname 'smithi064' 2024-09-21T04:11:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.890+0000 7f694efee740 1 --2- 172.21.15.64:0/1657021834 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b7970a7d80 0x55b7974abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.890+0000 7f694efee740 4 client.26696 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:11:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.890+0000 7f694efee740 1 -- 172.21.15.64:0/1657021834 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b797405640 con 0x55b7970a7d80 2024-09-21T04:11:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.890+0000 7f6945543640 1 --2- 172.21.15.64:0/1657021834 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b7970a7d80 0x55b7974abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.890+0000 7f69297fa640 10 client.26696 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.890+0000 7f6945543640 1 --2- 172.21.15.64:0/1657021834 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b7970a7d80 0x55b7974abdf0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a375afbd78f57a23 server_cookie=90bd05aa108c0126 in_seq=0 out_seq=0 2024-09-21T04:11:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.891+0000 7f69297fa640 1 -- 172.21.15.64:0/1657021834 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b797405640 con 0x55b7970a7d80 2024-09-21T04:11:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.891+0000 7f69297fa640 10 client.26696 handle_command_reply: tid=0 2024-09-21T04:11:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.891+0000 7f694efee740 1 client.26696 shutdown 2024-09-21T04:11:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.891+0000 7f694efee740 1 -- 172.21.15.64:0/1657021834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7970c02f0 msgr2=0x55b79749a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.891+0000 7f694efee740 1 --2- 172.21.15.64:0/1657021834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7970c02f0 0x55b79749a040 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6938009620 tx=0x7f69380040e0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.891+0000 7f694efee740 1 -- 172.21.15.64:0/1657021834 shutdown_connections 2024-09-21T04:11:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.891+0000 7f694efee740 1 --2- 172.21.15.64:0/1657021834 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b7970a7d80 0x55b7974abdf0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.891+0000 7f694efee740 1 --2- 172.21.15.64:0/1657021834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7970e91b0 0x55b7974a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.891+0000 7f694efee740 1 --2- 172.21.15.64:0/1657021834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7970c84d0 0x55b79749a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.897 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.891+0000 7f69297fa640 0 client.26696 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.891+0000 7f694efee740 1 --2- 172.21.15.64:0/1657021834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7970c02f0 0x55b79749a040 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.891+0000 7f694efee740 1 -- 172.21.15.64:0/1657021834 >> 172.21.15.64:0/1657021834 conn(0x55b7970dd300 msgr2=0x55b79748f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.892+0000 7f694efee740 1 -- 172.21.15.64:0/1657021834 shutdown_connections 2024-09-21T04:11:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.892+0000 7f694efee740 1 -- 172.21.15.64:0/1657021834 wait complete. 2024-09-21T04:11:31.898 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.892+0000 7f694efee740 20 client.26696 trim_cache size 0 max 16384 2024-09-21T04:11:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.924+0000 7f694efee740 1 Processor -- start 2024-09-21T04:11:31.927 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.924+0000 7f694efee740 1 -- start start 2024-09-21T04:11:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.925+0000 7f694efee740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7970a7d80 0x55b7974900f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.925+0000 7f694efee740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7970c02f0 0x55b7974a61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.925+0000 7f694efee740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7970c84d0 0x55b797497210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.925+0000 7f694efee740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f69401b43d0 con 0x55b7970a7d80 2024-09-21T04:11:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.925+0000 7f694efee740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f69401b1c10 con 0x55b7970c84d0 2024-09-21T04:11:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.925+0000 7f694efee740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f69401afa80 con 0x55b7970c02f0 2024-09-21T04:11:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.925+0000 7f6946545640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7970a7d80 0x55b7974900f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:31.928 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.925+0000 7f6946545640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7970a7d80 0x55b7974900f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46514/0 (socket says 172.21.15.64:46514) 2024-09-21T04:11:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.925+0000 7f6946545640 1 -- 172.21.15.64:0/3118396064 learned_addr learned my addr 172.21.15.64:0/3118396064 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.925+0000 7f6946545640 1 --2- 172.21.15.64:0/3118396064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7970c84d0 0x55b797497210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:31.929 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.925+0000 7f6945543640 1 --2- 172.21.15.64:0/3118396064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7970c02f0 0x55b7974a61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.926+0000 7f6946545640 1 -- 172.21.15.64:0/3118396064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7970c02f0 msgr2=0x55b7974a61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.926+0000 7f6946545640 1 --2- 172.21.15.64:0/3118396064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7970c02f0 0x55b7974a61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.926+0000 7f6946545640 1 -- 172.21.15.64:0/3118396064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7970c84d0 msgr2=0x55b797497210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.926+0000 7f6946545640 1 --2- 172.21.15.64:0/3118396064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7970c84d0 0x55b797497210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.926+0000 7f6946545640 1 -- 172.21.15.64:0/3118396064 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b797498520 con 0x55b7970a7d80 2024-09-21T04:11:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.926+0000 7f6945543640 1 --2- 172.21.15.64:0/3118396064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7970c02f0 0x55b7974a61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:11:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.926+0000 7f6946545640 1 --2- 172.21.15.64:0/3118396064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7970a7d80 0x55b7974900f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6938005e00 tx=0x7f6938005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa6632958d7437c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.927+0000 7f692b7fe640 1 -- 172.21.15.64:0/3118396064 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6938003970 con 0x55b7970a7d80 2024-09-21T04:11:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.927+0000 7f692b7fe640 1 -- 172.21.15.64:0/3118396064 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6938036070 con 0x55b7970a7d80 2024-09-21T04:11:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.927+0000 7f692b7fe640 1 -- 172.21.15.64:0/3118396064 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6938004200 con 0x55b7970a7d80 2024-09-21T04:11:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.928+0000 7f694efee740 1 -- 172.21.15.64:0/3118396064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7970a7d80 msgr2=0x55b7974900f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.928+0000 7f694efee740 1 --2- 172.21.15.64:0/3118396064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7970a7d80 0x55b7974900f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6938005e00 tx=0x7f6938005fd0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.928+0000 7f694efee740 1 -- 172.21.15.64:0/3118396064 shutdown_connections 2024-09-21T04:11:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.928+0000 7f694efee740 1 --2- 172.21.15.64:0/3118396064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7970c84d0 0x55b797497210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.928+0000 7f694efee740 1 --2- 172.21.15.64:0/3118396064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7970c02f0 0x55b7974a61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.928+0000 7f694efee740 1 --2- 172.21.15.64:0/3118396064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7970a7d80 0x55b7974900f0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.928+0000 7f694efee740 1 -- 172.21.15.64:0/3118396064 >> 172.21.15.64:0/3118396064 conn(0x55b7970dd300 msgr2=0x55b79748f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.928+0000 7f694efee740 1 -- 172.21.15.64:0/3118396064 shutdown_connections 2024-09-21T04:11:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.928+0000 7f694efee740 1 -- 172.21.15.64:0/3118396064 wait complete. 2024-09-21T04:11:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.929+0000 7f694efee740 1 -- start start 2024-09-21T04:11:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.929+0000 7f694efee740 1 Processor -- start 2024-09-21T04:11:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.930+0000 7f694efee740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7970a7d80 0x55b79749f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.930+0000 7f694efee740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7970c02f0 0x55b79749fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.930+0000 7f694efee740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7970c84d0 0x55b7975146e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.930+0000 7f694efee740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f69401b00d0 con 0x55b7970c02f0 2024-09-21T04:11:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.930+0000 7f694efee740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f69401b1540 con 0x55b7970c84d0 2024-09-21T04:11:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.930+0000 7f694efee740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f69401b07a0 con 0x55b7970a7d80 2024-09-21T04:11:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.930+0000 7f694efee740 10 client.0 _ll_register_callbacks cb 0x7f69300097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:11:31.933 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.930+0000 7f6946545640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7970a7d80 0x55b79749f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.930+0000 7f6945543640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7970c02f0 0x55b79749fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.930+0000 7f6946545640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7970a7d80 0x55b79749f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:46888/0 (socket says 172.21.15.64:46888) 2024-09-21T04:11:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.930+0000 7f6946545640 1 -- 172.21.15.64:0/809643195 learned_addr learned my addr 172.21.15.64:0/809643195 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.930+0000 7f6946545640 1 --2- 172.21.15.64:0/809643195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7970c84d0 0x55b7975146e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.931+0000 7f6945543640 1 -- 172.21.15.64:0/809643195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7970a7d80 msgr2=0x55b79749f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.931+0000 7f6945543640 1 --2- 172.21.15.64:0/809643195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7970a7d80 0x55b79749f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.931+0000 7f6945543640 1 -- 172.21.15.64:0/809643195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7970c84d0 msgr2=0x55b7975146e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.931+0000 7f6945543640 1 --2- 172.21.15.64:0/809643195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7970c84d0 0x55b7975146e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.931+0000 7f6945543640 1 -- 172.21.15.64:0/809643195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b797514ca0 con 0x55b7970c02f0 2024-09-21T04:11:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.931+0000 7f6946545640 1 --2- 172.21.15.64:0/809643195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7970a7d80 0x55b79749f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:11:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.931+0000 7f6945543640 1 --2- 172.21.15.64:0/809643195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7970c02f0 0x55b79749fef0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x55b7974ac3d0 tx=0x7f6930010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910c2740259b78d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.931+0000 7f69297fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:11:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.934+0000 7f69297fa640 1 -- 172.21.15.64:0/809643195 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6930028020 con 0x55b7970c02f0 2024-09-21T04:11:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.934+0000 7f69297fa640 1 -- 172.21.15.64:0/809643195 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7f6930004230 con 0x55b7970c02f0 2024-09-21T04:11:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.934+0000 7f694efee740 1 -- 172.21.15.64:0/809643195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6938036070 con 0x55b7970c02f0 2024-09-21T04:11:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.934+0000 7f69297fa640 1 -- 172.21.15.64:0/809643195 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6930004430 con 0x55b7970c02f0 2024-09-21T04:11:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.934+0000 7f694efee740 10 client.17028 fetch_fsmap learned FSMap version 39 2024-09-21T04:11:31.937 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.934+0000 7f694efee740 1 -- 172.21.15.64:0/809643195 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b797515f90 con 0x55b7970c02f0 2024-09-21T04:11:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.935+0000 7f69297fa640 1 -- 172.21.15.64:0/809643195 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6938036070 con 0x55b7970c02f0 2024-09-21T04:11:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.935+0000 7f69297fa640 1 -- 172.21.15.64:0/809643195 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f6930023070 con 0x55b7970c02f0 2024-09-21T04:11:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.935+0000 7f694efee740 10 client.17028 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T04:11:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.935+0000 7f694efee740 10 client.17028 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:11:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.935+0000 7f694efee740 20 client.17028 populate_metadata read hostname 'smithi064' 2024-09-21T04:11:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.935+0000 7f694efee740 1 --2- 172.21.15.64:0/809643195 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b7970e91b0 0x55b7975187b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.935+0000 7f694efee740 4 client.17028 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:11:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.935+0000 7f694efee740 1 -- 172.21.15.64:0/809643195 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b7974010f0 con 0x55b7970e91b0 2024-09-21T04:11:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.936+0000 7f6946545640 1 --2- 172.21.15.64:0/809643195 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b7970e91b0 0x55b7975187b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.936+0000 7f69297fa640 10 client.17028 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:31.939 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.936+0000 7f6946545640 1 --2- 172.21.15.64:0/809643195 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b7970e91b0 0x55b7975187b0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4b9164eb3ee4a87 server_cookie=d85e0b762b10622f in_seq=0 out_seq=0 2024-09-21T04:11:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.957+0000 7f69297fa640 1 -- 172.21.15.64:0/809643195 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b7974010f0 con 0x55b7970e91b0 2024-09-21T04:11:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.957+0000 7f69297fa640 10 client.17028 handle_command_reply: tid=0 2024-09-21T04:11:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.957+0000 7f694efee740 1 client.17028 shutdown 2024-09-21T04:11:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.958+0000 7f694efee740 1 -- 172.21.15.64:0/809643195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7970c02f0 msgr2=0x55b79749fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:31.961 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.958+0000 7f694efee740 1 --2- 172.21.15.64:0/809643195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7970c02f0 0x55b79749fef0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x55b7974ac3d0 tx=0x7f6930010c10 comp rx=0 tx=0).stop 2024-09-21T04:11:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.959+0000 7f694efee740 1 -- 172.21.15.64:0/809643195 shutdown_connections 2024-09-21T04:11:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.959+0000 7f694efee740 1 --2- 172.21.15.64:0/809643195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7970c84d0 0x55b7975146e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.959+0000 7f694efee740 1 --2- 172.21.15.64:0/809643195 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b7970e91b0 0x55b7975187b0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.959+0000 7f69297fa640 0 client.17028 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.959+0000 7f694efee740 1 --2- 172.21.15.64:0/809643195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7970c02f0 0x55b79749fef0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.959+0000 7f694efee740 1 --2- 172.21.15.64:0/809643195 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7970a7d80 0x55b79749f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.959+0000 7f694efee740 1 -- 172.21.15.64:0/809643195 >> 172.21.15.64:0/809643195 conn(0x55b7970dd300 msgr2=0x55b79748aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.962+0000 7f694efee740 1 -- 172.21.15.64:0/809643195 shutdown_connections 2024-09-21T04:11:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.962+0000 7f694efee740 1 -- 172.21.15.64:0/809643195 wait complete. 2024-09-21T04:11:31.965 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.962+0000 7f694efee740 20 client.17028 trim_cache size 0 max 16384 2024-09-21T04:11:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:11:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:11:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:11:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:11:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.964+0000 7f6947fa8640 1 -- 172.21.15.64:0/3483573925 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f692c0b6700 msgr2=0x7f692c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.964+0000 7f6947fa8640 1 --2- 172.21.15.64:0/3483573925 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f692c0b6700 0x7f692c0b8c10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x55b79748d630 tx=0x7f693c005760 comp rx=0 tx=0).stop 2024-09-21T04:11:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.964+0000 7f6947fa8640 1 -- 172.21.15.64:0/3483573925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6940104930 msgr2=0x7f694019a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.964+0000 7f6947fa8640 1 --2- 172.21.15.64:0/3483573925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6940104930 0x7f694019a3b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f693c00e9f0 tx=0x7f693c00eeb0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.964+0000 7f6947fa8640 1 -- 172.21.15.64:0/3483573925 shutdown_connections 2024-09-21T04:11:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.964+0000 7f6947fa8640 1 --2- 172.21.15.64:0/3483573925 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f692c0b6700 0x7f692c0b8c10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.965+0000 7f6947fa8640 1 --2- 172.21.15.64:0/3483573925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f694010dd90 0x7f694019fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.965+0000 7f6947fa8640 1 --2- 172.21.15.64:0/3483573925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6940105280 0x7f694019a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.965+0000 7f6947fa8640 1 --2- 172.21.15.64:0/3483573925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6940104930 0x7f694019a3b0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.965+0000 7f6947fa8640 1 -- 172.21.15.64:0/3483573925 >> 172.21.15.64:0/3483573925 conn(0x7f69400fe3c0 msgr2=0x7f69400ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:31.968 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.965+0000 7f6947fa8640 1 -- 172.21.15.64:0/3483573925 shutdown_connections 2024-09-21T04:11:31.970 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:31.968+0000 7f6947fa8640 1 -- 172.21.15.64:0/3483573925 wait complete. 2024-09-21T04:11:31.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.975+0000 7ff94cf16640 1 -- 172.21.15.64:0/1163862254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff94810a9e0 msgr2=0x7ff94810adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:31.978 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.975+0000 7ff94cf16640 1 --2- 172.21.15.64:0/1163862254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff94810a9e0 0x7ff94810adf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009900 tx=0x7ff93c02a410 comp rx=0 tx=0).stop 2024-09-21T04:11:31.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.976+0000 7ff94cf16640 1 -- 172.21.15.64:0/1163862254 shutdown_connections 2024-09-21T04:11:31.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.976+0000 7ff94cf16640 1 --2- 172.21.15.64:0/1163862254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff94811cc30 0x7ff94811f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.976+0000 7ff94cf16640 1 --2- 172.21.15.64:0/1163862254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff94810a9e0 0x7ff94810adf0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.979 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.976+0000 7ff94cf16640 1 --2- 172.21.15.64:0/1163862254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff948111f40 0x7ff94810a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.976+0000 7ff94cf16640 1 -- 172.21.15.64:0/1163862254 >> 172.21.15.64:0/1163862254 conn(0x7ff948070ec0 msgr2=0x7ff9480712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:31.980 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.977+0000 7ff94cf16640 1 -- 172.21.15.64:0/1163862254 shutdown_connections 2024-09-21T04:11:31.981 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.979+0000 7ff94cf16640 1 -- 172.21.15.64:0/1163862254 wait complete. 2024-09-21T04:11:31.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.980+0000 7ff94cf16640 1 Processor -- start 2024-09-21T04:11:31.983 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.980+0000 7ff94cf16640 1 -- start start 2024-09-21T04:11:31.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.981+0000 7ff94cf16640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff94810a9e0 0x7ff9481a9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.981+0000 7ff94cf16640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff948111f40 0x7ff9481a98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.984 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.981+0000 7ff94cf16640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff94811cc30 0x7ff9481aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.982+0000 7ff94cf16640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff948122980 con 0x7ff94811cc30 2024-09-21T04:11:31.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.982+0000 7ff94cf16640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff948122800 con 0x7ff94810a9e0 2024-09-21T04:11:31.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.982+0000 7ff945d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff948111f40 0x7ff9481a98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:31.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.982+0000 7ff945d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff948111f40 0x7ff9481a98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:46912/0 (socket says 172.21.15.64:46912) 2024-09-21T04:11:31.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.982+0000 7ff945d9b640 1 -- 172.21.15.64:0/4288471721 learned_addr learned my addr 172.21.15.64:0/4288471721 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:31.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.982+0000 7ff94cf16640 1 -- 172.21.15.64:0/4288471721 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff948122b00 con 0x7ff948111f40 2024-09-21T04:11:31.985 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.982+0000 7ff946d9d640 1 --2- 172.21.15.64:0/4288471721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff94811cc30 0x7ff9481aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:31.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.983+0000 7ff945d9b640 1 -- 172.21.15.64:0/4288471721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff94810a9e0 msgr2=0x7ff9481a9330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T04:11:31.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.983+0000 7ff945d9b640 1 --2- 172.21.15.64:0/4288471721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff94810a9e0 0x7ff9481a9330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.983+0000 7ff945d9b640 1 -- 172.21.15.64:0/4288471721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff94811cc30 msgr2=0x7ff9481aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:31.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.983+0000 7ff945d9b640 1 --2- 172.21.15.64:0/4288471721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff94811cc30 0x7ff9481aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.986 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.983+0000 7ff945d9b640 1 -- 172.21.15.64:0/4288471721 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9481af2a0 con 0x7ff948111f40 2024-09-21T04:11:31.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.983+0000 7ff945d9b640 1 --2- 172.21.15.64:0/4288471721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff948111f40 0x7ff9481a98d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff93c005ab0 tx=0x7ff93c003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:31.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.984+0000 7ff9377fe640 1 -- 172.21.15.64:0/4288471721 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c03f070 con 0x7ff948111f40 2024-09-21T04:11:31.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.984+0000 7ff9377fe640 1 -- 172.21.15.64:0/4288471721 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff93c031b50 con 0x7ff948111f40 2024-09-21T04:11:31.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.984+0000 7ff94cf16640 1 -- 172.21.15.64:0/4288471721 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9481af530 con 0x7ff948111f40 2024-09-21T04:11:31.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.984+0000 7ff9377fe640 1 -- 172.21.15.64:0/4288471721 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c035030 con 0x7ff948111f40 2024-09-21T04:11:31.987 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.984+0000 7ff94cf16640 1 -- 172.21.15.64:0/4288471721 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9481b5cf0 con 0x7ff948111f40 2024-09-21T04:11:31.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.988+0000 7ff9377fe640 1 -- 172.21.15.64:0/4288471721 <== mon.1 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7ff93c042070 con 0x7ff948111f40 2024-09-21T04:11:31.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.990+0000 7ff9377fe640 1 --2- 172.21.15.64:0/4288471721 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff9300b6850 0x7ff9300b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.990+0000 7ff95348a740 1 Processor -- start 2024-09-21T04:11:31.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.990+0000 7ff9377fe640 1 -- 172.21.15.64:0/4288471721 <== mon.1 v2:172.21.15.131:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7ff93c108cf0 con 0x7ff948111f40 2024-09-21T04:11:31.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.990+0000 7ff94659c640 1 --2- 172.21.15.64:0/4288471721 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff9300b6850 0x7ff9300b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:31.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.991+0000 7ff95348a740 1 -- start start 2024-09-21T04:11:31.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.991+0000 7ff94659c640 1 --2- 172.21.15.64:0/4288471721 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff9300b6850 0x7ff9300b8d60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff938009670 tx=0x7ff938006bd0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:31.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.991+0000 7ff95348a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d6d62070 0x5596d7109270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.991+0000 7ff95348a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596d6d391b0 0x5596d71097b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.991+0000 7ff95348a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596d6d41390 0x5596d7109cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:31.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.991+0000 7ff95348a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5596d707fe50 con 0x5596d6d41390 2024-09-21T04:11:31.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.991+0000 7ff95348a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5596d706f280 con 0x5596d6d62070 2024-09-21T04:11:31.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.991+0000 7ff95348a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5596d7073090 con 0x5596d6d391b0 2024-09-21T04:11:31.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.992+0000 7ff94659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d6d62070 0x5596d7109270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:31.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.992+0000 7ff94659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d6d62070 0x5596d7109270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:55776/0 (socket says 172.21.15.64:55776) 2024-09-21T04:11:31.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.992+0000 7ff94659c640 1 -- 172.21.15.64:0/3315592974 learned_addr learned my addr 172.21.15.64:0/3315592974 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:31.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.992+0000 7ff945d9b640 1 --2- 172.21.15.64:0/3315592974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596d6d391b0 0x5596d71097b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:31.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.993+0000 7ff94659c640 1 -- 172.21.15.64:0/3315592974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596d6d391b0 msgr2=0x5596d71097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:31.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.993+0000 7ff94659c640 1 --2- 172.21.15.64:0/3315592974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596d6d391b0 0x5596d71097b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.993+0000 7ff94659c640 1 -- 172.21.15.64:0/3315592974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596d6d41390 msgr2=0x5596d7109cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:31.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.993+0000 7ff94659c640 1 --2- 172.21.15.64:0/3315592974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596d6d41390 0x5596d7109cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:31.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.993+0000 7ff94659c640 1 -- 172.21.15.64:0/3315592974 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596d6dcd650 con 0x5596d6d62070 2024-09-21T04:11:31.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.993+0000 7ff94659c640 1 --2- 172.21.15.64:0/3315592974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d6d62070 0x5596d7109270 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff938013890 tx=0x7ff93800e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68130430d0a5ba84 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:31.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.994+0000 7ff934ff9640 1 -- 172.21.15.64:0/3315592974 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93800bcf0 con 0x5596d6d62070 2024-09-21T04:11:31.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.994+0000 7ff934ff9640 1 -- 172.21.15.64:0/3315592974 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff938022070 con 0x5596d6d62070 2024-09-21T04:11:31.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.994+0000 7ff934ff9640 1 -- 172.21.15.64:0/3315592974 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9380125f0 con 0x5596d6d62070 2024-09-21T04:11:31.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.996+0000 7ff95348a740 1 -- 172.21.15.64:0/3315592974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d6d62070 msgr2=0x5596d7109270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:31.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.996+0000 7ff95348a740 1 --2- 172.21.15.64:0/3315592974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d6d62070 0x5596d7109270 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff938013890 tx=0x7ff93800e000 comp rx=0 tx=0).stop 2024-09-21T04:11:32.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.997+0000 7ff95348a740 1 -- 172.21.15.64:0/3315592974 shutdown_connections 2024-09-21T04:11:32.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.997+0000 7ff95348a740 1 --2- 172.21.15.64:0/3315592974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596d6d41390 0x5596d7109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:32.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.997+0000 7ff95348a740 1 --2- 172.21.15.64:0/3315592974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596d6d391b0 0x5596d71097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:32.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.997+0000 7ff95348a740 1 --2- 172.21.15.64:0/3315592974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d6d62070 0x5596d7109270 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:32.000 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.997+0000 7ff95348a740 1 -- 172.21.15.64:0/3315592974 >> 172.21.15.64:0/3315592974 conn(0x5596d6d56460 msgr2=0x5596d7107160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:32.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.998+0000 7ff95348a740 1 -- 172.21.15.64:0/3315592974 shutdown_connections 2024-09-21T04:11:32.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.998+0000 7ff95348a740 1 -- 172.21.15.64:0/3315592974 wait complete. 2024-09-21T04:11:32.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.998+0000 7ff95348a740 1 -- start start 2024-09-21T04:11:32.001 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.999+0000 7ff95348a740 1 Processor -- start 2024-09-21T04:11:32.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.999+0000 7ff95348a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d6d391b0 0x5596d7125890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:32.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:31.999+0000 7ff95348a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596d6d41390 0x5596d7125dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:32.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.000+0000 7ff94659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d6d391b0 0x5596d7125890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:32.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.000+0000 7ff94659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d6d391b0 0x5596d7125890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:55790/0 (socket says 172.21.15.64:55790) 2024-09-21T04:11:32.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.000+0000 7ff94659c640 1 -- 172.21.15.64:0/3313302054 learned_addr learned my addr 172.21.15.64:0/3313302054 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:32.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.000+0000 7ff95348a740 1 --2- 172.21.15.64:0/3313302054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596d6d62070 0x5596d7126310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:32.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.000+0000 7ff95348a740 1 -- 172.21.15.64:0/3313302054 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5596d6ed0700 con 0x5596d6d41390 2024-09-21T04:11:32.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.000+0000 7ff95348a740 1 -- 172.21.15.64:0/3313302054 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5596d700a110 con 0x5596d6d391b0 2024-09-21T04:11:32.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.000+0000 7ff95348a740 1 -- 172.21.15.64:0/3313302054 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5596d71288f0 con 0x5596d6d62070 2024-09-21T04:11:32.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.000+0000 7ff946d9d640 1 --2- 172.21.15.64:0/3313302054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596d6d62070 0x5596d7126310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:32.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.000+0000 7ff946d9d640 1 -- 172.21.15.64:0/3313302054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d6d391b0 msgr2=0x5596d7125890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:32.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.000+0000 7ff946d9d640 1 --2- 172.21.15.64:0/3313302054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d6d391b0 0x5596d7125890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:32.004 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.000+0000 7ff946d9d640 1 -- 172.21.15.64:0/3313302054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596d6d41390 msgr2=0x5596d7125dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:32.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.000+0000 7ff946d9d640 1 --2- 172.21.15.64:0/3313302054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596d6d41390 0x5596d7125dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:32.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.000+0000 7ff946d9d640 1 -- 172.21.15.64:0/3313302054 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596d7037c40 con 0x5596d6d62070 2024-09-21T04:11:32.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.000+0000 7ff95348a740 10 client.0 _ll_register_callbacks cb 0x5596d7023990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:11:32.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.001+0000 7ff946d9d640 1 --2- 172.21.15.64:0/3313302054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596d6d62070 0x5596d7126310 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff940011940 tx=0x7ff940011e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a6089ba2663b320 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:32.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.001+0000 7ff92dffb640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-21T04:11:32.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.001+0000 7ff95348a740 1 -- 172.21.15.64:0/3313302054 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff938022070 con 0x5596d6d62070 2024-09-21T04:11:32.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.001+0000 7ff92dffb640 1 -- 172.21.15.64:0/3313302054 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940003e40 con 0x5596d6d62070 2024-09-21T04:11:32.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.001+0000 7ff92dffb640 1 -- 172.21.15.64:0/3313302054 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7ff940004b20 con 0x5596d6d62070 2024-09-21T04:11:32.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.001+0000 7ff92dffb640 1 -- 172.21.15.64:0/3313302054 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff940004d60 con 0x5596d6d62070 2024-09-21T04:11:32.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.002+0000 7ff95348a740 10 client.26572 fetch_fsmap learned FSMap version 39 2024-09-21T04:11:32.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.002+0000 7ff95348a740 1 -- 172.21.15.64:0/3313302054 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=39}) -- 0x5596d7134580 con 0x5596d6d62070 2024-09-21T04:11:32.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.003+0000 7ff92dffb640 1 -- 172.21.15.64:0/3313302054 <== mon.1 v2:172.21.15.131:3300/0 4 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7ff940018070 con 0x5596d6d62070 2024-09-21T04:11:32.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.003+0000 7ff95348a740 10 client.26572 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T04:11:32.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.003+0000 7ff95348a740 10 client.26572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:11:32.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.003+0000 7ff95348a740 20 client.26572 populate_metadata read hostname 'smithi064' 2024-09-21T04:11:32.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.003+0000 7ff95348a740 1 --2- 172.21.15.64:0/3313302054 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5596d6d209f0 0x5596d71379d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:32.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.003+0000 7ff95348a740 4 client.26572 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:11:32.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.003+0000 7ff95348a740 1 -- 172.21.15.64:0/3313302054 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff938022070 con 0x5596d6d209f0 2024-09-21T04:11:32.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.004+0000 7ff94659c640 1 --2- 172.21.15.64:0/3313302054 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5596d6d209f0 0x5596d71379d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:32.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.006+0000 7ff94659c640 1 --2- 172.21.15.64:0/3313302054 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5596d6d209f0 0x5596d71379d0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46a718a3edeb612 server_cookie=c03438e8c83ac259 in_seq=0 out_seq=0 2024-09-21T04:11:32.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.007+0000 7ff92dffb640 10 client.26572 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:32.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.008+0000 7ff92dffb640 1 -- 172.21.15.64:0/3313302054 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7ff938022070 con 0x5596d6d209f0 2024-09-21T04:11:32.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.008+0000 7ff92dffb640 10 client.26572 handle_command_reply: tid=0 2024-09-21T04:11:32.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.008+0000 7ff95348a740 1 client.26572 shutdown 2024-09-21T04:11:32.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.008+0000 7ff95348a740 1 -- 172.21.15.64:0/3313302054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596d6d62070 msgr2=0x5596d7126310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:32.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.008+0000 7ff95348a740 1 --2- 172.21.15.64:0/3313302054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596d6d62070 0x5596d7126310 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff940011940 tx=0x7ff940011e00 comp rx=0 tx=0).stop 2024-09-21T04:11:32.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.008+0000 7ff95348a740 1 -- 172.21.15.64:0/3313302054 shutdown_connections 2024-09-21T04:11:32.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.008+0000 7ff95348a740 1 --2- 172.21.15.64:0/3313302054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596d6d62070 0x5596d7126310 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:32.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.009+0000 7ff95348a740 1 --2- 172.21.15.64:0/3313302054 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5596d6d209f0 0x5596d71379d0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:32.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.009+0000 7ff95348a740 1 --2- 172.21.15.64:0/3313302054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596d6d41390 0x5596d7125dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:32.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.009+0000 7ff92dffb640 0 client.26572 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:32.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.009+0000 7ff95348a740 1 --2- 172.21.15.64:0/3313302054 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d6d391b0 0x5596d7125890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:32.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.009+0000 7ff95348a740 1 -- 172.21.15.64:0/3313302054 >> 172.21.15.64:0/3313302054 conn(0x5596d6d56460 msgr2=0x5596d710c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:32.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.010+0000 7ff95348a740 1 -- 172.21.15.64:0/3313302054 shutdown_connections 2024-09-21T04:11:32.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.010+0000 7ff95348a740 1 -- 172.21.15.64:0/3313302054 wait complete. 2024-09-21T04:11:32.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.010+0000 7ff95348a740 20 client.26572 trim_cache size 0 max 16384 2024-09-21T04:11:32.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.047+0000 7ff95348a740 1 Processor -- start 2024-09-21T04:11:32.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.047+0000 7ff95348a740 1 -- start start 2024-09-21T04:11:32.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.047+0000 7ff95348a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d6d209f0 0x5596d7124780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:32.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.047+0000 7ff95348a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596d6d391b0 0x5596d710cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:32.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.048+0000 7ff95348a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596d6d41390 0x5596d71274c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:32.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.048+0000 7ff95348a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff9280113b0 con 0x5596d6d391b0 2024-09-21T04:11:32.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.048+0000 7ff95348a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff92800ebf0 con 0x5596d6d209f0 2024-09-21T04:11:32.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.048+0000 7ff95348a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff92800ca60 con 0x5596d6d41390 2024-09-21T04:11:32.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.048+0000 7ff945d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596d6d391b0 0x5596d710cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:32.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.048+0000 7ff945d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596d6d391b0 0x5596d710cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46586/0 (socket says 172.21.15.64:46586) 2024-09-21T04:11:32.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.048+0000 7ff945d9b640 1 -- 172.21.15.64:0/3239486498 learned_addr learned my addr 172.21.15.64:0/3239486498 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:32.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.048+0000 7ff946d9d640 1 --2- 172.21.15.64:0/3239486498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596d6d41390 0x5596d71274c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:32.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.048+0000 7ff945d9b640 1 -- 172.21.15.64:0/3239486498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596d6d41390 msgr2=0x5596d71274c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:32.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.048+0000 7ff945d9b640 1 --2- 172.21.15.64:0/3239486498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596d6d41390 0x5596d71274c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:32.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.048+0000 7ff945d9b640 1 -- 172.21.15.64:0/3239486498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d6d209f0 msgr2=0x5596d7124780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:32.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.048+0000 7ff945d9b640 1 --2- 172.21.15.64:0/3239486498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d6d209f0 0x5596d7124780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:32.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.048+0000 7ff945d9b640 1 -- 172.21.15.64:0/3239486498 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596d710c0a0 con 0x5596d6d391b0 2024-09-21T04:11:32.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.049+0000 7ff945d9b640 1 --2- 172.21.15.64:0/3239486498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596d6d391b0 0x5596d710cd00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff93c106320 tx=0x7ff93c0ce490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7313023f7da6e323 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:32.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.049+0000 7ff92ffff640 1 -- 172.21.15.64:0/3239486498 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c0ce590 con 0x5596d6d391b0 2024-09-21T04:11:32.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.049+0000 7ff92ffff640 1 -- 172.21.15.64:0/3239486498 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff93c0ca040 con 0x5596d6d391b0 2024-09-21T04:11:32.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.049+0000 7ff92ffff640 1 -- 172.21.15.64:0/3239486498 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c0d8010 con 0x5596d6d391b0 2024-09-21T04:11:32.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.050+0000 7ff95348a740 1 -- 172.21.15.64:0/3239486498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596d6d391b0 msgr2=0x5596d710cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:32.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.050+0000 7ff95348a740 1 --2- 172.21.15.64:0/3239486498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596d6d391b0 0x5596d710cd00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff93c106320 tx=0x7ff93c0ce490 comp rx=0 tx=0).stop 2024-09-21T04:11:32.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.052+0000 7ff95348a740 1 -- 172.21.15.64:0/3239486498 shutdown_connections 2024-09-21T04:11:32.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.052+0000 7ff95348a740 1 --2- 172.21.15.64:0/3239486498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596d6d41390 0x5596d71274c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:32.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.052+0000 7ff95348a740 1 --2- 172.21.15.64:0/3239486498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596d6d391b0 0x5596d710cd00 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:32.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.052+0000 7ff95348a740 1 --2- 172.21.15.64:0/3239486498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d6d209f0 0x5596d7124780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:32.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.052+0000 7ff95348a740 1 -- 172.21.15.64:0/3239486498 >> 172.21.15.64:0/3239486498 conn(0x5596d6d56460 msgr2=0x5596d7126db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:32.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.052+0000 7ff95348a740 1 -- 172.21.15.64:0/3239486498 shutdown_connections 2024-09-21T04:11:32.055 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.052+0000 7ff95348a740 1 -- 172.21.15.64:0/3239486498 wait complete. 2024-09-21T04:11:32.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.053+0000 7ff95348a740 1 -- start start 2024-09-21T04:11:32.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.053+0000 7ff95348a740 1 Processor -- start 2024-09-21T04:11:32.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.054+0000 7ff95348a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d6d209f0 0x5596d712bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:32.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.054+0000 7ff95348a740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596d6d391b0 0x5596d712c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:32.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.054+0000 7ff95348a740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596d6d41390 0x5596d712c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:32.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.054+0000 7ff95348a740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff92800d0b0 con 0x5596d6d391b0 2024-09-21T04:11:32.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.054+0000 7ff95348a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff92800e520 con 0x5596d6d209f0 2024-09-21T04:11:32.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.054+0000 7ff95348a740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff92800d780 con 0x5596d6d41390 2024-09-21T04:11:32.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.054+0000 7ff95348a740 10 client.0 _ll_register_callbacks cb 0x7ff940009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:11:32.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.054+0000 7ff94659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d6d209f0 0x5596d712bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:32.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.054+0000 7ff94659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d6d209f0 0x5596d712bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:55802/0 (socket says 172.21.15.64:55802) 2024-09-21T04:11:32.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.054+0000 7ff94659c640 1 -- 172.21.15.64:0/3198194467 learned_addr learned my addr 172.21.15.64:0/3198194467 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:32.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.055+0000 7ff945d9b640 1 --2- 172.21.15.64:0/3198194467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596d6d391b0 0x5596d712c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:32.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.055+0000 7ff946d9d640 1 --2- 172.21.15.64:0/3198194467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596d6d41390 0x5596d712c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:32.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.055+0000 7ff94659c640 1 -- 172.21.15.64:0/3198194467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596d6d41390 msgr2=0x5596d712c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:32.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.055+0000 7ff94659c640 1 --2- 172.21.15.64:0/3198194467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596d6d41390 0x5596d712c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:32.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.055+0000 7ff94659c640 1 -- 172.21.15.64:0/3198194467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596d6d391b0 msgr2=0x5596d712c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:32.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.056+0000 7ff94659c640 1 --2- 172.21.15.64:0/3198194467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596d6d391b0 0x5596d712c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:32.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.056+0000 7ff94659c640 1 -- 172.21.15.64:0/3198194467 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596d710b9d0 con 0x5596d6d209f0 2024-09-21T04:11:32.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.056+0000 7ff945d9b640 1 --2- 172.21.15.64:0/3198194467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596d6d391b0 0x5596d712c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:11:32.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.056+0000 7ff94659c640 1 --2- 172.21.15.64:0/3198194467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d6d209f0 0x5596d712bde0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff9380123b0 tx=0x7ff938012760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a80f2a8ed72bf244 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:32.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.056+0000 7ff92dffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:11:32.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.057+0000 7ff92dffb640 1 -- 172.21.15.64:0/3198194467 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93800bd00 con 0x5596d6d209f0 2024-09-21T04:11:32.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.057+0000 7ff92dffb640 1 -- 172.21.15.64:0/3198194467 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7ff938004040 con 0x5596d6d209f0 2024-09-21T04:11:32.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.057+0000 7ff95348a740 1 -- 172.21.15.64:0/3198194467 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff93c0ca040 con 0x5596d6d209f0 2024-09-21T04:11:32.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.057+0000 7ff92dffb640 1 -- 172.21.15.64:0/3198194467 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff93800bea0 con 0x5596d6d209f0 2024-09-21T04:11:32.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.057+0000 7ff95348a740 10 client.26711 fetch_fsmap learned FSMap version 39 2024-09-21T04:11:32.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.057+0000 7ff95348a740 1 -- 172.21.15.64:0/3198194467 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5596d71a2610 con 0x5596d6d209f0 2024-09-21T04:11:32.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.058+0000 7ff92dffb640 1 -- 172.21.15.64:0/3198194467 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff93c0ca040 con 0x5596d6d209f0 2024-09-21T04:11:32.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.058+0000 7ff92dffb640 1 -- 172.21.15.64:0/3198194467 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7ff938012070 con 0x5596d6d209f0 2024-09-21T04:11:32.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.058+0000 7ff95348a740 10 client.26711 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T04:11:32.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.058+0000 7ff95348a740 10 client.26711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:11:32.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.058+0000 7ff95348a740 20 client.26711 populate_metadata read hostname 'smithi064' 2024-09-21T04:11:32.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.058+0000 7ff95348a740 1 --2- 172.21.15.64:0/3198194467 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5596d6d62070 0x5596d71a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:32.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.058+0000 7ff95348a740 4 client.26711 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T04:11:32.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.058+0000 7ff95348a740 1 -- 172.21.15.64:0/3198194467 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5596d7081da0 con 0x5596d6d62070 2024-09-21T04:11:32.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.060+0000 7ff945d9b640 1 --2- 172.21.15.64:0/3198194467 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5596d6d62070 0x5596d71a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:32.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.061+0000 7ff945d9b640 1 --2- 172.21.15.64:0/3198194467 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5596d6d62070 0x5596d71a4ab0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a351ef702341767 server_cookie=55854c376e89ccdd in_seq=0 out_seq=0 2024-09-21T04:11:32.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.061+0000 7ff92dffb640 10 client.26711 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:32.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.062+0000 7ff92dffb640 1 -- 172.21.15.64:0/3198194467 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2930 (crc 0 0 0) 0x5596d7081da0 con 0x5596d6d62070 2024-09-21T04:11:32.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.062+0000 7ff92dffb640 10 client.26711 handle_command_reply: tid=0 2024-09-21T04:11:32.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.062+0000 7ff95348a740 1 client.26711 shutdown 2024-09-21T04:11:32.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.062+0000 7ff95348a740 1 -- 172.21.15.64:0/3198194467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d6d209f0 msgr2=0x5596d712bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:32.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.062+0000 7ff95348a740 1 --2- 172.21.15.64:0/3198194467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d6d209f0 0x5596d712bde0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff9380123b0 tx=0x7ff938012760 comp rx=0 tx=0).stop 2024-09-21T04:11:32.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.062+0000 7ff95348a740 1 -- 172.21.15.64:0/3198194467 shutdown_connections 2024-09-21T04:11:32.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.062+0000 7ff95348a740 1 --2- 172.21.15.64:0/3198194467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5596d6d41390 0x5596d712c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:32.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.062+0000 7ff95348a740 1 --2- 172.21.15.64:0/3198194467 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5596d6d62070 0x5596d71a4ab0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:32.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.062+0000 7ff95348a740 1 --2- 172.21.15.64:0/3198194467 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596d6d391b0 0x5596d712c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:32.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.062+0000 7ff95348a740 1 --2- 172.21.15.64:0/3198194467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5596d6d209f0 0x5596d712bde0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:32.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.062+0000 7ff95348a740 1 -- 172.21.15.64:0/3198194467 >> 172.21.15.64:0/3198194467 conn(0x5596d6d56460 msgr2=0x5596d710cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:32.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.062+0000 7ff92dffb640 0 client.26711 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:32.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.062+0000 7ff95348a740 1 -- 172.21.15.64:0/3198194467 shutdown_connections 2024-09-21T04:11:32.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.062+0000 7ff95348a740 1 -- 172.21.15.64:0/3198194467 wait complete. 2024-09-21T04:11:32.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.063+0000 7ff95348a740 20 client.26711 trim_cache size 0 max 16384 2024-09-21T04:11:32.067 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:11:32.067 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-21T04:11:32.067 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {}, 2024-09-21T04:11:32.067 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_stats": { 2024-09-21T04:11:32.067 INFO:teuthology.orchestra.run.smithi064.stdout: "1b23713d-ea3a-4a86-a87f-65f6ab82671d": { 2024-09-21T04:11:32.067 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:10:58.352639+0000", 2024-09-21T04:11:32.067 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:32.067 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:32.067 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:32.067 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:32.068 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:32.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:32.068 INFO:teuthology.orchestra.run.smithi064.stdout: "2c0523fb-aae4-43fa-9568-136a9b640051": { 2024-09-21T04:11:32.068 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:09:39.808606+0000", 2024-09-21T04:11:32.068 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:32.068 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:32.068 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:32.068 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:32.068 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:32.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:32.068 INFO:teuthology.orchestra.run.smithi064.stdout: "bed40d88-2102-43a6-ac2d-b89d5a2264d0": { 2024-09-21T04:11:32.068 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:11.363340+0000", 2024-09-21T04:11:32.068 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:32.068 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:32.068 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:32.069 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:32.069 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:32.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:32.069 INFO:teuthology.orchestra.run.smithi064.stdout: "e9242f88-6599-40eb-adf1-bdb3e946a1e0": { 2024-09-21T04:11:32.069 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:05:39.744646+0000", 2024-09-21T04:11:32.069 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:32.069 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:32.069 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:32.069 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:32.069 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:32.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:32.069 INFO:teuthology.orchestra.run.smithi064.stdout: "6bde0fe5-f775-44b1-ad0c-f9e596574dc0": { 2024-09-21T04:11:32.069 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:06:23.835536+0000", 2024-09-21T04:11:32.070 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:32.070 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:32.070 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:32.070 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:32.070 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:32.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:32.070 INFO:teuthology.orchestra.run.smithi064.stdout: "b52b950c-0f97-49ec-a37c-1a8e04348ce8": { 2024-09-21T04:11:32.070 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:54.003800+0000", 2024-09-21T04:11:32.070 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:32.070 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:32.070 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:32.070 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:32.070 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:32.070 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:32.071 INFO:teuthology.orchestra.run.smithi064.stdout: "e2c2eed8-9292-4bf6-8a39-13ad27da5acb": { 2024-09-21T04:11:32.071 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:08:47.067932+0000", 2024-09-21T04:11:32.071 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:32.071 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:32.071 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:32.071 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:32.071 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:32.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:32.071 INFO:teuthology.orchestra.run.smithi064.stdout: "75db1bc5-612e-45aa-bad2-cc90feeaa840": { 2024-09-21T04:11:32.071 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:59.123138+0000", 2024-09-21T04:11:32.071 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:32.071 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:32.071 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:32.071 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:32.072 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:32.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:32.072 INFO:teuthology.orchestra.run.smithi064.stdout: "0ab717a7-36c6-4dca-94e7-af0bdf2aa112": { 2024-09-21T04:11:32.072 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:42.691086+0000", 2024-09-21T04:11:32.072 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:32.072 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:32.072 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:32.072 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:32.072 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:32.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:32.072 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:11:32.073 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:03:59.570174+0000", 2024-09-21T04:11:32.073 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:32.073 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:32.073 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:32.073 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:32.073 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:32.073 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:11:32.073 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:11:32.073 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:11:32.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.064+0000 7ff94cf16640 1 -- 172.21.15.64:0/4288471721 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff9300b6850 msgr2=0x7ff9300b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:32.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.064+0000 7ff94cf16640 1 --2- 172.21.15.64:0/4288471721 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff9300b6850 0x7ff9300b8d60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff938009670 tx=0x7ff938006bd0 comp rx=0 tx=0).stop 2024-09-21T04:11:32.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.064+0000 7ff94cf16640 1 -- 172.21.15.64:0/4288471721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff948111f40 msgr2=0x7ff9481a98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:32.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.064+0000 7ff94cf16640 1 --2- 172.21.15.64:0/4288471721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff948111f40 0x7ff9481a98d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff93c005ab0 tx=0x7ff93c003b90 comp rx=0 tx=0).stop 2024-09-21T04:11:32.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.064+0000 7ff94cf16640 1 -- 172.21.15.64:0/4288471721 shutdown_connections 2024-09-21T04:11:32.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.064+0000 7ff94cf16640 1 --2- 172.21.15.64:0/4288471721 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff9300b6850 0x7ff9300b8d60 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:32.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.064+0000 7ff94cf16640 1 --2- 172.21.15.64:0/4288471721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff94811cc30 0x7ff9481aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:32.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.064+0000 7ff94cf16640 1 --2- 172.21.15.64:0/4288471721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff948111f40 0x7ff9481a98d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:32.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.064+0000 7ff94cf16640 1 --2- 172.21.15.64:0/4288471721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff94810a9e0 0x7ff9481a9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:32.074 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.064+0000 7ff94cf16640 1 -- 172.21.15.64:0/4288471721 >> 172.21.15.64:0/4288471721 conn(0x7ff948070ec0 msgr2=0x7ff94811e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:32.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.065+0000 7ff94cf16640 1 -- 172.21.15.64:0/4288471721 shutdown_connections 2024-09-21T04:11:32.075 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:32.065+0000 7ff94cf16640 1 -- 172.21.15.64:0/4288471721 wait complete. 2024-09-21T04:11:32.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:32 smithi131 ceph-mon[35193]: pgmap v550: 97 pgs: 97 active+clean; 803 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 393 op/s 2024-09-21T04:11:32.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:32 smithi119 ceph-mon[34811]: pgmap v550: 97 pgs: 97 active+clean; 803 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 393 op/s 2024-09-21T04:11:32.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:32 smithi064 ceph-mon[31180]: pgmap v550: 97 pgs: 97 active+clean; 803 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 393 op/s 2024-09-21T04:11:33.027 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T04:11:33.027 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:11:33.349 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:11:33.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.851+0000 7efd8d719640 1 -- 172.21.15.64:0/3771307111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd88108cf0 msgr2=0x7efd8810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:33.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.851+0000 7efd8d719640 1 --2- 172.21.15.64:0/3771307111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd88108cf0 0x7efd8810f520 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7efd78009ca0 tx=0x7efd7802a8c0 comp rx=0 tx=0).stop 2024-09-21T04:11:33.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.851+0000 7efd8d719640 1 -- 172.21.15.64:0/3771307111 shutdown_connections 2024-09-21T04:11:33.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.851+0000 7efd8d719640 1 --2- 172.21.15.64:0/3771307111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd88108cf0 0x7efd8810f520 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:33.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.851+0000 7efd8d719640 1 --2- 172.21.15.64:0/3771307111 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd881083a0 0x7efd881087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:33.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.851+0000 7efd8d719640 1 --2- 172.21.15.64:0/3771307111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd88102a60 0x7efd88102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:33.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.851+0000 7efd8d719640 1 -- 172.21.15.64:0/3771307111 >> 172.21.15.64:0/3771307111 conn(0x7efd88079730 msgr2=0x7efd88079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:33.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.851+0000 7efd8d719640 1 -- 172.21.15.64:0/3771307111 shutdown_connections 2024-09-21T04:11:33.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.851+0000 7efd8d719640 1 -- 172.21.15.64:0/3771307111 wait complete. 2024-09-21T04:11:33.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.852+0000 7efd8d719640 1 Processor -- start 2024-09-21T04:11:33.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.852+0000 7efd8d719640 1 -- start start 2024-09-21T04:11:33.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.852+0000 7efd8d719640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd88102a60 0x7efd8819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:33.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.852+0000 7efd8d719640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd881083a0 0x7efd8819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:33.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.852+0000 7efd8d719640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd88108cf0 0x7efd8819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:33.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.852+0000 7efd8d719640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efd88112960 con 0x7efd88102a60 2024-09-21T04:11:33.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.852+0000 7efd8d719640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efd881127e0 con 0x7efd88108cf0 2024-09-21T04:11:33.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.852+0000 7efd8d719640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efd88112ae0 con 0x7efd881083a0 2024-09-21T04:11:33.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.852+0000 7efd86ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd88102a60 0x7efd8819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:33.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.852+0000 7efd86ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd88102a60 0x7efd8819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46622/0 (socket says 172.21.15.64:46622) 2024-09-21T04:11:33.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.852+0000 7efd86ffd640 1 -- 172.21.15.64:0/841634053 learned_addr learned my addr 172.21.15.64:0/841634053 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:33.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.852+0000 7efd877fe640 1 --2- 172.21.15.64:0/841634053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd88108cf0 0x7efd8819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:33.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.853+0000 7efd867fc640 1 --2- 172.21.15.64:0/841634053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd881083a0 0x7efd8819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:33.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.853+0000 7efd86ffd640 1 -- 172.21.15.64:0/841634053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd881083a0 msgr2=0x7efd8819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:33.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.853+0000 7efd86ffd640 1 --2- 172.21.15.64:0/841634053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd881083a0 0x7efd8819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:33.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.853+0000 7efd86ffd640 1 -- 172.21.15.64:0/841634053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd88108cf0 msgr2=0x7efd8819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:33.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.853+0000 7efd86ffd640 1 --2- 172.21.15.64:0/841634053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd88108cf0 0x7efd8819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:33.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.853+0000 7efd86ffd640 1 -- 172.21.15.64:0/841634053 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd881a0410 con 0x7efd88102a60 2024-09-21T04:11:33.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.853+0000 7efd867fc640 1 --2- 172.21.15.64:0/841634053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd881083a0 0x7efd8819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:11:33.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.853+0000 7efd86ffd640 1 --2- 172.21.15.64:0/841634053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd88102a60 0x7efd8819a4a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7efd8000e9f0 tx=0x7efd8000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:33.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.853+0000 7efd7ffff640 1 -- 172.21.15.64:0/841634053 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8000cd80 con 0x7efd88102a60 2024-09-21T04:11:33.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.853+0000 7efd8d719640 1 -- 172.21.15.64:0/841634053 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd881a0700 con 0x7efd88102a60 2024-09-21T04:11:33.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.853+0000 7efd7ffff640 1 -- 172.21.15.64:0/841634053 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd800108e0 con 0x7efd88102a60 2024-09-21T04:11:33.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.853+0000 7efd7ffff640 1 -- 172.21.15.64:0/841634053 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80018980 con 0x7efd88102a60 2024-09-21T04:11:33.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.853+0000 7efd8d719640 1 -- 172.21.15.64:0/841634053 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd881a6e60 con 0x7efd88102a60 2024-09-21T04:11:33.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.854+0000 7efd8d719640 1 -- 172.21.15.64:0/841634053 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd88102e70 con 0x7efd88102a60 2024-09-21T04:11:33.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.856+0000 7efd7ffff640 1 -- 172.21.15.64:0/841634053 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7efd80010430 con 0x7efd88102a60 2024-09-21T04:11:33.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.856+0000 7efd7ffff640 1 --2- 172.21.15.64:0/841634053 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7efd740b6680 0x7efd740b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:33.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.857+0000 7efd867fc640 1 --2- 172.21.15.64:0/841634053 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7efd740b6680 0x7efd740b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:33.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.857+0000 7efd7ffff640 1 -- 172.21.15.64:0/841634053 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7efd800e7b60 con 0x7efd88102a60 2024-09-21T04:11:33.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.857+0000 7efd867fc640 1 --2- 172.21.15.64:0/841634053 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7efd740b6680 0x7efd740b8b90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efd70005fd0 tx=0x7efd700094d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:33.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:33.861+0000 7efd7ffff640 1 -- 172.21.15.64:0/841634053 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efd800ae050 con 0x7efd88102a60 2024-09-21T04:11:34.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:34.167+0000 7efd8d719640 1 -- 172.21.15.64:0/841634053 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd8802e640 con 0x7efd88102a60 2024-09-21T04:11:34.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:34.168+0000 7efd7ffff640 1 -- 172.21.15.64:0/841634053 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11916 (secure 0 0 0) 0x7efd800b2430 con 0x7efd88102a60 2024-09-21T04:11:34.171 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:11:34.171 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":39,"btime":"2024-09-21T04:11:18:259202+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14805,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/97995119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":97995119},{"type":"v1","addr":"172.21.15.64:6839","nonce":97995119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/390891068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":390891068},{"type":"v1","addr":"172.21.15.64:6841","nonce":390891068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:02:03.354744+0000","modified":"2024-09-21T04:11:17.548265+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24710,"mds_1":24680,"mds_2":24661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24634":{"gid":24634,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6839/782190424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782190424},{"type":"v1","addr":"172.21.15.131:6839","nonce":782190424}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24680":{"gid":24680,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24710":{"gid":24710,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6839/1319793636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1319793636},{"type":"v1","addr":"172.21.15.119:6839","nonce":1319793636}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24710,"qdb_cluster":[24710,24680,24661]},"id":1}]} 2024-09-21T04:11:34.171 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 39 2024-09-21T04:11:34.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:34.170+0000 7efd8d719640 1 -- 172.21.15.64:0/841634053 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7efd740b6680 msgr2=0x7efd740b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:34.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:34.170+0000 7efd8d719640 1 --2- 172.21.15.64:0/841634053 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7efd740b6680 0x7efd740b8b90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efd70005fd0 tx=0x7efd700094d0 comp rx=0 tx=0).stop 2024-09-21T04:11:34.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:34.170+0000 7efd8d719640 1 -- 172.21.15.64:0/841634053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd88102a60 msgr2=0x7efd8819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:34.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:34.170+0000 7efd8d719640 1 --2- 172.21.15.64:0/841634053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd88102a60 0x7efd8819a4a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7efd8000e9f0 tx=0x7efd8000eeb0 comp rx=0 tx=0).stop 2024-09-21T04:11:34.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:34.170+0000 7efd8d719640 1 -- 172.21.15.64:0/841634053 shutdown_connections 2024-09-21T04:11:34.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:34.170+0000 7efd8d719640 1 --2- 172.21.15.64:0/841634053 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7efd740b6680 0x7efd740b8b90 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:34.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:34.170+0000 7efd8d719640 1 --2- 172.21.15.64:0/841634053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd88108cf0 0x7efd8819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:34.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:34.170+0000 7efd8d719640 1 --2- 172.21.15.64:0/841634053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd881083a0 0x7efd8819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:34.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:34.170+0000 7efd8d719640 1 --2- 172.21.15.64:0/841634053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd88102a60 0x7efd8819a4a0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:34.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:34.170+0000 7efd8d719640 1 -- 172.21.15.64:0/841634053 >> 172.21.15.64:0/841634053 conn(0x7efd88079730 msgr2=0x7efd8806b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:34.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:34.170+0000 7efd8d719640 1 -- 172.21.15.64:0/841634053 shutdown_connections 2024-09-21T04:11:34.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:34.171+0000 7efd8d719640 1 -- 172.21.15.64:0/841634053 wait complete. 2024-09-21T04:11:34.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:34 smithi064 ceph-mon[31180]: pgmap v551: 97 pgs: 97 active+clean; 818 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.7 MiB/s wr, 547 op/s 2024-09-21T04:11:34.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:34 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/841634053' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:11:34.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:34 smithi131 ceph-mon[35193]: pgmap v551: 97 pgs: 97 active+clean; 818 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.7 MiB/s wr, 547 op/s 2024-09-21T04:11:34.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:34 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/841634053' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:11:34.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:34 smithi119 ceph-mon[34811]: pgmap v551: 97 pgs: 97 active+clean; 818 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.7 MiB/s wr, 547 op/s 2024-09-21T04:11:34.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:34 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/841634053' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:11:35.299 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T04:11:35.558 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:11:36.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.300+0000 7fe7a2a73640 1 -- 172.21.15.64:0/1580031153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe79c110180 msgr2=0x7fe79c1125a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:36.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.300+0000 7fe7a2a73640 1 --2- 172.21.15.64:0/1580031153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe79c110180 0x7fe79c1125a0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe78c009ca0 tx=0x7fe78c02a950 comp rx=0 tx=0).stop 2024-09-21T04:11:36.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.300+0000 7fe7a2a73640 1 -- 172.21.15.64:0/1580031153 shutdown_connections 2024-09-21T04:11:36.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.300+0000 7fe7a2a73640 1 --2- 172.21.15.64:0/1580031153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe79c110180 0x7fe79c1125a0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.300+0000 7fe7a2a73640 1 --2- 172.21.15.64:0/1580031153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe79c102f20 0x7fe79c10fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.300+0000 7fe7a2a73640 1 --2- 172.21.15.64:0/1580031153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe79c1025d0 0x7fe79c1029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.300+0000 7fe7a2a73640 1 -- 172.21.15.64:0/1580031153 >> 172.21.15.64:0/1580031153 conn(0x7fe79c079730 msgr2=0x7fe79c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:36.725 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.300+0000 7fe7a2a73640 1 -- 172.21.15.64:0/1580031153 shutdown_connections 2024-09-21T04:11:36.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.300+0000 7fe7a2a73640 1 -- 172.21.15.64:0/1580031153 wait complete. 2024-09-21T04:11:36.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.301+0000 7fe7a2a73640 1 Processor -- start 2024-09-21T04:11:36.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.301+0000 7fe7a2a73640 1 -- start start 2024-09-21T04:11:36.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.301+0000 7fe7a2a73640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe79c1025d0 0x7fe79c19c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:36.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.301+0000 7fe7a2a73640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe79c102f20 0x7fe79c19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:36.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.301+0000 7fe7a2a73640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe79c110180 0x7fe79c1a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:36.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.301+0000 7fe7a2a73640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c115cc0 con 0x7fe79c102f20 2024-09-21T04:11:36.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.301+0000 7fe7a2a73640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c115b40 con 0x7fe79c110180 2024-09-21T04:11:36.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.302+0000 7fe7a2a73640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c115e40 con 0x7fe79c1025d0 2024-09-21T04:11:36.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.302+0000 7fe79bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe79c102f20 0x7fe79c19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:36.726 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.302+0000 7fe79bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe79c102f20 0x7fe79c19cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46632/0 (socket says 172.21.15.64:46632) 2024-09-21T04:11:36.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.302+0000 7fe79bfff640 1 -- 172.21.15.64:0/4047365913 learned_addr learned my addr 172.21.15.64:0/4047365913 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:36.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.302+0000 7fe7a080f640 1 --2- 172.21.15.64:0/4047365913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe79c1025d0 0x7fe79c19c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:36.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.302+0000 7fe7a1010640 1 --2- 172.21.15.64:0/4047365913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe79c110180 0x7fe79c1a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:36.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.302+0000 7fe79bfff640 1 -- 172.21.15.64:0/4047365913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe79c1025d0 msgr2=0x7fe79c19c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:36.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.302+0000 7fe79bfff640 1 --2- 172.21.15.64:0/4047365913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe79c1025d0 0x7fe79c19c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.302+0000 7fe79bfff640 1 -- 172.21.15.64:0/4047365913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe79c110180 msgr2=0x7fe79c1a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:36.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.302+0000 7fe79bfff640 1 --2- 172.21.15.64:0/4047365913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe79c110180 0x7fe79c1a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.302+0000 7fe79bfff640 1 -- 172.21.15.64:0/4047365913 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79c1a2540 con 0x7fe79c102f20 2024-09-21T04:11:36.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.302+0000 7fe79bfff640 1 --2- 172.21.15.64:0/4047365913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe79c102f20 0x7fe79c19cb70 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe78400e9f0 tx=0x7fe78400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:36.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.302+0000 7fe799ffb640 1 -- 172.21.15.64:0/4047365913 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78400cd80 con 0x7fe79c102f20 2024-09-21T04:11:36.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.302+0000 7fe799ffb640 1 -- 172.21.15.64:0/4047365913 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7840108e0 con 0x7fe79c102f20 2024-09-21T04:11:36.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.302+0000 7fe7a2a73640 1 -- 172.21.15.64:0/4047365913 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79c1a2830 con 0x7fe79c102f20 2024-09-21T04:11:36.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.302+0000 7fe799ffb640 1 -- 172.21.15.64:0/4047365913 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe784018980 con 0x7fe79c102f20 2024-09-21T04:11:36.727 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.303+0000 7fe7a2a73640 1 -- 172.21.15.64:0/4047365913 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe79c1a8f90 con 0x7fe79c102f20 2024-09-21T04:11:36.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.305+0000 7fe7a8fe7740 1 Processor -- start 2024-09-21T04:11:36.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.305+0000 7fe7a8fe7740 1 -- start start 2024-09-21T04:11:36.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.305+0000 7fe7a8fe7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560622ef3070 0x56062329a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:36.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.305+0000 7fe799ffb640 1 -- 172.21.15.64:0/4047365913 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fe784010430 con 0x7fe79c102f20 2024-09-21T04:11:36.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.305+0000 7fe7a8fe7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622eca1b0 0x56062329a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:36.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.305+0000 7fe7a8fe7740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560622ed2390 0x56062329ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:36.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.306+0000 7fe7a8fe7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560623210ea0 con 0x560622eca1b0 2024-09-21T04:11:36.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.306+0000 7fe7a8fe7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5606232002d0 con 0x560622ef3070 2024-09-21T04:11:36.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.306+0000 7fe7a8fe7740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5606232040e0 con 0x560622ed2390 2024-09-21T04:11:36.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.306+0000 7fe7a080f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622eca1b0 0x56062329a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:36.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.306+0000 7fe7a1010640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560622ef3070 0x56062329a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:36.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.306+0000 7fe79bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560622ed2390 0x56062329ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:36.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.306+0000 7fe7a1010640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560622ef3070 0x56062329a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:55854/0 (socket says 172.21.15.64:55854) 2024-09-21T04:11:36.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.306+0000 7fe7a1010640 1 -- 172.21.15.64:0/3242300538 learned_addr learned my addr 172.21.15.64:0/3242300538 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:36.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.306+0000 7fe7a080f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622eca1b0 0x56062329a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46646/0 (socket says 172.21.15.64:46646) 2024-09-21T04:11:36.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.306+0000 7fe7a1010640 1 -- 172.21.15.64:0/3242300538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560622ed2390 msgr2=0x56062329ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:36.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.306+0000 7fe7a1010640 1 --2- 172.21.15.64:0/3242300538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560622ed2390 0x56062329ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.306+0000 7fe7a1010640 1 -- 172.21.15.64:0/3242300538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622eca1b0 msgr2=0x56062329a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:36.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.306+0000 7fe7a1010640 1 --2- 172.21.15.64:0/3242300538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622eca1b0 0x56062329a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.306+0000 7fe7a1010640 1 -- 172.21.15.64:0/3242300538 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560622f5e650 con 0x560622ef3070 2024-09-21T04:11:36.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.306+0000 7fe7a080f640 1 --2- 172.21.15.64:0/3242300538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622eca1b0 0x56062329a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:11:36.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.306+0000 7fe7a1010640 1 --2- 172.21.15.64:0/3242300538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560622ef3070 0x56062329a2c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe78c009ca0 tx=0x7fe78c02f4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b8633bc2144c7e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:36.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.307+0000 7fe792ffd640 1 -- 172.21.15.64:0/3242300538 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c048070 con 0x560622ef3070 2024-09-21T04:11:36.729 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.307+0000 7fe799ffb640 1 --2- 172.21.15.64:0/4047365913 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe7880b6700 0x7fe7880b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:36.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.307+0000 7fe792ffd640 1 -- 172.21.15.64:0/3242300538 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe78c003e20 con 0x560622ef3070 2024-09-21T04:11:36.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.307+0000 7fe792ffd640 1 -- 172.21.15.64:0/3242300538 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c034040 con 0x560622ef3070 2024-09-21T04:11:36.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.307+0000 7fe7a080f640 1 --2- 172.21.15.64:0/4047365913 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe7880b6700 0x7fe7880b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:36.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.307+0000 7fe799ffb640 1 -- 172.21.15.64:0/4047365913 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7fe7840e7a30 con 0x7fe79c102f20 2024-09-21T04:11:36.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.307+0000 7fe7a8fe7740 1 -- 172.21.15.64:0/3242300538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560622ef3070 msgr2=0x56062329a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:36.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.307+0000 7fe7a080f640 1 --2- 172.21.15.64:0/4047365913 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe7880b6700 0x7fe7880b8c10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x560623298f40 tx=0x7fe79400a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:36.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.308+0000 7fe7a8fe7740 1 --2- 172.21.15.64:0/3242300538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560622ef3070 0x56062329a2c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe78c009ca0 tx=0x7fe78c02f4e0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.308+0000 7fe7a8fe7740 1 -- 172.21.15.64:0/3242300538 shutdown_connections 2024-09-21T04:11:36.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.308+0000 7fe7a8fe7740 1 --2- 172.21.15.64:0/3242300538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560622ed2390 0x56062329ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.308+0000 7fe7a8fe7740 1 --2- 172.21.15.64:0/3242300538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622eca1b0 0x56062329a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.308+0000 7fe7a8fe7740 1 --2- 172.21.15.64:0/3242300538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560622ef3070 0x56062329a2c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.308+0000 7fe7a8fe7740 1 -- 172.21.15.64:0/3242300538 >> 172.21.15.64:0/3242300538 conn(0x560622ee7460 msgr2=0x5606232981b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:36.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.308+0000 7fe7a8fe7740 1 -- 172.21.15.64:0/3242300538 shutdown_connections 2024-09-21T04:11:36.730 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.308+0000 7fe7a8fe7740 1 -- 172.21.15.64:0/3242300538 wait complete. 2024-09-21T04:11:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.308+0000 7fe7a8fe7740 1 -- start start 2024-09-21T04:11:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.308+0000 7fe7a8fe7740 1 Processor -- start 2024-09-21T04:11:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.309+0000 7fe7a8fe7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560622eca1b0 0x5606232a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.309+0000 7fe7a8fe7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622ed2390 0x5606232a5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.309+0000 7fe7a8fe7740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560622ef3070 0x5606232b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.309+0000 7fe7a8fe7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560623061700 con 0x560622ed2390 2024-09-21T04:11:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.309+0000 7fe7a8fe7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56062319b160 con 0x560622eca1b0 2024-09-21T04:11:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.309+0000 7fe7a8fe7740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5606232a8a40 con 0x560622ef3070 2024-09-21T04:11:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.309+0000 7fe7a8fe7740 10 client.0 _ll_register_callbacks cb 0x5606231b49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:11:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.309+0000 7fe79bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622ed2390 0x5606232a5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.309+0000 7fe7a080f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560622eca1b0 0x5606232a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.309+0000 7fe7a080f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560622eca1b0 0x5606232a59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:55864/0 (socket says 172.21.15.64:55864) 2024-09-21T04:11:36.731 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.309+0000 7fe79bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622ed2390 0x5606232a5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46654/0 (socket says 172.21.15.64:46654) 2024-09-21T04:11:36.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.309+0000 7fe7a080f640 1 -- 172.21.15.64:0/862866681 learned_addr learned my addr 172.21.15.64:0/862866681 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:36.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.309+0000 7fe7a1010640 1 --2- 172.21.15.64:0/862866681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560622ef3070 0x5606232b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:36.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.309+0000 7fe7a080f640 1 -- 172.21.15.64:0/862866681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560622ef3070 msgr2=0x5606232b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:36.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.309+0000 7fe7a080f640 1 --2- 172.21.15.64:0/862866681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560622ef3070 0x5606232b1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.309+0000 7fe7a080f640 1 -- 172.21.15.64:0/862866681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622ed2390 msgr2=0x5606232a5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:36.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.309+0000 7fe7a080f640 1 --2- 172.21.15.64:0/862866681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622ed2390 0x5606232a5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.309+0000 7fe7a080f640 1 -- 172.21.15.64:0/862866681 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606231c8c90 con 0x560622eca1b0 2024-09-21T04:11:36.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.309+0000 7fe79bfff640 1 --2- 172.21.15.64:0/862866681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622ed2390 0x5606232a5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:11:36.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.309+0000 7fe7a1010640 1 --2- 172.21.15.64:0/862866681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560622ef3070 0x5606232b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:11:36.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.310+0000 7fe7a080f640 1 --2- 172.21.15.64:0/862866681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560622eca1b0 0x5606232a59d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe79401ca00 tx=0x7fe794004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aab7d11f692ba36 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:36.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.310+0000 7fe77bfff640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:11:36.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.310+0000 7fe77bfff640 1 -- 172.21.15.64:0/862866681 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79402c060 con 0x560622eca1b0 2024-09-21T04:11:36.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.310+0000 7fe77bfff640 1 -- 172.21.15.64:0/862866681 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7fe794016030 con 0x560622eca1b0 2024-09-21T04:11:36.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.310+0000 7fe7a8fe7740 1 -- 172.21.15.64:0/862866681 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe78c003e20 con 0x560622eca1b0 2024-09-21T04:11:36.732 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.310+0000 7fe77bfff640 1 -- 172.21.15.64:0/862866681 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe794027420 con 0x560622eca1b0 2024-09-21T04:11:36.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.310+0000 7fe7a8fe7740 10 client.26729 fetch_fsmap learned FSMap version 39 2024-09-21T04:11:36.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.310+0000 7fe7a8fe7740 1 -- 172.21.15.64:0/862866681 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5606232b4860 con 0x560622eca1b0 2024-09-21T04:11:36.733 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.310+0000 7fe77bfff640 1 -- 172.21.15.64:0/862866681 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe78c003e20 con 0x560622eca1b0 2024-09-21T04:11:36.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.310+0000 7fe77bfff640 1 -- 172.21.15.64:0/862866681 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fe79401a560 con 0x560622eca1b0 2024-09-21T04:11:36.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.310+0000 7fe7a8fe7740 10 client.26729 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T04:11:36.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.310+0000 7fe7a8fe7740 10 client.26729 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:11:36.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.311+0000 7fe7a8fe7740 20 client.26729 populate_metadata read hostname 'smithi064' 2024-09-21T04:11:36.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.311+0000 7fe7a8fe7740 1 --2- 172.21.15.64:0/862866681 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x560622eb19f0 0x5606232b7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:36.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.311+0000 7fe7a8fe7740 4 client.26729 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:11:36.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.311+0000 7fe7a8fe7740 1 -- 172.21.15.64:0/862866681 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560623212df0 con 0x560622eb19f0 2024-09-21T04:11:36.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.311+0000 7fe79bfff640 1 --2- 172.21.15.64:0/862866681 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x560622eb19f0 0x5606232b7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:36.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.311+0000 7fe79bfff640 1 --2- 172.21.15.64:0/862866681 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x560622eb19f0 0x5606232b7cc0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b56273f3cfd9705 server_cookie=26370d9f37e2f775 in_seq=0 out_seq=0 2024-09-21T04:11:36.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.311+0000 7fe77bfff640 10 client.26729 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:11:36.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.312+0000 7fe77bfff640 1 -- 172.21.15.64:0/862866681 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560623212df0 con 0x560622eb19f0 2024-09-21T04:11:36.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.312+0000 7fe77bfff640 10 client.26729 handle_command_reply: tid=0 2024-09-21T04:11:36.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.312+0000 7fe7a8fe7740 1 client.26729 shutdown 2024-09-21T04:11:36.734 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.312+0000 7fe7a8fe7740 1 -- 172.21.15.64:0/862866681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560622eca1b0 msgr2=0x5606232a59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:36.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.312+0000 7fe7a8fe7740 1 --2- 172.21.15.64:0/862866681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560622eca1b0 0x5606232a59d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe79401ca00 tx=0x7fe794004730 comp rx=0 tx=0).stop 2024-09-21T04:11:36.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.312+0000 7fe7a8fe7740 1 -- 172.21.15.64:0/862866681 shutdown_connections 2024-09-21T04:11:36.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.312+0000 7fe7a8fe7740 1 --2- 172.21.15.64:0/862866681 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x560622eb19f0 0x5606232b7cc0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.312+0000 7fe7a8fe7740 1 --2- 172.21.15.64:0/862866681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560622ef3070 0x5606232b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.312+0000 7fe7a8fe7740 1 --2- 172.21.15.64:0/862866681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622ed2390 0x5606232a5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.312+0000 7fe77bfff640 0 client.26729 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:11:36.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.312+0000 7fe7a8fe7740 1 --2- 172.21.15.64:0/862866681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560622eca1b0 0x5606232a59d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.312+0000 7fe7a8fe7740 1 -- 172.21.15.64:0/862866681 >> 172.21.15.64:0/862866681 conn(0x560622ee7460 msgr2=0x56062329d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:36.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.312+0000 7fe7a8fe7740 1 -- 172.21.15.64:0/862866681 shutdown_connections 2024-09-21T04:11:36.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.312+0000 7fe7a8fe7740 1 -- 172.21.15.64:0/862866681 wait complete. 2024-09-21T04:11:36.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.312+0000 7fe7a8fe7740 20 client.26729 trim_cache size 0 max 16384 2024-09-21T04:11:36.737 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:36 smithi131 ceph-mon[35193]: pgmap v552: 97 pgs: 97 active+clean; 818 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 340 op/s 2024-09-21T04:11:36.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.326+0000 7fe7a8fe7740 1 Processor -- start 2024-09-21T04:11:36.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.326+0000 7fe7a8fe7740 1 -- start start 2024-09-21T04:11:36.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.326+0000 7fe7a8fe7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622eca1b0 0x5606232a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:36.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.326+0000 7fe7a8fe7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560622ed2390 0x56062329d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:36.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.326+0000 7fe7a8fe7740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560622ef3070 0x56062329d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:36.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.326+0000 7fe7a8fe7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c1b4300 con 0x560622eca1b0 2024-09-21T04:11:36.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.326+0000 7fe7a8fe7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c1b1b40 con 0x560622ed2390 2024-09-21T04:11:36.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.326+0000 7fe7a8fe7740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c1af9b0 con 0x560622ef3070 2024-09-21T04:11:36.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.326+0000 7fe7a080f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622eca1b0 0x5606232a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:36.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.327+0000 7fe7a080f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622eca1b0 0x5606232a5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46664/0 (socket says 172.21.15.64:46664) 2024-09-21T04:11:36.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.327+0000 7fe7a080f640 1 -- 172.21.15.64:0/2093221092 learned_addr learned my addr 172.21.15.64:0/2093221092 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:36.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.327+0000 7fe7a1010640 1 --2- 172.21.15.64:0/2093221092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560622ef3070 0x56062329d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:36.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.327+0000 7fe79bfff640 1 --2- 172.21.15.64:0/2093221092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560622ed2390 0x56062329d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:36.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.327+0000 7fe7a080f640 1 -- 172.21.15.64:0/2093221092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560622ef3070 msgr2=0x56062329d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:36.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.327+0000 7fe7a080f640 1 --2- 172.21.15.64:0/2093221092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560622ef3070 0x56062329d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.327+0000 7fe7a080f640 1 -- 172.21.15.64:0/2093221092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560622ed2390 msgr2=0x56062329d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:36.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.327+0000 7fe7a080f640 1 --2- 172.21.15.64:0/2093221092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560622ed2390 0x56062329d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.327+0000 7fe7a080f640 1 -- 172.21.15.64:0/2093221092 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56062329c7e0 con 0x560622eca1b0 2024-09-21T04:11:36.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.327+0000 7fe7a1010640 1 --2- 172.21.15.64:0/2093221092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560622ef3070 0x56062329d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:11:36.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.327+0000 7fe79bfff640 1 --2- 172.21.15.64:0/2093221092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560622ed2390 0x56062329d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:11:36.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.327+0000 7fe7a080f640 1 --2- 172.21.15.64:0/2093221092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622eca1b0 0x5606232a5e90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe79401c9d0 tx=0x7fe79401add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c1686ea0595822 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:36.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.327+0000 7fe7927fc640 1 -- 172.21.15.64:0/2093221092 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79401a3a0 con 0x560622eca1b0 2024-09-21T04:11:36.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.327+0000 7fe7927fc640 1 -- 172.21.15.64:0/2093221092 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7940054d0 con 0x560622eca1b0 2024-09-21T04:11:36.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.327+0000 7fe7927fc640 1 -- 172.21.15.64:0/2093221092 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794004c40 con 0x560622eca1b0 2024-09-21T04:11:36.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.328+0000 7fe7a8fe7740 1 -- 172.21.15.64:0/2093221092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622eca1b0 msgr2=0x5606232a5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:36.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.328+0000 7fe7a8fe7740 1 --2- 172.21.15.64:0/2093221092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622eca1b0 0x5606232a5e90 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe79401c9d0 tx=0x7fe79401add0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.328+0000 7fe7a8fe7740 1 -- 172.21.15.64:0/2093221092 shutdown_connections 2024-09-21T04:11:36.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.328+0000 7fe7a8fe7740 1 --2- 172.21.15.64:0/2093221092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560622ef3070 0x56062329d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.328+0000 7fe7a8fe7740 1 --2- 172.21.15.64:0/2093221092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560622ed2390 0x56062329d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.328+0000 7fe7a8fe7740 1 --2- 172.21.15.64:0/2093221092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622eca1b0 0x5606232a5e90 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.328+0000 7fe7a8fe7740 1 -- 172.21.15.64:0/2093221092 >> 172.21.15.64:0/2093221092 conn(0x560622ee7460 msgr2=0x5606232a5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:36.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.329+0000 7fe7a8fe7740 1 -- 172.21.15.64:0/2093221092 shutdown_connections 2024-09-21T04:11:36.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.329+0000 7fe7a8fe7740 1 -- 172.21.15.64:0/2093221092 wait complete. 2024-09-21T04:11:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.329+0000 7fe7a8fe7740 1 -- start start 2024-09-21T04:11:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.329+0000 7fe7a8fe7740 1 Processor -- start 2024-09-21T04:11:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.330+0000 7fe7a8fe7740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622eca1b0 0x5606232ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.330+0000 7fe7a8fe7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560622ed2390 0x5606232a9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.330+0000 7fe7a8fe7740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560622ef3070 0x5606232a9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.330+0000 7fe7a8fe7740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c1b0000 con 0x560622eca1b0 2024-09-21T04:11:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.330+0000 7fe7a8fe7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c1b1470 con 0x560622ed2390 2024-09-21T04:11:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.330+0000 7fe7a8fe7740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c1b06d0 con 0x560622ef3070 2024-09-21T04:11:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.330+0000 7fe7a8fe7740 10 client.0 _ll_register_callbacks cb 0x7fe79401c890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:11:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.330+0000 7fe7a080f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622eca1b0 0x5606232ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.330+0000 7fe7a080f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622eca1b0 0x5606232ae4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46670/0 (socket says 172.21.15.64:46670) 2024-09-21T04:11:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.330+0000 7fe7a080f640 1 -- 172.21.15.64:0/2292706763 learned_addr learned my addr 172.21.15.64:0/2292706763 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.330+0000 7fe7a1010640 1 --2- 172.21.15.64:0/2292706763 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560622ef3070 0x5606232a9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:36.744 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.330+0000 7fe79bfff640 1 --2- 172.21.15.64:0/2292706763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560622ed2390 0x5606232a9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.330+0000 7fe7a080f640 1 -- 172.21.15.64:0/2292706763 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560622ef3070 msgr2=0x5606232a9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.330+0000 7fe7a080f640 1 --2- 172.21.15.64:0/2292706763 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560622ef3070 0x5606232a9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.330+0000 7fe7a080f640 1 -- 172.21.15.64:0/2292706763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560622ed2390 msgr2=0x5606232a9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.330+0000 7fe7a080f640 1 --2- 172.21.15.64:0/2292706763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560622ed2390 0x5606232a9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.330+0000 7fe7a080f640 1 -- 172.21.15.64:0/2292706763 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606232aa610 con 0x560622eca1b0 2024-09-21T04:11:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.330+0000 7fe7a080f640 1 --2- 172.21.15.64:0/2292706763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622eca1b0 0x5606232ae4b0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe79401aa50 tx=0x7fe79400a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1d483d106f6f5b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.330+0000 7fe77bfff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:11:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.331+0000 7fe77bfff640 1 -- 172.21.15.64:0/2292706763 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794045060 con 0x560622eca1b0 2024-09-21T04:11:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.331+0000 7fe77bfff640 1 -- 172.21.15.64:0/2292706763 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7fe79401a3a0 con 0x560622eca1b0 2024-09-21T04:11:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.331+0000 7fe7a8fe7740 1 -- 172.21.15.64:0/2292706763 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7940054d0 con 0x560622eca1b0 2024-09-21T04:11:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.331+0000 7fe77bfff640 1 -- 172.21.15.64:0/2292706763 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe794004840 con 0x560622eca1b0 2024-09-21T04:11:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.331+0000 7fe7a8fe7740 10 client.17061 fetch_fsmap learned FSMap version 39 2024-09-21T04:11:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.331+0000 7fe7a8fe7740 1 -- 172.21.15.64:0/2292706763 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560623322550 con 0x560622eca1b0 2024-09-21T04:11:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.331+0000 7fe77bfff640 1 -- 172.21.15.64:0/2292706763 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7940054d0 con 0x560622eca1b0 2024-09-21T04:11:36.745 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.331+0000 7fe77bfff640 1 -- 172.21.15.64:0/2292706763 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fe794036030 con 0x560622eca1b0 2024-09-21T04:11:36.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.332+0000 7fe7a8fe7740 10 client.17061 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T04:11:36.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.332+0000 7fe7a8fe7740 10 client.17061 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:11:36.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.332+0000 7fe7a8fe7740 20 client.17061 populate_metadata read hostname 'smithi064' 2024-09-21T04:11:36.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.332+0000 7fe7a8fe7740 1 --2- 172.21.15.64:0/2292706763 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x560622e86da0 0x560623324c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:36.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.332+0000 7fe7a8fe7740 4 client.17061 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:11:36.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.332+0000 7fe7a8fe7740 1 -- 172.21.15.64:0/2292706763 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5606232a1580 con 0x560622e86da0 2024-09-21T04:11:36.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.332+0000 7fe79bfff640 1 --2- 172.21.15.64:0/2292706763 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x560622e86da0 0x560623324c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:36.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.332+0000 7fe77bfff640 10 client.17061 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:11:36.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.332+0000 7fe79bfff640 1 --2- 172.21.15.64:0/2292706763 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x560622e86da0 0x560623324c90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1a57de6914366206 server_cookie=bc122d22aa5cc339 in_seq=0 out_seq=0 2024-09-21T04:11:36.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.333+0000 7fe77bfff640 1 -- 172.21.15.64:0/2292706763 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5606232a1580 con 0x560622e86da0 2024-09-21T04:11:36.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.333+0000 7fe77bfff640 10 client.17061 handle_command_reply: tid=0 2024-09-21T04:11:36.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.333+0000 7fe7a8fe7740 1 client.17061 shutdown 2024-09-21T04:11:36.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.333+0000 7fe7a8fe7740 1 -- 172.21.15.64:0/2292706763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622eca1b0 msgr2=0x5606232ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:36.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.333+0000 7fe7a8fe7740 1 --2- 172.21.15.64:0/2292706763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622eca1b0 0x5606232ae4b0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe79401aa50 tx=0x7fe79400a640 comp rx=0 tx=0).stop 2024-09-21T04:11:36.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.333+0000 7fe7a8fe7740 1 -- 172.21.15.64:0/2292706763 shutdown_connections 2024-09-21T04:11:36.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.333+0000 7fe7a8fe7740 1 --2- 172.21.15.64:0/2292706763 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x560622e86da0 0x560623324c90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.333+0000 7fe7a8fe7740 1 --2- 172.21.15.64:0/2292706763 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560622ef3070 0x5606232a9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.333+0000 7fe7a8fe7740 1 --2- 172.21.15.64:0/2292706763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560622ed2390 0x5606232a9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.333+0000 7fe7a8fe7740 1 --2- 172.21.15.64:0/2292706763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560622eca1b0 0x5606232ae4b0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.333+0000 7fe77bfff640 0 client.17061 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:11:36.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.333+0000 7fe7a8fe7740 1 -- 172.21.15.64:0/2292706763 >> 172.21.15.64:0/2292706763 conn(0x560622ee7460 msgr2=0x5606232a3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:36.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.333+0000 7fe7a8fe7740 1 -- 172.21.15.64:0/2292706763 shutdown_connections 2024-09-21T04:11:36.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.333+0000 7fe7a8fe7740 1 -- 172.21.15.64:0/2292706763 wait complete. 2024-09-21T04:11:36.747 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.333+0000 7fe7a8fe7740 20 client.17061 trim_cache size 0 max 16384 2024-09-21T04:11:36.747 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:11:36.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.334+0000 7fe7a2a73640 1 -- 172.21.15.64:0/4047365913 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe7880b6700 msgr2=0x7fe7880b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:36.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.334+0000 7fe7a2a73640 1 --2- 172.21.15.64:0/4047365913 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe7880b6700 0x7fe7880b8c10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x560623298f40 tx=0x7fe79400a2d0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.334+0000 7fe7a2a73640 1 -- 172.21.15.64:0/4047365913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe79c102f20 msgr2=0x7fe79c19cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:36.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.334+0000 7fe7a2a73640 1 --2- 172.21.15.64:0/4047365913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe79c102f20 0x7fe79c19cb70 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe78400e9f0 tx=0x7fe78400eeb0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.334+0000 7fe7a2a73640 1 -- 172.21.15.64:0/4047365913 shutdown_connections 2024-09-21T04:11:36.748 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.334+0000 7fe7a2a73640 1 --2- 172.21.15.64:0/4047365913 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe7880b6700 0x7fe7880b8c10 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.334+0000 7fe7a2a73640 1 --2- 172.21.15.64:0/4047365913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe79c110180 0x7fe79c1a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.334+0000 7fe7a2a73640 1 --2- 172.21.15.64:0/4047365913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe79c102f20 0x7fe79c19cb70 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.334+0000 7fe7a2a73640 1 --2- 172.21.15.64:0/4047365913 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe79c1025d0 0x7fe79c19c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:36.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.334+0000 7fe7a2a73640 1 -- 172.21.15.64:0/4047365913 >> 172.21.15.64:0/4047365913 conn(0x7fe79c079730 msgr2=0x7fe79c109c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:36.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.334+0000 7fe7a2a73640 1 -- 172.21.15.64:0/4047365913 shutdown_connections 2024-09-21T04:11:36.749 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:36.334+0000 7fe7a2a73640 1 -- 172.21.15.64:0/4047365913 wait complete. 2024-09-21T04:11:36.749 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:36 smithi064 ceph-mon[31180]: pgmap v552: 97 pgs: 97 active+clean; 818 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 340 op/s 2024-09-21T04:11:36.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:36 smithi119 ceph-mon[34811]: pgmap v552: 97 pgs: 97 active+clean; 818 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 340 op/s 2024-09-21T04:11:37.452 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T04:11:37.710 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:11:38.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.233+0000 7fcfbbf24640 1 -- 172.21.15.64:0/2349157673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfb41094b0 msgr2=0x7fcfb41098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:38.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.233+0000 7fcfbbf24640 1 --2- 172.21.15.64:0/2349157673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfb41094b0 0x7fcfb41098c0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009900 tx=0x7fcfa402a410 comp rx=0 tx=0).stop 2024-09-21T04:11:38.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.233+0000 7fcfbbf24640 1 -- 172.21.15.64:0/2349157673 shutdown_connections 2024-09-21T04:11:38.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.233+0000 7fcfbbf24640 1 --2- 172.21.15.64:0/2349157673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfb4109e00 0x7fcfb4110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.233+0000 7fcfbbf24640 1 --2- 172.21.15.64:0/2349157673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfb41094b0 0x7fcfb41098c0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.233+0000 7fcfbbf24640 1 --2- 172.21.15.64:0/2349157673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfb4101f70 0x7fcfb4102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.233+0000 7fcfbbf24640 1 -- 172.21.15.64:0/2349157673 >> 172.21.15.64:0/2349157673 conn(0x7fcfb4077c60 msgr2=0x7fcfb40780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:38.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.233+0000 7fcfbbf24640 1 -- 172.21.15.64:0/2349157673 shutdown_connections 2024-09-21T04:11:38.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.233+0000 7fcfbbf24640 1 -- 172.21.15.64:0/2349157673 wait complete. 2024-09-21T04:11:38.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.234+0000 7fcfbbf24640 1 Processor -- start 2024-09-21T04:11:38.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.234+0000 7fcfbbf24640 1 -- start start 2024-09-21T04:11:38.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.234+0000 7fcfbbf24640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfb4101f70 0x7fcfb419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:38.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.234+0000 7fcfbbf24640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfb41094b0 0x7fcfb419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:38.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.234+0000 7fcfbbf24640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfb4109e00 0x7fcfb419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:38.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.234+0000 7fcfbbf24640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb4113ac0 con 0x7fcfb4101f70 2024-09-21T04:11:38.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.234+0000 7fcfbbf24640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb4113940 con 0x7fcfb41094b0 2024-09-21T04:11:38.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.234+0000 7fcfbbf24640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb4113c40 con 0x7fcfb4109e00 2024-09-21T04:11:38.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.234+0000 7fcfb9cc0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfb4101f70 0x7fcfb419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:38.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.235+0000 7fcfb9cc0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfb4101f70 0x7fcfb419a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46692/0 (socket says 172.21.15.64:46692) 2024-09-21T04:11:38.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.235+0000 7fcfb94bf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfb41094b0 0x7fcfb419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:38.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.235+0000 7fcfb9cc0640 1 -- 172.21.15.64:0/1751541354 learned_addr learned my addr 172.21.15.64:0/1751541354 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:38.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.235+0000 7fcfba4c1640 1 --2- 172.21.15.64:0/1751541354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfb4109e00 0x7fcfb419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:38.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.235+0000 7fcfb9cc0640 1 -- 172.21.15.64:0/1751541354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfb4109e00 msgr2=0x7fcfb419fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:38.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.235+0000 7fcfb9cc0640 1 --2- 172.21.15.64:0/1751541354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfb4109e00 0x7fcfb419fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.235+0000 7fcfb9cc0640 1 -- 172.21.15.64:0/1751541354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfb41094b0 msgr2=0x7fcfb419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:38.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.235+0000 7fcfb9cc0640 1 --2- 172.21.15.64:0/1751541354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfb41094b0 0x7fcfb419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.235+0000 7fcfb9cc0640 1 -- 172.21.15.64:0/1751541354 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb41a0460 con 0x7fcfb4101f70 2024-09-21T04:11:38.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.235+0000 7fcfb94bf640 1 --2- 172.21.15.64:0/1751541354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfb41094b0 0x7fcfb419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:11:38.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.235+0000 7fcfb9cc0640 1 --2- 172.21.15.64:0/1751541354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfb4101f70 0x7fcfb419a4f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000e9f0 tx=0x7fcfb000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:38.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.235+0000 7fcfaaffd640 1 -- 172.21.15.64:0/1751541354 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb000cd80 con 0x7fcfb4101f70 2024-09-21T04:11:38.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.235+0000 7fcfaaffd640 1 -- 172.21.15.64:0/1751541354 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfb00108e0 con 0x7fcfb4101f70 2024-09-21T04:11:38.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.235+0000 7fcfbbf24640 1 -- 172.21.15.64:0/1751541354 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb41a0750 con 0x7fcfb4101f70 2024-09-21T04:11:38.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.235+0000 7fcfaaffd640 1 -- 172.21.15.64:0/1751541354 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0018980 con 0x7fcfb4101f70 2024-09-21T04:11:38.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.235+0000 7fcfbbf24640 1 -- 172.21.15.64:0/1751541354 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb41a6eb0 con 0x7fcfb4101f70 2024-09-21T04:11:38.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.238+0000 7fcfc2498740 1 Processor -- start 2024-09-21T04:11:38.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.239+0000 7fcfc2498740 1 -- start start 2024-09-21T04:11:38.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.239+0000 7fcfc2498740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64e85b070 0x55c64ec026a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:38.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.239+0000 7fcfc2498740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c64e8321b0 0x55c64ec02be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:38.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.239+0000 7fcfaaffd640 1 -- 172.21.15.64:0/1751541354 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fcfb0010430 con 0x7fcfb4101f70 2024-09-21T04:11:38.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.239+0000 7fcfc2498740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c64e83a390 0x55c64ec03120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:38.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.239+0000 7fcfc2498740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c64eb78ea0 con 0x55c64e8321b0 2024-09-21T04:11:38.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.239+0000 7fcfc2498740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c64eb682d0 con 0x55c64e85b070 2024-09-21T04:11:38.250 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.239+0000 7fcfc2498740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c64eb6c0e0 con 0x55c64e83a390 2024-09-21T04:11:38.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.239+0000 7fcfb9cc0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c64e8321b0 0x55c64ec02be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:38.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.239+0000 7fcfba4c1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64e85b070 0x55c64ec026a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:38.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.239+0000 7fcfba4c1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64e85b070 0x55c64ec026a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:55908/0 (socket says 172.21.15.64:55908) 2024-09-21T04:11:38.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.239+0000 7fcfba4c1640 1 -- 172.21.15.64:0/3153696396 learned_addr learned my addr 172.21.15.64:0/3153696396 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:38.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.239+0000 7fcfb94bf640 1 --2- 172.21.15.64:0/3153696396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c64e83a390 0x55c64ec03120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:38.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.240+0000 7fcfb9cc0640 1 -- 172.21.15.64:0/3153696396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c64e83a390 msgr2=0x55c64ec03120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:38.251 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.240+0000 7fcfb9cc0640 1 --2- 172.21.15.64:0/3153696396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c64e83a390 0x55c64ec03120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.240+0000 7fcfb9cc0640 1 -- 172.21.15.64:0/3153696396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64e85b070 msgr2=0x55c64ec026a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:38.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.240+0000 7fcfb9cc0640 1 --2- 172.21.15.64:0/3153696396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64e85b070 0x55c64ec026a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.240+0000 7fcfb9cc0640 1 -- 172.21.15.64:0/3153696396 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c64e8c6650 con 0x55c64e8321b0 2024-09-21T04:11:38.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.240+0000 7fcfba4c1640 1 --2- 172.21.15.64:0/3153696396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64e85b070 0x55c64ec026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:11:38.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.240+0000 7fcfb9cc0640 1 --2- 172.21.15.64:0/3153696396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c64e8321b0 0x55c64ec02be0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00e5120 tx=0x7fcfb00f0570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2551a83a92b139fd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:38.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.240+0000 7fcf9ffff640 1 -- 172.21.15.64:0/3153696396 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb00ebe40 con 0x55c64e8321b0 2024-09-21T04:11:38.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.240+0000 7fcf9ffff640 1 -- 172.21.15.64:0/3153696396 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfb00ee070 con 0x55c64e8321b0 2024-09-21T04:11:38.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.240+0000 7fcf9ffff640 1 -- 172.21.15.64:0/3153696396 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb00f4620 con 0x55c64e8321b0 2024-09-21T04:11:38.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.240+0000 7fcfaaffd640 1 --2- 172.21.15.64:0/1751541354 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcfa00b6700 0x7fcfa00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:38.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.240+0000 7fcfba4c1640 1 --2- 172.21.15.64:0/1751541354 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcfa00b6700 0x7fcfa00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:38.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.241+0000 7fcfaaffd640 1 -- 172.21.15.64:0/1751541354 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7fcfb00e7a30 con 0x7fcfb4101f70 2024-09-21T04:11:38.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.241+0000 7fcfc2498740 1 -- 172.21.15.64:0/3153696396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c64e8321b0 msgr2=0x55c64ec02be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:38.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.241+0000 7fcfc2498740 1 --2- 172.21.15.64:0/3153696396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c64e8321b0 0x55c64ec02be0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00e5120 tx=0x7fcfb00f0570 comp rx=0 tx=0).stop 2024-09-21T04:11:38.252 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.241+0000 7fcfba4c1640 1 --2- 172.21.15.64:0/1751541354 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcfa00b6700 0x7fcfa00b8c10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcfac010620 tx=0x7fcfac00a4d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:38.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.241+0000 7fcfc2498740 1 -- 172.21.15.64:0/3153696396 shutdown_connections 2024-09-21T04:11:38.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.241+0000 7fcfc2498740 1 --2- 172.21.15.64:0/3153696396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c64e83a390 0x55c64ec03120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.241+0000 7fcfc2498740 1 --2- 172.21.15.64:0/3153696396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c64e8321b0 0x55c64ec02be0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.241+0000 7fcfc2498740 1 --2- 172.21.15.64:0/3153696396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64e85b070 0x55c64ec026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.241+0000 7fcfc2498740 1 -- 172.21.15.64:0/3153696396 >> 172.21.15.64:0/3153696396 conn(0x55c64e84f460 msgr2=0x55c64ec00590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:38.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.242+0000 7fcfc2498740 1 -- 172.21.15.64:0/3153696396 shutdown_connections 2024-09-21T04:11:38.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.242+0000 7fcfc2498740 1 -- 172.21.15.64:0/3153696396 wait complete. 2024-09-21T04:11:38.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.242+0000 7fcfc2498740 1 -- start start 2024-09-21T04:11:38.253 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.242+0000 7fcfc2498740 1 Processor -- start 2024-09-21T04:11:38.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.243+0000 7fcfc2498740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c64e8321b0 0x55c64ec0dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:38.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.243+0000 7fcfc2498740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64e83a390 0x55c64ec0e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:38.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.243+0000 7fcfc2498740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c64e85b070 0x55c64ec1a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:38.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.243+0000 7fcfc2498740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c64e9c9700 con 0x55c64e8321b0 2024-09-21T04:11:38.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.243+0000 7fcfc2498740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c64eb03160 con 0x55c64e83a390 2024-09-21T04:11:38.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.243+0000 7fcfc2498740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c64ec10e80 con 0x55c64e85b070 2024-09-21T04:11:38.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.243+0000 7fcfc2498740 10 client.0 _ll_register_callbacks cb 0x55c64eb1c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:11:38.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.243+0000 7fcfb9cc0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c64e8321b0 0x55c64ec0dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:38.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.243+0000 7fcfb9cc0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c64e8321b0 0x55c64ec0dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46716/0 (socket says 172.21.15.64:46716) 2024-09-21T04:11:38.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.243+0000 7fcfb9cc0640 1 -- 172.21.15.64:0/1919172764 learned_addr learned my addr 172.21.15.64:0/1919172764 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:38.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.243+0000 7fcfb94bf640 1 --2- 172.21.15.64:0/1919172764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64e83a390 0x55c64ec0e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:38.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.243+0000 7fcfba4c1640 1 --2- 172.21.15.64:0/1919172764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c64e85b070 0x55c64ec1a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:38.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.243+0000 7fcfb9cc0640 1 -- 172.21.15.64:0/1919172764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c64e85b070 msgr2=0x55c64ec1a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:38.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.243+0000 7fcfb9cc0640 1 --2- 172.21.15.64:0/1919172764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c64e85b070 0x55c64ec1a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.243+0000 7fcfb9cc0640 1 -- 172.21.15.64:0/1919172764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64e83a390 msgr2=0x55c64ec0e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:38.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.243+0000 7fcfb9cc0640 1 --2- 172.21.15.64:0/1919172764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64e83a390 0x55c64ec0e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.243+0000 7fcfb9cc0640 1 -- 172.21.15.64:0/1919172764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c64eb30c90 con 0x55c64e8321b0 2024-09-21T04:11:38.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.243+0000 7fcfba4c1640 1 --2- 172.21.15.64:0/1919172764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c64e85b070 0x55c64ec1a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:11:38.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.243+0000 7fcfb9cc0640 1 --2- 172.21.15.64:0/1919172764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c64e8321b0 0x55c64ec0dda0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00ad860 tx=0x7fcfb0009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9ef5df3dc7f44f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:38.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.243+0000 7fcf9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:11:38.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.243+0000 7fcf9d7fa640 1 -- 172.21.15.64:0/1919172764 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb00a9f20 con 0x55c64e8321b0 2024-09-21T04:11:38.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.244+0000 7fcf9d7fa640 1 -- 172.21.15.64:0/1919172764 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7fcfb00a9b40 con 0x55c64e8321b0 2024-09-21T04:11:38.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.244+0000 7fcfc2498740 1 -- 172.21.15.64:0/1919172764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfb00ee070 con 0x55c64e8321b0 2024-09-21T04:11:38.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.244+0000 7fcf9d7fa640 1 -- 172.21.15.64:0/1919172764 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcfb00ad040 con 0x55c64e8321b0 2024-09-21T04:11:38.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.244+0000 7fcfc2498740 10 client.17073 fetch_fsmap learned FSMap version 39 2024-09-21T04:11:38.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.244+0000 7fcfc2498740 1 -- 172.21.15.64:0/1919172764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c64ec1cb70 con 0x55c64e8321b0 2024-09-21T04:11:38.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.244+0000 7fcf9d7fa640 1 -- 172.21.15.64:0/1919172764 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfb00ee070 con 0x55c64e8321b0 2024-09-21T04:11:38.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.244+0000 7fcf9d7fa640 1 -- 172.21.15.64:0/1919172764 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fcfb00ebc40 con 0x55c64e8321b0 2024-09-21T04:11:38.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.244+0000 7fcfc2498740 10 client.17073 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T04:11:38.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.244+0000 7fcfc2498740 10 client.17073 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:11:38.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.244+0000 7fcfc2498740 20 client.17073 populate_metadata read hostname 'smithi064' 2024-09-21T04:11:38.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.244+0000 7fcfc2498740 1 --2- 172.21.15.64:0/1919172764 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55c64e8199f0 0x55c64ec1fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:38.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.244+0000 7fcfc2498740 4 client.17073 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:11:38.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.244+0000 7fcfc2498740 1 -- 172.21.15.64:0/1919172764 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c64eb7adf0 con 0x55c64e8199f0 2024-09-21T04:11:38.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.244+0000 7fcfb94bf640 1 --2- 172.21.15.64:0/1919172764 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55c64e8199f0 0x55c64ec1fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:38.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.245+0000 7fcf9d7fa640 10 client.17073 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:11:38.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.245+0000 7fcfb94bf640 1 --2- 172.21.15.64:0/1919172764 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55c64e8199f0 0x55c64ec1fd00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e096703b66ee5a8f server_cookie=f249417805b11f59 in_seq=0 out_seq=0 2024-09-21T04:11:38.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.246+0000 7fcf9d7fa640 1 -- 172.21.15.64:0/1919172764 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c64eb7adf0 con 0x55c64e8199f0 2024-09-21T04:11:38.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.246+0000 7fcf9d7fa640 10 client.17073 handle_command_reply: tid=0 2024-09-21T04:11:38.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.246+0000 7fcfc2498740 1 client.17073 shutdown 2024-09-21T04:11:38.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.246+0000 7fcfc2498740 1 -- 172.21.15.64:0/1919172764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c64e8321b0 msgr2=0x55c64ec0dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:38.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.246+0000 7fcfc2498740 1 --2- 172.21.15.64:0/1919172764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c64e8321b0 0x55c64ec0dda0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00ad860 tx=0x7fcfb0009760 comp rx=0 tx=0).stop 2024-09-21T04:11:38.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.246+0000 7fcfc2498740 1 -- 172.21.15.64:0/1919172764 shutdown_connections 2024-09-21T04:11:38.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.246+0000 7fcfc2498740 1 --2- 172.21.15.64:0/1919172764 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55c64e8199f0 0x55c64ec1fd00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.246+0000 7fcfc2498740 1 --2- 172.21.15.64:0/1919172764 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c64e85b070 0x55c64ec1a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.246+0000 7fcfc2498740 1 --2- 172.21.15.64:0/1919172764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64e83a390 0x55c64ec0e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.246+0000 7fcf9d7fa640 0 client.17073 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:11:38.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.246+0000 7fcfc2498740 1 --2- 172.21.15.64:0/1919172764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c64e8321b0 0x55c64ec0dda0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.246+0000 7fcfc2498740 1 -- 172.21.15.64:0/1919172764 >> 172.21.15.64:0/1919172764 conn(0x55c64e84f460 msgr2=0x55c64ec05880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:38.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.246+0000 7fcfc2498740 1 -- 172.21.15.64:0/1919172764 shutdown_connections 2024-09-21T04:11:38.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.246+0000 7fcfc2498740 1 -- 172.21.15.64:0/1919172764 wait complete. 2024-09-21T04:11:38.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.246+0000 7fcfc2498740 20 client.17073 trim_cache size 0 max 16384 2024-09-21T04:11:38.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.260+0000 7fcfc2498740 1 Processor -- start 2024-09-21T04:11:38.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.260+0000 7fcfc2498740 1 -- start start 2024-09-21T04:11:38.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.260+0000 7fcfc2498740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c64e8321b0 0x55c64ec04090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:38.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.260+0000 7fcfc2498740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c64e83a390 0x55c64ec0f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:38.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.260+0000 7fcfc2498740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64e85b070 0x55c64ec1a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:38.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.260+0000 7fcfc2498740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41b2340 con 0x55c64e83a390 2024-09-21T04:11:38.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.260+0000 7fcfc2498740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41afb80 con 0x55c64e85b070 2024-09-21T04:11:38.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.260+0000 7fcfc2498740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41ad9f0 con 0x55c64e8321b0 2024-09-21T04:11:38.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.260+0000 7fcfb94bf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c64e83a390 0x55c64ec0f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:38.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.260+0000 7fcfb94bf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c64e83a390 0x55c64ec0f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46732/0 (socket says 172.21.15.64:46732) 2024-09-21T04:11:38.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.260+0000 7fcfb94bf640 1 -- 172.21.15.64:0/2912143819 learned_addr learned my addr 172.21.15.64:0/2912143819 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:38.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.260+0000 7fcfba4c1640 1 --2- 172.21.15.64:0/2912143819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64e85b070 0x55c64ec1a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:38.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.260+0000 7fcfb9cc0640 1 --2- 172.21.15.64:0/2912143819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c64e8321b0 0x55c64ec04090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:38.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.260+0000 7fcfb94bf640 1 -- 172.21.15.64:0/2912143819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c64e8321b0 msgr2=0x55c64ec04090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:38.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.260+0000 7fcfb94bf640 1 --2- 172.21.15.64:0/2912143819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c64e8321b0 0x55c64ec04090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.261+0000 7fcfb94bf640 1 -- 172.21.15.64:0/2912143819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64e85b070 msgr2=0x55c64ec1a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:38.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.261+0000 7fcfb94bf640 1 --2- 172.21.15.64:0/2912143819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64e85b070 0x55c64ec1a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.261+0000 7fcfb94bf640 1 -- 172.21.15.64:0/2912143819 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c64ec04990 con 0x55c64e83a390 2024-09-21T04:11:38.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.261+0000 7fcfb94bf640 1 --2- 172.21.15.64:0/2912143819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c64e83a390 0x55c64ec0f9d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40096c0 tx=0x7fcfa4031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5f16923c0c9e53 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:38.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.261+0000 7fcf9f7fe640 1 -- 172.21.15.64:0/2912143819 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4035040 con 0x55c64e83a390 2024-09-21T04:11:38.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.261+0000 7fcf9f7fe640 1 -- 172.21.15.64:0/2912143819 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfa40054f0 con 0x55c64e83a390 2024-09-21T04:11:38.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.261+0000 7fcf9f7fe640 1 -- 172.21.15.64:0/2912143819 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4004c00 con 0x55c64e83a390 2024-09-21T04:11:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.262+0000 7fcfc2498740 1 -- 172.21.15.64:0/2912143819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c64e83a390 msgr2=0x55c64ec0f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.262+0000 7fcfc2498740 1 --2- 172.21.15.64:0/2912143819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c64e83a390 0x55c64ec0f9d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40096c0 tx=0x7fcfa4031400 comp rx=0 tx=0).stop 2024-09-21T04:11:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.262+0000 7fcfc2498740 1 -- 172.21.15.64:0/2912143819 shutdown_connections 2024-09-21T04:11:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.262+0000 7fcfc2498740 1 --2- 172.21.15.64:0/2912143819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64e85b070 0x55c64ec1a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.262+0000 7fcfc2498740 1 --2- 172.21.15.64:0/2912143819 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c64e83a390 0x55c64ec0f9d0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.262+0000 7fcfc2498740 1 --2- 172.21.15.64:0/2912143819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c64e8321b0 0x55c64ec04090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.262+0000 7fcfc2498740 1 -- 172.21.15.64:0/2912143819 >> 172.21.15.64:0/2912143819 conn(0x55c64e84f460 msgr2=0x55c64ec0f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.262+0000 7fcfc2498740 1 -- 172.21.15.64:0/2912143819 shutdown_connections 2024-09-21T04:11:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.262+0000 7fcfc2498740 1 -- 172.21.15.64:0/2912143819 wait complete. 2024-09-21T04:11:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.262+0000 7fcfc2498740 1 -- start start 2024-09-21T04:11:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.262+0000 7fcfc2498740 1 Processor -- start 2024-09-21T04:11:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.263+0000 7fcfc2498740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64e8321b0 0x55c64ec21140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:38.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.263+0000 7fcfc2498740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c64e83a390 0x55c64ec216c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.263+0000 7fcfc2498740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c64e85b070 0x55c64ec89480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.263+0000 7fcfc2498740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41ae040 con 0x55c64e83a390 2024-09-21T04:11:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.263+0000 7fcfc2498740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41af4b0 con 0x55c64e8321b0 2024-09-21T04:11:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.263+0000 7fcfc2498740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb41ae710 con 0x55c64e85b070 2024-09-21T04:11:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.263+0000 7fcfc2498740 10 client.0 _ll_register_callbacks cb 0x7fcfac00a830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:11:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.263+0000 7fcfb94bf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c64e83a390 0x55c64ec216c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.263+0000 7fcfb94bf640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c64e83a390 0x55c64ec216c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46746/0 (socket says 172.21.15.64:46746) 2024-09-21T04:11:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.263+0000 7fcfb94bf640 1 -- 172.21.15.64:0/3315895946 learned_addr learned my addr 172.21.15.64:0/3315895946 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.263+0000 7fcfb9cc0640 1 --2- 172.21.15.64:0/3315895946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64e8321b0 0x55c64ec21140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.263+0000 7fcfba4c1640 1 --2- 172.21.15.64:0/3315895946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c64e85b070 0x55c64ec89480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.263+0000 7fcfb94bf640 1 -- 172.21.15.64:0/3315895946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c64e85b070 msgr2=0x55c64ec89480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.263+0000 7fcfb94bf640 1 --2- 172.21.15.64:0/3315895946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c64e85b070 0x55c64ec89480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.263+0000 7fcfb94bf640 1 -- 172.21.15.64:0/3315895946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64e8321b0 msgr2=0x55c64ec21140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:38.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.263+0000 7fcfb94bf640 1 --2- 172.21.15.64:0/3315895946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64e8321b0 0x55c64ec21140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.263+0000 7fcfb94bf640 1 -- 172.21.15.64:0/3315895946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c64ec899c0 con 0x55c64e83a390 2024-09-21T04:11:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.263+0000 7fcfb9cc0640 1 --2- 172.21.15.64:0/3315895946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64e8321b0 0x55c64ec21140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:11:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.264+0000 7fcfb94bf640 1 --2- 172.21.15.64:0/3315895946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c64e83a390 0x55c64ec216c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009420 tx=0x7fcfa4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bcffaaaf76661cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.264+0000 7fcf9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:11:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.264+0000 7fcf9d7fa640 1 -- 172.21.15.64:0/3315895946 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4053070 con 0x55c64e83a390 2024-09-21T04:11:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.264+0000 7fcf9d7fa640 1 -- 172.21.15.64:0/3315895946 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7fcfa40035f0 con 0x55c64e83a390 2024-09-21T04:11:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.264+0000 7fcfc2498740 1 -- 172.21.15.64:0/3315895946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfa40054f0 con 0x55c64e83a390 2024-09-21T04:11:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.264+0000 7fcf9d7fa640 1 -- 172.21.15.64:0/3315895946 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcfa4035040 con 0x55c64e83a390 2024-09-21T04:11:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.264+0000 7fcfc2498740 10 client.17079 fetch_fsmap learned FSMap version 39 2024-09-21T04:11:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.264+0000 7fcfc2498740 1 -- 172.21.15.64:0/3315895946 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c64ec8ab10 con 0x55c64e83a390 2024-09-21T04:11:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.264+0000 7fcf9d7fa640 1 -- 172.21.15.64:0/3315895946 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfa40054f0 con 0x55c64e83a390 2024-09-21T04:11:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.264+0000 7fcf9d7fa640 1 -- 172.21.15.64:0/3315895946 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fcfa403d9b0 con 0x55c64e83a390 2024-09-21T04:11:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.264+0000 7fcfc2498740 10 client.17079 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T04:11:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.264+0000 7fcfc2498740 10 client.17079 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:11:38.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.264+0000 7fcfc2498740 20 client.17079 populate_metadata read hostname 'smithi064' 2024-09-21T04:11:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.265+0000 7fcfc2498740 1 --2- 172.21.15.64:0/3315895946 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55c64e7eeda0 0x55c64ec8d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.265+0000 7fcfc2498740 4 client.17079 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:11:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.265+0000 7fcfc2498740 1 -- 172.21.15.64:0/3315895946 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c64eb768a0 con 0x55c64e7eeda0 2024-09-21T04:11:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.265+0000 7fcfb9cc0640 1 --2- 172.21.15.64:0/3315895946 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55c64e7eeda0 0x55c64ec8d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.265+0000 7fcf9d7fa640 10 client.17079 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:11:38.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.265+0000 7fcfb9cc0640 1 --2- 172.21.15.64:0/3315895946 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55c64e7eeda0 0x55c64ec8d240 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=19434b353cf07988 server_cookie=efe1f262765c402b in_seq=0 out_seq=0 2024-09-21T04:11:38.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.267+0000 7fcf9d7fa640 1 -- 172.21.15.64:0/3315895946 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c64eb768a0 con 0x55c64e7eeda0 2024-09-21T04:11:38.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.267+0000 7fcf9d7fa640 10 client.17079 handle_command_reply: tid=0 2024-09-21T04:11:38.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.267+0000 7fcfc2498740 1 client.17079 shutdown 2024-09-21T04:11:38.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.267+0000 7fcfc2498740 1 -- 172.21.15.64:0/3315895946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c64e83a390 msgr2=0x55c64ec216c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:38.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.267+0000 7fcfc2498740 1 --2- 172.21.15.64:0/3315895946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c64e83a390 0x55c64ec216c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009420 tx=0x7fcfa4004770 comp rx=0 tx=0).stop 2024-09-21T04:11:38.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.267+0000 7fcfc2498740 1 -- 172.21.15.64:0/3315895946 shutdown_connections 2024-09-21T04:11:38.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.267+0000 7fcfc2498740 1 --2- 172.21.15.64:0/3315895946 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55c64e7eeda0 0x55c64ec8d240 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.267+0000 7fcfc2498740 1 --2- 172.21.15.64:0/3315895946 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c64e85b070 0x55c64ec89480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.267+0000 7fcfc2498740 1 --2- 172.21.15.64:0/3315895946 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c64e83a390 0x55c64ec216c0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.267+0000 7fcf9d7fa640 0 client.17079 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:11:38.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.267+0000 7fcfc2498740 1 --2- 172.21.15.64:0/3315895946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64e8321b0 0x55c64ec21140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.267+0000 7fcfc2498740 1 -- 172.21.15.64:0/3315895946 >> 172.21.15.64:0/3315895946 conn(0x55c64e84f460 msgr2=0x55c64ec03e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:38.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.267+0000 7fcfc2498740 1 -- 172.21.15.64:0/3315895946 shutdown_connections 2024-09-21T04:11:38.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.267+0000 7fcfc2498740 1 -- 172.21.15.64:0/3315895946 wait complete. 2024-09-21T04:11:38.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.267+0000 7fcfc2498740 20 client.17079 trim_cache size 0 max 16384 2024-09-21T04:11:38.274 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:11:38.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.268+0000 7fcfbbf24640 1 -- 172.21.15.64:0/1751541354 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcfa00b6700 msgr2=0x7fcfa00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:38.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.268+0000 7fcfbbf24640 1 --2- 172.21.15.64:0/1751541354 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcfa00b6700 0x7fcfa00b8c10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcfac010620 tx=0x7fcfac00a4d0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.268+0000 7fcfbbf24640 1 -- 172.21.15.64:0/1751541354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfb4101f70 msgr2=0x7fcfb419a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:38.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.268+0000 7fcfbbf24640 1 --2- 172.21.15.64:0/1751541354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfb4101f70 0x7fcfb419a4f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000e9f0 tx=0x7fcfb000eeb0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.268+0000 7fcfbbf24640 1 -- 172.21.15.64:0/1751541354 shutdown_connections 2024-09-21T04:11:38.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.268+0000 7fcfbbf24640 1 --2- 172.21.15.64:0/1751541354 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcfa00b6700 0x7fcfa00b8c10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.268+0000 7fcfbbf24640 1 --2- 172.21.15.64:0/1751541354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfb4109e00 0x7fcfb419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.268+0000 7fcfbbf24640 1 --2- 172.21.15.64:0/1751541354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfb41094b0 0x7fcfb419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.268+0000 7fcfbbf24640 1 --2- 172.21.15.64:0/1751541354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcfb4101f70 0x7fcfb419a4f0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:38.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.268+0000 7fcfbbf24640 1 -- 172.21.15.64:0/1751541354 >> 172.21.15.64:0/1751541354 conn(0x7fcfb4077c60 msgr2=0x7fcfb41054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:38.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.268+0000 7fcfbbf24640 1 -- 172.21.15.64:0/1751541354 shutdown_connections 2024-09-21T04:11:38.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:38.268+0000 7fcfbbf24640 1 -- 172.21.15.64:0/1751541354 wait complete. 2024-09-21T04:11:38.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:38 smithi064 ceph-mon[31180]: pgmap v553: 97 pgs: 97 active+clean; 825 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.6 MiB/s wr, 546 op/s 2024-09-21T04:11:38.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:38 smithi131 ceph-mon[35193]: pgmap v553: 97 pgs: 97 active+clean; 825 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.6 MiB/s wr, 546 op/s 2024-09-21T04:11:38.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:38 smithi119 ceph-mon[34811]: pgmap v553: 97 pgs: 97 active+clean; 825 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.6 MiB/s wr, 546 op/s 2024-09-21T04:11:39.003 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T04:11:39.307 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:11:39.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.810+0000 7f1605730640 1 -- 172.21.15.64:0/960893813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16001083d0 msgr2=0x7f16001087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:39.813 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.810+0000 7f1605730640 1 --2- 172.21.15.64:0/960893813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16001083d0 0x7f16001087e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009900 tx=0x7f15e802a410 comp rx=0 tx=0).stop 2024-09-21T04:11:39.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.810+0000 7f1605730640 1 -- 172.21.15.64:0/960893813 shutdown_connections 2024-09-21T04:11:39.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.810+0000 7f1605730640 1 --2- 172.21.15.64:0/960893813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1600108d20 0x7f160010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.810+0000 7f1605730640 1 --2- 172.21.15.64:0/960893813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16001083d0 0x7f16001087e0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.810+0000 7f1605730640 1 --2- 172.21.15.64:0/960893813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1600102ae0 0x7f1600102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.810+0000 7f1605730640 1 -- 172.21.15.64:0/960893813 >> 172.21.15.64:0/960893813 conn(0x7f16000797b0 msgr2=0x7f1600079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:39.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.810+0000 7f1605730640 1 -- 172.21.15.64:0/960893813 shutdown_connections 2024-09-21T04:11:39.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.810+0000 7f1605730640 1 -- 172.21.15.64:0/960893813 wait complete. 2024-09-21T04:11:39.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.810+0000 7f1605730640 1 Processor -- start 2024-09-21T04:11:39.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.810+0000 7f1605730640 1 -- start start 2024-09-21T04:11:39.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.811+0000 7f1605730640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1600102ae0 0x7f16000ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:39.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.811+0000 7f1605730640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16001083d0 0x7f16000ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:39.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.811+0000 7f1605730640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1600108d20 0x7f16001000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:39.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.811+0000 7f1605730640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f16001129e0 con 0x7f1600108d20 2024-09-21T04:11:39.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.811+0000 7f1605730640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1600112860 con 0x7f1600102ae0 2024-09-21T04:11:39.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.811+0000 7f1605730640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1600112b60 con 0x7f16001083d0 2024-09-21T04:11:39.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.811+0000 7f15feffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1600102ae0 0x7f16000ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:39.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.811+0000 7f15ff7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1600108d20 0x7f16001000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:39.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.811+0000 7f15f7fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16001083d0 0x7f16000ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:39.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.811+0000 7f15feffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1600102ae0 0x7f16000ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:55948/0 (socket says 172.21.15.64:55948) 2024-09-21T04:11:39.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.811+0000 7f15feffd640 1 -- 172.21.15.64:0/1885150816 learned_addr learned my addr 172.21.15.64:0/1885150816 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:39.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.811+0000 7f15ff7fe640 1 -- 172.21.15.64:0/1885150816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16001083d0 msgr2=0x7f16000ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:39.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.811+0000 7f15ff7fe640 1 --2- 172.21.15.64:0/1885150816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16001083d0 0x7f16000ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.811+0000 7f15ff7fe640 1 -- 172.21.15.64:0/1885150816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1600102ae0 msgr2=0x7f16000ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:39.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.811+0000 7f15ff7fe640 1 --2- 172.21.15.64:0/1885150816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1600102ae0 0x7f16000ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.811+0000 7f15ff7fe640 1 -- 172.21.15.64:0/1885150816 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16001a8780 con 0x7f1600108d20 2024-09-21T04:11:39.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.811+0000 7f15feffd640 1 --2- 172.21.15.64:0/1885150816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1600102ae0 0x7f16000ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:11:39.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.812+0000 7f15ff7fe640 1 --2- 172.21.15.64:0/1885150816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1600108d20 0x7f16001000f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f15f8009fd0 tx=0x7f15f800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:39.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.812+0000 7f15fcff9640 1 -- 172.21.15.64:0/1885150816 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f800ce20 con 0x7f1600108d20 2024-09-21T04:11:39.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.812+0000 7f1605730640 1 -- 172.21.15.64:0/1885150816 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16001a8a70 con 0x7f1600108d20 2024-09-21T04:11:39.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.812+0000 7f15fcff9640 1 -- 172.21.15.64:0/1885150816 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15f8005020 con 0x7f1600108d20 2024-09-21T04:11:39.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.812+0000 7f15fcff9640 1 -- 172.21.15.64:0/1885150816 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f8010690 con 0x7f1600108d20 2024-09-21T04:11:39.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.812+0000 7f1605730640 1 -- 172.21.15.64:0/1885150816 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16001a8fd0 con 0x7f1600108d20 2024-09-21T04:11:39.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.814+0000 7f160bca4740 1 Processor -- start 2024-09-21T04:11:39.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.815+0000 7f15fcff9640 1 -- 172.21.15.64:0/1885150816 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f15f80054d0 con 0x7f1600108d20 2024-09-21T04:11:39.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.815+0000 7f160bca4740 1 -- start start 2024-09-21T04:11:39.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.815+0000 7f160bca4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a97266070 0x556a9760d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:39.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.815+0000 7f160bca4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9723d1b0 0x556a9760dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:39.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.815+0000 7f160bca4740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a97245390 0x556a9760e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:39.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.815+0000 7f160bca4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556a97583ea0 con 0x556a9723d1b0 2024-09-21T04:11:39.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.815+0000 7f160bca4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556a975732d0 con 0x556a97266070 2024-09-21T04:11:39.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.815+0000 7f160bca4740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556a975770e0 con 0x556a97245390 2024-09-21T04:11:39.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.815+0000 7f15f7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a97266070 0x556a9760d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:39.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.815+0000 7f15f7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a97266070 0x556a9760d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:55960/0 (socket says 172.21.15.64:55960) 2024-09-21T04:11:39.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.815+0000 7f15f7fff640 1 -- 172.21.15.64:0/2637253833 learned_addr learned my addr 172.21.15.64:0/2637253833 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:39.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.815+0000 7f15feffd640 1 --2- 172.21.15.64:0/2637253833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9723d1b0 0x556a9760dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:39.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.816+0000 7f15f7fff640 1 --2- 172.21.15.64:0/2637253833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a97245390 0x556a9760e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:39.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.816+0000 7f15feffd640 1 -- 172.21.15.64:0/2637253833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a97245390 msgr2=0x556a9760e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:39.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.816+0000 7f15feffd640 1 --2- 172.21.15.64:0/2637253833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a97245390 0x556a9760e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.816+0000 7f15feffd640 1 -- 172.21.15.64:0/2637253833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a97266070 msgr2=0x556a9760d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:39.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.816+0000 7f15feffd640 1 --2- 172.21.15.64:0/2637253833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a97266070 0x556a9760d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.816+0000 7f15feffd640 1 -- 172.21.15.64:0/2637253833 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a972d1650 con 0x556a9723d1b0 2024-09-21T04:11:39.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.816+0000 7f15f7fff640 1 --2- 172.21.15.64:0/2637253833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a97245390 0x556a9760e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:11:39.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.816+0000 7f15feffd640 1 --2- 172.21.15.64:0/2637253833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9723d1b0 0x556a9760dbe0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f16001010c0 tx=0x7f15f000f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb8f9a609385c528 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:39.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.816+0000 7f15f57fa640 1 -- 172.21.15.64:0/2637253833 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0004030 con 0x556a9723d1b0 2024-09-21T04:11:39.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.816+0000 7f15f57fa640 1 -- 172.21.15.64:0/2637253833 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15f0005020 con 0x556a9723d1b0 2024-09-21T04:11:39.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.816+0000 7f15f57fa640 1 -- 172.21.15.64:0/2637253833 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0004c60 con 0x556a9723d1b0 2024-09-21T04:11:39.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.816+0000 7f15fcff9640 1 --2- 172.21.15.64:0/1885150816 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f15ec0b6700 0x7f15ec0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:39.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.817+0000 7f15f7fff640 1 --2- 172.21.15.64:0/1885150816 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f15ec0b6700 0x7f15ec0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:39.822 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.817+0000 7f15fcff9640 1 -- 172.21.15.64:0/1885150816 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7f15f80e7ec0 con 0x7f1600108d20 2024-09-21T04:11:39.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.817+0000 7f160bca4740 1 -- 172.21.15.64:0/2637253833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9723d1b0 msgr2=0x556a9760dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:39.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.817+0000 7f15f7fff640 1 --2- 172.21.15.64:0/1885150816 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f15ec0b6700 0x7f15ec0b8c10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f15e80096c0 tx=0x7f15e8000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:39.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.817+0000 7f160bca4740 1 --2- 172.21.15.64:0/2637253833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9723d1b0 0x556a9760dbe0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f16001010c0 tx=0x7f15f000f920 comp rx=0 tx=0).stop 2024-09-21T04:11:39.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.818+0000 7f160bca4740 1 -- 172.21.15.64:0/2637253833 shutdown_connections 2024-09-21T04:11:39.826 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.818+0000 7f160bca4740 1 --2- 172.21.15.64:0/2637253833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a97245390 0x556a9760e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.818+0000 7f160bca4740 1 --2- 172.21.15.64:0/2637253833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9723d1b0 0x556a9760dbe0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.818+0000 7f160bca4740 1 --2- 172.21.15.64:0/2637253833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a97266070 0x556a9760d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.818+0000 7f160bca4740 1 -- 172.21.15.64:0/2637253833 >> 172.21.15.64:0/2637253833 conn(0x556a9725a460 msgr2=0x556a9760b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:39.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.818+0000 7f160bca4740 1 -- 172.21.15.64:0/2637253833 shutdown_connections 2024-09-21T04:11:39.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.818+0000 7f160bca4740 1 -- 172.21.15.64:0/2637253833 wait complete. 2024-09-21T04:11:39.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.818+0000 7f160bca4740 1 -- start start 2024-09-21T04:11:39.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.818+0000 7f160bca4740 1 Processor -- start 2024-09-21T04:11:39.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.819+0000 7f160bca4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9723d1b0 0x556a97618d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:39.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.819+0000 7f160bca4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a97245390 0x556a97619260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:39.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.819+0000 7f160bca4740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a97266070 0x556a976197c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:39.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.819+0000 7f160bca4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556a973d4700 con 0x556a9723d1b0 2024-09-21T04:11:39.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.819+0000 7f160bca4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556a9750e160 con 0x556a97245390 2024-09-21T04:11:39.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.819+0000 7f160bca4740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556a9761be80 con 0x556a97266070 2024-09-21T04:11:39.827 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.819+0000 7f15feffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9723d1b0 0x556a97618d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.819+0000 7f15feffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9723d1b0 0x556a97618d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46772/0 (socket says 172.21.15.64:46772) 2024-09-21T04:11:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.819+0000 7f15feffd640 1 -- 172.21.15.64:0/1846774776 learned_addr learned my addr 172.21.15.64:0/1846774776 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.819+0000 7f160bca4740 10 client.0 _ll_register_callbacks cb 0x556a975279e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:11:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.819+0000 7f15f7fff640 1 --2- 172.21.15.64:0/1846774776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a97245390 0x556a97619260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.819+0000 7f15ff7fe640 1 --2- 172.21.15.64:0/1846774776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a97266070 0x556a976197c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.819+0000 7f15f7fff640 1 -- 172.21.15.64:0/1846774776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a97266070 msgr2=0x556a976197c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.819+0000 7f15f7fff640 1 --2- 172.21.15.64:0/1846774776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a97266070 0x556a976197c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.819+0000 7f15f7fff640 1 -- 172.21.15.64:0/1846774776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9723d1b0 msgr2=0x556a97618d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.819+0000 7f15f7fff640 1 --2- 172.21.15.64:0/1846774776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9723d1b0 0x556a97618d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.819+0000 7f15f7fff640 1 -- 172.21.15.64:0/1846774776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a9753bc90 con 0x556a97245390 2024-09-21T04:11:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.819+0000 7f15ff7fe640 1 --2- 172.21.15.64:0/1846774776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a97266070 0x556a976197c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:11:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.819+0000 7f15feffd640 1 --2- 172.21.15.64:0/1846774776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9723d1b0 0x556a97618d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:11:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.819+0000 7f15f7fff640 1 --2- 172.21.15.64:0/1846774776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a97245390 0x556a97619260 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f15e8006010 tx=0x7f15e802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=473922c553502b99 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.819+0000 7f15e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:11:39.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.820+0000 7f15e67fc640 1 -- 172.21.15.64:0/1846774776 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8035040 con 0x556a97245390 2024-09-21T04:11:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.820+0000 7f160bca4740 1 -- 172.21.15.64:0/1846774776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15f0005020 con 0x556a97245390 2024-09-21T04:11:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.820+0000 7f15e67fc640 1 -- 172.21.15.64:0/1846774776 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7f15e8004450 con 0x556a97245390 2024-09-21T04:11:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.820+0000 7f15e67fc640 1 -- 172.21.15.64:0/1846774776 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f15e8005040 con 0x556a97245390 2024-09-21T04:11:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.820+0000 7f160bca4740 10 client.26762 fetch_fsmap learned FSMap version 39 2024-09-21T04:11:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.820+0000 7f160bca4740 1 -- 172.21.15.64:0/1846774776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556a97627af0 con 0x556a97245390 2024-09-21T04:11:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.820+0000 7f15e67fc640 1 -- 172.21.15.64:0/1846774776 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15f0005020 con 0x556a97245390 2024-09-21T04:11:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.820+0000 7f15e67fc640 1 -- 172.21.15.64:0/1846774776 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f15e80035f0 con 0x556a97245390 2024-09-21T04:11:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.820+0000 7f160bca4740 10 client.26762 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T04:11:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.820+0000 7f160bca4740 10 client.26762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:11:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.820+0000 7f160bca4740 20 client.26762 populate_metadata read hostname 'smithi064' 2024-09-21T04:11:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.820+0000 7f160bca4740 1 --2- 172.21.15.64:0/1846774776 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556a972249f0 0x556a9762afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.820+0000 7f160bca4740 4 client.26762 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:11:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.820+0000 7f160bca4740 1 -- 172.21.15.64:0/1846774776 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a97585df0 con 0x556a972249f0 2024-09-21T04:11:39.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.821+0000 7f15feffd640 1 --2- 172.21.15.64:0/1846774776 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556a972249f0 0x556a9762afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.821+0000 7f15feffd640 1 --2- 172.21.15.64:0/1846774776 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556a972249f0 0x556a9762afa0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cba382cc09359457 server_cookie=5aed6ff548ee6155 in_seq=0 out_seq=0 2024-09-21T04:11:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.821+0000 7f15e67fc640 10 client.26762 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.822+0000 7f15e67fc640 1 -- 172.21.15.64:0/1846774776 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556a97585df0 con 0x556a972249f0 2024-09-21T04:11:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.822+0000 7f15e67fc640 10 client.26762 handle_command_reply: tid=0 2024-09-21T04:11:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.822+0000 7f160bca4740 1 client.26762 shutdown 2024-09-21T04:11:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.822+0000 7f160bca4740 1 -- 172.21.15.64:0/1846774776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a97245390 msgr2=0x556a97619260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.822+0000 7f160bca4740 1 --2- 172.21.15.64:0/1846774776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a97245390 0x556a97619260 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f15e8006010 tx=0x7f15e802a410 comp rx=0 tx=0).stop 2024-09-21T04:11:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.822+0000 7f160bca4740 1 -- 172.21.15.64:0/1846774776 shutdown_connections 2024-09-21T04:11:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.822+0000 7f160bca4740 1 --2- 172.21.15.64:0/1846774776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a97266070 0x556a976197c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.822+0000 7f160bca4740 1 --2- 172.21.15.64:0/1846774776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a97245390 0x556a97619260 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.822+0000 7f160bca4740 1 --2- 172.21.15.64:0/1846774776 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556a972249f0 0x556a9762afa0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.822+0000 7f160bca4740 1 --2- 172.21.15.64:0/1846774776 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a9723d1b0 0x556a97618d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.822+0000 7f160bca4740 1 -- 172.21.15.64:0/1846774776 >> 172.21.15.64:0/1846774776 conn(0x556a9725a460 msgr2=0x556a97610830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.822+0000 7f15e67fc640 0 client.26762 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:39.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.822+0000 7f160bca4740 1 -- 172.21.15.64:0/1846774776 shutdown_connections 2024-09-21T04:11:39.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.822+0000 7f160bca4740 1 -- 172.21.15.64:0/1846774776 wait complete. 2024-09-21T04:11:39.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.822+0000 7f160bca4740 20 client.26762 trim_cache size 0 max 16384 2024-09-21T04:11:39.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.835+0000 7f160bca4740 1 Processor -- start 2024-09-21T04:11:39.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.836+0000 7f160bca4740 1 -- start start 2024-09-21T04:11:39.840 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.836+0000 7f160bca4740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a9723d1b0 0x556a9760f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:39.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.836+0000 7f160bca4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a97245390 0x556a9761a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:39.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.836+0000 7f160bca4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a97266070 0x556a9760abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:39.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.836+0000 7f160bca4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f16001b0090 con 0x556a97266070 2024-09-21T04:11:39.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.836+0000 7f160bca4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f16001ad8d0 con 0x556a97245390 2024-09-21T04:11:39.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.836+0000 7f160bca4740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f16001ab740 con 0x556a9723d1b0 2024-09-21T04:11:39.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.836+0000 7f15ff7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a97266070 0x556a9760abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:39.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.836+0000 7f15f7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a97245390 0x556a9761a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:39.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.836+0000 7f15ff7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a97266070 0x556a9760abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46778/0 (socket says 172.21.15.64:46778) 2024-09-21T04:11:39.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.836+0000 7f15ff7fe640 1 -- 172.21.15.64:0/279021860 learned_addr learned my addr 172.21.15.64:0/279021860 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:39.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.836+0000 7f15feffd640 1 --2- 172.21.15.64:0/279021860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a9723d1b0 0x556a9760f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:39.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.836+0000 7f15ff7fe640 1 -- 172.21.15.64:0/279021860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a9723d1b0 msgr2=0x556a9760f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:39.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.836+0000 7f15ff7fe640 1 --2- 172.21.15.64:0/279021860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a9723d1b0 0x556a9760f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.836+0000 7f15ff7fe640 1 -- 172.21.15.64:0/279021860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a97245390 msgr2=0x556a9761a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:39.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.836+0000 7f15ff7fe640 1 --2- 172.21.15.64:0/279021860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a97245390 0x556a9761a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.837+0000 7f15ff7fe640 1 -- 172.21.15.64:0/279021860 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a976250a0 con 0x556a97266070 2024-09-21T04:11:39.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.837+0000 7f15f7fff640 1 --2- 172.21.15.64:0/279021860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a97245390 0x556a9761a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:11:39.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.837+0000 7f15ff7fe640 1 --2- 172.21.15.64:0/279021860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a97266070 0x556a9760abc0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f15f80adbe0 tx=0x7f15f80a93c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864df620b77e1c33 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:39.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.837+0000 7f15f4ff9640 1 -- 172.21.15.64:0/279021860 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f80ad760 con 0x556a97266070 2024-09-21T04:11:39.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.837+0000 7f15f4ff9640 1 -- 172.21.15.64:0/279021860 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15f80ae350 con 0x556a97266070 2024-09-21T04:11:39.842 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.837+0000 7f15f4ff9640 1 -- 172.21.15.64:0/279021860 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f80e57d0 con 0x556a97266070 2024-09-21T04:11:39.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.838+0000 7f160bca4740 1 -- 172.21.15.64:0/279021860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a97266070 msgr2=0x556a9760abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:39.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.838+0000 7f160bca4740 1 --2- 172.21.15.64:0/279021860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a97266070 0x556a9760abc0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f15f80adbe0 tx=0x7f15f80a93c0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.838+0000 7f160bca4740 1 -- 172.21.15.64:0/279021860 shutdown_connections 2024-09-21T04:11:39.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.838+0000 7f160bca4740 1 --2- 172.21.15.64:0/279021860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a97266070 0x556a9760abc0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.838+0000 7f160bca4740 1 --2- 172.21.15.64:0/279021860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a97245390 0x556a9761a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.838+0000 7f160bca4740 1 --2- 172.21.15.64:0/279021860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a9723d1b0 0x556a9760f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.838+0000 7f160bca4740 1 -- 172.21.15.64:0/279021860 >> 172.21.15.64:0/279021860 conn(0x556a9725a460 msgr2=0x556a9761a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:39.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.838+0000 7f160bca4740 1 -- 172.21.15.64:0/279021860 shutdown_connections 2024-09-21T04:11:39.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.838+0000 7f160bca4740 1 -- 172.21.15.64:0/279021860 wait complete. 2024-09-21T04:11:39.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.838+0000 7f160bca4740 1 -- start start 2024-09-21T04:11:39.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.838+0000 7f160bca4740 1 Processor -- start 2024-09-21T04:11:39.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.839+0000 7f160bca4740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a9723d1b0 0x556a97621830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:39.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.839+0000 7f160bca4740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a97245390 0x556a9761cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:39.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.839+0000 7f160bca4740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a97266070 0x556a9761d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:39.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.839+0000 7f160bca4740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f16001abd90 con 0x556a97245390 2024-09-21T04:11:39.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.839+0000 7f160bca4740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f16001ad200 con 0x556a9723d1b0 2024-09-21T04:11:39.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.839+0000 7f160bca4740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f16001ac460 con 0x556a97266070 2024-09-21T04:11:39.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.839+0000 7f160bca4740 10 client.0 _ll_register_callbacks cb 0x7f15e8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:11:39.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.839+0000 7f15f7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a97245390 0x556a9761cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:39.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.839+0000 7f15feffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a9723d1b0 0x556a97621830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:39.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.839+0000 7f15f7fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a97245390 0x556a9761cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:46788/0 (socket says 172.21.15.64:46788) 2024-09-21T04:11:39.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.839+0000 7f15feffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a9723d1b0 0x556a97621830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:55996/0 (socket says 172.21.15.64:55996) 2024-09-21T04:11:39.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.839+0000 7f15f7fff640 1 -- 172.21.15.64:0/2300091832 learned_addr learned my addr 172.21.15.64:0/2300091832 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:39.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.839+0000 7f15ff7fe640 1 --2- 172.21.15.64:0/2300091832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a97266070 0x556a9761d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:39.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.839+0000 7f15f7fff640 1 -- 172.21.15.64:0/2300091832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a97266070 msgr2=0x556a9761d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:39.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.839+0000 7f15f7fff640 1 --2- 172.21.15.64:0/2300091832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a97266070 0x556a9761d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.839+0000 7f15f7fff640 1 -- 172.21.15.64:0/2300091832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a9723d1b0 msgr2=0x556a97621830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:39.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.839+0000 7f15f7fff640 1 --2- 172.21.15.64:0/2300091832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a9723d1b0 0x556a97621830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.839+0000 7f15f7fff640 1 -- 172.21.15.64:0/2300091832 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a9761d910 con 0x556a97245390 2024-09-21T04:11:39.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.839+0000 7f15feffd640 1 --2- 172.21.15.64:0/2300091832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a9723d1b0 0x556a97621830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:11:39.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.839+0000 7f15f7fff640 1 --2- 172.21.15.64:0/2300091832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a97245390 0x556a9761cce0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f15e8003a60 tx=0x7f15e8003800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9d5733fd1c8abd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:39.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.839+0000 7f15e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:11:39.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.840+0000 7f15e67fc640 1 -- 172.21.15.64:0/2300091832 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8063070 con 0x556a97245390 2024-09-21T04:11:39.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.840+0000 7f15e67fc640 1 -- 172.21.15.64:0/2300091832 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7f15e80051e0 con 0x556a97245390 2024-09-21T04:11:39.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.840+0000 7f160bca4740 1 -- 172.21.15.64:0/2300091832 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15f80ae350 con 0x556a97245390 2024-09-21T04:11:39.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.840+0000 7f15e67fc640 1 -- 172.21.15.64:0/2300091832 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f15e806f050 con 0x556a97245390 2024-09-21T04:11:39.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.840+0000 7f160bca4740 10 client.17097 fetch_fsmap learned FSMap version 39 2024-09-21T04:11:39.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.840+0000 7f160bca4740 1 -- 172.21.15.64:0/2300091832 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556a97695850 con 0x556a97245390 2024-09-21T04:11:39.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.840+0000 7f15e67fc640 1 -- 172.21.15.64:0/2300091832 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15f80ae350 con 0x556a97245390 2024-09-21T04:11:39.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.840+0000 7f15e67fc640 1 -- 172.21.15.64:0/2300091832 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f15e804b4e0 con 0x556a97245390 2024-09-21T04:11:39.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.840+0000 7f160bca4740 10 client.17097 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T04:11:39.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.840+0000 7f160bca4740 10 client.17097 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:11:39.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.840+0000 7f160bca4740 20 client.17097 populate_metadata read hostname 'smithi064' 2024-09-21T04:11:39.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.841+0000 7f160bca4740 1 --2- 172.21.15.64:0/2300091832 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556a971f9da0 0x556a976980a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:39.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.841+0000 7f160bca4740 4 client.17097 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:11:39.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.841+0000 7f160bca4740 1 -- 172.21.15.64:0/2300091832 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556a976148d0 con 0x556a971f9da0 2024-09-21T04:11:39.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.841+0000 7f15feffd640 1 --2- 172.21.15.64:0/2300091832 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556a971f9da0 0x556a976980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:39.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.841+0000 7f15e67fc640 10 client.17097 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:39.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.841+0000 7f15feffd640 1 --2- 172.21.15.64:0/2300091832 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556a971f9da0 0x556a976980a0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d875b2f8ccbb4e2 server_cookie=391f4103814e5367 in_seq=0 out_seq=0 2024-09-21T04:11:39.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.841+0000 7f15e67fc640 1 -- 172.21.15.64:0/2300091832 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556a976148d0 con 0x556a971f9da0 2024-09-21T04:11:39.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.841+0000 7f15e67fc640 10 client.17097 handle_command_reply: tid=0 2024-09-21T04:11:39.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.841+0000 7f160bca4740 1 client.17097 shutdown 2024-09-21T04:11:39.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.841+0000 7f160bca4740 1 -- 172.21.15.64:0/2300091832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a97245390 msgr2=0x556a9761cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:39.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.841+0000 7f160bca4740 1 --2- 172.21.15.64:0/2300091832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a97245390 0x556a9761cce0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f15e8003a60 tx=0x7f15e8003800 comp rx=0 tx=0).stop 2024-09-21T04:11:39.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.842+0000 7f160bca4740 1 -- 172.21.15.64:0/2300091832 shutdown_connections 2024-09-21T04:11:39.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.842+0000 7f160bca4740 1 --2- 172.21.15.64:0/2300091832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556a97266070 0x556a9761d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.842+0000 7f160bca4740 1 --2- 172.21.15.64:0/2300091832 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556a971f9da0 0x556a976980a0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.842+0000 7f160bca4740 1 --2- 172.21.15.64:0/2300091832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556a97245390 0x556a9761cce0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.842+0000 7f160bca4740 1 --2- 172.21.15.64:0/2300091832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a9723d1b0 0x556a97621830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.842+0000 7f15e67fc640 0 client.17097 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:39.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.842+0000 7f160bca4740 1 -- 172.21.15.64:0/2300091832 >> 172.21.15.64:0/2300091832 conn(0x556a9725a460 msgr2=0x556a9760edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:39.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.842+0000 7f160bca4740 1 -- 172.21.15.64:0/2300091832 shutdown_connections 2024-09-21T04:11:39.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.842+0000 7f160bca4740 1 -- 172.21.15.64:0/2300091832 wait complete. 2024-09-21T04:11:39.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.842+0000 7f160bca4740 20 client.17097 trim_cache size 0 max 16384 2024-09-21T04:11:39.850 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:11:39.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.842+0000 7f1605730640 1 -- 172.21.15.64:0/1885150816 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f15ec0b6700 msgr2=0x7f15ec0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:39.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.842+0000 7f1605730640 1 --2- 172.21.15.64:0/1885150816 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f15ec0b6700 0x7f15ec0b8c10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f15e80096c0 tx=0x7f15e8000bc0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.842+0000 7f1605730640 1 -- 172.21.15.64:0/1885150816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1600108d20 msgr2=0x7f16001000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:39.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.842+0000 7f1605730640 1 --2- 172.21.15.64:0/1885150816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1600108d20 0x7f16001000f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f15f8009fd0 tx=0x7f15f800ef10 comp rx=0 tx=0).stop 2024-09-21T04:11:39.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.842+0000 7f1605730640 1 -- 172.21.15.64:0/1885150816 shutdown_connections 2024-09-21T04:11:39.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.842+0000 7f1605730640 1 --2- 172.21.15.64:0/1885150816 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f15ec0b6700 0x7f15ec0b8c10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.842+0000 7f1605730640 1 --2- 172.21.15.64:0/1885150816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1600108d20 0x7f16001000f0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.842+0000 7f1605730640 1 --2- 172.21.15.64:0/1885150816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16001083d0 0x7f16000ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.842+0000 7f1605730640 1 --2- 172.21.15.64:0/1885150816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1600102ae0 0x7f16000ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:39.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.842+0000 7f1605730640 1 -- 172.21.15.64:0/1885150816 >> 172.21.15.64:0/1885150816 conn(0x7f16000797b0 msgr2=0x7f1600101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:39.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.843+0000 7f1605730640 1 -- 172.21.15.64:0/1885150816 shutdown_connections 2024-09-21T04:11:39.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:39.843+0000 7f1605730640 1 -- 172.21.15.64:0/1885150816 wait complete. 2024-09-21T04:11:40.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:40 smithi064 ceph-mon[31180]: pgmap v554: 97 pgs: 97 active+clean; 813 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.8 MiB/s wr, 470 op/s 2024-09-21T04:11:40.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:40 smithi131 ceph-mon[35193]: pgmap v554: 97 pgs: 97 active+clean; 813 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.8 MiB/s wr, 470 op/s 2024-09-21T04:11:40.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:40 smithi119 ceph-mon[34811]: pgmap v554: 97 pgs: 97 active+clean; 813 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.8 MiB/s wr, 470 op/s 2024-09-21T04:11:41.574 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T04:11:41.574 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T04:11:41.858 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:11:42.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:42 smithi064 ceph-mon[31180]: pgmap v555: 97 pgs: 97 active+clean; 813 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 451 op/s 2024-09-21T04:11:42.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:42 smithi131 ceph-mon[35193]: pgmap v555: 97 pgs: 97 active+clean; 813 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 451 op/s 2024-09-21T04:11:42.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.342+0000 7f47db3bb640 1 -- 172.21.15.64:0/4152062094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d4102a60 msgr2=0x7f47d4102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:42.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.342+0000 7f47db3bb640 1 --2- 172.21.15.64:0/4152062094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d4102a60 0x7f47d4102e70 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f47d0009900 tx=0x7f47d002a410 comp rx=0 tx=0).stop 2024-09-21T04:11:42.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.342+0000 7f47db3bb640 1 -- 172.21.15.64:0/4152062094 shutdown_connections 2024-09-21T04:11:42.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.342+0000 7f47db3bb640 1 --2- 172.21.15.64:0/4152062094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47d4108cf0 0x7f47d410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.342+0000 7f47db3bb640 1 --2- 172.21.15.64:0/4152062094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47d41083a0 0x7f47d41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.342+0000 7f47db3bb640 1 --2- 172.21.15.64:0/4152062094 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d4102a60 0x7f47d4102e70 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.342+0000 7f47db3bb640 1 -- 172.21.15.64:0/4152062094 >> 172.21.15.64:0/4152062094 conn(0x7f47d4079730 msgr2=0x7f47d4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:42.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.342+0000 7f47db3bb640 1 -- 172.21.15.64:0/4152062094 shutdown_connections 2024-09-21T04:11:42.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.342+0000 7f47db3bb640 1 -- 172.21.15.64:0/4152062094 wait complete. 2024-09-21T04:11:42.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.343+0000 7f47db3bb640 1 Processor -- start 2024-09-21T04:11:42.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.343+0000 7f47db3bb640 1 -- start start 2024-09-21T04:11:42.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.343+0000 7f47db3bb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d4102a60 0x7f47d419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:42.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.343+0000 7f47db3bb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47d41083a0 0x7f47d419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:42.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.343+0000 7f47db3bb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47d4108cf0 0x7f47d419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:42.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.343+0000 7f47db3bb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f47d4112960 con 0x7f47d4102a60 2024-09-21T04:11:42.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.343+0000 7f47db3bb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f47d41127e0 con 0x7f47d4108cf0 2024-09-21T04:11:42.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.343+0000 7f47db3bb640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f47d4112ae0 con 0x7f47d41083a0 2024-09-21T04:11:42.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.343+0000 7f47d9157640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d4102a60 0x7f47d419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:42.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.343+0000 7f47d9157640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d4102a60 0x7f47d419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41032/0 (socket says 172.21.15.64:41032) 2024-09-21T04:11:42.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.343+0000 7f47d9157640 1 -- 172.21.15.64:0/4244661429 learned_addr learned my addr 172.21.15.64:0/4244661429 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:42.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.343+0000 7f47d9958640 1 --2- 172.21.15.64:0/4244661429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47d4108cf0 0x7f47d419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:42.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.343+0000 7f47d8956640 1 --2- 172.21.15.64:0/4244661429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47d41083a0 0x7f47d419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:42.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.344+0000 7f47d9157640 1 -- 172.21.15.64:0/4244661429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47d41083a0 msgr2=0x7f47d419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:42.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.344+0000 7f47d9157640 1 --2- 172.21.15.64:0/4244661429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47d41083a0 0x7f47d419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.344+0000 7f47d9157640 1 -- 172.21.15.64:0/4244661429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47d4108cf0 msgr2=0x7f47d419fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:42.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.344+0000 7f47d9157640 1 --2- 172.21.15.64:0/4244661429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47d4108cf0 0x7f47d419fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.344+0000 7f47d9157640 1 -- 172.21.15.64:0/4244661429 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d41a03b0 con 0x7f47d4102a60 2024-09-21T04:11:42.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.344+0000 7f47d8956640 1 --2- 172.21.15.64:0/4244661429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47d41083a0 0x7f47d419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:11:42.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.344+0000 7f47d9958640 1 --2- 172.21.15.64:0/4244661429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47d4108cf0 0x7f47d419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:11:42.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.344+0000 7f47d9157640 1 --2- 172.21.15.64:0/4244661429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d4102a60 0x7f47d419a440 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f47d0005940 tx=0x7f47d00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:42.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.344+0000 7f47ca7fc640 1 -- 172.21.15.64:0/4244661429 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d003f030 con 0x7f47d4102a60 2024-09-21T04:11:42.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.344+0000 7f47ca7fc640 1 -- 172.21.15.64:0/4244661429 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47d00044c0 con 0x7f47d4102a60 2024-09-21T04:11:42.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.344+0000 7f47db3bb640 1 -- 172.21.15.64:0/4244661429 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d41a0640 con 0x7f47d4102a60 2024-09-21T04:11:42.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.344+0000 7f47ca7fc640 1 -- 172.21.15.64:0/4244661429 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d0035070 con 0x7f47d4102a60 2024-09-21T04:11:42.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.344+0000 7f47db3bb640 1 -- 172.21.15.64:0/4244661429 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d41a6e00 con 0x7f47d4102a60 2024-09-21T04:11:42.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.347+0000 7f47ca7fc640 1 -- 172.21.15.64:0/4244661429 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f47d0042030 con 0x7f47d4102a60 2024-09-21T04:11:42.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.347+0000 7f47e192f740 1 Processor -- start 2024-09-21T04:11:42.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.347+0000 7f47e192f740 1 -- start start 2024-09-21T04:11:42.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.347+0000 7f47e192f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4bc409350 0x55c4bc7b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:42.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.347+0000 7f47e192f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4bc3e0490 0x55c4bc7b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:42.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.347+0000 7f47e192f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4bc3e8670 0x55c4bc7b0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:42.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.347+0000 7f47e192f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c4bc726e60 con 0x55c4bc409350 2024-09-21T04:11:42.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.347+0000 7f47e192f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c4bc716290 con 0x55c4bc3e0490 2024-09-21T04:11:42.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.347+0000 7f47e192f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c4bc71a0a0 con 0x55c4bc3e8670 2024-09-21T04:11:42.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.347+0000 7f47d9958640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4bc409350 0x55c4bc7b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:42.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.348+0000 7f47d9157640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4bc3e0490 0x55c4bc7b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:42.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.348+0000 7f47d9958640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4bc409350 0x55c4bc7b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41034/0 (socket says 172.21.15.64:41034) 2024-09-21T04:11:42.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.348+0000 7f47d9958640 1 -- 172.21.15.64:0/2070998967 learned_addr learned my addr 172.21.15.64:0/2070998967 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:42.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.348+0000 7f47d8956640 1 --2- 172.21.15.64:0/2070998967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4bc3e8670 0x55c4bc7b0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:42.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.348+0000 7f47d9958640 1 -- 172.21.15.64:0/2070998967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4bc3e8670 msgr2=0x55c4bc7b0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:42.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.348+0000 7f47d9958640 1 --2- 172.21.15.64:0/2070998967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4bc3e8670 0x55c4bc7b0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.348+0000 7f47d9958640 1 -- 172.21.15.64:0/2070998967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4bc3e0490 msgr2=0x55c4bc7b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:42.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.348+0000 7f47d9958640 1 --2- 172.21.15.64:0/2070998967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4bc3e0490 0x55c4bc7b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.348+0000 7f47d9958640 1 -- 172.21.15.64:0/2070998967 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4bc474930 con 0x55c4bc409350 2024-09-21T04:11:42.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.348+0000 7f47d9157640 1 --2- 172.21.15.64:0/2070998967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4bc3e0490 0x55c4bc7b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:11:42.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.348+0000 7f47ca7fc640 1 --2- 172.21.15.64:0/4244661429 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f47c00b6680 0x7f47c00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:42.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.348+0000 7f47d9958640 1 --2- 172.21.15.64:0/2070998967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4bc409350 0x55c4bc7b0430 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0117e0 tx=0x7f47cc011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a1d2432760cd2b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:42.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.348+0000 7f47d9157640 1 --2- 172.21.15.64:0/4244661429 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f47c00b6680 0x7f47c00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:42.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.348+0000 7f47ca7fc640 1 -- 172.21.15.64:0/4244661429 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7f47d0107b70 con 0x7f47d4102a60 2024-09-21T04:11:42.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.348+0000 7f47bf7fe640 1 -- 172.21.15.64:0/2070998967 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc011310 con 0x55c4bc409350 2024-09-21T04:11:42.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.349+0000 7f47bf7fe640 1 -- 172.21.15.64:0/2070998967 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47cc003e20 con 0x55c4bc409350 2024-09-21T04:11:42.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.349+0000 7f47bf7fe640 1 -- 172.21.15.64:0/2070998967 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc004450 con 0x55c4bc409350 2024-09-21T04:11:42.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.349+0000 7f47e192f740 1 -- 172.21.15.64:0/2070998967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4bc409350 msgr2=0x55c4bc7b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:42.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.349+0000 7f47d9157640 1 --2- 172.21.15.64:0/4244661429 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f47c00b6680 0x7f47c00b8b90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f47d0105310 tx=0x7f47d010b6c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:42.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.349+0000 7f47e192f740 1 --2- 172.21.15.64:0/2070998967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4bc409350 0x55c4bc7b0430 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0117e0 tx=0x7f47cc011ca0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.349+0000 7f47e192f740 1 -- 172.21.15.64:0/2070998967 shutdown_connections 2024-09-21T04:11:42.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.349+0000 7f47e192f740 1 --2- 172.21.15.64:0/2070998967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4bc3e8670 0x55c4bc7b0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.349+0000 7f47e192f740 1 --2- 172.21.15.64:0/2070998967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4bc3e0490 0x55c4bc7b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.349+0000 7f47e192f740 1 --2- 172.21.15.64:0/2070998967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4bc409350 0x55c4bc7b0430 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.350+0000 7f47e192f740 1 -- 172.21.15.64:0/2070998967 >> 172.21.15.64:0/2070998967 conn(0x55c4bc3fd740 msgr2=0x55c4bc7ae320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:42.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.350+0000 7f47e192f740 1 -- 172.21.15.64:0/2070998967 shutdown_connections 2024-09-21T04:11:42.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.350+0000 7f47e192f740 1 -- 172.21.15.64:0/2070998967 wait complete. 2024-09-21T04:11:42.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.350+0000 7f47e192f740 1 -- start start 2024-09-21T04:11:42.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.350+0000 7f47e192f740 1 Processor -- start 2024-09-21T04:11:42.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.351+0000 7f47e192f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4bc3e0490 0x55c4bc7c03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:42.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.351+0000 7f47e192f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4bc3e8670 0x55c4bc7c0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:42.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.351+0000 7f47e192f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4bc409350 0x55c4bc7cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:42.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.351+0000 7f47e192f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c4bc577890 con 0x55c4bc3e8670 2024-09-21T04:11:42.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.351+0000 7f47e192f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c4bc6b0d40 con 0x55c4bc3e0490 2024-09-21T04:11:42.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.351+0000 7f47e192f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c4bc7c2f70 con 0x55c4bc409350 2024-09-21T04:11:42.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.351+0000 7f47d8956640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4bc3e8670 0x55c4bc7c0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:42.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.351+0000 7f47e192f740 10 client.0 _ll_register_callbacks cb 0x55c4bc714c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:11:42.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.351+0000 7f47d9958640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4bc3e0490 0x55c4bc7c03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:42.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.351+0000 7f47d9958640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4bc3e0490 0x55c4bc7c03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:37268/0 (socket says 172.21.15.64:37268) 2024-09-21T04:11:42.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.351+0000 7f47d9958640 1 -- 172.21.15.64:0/1012767471 learned_addr learned my addr 172.21.15.64:0/1012767471 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:42.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.351+0000 7f47d9958640 1 --2- 172.21.15.64:0/1012767471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4bc409350 0x55c4bc7cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:42.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.351+0000 7f47d9958640 1 -- 172.21.15.64:0/1012767471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4bc409350 msgr2=0x55c4bc7cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:42.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.351+0000 7f47d9958640 1 --2- 172.21.15.64:0/1012767471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4bc409350 0x55c4bc7cc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.351+0000 7f47d9958640 1 -- 172.21.15.64:0/1012767471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4bc3e8670 msgr2=0x55c4bc7c0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:42.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.351+0000 7f47d9958640 1 --2- 172.21.15.64:0/1012767471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4bc3e8670 0x55c4bc7c0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.351+0000 7f47d9958640 1 -- 172.21.15.64:0/1012767471 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4bc7cc830 con 0x55c4bc3e0490 2024-09-21T04:11:42.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.351+0000 7f47d9958640 1 --2- 172.21.15.64:0/1012767471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4bc3e0490 0x55c4bc7c03f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f47cc003df0 tx=0x7f47cc005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73fdf116f9c834df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:42.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.351+0000 7f47bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:11:42.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.352+0000 7f47bcff9640 1 -- 172.21.15.64:0/1012767471 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc020070 con 0x55c4bc3e0490 2024-09-21T04:11:42.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.352+0000 7f47e192f740 1 -- 172.21.15.64:0/1012767471 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47cc003e20 con 0x55c4bc3e0490 2024-09-21T04:11:42.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.352+0000 7f47bcff9640 1 -- 172.21.15.64:0/1012767471 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7f47cc00fd00 con 0x55c4bc3e0490 2024-09-21T04:11:42.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.352+0000 7f47bcff9640 1 -- 172.21.15.64:0/1012767471 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f47cc032050 con 0x55c4bc3e0490 2024-09-21T04:11:42.647 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.352+0000 7f47e192f740 10 client.26780 fetch_fsmap learned FSMap version 39 2024-09-21T04:11:42.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.352+0000 7f47e192f740 1 -- 172.21.15.64:0/1012767471 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c4bc7cf160 con 0x55c4bc3e0490 2024-09-21T04:11:42.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.352+0000 7f47bcff9640 1 -- 172.21.15.64:0/1012767471 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47cc003e20 con 0x55c4bc3e0490 2024-09-21T04:11:42.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.352+0000 7f47bcff9640 1 -- 172.21.15.64:0/1012767471 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f47cc018020 con 0x55c4bc3e0490 2024-09-21T04:11:42.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.352+0000 7f47e192f740 10 client.26780 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T04:11:42.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.352+0000 7f47e192f740 10 client.26780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:11:42.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.352+0000 7f47e192f740 20 client.26780 populate_metadata read hostname 'smithi064' 2024-09-21T04:11:42.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.353+0000 7f47e192f740 1 --2- 172.21.15.64:0/1012767471 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55c4bc3c7cd0 0x55c4bc7d24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:42.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.353+0000 7f47e192f740 4 client.26780 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:11:42.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.353+0000 7f47e192f740 1 -- 172.21.15.64:0/1012767471 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4bc728db0 con 0x55c4bc3c7cd0 2024-09-21T04:11:42.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.353+0000 7f47d8956640 1 --2- 172.21.15.64:0/1012767471 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55c4bc3c7cd0 0x55c4bc7d24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:42.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.353+0000 7f47d8956640 1 --2- 172.21.15.64:0/1012767471 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55c4bc3c7cd0 0x55c4bc7d24c0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e6ab1d3e3d74422 server_cookie=8e0b65d8e47e0f in_seq=0 out_seq=0 2024-09-21T04:11:42.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.353+0000 7f47bcff9640 10 client.26780 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:42.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.354+0000 7f47bcff9640 1 -- 172.21.15.64:0/1012767471 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c4bc728db0 con 0x55c4bc3c7cd0 2024-09-21T04:11:42.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.354+0000 7f47bcff9640 10 client.26780 handle_command_reply: tid=0 2024-09-21T04:11:42.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.354+0000 7f47e192f740 1 client.26780 shutdown 2024-09-21T04:11:42.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.354+0000 7f47e192f740 1 -- 172.21.15.64:0/1012767471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4bc3e0490 msgr2=0x55c4bc7c03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:42.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.354+0000 7f47e192f740 1 --2- 172.21.15.64:0/1012767471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4bc3e0490 0x55c4bc7c03f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f47cc003df0 tx=0x7f47cc005bd0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.354+0000 7f47e192f740 1 -- 172.21.15.64:0/1012767471 shutdown_connections 2024-09-21T04:11:42.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.354+0000 7f47e192f740 1 --2- 172.21.15.64:0/1012767471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4bc409350 0x55c4bc7cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.354+0000 7f47e192f740 1 --2- 172.21.15.64:0/1012767471 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55c4bc3c7cd0 0x55c4bc7d24c0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.354+0000 7f47e192f740 1 --2- 172.21.15.64:0/1012767471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4bc3e8670 0x55c4bc7c0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.354+0000 7f47e192f740 1 --2- 172.21.15.64:0/1012767471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4bc3e0490 0x55c4bc7c03f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.354+0000 7f47bcff9640 0 client.26780 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:42.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.354+0000 7f47e192f740 1 -- 172.21.15.64:0/1012767471 >> 172.21.15.64:0/1012767471 conn(0x55c4bc3fd740 msgr2=0x55c4bc7ac850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:42.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.354+0000 7f47e192f740 1 -- 172.21.15.64:0/1012767471 shutdown_connections 2024-09-21T04:11:42.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.354+0000 7f47e192f740 1 -- 172.21.15.64:0/1012767471 wait complete. 2024-09-21T04:11:42.649 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.354+0000 7f47e192f740 20 client.26780 trim_cache size 0 max 16384 2024-09-21T04:11:42.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.373+0000 7f47e192f740 1 Processor -- start 2024-09-21T04:11:42.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.373+0000 7f47e192f740 1 -- start start 2024-09-21T04:11:42.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.373+0000 7f47e192f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4bc3e0490 0x55c4bc7bea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:42.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.373+0000 7f47e192f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4bc3e8670 0x55c4bc7befb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:42.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.373+0000 7f47e192f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4bc409350 0x55c4bc7adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:42.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.373+0000 7f47e192f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f47d41affb0 con 0x55c4bc3e0490 2024-09-21T04:11:42.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.373+0000 7f47e192f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f47d41ad7f0 con 0x55c4bc409350 2024-09-21T04:11:42.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.373+0000 7f47e192f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f47d41ab660 con 0x55c4bc3e8670 2024-09-21T04:11:42.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.373+0000 7f47d9958640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4bc3e0490 0x55c4bc7bea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:42.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.373+0000 7f47d9958640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4bc3e0490 0x55c4bc7bea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41066/0 (socket says 172.21.15.64:41066) 2024-09-21T04:11:42.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.373+0000 7f47d9958640 1 -- 172.21.15.64:0/4079197025 learned_addr learned my addr 172.21.15.64:0/4079197025 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:42.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.374+0000 7f47d9958640 1 --2- 172.21.15.64:0/4079197025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4bc409350 0x55c4bc7adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:42.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.374+0000 7f47d8956640 1 --2- 172.21.15.64:0/4079197025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4bc3e8670 0x55c4bc7befb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:42.650 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.374+0000 7f47d9958640 1 -- 172.21.15.64:0/4079197025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4bc3e8670 msgr2=0x55c4bc7befb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.374+0000 7f47d9958640 1 --2- 172.21.15.64:0/4079197025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4bc3e8670 0x55c4bc7befb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.374+0000 7f47d9958640 1 -- 172.21.15.64:0/4079197025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4bc409350 msgr2=0x55c4bc7adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.374+0000 7f47d9958640 1 --2- 172.21.15.64:0/4079197025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4bc409350 0x55c4bc7adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.374+0000 7f47d9958640 1 -- 172.21.15.64:0/4079197025 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4bc7c0d90 con 0x55c4bc3e0490 2024-09-21T04:11:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.374+0000 7f47d8956640 1 --2- 172.21.15.64:0/4079197025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4bc3e8670 0x55c4bc7befb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:11:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.374+0000 7f47d9958640 1 --2- 172.21.15.64:0/4079197025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4bc3e0490 0x55c4bc7bea70 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f47cc006010 tx=0x7f47cc005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b9451255e82fd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.374+0000 7f47beffd640 1 -- 172.21.15.64:0/4079197025 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc020070 con 0x55c4bc3e0490 2024-09-21T04:11:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.374+0000 7f47beffd640 1 -- 172.21.15.64:0/4079197025 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47cc00f8c0 con 0x55c4bc3e0490 2024-09-21T04:11:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.374+0000 7f47beffd640 1 -- 172.21.15.64:0/4079197025 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc01ce60 con 0x55c4bc3e0490 2024-09-21T04:11:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.375+0000 7f47e192f740 1 -- 172.21.15.64:0/4079197025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4bc3e0490 msgr2=0x55c4bc7bea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.375+0000 7f47e192f740 1 --2- 172.21.15.64:0/4079197025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4bc3e0490 0x55c4bc7bea70 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f47cc006010 tx=0x7f47cc005cf0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.375+0000 7f47e192f740 1 -- 172.21.15.64:0/4079197025 shutdown_connections 2024-09-21T04:11:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.375+0000 7f47e192f740 1 --2- 172.21.15.64:0/4079197025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4bc409350 0x55c4bc7adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.375+0000 7f47e192f740 1 --2- 172.21.15.64:0/4079197025 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4bc3e8670 0x55c4bc7befb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.651 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.375+0000 7f47e192f740 1 --2- 172.21.15.64:0/4079197025 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4bc3e0490 0x55c4bc7bea70 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.375+0000 7f47e192f740 1 -- 172.21.15.64:0/4079197025 >> 172.21.15.64:0/4079197025 conn(0x55c4bc3fd740 msgr2=0x55c4bc7b55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:42.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.375+0000 7f47e192f740 1 -- 172.21.15.64:0/4079197025 shutdown_connections 2024-09-21T04:11:42.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.375+0000 7f47e192f740 1 -- 172.21.15.64:0/4079197025 wait complete. 2024-09-21T04:11:42.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.375+0000 7f47e192f740 1 -- start start 2024-09-21T04:11:42.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.375+0000 7f47e192f740 1 Processor -- start 2024-09-21T04:11:42.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.376+0000 7f47e192f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4bc3e0490 0x55c4bc7c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:42.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.376+0000 7f47e192f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4bc3e8670 0x55c4bc7c8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:42.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.376+0000 7f47e192f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4bc409350 0x55c4bc7c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:42.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.376+0000 7f47e192f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f47d41abcb0 con 0x55c4bc3e8670 2024-09-21T04:11:42.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.376+0000 7f47e192f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f47d41ad120 con 0x55c4bc3e0490 2024-09-21T04:11:42.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.376+0000 7f47e192f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f47d41ac380 con 0x55c4bc409350 2024-09-21T04:11:42.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.376+0000 7f47e192f740 10 client.0 _ll_register_callbacks cb 0x7f47d010b590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:11:42.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.376+0000 7f47d9958640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4bc3e0490 0x55c4bc7c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:42.652 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.376+0000 7f47d8956640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4bc3e8670 0x55c4bc7c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.376+0000 7f47d9958640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4bc3e0490 0x55c4bc7c81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:37280/0 (socket says 172.21.15.64:37280) 2024-09-21T04:11:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.376+0000 7f47d9958640 1 -- 172.21.15.64:0/226194559 learned_addr learned my addr 172.21.15.64:0/226194559 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.376+0000 7f47d9958640 1 --2- 172.21.15.64:0/226194559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4bc409350 0x55c4bc7c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.376+0000 7f47d9958640 1 -- 172.21.15.64:0/226194559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4bc409350 msgr2=0x55c4bc7c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.376+0000 7f47d9958640 1 --2- 172.21.15.64:0/226194559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4bc409350 0x55c4bc7c31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.376+0000 7f47d9958640 1 -- 172.21.15.64:0/226194559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4bc3e8670 msgr2=0x55c4bc7c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.376+0000 7f47d9958640 1 --2- 172.21.15.64:0/226194559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4bc3e8670 0x55c4bc7c8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.376+0000 7f47d9958640 1 -- 172.21.15.64:0/226194559 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4bc7c8e70 con 0x55c4bc3e0490 2024-09-21T04:11:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.377+0000 7f47d9958640 1 --2- 172.21.15.64:0/226194559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4bc3e0490 0x55c4bc7c81f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00ff70 tx=0x7f47cc009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d6af6b63d2b977 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.377+0000 7f47bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:11:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.377+0000 7f47bcff9640 1 -- 172.21.15.64:0/226194559 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc020070 con 0x55c4bc3e0490 2024-09-21T04:11:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.377+0000 7f47e192f740 1 -- 172.21.15.64:0/226194559 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47cc00f8c0 con 0x55c4bc3e0490 2024-09-21T04:11:42.653 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.377+0000 7f47bcff9640 1 -- 172.21.15.64:0/226194559 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7f47cc0053b0 con 0x55c4bc3e0490 2024-09-21T04:11:42.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.377+0000 7f47bcff9640 1 -- 172.21.15.64:0/226194559 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f47cc00fd00 con 0x55c4bc3e0490 2024-09-21T04:11:42.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.377+0000 7f47e192f740 10 client.26786 fetch_fsmap learned FSMap version 39 2024-09-21T04:11:42.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.377+0000 7f47e192f740 1 -- 172.21.15.64:0/226194559 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c4bc83d290 con 0x55c4bc3e0490 2024-09-21T04:11:42.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.377+0000 7f47bcff9640 1 -- 172.21.15.64:0/226194559 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47cc00f8c0 con 0x55c4bc3e0490 2024-09-21T04:11:42.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.377+0000 7f47bcff9640 1 -- 172.21.15.64:0/226194559 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f47cc018040 con 0x55c4bc3e0490 2024-09-21T04:11:42.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.377+0000 7f47e192f740 10 client.26786 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T04:11:42.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.377+0000 7f47e192f740 10 client.26786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:11:42.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.377+0000 7f47e192f740 20 client.26786 populate_metadata read hostname 'smithi064' 2024-09-21T04:11:42.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.377+0000 7f47e192f740 1 --2- 172.21.15.64:0/226194559 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55c4bc39d080 0x55c4bc83fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:42.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.378+0000 7f47e192f740 4 client.26786 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T04:11:42.654 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.378+0000 7f47e192f740 1 -- 172.21.15.64:0/226194559 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c4bc724860 con 0x55c4bc39d080 2024-09-21T04:11:42.654 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:11:42.655 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-21T04:11:42.655 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "71f3f255-4837-45e9-92bc-4b3abb5c20eb", 2024-09-21T04:11:42.655 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-21T04:11:42.655 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:11:42.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.378+0000 7f47d8956640 1 --2- 172.21.15.64:0/226194559 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55c4bc39d080 0x55c4bc83fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:42.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.378+0000 7f47d8956640 1 --2- 172.21.15.64:0/226194559 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55c4bc39d080 0x55c4bc83fae0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6ddae762c60ddf server_cookie=8e6585d21cd5824f in_seq=0 out_seq=0 2024-09-21T04:11:42.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.378+0000 7f47bcff9640 10 client.26786 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:42.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.379+0000 7f47bcff9640 1 -- 172.21.15.64:0/226194559 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c4bc724860 con 0x55c4bc39d080 2024-09-21T04:11:42.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.379+0000 7f47bcff9640 10 client.26786 handle_command_reply: tid=0 2024-09-21T04:11:42.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.379+0000 7f47e192f740 1 client.26786 shutdown 2024-09-21T04:11:42.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.379+0000 7f47e192f740 1 -- 172.21.15.64:0/226194559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4bc3e0490 msgr2=0x55c4bc7c81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:42.655 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.379+0000 7f47e192f740 1 --2- 172.21.15.64:0/226194559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4bc3e0490 0x55c4bc7c81f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00ff70 tx=0x7f47cc009610 comp rx=0 tx=0).stop 2024-09-21T04:11:42.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.379+0000 7f47e192f740 1 -- 172.21.15.64:0/226194559 shutdown_connections 2024-09-21T04:11:42.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.379+0000 7f47e192f740 1 --2- 172.21.15.64:0/226194559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c4bc409350 0x55c4bc7c31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.379+0000 7f47e192f740 1 --2- 172.21.15.64:0/226194559 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55c4bc39d080 0x55c4bc83fae0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.379+0000 7f47e192f740 1 --2- 172.21.15.64:0/226194559 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4bc3e8670 0x55c4bc7c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.379+0000 7f47e192f740 1 --2- 172.21.15.64:0/226194559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c4bc3e0490 0x55c4bc7c81f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.379+0000 7f47bcff9640 0 client.26786 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:42.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.379+0000 7f47e192f740 1 -- 172.21.15.64:0/226194559 >> 172.21.15.64:0/226194559 conn(0x55c4bc3fd740 msgr2=0x55c4bc7adc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:42.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.379+0000 7f47e192f740 1 -- 172.21.15.64:0/226194559 shutdown_connections 2024-09-21T04:11:42.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.379+0000 7f47e192f740 1 -- 172.21.15.64:0/226194559 wait complete. 2024-09-21T04:11:42.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.379+0000 7f47e192f740 20 client.26786 trim_cache size 0 max 16384 2024-09-21T04:11:42.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.380+0000 7f47db3bb640 1 -- 172.21.15.64:0/4244661429 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f47c00b6680 msgr2=0x7f47c00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:42.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.380+0000 7f47db3bb640 1 --2- 172.21.15.64:0/4244661429 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f47c00b6680 0x7f47c00b8b90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f47d0105310 tx=0x7f47d010b6c0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.380+0000 7f47db3bb640 1 -- 172.21.15.64:0/4244661429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d4102a60 msgr2=0x7f47d419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:42.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.380+0000 7f47db3bb640 1 --2- 172.21.15.64:0/4244661429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d4102a60 0x7f47d419a440 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f47d0005940 tx=0x7f47d00040c0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.656 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.380+0000 7f47db3bb640 1 -- 172.21.15.64:0/4244661429 shutdown_connections 2024-09-21T04:11:42.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.380+0000 7f47db3bb640 1 --2- 172.21.15.64:0/4244661429 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f47c00b6680 0x7f47c00b8b90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.380+0000 7f47db3bb640 1 --2- 172.21.15.64:0/4244661429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47d4108cf0 0x7f47d419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.380+0000 7f47db3bb640 1 --2- 172.21.15.64:0/4244661429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47d41083a0 0x7f47d419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.380+0000 7f47db3bb640 1 --2- 172.21.15.64:0/4244661429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f47d4102a60 0x7f47d419a440 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:42.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.380+0000 7f47db3bb640 1 -- 172.21.15.64:0/4244661429 >> 172.21.15.64:0/4244661429 conn(0x7f47d4079730 msgr2=0x7f47d4101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:42.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.380+0000 7f47db3bb640 1 -- 172.21.15.64:0/4244661429 shutdown_connections 2024-09-21T04:11:42.657 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:42.380+0000 7f47db3bb640 1 -- 172.21.15.64:0/4244661429 wait complete. 2024-09-21T04:11:42.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:42 smithi119 ceph-mon[34811]: pgmap v555: 97 pgs: 97 active+clean; 813 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 451 op/s 2024-09-21T04:11:43.380 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:43 smithi064 ceph-mon[31180]: scrub queued for path: / 2024-09-21T04:11:43.380 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:43 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:11:43.380 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:43 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:11:43.380 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:43 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:11:43.380 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:43 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:11:43.380 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:43 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:11:43.380 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:43 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:11:43.380 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:43 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:11:43.380 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:43 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:11:43.408 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T04:11:43.683 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:11:43.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:43 smithi119 ceph-mon[34811]: scrub queued for path: / 2024-09-21T04:11:43.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:43 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:11:43.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:43 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:11:43.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:43 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:11:43.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:43 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:11:43.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:43 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:11:43.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:43 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:11:43.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:43 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:11:43.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:43 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:11:43.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:43 smithi131 ceph-mon[35193]: scrub queued for path: / 2024-09-21T04:11:43.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:43 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:11:43.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:43 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:11:43.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:43 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:11:43.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:43 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:11:43.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:43 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:11:43.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:43 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:11:43.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:43 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:11:43.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:43 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:11:44.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.253+0000 7f3630f9a640 1 -- 172.21.15.64:0/4223769412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f362c109e00 msgr2=0x7f362c110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:44.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.253+0000 7f3630f9a640 1 --2- 172.21.15.64:0/4223769412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f362c109e00 0x7f362c110630 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f3624009ca0 tx=0x7f362402a8c0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.254+0000 7f3630f9a640 1 -- 172.21.15.64:0/4223769412 shutdown_connections 2024-09-21T04:11:44.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.254+0000 7f3630f9a640 1 --2- 172.21.15.64:0/4223769412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f362c109e00 0x7f362c110630 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.254+0000 7f3630f9a640 1 --2- 172.21.15.64:0/4223769412 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f362c1094b0 0x7f362c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.254+0000 7f3630f9a640 1 --2- 172.21.15.64:0/4223769412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f362c103bc0 0x7f362c103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.254+0000 7f3630f9a640 1 -- 172.21.15.64:0/4223769412 >> 172.21.15.64:0/4223769412 conn(0x7f362c07b910 msgr2=0x7f362c07bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:44.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.254+0000 7f3630f9a640 1 -- 172.21.15.64:0/4223769412 shutdown_connections 2024-09-21T04:11:44.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.254+0000 7f3630f9a640 1 -- 172.21.15.64:0/4223769412 wait complete. 2024-09-21T04:11:44.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.255+0000 7f3630f9a640 1 Processor -- start 2024-09-21T04:11:44.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.255+0000 7f3630f9a640 1 -- start start 2024-09-21T04:11:44.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.255+0000 7f3630f9a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f362c103bc0 0x7f362c19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:44.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.255+0000 7f3630f9a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f362c1094b0 0x7f362c19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:44.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.255+0000 7f3630f9a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f362c109e00 0x7f362c19fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:44.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.255+0000 7f3630f9a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f362c113ac0 con 0x7f362c109e00 2024-09-21T04:11:44.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.255+0000 7f3630f9a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f362c113940 con 0x7f362c1094b0 2024-09-21T04:11:44.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.255+0000 7f3630f9a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f362c113c40 con 0x7f362c103bc0 2024-09-21T04:11:44.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.255+0000 7f3629d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f362c1094b0 0x7f362c19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:44.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.255+0000 7f362ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f362c109e00 0x7f362c19fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:44.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.255+0000 7f3629d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f362c1094b0 0x7f362c19aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:37310/0 (socket says 172.21.15.64:37310) 2024-09-21T04:11:44.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.255+0000 7f3629d9b640 1 -- 172.21.15.64:0/2405455503 learned_addr learned my addr 172.21.15.64:0/2405455503 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:44.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.255+0000 7f362a59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f362c103bc0 0x7f362c19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:44.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.255+0000 7f362ad9d640 1 --2- 172.21.15.64:0/2405455503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f362c109e00 0x7f362c19fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41084/0 (socket says 172.21.15.64:41084) 2024-09-21T04:11:44.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.255+0000 7f3629d9b640 1 -- 172.21.15.64:0/2405455503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f362c103bc0 msgr2=0x7f362c19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:44.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.255+0000 7f3629d9b640 1 --2- 172.21.15.64:0/2405455503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f362c103bc0 0x7f362c19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.255+0000 7f3629d9b640 1 -- 172.21.15.64:0/2405455503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f362c109e00 msgr2=0x7f362c19fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:44.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.255+0000 7f3629d9b640 1 --2- 172.21.15.64:0/2405455503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f362c109e00 0x7f362c19fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.256+0000 7f3629d9b640 1 -- 172.21.15.64:0/2405455503 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f362c1a0460 con 0x7f362c1094b0 2024-09-21T04:11:44.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.256+0000 7f362ad9d640 1 --2- 172.21.15.64:0/2405455503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f362c109e00 0x7f362c19fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:11:44.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.256+0000 7f3629d9b640 1 --2- 172.21.15.64:0/2405455503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f362c1094b0 0x7f362c19aa90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f361400d750 tx=0x7f361400dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:44.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.256+0000 7f36237fe640 1 -- 172.21.15.64:0/2405455503 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3614017070 con 0x7f362c1094b0 2024-09-21T04:11:44.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.256+0000 7f3630f9a640 1 -- 172.21.15.64:0/2405455503 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f362c1a0750 con 0x7f362c1094b0 2024-09-21T04:11:44.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.256+0000 7f36237fe640 1 -- 172.21.15.64:0/2405455503 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36140054d0 con 0x7f362c1094b0 2024-09-21T04:11:44.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.256+0000 7f36237fe640 1 -- 172.21.15.64:0/2405455503 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3614004c30 con 0x7f362c1094b0 2024-09-21T04:11:44.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.256+0000 7f3630f9a640 1 -- 172.21.15.64:0/2405455503 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f362c1a6eb0 con 0x7f362c1094b0 2024-09-21T04:11:44.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.259+0000 7f363750e740 1 Processor -- start 2024-09-21T04:11:44.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.259+0000 7f363750e740 1 -- start start 2024-09-21T04:11:44.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.260+0000 7f363750e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca87070 0x5556dce2e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:44.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.260+0000 7f363750e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556dca5e1b0 0x5556dce2eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:44.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.260+0000 7f363750e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556dca66390 0x5556dce2f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:44.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.260+0000 7f363750e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5556dcda4e50 con 0x5556dca87070 2024-09-21T04:11:44.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.260+0000 7f363750e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5556dcd94280 con 0x5556dca5e1b0 2024-09-21T04:11:44.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.260+0000 7f363750e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5556dcd98090 con 0x5556dca66390 2024-09-21T04:11:44.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.260+0000 7f362ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca87070 0x5556dce2e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:44.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.260+0000 7f362ad9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca87070 0x5556dce2e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41090/0 (socket says 172.21.15.64:41090) 2024-09-21T04:11:44.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.260+0000 7f362ad9d640 1 -- 172.21.15.64:0/2200982541 learned_addr learned my addr 172.21.15.64:0/2200982541 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:44.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.260+0000 7f362a59c640 1 --2- 172.21.15.64:0/2200982541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556dca5e1b0 0x5556dce2eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:44.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.260+0000 7f36237fe640 1 -- 172.21.15.64:0/2405455503 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f3614004dd0 con 0x7f362c1094b0 2024-09-21T04:11:44.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.260+0000 7f362ad9d640 1 -- 172.21.15.64:0/2200982541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556dca66390 msgr2=0x5556dce2f0d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T04:11:44.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.260+0000 7f362ad9d640 1 --2- 172.21.15.64:0/2200982541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556dca66390 0x5556dce2f0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.260+0000 7f362ad9d640 1 -- 172.21.15.64:0/2200982541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556dca5e1b0 msgr2=0x5556dce2eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:44.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.260+0000 7f362ad9d640 1 --2- 172.21.15.64:0/2200982541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556dca5e1b0 0x5556dce2eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.260+0000 7f362ad9d640 1 -- 172.21.15.64:0/2200982541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556dcaf2650 con 0x5556dca87070 2024-09-21T04:11:44.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.260+0000 7f362a59c640 1 --2- 172.21.15.64:0/2200982541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556dca5e1b0 0x5556dce2eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:11:44.264 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.261+0000 7f362ad9d640 1 --2- 172.21.15.64:0/2200982541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca87070 0x5556dce2e650 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f3624033040 tx=0x7f362402f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9d68fb3c3c8f6c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:44.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.261+0000 7f3620ff9640 1 -- 172.21.15.64:0/2200982541 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3624031650 con 0x5556dca87070 2024-09-21T04:11:44.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.261+0000 7f3620ff9640 1 -- 172.21.15.64:0/2200982541 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3624036040 con 0x5556dca87070 2024-09-21T04:11:44.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.261+0000 7f3620ff9640 1 -- 172.21.15.64:0/2200982541 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36240044e0 con 0x5556dca87070 2024-09-21T04:11:44.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.261+0000 7f36237fe640 1 --2- 172.21.15.64:0/2405455503 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f36180b6680 0x7f36180b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:44.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.261+0000 7f362a59c640 1 --2- 172.21.15.64:0/2405455503 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f36180b6680 0x7f36180b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:44.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.261+0000 7f36237fe640 1 -- 172.21.15.64:0/2405455503 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7f36140e7050 con 0x7f362c1094b0 2024-09-21T04:11:44.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.262+0000 7f363750e740 1 -- 172.21.15.64:0/2200982541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca87070 msgr2=0x5556dce2e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:44.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.262+0000 7f362a59c640 1 --2- 172.21.15.64:0/2405455503 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f36180b6680 0x7f36180b8b90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x5556dce2d2d0 tx=0x7f361c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:44.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.262+0000 7f363750e740 1 --2- 172.21.15.64:0/2200982541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca87070 0x5556dce2e650 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f3624033040 tx=0x7f362402f3c0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.262+0000 7f363750e740 1 -- 172.21.15.64:0/2200982541 shutdown_connections 2024-09-21T04:11:44.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.262+0000 7f363750e740 1 --2- 172.21.15.64:0/2200982541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556dca66390 0x5556dce2f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.262+0000 7f363750e740 1 --2- 172.21.15.64:0/2200982541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556dca5e1b0 0x5556dce2eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.262+0000 7f363750e740 1 --2- 172.21.15.64:0/2200982541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca87070 0x5556dce2e650 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.262+0000 7f363750e740 1 -- 172.21.15.64:0/2200982541 >> 172.21.15.64:0/2200982541 conn(0x5556dca7b460 msgr2=0x5556dce2c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:44.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.262+0000 7f363750e740 1 -- 172.21.15.64:0/2200982541 shutdown_connections 2024-09-21T04:11:44.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.262+0000 7f363750e740 1 -- 172.21.15.64:0/2200982541 wait complete. 2024-09-21T04:11:44.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.263+0000 7f363750e740 1 -- start start 2024-09-21T04:11:44.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.263+0000 7f363750e740 1 Processor -- start 2024-09-21T04:11:44.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.263+0000 7f363750e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca5e1b0 0x5556dce39de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:44.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.263+0000 7f363750e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556dca66390 0x5556dce3a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:44.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.263+0000 7f363750e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556dca87070 0x5556dce45f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:44.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.263+0000 7f363750e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5556dcbf5700 con 0x5556dca5e1b0 2024-09-21T04:11:44.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.263+0000 7f363750e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5556dcd2f110 con 0x5556dca66390 2024-09-21T04:11:44.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.263+0000 7f363750e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5556dce3cc50 con 0x5556dca87070 2024-09-21T04:11:44.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.263+0000 7f363750e740 10 client.0 _ll_register_callbacks cb 0x5556dcd48990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:11:44.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.263+0000 7f362a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca5e1b0 0x5556dce39de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:44.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.263+0000 7f362a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca5e1b0 0x5556dce39de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41102/0 (socket says 172.21.15.64:41102) 2024-09-21T04:11:44.269 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.263+0000 7f362a59c640 1 -- 172.21.15.64:0/1131159713 learned_addr learned my addr 172.21.15.64:0/1131159713 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:44.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.263+0000 7f3629d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556dca66390 0x5556dce3a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:44.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.264+0000 7f362ad9d640 1 --2- 172.21.15.64:0/1131159713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556dca87070 0x5556dce45f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:44.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.264+0000 7f3629d9b640 1 -- 172.21.15.64:0/1131159713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556dca87070 msgr2=0x5556dce45f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:44.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.264+0000 7f3629d9b640 1 --2- 172.21.15.64:0/1131159713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556dca87070 0x5556dce45f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.264+0000 7f3629d9b640 1 -- 172.21.15.64:0/1131159713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca5e1b0 msgr2=0x5556dce39de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:44.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.264+0000 7f3629d9b640 1 --2- 172.21.15.64:0/1131159713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca5e1b0 0x5556dce39de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.264+0000 7f3629d9b640 1 -- 172.21.15.64:0/1131159713 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556dcd5cc40 con 0x5556dca66390 2024-09-21T04:11:44.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.264+0000 7f362a59c640 1 --2- 172.21.15.64:0/1131159713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca5e1b0 0x5556dce39de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:11:44.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.264+0000 7f362ad9d640 1 --2- 172.21.15.64:0/1131159713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556dca87070 0x5556dce45f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:11:44.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.264+0000 7f3629d9b640 1 --2- 172.21.15.64:0/1131159713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556dca66390 0x5556dce3a320 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f36140e4740 tx=0x7f36140acdc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d92fc19c37f9419 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:44.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.264+0000 7f3611ffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:11:44.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.264+0000 7f3611ffb640 1 -- 172.21.15.64:0/1131159713 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36140af030 con 0x5556dca66390 2024-09-21T04:11:44.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.264+0000 7f363750e740 1 -- 172.21.15.64:0/1131159713 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3624036040 con 0x5556dca66390 2024-09-21T04:11:44.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.264+0000 7f3611ffb640 1 -- 172.21.15.64:0/1131159713 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7f36140a9160 con 0x5556dca66390 2024-09-21T04:11:44.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.264+0000 7f3611ffb640 1 -- 172.21.15.64:0/1131159713 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3614020070 con 0x5556dca66390 2024-09-21T04:11:44.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.264+0000 7f363750e740 10 client.26798 fetch_fsmap learned FSMap version 39 2024-09-21T04:11:44.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.264+0000 7f363750e740 1 -- 172.21.15.64:0/1131159713 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5556dce48a40 con 0x5556dca66390 2024-09-21T04:11:44.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.264+0000 7f3611ffb640 1 -- 172.21.15.64:0/1131159713 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3624036040 con 0x5556dca66390 2024-09-21T04:11:44.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.265+0000 7f3611ffb640 1 -- 172.21.15.64:0/1131159713 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f36140d4190 con 0x5556dca66390 2024-09-21T04:11:44.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.265+0000 7f363750e740 10 client.26798 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T04:11:44.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.265+0000 7f363750e740 10 client.26798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:11:44.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.265+0000 7f363750e740 20 client.26798 populate_metadata read hostname 'smithi064' 2024-09-21T04:11:44.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.265+0000 7f363750e740 1 --2- 172.21.15.64:0/1131159713 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5556dca459f0 0x5556dce4c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:44.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.265+0000 7f363750e740 4 client.26798 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:11:44.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.265+0000 7f363750e740 1 -- 172.21.15.64:0/1131159713 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556dcda6da0 con 0x5556dca459f0 2024-09-21T04:11:44.271 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.265+0000 7f362a59c640 1 --2- 172.21.15.64:0/1131159713 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5556dca459f0 0x5556dce4c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:44.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.265+0000 7f362a59c640 1 --2- 172.21.15.64:0/1131159713 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5556dca459f0 0x5556dce4c030 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=297797d32f4389d3 server_cookie=5130ddc6f99e4a67 in_seq=0 out_seq=0 2024-09-21T04:11:44.272 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.265+0000 7f3611ffb640 10 client.26798 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:44.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.266+0000 7f3611ffb640 1 -- 172.21.15.64:0/1131159713 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5556dcda6da0 con 0x5556dca459f0 2024-09-21T04:11:44.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.266+0000 7f3611ffb640 10 client.26798 handle_command_reply: tid=0 2024-09-21T04:11:44.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.266+0000 7f363750e740 1 client.26798 shutdown 2024-09-21T04:11:44.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.266+0000 7f363750e740 1 -- 172.21.15.64:0/1131159713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556dca66390 msgr2=0x5556dce3a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:44.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.266+0000 7f363750e740 1 --2- 172.21.15.64:0/1131159713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556dca66390 0x5556dce3a320 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f36140e4740 tx=0x7f36140acdc0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.266+0000 7f363750e740 1 -- 172.21.15.64:0/1131159713 shutdown_connections 2024-09-21T04:11:44.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.266+0000 7f363750e740 1 --2- 172.21.15.64:0/1131159713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556dca87070 0x5556dce45f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.266+0000 7f363750e740 1 --2- 172.21.15.64:0/1131159713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556dca66390 0x5556dce3a320 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.266+0000 7f363750e740 1 --2- 172.21.15.64:0/1131159713 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5556dca459f0 0x5556dce4c030 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.266+0000 7f363750e740 1 --2- 172.21.15.64:0/1131159713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca5e1b0 0x5556dce39de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.266+0000 7f3611ffb640 0 client.26798 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:44.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.266+0000 7f363750e740 1 -- 172.21.15.64:0/1131159713 >> 172.21.15.64:0/1131159713 conn(0x5556dca7b460 msgr2=0x5556dce31830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:44.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.266+0000 7f363750e740 1 -- 172.21.15.64:0/1131159713 shutdown_connections 2024-09-21T04:11:44.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.266+0000 7f363750e740 1 -- 172.21.15.64:0/1131159713 wait complete. 2024-09-21T04:11:44.273 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.267+0000 7f363750e740 20 client.26798 trim_cache size 0 max 16384 2024-09-21T04:11:44.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.284+0000 7f363750e740 1 Processor -- start 2024-09-21T04:11:44.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.284+0000 7f363750e740 1 -- start start 2024-09-21T04:11:44.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.285+0000 7f363750e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556dca5e1b0 0x5556dce32110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:44.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.285+0000 7f363750e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca66390 0x5556dce39d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:44.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.285+0000 7f363750e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556dca87070 0x5556dce42280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:44.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.285+0000 7f363750e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1b21f0 con 0x5556dca66390 2024-09-21T04:11:44.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.285+0000 7f363750e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1afa30 con 0x5556dca5e1b0 2024-09-21T04:11:44.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.285+0000 7f363750e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1ad8a0 con 0x5556dca87070 2024-09-21T04:11:44.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.285+0000 7f3629d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca66390 0x5556dce39d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:44.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.285+0000 7f3629d9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca66390 0x5556dce39d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41116/0 (socket says 172.21.15.64:41116) 2024-09-21T04:11:44.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.285+0000 7f3629d9b640 1 -- 172.21.15.64:0/1850646394 learned_addr learned my addr 172.21.15.64:0/1850646394 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:44.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.285+0000 7f362a59c640 1 --2- 172.21.15.64:0/1850646394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556dca5e1b0 0x5556dce32110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:44.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.285+0000 7f362ad9d640 1 --2- 172.21.15.64:0/1850646394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556dca87070 0x5556dce42280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:44.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.285+0000 7f3629d9b640 1 -- 172.21.15.64:0/1850646394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556dca87070 msgr2=0x5556dce42280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:44.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.285+0000 7f3629d9b640 1 --2- 172.21.15.64:0/1850646394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556dca87070 0x5556dce42280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.285+0000 7f3629d9b640 1 -- 172.21.15.64:0/1850646394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556dca5e1b0 msgr2=0x5556dce32110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:44.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.285+0000 7f3629d9b640 1 --2- 172.21.15.64:0/1850646394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556dca5e1b0 0x5556dce32110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.285+0000 7f3629d9b640 1 -- 172.21.15.64:0/1850646394 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556dce2d300 con 0x5556dca66390 2024-09-21T04:11:44.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.285+0000 7f362a59c640 1 --2- 172.21.15.64:0/1850646394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556dca5e1b0 0x5556dce32110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:11:44.289 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.285+0000 7f3629d9b640 1 --2- 172.21.15.64:0/1850646394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca66390 0x5556dce39d50 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f36140acbf0 tx=0x7f3614009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f45516d22b60ae3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:44.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.286+0000 7f3613fff640 1 -- 172.21.15.64:0/1850646394 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36140af070 con 0x5556dca66390 2024-09-21T04:11:44.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.286+0000 7f3613fff640 1 -- 172.21.15.64:0/1850646394 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36140cee50 con 0x5556dca66390 2024-09-21T04:11:44.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.286+0000 7f3613fff640 1 -- 172.21.15.64:0/1850646394 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36140a9160 con 0x5556dca66390 2024-09-21T04:11:44.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.286+0000 7f363750e740 1 -- 172.21.15.64:0/1850646394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca66390 msgr2=0x5556dce39d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:44.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.286+0000 7f363750e740 1 --2- 172.21.15.64:0/1850646394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca66390 0x5556dce39d50 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f36140acbf0 tx=0x7f3614009f90 comp rx=0 tx=0).stop 2024-09-21T04:11:44.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.286+0000 7f363750e740 1 -- 172.21.15.64:0/1850646394 shutdown_connections 2024-09-21T04:11:44.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.286+0000 7f363750e740 1 --2- 172.21.15.64:0/1850646394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556dca87070 0x5556dce42280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.286+0000 7f363750e740 1 --2- 172.21.15.64:0/1850646394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca66390 0x5556dce39d50 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.286+0000 7f363750e740 1 --2- 172.21.15.64:0/1850646394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556dca5e1b0 0x5556dce32110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.286+0000 7f363750e740 1 -- 172.21.15.64:0/1850646394 >> 172.21.15.64:0/1850646394 conn(0x5556dca7b460 msgr2=0x5556dce38660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:44.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.286+0000 7f363750e740 1 -- 172.21.15.64:0/1850646394 shutdown_connections 2024-09-21T04:11:44.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.287+0000 7f363750e740 1 -- 172.21.15.64:0/1850646394 wait complete. 2024-09-21T04:11:44.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.287+0000 7f363750e740 1 -- start start 2024-09-21T04:11:44.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.287+0000 7f363750e740 1 Processor -- start 2024-09-21T04:11:44.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.287+0000 7f363750e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca5e1b0 0x5556dceaeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:44.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.287+0000 7f363750e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556dca66390 0x5556dceaf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:44.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.287+0000 7f363750e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556dca87070 0x5556dceb3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:44.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.287+0000 7f363750e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1adef0 con 0x5556dca5e1b0 2024-09-21T04:11:44.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.287+0000 7f363750e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1af360 con 0x5556dca66390 2024-09-21T04:11:44.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.287+0000 7f363750e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1ae5c0 con 0x5556dca87070 2024-09-21T04:11:44.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.287+0000 7f363750e740 10 client.0 _ll_register_callbacks cb 0x7f361400d6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:11:44.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.288+0000 7f362a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca5e1b0 0x5556dceaeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:44.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.288+0000 7f362a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca5e1b0 0x5556dceaeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41120/0 (socket says 172.21.15.64:41120) 2024-09-21T04:11:44.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.288+0000 7f362a59c640 1 -- 172.21.15.64:0/111327890 learned_addr learned my addr 172.21.15.64:0/111327890 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:44.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.288+0000 7f3629d9b640 1 --2- 172.21.15.64:0/111327890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556dca66390 0x5556dceaf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:44.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.288+0000 7f362ad9d640 1 --2- 172.21.15.64:0/111327890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556dca87070 0x5556dceb3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:44.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.288+0000 7f362a59c640 1 -- 172.21.15.64:0/111327890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556dca87070 msgr2=0x5556dceb3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:44.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.288+0000 7f362a59c640 1 --2- 172.21.15.64:0/111327890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556dca87070 0x5556dceb3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.288+0000 7f362a59c640 1 -- 172.21.15.64:0/111327890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556dca66390 msgr2=0x5556dceaf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:44.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.288+0000 7f362a59c640 1 --2- 172.21.15.64:0/111327890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556dca66390 0x5556dceaf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.288+0000 7f362a59c640 1 -- 172.21.15.64:0/111327890 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556dceb40c0 con 0x5556dca5e1b0 2024-09-21T04:11:44.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.288+0000 7f362a59c640 1 --2- 172.21.15.64:0/111327890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca5e1b0 0x5556dceaeb90 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f361c012cb0 tx=0x7f361c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51471ff9e3ed863c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:44.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.288+0000 7f3611ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:11:44.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.288+0000 7f3611ffb640 1 -- 172.21.15.64:0/111327890 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361c036060 con 0x5556dca5e1b0 2024-09-21T04:11:44.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.288+0000 7f3611ffb640 1 -- 172.21.15.64:0/111327890 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7f361c018030 con 0x5556dca5e1b0 2024-09-21T04:11:44.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.288+0000 7f363750e740 1 -- 172.21.15.64:0/111327890 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36140cee50 con 0x5556dca5e1b0 2024-09-21T04:11:44.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.289+0000 7f3611ffb640 1 -- 172.21.15.64:0/111327890 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f361c01f430 con 0x5556dca5e1b0 2024-09-21T04:11:44.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.289+0000 7f363750e740 10 client.17133 fetch_fsmap learned FSMap version 39 2024-09-21T04:11:44.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.289+0000 7f363750e740 1 -- 172.21.15.64:0/111327890 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5556dcebb460 con 0x5556dca5e1b0 2024-09-21T04:11:44.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.289+0000 7f3611ffb640 1 -- 172.21.15.64:0/111327890 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36140cee50 con 0x5556dca5e1b0 2024-09-21T04:11:44.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.289+0000 7f3611ffb640 1 -- 172.21.15.64:0/111327890 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f361c0315c0 con 0x5556dca5e1b0 2024-09-21T04:11:44.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.289+0000 7f363750e740 10 client.17133 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T04:11:44.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.289+0000 7f363750e740 10 client.17133 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:11:44.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.289+0000 7f363750e740 20 client.17133 populate_metadata read hostname 'smithi064' 2024-09-21T04:11:44.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.289+0000 7f363750e740 1 --2- 172.21.15.64:0/111327890 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5556dca1ada0 0x5556dcebdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:44.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.289+0000 7f363750e740 4 client.17133 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T04:11:44.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.289+0000 7f363750e740 1 -- 172.21.15.64:0/111327890 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5556dcda2850 con 0x5556dca1ada0 2024-09-21T04:11:44.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.289+0000 7f3629d9b640 1 --2- 172.21.15.64:0/111327890 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5556dca1ada0 0x5556dcebdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:44.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.290+0000 7f3629d9b640 1 --2- 172.21.15.64:0/111327890 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5556dca1ada0 0x5556dcebdbb0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6feae5888d5cee00 server_cookie=f773337f7aac7b4 in_seq=0 out_seq=0 2024-09-21T04:11:44.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.290+0000 7f3611ffb640 10 client.17133 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:44.296 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.290+0000 7f3611ffb640 1 -- 172.21.15.64:0/111327890 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3410 (crc 0 0 0) 0x5556dcda2850 con 0x5556dca1ada0 2024-09-21T04:11:44.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.290+0000 7f3611ffb640 10 client.17133 handle_command_reply: tid=0 2024-09-21T04:11:44.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.290+0000 7f363750e740 1 client.17133 shutdown 2024-09-21T04:11:44.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.290+0000 7f363750e740 1 -- 172.21.15.64:0/111327890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca5e1b0 msgr2=0x5556dceaeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:44.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.290+0000 7f363750e740 1 --2- 172.21.15.64:0/111327890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca5e1b0 0x5556dceaeb90 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f361c012cb0 tx=0x7f361c0046c0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.290+0000 7f363750e740 1 -- 172.21.15.64:0/111327890 shutdown_connections 2024-09-21T04:11:44.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.290+0000 7f363750e740 1 --2- 172.21.15.64:0/111327890 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556dca87070 0x5556dceb3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.290+0000 7f363750e740 1 --2- 172.21.15.64:0/111327890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5556dca66390 0x5556dceaf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.290+0000 7f363750e740 1 --2- 172.21.15.64:0/111327890 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5556dca1ada0 0x5556dcebdbb0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.290+0000 7f363750e740 1 --2- 172.21.15.64:0/111327890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5556dca5e1b0 0x5556dceaeb90 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.290+0000 7f3611ffb640 0 client.17133 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:44.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.290+0000 7f363750e740 1 -- 172.21.15.64:0/111327890 >> 172.21.15.64:0/111327890 conn(0x5556dca7b460 msgr2=0x5556dce31750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:44.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.290+0000 7f363750e740 1 -- 172.21.15.64:0/111327890 shutdown_connections 2024-09-21T04:11:44.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.290+0000 7f363750e740 1 -- 172.21.15.64:0/111327890 wait complete. 2024-09-21T04:11:44.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.290+0000 7f363750e740 20 client.17133 trim_cache size 0 max 16384 2024-09-21T04:11:44.298 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:11:44.298 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T04:11:44.298 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-09-21T04:11:44.298 INFO:teuthology.orchestra.run.smithi064.stdout: "71f3f255-4837-45e9-92bc-4b3abb5c20eb": { 2024-09-21T04:11:44.298 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:44.298 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "71f3f255-4837-45e9-92bc-4b3abb5c20eb", 2024-09-21T04:11:44.298 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-09-21T04:11:44.298 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:11:44.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:44.298 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_stats": { 2024-09-21T04:11:44.298 INFO:teuthology.orchestra.run.smithi064.stdout: "1b23713d-ea3a-4a86-a87f-65f6ab82671d": { 2024-09-21T04:11:44.299 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:10:58.352639+0000", 2024-09-21T04:11:44.299 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:44.299 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:44.299 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:44.299 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:44.299 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:44.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:44.299 INFO:teuthology.orchestra.run.smithi064.stdout: "71f3f255-4837-45e9-92bc-4b3abb5c20eb": { 2024-09-21T04:11:44.299 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:11:42.378549+0000", 2024-09-21T04:11:44.299 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:44.299 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:44.299 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:44.299 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:44.299 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:44.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:44.300 INFO:teuthology.orchestra.run.smithi064.stdout: "2c0523fb-aae4-43fa-9568-136a9b640051": { 2024-09-21T04:11:44.300 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:09:39.808606+0000", 2024-09-21T04:11:44.300 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:44.300 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:44.300 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:44.300 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:44.300 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:44.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:44.300 INFO:teuthology.orchestra.run.smithi064.stdout: "bed40d88-2102-43a6-ac2d-b89d5a2264d0": { 2024-09-21T04:11:44.300 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:11.363340+0000", 2024-09-21T04:11:44.300 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:44.300 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:44.300 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:44.301 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:44.301 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:44.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:44.301 INFO:teuthology.orchestra.run.smithi064.stdout: "e9242f88-6599-40eb-adf1-bdb3e946a1e0": { 2024-09-21T04:11:44.301 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:05:39.744646+0000", 2024-09-21T04:11:44.301 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:44.301 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:44.301 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:44.301 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:44.301 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:44.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:44.301 INFO:teuthology.orchestra.run.smithi064.stdout: "6bde0fe5-f775-44b1-ad0c-f9e596574dc0": { 2024-09-21T04:11:44.301 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:06:23.835536+0000", 2024-09-21T04:11:44.301 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:44.301 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:44.302 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:44.302 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:44.302 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:44.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:44.302 INFO:teuthology.orchestra.run.smithi064.stdout: "b52b950c-0f97-49ec-a37c-1a8e04348ce8": { 2024-09-21T04:11:44.302 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:54.003800+0000", 2024-09-21T04:11:44.302 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:44.302 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:44.302 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:44.302 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:44.302 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:44.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:44.302 INFO:teuthology.orchestra.run.smithi064.stdout: "e2c2eed8-9292-4bf6-8a39-13ad27da5acb": { 2024-09-21T04:11:44.303 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:08:47.067932+0000", 2024-09-21T04:11:44.303 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:44.303 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:44.303 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:44.303 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:44.303 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:44.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:44.303 INFO:teuthology.orchestra.run.smithi064.stdout: "75db1bc5-612e-45aa-bad2-cc90feeaa840": { 2024-09-21T04:11:44.303 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:59.123138+0000", 2024-09-21T04:11:44.303 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:44.303 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:44.303 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:44.303 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:44.303 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:44.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:44.304 INFO:teuthology.orchestra.run.smithi064.stdout: "0ab717a7-36c6-4dca-94e7-af0bdf2aa112": { 2024-09-21T04:11:44.304 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:42.691086+0000", 2024-09-21T04:11:44.304 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:44.304 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:44.304 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:44.304 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:44.304 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:44.304 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:11:44.304 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:11:44.304 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:03:59.570174+0000", 2024-09-21T04:11:44.304 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:11:44.304 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:11:44.304 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:11:44.304 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:11:44.305 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:11:44.305 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:11:44.305 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:11:44.305 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:11:44.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.291+0000 7f3630f9a640 1 -- 172.21.15.64:0/2405455503 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f36180b6680 msgr2=0x7f36180b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:44.305 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.291+0000 7f3630f9a640 1 --2- 172.21.15.64:0/2405455503 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f36180b6680 0x7f36180b8b90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x5556dce2d2d0 tx=0x7f361c005f90 comp rx=0 tx=0).stop 2024-09-21T04:11:44.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.291+0000 7f3630f9a640 1 -- 172.21.15.64:0/2405455503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f362c1094b0 msgr2=0x7f362c19aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:44.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.291+0000 7f3630f9a640 1 --2- 172.21.15.64:0/2405455503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f362c1094b0 0x7f362c19aa90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f361400d750 tx=0x7f361400dc10 comp rx=0 tx=0).stop 2024-09-21T04:11:44.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.291+0000 7f3630f9a640 1 -- 172.21.15.64:0/2405455503 shutdown_connections 2024-09-21T04:11:44.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.291+0000 7f3630f9a640 1 --2- 172.21.15.64:0/2405455503 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f36180b6680 0x7f36180b8b90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.291+0000 7f3630f9a640 1 --2- 172.21.15.64:0/2405455503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f362c109e00 0x7f362c19fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.291+0000 7f3630f9a640 1 --2- 172.21.15.64:0/2405455503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f362c1094b0 0x7f362c19aa90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.291+0000 7f3630f9a640 1 --2- 172.21.15.64:0/2405455503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f362c103bc0 0x7f362c19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:44.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.291+0000 7f3630f9a640 1 -- 172.21.15.64:0/2405455503 >> 172.21.15.64:0/2405455503 conn(0x7f362c07b910 msgr2=0x7f362c107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:44.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.291+0000 7f3630f9a640 1 -- 172.21.15.64:0/2405455503 shutdown_connections 2024-09-21T04:11:44.306 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:11:44.291+0000 7f3630f9a640 1 -- 172.21.15.64:0/2405455503 wait complete. 2024-09-21T04:11:44.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:44 smithi064 ceph-mon[31180]: pgmap v556: 97 pgs: 97 active+clean; 831 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.9 MiB/s wr, 654 op/s 2024-09-21T04:11:44.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:44 smithi119 ceph-mon[34811]: pgmap v556: 97 pgs: 97 active+clean; 831 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.9 MiB/s wr, 654 op/s 2024-09-21T04:11:44.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:44 smithi131 ceph-mon[35193]: pgmap v556: 97 pgs: 97 active+clean; 831 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.9 MiB/s wr, 654 op/s 2024-09-21T04:11:45.015 INFO:tasks.cephfs.filesystem:scrub status for tag:71f3f255-4837-45e9-92bc-4b3abb5c20eb - {'path': '/', 'tag': '71f3f255-4837-45e9-92bc-4b3abb5c20eb', 'options': 'recursive,force'} 2024-09-21T04:11:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.084+0000 7f7366a8b640 1 -- 172.21.15.64:0/3125583519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7360106b50 msgr2=0x7f73601105c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.084+0000 7f7366a8b640 1 --2- 172.21.15.64:0/3125583519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7360106b50 0x7f73601105c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f7358009900 tx=0x7f735802a410 comp rx=0 tx=0).stop 2024-09-21T04:11:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.084+0000 7f7366a8b640 1 -- 172.21.15.64:0/3125583519 shutdown_connections 2024-09-21T04:11:45.087 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.084+0000 7f7366a8b640 1 --2- 172.21.15.64:0/3125583519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7360106b50 0x7f73601105c0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.084+0000 7f7366a8b640 1 --2- 172.21.15.64:0/3125583519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7360106200 0x7f7360106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.084+0000 7f7366a8b640 1 --2- 172.21.15.64:0/3125583519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7360102090 0x7f73601024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.084+0000 7f7366a8b640 1 -- 172.21.15.64:0/3125583519 >> 172.21.15.64:0/3125583519 conn(0x7f7360079ec0 msgr2=0x7f736007a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.084+0000 7f7366a8b640 1 -- 172.21.15.64:0/3125583519 shutdown_connections 2024-09-21T04:11:45.088 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.084+0000 7f7366a8b640 1 -- 172.21.15.64:0/3125583519 wait complete. 2024-09-21T04:11:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.085+0000 7f7366a8b640 1 Processor -- start 2024-09-21T04:11:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.085+0000 7f7366a8b640 1 -- start start 2024-09-21T04:11:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.085+0000 7f7366a8b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7360102090 0x7f736019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.085+0000 7f7366a8b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7360106200 0x7f736019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.085+0000 7f7366a8b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7360106b50 0x7f736019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.085+0000 7f7366a8b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7360113a70 con 0x7f7360106200 2024-09-21T04:11:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.085+0000 7f7366a8b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f73601138f0 con 0x7f7360102090 2024-09-21T04:11:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.085+0000 7f7366a8b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7360113bf0 con 0x7f7360106b50 2024-09-21T04:11:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.086+0000 7f735ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7360106200 0x7f736019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.086+0000 7f735ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7360106200 0x7f736019a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41144/0 (socket says 172.21.15.64:41144) 2024-09-21T04:11:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.086+0000 7f7364827640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7360102090 0x7f736019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.086+0000 7f735ffff640 1 -- 172.21.15.64:0/832899011 learned_addr learned my addr 172.21.15.64:0/832899011 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:45.090 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.086+0000 7f7365028640 1 --2- 172.21.15.64:0/832899011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7360106b50 0x7f736019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.086+0000 7f735ffff640 1 -- 172.21.15.64:0/832899011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7360106b50 msgr2=0x7f736019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.086+0000 7f735ffff640 1 --2- 172.21.15.64:0/832899011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7360106b50 0x7f736019fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.086+0000 7f735ffff640 1 -- 172.21.15.64:0/832899011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7360102090 msgr2=0x7f736019a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.086+0000 7f735ffff640 1 --2- 172.21.15.64:0/832899011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7360102090 0x7f736019a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.086+0000 7f735ffff640 1 -- 172.21.15.64:0/832899011 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73601a0380 con 0x7f7360106200 2024-09-21T04:11:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.086+0000 7f735ffff640 1 --2- 172.21.15.64:0/832899011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7360106200 0x7f736019a9f0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f735000eb90 tx=0x7f735000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.086+0000 7f735dffb640 1 -- 172.21.15.64:0/832899011 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7350004d20 con 0x7f7360106200 2024-09-21T04:11:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.086+0000 7f7366a8b640 1 -- 172.21.15.64:0/832899011 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73601a0670 con 0x7f7360106200 2024-09-21T04:11:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.086+0000 7f735dffb640 1 -- 172.21.15.64:0/832899011 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f735000ee90 con 0x7f7360106200 2024-09-21T04:11:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.086+0000 7f735dffb640 1 -- 172.21.15.64:0/832899011 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73500107f0 con 0x7f7360106200 2024-09-21T04:11:45.091 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.087+0000 7f7366a8b640 1 -- 172.21.15.64:0/832899011 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73601a6e00 con 0x7f7360106200 2024-09-21T04:11:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.089+0000 7f736d9f5740 1 Processor -- start 2024-09-21T04:11:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.089+0000 7f735dffb640 1 -- 172.21.15.64:0/832899011 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f7350004fc0 con 0x7f7360106200 2024-09-21T04:11:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.089+0000 7f736d9f5740 1 -- start start 2024-09-21T04:11:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.090+0000 7f736d9f5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565205a9d1b0 0x565205e429b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.090+0000 7f736d9f5740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565205a742f0 0x565205e42ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.090+0000 7f736d9f5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565205a7c4d0 0x565205e43430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.090+0000 7f736d9f5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x565205db76f0 con 0x565205a9d1b0 2024-09-21T04:11:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.090+0000 7f736d9f5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x565205da6b20 con 0x565205a7c4d0 2024-09-21T04:11:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.090+0000 7f736d9f5740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x565205daa930 con 0x565205a742f0 2024-09-21T04:11:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.090+0000 7f7364827640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565205a742f0 0x565205e42ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.090+0000 7f7364827640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565205a742f0 0x565205e42ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:46530/0 (socket says 172.21.15.64:46530) 2024-09-21T04:11:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.090+0000 7f7364827640 1 -- 172.21.15.64:0/1209306368 learned_addr learned my addr 172.21.15.64:0/1209306368 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.090+0000 7f7365028640 1 --2- 172.21.15.64:0/1209306368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565205a9d1b0 0x565205e429b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.090+0000 7f735ffff640 1 --2- 172.21.15.64:0/1209306368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565205a7c4d0 0x565205e43430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.090+0000 7f7364827640 1 -- 172.21.15.64:0/1209306368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565205a7c4d0 msgr2=0x565205e43430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.090+0000 7f7364827640 1 --2- 172.21.15.64:0/1209306368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565205a7c4d0 0x565205e43430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.090+0000 7f7364827640 1 -- 172.21.15.64:0/1209306368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565205a9d1b0 msgr2=0x565205e429b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.090+0000 7f7364827640 1 --2- 172.21.15.64:0/1209306368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565205a9d1b0 0x565205e429b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.090+0000 7f7364827640 1 -- 172.21.15.64:0/1209306368 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565205d6c530 con 0x565205a742f0 2024-09-21T04:11:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.090+0000 7f735ffff640 1 --2- 172.21.15.64:0/1209306368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565205a7c4d0 0x565205e43430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:11:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.091+0000 7f7364827640 1 --2- 172.21.15.64:0/1209306368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565205a742f0 0x565205e42ef0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7348009cd0 tx=0x7f7348011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=318a05fff2f8721d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.091+0000 7f7356ffd640 1 -- 172.21.15.64:0/1209306368 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348004220 con 0x565205a742f0 2024-09-21T04:11:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.091+0000 7f735dffb640 1 --2- 172.21.15.64:0/832899011 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f734c0b6680 0x7f734c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.091+0000 7f7356ffd640 1 -- 172.21.15.64:0/1209306368 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73480043c0 con 0x565205a742f0 2024-09-21T04:11:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.091+0000 7f7356ffd640 1 -- 172.21.15.64:0/1209306368 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348019830 con 0x565205a742f0 2024-09-21T04:11:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.091+0000 7f7365028640 1 --2- 172.21.15.64:0/832899011 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f734c0b6680 0x7f734c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:45.096 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.091+0000 7f735dffb640 1 -- 172.21.15.64:0/832899011 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7f73500e7ed0 con 0x7f7360106200 2024-09-21T04:11:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.092+0000 7f736d9f5740 1 -- 172.21.15.64:0/1209306368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565205a742f0 msgr2=0x565205e42ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.092+0000 7f736d9f5740 1 --2- 172.21.15.64:0/1209306368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565205a742f0 0x565205e42ef0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7348009cd0 tx=0x7f7348011810 comp rx=0 tx=0).stop 2024-09-21T04:11:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.092+0000 7f7365028640 1 --2- 172.21.15.64:0/832899011 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f734c0b6680 0x7f734c0b8b90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7358006010 tx=0x7f7358005a40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.092+0000 7f736d9f5740 1 -- 172.21.15.64:0/1209306368 shutdown_connections 2024-09-21T04:11:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.092+0000 7f736d9f5740 1 --2- 172.21.15.64:0/1209306368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565205a7c4d0 0x565205e43430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.092+0000 7f736d9f5740 1 --2- 172.21.15.64:0/1209306368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565205a742f0 0x565205e42ef0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.099 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.092+0000 7f736d9f5740 1 --2- 172.21.15.64:0/1209306368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565205a9d1b0 0x565205e429b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.092+0000 7f736d9f5740 1 -- 172.21.15.64:0/1209306368 >> 172.21.15.64:0/1209306368 conn(0x565205a91300 msgr2=0x565205e408a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.092+0000 7f736d9f5740 1 -- 172.21.15.64:0/1209306368 shutdown_connections 2024-09-21T04:11:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.092+0000 7f736d9f5740 1 -- 172.21.15.64:0/1209306368 wait complete. 2024-09-21T04:11:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.092+0000 7f736d9f5740 1 -- start start 2024-09-21T04:11:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.092+0000 7f736d9f5740 1 Processor -- start 2024-09-21T04:11:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.093+0000 7f736d9f5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565205a742f0 0x565205e4e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.093+0000 7f736d9f5740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565205a7c4d0 0x565205e4e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.093+0000 7f736d9f5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565205a9d1b0 0x565205e5e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.093+0000 7f736d9f5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x565205cdeab0 con 0x565205a9d1b0 2024-09-21T04:11:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.093+0000 7f736d9f5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x565205d4c730 con 0x565205a742f0 2024-09-21T04:11:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.093+0000 7f736d9f5740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x565205e51130 con 0x565205a7c4d0 2024-09-21T04:11:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.093+0000 7f736d9f5740 10 client.0 _ll_register_callbacks cb 0x565205da06b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:11:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.093+0000 7f7365028640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565205a9d1b0 0x565205e5e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:45.100 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.093+0000 7f7364827640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565205a742f0 0x565205e4e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.093+0000 7f735ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565205a7c4d0 0x565205e4e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.093+0000 7f7364827640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565205a742f0 0x565205e4e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:37370/0 (socket says 172.21.15.64:37370) 2024-09-21T04:11:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.093+0000 7f7364827640 1 -- 172.21.15.64:0/2248670525 learned_addr learned my addr 172.21.15.64:0/2248670525 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.093+0000 7f7365028640 1 --2- 172.21.15.64:0/2248670525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565205a9d1b0 0x565205e5e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41154/0 (socket says 172.21.15.64:41154) 2024-09-21T04:11:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.093+0000 7f735ffff640 1 -- 172.21.15.64:0/2248670525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565205a742f0 msgr2=0x565205e4e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.093+0000 7f735ffff640 1 --2- 172.21.15.64:0/2248670525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565205a742f0 0x565205e4e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.093+0000 7f735ffff640 1 -- 172.21.15.64:0/2248670525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565205a9d1b0 msgr2=0x565205e5e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.093+0000 7f735ffff640 1 --2- 172.21.15.64:0/2248670525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565205a9d1b0 0x565205e5e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.093+0000 7f735ffff640 1 -- 172.21.15.64:0/2248670525 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565205b08790 con 0x565205a7c4d0 2024-09-21T04:11:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.093+0000 7f7364827640 1 --2- 172.21.15.64:0/2248670525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565205a742f0 0x565205e4e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:11:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.094+0000 7f735ffff640 1 --2- 172.21.15.64:0/2248670525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565205a7c4d0 0x565205e4e570 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f73500e55c0 tx=0x7f7350005790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a535118d6954f068 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.094+0000 7f733ffff640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-21T04:11:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.094+0000 7f733ffff640 1 -- 172.21.15.64:0/2248670525 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73500ad610 con 0x565205a7c4d0 2024-09-21T04:11:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.094+0000 7f736d9f5740 1 -- 172.21.15.64:0/2248670525 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73480043c0 con 0x565205a7c4d0 2024-09-21T04:11:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.094+0000 7f733ffff640 1 -- 172.21.15.64:0/2248670525 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7f73500d10e0 con 0x565205a7c4d0 2024-09-21T04:11:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.094+0000 7f733ffff640 1 -- 172.21.15.64:0/2248670525 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f73500ad430 con 0x565205a7c4d0 2024-09-21T04:11:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.094+0000 7f736d9f5740 10 client.26662 fetch_fsmap learned FSMap version 39 2024-09-21T04:11:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.094+0000 7f736d9f5740 1 -- 172.21.15.64:0/2248670525 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x565205b81620 con 0x565205a7c4d0 2024-09-21T04:11:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.094+0000 7f733ffff640 1 -- 172.21.15.64:0/2248670525 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73480043c0 con 0x565205a7c4d0 2024-09-21T04:11:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.094+0000 7f733ffff640 1 -- 172.21.15.64:0/2248670525 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f73500b5b20 con 0x565205a7c4d0 2024-09-21T04:11:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.094+0000 7f736d9f5740 10 client.26662 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T04:11:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.095+0000 7f736d9f5740 10 client.26662 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:11:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.095+0000 7f736d9f5740 20 client.26662 populate_metadata read hostname 'smithi064' 2024-09-21T04:11:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.095+0000 7f736d9f5740 1 --2- 172.21.15.64:0/2248670525 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x565205a5bd80 0x565205e645e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.095+0000 7f736d9f5740 4 client.26662 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:11:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.095+0000 7f736d9f5740 1 -- 172.21.15.64:0/2248670525 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565205db9640 con 0x565205a5bd80 2024-09-21T04:11:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.095+0000 7f7364827640 1 --2- 172.21.15.64:0/2248670525 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x565205a5bd80 0x565205e645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.095+0000 7f733ffff640 10 client.26662 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:45.103 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.095+0000 7f7364827640 1 --2- 172.21.15.64:0/2248670525 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x565205a5bd80 0x565205e645e0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47d0633a6504eeac server_cookie=cb87789627e0fe9e in_seq=0 out_seq=0 2024-09-21T04:11:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.097+0000 7f733ffff640 1 -- 172.21.15.64:0/2248670525 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565205db9640 con 0x565205a5bd80 2024-09-21T04:11:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.097+0000 7f733ffff640 10 client.26662 handle_command_reply: tid=0 2024-09-21T04:11:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.097+0000 7f736d9f5740 1 client.26662 shutdown 2024-09-21T04:11:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.097+0000 7f736d9f5740 1 -- 172.21.15.64:0/2248670525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565205a7c4d0 msgr2=0x565205e4e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.097+0000 7f736d9f5740 1 --2- 172.21.15.64:0/2248670525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565205a7c4d0 0x565205e4e570 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f73500e55c0 tx=0x7f7350005790 comp rx=0 tx=0).stop 2024-09-21T04:11:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.097+0000 7f736d9f5740 1 -- 172.21.15.64:0/2248670525 shutdown_connections 2024-09-21T04:11:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.097+0000 7f736d9f5740 1 --2- 172.21.15.64:0/2248670525 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x565205a5bd80 0x565205e645e0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.097+0000 7f736d9f5740 1 --2- 172.21.15.64:0/2248670525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565205a9d1b0 0x565205e5e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.097+0000 7f733ffff640 0 client.26662 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.097+0000 7f736d9f5740 1 --2- 172.21.15.64:0/2248670525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565205a7c4d0 0x565205e4e570 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.097+0000 7f736d9f5740 1 --2- 172.21.15.64:0/2248670525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565205a742f0 0x565205e4e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.097+0000 7f736d9f5740 1 -- 172.21.15.64:0/2248670525 >> 172.21.15.64:0/2248670525 conn(0x565205a91300 msgr2=0x565205e439e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.097+0000 7f736d9f5740 1 -- 172.21.15.64:0/2248670525 shutdown_connections 2024-09-21T04:11:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.097+0000 7f736d9f5740 1 -- 172.21.15.64:0/2248670525 wait complete. 2024-09-21T04:11:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.097+0000 7f736d9f5740 20 client.26662 trim_cache size 0 max 16384 2024-09-21T04:11:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.112+0000 7f736d9f5740 1 Processor -- start 2024-09-21T04:11:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.112+0000 7f736d9f5740 1 -- start start 2024-09-21T04:11:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.112+0000 7f736d9f5740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565205a742f0 0x565205e440f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.112+0000 7f736d9f5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565205a7c4d0 0x565205e5e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.112+0000 7f736d9f5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565205a9d1b0 0x565205e4b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.112+0000 7f736d9f5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f73601affb0 con 0x565205a7c4d0 2024-09-21T04:11:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.112+0000 7f736d9f5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f73601ad7f0 con 0x565205a9d1b0 2024-09-21T04:11:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.112+0000 7f736d9f5740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f73601ab660 con 0x565205a742f0 2024-09-21T04:11:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.113+0000 7f735ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565205a7c4d0 0x565205e5e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.113+0000 7f7365028640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565205a9d1b0 0x565205e4b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.113+0000 7f7364827640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565205a742f0 0x565205e440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.113+0000 7f735ffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565205a7c4d0 0x565205e5e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41160/0 (socket says 172.21.15.64:41160) 2024-09-21T04:11:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.113+0000 7f735ffff640 1 -- 172.21.15.64:0/566639899 learned_addr learned my addr 172.21.15.64:0/566639899 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.113+0000 7f735ffff640 1 -- 172.21.15.64:0/566639899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565205a742f0 msgr2=0x565205e440f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.113+0000 7f735ffff640 1 --2- 172.21.15.64:0/566639899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565205a742f0 0x565205e440f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.113+0000 7f735ffff640 1 -- 172.21.15.64:0/566639899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565205a9d1b0 msgr2=0x565205e4b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.113+0000 7f735ffff640 1 --2- 172.21.15.64:0/566639899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565205a9d1b0 0x565205e4b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.113+0000 7f735ffff640 1 -- 172.21.15.64:0/566639899 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565205e51010 con 0x565205a7c4d0 2024-09-21T04:11:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.113+0000 7f735ffff640 1 --2- 172.21.15.64:0/566639899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565205a7c4d0 0x565205e5e810 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f73500aaf30 tx=0x7f73500d1450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bfda7fa04493348 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.113+0000 7f73567fc640 1 -- 172.21.15.64:0/566639899 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73500ad610 con 0x565205a7c4d0 2024-09-21T04:11:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.113+0000 7f73567fc640 1 -- 172.21.15.64:0/566639899 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73500ad7d0 con 0x565205a7c4d0 2024-09-21T04:11:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.113+0000 7f73567fc640 1 -- 172.21.15.64:0/566639899 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73500a9b60 con 0x565205a7c4d0 2024-09-21T04:11:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.114+0000 7f736d9f5740 1 -- 172.21.15.64:0/566639899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565205a7c4d0 msgr2=0x565205e5e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.114+0000 7f736d9f5740 1 --2- 172.21.15.64:0/566639899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565205a7c4d0 0x565205e5e810 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f73500aaf30 tx=0x7f73500d1450 comp rx=0 tx=0).stop 2024-09-21T04:11:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.114+0000 7f736d9f5740 1 -- 172.21.15.64:0/566639899 shutdown_connections 2024-09-21T04:11:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.114+0000 7f736d9f5740 1 --2- 172.21.15.64:0/566639899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565205a9d1b0 0x565205e4b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.114+0000 7f736d9f5740 1 --2- 172.21.15.64:0/566639899 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565205a7c4d0 0x565205e5e810 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.114+0000 7f736d9f5740 1 --2- 172.21.15.64:0/566639899 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565205a742f0 0x565205e440f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.114+0000 7f736d9f5740 1 -- 172.21.15.64:0/566639899 >> 172.21.15.64:0/566639899 conn(0x565205a91300 msgr2=0x565205e439e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.114+0000 7f736d9f5740 1 -- 172.21.15.64:0/566639899 shutdown_connections 2024-09-21T04:11:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.114+0000 7f736d9f5740 1 -- 172.21.15.64:0/566639899 wait complete. 2024-09-21T04:11:45.121 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.114+0000 7f736d9f5740 1 -- start start 2024-09-21T04:11:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.115+0000 7f736d9f5740 1 Processor -- start 2024-09-21T04:11:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.115+0000 7f736d9f5740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565205a742f0 0x565205e5ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.115+0000 7f736d9f5740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565205a7c4d0 0x565205e58230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.115+0000 7f736d9f5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565205a9d1b0 0x565205e58770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.115+0000 7f736d9f5740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f73601abcb0 con 0x565205a742f0 2024-09-21T04:11:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.115+0000 7f736d9f5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f73601ad120 con 0x565205a9d1b0 2024-09-21T04:11:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.115+0000 7f736d9f5740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f73601ac380 con 0x565205a7c4d0 2024-09-21T04:11:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.115+0000 7f736d9f5740 10 client.0 _ll_register_callbacks cb 0x7f735000ce70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:11:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.115+0000 7f7364827640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565205a742f0 0x565205e5ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.115+0000 7f7364827640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565205a742f0 0x565205e5ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:41162/0 (socket says 172.21.15.64:41162) 2024-09-21T04:11:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.115+0000 7f7364827640 1 -- 172.21.15.64:0/2021309200 learned_addr learned my addr 172.21.15.64:0/2021309200 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:11:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.115+0000 7f7365028640 1 --2- 172.21.15.64:0/2021309200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565205a9d1b0 0x565205e58770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.115+0000 7f735ffff640 1 --2- 172.21.15.64:0/2021309200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565205a7c4d0 0x565205e58230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.115+0000 7f7364827640 1 -- 172.21.15.64:0/2021309200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565205a7c4d0 msgr2=0x565205e58230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.115+0000 7f7364827640 1 --2- 172.21.15.64:0/2021309200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565205a7c4d0 0x565205e58230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.115+0000 7f7364827640 1 -- 172.21.15.64:0/2021309200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565205a9d1b0 msgr2=0x565205e58770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:11:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.115+0000 7f7364827640 1 --2- 172.21.15.64:0/2021309200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565205a9d1b0 0x565205e58770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.115+0000 7f7364827640 1 -- 172.21.15.64:0/2021309200 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565205e58cb0 con 0x565205a742f0 2024-09-21T04:11:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.116+0000 7f7364827640 1 --2- 172.21.15.64:0/2021309200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565205a742f0 0x565205e5ad70 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f7348009bc0 tx=0x7f7348009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f8071e9158fd3a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:11:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.116+0000 7f733ffff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:11:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.116+0000 7f733ffff640 1 -- 172.21.15.64:0/2021309200 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348003a00 con 0x565205a742f0 2024-09-21T04:11:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.116+0000 7f733ffff640 1 -- 172.21.15.64:0/2021309200 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8615+0+0 (secure 0 0 0) 0x7f7348005040 con 0x565205a742f0 2024-09-21T04:11:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.116+0000 7f736d9f5740 1 -- 172.21.15.64:0/2021309200 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73500ad7d0 con 0x565205a742f0 2024-09-21T04:11:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.116+0000 7f733ffff640 1 -- 172.21.15.64:0/2021309200 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7348005c50 con 0x565205a742f0 2024-09-21T04:11:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.116+0000 7f736d9f5740 10 client.17151 fetch_fsmap learned FSMap version 39 2024-09-21T04:11:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.116+0000 7f736d9f5740 1 -- 172.21.15.64:0/2021309200 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x565205ecec40 con 0x565205a742f0 2024-09-21T04:11:45.123 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.116+0000 7f733ffff640 1 -- 172.21.15.64:0/2021309200 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73500ad7d0 con 0x565205a742f0 2024-09-21T04:11:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.116+0000 7f733ffff640 1 -- 172.21.15.64:0/2021309200 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f7348019a00 con 0x565205a742f0 2024-09-21T04:11:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.117+0000 7f736d9f5740 10 client.17151 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T04:11:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.117+0000 7f736d9f5740 10 client.17151 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T04:11:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.117+0000 7f736d9f5740 20 client.17151 populate_metadata read hostname 'smithi064' 2024-09-21T04:11:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.117+0000 7f736d9f5740 1 --2- 172.21.15.64:0/2021309200 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x565205a311c0 0x565205ed1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:11:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.117+0000 7f736d9f5740 4 client.17151 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T04:11:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.117+0000 7f736d9f5740 1 -- 172.21.15.64:0/2021309200 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565205db50f0 con 0x565205a311c0 2024-09-21T04:11:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.117+0000 7f735ffff640 1 --2- 172.21.15.64:0/2021309200 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x565205a311c0 0x565205ed1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:11:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.117+0000 7f733ffff640 10 client.17151 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:45.124 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.117+0000 7f735ffff640 1 --2- 172.21.15.64:0/2021309200 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x565205a311c0 0x565205ed1290 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8df9bf7bd7302b05 server_cookie=fb7408568fd28091 in_seq=0 out_seq=0 2024-09-21T04:11:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.395+0000 7f733ffff640 1 -- 172.21.15.64:0/2021309200 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565205db50f0 con 0x565205a311c0 2024-09-21T04:11:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.395+0000 7f733ffff640 10 client.17151 handle_command_reply: tid=0 2024-09-21T04:11:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.395+0000 7f736d9f5740 1 client.17151 shutdown 2024-09-21T04:11:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.395+0000 7f736d9f5740 1 -- 172.21.15.64:0/2021309200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565205a742f0 msgr2=0x565205e5ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.395+0000 7f736d9f5740 1 --2- 172.21.15.64:0/2021309200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565205a742f0 0x565205e5ad70 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f7348009bc0 tx=0x7f7348009310 comp rx=0 tx=0).stop 2024-09-21T04:11:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.395+0000 7f736d9f5740 1 -- 172.21.15.64:0/2021309200 shutdown_connections 2024-09-21T04:11:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.395+0000 7f736d9f5740 1 --2- 172.21.15.64:0/2021309200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x565205a9d1b0 0x565205e58770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.395+0000 7f736d9f5740 1 --2- 172.21.15.64:0/2021309200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x565205a7c4d0 0x565205e58230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.395+0000 7f736d9f5740 1 --2- 172.21.15.64:0/2021309200 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x565205a311c0 0x565205ed1290 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.395+0000 7f733ffff640 0 client.17151 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:11:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.395+0000 7f736d9f5740 1 --2- 172.21.15.64:0/2021309200 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565205a742f0 0x565205e5ad70 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.395+0000 7f736d9f5740 1 -- 172.21.15.64:0/2021309200 >> 172.21.15.64:0/2021309200 conn(0x565205a91300 msgr2=0x565205e40880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.395+0000 7f736d9f5740 1 -- 172.21.15.64:0/2021309200 shutdown_connections 2024-09-21T04:11:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.395+0000 7f736d9f5740 1 -- 172.21.15.64:0/2021309200 wait complete. 2024-09-21T04:11:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.395+0000 7f736d9f5740 20 client.17151 trim_cache size 0 max 16384 2024-09-21T04:11:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:{ 2024-09-21T04:11:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "message": "", 2024-09-21T04:11:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout: "return_code": 0 2024-09-21T04:11:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stdout:} 2024-09-21T04:11:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.396+0000 7f7366a8b640 1 -- 172.21.15.64:0/832899011 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f734c0b6680 msgr2=0x7f734c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.396+0000 7f7366a8b640 1 --2- 172.21.15.64:0/832899011 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f734c0b6680 0x7f734c0b8b90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7358006010 tx=0x7f7358005a40 comp rx=0 tx=0).stop 2024-09-21T04:11:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.396+0000 7f7366a8b640 1 -- 172.21.15.64:0/832899011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7360106200 msgr2=0x7f736019a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:11:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.396+0000 7f7366a8b640 1 --2- 172.21.15.64:0/832899011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7360106200 0x7f736019a9f0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f735000eb90 tx=0x7f735000c6a0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.396+0000 7f7366a8b640 1 -- 172.21.15.64:0/832899011 shutdown_connections 2024-09-21T04:11:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.396+0000 7f7366a8b640 1 --2- 172.21.15.64:0/832899011 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f734c0b6680 0x7f734c0b8b90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.396+0000 7f7366a8b640 1 --2- 172.21.15.64:0/832899011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7360106b50 0x7f736019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.396+0000 7f7366a8b640 1 --2- 172.21.15.64:0/832899011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7360106200 0x7f736019a9f0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.396+0000 7f7366a8b640 1 --2- 172.21.15.64:0/832899011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7360102090 0x7f736019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:11:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.396+0000 7f7366a8b640 1 -- 172.21.15.64:0/832899011 >> 172.21.15.64:0/832899011 conn(0x7f7360079ec0 msgr2=0x7f73601053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:11:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.396+0000 7f7366a8b640 1 -- 172.21.15.64:0/832899011 shutdown_connections 2024-09-21T04:11:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi064.front.sepia.ceph.com.smithi064.stderr:2024-09-21T04:11:45.396+0000 7f7366a8b640 1 -- 172.21.15.64:0/832899011 wait complete. 2024-09-21T04:11:46.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:46 smithi119 ceph-mon[34811]: pgmap v557: 97 pgs: 97 active+clean; 831 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 483 op/s 2024-09-21T04:11:46.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:46 smithi064 ceph-mon[31180]: pgmap v557: 97 pgs: 97 active+clean; 831 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 483 op/s 2024-09-21T04:11:46.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:46 smithi131 ceph-mon[35193]: pgmap v557: 97 pgs: 97 active+clean; 831 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 483 op/s 2024-09-21T04:11:48.605 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:48 smithi131 ceph-mon[35193]: pgmap v558: 97 pgs: 97 active+clean; 803 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 782 op/s 2024-09-21T04:11:48.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:48 smithi119 ceph-mon[34811]: pgmap v558: 97 pgs: 97 active+clean; 803 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 782 op/s 2024-09-21T04:11:48.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:48 smithi064 ceph-mon[31180]: pgmap v558: 97 pgs: 97 active+clean; 803 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 782 op/s 2024-09-21T04:11:50.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:50 smithi119 ceph-mon[34811]: pgmap v559: 97 pgs: 97 active+clean; 796 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 665 op/s 2024-09-21T04:11:50.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:50 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:11:50.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:50 smithi064 ceph-mon[31180]: pgmap v559: 97 pgs: 97 active+clean; 796 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 665 op/s 2024-09-21T04:11:50.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:50 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:11:50.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:50 smithi131 ceph-mon[35193]: pgmap v559: 97 pgs: 97 active+clean; 796 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 665 op/s 2024-09-21T04:11:50.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:50 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:11:52.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:52 smithi119 ceph-mon[34811]: pgmap v560: 97 pgs: 97 active+clean; 796 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 642 op/s 2024-09-21T04:11:52.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:52 smithi064 ceph-mon[31180]: pgmap v560: 97 pgs: 97 active+clean; 796 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 642 op/s 2024-09-21T04:11:52.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:52 smithi131 ceph-mon[35193]: pgmap v560: 97 pgs: 97 active+clean; 796 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 642 op/s 2024-09-21T04:11:54.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:54 smithi119 ceph-mon[34811]: pgmap v561: 97 pgs: 97 active+clean; 816 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.4 MiB/s wr, 900 op/s 2024-09-21T04:11:54.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:54 smithi064 ceph-mon[31180]: pgmap v561: 97 pgs: 97 active+clean; 816 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.4 MiB/s wr, 900 op/s 2024-09-21T04:11:54.779 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2024-09-21T04:11:54.779 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-21T04:11:54.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:54 smithi131 ceph-mon[35193]: pgmap v561: 97 pgs: 97 active+clean; 816 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.4 MiB/s wr, 900 op/s 2024-09-21T04:11:55.152 DEBUG:teuthology.parallel:result is None 2024-09-21T04:11:55.153 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T04:11:55.207 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T04:11:55.207 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-21T04:11:55.223 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi064.front.sepia.ceph.com 2024-09-21T04:11:55.233 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-21T04:11:55.242 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-21T04:11:55.243 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T04:11:55.243 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi064.front.sepia.ceph.com 2024-09-21T04:11:55.243 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-21T04:11:55.311 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-21T04:11:56.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:56 smithi119 ceph-mon[34811]: pgmap v562: 97 pgs: 97 active+clean; 816 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 687 op/s 2024-09-21T04:11:56.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:56 smithi064 ceph-mon[31180]: pgmap v562: 97 pgs: 97 active+clean; 816 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 687 op/s 2024-09-21T04:11:56.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:56 smithi131 ceph-mon[35193]: pgmap v562: 97 pgs: 97 active+clean; 816 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 687 op/s 2024-09-21T04:11:58.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:58 smithi119 ceph-mon[34811]: pgmap v563: 97 pgs: 97 active+clean; 797 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.0 MiB/s wr, 813 op/s 2024-09-21T04:11:58.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:58 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:11:58.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:58 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:11:58.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:11:58 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:11:58.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:58 smithi064 ceph-mon[31180]: pgmap v563: 97 pgs: 97 active+clean; 797 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.0 MiB/s wr, 813 op/s 2024-09-21T04:11:58.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:58 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:11:58.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:58 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:11:58.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:11:58 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:11:58.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:58 smithi131 ceph-mon[35193]: pgmap v563: 97 pgs: 97 active+clean; 797 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.0 MiB/s wr, 813 op/s 2024-09-21T04:11:58.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:58 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:11:58.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:58 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:11:58.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:11:58 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:12:00.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:00 smithi119 ceph-mon[34811]: pgmap v564: 97 pgs: 97 active+clean; 785 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 540 op/s 2024-09-21T04:12:00.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:00 smithi119 ceph-mon[34811]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:12:00.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:00 smithi064 ceph-mon[31180]: pgmap v564: 97 pgs: 97 active+clean; 785 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 540 op/s 2024-09-21T04:12:00.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:00 smithi064 ceph-mon[31180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:12:00.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:00 smithi131 ceph-mon[35193]: pgmap v564: 97 pgs: 97 active+clean; 785 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 540 op/s 2024-09-21T04:12:00.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:00 smithi131 ceph-mon[35193]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:12:02.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:02 smithi119 ceph-mon[34811]: pgmap v565: 97 pgs: 97 active+clean; 785 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 454 op/s 2024-09-21T04:12:02.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:02 smithi064 ceph-mon[31180]: pgmap v565: 97 pgs: 97 active+clean; 785 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 454 op/s 2024-09-21T04:12:02.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:02 smithi131 ceph-mon[35193]: pgmap v565: 97 pgs: 97 active+clean; 785 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 454 op/s 2024-09-21T04:12:04.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:04 smithi119 ceph-mon[34811]: pgmap v566: 97 pgs: 97 active+clean; 785 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 446 op/s 2024-09-21T04:12:04.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:04 smithi064 ceph-mon[31180]: pgmap v566: 97 pgs: 97 active+clean; 785 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 446 op/s 2024-09-21T04:12:04.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:04 smithi131 ceph-mon[35193]: pgmap v566: 97 pgs: 97 active+clean; 785 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 446 op/s 2024-09-21T04:12:06.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:06 smithi119 ceph-mon[34811]: pgmap v567: 97 pgs: 97 active+clean; 785 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 175 op/s 2024-09-21T04:12:06.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:06 smithi064 ceph-mon[31180]: pgmap v567: 97 pgs: 97 active+clean; 785 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 175 op/s 2024-09-21T04:12:06.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:06 smithi131 ceph-mon[35193]: pgmap v567: 97 pgs: 97 active+clean; 785 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 175 op/s 2024-09-21T04:12:08.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:08 smithi119 ceph-mon[34811]: pgmap v568: 97 pgs: 97 active+clean; 785 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 176 op/s 2024-09-21T04:12:08.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:08 smithi064 ceph-mon[31180]: pgmap v568: 97 pgs: 97 active+clean; 785 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 176 op/s 2024-09-21T04:12:08.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:08 smithi131 ceph-mon[35193]: pgmap v568: 97 pgs: 97 active+clean; 785 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 176 op/s 2024-09-21T04:12:09.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:09 smithi119 ceph-mon[34811]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T04:12:09.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:09 smithi064 ceph-mon[31180]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T04:12:09.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:09 smithi131 ceph-mon[35193]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T04:12:10.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:10 smithi119 ceph-mon[34811]: pgmap v570: 97 pgs: 97 active+clean; 785 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 66 KiB/s wr, 33 op/s 2024-09-21T04:12:10.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:10 smithi064 ceph-mon[31180]: pgmap v570: 97 pgs: 97 active+clean; 785 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 66 KiB/s wr, 33 op/s 2024-09-21T04:12:10.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:10 smithi131 ceph-mon[35193]: pgmap v570: 97 pgs: 97 active+clean; 785 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 66 KiB/s wr, 33 op/s 2024-09-21T04:12:12.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:12 smithi119 ceph-mon[34811]: pgmap v571: 97 pgs: 97 active+clean; 785 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 66 KiB/s wr, 30 op/s 2024-09-21T04:12:12.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:12 smithi064 ceph-mon[31180]: pgmap v571: 97 pgs: 97 active+clean; 785 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 66 KiB/s wr, 30 op/s 2024-09-21T04:12:12.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:12 smithi131 ceph-mon[35193]: pgmap v571: 97 pgs: 97 active+clean; 785 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 66 KiB/s wr, 30 op/s 2024-09-21T04:12:14.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:14 smithi119 ceph-mon[34811]: pgmap v572: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 KiB/s wr, 8 op/s 2024-09-21T04:12:14.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:14 smithi064 ceph-mon[31180]: pgmap v572: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 KiB/s wr, 8 op/s 2024-09-21T04:12:14.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:14 smithi131 ceph-mon[35193]: pgmap v572: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 KiB/s wr, 8 op/s 2024-09-21T04:12:15.017 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T04:12:15.284 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:12:15.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.776+0000 7fc1e2044640 1 -- 172.21.15.64:0/1384922511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1dc109e00 msgr2=0x7fc1dc110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:15.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.776+0000 7fc1e2044640 1 --2- 172.21.15.64:0/1384922511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1dc109e00 0x7fc1dc110630 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4009ca0 tx=0x7fc1d402a8e0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.776+0000 7fc1e2044640 1 -- 172.21.15.64:0/1384922511 shutdown_connections 2024-09-21T04:12:15.782 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.776+0000 7fc1e2044640 1 --2- 172.21.15.64:0/1384922511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1dc109e00 0x7fc1dc110630 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.776+0000 7fc1e2044640 1 --2- 172.21.15.64:0/1384922511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1dc1094b0 0x7fc1dc1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.776+0000 7fc1e2044640 1 --2- 172.21.15.64:0/1384922511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1dc103bc0 0x7fc1dc103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.776+0000 7fc1e2044640 1 -- 172.21.15.64:0/1384922511 >> 172.21.15.64:0/1384922511 conn(0x7fc1dc07b910 msgr2=0x7fc1dc07bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:15.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.776+0000 7fc1e2044640 1 -- 172.21.15.64:0/1384922511 shutdown_connections 2024-09-21T04:12:15.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.776+0000 7fc1e2044640 1 -- 172.21.15.64:0/1384922511 wait complete. 2024-09-21T04:12:15.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.776+0000 7fc1e2044640 1 Processor -- start 2024-09-21T04:12:15.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.777+0000 7fc1e2044640 1 -- start start 2024-09-21T04:12:15.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.777+0000 7fc1e2044640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1dc103bc0 0x7fc1dc19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:15.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.777+0000 7fc1e2044640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1dc1094b0 0x7fc1dc19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:15.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.777+0000 7fc1e2044640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1dc109e00 0x7fc1dc19fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:15.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.777+0000 7fc1e2044640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc113b80 con 0x7fc1dc109e00 2024-09-21T04:12:15.783 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.777+0000 7fc1e2044640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc113a00 con 0x7fc1dc1094b0 2024-09-21T04:12:15.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.777+0000 7fc1e2044640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc113d00 con 0x7fc1dc103bc0 2024-09-21T04:12:15.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.777+0000 7fc1dbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1dc109e00 0x7fc1dc19fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:15.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.777+0000 7fc1dbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1dc109e00 0x7fc1dc19fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54700/0 (socket says 172.21.15.64:54700) 2024-09-21T04:12:15.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.777+0000 7fc1dbfff640 1 -- 172.21.15.64:0/2923332640 learned_addr learned my addr 172.21.15.64:0/2923332640 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:15.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.777+0000 7fc1db7fe640 1 --2- 172.21.15.64:0/2923332640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1dc103bc0 0x7fc1dc19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:15.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.777+0000 7fc1daffd640 1 --2- 172.21.15.64:0/2923332640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1dc1094b0 0x7fc1dc19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:15.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.777+0000 7fc1dbfff640 1 -- 172.21.15.64:0/2923332640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1dc103bc0 msgr2=0x7fc1dc19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:15.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.777+0000 7fc1dbfff640 1 --2- 172.21.15.64:0/2923332640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1dc103bc0 0x7fc1dc19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.777+0000 7fc1dbfff640 1 -- 172.21.15.64:0/2923332640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1dc1094b0 msgr2=0x7fc1dc19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:15.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.777+0000 7fc1dbfff640 1 --2- 172.21.15.64:0/2923332640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1dc1094b0 0x7fc1dc19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.777+0000 7fc1dbfff640 1 -- 172.21.15.64:0/2923332640 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1dc1a0500 con 0x7fc1dc109e00 2024-09-21T04:12:15.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.777+0000 7fc1daffd640 1 --2- 172.21.15.64:0/2923332640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1dc1094b0 0x7fc1dc19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:12:15.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.778+0000 7fc1dbfff640 1 --2- 172.21.15.64:0/2923332640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1dc109e00 0x7fc1dc19fdd0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4005cc0 tx=0x7fc1d4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:15.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.778+0000 7fc1d8ff9640 1 -- 172.21.15.64:0/2923332640 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d4004040 con 0x7fc1dc109e00 2024-09-21T04:12:15.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.778+0000 7fc1e2044640 1 -- 172.21.15.64:0/2923332640 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1dc1a0790 con 0x7fc1dc109e00 2024-09-21T04:12:15.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.778+0000 7fc1d8ff9640 1 -- 172.21.15.64:0/2923332640 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1d4004690 con 0x7fc1dc109e00 2024-09-21T04:12:15.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.778+0000 7fc1d8ff9640 1 -- 172.21.15.64:0/2923332640 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d4030e70 con 0x7fc1dc109e00 2024-09-21T04:12:15.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.778+0000 7fc1e2044640 1 -- 172.21.15.64:0/2923332640 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1dc1a6f50 con 0x7fc1dc109e00 2024-09-21T04:12:15.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.781+0000 7fc1e85b8740 1 Processor -- start 2024-09-21T04:12:15.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.781+0000 7fc1d8ff9640 1 -- 172.21.15.64:0/2923332640 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fc1d40305e0 con 0x7fc1dc109e00 2024-09-21T04:12:15.789 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.781+0000 7fc1e85b8740 1 -- start start 2024-09-21T04:12:15.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.782+0000 7fc1e85b8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8c66e9070 0x55f8c6a90270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:15.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.782+0000 7fc1e85b8740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8c66c01b0 0x55f8c6a907b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:15.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.782+0000 7fc1e85b8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c8390 0x55f8c6a90cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:15.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.782+0000 7fc1e85b8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f8c6a06e50 con 0x55f8c66c8390 2024-09-21T04:12:15.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.782+0000 7fc1e85b8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f8c69f6280 con 0x55f8c66e9070 2024-09-21T04:12:15.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.782+0000 7fc1e85b8740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f8c69fa090 con 0x55f8c66c01b0 2024-09-21T04:12:15.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.782+0000 7fc1daffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c8390 0x55f8c6a90cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:15.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.782+0000 7fc1daffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c8390 0x55f8c6a90cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54702/0 (socket says 172.21.15.64:54702) 2024-09-21T04:12:15.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.782+0000 7fc1daffd640 1 -- 172.21.15.64:0/3783077060 learned_addr learned my addr 172.21.15.64:0/3783077060 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:15.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.782+0000 7fc1db7fe640 1 --2- 172.21.15.64:0/3783077060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8c66c01b0 0x55f8c6a907b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:15.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.782+0000 7fc1daffd640 1 --2- 172.21.15.64:0/3783077060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8c66e9070 0x55f8c6a90270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:15.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.782+0000 7fc1daffd640 1 -- 172.21.15.64:0/3783077060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8c66c01b0 msgr2=0x55f8c6a907b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:15.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.782+0000 7fc1d8ff9640 1 --2- 172.21.15.64:0/2923332640 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc1c40b6680 0x7fc1c40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.782+0000 7fc1daffd640 1 --2- 172.21.15.64:0/3783077060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8c66c01b0 0x55f8c6a907b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.782+0000 7fc1daffd640 1 -- 172.21.15.64:0/3783077060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8c66e9070 msgr2=0x55f8c6a90270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.782+0000 7fc1daffd640 1 --2- 172.21.15.64:0/3783077060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8c66e9070 0x55f8c6a90270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.782+0000 7fc1daffd640 1 -- 172.21.15.64:0/3783077060 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8c6754650 con 0x55f8c66c8390 2024-09-21T04:12:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.782+0000 7fc1dbfff640 1 --2- 172.21.15.64:0/2923332640 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc1c40b6680 0x7fc1c40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.783+0000 7fc1daffd640 1 --2- 172.21.15.64:0/3783077060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c8390 0x55f8c6a90cf0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fc1d000ae10 tx=0x7fc1d000a2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44af8261aa74e1bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.783+0000 7fc1d8ff9640 1 -- 172.21.15.64:0/2923332640 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8659+0+0 (secure 0 0 0) 0x7fc1d4108c70 con 0x7fc1dc109e00 2024-09-21T04:12:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.783+0000 7fc1c1ffb640 1 -- 172.21.15.64:0/3783077060 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d001c070 con 0x55f8c66c8390 2024-09-21T04:12:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.783+0000 7fc1c1ffb640 1 -- 172.21.15.64:0/3783077060 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1d0011bf0 con 0x55f8c66c8390 2024-09-21T04:12:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.783+0000 7fc1c1ffb640 1 -- 172.21.15.64:0/3783077060 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0014450 con 0x55f8c66c8390 2024-09-21T04:12:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.784+0000 7fc1e85b8740 1 -- 172.21.15.64:0/3783077060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c8390 msgr2=0x55f8c6a90cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.784+0000 7fc1dbfff640 1 --2- 172.21.15.64:0/2923332640 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc1c40b6680 0x7fc1c40b8b90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc1d41063e0 tx=0x7fc1d409b6e0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.784+0000 7fc1e85b8740 1 --2- 172.21.15.64:0/3783077060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c8390 0x55f8c6a90cf0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fc1d000ae10 tx=0x7fc1d000a2f0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.784+0000 7fc1e85b8740 1 -- 172.21.15.64:0/3783077060 shutdown_connections 2024-09-21T04:12:15.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.784+0000 7fc1e85b8740 1 --2- 172.21.15.64:0/3783077060 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c8390 0x55f8c6a90cf0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.784+0000 7fc1e85b8740 1 --2- 172.21.15.64:0/3783077060 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8c66c01b0 0x55f8c6a907b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.784+0000 7fc1e85b8740 1 --2- 172.21.15.64:0/3783077060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8c66e9070 0x55f8c6a90270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.784+0000 7fc1e85b8740 1 -- 172.21.15.64:0/3783077060 >> 172.21.15.64:0/3783077060 conn(0x55f8c66dd460 msgr2=0x55f8c6a8e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.784+0000 7fc1e85b8740 1 -- 172.21.15.64:0/3783077060 shutdown_connections 2024-09-21T04:12:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.784+0000 7fc1e85b8740 1 -- 172.21.15.64:0/3783077060 wait complete. 2024-09-21T04:12:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.784+0000 7fc1e85b8740 1 -- start start 2024-09-21T04:12:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.785+0000 7fc1e85b8740 1 Processor -- start 2024-09-21T04:12:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.785+0000 7fc1e85b8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c01b0 0x55f8c6a9b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.785+0000 7fc1e85b8740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8c66c8390 0x55f8c6a9be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.785+0000 7fc1e85b8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8c66e9070 0x55f8c6aa7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.785+0000 7fc1e85b8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f8c6857700 con 0x55f8c66c01b0 2024-09-21T04:12:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.785+0000 7fc1e85b8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f8c6991110 con 0x55f8c66e9070 2024-09-21T04:12:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.785+0000 7fc1e85b8740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f8c6a9ea50 con 0x55f8c66c8390 2024-09-21T04:12:15.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.785+0000 7fc1e85b8740 10 client.0 _ll_register_callbacks cb 0x55f8c69aa990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:12:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.785+0000 7fc1daffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c01b0 0x55f8c6a9b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.785+0000 7fc1daffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c01b0 0x55f8c6a9b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54704/0 (socket says 172.21.15.64:54704) 2024-09-21T04:12:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.785+0000 7fc1daffd640 1 -- 172.21.15.64:0/1788834792 learned_addr learned my addr 172.21.15.64:0/1788834792 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.785+0000 7fc1db7fe640 1 --2- 172.21.15.64:0/1788834792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8c66c8390 0x55f8c6a9be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.785+0000 7fc1daffd640 1 -- 172.21.15.64:0/1788834792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8c66c8390 msgr2=0x55f8c6a9be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.785+0000 7fc1daffd640 1 --2- 172.21.15.64:0/1788834792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8c66c8390 0x55f8c6a9be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.785+0000 7fc1daffd640 1 -- 172.21.15.64:0/1788834792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8c66e9070 msgr2=0x55f8c6aa7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.785+0000 7fc1daffd640 1 --2- 172.21.15.64:0/1788834792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8c66e9070 0x55f8c6aa7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.785+0000 7fc1daffd640 1 -- 172.21.15.64:0/1788834792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8c69bec40 con 0x55f8c66c01b0 2024-09-21T04:12:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.786+0000 7fc1db7fe640 1 --2- 172.21.15.64:0/1788834792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8c66c8390 0x55f8c6a9be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:12:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.786+0000 7fc1daffd640 1 --2- 172.21.15.64:0/1788834792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c01b0 0x55f8c6a9b900 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fc1d000a460 tx=0x7fc1d000a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d85aadd6f1529a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.786+0000 7fc1baffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:12:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.786+0000 7fc1baffd640 1 -- 172.21.15.64:0/1788834792 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0027060 con 0x55f8c66c01b0 2024-09-21T04:12:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.786+0000 7fc1baffd640 1 -- 172.21.15.64:0/1788834792 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8659+0+0 (secure 0 0 0) 0x7fc1d00223f0 con 0x55f8c66c01b0 2024-09-21T04:12:15.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.786+0000 7fc1e85b8740 1 -- 172.21.15.64:0/1788834792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1d0011bf0 con 0x55f8c66c01b0 2024-09-21T04:12:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.786+0000 7fc1baffd640 1 -- 172.21.15.64:0/1788834792 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc1d001c040 con 0x55f8c66c01b0 2024-09-21T04:12:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.786+0000 7fc1e85b8740 10 client.17163 fetch_fsmap learned FSMap version 40 2024-09-21T04:12:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.786+0000 7fc1e85b8740 1 -- 172.21.15.64:0/1788834792 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f8c6aaa730 con 0x55f8c66c01b0 2024-09-21T04:12:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.786+0000 7fc1baffd640 1 -- 172.21.15.64:0/1788834792 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1d0011bf0 con 0x55f8c66c01b0 2024-09-21T04:12:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.787+0000 7fc1baffd640 1 -- 172.21.15.64:0/1788834792 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7fc1d0015040 con 0x55f8c66c01b0 2024-09-21T04:12:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.787+0000 7fc1e85b8740 10 client.17163 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:12:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.787+0000 7fc1e85b8740 10 client.17163 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:12:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.787+0000 7fc1e85b8740 20 client.17163 populate_metadata read hostname 'smithi064' 2024-09-21T04:12:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.787+0000 7fc1e85b8740 1 --2- 172.21.15.64:0/1788834792 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f8c66a79f0 0x55f8c6aada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.787+0000 7fc1e85b8740 4 client.17163 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:12:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.787+0000 7fc1e85b8740 1 -- 172.21.15.64:0/1788834792 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8c6a08da0 con 0x55f8c66a79f0 2024-09-21T04:12:15.794 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.787+0000 7fc1db7fe640 1 --2- 172.21.15.64:0/1788834792 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f8c66a79f0 0x55f8c6aada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:15.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.787+0000 7fc1db7fe640 1 --2- 172.21.15.64:0/1788834792 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f8c66a79f0 0x55f8c6aada90 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e678410ea810f83b server_cookie=92bcbfb000a8daef in_seq=0 out_seq=0 2024-09-21T04:12:15.795 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.787+0000 7fc1baffd640 10 client.17163 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:12:15.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.788+0000 7fc1baffd640 1 -- 172.21.15.64:0/1788834792 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f8c6a08da0 con 0x55f8c66a79f0 2024-09-21T04:12:15.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.788+0000 7fc1baffd640 10 client.17163 handle_command_reply: tid=0 2024-09-21T04:12:15.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.788+0000 7fc1e85b8740 1 client.17163 shutdown 2024-09-21T04:12:15.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.789+0000 7fc1e85b8740 1 -- 172.21.15.64:0/1788834792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c01b0 msgr2=0x55f8c6a9b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:15.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.789+0000 7fc1e85b8740 1 --2- 172.21.15.64:0/1788834792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c01b0 0x55f8c6a9b900 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fc1d000a460 tx=0x7fc1d000a490 comp rx=0 tx=0).stop 2024-09-21T04:12:15.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.789+0000 7fc1e85b8740 1 -- 172.21.15.64:0/1788834792 shutdown_connections 2024-09-21T04:12:15.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.789+0000 7fc1e85b8740 1 --2- 172.21.15.64:0/1788834792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8c66e9070 0x55f8c6aa7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.789+0000 7fc1e85b8740 1 --2- 172.21.15.64:0/1788834792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8c66c8390 0x55f8c6a9be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.789+0000 7fc1e85b8740 1 --2- 172.21.15.64:0/1788834792 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f8c66a79f0 0x55f8c6aada90 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.789+0000 7fc1baffd640 0 client.17163 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:12:15.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.789+0000 7fc1e85b8740 1 --2- 172.21.15.64:0/1788834792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c01b0 0x55f8c6a9b900 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.789+0000 7fc1e85b8740 1 -- 172.21.15.64:0/1788834792 >> 172.21.15.64:0/1788834792 conn(0x55f8c66dd460 msgr2=0x55f8c6a93450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:15.796 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.789+0000 7fc1e85b8740 1 -- 172.21.15.64:0/1788834792 shutdown_connections 2024-09-21T04:12:15.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.789+0000 7fc1e85b8740 1 -- 172.21.15.64:0/1788834792 wait complete. 2024-09-21T04:12:15.797 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.789+0000 7fc1e85b8740 20 client.17163 trim_cache size 0 max 16384 2024-09-21T04:12:15.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.807+0000 7fc1e85b8740 1 Processor -- start 2024-09-21T04:12:15.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.807+0000 7fc1e85b8740 1 -- start start 2024-09-21T04:12:15.810 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.807+0000 7fc1e85b8740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8c66a79f0 0x55f8c6a9a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:15.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.807+0000 7fc1e85b8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c01b0 0x55f8c6a919b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:15.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.807+0000 7fc1e85b8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8c66c8390 0x55f8c6aa7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:15.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.807+0000 7fc1e85b8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc1b23c0 con 0x55f8c66c01b0 2024-09-21T04:12:15.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.807+0000 7fc1e85b8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc1afbd0 con 0x55f8c66c8390 2024-09-21T04:12:15.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.807+0000 7fc1e85b8740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc1ada40 con 0x55f8c66a79f0 2024-09-21T04:12:15.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.807+0000 7fc1db7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c01b0 0x55f8c6a919b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:15.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.807+0000 7fc1db7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c01b0 0x55f8c6a919b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54710/0 (socket says 172.21.15.64:54710) 2024-09-21T04:12:15.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.807+0000 7fc1db7fe640 1 -- 172.21.15.64:0/1207739108 learned_addr learned my addr 172.21.15.64:0/1207739108 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:15.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.807+0000 7fc1db7fe640 1 -- 172.21.15.64:0/1207739108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8c66a79f0 msgr2=0x55f8c6a9a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:15.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.807+0000 7fc1daffd640 1 --2- 172.21.15.64:0/1207739108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8c66a79f0 0x55f8c6a9a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:15.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.807+0000 7fc1db7fe640 1 --2- 172.21.15.64:0/1207739108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8c66a79f0 0x55f8c6a9a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.807+0000 7fc1daffd640 1 --2- 172.21.15.64:0/1207739108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8c66c8390 0x55f8c6aa7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:15.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.807+0000 7fc1db7fe640 1 -- 172.21.15.64:0/1207739108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8c66c8390 msgr2=0x55f8c6aa7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:15.811 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.807+0000 7fc1db7fe640 1 --2- 172.21.15.64:0/1207739108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8c66c8390 0x55f8c6aa7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.807+0000 7fc1db7fe640 1 -- 172.21.15.64:0/1207739108 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8c6a89580 con 0x55f8c66c01b0 2024-09-21T04:12:15.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.808+0000 7fc1db7fe640 1 --2- 172.21.15.64:0/1207739108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c01b0 0x55f8c6a919b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc00b500 tx=0x7fc1cc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f376adbc1edff1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:15.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.808+0000 7fc1c17fa640 1 -- 172.21.15.64:0/1207739108 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc022070 con 0x55f8c66c01b0 2024-09-21T04:12:15.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.808+0000 7fc1c17fa640 1 -- 172.21.15.64:0/1207739108 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1cc0143b0 con 0x55f8c66c01b0 2024-09-21T04:12:15.812 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.808+0000 7fc1c17fa640 1 -- 172.21.15.64:0/1207739108 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc0255c0 con 0x55f8c66c01b0 2024-09-21T04:12:15.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.808+0000 7fc1e85b8740 1 -- 172.21.15.64:0/1207739108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c01b0 msgr2=0x55f8c6a919b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:15.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.808+0000 7fc1e85b8740 1 --2- 172.21.15.64:0/1207739108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c01b0 0x55f8c6a919b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc00b500 tx=0x7fc1cc008040 comp rx=0 tx=0).stop 2024-09-21T04:12:15.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.809+0000 7fc1e85b8740 1 -- 172.21.15.64:0/1207739108 shutdown_connections 2024-09-21T04:12:15.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.809+0000 7fc1e85b8740 1 --2- 172.21.15.64:0/1207739108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8c66c8390 0x55f8c6aa7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.809+0000 7fc1e85b8740 1 --2- 172.21.15.64:0/1207739108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c01b0 0x55f8c6a919b0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.809+0000 7fc1e85b8740 1 --2- 172.21.15.64:0/1207739108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8c66a79f0 0x55f8c6a9a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.809+0000 7fc1e85b8740 1 -- 172.21.15.64:0/1207739108 >> 172.21.15.64:0/1207739108 conn(0x55f8c66dd460 msgr2=0x55f8c6a912a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:15.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.809+0000 7fc1e85b8740 1 -- 172.21.15.64:0/1207739108 shutdown_connections 2024-09-21T04:12:15.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.809+0000 7fc1e85b8740 1 -- 172.21.15.64:0/1207739108 wait complete. 2024-09-21T04:12:15.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.809+0000 7fc1e85b8740 1 -- start start 2024-09-21T04:12:15.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.809+0000 7fc1e85b8740 1 Processor -- start 2024-09-21T04:12:15.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.810+0000 7fc1e85b8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8c66a79f0 0x55f8c6aaea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:15.815 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.810+0000 7fc1e85b8740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c01b0 0x55f8c6aaef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:15.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.810+0000 7fc1e85b8740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8c66c8390 0x55f8c6aaf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:15.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.810+0000 7fc1e85b8740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc1ae090 con 0x55f8c66c01b0 2024-09-21T04:12:15.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.810+0000 7fc1e85b8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc1af500 con 0x55f8c66a79f0 2024-09-21T04:12:15.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.810+0000 7fc1e85b8740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc1ae760 con 0x55f8c66c8390 2024-09-21T04:12:15.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.810+0000 7fc1e85b8740 10 client.0 _ll_register_callbacks cb 0x7fc1d001a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:12:15.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.810+0000 7fc1db7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c01b0 0x55f8c6aaef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:15.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.810+0000 7fc1db7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c01b0 0x55f8c6aaef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54718/0 (socket says 172.21.15.64:54718) 2024-09-21T04:12:15.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.810+0000 7fc1db7fe640 1 -- 172.21.15.64:0/1204880213 learned_addr learned my addr 172.21.15.64:0/1204880213 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:15.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.810+0000 7fc1daffd640 1 --2- 172.21.15.64:0/1204880213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8c66c8390 0x55f8c6aaf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:15.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.810+0000 7fc1daffd640 1 --2- 172.21.15.64:0/1204880213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8c66a79f0 0x55f8c6aaea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:15.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.810+0000 7fc1db7fe640 1 -- 172.21.15.64:0/1204880213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8c66c8390 msgr2=0x55f8c6aaf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:15.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.810+0000 7fc1db7fe640 1 --2- 172.21.15.64:0/1204880213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8c66c8390 0x55f8c6aaf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.810+0000 7fc1db7fe640 1 -- 172.21.15.64:0/1204880213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8c66a79f0 msgr2=0x55f8c6aaea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:15.816 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.810+0000 7fc1db7fe640 1 --2- 172.21.15.64:0/1204880213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8c66a79f0 0x55f8c6aaea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.810+0000 7fc1db7fe640 1 -- 172.21.15.64:0/1204880213 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8c6b17210 con 0x55f8c66c01b0 2024-09-21T04:12:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.810+0000 7fc1daffd640 1 --2- 172.21.15.64:0/1204880213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8c66a79f0 0x55f8c6aaea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:12:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.810+0000 7fc1db7fe640 1 --2- 172.21.15.64:0/1204880213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c01b0 0x55f8c6aaef90 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc016c60 tx=0x7fc1cc016c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23d5309473b55cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.810+0000 7fc1baffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:12:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.811+0000 7fc1baffd640 1 -- 172.21.15.64:0/1204880213 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc02b020 con 0x55f8c66c01b0 2024-09-21T04:12:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.811+0000 7fc1baffd640 1 -- 172.21.15.64:0/1204880213 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8659+0+0 (secure 0 0 0) 0x7fc1cc014770 con 0x55f8c66c01b0 2024-09-21T04:12:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.811+0000 7fc1e85b8740 1 -- 172.21.15.64:0/1204880213 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1cc0143b0 con 0x55f8c66c01b0 2024-09-21T04:12:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.811+0000 7fc1baffd640 1 -- 172.21.15.64:0/1204880213 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc1cc022070 con 0x55f8c66c01b0 2024-09-21T04:12:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.811+0000 7fc1e85b8740 10 client.17169 fetch_fsmap learned FSMap version 40 2024-09-21T04:12:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.811+0000 7fc1e85b8740 1 -- 172.21.15.64:0/1204880213 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f8c6b18430 con 0x55f8c66c01b0 2024-09-21T04:12:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.811+0000 7fc1baffd640 1 -- 172.21.15.64:0/1204880213 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1cc0143b0 con 0x55f8c66c01b0 2024-09-21T04:12:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.811+0000 7fc1baffd640 1 -- 172.21.15.64:0/1204880213 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7fc1cc02f3a0 con 0x55f8c66c01b0 2024-09-21T04:12:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.811+0000 7fc1e85b8740 10 client.17169 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:12:15.817 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.811+0000 7fc1e85b8740 10 client.17169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:12:15.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.811+0000 7fc1e85b8740 20 client.17169 populate_metadata read hostname 'smithi064' 2024-09-21T04:12:15.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.811+0000 7fc1e85b8740 1 --2- 172.21.15.64:0/1204880213 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f8c66e9070 0x55f8c6b1ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:15.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.811+0000 7fc1e85b8740 4 client.17169 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T04:12:15.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.811+0000 7fc1e85b8740 1 -- 172.21.15.64:0/1204880213 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f8c6a04850 con 0x55f8c66e9070 2024-09-21T04:12:15.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.812+0000 7fc1daffd640 1 --2- 172.21.15.64:0/1204880213 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f8c66e9070 0x55f8c6b1ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:15.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.812+0000 7fc1baffd640 10 client.17169 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:12:15.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.812+0000 7fc1daffd640 1 --2- 172.21.15.64:0/1204880213 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f8c66e9070 0x55f8c6b1ac40 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63b4c6a853eb1314 server_cookie=fafbb8e7dc65fea5 in_seq=0 out_seq=0 2024-09-21T04:12:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.812+0000 7fc1baffd640 1 -- 172.21.15.64:0/1204880213 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3214 (crc 0 0 0) 0x55f8c6a04850 con 0x55f8c66e9070 2024-09-21T04:12:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.813+0000 7fc1baffd640 10 client.17169 handle_command_reply: tid=0 2024-09-21T04:12:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.813+0000 7fc1e85b8740 1 client.17169 shutdown 2024-09-21T04:12:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.813+0000 7fc1e85b8740 1 -- 172.21.15.64:0/1204880213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c01b0 msgr2=0x55f8c6aaef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.813+0000 7fc1e85b8740 1 --2- 172.21.15.64:0/1204880213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c01b0 0x55f8c6aaef90 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc016c60 tx=0x7fc1cc016c90 comp rx=0 tx=0).stop 2024-09-21T04:12:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.813+0000 7fc1e85b8740 1 -- 172.21.15.64:0/1204880213 shutdown_connections 2024-09-21T04:12:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.813+0000 7fc1e85b8740 1 --2- 172.21.15.64:0/1204880213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8c66c8390 0x55f8c6aaf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.819 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.813+0000 7fc1e85b8740 1 --2- 172.21.15.64:0/1204880213 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55f8c66e9070 0x55f8c6b1ac40 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.813+0000 7fc1e85b8740 1 --2- 172.21.15.64:0/1204880213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f8c66c01b0 0x55f8c6aaef90 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.813+0000 7fc1e85b8740 1 --2- 172.21.15.64:0/1204880213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f8c66a79f0 0x55f8c6aaea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.813+0000 7fc1e85b8740 1 -- 172.21.15.64:0/1204880213 >> 172.21.15.64:0/1204880213 conn(0x55f8c66dd460 msgr2=0x55f8c6a95380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.813+0000 7fc1baffd640 0 client.17169 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:12:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.813+0000 7fc1e85b8740 1 -- 172.21.15.64:0/1204880213 shutdown_connections 2024-09-21T04:12:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.813+0000 7fc1e85b8740 1 -- 172.21.15.64:0/1204880213 wait complete. 2024-09-21T04:12:15.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.813+0000 7fc1e85b8740 20 client.17169 trim_cache size 0 max 16384 2024-09-21T04:12:15.820 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:12:15.820 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-21T04:12:15.820 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {}, 2024-09-21T04:12:15.821 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_stats": { 2024-09-21T04:12:15.821 INFO:teuthology.orchestra.run.smithi064.stdout: "1b23713d-ea3a-4a86-a87f-65f6ab82671d": { 2024-09-21T04:12:15.821 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:10:58.352639+0000", 2024-09-21T04:12:15.821 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:15.821 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:15.821 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:15.821 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:15.821 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:15.821 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:15.821 INFO:teuthology.orchestra.run.smithi064.stdout: "71f3f255-4837-45e9-92bc-4b3abb5c20eb": { 2024-09-21T04:12:15.821 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:11:42.378549+0000", 2024-09-21T04:12:15.821 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:15.821 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:15.821 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:15.822 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:15.822 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:15.822 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:15.822 INFO:teuthology.orchestra.run.smithi064.stdout: "2c0523fb-aae4-43fa-9568-136a9b640051": { 2024-09-21T04:12:15.822 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:09:39.808606+0000", 2024-09-21T04:12:15.822 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:15.822 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:15.822 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:15.822 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:15.822 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:15.822 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:15.822 INFO:teuthology.orchestra.run.smithi064.stdout: "bed40d88-2102-43a6-ac2d-b89d5a2264d0": { 2024-09-21T04:12:15.822 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:11.363340+0000", 2024-09-21T04:12:15.823 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:15.823 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:15.823 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:15.823 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:15.823 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:15.823 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:15.823 INFO:teuthology.orchestra.run.smithi064.stdout: "e9242f88-6599-40eb-adf1-bdb3e946a1e0": { 2024-09-21T04:12:15.823 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:05:39.744646+0000", 2024-09-21T04:12:15.823 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:15.823 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:15.823 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:15.823 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:15.823 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:15.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:15.824 INFO:teuthology.orchestra.run.smithi064.stdout: "6bde0fe5-f775-44b1-ad0c-f9e596574dc0": { 2024-09-21T04:12:15.824 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:06:23.835536+0000", 2024-09-21T04:12:15.824 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:15.824 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:15.824 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:15.824 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:15.824 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:15.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:15.824 INFO:teuthology.orchestra.run.smithi064.stdout: "b52b950c-0f97-49ec-a37c-1a8e04348ce8": { 2024-09-21T04:12:15.824 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:54.003800+0000", 2024-09-21T04:12:15.824 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:15.824 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:15.824 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:15.824 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:15.825 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:15.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:15.825 INFO:teuthology.orchestra.run.smithi064.stdout: "e2c2eed8-9292-4bf6-8a39-13ad27da5acb": { 2024-09-21T04:12:15.825 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:08:47.067932+0000", 2024-09-21T04:12:15.825 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:15.825 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:15.825 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:15.825 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:15.825 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:15.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:15.825 INFO:teuthology.orchestra.run.smithi064.stdout: "75db1bc5-612e-45aa-bad2-cc90feeaa840": { 2024-09-21T04:12:15.825 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:59.123138+0000", 2024-09-21T04:12:15.825 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:15.825 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:15.826 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:15.826 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:15.826 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:15.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:15.826 INFO:teuthology.orchestra.run.smithi064.stdout: "0ab717a7-36c6-4dca-94e7-af0bdf2aa112": { 2024-09-21T04:12:15.826 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:42.691086+0000", 2024-09-21T04:12:15.826 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:15.826 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:15.826 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:15.826 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:15.826 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:15.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:15.826 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:12:15.827 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:03:59.570174+0000", 2024-09-21T04:12:15.827 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:15.827 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:15.827 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:15.827 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:15.827 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:15.827 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:12:15.827 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:12:15.827 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:12:15.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.814+0000 7fc1e2044640 1 -- 172.21.15.64:0/2923332640 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc1c40b6680 msgr2=0x7fc1c40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:15.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.814+0000 7fc1e2044640 1 --2- 172.21.15.64:0/2923332640 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc1c40b6680 0x7fc1c40b8b90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc1d41063e0 tx=0x7fc1d409b6e0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.814+0000 7fc1e2044640 1 -- 172.21.15.64:0/2923332640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1dc109e00 msgr2=0x7fc1dc19fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:15.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.814+0000 7fc1e2044640 1 --2- 172.21.15.64:0/2923332640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1dc109e00 0x7fc1dc19fdd0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4005cc0 tx=0x7fc1d4003ba0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.814+0000 7fc1e2044640 1 -- 172.21.15.64:0/2923332640 shutdown_connections 2024-09-21T04:12:15.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.814+0000 7fc1e2044640 1 --2- 172.21.15.64:0/2923332640 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fc1c40b6680 0x7fc1c40b8b90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.814+0000 7fc1e2044640 1 --2- 172.21.15.64:0/2923332640 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc1dc109e00 0x7fc1dc19fdd0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.814+0000 7fc1e2044640 1 --2- 172.21.15.64:0/2923332640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1dc1094b0 0x7fc1dc19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.814+0000 7fc1e2044640 1 --2- 172.21.15.64:0/2923332640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc1dc103bc0 0x7fc1dc19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:15.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.814+0000 7fc1e2044640 1 -- 172.21.15.64:0/2923332640 >> 172.21.15.64:0/2923332640 conn(0x7fc1dc07b910 msgr2=0x7fc1dc1075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:15.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.814+0000 7fc1e2044640 1 -- 172.21.15.64:0/2923332640 shutdown_connections 2024-09-21T04:12:15.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:15.814+0000 7fc1e2044640 1 -- 172.21.15.64:0/2923332640 wait complete. 2024-09-21T04:12:16.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:16 smithi064 ceph-mon[31180]: pgmap v573: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 KiB/s wr, 8 op/s 2024-09-21T04:12:16.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:16 smithi064 ceph-mon[31180]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T04:12:16.613 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T04:12:16.614 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:12:16.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:16 smithi119 ceph-mon[34811]: pgmap v573: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 KiB/s wr, 8 op/s 2024-09-21T04:12:16.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:16 smithi119 ceph-mon[34811]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T04:12:16.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:16 smithi131 ceph-mon[35193]: pgmap v573: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 KiB/s wr, 8 op/s 2024-09-21T04:12:16.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:16 smithi131 ceph-mon[35193]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T04:12:16.875 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:12:17.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.366+0000 7f48aaf95640 1 -- 172.21.15.64:0/1315093983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48a41025f0 msgr2=0x7f48a4102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:17.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.366+0000 7f48aaf95640 1 --2- 172.21.15.64:0/1315093983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48a41025f0 0x7f48a4102a00 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4898009900 tx=0x7f489802a480 comp rx=0 tx=0).stop 2024-09-21T04:12:17.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.366+0000 7f48aaf95640 1 -- 172.21.15.64:0/1315093983 shutdown_connections 2024-09-21T04:12:17.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.366+0000 7f48aaf95640 1 --2- 172.21.15.64:0/1315093983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a41101a0 0x7f48a41125c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:17.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.366+0000 7f48aaf95640 1 --2- 172.21.15.64:0/1315093983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48a4102f40 0x7f48a410fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:17.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.366+0000 7f48aaf95640 1 --2- 172.21.15.64:0/1315093983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48a41025f0 0x7f48a4102a00 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:17.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.366+0000 7f48aaf95640 1 -- 172.21.15.64:0/1315093983 >> 172.21.15.64:0/1315093983 conn(0x7f48a4079740 msgr2=0x7f48a4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:17.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.366+0000 7f48aaf95640 1 -- 172.21.15.64:0/1315093983 shutdown_connections 2024-09-21T04:12:17.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.366+0000 7f48aaf95640 1 -- 172.21.15.64:0/1315093983 wait complete. 2024-09-21T04:12:17.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.367+0000 7f48aaf95640 1 Processor -- start 2024-09-21T04:12:17.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.367+0000 7f48aaf95640 1 -- start start 2024-09-21T04:12:17.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.367+0000 7f48aaf95640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48a41025f0 0x7f48a419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:17.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.367+0000 7f48aaf95640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a4102f40 0x7f48a419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:17.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.367+0000 7f48aaf95640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48a41101a0 0x7f48a41a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:17.377 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.367+0000 7f48aaf95640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f48a4115c90 con 0x7f48a41101a0 2024-09-21T04:12:17.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.367+0000 7f48aaf95640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f48a4115b10 con 0x7f48a4102f40 2024-09-21T04:12:17.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.367+0000 7f48aaf95640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f48a4115e10 con 0x7f48a41025f0 2024-09-21T04:12:17.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.367+0000 7f48a9532640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48a41101a0 0x7f48a41a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:17.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.367+0000 7f48a9532640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48a41101a0 0x7f48a41a1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54728/0 (socket says 172.21.15.64:54728) 2024-09-21T04:12:17.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.367+0000 7f48a9532640 1 -- 172.21.15.64:0/494466502 learned_addr learned my addr 172.21.15.64:0/494466502 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:17.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.367+0000 7f48a3fff640 1 --2- 172.21.15.64:0/494466502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a4102f40 0x7f48a419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:17.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.367+0000 7f48a8d31640 1 --2- 172.21.15.64:0/494466502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48a41025f0 0x7f48a419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:17.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.367+0000 7f48a9532640 1 -- 172.21.15.64:0/494466502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48a41025f0 msgr2=0x7f48a419c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:17.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.367+0000 7f48a9532640 1 --2- 172.21.15.64:0/494466502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48a41025f0 0x7f48a419c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:17.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.367+0000 7f48a9532640 1 -- 172.21.15.64:0/494466502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a4102f40 msgr2=0x7f48a419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:17.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.367+0000 7f48a9532640 1 --2- 172.21.15.64:0/494466502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a4102f40 0x7f48a419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:17.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.367+0000 7f48a9532640 1 -- 172.21.15.64:0/494466502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a41a25d0 con 0x7f48a41101a0 2024-09-21T04:12:17.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.367+0000 7f48a3fff640 1 --2- 172.21.15.64:0/494466502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a4102f40 0x7f48a419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:12:17.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.368+0000 7f48a9532640 1 --2- 172.21.15.64:0/494466502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48a41101a0 0x7f48a41a1ea0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f489c00eaa0 tx=0x7f489c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:17.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.368+0000 7f48a1ffb640 1 -- 172.21.15.64:0/494466502 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489c00cdb0 con 0x7f48a41101a0 2024-09-21T04:12:17.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.368+0000 7f48a1ffb640 1 -- 172.21.15.64:0/494466502 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f489c005020 con 0x7f48a41101a0 2024-09-21T04:12:17.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.368+0000 7f48aaf95640 1 -- 172.21.15.64:0/494466502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a41a28c0 con 0x7f48a41101a0 2024-09-21T04:12:17.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.368+0000 7f48a1ffb640 1 -- 172.21.15.64:0/494466502 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489c0116d0 con 0x7f48a41101a0 2024-09-21T04:12:17.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.368+0000 7f48aaf95640 1 -- 172.21.15.64:0/494466502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a41a9020 con 0x7f48a41101a0 2024-09-21T04:12:17.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.369+0000 7f48aaf95640 1 -- 172.21.15.64:0/494466502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48a406c240 con 0x7f48a41101a0 2024-09-21T04:12:17.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.370+0000 7f48a1ffb640 1 -- 172.21.15.64:0/494466502 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f489c0051d0 con 0x7f48a41101a0 2024-09-21T04:12:17.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.371+0000 7f48a1ffb640 1 --2- 172.21.15.64:0/494466502 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f48940b6700 0x7f48940b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:17.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.371+0000 7f48a8d31640 1 --2- 172.21.15.64:0/494466502 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f48940b6700 0x7f48940b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:17.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.371+0000 7f48a1ffb640 1 -- 172.21.15.64:0/494466502 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7f489c0e7980 con 0x7f48a41101a0 2024-09-21T04:12:17.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.372+0000 7f48a8d31640 1 --2- 172.21.15.64:0/494466502 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f48940b6700 0x7f48940b8c10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4898005f70 tx=0x7f4898000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:17.380 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.376+0000 7f48a1ffb640 1 -- 172.21.15.64:0/494466502 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f489c016030 con 0x7f48a41101a0 2024-09-21T04:12:17.606 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.604+0000 7f48aaf95640 1 -- 172.21.15.64:0/494466502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48a402e640 con 0x7f48a41101a0 2024-09-21T04:12:17.607 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.605+0000 7f48a1ffb640 1 -- 172.21.15.64:0/494466502 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11918 (secure 0 0 0) 0x7f489c0ade70 con 0x7f48a41101a0 2024-09-21T04:12:17.607 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:12:17.608 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":40,"btime":"2024-09-21T04:11:59:330881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14805,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/97995119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":97995119},{"type":"v1","addr":"172.21.15.64:6839","nonce":97995119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/390891068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":390891068},{"type":"v1","addr":"172.21.15.64:6841","nonce":390891068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:02:03.354744+0000","modified":"2024-09-21T04:11:59.188679+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24710,"mds_1":24680,"mds_2":24661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24634":{"gid":24634,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6839/782190424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782190424},{"type":"v1","addr":"172.21.15.131:6839","nonce":782190424}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24680":{"gid":24680,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24710":{"gid":24710,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6839/1319793636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1319793636},{"type":"v1","addr":"172.21.15.119:6839","nonce":1319793636}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24710,"qdb_cluster":[24661,24680,24710]},"id":1}]} 2024-09-21T04:12:17.608 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 40 2024-09-21T04:12:17.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.608+0000 7f48aaf95640 1 -- 172.21.15.64:0/494466502 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f48940b6700 msgr2=0x7f48940b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:17.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.608+0000 7f48aaf95640 1 --2- 172.21.15.64:0/494466502 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f48940b6700 0x7f48940b8c10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4898005f70 tx=0x7f4898000bc0 comp rx=0 tx=0).stop 2024-09-21T04:12:17.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.608+0000 7f48aaf95640 1 -- 172.21.15.64:0/494466502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48a41101a0 msgr2=0x7f48a41a1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:17.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.608+0000 7f48aaf95640 1 --2- 172.21.15.64:0/494466502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48a41101a0 0x7f48a41a1ea0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f489c00eaa0 tx=0x7f489c00ef60 comp rx=0 tx=0).stop 2024-09-21T04:12:17.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.608+0000 7f48aaf95640 1 -- 172.21.15.64:0/494466502 shutdown_connections 2024-09-21T04:12:17.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.608+0000 7f48aaf95640 1 --2- 172.21.15.64:0/494466502 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f48940b6700 0x7f48940b8c10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:17.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.608+0000 7f48aaf95640 1 --2- 172.21.15.64:0/494466502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48a41101a0 0x7f48a41a1ea0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:17.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.608+0000 7f48aaf95640 1 --2- 172.21.15.64:0/494466502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48a4102f40 0x7f48a419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:17.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.608+0000 7f48aaf95640 1 --2- 172.21.15.64:0/494466502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48a41025f0 0x7f48a419c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:17.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.608+0000 7f48aaf95640 1 -- 172.21.15.64:0/494466502 >> 172.21.15.64:0/494466502 conn(0x7f48a4079740 msgr2=0x7f48a4111620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:17.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.608+0000 7f48aaf95640 1 -- 172.21.15.64:0/494466502 shutdown_connections 2024-09-21T04:12:17.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:17.608+0000 7f48aaf95640 1 -- 172.21.15.64:0/494466502 wait complete. 2024-09-21T04:12:18.325 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T04:12:18.589 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:12:18.710 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:18 smithi064 ceph-mon[31180]: pgmap v575: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 985 B/s wr, 5 op/s 2024-09-21T04:12:18.710 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:18 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/494466502' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:12:18.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:18 smithi119 ceph-mon[34811]: pgmap v575: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 985 B/s wr, 5 op/s 2024-09-21T04:12:18.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:18 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/494466502' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:12:18.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:18 smithi131 ceph-mon[35193]: pgmap v575: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 985 B/s wr, 5 op/s 2024-09-21T04:12:18.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:18 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/494466502' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:12:19.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.085+0000 7fad7b048640 1 -- 172.21.15.64:0/2155265262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad74103b60 msgr2=0x7fad74103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:19.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.085+0000 7fad7b048640 1 --2- 172.21.15.64:0/2155265262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad74103b60 0x7fad74103f70 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fad68009900 tx=0x7fad6802a410 comp rx=0 tx=0).stop 2024-09-21T04:12:19.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.086+0000 7fad7b048640 1 -- 172.21.15.64:0/2155265262 shutdown_connections 2024-09-21T04:12:19.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.086+0000 7fad7b048640 1 --2- 172.21.15.64:0/2155265262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad74109df0 0x7fad74110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.086+0000 7fad7b048640 1 --2- 172.21.15.64:0/2155265262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad741094a0 0x7fad741098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.086+0000 7fad7b048640 1 --2- 172.21.15.64:0/2155265262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad74103b60 0x7fad74103f70 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.086+0000 7fad7b048640 1 -- 172.21.15.64:0/2155265262 >> 172.21.15.64:0/2155265262 conn(0x7fad7407b850 msgr2=0x7fad7407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:19.095 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.086+0000 7fad7b048640 1 -- 172.21.15.64:0/2155265262 shutdown_connections 2024-09-21T04:12:19.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.086+0000 7fad7b048640 1 -- 172.21.15.64:0/2155265262 wait complete. 2024-09-21T04:12:19.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.086+0000 7fad7b048640 1 Processor -- start 2024-09-21T04:12:19.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.086+0000 7fad7b048640 1 -- start start 2024-09-21T04:12:19.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.087+0000 7fad7b048640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad74103b60 0x7fad7419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:19.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.087+0000 7fad7b048640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad741094a0 0x7fad7419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:19.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.087+0000 7fad7b048640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad74109df0 0x7fad7419fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:19.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.087+0000 7fad7b048640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fad74113b70 con 0x7fad74109df0 2024-09-21T04:12:19.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.087+0000 7fad7b048640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fad741139f0 con 0x7fad741094a0 2024-09-21T04:12:19.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.087+0000 7fad7b048640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fad74113cf0 con 0x7fad74103b60 2024-09-21T04:12:19.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.087+0000 7fad795e5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad74109df0 0x7fad7419fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:19.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.087+0000 7fad78de4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad74103b60 0x7fad7419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:19.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.087+0000 7fad795e5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad74109df0 0x7fad7419fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54746/0 (socket says 172.21.15.64:54746) 2024-09-21T04:12:19.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.087+0000 7fad795e5640 1 -- 172.21.15.64:0/3049502919 learned_addr learned my addr 172.21.15.64:0/3049502919 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:19.097 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.087+0000 7fad73fff640 1 --2- 172.21.15.64:0/3049502919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad741094a0 0x7fad7419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:19.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.087+0000 7fad795e5640 1 -- 172.21.15.64:0/3049502919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad74103b60 msgr2=0x7fad7419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:19.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.087+0000 7fad795e5640 1 --2- 172.21.15.64:0/3049502919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad74103b60 0x7fad7419a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.087+0000 7fad795e5640 1 -- 172.21.15.64:0/3049502919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad741094a0 msgr2=0x7fad7419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:19.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.087+0000 7fad795e5640 1 --2- 172.21.15.64:0/3049502919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad741094a0 0x7fad7419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.087+0000 7fad795e5640 1 -- 172.21.15.64:0/3049502919 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad741a0530 con 0x7fad74109df0 2024-09-21T04:12:19.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.087+0000 7fad795e5640 1 --2- 172.21.15.64:0/3049502919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad74109df0 0x7fad7419fe00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00b970 tx=0x7fad6c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:19.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.088+0000 7fad71ffb640 1 -- 172.21.15.64:0/3049502919 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c019070 con 0x7fad74109df0 2024-09-21T04:12:19.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.088+0000 7fad7b048640 1 -- 172.21.15.64:0/3049502919 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad741a0820 con 0x7fad74109df0 2024-09-21T04:12:19.098 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.088+0000 7fad71ffb640 1 -- 172.21.15.64:0/3049502919 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad6c014440 con 0x7fad74109df0 2024-09-21T04:12:19.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.088+0000 7fad71ffb640 1 -- 172.21.15.64:0/3049502919 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c0075d0 con 0x7fad74109df0 2024-09-21T04:12:19.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.088+0000 7fad7b048640 1 -- 172.21.15.64:0/3049502919 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad741a6f80 con 0x7fad74109df0 2024-09-21T04:12:19.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.090+0000 7fad815bc740 1 Processor -- start 2024-09-21T04:12:19.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.090+0000 7fad71ffb640 1 -- 172.21.15.64:0/3049502919 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fad6c01c070 con 0x7fad74109df0 2024-09-21T04:12:19.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.090+0000 7fad815bc740 1 -- start start 2024-09-21T04:12:19.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.091+0000 7fad815bc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b0ed06070 0x559b0f0ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:19.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.091+0000 7fad815bc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b0ecdd1b0 0x559b0f0ad800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:19.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.091+0000 7fad815bc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b0ece5390 0x559b0f0add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:19.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.091+0000 7fad815bc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559b0f023ea0 con 0x559b0ece5390 2024-09-21T04:12:19.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.091+0000 7fad815bc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559b0f0132d0 con 0x559b0ed06070 2024-09-21T04:12:19.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.091+0000 7fad815bc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559b0f0170e0 con 0x559b0ecdd1b0 2024-09-21T04:12:19.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.091+0000 7fad73fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b0ece5390 0x559b0f0add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:19.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.091+0000 7fad73fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b0ece5390 0x559b0f0add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54750/0 (socket says 172.21.15.64:54750) 2024-09-21T04:12:19.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.091+0000 7fad73fff640 1 -- 172.21.15.64:0/3314781269 learned_addr learned my addr 172.21.15.64:0/3314781269 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:19.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.091+0000 7fad78de4640 1 --2- 172.21.15.64:0/3314781269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b0ecdd1b0 0x559b0f0ad800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:19.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.091+0000 7fad73fff640 1 --2- 172.21.15.64:0/3314781269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b0ed06070 0x559b0f0ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:19.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.091+0000 7fad71ffb640 1 --2- 172.21.15.64:0/3049502919 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fad640b6680 0x7fad640b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:19.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.091+0000 7fad78de4640 1 -- 172.21.15.64:0/3314781269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b0ed06070 msgr2=0x559b0f0ad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:19.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.091+0000 7fad78de4640 1 --2- 172.21.15.64:0/3314781269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b0ed06070 0x559b0f0ad2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.091+0000 7fad78de4640 1 -- 172.21.15.64:0/3314781269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b0ece5390 msgr2=0x559b0f0add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:19.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.091+0000 7fad78de4640 1 --2- 172.21.15.64:0/3314781269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b0ece5390 0x559b0f0add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.091+0000 7fad78de4640 1 -- 172.21.15.64:0/3314781269 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b0ed71650 con 0x559b0ecdd1b0 2024-09-21T04:12:19.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.091+0000 7fad795e5640 1 --2- 172.21.15.64:0/3049502919 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fad640b6680 0x7fad640b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:19.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.092+0000 7fad78de4640 1 --2- 172.21.15.64:0/3314781269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b0ecdd1b0 0x559b0f0ad800 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fad68009230 tx=0x7fad68031e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da0f299bcab04b42 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:19.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.092+0000 7fad71ffb640 1 -- 172.21.15.64:0/3049502919 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7fad6c0e6ba0 con 0x7fad74109df0 2024-09-21T04:12:19.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.092+0000 7fad5effd640 1 -- 172.21.15.64:0/3314781269 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad68040070 con 0x559b0ecdd1b0 2024-09-21T04:12:19.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.092+0000 7fad795e5640 1 --2- 172.21.15.64:0/3049502919 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fad640b6680 0x7fad640b8b90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0ac740 tx=0x7fad6c0a9110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:19.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.092+0000 7fad5effd640 1 -- 172.21.15.64:0/3314781269 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad680316a0 con 0x559b0ecdd1b0 2024-09-21T04:12:19.101 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.092+0000 7fad5effd640 1 -- 172.21.15.64:0/3314781269 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad68035040 con 0x559b0ecdd1b0 2024-09-21T04:12:19.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.093+0000 7fad815bc740 1 -- 172.21.15.64:0/3314781269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b0ecdd1b0 msgr2=0x559b0f0ad800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:19.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.093+0000 7fad815bc740 1 --2- 172.21.15.64:0/3314781269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b0ecdd1b0 0x559b0f0ad800 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fad68009230 tx=0x7fad68031e50 comp rx=0 tx=0).stop 2024-09-21T04:12:19.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.093+0000 7fad815bc740 1 -- 172.21.15.64:0/3314781269 shutdown_connections 2024-09-21T04:12:19.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.093+0000 7fad815bc740 1 --2- 172.21.15.64:0/3314781269 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b0ece5390 0x559b0f0add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.093+0000 7fad815bc740 1 --2- 172.21.15.64:0/3314781269 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b0ecdd1b0 0x559b0f0ad800 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.093+0000 7fad815bc740 1 --2- 172.21.15.64:0/3314781269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b0ed06070 0x559b0f0ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.093+0000 7fad815bc740 1 -- 172.21.15.64:0/3314781269 >> 172.21.15.64:0/3314781269 conn(0x559b0ecfa460 msgr2=0x559b0f0ab1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:19.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.093+0000 7fad815bc740 1 -- 172.21.15.64:0/3314781269 shutdown_connections 2024-09-21T04:12:19.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.093+0000 7fad815bc740 1 -- 172.21.15.64:0/3314781269 wait complete. 2024-09-21T04:12:19.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.094+0000 7fad815bc740 1 -- start start 2024-09-21T04:12:19.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.094+0000 7fad815bc740 1 Processor -- start 2024-09-21T04:12:19.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.094+0000 7fad815bc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b0ecdd1b0 0x559b0f0b8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:19.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.094+0000 7fad815bc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b0ece5390 0x559b0f0b8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:19.104 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.094+0000 7fad815bc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b0ed06070 0x559b0f0b93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:19.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.094+0000 7fad815bc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559b0ee74700 con 0x559b0ecdd1b0 2024-09-21T04:12:19.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.094+0000 7fad815bc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559b0efae160 con 0x559b0ed06070 2024-09-21T04:12:19.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.094+0000 7fad815bc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559b0f0bba40 con 0x559b0ece5390 2024-09-21T04:12:19.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.094+0000 7fad815bc740 10 client.0 _ll_register_callbacks cb 0x559b0efc79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:12:19.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.094+0000 7fad73fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b0ecdd1b0 0x559b0f0b8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:19.105 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.094+0000 7fad73fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b0ecdd1b0 0x559b0f0b8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54760/0 (socket says 172.21.15.64:54760) 2024-09-21T04:12:19.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.094+0000 7fad73fff640 1 -- 172.21.15.64:0/981716925 learned_addr learned my addr 172.21.15.64:0/981716925 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:19.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.095+0000 7fad78de4640 1 --2- 172.21.15.64:0/981716925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b0ece5390 0x559b0f0b8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:19.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.095+0000 7fad73fff640 1 -- 172.21.15.64:0/981716925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b0ece5390 msgr2=0x559b0f0b8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:19.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.095+0000 7fad73fff640 1 --2- 172.21.15.64:0/981716925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b0ece5390 0x559b0f0b8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.095+0000 7fad73fff640 1 -- 172.21.15.64:0/981716925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b0ed06070 msgr2=0x559b0f0b93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:19.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.095+0000 7fad73fff640 1 --2- 172.21.15.64:0/981716925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b0ed06070 0x559b0f0b93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.095+0000 7fad73fff640 1 -- 172.21.15.64:0/981716925 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b0efdbc90 con 0x559b0ecdd1b0 2024-09-21T04:12:19.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.095+0000 7fad73fff640 1 --2- 172.21.15.64:0/981716925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b0ecdd1b0 0x559b0f0b8910 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fad60009c40 tx=0x7fad60017ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161c61546a02b842 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:19.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.095+0000 7fad53fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:12:19.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.095+0000 7fad53fff640 1 -- 172.21.15.64:0/981716925 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6001c070 con 0x559b0ecdd1b0 2024-09-21T04:12:19.106 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.095+0000 7fad53fff640 1 -- 172.21.15.64:0/981716925 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7fad60016be0 con 0x559b0ecdd1b0 2024-09-21T04:12:19.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.095+0000 7fad815bc740 1 -- 172.21.15.64:0/981716925 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad680316a0 con 0x559b0ecdd1b0 2024-09-21T04:12:19.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.095+0000 7fad53fff640 1 -- 172.21.15.64:0/981716925 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fad60028050 con 0x559b0ecdd1b0 2024-09-21T04:12:19.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.095+0000 7fad815bc740 10 client.17187 fetch_fsmap learned FSMap version 40 2024-09-21T04:12:19.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.095+0000 7fad815bc740 1 -- 172.21.15.64:0/981716925 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559b0f0c7750 con 0x559b0ecdd1b0 2024-09-21T04:12:19.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.096+0000 7fad53fff640 1 -- 172.21.15.64:0/981716925 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad680316a0 con 0x559b0ecdd1b0 2024-09-21T04:12:19.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.096+0000 7fad53fff640 1 -- 172.21.15.64:0/981716925 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7fad600203b0 con 0x559b0ecdd1b0 2024-09-21T04:12:19.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.096+0000 7fad815bc740 10 client.17187 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:12:19.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.096+0000 7fad815bc740 10 client.17187 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:12:19.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.096+0000 7fad815bc740 20 client.17187 populate_metadata read hostname 'smithi064' 2024-09-21T04:12:19.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.096+0000 7fad815bc740 1 --2- 172.21.15.64:0/981716925 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x559b0ecc49f0 0x559b0f0cac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:19.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.096+0000 7fad815bc740 4 client.17187 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:12:19.107 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.096+0000 7fad815bc740 1 -- 172.21.15.64:0/981716925 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b0f025df0 con 0x559b0ecc49f0 2024-09-21T04:12:19.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.096+0000 7fad78de4640 1 --2- 172.21.15.64:0/981716925 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x559b0ecc49f0 0x559b0f0cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:19.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.097+0000 7fad78de4640 1 --2- 172.21.15.64:0/981716925 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x559b0ecc49f0 0x559b0f0cac90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ad09e4f6150be3a server_cookie=d6bafbda40f10969 in_seq=0 out_seq=0 2024-09-21T04:12:19.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.097+0000 7fad53fff640 10 client.17187 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:12:19.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.098+0000 7fad53fff640 1 -- 172.21.15.64:0/981716925 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559b0f025df0 con 0x559b0ecc49f0 2024-09-21T04:12:19.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.098+0000 7fad53fff640 10 client.17187 handle_command_reply: tid=0 2024-09-21T04:12:19.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.098+0000 7fad815bc740 1 client.17187 shutdown 2024-09-21T04:12:19.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.098+0000 7fad815bc740 1 -- 172.21.15.64:0/981716925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b0ecdd1b0 msgr2=0x559b0f0b8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:19.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.098+0000 7fad815bc740 1 --2- 172.21.15.64:0/981716925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b0ecdd1b0 0x559b0f0b8910 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fad60009c40 tx=0x7fad60017ec0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.098+0000 7fad815bc740 1 -- 172.21.15.64:0/981716925 shutdown_connections 2024-09-21T04:12:19.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.098+0000 7fad815bc740 1 --2- 172.21.15.64:0/981716925 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x559b0ecc49f0 0x559b0f0cac90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.108 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.098+0000 7fad815bc740 1 --2- 172.21.15.64:0/981716925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b0ed06070 0x559b0f0b93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.098+0000 7fad815bc740 1 --2- 172.21.15.64:0/981716925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b0ece5390 0x559b0f0b8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.098+0000 7fad815bc740 1 --2- 172.21.15.64:0/981716925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b0ecdd1b0 0x559b0f0b8910 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.098+0000 7fad815bc740 1 -- 172.21.15.64:0/981716925 >> 172.21.15.64:0/981716925 conn(0x559b0ecfa460 msgr2=0x559b0f0b0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:19.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.098+0000 7fad53fff640 0 client.17187 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:12:19.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.098+0000 7fad815bc740 1 -- 172.21.15.64:0/981716925 shutdown_connections 2024-09-21T04:12:19.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.098+0000 7fad815bc740 1 -- 172.21.15.64:0/981716925 wait complete. 2024-09-21T04:12:19.109 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.098+0000 7fad815bc740 20 client.17187 trim_cache size 0 max 16384 2024-09-21T04:12:19.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.112+0000 7fad815bc740 1 Processor -- start 2024-09-21T04:12:19.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.112+0000 7fad815bc740 1 -- start start 2024-09-21T04:12:19.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.112+0000 7fad815bc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b0ecdd1b0 0x559b0f0aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:19.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.112+0000 7fad815bc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b0ece5390 0x559b0f0aa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:19.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.112+0000 7fad815bc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b0ed06070 0x559b0f0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:19.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.112+0000 7fad815bc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fad741b2320 con 0x559b0ecdd1b0 2024-09-21T04:12:19.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.112+0000 7fad815bc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fad741afb30 con 0x559b0ed06070 2024-09-21T04:12:19.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.112+0000 7fad815bc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fad741ad9a0 con 0x559b0ece5390 2024-09-21T04:12:19.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.112+0000 7fad73fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b0ecdd1b0 0x559b0f0aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:19.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.112+0000 7fad73fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b0ecdd1b0 0x559b0f0aea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54774/0 (socket says 172.21.15.64:54774) 2024-09-21T04:12:19.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.112+0000 7fad73fff640 1 -- 172.21.15.64:0/2377873700 learned_addr learned my addr 172.21.15.64:0/2377873700 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:19.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.112+0000 7fad73fff640 1 -- 172.21.15.64:0/2377873700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b0ece5390 msgr2=0x559b0f0aa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:19.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.112+0000 7fad73fff640 1 --2- 172.21.15.64:0/2377873700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b0ece5390 0x559b0f0aa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.112+0000 7fad73fff640 1 -- 172.21.15.64:0/2377873700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b0ed06070 msgr2=0x559b0f0aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:19.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.112+0000 7fad73fff640 1 --2- 172.21.15.64:0/2377873700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b0ed06070 0x559b0f0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.112+0000 7fad73fff640 1 -- 172.21.15.64:0/2377873700 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b0f0af5b0 con 0x559b0ecdd1b0 2024-09-21T04:12:19.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.112+0000 7fad73fff640 1 --2- 172.21.15.64:0/2377873700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b0ecdd1b0 0x559b0f0aea00 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fad60020920 tx=0x7fad60006e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3d0d16fb5dcdb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:19.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.113+0000 7fad5e7fc640 1 -- 172.21.15.64:0/2377873700 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad60025070 con 0x559b0ecdd1b0 2024-09-21T04:12:19.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.113+0000 7fad5e7fc640 1 -- 172.21.15.64:0/2377873700 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad600209a0 con 0x559b0ecdd1b0 2024-09-21T04:12:19.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.113+0000 7fad5e7fc640 1 -- 172.21.15.64:0/2377873700 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6001c070 con 0x559b0ecdd1b0 2024-09-21T04:12:19.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.113+0000 7fad815bc740 1 -- 172.21.15.64:0/2377873700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b0ecdd1b0 msgr2=0x559b0f0aea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:19.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.113+0000 7fad815bc740 1 --2- 172.21.15.64:0/2377873700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b0ecdd1b0 0x559b0f0aea00 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fad60020920 tx=0x7fad60006e40 comp rx=0 tx=0).stop 2024-09-21T04:12:19.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.114+0000 7fad815bc740 1 -- 172.21.15.64:0/2377873700 shutdown_connections 2024-09-21T04:12:19.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.114+0000 7fad815bc740 1 --2- 172.21.15.64:0/2377873700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b0ed06070 0x559b0f0aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.114+0000 7fad815bc740 1 --2- 172.21.15.64:0/2377873700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b0ece5390 0x559b0f0aa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.114+0000 7fad815bc740 1 --2- 172.21.15.64:0/2377873700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b0ecdd1b0 0x559b0f0aea00 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.114+0000 7fad815bc740 1 -- 172.21.15.64:0/2377873700 >> 172.21.15.64:0/2377873700 conn(0x559b0ecfa460 msgr2=0x559b0f0ae2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:19.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.114+0000 7fad815bc740 1 -- 172.21.15.64:0/2377873700 shutdown_connections 2024-09-21T04:12:19.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.114+0000 7fad815bc740 1 -- 172.21.15.64:0/2377873700 wait complete. 2024-09-21T04:12:19.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.114+0000 7fad815bc740 1 -- start start 2024-09-21T04:12:19.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.114+0000 7fad815bc740 1 Processor -- start 2024-09-21T04:12:19.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.115+0000 7fad815bc740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b0ecdd1b0 0x559b0f0bd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:19.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.115+0000 7fad815bc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b0ece5390 0x559b0f0bdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:19.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.115+0000 7fad815bc740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b0ed06070 0x559b0f12e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:19.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.115+0000 7fad815bc740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fad741adff0 con 0x559b0ed06070 2024-09-21T04:12:19.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.115+0000 7fad815bc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fad741af460 con 0x559b0ece5390 2024-09-21T04:12:19.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.115+0000 7fad815bc740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fad741ae6c0 con 0x559b0ecdd1b0 2024-09-21T04:12:19.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.115+0000 7fad815bc740 10 client.0 _ll_register_callbacks cb 0x7fad60009280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:12:19.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.115+0000 7fad73fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b0ed06070 0x559b0f12e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:19.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.115+0000 7fad78de4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b0ece5390 0x559b0f0bdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:19.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.115+0000 7fad73fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b0ed06070 0x559b0f12e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54788/0 (socket says 172.21.15.64:54788) 2024-09-21T04:12:19.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.115+0000 7fad73fff640 1 -- 172.21.15.64:0/3637759674 learned_addr learned my addr 172.21.15.64:0/3637759674 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:19.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.115+0000 7fad73fff640 1 --2- 172.21.15.64:0/3637759674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b0ecdd1b0 0x559b0f0bd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:19.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.115+0000 7fad73fff640 1 -- 172.21.15.64:0/3637759674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b0ecdd1b0 msgr2=0x559b0f0bd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:19.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.115+0000 7fad73fff640 1 --2- 172.21.15.64:0/3637759674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b0ecdd1b0 0x559b0f0bd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.115+0000 7fad73fff640 1 -- 172.21.15.64:0/3637759674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b0ece5390 msgr2=0x559b0f0bdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:19.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.115+0000 7fad73fff640 1 --2- 172.21.15.64:0/3637759674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b0ece5390 0x559b0f0bdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.115+0000 7fad73fff640 1 -- 172.21.15.64:0/3637759674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b0f12ec90 con 0x559b0ed06070 2024-09-21T04:12:19.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.115+0000 7fad73fff640 1 --2- 172.21.15.64:0/3637759674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b0ed06070 0x559b0f12e620 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fad60009570 tx=0x7fad60016e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4040f611d0ee179 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:19.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.115+0000 7fad53fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:12:19.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.115+0000 7fad53fff640 1 -- 172.21.15.64:0/3637759674 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6001c030 con 0x559b0ed06070 2024-09-21T04:12:19.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.116+0000 7fad53fff640 1 -- 172.21.15.64:0/3637759674 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7fad60020470 con 0x559b0ed06070 2024-09-21T04:12:19.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.116+0000 7fad815bc740 1 -- 172.21.15.64:0/3637759674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad600209a0 con 0x559b0ed06070 2024-09-21T04:12:19.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.116+0000 7fad53fff640 1 -- 172.21.15.64:0/3637759674 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fad60025040 con 0x559b0ed06070 2024-09-21T04:12:19.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.116+0000 7fad815bc740 10 client.17193 fetch_fsmap learned FSMap version 40 2024-09-21T04:12:19.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.116+0000 7fad815bc740 1 -- 172.21.15.64:0/3637759674 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559b0f135fd0 con 0x559b0ed06070 2024-09-21T04:12:19.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.116+0000 7fad53fff640 1 -- 172.21.15.64:0/3637759674 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad600209a0 con 0x559b0ed06070 2024-09-21T04:12:19.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.116+0000 7fad53fff640 1 -- 172.21.15.64:0/3637759674 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7fad600214d0 con 0x559b0ed06070 2024-09-21T04:12:19.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.116+0000 7fad815bc740 10 client.17193 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:12:19.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.116+0000 7fad815bc740 10 client.17193 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:12:19.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.116+0000 7fad815bc740 20 client.17193 populate_metadata read hostname 'smithi064' 2024-09-21T04:12:19.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.116+0000 7fad815bc740 1 --2- 172.21.15.64:0/3637759674 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x559b0ec99da0 0x559b0f138640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:19.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.116+0000 7fad815bc740 4 client.17193 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:12:19.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.116+0000 7fad815bc740 1 -- 172.21.15.64:0/3637759674 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b0f0218a0 con 0x559b0ec99da0 2024-09-21T04:12:19.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.117+0000 7fad78de4640 1 --2- 172.21.15.64:0/3637759674 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x559b0ec99da0 0x559b0f138640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:19.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.117+0000 7fad53fff640 10 client.17193 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:12:19.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.117+0000 7fad78de4640 1 --2- 172.21.15.64:0/3637759674 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x559b0ec99da0 0x559b0f138640 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d20a055e57a850c1 server_cookie=a4be88a29f015ca6 in_seq=0 out_seq=0 2024-09-21T04:12:19.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.117+0000 7fad53fff640 1 -- 172.21.15.64:0/3637759674 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b0f0218a0 con 0x559b0ec99da0 2024-09-21T04:12:19.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.117+0000 7fad53fff640 10 client.17193 handle_command_reply: tid=0 2024-09-21T04:12:19.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.117+0000 7fad815bc740 1 client.17193 shutdown 2024-09-21T04:12:19.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.117+0000 7fad815bc740 1 -- 172.21.15.64:0/3637759674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b0ed06070 msgr2=0x559b0f12e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:19.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.117+0000 7fad815bc740 1 --2- 172.21.15.64:0/3637759674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b0ed06070 0x559b0f12e620 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fad60009570 tx=0x7fad60016e40 comp rx=0 tx=0).stop 2024-09-21T04:12:19.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.117+0000 7fad815bc740 1 -- 172.21.15.64:0/3637759674 shutdown_connections 2024-09-21T04:12:19.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.117+0000 7fad815bc740 1 --2- 172.21.15.64:0/3637759674 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x559b0ec99da0 0x559b0f138640 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.117+0000 7fad815bc740 1 --2- 172.21.15.64:0/3637759674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559b0ed06070 0x559b0f12e620 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.117+0000 7fad815bc740 1 --2- 172.21.15.64:0/3637759674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b0ece5390 0x559b0f0bdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.117+0000 7fad815bc740 1 --2- 172.21.15.64:0/3637759674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559b0ecdd1b0 0x559b0f0bd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.117+0000 7fad53fff640 0 client.17193 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:12:19.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.117+0000 7fad815bc740 1 -- 172.21.15.64:0/3637759674 >> 172.21.15.64:0/3637759674 conn(0x559b0ecfa460 msgr2=0x559b0f0b9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:19.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.118+0000 7fad815bc740 1 -- 172.21.15.64:0/3637759674 shutdown_connections 2024-09-21T04:12:19.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.118+0000 7fad815bc740 1 -- 172.21.15.64:0/3637759674 wait complete. 2024-09-21T04:12:19.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.118+0000 7fad815bc740 20 client.17193 trim_cache size 0 max 16384 2024-09-21T04:12:19.127 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:12:19.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.118+0000 7fad7b048640 1 -- 172.21.15.64:0/3049502919 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fad640b6680 msgr2=0x7fad640b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:19.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.118+0000 7fad7b048640 1 --2- 172.21.15.64:0/3049502919 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fad640b6680 0x7fad640b8b90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0ac740 tx=0x7fad6c0a9110 comp rx=0 tx=0).stop 2024-09-21T04:12:19.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.118+0000 7fad7b048640 1 -- 172.21.15.64:0/3049502919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad74109df0 msgr2=0x7fad7419fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:19.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.118+0000 7fad7b048640 1 --2- 172.21.15.64:0/3049502919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad74109df0 0x7fad7419fe00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00b970 tx=0x7fad6c00bd70 comp rx=0 tx=0).stop 2024-09-21T04:12:19.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.118+0000 7fad7b048640 1 -- 172.21.15.64:0/3049502919 shutdown_connections 2024-09-21T04:12:19.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.118+0000 7fad7b048640 1 --2- 172.21.15.64:0/3049502919 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fad640b6680 0x7fad640b8b90 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.118+0000 7fad7b048640 1 --2- 172.21.15.64:0/3049502919 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad74109df0 0x7fad7419fe00 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.118+0000 7fad7b048640 1 --2- 172.21.15.64:0/3049502919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad741094a0 0x7fad7419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.118+0000 7fad7b048640 1 --2- 172.21.15.64:0/3049502919 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad74103b60 0x7fad7419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:19.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.118+0000 7fad7b048640 1 -- 172.21.15.64:0/3049502919 >> 172.21.15.64:0/3049502919 conn(0x7fad7407b850 msgr2=0x7fad741075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:19.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.119+0000 7fad7b048640 1 -- 172.21.15.64:0/3049502919 shutdown_connections 2024-09-21T04:12:19.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:19.119+0000 7fad7b048640 1 -- 172.21.15.64:0/3049502919 wait complete. 2024-09-21T04:12:19.754 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T04:12:20.016 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:12:20.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.501+0000 7f5ff23f8640 1 -- 172.21.15.64:0/3831975477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fec108d10 msgr2=0x7f5fec10f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:20.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.501+0000 7f5ff23f8640 1 --2- 172.21.15.64:0/3831975477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fec108d10 0x7f5fec10f540 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009ca0 tx=0x7f5fe402a8c0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.501+0000 7f5ff23f8640 1 -- 172.21.15.64:0/3831975477 shutdown_connections 2024-09-21T04:12:20.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.501+0000 7f5ff23f8640 1 --2- 172.21.15.64:0/3831975477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fec108d10 0x7f5fec10f540 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.501+0000 7f5ff23f8640 1 --2- 172.21.15.64:0/3831975477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fec1083c0 0x7f5fec1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.501+0000 7f5ff23f8640 1 --2- 172.21.15.64:0/3831975477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fec102ad0 0x7f5fec102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.501+0000 7f5ff23f8640 1 -- 172.21.15.64:0/3831975477 >> 172.21.15.64:0/3831975477 conn(0x7f5fec0796d0 msgr2=0x7f5fec079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:20.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.501+0000 7f5ff23f8640 1 -- 172.21.15.64:0/3831975477 shutdown_connections 2024-09-21T04:12:20.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.501+0000 7f5ff23f8640 1 -- 172.21.15.64:0/3831975477 wait complete. 2024-09-21T04:12:20.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.501+0000 7f5ff23f8640 1 Processor -- start 2024-09-21T04:12:20.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.501+0000 7f5ff23f8640 1 -- start start 2024-09-21T04:12:20.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.502+0000 7f5ff23f8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fec102ad0 0x7f5fec19c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.502+0000 7f5ff23f8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fec1083c0 0x7f5fec19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.502+0000 7f5ff23f8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fec108d10 0x7f5fec1a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.502+0000 7f5ff23f8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec1129d0 con 0x7f5fec102ad0 2024-09-21T04:12:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.502+0000 7f5ff23f8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec112850 con 0x7f5fec1083c0 2024-09-21T04:12:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.502+0000 7f5ff23f8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec112b50 con 0x7f5fec108d10 2024-09-21T04:12:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.502+0000 7f5febfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fec102ad0 0x7f5fec19c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.502+0000 7f5febfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fec102ad0 0x7f5fec19c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54798/0 (socket says 172.21.15.64:54798) 2024-09-21T04:12:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.502+0000 7f5febfff640 1 -- 172.21.15.64:0/2392656892 learned_addr learned my addr 172.21.15.64:0/2392656892 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.502+0000 7f5feb7fe640 1 --2- 172.21.15.64:0/2392656892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fec1083c0 0x7f5fec19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.502+0000 7f5ff0995640 1 --2- 172.21.15.64:0/2392656892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fec108d10 0x7f5fec1a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.502+0000 7f5febfff640 1 -- 172.21.15.64:0/2392656892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fec108d10 msgr2=0x7f5fec1a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.502+0000 7f5febfff640 1 --2- 172.21.15.64:0/2392656892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fec108d10 0x7f5fec1a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.502+0000 7f5febfff640 1 -- 172.21.15.64:0/2392656892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fec1083c0 msgr2=0x7f5fec19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:20.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.502+0000 7f5febfff640 1 --2- 172.21.15.64:0/2392656892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fec1083c0 0x7f5fec19cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.502+0000 7f5febfff640 1 -- 172.21.15.64:0/2392656892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fec1a25c0 con 0x7f5fec102ad0 2024-09-21T04:12:20.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.503+0000 7f5febfff640 1 --2- 172.21.15.64:0/2392656892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fec102ad0 0x7f5fec19c650 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00d950 tx=0x7f5fdc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:20.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.503+0000 7f5fe97fa640 1 -- 172.21.15.64:0/2392656892 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc017070 con 0x7f5fec102ad0 2024-09-21T04:12:20.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.503+0000 7f5ff23f8640 1 -- 172.21.15.64:0/2392656892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fec1a28b0 con 0x7f5fec102ad0 2024-09-21T04:12:20.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.503+0000 7f5fe97fa640 1 -- 172.21.15.64:0/2392656892 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fdc009c30 con 0x7f5fec102ad0 2024-09-21T04:12:20.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.503+0000 7f5fe97fa640 1 -- 172.21.15.64:0/2392656892 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc00f620 con 0x7f5fec102ad0 2024-09-21T04:12:20.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.503+0000 7f5ff23f8640 1 -- 172.21.15.64:0/2392656892 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fec1a9010 con 0x7f5fec102ad0 2024-09-21T04:12:20.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.506+0000 7f5ff896c740 1 Processor -- start 2024-09-21T04:12:20.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.506+0000 7f5ff896c740 1 -- start start 2024-09-21T04:12:20.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.506+0000 7f5ff896c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a06992f070 0x55a069cd66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:20.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.507+0000 7f5ff896c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0699061b0 0x55a069cd6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:20.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.507+0000 7f5ff896c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a06990e390 0x55a069cd7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:20.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.507+0000 7f5ff896c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a069c4cea0 con 0x55a06990e390 2024-09-21T04:12:20.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.507+0000 7f5ff896c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a069c3c2d0 con 0x55a0699061b0 2024-09-21T04:12:20.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.507+0000 7f5ff896c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a069c400e0 con 0x55a06992f070 2024-09-21T04:12:20.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.507+0000 7f5fe97fa640 1 -- 172.21.15.64:0/2392656892 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f5fdc010460 con 0x7f5fec102ad0 2024-09-21T04:12:20.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.507+0000 7f5feb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a06990e390 0x55a069cd7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:20.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.507+0000 7f5feb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a06990e390 0x55a069cd7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54802/0 (socket says 172.21.15.64:54802) 2024-09-21T04:12:20.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.507+0000 7f5feb7fe640 1 -- 172.21.15.64:0/2300991587 learned_addr learned my addr 172.21.15.64:0/2300991587 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:20.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.507+0000 7f5ff0995640 1 --2- 172.21.15.64:0/2300991587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a06992f070 0x55a069cd66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:20.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.507+0000 7f5febfff640 1 --2- 172.21.15.64:0/2300991587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0699061b0 0x55a069cd6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:20.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.507+0000 7f5feb7fe640 1 -- 172.21.15.64:0/2300991587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a06992f070 msgr2=0x55a069cd66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:20.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.507+0000 7f5feb7fe640 1 --2- 172.21.15.64:0/2300991587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a06992f070 0x55a069cd66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.507+0000 7f5feb7fe640 1 -- 172.21.15.64:0/2300991587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0699061b0 msgr2=0x55a069cd6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:20.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.507+0000 7f5feb7fe640 1 --2- 172.21.15.64:0/2300991587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0699061b0 0x55a069cd6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.507+0000 7f5feb7fe640 1 -- 172.21.15.64:0/2300991587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a06999a650 con 0x55a06990e390 2024-09-21T04:12:20.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.507+0000 7f5febfff640 1 --2- 172.21.15.64:0/2300991587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0699061b0 0x55a069cd6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:12:20.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.507+0000 7f5feb7fe640 1 --2- 172.21.15.64:0/2300991587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a06990e390 0x55a069cd7120 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000bc30 tx=0x7f5fe00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3531cff75dd27de8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:20.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.508+0000 7f5fd27fc640 1 -- 172.21.15.64:0/2300991587 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe001c480 con 0x55a06990e390 2024-09-21T04:12:20.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.508+0000 7f5fd27fc640 1 -- 172.21.15.64:0/2300991587 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fe001cad0 con 0x55a06990e390 2024-09-21T04:12:20.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.508+0000 7f5fd27fc640 1 -- 172.21.15.64:0/2300991587 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0011b20 con 0x55a06990e390 2024-09-21T04:12:20.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.508+0000 7f5fe97fa640 1 --2- 172.21.15.64:0/2392656892 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5fd40b6680 0x7f5fd40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:20.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.508+0000 7f5ff0995640 1 --2- 172.21.15.64:0/2392656892 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5fd40b6680 0x7f5fd40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:20.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.508+0000 7f5fe97fa640 1 -- 172.21.15.64:0/2392656892 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7f5fdc0e7340 con 0x7f5fec102ad0 2024-09-21T04:12:20.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.509+0000 7f5ff896c740 1 -- 172.21.15.64:0/2300991587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a06990e390 msgr2=0x55a069cd7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:20.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.509+0000 7f5ff0995640 1 --2- 172.21.15.64:0/2392656892 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5fd40b6680 0x7f5fd40b8b90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5fe402f500 tx=0x7f5fe402f370 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:20.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.509+0000 7f5ff896c740 1 --2- 172.21.15.64:0/2300991587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a06990e390 0x55a069cd7120 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000bc30 tx=0x7f5fe00107a0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.509+0000 7f5ff896c740 1 -- 172.21.15.64:0/2300991587 shutdown_connections 2024-09-21T04:12:20.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.509+0000 7f5ff896c740 1 --2- 172.21.15.64:0/2300991587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a06990e390 0x55a069cd7120 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.509+0000 7f5ff896c740 1 --2- 172.21.15.64:0/2300991587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0699061b0 0x55a069cd6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.509+0000 7f5ff896c740 1 --2- 172.21.15.64:0/2300991587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a06992f070 0x55a069cd66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.509+0000 7f5ff896c740 1 -- 172.21.15.64:0/2300991587 >> 172.21.15.64:0/2300991587 conn(0x55a069923460 msgr2=0x55a069cd4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:20.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.509+0000 7f5ff896c740 1 -- 172.21.15.64:0/2300991587 shutdown_connections 2024-09-21T04:12:20.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.509+0000 7f5ff896c740 1 -- 172.21.15.64:0/2300991587 wait complete. 2024-09-21T04:12:20.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.509+0000 7f5ff896c740 1 -- start start 2024-09-21T04:12:20.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.510+0000 7f5ff896c740 1 Processor -- start 2024-09-21T04:12:20.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.510+0000 7f5ff896c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0699061b0 0x55a069ce1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:20.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.510+0000 7f5ff896c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a06990e390 0x55a069ce22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:20.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.510+0000 7f5ff896c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a06992f070 0x55a069cee170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:20.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.510+0000 7f5ff896c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a069a9d700 con 0x55a0699061b0 2024-09-21T04:12:20.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.510+0000 7f5ff896c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a069bd7160 con 0x55a06992f070 2024-09-21T04:12:20.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.510+0000 7f5ff896c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a069ce4e80 con 0x55a06990e390 2024-09-21T04:12:20.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.510+0000 7f5ff896c740 10 client.0 _ll_register_callbacks cb 0x55a069bf09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:12:20.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.510+0000 7f5febfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0699061b0 0x55a069ce1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:20.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.510+0000 7f5febfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0699061b0 0x55a069ce1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54808/0 (socket says 172.21.15.64:54808) 2024-09-21T04:12:20.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.510+0000 7f5febfff640 1 -- 172.21.15.64:0/1899924577 learned_addr learned my addr 172.21.15.64:0/1899924577 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:20.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.510+0000 7f5ff0995640 1 --2- 172.21.15.64:0/1899924577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a06992f070 0x55a069cee170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:20.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.510+0000 7f5feb7fe640 1 --2- 172.21.15.64:0/1899924577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a06990e390 0x55a069ce22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:20.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.510+0000 7f5febfff640 1 -- 172.21.15.64:0/1899924577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a06990e390 msgr2=0x55a069ce22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:20.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.510+0000 7f5febfff640 1 --2- 172.21.15.64:0/1899924577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a06990e390 0x55a069ce22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.510+0000 7f5febfff640 1 -- 172.21.15.64:0/1899924577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a06992f070 msgr2=0x55a069cee170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:20.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.510+0000 7f5febfff640 1 --2- 172.21.15.64:0/1899924577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a06992f070 0x55a069cee170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.510+0000 7f5febfff640 1 -- 172.21.15.64:0/1899924577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a069c04c90 con 0x55a0699061b0 2024-09-21T04:12:20.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.510+0000 7f5feb7fe640 1 --2- 172.21.15.64:0/1899924577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a06990e390 0x55a069ce22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:12:20.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.511+0000 7f5febfff640 1 --2- 172.21.15.64:0/1899924577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0699061b0 0x55a069ce1da0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc000c00 tx=0x7f5fdc00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c9783f5cc7b25a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:20.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.511+0000 7f5fcb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:12:20.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.511+0000 7f5fcb7fe640 1 -- 172.21.15.64:0/1899924577 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc020070 con 0x55a0699061b0 2024-09-21T04:12:20.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.511+0000 7f5fcb7fe640 1 -- 172.21.15.64:0/1899924577 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7f5fdc0a8b30 con 0x55a0699061b0 2024-09-21T04:12:20.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.511+0000 7f5ff896c740 1 -- 172.21.15.64:0/1899924577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fe001cad0 con 0x55a0699061b0 2024-09-21T04:12:20.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.511+0000 7f5fcb7fe640 1 -- 172.21.15.64:0/1899924577 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5fdc0acb30 con 0x55a0699061b0 2024-09-21T04:12:20.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.511+0000 7f5ff896c740 10 client.17205 fetch_fsmap learned FSMap version 40 2024-09-21T04:12:20.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.511+0000 7f5ff896c740 1 -- 172.21.15.64:0/1899924577 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a069cf0bf0 con 0x55a0699061b0 2024-09-21T04:12:20.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.511+0000 7f5fcb7fe640 1 -- 172.21.15.64:0/1899924577 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fe001cad0 con 0x55a0699061b0 2024-09-21T04:12:20.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.511+0000 7f5fcb7fe640 1 -- 172.21.15.64:0/1899924577 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f5fdc0a8d30 con 0x55a0699061b0 2024-09-21T04:12:20.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.512+0000 7f5ff896c740 10 client.17205 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:12:20.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.512+0000 7f5ff896c740 10 client.17205 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:12:20.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.512+0000 7f5ff896c740 20 client.17205 populate_metadata read hostname 'smithi064' 2024-09-21T04:12:20.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.512+0000 7f5ff896c740 1 --2- 172.21.15.64:0/1899924577 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55a0698ed9f0 0x55a069cf4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:20.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.512+0000 7f5ff896c740 4 client.17205 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:12:20.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.512+0000 7f5ff896c740 1 -- 172.21.15.64:0/1899924577 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a069c4edf0 con 0x55a0698ed9f0 2024-09-21T04:12:20.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.512+0000 7f5feb7fe640 1 --2- 172.21.15.64:0/1899924577 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55a0698ed9f0 0x55a069cf4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:20.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.512+0000 7f5fcb7fe640 10 client.17205 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:12:20.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.512+0000 7f5feb7fe640 1 --2- 172.21.15.64:0/1899924577 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55a0698ed9f0 0x55a069cf4120 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=11c970aab8e26654 server_cookie=22fe08ed1c0082b3 in_seq=0 out_seq=0 2024-09-21T04:12:20.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.513+0000 7f5fcb7fe640 1 -- 172.21.15.64:0/1899924577 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a069c4edf0 con 0x55a0698ed9f0 2024-09-21T04:12:20.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.513+0000 7f5fcb7fe640 10 client.17205 handle_command_reply: tid=0 2024-09-21T04:12:20.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.513+0000 7f5ff896c740 1 client.17205 shutdown 2024-09-21T04:12:20.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.514+0000 7f5ff896c740 1 -- 172.21.15.64:0/1899924577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0699061b0 msgr2=0x55a069ce1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:20.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.514+0000 7f5ff896c740 1 --2- 172.21.15.64:0/1899924577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0699061b0 0x55a069ce1da0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc000c00 tx=0x7f5fdc00bf60 comp rx=0 tx=0).stop 2024-09-21T04:12:20.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.514+0000 7f5ff896c740 1 -- 172.21.15.64:0/1899924577 shutdown_connections 2024-09-21T04:12:20.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.514+0000 7f5ff896c740 1 --2- 172.21.15.64:0/1899924577 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55a0698ed9f0 0x55a069cf4120 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.514+0000 7f5ff896c740 1 --2- 172.21.15.64:0/1899924577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a06992f070 0x55a069cee170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.514+0000 7f5ff896c740 1 --2- 172.21.15.64:0/1899924577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a06990e390 0x55a069ce22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.514+0000 7f5ff896c740 1 --2- 172.21.15.64:0/1899924577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a0699061b0 0x55a069ce1da0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.514+0000 7f5ff896c740 1 -- 172.21.15.64:0/1899924577 >> 172.21.15.64:0/1899924577 conn(0x55a069923460 msgr2=0x55a069cd9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:20.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.514+0000 7f5fcb7fe640 0 client.17205 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:12:20.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.514+0000 7f5ff896c740 1 -- 172.21.15.64:0/1899924577 shutdown_connections 2024-09-21T04:12:20.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.514+0000 7f5ff896c740 1 -- 172.21.15.64:0/1899924577 wait complete. 2024-09-21T04:12:20.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.514+0000 7f5ff896c740 20 client.17205 trim_cache size 0 max 16384 2024-09-21T04:12:20.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.527+0000 7f5ff896c740 1 Processor -- start 2024-09-21T04:12:20.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.527+0000 7f5ff896c740 1 -- start start 2024-09-21T04:12:20.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.527+0000 7f5ff896c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0699061b0 0x55a069ce39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:20.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.527+0000 7f5ff896c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a06990e390 0x55a069cd3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:20.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.527+0000 7f5ff896c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a06992f070 0x55a069cd4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:20.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.527+0000 7f5ff896c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec1b4410 con 0x55a06990e390 2024-09-21T04:12:20.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.527+0000 7f5ff896c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec1b1c20 con 0x55a06992f070 2024-09-21T04:12:20.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.527+0000 7f5ff896c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec1afa90 con 0x55a0699061b0 2024-09-21T04:12:20.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.527+0000 7f5feb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a06990e390 0x55a069cd3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:20.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.527+0000 7f5feb7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a06990e390 0x55a069cd3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54810/0 (socket says 172.21.15.64:54810) 2024-09-21T04:12:20.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.527+0000 7f5feb7fe640 1 -- 172.21.15.64:0/1228087292 learned_addr learned my addr 172.21.15.64:0/1228087292 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:20.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.528+0000 7f5febfff640 1 --2- 172.21.15.64:0/1228087292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0699061b0 0x55a069ce39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:20.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.528+0000 7f5ff0995640 1 --2- 172.21.15.64:0/1228087292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a06992f070 0x55a069cd4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:20.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.528+0000 7f5feb7fe640 1 -- 172.21.15.64:0/1228087292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0699061b0 msgr2=0x55a069ce39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:20.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.528+0000 7f5feb7fe640 1 --2- 172.21.15.64:0/1228087292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0699061b0 0x55a069ce39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.528+0000 7f5feb7fe640 1 -- 172.21.15.64:0/1228087292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a06992f070 msgr2=0x55a069cd4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:20.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.528+0000 7f5feb7fe640 1 --2- 172.21.15.64:0/1228087292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a06992f070 0x55a069cd4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.528+0000 7f5feb7fe640 1 -- 172.21.15.64:0/1228087292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a069cd8990 con 0x55a06990e390 2024-09-21T04:12:20.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.528+0000 7f5febfff640 1 --2- 172.21.15.64:0/1228087292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0699061b0 0x55a069ce39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:12:20.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.528+0000 7f5ff0995640 1 --2- 172.21.15.64:0/1228087292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a06992f070 0x55a069cd4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:12:20.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.528+0000 7f5feb7fe640 1 --2- 172.21.15.64:0/1228087292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a06990e390 0x55a069cd3bc0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000bc00 tx=0x7f5fe0013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93c15730a5b7b8f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:20.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.528+0000 7f5fd1ffb640 1 -- 172.21.15.64:0/1228087292 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0005040 con 0x55a06990e390 2024-09-21T04:12:20.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.528+0000 7f5fd1ffb640 1 -- 172.21.15.64:0/1228087292 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fe00035f0 con 0x55a06990e390 2024-09-21T04:12:20.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.528+0000 7f5fd1ffb640 1 -- 172.21.15.64:0/1228087292 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0011840 con 0x55a06990e390 2024-09-21T04:12:20.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.529+0000 7f5ff896c740 1 -- 172.21.15.64:0/1228087292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a06990e390 msgr2=0x55a069cd3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:20.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.529+0000 7f5ff896c740 1 --2- 172.21.15.64:0/1228087292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a06990e390 0x55a069cd3bc0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000bc00 tx=0x7f5fe0013040 comp rx=0 tx=0).stop 2024-09-21T04:12:20.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.529+0000 7f5ff896c740 1 -- 172.21.15.64:0/1228087292 shutdown_connections 2024-09-21T04:12:20.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.529+0000 7f5ff896c740 1 --2- 172.21.15.64:0/1228087292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a06992f070 0x55a069cd4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.529+0000 7f5ff896c740 1 --2- 172.21.15.64:0/1228087292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a06990e390 0x55a069cd3bc0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.529+0000 7f5ff896c740 1 --2- 172.21.15.64:0/1228087292 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0699061b0 0x55a069ce39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.529+0000 7f5ff896c740 1 -- 172.21.15.64:0/1228087292 >> 172.21.15.64:0/1228087292 conn(0x55a069923460 msgr2=0x55a069ce32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:20.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.529+0000 7f5ff896c740 1 -- 172.21.15.64:0/1228087292 shutdown_connections 2024-09-21T04:12:20.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.529+0000 7f5ff896c740 1 -- 172.21.15.64:0/1228087292 wait complete. 2024-09-21T04:12:20.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.529+0000 7f5ff896c740 1 -- start start 2024-09-21T04:12:20.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.530+0000 7f5ff896c740 1 Processor -- start 2024-09-21T04:12:20.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.530+0000 7f5ff896c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0699061b0 0x55a069ceaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:20.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.530+0000 7f5ff896c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a06990e390 0x55a069ce6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:20.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.530+0000 7f5ff896c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a06992f070 0x55a069ce6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:20.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.530+0000 7f5ff896c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec1b00e0 con 0x55a06992f070 2024-09-21T04:12:20.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.530+0000 7f5ff896c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec1b1550 con 0x55a06990e390 2024-09-21T04:12:20.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.530+0000 7f5ff896c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec1b07b0 con 0x55a0699061b0 2024-09-21T04:12:20.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.530+0000 7f5ff896c740 10 client.0 _ll_register_callbacks cb 0x7f5fdc00df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:12:20.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.530+0000 7f5ff0995640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a06992f070 0x55a069ce6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:20.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.530+0000 7f5ff0995640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a06992f070 0x55a069ce6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54822/0 (socket says 172.21.15.64:54822) 2024-09-21T04:12:20.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.530+0000 7f5ff0995640 1 -- 172.21.15.64:0/857345546 learned_addr learned my addr 172.21.15.64:0/857345546 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:20.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.530+0000 7f5feb7fe640 1 --2- 172.21.15.64:0/857345546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a06990e390 0x55a069ce6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:20.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.530+0000 7f5febfff640 1 --2- 172.21.15.64:0/857345546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0699061b0 0x55a069ceaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:20.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.530+0000 7f5ff0995640 1 -- 172.21.15.64:0/857345546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0699061b0 msgr2=0x55a069ceaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:20.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.530+0000 7f5ff0995640 1 --2- 172.21.15.64:0/857345546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0699061b0 0x55a069ceaaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.530+0000 7f5ff0995640 1 -- 172.21.15.64:0/857345546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a06990e390 msgr2=0x55a069ce6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.530+0000 7f5ff0995640 1 --2- 172.21.15.64:0/857345546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a06990e390 0x55a069ce6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.530+0000 7f5ff0995640 1 -- 172.21.15.64:0/857345546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a069d5db20 con 0x55a06992f070 2024-09-21T04:12:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.530+0000 7f5febfff640 1 --2- 172.21.15.64:0/857345546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0699061b0 0x55a069ceaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:12:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.531+0000 7f5ff0995640 1 --2- 172.21.15.64:0/857345546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a06992f070 0x55a069ce6710 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4035000 tx=0x7f5fe402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805c6494e22f5fa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.531+0000 7f5fcb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:12:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.531+0000 7f5fcb7fe640 1 -- 172.21.15.64:0/857345546 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4058060 con 0x55a06992f070 2024-09-21T04:12:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.531+0000 7f5ff896c740 1 -- 172.21.15.64:0/857345546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fe00035f0 con 0x55a06992f070 2024-09-21T04:12:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.531+0000 7f5fcb7fe640 1 -- 172.21.15.64:0/857345546 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7f5fe4004250 con 0x55a06992f070 2024-09-21T04:12:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.531+0000 7f5fcb7fe640 1 -- 172.21.15.64:0/857345546 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5fe4031e60 con 0x55a06992f070 2024-09-21T04:12:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.531+0000 7f5ff896c740 10 client.17211 fetch_fsmap learned FSMap version 40 2024-09-21T04:12:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.531+0000 7f5ff896c740 1 -- 172.21.15.64:0/857345546 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a069d5ec10 con 0x55a06992f070 2024-09-21T04:12:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.531+0000 7f5fcb7fe640 1 -- 172.21.15.64:0/857345546 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fe00035f0 con 0x55a06992f070 2024-09-21T04:12:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.531+0000 7f5fcb7fe640 1 -- 172.21.15.64:0/857345546 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f5fe4052070 con 0x55a06992f070 2024-09-21T04:12:20.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.531+0000 7f5ff896c740 10 client.17211 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:12:20.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.531+0000 7f5ff896c740 10 client.17211 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:12:20.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.532+0000 7f5ff896c740 20 client.17211 populate_metadata read hostname 'smithi064' 2024-09-21T04:12:20.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.532+0000 7f5ff896c740 1 --2- 172.21.15.64:0/857345546 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55a0698c2da0 0x55a069d61490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:20.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.532+0000 7f5ff896c740 4 client.17211 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:12:20.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.532+0000 7f5ff896c740 1 -- 172.21.15.64:0/857345546 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a069c4a8a0 con 0x55a0698c2da0 2024-09-21T04:12:20.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.532+0000 7f5febfff640 1 --2- 172.21.15.64:0/857345546 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55a0698c2da0 0x55a069d61490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:20.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.532+0000 7f5fcb7fe640 10 client.17211 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:12:20.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.532+0000 7f5febfff640 1 --2- 172.21.15.64:0/857345546 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55a0698c2da0 0x55a069d61490 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=12b83780ada7217f server_cookie=93e836dd953597c9 in_seq=0 out_seq=0 2024-09-21T04:12:20.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.532+0000 7f5fcb7fe640 1 -- 172.21.15.64:0/857345546 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a069c4a8a0 con 0x55a0698c2da0 2024-09-21T04:12:20.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.533+0000 7f5fcb7fe640 10 client.17211 handle_command_reply: tid=0 2024-09-21T04:12:20.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.533+0000 7f5ff896c740 1 client.17211 shutdown 2024-09-21T04:12:20.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.533+0000 7f5ff896c740 1 -- 172.21.15.64:0/857345546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a06992f070 msgr2=0x55a069ce6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:20.539 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.533+0000 7f5ff896c740 1 --2- 172.21.15.64:0/857345546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a06992f070 0x55a069ce6710 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4035000 tx=0x7f5fe402a8c0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.533+0000 7f5ff896c740 1 -- 172.21.15.64:0/857345546 shutdown_connections 2024-09-21T04:12:20.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.533+0000 7f5ff896c740 1 --2- 172.21.15.64:0/857345546 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55a0698c2da0 0x55a069d61490 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.533+0000 7f5ff896c740 1 --2- 172.21.15.64:0/857345546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a06992f070 0x55a069ce6710 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.533+0000 7f5ff896c740 1 --2- 172.21.15.64:0/857345546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a06990e390 0x55a069ce6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.533+0000 7f5fcb7fe640 0 client.17211 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:12:20.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.533+0000 7f5ff896c740 1 --2- 172.21.15.64:0/857345546 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a0699061b0 0x55a069ceaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.533+0000 7f5ff896c740 1 -- 172.21.15.64:0/857345546 >> 172.21.15.64:0/857345546 conn(0x55a069923460 msgr2=0x55a069cd75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:20.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.533+0000 7f5ff896c740 1 -- 172.21.15.64:0/857345546 shutdown_connections 2024-09-21T04:12:20.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.533+0000 7f5ff896c740 1 -- 172.21.15.64:0/857345546 wait complete. 2024-09-21T04:12:20.540 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.533+0000 7f5ff896c740 20 client.17211 trim_cache size 0 max 16384 2024-09-21T04:12:20.540 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:12:20.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.533+0000 7f5ff23f8640 1 -- 172.21.15.64:0/2392656892 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5fd40b6680 msgr2=0x7f5fd40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:20.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.533+0000 7f5ff23f8640 1 --2- 172.21.15.64:0/2392656892 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5fd40b6680 0x7f5fd40b8b90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5fe402f500 tx=0x7f5fe402f370 comp rx=0 tx=0).stop 2024-09-21T04:12:20.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.533+0000 7f5ff23f8640 1 -- 172.21.15.64:0/2392656892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fec102ad0 msgr2=0x7f5fec19c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:20.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.533+0000 7f5ff23f8640 1 --2- 172.21.15.64:0/2392656892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fec102ad0 0x7f5fec19c650 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00d950 tx=0x7f5fdc00de10 comp rx=0 tx=0).stop 2024-09-21T04:12:20.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.534+0000 7f5ff23f8640 1 -- 172.21.15.64:0/2392656892 shutdown_connections 2024-09-21T04:12:20.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.534+0000 7f5ff23f8640 1 --2- 172.21.15.64:0/2392656892 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f5fd40b6680 0x7f5fd40b8b90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.534+0000 7f5ff23f8640 1 --2- 172.21.15.64:0/2392656892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5fec108d10 0x7f5fec1a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.534+0000 7f5ff23f8640 1 --2- 172.21.15.64:0/2392656892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fec1083c0 0x7f5fec19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.534+0000 7f5ff23f8640 1 --2- 172.21.15.64:0/2392656892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fec102ad0 0x7f5fec19c650 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.534+0000 7f5ff23f8640 1 -- 172.21.15.64:0/2392656892 >> 172.21.15.64:0/2392656892 conn(0x7f5fec0796d0 msgr2=0x7f5fec06bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.534+0000 7f5ff23f8640 1 -- 172.21.15.64:0/2392656892 shutdown_connections 2024-09-21T04:12:20.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:20.534+0000 7f5ff23f8640 1 -- 172.21.15.64:0/2392656892 wait complete. 2024-09-21T04:12:20.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:20 smithi119 ceph-mon[34811]: pgmap v576: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 921 B/s wr, 6 op/s 2024-09-21T04:12:20.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:20 smithi064 ceph-mon[31180]: pgmap v576: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 921 B/s wr, 6 op/s 2024-09-21T04:12:20.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:20 smithi131 ceph-mon[35193]: pgmap v576: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 921 B/s wr, 6 op/s 2024-09-21T04:12:21.283 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T04:12:21.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:21 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:12:21.550 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:12:21.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:21 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:12:21.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:21 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:12:22.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.049+0000 7fa60cfab640 1 -- 172.21.15.64:0/3300621265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa608102ae0 msgr2=0x7fa608102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:22.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.049+0000 7fa60cfab640 1 --2- 172.21.15.64:0/3300621265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa608102ae0 0x7fa608102ef0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8009900 tx=0x7fa5f802a410 comp rx=0 tx=0).stop 2024-09-21T04:12:22.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.049+0000 7fa60cfab640 1 -- 172.21.15.64:0/3300621265 shutdown_connections 2024-09-21T04:12:22.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.049+0000 7fa60cfab640 1 --2- 172.21.15.64:0/3300621265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa608108d20 0x7fa60810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.049+0000 7fa60cfab640 1 --2- 172.21.15.64:0/3300621265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6081083d0 0x7fa6081087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.049+0000 7fa60cfab640 1 --2- 172.21.15.64:0/3300621265 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa608102ae0 0x7fa608102ef0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.049+0000 7fa60cfab640 1 -- 172.21.15.64:0/3300621265 >> 172.21.15.64:0/3300621265 conn(0x7fa608079760 msgr2=0x7fa608079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:22.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.049+0000 7fa60cfab640 1 -- 172.21.15.64:0/3300621265 shutdown_connections 2024-09-21T04:12:22.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.049+0000 7fa60cfab640 1 -- 172.21.15.64:0/3300621265 wait complete. 2024-09-21T04:12:22.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.050+0000 7fa60cfab640 1 Processor -- start 2024-09-21T04:12:22.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.050+0000 7fa60cfab640 1 -- start start 2024-09-21T04:12:22.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.050+0000 7fa60cfab640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa608102ae0 0x7fa608199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:22.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.050+0000 7fa60cfab640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6081083d0 0x7fa608199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:22.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.050+0000 7fa60cfab640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa608108d20 0x7fa60819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:22.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.050+0000 7fa60cfab640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa608112990 con 0x7fa608108d20 2024-09-21T04:12:22.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.050+0000 7fa60cfab640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa608112810 con 0x7fa608102ae0 2024-09-21T04:12:22.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.050+0000 7fa60cfab640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa608112b10 con 0x7fa6081083d0 2024-09-21T04:12:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.050+0000 7fa606d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa608108d20 0x7fa60819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.050+0000 7fa606d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa608108d20 0x7fa60819eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60210/0 (socket says 172.21.15.64:60210) 2024-09-21T04:12:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.050+0000 7fa606d9d640 1 -- 172.21.15.64:0/2399355964 learned_addr learned my addr 172.21.15.64:0/2399355964 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.051+0000 7fa5fffff640 1 --2- 172.21.15.64:0/2399355964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6081083d0 0x7fa608199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.051+0000 7fa60659c640 1 --2- 172.21.15.64:0/2399355964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa608102ae0 0x7fa608199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.051+0000 7fa606d9d640 1 -- 172.21.15.64:0/2399355964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6081083d0 msgr2=0x7fa608199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.051+0000 7fa606d9d640 1 --2- 172.21.15.64:0/2399355964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6081083d0 0x7fa608199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.051+0000 7fa606d9d640 1 -- 172.21.15.64:0/2399355964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa608102ae0 msgr2=0x7fa608199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.051+0000 7fa606d9d640 1 --2- 172.21.15.64:0/2399355964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa608102ae0 0x7fa608199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.051+0000 7fa606d9d640 1 -- 172.21.15.64:0/2399355964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa60819f2d0 con 0x7fa608108d20 2024-09-21T04:12:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.051+0000 7fa606d9d640 1 --2- 172.21.15.64:0/2399355964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa608108d20 0x7fa60819eba0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa60000eaa0 tx=0x7fa60000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.051+0000 7fa5ff7fe640 1 -- 172.21.15.64:0/2399355964 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60000cdb0 con 0x7fa608108d20 2024-09-21T04:12:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.051+0000 7fa5ff7fe640 1 -- 172.21.15.64:0/2399355964 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa600005020 con 0x7fa608108d20 2024-09-21T04:12:22.059 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.051+0000 7fa60cfab640 1 -- 172.21.15.64:0/2399355964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa60819f5c0 con 0x7fa608108d20 2024-09-21T04:12:22.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.051+0000 7fa5ff7fe640 1 -- 172.21.15.64:0/2399355964 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6000116d0 con 0x7fa608108d20 2024-09-21T04:12:22.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.051+0000 7fa60cfab640 1 -- 172.21.15.64:0/2399355964 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6081a5d20 con 0x7fa608108d20 2024-09-21T04:12:22.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.054+0000 7fa61351f740 1 Processor -- start 2024-09-21T04:12:22.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.054+0000 7fa5ff7fe640 1 -- 172.21.15.64:0/2399355964 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fa6000051d0 con 0x7fa608108d20 2024-09-21T04:12:22.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.054+0000 7fa61351f740 1 -- start start 2024-09-21T04:12:22.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.055+0000 7fa61351f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d7e070 0x5621281252c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:22.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.055+0000 7fa61351f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562127d551b0 0x562128125800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:22.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.055+0000 7fa61351f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562127d5d390 0x562128125d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:22.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.055+0000 7fa61351f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56212809bea0 con 0x562127d7e070 2024-09-21T04:12:22.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.055+0000 7fa61351f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56212808b2d0 con 0x562127d5d390 2024-09-21T04:12:22.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.055+0000 7fa61351f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56212808f0e0 con 0x562127d551b0 2024-09-21T04:12:22.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.055+0000 7fa5fffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d7e070 0x5621281252c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:22.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.055+0000 7fa5fffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d7e070 0x5621281252c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60216/0 (socket says 172.21.15.64:60216) 2024-09-21T04:12:22.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.055+0000 7fa5fffff640 1 -- 172.21.15.64:0/3794041378 learned_addr learned my addr 172.21.15.64:0/3794041378 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:22.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.055+0000 7fa60659c640 1 --2- 172.21.15.64:0/3794041378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562127d551b0 0x562128125800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:22.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.055+0000 7fa5fffff640 1 --2- 172.21.15.64:0/3794041378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562127d5d390 0x562128125d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:22.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.055+0000 7fa5ff7fe640 1 --2- 172.21.15.64:0/2399355964 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa5f40b6680 0x7fa5f40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:22.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.055+0000 7fa5fffff640 1 -- 172.21.15.64:0/3794041378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562127d551b0 msgr2=0x562128125800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:22.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.055+0000 7fa5fffff640 1 --2- 172.21.15.64:0/3794041378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562127d551b0 0x562128125800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.055+0000 7fa5fffff640 1 -- 172.21.15.64:0/3794041378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562127d5d390 msgr2=0x562128125d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:22.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.055+0000 7fa5fffff640 1 --2- 172.21.15.64:0/3794041378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562127d5d390 0x562128125d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.056+0000 7fa5fffff640 1 -- 172.21.15.64:0/3794041378 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562127de9650 con 0x562127d7e070 2024-09-21T04:12:22.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.056+0000 7fa606d9d640 1 --2- 172.21.15.64:0/2399355964 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa5f40b6680 0x7fa5f40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:22.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.056+0000 7fa60659c640 1 --2- 172.21.15.64:0/3794041378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562127d551b0 0x562128125800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:12:22.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.056+0000 7fa5ff7fe640 1 -- 172.21.15.64:0/2399355964 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7fa6000e78f0 con 0x7fa608108d20 2024-09-21T04:12:22.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.056+0000 7fa5fffff640 1 --2- 172.21.15.64:0/3794041378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d7e070 0x5621281252c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000a520 tx=0x7fa5f000a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2868725520eba30 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.056+0000 7fa5fcff9640 1 -- 172.21.15.64:0/3794041378 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0004020 con 0x562127d7e070 2024-09-21T04:12:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.056+0000 7fa5fcff9640 1 -- 172.21.15.64:0/3794041378 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5f00054d0 con 0x562127d7e070 2024-09-21T04:12:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.056+0000 7fa5fcff9640 1 -- 172.21.15.64:0/3794041378 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0004aa0 con 0x562127d7e070 2024-09-21T04:12:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.057+0000 7fa61351f740 1 -- 172.21.15.64:0/3794041378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d7e070 msgr2=0x5621281252c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.057+0000 7fa61351f740 1 --2- 172.21.15.64:0/3794041378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d7e070 0x5621281252c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000a520 tx=0x7fa5f000a690 comp rx=0 tx=0).stop 2024-09-21T04:12:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.057+0000 7fa606d9d640 1 --2- 172.21.15.64:0/2399355964 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa5f40b6680 0x7fa5f40b8b90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa6000e5090 tx=0x7fa60007a390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.057+0000 7fa61351f740 1 -- 172.21.15.64:0/3794041378 shutdown_connections 2024-09-21T04:12:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.057+0000 7fa61351f740 1 --2- 172.21.15.64:0/3794041378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562127d5d390 0x562128125d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.057+0000 7fa61351f740 1 --2- 172.21.15.64:0/3794041378 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562127d551b0 0x562128125800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.057+0000 7fa61351f740 1 --2- 172.21.15.64:0/3794041378 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d7e070 0x5621281252c0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.057+0000 7fa61351f740 1 -- 172.21.15.64:0/3794041378 >> 172.21.15.64:0/3794041378 conn(0x562127d72460 msgr2=0x5621281231b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.057+0000 7fa61351f740 1 -- 172.21.15.64:0/3794041378 shutdown_connections 2024-09-21T04:12:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.057+0000 7fa61351f740 1 -- 172.21.15.64:0/3794041378 wait complete. 2024-09-21T04:12:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.058+0000 7fa61351f740 1 -- start start 2024-09-21T04:12:22.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.058+0000 7fa61351f740 1 Processor -- start 2024-09-21T04:12:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.058+0000 7fa61351f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562127d551b0 0x562128132ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.058+0000 7fa61351f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d5d390 0x562128133020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.058+0000 7fa61351f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562127d7e070 0x562128133560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.058+0000 7fa61351f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562127eec700 con 0x562127d5d390 2024-09-21T04:12:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.058+0000 7fa61351f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562128026160 con 0x562127d551b0 2024-09-21T04:12:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.058+0000 7fa61351f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562128135ba0 con 0x562127d7e070 2024-09-21T04:12:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.058+0000 7fa61351f740 10 client.0 _ll_register_callbacks cb 0x56212803f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:12:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.058+0000 7fa60659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d5d390 0x562128133020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.058+0000 7fa60659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d5d390 0x562128133020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60228/0 (socket says 172.21.15.64:60228) 2024-09-21T04:12:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.058+0000 7fa60659c640 1 -- 172.21.15.64:0/3913910418 learned_addr learned my addr 172.21.15.64:0/3913910418 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.058+0000 7fa5fffff640 1 --2- 172.21.15.64:0/3913910418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562127d551b0 0x562128132ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.058+0000 7fa5fffff640 1 --2- 172.21.15.64:0/3913910418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562127d7e070 0x562128133560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.059+0000 7fa60659c640 1 -- 172.21.15.64:0/3913910418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562127d7e070 msgr2=0x562128133560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:22.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.059+0000 7fa60659c640 1 --2- 172.21.15.64:0/3913910418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562127d7e070 0x562128133560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.059+0000 7fa60659c640 1 -- 172.21.15.64:0/3913910418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562127d551b0 msgr2=0x562128132ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:22.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.059+0000 7fa60659c640 1 --2- 172.21.15.64:0/3913910418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562127d551b0 0x562128132ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.059+0000 7fa60659c640 1 -- 172.21.15.64:0/3913910418 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562128053c90 con 0x562127d5d390 2024-09-21T04:12:22.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.059+0000 7fa5fffff640 1 --2- 172.21.15.64:0/3913910418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562127d7e070 0x562128133560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:12:22.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.059+0000 7fa60659c640 1 --2- 172.21.15.64:0/3913910418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d5d390 0x562128133020 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fa5f802a930 tx=0x7fa5f8031830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ad286681c78b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:22.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.059+0000 7fa5edffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:12:22.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.059+0000 7fa5edffb640 1 -- 172.21.15.64:0/3913910418 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8035040 con 0x562127d5d390 2024-09-21T04:12:22.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.059+0000 7fa5edffb640 1 -- 172.21.15.64:0/3913910418 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7fa5f8004240 con 0x562127d5d390 2024-09-21T04:12:22.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.059+0000 7fa61351f740 1 -- 172.21.15.64:0/3913910418 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5f00054d0 con 0x562127d5d390 2024-09-21T04:12:22.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.059+0000 7fa5edffb640 1 -- 172.21.15.64:0/3913910418 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa5f8004440 con 0x562127d5d390 2024-09-21T04:12:22.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.059+0000 7fa61351f740 10 client.17223 fetch_fsmap learned FSMap version 40 2024-09-21T04:12:22.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.059+0000 7fa61351f740 1 -- 172.21.15.64:0/3913910418 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x562128141810 con 0x562127d5d390 2024-09-21T04:12:22.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.059+0000 7fa5edffb640 1 -- 172.21.15.64:0/3913910418 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5f00054d0 con 0x562127d5d390 2024-09-21T04:12:22.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.060+0000 7fa5edffb640 1 -- 172.21.15.64:0/3913910418 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7fa5f8004bc0 con 0x562127d5d390 2024-09-21T04:12:22.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.060+0000 7fa61351f740 10 client.17223 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:12:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.060+0000 7fa61351f740 10 client.17223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:12:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.060+0000 7fa61351f740 20 client.17223 populate_metadata read hostname 'smithi064' 2024-09-21T04:12:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.060+0000 7fa61351f740 1 --2- 172.21.15.64:0/3913910418 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x562127d3c9f0 0x562128144d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.060+0000 7fa61351f740 4 client.17223 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:12:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.060+0000 7fa61351f740 1 -- 172.21.15.64:0/3913910418 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56212809ddf0 con 0x562127d3c9f0 2024-09-21T04:12:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.060+0000 7fa5fffff640 1 --2- 172.21.15.64:0/3913910418 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x562127d3c9f0 0x562128144d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.061+0000 7fa5fffff640 1 --2- 172.21.15.64:0/3913910418 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x562127d3c9f0 0x562128144d70 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c606d3d33c618f9b server_cookie=9f0ac4ce7bd4fddf in_seq=0 out_seq=0 2024-09-21T04:12:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.061+0000 7fa5edffb640 10 client.17223 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:12:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.061+0000 7fa5edffb640 1 -- 172.21.15.64:0/3913910418 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56212809ddf0 con 0x562127d3c9f0 2024-09-21T04:12:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.061+0000 7fa5edffb640 10 client.17223 handle_command_reply: tid=0 2024-09-21T04:12:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.061+0000 7fa61351f740 1 client.17223 shutdown 2024-09-21T04:12:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.061+0000 7fa61351f740 1 -- 172.21.15.64:0/3913910418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d5d390 msgr2=0x562128133020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.061+0000 7fa61351f740 1 --2- 172.21.15.64:0/3913910418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d5d390 0x562128133020 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fa5f802a930 tx=0x7fa5f8031830 comp rx=0 tx=0).stop 2024-09-21T04:12:22.069 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.062+0000 7fa61351f740 1 -- 172.21.15.64:0/3913910418 shutdown_connections 2024-09-21T04:12:22.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.062+0000 7fa61351f740 1 --2- 172.21.15.64:0/3913910418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562127d7e070 0x562128133560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.062+0000 7fa61351f740 1 --2- 172.21.15.64:0/3913910418 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x562127d3c9f0 0x562128144d70 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.062+0000 7fa61351f740 1 --2- 172.21.15.64:0/3913910418 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d5d390 0x562128133020 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.062+0000 7fa61351f740 1 --2- 172.21.15.64:0/3913910418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562127d551b0 0x562128132ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.062+0000 7fa5edffb640 0 client.17223 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:12:22.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.062+0000 7fa61351f740 1 -- 172.21.15.64:0/3913910418 >> 172.21.15.64:0/3913910418 conn(0x562127d72460 msgr2=0x562128121750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:22.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.062+0000 7fa61351f740 1 -- 172.21.15.64:0/3913910418 shutdown_connections 2024-09-21T04:12:22.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.062+0000 7fa61351f740 1 -- 172.21.15.64:0/3913910418 wait complete. 2024-09-21T04:12:22.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.062+0000 7fa61351f740 20 client.17223 trim_cache size 0 max 16384 2024-09-21T04:12:22.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.075+0000 7fa61351f740 1 Processor -- start 2024-09-21T04:12:22.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.075+0000 7fa61351f740 1 -- start start 2024-09-21T04:12:22.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.075+0000 7fa61351f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562127d551b0 0x562128123150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:22.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.075+0000 7fa61351f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d5d390 0x562128134710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:22.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.075+0000 7fa61351f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562127d7e070 0x56212812fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:22.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.075+0000 7fa61351f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa6081b0070 con 0x562127d5d390 2024-09-21T04:12:22.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.075+0000 7fa61351f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa6081ad880 con 0x562127d7e070 2024-09-21T04:12:22.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.075+0000 7fa61351f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa6081ab6f0 con 0x562127d551b0 2024-09-21T04:12:22.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.075+0000 7fa60659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d5d390 0x562128134710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:22.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.075+0000 7fa60659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d5d390 0x562128134710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60240/0 (socket says 172.21.15.64:60240) 2024-09-21T04:12:22.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.075+0000 7fa60659c640 1 -- 172.21.15.64:0/2463588553 learned_addr learned my addr 172.21.15.64:0/2463588553 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:22.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.076+0000 7fa5fffff640 1 --2- 172.21.15.64:0/2463588553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562127d551b0 0x562128123150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:22.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.076+0000 7fa5fffff640 1 --2- 172.21.15.64:0/2463588553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562127d7e070 0x56212812fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:22.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.076+0000 7fa60659c640 1 -- 172.21.15.64:0/2463588553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562127d551b0 msgr2=0x562128123150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:22.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.076+0000 7fa60659c640 1 --2- 172.21.15.64:0/2463588553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562127d551b0 0x562128123150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.076+0000 7fa60659c640 1 -- 172.21.15.64:0/2463588553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562127d7e070 msgr2=0x56212812fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:22.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.076+0000 7fa60659c640 1 --2- 172.21.15.64:0/2463588553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562127d7e070 0x56212812fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.076+0000 7fa60659c640 1 -- 172.21.15.64:0/2463588553 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562128130fe0 con 0x562127d5d390 2024-09-21T04:12:22.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.076+0000 7fa5fffff640 1 --2- 172.21.15.64:0/2463588553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562127d551b0 0x562128123150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:12:22.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.076+0000 7fa60659c640 1 --2- 172.21.15.64:0/2463588553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d5d390 0x562128134710 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8006010 tx=0x7fa5f802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e510deeb58f66e07 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:22.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.076+0000 7fa5effff640 1 -- 172.21.15.64:0/2463588553 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8051070 con 0x562127d5d390 2024-09-21T04:12:22.080 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.076+0000 7fa5effff640 1 -- 172.21.15.64:0/2463588553 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5f8003c60 con 0x562127d5d390 2024-09-21T04:12:22.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.076+0000 7fa5effff640 1 -- 172.21.15.64:0/2463588553 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8035040 con 0x562127d5d390 2024-09-21T04:12:22.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.077+0000 7fa61351f740 1 -- 172.21.15.64:0/2463588553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d5d390 msgr2=0x562128134710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:22.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.077+0000 7fa61351f740 1 --2- 172.21.15.64:0/2463588553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d5d390 0x562128134710 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8006010 tx=0x7fa5f802a410 comp rx=0 tx=0).stop 2024-09-21T04:12:22.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.077+0000 7fa61351f740 1 -- 172.21.15.64:0/2463588553 shutdown_connections 2024-09-21T04:12:22.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.077+0000 7fa61351f740 1 --2- 172.21.15.64:0/2463588553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562127d7e070 0x56212812fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.077+0000 7fa61351f740 1 --2- 172.21.15.64:0/2463588553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d5d390 0x562128134710 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.077+0000 7fa61351f740 1 --2- 172.21.15.64:0/2463588553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562127d551b0 0x562128123150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.077+0000 7fa61351f740 1 -- 172.21.15.64:0/2463588553 >> 172.21.15.64:0/2463588553 conn(0x562127d72460 msgr2=0x562128134000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:22.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.077+0000 7fa61351f740 1 -- 172.21.15.64:0/2463588553 shutdown_connections 2024-09-21T04:12:22.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.077+0000 7fa61351f740 1 -- 172.21.15.64:0/2463588553 wait complete. 2024-09-21T04:12:22.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.077+0000 7fa61351f740 1 -- start start 2024-09-21T04:12:22.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.077+0000 7fa61351f740 1 Processor -- start 2024-09-21T04:12:22.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.078+0000 7fa61351f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d551b0 0x562128137000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:22.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.078+0000 7fa61351f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562127d5d390 0x562128137540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:22.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.078+0000 7fa61351f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562127d7e070 0x562128137a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:22.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.078+0000 7fa61351f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa6081abd40 con 0x562127d551b0 2024-09-21T04:12:22.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.078+0000 7fa61351f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa6081ad1b0 con 0x562127d5d390 2024-09-21T04:12:22.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.078+0000 7fa61351f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa6081ac410 con 0x562127d7e070 2024-09-21T04:12:22.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.078+0000 7fa61351f740 10 client.0 _ll_register_callbacks cb 0x7fa5f8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:12:22.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.078+0000 7fa5fffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d551b0 0x562128137000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:22.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.078+0000 7fa5fffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d551b0 0x562128137000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60244/0 (socket says 172.21.15.64:60244) 2024-09-21T04:12:22.086 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.078+0000 7fa5fffff640 1 -- 172.21.15.64:0/24654958 learned_addr learned my addr 172.21.15.64:0/24654958 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:22.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.078+0000 7fa60659c640 1 --2- 172.21.15.64:0/24654958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562127d5d390 0x562128137540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:22.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.078+0000 7fa5fffff640 1 -- 172.21.15.64:0/24654958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562127d7e070 msgr2=0x562128137a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:22.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.078+0000 7fa5fffff640 1 --2- 172.21.15.64:0/24654958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562127d7e070 0x562128137a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.078+0000 7fa5fffff640 1 -- 172.21.15.64:0/24654958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562127d5d390 msgr2=0x562128137540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:22.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.078+0000 7fa5fffff640 1 --2- 172.21.15.64:0/24654958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562127d5d390 0x562128137540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.078+0000 7fa5fffff640 1 -- 172.21.15.64:0/24654958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621281ae930 con 0x562127d551b0 2024-09-21T04:12:22.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.078+0000 7fa5fffff640 1 --2- 172.21.15.64:0/24654958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d551b0 0x562128137000 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0004570 tx=0x7fa5f000af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82789e216fccb58 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:22.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.079+0000 7fa5edffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:12:22.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.079+0000 7fa5edffb640 1 -- 172.21.15.64:0/24654958 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f000a590 con 0x562127d551b0 2024-09-21T04:12:22.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.079+0000 7fa5edffb640 1 -- 172.21.15.64:0/24654958 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7fa5f002ef90 con 0x562127d551b0 2024-09-21T04:12:22.087 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.079+0000 7fa61351f740 1 -- 172.21.15.64:0/24654958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5f8003c60 con 0x562127d551b0 2024-09-21T04:12:22.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.079+0000 7fa5edffb640 1 -- 172.21.15.64:0/24654958 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa5f0011260 con 0x562127d551b0 2024-09-21T04:12:22.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.079+0000 7fa61351f740 10 client.17229 fetch_fsmap learned FSMap version 40 2024-09-21T04:12:22.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.079+0000 7fa61351f740 1 -- 172.21.15.64:0/24654958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5621281afbb0 con 0x562127d551b0 2024-09-21T04:12:22.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.079+0000 7fa5edffb640 1 -- 172.21.15.64:0/24654958 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5f8003c60 con 0x562127d551b0 2024-09-21T04:12:22.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.079+0000 7fa5edffb640 1 -- 172.21.15.64:0/24654958 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7fa5f003ca80 con 0x562127d551b0 2024-09-21T04:12:22.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.079+0000 7fa61351f740 10 client.17229 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:12:22.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.079+0000 7fa61351f740 10 client.17229 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:12:22.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.079+0000 7fa61351f740 20 client.17229 populate_metadata read hostname 'smithi064' 2024-09-21T04:12:22.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.079+0000 7fa61351f740 1 --2- 172.21.15.64:0/24654958 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x562127d11da0 0x5621281b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:22.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.079+0000 7fa61351f740 4 client.17229 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:12:22.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.079+0000 7fa61351f740 1 -- 172.21.15.64:0/24654958 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5621280998a0 con 0x562127d11da0 2024-09-21T04:12:22.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.080+0000 7fa60659c640 1 --2- 172.21.15.64:0/24654958 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x562127d11da0 0x5621281b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:22.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.080+0000 7fa60659c640 1 --2- 172.21.15.64:0/24654958 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x562127d11da0 0x5621281b2200 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2375ef2d99b95d5 server_cookie=b6a839c4623308da in_seq=0 out_seq=0 2024-09-21T04:12:22.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.080+0000 7fa5edffb640 10 client.17229 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:12:22.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.080+0000 7fa5edffb640 1 -- 172.21.15.64:0/24654958 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5621280998a0 con 0x562127d11da0 2024-09-21T04:12:22.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.080+0000 7fa5edffb640 10 client.17229 handle_command_reply: tid=0 2024-09-21T04:12:22.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.080+0000 7fa61351f740 1 client.17229 shutdown 2024-09-21T04:12:22.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.080+0000 7fa61351f740 1 -- 172.21.15.64:0/24654958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d551b0 msgr2=0x562128137000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:22.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.080+0000 7fa61351f740 1 --2- 172.21.15.64:0/24654958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d551b0 0x562128137000 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0004570 tx=0x7fa5f000af90 comp rx=0 tx=0).stop 2024-09-21T04:12:22.089 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.081+0000 7fa61351f740 1 -- 172.21.15.64:0/24654958 shutdown_connections 2024-09-21T04:12:22.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.081+0000 7fa61351f740 1 --2- 172.21.15.64:0/24654958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562127d7e070 0x562128137a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.081+0000 7fa61351f740 1 --2- 172.21.15.64:0/24654958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562127d5d390 0x562128137540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.081+0000 7fa61351f740 1 --2- 172.21.15.64:0/24654958 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x562127d11da0 0x5621281b2200 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.081+0000 7fa61351f740 1 --2- 172.21.15.64:0/24654958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562127d551b0 0x562128137000 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.081+0000 7fa61351f740 1 -- 172.21.15.64:0/24654958 >> 172.21.15.64:0/24654958 conn(0x562127d72460 msgr2=0x562128133ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:22.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.081+0000 7fa5edffb640 0 client.17229 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:12:22.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.081+0000 7fa61351f740 1 -- 172.21.15.64:0/24654958 shutdown_connections 2024-09-21T04:12:22.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.081+0000 7fa61351f740 1 -- 172.21.15.64:0/24654958 wait complete. 2024-09-21T04:12:22.090 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.081+0000 7fa61351f740 20 client.17229 trim_cache size 0 max 16384 2024-09-21T04:12:22.090 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:12:22.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.081+0000 7fa60cfab640 1 -- 172.21.15.64:0/2399355964 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa5f40b6680 msgr2=0x7fa5f40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:22.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.081+0000 7fa60cfab640 1 --2- 172.21.15.64:0/2399355964 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa5f40b6680 0x7fa5f40b8b90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa6000e5090 tx=0x7fa60007a390 comp rx=0 tx=0).stop 2024-09-21T04:12:22.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.081+0000 7fa60cfab640 1 -- 172.21.15.64:0/2399355964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa608108d20 msgr2=0x7fa60819eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:22.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.081+0000 7fa60cfab640 1 --2- 172.21.15.64:0/2399355964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa608108d20 0x7fa60819eba0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa60000eaa0 tx=0x7fa60000ef60 comp rx=0 tx=0).stop 2024-09-21T04:12:22.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.081+0000 7fa60cfab640 1 -- 172.21.15.64:0/2399355964 shutdown_connections 2024-09-21T04:12:22.091 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.081+0000 7fa60cfab640 1 --2- 172.21.15.64:0/2399355964 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa5f40b6680 0x7fa5f40b8b90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.081+0000 7fa60cfab640 1 --2- 172.21.15.64:0/2399355964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa608108d20 0x7fa60819eba0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.081+0000 7fa60cfab640 1 --2- 172.21.15.64:0/2399355964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6081083d0 0x7fa608199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.081+0000 7fa60cfab640 1 --2- 172.21.15.64:0/2399355964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa608102ae0 0x7fa608199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:22.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.081+0000 7fa60cfab640 1 -- 172.21.15.64:0/2399355964 >> 172.21.15.64:0/2399355964 conn(0x7fa608079760 msgr2=0x7fa608101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:22.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.082+0000 7fa60cfab640 1 -- 172.21.15.64:0/2399355964 shutdown_connections 2024-09-21T04:12:22.092 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:22.082+0000 7fa60cfab640 1 -- 172.21.15.64:0/2399355964 wait complete. 2024-09-21T04:12:22.737 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:22 smithi064 ceph-mon[31180]: pgmap v577: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 921 B/s wr, 6 op/s 2024-09-21T04:12:22.737 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:22 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:12:22.737 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:22 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:12:22.737 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:22 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:12:22.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:22 smithi119 ceph-mon[34811]: pgmap v577: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 921 B/s wr, 6 op/s 2024-09-21T04:12:22.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:22 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:12:22.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:22 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:12:22.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:22 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:12:22.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:22 smithi131 ceph-mon[35193]: pgmap v577: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 921 B/s wr, 6 op/s 2024-09-21T04:12:22.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:22 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:12:22.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:22 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:12:22.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:22 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:12:23.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:23 smithi119 ceph-mon[34811]: pgmap v578: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 818 B/s wr, 6 op/s 2024-09-21T04:12:23.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:23 smithi064 ceph-mon[31180]: pgmap v578: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 818 B/s wr, 6 op/s 2024-09-21T04:12:23.767 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T04:12:23.767 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T04:12:23.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:23 smithi131 ceph-mon[35193]: pgmap v578: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 818 B/s wr, 6 op/s 2024-09-21T04:12:24.037 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:12:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.517+0000 7fbb2b0ef640 1 -- 172.21.15.64:0/2570703391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb24102100 msgr2=0x7fbb24102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.517+0000 7fbb2b0ef640 1 --2- 172.21.15.64:0/2570703391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb24102100 0x7fbb24102510 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fbb18009900 tx=0x7fbb1802a410 comp rx=0 tx=0).stop 2024-09-21T04:12:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.517+0000 7fbb2b0ef640 1 -- 172.21.15.64:0/2570703391 shutdown_connections 2024-09-21T04:12:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.517+0000 7fbb2b0ef640 1 --2- 172.21.15.64:0/2570703391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb24107c50 0x7fbb24110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.517+0000 7fbb2b0ef640 1 --2- 172.21.15.64:0/2570703391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb24107300 0x7fbb24107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.517+0000 7fbb2b0ef640 1 --2- 172.21.15.64:0/2570703391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb24102100 0x7fbb24102510 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.517+0000 7fbb2b0ef640 1 -- 172.21.15.64:0/2570703391 >> 172.21.15.64:0/2570703391 conn(0x7fbb24079e40 msgr2=0x7fbb2407a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.517+0000 7fbb2b0ef640 1 -- 172.21.15.64:0/2570703391 shutdown_connections 2024-09-21T04:12:24.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.518+0000 7fbb2b0ef640 1 -- 172.21.15.64:0/2570703391 wait complete. 2024-09-21T04:12:24.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.518+0000 7fbb2b0ef640 1 Processor -- start 2024-09-21T04:12:24.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.518+0000 7fbb2b0ef640 1 -- start start 2024-09-21T04:12:24.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.519+0000 7fbb2b0ef640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb24102100 0x7fbb241030a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:24.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.519+0000 7fbb2b0ef640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb24107300 0x7fbb241035e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:24.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.519+0000 7fbb2b0ef640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb24107c50 0x7fbb24103b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:24.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.519+0000 7fbb2b0ef640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbb24113b00 con 0x7fbb24107300 2024-09-21T04:12:24.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.519+0000 7fbb2b0ef640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbb24113980 con 0x7fbb24102100 2024-09-21T04:12:24.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.519+0000 7fbb2b0ef640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbb24113c80 con 0x7fbb24107c50 2024-09-21T04:12:24.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.519+0000 7fbb23fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb24107300 0x7fbb241035e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:24.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.519+0000 7fbb23fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb24107300 0x7fbb241035e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60266/0 (socket says 172.21.15.64:60266) 2024-09-21T04:12:24.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.519+0000 7fbb23fff640 1 -- 172.21.15.64:0/3105143180 learned_addr learned my addr 172.21.15.64:0/3105143180 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:24.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.519+0000 7fbb28e8b640 1 --2- 172.21.15.64:0/3105143180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb24102100 0x7fbb241030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:24.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.519+0000 7fbb2968c640 1 --2- 172.21.15.64:0/3105143180 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb24107c50 0x7fbb24103b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:24.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.519+0000 7fbb23fff640 1 -- 172.21.15.64:0/3105143180 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb24107c50 msgr2=0x7fbb24103b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:24.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.519+0000 7fbb23fff640 1 --2- 172.21.15.64:0/3105143180 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb24107c50 0x7fbb24103b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.519+0000 7fbb23fff640 1 -- 172.21.15.64:0/3105143180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb24102100 msgr2=0x7fbb241030a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:24.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.519+0000 7fbb23fff640 1 --2- 172.21.15.64:0/3105143180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb24102100 0x7fbb241030a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.519+0000 7fbb23fff640 1 -- 172.21.15.64:0/3105143180 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb241a87b0 con 0x7fbb24107300 2024-09-21T04:12:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.519+0000 7fbb28e8b640 1 --2- 172.21.15.64:0/3105143180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb24102100 0x7fbb241030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:12:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.519+0000 7fbb23fff640 1 --2- 172.21.15.64:0/3105143180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb24107300 0x7fbb241035e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000d950 tx=0x7fbb1000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.520+0000 7fbb21ffb640 1 -- 172.21.15.64:0/3105143180 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10017070 con 0x7fbb24107300 2024-09-21T04:12:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.520+0000 7fbb2b0ef640 1 -- 172.21.15.64:0/3105143180 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb241a8aa0 con 0x7fbb24107300 2024-09-21T04:12:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.520+0000 7fbb21ffb640 1 -- 172.21.15.64:0/3105143180 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb10009c30 con 0x7fbb24107300 2024-09-21T04:12:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.520+0000 7fbb21ffb640 1 -- 172.21.15.64:0/3105143180 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb100047a0 con 0x7fbb24107300 2024-09-21T04:12:24.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.520+0000 7fbb2b0ef640 1 -- 172.21.15.64:0/3105143180 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb241a9030 con 0x7fbb24107300 2024-09-21T04:12:24.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.523+0000 7fbb31663740 1 Processor -- start 2024-09-21T04:12:24.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.523+0000 7fbb31663740 1 -- start start 2024-09-21T04:12:24.526 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.523+0000 7fbb21ffb640 1 -- 172.21.15.64:0/3105143180 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fbb10004020 con 0x7fbb24107300 2024-09-21T04:12:24.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.523+0000 7fbb31663740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cf3350 0x55e91a09a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:24.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.523+0000 7fbb31663740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e919cca490 0x55e91a09a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:24.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.523+0000 7fbb31663740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e919cd2670 0x55e91a09aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:24.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.523+0000 7fbb31663740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e91a010e60 con 0x55e919cf3350 2024-09-21T04:12:24.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.523+0000 7fbb31663740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e91a000290 con 0x55e919cd2670 2024-09-21T04:12:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.523+0000 7fbb31663740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e91a0040a0 con 0x55e919cca490 2024-09-21T04:12:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.524+0000 7fbb2968c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cf3350 0x55e91a09a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.524+0000 7fbb2968c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cf3350 0x55e91a09a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60274/0 (socket says 172.21.15.64:60274) 2024-09-21T04:12:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.524+0000 7fbb2968c640 1 -- 172.21.15.64:0/1036835802 learned_addr learned my addr 172.21.15.64:0/1036835802 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.524+0000 7fbb28e8b640 1 --2- 172.21.15.64:0/1036835802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e919cca490 0x55e91a09a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.524+0000 7fbb23fff640 1 --2- 172.21.15.64:0/1036835802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e919cd2670 0x55e91a09aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.524+0000 7fbb2968c640 1 -- 172.21.15.64:0/1036835802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e919cca490 msgr2=0x55e91a09a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.524+0000 7fbb2968c640 1 --2- 172.21.15.64:0/1036835802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e919cca490 0x55e91a09a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.524+0000 7fbb2968c640 1 -- 172.21.15.64:0/1036835802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e919cd2670 msgr2=0x55e91a09aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.524+0000 7fbb2968c640 1 --2- 172.21.15.64:0/1036835802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e919cd2670 0x55e91a09aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.524+0000 7fbb2968c640 1 -- 172.21.15.64:0/1036835802 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e919d5e930 con 0x55e919cf3350 2024-09-21T04:12:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.524+0000 7fbb2968c640 1 --2- 172.21.15.64:0/1036835802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cf3350 0x55e91a09a430 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c00ae10 tx=0x7fbb1c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b770ded9c927da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.524+0000 7fbb21ffb640 1 --2- 172.21.15.64:0/3105143180 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbb140b6680 0x7fbb140b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:24.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.524+0000 7fbb0effd640 1 -- 172.21.15.64:0/1036835802 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c003b70 con 0x55e919cf3350 2024-09-21T04:12:24.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.525+0000 7fbb0effd640 1 -- 172.21.15.64:0/1036835802 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb1c003d10 con 0x55e919cf3350 2024-09-21T04:12:24.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.525+0000 7fbb0effd640 1 -- 172.21.15.64:0/1036835802 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c0109c0 con 0x55e919cf3350 2024-09-21T04:12:24.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.525+0000 7fbb28e8b640 1 --2- 172.21.15.64:0/3105143180 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbb140b6680 0x7fbb140b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:24.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.525+0000 7fbb21ffb640 1 -- 172.21.15.64:0/3105143180 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7fbb100e7470 con 0x7fbb24107300 2024-09-21T04:12:24.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.525+0000 7fbb31663740 1 -- 172.21.15.64:0/1036835802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cf3350 msgr2=0x55e91a09a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:24.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.525+0000 7fbb28e8b640 1 --2- 172.21.15.64:0/3105143180 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbb140b6680 0x7fbb140b8b90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbb18000c00 tx=0x7fbb18009640 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:24.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.525+0000 7fbb31663740 1 --2- 172.21.15.64:0/1036835802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cf3350 0x55e91a09a430 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c00ae10 tx=0x7fbb1c00f790 comp rx=0 tx=0).stop 2024-09-21T04:12:24.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.525+0000 7fbb31663740 1 -- 172.21.15.64:0/1036835802 shutdown_connections 2024-09-21T04:12:24.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.525+0000 7fbb31663740 1 --2- 172.21.15.64:0/1036835802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e919cd2670 0x55e91a09aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.525+0000 7fbb31663740 1 --2- 172.21.15.64:0/1036835802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e919cca490 0x55e91a09a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.525+0000 7fbb31663740 1 --2- 172.21.15.64:0/1036835802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cf3350 0x55e91a09a430 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.525+0000 7fbb31663740 1 -- 172.21.15.64:0/1036835802 >> 172.21.15.64:0/1036835802 conn(0x55e919ce7740 msgr2=0x55e91a098320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:24.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.526+0000 7fbb31663740 1 -- 172.21.15.64:0/1036835802 shutdown_connections 2024-09-21T04:12:24.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.526+0000 7fbb31663740 1 -- 172.21.15.64:0/1036835802 wait complete. 2024-09-21T04:12:24.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.526+0000 7fbb31663740 1 -- start start 2024-09-21T04:12:24.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.526+0000 7fbb31663740 1 Processor -- start 2024-09-21T04:12:24.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.526+0000 7fbb31663740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e919cca490 0x55e91a0a5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:24.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.526+0000 7fbb31663740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cd2670 0x55e91a0a6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:24.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.526+0000 7fbb31663740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e919cf3350 0x55e91a0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:24.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.526+0000 7fbb31663740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e919e61890 con 0x55e919cd2670 2024-09-21T04:12:24.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.527+0000 7fbb31663740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e919f9ad40 con 0x55e919cf3350 2024-09-21T04:12:24.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.527+0000 7fbb31663740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e91a0a8c10 con 0x55e919cca490 2024-09-21T04:12:24.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.527+0000 7fbb31663740 10 client.0 _ll_register_callbacks cb 0x55e919ffec80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:12:24.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.527+0000 7fbb23fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cd2670 0x55e91a0a6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:24.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.527+0000 7fbb23fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cd2670 0x55e91a0a6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60280/0 (socket says 172.21.15.64:60280) 2024-09-21T04:12:24.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.527+0000 7fbb23fff640 1 -- 172.21.15.64:0/4223541372 learned_addr learned my addr 172.21.15.64:0/4223541372 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:24.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.527+0000 7fbb2968c640 1 --2- 172.21.15.64:0/4223541372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e919cf3350 0x55e91a0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:24.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.527+0000 7fbb28e8b640 1 --2- 172.21.15.64:0/4223541372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e919cca490 0x55e91a0a5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:24.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.527+0000 7fbb23fff640 1 -- 172.21.15.64:0/4223541372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e919cca490 msgr2=0x55e91a0a5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:24.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.527+0000 7fbb23fff640 1 --2- 172.21.15.64:0/4223541372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e919cca490 0x55e91a0a5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.527+0000 7fbb23fff640 1 -- 172.21.15.64:0/4223541372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e919cf3350 msgr2=0x55e91a0b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:24.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.527+0000 7fbb23fff640 1 --2- 172.21.15.64:0/4223541372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e919cf3350 0x55e91a0b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.527+0000 7fbb23fff640 1 -- 172.21.15.64:0/4223541372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e91a0b24c0 con 0x55e919cd2670 2024-09-21T04:12:24.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.527+0000 7fbb23fff640 1 --2- 172.21.15.64:0/4223541372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cd2670 0x55e91a0a6520 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fbb10005fd0 tx=0x7fbb100056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68b826588116075 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:24.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.527+0000 7fbb03fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:12:24.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.527+0000 7fbb03fff640 1 -- 172.21.15.64:0/4223541372 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb100ad030 con 0x55e919cd2670 2024-09-21T04:12:24.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.527+0000 7fbb03fff640 1 -- 172.21.15.64:0/4223541372 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7fbb100a8bb0 con 0x55e919cd2670 2024-09-21T04:12:24.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.527+0000 7fbb31663740 1 -- 172.21.15.64:0/4223541372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb1c003d10 con 0x55e919cd2670 2024-09-21T04:12:24.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.528+0000 7fbb03fff640 1 -- 172.21.15.64:0/4223541372 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbb100ac9d0 con 0x55e919cd2670 2024-09-21T04:12:24.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.528+0000 7fbb31663740 10 client.17241 fetch_fsmap learned FSMap version 40 2024-09-21T04:12:24.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.528+0000 7fbb31663740 1 -- 172.21.15.64:0/4223541372 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e91a0b4df0 con 0x55e919cd2670 2024-09-21T04:12:24.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.528+0000 7fbb03fff640 1 -- 172.21.15.64:0/4223541372 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb1c003d10 con 0x55e919cd2670 2024-09-21T04:12:24.534 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.528+0000 7fbb03fff640 1 -- 172.21.15.64:0/4223541372 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7fbb100d0be0 con 0x55e919cd2670 2024-09-21T04:12:24.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.528+0000 7fbb31663740 10 client.17241 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:12:24.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.528+0000 7fbb31663740 10 client.17241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:12:24.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.528+0000 7fbb31663740 20 client.17241 populate_metadata read hostname 'smithi064' 2024-09-21T04:12:24.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.528+0000 7fbb31663740 1 --2- 172.21.15.64:0/4223541372 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55e919cb1cd0 0x55e91a0b8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:24.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.528+0000 7fbb31663740 4 client.17241 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:12:24.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.528+0000 7fbb31663740 1 -- 172.21.15.64:0/4223541372 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e91a012db0 con 0x55e919cb1cd0 2024-09-21T04:12:24.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.528+0000 7fbb28e8b640 1 --2- 172.21.15.64:0/4223541372 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55e919cb1cd0 0x55e91a0b8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:24.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.529+0000 7fbb03fff640 10 client.17241 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:12:24.535 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.529+0000 7fbb28e8b640 1 --2- 172.21.15.64:0/4223541372 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55e919cb1cd0 0x55e91a0b8360 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dc756271cb684b6 server_cookie=8a8049462e9659d8 in_seq=0 out_seq=0 2024-09-21T04:12:24.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.530+0000 7fbb03fff640 1 -- 172.21.15.64:0/4223541372 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e91a012db0 con 0x55e919cb1cd0 2024-09-21T04:12:24.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.530+0000 7fbb03fff640 10 client.17241 handle_command_reply: tid=0 2024-09-21T04:12:24.536 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.530+0000 7fbb31663740 1 client.17241 shutdown 2024-09-21T04:12:24.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.530+0000 7fbb31663740 1 -- 172.21.15.64:0/4223541372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cd2670 msgr2=0x55e91a0a6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:24.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.530+0000 7fbb31663740 1 --2- 172.21.15.64:0/4223541372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cd2670 0x55e91a0a6520 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fbb10005fd0 tx=0x7fbb100056f0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.530+0000 7fbb31663740 1 -- 172.21.15.64:0/4223541372 shutdown_connections 2024-09-21T04:12:24.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.530+0000 7fbb31663740 1 --2- 172.21.15.64:0/4223541372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e919cf3350 0x55e91a0b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.530+0000 7fbb31663740 1 --2- 172.21.15.64:0/4223541372 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55e919cb1cd0 0x55e91a0b8360 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.530+0000 7fbb03fff640 0 client.17241 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:12:24.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.530+0000 7fbb31663740 1 --2- 172.21.15.64:0/4223541372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cd2670 0x55e91a0a6520 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.530+0000 7fbb31663740 1 --2- 172.21.15.64:0/4223541372 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e919cca490 0x55e91a0a5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.530+0000 7fbb31663740 1 -- 172.21.15.64:0/4223541372 >> 172.21.15.64:0/4223541372 conn(0x55e919ce7740 msgr2=0x55e91a09d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:24.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.530+0000 7fbb31663740 1 -- 172.21.15.64:0/4223541372 shutdown_connections 2024-09-21T04:12:24.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.530+0000 7fbb31663740 1 -- 172.21.15.64:0/4223541372 wait complete. 2024-09-21T04:12:24.537 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.530+0000 7fbb31663740 20 client.17241 trim_cache size 0 max 16384 2024-09-21T04:12:24.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.548+0000 7fbb31663740 1 Processor -- start 2024-09-21T04:12:24.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.548+0000 7fbb31663740 1 -- start start 2024-09-21T04:12:24.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.548+0000 7fbb31663740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cb1cd0 0x55e91a0a1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:24.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.548+0000 7fbb31663740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e919cca490 0x55e91a0b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:24.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.548+0000 7fbb31663740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e919cd2670 0x55e91a09be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:24.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.548+0000 7fbb31663740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbb241b4570 con 0x55e919cb1cd0 2024-09-21T04:12:24.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.548+0000 7fbb31663740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbb241b1d80 con 0x55e919cca490 2024-09-21T04:12:24.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.548+0000 7fbb31663740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbb241afbf0 con 0x55e919cd2670 2024-09-21T04:12:24.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.548+0000 7fbb28e8b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cb1cd0 0x55e91a0a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:24.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.548+0000 7fbb28e8b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cb1cd0 0x55e91a0a1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60282/0 (socket says 172.21.15.64:60282) 2024-09-21T04:12:24.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.548+0000 7fbb28e8b640 1 -- 172.21.15.64:0/272885597 learned_addr learned my addr 172.21.15.64:0/272885597 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:24.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.548+0000 7fbb23fff640 1 --2- 172.21.15.64:0/272885597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e919cca490 0x55e91a0b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:24.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.548+0000 7fbb2968c640 1 --2- 172.21.15.64:0/272885597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e919cd2670 0x55e91a09be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:24.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.549+0000 7fbb28e8b640 1 -- 172.21.15.64:0/272885597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e919cd2670 msgr2=0x55e91a09be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:24.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.549+0000 7fbb28e8b640 1 --2- 172.21.15.64:0/272885597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e919cd2670 0x55e91a09be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.549+0000 7fbb28e8b640 1 -- 172.21.15.64:0/272885597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e919cca490 msgr2=0x55e91a0b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:24.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.549+0000 7fbb28e8b640 1 --2- 172.21.15.64:0/272885597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e919cca490 0x55e91a0b1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.549+0000 7fbb28e8b640 1 -- 172.21.15.64:0/272885597 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e91a0a70a0 con 0x55e919cb1cd0 2024-09-21T04:12:24.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.549+0000 7fbb28e8b640 1 --2- 172.21.15.64:0/272885597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cb1cd0 0x55e91a0a1590 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fbb18009900 tx=0x7fbb1803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=669c47e421f82567 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:24.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.549+0000 7fbb0e7fc640 1 -- 172.21.15.64:0/272885597 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18035050 con 0x55e919cb1cd0 2024-09-21T04:12:24.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.549+0000 7fbb0e7fc640 1 -- 172.21.15.64:0/272885597 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb18054070 con 0x55e919cb1cd0 2024-09-21T04:12:24.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.549+0000 7fbb0e7fc640 1 -- 172.21.15.64:0/272885597 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb180049b0 con 0x55e919cb1cd0 2024-09-21T04:12:24.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.550+0000 7fbb31663740 1 -- 172.21.15.64:0/272885597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cb1cd0 msgr2=0x55e91a0a1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:24.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.550+0000 7fbb31663740 1 --2- 172.21.15.64:0/272885597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cb1cd0 0x55e91a0a1590 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fbb18009900 tx=0x7fbb1803d000 comp rx=0 tx=0).stop 2024-09-21T04:12:24.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.550+0000 7fbb31663740 1 -- 172.21.15.64:0/272885597 shutdown_connections 2024-09-21T04:12:24.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.550+0000 7fbb31663740 1 --2- 172.21.15.64:0/272885597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e919cd2670 0x55e91a09be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.550+0000 7fbb31663740 1 --2- 172.21.15.64:0/272885597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e919cca490 0x55e91a0b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.550+0000 7fbb31663740 1 --2- 172.21.15.64:0/272885597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cb1cd0 0x55e91a0a1590 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.550+0000 7fbb31663740 1 -- 172.21.15.64:0/272885597 >> 172.21.15.64:0/272885597 conn(0x55e919ce7740 msgr2=0x55e91a0a7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:24.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.550+0000 7fbb31663740 1 -- 172.21.15.64:0/272885597 shutdown_connections 2024-09-21T04:12:24.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.550+0000 7fbb31663740 1 -- 172.21.15.64:0/272885597 wait complete. 2024-09-21T04:12:24.556 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.550+0000 7fbb31663740 1 -- start start 2024-09-21T04:12:24.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.550+0000 7fbb31663740 1 Processor -- start 2024-09-21T04:12:24.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.551+0000 7fbb31663740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cb1cd0 0x55e91a0aa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:24.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.551+0000 7fbb31663740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e919cca490 0x55e91a0aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:24.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.551+0000 7fbb31663740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e919cd2670 0x55e91a0aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:24.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.551+0000 7fbb31663740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbb241b0240 con 0x55e919cb1cd0 2024-09-21T04:12:24.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.551+0000 7fbb31663740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbb241b16b0 con 0x55e919cd2670 2024-09-21T04:12:24.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.551+0000 7fbb31663740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbb241b0910 con 0x55e919cca490 2024-09-21T04:12:24.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.551+0000 7fbb31663740 10 client.0 _ll_register_callbacks cb 0x7fbb100eab40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:12:24.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.551+0000 7fbb28e8b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cb1cd0 0x55e91a0aa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:24.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.551+0000 7fbb28e8b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cb1cd0 0x55e91a0aa030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60288/0 (socket says 172.21.15.64:60288) 2024-09-21T04:12:24.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.551+0000 7fbb28e8b640 1 -- 172.21.15.64:0/4149707960 learned_addr learned my addr 172.21.15.64:0/4149707960 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:24.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.551+0000 7fbb23fff640 1 --2- 172.21.15.64:0/4149707960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e919cca490 0x55e91a0aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:24.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.551+0000 7fbb2968c640 1 --2- 172.21.15.64:0/4149707960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e919cd2670 0x55e91a0aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:24.557 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.551+0000 7fbb28e8b640 1 -- 172.21.15.64:0/4149707960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e919cca490 msgr2=0x55e91a0aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:24.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.551+0000 7fbb28e8b640 1 --2- 172.21.15.64:0/4149707960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e919cca490 0x55e91a0aa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.551+0000 7fbb28e8b640 1 -- 172.21.15.64:0/4149707960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e919cd2670 msgr2=0x55e91a0aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:24.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.551+0000 7fbb28e8b640 1 --2- 172.21.15.64:0/4149707960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e919cd2670 0x55e91a0aaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.551+0000 7fbb28e8b640 1 -- 172.21.15.64:0/4149707960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e91a121f00 con 0x55e919cb1cd0 2024-09-21T04:12:24.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.552+0000 7fbb28e8b640 1 --2- 172.21.15.64:0/4149707960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cb1cd0 0x55e91a0aa030 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fbb18031a60 tx=0x7fbb18005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b312382c1610ce2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:24.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.552+0000 7fbb03fff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:12:24.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.552+0000 7fbb03fff640 1 -- 172.21.15.64:0/4149707960 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18035030 con 0x55e919cb1cd0 2024-09-21T04:12:24.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.552+0000 7fbb03fff640 1 -- 172.21.15.64:0/4149707960 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7fbb1804e070 con 0x55e919cb1cd0 2024-09-21T04:12:24.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.552+0000 7fbb31663740 1 -- 172.21.15.64:0/4149707960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb18054070 con 0x55e919cb1cd0 2024-09-21T04:12:24.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.552+0000 7fbb03fff640 1 -- 172.21.15.64:0/4149707960 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbb18065050 con 0x55e919cb1cd0 2024-09-21T04:12:24.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.552+0000 7fbb31663740 10 client.17247 fetch_fsmap learned FSMap version 40 2024-09-21T04:12:24.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.552+0000 7fbb31663740 1 -- 172.21.15.64:0/4149707960 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e91a1231c0 con 0x55e919cb1cd0 2024-09-21T04:12:24.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.552+0000 7fbb03fff640 1 -- 172.21.15.64:0/4149707960 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb18054070 con 0x55e919cb1cd0 2024-09-21T04:12:24.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.552+0000 7fbb03fff640 1 -- 172.21.15.64:0/4149707960 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7fbb180035f0 con 0x55e919cb1cd0 2024-09-21T04:12:24.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.552+0000 7fbb31663740 10 client.17247 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:12:24.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.552+0000 7fbb31663740 10 client.17247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:12:24.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.552+0000 7fbb31663740 20 client.17247 populate_metadata read hostname 'smithi064' 2024-09-21T04:12:24.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.552+0000 7fbb31663740 1 --2- 172.21.15.64:0/4149707960 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55e919cf3350 0x55e91a125940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:24.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.552+0000 7fbb31663740 4 client.17247 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T04:12:24.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.552+0000 7fbb31663740 1 -- 172.21.15.64:0/4149707960 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e91a00e860 con 0x55e919cf3350 2024-09-21T04:12:24.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.553+0000 7fbb23fff640 1 --2- 172.21.15.64:0/4149707960 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55e919cf3350 0x55e91a125940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:24.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.553+0000 7fbb23fff640 1 --2- 172.21.15.64:0/4149707960 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55e919cf3350 0x55e91a125940 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51bd1459e2134a75 server_cookie=b42cf6b0173e24e1 in_seq=0 out_seq=0 2024-09-21T04:12:24.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.553+0000 7fbb03fff640 10 client.17247 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:12:24.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.554+0000 7fbb03fff640 1 -- 172.21.15.64:0/4149707960 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e91a00e860 con 0x55e919cf3350 2024-09-21T04:12:24.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.554+0000 7fbb03fff640 10 client.17247 handle_command_reply: tid=0 2024-09-21T04:12:24.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.554+0000 7fbb31663740 1 client.17247 shutdown 2024-09-21T04:12:24.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.554+0000 7fbb31663740 1 -- 172.21.15.64:0/4149707960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cb1cd0 msgr2=0x55e91a0aa030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:24.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.554+0000 7fbb31663740 1 --2- 172.21.15.64:0/4149707960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cb1cd0 0x55e91a0aa030 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fbb18031a60 tx=0x7fbb18005770 comp rx=0 tx=0).stop 2024-09-21T04:12:24.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.554+0000 7fbb31663740 1 -- 172.21.15.64:0/4149707960 shutdown_connections 2024-09-21T04:12:24.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.554+0000 7fbb31663740 1 --2- 172.21.15.64:0/4149707960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e919cd2670 0x55e91a0aaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.554+0000 7fbb31663740 1 --2- 172.21.15.64:0/4149707960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e919cca490 0x55e91a0aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.554+0000 7fbb31663740 1 --2- 172.21.15.64:0/4149707960 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55e919cf3350 0x55e91a125940 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.554+0000 7fbb31663740 1 --2- 172.21.15.64:0/4149707960 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e919cb1cd0 0x55e91a0aa030 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.554+0000 7fbb31663740 1 -- 172.21.15.64:0/4149707960 >> 172.21.15.64:0/4149707960 conn(0x55e919ce7740 msgr2=0x55e91a09bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:24.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.554+0000 7fbb03fff640 0 client.17247 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:12:24.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.555+0000 7fbb31663740 1 -- 172.21.15.64:0/4149707960 shutdown_connections 2024-09-21T04:12:24.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.555+0000 7fbb31663740 1 -- 172.21.15.64:0/4149707960 wait complete. 2024-09-21T04:12:24.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.555+0000 7fbb31663740 20 client.17247 trim_cache size 0 max 16384 2024-09-21T04:12:24.561 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:12:24.562 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-21T04:12:24.562 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "dd25622a-26f0-4838-b998-85d0603c83b6", 2024-09-21T04:12:24.562 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-21T04:12:24.562 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:12:24.562 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.555+0000 7fbb2b0ef640 1 -- 172.21.15.64:0/3105143180 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbb140b6680 msgr2=0x7fbb140b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:24.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.555+0000 7fbb2b0ef640 1 --2- 172.21.15.64:0/3105143180 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbb140b6680 0x7fbb140b8b90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbb18000c00 tx=0x7fbb18009640 comp rx=0 tx=0).stop 2024-09-21T04:12:24.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.555+0000 7fbb2b0ef640 1 -- 172.21.15.64:0/3105143180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb24107300 msgr2=0x7fbb241035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:24.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.555+0000 7fbb2b0ef640 1 --2- 172.21.15.64:0/3105143180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb24107300 0x7fbb241035e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000d950 tx=0x7fbb1000de10 comp rx=0 tx=0).stop 2024-09-21T04:12:24.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.555+0000 7fbb2b0ef640 1 -- 172.21.15.64:0/3105143180 shutdown_connections 2024-09-21T04:12:24.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.555+0000 7fbb2b0ef640 1 --2- 172.21.15.64:0/3105143180 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fbb140b6680 0x7fbb140b8b90 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.555+0000 7fbb2b0ef640 1 --2- 172.21.15.64:0/3105143180 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb24107c50 0x7fbb24103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.555+0000 7fbb2b0ef640 1 --2- 172.21.15.64:0/3105143180 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb24107300 0x7fbb241035e0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.555+0000 7fbb2b0ef640 1 --2- 172.21.15.64:0/3105143180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb24102100 0x7fbb241030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:24.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.556+0000 7fbb2b0ef640 1 -- 172.21.15.64:0/3105143180 >> 172.21.15.64:0/3105143180 conn(0x7fbb24079e40 msgr2=0x7fbb241060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:24.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.556+0000 7fbb2b0ef640 1 -- 172.21.15.64:0/3105143180 shutdown_connections 2024-09-21T04:12:24.563 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:24.556+0000 7fbb2b0ef640 1 -- 172.21.15.64:0/3105143180 wait complete. 2024-09-21T04:12:25.242 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T04:12:25.497 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:12:26.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:25 smithi119 ceph-mon[34811]: scrub queued for path: / 2024-09-21T04:12:26.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:25 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:12:26.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:25 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:12:26.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:25 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:12:26.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:25 smithi119 ceph-mon[34811]: pgmap v579: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 818 B/s wr, 6 op/s 2024-09-21T04:12:26.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:25 smithi064 ceph-mon[31180]: scrub queued for path: / 2024-09-21T04:12:26.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:25 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:12:26.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:25 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:12:26.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:25 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:12:26.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:25 smithi064 ceph-mon[31180]: pgmap v579: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 818 B/s wr, 6 op/s 2024-09-21T04:12:26.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:25 smithi131 ceph-mon[35193]: scrub queued for path: / 2024-09-21T04:12:26.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:25 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:12:26.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:25 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:12:26.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:25 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:12:26.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:25 smithi131 ceph-mon[35193]: pgmap v579: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 818 B/s wr, 6 op/s 2024-09-21T04:12:26.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.143+0000 7f2bd7e97640 1 -- 172.21.15.64:0/1922484538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bd01083a0 msgr2=0x7f2bd01087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:26.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.143+0000 7f2bd7e97640 1 --2- 172.21.15.64:0/1922484538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bd01083a0 0x7f2bd01087b0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009900 tx=0x7f2bc002a410 comp rx=0 tx=0).stop 2024-09-21T04:12:26.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.143+0000 7f2bd7e97640 1 -- 172.21.15.64:0/1922484538 shutdown_connections 2024-09-21T04:12:26.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.143+0000 7f2bd7e97640 1 --2- 172.21.15.64:0/1922484538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bd0108cf0 0x7f2bd010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.143+0000 7f2bd7e97640 1 --2- 172.21.15.64:0/1922484538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bd01083a0 0x7f2bd01087b0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.143+0000 7f2bd7e97640 1 --2- 172.21.15.64:0/1922484538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2bd0102a60 0x7f2bd0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.143+0000 7f2bd7e97640 1 -- 172.21.15.64:0/1922484538 >> 172.21.15.64:0/1922484538 conn(0x7f2bd0079730 msgr2=0x7f2bd0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:26.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.144+0000 7f2bd7e97640 1 -- 172.21.15.64:0/1922484538 shutdown_connections 2024-09-21T04:12:26.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.144+0000 7f2bd7e97640 1 -- 172.21.15.64:0/1922484538 wait complete. 2024-09-21T04:12:26.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.144+0000 7f2bd7e97640 1 Processor -- start 2024-09-21T04:12:26.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.144+0000 7f2bd7e97640 1 -- start start 2024-09-21T04:12:26.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.145+0000 7f2bd7e97640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bd0102a60 0x7f2bd019b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:26.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.145+0000 7f2bd7e97640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2bd01083a0 0x7f2bd019b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:26.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.145+0000 7f2bd7e97640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bd0108cf0 0x7f2bd019be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:26.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.145+0000 7f2bd7e97640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd01129a0 con 0x7f2bd0108cf0 2024-09-21T04:12:26.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.145+0000 7f2bd7e97640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd0112820 con 0x7f2bd0102a60 2024-09-21T04:12:26.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.145+0000 7f2bd7e97640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd0112b20 con 0x7f2bd01083a0 2024-09-21T04:12:26.152 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.145+0000 7f2bd6434640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bd0108cf0 0x7f2bd019be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:26.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.145+0000 7f2bd6434640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bd0108cf0 0x7f2bd019be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60308/0 (socket says 172.21.15.64:60308) 2024-09-21T04:12:26.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.145+0000 7f2bd6434640 1 -- 172.21.15.64:0/2564968178 learned_addr learned my addr 172.21.15.64:0/2564968178 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:26.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.145+0000 7f2bd5c33640 1 --2- 172.21.15.64:0/2564968178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bd0102a60 0x7f2bd019b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:26.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.145+0000 7f2bd5432640 1 --2- 172.21.15.64:0/2564968178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2bd01083a0 0x7f2bd019b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:26.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.145+0000 7f2bd6434640 1 -- 172.21.15.64:0/2564968178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2bd01083a0 msgr2=0x7f2bd019b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:26.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.145+0000 7f2bd6434640 1 --2- 172.21.15.64:0/2564968178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2bd01083a0 0x7f2bd019b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.145+0000 7f2bd6434640 1 -- 172.21.15.64:0/2564968178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bd0102a60 msgr2=0x7f2bd019b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:26.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.145+0000 7f2bd6434640 1 --2- 172.21.15.64:0/2564968178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bd0102a60 0x7f2bd019b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.145+0000 7f2bd6434640 1 -- 172.21.15.64:0/2564968178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd01a6680 con 0x7f2bd0108cf0 2024-09-21T04:12:26.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.145+0000 7f2bd6434640 1 --2- 172.21.15.64:0/2564968178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bd0108cf0 0x7f2bd019be70 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800b970 tx=0x7f2bc800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:26.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.146+0000 7f2bc6ffd640 1 -- 172.21.15.64:0/2564968178 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8019070 con 0x7f2bd0108cf0 2024-09-21T04:12:26.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.146+0000 7f2bc6ffd640 1 -- 172.21.15.64:0/2564968178 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bc8014440 con 0x7f2bd0108cf0 2024-09-21T04:12:26.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.146+0000 7f2bd7e97640 1 -- 172.21.15.64:0/2564968178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd01a6970 con 0x7f2bd0108cf0 2024-09-21T04:12:26.153 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.146+0000 7f2bc6ffd640 1 -- 172.21.15.64:0/2564968178 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc80075d0 con 0x7f2bd0108cf0 2024-09-21T04:12:26.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.146+0000 7f2bd7e97640 1 -- 172.21.15.64:0/2564968178 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd01a6e30 con 0x7f2bd0108cf0 2024-09-21T04:12:26.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.148+0000 7f2bde40b740 1 Processor -- start 2024-09-21T04:12:26.154 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.149+0000 7f2bc6ffd640 1 -- 172.21.15.64:0/2564968178 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f2bc801c070 con 0x7f2bd0108cf0 2024-09-21T04:12:26.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.149+0000 7f2bde40b740 1 -- start start 2024-09-21T04:12:26.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.149+0000 7f2bde40b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a740b1e070 0x55a740ec5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:26.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.149+0000 7f2bde40b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a740af51b0 0x55a740ec57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:26.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.149+0000 7f2bde40b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a740afd390 0x55a740ec5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:26.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.149+0000 7f2bde40b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a740e3be50 con 0x55a740afd390 2024-09-21T04:12:26.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.149+0000 7f2bde40b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a740e2b280 con 0x55a740b1e070 2024-09-21T04:12:26.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.149+0000 7f2bde40b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a740e2f090 con 0x55a740af51b0 2024-09-21T04:12:26.161 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.150+0000 7f2bd5432640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a740afd390 0x55a740ec5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:26.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.150+0000 7f2bd5432640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a740b1e070 0x55a740ec5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:26.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.150+0000 7f2bd5432640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a740b1e070 0x55a740ec5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:41628/0 (socket says 172.21.15.64:41628) 2024-09-21T04:12:26.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.150+0000 7f2bd5432640 1 -- 172.21.15.64:0/754763951 learned_addr learned my addr 172.21.15.64:0/754763951 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:26.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.150+0000 7f2bd5c33640 1 --2- 172.21.15.64:0/754763951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a740af51b0 0x55a740ec57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:26.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.150+0000 7f2bc6ffd640 1 --2- 172.21.15.64:0/2564968178 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2bbc0b66b0 0x7f2bbc0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:26.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.150+0000 7f2bd5432640 1 -- 172.21.15.64:0/754763951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a740af51b0 msgr2=0x55a740ec57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:26.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.150+0000 7f2bd5432640 1 --2- 172.21.15.64:0/754763951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a740af51b0 0x55a740ec57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.150+0000 7f2bd5432640 1 -- 172.21.15.64:0/754763951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a740afd390 msgr2=0x55a740ec5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:26.162 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.150+0000 7f2bd5432640 1 --2- 172.21.15.64:0/754763951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a740afd390 0x55a740ec5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.150+0000 7f2bd6434640 1 --2- 172.21.15.64:0/2564968178 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2bbc0b66b0 0x7f2bbc0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:26.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.150+0000 7f2bd5432640 1 -- 172.21.15.64:0/754763951 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a740b89650 con 0x55a740b1e070 2024-09-21T04:12:26.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.150+0000 7f2bc6ffd640 1 -- 172.21.15.64:0/2564968178 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7f2bc80e6c50 con 0x7f2bd0108cf0 2024-09-21T04:12:26.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.150+0000 7f2bd5432640 1 --2- 172.21.15.64:0/754763951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a740b1e070 0x55a740ec5270 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2bd006aaa0 tx=0x7f2bc0031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9953de20bceb815c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:26.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.151+0000 7f2bbbfff640 1 -- 172.21.15.64:0/754763951 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0035050 con 0x55a740b1e070 2024-09-21T04:12:26.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.151+0000 7f2bbbfff640 1 -- 172.21.15.64:0/754763951 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bc00035f0 con 0x55a740b1e070 2024-09-21T04:12:26.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.151+0000 7f2bbbfff640 1 -- 172.21.15.64:0/754763951 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc00044e0 con 0x55a740b1e070 2024-09-21T04:12:26.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.151+0000 7f2bde40b740 1 -- 172.21.15.64:0/754763951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a740b1e070 msgr2=0x55a740ec5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:26.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.151+0000 7f2bd6434640 1 --2- 172.21.15.64:0/2564968178 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2bbc0b66b0 0x7f2bbc0b8bc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80e43f0 tx=0x7f2bc80797a0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:26.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.151+0000 7f2bde40b740 1 --2- 172.21.15.64:0/754763951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a740b1e070 0x55a740ec5270 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2bd006aaa0 tx=0x7f2bc0031400 comp rx=0 tx=0).stop 2024-09-21T04:12:26.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.152+0000 7f2bde40b740 1 -- 172.21.15.64:0/754763951 shutdown_connections 2024-09-21T04:12:26.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.152+0000 7f2bde40b740 1 --2- 172.21.15.64:0/754763951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a740afd390 0x55a740ec5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.152+0000 7f2bde40b740 1 --2- 172.21.15.64:0/754763951 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a740af51b0 0x55a740ec57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.152+0000 7f2bde40b740 1 --2- 172.21.15.64:0/754763951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a740b1e070 0x55a740ec5270 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.152+0000 7f2bde40b740 1 -- 172.21.15.64:0/754763951 >> 172.21.15.64:0/754763951 conn(0x55a740b12460 msgr2=0x55a740ec3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:26.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.152+0000 7f2bde40b740 1 -- 172.21.15.64:0/754763951 shutdown_connections 2024-09-21T04:12:26.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.152+0000 7f2bde40b740 1 -- 172.21.15.64:0/754763951 wait complete. 2024-09-21T04:12:26.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.152+0000 7f2bde40b740 1 -- start start 2024-09-21T04:12:26.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.152+0000 7f2bde40b740 1 Processor -- start 2024-09-21T04:12:26.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.153+0000 7f2bde40b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a740af51b0 0x55a740ed4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:26.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.153+0000 7f2bde40b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a740afd390 0x55a740ed5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:26.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.153+0000 7f2bde40b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a740b1e070 0x55a740ed56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:26.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.153+0000 7f2bde40b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a740c8c700 con 0x55a740af51b0 2024-09-21T04:12:26.165 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.153+0000 7f2bde40b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a740dc6110 con 0x55a740afd390 2024-09-21T04:12:26.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.153+0000 7f2bde40b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a740ed7d20 con 0x55a740b1e070 2024-09-21T04:12:26.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.153+0000 7f2bde40b740 10 client.0 _ll_register_callbacks cb 0x55a740ddf990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:12:26.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.153+0000 7f2bd5432640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a740af51b0 0x55a740ed4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:26.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.153+0000 7f2bd5432640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a740af51b0 0x55a740ed4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60332/0 (socket says 172.21.15.64:60332) 2024-09-21T04:12:26.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.153+0000 7f2bd5432640 1 -- 172.21.15.64:0/2286839727 learned_addr learned my addr 172.21.15.64:0/2286839727 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:26.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.153+0000 7f2bd5c33640 1 --2- 172.21.15.64:0/2286839727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a740afd390 0x55a740ed5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:26.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.153+0000 7f2bd5432640 1 --2- 172.21.15.64:0/2286839727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a740b1e070 0x55a740ed56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:26.166 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.153+0000 7f2bd5432640 1 -- 172.21.15.64:0/2286839727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a740b1e070 msgr2=0x55a740ed56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:26.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.153+0000 7f2bd5432640 1 --2- 172.21.15.64:0/2286839727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a740b1e070 0x55a740ed56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.153+0000 7f2bd5432640 1 -- 172.21.15.64:0/2286839727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a740afd390 msgr2=0x55a740ed5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:26.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.153+0000 7f2bd5432640 1 --2- 172.21.15.64:0/2286839727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a740afd390 0x55a740ed5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.153+0000 7f2bd5432640 1 -- 172.21.15.64:0/2286839727 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a740df3c40 con 0x55a740af51b0 2024-09-21T04:12:26.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.153+0000 7f2bd5c33640 1 --2- 172.21.15.64:0/2286839727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a740afd390 0x55a740ed5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:12:26.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.153+0000 7f2bd5432640 1 --2- 172.21.15.64:0/2286839727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a740af51b0 0x55a740ed4c40 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0005d00 tx=0x7f2bc0005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491e301a9558c8c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:26.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.153+0000 7f2bb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:12:26.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.154+0000 7f2bb97fa640 1 -- 172.21.15.64:0/2286839727 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0035070 con 0x55a740af51b0 2024-09-21T04:12:26.167 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.154+0000 7f2bde40b740 1 -- 172.21.15.64:0/2286839727 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bc00035f0 con 0x55a740af51b0 2024-09-21T04:12:26.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.154+0000 7f2bb97fa640 1 -- 172.21.15.64:0/2286839727 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7f2bc0004000 con 0x55a740af51b0 2024-09-21T04:12:26.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.154+0000 7f2bb97fa640 1 -- 172.21.15.64:0/2286839727 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f2bc005a050 con 0x55a740af51b0 2024-09-21T04:12:26.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.154+0000 7f2bde40b740 10 client.17259 fetch_fsmap learned FSMap version 40 2024-09-21T04:12:26.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.154+0000 7f2bde40b740 1 -- 172.21.15.64:0/2286839727 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a740ee39a0 con 0x55a740af51b0 2024-09-21T04:12:26.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.154+0000 7f2bb97fa640 1 -- 172.21.15.64:0/2286839727 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bc00035f0 con 0x55a740af51b0 2024-09-21T04:12:26.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.154+0000 7f2bb97fa640 1 -- 172.21.15.64:0/2286839727 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f2bc0004750 con 0x55a740af51b0 2024-09-21T04:12:26.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.154+0000 7f2bde40b740 10 client.17259 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:12:26.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.154+0000 7f2bde40b740 10 client.17259 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:12:26.168 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.154+0000 7f2bde40b740 20 client.17259 populate_metadata read hostname 'smithi064' 2024-09-21T04:12:26.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.154+0000 7f2bde40b740 1 --2- 172.21.15.64:0/2286839727 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a740adc9f0 0x55a740ee6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:26.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.154+0000 7f2bde40b740 4 client.17259 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:12:26.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.154+0000 7f2bde40b740 1 -- 172.21.15.64:0/2286839727 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a740e3dda0 con 0x55a740adc9f0 2024-09-21T04:12:26.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.155+0000 7f2bd5c33640 1 --2- 172.21.15.64:0/2286839727 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a740adc9f0 0x55a740ee6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:26.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.155+0000 7f2bd5c33640 1 --2- 172.21.15.64:0/2286839727 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a740adc9f0 0x55a740ee6d00 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3782debd83d01784 server_cookie=92ae532e1af6f74c in_seq=0 out_seq=0 2024-09-21T04:12:26.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.155+0000 7f2bb97fa640 10 client.17259 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:12:26.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.156+0000 7f2bb97fa640 1 -- 172.21.15.64:0/2286839727 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a740e3dda0 con 0x55a740adc9f0 2024-09-21T04:12:26.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.156+0000 7f2bb97fa640 10 client.17259 handle_command_reply: tid=0 2024-09-21T04:12:26.169 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.156+0000 7f2bde40b740 1 client.17259 shutdown 2024-09-21T04:12:26.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.156+0000 7f2bde40b740 1 -- 172.21.15.64:0/2286839727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a740af51b0 msgr2=0x55a740ed4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:26.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.156+0000 7f2bde40b740 1 --2- 172.21.15.64:0/2286839727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a740af51b0 0x55a740ed4c40 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0005d00 tx=0x7f2bc0005c90 comp rx=0 tx=0).stop 2024-09-21T04:12:26.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.156+0000 7f2bde40b740 1 -- 172.21.15.64:0/2286839727 shutdown_connections 2024-09-21T04:12:26.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.156+0000 7f2bde40b740 1 --2- 172.21.15.64:0/2286839727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a740b1e070 0x55a740ed56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.156+0000 7f2bde40b740 1 --2- 172.21.15.64:0/2286839727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a740afd390 0x55a740ed5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.156+0000 7f2bde40b740 1 --2- 172.21.15.64:0/2286839727 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a740adc9f0 0x55a740ee6d00 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.156+0000 7f2bde40b740 1 --2- 172.21.15.64:0/2286839727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a740af51b0 0x55a740ed4c40 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.156+0000 7f2bde40b740 1 -- 172.21.15.64:0/2286839727 >> 172.21.15.64:0/2286839727 conn(0x55a740b12460 msgr2=0x55a740ec8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:26.170 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.156+0000 7f2bb97fa640 0 client.17259 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:12:26.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.156+0000 7f2bde40b740 1 -- 172.21.15.64:0/2286839727 shutdown_connections 2024-09-21T04:12:26.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.156+0000 7f2bde40b740 1 -- 172.21.15.64:0/2286839727 wait complete. 2024-09-21T04:12:26.171 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.156+0000 7f2bde40b740 20 client.17259 trim_cache size 0 max 16384 2024-09-21T04:12:26.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.174+0000 7f2bde40b740 1 Processor -- start 2024-09-21T04:12:26.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.174+0000 7f2bde40b740 1 -- start start 2024-09-21T04:12:26.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.174+0000 7f2bde40b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a740af51b0 0x55a740ec8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:26.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.174+0000 7f2bde40b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a740afd390 0x55a740ec2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:26.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.174+0000 7f2bde40b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a740b1e070 0x55a740ec2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:26.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.174+0000 7f2bde40b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd01b0140 con 0x55a740b1e070 2024-09-21T04:12:26.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.174+0000 7f2bde40b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd01ad950 con 0x55a740af51b0 2024-09-21T04:12:26.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.174+0000 7f2bde40b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd01ab7c0 con 0x55a740afd390 2024-09-21T04:12:26.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.174+0000 7f2bd5432640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a740b1e070 0x55a740ec2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:26.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.174+0000 7f2bd5432640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a740b1e070 0x55a740ec2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60336/0 (socket says 172.21.15.64:60336) 2024-09-21T04:12:26.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.174+0000 7f2bd5432640 1 -- 172.21.15.64:0/924527950 learned_addr learned my addr 172.21.15.64:0/924527950 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:26.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.174+0000 7f2bd5c33640 1 --2- 172.21.15.64:0/924527950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a740afd390 0x55a740ec2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:26.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.175+0000 7f2bd5432640 1 --2- 172.21.15.64:0/924527950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a740af51b0 0x55a740ec8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:26.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.175+0000 7f2bd5432640 1 -- 172.21.15.64:0/924527950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a740afd390 msgr2=0x55a740ec2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:26.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.175+0000 7f2bd5432640 1 --2- 172.21.15.64:0/924527950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a740afd390 0x55a740ec2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.175+0000 7f2bd5432640 1 -- 172.21.15.64:0/924527950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a740af51b0 msgr2=0x55a740ec8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:26.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.175+0000 7f2bd5432640 1 --2- 172.21.15.64:0/924527950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a740af51b0 0x55a740ec8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.175+0000 7f2bd5432640 1 -- 172.21.15.64:0/924527950 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a740ec3690 con 0x55a740b1e070 2024-09-21T04:12:26.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.175+0000 7f2bd5432640 1 --2- 172.21.15.64:0/924527950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a740b1e070 0x55a740ec2c50 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0000c00 tx=0x7f2bc0031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c65ace93cf9fef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:26.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.175+0000 7f2bbb7fe640 1 -- 172.21.15.64:0/924527950 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0035070 con 0x55a740b1e070 2024-09-21T04:12:26.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.175+0000 7f2bbb7fe640 1 -- 172.21.15.64:0/924527950 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bc00035f0 con 0x55a740b1e070 2024-09-21T04:12:26.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.175+0000 7f2bbb7fe640 1 -- 172.21.15.64:0/924527950 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0031b90 con 0x55a740b1e070 2024-09-21T04:12:26.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.176+0000 7f2bde40b740 1 -- 172.21.15.64:0/924527950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a740b1e070 msgr2=0x55a740ec2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:26.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.176+0000 7f2bde40b740 1 --2- 172.21.15.64:0/924527950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a740b1e070 0x55a740ec2c50 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0000c00 tx=0x7f2bc0031400 comp rx=0 tx=0).stop 2024-09-21T04:12:26.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.176+0000 7f2bde40b740 1 -- 172.21.15.64:0/924527950 shutdown_connections 2024-09-21T04:12:26.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.176+0000 7f2bde40b740 1 --2- 172.21.15.64:0/924527950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a740b1e070 0x55a740ec2c50 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.176+0000 7f2bde40b740 1 --2- 172.21.15.64:0/924527950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a740afd390 0x55a740ec2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.176+0000 7f2bde40b740 1 --2- 172.21.15.64:0/924527950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a740af51b0 0x55a740ec8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.176+0000 7f2bde40b740 1 -- 172.21.15.64:0/924527950 >> 172.21.15.64:0/924527950 conn(0x55a740b12460 msgr2=0x55a740ed6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:26.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.176+0000 7f2bde40b740 1 -- 172.21.15.64:0/924527950 shutdown_connections 2024-09-21T04:12:26.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.176+0000 7f2bde40b740 1 -- 172.21.15.64:0/924527950 wait complete. 2024-09-21T04:12:26.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.176+0000 7f2bde40b740 1 -- start start 2024-09-21T04:12:26.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.176+0000 7f2bde40b740 1 Processor -- start 2024-09-21T04:12:26.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.177+0000 7f2bde40b740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a740af51b0 0x55a740eea6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:26.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.177+0000 7f2bde40b740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a740afd390 0x55a740ee7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:26.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.177+0000 7f2bde40b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a740b1e070 0x55a740ee8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:26.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.177+0000 7f2bde40b740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd01abe10 con 0x55a740af51b0 2024-09-21T04:12:26.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.177+0000 7f2bde40b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd01ad280 con 0x55a740b1e070 2024-09-21T04:12:26.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.177+0000 7f2bde40b740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd01ac4e0 con 0x55a740afd390 2024-09-21T04:12:26.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.177+0000 7f2bde40b740 10 client.0 _ll_register_callbacks cb 0x7f2bc0032e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:12:26.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.177+0000 7f2bd5432640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a740af51b0 0x55a740eea6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:26.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.177+0000 7f2bd5432640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a740af51b0 0x55a740eea6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60352/0 (socket says 172.21.15.64:60352) 2024-09-21T04:12:26.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.177+0000 7f2bd5432640 1 -- 172.21.15.64:0/2157000525 learned_addr learned my addr 172.21.15.64:0/2157000525 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:26.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.177+0000 7f2bd5c33640 1 --2- 172.21.15.64:0/2157000525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a740afd390 0x55a740ee7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:26.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.177+0000 7f2bd5432640 1 -- 172.21.15.64:0/2157000525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a740afd390 msgr2=0x55a740ee7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:26.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.177+0000 7f2bd5432640 1 --2- 172.21.15.64:0/2157000525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a740afd390 0x55a740ee7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.177+0000 7f2bd5432640 1 -- 172.21.15.64:0/2157000525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a740b1e070 msgr2=0x55a740ee8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:26.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.177+0000 7f2bd5432640 1 --2- 172.21.15.64:0/2157000525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a740b1e070 0x55a740ee8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.177+0000 7f2bd5432640 1 -- 172.21.15.64:0/2157000525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a740f50480 con 0x55a740af51b0 2024-09-21T04:12:26.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.178+0000 7f2bd5432640 1 --2- 172.21.15.64:0/2157000525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a740af51b0 0x55a740eea6c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0001eb0 tx=0x7f2bc0032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05bfb386c26f7dd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:26.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.178+0000 7f2bb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:12:26.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.178+0000 7f2bb97fa640 1 -- 172.21.15.64:0/2157000525 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0064020 con 0x55a740af51b0 2024-09-21T04:12:26.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.178+0000 7f2bb97fa640 1 -- 172.21.15.64:0/2157000525 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7f2bc0003f80 con 0x55a740af51b0 2024-09-21T04:12:26.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.178+0000 7f2bde40b740 1 -- 172.21.15.64:0/2157000525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bc00035f0 con 0x55a740af51b0 2024-09-21T04:12:26.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.178+0000 7f2bb97fa640 1 -- 172.21.15.64:0/2157000525 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f2bc0035070 con 0x55a740af51b0 2024-09-21T04:12:26.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.178+0000 7f2bde40b740 10 client.17265 fetch_fsmap learned FSMap version 40 2024-09-21T04:12:26.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.178+0000 7f2bde40b740 1 -- 172.21.15.64:0/2157000525 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a740f51580 con 0x55a740af51b0 2024-09-21T04:12:26.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.178+0000 7f2bb97fa640 1 -- 172.21.15.64:0/2157000525 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bc00035f0 con 0x55a740af51b0 2024-09-21T04:12:26.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.178+0000 7f2bb97fa640 1 -- 172.21.15.64:0/2157000525 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f2bc0004f70 con 0x55a740af51b0 2024-09-21T04:12:26.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.178+0000 7f2bde40b740 10 client.17265 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:12:26.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.178+0000 7f2bde40b740 10 client.17265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:12:26.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.178+0000 7f2bde40b740 20 client.17265 populate_metadata read hostname 'smithi064' 2024-09-21T04:12:26.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.178+0000 7f2bde40b740 1 --2- 172.21.15.64:0/2157000525 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a740ab1da0 0x55a740f53ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:26.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.178+0000 7f2bde40b740 4 client.17265 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T04:12:26.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.178+0000 7f2bde40b740 1 -- 172.21.15.64:0/2157000525 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a740e39850 con 0x55a740ab1da0 2024-09-21T04:12:26.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.179+0000 7f2bd5c33640 1 --2- 172.21.15.64:0/2157000525 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a740ab1da0 0x55a740f53ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:26.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.179+0000 7f2bd5c33640 1 --2- 172.21.15.64:0/2157000525 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a740ab1da0 0x55a740f53ab0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a8ad78e5871f039 server_cookie=2ce180a7de05d9e6 in_seq=0 out_seq=0 2024-09-21T04:12:26.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.179+0000 7f2bb97fa640 10 client.17265 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:12:26.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.179+0000 7f2bb97fa640 1 -- 172.21.15.64:0/2157000525 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3694 (crc 0 0 0) 0x55a740e39850 con 0x55a740ab1da0 2024-09-21T04:12:26.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.179+0000 7f2bb97fa640 10 client.17265 handle_command_reply: tid=0 2024-09-21T04:12:26.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.179+0000 7f2bde40b740 1 client.17265 shutdown 2024-09-21T04:12:26.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.180+0000 7f2bde40b740 1 -- 172.21.15.64:0/2157000525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a740af51b0 msgr2=0x55a740eea6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:26.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.180+0000 7f2bde40b740 1 --2- 172.21.15.64:0/2157000525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a740af51b0 0x55a740eea6c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0001eb0 tx=0x7f2bc0032f60 comp rx=0 tx=0).stop 2024-09-21T04:12:26.186 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.180+0000 7f2bde40b740 1 -- 172.21.15.64:0/2157000525 shutdown_connections 2024-09-21T04:12:26.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.180+0000 7f2bde40b740 1 --2- 172.21.15.64:0/2157000525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a740b1e070 0x55a740ee8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.180+0000 7f2bde40b740 1 --2- 172.21.15.64:0/2157000525 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a740afd390 0x55a740ee7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.180+0000 7f2bde40b740 1 --2- 172.21.15.64:0/2157000525 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55a740ab1da0 0x55a740f53ab0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.180+0000 7f2bde40b740 1 --2- 172.21.15.64:0/2157000525 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a740af51b0 0x55a740eea6c0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.180+0000 7f2bde40b740 1 -- 172.21.15.64:0/2157000525 >> 172.21.15.64:0/2157000525 conn(0x55a740b12460 msgr2=0x55a740edffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:26.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.180+0000 7f2bb97fa640 0 client.17265 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:12:26.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.180+0000 7f2bde40b740 1 -- 172.21.15.64:0/2157000525 shutdown_connections 2024-09-21T04:12:26.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.180+0000 7f2bde40b740 1 -- 172.21.15.64:0/2157000525 wait complete. 2024-09-21T04:12:26.187 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.180+0000 7f2bde40b740 20 client.17265 trim_cache size 0 max 16384 2024-09-21T04:12:26.187 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:12:26.188 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T04:12:26.188 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-09-21T04:12:26.188 INFO:teuthology.orchestra.run.smithi064.stdout: "dd25622a-26f0-4838-b998-85d0603c83b6": { 2024-09-21T04:12:26.188 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:26.188 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "dd25622a-26f0-4838-b998-85d0603c83b6", 2024-09-21T04:12:26.188 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-09-21T04:12:26.188 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:12:26.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:26.188 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_stats": { 2024-09-21T04:12:26.188 INFO:teuthology.orchestra.run.smithi064.stdout: "1b23713d-ea3a-4a86-a87f-65f6ab82671d": { 2024-09-21T04:12:26.188 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:10:58.352639+0000", 2024-09-21T04:12:26.189 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:26.189 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:26.189 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:26.189 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:26.189 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:26.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:26.189 INFO:teuthology.orchestra.run.smithi064.stdout: "dd25622a-26f0-4838-b998-85d0603c83b6": { 2024-09-21T04:12:26.189 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:12:24.553513+0000", 2024-09-21T04:12:26.189 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:26.189 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:26.189 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:26.189 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:26.189 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:26.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:26.189 INFO:teuthology.orchestra.run.smithi064.stdout: "71f3f255-4837-45e9-92bc-4b3abb5c20eb": { 2024-09-21T04:12:26.190 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:11:42.378549+0000", 2024-09-21T04:12:26.190 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:26.190 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:26.190 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:26.190 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:26.190 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:26.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:26.190 INFO:teuthology.orchestra.run.smithi064.stdout: "2c0523fb-aae4-43fa-9568-136a9b640051": { 2024-09-21T04:12:26.190 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:09:39.808606+0000", 2024-09-21T04:12:26.190 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:26.190 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:26.190 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:26.190 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:26.190 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:26.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:26.191 INFO:teuthology.orchestra.run.smithi064.stdout: "bed40d88-2102-43a6-ac2d-b89d5a2264d0": { 2024-09-21T04:12:26.191 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:11.363340+0000", 2024-09-21T04:12:26.191 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:26.191 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:26.191 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:26.191 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:26.191 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:26.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:26.192 INFO:teuthology.orchestra.run.smithi064.stdout: "e9242f88-6599-40eb-adf1-bdb3e946a1e0": { 2024-09-21T04:12:26.192 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:05:39.744646+0000", 2024-09-21T04:12:26.192 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:26.192 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:26.192 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:26.192 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:26.192 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:26.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:26.193 INFO:teuthology.orchestra.run.smithi064.stdout: "6bde0fe5-f775-44b1-ad0c-f9e596574dc0": { 2024-09-21T04:12:26.193 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:06:23.835536+0000", 2024-09-21T04:12:26.193 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:26.193 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:26.193 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:26.193 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:26.193 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:26.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:26.193 INFO:teuthology.orchestra.run.smithi064.stdout: "b52b950c-0f97-49ec-a37c-1a8e04348ce8": { 2024-09-21T04:12:26.193 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:54.003800+0000", 2024-09-21T04:12:26.193 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:26.193 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:26.193 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:26.193 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:26.194 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:26.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:26.194 INFO:teuthology.orchestra.run.smithi064.stdout: "e2c2eed8-9292-4bf6-8a39-13ad27da5acb": { 2024-09-21T04:12:26.194 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:08:47.067932+0000", 2024-09-21T04:12:26.194 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:26.194 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:26.194 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:26.194 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:26.194 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:26.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:26.194 INFO:teuthology.orchestra.run.smithi064.stdout: "75db1bc5-612e-45aa-bad2-cc90feeaa840": { 2024-09-21T04:12:26.195 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:59.123138+0000", 2024-09-21T04:12:26.195 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:26.195 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:26.195 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:26.195 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:26.195 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:26.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:26.195 INFO:teuthology.orchestra.run.smithi064.stdout: "0ab717a7-36c6-4dca-94e7-af0bdf2aa112": { 2024-09-21T04:12:26.195 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:42.691086+0000", 2024-09-21T04:12:26.195 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:26.195 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:26.195 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:26.196 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:26.196 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:26.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:26.196 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:12:26.196 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:03:59.570174+0000", 2024-09-21T04:12:26.196 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:26.196 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:26.196 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:26.196 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:26.196 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:26.196 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:12:26.196 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:12:26.196 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:12:26.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.181+0000 7f2bd7e97640 1 -- 172.21.15.64:0/2564968178 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2bbc0b66b0 msgr2=0x7f2bbc0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:26.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.181+0000 7f2bd7e97640 1 --2- 172.21.15.64:0/2564968178 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2bbc0b66b0 0x7f2bbc0b8bc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80e43f0 tx=0x7f2bc80797a0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.181+0000 7f2bd7e97640 1 -- 172.21.15.64:0/2564968178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bd0108cf0 msgr2=0x7f2bd019be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:26.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.181+0000 7f2bd7e97640 1 --2- 172.21.15.64:0/2564968178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bd0108cf0 0x7f2bd019be70 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800b970 tx=0x7f2bc800be30 comp rx=0 tx=0).stop 2024-09-21T04:12:26.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.181+0000 7f2bd7e97640 1 -- 172.21.15.64:0/2564968178 shutdown_connections 2024-09-21T04:12:26.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.181+0000 7f2bd7e97640 1 --2- 172.21.15.64:0/2564968178 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2bbc0b66b0 0x7f2bbc0b8bc0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.181+0000 7f2bd7e97640 1 --2- 172.21.15.64:0/2564968178 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2bd0108cf0 0x7f2bd019be70 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.181+0000 7f2bd7e97640 1 --2- 172.21.15.64:0/2564968178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2bd01083a0 0x7f2bd019b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.181+0000 7f2bd7e97640 1 --2- 172.21.15.64:0/2564968178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bd0102a60 0x7f2bd019b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:26.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.181+0000 7f2bd7e97640 1 -- 172.21.15.64:0/2564968178 >> 172.21.15.64:0/2564968178 conn(0x7f2bd0079730 msgr2=0x7f2bd0105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:26.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.181+0000 7f2bd7e97640 1 -- 172.21.15.64:0/2564968178 shutdown_connections 2024-09-21T04:12:26.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:26.181+0000 7f2bd7e97640 1 -- 172.21.15.64:0/2564968178 wait complete. 2024-09-21T04:12:27.104 INFO:tasks.cephfs.filesystem:scrub status for tag:dd25622a-26f0-4838-b998-85d0603c83b6 - {'path': '/', 'tag': 'dd25622a-26f0-4838-b998-85d0603c83b6', 'options': 'recursive,force'} 2024-09-21T04:12:28.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:27 smithi064 ceph-mon[31180]: pgmap v580: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 97 B/s wr, 8 op/s 2024-09-21T04:12:28.014 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:27 smithi119 ceph-mon[34811]: pgmap v580: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 97 B/s wr, 8 op/s 2024-09-21T04:12:28.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:27 smithi131 ceph-mon[35193]: pgmap v580: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 97 B/s wr, 8 op/s 2024-09-21T04:12:29.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:28 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:12:29.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:28 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:12:29.055 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:28 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:12:30.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:29 smithi119 ceph-mon[34811]: pgmap v581: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T04:12:30.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:29 smithi064 ceph-mon[31180]: pgmap v581: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T04:12:30.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:29 smithi131 ceph-mon[35193]: pgmap v581: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T04:12:32.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:31 smithi119 ceph-mon[34811]: pgmap v582: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 7 op/s 2024-09-21T04:12:32.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:31 smithi064 ceph-mon[31180]: pgmap v582: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 7 op/s 2024-09-21T04:12:32.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:31 smithi131 ceph-mon[35193]: pgmap v582: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 7 op/s 2024-09-21T04:12:34.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:33 smithi119 ceph-mon[34811]: pgmap v583: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 10 op/s 2024-09-21T04:12:34.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:33 smithi064 ceph-mon[31180]: pgmap v583: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 10 op/s 2024-09-21T04:12:34.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:33 smithi131 ceph-mon[35193]: pgmap v583: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 10 op/s 2024-09-21T04:12:36.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:35 smithi119 ceph-mon[34811]: pgmap v584: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 7 op/s 2024-09-21T04:12:36.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:35 smithi064 ceph-mon[31180]: pgmap v584: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 7 op/s 2024-09-21T04:12:36.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:35 smithi131 ceph-mon[35193]: pgmap v584: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 7 op/s 2024-09-21T04:12:38.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:37 smithi119 ceph-mon[34811]: pgmap v585: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 8 op/s 2024-09-21T04:12:38.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:37 smithi064 ceph-mon[31180]: pgmap v585: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 8 op/s 2024-09-21T04:12:38.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:37 smithi131 ceph-mon[35193]: pgmap v585: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 8 op/s 2024-09-21T04:12:40.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:39 smithi119 ceph-mon[34811]: pgmap v586: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 7 op/s 2024-09-21T04:12:40.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:39 smithi064 ceph-mon[31180]: pgmap v586: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 7 op/s 2024-09-21T04:12:40.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:39 smithi131 ceph-mon[35193]: pgmap v586: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 7 op/s 2024-09-21T04:12:42.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:41 smithi119 ceph-mon[34811]: pgmap v587: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:12:42.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:41 smithi064 ceph-mon[31180]: pgmap v587: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:12:42.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:41 smithi131 ceph-mon[35193]: pgmap v587: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:12:44.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:43 smithi119 ceph-mon[34811]: pgmap v588: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T04:12:44.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:43 smithi064 ceph-mon[31180]: pgmap v588: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T04:12:44.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:43 smithi131 ceph-mon[35193]: pgmap v588: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T04:12:46.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:45 smithi119 ceph-mon[34811]: pgmap v589: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:12:46.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:45 smithi064 ceph-mon[31180]: pgmap v589: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:12:46.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:45 smithi131 ceph-mon[35193]: pgmap v589: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:12:48.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:47 smithi119 ceph-mon[34811]: pgmap v590: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T04:12:48.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:47 smithi064 ceph-mon[31180]: pgmap v590: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T04:12:48.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:47 smithi131 ceph-mon[35193]: pgmap v590: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T04:12:50.012 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:49 smithi064 ceph-mon[31180]: pgmap v591: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T04:12:50.054 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:49 smithi131 ceph-mon[35193]: pgmap v591: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T04:12:50.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:49 smithi119 ceph-mon[34811]: pgmap v591: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T04:12:52.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:51 smithi131 ceph-mon[35193]: pgmap v592: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:12:52.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:51 smithi119 ceph-mon[34811]: pgmap v592: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:12:52.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:51 smithi064 ceph-mon[31180]: pgmap v592: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:12:54.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:53 smithi131 ceph-mon[35193]: pgmap v593: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T04:12:54.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:53 smithi119 ceph-mon[34811]: pgmap v593: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T04:12:54.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:53 smithi064 ceph-mon[31180]: pgmap v593: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T04:12:56.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:55 smithi131 ceph-mon[35193]: pgmap v594: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:12:56.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:55 smithi119 ceph-mon[34811]: pgmap v594: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:12:56.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:55 smithi064 ceph-mon[31180]: pgmap v594: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:12:57.106 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T04:12:57.383 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:12:58.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:57 smithi131 ceph-mon[35193]: pgmap v595: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T04:12:58.099 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:57 smithi064 ceph-mon[31180]: pgmap v595: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T04:12:58.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.096+0000 7f2dd8d1a640 1 -- 172.21.15.64:0/1631283701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd4108cf0 msgr2=0x7f2dd410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:58.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.096+0000 7f2dd8d1a640 1 --2- 172.21.15.64:0/1631283701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd4108cf0 0x7f2dd410f520 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009ca0 tx=0x7f2dcc02a8c0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.096+0000 7f2dd8d1a640 1 -- 172.21.15.64:0/1631283701 shutdown_connections 2024-09-21T04:12:58.120 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.096+0000 7f2dd8d1a640 1 --2- 172.21.15.64:0/1631283701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd4108cf0 0x7f2dd410f520 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.096+0000 7f2dd8d1a640 1 --2- 172.21.15.64:0/1631283701 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dd41083a0 0x7f2dd41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.096+0000 7f2dd8d1a640 1 --2- 172.21.15.64:0/1631283701 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2dd4102a60 0x7f2dd4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.096+0000 7f2dd8d1a640 1 -- 172.21.15.64:0/1631283701 >> 172.21.15.64:0/1631283701 conn(0x7f2dd4079730 msgr2=0x7f2dd4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:58.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.096+0000 7f2dd8d1a640 1 -- 172.21.15.64:0/1631283701 shutdown_connections 2024-09-21T04:12:58.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.096+0000 7f2dd8d1a640 1 -- 172.21.15.64:0/1631283701 wait complete. 2024-09-21T04:12:58.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.097+0000 7f2dd8d1a640 1 Processor -- start 2024-09-21T04:12:58.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.097+0000 7f2dd8d1a640 1 -- start start 2024-09-21T04:12:58.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.098+0000 7f2dd8d1a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd4102a60 0x7f2dd40ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:58.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.098+0000 7f2dd8d1a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2dd41083a0 0x7f2dd40ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:58.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.098+0000 7f2dd8d1a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dd4108cf0 0x7f2dd41000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:58.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.098+0000 7f2dd8d1a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd4112a10 con 0x7f2dd4102a60 2024-09-21T04:12:58.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.098+0000 7f2dd8d1a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd4112890 con 0x7f2dd4108cf0 2024-09-21T04:12:58.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.098+0000 7f2dd8d1a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd4112b90 con 0x7f2dd41083a0 2024-09-21T04:12:58.121 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.098+0000 7f2dd259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd4102a60 0x7f2dd40ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:58.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.098+0000 7f2dd259c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd4102a60 0x7f2dd40ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39704/0 (socket says 172.21.15.64:39704) 2024-09-21T04:12:58.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.098+0000 7f2dd259c640 1 -- 172.21.15.64:0/325411239 learned_addr learned my addr 172.21.15.64:0/325411239 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:58.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.098+0000 7f2dd2d9d640 1 --2- 172.21.15.64:0/325411239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dd4108cf0 0x7f2dd41000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:58.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.098+0000 7f2dcbfff640 1 --2- 172.21.15.64:0/325411239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2dd41083a0 0x7f2dd40ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:58.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.098+0000 7f2dd259c640 1 -- 172.21.15.64:0/325411239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2dd41083a0 msgr2=0x7f2dd40ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:58.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.098+0000 7f2dd259c640 1 --2- 172.21.15.64:0/325411239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2dd41083a0 0x7f2dd40ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.098+0000 7f2dd259c640 1 -- 172.21.15.64:0/325411239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dd4108cf0 msgr2=0x7f2dd41000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:58.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.098+0000 7f2dd259c640 1 --2- 172.21.15.64:0/325411239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dd4108cf0 0x7f2dd41000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.098+0000 7f2dd259c640 1 -- 172.21.15.64:0/325411239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd41a8710 con 0x7f2dd4102a60 2024-09-21T04:12:58.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.098+0000 7f2dd259c640 1 --2- 172.21.15.64:0/325411239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd4102a60 0x7f2dd40ff670 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400b4a0 tx=0x7f2dc400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:58.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.099+0000 7f2dcb7fe640 1 -- 172.21.15.64:0/325411239 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4014070 con 0x7f2dd4102a60 2024-09-21T04:12:58.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.099+0000 7f2dcb7fe640 1 -- 172.21.15.64:0/325411239 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dc4010ac0 con 0x7f2dd4102a60 2024-09-21T04:12:58.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.099+0000 7f2dd8d1a640 1 -- 172.21.15.64:0/325411239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dd41a8a00 con 0x7f2dd4102a60 2024-09-21T04:12:58.122 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.099+0000 7f2dcb7fe640 1 -- 172.21.15.64:0/325411239 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4017c30 con 0x7f2dd4102a60 2024-09-21T04:12:58.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.099+0000 7f2dd8d1a640 1 -- 172.21.15.64:0/325411239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dd41a8fc0 con 0x7f2dd4102a60 2024-09-21T04:12:58.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.101+0000 7f2ddf28e740 1 Processor -- start 2024-09-21T04:12:58.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.101+0000 7f2ddf28e740 1 -- start start 2024-09-21T04:12:58.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.102+0000 7f2ddf28e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea3ec12070 0x55ea3efb9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:58.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.102+0000 7f2dcb7fe640 1 -- 172.21.15.64:0/325411239 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f2dc4017070 con 0x7f2dd4102a60 2024-09-21T04:12:58.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.102+0000 7f2ddf28e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea3ebe91b0 0x55ea3efb97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:58.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.102+0000 7f2ddf28e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ebf1390 0x55ea3efb9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:58.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.102+0000 7f2ddf28e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ea3ef2fe50 con 0x55ea3ebf1390 2024-09-21T04:12:58.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.102+0000 7f2ddf28e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ea3ef1f280 con 0x55ea3ebe91b0 2024-09-21T04:12:58.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.102+0000 7f2ddf28e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ea3ef23090 con 0x55ea3ec12070 2024-09-21T04:12:58.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.102+0000 7f2dcbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ebf1390 0x55ea3efb9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:58.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.102+0000 7f2dcbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ebf1390 0x55ea3efb9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39716/0 (socket says 172.21.15.64:39716) 2024-09-21T04:12:58.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.102+0000 7f2dcbfff640 1 -- 172.21.15.64:0/2778719485 learned_addr learned my addr 172.21.15.64:0/2778719485 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:58.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.102+0000 7f2dd2d9d640 1 --2- 172.21.15.64:0/2778719485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea3ec12070 0x55ea3efb9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:58.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.102+0000 7f2dd259c640 1 --2- 172.21.15.64:0/2778719485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea3ebe91b0 0x55ea3efb97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:58.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.102+0000 7f2dcbfff640 1 -- 172.21.15.64:0/2778719485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea3ec12070 msgr2=0x55ea3efb9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:58.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.102+0000 7f2dcbfff640 1 --2- 172.21.15.64:0/2778719485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea3ec12070 0x55ea3efb9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.102+0000 7f2dcbfff640 1 -- 172.21.15.64:0/2778719485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea3ebe91b0 msgr2=0x55ea3efb97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:58.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.102+0000 7f2dcbfff640 1 --2- 172.21.15.64:0/2778719485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea3ebe91b0 0x55ea3efb97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.102+0000 7f2dcbfff640 1 -- 172.21.15.64:0/2778719485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea3ec7d650 con 0x55ea3ebf1390 2024-09-21T04:12:58.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.102+0000 7f2dcbfff640 1 --2- 172.21.15.64:0/2778719485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ebf1390 0x55ea3efb9cf0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc00bc70 tx=0x7f2dbc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48216db7f50586b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:58.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.103+0000 7f2dcb7fe640 1 --2- 172.21.15.64:0/325411239 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2dc00b6680 0x7f2dc00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:58.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.103+0000 7f2dc8ff9640 1 -- 172.21.15.64:0/2778719485 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc005220 con 0x55ea3ebf1390 2024-09-21T04:12:58.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.103+0000 7f2dc8ff9640 1 -- 172.21.15.64:0/2778719485 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dbc00be20 con 0x55ea3ebf1390 2024-09-21T04:12:58.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.103+0000 7f2dc8ff9640 1 -- 172.21.15.64:0/2778719485 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc011640 con 0x55ea3ebf1390 2024-09-21T04:12:58.124 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.103+0000 7f2dd2d9d640 1 --2- 172.21.15.64:0/325411239 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2dc00b6680 0x7f2dc00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:58.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.103+0000 7f2dcb7fe640 1 -- 172.21.15.64:0/325411239 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7f2dc40e6cd0 con 0x7f2dd4102a60 2024-09-21T04:12:58.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.104+0000 7f2ddf28e740 1 -- 172.21.15.64:0/2778719485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ebf1390 msgr2=0x55ea3efb9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:58.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.104+0000 7f2dd2d9d640 1 --2- 172.21.15.64:0/325411239 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2dc00b6680 0x7f2dc00b8b90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc006010 tx=0x7f2dcc005d80 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:58.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.104+0000 7f2ddf28e740 1 --2- 172.21.15.64:0/2778719485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ebf1390 0x55ea3efb9cf0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc00bc70 tx=0x7f2dbc0107a0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.104+0000 7f2ddf28e740 1 -- 172.21.15.64:0/2778719485 shutdown_connections 2024-09-21T04:12:58.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.104+0000 7f2ddf28e740 1 --2- 172.21.15.64:0/2778719485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ebf1390 0x55ea3efb9cf0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.104+0000 7f2ddf28e740 1 --2- 172.21.15.64:0/2778719485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea3ebe91b0 0x55ea3efb97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.104+0000 7f2ddf28e740 1 --2- 172.21.15.64:0/2778719485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea3ec12070 0x55ea3efb9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.104+0000 7f2ddf28e740 1 -- 172.21.15.64:0/2778719485 >> 172.21.15.64:0/2778719485 conn(0x55ea3ec06460 msgr2=0x55ea3efb7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:58.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.104+0000 7f2ddf28e740 1 -- 172.21.15.64:0/2778719485 shutdown_connections 2024-09-21T04:12:58.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.104+0000 7f2ddf28e740 1 -- 172.21.15.64:0/2778719485 wait complete. 2024-09-21T04:12:58.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.104+0000 7f2ddf28e740 1 -- start start 2024-09-21T04:12:58.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.104+0000 7f2ddf28e740 1 Processor -- start 2024-09-21T04:12:58.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.105+0000 7f2ddf28e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea3ebe91b0 0x55ea3efc4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:58.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.105+0000 7f2ddf28e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea3ebf1390 0x55ea3efc4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:58.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.105+0000 7f2ddf28e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ec12070 0x55ea3efd0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:58.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.105+0000 7f2ddf28e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ea3ed80700 con 0x55ea3ec12070 2024-09-21T04:12:58.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.105+0000 7f2ddf28e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ea3eeba110 con 0x55ea3ebf1390 2024-09-21T04:12:58.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.105+0000 7f2ddf28e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ea3efc7a50 con 0x55ea3ebe91b0 2024-09-21T04:12:58.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.105+0000 7f2ddf28e740 10 client.0 _ll_register_callbacks cb 0x55ea3eed3990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:12:58.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.105+0000 7f2dd2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ec12070 0x55ea3efd0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:58.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.105+0000 7f2dd2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ec12070 0x55ea3efd0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39722/0 (socket says 172.21.15.64:39722) 2024-09-21T04:12:58.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.105+0000 7f2dd2d9d640 1 -- 172.21.15.64:0/3095346984 learned_addr learned my addr 172.21.15.64:0/3095346984 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:58.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.105+0000 7f2dcbfff640 1 --2- 172.21.15.64:0/3095346984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea3ebf1390 0x55ea3efc4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:58.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.105+0000 7f2dd259c640 1 --2- 172.21.15.64:0/3095346984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea3ebe91b0 0x55ea3efc4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:58.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.105+0000 7f2dd2d9d640 1 -- 172.21.15.64:0/3095346984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea3ebe91b0 msgr2=0x55ea3efc4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:58.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.105+0000 7f2dd2d9d640 1 --2- 172.21.15.64:0/3095346984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea3ebe91b0 0x55ea3efc4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.105+0000 7f2dd2d9d640 1 -- 172.21.15.64:0/3095346984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea3ebf1390 msgr2=0x55ea3efc4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:58.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.105+0000 7f2dd2d9d640 1 --2- 172.21.15.64:0/3095346984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea3ebf1390 0x55ea3efc4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.105+0000 7f2dd2d9d640 1 -- 172.21.15.64:0/3095346984 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea3eee7c40 con 0x55ea3ec12070 2024-09-21T04:12:58.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.105+0000 7f2dd2d9d640 1 --2- 172.21.15.64:0/3095346984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ec12070 0x55ea3efd0d40 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc035000 tx=0x7f2dcc02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1adc02665905423 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:58.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.105+0000 7f2db9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:12:58.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.106+0000 7f2db9ffb640 1 -- 172.21.15.64:0/3095346984 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc02f610 con 0x55ea3ec12070 2024-09-21T04:12:58.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.106+0000 7f2ddf28e740 1 -- 172.21.15.64:0/3095346984 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dbc00be20 con 0x55ea3ec12070 2024-09-21T04:12:58.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.106+0000 7f2db9ffb640 1 -- 172.21.15.64:0/3095346984 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7f2dcc02f7b0 con 0x55ea3ec12070 2024-09-21T04:12:58.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.106+0000 7f2db9ffb640 1 -- 172.21.15.64:0/3095346984 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f2dcc02f9b0 con 0x55ea3ec12070 2024-09-21T04:12:58.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.106+0000 7f2ddf28e740 10 client.17277 fetch_fsmap learned FSMap version 40 2024-09-21T04:12:58.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.106+0000 7f2ddf28e740 1 -- 172.21.15.64:0/3095346984 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55ea3efd3760 con 0x55ea3ec12070 2024-09-21T04:12:58.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.106+0000 7f2db9ffb640 1 -- 172.21.15.64:0/3095346984 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dbc00be20 con 0x55ea3ec12070 2024-09-21T04:12:58.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.106+0000 7f2db9ffb640 1 -- 172.21.15.64:0/3095346984 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f2dcc051070 con 0x55ea3ec12070 2024-09-21T04:12:58.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.106+0000 7f2ddf28e740 10 client.17277 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:12:58.127 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.106+0000 7f2ddf28e740 10 client.17277 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:12:58.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.106+0000 7f2ddf28e740 20 client.17277 populate_metadata read hostname 'smithi064' 2024-09-21T04:12:58.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.106+0000 7f2ddf28e740 1 --2- 172.21.15.64:0/3095346984 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55ea3ebd09f0 0x55ea3efd6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:58.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.106+0000 7f2ddf28e740 4 client.17277 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:12:58.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.106+0000 7f2ddf28e740 1 -- 172.21.15.64:0/3095346984 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea3ef31da0 con 0x55ea3ebd09f0 2024-09-21T04:12:58.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.107+0000 7f2dd259c640 1 --2- 172.21.15.64:0/3095346984 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55ea3ebd09f0 0x55ea3efd6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:58.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.107+0000 7f2db9ffb640 10 client.17277 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:12:58.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.107+0000 7f2dd259c640 1 --2- 172.21.15.64:0/3095346984 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55ea3ebd09f0 0x55ea3efd6b80 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd08d6d24a7ef181 server_cookie=19be5fd65c12d390 in_seq=0 out_seq=0 2024-09-21T04:12:58.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.108+0000 7f2db9ffb640 1 -- 172.21.15.64:0/3095346984 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ea3ef31da0 con 0x55ea3ebd09f0 2024-09-21T04:12:58.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.108+0000 7f2db9ffb640 10 client.17277 handle_command_reply: tid=0 2024-09-21T04:12:58.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.108+0000 7f2ddf28e740 1 client.17277 shutdown 2024-09-21T04:12:58.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.108+0000 7f2ddf28e740 1 -- 172.21.15.64:0/3095346984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ec12070 msgr2=0x55ea3efd0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:58.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.108+0000 7f2ddf28e740 1 --2- 172.21.15.64:0/3095346984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ec12070 0x55ea3efd0d40 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc035000 tx=0x7f2dcc02a8c0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.128 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.108+0000 7f2ddf28e740 1 -- 172.21.15.64:0/3095346984 shutdown_connections 2024-09-21T04:12:58.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.108+0000 7f2ddf28e740 1 --2- 172.21.15.64:0/3095346984 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55ea3ebd09f0 0x55ea3efd6b80 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.108+0000 7f2ddf28e740 1 --2- 172.21.15.64:0/3095346984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ec12070 0x55ea3efd0d40 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.108+0000 7f2ddf28e740 1 --2- 172.21.15.64:0/3095346984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea3ebf1390 0x55ea3efc4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.108+0000 7f2ddf28e740 1 --2- 172.21.15.64:0/3095346984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea3ebe91b0 0x55ea3efc4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.108+0000 7f2ddf28e740 1 -- 172.21.15.64:0/3095346984 >> 172.21.15.64:0/3095346984 conn(0x55ea3ec06460 msgr2=0x55ea3efbc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:58.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.108+0000 7f2db9ffb640 0 client.17277 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:12:58.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.108+0000 7f2ddf28e740 1 -- 172.21.15.64:0/3095346984 shutdown_connections 2024-09-21T04:12:58.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.108+0000 7f2ddf28e740 1 -- 172.21.15.64:0/3095346984 wait complete. 2024-09-21T04:12:58.129 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.108+0000 7f2ddf28e740 20 client.17277 trim_cache size 0 max 16384 2024-09-21T04:12:58.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.126+0000 7f2ddf28e740 1 Processor -- start 2024-09-21T04:12:58.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.126+0000 7f2ddf28e740 1 -- start start 2024-09-21T04:12:58.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.126+0000 7f2ddf28e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea3ebe91b0 0x55ea3efc4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:58.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.126+0000 7f2ddf28e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ebf1390 0x55ea3efd0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:58.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.126+0000 7f2ddf28e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea3ec12070 0x55ea3efc1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:58.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.126+0000 7f2ddf28e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd41b4520 con 0x55ea3ebf1390 2024-09-21T04:12:58.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.126+0000 7f2ddf28e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd41b1d30 con 0x55ea3ec12070 2024-09-21T04:12:58.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.126+0000 7f2ddf28e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd41afba0 con 0x55ea3ebe91b0 2024-09-21T04:12:58.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.126+0000 7f2dcbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ebf1390 0x55ea3efd0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:58.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.126+0000 7f2dcbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ebf1390 0x55ea3efd0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39730/0 (socket says 172.21.15.64:39730) 2024-09-21T04:12:58.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.126+0000 7f2dcbfff640 1 -- 172.21.15.64:0/131123656 learned_addr learned my addr 172.21.15.64:0/131123656 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:58.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.126+0000 7f2dcbfff640 1 -- 172.21.15.64:0/131123656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea3ebe91b0 msgr2=0x55ea3efc4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:58.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.126+0000 7f2dcbfff640 1 --2- 172.21.15.64:0/131123656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea3ebe91b0 0x55ea3efc4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.126+0000 7f2dd2d9d640 1 --2- 172.21.15.64:0/131123656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea3ec12070 0x55ea3efc1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:58.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.126+0000 7f2dcbfff640 1 -- 172.21.15.64:0/131123656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea3ec12070 msgr2=0x55ea3efc1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:58.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.126+0000 7f2dcbfff640 1 --2- 172.21.15.64:0/131123656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea3ec12070 0x55ea3efc1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.126+0000 7f2dcbfff640 1 -- 172.21.15.64:0/131123656 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea3efc3890 con 0x55ea3ebf1390 2024-09-21T04:12:58.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.126+0000 7f2dd2d9d640 1 --2- 172.21.15.64:0/131123656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea3ec12070 0x55ea3efc1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:12:58.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.127+0000 7f2dcbfff640 1 --2- 172.21.15.64:0/131123656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ebf1390 0x55ea3efd0c40 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc00bc70 tx=0x7f2dbc005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b3d90ab00fe107 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:58.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.127+0000 7f2dbbfff640 1 -- 172.21.15.64:0/131123656 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc02b070 con 0x55ea3ebf1390 2024-09-21T04:12:58.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.127+0000 7f2dbbfff640 1 -- 172.21.15.64:0/131123656 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dbc0035f0 con 0x55ea3ebf1390 2024-09-21T04:12:58.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.127+0000 7f2dbbfff640 1 -- 172.21.15.64:0/131123656 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc011680 con 0x55ea3ebf1390 2024-09-21T04:12:58.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.128+0000 7f2ddf28e740 1 -- 172.21.15.64:0/131123656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ebf1390 msgr2=0x55ea3efd0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:58.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.128+0000 7f2ddf28e740 1 --2- 172.21.15.64:0/131123656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ebf1390 0x55ea3efd0c40 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc00bc70 tx=0x7f2dbc005040 comp rx=0 tx=0).stop 2024-09-21T04:12:58.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.128+0000 7f2ddf28e740 1 -- 172.21.15.64:0/131123656 shutdown_connections 2024-09-21T04:12:58.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.128+0000 7f2ddf28e740 1 --2- 172.21.15.64:0/131123656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea3ec12070 0x55ea3efc1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.128+0000 7f2ddf28e740 1 --2- 172.21.15.64:0/131123656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ebf1390 0x55ea3efd0c40 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.128+0000 7f2ddf28e740 1 --2- 172.21.15.64:0/131123656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea3ebe91b0 0x55ea3efc4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.128+0000 7f2ddf28e740 1 -- 172.21.15.64:0/131123656 >> 172.21.15.64:0/131123656 conn(0x55ea3ec06460 msgr2=0x55ea3efc46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:58.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.128+0000 7f2ddf28e740 1 -- 172.21.15.64:0/131123656 shutdown_connections 2024-09-21T04:12:58.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.128+0000 7f2ddf28e740 1 -- 172.21.15.64:0/131123656 wait complete. 2024-09-21T04:12:58.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.128+0000 7f2ddf28e740 1 -- start start 2024-09-21T04:12:58.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.128+0000 7f2ddf28e740 1 Processor -- start 2024-09-21T04:12:58.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.129+0000 7f2ddf28e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea3ebe91b0 0x55ea3efc8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:58.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.129+0000 7f2ddf28e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea3ebf1390 0x55ea3efc9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:58.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.129+0000 7f2ddf28e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ec12070 0x55ea3efc96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:58.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.129+0000 7f2ddf28e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd41b01f0 con 0x55ea3ec12070 2024-09-21T04:12:58.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.129+0000 7f2ddf28e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd41b1660 con 0x55ea3ebe91b0 2024-09-21T04:12:58.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.129+0000 7f2ddf28e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd41b08c0 con 0x55ea3ebf1390 2024-09-21T04:12:58.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.129+0000 7f2ddf28e740 10 client.0 _ll_register_callbacks cb 0x7f2dcc031450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:12:58.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.129+0000 7f2dd2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ec12070 0x55ea3efc96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:58.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.129+0000 7f2dd259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea3ebe91b0 0x55ea3efc8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:58.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.129+0000 7f2dd259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea3ebe91b0 0x55ea3efc8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:57784/0 (socket says 172.21.15.64:57784) 2024-09-21T04:12:58.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.129+0000 7f2dd2d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ec12070 0x55ea3efc96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39738/0 (socket says 172.21.15.64:39738) 2024-09-21T04:12:58.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.129+0000 7f2dd259c640 1 -- 172.21.15.64:0/2234520145 learned_addr learned my addr 172.21.15.64:0/2234520145 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:58.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.129+0000 7f2dcbfff640 1 --2- 172.21.15.64:0/2234520145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea3ebf1390 0x55ea3efc9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:58.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.129+0000 7f2dd2d9d640 1 -- 172.21.15.64:0/2234520145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea3ebf1390 msgr2=0x55ea3efc9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:58.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.129+0000 7f2dd2d9d640 1 --2- 172.21.15.64:0/2234520145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea3ebf1390 0x55ea3efc9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.129+0000 7f2dd2d9d640 1 -- 172.21.15.64:0/2234520145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea3ebe91b0 msgr2=0x55ea3efc8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:12:58.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.129+0000 7f2dd2d9d640 1 --2- 172.21.15.64:0/2234520145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea3ebe91b0 0x55ea3efc8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.129+0000 7f2dd2d9d640 1 -- 172.21.15.64:0/2234520145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea3f040580 con 0x55ea3ec12070 2024-09-21T04:12:58.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.129+0000 7f2dd259c640 1 --2- 172.21.15.64:0/2234520145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea3ebe91b0 0x55ea3efc8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:12:58.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.129+0000 7f2dd2d9d640 1 --2- 172.21.15.64:0/2234520145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ec12070 0x55ea3efc96c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc02ade0 tx=0x7f2dcc004160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4cd20ca1b169538 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:58.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.129+0000 7f2db9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:12:58.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.130+0000 7f2db9ffb640 1 -- 172.21.15.64:0/2234520145 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc02fd50 con 0x55ea3ec12070 2024-09-21T04:12:58.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.130+0000 7f2db9ffb640 1 -- 172.21.15.64:0/2234520145 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7f2dcc004250 con 0x55ea3ec12070 2024-09-21T04:12:58.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.130+0000 7f2ddf28e740 1 -- 172.21.15.64:0/2234520145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dbc0035f0 con 0x55ea3ec12070 2024-09-21T04:12:58.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.130+0000 7f2db9ffb640 1 -- 172.21.15.64:0/2234520145 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f2dcc069050 con 0x55ea3ec12070 2024-09-21T04:12:58.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.130+0000 7f2ddf28e740 10 client.17283 fetch_fsmap learned FSMap version 40 2024-09-21T04:12:58.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.130+0000 7f2ddf28e740 1 -- 172.21.15.64:0/2234520145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55ea3f041750 con 0x55ea3ec12070 2024-09-21T04:12:58.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.130+0000 7f2db9ffb640 1 -- 172.21.15.64:0/2234520145 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dbc0035f0 con 0x55ea3ec12070 2024-09-21T04:12:58.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.130+0000 7f2db9ffb640 1 -- 172.21.15.64:0/2234520145 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f2dcc062060 con 0x55ea3ec12070 2024-09-21T04:12:58.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.130+0000 7f2ddf28e740 10 client.17283 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:12:58.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.130+0000 7f2ddf28e740 10 client.17283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:12:58.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.130+0000 7f2ddf28e740 20 client.17283 populate_metadata read hostname 'smithi064' 2024-09-21T04:12:58.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.130+0000 7f2ddf28e740 1 --2- 172.21.15.64:0/2234520145 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55ea3eba5da0 0x55ea3f043f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:58.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.130+0000 7f2ddf28e740 4 client.17283 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T04:12:58.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.130+0000 7f2ddf28e740 1 -- 172.21.15.64:0/2234520145 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ea3ef2d850 con 0x55ea3eba5da0 2024-09-21T04:12:58.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.131+0000 7f2dd259c640 1 --2- 172.21.15.64:0/2234520145 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55ea3eba5da0 0x55ea3f043f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:58.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.131+0000 7f2dd259c640 1 --2- 172.21.15.64:0/2234520145 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55ea3eba5da0 0x55ea3f043f00 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fc9d7d199ca51f8 server_cookie=f0e86a0991754c7e in_seq=0 out_seq=0 2024-09-21T04:12:58.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.131+0000 7f2db9ffb640 10 client.17283 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:12:58.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.131+0000 7f2db9ffb640 1 -- 172.21.15.64:0/2234520145 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3498 (crc 0 0 0) 0x55ea3ef2d850 con 0x55ea3eba5da0 2024-09-21T04:12:58.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.131+0000 7f2db9ffb640 10 client.17283 handle_command_reply: tid=0 2024-09-21T04:12:58.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.131+0000 7f2ddf28e740 1 client.17283 shutdown 2024-09-21T04:12:58.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.132+0000 7f2ddf28e740 1 -- 172.21.15.64:0/2234520145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ec12070 msgr2=0x55ea3efc96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:58.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.132+0000 7f2ddf28e740 1 --2- 172.21.15.64:0/2234520145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ec12070 0x55ea3efc96c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc02ade0 tx=0x7f2dcc004160 comp rx=0 tx=0).stop 2024-09-21T04:12:58.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.132+0000 7f2ddf28e740 1 -- 172.21.15.64:0/2234520145 shutdown_connections 2024-09-21T04:12:58.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.132+0000 7f2ddf28e740 1 --2- 172.21.15.64:0/2234520145 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55ea3eba5da0 0x55ea3f043f00 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.132+0000 7f2ddf28e740 1 --2- 172.21.15.64:0/2234520145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ea3ec12070 0x55ea3efc96c0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.132+0000 7f2ddf28e740 1 --2- 172.21.15.64:0/2234520145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ea3ebf1390 0x55ea3efc9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.132+0000 7f2ddf28e740 1 --2- 172.21.15.64:0/2234520145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ea3ebe91b0 0x55ea3efc8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.132+0000 7f2ddf28e740 1 -- 172.21.15.64:0/2234520145 >> 172.21.15.64:0/2234520145 conn(0x55ea3ec06460 msgr2=0x55ea3efc5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:58.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.132+0000 7f2db9ffb640 0 client.17283 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:12:58.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.132+0000 7f2ddf28e740 1 -- 172.21.15.64:0/2234520145 shutdown_connections 2024-09-21T04:12:58.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.132+0000 7f2ddf28e740 1 -- 172.21.15.64:0/2234520145 wait complete. 2024-09-21T04:12:58.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.132+0000 7f2ddf28e740 20 client.17283 trim_cache size 0 max 16384 2024-09-21T04:12:58.141 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:12:58.141 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-21T04:12:58.141 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {}, 2024-09-21T04:12:58.141 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_stats": { 2024-09-21T04:12:58.141 INFO:teuthology.orchestra.run.smithi064.stdout: "1b23713d-ea3a-4a86-a87f-65f6ab82671d": { 2024-09-21T04:12:58.141 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:10:58.352639+0000", 2024-09-21T04:12:58.141 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:58.141 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:58.141 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:58.141 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:58.141 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:58.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:58.142 INFO:teuthology.orchestra.run.smithi064.stdout: "dd25622a-26f0-4838-b998-85d0603c83b6": { 2024-09-21T04:12:58.142 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:12:24.553513+0000", 2024-09-21T04:12:58.142 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:58.142 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:58.142 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:58.142 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:58.142 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:58.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:58.142 INFO:teuthology.orchestra.run.smithi064.stdout: "71f3f255-4837-45e9-92bc-4b3abb5c20eb": { 2024-09-21T04:12:58.142 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:11:42.378549+0000", 2024-09-21T04:12:58.142 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:58.142 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:58.142 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:58.143 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:58.143 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:58.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:58.143 INFO:teuthology.orchestra.run.smithi064.stdout: "2c0523fb-aae4-43fa-9568-136a9b640051": { 2024-09-21T04:12:58.143 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:09:39.808606+0000", 2024-09-21T04:12:58.143 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:58.143 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:58.143 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:58.143 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:58.143 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:58.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:58.143 INFO:teuthology.orchestra.run.smithi064.stdout: "bed40d88-2102-43a6-ac2d-b89d5a2264d0": { 2024-09-21T04:12:58.143 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:11.363340+0000", 2024-09-21T04:12:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: "e9242f88-6599-40eb-adf1-bdb3e946a1e0": { 2024-09-21T04:12:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:05:39.744646+0000", 2024-09-21T04:12:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:58.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:58.145 INFO:teuthology.orchestra.run.smithi064.stdout: "6bde0fe5-f775-44b1-ad0c-f9e596574dc0": { 2024-09-21T04:12:58.145 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:06:23.835536+0000", 2024-09-21T04:12:58.145 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:58.145 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:58.145 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:58.145 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:58.145 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:58.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:58.145 INFO:teuthology.orchestra.run.smithi064.stdout: "b52b950c-0f97-49ec-a37c-1a8e04348ce8": { 2024-09-21T04:12:58.145 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:54.003800+0000", 2024-09-21T04:12:58.145 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:58.145 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:58.145 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:58.145 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:58.146 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:58.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:58.146 INFO:teuthology.orchestra.run.smithi064.stdout: "e2c2eed8-9292-4bf6-8a39-13ad27da5acb": { 2024-09-21T04:12:58.146 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:08:47.067932+0000", 2024-09-21T04:12:58.146 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:58.146 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:58.146 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:58.146 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:58.146 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:58.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:58.146 INFO:teuthology.orchestra.run.smithi064.stdout: "75db1bc5-612e-45aa-bad2-cc90feeaa840": { 2024-09-21T04:12:58.146 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:59.123138+0000", 2024-09-21T04:12:58.146 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:58.146 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:58.146 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:58.147 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:58.147 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:58.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:58.147 INFO:teuthology.orchestra.run.smithi064.stdout: "0ab717a7-36c6-4dca-94e7-af0bdf2aa112": { 2024-09-21T04:12:58.147 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:42.691086+0000", 2024-09-21T04:12:58.147 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:58.147 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:58.147 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:58.147 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:58.147 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:58.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:12:58.147 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:12:58.147 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:03:59.570174+0000", 2024-09-21T04:12:58.148 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:12:58.148 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:12:58.148 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:12:58.148 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:12:58.148 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:12:58.148 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:12:58.148 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:12:58.148 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:12:58.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.132+0000 7f2dd8d1a640 1 -- 172.21.15.64:0/325411239 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2dc00b6680 msgr2=0x7f2dc00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:58.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.132+0000 7f2dd8d1a640 1 --2- 172.21.15.64:0/325411239 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2dc00b6680 0x7f2dc00b8b90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc006010 tx=0x7f2dcc005d80 comp rx=0 tx=0).stop 2024-09-21T04:12:58.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.132+0000 7f2dd8d1a640 1 -- 172.21.15.64:0/325411239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd4102a60 msgr2=0x7f2dd40ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:58.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.132+0000 7f2dd8d1a640 1 --2- 172.21.15.64:0/325411239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd4102a60 0x7f2dd40ff670 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400b4a0 tx=0x7f2dc400b960 comp rx=0 tx=0).stop 2024-09-21T04:12:58.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.132+0000 7f2dd8d1a640 1 -- 172.21.15.64:0/325411239 shutdown_connections 2024-09-21T04:12:58.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.132+0000 7f2dd8d1a640 1 --2- 172.21.15.64:0/325411239 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2dc00b6680 0x7f2dc00b8b90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.132+0000 7f2dd8d1a640 1 --2- 172.21.15.64:0/325411239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dd4108cf0 0x7f2dd41000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.132+0000 7f2dd8d1a640 1 --2- 172.21.15.64:0/325411239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2dd41083a0 0x7f2dd40ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.132+0000 7f2dd8d1a640 1 --2- 172.21.15.64:0/325411239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2dd4102a60 0x7f2dd40ff670 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:58.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.132+0000 7f2dd8d1a640 1 -- 172.21.15.64:0/325411239 >> 172.21.15.64:0/325411239 conn(0x7f2dd4079730 msgr2=0x7f2dd4101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:58.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.133+0000 7f2dd8d1a640 1 -- 172.21.15.64:0/325411239 shutdown_connections 2024-09-21T04:12:58.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:58.133+0000 7f2dd8d1a640 1 -- 172.21.15.64:0/325411239 wait complete. 2024-09-21T04:12:58.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:57 smithi119 ceph-mon[34811]: pgmap v595: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T04:12:58.989 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T04:12:58.990 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:12:59.255 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:12:59.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.897+0000 7f8247b6b640 1 -- 172.21.15.64:0/3028543480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8240108d30 msgr2=0x7f824010f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:59.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.897+0000 7f8247b6b640 1 --2- 172.21.15.64:0/3028543480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8240108d30 0x7f824010f560 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8238009ca0 tx=0x7f823802a8c0 comp rx=0 tx=0).stop 2024-09-21T04:12:59.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.898+0000 7f8247b6b640 1 -- 172.21.15.64:0/3028543480 shutdown_connections 2024-09-21T04:12:59.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.898+0000 7f8247b6b640 1 --2- 172.21.15.64:0/3028543480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8240108d30 0x7f824010f560 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:59.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.898+0000 7f8247b6b640 1 --2- 172.21.15.64:0/3028543480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82401083e0 0x7f82401087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:59.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.898+0000 7f8247b6b640 1 --2- 172.21.15.64:0/3028543480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8240102aa0 0x7f8240102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:59.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.898+0000 7f8247b6b640 1 -- 172.21.15.64:0/3028543480 >> 172.21.15.64:0/3028543480 conn(0x7f8240079750 msgr2=0x7f8240079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:12:59.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.898+0000 7f8247b6b640 1 -- 172.21.15.64:0/3028543480 shutdown_connections 2024-09-21T04:12:59.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.898+0000 7f8247b6b640 1 -- 172.21.15.64:0/3028543480 wait complete. 2024-09-21T04:12:59.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.898+0000 7f8247b6b640 1 Processor -- start 2024-09-21T04:12:59.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.898+0000 7f8247b6b640 1 -- start start 2024-09-21T04:12:59.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.899+0000 7f8247b6b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8240102aa0 0x7f824019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:59.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.899+0000 7f8247b6b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82401083e0 0x7f824019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:59.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.899+0000 7f8247b6b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8240108d30 0x7f824019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:59.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.899+0000 7f8247b6b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8240113b10 con 0x7f8240102aa0 2024-09-21T04:12:59.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.899+0000 7f8247b6b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8240113990 con 0x7f82401083e0 2024-09-21T04:12:59.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.899+0000 7f8247b6b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8240113c90 con 0x7f8240108d30 2024-09-21T04:12:59.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.899+0000 7f8245907640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8240102aa0 0x7f824019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:59.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.899+0000 7f8245907640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8240102aa0 0x7f824019a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:39756/0 (socket says 172.21.15.64:39756) 2024-09-21T04:12:59.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.899+0000 7f8245907640 1 -- 172.21.15.64:0/2071561267 learned_addr learned my addr 172.21.15.64:0/2071561267 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:12:59.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.899+0000 7f8245106640 1 --2- 172.21.15.64:0/2071561267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82401083e0 0x7f824019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:59.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.899+0000 7f8246108640 1 --2- 172.21.15.64:0/2071561267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8240108d30 0x7f824019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:59.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.899+0000 7f8245907640 1 -- 172.21.15.64:0/2071561267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8240108d30 msgr2=0x7f824019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:59.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.899+0000 7f8245907640 1 --2- 172.21.15.64:0/2071561267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8240108d30 0x7f824019fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:59.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.899+0000 7f8245907640 1 -- 172.21.15.64:0/2071561267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82401083e0 msgr2=0x7f824019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:12:59.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.899+0000 7f8245907640 1 --2- 172.21.15.64:0/2071561267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82401083e0 0x7f824019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:12:59.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.899+0000 7f8245907640 1 -- 172.21.15.64:0/2071561267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82401a02f0 con 0x7f8240102aa0 2024-09-21T04:12:59.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.899+0000 7f8245907640 1 --2- 172.21.15.64:0/2071561267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8240102aa0 0x7f824019a440 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f823400c920 tx=0x7f823400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:59.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.899+0000 7f8232ffd640 1 -- 172.21.15.64:0/2071561267 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234007c70 con 0x7f8240102aa0 2024-09-21T04:12:59.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.899+0000 7f8232ffd640 1 -- 172.21.15.64:0/2071561267 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8234007e10 con 0x7f8240102aa0 2024-09-21T04:12:59.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.900+0000 7f8247b6b640 1 -- 172.21.15.64:0/2071561267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82401a05e0 con 0x7f8240102aa0 2024-09-21T04:12:59.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.900+0000 7f8232ffd640 1 -- 172.21.15.64:0/2071561267 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234018820 con 0x7f8240102aa0 2024-09-21T04:12:59.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.900+0000 7f8247b6b640 1 -- 172.21.15.64:0/2071561267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82401a6e00 con 0x7f8240102aa0 2024-09-21T04:12:59.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.901+0000 7f8247b6b640 1 -- 172.21.15.64:0/2071561267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8240102eb0 con 0x7f8240102aa0 2024-09-21T04:12:59.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.903+0000 7f8232ffd640 1 -- 172.21.15.64:0/2071561267 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f823400f450 con 0x7f8240102aa0 2024-09-21T04:12:59.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.904+0000 7f8232ffd640 1 --2- 172.21.15.64:0/2071561267 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f822c0b6730 0x7f822c0b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:12:59.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.904+0000 7f8245106640 1 --2- 172.21.15.64:0/2071561267 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f822c0b6730 0x7f822c0b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:12:59.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.904+0000 7f8232ffd640 1 -- 172.21.15.64:0/2071561267 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7f82340e7b80 con 0x7f8240102aa0 2024-09-21T04:12:59.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.904+0000 7f8245106640 1 --2- 172.21.15.64:0/2071561267 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f822c0b6730 0x7f822c0b8c40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8240105ea0 tx=0x7f823c00b590 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:12:59.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:12:59.907+0000 7f8232ffd640 1 -- 172.21.15.64:0/2071561267 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8234014030 con 0x7f8240102aa0 2024-09-21T04:13:00.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:12:59 smithi131 ceph-mon[35193]: pgmap v596: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T04:13:00.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:00.133+0000 7f8247b6b640 1 -- 172.21.15.64:0/2071561267 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f824019b570 con 0x7f8240102aa0 2024-09-21T04:13:00.136 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:12:59 smithi064 ceph-mon[31180]: pgmap v596: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T04:13:00.136 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:00.134+0000 7f8232ffd640 1 -- 172.21.15.64:0/2071561267 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11918 (secure 0 0 0) 0x7f82340ae180 con 0x7f8240102aa0 2024-09-21T04:13:00.137 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:13:00.137 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":40,"btime":"2024-09-21T04:11:59:330881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14805,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/97995119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":97995119},{"type":"v1","addr":"172.21.15.64:6839","nonce":97995119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/390891068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":390891068},{"type":"v1","addr":"172.21.15.64:6841","nonce":390891068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:02:03.354744+0000","modified":"2024-09-21T04:11:59.188679+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24710,"mds_1":24680,"mds_2":24661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24634":{"gid":24634,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6839/782190424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782190424},{"type":"v1","addr":"172.21.15.131:6839","nonce":782190424}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24680":{"gid":24680,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24710":{"gid":24710,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6839/1319793636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1319793636},{"type":"v1","addr":"172.21.15.119:6839","nonce":1319793636}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24710,"qdb_cluster":[24661,24680,24710]},"id":1}]} 2024-09-21T04:13:00.137 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 40 2024-09-21T04:13:00.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:00.136+0000 7f8247b6b640 1 -- 172.21.15.64:0/2071561267 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f822c0b6730 msgr2=0x7f822c0b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:00.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:00.136+0000 7f8247b6b640 1 --2- 172.21.15.64:0/2071561267 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f822c0b6730 0x7f822c0b8c40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8240105ea0 tx=0x7f823c00b590 comp rx=0 tx=0).stop 2024-09-21T04:13:00.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:00.136+0000 7f8247b6b640 1 -- 172.21.15.64:0/2071561267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8240102aa0 msgr2=0x7f824019a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:00.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:00.136+0000 7f8247b6b640 1 --2- 172.21.15.64:0/2071561267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8240102aa0 0x7f824019a440 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f823400c920 tx=0x7f823400cde0 comp rx=0 tx=0).stop 2024-09-21T04:13:00.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:00.136+0000 7f8247b6b640 1 -- 172.21.15.64:0/2071561267 shutdown_connections 2024-09-21T04:13:00.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:00.136+0000 7f8247b6b640 1 --2- 172.21.15.64:0/2071561267 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f822c0b6730 0x7f822c0b8c40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:00.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:00.136+0000 7f8247b6b640 1 --2- 172.21.15.64:0/2071561267 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8240108d30 0x7f824019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:00.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:00.136+0000 7f8247b6b640 1 --2- 172.21.15.64:0/2071561267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82401083e0 0x7f824019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:00.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:00.136+0000 7f8247b6b640 1 --2- 172.21.15.64:0/2071561267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8240102aa0 0x7f824019a440 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:00.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:00.136+0000 7f8247b6b640 1 -- 172.21.15.64:0/2071561267 >> 172.21.15.64:0/2071561267 conn(0x7f8240079750 msgr2=0x7f8240104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:00.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:00.136+0000 7f8247b6b640 1 -- 172.21.15.64:0/2071561267 shutdown_connections 2024-09-21T04:13:00.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:00.136+0000 7f8247b6b640 1 -- 172.21.15.64:0/2071561267 wait complete. 2024-09-21T04:13:00.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:12:59 smithi119 ceph-mon[34811]: pgmap v596: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T04:13:00.971 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T04:13:01.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:00 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2071561267' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:13:01.232 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:13:01.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:00 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2071561267' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:13:01.261 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:00 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2071561267' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:13:01.843 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.837+0000 7f164b8a8640 1 -- 172.21.15.64:0/920290527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16441094b0 msgr2=0x7f16441098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:01.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.837+0000 7f164b8a8640 1 --2- 172.21.15.64:0/920290527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16441094b0 0x7f16441098c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f1638009900 tx=0x7f163802a410 comp rx=0 tx=0).stop 2024-09-21T04:13:01.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.837+0000 7f164b8a8640 1 -- 172.21.15.64:0/920290527 shutdown_connections 2024-09-21T04:13:01.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.837+0000 7f164b8a8640 1 --2- 172.21.15.64:0/920290527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1644109e00 0x7f1644110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.837+0000 7f164b8a8640 1 --2- 172.21.15.64:0/920290527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16441094b0 0x7f16441098c0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.837+0000 7f164b8a8640 1 --2- 172.21.15.64:0/920290527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1644103bc0 0x7f1644103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.837+0000 7f164b8a8640 1 -- 172.21.15.64:0/920290527 >> 172.21.15.64:0/920290527 conn(0x7f164407b910 msgr2=0x7f164407bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:01.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.837+0000 7f164b8a8640 1 -- 172.21.15.64:0/920290527 shutdown_connections 2024-09-21T04:13:01.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.837+0000 7f164b8a8640 1 -- 172.21.15.64:0/920290527 wait complete. 2024-09-21T04:13:01.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.838+0000 7f164b8a8640 1 Processor -- start 2024-09-21T04:13:01.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.838+0000 7f164b8a8640 1 -- start start 2024-09-21T04:13:01.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.838+0000 7f164b8a8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1644103bc0 0x7f164419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:01.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.838+0000 7f164b8a8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16441094b0 0x7f164419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:01.844 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.838+0000 7f164b8a8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1644109e00 0x7f164419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:01.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.838+0000 7f164b8a8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1644113ac0 con 0x7f16441094b0 2024-09-21T04:13:01.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.838+0000 7f164b8a8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1644113940 con 0x7f1644109e00 2024-09-21T04:13:01.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.838+0000 7f164b8a8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1644113c40 con 0x7f1644103bc0 2024-09-21T04:13:01.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.838+0000 7f1648e43640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16441094b0 0x7f164419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:01.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.838+0000 7f1648e43640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16441094b0 0x7f164419aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42014/0 (socket says 172.21.15.64:42014) 2024-09-21T04:13:01.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.838+0000 7f1648e43640 1 -- 172.21.15.64:0/1359872533 learned_addr learned my addr 172.21.15.64:0/1359872533 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:01.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.838+0000 7f1649e45640 1 --2- 172.21.15.64:0/1359872533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1644109e00 0x7f164419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:01.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.839+0000 7f1649644640 1 --2- 172.21.15.64:0/1359872533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1644103bc0 0x7f164419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:01.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.839+0000 7f1648e43640 1 -- 172.21.15.64:0/1359872533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1644103bc0 msgr2=0x7f164419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:01.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.839+0000 7f1648e43640 1 --2- 172.21.15.64:0/1359872533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1644103bc0 0x7f164419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.839+0000 7f1648e43640 1 -- 172.21.15.64:0/1359872533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1644109e00 msgr2=0x7f164419fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:01.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.839+0000 7f1648e43640 1 --2- 172.21.15.64:0/1359872533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1644109e00 0x7f164419fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.839+0000 7f1648e43640 1 -- 172.21.15.64:0/1359872533 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16441a03e0 con 0x7f16441094b0 2024-09-21T04:13:01.845 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.839+0000 7f1649644640 1 --2- 172.21.15.64:0/1359872533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1644103bc0 0x7f164419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:13:01.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.839+0000 7f1648e43640 1 --2- 172.21.15.64:0/1359872533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16441094b0 0x7f164419aa10 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f163802a930 tx=0x7f1638004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:01.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.839+0000 7f16427fc640 1 -- 172.21.15.64:0/1359872533 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163803f030 con 0x7f16441094b0 2024-09-21T04:13:01.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.839+0000 7f16427fc640 1 -- 172.21.15.64:0/1359872533 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1638003aa0 con 0x7f16441094b0 2024-09-21T04:13:01.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.839+0000 7f16427fc640 1 -- 172.21.15.64:0/1359872533 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1638035070 con 0x7f16441094b0 2024-09-21T04:13:01.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.839+0000 7f164b8a8640 1 -- 172.21.15.64:0/1359872533 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16441a0670 con 0x7f16441094b0 2024-09-21T04:13:01.846 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.839+0000 7f164b8a8640 1 -- 172.21.15.64:0/1359872533 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16441a6e30 con 0x7f16441094b0 2024-09-21T04:13:01.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.842+0000 7f1651e1c740 1 Processor -- start 2024-09-21T04:13:01.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.843+0000 7f1651e1c740 1 -- start start 2024-09-21T04:13:01.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.843+0000 7f16427fc640 1 -- 172.21.15.64:0/1359872533 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f1638042030 con 0x7f16441094b0 2024-09-21T04:13:01.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.843+0000 7f1651e1c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e143f99070 0x55e1443402c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:01.848 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.843+0000 7f1651e1c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f701b0 0x55e144340800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:01.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.843+0000 7f1651e1c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e143f78390 0x55e144340d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:01.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.843+0000 7f1651e1c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e1442b6ea0 con 0x55e143f701b0 2024-09-21T04:13:01.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.843+0000 7f1651e1c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e1442a62d0 con 0x55e143f99070 2024-09-21T04:13:01.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.843+0000 7f1651e1c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e1442aa0e0 con 0x55e143f78390 2024-09-21T04:13:01.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.843+0000 7f1649644640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f701b0 0x55e144340800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:01.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.843+0000 7f1649644640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f701b0 0x55e144340800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42030/0 (socket says 172.21.15.64:42030) 2024-09-21T04:13:01.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.843+0000 7f1649644640 1 -- 172.21.15.64:0/3141591613 learned_addr learned my addr 172.21.15.64:0/3141591613 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:01.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.843+0000 7f1649e45640 1 --2- 172.21.15.64:0/3141591613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e143f99070 0x55e1443402c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:01.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.843+0000 7f1648e43640 1 --2- 172.21.15.64:0/3141591613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e143f78390 0x55e144340d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:01.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.844+0000 7f1649644640 1 -- 172.21.15.64:0/3141591613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e143f78390 msgr2=0x55e144340d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:01.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.844+0000 7f1649644640 1 --2- 172.21.15.64:0/3141591613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e143f78390 0x55e144340d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.844+0000 7f1649644640 1 -- 172.21.15.64:0/3141591613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e143f99070 msgr2=0x55e1443402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:01.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.844+0000 7f1649644640 1 --2- 172.21.15.64:0/3141591613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e143f99070 0x55e1443402c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.849 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.844+0000 7f1649644640 1 -- 172.21.15.64:0/3141591613 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e144004650 con 0x55e143f701b0 2024-09-21T04:13:01.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.844+0000 7f1649e45640 1 --2- 172.21.15.64:0/3141591613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e143f99070 0x55e1443402c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:13:01.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.844+0000 7f1649644640 1 --2- 172.21.15.64:0/3141591613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f701b0 0x55e144340800 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f163c009c50 tx=0x7f163c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358b08181f555928 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:01.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.844+0000 7f162b7fe640 1 -- 172.21.15.64:0/3141591613 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c016070 con 0x55e143f701b0 2024-09-21T04:13:01.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.844+0000 7f162b7fe640 1 -- 172.21.15.64:0/3141591613 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f163c011920 con 0x55e143f701b0 2024-09-21T04:13:01.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.844+0000 7f162b7fe640 1 -- 172.21.15.64:0/3141591613 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c010810 con 0x55e143f701b0 2024-09-21T04:13:01.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.844+0000 7f16427fc640 1 --2- 172.21.15.64:0/1359872533 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f162c0b6680 0x7f162c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:01.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.844+0000 7f1649e45640 1 --2- 172.21.15.64:0/1359872533 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f162c0b6680 0x7f162c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:01.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.844+0000 7f16427fc640 1 -- 172.21.15.64:0/1359872533 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7f1638107b60 con 0x7f16441094b0 2024-09-21T04:13:01.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.845+0000 7f1651e1c740 1 -- 172.21.15.64:0/3141591613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f701b0 msgr2=0x55e144340800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:01.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.845+0000 7f1649e45640 1 --2- 172.21.15.64:0/1359872533 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f162c0b6680 0x7f162c0b8b90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1634005fd0 tx=0x7f1634009600 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:01.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.845+0000 7f1651e1c740 1 --2- 172.21.15.64:0/3141591613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f701b0 0x55e144340800 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f163c009c50 tx=0x7f163c00e6c0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.845+0000 7f1651e1c740 1 -- 172.21.15.64:0/3141591613 shutdown_connections 2024-09-21T04:13:01.850 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.845+0000 7f1651e1c740 1 --2- 172.21.15.64:0/3141591613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e143f78390 0x55e144340d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.845+0000 7f1651e1c740 1 --2- 172.21.15.64:0/3141591613 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f701b0 0x55e144340800 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.845+0000 7f1651e1c740 1 --2- 172.21.15.64:0/3141591613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e143f99070 0x55e1443402c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.845+0000 7f1651e1c740 1 -- 172.21.15.64:0/3141591613 >> 172.21.15.64:0/3141591613 conn(0x55e143f8d460 msgr2=0x55e14433e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:01.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.845+0000 7f1651e1c740 1 -- 172.21.15.64:0/3141591613 shutdown_connections 2024-09-21T04:13:01.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.845+0000 7f1651e1c740 1 -- 172.21.15.64:0/3141591613 wait complete. 2024-09-21T04:13:01.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.846+0000 7f1651e1c740 1 -- start start 2024-09-21T04:13:01.851 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.846+0000 7f1651e1c740 1 Processor -- start 2024-09-21T04:13:01.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.846+0000 7f1651e1c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f701b0 0x55e14434b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:01.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.846+0000 7f1651e1c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e143f78390 0x55e14434be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:01.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.846+0000 7f1651e1c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e143f99070 0x55e14434c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:01.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.846+0000 7f1651e1c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e144107700 con 0x55e143f701b0 2024-09-21T04:13:01.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.846+0000 7f1651e1c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e144241160 con 0x55e143f99070 2024-09-21T04:13:01.854 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.846+0000 7f1651e1c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e144352eb0 con 0x55e143f78390 2024-09-21T04:13:01.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.846+0000 7f1649644640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f701b0 0x55e14434b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:01.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.846+0000 7f1651e1c740 10 client.0 _ll_register_callbacks cb 0x55e14425a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:01.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.846+0000 7f1649644640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f701b0 0x55e14434b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42038/0 (socket says 172.21.15.64:42038) 2024-09-21T04:13:01.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.846+0000 7f1649644640 1 -- 172.21.15.64:0/2936908573 learned_addr learned my addr 172.21.15.64:0/2936908573 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:01.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.846+0000 7f1649e45640 1 --2- 172.21.15.64:0/2936908573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e143f99070 0x55e14434c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:01.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.847+0000 7f1648e43640 1 --2- 172.21.15.64:0/2936908573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e143f78390 0x55e14434be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:01.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.847+0000 7f1649644640 1 -- 172.21.15.64:0/2936908573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e143f78390 msgr2=0x55e14434be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:01.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.847+0000 7f1649644640 1 --2- 172.21.15.64:0/2936908573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e143f78390 0x55e14434be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.847+0000 7f1649644640 1 -- 172.21.15.64:0/2936908573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e143f99070 msgr2=0x55e14434c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:01.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.847+0000 7f1649644640 1 --2- 172.21.15.64:0/2936908573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e143f99070 0x55e14434c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.847+0000 7f1649644640 1 -- 172.21.15.64:0/2936908573 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e14426ec90 con 0x55e143f701b0 2024-09-21T04:13:01.855 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.847+0000 7f1648e43640 1 --2- 172.21.15.64:0/2936908573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e143f78390 0x55e14434be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:13:01.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.847+0000 7f1649644640 1 --2- 172.21.15.64:0/2936908573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f701b0 0x55e14434b8f0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f163c0118e0 tx=0x7f163c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d2e550d2518002 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:01.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.847+0000 7f1628ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:13:01.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.847+0000 7f1628ff9640 1 -- 172.21.15.64:0/2936908573 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c016050 con 0x55e143f701b0 2024-09-21T04:13:01.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.847+0000 7f1651e1c740 1 -- 172.21.15.64:0/2936908573 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f163c011920 con 0x55e143f701b0 2024-09-21T04:13:01.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.847+0000 7f1628ff9640 1 -- 172.21.15.64:0/2936908573 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7f163c005040 con 0x55e143f701b0 2024-09-21T04:13:01.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.847+0000 7f1628ff9640 1 -- 172.21.15.64:0/2936908573 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f163c030050 con 0x55e143f701b0 2024-09-21T04:13:01.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.847+0000 7f1651e1c740 10 client.17301 fetch_fsmap learned FSMap version 40 2024-09-21T04:13:01.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.848+0000 7f1651e1c740 1 -- 172.21.15.64:0/2936908573 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e14435eb30 con 0x55e143f701b0 2024-09-21T04:13:01.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.848+0000 7f1628ff9640 1 -- 172.21.15.64:0/2936908573 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f163c011920 con 0x55e143f701b0 2024-09-21T04:13:01.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.848+0000 7f1628ff9640 1 -- 172.21.15.64:0/2936908573 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f163c02b030 con 0x55e143f701b0 2024-09-21T04:13:01.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.848+0000 7f1651e1c740 10 client.17301 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:13:01.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.848+0000 7f1651e1c740 10 client.17301 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:13:01.856 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.848+0000 7f1651e1c740 20 client.17301 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:01.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.848+0000 7f1651e1c740 1 --2- 172.21.15.64:0/2936908573 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55e143f579f0 0x55e144361ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:01.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.848+0000 7f1651e1c740 4 client.17301 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:13:01.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.848+0000 7f1651e1c740 1 -- 172.21.15.64:0/2936908573 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1442b8df0 con 0x55e143f579f0 2024-09-21T04:13:01.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.848+0000 7f1648e43640 1 --2- 172.21.15.64:0/2936908573 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55e143f579f0 0x55e144361ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:01.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.849+0000 7f1648e43640 1 --2- 172.21.15.64:0/2936908573 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55e143f579f0 0x55e144361ef0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=11c699c644195ac4 server_cookie=56141fc32e1397e8 in_seq=0 out_seq=0 2024-09-21T04:13:01.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.849+0000 7f1628ff9640 10 client.17301 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:13:01.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.850+0000 7f1628ff9640 1 -- 172.21.15.64:0/2936908573 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e1442b8df0 con 0x55e143f579f0 2024-09-21T04:13:01.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.850+0000 7f1628ff9640 10 client.17301 handle_command_reply: tid=0 2024-09-21T04:13:01.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.850+0000 7f1651e1c740 1 client.17301 shutdown 2024-09-21T04:13:01.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.850+0000 7f1651e1c740 1 -- 172.21.15.64:0/2936908573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f701b0 msgr2=0x55e14434b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:01.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.850+0000 7f1651e1c740 1 --2- 172.21.15.64:0/2936908573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f701b0 0x55e14434b8f0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f163c0118e0 tx=0x7f163c005850 comp rx=0 tx=0).stop 2024-09-21T04:13:01.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.850+0000 7f1651e1c740 1 -- 172.21.15.64:0/2936908573 shutdown_connections 2024-09-21T04:13:01.857 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.850+0000 7f1651e1c740 1 --2- 172.21.15.64:0/2936908573 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55e143f579f0 0x55e144361ef0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.850+0000 7f1651e1c740 1 --2- 172.21.15.64:0/2936908573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e143f99070 0x55e14434c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.850+0000 7f1651e1c740 1 --2- 172.21.15.64:0/2936908573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e143f78390 0x55e14434be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.850+0000 7f1651e1c740 1 --2- 172.21.15.64:0/2936908573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f701b0 0x55e14434b8f0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.850+0000 7f1651e1c740 1 -- 172.21.15.64:0/2936908573 >> 172.21.15.64:0/2936908573 conn(0x55e143f8d460 msgr2=0x55e144343450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:01.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.850+0000 7f1628ff9640 0 client.17301 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:13:01.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.850+0000 7f1651e1c740 1 -- 172.21.15.64:0/2936908573 shutdown_connections 2024-09-21T04:13:01.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.850+0000 7f1651e1c740 1 -- 172.21.15.64:0/2936908573 wait complete. 2024-09-21T04:13:01.858 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.850+0000 7f1651e1c740 20 client.17301 trim_cache size 0 max 16384 2024-09-21T04:13:01.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.863+0000 7f1651e1c740 1 Processor -- start 2024-09-21T04:13:01.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.863+0000 7f1651e1c740 1 -- start start 2024-09-21T04:13:01.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.864+0000 7f1651e1c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e143f701b0 0x55e144341a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:01.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.864+0000 7f1651e1c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e143f78390 0x55e144348ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:01.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.864+0000 7f1651e1c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f99070 0x55e144349020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:01.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.864+0000 7f1651e1c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f16441affe0 con 0x55e143f99070 2024-09-21T04:13:01.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.864+0000 7f1651e1c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f16441ad7f0 con 0x55e143f78390 2024-09-21T04:13:01.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.864+0000 7f1651e1c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f16441ab660 con 0x55e143f701b0 2024-09-21T04:13:01.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.864+0000 7f1649e45640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f99070 0x55e144349020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:01.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.864+0000 7f1649e45640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f99070 0x55e144349020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42046/0 (socket says 172.21.15.64:42046) 2024-09-21T04:13:01.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.864+0000 7f1649e45640 1 -- 172.21.15.64:0/627503955 learned_addr learned my addr 172.21.15.64:0/627503955 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:01.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.864+0000 7f1649644640 1 --2- 172.21.15.64:0/627503955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e143f701b0 0x55e144341a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:01.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.864+0000 7f1648e43640 1 --2- 172.21.15.64:0/627503955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e143f78390 0x55e144348ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:01.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.864+0000 7f1649e45640 1 -- 172.21.15.64:0/627503955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e143f701b0 msgr2=0x55e144341a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:01.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.864+0000 7f1649e45640 1 --2- 172.21.15.64:0/627503955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e143f701b0 0x55e144341a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.864+0000 7f1649e45640 1 -- 172.21.15.64:0/627503955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e143f78390 msgr2=0x55e144348ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:01.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.864+0000 7f1649e45640 1 --2- 172.21.15.64:0/627503955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e143f78390 0x55e144348ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.864+0000 7f1649e45640 1 -- 172.21.15.64:0/627503955 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e144338440 con 0x55e143f99070 2024-09-21T04:13:01.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.864+0000 7f1649e45640 1 --2- 172.21.15.64:0/627503955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f99070 0x55e144349020 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f1634005f40 tx=0x7f1634016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436290ad6a87a4b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:01.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.865+0000 7f162affd640 1 -- 172.21.15.64:0/627503955 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163401d430 con 0x55e143f99070 2024-09-21T04:13:01.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.865+0000 7f162affd640 1 -- 172.21.15.64:0/627503955 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f163401da80 con 0x55e143f99070 2024-09-21T04:13:01.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.865+0000 7f162affd640 1 -- 172.21.15.64:0/627503955 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634020c70 con 0x55e143f99070 2024-09-21T04:13:01.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.865+0000 7f1651e1c740 1 -- 172.21.15.64:0/627503955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f99070 msgr2=0x55e144349020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:01.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.865+0000 7f1651e1c740 1 --2- 172.21.15.64:0/627503955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f99070 0x55e144349020 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f1634005f40 tx=0x7f1634016000 comp rx=0 tx=0).stop 2024-09-21T04:13:01.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.865+0000 7f1651e1c740 1 -- 172.21.15.64:0/627503955 shutdown_connections 2024-09-21T04:13:01.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.865+0000 7f1651e1c740 1 --2- 172.21.15.64:0/627503955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f99070 0x55e144349020 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.865+0000 7f1651e1c740 1 --2- 172.21.15.64:0/627503955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e143f78390 0x55e144348ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.865+0000 7f1651e1c740 1 --2- 172.21.15.64:0/627503955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e143f701b0 0x55e144341a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.865+0000 7f1651e1c740 1 -- 172.21.15.64:0/627503955 >> 172.21.15.64:0/627503955 conn(0x55e143f8d460 msgr2=0x55e1443412f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:01.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.866+0000 7f1651e1c740 1 -- 172.21.15.64:0/627503955 shutdown_connections 2024-09-21T04:13:01.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.866+0000 7f1651e1c740 1 -- 172.21.15.64:0/627503955 wait complete. 2024-09-21T04:13:01.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.866+0000 7f1651e1c740 1 -- start start 2024-09-21T04:13:01.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.866+0000 7f1651e1c740 1 Processor -- start 2024-09-21T04:13:01.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.867+0000 7f1651e1c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e143f701b0 0x55e1443587e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:01.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.867+0000 7f1651e1c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e143f78390 0x55e144353d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:01.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.867+0000 7f1651e1c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f99070 0x55e1443542c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:01.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.867+0000 7f1651e1c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f16441abcb0 con 0x55e143f99070 2024-09-21T04:13:01.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.867+0000 7f1651e1c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f16441ad120 con 0x55e143f78390 2024-09-21T04:13:01.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.867+0000 7f1651e1c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f16441ac380 con 0x55e143f701b0 2024-09-21T04:13:01.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.867+0000 7f1651e1c740 10 client.0 _ll_register_callbacks cb 0x7f1634009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:01.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.867+0000 7f1649e45640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f99070 0x55e1443542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:01.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.867+0000 7f1649e45640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f99070 0x55e1443542c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42050/0 (socket says 172.21.15.64:42050) 2024-09-21T04:13:01.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.867+0000 7f1649e45640 1 -- 172.21.15.64:0/1733438880 learned_addr learned my addr 172.21.15.64:0/1733438880 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:01.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.867+0000 7f1648e43640 1 --2- 172.21.15.64:0/1733438880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e143f78390 0x55e144353d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:01.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.867+0000 7f1649644640 1 --2- 172.21.15.64:0/1733438880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e143f701b0 0x55e1443587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:01.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.867+0000 7f1649e45640 1 -- 172.21.15.64:0/1733438880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e143f701b0 msgr2=0x55e1443587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:01.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.867+0000 7f1649e45640 1 --2- 172.21.15.64:0/1733438880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e143f701b0 0x55e1443587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.867+0000 7f1649e45640 1 -- 172.21.15.64:0/1733438880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e143f78390 msgr2=0x55e144353d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:01.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.867+0000 7f1649e45640 1 --2- 172.21.15.64:0/1733438880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e143f78390 0x55e144353d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.867+0000 7f1649e45640 1 -- 172.21.15.64:0/1733438880 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e144354930 con 0x55e143f99070 2024-09-21T04:13:01.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.867+0000 7f1649e45640 1 --2- 172.21.15.64:0/1733438880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f99070 0x55e1443542c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f163401d6e0 tx=0x7f163401dfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17003bb2449df258 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:01.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.867+0000 7f1628ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:13:01.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.868+0000 7f1628ff9640 1 -- 172.21.15.64:0/1733438880 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634020420 con 0x55e143f99070 2024-09-21T04:13:01.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.868+0000 7f1628ff9640 1 -- 172.21.15.64:0/1733438880 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7f163402f040 con 0x55e143f99070 2024-09-21T04:13:01.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.868+0000 7f1651e1c740 1 -- 172.21.15.64:0/1733438880 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f163401da80 con 0x55e143f99070 2024-09-21T04:13:01.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.868+0000 7f1628ff9640 1 -- 172.21.15.64:0/1733438880 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f163401d8c0 con 0x55e143f99070 2024-09-21T04:13:01.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.868+0000 7f1651e1c740 10 client.17307 fetch_fsmap learned FSMap version 40 2024-09-21T04:13:01.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.868+0000 7f1651e1c740 1 -- 172.21.15.64:0/1733438880 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e1443cc4f0 con 0x55e143f99070 2024-09-21T04:13:01.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.868+0000 7f1628ff9640 1 -- 172.21.15.64:0/1733438880 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f163401da80 con 0x55e143f99070 2024-09-21T04:13:01.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.868+0000 7f1628ff9640 1 -- 172.21.15.64:0/1733438880 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f163400f040 con 0x55e143f99070 2024-09-21T04:13:01.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.868+0000 7f1651e1c740 10 client.17307 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:13:01.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.868+0000 7f1651e1c740 10 client.17307 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:13:01.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.868+0000 7f1651e1c740 20 client.17307 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:01.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.868+0000 7f1651e1c740 1 --2- 172.21.15.64:0/1733438880 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55e143f2cda0 0x55e1443cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:01.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.868+0000 7f1651e1c740 4 client.17307 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:13:01.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.868+0000 7f1651e1c740 1 -- 172.21.15.64:0/1733438880 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e1442b48a0 con 0x55e143f2cda0 2024-09-21T04:13:01.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.869+0000 7f1649644640 1 --2- 172.21.15.64:0/1733438880 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55e143f2cda0 0x55e1443cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:01.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.869+0000 7f1649644640 1 --2- 172.21.15.64:0/1733438880 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55e143f2cda0 0x55e1443cec80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=38c72a5b543fce7d server_cookie=3a0bd404d4eb9df in_seq=0 out_seq=0 2024-09-21T04:13:01.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.869+0000 7f1628ff9640 10 client.17307 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:13:01.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.869+0000 7f1628ff9640 1 -- 172.21.15.64:0/1733438880 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e1442b48a0 con 0x55e143f2cda0 2024-09-21T04:13:01.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.869+0000 7f1628ff9640 10 client.17307 handle_command_reply: tid=0 2024-09-21T04:13:01.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.869+0000 7f1651e1c740 1 client.17307 shutdown 2024-09-21T04:13:01.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.869+0000 7f1651e1c740 1 -- 172.21.15.64:0/1733438880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f99070 msgr2=0x55e1443542c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:01.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.869+0000 7f1651e1c740 1 --2- 172.21.15.64:0/1733438880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f99070 0x55e1443542c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f163401d6e0 tx=0x7f163401dfd0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.869+0000 7f1651e1c740 1 -- 172.21.15.64:0/1733438880 shutdown_connections 2024-09-21T04:13:01.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.869+0000 7f1651e1c740 1 --2- 172.21.15.64:0/1733438880 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55e143f2cda0 0x55e1443cec80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.869+0000 7f1651e1c740 1 --2- 172.21.15.64:0/1733438880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e143f99070 0x55e1443542c0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.869+0000 7f1651e1c740 1 --2- 172.21.15.64:0/1733438880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e143f78390 0x55e144353d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.869+0000 7f1651e1c740 1 --2- 172.21.15.64:0/1733438880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e143f701b0 0x55e1443587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.869+0000 7f1651e1c740 1 -- 172.21.15.64:0/1733438880 >> 172.21.15.64:0/1733438880 conn(0x55e143f8d460 msgr2=0x55e1443453d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:01.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.869+0000 7f1628ff9640 0 client.17307 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:13:01.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.870+0000 7f1651e1c740 1 -- 172.21.15.64:0/1733438880 shutdown_connections 2024-09-21T04:13:01.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.870+0000 7f1651e1c740 1 -- 172.21.15.64:0/1733438880 wait complete. 2024-09-21T04:13:01.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.870+0000 7f1651e1c740 20 client.17307 trim_cache size 0 max 16384 2024-09-21T04:13:01.877 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:13:01.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.870+0000 7f164b8a8640 1 -- 172.21.15.64:0/1359872533 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f162c0b6680 msgr2=0x7f162c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:01.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.870+0000 7f164b8a8640 1 --2- 172.21.15.64:0/1359872533 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f162c0b6680 0x7f162c0b8b90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1634005fd0 tx=0x7f1634009600 comp rx=0 tx=0).stop 2024-09-21T04:13:01.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.870+0000 7f164b8a8640 1 -- 172.21.15.64:0/1359872533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16441094b0 msgr2=0x7f164419aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:01.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.870+0000 7f164b8a8640 1 --2- 172.21.15.64:0/1359872533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16441094b0 0x7f164419aa10 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f163802a930 tx=0x7f1638004270 comp rx=0 tx=0).stop 2024-09-21T04:13:01.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.870+0000 7f164b8a8640 1 -- 172.21.15.64:0/1359872533 shutdown_connections 2024-09-21T04:13:01.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.870+0000 7f164b8a8640 1 --2- 172.21.15.64:0/1359872533 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f162c0b6680 0x7f162c0b8b90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.870+0000 7f164b8a8640 1 --2- 172.21.15.64:0/1359872533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1644109e00 0x7f164419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.870+0000 7f164b8a8640 1 --2- 172.21.15.64:0/1359872533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16441094b0 0x7f164419aa10 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.870+0000 7f164b8a8640 1 --2- 172.21.15.64:0/1359872533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1644103bc0 0x7f164419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:01.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.870+0000 7f164b8a8640 1 -- 172.21.15.64:0/1359872533 >> 172.21.15.64:0/1359872533 conn(0x7f164407b910 msgr2=0x7f1644105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:01.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.870+0000 7f164b8a8640 1 -- 172.21.15.64:0/1359872533 shutdown_connections 2024-09-21T04:13:01.879 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:01.870+0000 7f164b8a8640 1 -- 172.21.15.64:0/1359872533 wait complete. 2024-09-21T04:13:02.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:01 smithi064 ceph-mon[31180]: pgmap v597: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:13:02.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:01 smithi131 ceph-mon[35193]: pgmap v597: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:13:02.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:01 smithi119 ceph-mon[34811]: pgmap v597: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:13:02.653 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T04:13:02.911 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:13:03.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.419+0000 7fccb7037640 1 -- 172.21.15.64:0/3487784246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccb0108cf0 msgr2=0x7fccb010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:03.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.419+0000 7fccb7037640 1 --2- 172.21.15.64:0/3487784246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccb0108cf0 0x7fccb010f520 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009900 tx=0x7fcca802a410 comp rx=0 tx=0).stop 2024-09-21T04:13:03.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.419+0000 7fccb7037640 1 -- 172.21.15.64:0/3487784246 shutdown_connections 2024-09-21T04:13:03.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.419+0000 7fccb7037640 1 --2- 172.21.15.64:0/3487784246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccb0108cf0 0x7fccb010f520 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.419+0000 7fccb7037640 1 --2- 172.21.15.64:0/3487784246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccb01083a0 0x7fccb01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.419+0000 7fccb7037640 1 --2- 172.21.15.64:0/3487784246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccb0102a60 0x7fccb0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.419+0000 7fccb7037640 1 -- 172.21.15.64:0/3487784246 >> 172.21.15.64:0/3487784246 conn(0x7fccb0079730 msgr2=0x7fccb0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:03.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.419+0000 7fccb7037640 1 -- 172.21.15.64:0/3487784246 shutdown_connections 2024-09-21T04:13:03.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.419+0000 7fccb7037640 1 -- 172.21.15.64:0/3487784246 wait complete. 2024-09-21T04:13:03.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.420+0000 7fccb7037640 1 Processor -- start 2024-09-21T04:13:03.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.420+0000 7fccb7037640 1 -- start start 2024-09-21T04:13:03.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.420+0000 7fccb7037640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccb0102a60 0x7fccb019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:03.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.420+0000 7fccb7037640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccb01083a0 0x7fccb019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:03.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.420+0000 7fccb7037640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccb0108cf0 0x7fccb019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:03.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.420+0000 7fccb7037640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fccb0113a80 con 0x7fccb01083a0 2024-09-21T04:13:03.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.420+0000 7fccb7037640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fccb0113900 con 0x7fccb0108cf0 2024-09-21T04:13:03.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.420+0000 7fccb7037640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fccb0113c00 con 0x7fccb0102a60 2024-09-21T04:13:03.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.421+0000 7fccaffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccb01083a0 0x7fccb019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:03.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.421+0000 7fccb4dd3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccb0102a60 0x7fccb019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:03.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.421+0000 7fccaffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccb01083a0 0x7fccb019aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42074/0 (socket says 172.21.15.64:42074) 2024-09-21T04:13:03.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.421+0000 7fccaffff640 1 -- 172.21.15.64:0/1313210309 learned_addr learned my addr 172.21.15.64:0/1313210309 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:03.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.421+0000 7fccb55d4640 1 --2- 172.21.15.64:0/1313210309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccb0108cf0 0x7fccb019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:03.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.421+0000 7fccaffff640 1 -- 172.21.15.64:0/1313210309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccb0102a60 msgr2=0x7fccb019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:03.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.421+0000 7fccaffff640 1 --2- 172.21.15.64:0/1313210309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccb0102a60 0x7fccb019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.439 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.421+0000 7fccaffff640 1 -- 172.21.15.64:0/1313210309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccb0108cf0 msgr2=0x7fccb019fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:03.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.421+0000 7fccaffff640 1 --2- 172.21.15.64:0/1313210309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccb0108cf0 0x7fccb019fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.421+0000 7fccaffff640 1 -- 172.21.15.64:0/1313210309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb01a0400 con 0x7fccb01083a0 2024-09-21T04:13:03.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.421+0000 7fccb55d4640 1 --2- 172.21.15.64:0/1313210309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccb0108cf0 0x7fccb019fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:13:03.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.421+0000 7fccaffff640 1 --2- 172.21.15.64:0/1313210309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccb01083a0 0x7fccb019aa30 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fcca000ea50 tx=0x7fcca000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:03.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.421+0000 7fccadffb640 1 -- 172.21.15.64:0/1313210309 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca000cdc0 con 0x7fccb01083a0 2024-09-21T04:13:03.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.421+0000 7fccb7037640 1 -- 172.21.15.64:0/1313210309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb01a06f0 con 0x7fccb01083a0 2024-09-21T04:13:03.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.421+0000 7fccadffb640 1 -- 172.21.15.64:0/1313210309 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcca0005020 con 0x7fccb01083a0 2024-09-21T04:13:03.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.421+0000 7fccadffb640 1 -- 172.21.15.64:0/1313210309 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca00116b0 con 0x7fccb01083a0 2024-09-21T04:13:03.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.422+0000 7fccb7037640 1 -- 172.21.15.64:0/1313210309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb01a6e50 con 0x7fccb01083a0 2024-09-21T04:13:03.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.425+0000 7fccbd5ab740 1 Processor -- start 2024-09-21T04:13:03.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.425+0000 7fccbd5ab740 1 -- start start 2024-09-21T04:13:03.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.425+0000 7fccadffb640 1 -- 172.21.15.64:0/1313210309 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fcca00051d0 con 0x7fccb01083a0 2024-09-21T04:13:03.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.425+0000 7fccbd5ab740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c285c5a070 0x55c2860016a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:03.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.425+0000 7fccbd5ab740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c285c311b0 0x55c286001be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:03.440 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.425+0000 7fccbd5ab740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c39390 0x55c286002120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:03.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.425+0000 7fccbd5ab740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c285f77ea0 con 0x55c285c39390 2024-09-21T04:13:03.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.425+0000 7fccbd5ab740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c285f672d0 con 0x55c285c5a070 2024-09-21T04:13:03.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.425+0000 7fccbd5ab740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c285f6b0e0 con 0x55c285c311b0 2024-09-21T04:13:03.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.425+0000 7fccaffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c39390 0x55c286002120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:03.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.425+0000 7fccaffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c39390 0x55c286002120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42086/0 (socket says 172.21.15.64:42086) 2024-09-21T04:13:03.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.425+0000 7fccb55d4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c285c5a070 0x55c2860016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:03.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.425+0000 7fccaffff640 1 -- 172.21.15.64:0/3186034297 learned_addr learned my addr 172.21.15.64:0/3186034297 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:03.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.426+0000 7fccb4dd3640 1 --2- 172.21.15.64:0/3186034297 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c285c311b0 0x55c286001be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:03.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.426+0000 7fccaffff640 1 -- 172.21.15.64:0/3186034297 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c285c311b0 msgr2=0x55c286001be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:03.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.426+0000 7fccaffff640 1 --2- 172.21.15.64:0/3186034297 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c285c311b0 0x55c286001be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.426+0000 7fccaffff640 1 -- 172.21.15.64:0/3186034297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c285c5a070 msgr2=0x55c2860016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:03.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.426+0000 7fccaffff640 1 --2- 172.21.15.64:0/3186034297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c285c5a070 0x55c2860016a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.441 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.426+0000 7fccaffff640 1 -- 172.21.15.64:0/3186034297 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c285cc5650 con 0x55c285c39390 2024-09-21T04:13:03.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.426+0000 7fccb55d4640 1 --2- 172.21.15.64:0/3186034297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c285c5a070 0x55c2860016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:13:03.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.426+0000 7fccadffb640 1 --2- 172.21.15.64:0/1313210309 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcc9c0b6680 0x7fcc9c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:03.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.426+0000 7fccaffff640 1 --2- 172.21.15.64:0/3186034297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c39390 0x55c286002120 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fcca00e5010 tx=0x7fcca00f04f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f045592e70b25b89 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:03.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.426+0000 7fccb55d4640 1 --2- 172.21.15.64:0/1313210309 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcc9c0b6680 0x7fcc9c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:03.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.426+0000 7fccadffb640 1 -- 172.21.15.64:0/1313210309 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7fcca00e79b0 con 0x7fccb01083a0 2024-09-21T04:13:03.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.426+0000 7fcca6ffd640 1 -- 172.21.15.64:0/3186034297 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca00e7e60 con 0x55c285c39390 2024-09-21T04:13:03.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.426+0000 7fcca6ffd640 1 -- 172.21.15.64:0/3186034297 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcca00ad2b0 con 0x55c285c39390 2024-09-21T04:13:03.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.426+0000 7fcca6ffd640 1 -- 172.21.15.64:0/3186034297 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca00ad0f0 con 0x55c285c39390 2024-09-21T04:13:03.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.427+0000 7fccbd5ab740 1 -- 172.21.15.64:0/3186034297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c39390 msgr2=0x55c286002120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:03.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.427+0000 7fccb55d4640 1 --2- 172.21.15.64:0/1313210309 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcc9c0b6680 0x7fcc9c0b8b90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcca8031440 tx=0x7fcca8005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:03.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.427+0000 7fccbd5ab740 1 --2- 172.21.15.64:0/3186034297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c39390 0x55c286002120 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fcca00e5010 tx=0x7fcca00f04f0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.427+0000 7fccbd5ab740 1 -- 172.21.15.64:0/3186034297 shutdown_connections 2024-09-21T04:13:03.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.427+0000 7fccbd5ab740 1 --2- 172.21.15.64:0/3186034297 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c39390 0x55c286002120 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.442 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.427+0000 7fccbd5ab740 1 --2- 172.21.15.64:0/3186034297 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c285c311b0 0x55c286001be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.427+0000 7fccbd5ab740 1 --2- 172.21.15.64:0/3186034297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c285c5a070 0x55c2860016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.427+0000 7fccbd5ab740 1 -- 172.21.15.64:0/3186034297 >> 172.21.15.64:0/3186034297 conn(0x55c285c4e460 msgr2=0x55c285fff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:03.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.427+0000 7fccbd5ab740 1 -- 172.21.15.64:0/3186034297 shutdown_connections 2024-09-21T04:13:03.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.427+0000 7fccbd5ab740 1 -- 172.21.15.64:0/3186034297 wait complete. 2024-09-21T04:13:03.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.428+0000 7fccbd5ab740 1 -- start start 2024-09-21T04:13:03.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.428+0000 7fccbd5ab740 1 Processor -- start 2024-09-21T04:13:03.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.428+0000 7fccbd5ab740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c285c311b0 0x55c28600ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:03.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.428+0000 7fccbd5ab740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c39390 0x55c28600d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:03.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.428+0000 7fccbd5ab740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c285c5a070 0x55c2860191a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:03.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.428+0000 7fccbd5ab740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c285dc8700 con 0x55c285c39390 2024-09-21T04:13:03.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.428+0000 7fccbd5ab740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c285f02160 con 0x55c285c5a070 2024-09-21T04:13:03.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.428+0000 7fccbd5ab740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c28600fe80 con 0x55c285c311b0 2024-09-21T04:13:03.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.428+0000 7fccbd5ab740 10 client.0 _ll_register_callbacks cb 0x55c285f1b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:03.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.429+0000 7fccaffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c39390 0x55c28600d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:03.443 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.429+0000 7fccaffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c39390 0x55c28600d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42090/0 (socket says 172.21.15.64:42090) 2024-09-21T04:13:03.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.429+0000 7fccaffff640 1 -- 172.21.15.64:0/948859901 learned_addr learned my addr 172.21.15.64:0/948859901 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:03.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.429+0000 7fccb4dd3640 1 --2- 172.21.15.64:0/948859901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c285c311b0 0x55c28600ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:03.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.429+0000 7fccb55d4640 1 --2- 172.21.15.64:0/948859901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c285c5a070 0x55c2860191a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:03.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.429+0000 7fccaffff640 1 -- 172.21.15.64:0/948859901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c285c311b0 msgr2=0x55c28600ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:03.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.429+0000 7fccaffff640 1 --2- 172.21.15.64:0/948859901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c285c311b0 0x55c28600ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.429+0000 7fccaffff640 1 -- 172.21.15.64:0/948859901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c285c5a070 msgr2=0x55c2860191a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:03.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.429+0000 7fccaffff640 1 --2- 172.21.15.64:0/948859901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c285c5a070 0x55c2860191a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.429+0000 7fccaffff640 1 -- 172.21.15.64:0/948859901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c285f2fc90 con 0x55c285c39390 2024-09-21T04:13:03.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.429+0000 7fccaffff640 1 --2- 172.21.15.64:0/948859901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c39390 0x55c28600d230 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fcca00ad730 tx=0x7fcca00eb8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c24fe7c70eda0c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:03.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.429+0000 7fcc8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:13:03.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.429+0000 7fcc8ffff640 1 -- 172.21.15.64:0/948859901 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca001f070 con 0x55c285c39390 2024-09-21T04:13:03.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.429+0000 7fcc8ffff640 1 -- 172.21.15.64:0/948859901 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7fcca0005510 con 0x55c285c39390 2024-09-21T04:13:03.444 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.429+0000 7fccbd5ab740 1 -- 172.21.15.64:0/948859901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcca00ad2b0 con 0x55c285c39390 2024-09-21T04:13:03.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.430+0000 7fcc8ffff640 1 -- 172.21.15.64:0/948859901 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fcca00e7e60 con 0x55c285c39390 2024-09-21T04:13:03.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.430+0000 7fccbd5ab740 10 client.17319 fetch_fsmap learned FSMap version 40 2024-09-21T04:13:03.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.430+0000 7fccbd5ab740 1 -- 172.21.15.64:0/948859901 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c28601bb60 con 0x55c285c39390 2024-09-21T04:13:03.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.430+0000 7fcc8ffff640 1 -- 172.21.15.64:0/948859901 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcca00ad2b0 con 0x55c285c39390 2024-09-21T04:13:03.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.430+0000 7fcc8ffff640 1 -- 172.21.15.64:0/948859901 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7fcca0086020 con 0x55c285c39390 2024-09-21T04:13:03.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.430+0000 7fccbd5ab740 10 client.17319 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:13:03.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.430+0000 7fccbd5ab740 10 client.17319 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:13:03.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.430+0000 7fccbd5ab740 20 client.17319 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:03.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.430+0000 7fccbd5ab740 1 --2- 172.21.15.64:0/948859901 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55c285c189f0 0x55c28601eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:03.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.430+0000 7fccbd5ab740 4 client.17319 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:13:03.445 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.430+0000 7fccbd5ab740 1 -- 172.21.15.64:0/948859901 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c285f79df0 con 0x55c285c189f0 2024-09-21T04:13:03.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.430+0000 7fccb4dd3640 1 --2- 172.21.15.64:0/948859901 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55c285c189f0 0x55c28601eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:03.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.431+0000 7fcc8ffff640 10 client.17319 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:13:03.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.431+0000 7fccb4dd3640 1 --2- 172.21.15.64:0/948859901 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55c285c189f0 0x55c28601eec0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1f676ca3007d31fb server_cookie=af4f1796a5f19604 in_seq=0 out_seq=0 2024-09-21T04:13:03.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.431+0000 7fcc8ffff640 1 -- 172.21.15.64:0/948859901 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c285f79df0 con 0x55c285c189f0 2024-09-21T04:13:03.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.432+0000 7fcc8ffff640 10 client.17319 handle_command_reply: tid=0 2024-09-21T04:13:03.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.432+0000 7fccbd5ab740 1 client.17319 shutdown 2024-09-21T04:13:03.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.432+0000 7fccbd5ab740 1 -- 172.21.15.64:0/948859901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c39390 msgr2=0x55c28600d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:03.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.432+0000 7fccbd5ab740 1 --2- 172.21.15.64:0/948859901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c39390 0x55c28600d230 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fcca00ad730 tx=0x7fcca00eb8f0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.432+0000 7fccbd5ab740 1 -- 172.21.15.64:0/948859901 shutdown_connections 2024-09-21T04:13:03.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.432+0000 7fccbd5ab740 1 --2- 172.21.15.64:0/948859901 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55c285c189f0 0x55c28601eec0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.432+0000 7fccbd5ab740 1 --2- 172.21.15.64:0/948859901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c285c5a070 0x55c2860191a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.432+0000 7fcc8ffff640 0 client.17319 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:13:03.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.432+0000 7fccbd5ab740 1 --2- 172.21.15.64:0/948859901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c39390 0x55c28600d230 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.432+0000 7fccbd5ab740 1 --2- 172.21.15.64:0/948859901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c285c311b0 0x55c28600ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.432+0000 7fccbd5ab740 1 -- 172.21.15.64:0/948859901 >> 172.21.15.64:0/948859901 conn(0x55c285c4e460 msgr2=0x55c286004830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:03.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.432+0000 7fccbd5ab740 1 -- 172.21.15.64:0/948859901 shutdown_connections 2024-09-21T04:13:03.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.432+0000 7fccbd5ab740 1 -- 172.21.15.64:0/948859901 wait complete. 2024-09-21T04:13:03.447 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.432+0000 7fccbd5ab740 20 client.17319 trim_cache size 0 max 16384 2024-09-21T04:13:03.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.445+0000 7fccbd5ab740 1 Processor -- start 2024-09-21T04:13:03.448 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.445+0000 7fccbd5ab740 1 -- start start 2024-09-21T04:13:03.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.445+0000 7fccbd5ab740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c285c189f0 0x55c28600d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:03.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.445+0000 7fccbd5ab740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c285c311b0 0x55c286019110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:03.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.446+0000 7fccbd5ab740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c39390 0x55c2860222d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:03.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.446+0000 7fccbd5ab740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fccb01b22c0 con 0x55c285c39390 2024-09-21T04:13:03.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.446+0000 7fccbd5ab740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fccb01afad0 con 0x55c285c189f0 2024-09-21T04:13:03.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.446+0000 7fccbd5ab740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fccb01ad940 con 0x55c285c311b0 2024-09-21T04:13:03.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.446+0000 7fccb55d4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c39390 0x55c2860222d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:03.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.446+0000 7fccb55d4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c39390 0x55c2860222d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42102/0 (socket says 172.21.15.64:42102) 2024-09-21T04:13:03.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.446+0000 7fccb55d4640 1 -- 172.21.15.64:0/1410894844 learned_addr learned my addr 172.21.15.64:0/1410894844 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:03.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.446+0000 7fccb4dd3640 1 --2- 172.21.15.64:0/1410894844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c285c189f0 0x55c28600d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:03.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.446+0000 7fccaffff640 1 --2- 172.21.15.64:0/1410894844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c285c311b0 0x55c286019110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:03.449 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.446+0000 7fccb55d4640 1 -- 172.21.15.64:0/1410894844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c285c311b0 msgr2=0x55c286019110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:03.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.446+0000 7fccb55d4640 1 --2- 172.21.15.64:0/1410894844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c285c311b0 0x55c286019110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.446+0000 7fccb55d4640 1 -- 172.21.15.64:0/1410894844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c285c189f0 msgr2=0x55c28600d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:03.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.446+0000 7fccb55d4640 1 --2- 172.21.15.64:0/1410894844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c285c189f0 0x55c28600d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.446+0000 7fccb55d4640 1 -- 172.21.15.64:0/1410894844 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c28600b1f0 con 0x55c285c39390 2024-09-21T04:13:03.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.446+0000 7fccb55d4640 1 --2- 172.21.15.64:0/1410894844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c39390 0x55c2860222d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fcca8033000 tx=0x7fcca803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a33882eb197302e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:03.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.446+0000 7fcca67fc640 1 -- 172.21.15.64:0/1410894844 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca804d070 con 0x55c285c39390 2024-09-21T04:13:03.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.447+0000 7fcca67fc640 1 -- 172.21.15.64:0/1410894844 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcca8005040 con 0x55c285c39390 2024-09-21T04:13:03.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.447+0000 7fcca67fc640 1 -- 172.21.15.64:0/1410894844 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca8035040 con 0x55c285c39390 2024-09-21T04:13:03.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.447+0000 7fccbd5ab740 1 -- 172.21.15.64:0/1410894844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c39390 msgr2=0x55c2860222d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:03.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.447+0000 7fccbd5ab740 1 --2- 172.21.15.64:0/1410894844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c39390 0x55c2860222d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fcca8033000 tx=0x7fcca803d000 comp rx=0 tx=0).stop 2024-09-21T04:13:03.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.447+0000 7fccbd5ab740 1 -- 172.21.15.64:0/1410894844 shutdown_connections 2024-09-21T04:13:03.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.447+0000 7fccbd5ab740 1 --2- 172.21.15.64:0/1410894844 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c39390 0x55c2860222d0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.447+0000 7fccbd5ab740 1 --2- 172.21.15.64:0/1410894844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c285c311b0 0x55c286019110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.447+0000 7fccbd5ab740 1 --2- 172.21.15.64:0/1410894844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c285c189f0 0x55c28600d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.447+0000 7fccbd5ab740 1 -- 172.21.15.64:0/1410894844 >> 172.21.15.64:0/1410894844 conn(0x55c285c4e460 msgr2=0x55c28600caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:03.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.447+0000 7fccbd5ab740 1 -- 172.21.15.64:0/1410894844 shutdown_connections 2024-09-21T04:13:03.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.447+0000 7fccbd5ab740 1 -- 172.21.15.64:0/1410894844 wait complete. 2024-09-21T04:13:03.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.448+0000 7fccbd5ab740 1 -- start start 2024-09-21T04:13:03.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.448+0000 7fccbd5ab740 1 Processor -- start 2024-09-21T04:13:03.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.448+0000 7fccbd5ab740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c285c189f0 0x55c286081010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:03.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.448+0000 7fccbd5ab740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c311b0 0x55c2860815f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:03.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.448+0000 7fccbd5ab740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c285c39390 0x55c286086610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:03.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.448+0000 7fccbd5ab740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fccb01adf90 con 0x55c285c311b0 2024-09-21T04:13:03.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.448+0000 7fccbd5ab740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fccb01af400 con 0x55c285c39390 2024-09-21T04:13:03.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.448+0000 7fccbd5ab740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fccb01ae660 con 0x55c285c189f0 2024-09-21T04:13:03.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.448+0000 7fccbd5ab740 10 client.0 _ll_register_callbacks cb 0x7fcca00ed050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:03.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.448+0000 7fccaffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c311b0 0x55c2860815f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:03.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.449+0000 7fccaffff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c311b0 0x55c2860815f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42116/0 (socket says 172.21.15.64:42116) 2024-09-21T04:13:03.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.449+0000 7fccaffff640 1 -- 172.21.15.64:0/3592139923 learned_addr learned my addr 172.21.15.64:0/3592139923 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:03.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.449+0000 7fccb4dd3640 1 --2- 172.21.15.64:0/3592139923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c285c189f0 0x55c286081010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:03.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.449+0000 7fccb55d4640 1 --2- 172.21.15.64:0/3592139923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c285c39390 0x55c286086610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:03.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.449+0000 7fccaffff640 1 -- 172.21.15.64:0/3592139923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c285c189f0 msgr2=0x55c286081010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:03.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.449+0000 7fccaffff640 1 --2- 172.21.15.64:0/3592139923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c285c189f0 0x55c286081010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.449+0000 7fccaffff640 1 -- 172.21.15.64:0/3592139923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c285c39390 msgr2=0x55c286086610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:03.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.449+0000 7fccaffff640 1 --2- 172.21.15.64:0/3592139923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c285c39390 0x55c286086610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.449+0000 7fccaffff640 1 -- 172.21.15.64:0/3592139923 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c286086c80 con 0x55c285c311b0 2024-09-21T04:13:03.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.449+0000 7fccb55d4640 1 --2- 172.21.15.64:0/3592139923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c285c39390 0x55c286086610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:13:03.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.449+0000 7fccaffff640 1 --2- 172.21.15.64:0/3592139923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c311b0 0x55c2860815f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fcca00b21f0 tx=0x7fcca00875e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934f6ced39b567cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:03.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.449+0000 7fcc8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:13:03.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.449+0000 7fcc8ffff640 1 -- 172.21.15.64:0/3592139923 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca001f070 con 0x55c285c311b0 2024-09-21T04:13:03.456 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.449+0000 7fcc8ffff640 1 -- 172.21.15.64:0/3592139923 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7fcca00b6f30 con 0x55c285c311b0 2024-09-21T04:13:03.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.449+0000 7fccbd5ab740 1 -- 172.21.15.64:0/3592139923 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcca8005040 con 0x55c285c311b0 2024-09-21T04:13:03.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.449+0000 7fcc8ffff640 1 -- 172.21.15.64:0/3592139923 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fcca00d3050 con 0x55c285c311b0 2024-09-21T04:13:03.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.449+0000 7fccbd5ab740 10 client.17325 fetch_fsmap learned FSMap version 40 2024-09-21T04:13:03.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.449+0000 7fccbd5ab740 1 -- 172.21.15.64:0/3592139923 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c28608e070 con 0x55c285c311b0 2024-09-21T04:13:03.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.450+0000 7fcc8ffff640 1 -- 172.21.15.64:0/3592139923 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcca8005040 con 0x55c285c311b0 2024-09-21T04:13:03.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.450+0000 7fcc8ffff640 1 -- 172.21.15.64:0/3592139923 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7fcca0086070 con 0x55c285c311b0 2024-09-21T04:13:03.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.450+0000 7fccbd5ab740 10 client.17325 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:13:03.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.450+0000 7fccbd5ab740 10 client.17325 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:13:03.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.450+0000 7fccbd5ab740 20 client.17325 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:03.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.450+0000 7fccbd5ab740 1 --2- 172.21.15.64:0/3592139923 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55c285c5a070 0x55c2860905a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:03.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.450+0000 7fccbd5ab740 4 client.17325 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:13:03.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.450+0000 7fccbd5ab740 1 -- 172.21.15.64:0/3592139923 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c285f758a0 con 0x55c285c5a070 2024-09-21T04:13:03.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.450+0000 7fccb4dd3640 1 --2- 172.21.15.64:0/3592139923 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55c285c5a070 0x55c2860905a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:03.457 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.450+0000 7fccb4dd3640 1 --2- 172.21.15.64:0/3592139923 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55c285c5a070 0x55c2860905a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c441256a1104135 server_cookie=d1cf31742c360f3b in_seq=0 out_seq=0 2024-09-21T04:13:03.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.450+0000 7fcc8ffff640 10 client.17325 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:13:03.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.451+0000 7fcc8ffff640 1 -- 172.21.15.64:0/3592139923 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c285f758a0 con 0x55c285c5a070 2024-09-21T04:13:03.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.451+0000 7fcc8ffff640 10 client.17325 handle_command_reply: tid=0 2024-09-21T04:13:03.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.451+0000 7fccbd5ab740 1 client.17325 shutdown 2024-09-21T04:13:03.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.451+0000 7fccbd5ab740 1 -- 172.21.15.64:0/3592139923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c311b0 msgr2=0x55c2860815f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:03.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.451+0000 7fccbd5ab740 1 --2- 172.21.15.64:0/3592139923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c311b0 0x55c2860815f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fcca00b21f0 tx=0x7fcca00875e0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.451+0000 7fccbd5ab740 1 -- 172.21.15.64:0/3592139923 shutdown_connections 2024-09-21T04:13:03.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.451+0000 7fccbd5ab740 1 --2- 172.21.15.64:0/3592139923 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x55c285c5a070 0x55c2860905a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.451+0000 7fccbd5ab740 1 --2- 172.21.15.64:0/3592139923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c285c39390 0x55c286086610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.451+0000 7fccbd5ab740 1 --2- 172.21.15.64:0/3592139923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c285c311b0 0x55c2860815f0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.451+0000 7fcc8ffff640 0 client.17325 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:13:03.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.451+0000 7fccbd5ab740 1 --2- 172.21.15.64:0/3592139923 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c285c189f0 0x55c286081010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.451+0000 7fccbd5ab740 1 -- 172.21.15.64:0/3592139923 >> 172.21.15.64:0/3592139923 conn(0x55c285c4e460 msgr2=0x55c2860026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:03.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.451+0000 7fccbd5ab740 1 -- 172.21.15.64:0/3592139923 shutdown_connections 2024-09-21T04:13:03.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.451+0000 7fccbd5ab740 1 -- 172.21.15.64:0/3592139923 wait complete. 2024-09-21T04:13:03.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.451+0000 7fccbd5ab740 20 client.17325 trim_cache size 0 max 16384 2024-09-21T04:13:03.459 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:13:03.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.452+0000 7fccb7037640 1 -- 172.21.15.64:0/1313210309 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcc9c0b6680 msgr2=0x7fcc9c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:03.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.452+0000 7fccb7037640 1 --2- 172.21.15.64:0/1313210309 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcc9c0b6680 0x7fcc9c0b8b90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcca8031440 tx=0x7fcca8005a20 comp rx=0 tx=0).stop 2024-09-21T04:13:03.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.452+0000 7fccb7037640 1 -- 172.21.15.64:0/1313210309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccb01083a0 msgr2=0x7fccb019aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:03.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.452+0000 7fccb7037640 1 --2- 172.21.15.64:0/1313210309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccb01083a0 0x7fccb019aa30 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fcca000ea50 tx=0x7fcca000ef10 comp rx=0 tx=0).stop 2024-09-21T04:13:03.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.452+0000 7fccb7037640 1 -- 172.21.15.64:0/1313210309 shutdown_connections 2024-09-21T04:13:03.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.452+0000 7fccb7037640 1 --2- 172.21.15.64:0/1313210309 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcc9c0b6680 0x7fcc9c0b8b90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.452+0000 7fccb7037640 1 --2- 172.21.15.64:0/1313210309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccb0108cf0 0x7fccb019fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.452+0000 7fccb7037640 1 --2- 172.21.15.64:0/1313210309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fccb01083a0 0x7fccb019aa30 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.452+0000 7fccb7037640 1 --2- 172.21.15.64:0/1313210309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccb0102a60 0x7fccb019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:03.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.452+0000 7fccb7037640 1 -- 172.21.15.64:0/1313210309 >> 172.21.15.64:0/1313210309 conn(0x7fccb0079730 msgr2=0x7fccb0104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:03.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.452+0000 7fccb7037640 1 -- 172.21.15.64:0/1313210309 shutdown_connections 2024-09-21T04:13:03.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:03.452+0000 7fccb7037640 1 -- 172.21.15.64:0/1313210309 wait complete. 2024-09-21T04:13:04.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:03 smithi131 ceph-mon[35193]: pgmap v598: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-21T04:13:04.193 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:03 smithi064 ceph-mon[31180]: pgmap v598: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-21T04:13:04.222 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T04:13:04.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:03 smithi119 ceph-mon[34811]: pgmap v598: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-21T04:13:04.480 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:13:04.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.973+0000 7f4f2db9b640 1 -- 172.21.15.64:0/3868208734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f28102a60 msgr2=0x7f4f28102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:04.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.973+0000 7f4f2db9b640 1 --2- 172.21.15.64:0/3868208734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f28102a60 0x7f4f28102e70 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f4f18009900 tx=0x7f4f1802a410 comp rx=0 tx=0).stop 2024-09-21T04:13:04.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.973+0000 7f4f2db9b640 1 -- 172.21.15.64:0/3868208734 shutdown_connections 2024-09-21T04:13:04.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.973+0000 7f4f2db9b640 1 --2- 172.21.15.64:0/3868208734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f28108cf0 0x7f4f2810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:04.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.973+0000 7f4f2db9b640 1 --2- 172.21.15.64:0/3868208734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f281083a0 0x7f4f281087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:04.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.973+0000 7f4f2db9b640 1 --2- 172.21.15.64:0/3868208734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f28102a60 0x7f4f28102e70 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:04.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.973+0000 7f4f2db9b640 1 -- 172.21.15.64:0/3868208734 >> 172.21.15.64:0/3868208734 conn(0x7f4f28079730 msgr2=0x7f4f28079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.973+0000 7f4f2db9b640 1 -- 172.21.15.64:0/3868208734 shutdown_connections 2024-09-21T04:13:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.973+0000 7f4f2db9b640 1 -- 172.21.15.64:0/3868208734 wait complete. 2024-09-21T04:13:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.974+0000 7f4f2db9b640 1 Processor -- start 2024-09-21T04:13:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.974+0000 7f4f2db9b640 1 -- start start 2024-09-21T04:13:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.974+0000 7f4f2db9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f28102a60 0x7f4f2819b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.974+0000 7f4f2db9b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f281083a0 0x7f4f2819b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.974+0000 7f4f2db9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f28108cf0 0x7f4f2819bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.974+0000 7f4f2db9b640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4f28112960 con 0x7f4f281083a0 2024-09-21T04:13:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.974+0000 7f4f2db9b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4f281127e0 con 0x7f4f28102a60 2024-09-21T04:13:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.974+0000 7f4f2db9b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4f28112ae0 con 0x7f4f28108cf0 2024-09-21T04:13:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.975+0000 7f4f26ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f281083a0 0x7f4f2819b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.975+0000 7f4f26ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f281083a0 0x7f4f2819b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42136/0 (socket says 172.21.15.64:42136) 2024-09-21T04:13:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.975+0000 7f4f26ffd640 1 -- 172.21.15.64:0/3660273600 learned_addr learned my addr 172.21.15.64:0/3660273600 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.975+0000 7f4f27fff640 1 --2- 172.21.15.64:0/3660273600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f28108cf0 0x7f4f2819bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:04.991 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.975+0000 7f4f277fe640 1 --2- 172.21.15.64:0/3660273600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f28102a60 0x7f4f2819b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:04.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.975+0000 7f4f26ffd640 1 -- 172.21.15.64:0/3660273600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f28108cf0 msgr2=0x7f4f2819bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:04.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.975+0000 7f4f26ffd640 1 --2- 172.21.15.64:0/3660273600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f28108cf0 0x7f4f2819bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:04.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.975+0000 7f4f26ffd640 1 -- 172.21.15.64:0/3660273600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f28102a60 msgr2=0x7f4f2819b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:04.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.975+0000 7f4f26ffd640 1 --2- 172.21.15.64:0/3660273600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f28102a60 0x7f4f2819b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:04.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.975+0000 7f4f26ffd640 1 -- 172.21.15.64:0/3660273600 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f281a64e0 con 0x7f4f281083a0 2024-09-21T04:13:04.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.975+0000 7f4f27fff640 1 --2- 172.21.15.64:0/3660273600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f28108cf0 0x7f4f2819bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:13:04.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.975+0000 7f4f26ffd640 1 --2- 172.21.15.64:0/3660273600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f281083a0 0x7f4f2819b7f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00d740 tx=0x7f4f1c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:04.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.975+0000 7f4f24ff9640 1 -- 172.21.15.64:0/3660273600 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c017070 con 0x7f4f281083a0 2024-09-21T04:13:04.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.975+0000 7f4f24ff9640 1 -- 172.21.15.64:0/3660273600 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f1c00de50 con 0x7f4f281083a0 2024-09-21T04:13:04.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.975+0000 7f4f2db9b640 1 -- 172.21.15.64:0/3660273600 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f281a67d0 con 0x7f4f281083a0 2024-09-21T04:13:04.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.975+0000 7f4f24ff9640 1 -- 172.21.15.64:0/3660273600 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c004ce0 con 0x7f4f281083a0 2024-09-21T04:13:04.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.975+0000 7f4f2db9b640 1 -- 172.21.15.64:0/3660273600 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f281a6cc0 con 0x7f4f281083a0 2024-09-21T04:13:04.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.978+0000 7f4f3410f740 1 Processor -- start 2024-09-21T04:13:04.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.978+0000 7f4f3410f740 1 -- start start 2024-09-21T04:13:04.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.978+0000 7f4f3410f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560aa955b070 0x560aa99026a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:04.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.978+0000 7f4f3410f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa95321b0 0x560aa9902be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:04.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.978+0000 7f4f3410f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa953a390 0x560aa9903120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:04.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.978+0000 7f4f3410f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560aa9878ea0 con 0x560aa95321b0 2024-09-21T04:13:04.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.978+0000 7f4f3410f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560aa98682d0 con 0x560aa953a390 2024-09-21T04:13:04.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.978+0000 7f4f3410f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560aa986c0e0 con 0x560aa955b070 2024-09-21T04:13:04.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.978+0000 7f4f277fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa95321b0 0x560aa9902be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:04.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.978+0000 7f4f277fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa95321b0 0x560aa9902be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42138/0 (socket says 172.21.15.64:42138) 2024-09-21T04:13:04.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.978+0000 7f4f277fe640 1 -- 172.21.15.64:0/4119159479 learned_addr learned my addr 172.21.15.64:0/4119159479 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:04.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.978+0000 7f4f24ff9640 1 -- 172.21.15.64:0/3660273600 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f4f1c004e80 con 0x7f4f281083a0 2024-09-21T04:13:04.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.979+0000 7f4f27fff640 1 --2- 172.21.15.64:0/4119159479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560aa955b070 0x560aa99026a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:04.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.979+0000 7f4f277fe640 1 -- 172.21.15.64:0/4119159479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560aa955b070 msgr2=0x560aa99026a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:04.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.979+0000 7f4f277fe640 1 --2- 172.21.15.64:0/4119159479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560aa955b070 0x560aa99026a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:04.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.979+0000 7f4f277fe640 1 -- 172.21.15.64:0/4119159479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa953a390 msgr2=0x560aa9903120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:04.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.979+0000 7f4f277fe640 1 --2- 172.21.15.64:0/4119159479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa953a390 0x560aa9903120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:04.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.979+0000 7f4f277fe640 1 -- 172.21.15.64:0/4119159479 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aa95c6650 con 0x560aa95321b0 2024-09-21T04:13:04.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.979+0000 7f4f27fff640 1 --2- 172.21.15.64:0/4119159479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560aa955b070 0x560aa99026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:13:04.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.979+0000 7f4f277fe640 1 --2- 172.21.15.64:0/4119159479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa95321b0 0x560aa9902be0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f4f1802a930 tx=0x7f4f18032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb1c7bd589d2a32 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:04.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.979+0000 7f4f0dffb640 1 -- 172.21.15.64:0/4119159479 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1803f070 con 0x560aa95321b0 2024-09-21T04:13:04.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.979+0000 7f4f0dffb640 1 -- 172.21.15.64:0/4119159479 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f180316a0 con 0x560aa95321b0 2024-09-21T04:13:04.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.979+0000 7f4f0dffb640 1 -- 172.21.15.64:0/4119159479 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f18035070 con 0x560aa95321b0 2024-09-21T04:13:04.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.979+0000 7f4f24ff9640 1 --2- 172.21.15.64:0/3660273600 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4f100b6680 0x7f4f100b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:04.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.979+0000 7f4f27fff640 1 --2- 172.21.15.64:0/3660273600 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4f100b6680 0x7f4f100b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:04.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.979+0000 7f4f24ff9640 1 -- 172.21.15.64:0/3660273600 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7f4f1c0e7170 con 0x7f4f281083a0 2024-09-21T04:13:04.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.980+0000 7f4f3410f740 1 -- 172.21.15.64:0/4119159479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa95321b0 msgr2=0x560aa9902be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:04.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.980+0000 7f4f27fff640 1 --2- 172.21.15.64:0/3660273600 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4f100b6680 0x7f4f100b8b90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4f20007fb0 tx=0x7f4f20007990 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:04.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.980+0000 7f4f3410f740 1 --2- 172.21.15.64:0/4119159479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa95321b0 0x560aa9902be0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f4f1802a930 tx=0x7f4f18032ce0 comp rx=0 tx=0).stop 2024-09-21T04:13:04.994 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.980+0000 7f4f3410f740 1 -- 172.21.15.64:0/4119159479 shutdown_connections 2024-09-21T04:13:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.980+0000 7f4f3410f740 1 --2- 172.21.15.64:0/4119159479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa953a390 0x560aa9903120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.980+0000 7f4f3410f740 1 --2- 172.21.15.64:0/4119159479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa95321b0 0x560aa9902be0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.980+0000 7f4f3410f740 1 --2- 172.21.15.64:0/4119159479 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560aa955b070 0x560aa99026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.980+0000 7f4f3410f740 1 -- 172.21.15.64:0/4119159479 >> 172.21.15.64:0/4119159479 conn(0x560aa954f460 msgr2=0x560aa9900590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.980+0000 7f4f3410f740 1 -- 172.21.15.64:0/4119159479 shutdown_connections 2024-09-21T04:13:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.980+0000 7f4f3410f740 1 -- 172.21.15.64:0/4119159479 wait complete. 2024-09-21T04:13:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.981+0000 7f4f3410f740 1 -- start start 2024-09-21T04:13:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.981+0000 7f4f3410f740 1 Processor -- start 2024-09-21T04:13:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.981+0000 7f4f3410f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa95321b0 0x560aa990de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.981+0000 7f4f3410f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa953a390 0x560aa990e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.981+0000 7f4f3410f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560aa955b070 0x560aa991a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.981+0000 7f4f3410f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560aa96c9700 con 0x560aa95321b0 2024-09-21T04:13:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.981+0000 7f4f3410f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560aa9803160 con 0x560aa953a390 2024-09-21T04:13:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.981+0000 7f4f3410f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560aa9910e20 con 0x560aa955b070 2024-09-21T04:13:04.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.981+0000 7f4f3410f740 10 client.0 _ll_register_callbacks cb 0x560aa981c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:04.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.981+0000 7f4f277fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa95321b0 0x560aa990de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:04.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.981+0000 7f4f277fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa95321b0 0x560aa990de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42144/0 (socket says 172.21.15.64:42144) 2024-09-21T04:13:04.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.981+0000 7f4f277fe640 1 -- 172.21.15.64:0/28524926 learned_addr learned my addr 172.21.15.64:0/28524926 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:04.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.982+0000 7f4f26ffd640 1 --2- 172.21.15.64:0/28524926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa953a390 0x560aa990e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:04.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.982+0000 7f4f27fff640 1 --2- 172.21.15.64:0/28524926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560aa955b070 0x560aa991a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:04.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.982+0000 7f4f277fe640 1 -- 172.21.15.64:0/28524926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560aa955b070 msgr2=0x560aa991a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:04.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.982+0000 7f4f277fe640 1 --2- 172.21.15.64:0/28524926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560aa955b070 0x560aa991a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:04.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.982+0000 7f4f277fe640 1 -- 172.21.15.64:0/28524926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa953a390 msgr2=0x560aa990e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:04.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.982+0000 7f4f277fe640 1 --2- 172.21.15.64:0/28524926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa953a390 0x560aa990e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:04.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.982+0000 7f4f277fe640 1 -- 172.21.15.64:0/28524926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aa9830c90 con 0x560aa95321b0 2024-09-21T04:13:04.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.982+0000 7f4f277fe640 1 --2- 172.21.15.64:0/28524926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa95321b0 0x560aa990de40 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f4f18005940 tx=0x7f4f18005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451819e74e3d966 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:04.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.982+0000 7f4f06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:13:04.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.982+0000 7f4f06ffd640 1 -- 172.21.15.64:0/28524926 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1804b060 con 0x560aa95321b0 2024-09-21T04:13:04.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.982+0000 7f4f06ffd640 1 -- 172.21.15.64:0/28524926 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7f4f18004440 con 0x560aa95321b0 2024-09-21T04:13:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.982+0000 7f4f3410f740 1 -- 172.21.15.64:0/28524926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f180316a0 con 0x560aa95321b0 2024-09-21T04:13:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.982+0000 7f4f06ffd640 1 -- 172.21.15.64:0/28524926 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f4f1803f030 con 0x560aa95321b0 2024-09-21T04:13:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.982+0000 7f4f3410f740 10 client.17337 fetch_fsmap learned FSMap version 40 2024-09-21T04:13:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.982+0000 7f4f3410f740 1 -- 172.21.15.64:0/28524926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560aa991cbe0 con 0x560aa95321b0 2024-09-21T04:13:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.983+0000 7f4f06ffd640 1 -- 172.21.15.64:0/28524926 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f180316a0 con 0x560aa95321b0 2024-09-21T04:13:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.983+0000 7f4f06ffd640 1 -- 172.21.15.64:0/28524926 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f4f1803b7d0 con 0x560aa95321b0 2024-09-21T04:13:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.983+0000 7f4f3410f740 10 client.17337 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:13:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.983+0000 7f4f3410f740 10 client.17337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:13:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.983+0000 7f4f3410f740 20 client.17337 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.983+0000 7f4f3410f740 1 --2- 172.21.15.64:0/28524926 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x560aa95199f0 0x560aa991ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.983+0000 7f4f3410f740 4 client.17337 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:13:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.983+0000 7f4f3410f740 1 -- 172.21.15.64:0/28524926 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560aa987adf0 con 0x560aa95199f0 2024-09-21T04:13:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.983+0000 7f4f26ffd640 1 --2- 172.21.15.64:0/28524926 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x560aa95199f0 0x560aa991ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.984+0000 7f4f26ffd640 1 --2- 172.21.15.64:0/28524926 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x560aa95199f0 0x560aa991ff40 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=259dda58a65d10f7 server_cookie=c304fbd5ec5264be in_seq=0 out_seq=0 2024-09-21T04:13:04.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.984+0000 7f4f06ffd640 10 client.17337 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:04.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.984+0000 7f4f06ffd640 1 -- 172.21.15.64:0/28524926 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560aa987adf0 con 0x560aa95199f0 2024-09-21T04:13:04.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.984+0000 7f4f06ffd640 10 client.17337 handle_command_reply: tid=0 2024-09-21T04:13:04.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.985+0000 7f4f3410f740 1 client.17337 shutdown 2024-09-21T04:13:04.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.985+0000 7f4f3410f740 1 -- 172.21.15.64:0/28524926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa95321b0 msgr2=0x560aa990de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:04.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.985+0000 7f4f3410f740 1 --2- 172.21.15.64:0/28524926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa95321b0 0x560aa990de40 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f4f18005940 tx=0x7f4f18005f30 comp rx=0 tx=0).stop 2024-09-21T04:13:04.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.985+0000 7f4f3410f740 1 -- 172.21.15.64:0/28524926 shutdown_connections 2024-09-21T04:13:04.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.985+0000 7f4f3410f740 1 --2- 172.21.15.64:0/28524926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560aa955b070 0x560aa991a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:04.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.985+0000 7f4f3410f740 1 --2- 172.21.15.64:0/28524926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa953a390 0x560aa990e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:04.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.985+0000 7f4f3410f740 1 --2- 172.21.15.64:0/28524926 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x560aa95199f0 0x560aa991ff40 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:04.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.985+0000 7f4f3410f740 1 --2- 172.21.15.64:0/28524926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa95321b0 0x560aa990de40 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:04.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.985+0000 7f4f06ffd640 0 client.17337 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:04.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.985+0000 7f4f3410f740 1 -- 172.21.15.64:0/28524926 >> 172.21.15.64:0/28524926 conn(0x560aa954f460 msgr2=0x560aa9905830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:04.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.985+0000 7f4f3410f740 1 -- 172.21.15.64:0/28524926 shutdown_connections 2024-09-21T04:13:04.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.985+0000 7f4f3410f740 1 -- 172.21.15.64:0/28524926 wait complete. 2024-09-21T04:13:04.999 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.985+0000 7f4f3410f740 20 client.17337 trim_cache size 0 max 16384 2024-09-21T04:13:05.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.998+0000 7f4f3410f740 1 Processor -- start 2024-09-21T04:13:05.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.998+0000 7f4f3410f740 1 -- start start 2024-09-21T04:13:05.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.999+0000 7f4f3410f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa95321b0 0x560aa9903de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:05.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.999+0000 7f4f3410f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa953a390 0x560aa991a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:05.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.999+0000 7f4f3410f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560aa955b070 0x560aa990b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:05.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.999+0000 7f4f3410f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4f281b2160 con 0x560aa95321b0 2024-09-21T04:13:05.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.999+0000 7f4f3410f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4f281af970 con 0x560aa953a390 2024-09-21T04:13:05.002 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.999+0000 7f4f3410f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4f281ad7e0 con 0x560aa955b070 2024-09-21T04:13:05.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.999+0000 7f4f277fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa95321b0 0x560aa9903de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:05.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.999+0000 7f4f277fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa95321b0 0x560aa9903de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42152/0 (socket says 172.21.15.64:42152) 2024-09-21T04:13:05.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.999+0000 7f4f277fe640 1 -- 172.21.15.64:0/740379904 learned_addr learned my addr 172.21.15.64:0/740379904 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:05.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.999+0000 7f4f26ffd640 1 --2- 172.21.15.64:0/740379904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa953a390 0x560aa991a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:05.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.999+0000 7f4f27fff640 1 --2- 172.21.15.64:0/740379904 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560aa955b070 0x560aa990b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:05.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.999+0000 7f4f277fe640 1 -- 172.21.15.64:0/740379904 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560aa955b070 msgr2=0x560aa990b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:05.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.999+0000 7f4f277fe640 1 --2- 172.21.15.64:0/740379904 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560aa955b070 0x560aa990b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:05.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.999+0000 7f4f277fe640 1 -- 172.21.15.64:0/740379904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa953a390 msgr2=0x560aa991a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:05.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.999+0000 7f4f277fe640 1 --2- 172.21.15.64:0/740379904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa953a390 0x560aa991a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:05.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.999+0000 7f4f277fe640 1 -- 172.21.15.64:0/740379904 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aa9900e80 con 0x560aa95321b0 2024-09-21T04:13:05.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.999+0000 7f4f277fe640 1 --2- 172.21.15.64:0/740379904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa95321b0 0x560aa9903de0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f4f18031440 tx=0x7f4f18001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36bab9c4df0a3d62 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:05.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:04.999+0000 7f4f0d7fa640 1 -- 172.21.15.64:0/740379904 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1805b070 con 0x560aa95321b0 2024-09-21T04:13:05.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.000+0000 7f4f0d7fa640 1 -- 172.21.15.64:0/740379904 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f18004440 con 0x560aa95321b0 2024-09-21T04:13:05.003 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.000+0000 7f4f0d7fa640 1 -- 172.21.15.64:0/740379904 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1804b070 con 0x560aa95321b0 2024-09-21T04:13:05.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.000+0000 7f4f3410f740 1 -- 172.21.15.64:0/740379904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa95321b0 msgr2=0x560aa9903de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:05.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.000+0000 7f4f3410f740 1 --2- 172.21.15.64:0/740379904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa95321b0 0x560aa9903de0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f4f18031440 tx=0x7f4f18001e70 comp rx=0 tx=0).stop 2024-09-21T04:13:05.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.000+0000 7f4f3410f740 1 -- 172.21.15.64:0/740379904 shutdown_connections 2024-09-21T04:13:05.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.000+0000 7f4f3410f740 1 --2- 172.21.15.64:0/740379904 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560aa955b070 0x560aa990b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:05.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.000+0000 7f4f3410f740 1 --2- 172.21.15.64:0/740379904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa953a390 0x560aa991a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:05.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.000+0000 7f4f3410f740 1 --2- 172.21.15.64:0/740379904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa95321b0 0x560aa9903de0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:05.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.000+0000 7f4f3410f740 1 -- 172.21.15.64:0/740379904 >> 172.21.15.64:0/740379904 conn(0x560aa954f460 msgr2=0x560aa99036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:05.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.000+0000 7f4f3410f740 1 -- 172.21.15.64:0/740379904 shutdown_connections 2024-09-21T04:13:05.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.000+0000 7f4f3410f740 1 -- 172.21.15.64:0/740379904 wait complete. 2024-09-21T04:13:05.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.001+0000 7f4f3410f740 1 -- start start 2024-09-21T04:13:05.007 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.001+0000 7f4f3410f740 1 Processor -- start 2024-09-21T04:13:05.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.001+0000 7f4f3410f740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560aa95321b0 0x560aa9921b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:05.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.001+0000 7f4f3410f740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa953a390 0x560aa99220e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:05.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.001+0000 7f4f3410f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa955b070 0x560aa99838a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:05.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.001+0000 7f4f3410f740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4f281ade30 con 0x560aa953a390 2024-09-21T04:13:05.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.001+0000 7f4f3410f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4f281af2a0 con 0x560aa955b070 2024-09-21T04:13:05.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.001+0000 7f4f3410f740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4f281ae500 con 0x560aa95321b0 2024-09-21T04:13:05.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.001+0000 7f4f3410f740 10 client.0 _ll_register_callbacks cb 0x7f4f18032bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:05.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.002+0000 7f4f26ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa953a390 0x560aa99220e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:05.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.002+0000 7f4f26ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa953a390 0x560aa99220e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42164/0 (socket says 172.21.15.64:42164) 2024-09-21T04:13:05.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.002+0000 7f4f26ffd640 1 -- 172.21.15.64:0/853489730 learned_addr learned my addr 172.21.15.64:0/853489730 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:05.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.002+0000 7f4f277fe640 1 --2- 172.21.15.64:0/853489730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560aa95321b0 0x560aa9921b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:05.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.002+0000 7f4f27fff640 1 --2- 172.21.15.64:0/853489730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa955b070 0x560aa99838a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:05.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.002+0000 7f4f26ffd640 1 -- 172.21.15.64:0/853489730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560aa95321b0 msgr2=0x560aa9921b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:05.008 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.002+0000 7f4f26ffd640 1 --2- 172.21.15.64:0/853489730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560aa95321b0 0x560aa9921b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.002+0000 7f4f26ffd640 1 -- 172.21.15.64:0/853489730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa955b070 msgr2=0x560aa99838a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.002+0000 7f4f26ffd640 1 --2- 172.21.15.64:0/853489730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa955b070 0x560aa99838a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.002+0000 7f4f26ffd640 1 -- 172.21.15.64:0/853489730 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aa9983f10 con 0x560aa953a390 2024-09-21T04:13:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.002+0000 7f4f277fe640 1 --2- 172.21.15.64:0/853489730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560aa95321b0 0x560aa9921b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:13:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.002+0000 7f4f26ffd640 1 --2- 172.21.15.64:0/853489730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa953a390 0x560aa99220e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0a95d0 tx=0x7f4f1c009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f98585dd743704c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.002+0000 7f4f06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:13:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.002+0000 7f4f06ffd640 1 -- 172.21.15.64:0/853489730 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c020040 con 0x560aa953a390 2024-09-21T04:13:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.002+0000 7f4f06ffd640 1 -- 172.21.15.64:0/853489730 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7f4f1c098180 con 0x560aa953a390 2024-09-21T04:13:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.002+0000 7f4f3410f740 1 -- 172.21.15.64:0/853489730 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f18004440 con 0x560aa953a390 2024-09-21T04:13:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.002+0000 7f4f06ffd640 1 -- 172.21.15.64:0/853489730 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f4f1c0b4e60 con 0x560aa953a390 2024-09-21T04:13:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.002+0000 7f4f3410f740 10 client.17343 fetch_fsmap learned FSMap version 40 2024-09-21T04:13:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.002+0000 7f4f3410f740 1 -- 172.21.15.64:0/853489730 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560aa998b300 con 0x560aa953a390 2024-09-21T04:13:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.003+0000 7f4f06ffd640 1 -- 172.21.15.64:0/853489730 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f18004440 con 0x560aa953a390 2024-09-21T04:13:05.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.003+0000 7f4f06ffd640 1 -- 172.21.15.64:0/853489730 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f4f1c0a8950 con 0x560aa953a390 2024-09-21T04:13:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.003+0000 7f4f3410f740 10 client.17343 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:13:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.003+0000 7f4f3410f740 10 client.17343 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:13:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.003+0000 7f4f3410f740 20 client.17343 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.003+0000 7f4f3410f740 1 --2- 172.21.15.64:0/853489730 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x560aa94eeda0 0x560aa998dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.003+0000 7f4f3410f740 4 client.17343 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:13:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.003+0000 7f4f3410f740 1 -- 172.21.15.64:0/853489730 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560aa98768a0 con 0x560aa94eeda0 2024-09-21T04:13:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.003+0000 7f4f277fe640 1 --2- 172.21.15.64:0/853489730 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x560aa94eeda0 0x560aa998dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.004+0000 7f4f06ffd640 10 client.17343 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.004+0000 7f4f277fe640 1 --2- 172.21.15.64:0/853489730 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x560aa94eeda0 0x560aa998dae0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efd150756eb3bb2 server_cookie=34fd5f937926d4ca in_seq=0 out_seq=0 2024-09-21T04:13:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.004+0000 7f4f06ffd640 1 -- 172.21.15.64:0/853489730 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560aa98768a0 con 0x560aa94eeda0 2024-09-21T04:13:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.004+0000 7f4f06ffd640 10 client.17343 handle_command_reply: tid=0 2024-09-21T04:13:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.004+0000 7f4f3410f740 1 client.17343 shutdown 2024-09-21T04:13:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.004+0000 7f4f3410f740 1 -- 172.21.15.64:0/853489730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa953a390 msgr2=0x560aa99220e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:05.010 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.004+0000 7f4f3410f740 1 --2- 172.21.15.64:0/853489730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa953a390 0x560aa99220e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0a95d0 tx=0x7f4f1c009e60 comp rx=0 tx=0).stop 2024-09-21T04:13:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.004+0000 7f4f3410f740 1 -- 172.21.15.64:0/853489730 shutdown_connections 2024-09-21T04:13:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.004+0000 7f4f3410f740 1 --2- 172.21.15.64:0/853489730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560aa955b070 0x560aa99838a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.004+0000 7f4f3410f740 1 --2- 172.21.15.64:0/853489730 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x560aa94eeda0 0x560aa998dae0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.004+0000 7f4f3410f740 1 --2- 172.21.15.64:0/853489730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560aa953a390 0x560aa99220e0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.004+0000 7f4f3410f740 1 --2- 172.21.15.64:0/853489730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560aa95321b0 0x560aa9921b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.004+0000 7f4f3410f740 1 -- 172.21.15.64:0/853489730 >> 172.21.15.64:0/853489730 conn(0x560aa954f460 msgr2=0x560aa9905f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.004+0000 7f4f06ffd640 0 client.17343 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.004+0000 7f4f3410f740 1 -- 172.21.15.64:0/853489730 shutdown_connections 2024-09-21T04:13:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.004+0000 7f4f3410f740 1 -- 172.21.15.64:0/853489730 wait complete. 2024-09-21T04:13:05.011 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.004+0000 7f4f3410f740 20 client.17343 trim_cache size 0 max 16384 2024-09-21T04:13:05.012 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:13:05.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.005+0000 7f4f2db9b640 1 -- 172.21.15.64:0/3660273600 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4f100b6680 msgr2=0x7f4f100b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:05.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.005+0000 7f4f2db9b640 1 --2- 172.21.15.64:0/3660273600 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4f100b6680 0x7f4f100b8b90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4f20007fb0 tx=0x7f4f20007990 comp rx=0 tx=0).stop 2024-09-21T04:13:05.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.005+0000 7f4f2db9b640 1 -- 172.21.15.64:0/3660273600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f281083a0 msgr2=0x7f4f2819b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:05.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.005+0000 7f4f2db9b640 1 --2- 172.21.15.64:0/3660273600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f281083a0 0x7f4f2819b7f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00d740 tx=0x7f4f1c00dc00 comp rx=0 tx=0).stop 2024-09-21T04:13:05.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.005+0000 7f4f2db9b640 1 -- 172.21.15.64:0/3660273600 shutdown_connections 2024-09-21T04:13:05.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.005+0000 7f4f2db9b640 1 --2- 172.21.15.64:0/3660273600 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f4f100b6680 0x7f4f100b8b90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:05.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.005+0000 7f4f2db9b640 1 --2- 172.21.15.64:0/3660273600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f28108cf0 0x7f4f2819bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:05.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.005+0000 7f4f2db9b640 1 --2- 172.21.15.64:0/3660273600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4f281083a0 0x7f4f2819b7f0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:05.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.005+0000 7f4f2db9b640 1 --2- 172.21.15.64:0/3660273600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f28102a60 0x7f4f2819b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:05.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.005+0000 7f4f2db9b640 1 -- 172.21.15.64:0/3660273600 >> 172.21.15.64:0/3660273600 conn(0x7f4f28079730 msgr2=0x7f4f28101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:05.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.005+0000 7f4f2db9b640 1 -- 172.21.15.64:0/3660273600 shutdown_connections 2024-09-21T04:13:05.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:05.005+0000 7f4f2db9b640 1 -- 172.21.15.64:0/3660273600 wait complete. 2024-09-21T04:13:06.011 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:05 smithi064 ceph-mon[31180]: pgmap v599: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-21T04:13:06.053 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:05 smithi131 ceph-mon[35193]: pgmap v599: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-21T04:13:06.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:05 smithi119 ceph-mon[34811]: pgmap v599: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-21T04:13:06.764 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T04:13:06.765 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T04:13:07.037 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:13:07.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.526+0000 7efcfc828640 1 -- 172.21.15.64:0/675092165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcf8109e40 msgr2=0x7efcf8110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:07.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.526+0000 7efcfc828640 1 --2- 172.21.15.64:0/675092165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcf8109e40 0x7efcf8110670 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009ca0 tx=0x7efcf002a8c0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.526+0000 7efcfc828640 1 -- 172.21.15.64:0/675092165 shutdown_connections 2024-09-21T04:13:07.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.526+0000 7efcfc828640 1 --2- 172.21.15.64:0/675092165 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcf8109e40 0x7efcf8110670 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.526+0000 7efcfc828640 1 --2- 172.21.15.64:0/675092165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efcf81094f0 0x7efcf8109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.526+0000 7efcfc828640 1 --2- 172.21.15.64:0/675092165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcf8103bb0 0x7efcf8103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.526+0000 7efcfc828640 1 -- 172.21.15.64:0/675092165 >> 172.21.15.64:0/675092165 conn(0x7efcf80ff600 msgr2=0x7efcf8101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:07.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.526+0000 7efcfc828640 1 -- 172.21.15.64:0/675092165 shutdown_connections 2024-09-21T04:13:07.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.526+0000 7efcfc828640 1 -- 172.21.15.64:0/675092165 wait complete. 2024-09-21T04:13:07.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.527+0000 7efcfc828640 1 Processor -- start 2024-09-21T04:13:07.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.527+0000 7efcfc828640 1 -- start start 2024-09-21T04:13:07.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.527+0000 7efcfc828640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcf8103bb0 0x7efcf819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:07.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.527+0000 7efcfc828640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efcf81094f0 0x7efcf819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:07.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.527+0000 7efcfc828640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcf8109e40 0x7efcf819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:07.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.527+0000 7efcfc828640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efcf8113b00 con 0x7efcf8103bb0 2024-09-21T04:13:07.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.527+0000 7efcfc828640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efcf8113980 con 0x7efcf8109e40 2024-09-21T04:13:07.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.527+0000 7efcfc828640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efcf8113c80 con 0x7efcf81094f0 2024-09-21T04:13:07.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.527+0000 7efcf659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcf8103bb0 0x7efcf819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:07.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.527+0000 7efcf659c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcf8103bb0 0x7efcf819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42174/0 (socket says 172.21.15.64:42174) 2024-09-21T04:13:07.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.527+0000 7efcf659c640 1 -- 172.21.15.64:0/2197516477 learned_addr learned my addr 172.21.15.64:0/2197516477 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:07.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.527+0000 7efcf5d9b640 1 --2- 172.21.15.64:0/2197516477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efcf81094f0 0x7efcf819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:07.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.528+0000 7efcf6d9d640 1 --2- 172.21.15.64:0/2197516477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcf8109e40 0x7efcf819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:07.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.528+0000 7efcf659c640 1 -- 172.21.15.64:0/2197516477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efcf81094f0 msgr2=0x7efcf819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:07.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.528+0000 7efcf659c640 1 --2- 172.21.15.64:0/2197516477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efcf81094f0 0x7efcf819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.528+0000 7efcf659c640 1 -- 172.21.15.64:0/2197516477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcf8109e40 msgr2=0x7efcf819fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:07.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.528+0000 7efcf659c640 1 --2- 172.21.15.64:0/2197516477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcf8109e40 0x7efcf819fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.528+0000 7efcf659c640 1 -- 172.21.15.64:0/2197516477 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf81a03e0 con 0x7efcf8103bb0 2024-09-21T04:13:07.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.528+0000 7efcf659c640 1 --2- 172.21.15.64:0/2197516477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcf8103bb0 0x7efcf819a470 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7efce800d950 tx=0x7efce800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:07.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.528+0000 7efce77fe640 1 -- 172.21.15.64:0/2197516477 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce8017030 con 0x7efcf8103bb0 2024-09-21T04:13:07.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.528+0000 7efce77fe640 1 -- 172.21.15.64:0/2197516477 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efce8004020 con 0x7efcf8103bb0 2024-09-21T04:13:07.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.528+0000 7efcfc828640 1 -- 172.21.15.64:0/2197516477 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcf81a06d0 con 0x7efcf8103bb0 2024-09-21T04:13:07.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.528+0000 7efce77fe640 1 -- 172.21.15.64:0/2197516477 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce80047a0 con 0x7efcf8103bb0 2024-09-21T04:13:07.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.528+0000 7efcfc828640 1 -- 172.21.15.64:0/2197516477 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcf81a6e30 con 0x7efcf8103bb0 2024-09-21T04:13:07.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.531+0000 7efd02d9c740 1 Processor -- start 2024-09-21T04:13:07.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.531+0000 7efce77fe640 1 -- 172.21.15.64:0/2197516477 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7efce8009a30 con 0x7efcf8103bb0 2024-09-21T04:13:07.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.531+0000 7efd02d9c740 1 -- start start 2024-09-21T04:13:07.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.531+0000 7efd02d9c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a47b350 0x55ae8a822430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:07.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.531+0000 7efd02d9c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8a452490 0x55ae8a822970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:07.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.531+0000 7efd02d9c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae8a45a670 0x55ae8a822eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:07.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.531+0000 7efd02d9c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ae8a798e60 con 0x55ae8a47b350 2024-09-21T04:13:07.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.531+0000 7efd02d9c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ae8a788290 con 0x55ae8a45a670 2024-09-21T04:13:07.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.531+0000 7efd02d9c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ae8a78c0a0 con 0x55ae8a452490 2024-09-21T04:13:07.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.531+0000 7efcf6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a47b350 0x55ae8a822430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:07.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.531+0000 7efcf6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a47b350 0x55ae8a822430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42188/0 (socket says 172.21.15.64:42188) 2024-09-21T04:13:07.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.531+0000 7efcf6d9d640 1 -- 172.21.15.64:0/2524226772 learned_addr learned my addr 172.21.15.64:0/2524226772 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:07.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.531+0000 7efcf5d9b640 1 --2- 172.21.15.64:0/2524226772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae8a45a670 0x55ae8a822eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:07.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.532+0000 7efcf659c640 1 --2- 172.21.15.64:0/2524226772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8a452490 0x55ae8a822970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:07.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.532+0000 7efcf6d9d640 1 -- 172.21.15.64:0/2524226772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8a452490 msgr2=0x55ae8a822970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:07.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.532+0000 7efcf6d9d640 1 --2- 172.21.15.64:0/2524226772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8a452490 0x55ae8a822970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.532+0000 7efcf6d9d640 1 -- 172.21.15.64:0/2524226772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae8a45a670 msgr2=0x55ae8a822eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:07.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.532+0000 7efcf6d9d640 1 --2- 172.21.15.64:0/2524226772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae8a45a670 0x55ae8a822eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.532+0000 7efcf6d9d640 1 -- 172.21.15.64:0/2524226772 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae8a4e6930 con 0x55ae8a47b350 2024-09-21T04:13:07.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.532+0000 7efce77fe640 1 --2- 172.21.15.64:0/2197516477 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7efce00b6700 0x7efce00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:07.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.532+0000 7efcf6d9d640 1 --2- 172.21.15.64:0/2524226772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a47b350 0x55ae8a822430 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7efcf0005cc0 tx=0x7efcf002f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52fc9a0a3fd94242 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:07.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.532+0000 7efcf659c640 1 --2- 172.21.15.64:0/2197516477 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7efce00b6700 0x7efce00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:07.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.532+0000 7efce77fe640 1 -- 172.21.15.64:0/2197516477 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7efce80e72d0 con 0x7efcf8103bb0 2024-09-21T04:13:07.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.532+0000 7efce4ff9640 1 -- 172.21.15.64:0/2524226772 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0031cf0 con 0x55ae8a47b350 2024-09-21T04:13:07.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.532+0000 7efce4ff9640 1 -- 172.21.15.64:0/2524226772 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcf0036040 con 0x55ae8a47b350 2024-09-21T04:13:07.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.532+0000 7efce4ff9640 1 -- 172.21.15.64:0/2524226772 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0004040 con 0x55ae8a47b350 2024-09-21T04:13:07.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.533+0000 7efd02d9c740 1 -- 172.21.15.64:0/2524226772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a47b350 msgr2=0x55ae8a822430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:07.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.533+0000 7efd02d9c740 1 --2- 172.21.15.64:0/2524226772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a47b350 0x55ae8a822430 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7efcf0005cc0 tx=0x7efcf002f420 comp rx=0 tx=0).stop 2024-09-21T04:13:07.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.533+0000 7efcf659c640 1 --2- 172.21.15.64:0/2197516477 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7efce00b6700 0x7efce00b8c10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7efce80e4a70 tx=0x7efce8094040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:07.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.533+0000 7efd02d9c740 1 -- 172.21.15.64:0/2524226772 shutdown_connections 2024-09-21T04:13:07.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.533+0000 7efd02d9c740 1 --2- 172.21.15.64:0/2524226772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae8a45a670 0x55ae8a822eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.533+0000 7efd02d9c740 1 --2- 172.21.15.64:0/2524226772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8a452490 0x55ae8a822970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.533+0000 7efd02d9c740 1 --2- 172.21.15.64:0/2524226772 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a47b350 0x55ae8a822430 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.533+0000 7efd02d9c740 1 -- 172.21.15.64:0/2524226772 >> 172.21.15.64:0/2524226772 conn(0x55ae8a46f740 msgr2=0x55ae8a820320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:07.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.534+0000 7efd02d9c740 1 -- 172.21.15.64:0/2524226772 shutdown_connections 2024-09-21T04:13:07.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.534+0000 7efd02d9c740 1 -- 172.21.15.64:0/2524226772 wait complete. 2024-09-21T04:13:07.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.534+0000 7efd02d9c740 1 -- start start 2024-09-21T04:13:07.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.534+0000 7efd02d9c740 1 Processor -- start 2024-09-21T04:13:07.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.534+0000 7efd02d9c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a452490 0x55ae8a82dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:07.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.534+0000 7efd02d9c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8a45a670 0x55ae8a82e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:07.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.534+0000 7efd02d9c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae8a47b350 0x55ae8a839ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:07.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.534+0000 7efd02d9c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ae8a5e9890 con 0x55ae8a452490 2024-09-21T04:13:07.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.534+0000 7efd02d9c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ae8a722d40 con 0x55ae8a47b350 2024-09-21T04:13:07.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.534+0000 7efd02d9c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ae8a830bb0 con 0x55ae8a45a670 2024-09-21T04:13:07.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.534+0000 7efd02d9c740 10 client.0 _ll_register_callbacks cb 0x55ae8a786c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:07.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.534+0000 7efcf6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a452490 0x55ae8a82dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:07.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.534+0000 7efcf6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a452490 0x55ae8a82dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42200/0 (socket says 172.21.15.64:42200) 2024-09-21T04:13:07.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.534+0000 7efcf6d9d640 1 -- 172.21.15.64:0/2707340213 learned_addr learned my addr 172.21.15.64:0/2707340213 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.535+0000 7efcf5d9b640 1 --2- 172.21.15.64:0/2707340213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8a45a670 0x55ae8a82e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.535+0000 7efcf6d9d640 1 --2- 172.21.15.64:0/2707340213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae8a47b350 0x55ae8a839ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.535+0000 7efcf6d9d640 1 -- 172.21.15.64:0/2707340213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8a45a670 msgr2=0x55ae8a82e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.535+0000 7efcf6d9d640 1 --2- 172.21.15.64:0/2707340213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8a45a670 0x55ae8a82e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.535+0000 7efcf6d9d640 1 -- 172.21.15.64:0/2707340213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae8a47b350 msgr2=0x55ae8a839ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.535+0000 7efcf6d9d640 1 --2- 172.21.15.64:0/2707340213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae8a47b350 0x55ae8a839ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.535+0000 7efcf6d9d640 1 -- 172.21.15.64:0/2707340213 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae8a83a440 con 0x55ae8a452490 2024-09-21T04:13:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.535+0000 7efcf6d9d640 1 --2- 172.21.15.64:0/2707340213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a452490 0x55ae8a82dfe0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7efcf002f500 tx=0x7efcf0003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422df678a66d050 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.535+0000 7efcddffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:13:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.535+0000 7efcddffb640 1 -- 172.21.15.64:0/2707340213 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0058060 con 0x55ae8a452490 2024-09-21T04:13:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.535+0000 7efcddffb640 1 -- 172.21.15.64:0/2707340213 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7efcf0031450 con 0x55ae8a452490 2024-09-21T04:13:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.535+0000 7efd02d9c740 1 -- 172.21.15.64:0/2707340213 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcf0036040 con 0x55ae8a452490 2024-09-21T04:13:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.535+0000 7efcddffb640 1 -- 172.21.15.64:0/2707340213 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7efcf0031650 con 0x55ae8a452490 2024-09-21T04:13:07.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.535+0000 7efd02d9c740 10 client.17355 fetch_fsmap learned FSMap version 40 2024-09-21T04:13:07.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.535+0000 7efd02d9c740 1 -- 172.21.15.64:0/2707340213 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55ae8a83cd70 con 0x55ae8a452490 2024-09-21T04:13:07.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.536+0000 7efcddffb640 1 -- 172.21.15.64:0/2707340213 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcf0036040 con 0x55ae8a452490 2024-09-21T04:13:07.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.536+0000 7efcddffb640 1 -- 172.21.15.64:0/2707340213 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7efcf002f660 con 0x55ae8a452490 2024-09-21T04:13:07.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.536+0000 7efd02d9c740 10 client.17355 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:13:07.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.536+0000 7efd02d9c740 10 client.17355 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:13:07.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.536+0000 7efd02d9c740 20 client.17355 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:07.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.536+0000 7efd02d9c740 1 --2- 172.21.15.64:0/2707340213 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55ae8a439cd0 0x55ae8a8400d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:07.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.536+0000 7efd02d9c740 4 client.17355 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:13:07.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.536+0000 7efd02d9c740 1 -- 172.21.15.64:0/2707340213 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae8a79adb0 con 0x55ae8a439cd0 2024-09-21T04:13:07.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.536+0000 7efcf5d9b640 1 --2- 172.21.15.64:0/2707340213 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55ae8a439cd0 0x55ae8a8400d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:07.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.537+0000 7efcddffb640 10 client.17355 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:07.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.537+0000 7efcf5d9b640 1 --2- 172.21.15.64:0/2707340213 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55ae8a439cd0 0x55ae8a8400d0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19458b16e4475539 server_cookie=a1a560a1da43462e in_seq=0 out_seq=0 2024-09-21T04:13:07.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.538+0000 7efcddffb640 1 -- 172.21.15.64:0/2707340213 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ae8a79adb0 con 0x55ae8a439cd0 2024-09-21T04:13:07.549 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.538+0000 7efcddffb640 10 client.17355 handle_command_reply: tid=0 2024-09-21T04:13:07.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.538+0000 7efd02d9c740 1 client.17355 shutdown 2024-09-21T04:13:07.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.538+0000 7efd02d9c740 1 -- 172.21.15.64:0/2707340213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a452490 msgr2=0x55ae8a82dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:07.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.538+0000 7efd02d9c740 1 --2- 172.21.15.64:0/2707340213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a452490 0x55ae8a82dfe0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7efcf002f500 tx=0x7efcf0003af0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.538+0000 7efd02d9c740 1 -- 172.21.15.64:0/2707340213 shutdown_connections 2024-09-21T04:13:07.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.538+0000 7efd02d9c740 1 --2- 172.21.15.64:0/2707340213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae8a47b350 0x55ae8a839ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.538+0000 7efd02d9c740 1 --2- 172.21.15.64:0/2707340213 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8a45a670 0x55ae8a82e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.538+0000 7efd02d9c740 1 --2- 172.21.15.64:0/2707340213 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55ae8a439cd0 0x55ae8a8400d0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.538+0000 7efd02d9c740 1 --2- 172.21.15.64:0/2707340213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a452490 0x55ae8a82dfe0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.538+0000 7efd02d9c740 1 -- 172.21.15.64:0/2707340213 >> 172.21.15.64:0/2707340213 conn(0x55ae8a46f740 msgr2=0x55ae8a825610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:07.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.538+0000 7efcddffb640 0 client.17355 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:07.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.538+0000 7efd02d9c740 1 -- 172.21.15.64:0/2707340213 shutdown_connections 2024-09-21T04:13:07.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.538+0000 7efd02d9c740 1 -- 172.21.15.64:0/2707340213 wait complete. 2024-09-21T04:13:07.550 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.539+0000 7efd02d9c740 20 client.17355 trim_cache size 0 max 16384 2024-09-21T04:13:07.559 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.556+0000 7efd02d9c740 1 Processor -- start 2024-09-21T04:13:07.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.556+0000 7efd02d9c740 1 -- start start 2024-09-21T04:13:07.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.556+0000 7efd02d9c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae8a452490 0x55ae8a839e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:07.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.556+0000 7efd02d9c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8a45a670 0x55ae8a828550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:07.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.556+0000 7efd02d9c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a47b350 0x55ae8a82afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:07.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.556+0000 7efd02d9c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efcf81b22c0 con 0x55ae8a47b350 2024-09-21T04:13:07.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.556+0000 7efd02d9c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efcf81afad0 con 0x55ae8a452490 2024-09-21T04:13:07.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.556+0000 7efd02d9c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efcf81ad940 con 0x55ae8a45a670 2024-09-21T04:13:07.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.557+0000 7efcf6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a47b350 0x55ae8a82afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:07.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.557+0000 7efcf6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a47b350 0x55ae8a82afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42202/0 (socket says 172.21.15.64:42202) 2024-09-21T04:13:07.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.557+0000 7efcf6d9d640 1 -- 172.21.15.64:0/1291400154 learned_addr learned my addr 172.21.15.64:0/1291400154 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:07.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.557+0000 7efcf6d9d640 1 --2- 172.21.15.64:0/1291400154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae8a452490 0x55ae8a839e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:07.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.557+0000 7efcf5d9b640 1 --2- 172.21.15.64:0/1291400154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8a45a670 0x55ae8a828550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:07.560 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.557+0000 7efcf6d9d640 1 -- 172.21.15.64:0/1291400154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8a45a670 msgr2=0x55ae8a828550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:07.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.557+0000 7efcf6d9d640 1 --2- 172.21.15.64:0/1291400154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8a45a670 0x55ae8a828550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.557+0000 7efcf6d9d640 1 -- 172.21.15.64:0/1291400154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae8a452490 msgr2=0x55ae8a839e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:07.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.557+0000 7efcf6d9d640 1 --2- 172.21.15.64:0/1291400154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae8a452490 0x55ae8a839e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.557+0000 7efcf6d9d640 1 -- 172.21.15.64:0/1291400154 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae8a825290 con 0x55ae8a47b350 2024-09-21T04:13:07.561 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.557+0000 7efcf6d9d640 1 --2- 172.21.15.64:0/1291400154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a47b350 0x55ae8a82afd0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7efcf002fa50 tx=0x7efcf0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad4f2cc99d03528 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:07.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.557+0000 7efcdffff640 1 -- 172.21.15.64:0/1291400154 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf00317f0 con 0x55ae8a47b350 2024-09-21T04:13:07.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.557+0000 7efcdffff640 1 -- 172.21.15.64:0/1291400154 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcf0036070 con 0x55ae8a47b350 2024-09-21T04:13:07.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.557+0000 7efcdffff640 1 -- 172.21.15.64:0/1291400154 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0030a70 con 0x55ae8a47b350 2024-09-21T04:13:07.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.558+0000 7efd02d9c740 1 -- 172.21.15.64:0/1291400154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a47b350 msgr2=0x55ae8a82afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:07.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.558+0000 7efd02d9c740 1 --2- 172.21.15.64:0/1291400154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a47b350 0x55ae8a82afd0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7efcf002fa50 tx=0x7efcf0009b20 comp rx=0 tx=0).stop 2024-09-21T04:13:07.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.558+0000 7efd02d9c740 1 -- 172.21.15.64:0/1291400154 shutdown_connections 2024-09-21T04:13:07.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.558+0000 7efd02d9c740 1 --2- 172.21.15.64:0/1291400154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a47b350 0x55ae8a82afd0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.558+0000 7efd02d9c740 1 --2- 172.21.15.64:0/1291400154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8a45a670 0x55ae8a828550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.558+0000 7efd02d9c740 1 --2- 172.21.15.64:0/1291400154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae8a452490 0x55ae8a839e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.558+0000 7efd02d9c740 1 -- 172.21.15.64:0/1291400154 >> 172.21.15.64:0/1291400154 conn(0x55ae8a46f740 msgr2=0x55ae8a82c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:07.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.558+0000 7efd02d9c740 1 -- 172.21.15.64:0/1291400154 shutdown_connections 2024-09-21T04:13:07.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.558+0000 7efd02d9c740 1 -- 172.21.15.64:0/1291400154 wait complete. 2024-09-21T04:13:07.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.559+0000 7efd02d9c740 1 -- start start 2024-09-21T04:13:07.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.559+0000 7efd02d9c740 1 Processor -- start 2024-09-21T04:13:07.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.559+0000 7efd02d9c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae8a452490 0x55ae8a843ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:07.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.559+0000 7efd02d9c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8a45a670 0x55ae8a840fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:07.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.559+0000 7efd02d9c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a47b350 0x55ae8a8416e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:07.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.559+0000 7efd02d9c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efcf81adf90 con 0x55ae8a47b350 2024-09-21T04:13:07.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.559+0000 7efd02d9c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efcf81af400 con 0x55ae8a452490 2024-09-21T04:13:07.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.559+0000 7efd02d9c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efcf81ae660 con 0x55ae8a45a670 2024-09-21T04:13:07.565 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.559+0000 7efd02d9c740 10 client.0 _ll_register_callbacks cb 0x7efcec009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:07.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.559+0000 7efcf6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a47b350 0x55ae8a8416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:07.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.559+0000 7efcf6d9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a47b350 0x55ae8a8416e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42208/0 (socket says 172.21.15.64:42208) 2024-09-21T04:13:07.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.559+0000 7efcf6d9d640 1 -- 172.21.15.64:0/4010234632 learned_addr learned my addr 172.21.15.64:0/4010234632 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:07.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.559+0000 7efcf6d9d640 1 --2- 172.21.15.64:0/4010234632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae8a452490 0x55ae8a843ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:07.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.559+0000 7efcf5d9b640 1 --2- 172.21.15.64:0/4010234632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8a45a670 0x55ae8a840fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:07.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.560+0000 7efcf6d9d640 1 -- 172.21.15.64:0/4010234632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8a45a670 msgr2=0x55ae8a840fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:07.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.560+0000 7efcf6d9d640 1 --2- 172.21.15.64:0/4010234632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8a45a670 0x55ae8a840fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.560+0000 7efcf6d9d640 1 -- 172.21.15.64:0/4010234632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae8a452490 msgr2=0x55ae8a843ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:07.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.560+0000 7efcf6d9d640 1 --2- 172.21.15.64:0/4010234632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae8a452490 0x55ae8a843ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.560+0000 7efcf6d9d640 1 -- 172.21.15.64:0/4010234632 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae8a8a9850 con 0x55ae8a47b350 2024-09-21T04:13:07.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.560+0000 7efcf6d9d640 1 --2- 172.21.15.64:0/4010234632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a47b350 0x55ae8a8416e0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7efcf0005e30 tx=0x7efcf00315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f4f19eb0749a53 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:07.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.560+0000 7efcddffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:13:07.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.560+0000 7efcddffb640 1 -- 172.21.15.64:0/4010234632 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0030a90 con 0x55ae8a47b350 2024-09-21T04:13:07.566 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.560+0000 7efd02d9c740 1 -- 172.21.15.64:0/4010234632 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcf0036070 con 0x55ae8a47b350 2024-09-21T04:13:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.560+0000 7efcddffb640 1 -- 172.21.15.64:0/4010234632 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8615+0+0 (secure 0 0 0) 0x7efcf0030c30 con 0x55ae8a47b350 2024-09-21T04:13:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.560+0000 7efcddffb640 1 -- 172.21.15.64:0/4010234632 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7efcf0030e30 con 0x55ae8a47b350 2024-09-21T04:13:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.560+0000 7efd02d9c740 10 client.17361 fetch_fsmap learned FSMap version 40 2024-09-21T04:13:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.560+0000 7efd02d9c740 1 -- 172.21.15.64:0/4010234632 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55ae8a8aa9b0 con 0x55ae8a47b350 2024-09-21T04:13:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.560+0000 7efcddffb640 1 -- 172.21.15.64:0/4010234632 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcf0036070 con 0x55ae8a47b350 2024-09-21T04:13:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.561+0000 7efcddffb640 1 -- 172.21.15.64:0/4010234632 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7efcf0045430 con 0x55ae8a47b350 2024-09-21T04:13:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.561+0000 7efd02d9c740 10 client.17361 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:13:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.561+0000 7efd02d9c740 10 client.17361 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:13:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.561+0000 7efd02d9c740 20 client.17361 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.561+0000 7efd02d9c740 1 --2- 172.21.15.64:0/4010234632 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55ae8a40f080 0x55ae8a8acf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.561+0000 7efd02d9c740 4 client.17361 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T04:13:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.561+0000 7efd02d9c740 1 -- 172.21.15.64:0/4010234632 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ae8a796860 con 0x55ae8a40f080 2024-09-21T04:13:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.561+0000 7efcf5d9b640 1 --2- 172.21.15.64:0/4010234632 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55ae8a40f080 0x55ae8a8acf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.561+0000 7efcf5d9b640 1 --2- 172.21.15.64:0/4010234632 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55ae8a40f080 0x55ae8a8acf90 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=764b9cc6e316e81c server_cookie=82909b7abd5ef261 in_seq=0 out_seq=0 2024-09-21T04:13:07.567 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.561+0000 7efcddffb640 10 client.17361 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:07.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.564+0000 7efcddffb640 1 -- 172.21.15.64:0/4010234632 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ae8a796860 con 0x55ae8a40f080 2024-09-21T04:13:07.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.564+0000 7efcddffb640 10 client.17361 handle_command_reply: tid=0 2024-09-21T04:13:07.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.564+0000 7efd02d9c740 1 client.17361 shutdown 2024-09-21T04:13:07.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.564+0000 7efd02d9c740 1 -- 172.21.15.64:0/4010234632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a47b350 msgr2=0x55ae8a8416e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:07.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.564+0000 7efd02d9c740 1 --2- 172.21.15.64:0/4010234632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a47b350 0x55ae8a8416e0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7efcf0005e30 tx=0x7efcf00315b0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.564+0000 7efd02d9c740 1 -- 172.21.15.64:0/4010234632 shutdown_connections 2024-09-21T04:13:07.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.564+0000 7efd02d9c740 1 --2- 172.21.15.64:0/4010234632 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55ae8a40f080 0x55ae8a8acf90 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.564+0000 7efd02d9c740 1 --2- 172.21.15.64:0/4010234632 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ae8a47b350 0x55ae8a8416e0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.564+0000 7efd02d9c740 1 --2- 172.21.15.64:0/4010234632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ae8a45a670 0x55ae8a840fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.564+0000 7efd02d9c740 1 --2- 172.21.15.64:0/4010234632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae8a452490 0x55ae8a843ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.564+0000 7efcddffb640 0 client.17361 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:07.569 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.564+0000 7efd02d9c740 1 -- 172.21.15.64:0/4010234632 >> 172.21.15.64:0/4010234632 conn(0x55ae8a46f740 msgr2=0x55ae8a82f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:07.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.564+0000 7efd02d9c740 1 -- 172.21.15.64:0/4010234632 shutdown_connections 2024-09-21T04:13:07.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.564+0000 7efd02d9c740 1 -- 172.21.15.64:0/4010234632 wait complete. 2024-09-21T04:13:07.570 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.564+0000 7efd02d9c740 20 client.17361 trim_cache size 0 max 16384 2024-09-21T04:13:07.570 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:13:07.570 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-21T04:13:07.570 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "424f2ae3-b0e9-4d20-a6cc-7cea3b92b066", 2024-09-21T04:13:07.570 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-21T04:13:07.570 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:13:07.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.565+0000 7efcfc828640 1 -- 172.21.15.64:0/2197516477 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7efce00b6700 msgr2=0x7efce00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:07.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.565+0000 7efcfc828640 1 --2- 172.21.15.64:0/2197516477 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7efce00b6700 0x7efce00b8c10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7efce80e4a70 tx=0x7efce8094040 comp rx=0 tx=0).stop 2024-09-21T04:13:07.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.565+0000 7efcfc828640 1 -- 172.21.15.64:0/2197516477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcf8103bb0 msgr2=0x7efcf819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:07.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.565+0000 7efcfc828640 1 --2- 172.21.15.64:0/2197516477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcf8103bb0 0x7efcf819a470 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7efce800d950 tx=0x7efce800de10 comp rx=0 tx=0).stop 2024-09-21T04:13:07.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.565+0000 7efcfc828640 1 -- 172.21.15.64:0/2197516477 shutdown_connections 2024-09-21T04:13:07.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.565+0000 7efcfc828640 1 --2- 172.21.15.64:0/2197516477 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7efce00b6700 0x7efce00b8c10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.565+0000 7efcfc828640 1 --2- 172.21.15.64:0/2197516477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcf8109e40 0x7efcf819fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.565+0000 7efcfc828640 1 --2- 172.21.15.64:0/2197516477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efcf81094f0 0x7efcf819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.565+0000 7efcfc828640 1 --2- 172.21.15.64:0/2197516477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcf8103bb0 0x7efcf819a470 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:07.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.565+0000 7efcfc828640 1 -- 172.21.15.64:0/2197516477 >> 172.21.15.64:0/2197516477 conn(0x7efcf80ff600 msgr2=0x7efcf8101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:07.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.565+0000 7efcfc828640 1 -- 172.21.15.64:0/2197516477 shutdown_connections 2024-09-21T04:13:07.571 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:07.565+0000 7efcfc828640 1 -- 172.21.15.64:0/2197516477 wait complete. 2024-09-21T04:13:08.302 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T04:13:08.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:08 smithi119 ceph-mon[34811]: pgmap v600: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-21T04:13:08.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:08 smithi119 ceph-mon[34811]: scrub queued for path: / 2024-09-21T04:13:08.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:08 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:13:08.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:08 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:13:08.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:08 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:13:08.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:08 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:13:08.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:08 smithi064 ceph-mon[31180]: pgmap v600: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-21T04:13:08.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:08 smithi064 ceph-mon[31180]: scrub queued for path: / 2024-09-21T04:13:08.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:08 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:13:08.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:08 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:13:08.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:08 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:13:08.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:08 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:13:08.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:08 smithi131 ceph-mon[35193]: pgmap v600: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-09-21T04:13:08.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:08 smithi131 ceph-mon[35193]: scrub queued for path: / 2024-09-21T04:13:08.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:08 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:13:08.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:08 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:13:08.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:08 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:13:08.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:08 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:13:08.777 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:13:09.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.362+0000 7f99ba6aa640 1 -- 172.21.15.64:0/515162023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99b41094d0 msgr2=0x7f99b41098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:09.365 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.363+0000 7f99ba6aa640 1 --2- 172.21.15.64:0/515162023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99b41094d0 0x7f99b41098e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009900 tx=0x7f99a402a440 comp rx=0 tx=0).stop 2024-09-21T04:13:09.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.364+0000 7f99ba6aa640 1 -- 172.21.15.64:0/515162023 shutdown_connections 2024-09-21T04:13:09.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.364+0000 7f99ba6aa640 1 --2- 172.21.15.64:0/515162023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b4109e20 0x7f99b4110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.366 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.364+0000 7f99ba6aa640 1 --2- 172.21.15.64:0/515162023 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99b41094d0 0x7f99b41098e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.364+0000 7f99ba6aa640 1 --2- 172.21.15.64:0/515162023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b4103b90 0x7f99b4103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.364+0000 7f99ba6aa640 1 -- 172.21.15.64:0/515162023 >> 172.21.15.64:0/515162023 conn(0x7f99b40ff5e0 msgr2=0x7f99b4101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:09.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.365+0000 7f99ba6aa640 1 -- 172.21.15.64:0/515162023 shutdown_connections 2024-09-21T04:13:09.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.366+0000 7f99ba6aa640 1 -- 172.21.15.64:0/515162023 wait complete. 2024-09-21T04:13:09.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.367+0000 7f99ba6aa640 1 Processor -- start 2024-09-21T04:13:09.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.367+0000 7f99ba6aa640 1 -- start start 2024-09-21T04:13:09.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.368+0000 7f99ba6aa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99b4103b90 0x7f99b4198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:09.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.368+0000 7f99ba6aa640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b41094d0 0x7f99b4198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:09.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.369+0000 7f99ba6aa640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b4109e20 0x7f99b419d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:09.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.369+0000 7f99b3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99b4103b90 0x7f99b4198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:09.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.369+0000 7f99b3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99b4103b90 0x7f99b4198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:55810/0 (socket says 172.21.15.64:55810) 2024-09-21T04:13:09.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.369+0000 7f99b3fff640 1 -- 172.21.15.64:0/1703138387 learned_addr learned my addr 172.21.15.64:0/1703138387 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:09.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.369+0000 7f99ba6aa640 1 -- 172.21.15.64:0/1703138387 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f99b41119e0 con 0x7f99b4109e20 2024-09-21T04:13:09.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.369+0000 7f99ba6aa640 1 -- 172.21.15.64:0/1703138387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f99b4111860 con 0x7f99b4103b90 2024-09-21T04:13:09.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.369+0000 7f99ba6aa640 1 -- 172.21.15.64:0/1703138387 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f99b4111b60 con 0x7f99b41094d0 2024-09-21T04:13:09.372 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.369+0000 7f99b8c47640 1 --2- 172.21.15.64:0/1703138387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b4109e20 0x7f99b419d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:09.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.370+0000 7f99b8c47640 1 -- 172.21.15.64:0/1703138387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b41094d0 msgr2=0x7f99b4198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:09.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.370+0000 7f99b8c47640 1 --2- 172.21.15.64:0/1703138387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b41094d0 0x7f99b4198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.370+0000 7f99b8c47640 1 -- 172.21.15.64:0/1703138387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99b4103b90 msgr2=0x7f99b4198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:09.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.370+0000 7f99b8c47640 1 --2- 172.21.15.64:0/1703138387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99b4103b90 0x7f99b4198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.370+0000 7f99b8c47640 1 -- 172.21.15.64:0/1703138387 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b419e0f0 con 0x7f99b4109e20 2024-09-21T04:13:09.373 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.370+0000 7f99b8c47640 1 --2- 172.21.15.64:0/1703138387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b4109e20 0x7f99b419d9c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00be20 tx=0x7f99ac00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:09.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.372+0000 7f99b3fff640 1 --2- 172.21.15.64:0/1703138387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99b4103b90 0x7f99b4198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:13:09.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.373+0000 7f99b17fa640 1 -- 172.21.15.64:0/1703138387 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ac019070 con 0x7f99b4109e20 2024-09-21T04:13:09.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.373+0000 7f99b17fa640 1 -- 172.21.15.64:0/1703138387 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99ac0148f0 con 0x7f99b4109e20 2024-09-21T04:13:09.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.373+0000 7f99b17fa640 1 -- 172.21.15.64:0/1703138387 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ac012810 con 0x7f99b4109e20 2024-09-21T04:13:09.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.373+0000 7f99ba6aa640 1 -- 172.21.15.64:0/1703138387 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b419e3e0 con 0x7f99b4109e20 2024-09-21T04:13:09.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.373+0000 7f99ba6aa640 1 -- 172.21.15.64:0/1703138387 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b41a4b40 con 0x7f99b4109e20 2024-09-21T04:13:09.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.377+0000 7f99c0c1e740 1 Processor -- start 2024-09-21T04:13:09.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.381+0000 7f99b17fa640 1 -- 172.21.15.64:0/1703138387 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f99ac01c070 con 0x7f99b4109e20 2024-09-21T04:13:09.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.381+0000 7f99c0c1e740 1 -- start start 2024-09-21T04:13:09.384 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.381+0000 7f99c0c1e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3970c070 0x55df39ab6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:09.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.381+0000 7f99c0c1e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df396e31b0 0x55df39ab3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:09.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.381+0000 7f99c0c1e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df396eb390 0x55df39ab3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:09.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.381+0000 7f99c0c1e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55df39a29e50 con 0x55df396e31b0 2024-09-21T04:13:09.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.381+0000 7f99c0c1e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55df39a19280 con 0x55df3970c070 2024-09-21T04:13:09.385 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.381+0000 7f99c0c1e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55df39a1d090 con 0x55df396eb390 2024-09-21T04:13:09.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.382+0000 7f99b37fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3970c070 0x55df39ab6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:09.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.382+0000 7f99b37fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3970c070 0x55df39ab6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:55822/0 (socket says 172.21.15.64:55822) 2024-09-21T04:13:09.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.382+0000 7f99b37fe640 1 -- 172.21.15.64:0/1394741306 learned_addr learned my addr 172.21.15.64:0/1394741306 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:09.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.382+0000 7f99b37fe640 1 --2- 172.21.15.64:0/1394741306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df396eb390 0x55df39ab3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:09.386 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.383+0000 7f99b17fa640 1 --2- 172.21.15.64:0/1703138387 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f99a00bac10 0x7f99a00bd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:09.387 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:09 smithi064 ceph-mon[31180]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T04:13:09.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.383+0000 7f99b8c47640 1 --2- 172.21.15.64:0/1703138387 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f99a00bac10 0x7f99a00bd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:09.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.384+0000 7f99b17fa640 1 -- 172.21.15.64:0/1703138387 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8659+0+0 (secure 0 0 0) 0x7f99ac0e6f30 con 0x7f99b4109e20 2024-09-21T04:13:09.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.384+0000 7f99b37fe640 1 -- 172.21.15.64:0/1394741306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df396eb390 msgr2=0x55df39ab3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:09.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.384+0000 7f99b37fe640 1 --2- 172.21.15.64:0/1394741306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df396eb390 0x55df39ab3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.384+0000 7f99b37fe640 1 -- 172.21.15.64:0/1394741306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df396e31b0 msgr2=0x55df39ab3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:09.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.384+0000 7f99b37fe640 1 --2- 172.21.15.64:0/1394741306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df396e31b0 0x55df39ab3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.384+0000 7f99b37fe640 1 -- 172.21.15.64:0/1394741306 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df39777650 con 0x55df3970c070 2024-09-21T04:13:09.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.384+0000 7f99b37fe640 1 --2- 172.21.15.64:0/1394741306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3970c070 0x55df39ab6340 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f99a40092a0 tx=0x7f99a4009230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89a985e48a86fa4d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:09.388 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.385+0000 7f99b8c47640 1 --2- 172.21.15.64:0/1703138387 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f99a00bac10 0x7f99a00bd120 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f99ac0ac990 tx=0x7f99ac004b50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:09.389 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.386+0000 7f99ba6aa640 1 -- 172.21.15.64:0/1394741306 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a4041030 con 0x55df3970c070 2024-09-21T04:13:09.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.386+0000 7f99ba6aa640 1 -- 172.21.15.64:0/1394741306 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99a4031850 con 0x55df3970c070 2024-09-21T04:13:09.390 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.386+0000 7f99ba6aa640 1 -- 172.21.15.64:0/1394741306 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a4035070 con 0x55df3970c070 2024-09-21T04:13:09.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.392+0000 7f99c0c1e740 1 -- 172.21.15.64:0/1394741306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3970c070 msgr2=0x55df39ab6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:09.394 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.392+0000 7f99c0c1e740 1 --2- 172.21.15.64:0/1394741306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3970c070 0x55df39ab6340 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f99a40092a0 tx=0x7f99a4009230 comp rx=0 tx=0).stop 2024-09-21T04:13:09.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.394+0000 7f99c0c1e740 1 -- 172.21.15.64:0/1394741306 shutdown_connections 2024-09-21T04:13:09.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.394+0000 7f99c0c1e740 1 --2- 172.21.15.64:0/1394741306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df396eb390 0x55df39ab3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.394+0000 7f99c0c1e740 1 --2- 172.21.15.64:0/1394741306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df396e31b0 0x55df39ab3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.394+0000 7f99c0c1e740 1 --2- 172.21.15.64:0/1394741306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3970c070 0x55df39ab6340 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.397 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.394+0000 7f99c0c1e740 1 -- 172.21.15.64:0/1394741306 >> 172.21.15.64:0/1394741306 conn(0x55df39700460 msgr2=0x55df39ab1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:09.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.395+0000 7f99c0c1e740 1 -- 172.21.15.64:0/1394741306 shutdown_connections 2024-09-21T04:13:09.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.395+0000 7f99c0c1e740 1 -- 172.21.15.64:0/1394741306 wait complete. 2024-09-21T04:13:09.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.395+0000 7f99c0c1e740 1 -- start start 2024-09-21T04:13:09.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.396+0000 7f99c0c1e740 1 Processor -- start 2024-09-21T04:13:09.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.396+0000 7f99c0c1e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df396e31b0 0x55df39abf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:09.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.396+0000 7f99c0c1e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df396eb390 0x55df39abf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:09.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.396+0000 7f99c0c1e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df3970c070 0x55df39ad38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:09.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.396+0000 7f99c0c1e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55df3987a700 con 0x55df396e31b0 2024-09-21T04:13:09.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.396+0000 7f99c0c1e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55df399b4110 con 0x55df396eb390 2024-09-21T04:13:09.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.396+0000 7f99c0c1e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55df39ac1da0 con 0x55df3970c070 2024-09-21T04:13:09.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.396+0000 7f99c0c1e740 10 client.0 _ll_register_callbacks cb 0x55df399cd990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:09.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.397+0000 7f99b3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df396eb390 0x55df39abf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:09.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.397+0000 7f99b3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df396eb390 0x55df39abf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:55836/0 (socket says 172.21.15.64:55836) 2024-09-21T04:13:09.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.397+0000 7f99b3fff640 1 -- 172.21.15.64:0/3360321432 learned_addr learned my addr 172.21.15.64:0/3360321432 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:09.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.399+0000 7f99b3fff640 1 -- 172.21.15.64:0/3360321432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df3970c070 msgr2=0x55df39ad38e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T04:13:09.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.399+0000 7f99b3fff640 1 --2- 172.21.15.64:0/3360321432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df3970c070 0x55df39ad38e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.399+0000 7f99b3fff640 1 -- 172.21.15.64:0/3360321432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df396e31b0 msgr2=0x55df39abf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:09.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.399+0000 7f99b3fff640 1 --2- 172.21.15.64:0/3360321432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df396e31b0 0x55df39abf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.399+0000 7f99b3fff640 1 -- 172.21.15.64:0/3360321432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df399e1c40 con 0x55df396eb390 2024-09-21T04:13:09.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.399+0000 7f99b3fff640 1 --2- 172.21.15.64:0/3360321432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df396eb390 0x55df39abf620 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f999c00c250 tx=0x7f999c00c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=427c3b104a238d6b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:09.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.399+0000 7f99937fe640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-09-21T04:13:09.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.400+0000 7f99937fe640 1 -- 172.21.15.64:0/3360321432 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c015070 con 0x55df396eb390 2024-09-21T04:13:09.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.400+0000 7f99937fe640 1 -- 172.21.15.64:0/3360321432 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8659+0+0 (secure 0 0 0) 0x7f999c011800 con 0x55df396eb390 2024-09-21T04:13:09.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.400+0000 7f99c0c1e740 1 -- 172.21.15.64:0/3360321432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99a4031850 con 0x55df396eb390 2024-09-21T04:13:09.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.400+0000 7f99937fe640 1 -- 172.21.15.64:0/3360321432 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f999c021050 con 0x55df396eb390 2024-09-21T04:13:09.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.400+0000 7f99c0c1e740 10 client.26990 fetch_fsmap learned FSMap version 40 2024-09-21T04:13:09.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.400+0000 7f99c0c1e740 1 -- 172.21.15.64:0/3360321432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55df39ac4680 con 0x55df396eb390 2024-09-21T04:13:09.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.404+0000 7f99937fe640 1 -- 172.21.15.64:0/3360321432 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99a4031850 con 0x55df396eb390 2024-09-21T04:13:09.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.404+0000 7f99937fe640 1 -- 172.21.15.64:0/3360321432 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f999c019bf0 con 0x55df396eb390 2024-09-21T04:13:09.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.405+0000 7f99c0c1e740 10 client.26990 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:13:09.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.405+0000 7f99c0c1e740 10 client.26990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:13:09.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.405+0000 7f99c0c1e740 20 client.26990 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:09.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.405+0000 7f99c0c1e740 1 --2- 172.21.15.64:0/3360321432 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55df396ca9f0 0x55df39ac7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:09.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.405+0000 7f99c0c1e740 4 client.26990 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:13:09.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.405+0000 7f99c0c1e740 1 -- 172.21.15.64:0/3360321432 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df39a2bda0 con 0x55df396ca9f0 2024-09-21T04:13:09.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.406+0000 7f99b37fe640 1 --2- 172.21.15.64:0/3360321432 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55df396ca9f0 0x55df39ac7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:09.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.407+0000 7f99b37fe640 1 --2- 172.21.15.64:0/3360321432 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55df396ca9f0 0x55df39ac7c10 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37871057aa687db2 server_cookie=9ec645765ade2629 in_seq=0 out_seq=0 2024-09-21T04:13:09.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.407+0000 7f99937fe640 10 client.26990 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:09.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.408+0000 7f99937fe640 1 -- 172.21.15.64:0/3360321432 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55df39a2bda0 con 0x55df396ca9f0 2024-09-21T04:13:09.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.408+0000 7f99937fe640 10 client.26990 handle_command_reply: tid=0 2024-09-21T04:13:09.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.409+0000 7f99c0c1e740 1 client.26990 shutdown 2024-09-21T04:13:09.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.409+0000 7f99c0c1e740 1 -- 172.21.15.64:0/3360321432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df396eb390 msgr2=0x55df39abf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:09.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.409+0000 7f99c0c1e740 1 --2- 172.21.15.64:0/3360321432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df396eb390 0x55df39abf620 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f999c00c250 tx=0x7f999c00c710 comp rx=0 tx=0).stop 2024-09-21T04:13:09.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.409+0000 7f99c0c1e740 1 -- 172.21.15.64:0/3360321432 shutdown_connections 2024-09-21T04:13:09.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.409+0000 7f99c0c1e740 1 --2- 172.21.15.64:0/3360321432 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df3970c070 0x55df39ad38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.409+0000 7f99c0c1e740 1 --2- 172.21.15.64:0/3360321432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df396eb390 0x55df39abf620 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f999c00c250 tx=0x7f999c00c710 comp rx=0 tx=0).stop 2024-09-21T04:13:09.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.409+0000 7f99c0c1e740 1 --2- 172.21.15.64:0/3360321432 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55df396ca9f0 0x55df39ac7c10 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.409+0000 7f99c0c1e740 1 --2- 172.21.15.64:0/3360321432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df396e31b0 0x55df39abf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.409+0000 7f99c0c1e740 1 -- 172.21.15.64:0/3360321432 >> 172.21.15.64:0/3360321432 conn(0x55df39700460 msgr2=0x55df39ab6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:09.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.409+0000 7f99937fe640 0 client.26990 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:09.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.410+0000 7f99c0c1e740 1 -- 172.21.15.64:0/3360321432 shutdown_connections 2024-09-21T04:13:09.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.410+0000 7f99c0c1e740 1 -- 172.21.15.64:0/3360321432 wait complete. 2024-09-21T04:13:09.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.410+0000 7f99c0c1e740 20 client.26990 trim_cache size 0 max 16384 2024-09-21T04:13:09.450 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.447+0000 7f99c0c1e740 1 Processor -- start 2024-09-21T04:13:09.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.448+0000 7f99c0c1e740 1 -- start start 2024-09-21T04:13:09.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.448+0000 7f99c0c1e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df396e31b0 0x55df39abe070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:09.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.448+0000 7f99c0c1e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3970c070 0x55df39abc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:09.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.448+0000 7f99c0c1e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df3969fda0 0x55df39abc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:09.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.448+0000 7f99c0c1e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f99b41b0200 con 0x55df396e31b0 2024-09-21T04:13:09.452 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.448+0000 7f99c0c1e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f99b41ada10 con 0x55df3970c070 2024-09-21T04:13:09.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.448+0000 7f99c0c1e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f99b41ab880 con 0x55df3969fda0 2024-09-21T04:13:09.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.449+0000 7f99b37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df396e31b0 0x55df39abe070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:09.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.449+0000 7f99b37fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df396e31b0 0x55df39abe070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:42246/0 (socket says 172.21.15.64:42246) 2024-09-21T04:13:09.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.449+0000 7f99b37fe640 1 -- 172.21.15.64:0/2610374364 learned_addr learned my addr 172.21.15.64:0/2610374364 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:09.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.449+0000 7f99b37fe640 1 --2- 172.21.15.64:0/2610374364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df3969fda0 0x55df39abc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:09.453 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.449+0000 7f99b3fff640 1 --2- 172.21.15.64:0/2610374364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3970c070 0x55df39abc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:09.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.451+0000 7f99b37fe640 1 -- 172.21.15.64:0/2610374364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3970c070 msgr2=0x55df39abc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:09.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.451+0000 7f99b37fe640 1 --2- 172.21.15.64:0/2610374364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3970c070 0x55df39abc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.451+0000 7f99b37fe640 1 -- 172.21.15.64:0/2610374364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df396e31b0 msgr2=0x55df39abe070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:09.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.451+0000 7f99b37fe640 1 --2- 172.21.15.64:0/2610374364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df396e31b0 0x55df39abe070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.451+0000 7f99b37fe640 1 -- 172.21.15.64:0/2610374364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df39ab7110 con 0x55df3969fda0 2024-09-21T04:13:09.454 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.451+0000 7f99b37fe640 1 --2- 172.21.15.64:0/2610374364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df3969fda0 0x55df39abc810 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f99a40058c0 tx=0x7f99a40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=248cdc2fa5649d05 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:09.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.451+0000 7f99a9ffb640 1 -- 172.21.15.64:0/2610374364 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a4035040 con 0x55df3969fda0 2024-09-21T04:13:09.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.452+0000 7f99a9ffb640 1 -- 172.21.15.64:0/2610374364 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99a4031c80 con 0x55df3969fda0 2024-09-21T04:13:09.455 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.452+0000 7f99a9ffb640 1 -- 172.21.15.64:0/2610374364 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a4004de0 con 0x55df3969fda0 2024-09-21T04:13:09.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.455+0000 7f99c0c1e740 1 -- 172.21.15.64:0/2610374364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df3969fda0 msgr2=0x55df39abc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:09.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.455+0000 7f99c0c1e740 1 --2- 172.21.15.64:0/2610374364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df3969fda0 0x55df39abc810 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f99a40058c0 tx=0x7f99a40046c0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.455+0000 7f99c0c1e740 1 -- 172.21.15.64:0/2610374364 shutdown_connections 2024-09-21T04:13:09.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.455+0000 7f99c0c1e740 1 --2- 172.21.15.64:0/2610374364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df3969fda0 0x55df39abc810 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.455+0000 7f99c0c1e740 1 --2- 172.21.15.64:0/2610374364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3970c070 0x55df39abc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.455+0000 7f99c0c1e740 1 --2- 172.21.15.64:0/2610374364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df396e31b0 0x55df39abe070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.458 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.455+0000 7f99c0c1e740 1 -- 172.21.15.64:0/2610374364 >> 172.21.15.64:0/2610374364 conn(0x55df39700460 msgr2=0x55df39abd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:09.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.456+0000 7f99c0c1e740 1 -- 172.21.15.64:0/2610374364 shutdown_connections 2024-09-21T04:13:09.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.456+0000 7f99c0c1e740 1 -- 172.21.15.64:0/2610374364 wait complete. 2024-09-21T04:13:09.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.457+0000 7f99c0c1e740 1 -- start start 2024-09-21T04:13:09.459 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.457+0000 7f99c0c1e740 1 Processor -- start 2024-09-21T04:13:09.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.458+0000 7f99c0c1e740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df3969fda0 0x55df39b467e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:09.460 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.458+0000 7f99c0c1e740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df396e31b0 0x55df39b46d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:09.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.458+0000 7f99c0c1e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3970c070 0x55df39acd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:09.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.458+0000 7f99c0c1e740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f99b41abed0 con 0x55df3969fda0 2024-09-21T04:13:09.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.458+0000 7f99b3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df396e31b0 0x55df39b46d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:09.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.458+0000 7f99c0c1e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f99b41ad340 con 0x55df3970c070 2024-09-21T04:13:09.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.459+0000 7f99c0c1e740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f99b41ac5a0 con 0x55df396e31b0 2024-09-21T04:13:09.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.459+0000 7f99b37fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3970c070 0x55df39acd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:09.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.459+0000 7f99b3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df396e31b0 0x55df39b46d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.64:39626/0 (socket says 172.21.15.64:39626) 2024-09-21T04:13:09.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.459+0000 7f99b3fff640 1 -- 172.21.15.64:0/1416026293 learned_addr learned my addr 172.21.15.64:0/1416026293 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:09.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.459+0000 7f99c0c1e740 10 client.0 _ll_register_callbacks cb 0x7f999c00a110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:09.462 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.459+0000 7f99b37fe640 1 --2- 172.21.15.64:0/1416026293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df3969fda0 0x55df39b467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:09.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.460+0000 7f99b3fff640 1 -- 172.21.15.64:0/1416026293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3970c070 msgr2=0x55df39acd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:09.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.460+0000 7f99b3fff640 1 --2- 172.21.15.64:0/1416026293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3970c070 0x55df39acd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.460+0000 7f99b3fff640 1 -- 172.21.15.64:0/1416026293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df3969fda0 msgr2=0x55df39b467e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:09.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.460+0000 7f99b3fff640 1 --2- 172.21.15.64:0/1416026293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df3969fda0 0x55df39b467e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.460+0000 7f99b3fff640 1 -- 172.21.15.64:0/1416026293 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df39acd860 con 0x55df396e31b0 2024-09-21T04:13:09.466 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.463+0000 7f99b3fff640 1 --2- 172.21.15.64:0/1416026293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df396e31b0 0x55df39b46d20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x55df39abf0b0 tx=0x7f999c00aca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=245636711ec8383c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:09.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.465+0000 7f99937fe640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-21T04:13:09.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.469+0000 7f99937fe640 1 -- 172.21.15.64:0/1416026293 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c015040 con 0x55df396e31b0 2024-09-21T04:13:09.471 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.469+0000 7f99937fe640 1 -- 172.21.15.64:0/1416026293 <== mon.1 v2:172.21.15.131:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8659+0+0 (secure 0 0 0) 0x7f999c010dc0 con 0x55df396e31b0 2024-09-21T04:13:09.472 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.470+0000 7f99c0c1e740 1 -- 172.21.15.64:0/1416026293 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99a4031c80 con 0x55df396e31b0 2024-09-21T04:13:09.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.471+0000 7f99937fe640 1 -- 172.21.15.64:0/1416026293 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f999c00a370 con 0x55df396e31b0 2024-09-21T04:13:09.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.472+0000 7f99c0c1e740 10 client.26845 fetch_fsmap learned FSMap version 40 2024-09-21T04:13:09.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.472+0000 7f99c0c1e740 1 -- 172.21.15.64:0/1416026293 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55df39b5e760 con 0x55df396e31b0 2024-09-21T04:13:09.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.473+0000 7f99937fe640 1 -- 172.21.15.64:0/1416026293 <== mon.1 v2:172.21.15.131:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99a4031c80 con 0x55df396e31b0 2024-09-21T04:13:09.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.473+0000 7f99937fe640 1 -- 172.21.15.64:0/1416026293 <== mon.1 v2:172.21.15.131:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f999c0315f0 con 0x55df396e31b0 2024-09-21T04:13:09.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.473+0000 7f99c0c1e740 10 client.26845 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:13:09.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.473+0000 7f99c0c1e740 10 client.26845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:13:09.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.473+0000 7f99c0c1e740 20 client.26845 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:09.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.473+0000 7f99c0c1e740 1 --2- 172.21.15.64:0/1416026293 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55df39b5eee0 0x55df39b61300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:09.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.473+0000 7f99c0c1e740 4 client.26845 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T04:13:09.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.473+0000 7f99c0c1e740 1 -- 172.21.15.64:0/1416026293 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55df39a27850 con 0x55df39b5eee0 2024-09-21T04:13:09.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.474+0000 7f99b37fe640 1 --2- 172.21.15.64:0/1416026293 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55df39b5eee0 0x55df39b61300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:09.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.475+0000 7f99937fe640 10 client.26845 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:09.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.475+0000 7f99b37fe640 1 --2- 172.21.15.64:0/1416026293 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55df39b5eee0 0x55df39b61300 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53281d08907519da server_cookie=eeed665e0c057863 in_seq=0 out_seq=0 2024-09-21T04:13:09.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.477+0000 7f99937fe640 1 -- 172.21.15.64:0/1416026293 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3782 (crc 0 0 0) 0x55df39a27850 con 0x55df39b5eee0 2024-09-21T04:13:09.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.477+0000 7f99937fe640 10 client.26845 handle_command_reply: tid=0 2024-09-21T04:13:09.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.477+0000 7f99c0c1e740 1 client.26845 shutdown 2024-09-21T04:13:09.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.478+0000 7f99c0c1e740 1 -- 172.21.15.64:0/1416026293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df396e31b0 msgr2=0x55df39b46d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:09.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.478+0000 7f99c0c1e740 1 --2- 172.21.15.64:0/1416026293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df396e31b0 0x55df39b46d20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x55df39abf0b0 tx=0x7f999c00aca0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.479+0000 7f99c0c1e740 1 -- 172.21.15.64:0/1416026293 shutdown_connections 2024-09-21T04:13:09.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.479+0000 7f99c0c1e740 1 --2- 172.21.15.64:0/1416026293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df3970c070 0x55df39acd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.479+0000 7f99c0c1e740 1 --2- 172.21.15.64:0/1416026293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df396e31b0 0x55df39b46d20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.479+0000 7f99c0c1e740 1 --2- 172.21.15.64:0/1416026293 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55df39b5eee0 0x55df39b61300 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.479+0000 7f99c0c1e740 1 --2- 172.21.15.64:0/1416026293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df3969fda0 0x55df39b467e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.479+0000 7f99937fe640 0 client.26845 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:09.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.479+0000 7f99c0c1e740 1 -- 172.21.15.64:0/1416026293 >> 172.21.15.64:0/1416026293 conn(0x55df39700460 msgr2=0x55df39ac03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:09.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.481+0000 7f99c0c1e740 1 -- 172.21.15.64:0/1416026293 shutdown_connections 2024-09-21T04:13:09.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.481+0000 7f99c0c1e740 1 -- 172.21.15.64:0/1416026293 wait complete. 2024-09-21T04:13:09.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.483+0000 7f99c0c1e740 20 client.26845 trim_cache size 0 max 16384 2024-09-21T04:13:09.486 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:13:09.486 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-21T04:13:09.486 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {}, 2024-09-21T04:13:09.486 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_stats": { 2024-09-21T04:13:09.486 INFO:teuthology.orchestra.run.smithi064.stdout: "1b23713d-ea3a-4a86-a87f-65f6ab82671d": { 2024-09-21T04:13:09.486 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:10:58.352639+0000", 2024-09-21T04:13:09.486 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:09.486 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:09.486 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:09.486 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:09.486 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:09.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:09.487 INFO:teuthology.orchestra.run.smithi064.stdout: "dd25622a-26f0-4838-b998-85d0603c83b6": { 2024-09-21T04:13:09.487 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:12:24.553513+0000", 2024-09-21T04:13:09.487 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:09.487 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:09.487 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:09.487 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:09.487 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:09.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:09.487 INFO:teuthology.orchestra.run.smithi064.stdout: "71f3f255-4837-45e9-92bc-4b3abb5c20eb": { 2024-09-21T04:13:09.487 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:11:42.378549+0000", 2024-09-21T04:13:09.487 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:09.487 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:09.487 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:09.488 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:09.488 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:09.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:09.488 INFO:teuthology.orchestra.run.smithi064.stdout: "2c0523fb-aae4-43fa-9568-136a9b640051": { 2024-09-21T04:13:09.488 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:09:39.808606+0000", 2024-09-21T04:13:09.488 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:09.488 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:09.488 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:09.488 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:09.488 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:09.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:09.488 INFO:teuthology.orchestra.run.smithi064.stdout: "bed40d88-2102-43a6-ac2d-b89d5a2264d0": { 2024-09-21T04:13:09.489 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:11.363340+0000", 2024-09-21T04:13:09.489 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:09.489 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:09.489 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:09.489 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:09.489 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:09.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:09.489 INFO:teuthology.orchestra.run.smithi064.stdout: "424f2ae3-b0e9-4d20-a6cc-7cea3b92b066": { 2024-09-21T04:13:09.489 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:13:07.561468+0000", 2024-09-21T04:13:09.489 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:09.489 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:09.489 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:09.489 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:09.490 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:09.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:09.490 INFO:teuthology.orchestra.run.smithi064.stdout: "e9242f88-6599-40eb-adf1-bdb3e946a1e0": { 2024-09-21T04:13:09.490 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:05:39.744646+0000", 2024-09-21T04:13:09.490 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:09.490 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:09.490 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:09.490 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:09.490 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:09.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:09.490 INFO:teuthology.orchestra.run.smithi064.stdout: "6bde0fe5-f775-44b1-ad0c-f9e596574dc0": { 2024-09-21T04:13:09.490 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:06:23.835536+0000", 2024-09-21T04:13:09.490 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:09.490 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:09.491 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:09.491 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:09.491 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:09.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:09.491 INFO:teuthology.orchestra.run.smithi064.stdout: "b52b950c-0f97-49ec-a37c-1a8e04348ce8": { 2024-09-21T04:13:09.491 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:54.003800+0000", 2024-09-21T04:13:09.491 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:09.491 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:09.491 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:09.491 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:09.491 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:09.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:09.491 INFO:teuthology.orchestra.run.smithi064.stdout: "e2c2eed8-9292-4bf6-8a39-13ad27da5acb": { 2024-09-21T04:13:09.491 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:08:47.067932+0000", 2024-09-21T04:13:09.491 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:09.492 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:09.492 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:09.492 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:09.492 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:09.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:09.492 INFO:teuthology.orchestra.run.smithi064.stdout: "75db1bc5-612e-45aa-bad2-cc90feeaa840": { 2024-09-21T04:13:09.492 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:59.123138+0000", 2024-09-21T04:13:09.492 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:09.492 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:09.492 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:09.492 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:09.493 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:09.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:09.493 INFO:teuthology.orchestra.run.smithi064.stdout: "0ab717a7-36c6-4dca-94e7-af0bdf2aa112": { 2024-09-21T04:13:09.493 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:42.691086+0000", 2024-09-21T04:13:09.493 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:09.493 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:09.493 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:09.493 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:09.493 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:09.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:09.493 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:13:09.493 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:03:59.570174+0000", 2024-09-21T04:13:09.493 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:09.494 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:09.494 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:09.494 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:09.494 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:09.494 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:13:09.494 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:13:09.494 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:13:09.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.485+0000 7f99aaffd640 1 -- 172.21.15.64:0/1703138387 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f99a00bac10 msgr2=0x7f99a00bd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:09.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.486+0000 7f99aaffd640 1 --2- 172.21.15.64:0/1703138387 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f99a00bac10 0x7f99a00bd120 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f99ac0ac990 tx=0x7f99ac004b50 comp rx=0 tx=0).stop 2024-09-21T04:13:09.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.486+0000 7f99aaffd640 1 -- 172.21.15.64:0/1703138387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b4109e20 msgr2=0x7f99b419d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:09.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.486+0000 7f99aaffd640 1 --2- 172.21.15.64:0/1703138387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b4109e20 0x7f99b419d9c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00be20 tx=0x7f99ac00bf10 comp rx=0 tx=0).stop 2024-09-21T04:13:09.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.486+0000 7f99aaffd640 1 -- 172.21.15.64:0/1703138387 shutdown_connections 2024-09-21T04:13:09.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.486+0000 7f99aaffd640 1 --2- 172.21.15.64:0/1703138387 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f99a00bac10 0x7f99a00bd120 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.486+0000 7f99aaffd640 1 --2- 172.21.15.64:0/1703138387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99b4109e20 0x7f99b419d9c0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.486+0000 7f99aaffd640 1 --2- 172.21.15.64:0/1703138387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b41094d0 0x7f99b4198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.486+0000 7f99aaffd640 1 --2- 172.21.15.64:0/1703138387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99b4103b90 0x7f99b4198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:09.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.486+0000 7f99aaffd640 1 -- 172.21.15.64:0/1703138387 >> 172.21.15.64:0/1703138387 conn(0x7f99b40ff5e0 msgr2=0x7f99b41069b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:09.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.486+0000 7f99aaffd640 1 -- 172.21.15.64:0/1703138387 shutdown_connections 2024-09-21T04:13:09.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:09.486+0000 7f99aaffd640 1 -- 172.21.15.64:0/1703138387 wait complete. 2024-09-21T04:13:09.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:09 smithi119 ceph-mon[34811]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T04:13:09.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:09 smithi131 ceph-mon[35193]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T04:13:10.269 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T04:13:10.269 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:13:10.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:10 smithi119 ceph-mon[34811]: pgmap v602: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-21T04:13:10.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:10 smithi064 ceph-mon[31180]: pgmap v602: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-21T04:13:10.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:10 smithi131 ceph-mon[35193]: pgmap v602: 97 pgs: 97 active+clean; 776 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-21T04:13:10.731 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:13:11.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.539+0000 7f9119041640 1 -- 172.21.15.64:0/1099164238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f910c0947a0 msgr2=0x7f910c094bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:11.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.539+0000 7f9119041640 1 --2- 172.21.15.64:0/1099164238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f910c0947a0 0x7f910c094bb0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f910800a910 tx=0x7f910802a610 comp rx=0 tx=0).stop 2024-09-21T04:13:11.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.541+0000 7f9119041640 1 -- 172.21.15.64:0/1099164238 shutdown_connections 2024-09-21T04:13:11.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.541+0000 7f9119041640 1 --2- 172.21.15.64:0/1099164238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f910c09aa30 0x7f910c0a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:11.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.541+0000 7f9119041640 1 --2- 172.21.15.64:0/1099164238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f910c09a0e0 0x7f910c09a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:11.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.541+0000 7f9119041640 1 --2- 172.21.15.64:0/1099164238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f910c0947a0 0x7f910c094bb0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:11.755 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.541+0000 7f9119041640 1 -- 172.21.15.64:0/1099164238 >> 172.21.15.64:0/1099164238 conn(0x7f910c00c4a0 msgr2=0x7f910c00c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:11.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.542+0000 7f9119041640 1 -- 172.21.15.64:0/1099164238 shutdown_connections 2024-09-21T04:13:11.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.542+0000 7f9119041640 1 -- 172.21.15.64:0/1099164238 wait complete. 2024-09-21T04:13:11.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.543+0000 7f9119041640 1 Processor -- start 2024-09-21T04:13:11.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.543+0000 7f9119041640 1 -- start start 2024-09-21T04:13:11.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.544+0000 7f9119041640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f910c0947a0 0x7f910c144070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:11.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.544+0000 7f9119041640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f910c09a0e0 0x7f910c144610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:11.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.544+0000 7f9119041640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f910c09aa30 0x7f910c1498b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:11.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.544+0000 7f9119041640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f910c0b4ed0 con 0x7f910c09aa30 2024-09-21T04:13:11.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.544+0000 7f9119041640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f910c0b4d50 con 0x7f910c0947a0 2024-09-21T04:13:11.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.544+0000 7f9119041640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f910c0b5050 con 0x7f910c09a0e0 2024-09-21T04:13:11.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.544+0000 7f9113fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f910c0947a0 0x7f910c144070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:11.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.544+0000 7f9118840640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f910c09aa30 0x7f910c1498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:11.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.544+0000 7f9113fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f910c0947a0 0x7f910c144070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:35864/0 (socket says 172.21.15.64:35864) 2024-09-21T04:13:11.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.544+0000 7f9113fff640 1 -- 172.21.15.64:0/2955751398 learned_addr learned my addr 172.21.15.64:0/2955751398 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:11.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.544+0000 7f91137fe640 1 --2- 172.21.15.64:0/2955751398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f910c09a0e0 0x7f910c144610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:11.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.545+0000 7f9113fff640 1 -- 172.21.15.64:0/2955751398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f910c09a0e0 msgr2=0x7f910c144610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:11.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.545+0000 7f9113fff640 1 --2- 172.21.15.64:0/2955751398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f910c09a0e0 0x7f910c144610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:11.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.545+0000 7f9113fff640 1 -- 172.21.15.64:0/2955751398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f910c09aa30 msgr2=0x7f910c1498b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:11.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.545+0000 7f9113fff640 1 --2- 172.21.15.64:0/2955751398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f910c09aa30 0x7f910c1498b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:11.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.545+0000 7f9113fff640 1 -- 172.21.15.64:0/2955751398 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910c149fe0 con 0x7f910c0947a0 2024-09-21T04:13:11.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.545+0000 7f9113fff640 1 --2- 172.21.15.64:0/2955751398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f910c0947a0 0x7f910c144070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f910800aa30 tx=0x7f9108031630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:11.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.545+0000 7f91117fa640 1 -- 172.21.15.64:0/2955751398 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108040020 con 0x7f910c0947a0 2024-09-21T04:13:11.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.546+0000 7f91117fa640 1 -- 172.21.15.64:0/2955751398 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9108004460 con 0x7f910c0947a0 2024-09-21T04:13:11.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.547+0000 7f91117fa640 1 -- 172.21.15.64:0/2955751398 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108035070 con 0x7f910c0947a0 2024-09-21T04:13:11.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.547+0000 7f9119041640 1 -- 172.21.15.64:0/2955751398 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f910c14a270 con 0x7f910c0947a0 2024-09-21T04:13:11.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.547+0000 7f9119041640 1 -- 172.21.15.64:0/2955751398 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f910c14a6d0 con 0x7f910c0947a0 2024-09-21T04:13:11.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.549+0000 7f9119041640 1 -- 172.21.15.64:0/2955751398 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f910c095a20 con 0x7f910c0947a0 2024-09-21T04:13:11.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.552+0000 7f91117fa640 1 -- 172.21.15.64:0/2955751398 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f9108043020 con 0x7f910c0947a0 2024-09-21T04:13:11.757 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.553+0000 7f91117fa640 1 --2- 172.21.15.64:0/2955751398 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f91040b6670 0x7f91040b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:11.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.553+0000 7f91137fe640 1 --2- 172.21.15.64:0/2955751398 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f91040b6670 0x7f91040b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:11.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.553+0000 7f91117fa640 1 -- 172.21.15.64:0/2955751398 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8659+0+0 (secure 0 0 0) 0x7f9108108530 con 0x7f910c0947a0 2024-09-21T04:13:11.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.554+0000 7f91137fe640 1 --2- 172.21.15.64:0/2955751398 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f91040b6670 0x7f91040b8b80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f910000b3c0 tx=0x7f910000d040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:11.758 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.558+0000 7f91117fa640 1 -- 172.21.15.64:0/2955751398 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f910804a020 con 0x7f910c0947a0 2024-09-21T04:13:11.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.937+0000 7f9119041640 1 -- 172.21.15.64:0/2955751398 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f910c094bb0 con 0x7f910c0947a0 2024-09-21T04:13:11.941 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.938+0000 7f91117fa640 1 -- 172.21.15.64:0/2955751398 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11918 (secure 0 0 0) 0x7f91080cea70 con 0x7f910c0947a0 2024-09-21T04:13:11.941 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:13:11.941 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":40,"btime":"2024-09-21T04:11:59:330881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14805,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/97995119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":97995119},{"type":"v1","addr":"172.21.15.64:6839","nonce":97995119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/390891068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":390891068},{"type":"v1","addr":"172.21.15.64:6841","nonce":390891068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:02:03.354744+0000","modified":"2024-09-21T04:11:59.188679+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24710,"mds_1":24680,"mds_2":24661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24634":{"gid":24634,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6839/782190424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782190424},{"type":"v1","addr":"172.21.15.131:6839","nonce":782190424}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24680":{"gid":24680,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24710":{"gid":24710,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6839/1319793636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1319793636},{"type":"v1","addr":"172.21.15.119:6839","nonce":1319793636}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24710,"qdb_cluster":[24661,24680,24710]},"id":1}]} 2024-09-21T04:13:11.941 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 40 2024-09-21T04:13:11.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.941+0000 7f9119041640 1 -- 172.21.15.64:0/2955751398 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f91040b6670 msgr2=0x7f91040b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:11.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.941+0000 7f9119041640 1 --2- 172.21.15.64:0/2955751398 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f91040b6670 0x7f91040b8b80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f910000b3c0 tx=0x7f910000d040 comp rx=0 tx=0).stop 2024-09-21T04:13:11.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.941+0000 7f9119041640 1 -- 172.21.15.64:0/2955751398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f910c0947a0 msgr2=0x7f910c144070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:11.944 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.941+0000 7f9119041640 1 --2- 172.21.15.64:0/2955751398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f910c0947a0 0x7f910c144070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f910800aa30 tx=0x7f9108031630 comp rx=0 tx=0).stop 2024-09-21T04:13:11.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.942+0000 7f9119041640 1 -- 172.21.15.64:0/2955751398 shutdown_connections 2024-09-21T04:13:11.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.942+0000 7f9119041640 1 --2- 172.21.15.64:0/2955751398 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f91040b6670 0x7f91040b8b80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:11.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.942+0000 7f9119041640 1 --2- 172.21.15.64:0/2955751398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f910c09aa30 0x7f910c1498b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:11.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.942+0000 7f9119041640 1 --2- 172.21.15.64:0/2955751398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f910c09a0e0 0x7f910c144610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:11.945 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.942+0000 7f9119041640 1 --2- 172.21.15.64:0/2955751398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f910c0947a0 0x7f910c144070 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:11.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.942+0000 7f9119041640 1 -- 172.21.15.64:0/2955751398 >> 172.21.15.64:0/2955751398 conn(0x7f910c00c4a0 msgr2=0x7f910c098130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:11.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.943+0000 7f9119041640 1 -- 172.21.15.64:0/2955751398 shutdown_connections 2024-09-21T04:13:11.946 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:11.943+0000 7f9119041640 1 -- 172.21.15.64:0/2955751398 wait complete. 2024-09-21T04:13:12.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:12 smithi119 ceph-mon[34811]: pgmap v603: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 775 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-21T04:13:12.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:12 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2955751398' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:13:12.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:12 smithi064 ceph-mon[31180]: pgmap v603: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 775 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-21T04:13:12.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:12 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2955751398' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:13:12.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:12 smithi131 ceph-mon[35193]: pgmap v603: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 775 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-09-21T04:13:12.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:12 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2955751398' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:13:12.636 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T04:13:12.898 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:13:13.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.397+0000 7f2d121ec640 1 -- 172.21.15.64:0/3702041536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d0c1094a0 msgr2=0x7f2d0c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:13.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.397+0000 7f2d121ec640 1 --2- 172.21.15.64:0/3702041536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d0c1094a0 0x7f2d0c1098b0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009900 tx=0x7f2d0002a410 comp rx=0 tx=0).stop 2024-09-21T04:13:13.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.397+0000 7f2d121ec640 1 -- 172.21.15.64:0/3702041536 shutdown_connections 2024-09-21T04:13:13.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.397+0000 7f2d121ec640 1 --2- 172.21.15.64:0/3702041536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d0c109df0 0x7f2d0c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.397+0000 7f2d121ec640 1 --2- 172.21.15.64:0/3702041536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d0c1094a0 0x7f2d0c1098b0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.397+0000 7f2d121ec640 1 --2- 172.21.15.64:0/3702041536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d0c103b60 0x7f2d0c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.397+0000 7f2d121ec640 1 -- 172.21.15.64:0/3702041536 >> 172.21.15.64:0/3702041536 conn(0x7f2d0c07b850 msgr2=0x7f2d0c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:13.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.397+0000 7f2d121ec640 1 -- 172.21.15.64:0/3702041536 shutdown_connections 2024-09-21T04:13:13.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.397+0000 7f2d121ec640 1 -- 172.21.15.64:0/3702041536 wait complete. 2024-09-21T04:13:13.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.398+0000 7f2d121ec640 1 Processor -- start 2024-09-21T04:13:13.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.398+0000 7f2d121ec640 1 -- start start 2024-09-21T04:13:13.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.398+0000 7f2d121ec640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d0c103b60 0x7f2d0c19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:13.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.398+0000 7f2d121ec640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d0c1094a0 0x7f2d0c19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:13.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.398+0000 7f2d121ec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d0c109df0 0x7f2d0c19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:13.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.398+0000 7f2d121ec640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c113ab0 con 0x7f2d0c1094a0 2024-09-21T04:13:13.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.398+0000 7f2d121ec640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c113930 con 0x7f2d0c109df0 2024-09-21T04:13:13.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.398+0000 7f2d121ec640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c113c30 con 0x7f2d0c103b60 2024-09-21T04:13:13.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.399+0000 7f2d0affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d0c1094a0 0x7f2d0c19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:13.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.399+0000 7f2d0affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d0c1094a0 0x7f2d0c19ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48396/0 (socket says 172.21.15.64:48396) 2024-09-21T04:13:13.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.399+0000 7f2d0affd640 1 -- 172.21.15.64:0/546740470 learned_addr learned my addr 172.21.15.64:0/546740470 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:13.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.399+0000 7f2d0b7fe640 1 --2- 172.21.15.64:0/546740470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d0c103b60 0x7f2d0c19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:13.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.399+0000 7f2d0bfff640 1 --2- 172.21.15.64:0/546740470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d0c109df0 0x7f2d0c19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:13.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.399+0000 7f2d0affd640 1 -- 172.21.15.64:0/546740470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d0c103b60 msgr2=0x7f2d0c19a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:13.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.399+0000 7f2d0affd640 1 --2- 172.21.15.64:0/546740470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d0c103b60 0x7f2d0c19a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.399+0000 7f2d0affd640 1 -- 172.21.15.64:0/546740470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d0c109df0 msgr2=0x7f2d0c19fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:13.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.399+0000 7f2d0affd640 1 --2- 172.21.15.64:0/546740470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d0c109df0 0x7f2d0c19fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.399+0000 7f2d0affd640 1 -- 172.21.15.64:0/546740470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0c1a0410 con 0x7f2d0c1094a0 2024-09-21T04:13:13.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.399+0000 7f2d0bfff640 1 --2- 172.21.15.64:0/546740470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d0c109df0 0x7f2d0c19fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:13:13.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.399+0000 7f2d0affd640 1 --2- 172.21.15.64:0/546740470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d0c1094a0 0x7f2d0c19ab00 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f2d0002a930 tx=0x7f2d00004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:13.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.399+0000 7f2d08ff9640 1 -- 172.21.15.64:0/546740470 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0003f030 con 0x7f2d0c1094a0 2024-09-21T04:13:13.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.399+0000 7f2d08ff9640 1 -- 172.21.15.64:0/546740470 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d00003aa0 con 0x7f2d0c1094a0 2024-09-21T04:13:13.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.399+0000 7f2d121ec640 1 -- 172.21.15.64:0/546740470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d0c1a06a0 con 0x7f2d0c1094a0 2024-09-21T04:13:13.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.399+0000 7f2d08ff9640 1 -- 172.21.15.64:0/546740470 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00035070 con 0x7f2d0c1094a0 2024-09-21T04:13:13.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.399+0000 7f2d121ec640 1 -- 172.21.15.64:0/546740470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d0c1a0b80 con 0x7f2d0c1094a0 2024-09-21T04:13:13.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.402+0000 7f2d08ff9640 1 -- 172.21.15.64:0/546740470 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f2d00042030 con 0x7f2d0c1094a0 2024-09-21T04:13:13.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.402+0000 7f2d18760740 1 Processor -- start 2024-09-21T04:13:13.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.402+0000 7f2d18760740 1 -- start start 2024-09-21T04:13:13.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.402+0000 7f2d18760740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563add75c070 0x563addb032c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:13.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.402+0000 7f2d18760740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563add7331b0 0x563addb03800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:13.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.402+0000 7f2d18760740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563add73b390 0x563addb03d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:13.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.402+0000 7f2d18760740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563adda79ea0 con 0x563add73b390 2024-09-21T04:13:13.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.403+0000 7f2d18760740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563adda692d0 con 0x563add7331b0 2024-09-21T04:13:13.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.403+0000 7f2d18760740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x563adda6d0e0 con 0x563add75c070 2024-09-21T04:13:13.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.403+0000 7f2d0bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563add75c070 0x563addb032c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:13.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.403+0000 7f2d0affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563add73b390 0x563addb03d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:13.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.403+0000 7f2d0b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563add7331b0 0x563addb03800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:13.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.403+0000 7f2d0b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563add7331b0 0x563addb03800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.64:35896/0 (socket says 172.21.15.64:35896) 2024-09-21T04:13:13.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.403+0000 7f2d0b7fe640 1 -- 172.21.15.64:0/1498105221 learned_addr learned my addr 172.21.15.64:0/1498105221 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:13.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.403+0000 7f2d0affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563add73b390 0x563addb03d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48412/0 (socket says 172.21.15.64:48412) 2024-09-21T04:13:13.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.403+0000 7f2d0bfff640 1 -- 172.21.15.64:0/1498105221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563add7331b0 msgr2=0x563addb03800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:13.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.403+0000 7f2d0bfff640 1 --2- 172.21.15.64:0/1498105221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563add7331b0 0x563addb03800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.403+0000 7f2d0bfff640 1 -- 172.21.15.64:0/1498105221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563add73b390 msgr2=0x563addb03d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:13.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.403+0000 7f2d0bfff640 1 --2- 172.21.15.64:0/1498105221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563add73b390 0x563addb03d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.403+0000 7f2d0bfff640 1 -- 172.21.15.64:0/1498105221 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563add7c7650 con 0x563add75c070 2024-09-21T04:13:13.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.403+0000 7f2d08ff9640 1 --2- 172.21.15.64:0/546740470 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2cf40b66b0 0x7f2cf40b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:13.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.403+0000 7f2d0b7fe640 1 --2- 172.21.15.64:0/546740470 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2cf40b66b0 0x7f2cf40b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:13.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.403+0000 7f2d0bfff640 1 --2- 172.21.15.64:0/1498105221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563add75c070 0x563addb032c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009d80 tx=0x7f2d0400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24b53ed65b8c54cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:13.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.403+0000 7f2d08ff9640 1 -- 172.21.15.64:0/546740470 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8659+0+0 (secure 0 0 0) 0x7f2d00107ea0 con 0x7f2d0c1094a0 2024-09-21T04:13:13.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.404+0000 7f2cf1ffb640 1 -- 172.21.15.64:0/1498105221 <== mon.1 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04004b20 con 0x563add75c070 2024-09-21T04:13:13.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.404+0000 7f2cf1ffb640 1 -- 172.21.15.64:0/1498105221 <== mon.1 v2:172.21.15.131:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d04004cc0 con 0x563add75c070 2024-09-21T04:13:13.411 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.404+0000 7f2cf1ffb640 1 -- 172.21.15.64:0/1498105221 <== mon.1 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04011710 con 0x563add75c070 2024-09-21T04:13:13.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.405+0000 7f2d18760740 1 -- 172.21.15.64:0/1498105221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563add75c070 msgr2=0x563addb032c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:13.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.405+0000 7f2d18760740 1 --2- 172.21.15.64:0/1498105221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563add75c070 0x563addb032c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009d80 tx=0x7f2d0400f730 comp rx=0 tx=0).stop 2024-09-21T04:13:13.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.405+0000 7f2d0b7fe640 1 --2- 172.21.15.64:0/546740470 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2cf40b66b0 0x7f2cf40b8bc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x563addb01f40 tx=0x7f2cfc00a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:13.414 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.405+0000 7f2d18760740 1 -- 172.21.15.64:0/1498105221 shutdown_connections 2024-09-21T04:13:13.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.405+0000 7f2d18760740 1 --2- 172.21.15.64:0/1498105221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563add73b390 0x563addb03d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.405+0000 7f2d18760740 1 --2- 172.21.15.64:0/1498105221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563add7331b0 0x563addb03800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.405+0000 7f2d18760740 1 --2- 172.21.15.64:0/1498105221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563add75c070 0x563addb032c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.405+0000 7f2d18760740 1 -- 172.21.15.64:0/1498105221 >> 172.21.15.64:0/1498105221 conn(0x563add750460 msgr2=0x563addb011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:13.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.405+0000 7f2d18760740 1 -- 172.21.15.64:0/1498105221 shutdown_connections 2024-09-21T04:13:13.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.405+0000 7f2d18760740 1 -- 172.21.15.64:0/1498105221 wait complete. 2024-09-21T04:13:13.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.405+0000 7f2d18760740 1 -- start start 2024-09-21T04:13:13.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.405+0000 7f2d18760740 1 Processor -- start 2024-09-21T04:13:13.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.406+0000 7f2d18760740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563add7331b0 0x563addb0e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:13.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.406+0000 7f2d18760740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563add73b390 0x563addb0eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:13.415 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.406+0000 7f2d18760740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563add75c070 0x563addb1f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:13.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.406+0000 7f2d18760740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563add8ca700 con 0x563add75c070 2024-09-21T04:13:13.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.406+0000 7f2d18760740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563adda04160 con 0x563add7331b0 2024-09-21T04:13:13.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.406+0000 7f2d18760740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x563addb15eb0 con 0x563add73b390 2024-09-21T04:13:13.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.406+0000 7f2d18760740 10 client.0 _ll_register_callbacks cb 0x563adda1d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:13.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.406+0000 7f2d0bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563add75c070 0x563addb1f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:13.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.406+0000 7f2d0bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563add75c070 0x563addb1f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48420/0 (socket says 172.21.15.64:48420) 2024-09-21T04:13:13.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.406+0000 7f2d0bfff640 1 -- 172.21.15.64:0/1470072421 learned_addr learned my addr 172.21.15.64:0/1470072421 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:13.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.406+0000 7f2d0b7fe640 1 --2- 172.21.15.64:0/1470072421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563add7331b0 0x563addb0e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:13.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.406+0000 7f2d0affd640 1 --2- 172.21.15.64:0/1470072421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563add73b390 0x563addb0eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:13.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.406+0000 7f2d0bfff640 1 -- 172.21.15.64:0/1470072421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563add73b390 msgr2=0x563addb0eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:13.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.406+0000 7f2d0bfff640 1 --2- 172.21.15.64:0/1470072421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563add73b390 0x563addb0eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.406+0000 7f2d0bfff640 1 -- 172.21.15.64:0/1470072421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563add7331b0 msgr2=0x563addb0e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:13.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.406+0000 7f2d0bfff640 1 --2- 172.21.15.64:0/1470072421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563add7331b0 0x563addb0e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.416 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.406+0000 7f2d0bfff640 1 -- 172.21.15.64:0/1470072421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563adda31c90 con 0x563add75c070 2024-09-21T04:13:13.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.406+0000 7f2d0bfff640 1 --2- 172.21.15.64:0/1470072421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563add75c070 0x563addb1f1d0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f2d04005920 tx=0x7f2d04009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2fcaed66ab9109 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:13.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.406+0000 7f2ceaffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:13:13.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.407+0000 7f2ceaffd640 1 -- 172.21.15.64:0/1470072421 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04029060 con 0x563add75c070 2024-09-21T04:13:13.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.407+0000 7f2ceaffd640 1 -- 172.21.15.64:0/1470072421 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8659+0+0 (secure 0 0 0) 0x7f2d04005080 con 0x563add75c070 2024-09-21T04:13:13.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.407+0000 7f2d18760740 1 -- 172.21.15.64:0/1470072421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d04004cc0 con 0x563add75c070 2024-09-21T04:13:13.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.407+0000 7f2ceaffd640 1 -- 172.21.15.64:0/1470072421 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f2d04030050 con 0x563add75c070 2024-09-21T04:13:13.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.407+0000 7f2d18760740 10 client.17388 fetch_fsmap learned FSMap version 40 2024-09-21T04:13:13.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.407+0000 7f2d18760740 1 -- 172.21.15.64:0/1470072421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563addb21c20 con 0x563add75c070 2024-09-21T04:13:13.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.407+0000 7f2ceaffd640 1 -- 172.21.15.64:0/1470072421 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d04004cc0 con 0x563add75c070 2024-09-21T04:13:13.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.407+0000 7f2ceaffd640 1 -- 172.21.15.64:0/1470072421 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f2d04011b70 con 0x563add75c070 2024-09-21T04:13:13.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.407+0000 7f2d18760740 10 client.17388 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:13:13.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.407+0000 7f2d18760740 10 client.17388 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:13:13.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.407+0000 7f2d18760740 20 client.17388 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:13.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.407+0000 7f2d18760740 1 --2- 172.21.15.64:0/1470072421 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x563add71a9f0 0x563addb25090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:13.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.407+0000 7f2d18760740 4 client.17388 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:13:13.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.407+0000 7f2d18760740 1 -- 172.21.15.64:0/1470072421 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563adda7bdf0 con 0x563add71a9f0 2024-09-21T04:13:13.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.408+0000 7f2d0b7fe640 1 --2- 172.21.15.64:0/1470072421 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x563add71a9f0 0x563addb25090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:13.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.408+0000 7f2ceaffd640 10 client.17388 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:13:13.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.408+0000 7f2d0b7fe640 1 --2- 172.21.15.64:0/1470072421 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x563add71a9f0 0x563addb25090 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f7d2f8411808d621 server_cookie=4e41044ee40102b4 in_seq=0 out_seq=0 2024-09-21T04:13:13.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.409+0000 7f2ceaffd640 1 -- 172.21.15.64:0/1470072421 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563adda7bdf0 con 0x563add71a9f0 2024-09-21T04:13:13.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.409+0000 7f2ceaffd640 10 client.17388 handle_command_reply: tid=0 2024-09-21T04:13:13.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.409+0000 7f2d18760740 1 client.17388 shutdown 2024-09-21T04:13:13.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.409+0000 7f2d18760740 1 -- 172.21.15.64:0/1470072421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563add75c070 msgr2=0x563addb1f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:13.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.409+0000 7f2d18760740 1 --2- 172.21.15.64:0/1470072421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563add75c070 0x563addb1f1d0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f2d04005920 tx=0x7f2d04009aa0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.409+0000 7f2d18760740 1 -- 172.21.15.64:0/1470072421 shutdown_connections 2024-09-21T04:13:13.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.409+0000 7f2d18760740 1 --2- 172.21.15.64:0/1470072421 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x563add71a9f0 0x563addb25090 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.409+0000 7f2d18760740 1 --2- 172.21.15.64:0/1470072421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563add75c070 0x563addb1f1d0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.409+0000 7f2d18760740 1 --2- 172.21.15.64:0/1470072421 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563add73b390 0x563addb0eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.409+0000 7f2d18760740 1 --2- 172.21.15.64:0/1470072421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563add7331b0 0x563addb0e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.409+0000 7f2ceaffd640 0 client.17388 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:13:13.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.409+0000 7f2d18760740 1 -- 172.21.15.64:0/1470072421 >> 172.21.15.64:0/1470072421 conn(0x563add750460 msgr2=0x563addb06450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:13.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.409+0000 7f2d18760740 1 -- 172.21.15.64:0/1470072421 shutdown_connections 2024-09-21T04:13:13.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.409+0000 7f2d18760740 1 -- 172.21.15.64:0/1470072421 wait complete. 2024-09-21T04:13:13.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.409+0000 7f2d18760740 20 client.17388 trim_cache size 0 max 16384 2024-09-21T04:13:13.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.422+0000 7f2d18760740 1 Processor -- start 2024-09-21T04:13:13.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.422+0000 7f2d18760740 1 -- start start 2024-09-21T04:13:13.426 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.423+0000 7f2d18760740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563add71a9f0 0x563addb0ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:13.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.423+0000 7f2d18760740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563add7331b0 0x563addb1f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:13.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.423+0000 7f2d18760740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563add73b390 0x563addb0bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:13.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.423+0000 7f2d18760740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c1b0140 con 0x563add71a9f0 2024-09-21T04:13:13.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.423+0000 7f2d18760740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c1ad950 con 0x563add73b390 2024-09-21T04:13:13.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.423+0000 7f2d18760740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c1ab7c0 con 0x563add7331b0 2024-09-21T04:13:13.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.423+0000 7f2d0b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563add71a9f0 0x563addb0ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:13.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.423+0000 7f2d0b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563add71a9f0 0x563addb0ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48434/0 (socket says 172.21.15.64:48434) 2024-09-21T04:13:13.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.423+0000 7f2d0b7fe640 1 -- 172.21.15.64:0/1688148926 learned_addr learned my addr 172.21.15.64:0/1688148926 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:13.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.423+0000 7f2d0affd640 1 --2- 172.21.15.64:0/1688148926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563add7331b0 0x563addb1f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:13.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.423+0000 7f2d0bfff640 1 --2- 172.21.15.64:0/1688148926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563add73b390 0x563addb0bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:13.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.423+0000 7f2d0b7fe640 1 -- 172.21.15.64:0/1688148926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563add7331b0 msgr2=0x563addb1f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:13.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.423+0000 7f2d0b7fe640 1 --2- 172.21.15.64:0/1688148926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563add7331b0 0x563addb1f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.423+0000 7f2d0b7fe640 1 -- 172.21.15.64:0/1688148926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563add73b390 msgr2=0x563addb0bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:13.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.423+0000 7f2d0b7fe640 1 --2- 172.21.15.64:0/1688148926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563add73b390 0x563addb0bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.427 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.423+0000 7f2d0b7fe640 1 -- 172.21.15.64:0/1688148926 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563addb055b0 con 0x563add71a9f0 2024-09-21T04:13:13.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.423+0000 7f2d0bfff640 1 --2- 172.21.15.64:0/1688148926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563add73b390 0x563addb0bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:13:13.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.424+0000 7f2d0b7fe640 1 --2- 172.21.15.64:0/1688148926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563add71a9f0 0x563addb0ee70 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00a570 tx=0x7f2cfc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2286bd7cee1ca70 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:13.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.424+0000 7f2cf17fa640 1 -- 172.21.15.64:0/1688148926 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc033070 con 0x563add71a9f0 2024-09-21T04:13:13.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.424+0000 7f2cf17fa640 1 -- 172.21.15.64:0/1688148926 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cfc01bde0 con 0x563add71a9f0 2024-09-21T04:13:13.428 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.424+0000 7f2cf17fa640 1 -- 172.21.15.64:0/1688148926 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc028780 con 0x563add71a9f0 2024-09-21T04:13:13.431 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.424+0000 7f2d18760740 1 -- 172.21.15.64:0/1688148926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563add71a9f0 msgr2=0x563addb0ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:13.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.424+0000 7f2d18760740 1 --2- 172.21.15.64:0/1688148926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563add71a9f0 0x563addb0ee70 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00a570 tx=0x7f2cfc019000 comp rx=0 tx=0).stop 2024-09-21T04:13:13.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.425+0000 7f2d18760740 1 -- 172.21.15.64:0/1688148926 shutdown_connections 2024-09-21T04:13:13.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.425+0000 7f2d18760740 1 --2- 172.21.15.64:0/1688148926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563add73b390 0x563addb0bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.425+0000 7f2d18760740 1 --2- 172.21.15.64:0/1688148926 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563add7331b0 0x563addb1f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.425+0000 7f2d18760740 1 --2- 172.21.15.64:0/1688148926 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563add71a9f0 0x563addb0ee70 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.425+0000 7f2d18760740 1 -- 172.21.15.64:0/1688148926 >> 172.21.15.64:0/1688148926 conn(0x563add750460 msgr2=0x563addb0e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:13.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.425+0000 7f2d18760740 1 -- 172.21.15.64:0/1688148926 shutdown_connections 2024-09-21T04:13:13.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.425+0000 7f2d18760740 1 -- 172.21.15.64:0/1688148926 wait complete. 2024-09-21T04:13:13.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.425+0000 7f2d18760740 1 -- start start 2024-09-21T04:13:13.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.425+0000 7f2d18760740 1 Processor -- start 2024-09-21T04:13:13.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.426+0000 7f2d18760740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563add71a9f0 0x563addb28a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:13.432 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.426+0000 7f2d18760740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563add7331b0 0x563addb25f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:13.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.426+0000 7f2d18760740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563add73b390 0x563addb265e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:13.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.426+0000 7f2d18760740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c1abe10 con 0x563add73b390 2024-09-21T04:13:13.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.426+0000 7f2d18760740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c1ad280 con 0x563add7331b0 2024-09-21T04:13:13.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.426+0000 7f2d18760740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c1ac4e0 con 0x563add71a9f0 2024-09-21T04:13:13.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.426+0000 7f2d18760740 10 client.0 _ll_register_callbacks cb 0x7f2d0010b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:13.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.426+0000 7f2d0bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563add73b390 0x563addb265e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:13.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.426+0000 7f2d0bfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563add73b390 0x563addb265e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48444/0 (socket says 172.21.15.64:48444) 2024-09-21T04:13:13.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.426+0000 7f2d0bfff640 1 -- 172.21.15.64:0/1786214117 learned_addr learned my addr 172.21.15.64:0/1786214117 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:13.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.426+0000 7f2d0affd640 1 --2- 172.21.15.64:0/1786214117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563add7331b0 0x563addb25f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:13.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.426+0000 7f2d0b7fe640 1 --2- 172.21.15.64:0/1786214117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563add71a9f0 0x563addb28a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:13.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.426+0000 7f2d0bfff640 1 -- 172.21.15.64:0/1786214117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563add71a9f0 msgr2=0x563addb28a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:13.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.426+0000 7f2d0bfff640 1 --2- 172.21.15.64:0/1786214117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563add71a9f0 0x563addb28a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.426+0000 7f2d0bfff640 1 -- 172.21.15.64:0/1786214117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563add7331b0 msgr2=0x563addb25f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:13.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.426+0000 7f2d0bfff640 1 --2- 172.21.15.64:0/1786214117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563add7331b0 0x563addb25f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.426+0000 7f2d0bfff640 1 -- 172.21.15.64:0/1786214117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563addb8e810 con 0x563add73b390 2024-09-21T04:13:13.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.426+0000 7f2d0bfff640 1 --2- 172.21.15.64:0/1786214117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563add73b390 0x563addb265e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f2d04002230 tx=0x7f2d04009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1d56ab23d52e91 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:13.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.426+0000 7f2ceaffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:13:13.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.427+0000 7f2ceaffd640 1 -- 172.21.15.64:0/1786214117 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04029050 con 0x563add73b390 2024-09-21T04:13:13.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.427+0000 7f2ceaffd640 1 -- 172.21.15.64:0/1786214117 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8659+0+0 (secure 0 0 0) 0x7f2d04024a80 con 0x563add73b390 2024-09-21T04:13:13.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.427+0000 7f2d18760740 1 -- 172.21.15.64:0/1786214117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cfc01bde0 con 0x563add73b390 2024-09-21T04:13:13.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.427+0000 7f2ceaffd640 1 -- 172.21.15.64:0/1786214117 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f2d0400fce0 con 0x563add73b390 2024-09-21T04:13:13.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.427+0000 7f2d18760740 10 client.17394 fetch_fsmap learned FSMap version 40 2024-09-21T04:13:13.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.427+0000 7f2d18760740 1 -- 172.21.15.64:0/1786214117 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563addb8f8e0 con 0x563add73b390 2024-09-21T04:13:13.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.427+0000 7f2ceaffd640 1 -- 172.21.15.64:0/1786214117 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cfc01bde0 con 0x563add73b390 2024-09-21T04:13:13.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.427+0000 7f2ceaffd640 1 -- 172.21.15.64:0/1786214117 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f2d04011440 con 0x563add73b390 2024-09-21T04:13:13.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.427+0000 7f2d18760740 10 client.17394 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:13:13.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.427+0000 7f2d18760740 10 client.17394 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:13:13.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.427+0000 7f2d18760740 20 client.17394 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:13.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.427+0000 7f2d18760740 1 --2- 172.21.15.64:0/1786214117 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x563add75c070 0x563addb92040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:13.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.427+0000 7f2d18760740 4 client.17394 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:13:13.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.427+0000 7f2d18760740 1 -- 172.21.15.64:0/1786214117 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563adda778a0 con 0x563add75c070 2024-09-21T04:13:13.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.428+0000 7f2d0b7fe640 1 --2- 172.21.15.64:0/1786214117 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x563add75c070 0x563addb92040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:13.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.428+0000 7f2d0b7fe640 1 --2- 172.21.15.64:0/1786214117 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x563add75c070 0x563addb92040 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=62f2d9365f5b1959 server_cookie=b50b228107e2c32f in_seq=0 out_seq=0 2024-09-21T04:13:13.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.428+0000 7f2ceaffd640 10 client.17394 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:13:13.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.428+0000 7f2ceaffd640 1 -- 172.21.15.64:0/1786214117 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563adda778a0 con 0x563add75c070 2024-09-21T04:13:13.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.428+0000 7f2ceaffd640 10 client.17394 handle_command_reply: tid=0 2024-09-21T04:13:13.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.428+0000 7f2d18760740 1 client.17394 shutdown 2024-09-21T04:13:13.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.429+0000 7f2d18760740 1 -- 172.21.15.64:0/1786214117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563add73b390 msgr2=0x563addb265e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:13.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.429+0000 7f2d18760740 1 --2- 172.21.15.64:0/1786214117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563add73b390 0x563addb265e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f2d04002230 tx=0x7f2d04009c80 comp rx=0 tx=0).stop 2024-09-21T04:13:13.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.429+0000 7f2d18760740 1 -- 172.21.15.64:0/1786214117 shutdown_connections 2024-09-21T04:13:13.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.429+0000 7f2d18760740 1 --2- 172.21.15.64:0/1786214117 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x563add75c070 0x563addb92040 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.429+0000 7f2d18760740 1 --2- 172.21.15.64:0/1786214117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563add73b390 0x563addb265e0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.429+0000 7f2d18760740 1 --2- 172.21.15.64:0/1786214117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563add7331b0 0x563addb25f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.429+0000 7f2d18760740 1 --2- 172.21.15.64:0/1786214117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563add71a9f0 0x563addb28a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.429+0000 7f2d18760740 1 -- 172.21.15.64:0/1786214117 >> 172.21.15.64:0/1786214117 conn(0x563add750460 msgr2=0x563addb0eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:13.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.429+0000 7f2ceaffd640 0 client.17394 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:13:13.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.429+0000 7f2d18760740 1 -- 172.21.15.64:0/1786214117 shutdown_connections 2024-09-21T04:13:13.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.429+0000 7f2d18760740 1 -- 172.21.15.64:0/1786214117 wait complete. 2024-09-21T04:13:13.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.429+0000 7f2d18760740 20 client.17394 trim_cache size 0 max 16384 2024-09-21T04:13:13.437 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:13:13.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.429+0000 7f2d121ec640 1 -- 172.21.15.64:0/546740470 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2cf40b66b0 msgr2=0x7f2cf40b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:13.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.429+0000 7f2d121ec640 1 --2- 172.21.15.64:0/546740470 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2cf40b66b0 0x7f2cf40b8bc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x563addb01f40 tx=0x7f2cfc00a2d0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.437 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.429+0000 7f2d121ec640 1 -- 172.21.15.64:0/546740470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d0c1094a0 msgr2=0x7f2d0c19ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:13.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.429+0000 7f2d121ec640 1 --2- 172.21.15.64:0/546740470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d0c1094a0 0x7f2d0c19ab00 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f2d0002a930 tx=0x7f2d00004270 comp rx=0 tx=0).stop 2024-09-21T04:13:13.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.430+0000 7f2d121ec640 1 -- 172.21.15.64:0/546740470 shutdown_connections 2024-09-21T04:13:13.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.430+0000 7f2d121ec640 1 --2- 172.21.15.64:0/546740470 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f2cf40b66b0 0x7f2cf40b8bc0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.430+0000 7f2d121ec640 1 --2- 172.21.15.64:0/546740470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d0c109df0 0x7f2d0c19fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.430+0000 7f2d121ec640 1 --2- 172.21.15.64:0/546740470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d0c1094a0 0x7f2d0c19ab00 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.430+0000 7f2d121ec640 1 --2- 172.21.15.64:0/546740470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2d0c103b60 0x7f2d0c19a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:13.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.430+0000 7f2d121ec640 1 -- 172.21.15.64:0/546740470 >> 172.21.15.64:0/546740470 conn(0x7f2d0c07b850 msgr2=0x7f2d0c1075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:13.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.430+0000 7f2d121ec640 1 -- 172.21.15.64:0/546740470 shutdown_connections 2024-09-21T04:13:13.438 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:13.430+0000 7f2d121ec640 1 -- 172.21.15.64:0/546740470 wait complete. 2024-09-21T04:13:14.387 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:14 smithi064 ceph-mon[31180]: pgmap v604: 97 pgs: 13 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 57 active+clean; 763 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 204 B/s wr, 7 op/s 2024-09-21T04:13:14.422 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T04:13:14.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:14 smithi119 ceph-mon[34811]: pgmap v604: 97 pgs: 13 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 57 active+clean; 763 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 204 B/s wr, 7 op/s 2024-09-21T04:13:14.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:14 smithi131 ceph-mon[35193]: pgmap v604: 97 pgs: 13 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 57 active+clean; 763 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 204 B/s wr, 7 op/s 2024-09-21T04:13:14.688 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:13:15.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.207+0000 7fa5c3dde640 1 -- 172.21.15.64:0/2662076620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5bc109e20 msgr2=0x7fa5bc110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:15.210 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.207+0000 7fa5c3dde640 1 --2- 172.21.15.64:0/2662076620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5bc109e20 0x7fa5bc110650 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009ca0 tx=0x7fa5b402a8c0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.207+0000 7fa5c3dde640 1 -- 172.21.15.64:0/2662076620 shutdown_connections 2024-09-21T04:13:15.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.207+0000 7fa5c3dde640 1 --2- 172.21.15.64:0/2662076620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5bc109e20 0x7fa5bc110650 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.207+0000 7fa5c3dde640 1 --2- 172.21.15.64:0/2662076620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5bc1094d0 0x7fa5bc1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.207+0000 7fa5c3dde640 1 --2- 172.21.15.64:0/2662076620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5bc103b90 0x7fa5bc103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.207+0000 7fa5c3dde640 1 -- 172.21.15.64:0/2662076620 >> 172.21.15.64:0/2662076620 conn(0x7fa5bc0ff600 msgr2=0x7fa5bc101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:15.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.207+0000 7fa5c3dde640 1 -- 172.21.15.64:0/2662076620 shutdown_connections 2024-09-21T04:13:15.211 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.208+0000 7fa5c3dde640 1 -- 172.21.15.64:0/2662076620 wait complete. 2024-09-21T04:13:15.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.208+0000 7fa5c3dde640 1 Processor -- start 2024-09-21T04:13:15.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.208+0000 7fa5c3dde640 1 -- start start 2024-09-21T04:13:15.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.208+0000 7fa5c3dde640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5bc103b90 0x7fa5bc19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:15.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.209+0000 7fa5c3dde640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5bc1094d0 0x7fa5bc19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:15.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.209+0000 7fa5c3dde640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5bc109e20 0x7fa5bc19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:15.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.209+0000 7fa5c3dde640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc113ae0 con 0x7fa5bc109e20 2024-09-21T04:13:15.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.209+0000 7fa5c3dde640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc113960 con 0x7fa5bc1094d0 2024-09-21T04:13:15.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.209+0000 7fa5c3dde640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc113c60 con 0x7fa5bc103b90 2024-09-21T04:13:15.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.209+0000 7fa5c237b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5bc109e20 0x7fa5bc19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:15.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.209+0000 7fa5c237b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5bc109e20 0x7fa5bc19fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48474/0 (socket says 172.21.15.64:48474) 2024-09-21T04:13:15.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.209+0000 7fa5c237b640 1 -- 172.21.15.64:0/1303333193 learned_addr learned my addr 172.21.15.64:0/1303333193 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:15.213 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.209+0000 7fa5c1379640 1 --2- 172.21.15.64:0/1303333193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5bc1094d0 0x7fa5bc19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:15.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.209+0000 7fa5c1b7a640 1 --2- 172.21.15.64:0/1303333193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5bc103b90 0x7fa5bc19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:15.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.209+0000 7fa5c237b640 1 -- 172.21.15.64:0/1303333193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5bc103b90 msgr2=0x7fa5bc19a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:15.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.209+0000 7fa5c237b640 1 --2- 172.21.15.64:0/1303333193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5bc103b90 0x7fa5bc19a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.209+0000 7fa5c237b640 1 -- 172.21.15.64:0/1303333193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5bc1094d0 msgr2=0x7fa5bc19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:15.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.209+0000 7fa5c237b640 1 --2- 172.21.15.64:0/1303333193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5bc1094d0 0x7fa5bc19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.209+0000 7fa5c237b640 1 -- 172.21.15.64:0/1303333193 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5bc1a02c0 con 0x7fa5bc109e20 2024-09-21T04:13:15.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.209+0000 7fa5c237b640 1 --2- 172.21.15.64:0/1303333193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5bc109e20 0x7fa5bc19fc50 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4005cc0 tx=0x7fa5b402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:15.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.209+0000 7fa5b2ffd640 1 -- 172.21.15.64:0/1303333193 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4003bf0 con 0x7fa5bc109e20 2024-09-21T04:13:15.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.210+0000 7fa5b2ffd640 1 -- 172.21.15.64:0/1303333193 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5b4003d90 con 0x7fa5bc109e20 2024-09-21T04:13:15.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.210+0000 7fa5c3dde640 1 -- 172.21.15.64:0/1303333193 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5bc1a0550 con 0x7fa5bc109e20 2024-09-21T04:13:15.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.210+0000 7fa5b2ffd640 1 -- 172.21.15.64:0/1303333193 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b402fb20 con 0x7fa5bc109e20 2024-09-21T04:13:15.214 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.210+0000 7fa5c3dde640 1 -- 172.21.15.64:0/1303333193 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5bc1a0a90 con 0x7fa5bc109e20 2024-09-21T04:13:15.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.212+0000 7fa5ca352740 1 Processor -- start 2024-09-21T04:13:15.215 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.213+0000 7fa5b2ffd640 1 -- 172.21.15.64:0/1303333193 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fa5b402fcc0 con 0x7fa5bc109e20 2024-09-21T04:13:15.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.213+0000 7fa5ca352740 1 -- start start 2024-09-21T04:13:15.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.213+0000 7fa5ca352740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e8548070 0x5611e88ef2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:15.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.213+0000 7fa5ca352740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e851f1b0 0x5611e88ef800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:15.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.213+0000 7fa5ca352740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611e8527390 0x5611e88efd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:15.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.213+0000 7fa5ca352740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5611e8865ea0 con 0x5611e851f1b0 2024-09-21T04:13:15.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.213+0000 7fa5ca352740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5611e88552d0 con 0x5611e8548070 2024-09-21T04:13:15.216 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.213+0000 7fa5ca352740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5611e88590e0 con 0x5611e8527390 2024-09-21T04:13:15.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.213+0000 7fa5c1b7a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e851f1b0 0x5611e88ef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:15.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.213+0000 7fa5c1b7a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e851f1b0 0x5611e88ef800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48488/0 (socket says 172.21.15.64:48488) 2024-09-21T04:13:15.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.213+0000 7fa5c1b7a640 1 -- 172.21.15.64:0/4109839459 learned_addr learned my addr 172.21.15.64:0/4109839459 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:15.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.214+0000 7fa5c1379640 1 --2- 172.21.15.64:0/4109839459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611e8527390 0x5611e88efd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:15.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.214+0000 7fa5b2ffd640 1 --2- 172.21.15.64:0/1303333193 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa5a80b66b0 0x7fa5a80b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:15.217 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.214+0000 7fa5c1379640 1 --2- 172.21.15.64:0/4109839459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e8548070 0x5611e88ef2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:15.218 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.214+0000 7fa5c1b7a640 1 -- 172.21.15.64:0/4109839459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611e8527390 msgr2=0x5611e88efd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:15.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.214+0000 7fa5c1b7a640 1 --2- 172.21.15.64:0/4109839459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611e8527390 0x5611e88efd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.214+0000 7fa5c1b7a640 1 -- 172.21.15.64:0/4109839459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e8548070 msgr2=0x5611e88ef2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:15.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.214+0000 7fa5c1b7a640 1 --2- 172.21.15.64:0/4109839459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e8548070 0x5611e88ef2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.214+0000 7fa5c1b7a640 1 -- 172.21.15.64:0/4109839459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611e85b3650 con 0x5611e851f1b0 2024-09-21T04:13:15.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.214+0000 7fa5c237b640 1 --2- 172.21.15.64:0/1303333193 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa5a80b66b0 0x7fa5a80b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:15.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.214+0000 7fa5b2ffd640 1 -- 172.21.15.64:0/1303333193 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8659+0+0 (secure 0 0 0) 0x7fa5b4108c40 con 0x7fa5bc109e20 2024-09-21T04:13:15.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.214+0000 7fa5c1b7a640 1 --2- 172.21.15.64:0/4109839459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e851f1b0 0x5611e88ef800 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800bd10 tx=0x7fa5b80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac733bb7d0e12945 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:15.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.214+0000 7fa5a7fff640 1 -- 172.21.15.64:0/4109839459 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8004030 con 0x5611e851f1b0 2024-09-21T04:13:15.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.214+0000 7fa5a7fff640 1 -- 172.21.15.64:0/4109839459 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5b80035f0 con 0x5611e851f1b0 2024-09-21T04:13:15.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.214+0000 7fa5a7fff640 1 -- 172.21.15.64:0/4109839459 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8011680 con 0x5611e851f1b0 2024-09-21T04:13:15.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.215+0000 7fa5ca352740 1 -- 172.21.15.64:0/4109839459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e851f1b0 msgr2=0x5611e88ef800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:15.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.215+0000 7fa5ca352740 1 --2- 172.21.15.64:0/4109839459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e851f1b0 0x5611e88ef800 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800bd10 tx=0x7fa5b80107a0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.215+0000 7fa5c237b640 1 --2- 172.21.15.64:0/1303333193 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa5a80b66b0 0x7fa5a80b8bc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa5b41063b0 tx=0x7fa5b409b7c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:15.219 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.215+0000 7fa5ca352740 1 -- 172.21.15.64:0/4109839459 shutdown_connections 2024-09-21T04:13:15.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.215+0000 7fa5ca352740 1 --2- 172.21.15.64:0/4109839459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611e8527390 0x5611e88efd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.215+0000 7fa5ca352740 1 --2- 172.21.15.64:0/4109839459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e851f1b0 0x5611e88ef800 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.215+0000 7fa5ca352740 1 --2- 172.21.15.64:0/4109839459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e8548070 0x5611e88ef2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.215+0000 7fa5ca352740 1 -- 172.21.15.64:0/4109839459 >> 172.21.15.64:0/4109839459 conn(0x5611e853c460 msgr2=0x5611e88ed1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:15.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.216+0000 7fa5ca352740 1 -- 172.21.15.64:0/4109839459 shutdown_connections 2024-09-21T04:13:15.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.216+0000 7fa5ca352740 1 -- 172.21.15.64:0/4109839459 wait complete. 2024-09-21T04:13:15.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.216+0000 7fa5ca352740 1 -- start start 2024-09-21T04:13:15.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.216+0000 7fa5ca352740 1 Processor -- start 2024-09-21T04:13:15.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.217+0000 7fa5ca352740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611e851f1b0 0x5611e88fed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:15.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.217+0000 7fa5ca352740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e8527390 0x5611e88ff260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:15.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.217+0000 7fa5ca352740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e8548070 0x5611e88ff7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:15.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.217+0000 7fa5ca352740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5611e86b6700 con 0x5611e8548070 2024-09-21T04:13:15.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.217+0000 7fa5ca352740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5611e87f0160 con 0x5611e8527390 2024-09-21T04:13:15.223 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.217+0000 7fa5ca352740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5611e8901e00 con 0x5611e851f1b0 2024-09-21T04:13:15.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.217+0000 7fa5ca352740 10 client.0 _ll_register_callbacks cb 0x5611e88099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:15.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.217+0000 7fa5c1379640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e8548070 0x5611e88ff7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:15.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.217+0000 7fa5c1379640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e8548070 0x5611e88ff7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48492/0 (socket says 172.21.15.64:48492) 2024-09-21T04:13:15.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.217+0000 7fa5c1379640 1 -- 172.21.15.64:0/1117318968 learned_addr learned my addr 172.21.15.64:0/1117318968 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:15.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.217+0000 7fa5c1379640 1 --2- 172.21.15.64:0/1117318968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611e851f1b0 0x5611e88fed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:15.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.217+0000 7fa5c1b7a640 1 --2- 172.21.15.64:0/1117318968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e8527390 0x5611e88ff260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:15.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.217+0000 7fa5c1379640 1 -- 172.21.15.64:0/1117318968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611e851f1b0 msgr2=0x5611e88fed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:15.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.217+0000 7fa5c1379640 1 --2- 172.21.15.64:0/1117318968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611e851f1b0 0x5611e88fed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.217+0000 7fa5c1379640 1 -- 172.21.15.64:0/1117318968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e8527390 msgr2=0x5611e88ff260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:15.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.217+0000 7fa5c1379640 1 --2- 172.21.15.64:0/1117318968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e8527390 0x5611e88ff260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.217+0000 7fa5c1379640 1 -- 172.21.15.64:0/1117318968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611e881dc90 con 0x5611e8548070 2024-09-21T04:13:15.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.217+0000 7fa5c1379640 1 --2- 172.21.15.64:0/1117318968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e8548070 0x5611e88ff7c0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00ada0 tx=0x7fa5ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad5616dd28a8582 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:15.224 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.217+0000 7fa5a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:13:15.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.217+0000 7fa5a57fa640 1 -- 172.21.15.64:0/1117318968 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac012260 con 0x5611e8548070 2024-09-21T04:13:15.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.218+0000 7fa5a57fa640 1 -- 172.21.15.64:0/1117318968 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8659+0+0 (secure 0 0 0) 0x7fa5ac012400 con 0x5611e8548070 2024-09-21T04:13:15.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.218+0000 7fa5ca352740 1 -- 172.21.15.64:0/1117318968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5b80035f0 con 0x5611e8548070 2024-09-21T04:13:15.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.218+0000 7fa5a57fa640 1 -- 172.21.15.64:0/1117318968 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa5ac005020 con 0x5611e8548070 2024-09-21T04:13:15.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.218+0000 7fa5ca352740 10 client.17406 fetch_fsmap learned FSMap version 40 2024-09-21T04:13:15.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.218+0000 7fa5ca352740 1 -- 172.21.15.64:0/1117318968 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5611e890daf0 con 0x5611e8548070 2024-09-21T04:13:15.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.218+0000 7fa5a57fa640 1 -- 172.21.15.64:0/1117318968 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5b80035f0 con 0x5611e8548070 2024-09-21T04:13:15.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.218+0000 7fa5a57fa640 1 -- 172.21.15.64:0/1117318968 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7fa5ac004cd0 con 0x5611e8548070 2024-09-21T04:13:15.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.218+0000 7fa5ca352740 10 client.17406 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:13:15.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.218+0000 7fa5ca352740 10 client.17406 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:13:15.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.218+0000 7fa5ca352740 20 client.17406 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:15.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.218+0000 7fa5ca352740 1 --2- 172.21.15.64:0/1117318968 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5611e85069f0 0x5611e8911120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:15.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.218+0000 7fa5ca352740 4 client.17406 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:13:15.225 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.218+0000 7fa5ca352740 1 -- 172.21.15.64:0/1117318968 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611e8867df0 con 0x5611e85069f0 2024-09-21T04:13:15.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.219+0000 7fa5c1b7a640 1 --2- 172.21.15.64:0/1117318968 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5611e85069f0 0x5611e8911120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:15.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.219+0000 7fa5c1b7a640 1 --2- 172.21.15.64:0/1117318968 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5611e85069f0 0x5611e8911120 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=33ace048e6bee868 server_cookie=f3852ed00b77c40d in_seq=0 out_seq=0 2024-09-21T04:13:15.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.219+0000 7fa5a57fa640 10 client.17406 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:13:15.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.220+0000 7fa5a57fa640 1 -- 172.21.15.64:0/1117318968 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5611e8867df0 con 0x5611e85069f0 2024-09-21T04:13:15.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.220+0000 7fa5a57fa640 10 client.17406 handle_command_reply: tid=0 2024-09-21T04:13:15.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.220+0000 7fa5ca352740 1 client.17406 shutdown 2024-09-21T04:13:15.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.220+0000 7fa5ca352740 1 -- 172.21.15.64:0/1117318968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e8548070 msgr2=0x5611e88ff7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:15.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.220+0000 7fa5ca352740 1 --2- 172.21.15.64:0/1117318968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e8548070 0x5611e88ff7c0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00ada0 tx=0x7fa5ac004730 comp rx=0 tx=0).stop 2024-09-21T04:13:15.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.220+0000 7fa5ca352740 1 -- 172.21.15.64:0/1117318968 shutdown_connections 2024-09-21T04:13:15.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.220+0000 7fa5ca352740 1 --2- 172.21.15.64:0/1117318968 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5611e85069f0 0x5611e8911120 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.220+0000 7fa5ca352740 1 --2- 172.21.15.64:0/1117318968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e8548070 0x5611e88ff7c0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.220+0000 7fa5ca352740 1 --2- 172.21.15.64:0/1117318968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e8527390 0x5611e88ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.226 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.220+0000 7fa5ca352740 1 --2- 172.21.15.64:0/1117318968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611e851f1b0 0x5611e88fed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.220+0000 7fa5a57fa640 0 client.17406 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:13:15.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.220+0000 7fa5ca352740 1 -- 172.21.15.64:0/1117318968 >> 172.21.15.64:0/1117318968 conn(0x5611e853c460 msgr2=0x5611e88eb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:15.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.220+0000 7fa5ca352740 1 -- 172.21.15.64:0/1117318968 shutdown_connections 2024-09-21T04:13:15.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.220+0000 7fa5ca352740 1 -- 172.21.15.64:0/1117318968 wait complete. 2024-09-21T04:13:15.227 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.220+0000 7fa5ca352740 20 client.17406 trim_cache size 0 max 16384 2024-09-21T04:13:15.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.233+0000 7fa5ca352740 1 Processor -- start 2024-09-21T04:13:15.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.233+0000 7fa5ca352740 1 -- start start 2024-09-21T04:13:15.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.234+0000 7fa5ca352740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e851f1b0 0x5611e88ecb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:15.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.234+0000 7fa5ca352740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611e8527390 0x5611e88ed050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:15.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.234+0000 7fa5ca352740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e8548070 0x5611e88fbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:15.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.234+0000 7fa5ca352740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc1b00b0 con 0x5611e851f1b0 2024-09-21T04:13:15.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.234+0000 7fa5ca352740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc1ad8c0 con 0x5611e8548070 2024-09-21T04:13:15.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.234+0000 7fa5ca352740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc1ab730 con 0x5611e8527390 2024-09-21T04:13:15.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.234+0000 7fa5c1379640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e851f1b0 0x5611e88ecb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:15.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.234+0000 7fa5c1379640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e851f1b0 0x5611e88ecb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48498/0 (socket says 172.21.15.64:48498) 2024-09-21T04:13:15.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.234+0000 7fa5c1379640 1 -- 172.21.15.64:0/545909239 learned_addr learned my addr 172.21.15.64:0/545909239 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:15.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.234+0000 7fa5c1379640 1 --2- 172.21.15.64:0/545909239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e8548070 0x5611e88fbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:15.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.234+0000 7fa5c1b7a640 1 --2- 172.21.15.64:0/545909239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611e8527390 0x5611e88ed050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:15.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.234+0000 7fa5c1379640 1 -- 172.21.15.64:0/545909239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611e8527390 msgr2=0x5611e88ed050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:15.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.234+0000 7fa5c1379640 1 --2- 172.21.15.64:0/545909239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611e8527390 0x5611e88ed050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.234+0000 7fa5c1379640 1 -- 172.21.15.64:0/545909239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e8548070 msgr2=0x5611e88fbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:15.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.234+0000 7fa5c1379640 1 --2- 172.21.15.64:0/545909239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e8548070 0x5611e88fbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.234+0000 7fa5c1379640 1 -- 172.21.15.64:0/545909239 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611e88f1bd0 con 0x5611e851f1b0 2024-09-21T04:13:15.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.234+0000 7fa5c1b7a640 1 --2- 172.21.15.64:0/545909239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611e8527390 0x5611e88ed050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:13:15.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.234+0000 7fa5c1379640 1 --2- 172.21.15.64:0/545909239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e851f1b0 0x5611e88ecb10 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac004700 tx=0x7fa5ac00af20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ae5d2f8c1690fe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:15.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.235+0000 7fa5a77fe640 1 -- 172.21.15.64:0/545909239 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac012460 con 0x5611e851f1b0 2024-09-21T04:13:15.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.235+0000 7fa5a77fe640 1 -- 172.21.15.64:0/545909239 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5ac005670 con 0x5611e851f1b0 2024-09-21T04:13:15.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.235+0000 7fa5a77fe640 1 -- 172.21.15.64:0/545909239 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac011c50 con 0x5611e851f1b0 2024-09-21T04:13:15.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.235+0000 7fa5ca352740 1 -- 172.21.15.64:0/545909239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e851f1b0 msgr2=0x5611e88ecb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:15.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.235+0000 7fa5ca352740 1 --2- 172.21.15.64:0/545909239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e851f1b0 0x5611e88ecb10 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac004700 tx=0x7fa5ac00af20 comp rx=0 tx=0).stop 2024-09-21T04:13:15.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.235+0000 7fa5ca352740 1 -- 172.21.15.64:0/545909239 shutdown_connections 2024-09-21T04:13:15.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.235+0000 7fa5ca352740 1 --2- 172.21.15.64:0/545909239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e8548070 0x5611e88fbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.243 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.235+0000 7fa5ca352740 1 --2- 172.21.15.64:0/545909239 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611e8527390 0x5611e88ed050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.235+0000 7fa5ca352740 1 --2- 172.21.15.64:0/545909239 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e851f1b0 0x5611e88ecb10 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.235+0000 7fa5ca352740 1 -- 172.21.15.64:0/545909239 >> 172.21.15.64:0/545909239 conn(0x5611e853c460 msgr2=0x5611e88fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:15.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.236+0000 7fa5ca352740 1 -- 172.21.15.64:0/545909239 shutdown_connections 2024-09-21T04:13:15.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.236+0000 7fa5ca352740 1 -- 172.21.15.64:0/545909239 wait complete. 2024-09-21T04:13:15.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.236+0000 7fa5ca352740 1 -- start start 2024-09-21T04:13:15.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.236+0000 7fa5ca352740 1 Processor -- start 2024-09-21T04:13:15.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.237+0000 7fa5ca352740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e851f1b0 0x5611e8902fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:15.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.237+0000 7fa5ca352740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e8527390 0x5611e8903520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:15.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.237+0000 7fa5ca352740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611e8548070 0x5611e8903a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:15.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.237+0000 7fa5ca352740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc1abd80 con 0x5611e851f1b0 2024-09-21T04:13:15.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.237+0000 7fa5ca352740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc1ad1f0 con 0x5611e8527390 2024-09-21T04:13:15.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.237+0000 7fa5ca352740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc1ac450 con 0x5611e8548070 2024-09-21T04:13:15.244 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.237+0000 7fa5ca352740 10 client.0 _ll_register_callbacks cb 0x7fa5ac00acd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:15.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.237+0000 7fa5c1379640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e851f1b0 0x5611e8902fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:15.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.237+0000 7fa5c1379640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e851f1b0 0x5611e8902fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48514/0 (socket says 172.21.15.64:48514) 2024-09-21T04:13:15.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.237+0000 7fa5c1379640 1 -- 172.21.15.64:0/4107341127 learned_addr learned my addr 172.21.15.64:0/4107341127 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:15.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.237+0000 7fa5c1b7a640 1 --2- 172.21.15.64:0/4107341127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e8527390 0x5611e8903520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:15.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.237+0000 7fa5c1379640 1 --2- 172.21.15.64:0/4107341127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611e8548070 0x5611e8903a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:15.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.237+0000 7fa5c1379640 1 -- 172.21.15.64:0/4107341127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611e8548070 msgr2=0x5611e8903a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:15.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.237+0000 7fa5c1379640 1 --2- 172.21.15.64:0/4107341127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611e8548070 0x5611e8903a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.237+0000 7fa5c1379640 1 -- 172.21.15.64:0/4107341127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e8527390 msgr2=0x5611e8903520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:15.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.237+0000 7fa5c1379640 1 --2- 172.21.15.64:0/4107341127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e8527390 0x5611e8903520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.237+0000 7fa5c1379640 1 -- 172.21.15.64:0/4107341127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611e897ab20 con 0x5611e851f1b0 2024-09-21T04:13:15.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.237+0000 7fa5c1b7a640 1 --2- 172.21.15.64:0/4107341127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e8527390 0x5611e8903520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:13:15.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.237+0000 7fa5c1379640 1 --2- 172.21.15.64:0/4107341127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e851f1b0 0x5611e8902fe0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00ac20 tx=0x7fa5ac00af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103e42090dca03ea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:15.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.237+0000 7fa5a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:13:15.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.238+0000 7fa5a57fa640 1 -- 172.21.15.64:0/4107341127 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac011910 con 0x5611e851f1b0 2024-09-21T04:13:15.245 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.238+0000 7fa5a57fa640 1 -- 172.21.15.64:0/4107341127 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8659+0+0 (secure 0 0 0) 0x7fa5ac011ab0 con 0x5611e851f1b0 2024-09-21T04:13:15.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.238+0000 7fa5ca352740 1 -- 172.21.15.64:0/4107341127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5ac005670 con 0x5611e851f1b0 2024-09-21T04:13:15.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.238+0000 7fa5a57fa640 1 -- 172.21.15.64:0/4107341127 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa5ac011cb0 con 0x5611e851f1b0 2024-09-21T04:13:15.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.238+0000 7fa5ca352740 10 client.17412 fetch_fsmap learned FSMap version 40 2024-09-21T04:13:15.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.238+0000 7fa5ca352740 1 -- 172.21.15.64:0/4107341127 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5611e897bc50 con 0x5611e851f1b0 2024-09-21T04:13:15.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.238+0000 7fa5a57fa640 1 -- 172.21.15.64:0/4107341127 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5ac005670 con 0x5611e851f1b0 2024-09-21T04:13:15.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.238+0000 7fa5a57fa640 1 -- 172.21.15.64:0/4107341127 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7fa5ac044050 con 0x5611e851f1b0 2024-09-21T04:13:15.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.238+0000 7fa5ca352740 10 client.17412 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:13:15.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.238+0000 7fa5ca352740 10 client.17412 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:13:15.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.238+0000 7fa5ca352740 20 client.17412 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:15.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.238+0000 7fa5ca352740 1 --2- 172.21.15.64:0/4107341127 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5611e84dbda0 0x5611e897e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:15.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.238+0000 7fa5ca352740 4 client.17412 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:13:15.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.238+0000 7fa5ca352740 1 -- 172.21.15.64:0/4107341127 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5611e88f1920 con 0x5611e84dbda0 2024-09-21T04:13:15.246 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.239+0000 7fa5c1b7a640 1 --2- 172.21.15.64:0/4107341127 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5611e84dbda0 0x5611e897e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:15.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.239+0000 7fa5c1b7a640 1 --2- 172.21.15.64:0/4107341127 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5611e84dbda0 0x5611e897e360 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d0b4fee6f2d2a1 server_cookie=29fbf95a93701206 in_seq=0 out_seq=0 2024-09-21T04:13:15.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.239+0000 7fa5a57fa640 10 client.17412 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:13:15.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.239+0000 7fa5a57fa640 1 -- 172.21.15.64:0/4107341127 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5611e88f1920 con 0x5611e84dbda0 2024-09-21T04:13:15.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.239+0000 7fa5a57fa640 10 client.17412 handle_command_reply: tid=0 2024-09-21T04:13:15.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.239+0000 7fa5ca352740 1 client.17412 shutdown 2024-09-21T04:13:15.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.239+0000 7fa5ca352740 1 -- 172.21.15.64:0/4107341127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e851f1b0 msgr2=0x5611e8902fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:15.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.239+0000 7fa5ca352740 1 --2- 172.21.15.64:0/4107341127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e851f1b0 0x5611e8902fe0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00ac20 tx=0x7fa5ac00af90 comp rx=0 tx=0).stop 2024-09-21T04:13:15.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.240+0000 7fa5ca352740 1 -- 172.21.15.64:0/4107341127 shutdown_connections 2024-09-21T04:13:15.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.240+0000 7fa5ca352740 1 --2- 172.21.15.64:0/4107341127 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x5611e84dbda0 0x5611e897e360 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.240+0000 7fa5ca352740 1 --2- 172.21.15.64:0/4107341127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5611e8548070 0x5611e8903a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.240+0000 7fa5ca352740 1 --2- 172.21.15.64:0/4107341127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5611e8527390 0x5611e8903520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.240+0000 7fa5ca352740 1 --2- 172.21.15.64:0/4107341127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611e851f1b0 0x5611e8902fe0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.240+0000 7fa5a57fa640 0 client.17412 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:13:15.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.240+0000 7fa5ca352740 1 -- 172.21.15.64:0/4107341127 >> 172.21.15.64:0/4107341127 conn(0x5611e853c460 msgr2=0x5611e88f02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:15.247 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.240+0000 7fa5ca352740 1 -- 172.21.15.64:0/4107341127 shutdown_connections 2024-09-21T04:13:15.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.240+0000 7fa5ca352740 1 -- 172.21.15.64:0/4107341127 wait complete. 2024-09-21T04:13:15.248 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.240+0000 7fa5ca352740 20 client.17412 trim_cache size 0 max 16384 2024-09-21T04:13:15.248 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:13:15.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.240+0000 7fa5c3dde640 1 -- 172.21.15.64:0/1303333193 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa5a80b66b0 msgr2=0x7fa5a80b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:15.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.240+0000 7fa5c3dde640 1 --2- 172.21.15.64:0/1303333193 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa5a80b66b0 0x7fa5a80b8bc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa5b41063b0 tx=0x7fa5b409b7c0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.240+0000 7fa5c3dde640 1 -- 172.21.15.64:0/1303333193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5bc109e20 msgr2=0x7fa5bc19fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:15.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.240+0000 7fa5c3dde640 1 --2- 172.21.15.64:0/1303333193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5bc109e20 0x7fa5bc19fc50 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4005cc0 tx=0x7fa5b402f380 comp rx=0 tx=0).stop 2024-09-21T04:13:15.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.240+0000 7fa5c3dde640 1 -- 172.21.15.64:0/1303333193 shutdown_connections 2024-09-21T04:13:15.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.240+0000 7fa5c3dde640 1 --2- 172.21.15.64:0/1303333193 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa5a80b66b0 0x7fa5a80b8bc0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.240+0000 7fa5c3dde640 1 --2- 172.21.15.64:0/1303333193 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa5bc109e20 0x7fa5bc19fc50 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.240+0000 7fa5c3dde640 1 --2- 172.21.15.64:0/1303333193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5bc1094d0 0x7fa5bc19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.240+0000 7fa5c3dde640 1 --2- 172.21.15.64:0/1303333193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5bc103b90 0x7fa5bc19a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:15.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.241+0000 7fa5c3dde640 1 -- 172.21.15.64:0/1303333193 >> 172.21.15.64:0/1303333193 conn(0x7fa5bc0ff600 msgr2=0x7fa5bc106aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:15.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.241+0000 7fa5c3dde640 1 -- 172.21.15.64:0/1303333193 shutdown_connections 2024-09-21T04:13:15.249 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:15.241+0000 7fa5c3dde640 1 -- 172.21.15.64:0/1303333193 wait complete. 2024-09-21T04:13:15.950 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T04:13:16.207 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:13:16.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:16 smithi064 ceph-mon[31180]: pgmap v605: 97 pgs: 13 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 57 active+clean; 763 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 204 B/s wr, 7 op/s 2024-09-21T04:13:16.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:16 smithi131 ceph-mon[35193]: pgmap v605: 97 pgs: 13 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 57 active+clean; 763 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 204 B/s wr, 7 op/s 2024-09-21T04:13:16.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:16 smithi119 ceph-mon[34811]: pgmap v605: 97 pgs: 13 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 57 active+clean; 763 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 204 B/s wr, 7 op/s 2024-09-21T04:13:16.875 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.869+0000 7ff69f8d8640 1 -- 172.21.15.64:0/837929841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff698104860 msgr2=0x7ff698104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:16.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.869+0000 7ff69f8d8640 1 --2- 172.21.15.64:0/837929841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff698104860 0x7ff698104c70 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff694009900 tx=0x7ff69402a410 comp rx=0 tx=0).stop 2024-09-21T04:13:16.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.869+0000 7ff69f8d8640 1 -- 172.21.15.64:0/837929841 shutdown_connections 2024-09-21T04:13:16.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.869+0000 7ff69f8d8640 1 --2- 172.21.15.64:0/837929841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff69810de10 0x7ff6981102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.869+0000 7ff69f8d8640 1 --2- 172.21.15.64:0/837929841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff698105290 0x7ff69810d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.869+0000 7ff69f8d8640 1 --2- 172.21.15.64:0/837929841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff698104860 0x7ff698104c70 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.869+0000 7ff69f8d8640 1 -- 172.21.15.64:0/837929841 >> 172.21.15.64:0/837929841 conn(0x7ff698079730 msgr2=0x7ff698079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:16.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.869+0000 7ff69f8d8640 1 -- 172.21.15.64:0/837929841 shutdown_connections 2024-09-21T04:13:16.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.869+0000 7ff69f8d8640 1 -- 172.21.15.64:0/837929841 wait complete. 2024-09-21T04:13:16.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.870+0000 7ff69f8d8640 1 Processor -- start 2024-09-21T04:13:16.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.870+0000 7ff69f8d8640 1 -- start start 2024-09-21T04:13:16.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.870+0000 7ff69f8d8640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff698104860 0x7ff698102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:16.876 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.870+0000 7ff69f8d8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff698105290 0x7ff698102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:16.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.870+0000 7ff69f8d8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff69810de10 0x7ff6981031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:16.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.870+0000 7ff69f8d8640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff698113a60 con 0x7ff698104860 2024-09-21T04:13:16.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.870+0000 7ff69f8d8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff6981138e0 con 0x7ff69810de10 2024-09-21T04:13:16.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.870+0000 7ff69f8d8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff698113be0 con 0x7ff698105290 2024-09-21T04:13:16.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.870+0000 7ff69d674640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff698104860 0x7ff698102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:16.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.870+0000 7ff69d674640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff698104860 0x7ff698102770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48532/0 (socket says 172.21.15.64:48532) 2024-09-21T04:13:16.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.870+0000 7ff69d674640 1 -- 172.21.15.64:0/1948118693 learned_addr learned my addr 172.21.15.64:0/1948118693 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:16.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.870+0000 7ff69ce73640 1 --2- 172.21.15.64:0/1948118693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff698105290 0x7ff698102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:16.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.871+0000 7ff69d674640 1 -- 172.21.15.64:0/1948118693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff698105290 msgr2=0x7ff698102cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:16.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.871+0000 7ff69d674640 1 --2- 172.21.15.64:0/1948118693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff698105290 0x7ff698102cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.871+0000 7ff69d674640 1 -- 172.21.15.64:0/1948118693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff69810de10 msgr2=0x7ff6981031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:16.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.871+0000 7ff69de75640 1 --2- 172.21.15.64:0/1948118693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff69810de10 0x7ff6981031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:16.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.871+0000 7ff69d674640 1 --2- 172.21.15.64:0/1948118693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff69810de10 0x7ff6981031f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.871+0000 7ff69d674640 1 -- 172.21.15.64:0/1948118693 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6981a8730 con 0x7ff698104860 2024-09-21T04:13:16.877 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.871+0000 7ff69ce73640 1 --2- 172.21.15.64:0/1948118693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff698105290 0x7ff698102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:13:16.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.871+0000 7ff69d674640 1 --2- 172.21.15.64:0/1948118693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff698104860 0x7ff698102770 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff694005940 tx=0x7ff694004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:16.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.871+0000 7ff68a7fc640 1 -- 172.21.15.64:0/1948118693 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69403f030 con 0x7ff698104860 2024-09-21T04:13:16.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.871+0000 7ff68a7fc640 1 -- 172.21.15.64:0/1948118693 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff694003aa0 con 0x7ff698104860 2024-09-21T04:13:16.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.871+0000 7ff69f8d8640 1 -- 172.21.15.64:0/1948118693 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6981a89c0 con 0x7ff698104860 2024-09-21T04:13:16.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.871+0000 7ff68a7fc640 1 -- 172.21.15.64:0/1948118693 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff694035070 con 0x7ff698104860 2024-09-21T04:13:16.878 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.871+0000 7ff69f8d8640 1 -- 172.21.15.64:0/1948118693 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6981a8ef0 con 0x7ff698104860 2024-09-21T04:13:16.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.874+0000 7ff6a5e4c740 1 Processor -- start 2024-09-21T04:13:16.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.874+0000 7ff68a7fc640 1 -- 172.21.15.64:0/1948118693 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7ff694042030 con 0x7ff698104860 2024-09-21T04:13:16.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.874+0000 7ff6a5e4c740 1 -- start start 2024-09-21T04:13:16.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.874+0000 7ff6a5e4c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556fb6813070 0x556fb6bba6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:16.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.874+0000 7ff6a5e4c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb67ea1b0 0x556fb6bbabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:16.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.874+0000 7ff6a5e4c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fb67f2390 0x556fb6bbb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:16.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.874+0000 7ff6a5e4c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556fb6b30ea0 con 0x556fb67ea1b0 2024-09-21T04:13:16.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.874+0000 7ff6a5e4c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556fb6b202d0 con 0x556fb67f2390 2024-09-21T04:13:16.881 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.874+0000 7ff6a5e4c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556fb6b240e0 con 0x556fb6813070 2024-09-21T04:13:16.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.875+0000 7ff69d674640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb67ea1b0 0x556fb6bbabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:16.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.875+0000 7ff69d674640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb67ea1b0 0x556fb6bbabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48534/0 (socket says 172.21.15.64:48534) 2024-09-21T04:13:16.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.875+0000 7ff69d674640 1 -- 172.21.15.64:0/2109048535 learned_addr learned my addr 172.21.15.64:0/2109048535 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:16.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.875+0000 7ff69de75640 1 --2- 172.21.15.64:0/2109048535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556fb6813070 0x556fb6bba6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:16.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.875+0000 7ff69ce73640 1 --2- 172.21.15.64:0/2109048535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fb67f2390 0x556fb6bbb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:16.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.875+0000 7ff69d674640 1 -- 172.21.15.64:0/2109048535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556fb6813070 msgr2=0x556fb6bba6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:16.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.875+0000 7ff69d674640 1 --2- 172.21.15.64:0/2109048535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556fb6813070 0x556fb6bba6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.875+0000 7ff69d674640 1 -- 172.21.15.64:0/2109048535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fb67f2390 msgr2=0x556fb6bbb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:16.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.875+0000 7ff69d674640 1 --2- 172.21.15.64:0/2109048535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fb67f2390 0x556fb6bbb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.875+0000 7ff69d674640 1 -- 172.21.15.64:0/2109048535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fb687e650 con 0x556fb67ea1b0 2024-09-21T04:13:16.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.875+0000 7ff69de75640 1 --2- 172.21.15.64:0/2109048535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556fb6813070 0x556fb6bba6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:13:16.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.875+0000 7ff69ce73640 1 --2- 172.21.15.64:0/2109048535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fb67f2390 0x556fb6bbb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:13:16.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.875+0000 7ff69d674640 1 --2- 172.21.15.64:0/2109048535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb67ea1b0 0x556fb6bbabe0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff6941054c0 tx=0x7ff69410bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d3b7aa0630635d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:16.882 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.875+0000 7ff68a7fc640 1 --2- 172.21.15.64:0/1948118693 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff6840b6700 0x7ff6840b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:16.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.876+0000 7ff6837fe640 1 -- 172.21.15.64:0/2109048535 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6940ca2a0 con 0x556fb67ea1b0 2024-09-21T04:13:16.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.876+0000 7ff69de75640 1 --2- 172.21.15.64:0/1948118693 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff6840b6700 0x7ff6840b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:16.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.876+0000 7ff6837fe640 1 -- 172.21.15.64:0/2109048535 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff694115070 con 0x556fb67ea1b0 2024-09-21T04:13:16.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.876+0000 7ff6837fe640 1 -- 172.21.15.64:0/2109048535 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6940cd3c0 con 0x556fb67ea1b0 2024-09-21T04:13:16.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.876+0000 7ff68a7fc640 1 -- 172.21.15.64:0/1948118693 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8659+0+0 (secure 0 0 0) 0x7ff694107e90 con 0x7ff698104860 2024-09-21T04:13:16.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.877+0000 7ff6a5e4c740 1 -- 172.21.15.64:0/2109048535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb67ea1b0 msgr2=0x556fb6bbabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:16.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.877+0000 7ff69de75640 1 --2- 172.21.15.64:0/1948118693 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff6840b6700 0x7ff6840b8c10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff690005fd0 tx=0x7ff690009620 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:16.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.877+0000 7ff6a5e4c740 1 --2- 172.21.15.64:0/2109048535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb67ea1b0 0x556fb6bbabe0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff6941054c0 tx=0x7ff69410bf90 comp rx=0 tx=0).stop 2024-09-21T04:13:16.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.877+0000 7ff6a5e4c740 1 -- 172.21.15.64:0/2109048535 shutdown_connections 2024-09-21T04:13:16.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.877+0000 7ff6a5e4c740 1 --2- 172.21.15.64:0/2109048535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fb67f2390 0x556fb6bbb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.877+0000 7ff6a5e4c740 1 --2- 172.21.15.64:0/2109048535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb67ea1b0 0x556fb6bbabe0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.877+0000 7ff6a5e4c740 1 --2- 172.21.15.64:0/2109048535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556fb6813070 0x556fb6bba6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.877+0000 7ff6a5e4c740 1 -- 172.21.15.64:0/2109048535 >> 172.21.15.64:0/2109048535 conn(0x556fb6807460 msgr2=0x556fb6bb8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:16.883 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.877+0000 7ff6a5e4c740 1 -- 172.21.15.64:0/2109048535 shutdown_connections 2024-09-21T04:13:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.877+0000 7ff6a5e4c740 1 -- 172.21.15.64:0/2109048535 wait complete. 2024-09-21T04:13:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.877+0000 7ff6a5e4c740 1 -- start start 2024-09-21T04:13:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.877+0000 7ff6a5e4c740 1 Processor -- start 2024-09-21T04:13:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.878+0000 7ff6a5e4c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb67ea1b0 0x556fb6bc5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.878+0000 7ff6a5e4c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fb67f2390 0x556fb6bc6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.878+0000 7ff6a5e4c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556fb6813070 0x556fb6bd1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.878+0000 7ff6a5e4c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556fb6981700 con 0x556fb67ea1b0 2024-09-21T04:13:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.878+0000 7ff6a5e4c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556fb6abb160 con 0x556fb67f2390 2024-09-21T04:13:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.878+0000 7ff6a5e4c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556fb6bc8c40 con 0x556fb6813070 2024-09-21T04:13:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.878+0000 7ff6a5e4c740 10 client.0 _ll_register_callbacks cb 0x556fb6ad49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.878+0000 7ff69d674640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb67ea1b0 0x556fb6bc5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.878+0000 7ff69d674640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb67ea1b0 0x556fb6bc5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48538/0 (socket says 172.21.15.64:48538) 2024-09-21T04:13:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.878+0000 7ff69d674640 1 -- 172.21.15.64:0/1784047867 learned_addr learned my addr 172.21.15.64:0/1784047867 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:16.884 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.878+0000 7ff69ce73640 1 --2- 172.21.15.64:0/1784047867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fb67f2390 0x556fb6bc6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:16.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.878+0000 7ff69de75640 1 --2- 172.21.15.64:0/1784047867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556fb6813070 0x556fb6bd1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:16.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.878+0000 7ff69d674640 1 -- 172.21.15.64:0/1784047867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556fb6813070 msgr2=0x556fb6bd1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:16.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.878+0000 7ff69d674640 1 --2- 172.21.15.64:0/1784047867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556fb6813070 0x556fb6bd1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.878+0000 7ff69d674640 1 -- 172.21.15.64:0/1784047867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fb67f2390 msgr2=0x556fb6bc6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:16.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.878+0000 7ff69d674640 1 --2- 172.21.15.64:0/1784047867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fb67f2390 0x556fb6bc6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.878+0000 7ff69d674640 1 -- 172.21.15.64:0/1784047867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fb6ae8c90 con 0x556fb67ea1b0 2024-09-21T04:13:16.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.879+0000 7ff69d674640 1 --2- 172.21.15.64:0/1784047867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb67ea1b0 0x556fb6bc5d40 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff6940caec0 tx=0x7ff694108190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52203d3e3ca9c8d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:16.885 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.879+0000 7ff680ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:13:16.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.879+0000 7ff680ff9640 1 -- 172.21.15.64:0/1784047867 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff694049030 con 0x556fb67ea1b0 2024-09-21T04:13:16.887 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.879+0000 7ff680ff9640 1 -- 172.21.15.64:0/1784047867 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8659+0+0 (secure 0 0 0) 0x7ff69410b4f0 con 0x556fb67ea1b0 2024-09-21T04:13:16.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.879+0000 7ff6a5e4c740 1 -- 172.21.15.64:0/1784047867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff694115070 con 0x556fb67ea1b0 2024-09-21T04:13:16.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.879+0000 7ff680ff9640 1 -- 172.21.15.64:0/1784047867 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff6940cd5a0 con 0x556fb67ea1b0 2024-09-21T04:13:16.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.879+0000 7ff6a5e4c740 10 client.17424 fetch_fsmap learned FSMap version 40 2024-09-21T04:13:16.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.879+0000 7ff6a5e4c740 1 -- 172.21.15.64:0/1784047867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556fb6bd48a0 con 0x556fb67ea1b0 2024-09-21T04:13:16.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.879+0000 7ff680ff9640 1 -- 172.21.15.64:0/1784047867 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff694115070 con 0x556fb67ea1b0 2024-09-21T04:13:16.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.879+0000 7ff680ff9640 1 -- 172.21.15.64:0/1784047867 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7ff6941057d0 con 0x556fb67ea1b0 2024-09-21T04:13:16.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.879+0000 7ff6a5e4c740 10 client.17424 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:13:16.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.879+0000 7ff6a5e4c740 10 client.17424 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:13:16.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.879+0000 7ff6a5e4c740 20 client.17424 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:16.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.879+0000 7ff6a5e4c740 1 --2- 172.21.15.64:0/1784047867 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556fb67d19f0 0x556fb6bd7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:16.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.879+0000 7ff6a5e4c740 4 client.17424 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:13:16.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.880+0000 7ff6a5e4c740 1 -- 172.21.15.64:0/1784047867 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fb6b32df0 con 0x556fb67d19f0 2024-09-21T04:13:16.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.880+0000 7ff69ce73640 1 --2- 172.21.15.64:0/1784047867 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556fb67d19f0 0x556fb6bd7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:16.888 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.880+0000 7ff680ff9640 10 client.17424 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.880+0000 7ff69ce73640 1 --2- 172.21.15.64:0/1784047867 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556fb67d19f0 0x556fb6bd7c00 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=161af87ab0f31ce2 server_cookie=bdb00b2a6a9cc98c in_seq=0 out_seq=0 2024-09-21T04:13:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.881+0000 7ff680ff9640 1 -- 172.21.15.64:0/1784047867 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556fb6b32df0 con 0x556fb67d19f0 2024-09-21T04:13:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.881+0000 7ff680ff9640 10 client.17424 handle_command_reply: tid=0 2024-09-21T04:13:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.881+0000 7ff6a5e4c740 1 client.17424 shutdown 2024-09-21T04:13:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.881+0000 7ff6a5e4c740 1 -- 172.21.15.64:0/1784047867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb67ea1b0 msgr2=0x556fb6bc5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.881+0000 7ff6a5e4c740 1 --2- 172.21.15.64:0/1784047867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb67ea1b0 0x556fb6bc5d40 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff6940caec0 tx=0x7ff694108190 comp rx=0 tx=0).stop 2024-09-21T04:13:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.881+0000 7ff6a5e4c740 1 -- 172.21.15.64:0/1784047867 shutdown_connections 2024-09-21T04:13:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.881+0000 7ff6a5e4c740 1 --2- 172.21.15.64:0/1784047867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556fb6813070 0x556fb6bd1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.881+0000 7ff6a5e4c740 1 --2- 172.21.15.64:0/1784047867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fb67f2390 0x556fb6bc6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.881+0000 7ff6a5e4c740 1 --2- 172.21.15.64:0/1784047867 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556fb67d19f0 0x556fb6bd7c00 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.881+0000 7ff6a5e4c740 1 --2- 172.21.15.64:0/1784047867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb67ea1b0 0x556fb6bc5d40 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.881+0000 7ff6a5e4c740 1 -- 172.21.15.64:0/1784047867 >> 172.21.15.64:0/1784047867 conn(0x556fb6807460 msgr2=0x556fb6bbd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.881+0000 7ff680ff9640 0 client.17424 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:16.889 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.881+0000 7ff6a5e4c740 1 -- 172.21.15.64:0/1784047867 shutdown_connections 2024-09-21T04:13:16.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.882+0000 7ff6a5e4c740 1 -- 172.21.15.64:0/1784047867 wait complete. 2024-09-21T04:13:16.890 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.882+0000 7ff6a5e4c740 20 client.17424 trim_cache size 0 max 16384 2024-09-21T04:13:16.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.895+0000 7ff6a5e4c740 1 Processor -- start 2024-09-21T04:13:16.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.895+0000 7ff6a5e4c740 1 -- start start 2024-09-21T04:13:16.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.895+0000 7ff6a5e4c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb6807460 0x556fb6bc6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:16.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.895+0000 7ff6a5e4c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fb67ea1b0 0x556fb6bb7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:16.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.895+0000 7ff6a5e4c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556fb67f2390 0x556fb6bb8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:16.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.895+0000 7ff6a5e4c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff6981b4510 con 0x556fb6807460 2024-09-21T04:13:16.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.895+0000 7ff6a5e4c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff6981b1d20 con 0x556fb67ea1b0 2024-09-21T04:13:16.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.895+0000 7ff6a5e4c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff6981afb90 con 0x556fb67f2390 2024-09-21T04:13:16.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.895+0000 7ff69d674640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb6807460 0x556fb6bc6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:16.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.895+0000 7ff69d674640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb6807460 0x556fb6bc6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48548/0 (socket says 172.21.15.64:48548) 2024-09-21T04:13:16.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.895+0000 7ff69d674640 1 -- 172.21.15.64:0/4142538681 learned_addr learned my addr 172.21.15.64:0/4142538681 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:16.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.895+0000 7ff69ce73640 1 --2- 172.21.15.64:0/4142538681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fb67ea1b0 0x556fb6bb7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:16.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.896+0000 7ff69de75640 1 --2- 172.21.15.64:0/4142538681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556fb67f2390 0x556fb6bb8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:16.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.896+0000 7ff69d674640 1 -- 172.21.15.64:0/4142538681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556fb67f2390 msgr2=0x556fb6bb8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:16.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.896+0000 7ff69d674640 1 --2- 172.21.15.64:0/4142538681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556fb67f2390 0x556fb6bb8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.896+0000 7ff69d674640 1 -- 172.21.15.64:0/4142538681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fb67ea1b0 msgr2=0x556fb6bb7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:16.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.896+0000 7ff69d674640 1 --2- 172.21.15.64:0/4142538681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fb67ea1b0 0x556fb6bb7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.896+0000 7ff69d674640 1 -- 172.21.15.64:0/4142538681 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fb6bbb4b0 con 0x556fb6807460 2024-09-21T04:13:16.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.896+0000 7ff69de75640 1 --2- 172.21.15.64:0/4142538681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556fb67f2390 0x556fb6bb8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:13:16.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.896+0000 7ff69ce73640 1 --2- 172.21.15.64:0/4142538681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fb67ea1b0 0x556fb6bb7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:13:16.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.896+0000 7ff69d674640 1 --2- 172.21.15.64:0/4142538681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb6807460 0x556fb6bc6200 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff694006010 tx=0x7ff69407ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28b7ab4fc1fe23f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:16.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.896+0000 7ff682ffd640 1 -- 172.21.15.64:0/4142538681 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff694049070 con 0x556fb6807460 2024-09-21T04:13:16.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.896+0000 7ff682ffd640 1 -- 172.21.15.64:0/4142538681 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff694115030 con 0x556fb6807460 2024-09-21T04:13:16.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.896+0000 7ff682ffd640 1 -- 172.21.15.64:0/4142538681 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6940cd790 con 0x556fb6807460 2024-09-21T04:13:16.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.897+0000 7ff6a5e4c740 1 -- 172.21.15.64:0/4142538681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb6807460 msgr2=0x556fb6bc6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:16.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.897+0000 7ff6a5e4c740 1 --2- 172.21.15.64:0/4142538681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb6807460 0x556fb6bc6200 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff694006010 tx=0x7ff69407ad40 comp rx=0 tx=0).stop 2024-09-21T04:13:16.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.897+0000 7ff6a5e4c740 1 -- 172.21.15.64:0/4142538681 shutdown_connections 2024-09-21T04:13:16.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.897+0000 7ff6a5e4c740 1 --2- 172.21.15.64:0/4142538681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556fb67f2390 0x556fb6bb8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.897+0000 7ff6a5e4c740 1 --2- 172.21.15.64:0/4142538681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fb67ea1b0 0x556fb6bb7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.897+0000 7ff6a5e4c740 1 --2- 172.21.15.64:0/4142538681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb6807460 0x556fb6bc6200 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.897+0000 7ff6a5e4c740 1 -- 172.21.15.64:0/4142538681 >> 172.21.15.64:0/4142538681 conn(0x556fb6bd2320 msgr2=0x556fb6bc5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:16.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.897+0000 7ff6a5e4c740 1 -- 172.21.15.64:0/4142538681 shutdown_connections 2024-09-21T04:13:16.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.897+0000 7ff6a5e4c740 1 -- 172.21.15.64:0/4142538681 wait complete. 2024-09-21T04:13:16.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.897+0000 7ff6a5e4c740 1 -- start start 2024-09-21T04:13:16.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.897+0000 7ff6a5e4c740 1 Processor -- start 2024-09-21T04:13:16.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.898+0000 7ff6a5e4c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fb67ea1b0 0x556fb6bce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:16.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.898+0000 7ff6a5e4c740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb67f2390 0x556fb6bc9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:16.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.898+0000 7ff6a5e4c740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556fb6807460 0x556fb6bca260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:16.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.898+0000 7ff6a5e4c740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff6981b01e0 con 0x556fb67f2390 2024-09-21T04:13:16.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.898+0000 7ff6a5e4c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff6981b1650 con 0x556fb67ea1b0 2024-09-21T04:13:16.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.898+0000 7ff6a5e4c740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff6981b08b0 con 0x556fb6807460 2024-09-21T04:13:16.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.898+0000 7ff6a5e4c740 10 client.0 _ll_register_callbacks cb 0x7ff68c0095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:16.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.898+0000 7ff69ce73640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb67f2390 0x556fb6bc9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:16.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.898+0000 7ff69ce73640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb67f2390 0x556fb6bc9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48554/0 (socket says 172.21.15.64:48554) 2024-09-21T04:13:16.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.898+0000 7ff69ce73640 1 -- 172.21.15.64:0/1194461530 learned_addr learned my addr 172.21.15.64:0/1194461530 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:16.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.898+0000 7ff69d674640 1 --2- 172.21.15.64:0/1194461530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fb67ea1b0 0x556fb6bce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:16.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.898+0000 7ff69de75640 1 --2- 172.21.15.64:0/1194461530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556fb6807460 0x556fb6bca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:16.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.898+0000 7ff69ce73640 1 -- 172.21.15.64:0/1194461530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556fb6807460 msgr2=0x556fb6bca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:16.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.898+0000 7ff69ce73640 1 --2- 172.21.15.64:0/1194461530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556fb6807460 0x556fb6bca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.898+0000 7ff69ce73640 1 -- 172.21.15.64:0/1194461530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fb67ea1b0 msgr2=0x556fb6bce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:16.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.898+0000 7ff69ce73640 1 --2- 172.21.15.64:0/1194461530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fb67ea1b0 0x556fb6bce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.898+0000 7ff69ce73640 1 -- 172.21.15.64:0/1194461530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fb6bca9d0 con 0x556fb67f2390 2024-09-21T04:13:16.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.899+0000 7ff69ce73640 1 --2- 172.21.15.64:0/1194461530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb67f2390 0x556fb6bc9c10 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff69806aaa0 tx=0x7ff68c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66760dc607641a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:16.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.899+0000 7ff680ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:13:16.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.899+0000 7ff680ff9640 1 -- 172.21.15.64:0/1194461530 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c012650 con 0x556fb67f2390 2024-09-21T04:13:16.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.899+0000 7ff6a5e4c740 1 -- 172.21.15.64:0/1194461530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff694115030 con 0x556fb67f2390 2024-09-21T04:13:16.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.899+0000 7ff680ff9640 1 -- 172.21.15.64:0/1194461530 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8659+0+0 (secure 0 0 0) 0x7ff68c0127f0 con 0x556fb67f2390 2024-09-21T04:13:16.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.899+0000 7ff680ff9640 1 -- 172.21.15.64:0/1194461530 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff68c0129f0 con 0x556fb67f2390 2024-09-21T04:13:16.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.899+0000 7ff6a5e4c740 10 client.17430 fetch_fsmap learned FSMap version 40 2024-09-21T04:13:16.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.899+0000 7ff6a5e4c740 1 -- 172.21.15.64:0/1194461530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556fb6c424e0 con 0x556fb67f2390 2024-09-21T04:13:16.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.899+0000 7ff680ff9640 1 -- 172.21.15.64:0/1194461530 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff694115030 con 0x556fb67f2390 2024-09-21T04:13:16.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.899+0000 7ff680ff9640 1 -- 172.21.15.64:0/1194461530 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7ff68c011440 con 0x556fb67f2390 2024-09-21T04:13:16.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.899+0000 7ff6a5e4c740 10 client.17430 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:13:16.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.899+0000 7ff6a5e4c740 10 client.17430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:13:16.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.899+0000 7ff6a5e4c740 20 client.17430 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:16.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.899+0000 7ff6a5e4c740 1 --2- 172.21.15.64:0/1194461530 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556fb6813070 0x556fb6c44d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:16.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.899+0000 7ff6a5e4c740 4 client.17430 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:13:16.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.899+0000 7ff6a5e4c740 1 -- 172.21.15.64:0/1194461530 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556fb6b2e8a0 con 0x556fb6813070 2024-09-21T04:13:16.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.900+0000 7ff69d674640 1 --2- 172.21.15.64:0/1194461530 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556fb6813070 0x556fb6c44d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:16.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.900+0000 7ff680ff9640 10 client.17430 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:16.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.900+0000 7ff69d674640 1 --2- 172.21.15.64:0/1194461530 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556fb6813070 0x556fb6c44d00 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fef5329fabcc59e9 server_cookie=4f4e72a2115760c6 in_seq=0 out_seq=0 2024-09-21T04:13:16.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.900+0000 7ff680ff9640 1 -- 172.21.15.64:0/1194461530 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556fb6b2e8a0 con 0x556fb6813070 2024-09-21T04:13:16.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.900+0000 7ff680ff9640 10 client.17430 handle_command_reply: tid=0 2024-09-21T04:13:16.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.900+0000 7ff6a5e4c740 1 client.17430 shutdown 2024-09-21T04:13:16.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.901+0000 7ff6a5e4c740 1 -- 172.21.15.64:0/1194461530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb67f2390 msgr2=0x556fb6bc9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:16.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.901+0000 7ff6a5e4c740 1 --2- 172.21.15.64:0/1194461530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb67f2390 0x556fb6bc9c10 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff69806aaa0 tx=0x7ff68c0035f0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.901+0000 7ff6a5e4c740 1 -- 172.21.15.64:0/1194461530 shutdown_connections 2024-09-21T04:13:16.907 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.901+0000 7ff6a5e4c740 1 --2- 172.21.15.64:0/1194461530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556fb6807460 0x556fb6bca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.901+0000 7ff6a5e4c740 1 --2- 172.21.15.64:0/1194461530 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x556fb6813070 0x556fb6c44d00 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.901+0000 7ff6a5e4c740 1 --2- 172.21.15.64:0/1194461530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fb67f2390 0x556fb6bc9c10 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.901+0000 7ff6a5e4c740 1 --2- 172.21.15.64:0/1194461530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556fb67ea1b0 0x556fb6bce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.901+0000 7ff6a5e4c740 1 -- 172.21.15.64:0/1194461530 >> 172.21.15.64:0/1194461530 conn(0x556fb6bd2320 msgr2=0x556fb6bbd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:16.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.901+0000 7ff680ff9640 0 client.17430 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:16.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.901+0000 7ff6a5e4c740 1 -- 172.21.15.64:0/1194461530 shutdown_connections 2024-09-21T04:13:16.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.901+0000 7ff6a5e4c740 1 -- 172.21.15.64:0/1194461530 wait complete. 2024-09-21T04:13:16.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.901+0000 7ff6a5e4c740 20 client.17430 trim_cache size 0 max 16384 2024-09-21T04:13:16.908 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:13:16.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.901+0000 7ff69f8d8640 1 -- 172.21.15.64:0/1948118693 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff6840b6700 msgr2=0x7ff6840b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:16.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.901+0000 7ff69f8d8640 1 --2- 172.21.15.64:0/1948118693 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff6840b6700 0x7ff6840b8c10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff690005fd0 tx=0x7ff690009620 comp rx=0 tx=0).stop 2024-09-21T04:13:16.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.901+0000 7ff69f8d8640 1 -- 172.21.15.64:0/1948118693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff698104860 msgr2=0x7ff698102770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:16.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.901+0000 7ff69f8d8640 1 --2- 172.21.15.64:0/1948118693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff698104860 0x7ff698102770 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff694005940 tx=0x7ff694004270 comp rx=0 tx=0).stop 2024-09-21T04:13:16.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.902+0000 7ff69f8d8640 1 -- 172.21.15.64:0/1948118693 shutdown_connections 2024-09-21T04:13:16.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.902+0000 7ff69f8d8640 1 --2- 172.21.15.64:0/1948118693 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff6840b6700 0x7ff6840b8c10 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.902+0000 7ff69f8d8640 1 --2- 172.21.15.64:0/1948118693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff69810de10 0x7ff6981031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.902+0000 7ff69f8d8640 1 --2- 172.21.15.64:0/1948118693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff698105290 0x7ff698102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.902+0000 7ff69f8d8640 1 --2- 172.21.15.64:0/1948118693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff698104860 0x7ff698102770 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:16.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.902+0000 7ff69f8d8640 1 -- 172.21.15.64:0/1948118693 >> 172.21.15.64:0/1948118693 conn(0x7ff698079730 msgr2=0x7ff6981008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:16.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.902+0000 7ff69f8d8640 1 -- 172.21.15.64:0/1948118693 shutdown_connections 2024-09-21T04:13:16.910 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:16.902+0000 7ff69f8d8640 1 -- 172.21.15.64:0/1948118693 wait complete. 2024-09-21T04:13:18.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:18 smithi064 ceph-mon[31180]: pgmap v606: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 204 B/s wr, 7 op/s 2024-09-21T04:13:18.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:18 smithi131 ceph-mon[35193]: pgmap v606: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 204 B/s wr, 7 op/s 2024-09-21T04:13:18.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:18 smithi119 ceph-mon[34811]: pgmap v606: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 204 B/s wr, 7 op/s 2024-09-21T04:13:18.818 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T04:13:18.819 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T04:13:19.090 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:13:19.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.883+0000 7f8e6789e640 1 -- 172.21.15.64:0/592187190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e601083a0 msgr2=0x7f8e601087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:19.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.883+0000 7f8e6789e640 1 --2- 172.21.15.64:0/592187190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e601083a0 0x7f8e601087b0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f8e50009900 tx=0x7f8e5002a410 comp rx=0 tx=0).stop 2024-09-21T04:13:19.912 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.883+0000 7f8e6789e640 1 -- 172.21.15.64:0/592187190 shutdown_connections 2024-09-21T04:13:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.883+0000 7f8e6789e640 1 --2- 172.21.15.64:0/592187190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e60108cf0 0x7f8e6010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.883+0000 7f8e6789e640 1 --2- 172.21.15.64:0/592187190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e601083a0 0x7f8e601087b0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.883+0000 7f8e6789e640 1 --2- 172.21.15.64:0/592187190 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e60102a60 0x7f8e60102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.883+0000 7f8e6789e640 1 -- 172.21.15.64:0/592187190 >> 172.21.15.64:0/592187190 conn(0x7f8e60079730 msgr2=0x7f8e60079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.883+0000 7f8e6789e640 1 -- 172.21.15.64:0/592187190 shutdown_connections 2024-09-21T04:13:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.883+0000 7f8e6789e640 1 -- 172.21.15.64:0/592187190 wait complete. 2024-09-21T04:13:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.884+0000 7f8e6789e640 1 Processor -- start 2024-09-21T04:13:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.884+0000 7f8e6789e640 1 -- start start 2024-09-21T04:13:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.884+0000 7f8e6789e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e60102a60 0x7f8e6019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.884+0000 7f8e6789e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e601083a0 0x7f8e6019b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.884+0000 7f8e6789e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e60108cf0 0x7f8e6019bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.884+0000 7f8e6789e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8e601129c0 con 0x7f8e601083a0 2024-09-21T04:13:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.884+0000 7f8e6789e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8e60112840 con 0x7f8e60108cf0 2024-09-21T04:13:19.913 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.884+0000 7f8e6789e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8e60112b40 con 0x7f8e60102a60 2024-09-21T04:13:19.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.884+0000 7f8e64e39640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e601083a0 0x7f8e6019b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:19.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.885+0000 7f8e64e39640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e601083a0 0x7f8e6019b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48562/0 (socket says 172.21.15.64:48562) 2024-09-21T04:13:19.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.885+0000 7f8e64e39640 1 -- 172.21.15.64:0/2551345470 learned_addr learned my addr 172.21.15.64:0/2551345470 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:19.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.885+0000 7f8e6563a640 1 --2- 172.21.15.64:0/2551345470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e60102a60 0x7f8e6019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:19.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.885+0000 7f8e65e3b640 1 --2- 172.21.15.64:0/2551345470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e60108cf0 0x7f8e6019bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:19.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.885+0000 7f8e64e39640 1 -- 172.21.15.64:0/2551345470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e60102a60 msgr2=0x7f8e6019b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:19.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.885+0000 7f8e64e39640 1 --2- 172.21.15.64:0/2551345470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e60102a60 0x7f8e6019b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.885+0000 7f8e64e39640 1 -- 172.21.15.64:0/2551345470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e60108cf0 msgr2=0x7f8e6019bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:19.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.885+0000 7f8e64e39640 1 --2- 172.21.15.64:0/2551345470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e60108cf0 0x7f8e6019bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.885+0000 7f8e64e39640 1 -- 172.21.15.64:0/2551345470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e601a6560 con 0x7f8e601083a0 2024-09-21T04:13:19.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.885+0000 7f8e6563a640 1 --2- 172.21.15.64:0/2551345470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e60102a60 0x7f8e6019b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:13:19.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.885+0000 7f8e64e39640 1 --2- 172.21.15.64:0/2551345470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e601083a0 0x7f8e6019b810 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f8e50032fd0 tx=0x7f8e50004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:19.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.885+0000 7f8e567fc640 1 -- 172.21.15.64:0/2551345470 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5003f030 con 0x7f8e601083a0 2024-09-21T04:13:19.914 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.885+0000 7f8e567fc640 1 -- 172.21.15.64:0/2551345470 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e50004490 con 0x7f8e601083a0 2024-09-21T04:13:19.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.885+0000 7f8e6789e640 1 -- 172.21.15.64:0/2551345470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e601a67f0 con 0x7f8e601083a0 2024-09-21T04:13:19.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.885+0000 7f8e567fc640 1 -- 172.21.15.64:0/2551345470 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e50035070 con 0x7f8e601083a0 2024-09-21T04:13:19.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.885+0000 7f8e6789e640 1 -- 172.21.15.64:0/2551345470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e601a6da0 con 0x7f8e601083a0 2024-09-21T04:13:19.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.888+0000 7f8e6de12740 1 Processor -- start 2024-09-21T04:13:19.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.888+0000 7f8e6de12740 1 -- start start 2024-09-21T04:13:19.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.889+0000 7f8e6de12740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a596350 0x55df4a93d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:19.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.889+0000 7f8e6de12740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df4a56d490 0x55df4a93d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:19.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.889+0000 7f8e6de12740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df4a575670 0x55df4a93deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:19.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.889+0000 7f8e6de12740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55df4a8b3e60 con 0x55df4a596350 2024-09-21T04:13:19.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.889+0000 7f8e6de12740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55df4a8a3290 con 0x55df4a575670 2024-09-21T04:13:19.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.889+0000 7f8e6de12740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55df4a8a70a0 con 0x55df4a56d490 2024-09-21T04:13:19.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.889+0000 7f8e567fc640 1 -- 172.21.15.64:0/2551345470 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f8e50042030 con 0x7f8e601083a0 2024-09-21T04:13:19.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.889+0000 7f8e65e3b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a596350 0x55df4a93d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:19.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.889+0000 7f8e6563a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df4a56d490 0x55df4a93d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:19.915 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.889+0000 7f8e65e3b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a596350 0x55df4a93d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48570/0 (socket says 172.21.15.64:48570) 2024-09-21T04:13:19.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.889+0000 7f8e65e3b640 1 -- 172.21.15.64:0/1409853216 learned_addr learned my addr 172.21.15.64:0/1409853216 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:19.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.889+0000 7f8e64e39640 1 --2- 172.21.15.64:0/1409853216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df4a575670 0x55df4a93deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:19.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.889+0000 7f8e65e3b640 1 -- 172.21.15.64:0/1409853216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df4a56d490 msgr2=0x55df4a93d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:19.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.889+0000 7f8e65e3b640 1 --2- 172.21.15.64:0/1409853216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df4a56d490 0x55df4a93d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.889+0000 7f8e65e3b640 1 -- 172.21.15.64:0/1409853216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df4a575670 msgr2=0x55df4a93deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:19.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.889+0000 7f8e65e3b640 1 --2- 172.21.15.64:0/1409853216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df4a575670 0x55df4a93deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.889+0000 7f8e65e3b640 1 -- 172.21.15.64:0/1409853216 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df4a601930 con 0x55df4a596350 2024-09-21T04:13:19.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.889+0000 7f8e64e39640 1 --2- 172.21.15.64:0/1409853216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df4a575670 0x55df4a93deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:13:19.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.889+0000 7f8e65e3b640 1 --2- 172.21.15.64:0/1409853216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a596350 0x55df4a93d430 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009f50 tx=0x7f8e5800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4256f994aedc6d12 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:19.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.890+0000 7f8e567fc640 1 --2- 172.21.15.64:0/2551345470 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8e4c0b6730 0x7f8e4c0b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:19.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.890+0000 7f8e4b7fe640 1 -- 172.21.15.64:0/1409853216 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5801a070 con 0x55df4a596350 2024-09-21T04:13:19.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.890+0000 7f8e4b7fe640 1 -- 172.21.15.64:0/1409853216 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e580158c0 con 0x55df4a596350 2024-09-21T04:13:19.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.890+0000 7f8e4b7fe640 1 -- 172.21.15.64:0/1409853216 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5800a820 con 0x55df4a596350 2024-09-21T04:13:19.916 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.890+0000 7f8e567fc640 1 -- 172.21.15.64:0/2551345470 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8659+0+0 (secure 0 0 0) 0x7f8e50107940 con 0x7f8e601083a0 2024-09-21T04:13:19.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.890+0000 7f8e6563a640 1 --2- 172.21.15.64:0/2551345470 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8e4c0b6730 0x7f8e4c0b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:19.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.891+0000 7f8e6de12740 1 -- 172.21.15.64:0/1409853216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a596350 msgr2=0x55df4a93d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:19.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.891+0000 7f8e6563a640 1 --2- 172.21.15.64:0/2551345470 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8e4c0b6730 0x7f8e4c0b8c40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c005fd0 tx=0x7f8e5c00b5c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:19.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.891+0000 7f8e6de12740 1 --2- 172.21.15.64:0/1409853216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a596350 0x55df4a93d430 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009f50 tx=0x7f8e5800c8b0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.891+0000 7f8e6de12740 1 -- 172.21.15.64:0/1409853216 shutdown_connections 2024-09-21T04:13:19.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.891+0000 7f8e6de12740 1 --2- 172.21.15.64:0/1409853216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df4a575670 0x55df4a93deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.891+0000 7f8e6de12740 1 --2- 172.21.15.64:0/1409853216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df4a56d490 0x55df4a93d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.891+0000 7f8e6de12740 1 --2- 172.21.15.64:0/1409853216 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a596350 0x55df4a93d430 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.891+0000 7f8e6de12740 1 -- 172.21.15.64:0/1409853216 >> 172.21.15.64:0/1409853216 conn(0x55df4a58a740 msgr2=0x55df4a93b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:19.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.891+0000 7f8e6de12740 1 -- 172.21.15.64:0/1409853216 shutdown_connections 2024-09-21T04:13:19.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.891+0000 7f8e6de12740 1 -- 172.21.15.64:0/1409853216 wait complete. 2024-09-21T04:13:19.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.891+0000 7f8e6de12740 1 -- start start 2024-09-21T04:13:19.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.891+0000 7f8e6de12740 1 Processor -- start 2024-09-21T04:13:19.917 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.892+0000 7f8e6de12740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df4a56d490 0x55df4a94d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:19.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.892+0000 7f8e6de12740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df4a575670 0x55df4a94d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:19.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.892+0000 7f8e6de12740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a596350 0x55df4a9592a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:19.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.892+0000 7f8e6de12740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55df4a704890 con 0x55df4a596350 2024-09-21T04:13:19.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.892+0000 7f8e6de12740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55df4a83dd40 con 0x55df4a575670 2024-09-21T04:13:19.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.892+0000 7f8e6de12740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55df4a94ff40 con 0x55df4a56d490 2024-09-21T04:13:19.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.892+0000 7f8e6de12740 10 client.0 _ll_register_callbacks cb 0x55df4a8a1c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:19.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.892+0000 7f8e65e3b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a596350 0x55df4a9592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:19.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.892+0000 7f8e65e3b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a596350 0x55df4a9592a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48582/0 (socket says 172.21.15.64:48582) 2024-09-21T04:13:19.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.892+0000 7f8e65e3b640 1 -- 172.21.15.64:0/502825834 learned_addr learned my addr 172.21.15.64:0/502825834 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:19.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.892+0000 7f8e6563a640 1 --2- 172.21.15.64:0/502825834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df4a56d490 0x55df4a94d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:19.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.892+0000 7f8e64e39640 1 --2- 172.21.15.64:0/502825834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df4a575670 0x55df4a94d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:19.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.892+0000 7f8e65e3b640 1 -- 172.21.15.64:0/502825834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df4a56d490 msgr2=0x55df4a94d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:19.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.892+0000 7f8e65e3b640 1 --2- 172.21.15.64:0/502825834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df4a56d490 0x55df4a94d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.918 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.892+0000 7f8e65e3b640 1 -- 172.21.15.64:0/502825834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df4a575670 msgr2=0x55df4a94d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:19.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.892+0000 7f8e65e3b640 1 --2- 172.21.15.64:0/502825834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df4a575670 0x55df4a94d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.892+0000 7f8e65e3b640 1 -- 172.21.15.64:0/502825834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df4a9597e0 con 0x55df4a596350 2024-09-21T04:13:19.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.892+0000 7f8e64e39640 1 --2- 172.21.15.64:0/502825834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df4a575670 0x55df4a94d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:13:19.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.892+0000 7f8e65e3b640 1 --2- 172.21.15.64:0/502825834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a596350 0x55df4a9592a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009a90 tx=0x7f8e58015ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0bb9247f215913a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:19.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.893+0000 7f8e48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:13:19.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.893+0000 7f8e48ff9640 1 -- 172.21.15.64:0/502825834 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5801a040 con 0x55df4a596350 2024-09-21T04:13:19.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.893+0000 7f8e48ff9640 1 -- 172.21.15.64:0/502825834 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8659+0+0 (secure 0 0 0) 0x7f8e58012040 con 0x55df4a596350 2024-09-21T04:13:19.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.893+0000 7f8e6de12740 1 -- 172.21.15.64:0/502825834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e580158c0 con 0x55df4a596350 2024-09-21T04:13:19.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.893+0000 7f8e48ff9640 1 -- 172.21.15.64:0/502825834 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8e5802d050 con 0x55df4a596350 2024-09-21T04:13:19.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.893+0000 7f8e6de12740 10 client.17442 fetch_fsmap learned FSMap version 40 2024-09-21T04:13:19.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.893+0000 7f8e6de12740 1 -- 172.21.15.64:0/502825834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55df4a95c110 con 0x55df4a596350 2024-09-21T04:13:19.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.893+0000 7f8e48ff9640 1 -- 172.21.15.64:0/502825834 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e580158c0 con 0x55df4a596350 2024-09-21T04:13:19.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.893+0000 7f8e48ff9640 1 -- 172.21.15.64:0/502825834 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f8e58020450 con 0x55df4a596350 2024-09-21T04:13:19.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.893+0000 7f8e6de12740 10 client.17442 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:13:19.919 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.893+0000 7f8e6de12740 10 client.17442 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:13:19.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.893+0000 7f8e6de12740 20 client.17442 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:19.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.893+0000 7f8e6de12740 1 --2- 172.21.15.64:0/502825834 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55df4a554cd0 0x55df4a95f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:19.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.893+0000 7f8e6de12740 4 client.17442 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:13:19.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.893+0000 7f8e6de12740 1 -- 172.21.15.64:0/502825834 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df4a8b5db0 con 0x55df4a554cd0 2024-09-21T04:13:19.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.894+0000 7f8e6563a640 1 --2- 172.21.15.64:0/502825834 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55df4a554cd0 0x55df4a95f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:19.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.894+0000 7f8e48ff9640 10 client.17442 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:19.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.894+0000 7f8e6563a640 1 --2- 172.21.15.64:0/502825834 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55df4a554cd0 0x55df4a95f520 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba21b8530f148d1b server_cookie=5dae003c1e6c6dae in_seq=0 out_seq=0 2024-09-21T04:13:19.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.895+0000 7f8e48ff9640 1 -- 172.21.15.64:0/502825834 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55df4a8b5db0 con 0x55df4a554cd0 2024-09-21T04:13:19.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.895+0000 7f8e48ff9640 10 client.17442 handle_command_reply: tid=0 2024-09-21T04:13:19.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.895+0000 7f8e6de12740 1 client.17442 shutdown 2024-09-21T04:13:19.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.895+0000 7f8e6de12740 1 -- 172.21.15.64:0/502825834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a596350 msgr2=0x55df4a9592a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:19.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.895+0000 7f8e6de12740 1 --2- 172.21.15.64:0/502825834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a596350 0x55df4a9592a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009a90 tx=0x7f8e58015ca0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.895+0000 7f8e6de12740 1 -- 172.21.15.64:0/502825834 shutdown_connections 2024-09-21T04:13:19.920 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.895+0000 7f8e6de12740 1 --2- 172.21.15.64:0/502825834 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55df4a554cd0 0x55df4a95f520 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.895+0000 7f8e6de12740 1 --2- 172.21.15.64:0/502825834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a596350 0x55df4a9592a0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.895+0000 7f8e6de12740 1 --2- 172.21.15.64:0/502825834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df4a575670 0x55df4a94d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.895+0000 7f8e48ff9640 0 client.17442 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:19.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.895+0000 7f8e6de12740 1 --2- 172.21.15.64:0/502825834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df4a56d490 0x55df4a94d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.895+0000 7f8e6de12740 1 -- 172.21.15.64:0/502825834 >> 172.21.15.64:0/502825834 conn(0x55df4a58a740 msgr2=0x55df4a940580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:19.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.896+0000 7f8e6de12740 1 -- 172.21.15.64:0/502825834 shutdown_connections 2024-09-21T04:13:19.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.896+0000 7f8e6de12740 1 -- 172.21.15.64:0/502825834 wait complete. 2024-09-21T04:13:19.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.896+0000 7f8e6de12740 20 client.17442 trim_cache size 0 max 16384 2024-09-21T04:13:19.925 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.914+0000 7f8e6de12740 1 Processor -- start 2024-09-21T04:13:19.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.914+0000 7f8e6de12740 1 -- start start 2024-09-21T04:13:19.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.914+0000 7f8e6de12740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df4a56d490 0x55df4a959210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:19.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.914+0000 7f8e6de12740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df4a575670 0x55df4a94a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:19.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.914+0000 7f8e6de12740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a596350 0x55df4a94a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:19.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.914+0000 7f8e6de12740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8e601b0180 con 0x55df4a596350 2024-09-21T04:13:19.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.914+0000 7f8e6de12740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8e601ad990 con 0x55df4a575670 2024-09-21T04:13:19.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.914+0000 7f8e6de12740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8e601ab800 con 0x55df4a56d490 2024-09-21T04:13:19.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.915+0000 7f8e65e3b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a596350 0x55df4a94a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:19.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.915+0000 7f8e65e3b640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a596350 0x55df4a94a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48590/0 (socket says 172.21.15.64:48590) 2024-09-21T04:13:19.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.915+0000 7f8e65e3b640 1 -- 172.21.15.64:0/3188341294 learned_addr learned my addr 172.21.15.64:0/3188341294 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:19.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.915+0000 7f8e6563a640 1 --2- 172.21.15.64:0/3188341294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df4a56d490 0x55df4a959210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:19.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.915+0000 7f8e64e39640 1 --2- 172.21.15.64:0/3188341294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df4a575670 0x55df4a94a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:19.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.915+0000 7f8e65e3b640 1 -- 172.21.15.64:0/3188341294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df4a56d490 msgr2=0x55df4a959210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.915+0000 7f8e65e3b640 1 --2- 172.21.15.64:0/3188341294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df4a56d490 0x55df4a959210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.915+0000 7f8e65e3b640 1 -- 172.21.15.64:0/3188341294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df4a575670 msgr2=0x55df4a94a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.915+0000 7f8e65e3b640 1 --2- 172.21.15.64:0/3188341294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df4a575670 0x55df4a94a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.915+0000 7f8e65e3b640 1 -- 172.21.15.64:0/3188341294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df4a94c0e0 con 0x55df4a596350 2024-09-21T04:13:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.915+0000 7f8e65e3b640 1 --2- 172.21.15.64:0/3188341294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a596350 0x55df4a94a8b0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800c8f0 tx=0x7f8e580049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6015397f22904683 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.915+0000 7f8e4affd640 1 -- 172.21.15.64:0/3188341294 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58015890 con 0x55df4a596350 2024-09-21T04:13:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.915+0000 7f8e4affd640 1 -- 172.21.15.64:0/3188341294 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e58009b60 con 0x55df4a596350 2024-09-21T04:13:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.915+0000 7f8e4affd640 1 -- 172.21.15.64:0/3188341294 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58020a90 con 0x55df4a596350 2024-09-21T04:13:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.916+0000 7f8e6de12740 1 -- 172.21.15.64:0/3188341294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a596350 msgr2=0x55df4a94a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.916+0000 7f8e6de12740 1 --2- 172.21.15.64:0/3188341294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a596350 0x55df4a94a8b0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800c8f0 tx=0x7f8e580049c0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.916+0000 7f8e6de12740 1 -- 172.21.15.64:0/3188341294 shutdown_connections 2024-09-21T04:13:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.916+0000 7f8e6de12740 1 --2- 172.21.15.64:0/3188341294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a596350 0x55df4a94a8b0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.916+0000 7f8e6de12740 1 --2- 172.21.15.64:0/3188341294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df4a575670 0x55df4a94a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.927 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.916+0000 7f8e6de12740 1 --2- 172.21.15.64:0/3188341294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df4a56d490 0x55df4a959210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.916+0000 7f8e6de12740 1 -- 172.21.15.64:0/3188341294 >> 172.21.15.64:0/3188341294 conn(0x55df4a58a740 msgr2=0x55df4a948330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.916+0000 7f8e6de12740 1 -- 172.21.15.64:0/3188341294 shutdown_connections 2024-09-21T04:13:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.916+0000 7f8e6de12740 1 -- 172.21.15.64:0/3188341294 wait complete. 2024-09-21T04:13:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.917+0000 7f8e6de12740 1 -- start start 2024-09-21T04:13:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.917+0000 7f8e6de12740 1 Processor -- start 2024-09-21T04:13:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.917+0000 7f8e6de12740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a56d490 0x55df4a962af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.917+0000 7f8e6de12740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df4a575670 0x55df4a960fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.917+0000 7f8e6de12740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df4a596350 0x55df4a961510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.917+0000 7f8e6de12740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8e601abe50 con 0x55df4a56d490 2024-09-21T04:13:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.917+0000 7f8e6de12740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8e601ad2c0 con 0x55df4a575670 2024-09-21T04:13:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.917+0000 7f8e6de12740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8e601ac520 con 0x55df4a596350 2024-09-21T04:13:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.917+0000 7f8e6de12740 10 client.0 _ll_register_callbacks cb 0x7f8e58018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.917+0000 7f8e6563a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a56d490 0x55df4a962af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:19.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.917+0000 7f8e6563a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a56d490 0x55df4a962af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48600/0 (socket says 172.21.15.64:48600) 2024-09-21T04:13:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.917+0000 7f8e6563a640 1 -- 172.21.15.64:0/3502361171 learned_addr learned my addr 172.21.15.64:0/3502361171 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.918+0000 7f8e65e3b640 1 --2- 172.21.15.64:0/3502361171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df4a596350 0x55df4a961510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.918+0000 7f8e64e39640 1 --2- 172.21.15.64:0/3502361171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df4a575670 0x55df4a960fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.918+0000 7f8e6563a640 1 -- 172.21.15.64:0/3502361171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df4a596350 msgr2=0x55df4a961510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.918+0000 7f8e6563a640 1 --2- 172.21.15.64:0/3502361171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df4a596350 0x55df4a961510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.918+0000 7f8e6563a640 1 -- 172.21.15.64:0/3502361171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df4a575670 msgr2=0x55df4a960fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.918+0000 7f8e6563a640 1 --2- 172.21.15.64:0/3502361171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df4a575670 0x55df4a960fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.918+0000 7f8e6563a640 1 -- 172.21.15.64:0/3502361171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df4a961a50 con 0x55df4a56d490 2024-09-21T04:13:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.918+0000 7f8e65e3b640 1 --2- 172.21.15.64:0/3502361171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df4a596350 0x55df4a961510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:13:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.918+0000 7f8e6563a640 1 --2- 172.21.15.64:0/3502361171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a56d490 0x55df4a962af0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c012fd0 tx=0x7f8e5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ad31b31c8040b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.918+0000 7f8e48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:13:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.918+0000 7f8e48ff9640 1 -- 172.21.15.64:0/3502361171 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c01f800 con 0x55df4a56d490 2024-09-21T04:13:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.918+0000 7f8e48ff9640 1 -- 172.21.15.64:0/3502361171 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8659+0+0 (secure 0 0 0) 0x7f8e5c01f9a0 con 0x55df4a56d490 2024-09-21T04:13:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.918+0000 7f8e6de12740 1 -- 172.21.15.64:0/3502361171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e58009b60 con 0x55df4a56d490 2024-09-21T04:13:19.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.919+0000 7f8e48ff9640 1 -- 172.21.15.64:0/3502361171 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8e5c020350 con 0x55df4a56d490 2024-09-21T04:13:19.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.919+0000 7f8e6de12740 10 client.17448 fetch_fsmap learned FSMap version 40 2024-09-21T04:13:19.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.919+0000 7f8e6de12740 1 -- 172.21.15.64:0/3502361171 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55df4a9caa70 con 0x55df4a56d490 2024-09-21T04:13:19.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.919+0000 7f8e48ff9640 1 -- 172.21.15.64:0/3502361171 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e58009b60 con 0x55df4a56d490 2024-09-21T04:13:19.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.919+0000 7f8e48ff9640 1 -- 172.21.15.64:0/3502361171 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f8e5c031ce0 con 0x55df4a56d490 2024-09-21T04:13:19.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.919+0000 7f8e6de12740 10 client.17448 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:13:19.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.919+0000 7f8e6de12740 10 client.17448 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:13:19.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.919+0000 7f8e6de12740 20 client.17448 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:19.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.919+0000 7f8e6de12740 1 --2- 172.21.15.64:0/3502361171 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55df4a52a080 0x55df4a9cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:19.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.919+0000 7f8e6de12740 4 client.17448 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T04:13:19.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.919+0000 7f8e6de12740 1 -- 172.21.15.64:0/3502361171 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55df4a8b1860 con 0x55df4a52a080 2024-09-21T04:13:19.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.919+0000 7f8e64e39640 1 --2- 172.21.15.64:0/3502361171 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55df4a52a080 0x55df4a9cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:19.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.920+0000 7f8e64e39640 1 --2- 172.21.15.64:0/3502361171 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55df4a52a080 0x55df4a9cd290 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=606a91a5675733d6 server_cookie=341ea06d3cf320ea in_seq=0 out_seq=0 2024-09-21T04:13:19.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.920+0000 7f8e48ff9640 10 client.17448 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:19.930 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.921+0000 7f8e48ff9640 1 -- 172.21.15.64:0/3502361171 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55df4a8b1860 con 0x55df4a52a080 2024-09-21T04:13:19.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.921+0000 7f8e48ff9640 10 client.17448 handle_command_reply: tid=0 2024-09-21T04:13:19.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.921+0000 7f8e6de12740 1 client.17448 shutdown 2024-09-21T04:13:19.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.922+0000 7f8e6de12740 1 -- 172.21.15.64:0/3502361171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a56d490 msgr2=0x55df4a962af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:19.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.922+0000 7f8e6de12740 1 --2- 172.21.15.64:0/3502361171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a56d490 0x55df4a962af0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c012fd0 tx=0x7f8e5c0046c0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.922+0000 7f8e6de12740 1 -- 172.21.15.64:0/3502361171 shutdown_connections 2024-09-21T04:13:19.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.922+0000 7f8e6de12740 1 --2- 172.21.15.64:0/3502361171 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55df4a596350 0x55df4a961510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.922+0000 7f8e6de12740 1 --2- 172.21.15.64:0/3502361171 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55df4a575670 0x55df4a960fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.922+0000 7f8e6de12740 1 --2- 172.21.15.64:0/3502361171 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55df4a52a080 0x55df4a9cd290 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.922+0000 7f8e6de12740 1 --2- 172.21.15.64:0/3502361171 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55df4a56d490 0x55df4a962af0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.922+0000 7f8e6de12740 1 -- 172.21.15.64:0/3502361171 >> 172.21.15.64:0/3502361171 conn(0x55df4a58a740 msgr2=0x55df4a948330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:19.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.922+0000 7f8e48ff9640 0 client.17448 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:19.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.922+0000 7f8e6de12740 1 -- 172.21.15.64:0/3502361171 shutdown_connections 2024-09-21T04:13:19.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.922+0000 7f8e6de12740 1 -- 172.21.15.64:0/3502361171 wait complete. 2024-09-21T04:13:19.931 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.922+0000 7f8e6de12740 20 client.17448 trim_cache size 0 max 16384 2024-09-21T04:13:19.932 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:13:19.932 INFO:teuthology.orchestra.run.smithi064.stdout: "return_code": 0, 2024-09-21T04:13:19.932 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_tag": "854413f1-df0a-47b9-aeff-200a0ffbb315", 2024-09-21T04:13:19.932 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": "asynchronous" 2024-09-21T04:13:19.932 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:13:19.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.922+0000 7f8e6789e640 1 -- 172.21.15.64:0/2551345470 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8e4c0b6730 msgr2=0x7f8e4c0b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:19.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.922+0000 7f8e6789e640 1 --2- 172.21.15.64:0/2551345470 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8e4c0b6730 0x7f8e4c0b8c40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c005fd0 tx=0x7f8e5c00b5c0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.922+0000 7f8e6789e640 1 -- 172.21.15.64:0/2551345470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e601083a0 msgr2=0x7f8e6019b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:19.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.922+0000 7f8e6789e640 1 --2- 172.21.15.64:0/2551345470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e601083a0 0x7f8e6019b810 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f8e50032fd0 tx=0x7f8e50004270 comp rx=0 tx=0).stop 2024-09-21T04:13:19.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.923+0000 7f8e6789e640 1 -- 172.21.15.64:0/2551345470 shutdown_connections 2024-09-21T04:13:19.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.923+0000 7f8e6789e640 1 --2- 172.21.15.64:0/2551345470 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f8e4c0b6730 0x7f8e4c0b8c40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.923+0000 7f8e6789e640 1 --2- 172.21.15.64:0/2551345470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e60108cf0 0x7f8e6019bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.923+0000 7f8e6789e640 1 --2- 172.21.15.64:0/2551345470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8e601083a0 0x7f8e6019b810 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.923+0000 7f8e6789e640 1 --2- 172.21.15.64:0/2551345470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e60102a60 0x7f8e6019b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:19.933 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.923+0000 7f8e6789e640 1 -- 172.21.15.64:0/2551345470 >> 172.21.15.64:0/2551345470 conn(0x7f8e60079730 msgr2=0x7f8e60104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.923+0000 7f8e6789e640 1 -- 172.21.15.64:0/2551345470 shutdown_connections 2024-09-21T04:13:19.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:19.923+0000 7f8e6789e640 1 -- 172.21.15.64:0/2551345470 wait complete. 2024-09-21T04:13:20.537 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:20 smithi064 ceph-mon[31180]: pgmap v607: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 195 B/s wr, 7 op/s 2024-09-21T04:13:20.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:20 smithi131 ceph-mon[35193]: pgmap v607: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 195 B/s wr, 7 op/s 2024-09-21T04:13:20.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:20 smithi119 ceph-mon[34811]: pgmap v607: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 195 B/s wr, 7 op/s 2024-09-21T04:13:20.696 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T04:13:20.955 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:13:21.396 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:21 smithi064 ceph-mon[31180]: scrub queued for path: / 2024-09-21T04:13:21.396 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:21 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:13:21.396 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:21 smithi064 ceph-mon[31180]: scrub summary: active paths [/] 2024-09-21T04:13:21.396 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:21 smithi064 ceph-mon[31180]: scrub summary: idle+waiting paths [/] 2024-09-21T04:13:21.397 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:21 smithi064 ceph-mon[31180]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T04:13:21.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.489+0000 7fee3469d640 1 -- 172.21.15.64:0/916764126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee2c102ae0 msgr2=0x7fee2c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:21.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.489+0000 7fee3469d640 1 --2- 172.21.15.64:0/916764126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee2c102ae0 0x7fee2c102ef0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fee28009900 tx=0x7fee2802a3f0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.489+0000 7fee3469d640 1 -- 172.21.15.64:0/916764126 shutdown_connections 2024-09-21T04:13:21.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.489+0000 7fee3469d640 1 --2- 172.21.15.64:0/916764126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee2c108d20 0x7fee2c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.489+0000 7fee3469d640 1 --2- 172.21.15.64:0/916764126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee2c1083d0 0x7fee2c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.489+0000 7fee3469d640 1 --2- 172.21.15.64:0/916764126 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee2c102ae0 0x7fee2c102ef0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.489+0000 7fee3469d640 1 -- 172.21.15.64:0/916764126 >> 172.21.15.64:0/916764126 conn(0x7fee2c079760 msgr2=0x7fee2c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:21.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.489+0000 7fee3469d640 1 -- 172.21.15.64:0/916764126 shutdown_connections 2024-09-21T04:13:21.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.489+0000 7fee3469d640 1 -- 172.21.15.64:0/916764126 wait complete. 2024-09-21T04:13:21.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.490+0000 7fee3469d640 1 Processor -- start 2024-09-21T04:13:21.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.490+0000 7fee3469d640 1 -- start start 2024-09-21T04:13:21.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.490+0000 7fee3469d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee2c102ae0 0x7fee2c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:21.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.490+0000 7fee3469d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee2c1083d0 0x7fee2c19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:21.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.490+0000 7fee3469d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee2c108d20 0x7fee2c19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:21.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.490+0000 7fee3469d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c112990 con 0x7fee2c102ae0 2024-09-21T04:13:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.490+0000 7fee3469d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c112810 con 0x7fee2c108d20 2024-09-21T04:13:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.490+0000 7fee3469d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c112b10 con 0x7fee2c1083d0 2024-09-21T04:13:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.490+0000 7fee32439640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee2c102ae0 0x7fee2c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.490+0000 7fee32439640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee2c102ae0 0x7fee2c19a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50192/0 (socket says 172.21.15.64:50192) 2024-09-21T04:13:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.490+0000 7fee32439640 1 -- 172.21.15.64:0/2677965415 learned_addr learned my addr 172.21.15.64:0/2677965415 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.490+0000 7fee31c38640 1 --2- 172.21.15.64:0/2677965415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee2c1083d0 0x7fee2c19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.490+0000 7fee32c3a640 1 --2- 172.21.15.64:0/2677965415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee2c108d20 0x7fee2c19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.490+0000 7fee32439640 1 -- 172.21.15.64:0/2677965415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee2c1083d0 msgr2=0x7fee2c19aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.490+0000 7fee32439640 1 --2- 172.21.15.64:0/2677965415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee2c1083d0 0x7fee2c19aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.490+0000 7fee32439640 1 -- 172.21.15.64:0/2677965415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee2c108d20 msgr2=0x7fee2c19fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.490+0000 7fee32439640 1 --2- 172.21.15.64:0/2677965415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee2c108d20 0x7fee2c19fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.491+0000 7fee32439640 1 -- 172.21.15.64:0/2677965415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2c1a0360 con 0x7fee2c102ae0 2024-09-21T04:13:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.491+0000 7fee32439640 1 --2- 172.21.15.64:0/2677965415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee2c102ae0 0x7fee2c19a4b0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fee28005940 tx=0x7fee280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.491+0000 7fee237fe640 1 -- 172.21.15.64:0/2677965415 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2803f030 con 0x7fee2c102ae0 2024-09-21T04:13:21.498 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.491+0000 7fee237fe640 1 -- 172.21.15.64:0/2677965415 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee28003cb0 con 0x7fee2c102ae0 2024-09-21T04:13:21.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.491+0000 7fee3469d640 1 -- 172.21.15.64:0/2677965415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee2c1a05f0 con 0x7fee2c102ae0 2024-09-21T04:13:21.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.491+0000 7fee237fe640 1 -- 172.21.15.64:0/2677965415 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee28035070 con 0x7fee2c102ae0 2024-09-21T04:13:21.499 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.491+0000 7fee3469d640 1 -- 172.21.15.64:0/2677965415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee2c1a0ad0 con 0x7fee2c102ae0 2024-09-21T04:13:21.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.494+0000 7fee3ac11740 1 Processor -- start 2024-09-21T04:13:21.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.494+0000 7fee3ac11740 1 -- start start 2024-09-21T04:13:21.505 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.494+0000 7fee237fe640 1 -- 172.21.15.64:0/2677965415 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fee28042030 con 0x7fee2c102ae0 2024-09-21T04:13:21.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.494+0000 7fee3ac11740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593d926f070 0x5593d9616650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:21.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.494+0000 7fee3ac11740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593d92461b0 0x5593d9616b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:21.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.494+0000 7fee3ac11740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d924e390 0x5593d96170d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:21.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.494+0000 7fee3ac11740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5593d958ce50 con 0x5593d924e390 2024-09-21T04:13:21.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.495+0000 7fee3ac11740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5593d957c280 con 0x5593d92461b0 2024-09-21T04:13:21.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.495+0000 7fee3ac11740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5593d9580090 con 0x5593d926f070 2024-09-21T04:13:21.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.495+0000 7fee31c38640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d924e390 0x5593d96170d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:21.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.495+0000 7fee31c38640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d924e390 0x5593d96170d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50198/0 (socket says 172.21.15.64:50198) 2024-09-21T04:13:21.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.495+0000 7fee31c38640 1 -- 172.21.15.64:0/58031586 learned_addr learned my addr 172.21.15.64:0/58031586 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:21.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.495+0000 7fee32439640 1 --2- 172.21.15.64:0/58031586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593d92461b0 0x5593d9616b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:21.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.495+0000 7fee32c3a640 1 --2- 172.21.15.64:0/58031586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593d926f070 0x5593d9616650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:21.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.495+0000 7fee31c38640 1 -- 172.21.15.64:0/58031586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593d926f070 msgr2=0x5593d9616650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:21.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.495+0000 7fee31c38640 1 --2- 172.21.15.64:0/58031586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593d926f070 0x5593d9616650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.495+0000 7fee31c38640 1 -- 172.21.15.64:0/58031586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593d92461b0 msgr2=0x5593d9616b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:21.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.495+0000 7fee31c38640 1 --2- 172.21.15.64:0/58031586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593d92461b0 0x5593d9616b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.495+0000 7fee31c38640 1 -- 172.21.15.64:0/58031586 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593d92da650 con 0x5593d924e390 2024-09-21T04:13:21.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.495+0000 7fee31c38640 1 --2- 172.21.15.64:0/58031586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d924e390 0x5593d96170d0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00bc80 tx=0x7fee1c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8215e7ac4fe44cb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:21.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.495+0000 7fee237fe640 1 --2- 172.21.15.64:0/2677965415 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fee180b66b0 0x7fee180b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:21.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.496+0000 7fee32c3a640 1 --2- 172.21.15.64:0/2677965415 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fee180b66b0 0x7fee180b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:21.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.496+0000 7fee20ff9640 1 -- 172.21.15.64:0/58031586 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c015070 con 0x5593d924e390 2024-09-21T04:13:21.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.496+0000 7fee237fe640 1 -- 172.21.15.64:0/2677965415 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8615+0+0 (secure 0 0 0) 0x7fee28107d50 con 0x7fee2c102ae0 2024-09-21T04:13:21.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.496+0000 7fee20ff9640 1 -- 172.21.15.64:0/58031586 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee1c01a910 con 0x5593d924e390 2024-09-21T04:13:21.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.496+0000 7fee20ff9640 1 -- 172.21.15.64:0/58031586 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c011860 con 0x5593d924e390 2024-09-21T04:13:21.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.497+0000 7fee3ac11740 1 -- 172.21.15.64:0/58031586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d924e390 msgr2=0x5593d96170d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:21.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.497+0000 7fee32c3a640 1 --2- 172.21.15.64:0/2677965415 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fee180b66b0 0x7fee180b8bc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fee24007e50 tx=0x7fee24007600 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:21.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.497+0000 7fee3ac11740 1 --2- 172.21.15.64:0/58031586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d924e390 0x5593d96170d0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00bc80 tx=0x7fee1c00f730 comp rx=0 tx=0).stop 2024-09-21T04:13:21.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.497+0000 7fee3ac11740 1 -- 172.21.15.64:0/58031586 shutdown_connections 2024-09-21T04:13:21.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.497+0000 7fee3ac11740 1 --2- 172.21.15.64:0/58031586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d924e390 0x5593d96170d0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.497+0000 7fee3ac11740 1 --2- 172.21.15.64:0/58031586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593d92461b0 0x5593d9616b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.497+0000 7fee3ac11740 1 --2- 172.21.15.64:0/58031586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593d926f070 0x5593d9616650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.497+0000 7fee3ac11740 1 -- 172.21.15.64:0/58031586 >> 172.21.15.64:0/58031586 conn(0x5593d9263460 msgr2=0x5593d9614540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:21.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.497+0000 7fee3ac11740 1 -- 172.21.15.64:0/58031586 shutdown_connections 2024-09-21T04:13:21.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.497+0000 7fee3ac11740 1 -- 172.21.15.64:0/58031586 wait complete. 2024-09-21T04:13:21.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.497+0000 7fee3ac11740 1 -- start start 2024-09-21T04:13:21.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.497+0000 7fee3ac11740 1 Processor -- start 2024-09-21T04:13:21.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.498+0000 7fee3ac11740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593d92461b0 0x5593d9626130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:21.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.498+0000 7fee3ac11740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d924e390 0x5593d9626670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:21.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.498+0000 7fee3ac11740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593d926f070 0x5593d96324f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:21.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.498+0000 7fee3ac11740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5593d93dd700 con 0x5593d924e390 2024-09-21T04:13:21.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.498+0000 7fee3ac11740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5593d9517110 con 0x5593d926f070 2024-09-21T04:13:21.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.498+0000 7fee3ac11740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5593d9629160 con 0x5593d92461b0 2024-09-21T04:13:21.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.498+0000 7fee3ac11740 10 client.0 _ll_register_callbacks cb 0x5593d9530990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:21.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.498+0000 7fee31c38640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d924e390 0x5593d9626670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:21.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.498+0000 7fee31c38640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d924e390 0x5593d9626670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50204/0 (socket says 172.21.15.64:50204) 2024-09-21T04:13:21.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.498+0000 7fee31c38640 1 -- 172.21.15.64:0/3067024008 learned_addr learned my addr 172.21.15.64:0/3067024008 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:21.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.498+0000 7fee32c3a640 1 --2- 172.21.15.64:0/3067024008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593d926f070 0x5593d96324f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:21.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.498+0000 7fee32439640 1 --2- 172.21.15.64:0/3067024008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593d92461b0 0x5593d9626130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:21.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.498+0000 7fee31c38640 1 -- 172.21.15.64:0/3067024008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593d92461b0 msgr2=0x5593d9626130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:21.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.498+0000 7fee31c38640 1 --2- 172.21.15.64:0/3067024008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593d92461b0 0x5593d9626130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.498+0000 7fee31c38640 1 -- 172.21.15.64:0/3067024008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593d926f070 msgr2=0x5593d96324f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:21.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.498+0000 7fee31c38640 1 --2- 172.21.15.64:0/3067024008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593d926f070 0x5593d96324f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.498+0000 7fee31c38640 1 -- 172.21.15.64:0/3067024008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593d9544c40 con 0x5593d924e390 2024-09-21T04:13:21.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.498+0000 7fee31c38640 1 --2- 172.21.15.64:0/3067024008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d924e390 0x5593d9626670 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fee1c005880 tx=0x7fee1c00bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642e08a083ba4ee6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:21.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.499+0000 7fee15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:13:21.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.499+0000 7fee15ffb640 1 -- 172.21.15.64:0/3067024008 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c015050 con 0x5593d924e390 2024-09-21T04:13:21.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.499+0000 7fee15ffb640 1 -- 172.21.15.64:0/3067024008 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8615+0+0 (secure 0 0 0) 0x7fee1c01aac0 con 0x5593d924e390 2024-09-21T04:13:21.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.499+0000 7fee3ac11740 1 -- 172.21.15.64:0/3067024008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee1c01a910 con 0x5593d924e390 2024-09-21T04:13:21.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.499+0000 7fee15ffb640 1 -- 172.21.15.64:0/3067024008 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fee1c031050 con 0x5593d924e390 2024-09-21T04:13:21.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.499+0000 7fee3ac11740 10 client.17460 fetch_fsmap learned FSMap version 40 2024-09-21T04:13:21.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.499+0000 7fee3ac11740 1 -- 172.21.15.64:0/3067024008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5593d9634f20 con 0x5593d924e390 2024-09-21T04:13:21.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.499+0000 7fee15ffb640 1 -- 172.21.15.64:0/3067024008 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee1c01a910 con 0x5593d924e390 2024-09-21T04:13:21.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.499+0000 7fee15ffb640 1 -- 172.21.15.64:0/3067024008 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7fee1c02c030 con 0x5593d924e390 2024-09-21T04:13:21.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.499+0000 7fee3ac11740 10 client.17460 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:13:21.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.499+0000 7fee3ac11740 10 client.17460 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:13:21.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.499+0000 7fee3ac11740 20 client.17460 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:21.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.499+0000 7fee3ac11740 1 --2- 172.21.15.64:0/3067024008 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5593d922d9f0 0x5593d9638330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:21.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.499+0000 7fee3ac11740 4 client.17460 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:13:21.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.499+0000 7fee3ac11740 1 -- 172.21.15.64:0/3067024008 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593d958eda0 con 0x5593d922d9f0 2024-09-21T04:13:21.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.500+0000 7fee32439640 1 --2- 172.21.15.64:0/3067024008 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5593d922d9f0 0x5593d9638330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:21.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.500+0000 7fee32439640 1 --2- 172.21.15.64:0/3067024008 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5593d922d9f0 0x5593d9638330 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea4edc049565a947 server_cookie=81f7e27f6a9398c in_seq=0 out_seq=0 2024-09-21T04:13:21.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.500+0000 7fee15ffb640 10 client.17460 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:21.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.501+0000 7fee15ffb640 1 -- 172.21.15.64:0/3067024008 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5593d958eda0 con 0x5593d922d9f0 2024-09-21T04:13:21.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.501+0000 7fee15ffb640 10 client.17460 handle_command_reply: tid=0 2024-09-21T04:13:21.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.501+0000 7fee3ac11740 1 client.17460 shutdown 2024-09-21T04:13:21.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.501+0000 7fee3ac11740 1 -- 172.21.15.64:0/3067024008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d924e390 msgr2=0x5593d9626670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:21.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.501+0000 7fee3ac11740 1 --2- 172.21.15.64:0/3067024008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d924e390 0x5593d9626670 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fee1c005880 tx=0x7fee1c00bf80 comp rx=0 tx=0).stop 2024-09-21T04:13:21.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.501+0000 7fee3ac11740 1 -- 172.21.15.64:0/3067024008 shutdown_connections 2024-09-21T04:13:21.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.501+0000 7fee3ac11740 1 --2- 172.21.15.64:0/3067024008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593d926f070 0x5593d96324f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.501+0000 7fee3ac11740 1 --2- 172.21.15.64:0/3067024008 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5593d922d9f0 0x5593d9638330 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.501+0000 7fee15ffb640 0 client.17460 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:21.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.501+0000 7fee3ac11740 1 --2- 172.21.15.64:0/3067024008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d924e390 0x5593d9626670 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.501+0000 7fee3ac11740 1 --2- 172.21.15.64:0/3067024008 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593d92461b0 0x5593d9626130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.501+0000 7fee3ac11740 1 -- 172.21.15.64:0/3067024008 >> 172.21.15.64:0/3067024008 conn(0x5593d9263460 msgr2=0x5593d96197e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:21.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.502+0000 7fee3ac11740 1 -- 172.21.15.64:0/3067024008 shutdown_connections 2024-09-21T04:13:21.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.502+0000 7fee3ac11740 1 -- 172.21.15.64:0/3067024008 wait complete. 2024-09-21T04:13:21.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.502+0000 7fee3ac11740 20 client.17460 trim_cache size 0 max 16384 2024-09-21T04:13:21.523 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.519+0000 7fee3ac11740 1 Processor -- start 2024-09-21T04:13:21.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.519+0000 7fee3ac11740 1 -- start start 2024-09-21T04:13:21.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.519+0000 7fee3ac11740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593d922d9f0 0x5593d96265f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:21.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.519+0000 7fee3ac11740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d92461b0 0x5593d9613b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:21.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.519+0000 7fee3ac11740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593d924e390 0x5593d96140b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:21.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.519+0000 7fee3ac11740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c1aff50 con 0x5593d92461b0 2024-09-21T04:13:21.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.519+0000 7fee3ac11740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c1ad760 con 0x5593d922d9f0 2024-09-21T04:13:21.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.519+0000 7fee3ac11740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c1ab5d0 con 0x5593d924e390 2024-09-21T04:13:21.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.519+0000 7fee31c38640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d92461b0 0x5593d9613b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:21.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.519+0000 7fee31c38640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d92461b0 0x5593d9613b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50210/0 (socket says 172.21.15.64:50210) 2024-09-21T04:13:21.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.519+0000 7fee31c38640 1 -- 172.21.15.64:0/881890659 learned_addr learned my addr 172.21.15.64:0/881890659 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:21.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.520+0000 7fee32439640 1 --2- 172.21.15.64:0/881890659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593d922d9f0 0x5593d96265f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:21.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.520+0000 7fee31c38640 1 -- 172.21.15.64:0/881890659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593d924e390 msgr2=0x5593d96140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:21.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.520+0000 7fee31c38640 1 --2- 172.21.15.64:0/881890659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593d924e390 0x5593d96140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.520+0000 7fee31c38640 1 -- 172.21.15.64:0/881890659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593d922d9f0 msgr2=0x5593d96265f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:21.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.520+0000 7fee31c38640 1 --2- 172.21.15.64:0/881890659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593d922d9f0 0x5593d96265f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.520+0000 7fee31c38640 1 -- 172.21.15.64:0/881890659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593d9625090 con 0x5593d92461b0 2024-09-21T04:13:21.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.520+0000 7fee31c38640 1 --2- 172.21.15.64:0/881890659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d92461b0 0x5593d9613b70 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fee1c0059f0 tx=0x7fee1c00b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d512fc984ec278a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:21.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.520+0000 7fee17fff640 1 -- 172.21.15.64:0/881890659 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c015040 con 0x5593d92461b0 2024-09-21T04:13:21.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.520+0000 7fee17fff640 1 -- 172.21.15.64:0/881890659 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee1c03a070 con 0x5593d92461b0 2024-09-21T04:13:21.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.520+0000 7fee17fff640 1 -- 172.21.15.64:0/881890659 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c01a8d0 con 0x5593d92461b0 2024-09-21T04:13:21.527 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.521+0000 7fee3ac11740 1 -- 172.21.15.64:0/881890659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d92461b0 msgr2=0x5593d9613b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:21.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.521+0000 7fee3ac11740 1 --2- 172.21.15.64:0/881890659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d92461b0 0x5593d9613b70 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fee1c0059f0 tx=0x7fee1c00b850 comp rx=0 tx=0).stop 2024-09-21T04:13:21.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.521+0000 7fee3ac11740 1 -- 172.21.15.64:0/881890659 shutdown_connections 2024-09-21T04:13:21.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.521+0000 7fee3ac11740 1 --2- 172.21.15.64:0/881890659 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593d924e390 0x5593d96140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.521+0000 7fee3ac11740 1 --2- 172.21.15.64:0/881890659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d92461b0 0x5593d9613b70 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.521+0000 7fee3ac11740 1 --2- 172.21.15.64:0/881890659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593d922d9f0 0x5593d96265f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.521+0000 7fee3ac11740 1 -- 172.21.15.64:0/881890659 >> 172.21.15.64:0/881890659 conn(0x5593d9263460 msgr2=0x5593d9625ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:21.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.521+0000 7fee3ac11740 1 -- 172.21.15.64:0/881890659 shutdown_connections 2024-09-21T04:13:21.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.521+0000 7fee3ac11740 1 -- 172.21.15.64:0/881890659 wait complete. 2024-09-21T04:13:21.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.521+0000 7fee3ac11740 1 -- start start 2024-09-21T04:13:21.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.521+0000 7fee3ac11740 1 Processor -- start 2024-09-21T04:13:21.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.522+0000 7fee3ac11740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593d922d9f0 0x5593d9639340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:21.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.522+0000 7fee3ac11740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d92461b0 0x5593d9639880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:21.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.522+0000 7fee3ac11740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593d924e390 0x5593d9639dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:21.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.522+0000 7fee3ac11740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c1abc20 con 0x5593d92461b0 2024-09-21T04:13:21.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.522+0000 7fee3ac11740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c1ad090 con 0x5593d924e390 2024-09-21T04:13:21.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.522+0000 7fee3ac11740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c1ac2f0 con 0x5593d922d9f0 2024-09-21T04:13:21.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.522+0000 7fee3ac11740 10 client.0 _ll_register_callbacks cb 0x7fee2810b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:21.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.522+0000 7fee31c38640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d92461b0 0x5593d9639880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:21.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.522+0000 7fee31c38640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d92461b0 0x5593d9639880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50222/0 (socket says 172.21.15.64:50222) 2024-09-21T04:13:21.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.522+0000 7fee31c38640 1 -- 172.21.15.64:0/4200500939 learned_addr learned my addr 172.21.15.64:0/4200500939 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:21.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.522+0000 7fee32c3a640 1 --2- 172.21.15.64:0/4200500939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593d924e390 0x5593d9639dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:21.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.522+0000 7fee32439640 1 --2- 172.21.15.64:0/4200500939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593d922d9f0 0x5593d9639340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:21.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.522+0000 7fee31c38640 1 -- 172.21.15.64:0/4200500939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593d922d9f0 msgr2=0x5593d9639340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:21.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.522+0000 7fee31c38640 1 --2- 172.21.15.64:0/4200500939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593d922d9f0 0x5593d9639340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.522+0000 7fee31c38640 1 -- 172.21.15.64:0/4200500939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593d924e390 msgr2=0x5593d9639dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:21.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.522+0000 7fee31c38640 1 --2- 172.21.15.64:0/4200500939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593d924e390 0x5593d9639dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.522+0000 7fee31c38640 1 -- 172.21.15.64:0/4200500939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593d96a1ab0 con 0x5593d92461b0 2024-09-21T04:13:21.529 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.522+0000 7fee32c3a640 1 --2- 172.21.15.64:0/4200500939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593d924e390 0x5593d9639dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:13:21.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.522+0000 7fee31c38640 1 --2- 172.21.15.64:0/4200500939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d92461b0 0x5593d9639880 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00fc30 tx=0x7fee1c00b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64c0a5c9b4c2303 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:21.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.522+0000 7fee15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:13:21.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.523+0000 7fee15ffb640 1 -- 172.21.15.64:0/4200500939 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c015040 con 0x5593d92461b0 2024-09-21T04:13:21.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.523+0000 7fee15ffb640 1 -- 172.21.15.64:0/4200500939 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8615+0+0 (secure 0 0 0) 0x7fee1c024680 con 0x5593d92461b0 2024-09-21T04:13:21.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.523+0000 7fee3ac11740 1 -- 172.21.15.64:0/4200500939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee1c03a070 con 0x5593d92461b0 2024-09-21T04:13:21.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.523+0000 7fee15ffb640 1 -- 172.21.15.64:0/4200500939 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fee1c00bad0 con 0x5593d92461b0 2024-09-21T04:13:21.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.523+0000 7fee3ac11740 10 client.17466 fetch_fsmap learned FSMap version 40 2024-09-21T04:13:21.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.523+0000 7fee3ac11740 1 -- 172.21.15.64:0/4200500939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5593d96a2d20 con 0x5593d92461b0 2024-09-21T04:13:21.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.523+0000 7fee15ffb640 1 -- 172.21.15.64:0/4200500939 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee1c03a070 con 0x5593d92461b0 2024-09-21T04:13:21.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.523+0000 7fee15ffb640 1 -- 172.21.15.64:0/4200500939 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7fee1c02c070 con 0x5593d92461b0 2024-09-21T04:13:21.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.523+0000 7fee3ac11740 10 client.17466 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T04:13:21.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.523+0000 7fee3ac11740 10 client.17466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:13:21.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.523+0000 7fee3ac11740 20 client.17466 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:21.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.523+0000 7fee3ac11740 1 --2- 172.21.15.64:0/4200500939 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5593d926f070 0x5593d96a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:21.530 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.523+0000 7fee3ac11740 4 client.17466 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T04:13:21.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.523+0000 7fee3ac11740 1 -- 172.21.15.64:0/4200500939 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5593d958a850 con 0x5593d926f070 2024-09-21T04:13:21.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.524+0000 7fee32439640 1 --2- 172.21.15.64:0/4200500939 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5593d926f070 0x5593d96a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:21.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.524+0000 7fee32439640 1 --2- 172.21.15.64:0/4200500939 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5593d926f070 0x5593d96a53d0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94e09aa3f81103a9 server_cookie=308becf022b20278 in_seq=0 out_seq=0 2024-09-21T04:13:21.531 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.524+0000 7fee15ffb640 10 client.17466 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.524+0000 7fee15ffb640 1 -- 172.21.15.64:0/4200500939 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4262 (crc 0 0 0) 0x5593d958a850 con 0x5593d926f070 2024-09-21T04:13:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.524+0000 7fee15ffb640 10 client.17466 handle_command_reply: tid=0 2024-09-21T04:13:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.524+0000 7fee3ac11740 1 client.17466 shutdown 2024-09-21T04:13:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.525+0000 7fee3ac11740 1 -- 172.21.15.64:0/4200500939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d92461b0 msgr2=0x5593d9639880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.525+0000 7fee3ac11740 1 --2- 172.21.15.64:0/4200500939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d92461b0 0x5593d9639880 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00fc30 tx=0x7fee1c00b510 comp rx=0 tx=0).stop 2024-09-21T04:13:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.525+0000 7fee3ac11740 1 -- 172.21.15.64:0/4200500939 shutdown_connections 2024-09-21T04:13:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.525+0000 7fee3ac11740 1 --2- 172.21.15.64:0/4200500939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593d924e390 0x5593d9639dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.525+0000 7fee3ac11740 1 --2- 172.21.15.64:0/4200500939 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5593d926f070 0x5593d96a53d0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.532 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.525+0000 7fee3ac11740 1 --2- 172.21.15.64:0/4200500939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593d92461b0 0x5593d9639880 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.525+0000 7fee3ac11740 1 --2- 172.21.15.64:0/4200500939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593d922d9f0 0x5593d9639340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.525+0000 7fee3ac11740 1 -- 172.21.15.64:0/4200500939 >> 172.21.15.64:0/4200500939 conn(0x5593d9263460 msgr2=0x5593d9617680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:21.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.525+0000 7fee15ffb640 0 client.17466 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:21.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.525+0000 7fee3ac11740 1 -- 172.21.15.64:0/4200500939 shutdown_connections 2024-09-21T04:13:21.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.525+0000 7fee3ac11740 1 -- 172.21.15.64:0/4200500939 wait complete. 2024-09-21T04:13:21.533 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.525+0000 7fee3ac11740 20 client.17466 trim_cache size 0 max 16384 2024-09-21T04:13:21.533 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:13:21.533 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T04:13:21.533 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": { 2024-09-21T04:13:21.533 INFO:teuthology.orchestra.run.smithi064.stdout: "854413f1-df0a-47b9-aeff-200a0ffbb315": { 2024-09-21T04:13:21.533 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: "tag": "854413f1-df0a-47b9-aeff-200a0ffbb315", 2024-09-21T04:13:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: "options": "recursive,force" 2024-09-21T04:13:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:13:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_stats": { 2024-09-21T04:13:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:13:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:03:59.570174+0000", 2024-09-21T04:13:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: "0ab717a7-36c6-4dca-94e7-af0bdf2aa112": { 2024-09-21T04:13:21.534 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:42.691086+0000", 2024-09-21T04:13:21.535 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:21.535 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:21.535 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:21.535 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:21.535 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:21.535 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:21.535 INFO:teuthology.orchestra.run.smithi064.stdout: "75db1bc5-612e-45aa-bad2-cc90feeaa840": { 2024-09-21T04:13:21.535 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:59.123138+0000", 2024-09-21T04:13:21.535 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:21.535 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:21.535 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:21.535 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:21.535 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:21.535 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:21.535 INFO:teuthology.orchestra.run.smithi064.stdout: "e2c2eed8-9292-4bf6-8a39-13ad27da5acb": { 2024-09-21T04:13:21.536 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:08:47.067932+0000", 2024-09-21T04:13:21.536 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:21.536 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:21.536 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:21.536 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:21.536 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:21.536 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:21.536 INFO:teuthology.orchestra.run.smithi064.stdout: "b52b950c-0f97-49ec-a37c-1a8e04348ce8": { 2024-09-21T04:13:21.536 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:54.003800+0000", 2024-09-21T04:13:21.536 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:21.536 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:21.536 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:21.536 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:21.536 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:21.536 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: "6bde0fe5-f775-44b1-ad0c-f9e596574dc0": { 2024-09-21T04:13:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:06:23.835536+0000", 2024-09-21T04:13:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: "e9242f88-6599-40eb-adf1-bdb3e946a1e0": { 2024-09-21T04:13:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:05:39.744646+0000", 2024-09-21T04:13:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:21.537 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: "854413f1-df0a-47b9-aeff-200a0ffbb315": { 2024-09-21T04:13:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:13:19.921168+0000", 2024-09-21T04:13:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: "424f2ae3-b0e9-4d20-a6cc-7cea3b92b066": { 2024-09-21T04:13:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:13:07.561468+0000", 2024-09-21T04:13:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:21.538 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: "bed40d88-2102-43a6-ac2d-b89d5a2264d0": { 2024-09-21T04:13:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:11.363340+0000", 2024-09-21T04:13:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: "71f3f255-4837-45e9-92bc-4b3abb5c20eb": { 2024-09-21T04:13:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:11:42.378549+0000", 2024-09-21T04:13:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:21.539 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: "dd25622a-26f0-4838-b998-85d0603c83b6": { 2024-09-21T04:13:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:12:24.553513+0000", 2024-09-21T04:13:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: "2c0523fb-aae4-43fa-9568-136a9b640051": { 2024-09-21T04:13:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:09:39.808606+0000", 2024-09-21T04:13:21.540 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: "1b23713d-ea3a-4a86-a87f-65f6ab82671d": { 2024-09-21T04:13:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:10:58.352639+0000", 2024-09-21T04:13:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:13:21.541 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:13:21.541 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:13:21.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.525+0000 7fee3469d640 1 -- 172.21.15.64:0/2677965415 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fee180b66b0 msgr2=0x7fee180b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:21.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.525+0000 7fee3469d640 1 --2- 172.21.15.64:0/2677965415 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fee180b66b0 0x7fee180b8bc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fee24007e50 tx=0x7fee24007600 comp rx=0 tx=0).stop 2024-09-21T04:13:21.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.525+0000 7fee3469d640 1 -- 172.21.15.64:0/2677965415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee2c102ae0 msgr2=0x7fee2c19a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:21.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.525+0000 7fee3469d640 1 --2- 172.21.15.64:0/2677965415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee2c102ae0 0x7fee2c19a4b0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fee28005940 tx=0x7fee280042f0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.525+0000 7fee3469d640 1 -- 172.21.15.64:0/2677965415 shutdown_connections 2024-09-21T04:13:21.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.525+0000 7fee3469d640 1 --2- 172.21.15.64:0/2677965415 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fee180b66b0 0x7fee180b8bc0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.525+0000 7fee3469d640 1 --2- 172.21.15.64:0/2677965415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee2c108d20 0x7fee2c19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.525+0000 7fee3469d640 1 --2- 172.21.15.64:0/2677965415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fee2c1083d0 0x7fee2c19aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.525+0000 7fee3469d640 1 --2- 172.21.15.64:0/2677965415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fee2c102ae0 0x7fee2c19a4b0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:21.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.525+0000 7fee3469d640 1 -- 172.21.15.64:0/2677965415 >> 172.21.15.64:0/2677965415 conn(0x7fee2c079760 msgr2=0x7fee2c101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:21.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.526+0000 7fee3469d640 1 -- 172.21.15.64:0/2677965415 shutdown_connections 2024-09-21T04:13:21.543 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:21.526+0000 7fee3469d640 1 -- 172.21.15.64:0/2677965415 wait complete. 2024-09-21T04:13:21.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:21 smithi131 ceph-mon[35193]: scrub queued for path: / 2024-09-21T04:13:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:21 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:13:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:21 smithi131 ceph-mon[35193]: scrub summary: active paths [/] 2024-09-21T04:13:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:21 smithi131 ceph-mon[35193]: scrub summary: idle+waiting paths [/] 2024-09-21T04:13:21.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:21 smithi131 ceph-mon[35193]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T04:13:21.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:21 smithi119 ceph-mon[34811]: scrub queued for path: / 2024-09-21T04:13:21.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:21 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:13:21.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:21 smithi119 ceph-mon[34811]: scrub summary: active paths [/] 2024-09-21T04:13:21.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:21 smithi119 ceph-mon[34811]: scrub summary: idle+waiting paths [/] 2024-09-21T04:13:21.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:21 smithi119 ceph-mon[34811]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T04:13:22.230 INFO:tasks.cephfs.filesystem:scrub status for tag:854413f1-df0a-47b9-aeff-200a0ffbb315 - {'path': '/', 'tag': '854413f1-df0a-47b9-aeff-200a0ffbb315', 'options': 'recursive,force'} 2024-09-21T04:13:22.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:22 smithi064 ceph-mon[31180]: pgmap v608: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T04:13:22.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:22 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:13:22.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:22 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:13:22.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:22 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:13:22.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:22 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:13:22.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:22 smithi131 ceph-mon[35193]: pgmap v608: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T04:13:22.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:22 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:13:22.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:22 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:13:22.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:22 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:13:22.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:22 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:13:22.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:22 smithi119 ceph-mon[34811]: pgmap v608: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-21T04:13:22.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:22 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:13:22.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:22 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:13:22.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:22 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:13:22.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:22 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:13:23.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:23 smithi131 ceph-mon[35193]: scrub summary: idle 2024-09-21T04:13:23.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:23 smithi119 ceph-mon[34811]: scrub summary: idle 2024-09-21T04:13:23.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:23 smithi064 ceph-mon[31180]: scrub summary: idle 2024-09-21T04:13:24.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:24 smithi131 ceph-mon[35193]: pgmap v610: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 102 B/s wr, 7 op/s 2024-09-21T04:13:24.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:24 smithi119 ceph-mon[34811]: pgmap v610: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 102 B/s wr, 7 op/s 2024-09-21T04:13:24.765 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:24 smithi064 ceph-mon[31180]: pgmap v610: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 102 B/s wr, 7 op/s 2024-09-21T04:13:26.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:26 smithi131 ceph-mon[35193]: pgmap v611: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 102 B/s wr, 7 op/s 2024-09-21T04:13:26.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:26 smithi119 ceph-mon[34811]: pgmap v611: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 102 B/s wr, 7 op/s 2024-09-21T04:13:26.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:26 smithi064 ceph-mon[31180]: pgmap v611: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 102 B/s wr, 7 op/s 2024-09-21T04:13:28.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:28 smithi131 ceph-mon[35193]: pgmap v612: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 102 B/s wr, 7 op/s 2024-09-21T04:13:28.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:28 smithi119 ceph-mon[34811]: pgmap v612: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 102 B/s wr, 7 op/s 2024-09-21T04:13:28.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:28 smithi064 ceph-mon[31180]: pgmap v612: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 102 B/s wr, 7 op/s 2024-09-21T04:13:30.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:30 smithi131 ceph-mon[35193]: pgmap v613: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s 2024-09-21T04:13:30.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:30 smithi119 ceph-mon[34811]: pgmap v613: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s 2024-09-21T04:13:30.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:30 smithi064 ceph-mon[31180]: pgmap v613: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s 2024-09-21T04:13:32.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:32 smithi131 ceph-mon[35193]: pgmap v614: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s 2024-09-21T04:13:32.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:32 smithi119 ceph-mon[34811]: pgmap v614: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s 2024-09-21T04:13:32.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:32 smithi064 ceph-mon[31180]: pgmap v614: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s 2024-09-21T04:13:34.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:34 smithi131 ceph-mon[35193]: pgmap v615: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-09-21T04:13:34.564 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:34 smithi064 ceph-mon[31180]: pgmap v615: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-09-21T04:13:34.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:34 smithi119 ceph-mon[34811]: pgmap v615: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-09-21T04:13:36.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:36 smithi131 ceph-mon[35193]: pgmap v616: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-21T04:13:36.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:36 smithi119 ceph-mon[34811]: pgmap v616: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-21T04:13:36.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:36 smithi064 ceph-mon[31180]: pgmap v616: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-21T04:13:38.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:38 smithi131 ceph-mon[35193]: pgmap v617: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T04:13:38.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:38 smithi119 ceph-mon[34811]: pgmap v617: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T04:13:38.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:38 smithi064 ceph-mon[31180]: pgmap v617: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T04:13:40.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:40 smithi119 ceph-mon[34811]: pgmap v618: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-21T04:13:40.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:40 smithi064 ceph-mon[31180]: pgmap v618: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-21T04:13:40.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:40 smithi131 ceph-mon[35193]: pgmap v618: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-21T04:13:42.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:42 smithi119 ceph-mon[34811]: pgmap v619: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-21T04:13:42.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:42 smithi064 ceph-mon[31180]: pgmap v619: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-21T04:13:42.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:42 smithi131 ceph-mon[35193]: pgmap v619: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-21T04:13:44.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:44 smithi119 ceph-mon[34811]: pgmap v620: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T04:13:44.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:44 smithi064 ceph-mon[31180]: pgmap v620: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T04:13:44.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:44 smithi131 ceph-mon[35193]: pgmap v620: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T04:13:46.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:46 smithi119 ceph-mon[34811]: pgmap v621: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-21T04:13:46.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:46 smithi064 ceph-mon[31180]: pgmap v621: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-21T04:13:46.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:46 smithi131 ceph-mon[35193]: pgmap v621: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-21T04:13:48.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:48 smithi119 ceph-mon[34811]: pgmap v622: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-21T04:13:48.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:48 smithi064 ceph-mon[31180]: pgmap v622: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-21T04:13:48.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:48 smithi131 ceph-mon[35193]: pgmap v622: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-21T04:13:49.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:49 smithi119 ceph-mon[34811]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:13:49.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:49 smithi064 ceph-mon[31180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:13:49.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:49 smithi131 ceph-mon[35193]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:13:50.658 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:50 smithi119 ceph-mon[34811]: pgmap v623: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-21T04:13:50.705 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:50 smithi131 ceph-mon[35193]: pgmap v623: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-21T04:13:50.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:50 smithi064 ceph-mon[31180]: pgmap v623: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-09-21T04:13:52.232 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T04:13:52.509 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:13:52.631 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:52 smithi064 ceph-mon[31180]: pgmap v624: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:13:52.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:52 smithi119 ceph-mon[34811]: pgmap v624: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:13:52.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:52 smithi131 ceph-mon[35193]: pgmap v624: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:13:53.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.148+0000 7f609e104640 1 -- 172.21.15.64:0/3964801475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6098102a90 msgr2=0x7f6098102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:53.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.148+0000 7f609e104640 1 --2- 172.21.15.64:0/3964801475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6098102a90 0x7f6098102ea0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f6088009900 tx=0x7f608802a3f0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.148+0000 7f609e104640 1 -- 172.21.15.64:0/3964801475 shutdown_connections 2024-09-21T04:13:53.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.148+0000 7f609e104640 1 --2- 172.21.15.64:0/3964801475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6098108d20 0x7f609810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.173 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.148+0000 7f609e104640 1 --2- 172.21.15.64:0/3964801475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f60981083d0 0x7f60981087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.148+0000 7f609e104640 1 --2- 172.21.15.64:0/3964801475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6098102a90 0x7f6098102ea0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.148+0000 7f609e104640 1 -- 172.21.15.64:0/3964801475 >> 172.21.15.64:0/3964801475 conn(0x7f6098079750 msgr2=0x7f6098079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:53.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.148+0000 7f609e104640 1 -- 172.21.15.64:0/3964801475 shutdown_connections 2024-09-21T04:13:53.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.148+0000 7f609e104640 1 -- 172.21.15.64:0/3964801475 wait complete. 2024-09-21T04:13:53.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.149+0000 7f609e104640 1 Processor -- start 2024-09-21T04:13:53.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.149+0000 7f609e104640 1 -- start start 2024-09-21T04:13:53.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.149+0000 7f609e104640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6098102a90 0x7f609819b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:53.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.149+0000 7f609e104640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60981083d0 0x7f609819b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:53.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.149+0000 7f609e104640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6098108d20 0x7f609819bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:53.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.149+0000 7f609e104640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6098112990 con 0x7f6098108d20 2024-09-21T04:13:53.174 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.149+0000 7f609e104640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6098112810 con 0x7f60981083d0 2024-09-21T04:13:53.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.149+0000 7f609e104640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6098112b10 con 0x7f6098102a90 2024-09-21T04:13:53.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.150+0000 7f6097fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6098108d20 0x7f609819bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:53.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.150+0000 7f6097fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6098108d20 0x7f609819bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34070/0 (socket says 172.21.15.64:34070) 2024-09-21T04:13:53.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.150+0000 7f6097fff640 1 -- 172.21.15.64:0/2715927693 learned_addr learned my addr 172.21.15.64:0/2715927693 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:53.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.150+0000 7f60977fe640 1 --2- 172.21.15.64:0/2715927693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6098102a90 0x7f609819b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:53.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.150+0000 7f6096ffd640 1 --2- 172.21.15.64:0/2715927693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60981083d0 0x7f609819b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:53.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.150+0000 7f6097fff640 1 -- 172.21.15.64:0/2715927693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6098102a90 msgr2=0x7f609819b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:53.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.150+0000 7f6097fff640 1 --2- 172.21.15.64:0/2715927693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6098102a90 0x7f609819b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.150+0000 7f6097fff640 1 -- 172.21.15.64:0/2715927693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60981083d0 msgr2=0x7f609819b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:53.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.150+0000 7f6097fff640 1 --2- 172.21.15.64:0/2715927693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60981083d0 0x7f609819b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.150+0000 7f6097fff640 1 -- 172.21.15.64:0/2715927693 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60981a6460 con 0x7f6098108d20 2024-09-21T04:13:53.175 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.150+0000 7f6097fff640 1 --2- 172.21.15.64:0/2715927693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6098108d20 0x7f609819bc60 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f609000eaa0 tx=0x7f609000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:53.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.150+0000 7f6094ff9640 1 -- 172.21.15.64:0/2715927693 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6090004d20 con 0x7f6098108d20 2024-09-21T04:13:53.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.151+0000 7f6094ff9640 1 -- 172.21.15.64:0/2715927693 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6090005020 con 0x7f6098108d20 2024-09-21T04:13:53.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.151+0000 7f6094ff9640 1 -- 172.21.15.64:0/2715927693 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60900116b0 con 0x7f6098108d20 2024-09-21T04:13:53.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.151+0000 7f609e104640 1 -- 172.21.15.64:0/2715927693 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60981a6750 con 0x7f6098108d20 2024-09-21T04:13:53.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.151+0000 7f609e104640 1 -- 172.21.15.64:0/2715927693 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60981a6c40 con 0x7f6098108d20 2024-09-21T04:13:53.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.154+0000 7f6094ff9640 1 -- 172.21.15.64:0/2715927693 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f60900051d0 con 0x7f6098108d20 2024-09-21T04:13:53.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.154+0000 7f60a4678740 1 Processor -- start 2024-09-21T04:13:53.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.154+0000 7f60a4678740 1 -- start start 2024-09-21T04:13:53.176 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.154+0000 7f60a4678740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643c6437070 0x5643c67e2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:53.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.154+0000 7f60a4678740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c640e1b0 0x5643c67e2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:53.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.154+0000 7f60a4678740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5643c6416390 0x5643c67e3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:53.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.154+0000 7f60a4678740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5643c6754e50 con 0x5643c640e1b0 2024-09-21T04:13:53.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.154+0000 7f60a4678740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5643c6744280 con 0x5643c6437070 2024-09-21T04:13:53.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.154+0000 7f60a4678740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5643c6748090 con 0x5643c6416390 2024-09-21T04:13:53.177 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.155+0000 7f60977fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c640e1b0 0x5643c67e2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:53.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.155+0000 7f60977fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c640e1b0 0x5643c67e2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34078/0 (socket says 172.21.15.64:34078) 2024-09-21T04:13:53.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.155+0000 7f60977fe640 1 -- 172.21.15.64:0/3246791515 learned_addr learned my addr 172.21.15.64:0/3246791515 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:53.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.155+0000 7f6094ff9640 1 --2- 172.21.15.64:0/2715927693 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f60800b6680 0x7f60800b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:53.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.155+0000 7f6096ffd640 1 --2- 172.21.15.64:0/3246791515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643c6437070 0x5643c67e2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:53.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.155+0000 7f6096ffd640 1 --2- 172.21.15.64:0/3246791515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5643c6416390 0x5643c67e3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:53.178 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.155+0000 7f6097fff640 1 --2- 172.21.15.64:0/2715927693 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f60800b6680 0x7f60800b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:53.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.155+0000 7f60977fe640 1 -- 172.21.15.64:0/3246791515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5643c6416390 msgr2=0x5643c67e3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:53.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.155+0000 7f60977fe640 1 --2- 172.21.15.64:0/3246791515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5643c6416390 0x5643c67e3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.155+0000 7f60977fe640 1 -- 172.21.15.64:0/3246791515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643c6437070 msgr2=0x5643c67e2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:53.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.155+0000 7f60977fe640 1 --2- 172.21.15.64:0/3246791515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643c6437070 0x5643c67e2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.155+0000 7f60977fe640 1 -- 172.21.15.64:0/3246791515 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643c64a2650 con 0x5643c640e1b0 2024-09-21T04:13:53.179 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.155+0000 7f6096ffd640 1 --2- 172.21.15.64:0/3246791515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5643c6416390 0x5643c67e3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:13:53.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.155+0000 7f6094ff9640 1 -- 172.21.15.64:0/2715927693 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8615+0+0 (secure 0 0 0) 0x7f60900e7920 con 0x7f6098108d20 2024-09-21T04:13:53.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.155+0000 7f60977fe640 1 --2- 172.21.15.64:0/3246791515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c640e1b0 0x5643c67e2bc0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f6088009900 tx=0x7f6088032df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da37d3fda5e3dc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:53.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.155+0000 7f6097fff640 1 --2- 172.21.15.64:0/2715927693 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f60800b6680 0x7f60800b8b90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6090052f20 tx=0x7f609007a3c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:53.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.156+0000 7f607dffb640 1 -- 172.21.15.64:0/3246791515 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6088040070 con 0x5643c640e1b0 2024-09-21T04:13:53.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.156+0000 7f607dffb640 1 -- 172.21.15.64:0/3246791515 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6088005db0 con 0x5643c640e1b0 2024-09-21T04:13:53.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.156+0000 7f607dffb640 1 -- 172.21.15.64:0/3246791515 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6088035040 con 0x5643c640e1b0 2024-09-21T04:13:53.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.156+0000 7f60a4678740 1 -- 172.21.15.64:0/3246791515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c640e1b0 msgr2=0x5643c67e2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:53.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.156+0000 7f60a4678740 1 --2- 172.21.15.64:0/3246791515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c640e1b0 0x5643c67e2bc0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f6088009900 tx=0x7f6088032df0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.156+0000 7f60a4678740 1 -- 172.21.15.64:0/3246791515 shutdown_connections 2024-09-21T04:13:53.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.156+0000 7f60a4678740 1 --2- 172.21.15.64:0/3246791515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5643c6416390 0x5643c67e3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.180 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.156+0000 7f60a4678740 1 --2- 172.21.15.64:0/3246791515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c640e1b0 0x5643c67e2bc0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.157+0000 7f60a4678740 1 --2- 172.21.15.64:0/3246791515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643c6437070 0x5643c67e2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.157+0000 7f60a4678740 1 -- 172.21.15.64:0/3246791515 >> 172.21.15.64:0/3246791515 conn(0x5643c642b460 msgr2=0x5643c67e0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:53.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.157+0000 7f60a4678740 1 -- 172.21.15.64:0/3246791515 shutdown_connections 2024-09-21T04:13:53.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.157+0000 7f60a4678740 1 -- 172.21.15.64:0/3246791515 wait complete. 2024-09-21T04:13:53.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.157+0000 7f60a4678740 1 -- start start 2024-09-21T04:13:53.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.157+0000 7f60a4678740 1 Processor -- start 2024-09-21T04:13:53.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.158+0000 7f60a4678740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c640e1b0 0x5643c67edd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:53.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.158+0000 7f60a4678740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643c6416390 0x5643c67ee290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:53.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.158+0000 7f60a4678740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5643c6437070 0x5643c67fa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:53.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.158+0000 7f60a4678740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5643c65a5700 con 0x5643c640e1b0 2024-09-21T04:13:53.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.158+0000 7f60a4678740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5643c66df110 con 0x5643c6416390 2024-09-21T04:13:53.181 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.158+0000 7f60a4678740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5643c67f0e00 con 0x5643c6437070 2024-09-21T04:13:53.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.158+0000 7f60a4678740 10 client.0 _ll_register_callbacks cb 0x5643c66f8990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:53.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.158+0000 7f6096ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c640e1b0 0x5643c67edd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:53.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.158+0000 7f6096ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c640e1b0 0x5643c67edd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34086/0 (socket says 172.21.15.64:34086) 2024-09-21T04:13:53.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.158+0000 7f6096ffd640 1 -- 172.21.15.64:0/3178311262 learned_addr learned my addr 172.21.15.64:0/3178311262 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:53.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.158+0000 7f60977fe640 1 --2- 172.21.15.64:0/3178311262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643c6416390 0x5643c67ee290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:53.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.158+0000 7f6096ffd640 1 --2- 172.21.15.64:0/3178311262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5643c6437070 0x5643c67fa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:53.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.158+0000 7f6096ffd640 1 -- 172.21.15.64:0/3178311262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5643c6437070 msgr2=0x5643c67fa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:53.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.158+0000 7f6096ffd640 1 --2- 172.21.15.64:0/3178311262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5643c6437070 0x5643c67fa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.158+0000 7f6096ffd640 1 -- 172.21.15.64:0/3178311262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643c6416390 msgr2=0x5643c67ee290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:53.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.158+0000 7f6096ffd640 1 --2- 172.21.15.64:0/3178311262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643c6416390 0x5643c67ee290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.182 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.158+0000 7f6096ffd640 1 -- 172.21.15.64:0/3178311262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643c670cc40 con 0x5643c640e1b0 2024-09-21T04:13:53.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.159+0000 7f6096ffd640 1 --2- 172.21.15.64:0/3178311262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c640e1b0 0x5643c67edd50 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f608c010ec0 tx=0x7f608c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d955533d7dc245 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:53.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.159+0000 7f6076ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:13:53.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.159+0000 7f6076ffd640 1 -- 172.21.15.64:0/3178311262 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c005020 con 0x5643c640e1b0 2024-09-21T04:13:53.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.159+0000 7f6076ffd640 1 -- 172.21.15.64:0/3178311262 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8615+0+0 (secure 0 0 0) 0x7f608c0051c0 con 0x5643c640e1b0 2024-09-21T04:13:53.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.159+0000 7f60a4678740 1 -- 172.21.15.64:0/3178311262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6088005db0 con 0x5643c640e1b0 2024-09-21T04:13:53.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.159+0000 7f6076ffd640 1 -- 172.21.15.64:0/3178311262 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f608c0053c0 con 0x5643c640e1b0 2024-09-21T04:13:53.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.159+0000 7f60a4678740 10 client.17478 fetch_fsmap learned FSMap version 41 2024-09-21T04:13:53.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.159+0000 7f60a4678740 1 -- 172.21.15.64:0/3178311262 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5643c67fcb30 con 0x5643c640e1b0 2024-09-21T04:13:53.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.159+0000 7f6076ffd640 1 -- 172.21.15.64:0/3178311262 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6088005db0 con 0x5643c640e1b0 2024-09-21T04:13:53.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.160+0000 7f6076ffd640 1 -- 172.21.15.64:0/3178311262 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f608c020660 con 0x5643c640e1b0 2024-09-21T04:13:53.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.160+0000 7f60a4678740 10 client.17478 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T04:13:53.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.160+0000 7f60a4678740 10 client.17478 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:13:53.183 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.160+0000 7f60a4678740 20 client.17478 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:53.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.160+0000 7f60a4678740 1 --2- 172.21.15.64:0/3178311262 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5643c63f59f0 0x5643c6800060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:53.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.160+0000 7f60a4678740 4 client.17478 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:13:53.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.160+0000 7f60a4678740 1 -- 172.21.15.64:0/3178311262 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643c6756da0 con 0x5643c63f59f0 2024-09-21T04:13:53.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.160+0000 7f60977fe640 1 --2- 172.21.15.64:0/3178311262 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5643c63f59f0 0x5643c6800060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:53.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.160+0000 7f6076ffd640 10 client.17478 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:53.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.161+0000 7f60977fe640 1 --2- 172.21.15.64:0/3178311262 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5643c63f59f0 0x5643c6800060 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ba176521c39cff0 server_cookie=4ec35f96be64fc64 in_seq=0 out_seq=0 2024-09-21T04:13:53.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.161+0000 7f6076ffd640 1 -- 172.21.15.64:0/3178311262 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5643c6756da0 con 0x5643c63f59f0 2024-09-21T04:13:53.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.161+0000 7f6076ffd640 10 client.17478 handle_command_reply: tid=0 2024-09-21T04:13:53.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.161+0000 7f60a4678740 1 client.17478 shutdown 2024-09-21T04:13:53.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.162+0000 7f60a4678740 1 -- 172.21.15.64:0/3178311262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c640e1b0 msgr2=0x5643c67edd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:53.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.162+0000 7f60a4678740 1 --2- 172.21.15.64:0/3178311262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c640e1b0 0x5643c67edd50 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f608c010ec0 tx=0x7f608c0046c0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.184 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.162+0000 7f60a4678740 1 -- 172.21.15.64:0/3178311262 shutdown_connections 2024-09-21T04:13:53.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.162+0000 7f60a4678740 1 --2- 172.21.15.64:0/3178311262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5643c6437070 0x5643c67fa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.162+0000 7f60a4678740 1 --2- 172.21.15.64:0/3178311262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643c6416390 0x5643c67ee290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.162+0000 7f60a4678740 1 --2- 172.21.15.64:0/3178311262 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5643c63f59f0 0x5643c6800060 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.162+0000 7f60a4678740 1 --2- 172.21.15.64:0/3178311262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c640e1b0 0x5643c67edd50 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.162+0000 7f6076ffd640 0 client.17478 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:53.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.162+0000 7f60a4678740 1 -- 172.21.15.64:0/3178311262 >> 172.21.15.64:0/3178311262 conn(0x5643c642b460 msgr2=0x5643c67e5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:53.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.162+0000 7f60a4678740 1 -- 172.21.15.64:0/3178311262 shutdown_connections 2024-09-21T04:13:53.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.162+0000 7f60a4678740 1 -- 172.21.15.64:0/3178311262 wait complete. 2024-09-21T04:13:53.185 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.162+0000 7f60a4678740 20 client.17478 trim_cache size 0 max 16384 2024-09-21T04:13:53.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.180+0000 7f60a4678740 1 Processor -- start 2024-09-21T04:13:53.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.180+0000 7f60a4678740 1 -- start start 2024-09-21T04:13:53.190 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.180+0000 7f60a4678740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c63f59f0 0x5643c67ee210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:53.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.180+0000 7f60a4678740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643c640e1b0 0x5643c67f9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:53.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.180+0000 7f60a4678740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5643c6416390 0x5643c67eaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:53.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.180+0000 7f60a4678740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f60981aff70 con 0x5643c63f59f0 2024-09-21T04:13:53.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.180+0000 7f60a4678740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f60981ad7b0 con 0x5643c640e1b0 2024-09-21T04:13:53.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.180+0000 7f60a4678740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f60981ab620 con 0x5643c6416390 2024-09-21T04:13:53.191 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.181+0000 7f6096ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c63f59f0 0x5643c67ee210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:53.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.181+0000 7f6096ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c63f59f0 0x5643c67ee210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34088/0 (socket says 172.21.15.64:34088) 2024-09-21T04:13:53.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.181+0000 7f6096ffd640 1 -- 172.21.15.64:0/200625353 learned_addr learned my addr 172.21.15.64:0/200625353 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:53.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.181+0000 7f60977fe640 1 --2- 172.21.15.64:0/200625353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643c640e1b0 0x5643c67f9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:53.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.181+0000 7f6096ffd640 1 --2- 172.21.15.64:0/200625353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5643c6416390 0x5643c67eaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:53.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.181+0000 7f6096ffd640 1 -- 172.21.15.64:0/200625353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5643c6416390 msgr2=0x5643c67eaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:53.192 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.181+0000 7f6096ffd640 1 --2- 172.21.15.64:0/200625353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5643c6416390 0x5643c67eaf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.181+0000 7f6096ffd640 1 -- 172.21.15.64:0/200625353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643c640e1b0 msgr2=0x5643c67f9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:53.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.181+0000 7f6096ffd640 1 --2- 172.21.15.64:0/200625353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643c640e1b0 0x5643c67f9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.181+0000 7f6096ffd640 1 -- 172.21.15.64:0/200625353 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643c67ec250 con 0x5643c63f59f0 2024-09-21T04:13:53.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.181+0000 7f6096ffd640 1 --2- 172.21.15.64:0/200625353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c63f59f0 0x5643c67ee210 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f608c001eb0 tx=0x7f608c00fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90a8fa7049359fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:53.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.181+0000 7f607d7fa640 1 -- 172.21.15.64:0/200625353 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c020460 con 0x5643c63f59f0 2024-09-21T04:13:53.193 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.181+0000 7f607d7fa640 1 -- 172.21.15.64:0/200625353 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f608c014030 con 0x5643c63f59f0 2024-09-21T04:13:53.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.182+0000 7f607d7fa640 1 -- 172.21.15.64:0/200625353 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c033db0 con 0x5643c63f59f0 2024-09-21T04:13:53.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.182+0000 7f60a4678740 1 -- 172.21.15.64:0/200625353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c63f59f0 msgr2=0x5643c67ee210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:53.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.182+0000 7f60a4678740 1 --2- 172.21.15.64:0/200625353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c63f59f0 0x5643c67ee210 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f608c001eb0 tx=0x7f608c00fc20 comp rx=0 tx=0).stop 2024-09-21T04:13:53.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.182+0000 7f60a4678740 1 -- 172.21.15.64:0/200625353 shutdown_connections 2024-09-21T04:13:53.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.182+0000 7f60a4678740 1 --2- 172.21.15.64:0/200625353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5643c6416390 0x5643c67eaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.194 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.182+0000 7f60a4678740 1 --2- 172.21.15.64:0/200625353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643c640e1b0 0x5643c67f9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.182+0000 7f60a4678740 1 --2- 172.21.15.64:0/200625353 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c63f59f0 0x5643c67ee210 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.182+0000 7f60a4678740 1 -- 172.21.15.64:0/200625353 >> 172.21.15.64:0/200625353 conn(0x5643c642b460 msgr2=0x5643c67edb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:53.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.183+0000 7f60a4678740 1 -- 172.21.15.64:0/200625353 shutdown_connections 2024-09-21T04:13:53.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.183+0000 7f60a4678740 1 -- 172.21.15.64:0/200625353 wait complete. 2024-09-21T04:13:53.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.183+0000 7f60a4678740 1 -- start start 2024-09-21T04:13:53.195 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.183+0000 7f60a4678740 1 Processor -- start 2024-09-21T04:13:53.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.184+0000 7f60a4678740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c63f59f0 0x5643c67f2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:53.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.184+0000 7f60a4678740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643c640e1b0 0x5643c67f2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:53.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.184+0000 7f60a4678740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5643c6416390 0x5643c67f2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:53.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.184+0000 7f60a4678740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f60981abc70 con 0x5643c63f59f0 2024-09-21T04:13:53.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.184+0000 7f60a4678740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f60981ad0e0 con 0x5643c640e1b0 2024-09-21T04:13:53.196 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.184+0000 7f60a4678740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f60981ac340 con 0x5643c6416390 2024-09-21T04:13:53.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.184+0000 7f60a4678740 10 client.0 _ll_register_callbacks cb 0x7f608c010ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:53.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.184+0000 7f6096ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c63f59f0 0x5643c67f2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:53.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.184+0000 7f6096ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c63f59f0 0x5643c67f2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34102/0 (socket says 172.21.15.64:34102) 2024-09-21T04:13:53.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.184+0000 7f6096ffd640 1 -- 172.21.15.64:0/4209981368 learned_addr learned my addr 172.21.15.64:0/4209981368 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:53.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.184+0000 7f60977fe640 1 --2- 172.21.15.64:0/4209981368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643c640e1b0 0x5643c67f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:53.197 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.184+0000 7f6096ffd640 1 --2- 172.21.15.64:0/4209981368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5643c6416390 0x5643c67f2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:53.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.184+0000 7f6096ffd640 1 -- 172.21.15.64:0/4209981368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5643c6416390 msgr2=0x5643c67f2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:53.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.184+0000 7f6096ffd640 1 --2- 172.21.15.64:0/4209981368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5643c6416390 0x5643c67f2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.184+0000 7f6096ffd640 1 -- 172.21.15.64:0/4209981368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643c640e1b0 msgr2=0x5643c67f2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:53.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.184+0000 7f6096ffd640 1 --2- 172.21.15.64:0/4209981368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643c640e1b0 0x5643c67f2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.184+0000 7f6096ffd640 1 -- 172.21.15.64:0/4209981368 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643c6869c60 con 0x5643c63f59f0 2024-09-21T04:13:53.198 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.184+0000 7f6096ffd640 1 --2- 172.21.15.64:0/4209981368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c63f59f0 0x5643c67f2220 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f608c005790 tx=0x7f608c0056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acde6dc24a78be96 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:53.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.184+0000 7f6076ffd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:13:53.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.185+0000 7f6076ffd640 1 -- 172.21.15.64:0/4209981368 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c0336d0 con 0x5643c63f59f0 2024-09-21T04:13:53.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.185+0000 7f6076ffd640 1 -- 172.21.15.64:0/4209981368 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8615+0+0 (secure 0 0 0) 0x7f608c033870 con 0x5643c63f59f0 2024-09-21T04:13:53.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.185+0000 7f60a4678740 1 -- 172.21.15.64:0/4209981368 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f608c014030 con 0x5643c63f59f0 2024-09-21T04:13:53.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.185+0000 7f6076ffd640 1 -- 172.21.15.64:0/4209981368 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f608c042410 con 0x5643c63f59f0 2024-09-21T04:13:53.199 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.185+0000 7f60a4678740 10 client.17484 fetch_fsmap learned FSMap version 41 2024-09-21T04:13:53.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.185+0000 7f60a4678740 1 -- 172.21.15.64:0/4209981368 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5643c686aef0 con 0x5643c63f59f0 2024-09-21T04:13:53.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.185+0000 7f6076ffd640 1 -- 172.21.15.64:0/4209981368 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f608c014030 con 0x5643c63f59f0 2024-09-21T04:13:53.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.185+0000 7f6076ffd640 1 -- 172.21.15.64:0/4209981368 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f608c028680 con 0x5643c63f59f0 2024-09-21T04:13:53.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.185+0000 7f60a4678740 10 client.17484 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T04:13:53.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.185+0000 7f60a4678740 10 client.17484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:13:53.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.186+0000 7f60a4678740 20 client.17484 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:53.200 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.186+0000 7f60a4678740 1 --2- 172.21.15.64:0/4209981368 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5643c6437070 0x5643c686d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:53.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.186+0000 7f60a4678740 4 client.17484 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T04:13:53.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.186+0000 7f60a4678740 1 -- 172.21.15.64:0/4209981368 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5643c6752850 con 0x5643c6437070 2024-09-21T04:13:53.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.186+0000 7f60977fe640 1 --2- 172.21.15.64:0/4209981368 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5643c6437070 0x5643c686d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:53.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.186+0000 7f60977fe640 1 --2- 172.21.15.64:0/4209981368 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5643c6437070 0x5643c686d510 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1076a75179ee787 server_cookie=895d8ba1938059a1 in_seq=0 out_seq=0 2024-09-21T04:13:53.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.186+0000 7f6076ffd640 10 client.17484 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:53.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.187+0000 7f6076ffd640 1 -- 172.21.15.64:0/4209981368 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4066 (crc 0 0 0) 0x5643c6752850 con 0x5643c6437070 2024-09-21T04:13:53.201 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.187+0000 7f6076ffd640 10 client.17484 handle_command_reply: tid=0 2024-09-21T04:13:53.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.187+0000 7f60a4678740 1 client.17484 shutdown 2024-09-21T04:13:53.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.187+0000 7f60a4678740 1 -- 172.21.15.64:0/4209981368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c63f59f0 msgr2=0x5643c67f2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:53.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.187+0000 7f60a4678740 1 --2- 172.21.15.64:0/4209981368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c63f59f0 0x5643c67f2220 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f608c005790 tx=0x7f608c0056e0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.187+0000 7f60a4678740 1 -- 172.21.15.64:0/4209981368 shutdown_connections 2024-09-21T04:13:53.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.187+0000 7f60a4678740 1 --2- 172.21.15.64:0/4209981368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5643c6416390 0x5643c67f2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.202 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.187+0000 7f60a4678740 1 --2- 172.21.15.64:0/4209981368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5643c640e1b0 0x5643c67f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.187+0000 7f60a4678740 1 --2- 172.21.15.64:0/4209981368 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x5643c6437070 0x5643c686d510 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.187+0000 7f60a4678740 1 --2- 172.21.15.64:0/4209981368 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5643c63f59f0 0x5643c67f2220 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.187+0000 7f60a4678740 1 -- 172.21.15.64:0/4209981368 >> 172.21.15.64:0/4209981368 conn(0x5643c642b460 msgr2=0x5643c67e36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:53.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.187+0000 7f6076ffd640 0 client.17484 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:53.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.187+0000 7f60a4678740 1 -- 172.21.15.64:0/4209981368 shutdown_connections 2024-09-21T04:13:53.203 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.187+0000 7f60a4678740 1 -- 172.21.15.64:0/4209981368 wait complete. 2024-09-21T04:13:53.204 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.187+0000 7f60a4678740 20 client.17484 trim_cache size 0 max 16384 2024-09-21T04:13:53.204 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:13:53.204 INFO:teuthology.orchestra.run.smithi064.stdout: "status": "no active scrubs running", 2024-09-21T04:13:53.205 INFO:teuthology.orchestra.run.smithi064.stdout: "scrubs": {}, 2024-09-21T04:13:53.205 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_stats": { 2024-09-21T04:13:53.205 INFO:teuthology.orchestra.run.smithi064.stdout: "cb0c7943-c962-4f34-86e2-8e678a340f05": { 2024-09-21T04:13:53.205 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:03:59.570174+0000", 2024-09-21T04:13:53.205 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:53.205 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:53.206 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:53.206 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:53.206 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:53.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:53.206 INFO:teuthology.orchestra.run.smithi064.stdout: "0ab717a7-36c6-4dca-94e7-af0bdf2aa112": { 2024-09-21T04:13:53.207 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:42.691086+0000", 2024-09-21T04:13:53.207 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:53.207 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:53.207 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:53.207 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:53.207 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:53.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:53.208 INFO:teuthology.orchestra.run.smithi064.stdout: "75db1bc5-612e-45aa-bad2-cc90feeaa840": { 2024-09-21T04:13:53.208 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:59.123138+0000", 2024-09-21T04:13:53.208 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:53.208 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:53.208 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:53.208 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:53.208 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:53.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:53.209 INFO:teuthology.orchestra.run.smithi064.stdout: "e2c2eed8-9292-4bf6-8a39-13ad27da5acb": { 2024-09-21T04:13:53.209 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:08:47.067932+0000", 2024-09-21T04:13:53.209 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:53.209 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:53.209 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:53.209 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:53.209 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:53.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:53.210 INFO:teuthology.orchestra.run.smithi064.stdout: "b52b950c-0f97-49ec-a37c-1a8e04348ce8": { 2024-09-21T04:13:53.210 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:04:54.003800+0000", 2024-09-21T04:13:53.210 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:53.210 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:53.210 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:53.210 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:53.210 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:53.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:53.211 INFO:teuthology.orchestra.run.smithi064.stdout: "6bde0fe5-f775-44b1-ad0c-f9e596574dc0": { 2024-09-21T04:13:53.211 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:06:23.835536+0000", 2024-09-21T04:13:53.211 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:53.211 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:53.211 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:53.211 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:53.212 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:53.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:53.212 INFO:teuthology.orchestra.run.smithi064.stdout: "e9242f88-6599-40eb-adf1-bdb3e946a1e0": { 2024-09-21T04:13:53.212 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:05:39.744646+0000", 2024-09-21T04:13:53.212 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:53.212 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:53.212 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:53.213 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:53.213 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:53.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:53.213 INFO:teuthology.orchestra.run.smithi064.stdout: "854413f1-df0a-47b9-aeff-200a0ffbb315": { 2024-09-21T04:13:53.213 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:13:19.921168+0000", 2024-09-21T04:13:53.213 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:53.214 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:53.214 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:53.214 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:53.214 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:53.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:53.214 INFO:teuthology.orchestra.run.smithi064.stdout: "424f2ae3-b0e9-4d20-a6cc-7cea3b92b066": { 2024-09-21T04:13:53.215 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:13:07.561468+0000", 2024-09-21T04:13:53.215 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:53.215 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:53.215 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:53.215 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:53.215 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:53.215 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:53.215 INFO:teuthology.orchestra.run.smithi064.stdout: "bed40d88-2102-43a6-ac2d-b89d5a2264d0": { 2024-09-21T04:13:53.215 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:07:11.363340+0000", 2024-09-21T04:13:53.215 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:53.215 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:53.216 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:53.216 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:53.216 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:53.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:53.216 INFO:teuthology.orchestra.run.smithi064.stdout: "71f3f255-4837-45e9-92bc-4b3abb5c20eb": { 2024-09-21T04:13:53.216 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:11:42.378549+0000", 2024-09-21T04:13:53.216 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:53.216 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:53.216 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:53.216 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:53.216 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:53.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:53.217 INFO:teuthology.orchestra.run.smithi064.stdout: "dd25622a-26f0-4838-b998-85d0603c83b6": { 2024-09-21T04:13:53.217 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:12:24.553513+0000", 2024-09-21T04:13:53.217 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:53.217 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:53.217 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:53.217 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:53.217 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:53.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:53.217 INFO:teuthology.orchestra.run.smithi064.stdout: "2c0523fb-aae4-43fa-9568-136a9b640051": { 2024-09-21T04:13:53.217 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:09:39.808606+0000", 2024-09-21T04:13:53.217 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:53.217 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:53.217 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:53.218 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:53.218 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:53.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:13:53.218 INFO:teuthology.orchestra.run.smithi064.stdout: "1b23713d-ea3a-4a86-a87f-65f6ab82671d": { 2024-09-21T04:13:53.218 INFO:teuthology.orchestra.run.smithi064.stdout: "start_time": "2024-09-21T04:10:58.352639+0000", 2024-09-21T04:13:53.218 INFO:teuthology.orchestra.run.smithi064.stdout: "path": "/", 2024-09-21T04:13:53.218 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_started": 0, 2024-09-21T04:13:53.218 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_passed": 0, 2024-09-21T04:13:53.218 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_failed": 0, 2024-09-21T04:13:53.218 INFO:teuthology.orchestra.run.smithi064.stdout: "uninline_skipped": 0 2024-09-21T04:13:53.218 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:13:53.219 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:13:53.219 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:13:53.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.188+0000 7f609e104640 1 -- 172.21.15.64:0/2715927693 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f60800b6680 msgr2=0x7f60800b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:53.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.188+0000 7f609e104640 1 --2- 172.21.15.64:0/2715927693 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f60800b6680 0x7f60800b8b90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6090052f20 tx=0x7f609007a3c0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.188+0000 7f609e104640 1 -- 172.21.15.64:0/2715927693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6098108d20 msgr2=0x7f609819bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:53.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.188+0000 7f609e104640 1 --2- 172.21.15.64:0/2715927693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6098108d20 0x7f609819bc60 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f609000eaa0 tx=0x7f609000eea0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.188+0000 7f609e104640 1 -- 172.21.15.64:0/2715927693 shutdown_connections 2024-09-21T04:13:53.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.188+0000 7f609e104640 1 --2- 172.21.15.64:0/2715927693 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f60800b6680 0x7f60800b8b90 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.188+0000 7f609e104640 1 --2- 172.21.15.64:0/2715927693 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6098108d20 0x7f609819bc60 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.188+0000 7f609e104640 1 --2- 172.21.15.64:0/2715927693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60981083d0 0x7f609819b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.220 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.188+0000 7f609e104640 1 --2- 172.21.15.64:0/2715927693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6098102a90 0x7f609819b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:53.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.188+0000 7f609e104640 1 -- 172.21.15.64:0/2715927693 >> 172.21.15.64:0/2715927693 conn(0x7f6098079750 msgr2=0x7f60981013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:53.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.188+0000 7f609e104640 1 -- 172.21.15.64:0/2715927693 shutdown_connections 2024-09-21T04:13:53.221 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:53.188+0000 7f609e104640 1 -- 172.21.15.64:0/2715927693 wait complete. 2024-09-21T04:13:53.990 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T04:13:53.990 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:13:54.250 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:13:54.674 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:54 smithi064 ceph-mon[31180]: pgmap v625: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T04:13:54.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:54 smithi119 ceph-mon[34811]: pgmap v625: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T04:13:54.790 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.764+0000 7f04e4fd0640 1 -- 172.21.15.64:0/2601999666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e0104880 msgr2=0x7f04e0104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:54.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.764+0000 7f04e4fd0640 1 --2- 172.21.15.64:0/2601999666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e0104880 0x7f04e0104c90 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009900 tx=0x7f04d002a410 comp rx=0 tx=0).stop 2024-09-21T04:13:54.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.764+0000 7f04e4fd0640 1 -- 172.21.15.64:0/2601999666 shutdown_connections 2024-09-21T04:13:54.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.764+0000 7f04e4fd0640 1 --2- 172.21.15.64:0/2601999666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04e010de30 0x7f04e0110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:54.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.764+0000 7f04e4fd0640 1 --2- 172.21.15.64:0/2601999666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04e01052b0 0x7f04e010d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:54.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.764+0000 7f04e4fd0640 1 --2- 172.21.15.64:0/2601999666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e0104880 0x7f04e0104c90 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:54.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.764+0000 7f04e4fd0640 1 -- 172.21.15.64:0/2601999666 >> 172.21.15.64:0/2601999666 conn(0x7f04e0079730 msgr2=0x7f04e0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:54.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.764+0000 7f04e4fd0640 1 -- 172.21.15.64:0/2601999666 shutdown_connections 2024-09-21T04:13:54.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.764+0000 7f04e4fd0640 1 -- 172.21.15.64:0/2601999666 wait complete. 2024-09-21T04:13:54.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.765+0000 7f04e4fd0640 1 Processor -- start 2024-09-21T04:13:54.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.765+0000 7f04e4fd0640 1 -- start start 2024-09-21T04:13:54.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.765+0000 7f04e4fd0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e0104880 0x7f04e019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:54.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.765+0000 7f04e4fd0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04e01052b0 0x7f04e019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:54.791 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.765+0000 7f04e4fd0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04e010de30 0x7f04e019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.765+0000 7f04e4fd0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f04e0113a70 con 0x7f04e0104880 2024-09-21T04:13:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.765+0000 7f04e4fd0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01138f0 con 0x7f04e01052b0 2024-09-21T04:13:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.765+0000 7f04e4fd0640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f04e0113bf0 con 0x7f04e010de30 2024-09-21T04:13:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.766+0000 7f04de59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e0104880 0x7f04e019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.766+0000 7f04de59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e0104880 0x7f04e019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34116/0 (socket says 172.21.15.64:34116) 2024-09-21T04:13:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.766+0000 7f04de59c640 1 -- 172.21.15.64:0/1481895978 learned_addr learned my addr 172.21.15.64:0/1481895978 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.766+0000 7f04ded9d640 1 --2- 172.21.15.64:0/1481895978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04e010de30 0x7f04e019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.766+0000 7f04ddd9b640 1 --2- 172.21.15.64:0/1481895978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04e01052b0 0x7f04e019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.766+0000 7f04de59c640 1 -- 172.21.15.64:0/1481895978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04e010de30 msgr2=0x7f04e019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.766+0000 7f04de59c640 1 --2- 172.21.15.64:0/1481895978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04e010de30 0x7f04e019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.766+0000 7f04de59c640 1 -- 172.21.15.64:0/1481895978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04e01052b0 msgr2=0x7f04e019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.766+0000 7f04de59c640 1 --2- 172.21.15.64:0/1481895978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04e01052b0 0x7f04e019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.766+0000 7f04de59c640 1 -- 172.21.15.64:0/1481895978 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e01a0410 con 0x7f04e0104880 2024-09-21T04:13:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.766+0000 7f04ded9d640 1 --2- 172.21.15.64:0/1481895978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04e010de30 0x7f04e019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:13:54.792 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.766+0000 7f04ddd9b640 1 --2- 172.21.15.64:0/1481895978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04e01052b0 0x7f04e019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:13:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.766+0000 7f04de59c640 1 --2- 172.21.15.64:0/1481895978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e0104880 0x7f04e019a4a0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f04d0032fd0 tx=0x7f04d0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.766+0000 7f04cf7fe640 1 -- 172.21.15.64:0/1481895978 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d003f030 con 0x7f04e0104880 2024-09-21T04:13:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.766+0000 7f04cf7fe640 1 -- 172.21.15.64:0/1481895978 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04d0004490 con 0x7f04e0104880 2024-09-21T04:13:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.766+0000 7f04e4fd0640 1 -- 172.21.15.64:0/1481895978 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e01a06a0 con 0x7f04e0104880 2024-09-21T04:13:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.766+0000 7f04cf7fe640 1 -- 172.21.15.64:0/1481895978 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0035070 con 0x7f04e0104880 2024-09-21T04:13:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.767+0000 7f04e4fd0640 1 -- 172.21.15.64:0/1481895978 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e01a6e60 con 0x7f04e0104880 2024-09-21T04:13:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.768+0000 7f04e4fd0640 1 -- 172.21.15.64:0/1481895978 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e006b4c0 con 0x7f04e0104880 2024-09-21T04:13:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.770+0000 7f04cf7fe640 1 -- 172.21.15.64:0/1481895978 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f04d0042030 con 0x7f04e0104880 2024-09-21T04:13:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.771+0000 7f04cf7fe640 1 --2- 172.21.15.64:0/1481895978 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f04c80b6700 0x7f04c80b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.771+0000 7f04ddd9b640 1 --2- 172.21.15.64:0/1481895978 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f04c80b6700 0x7f04c80b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.771+0000 7f04cf7fe640 1 -- 172.21.15.64:0/1481895978 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8615+0+0 (secure 0 0 0) 0x7f04d01080f0 con 0x7f04e0104880 2024-09-21T04:13:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.771+0000 7f04ddd9b640 1 --2- 172.21.15.64:0/1481895978 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f04c80b6700 0x7f04c80b8c10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f04d4005d00 tx=0x7f04d4005c70 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:54.793 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:54.774+0000 7f04cf7fe640 1 -- 172.21.15.64:0/1481895978 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f04d00ce5e0 con 0x7f04e0104880 2024-09-21T04:13:54.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:54 smithi131 ceph-mon[35193]: pgmap v625: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-21T04:13:55.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:55.006+0000 7f04e4fd0640 1 -- 172.21.15.64:0/1481895978 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04e0108fc0 con 0x7f04e0104880 2024-09-21T04:13:55.009 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:55.007+0000 7f04cf7fe640 1 -- 172.21.15.64:0/1481895978 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11916 (secure 0 0 0) 0x7f04d00d29c0 con 0x7f04e0104880 2024-09-21T04:13:55.010 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:13:55.010 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":41,"btime":"2024-09-21T04:13:48:314220+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14805,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/97995119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":97995119},{"type":"v1","addr":"172.21.15.64:6839","nonce":97995119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/390891068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":390891068},{"type":"v1","addr":"172.21.15.64:6841","nonce":390891068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:02:03.354744+0000","modified":"2024-09-21T04:13:47.553828+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24710,"mds_1":24680,"mds_2":24661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24634":{"gid":24634,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6839/782190424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782190424},{"type":"v1","addr":"172.21.15.131:6839","nonce":782190424}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24680":{"gid":24680,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24710":{"gid":24710,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6839/1319793636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1319793636},{"type":"v1","addr":"172.21.15.119:6839","nonce":1319793636}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24710,"qdb_cluster":[24710,24680,24661]},"id":1}]} 2024-09-21T04:13:55.010 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 41 2024-09-21T04:13:55.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:55.009+0000 7f04e4fd0640 1 -- 172.21.15.64:0/1481895978 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f04c80b6700 msgr2=0x7f04c80b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:55.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:55.009+0000 7f04e4fd0640 1 --2- 172.21.15.64:0/1481895978 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f04c80b6700 0x7f04c80b8c10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f04d4005d00 tx=0x7f04d4005c70 comp rx=0 tx=0).stop 2024-09-21T04:13:55.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:55.009+0000 7f04e4fd0640 1 -- 172.21.15.64:0/1481895978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e0104880 msgr2=0x7f04e019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:55.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:55.009+0000 7f04e4fd0640 1 --2- 172.21.15.64:0/1481895978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e0104880 0x7f04e019a4a0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f04d0032fd0 tx=0x7f04d0004270 comp rx=0 tx=0).stop 2024-09-21T04:13:55.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:55.009+0000 7f04e4fd0640 1 -- 172.21.15.64:0/1481895978 shutdown_connections 2024-09-21T04:13:55.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:55.009+0000 7f04e4fd0640 1 --2- 172.21.15.64:0/1481895978 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f04c80b6700 0x7f04c80b8c10 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:55.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:55.009+0000 7f04e4fd0640 1 --2- 172.21.15.64:0/1481895978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04e010de30 0x7f04e019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:55.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:55.009+0000 7f04e4fd0640 1 --2- 172.21.15.64:0/1481895978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04e01052b0 0x7f04e019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:55.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:55.009+0000 7f04e4fd0640 1 --2- 172.21.15.64:0/1481895978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e0104880 0x7f04e019a4a0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:55.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:55.009+0000 7f04e4fd0640 1 -- 172.21.15.64:0/1481895978 >> 172.21.15.64:0/1481895978 conn(0x7f04e0079730 msgr2=0x7f04e0100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:55.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:55.010+0000 7f04e4fd0640 1 -- 172.21.15.64:0/1481895978 shutdown_connections 2024-09-21T04:13:55.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:55.010+0000 7f04e4fd0640 1 -- 172.21.15.64:0/1481895978 wait complete. 2024-09-21T04:13:55.613 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:55 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1481895978' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:13:55.642 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T04:13:55.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:55 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1481895978' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:13:55.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:55 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1481895978' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:13:55.903 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:13:56.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.382+0000 7ff315046640 1 -- 172.21.15.64:0/3716594658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3101049d0 msgr2=0x7ff310104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:56.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.382+0000 7ff315046640 1 --2- 172.21.15.64:0/3716594658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3101049d0 0x7ff310104de0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7ff300009900 tx=0x7ff30002a410 comp rx=0 tx=0).stop 2024-09-21T04:13:56.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.382+0000 7ff315046640 1 -- 172.21.15.64:0/3716594658 shutdown_connections 2024-09-21T04:13:56.400 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.382+0000 7ff315046640 1 --2- 172.21.15.64:0/3716594658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff31010de30 0x7ff310110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.382+0000 7ff315046640 1 --2- 172.21.15.64:0/3716594658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff310105320 0x7ff31010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.382+0000 7ff315046640 1 --2- 172.21.15.64:0/3716594658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3101049d0 0x7ff310104de0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.382+0000 7ff315046640 1 -- 172.21.15.64:0/3716594658 >> 172.21.15.64:0/3716594658 conn(0x7ff310079740 msgr2=0x7ff310079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.382+0000 7ff315046640 1 -- 172.21.15.64:0/3716594658 shutdown_connections 2024-09-21T04:13:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.382+0000 7ff315046640 1 -- 172.21.15.64:0/3716594658 wait complete. 2024-09-21T04:13:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.383+0000 7ff315046640 1 Processor -- start 2024-09-21T04:13:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.383+0000 7ff315046640 1 -- start start 2024-09-21T04:13:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.384+0000 7ff315046640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3101049d0 0x7ff310102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.384+0000 7ff315046640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff310105320 0x7ff310102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.384+0000 7ff315046640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff31010de30 0x7ff310103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.384+0000 7ff315046640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff310113af0 con 0x7ff31010de30 2024-09-21T04:13:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.384+0000 7ff315046640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff310113970 con 0x7ff310105320 2024-09-21T04:13:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.384+0000 7ff315046640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff310113c70 con 0x7ff3101049d0 2024-09-21T04:13:56.401 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.384+0000 7ff30f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff31010de30 0x7ff310103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.384+0000 7ff30f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff31010de30 0x7ff310103190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34122/0 (socket says 172.21.15.64:34122) 2024-09-21T04:13:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.384+0000 7ff30f59e640 1 -- 172.21.15.64:0/1631257278 learned_addr learned my addr 172.21.15.64:0/1631257278 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.384+0000 7ff30e59c640 1 --2- 172.21.15.64:0/1631257278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff310105320 0x7ff310102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.384+0000 7ff30ed9d640 1 --2- 172.21.15.64:0/1631257278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3101049d0 0x7ff310102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.384+0000 7ff30f59e640 1 -- 172.21.15.64:0/1631257278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3101049d0 msgr2=0x7ff310102710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.384+0000 7ff30f59e640 1 --2- 172.21.15.64:0/1631257278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3101049d0 0x7ff310102710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.384+0000 7ff30f59e640 1 -- 172.21.15.64:0/1631257278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff310105320 msgr2=0x7ff310102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.384+0000 7ff30f59e640 1 --2- 172.21.15.64:0/1631257278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff310105320 0x7ff310102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.384+0000 7ff30f59e640 1 -- 172.21.15.64:0/1631257278 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3101a8810 con 0x7ff31010de30 2024-09-21T04:13:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.384+0000 7ff30ed9d640 1 --2- 172.21.15.64:0/1631257278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3101049d0 0x7ff310102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:13:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.385+0000 7ff30f59e640 1 --2- 172.21.15.64:0/1631257278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff31010de30 0x7ff310103190 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7ff30800cce0 tx=0x7ff30800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.385+0000 7ff2f7fff640 1 -- 172.21.15.64:0/1631257278 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30800f450 con 0x7ff31010de30 2024-09-21T04:13:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.385+0000 7ff315046640 1 -- 172.21.15.64:0/1631257278 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3101a8b00 con 0x7ff31010de30 2024-09-21T04:13:56.402 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.385+0000 7ff2f7fff640 1 -- 172.21.15.64:0/1631257278 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff30800faa0 con 0x7ff31010de30 2024-09-21T04:13:56.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.385+0000 7ff2f7fff640 1 -- 172.21.15.64:0/1631257278 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308018a80 con 0x7ff31010de30 2024-09-21T04:13:56.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.385+0000 7ff315046640 1 -- 172.21.15.64:0/1631257278 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3101a9040 con 0x7ff31010de30 2024-09-21T04:13:56.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.388+0000 7ff31b5ba740 1 Processor -- start 2024-09-21T04:13:56.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.388+0000 7ff2f7fff640 1 -- 172.21.15.64:0/1631257278 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7ff30800fc50 con 0x7ff31010de30 2024-09-21T04:13:56.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.388+0000 7ff31b5ba740 1 -- start start 2024-09-21T04:13:56.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.388+0000 7ff31b5ba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55557a781070 0x55557ab286a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:56.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.388+0000 7ff31b5ba740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a7581b0 0x55557ab28be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:56.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.388+0000 7ff31b5ba740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55557a760390 0x55557ab29120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:56.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.388+0000 7ff31b5ba740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55557aa9eea0 con 0x55557a7581b0 2024-09-21T04:13:56.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.388+0000 7ff31b5ba740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55557aa8e2d0 con 0x55557a781070 2024-09-21T04:13:56.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.388+0000 7ff31b5ba740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55557aa920e0 con 0x55557a760390 2024-09-21T04:13:56.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.388+0000 7ff30ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a7581b0 0x55557ab28be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:56.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.389+0000 7ff30ed9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a7581b0 0x55557ab28be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34130/0 (socket says 172.21.15.64:34130) 2024-09-21T04:13:56.403 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.389+0000 7ff30ed9d640 1 -- 172.21.15.64:0/1066082816 learned_addr learned my addr 172.21.15.64:0/1066082816 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:56.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.389+0000 7ff30e59c640 1 --2- 172.21.15.64:0/1066082816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55557a760390 0x55557ab29120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:56.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.389+0000 7ff30e59c640 1 --2- 172.21.15.64:0/1066082816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55557a781070 0x55557ab286a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:56.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.389+0000 7ff30ed9d640 1 -- 172.21.15.64:0/1066082816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55557a760390 msgr2=0x55557ab29120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:56.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.389+0000 7ff30ed9d640 1 --2- 172.21.15.64:0/1066082816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55557a760390 0x55557ab29120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.389+0000 7ff30ed9d640 1 -- 172.21.15.64:0/1066082816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55557a781070 msgr2=0x55557ab286a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:56.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.389+0000 7ff30ed9d640 1 --2- 172.21.15.64:0/1066082816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55557a781070 0x55557ab286a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.389+0000 7ff30ed9d640 1 -- 172.21.15.64:0/1066082816 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55557a7ec650 con 0x55557a7581b0 2024-09-21T04:13:56.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.389+0000 7ff2f7fff640 1 --2- 172.21.15.64:0/1631257278 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff2f80b6680 0x7ff2f80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:56.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.389+0000 7ff30e59c640 1 --2- 172.21.15.64:0/1631257278 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff2f80b6680 0x7ff2f80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:56.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.389+0000 7ff30ed9d640 1 --2- 172.21.15.64:0/1066082816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a7581b0 0x55557ab28be0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7ff300009900 tx=0x7ff300031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25702c07391bf35 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:56.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.389+0000 7ff2f7fff640 1 -- 172.21.15.64:0/1631257278 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8615+0+0 (secure 0 0 0) 0x7ff3080e7c40 con 0x7ff31010de30 2024-09-21T04:13:56.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.389+0000 7ff2f57fa640 1 -- 172.21.15.64:0/1066082816 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30003f070 con 0x55557a7581b0 2024-09-21T04:13:56.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.389+0000 7ff2f57fa640 1 -- 172.21.15.64:0/1066082816 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3000316a0 con 0x55557a7581b0 2024-09-21T04:13:56.404 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.389+0000 7ff2f57fa640 1 -- 172.21.15.64:0/1066082816 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300035070 con 0x55557a7581b0 2024-09-21T04:13:56.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.390+0000 7ff31b5ba740 1 -- 172.21.15.64:0/1066082816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a7581b0 msgr2=0x55557ab28be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:56.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.390+0000 7ff31b5ba740 1 --2- 172.21.15.64:0/1066082816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a7581b0 0x55557ab28be0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7ff300009900 tx=0x7ff300031e60 comp rx=0 tx=0).stop 2024-09-21T04:13:56.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.390+0000 7ff30e59c640 1 --2- 172.21.15.64:0/1631257278 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff2f80b6680 0x7ff2f80b8b90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff310079fd0 tx=0x7ff3040046c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:56.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.390+0000 7ff31b5ba740 1 -- 172.21.15.64:0/1066082816 shutdown_connections 2024-09-21T04:13:56.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.390+0000 7ff31b5ba740 1 --2- 172.21.15.64:0/1066082816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55557a760390 0x55557ab29120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.390+0000 7ff31b5ba740 1 --2- 172.21.15.64:0/1066082816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a7581b0 0x55557ab28be0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.390+0000 7ff31b5ba740 1 --2- 172.21.15.64:0/1066082816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55557a781070 0x55557ab286a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.390+0000 7ff31b5ba740 1 -- 172.21.15.64:0/1066082816 >> 172.21.15.64:0/1066082816 conn(0x55557a775460 msgr2=0x55557ab26590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:56.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.390+0000 7ff31b5ba740 1 -- 172.21.15.64:0/1066082816 shutdown_connections 2024-09-21T04:13:56.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.390+0000 7ff31b5ba740 1 -- 172.21.15.64:0/1066082816 wait complete. 2024-09-21T04:13:56.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.391+0000 7ff31b5ba740 1 -- start start 2024-09-21T04:13:56.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.391+0000 7ff31b5ba740 1 Processor -- start 2024-09-21T04:13:56.405 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.391+0000 7ff31b5ba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55557a7581b0 0x55557ab33dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:56.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.391+0000 7ff31b5ba740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55557a760390 0x55557ab34300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:56.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.391+0000 7ff31b5ba740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a781070 0x55557ab40140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:56.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.391+0000 7ff31b5ba740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55557a8ef700 con 0x55557a781070 2024-09-21T04:13:56.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.391+0000 7ff31b5ba740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55557aa29160 con 0x55557a7581b0 2024-09-21T04:13:56.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.391+0000 7ff31b5ba740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55557ab36e20 con 0x55557a760390 2024-09-21T04:13:56.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.391+0000 7ff31b5ba740 10 client.0 _ll_register_callbacks cb 0x55557aa429e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:56.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.391+0000 7ff30f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a781070 0x55557ab40140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:56.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.391+0000 7ff30f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a781070 0x55557ab40140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34134/0 (socket says 172.21.15.64:34134) 2024-09-21T04:13:56.406 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.391+0000 7ff30f59e640 1 -- 172.21.15.64:0/2489057416 learned_addr learned my addr 172.21.15.64:0/2489057416 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:56.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.392+0000 7ff30e59c640 1 --2- 172.21.15.64:0/2489057416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55557a760390 0x55557ab34300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:56.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.392+0000 7ff30ed9d640 1 --2- 172.21.15.64:0/2489057416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55557a7581b0 0x55557ab33dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:56.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.392+0000 7ff30f59e640 1 -- 172.21.15.64:0/2489057416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55557a760390 msgr2=0x55557ab34300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:56.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.392+0000 7ff30f59e640 1 --2- 172.21.15.64:0/2489057416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55557a760390 0x55557ab34300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.392+0000 7ff30f59e640 1 -- 172.21.15.64:0/2489057416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55557a7581b0 msgr2=0x55557ab33dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:56.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.392+0000 7ff30f59e640 1 --2- 172.21.15.64:0/2489057416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55557a7581b0 0x55557ab33dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.392+0000 7ff30f59e640 1 -- 172.21.15.64:0/2489057416 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55557aa56c90 con 0x55557a781070 2024-09-21T04:13:56.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.392+0000 7ff30e59c640 1 --2- 172.21.15.64:0/2489057416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55557a760390 0x55557ab34300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:13:56.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.392+0000 7ff30ed9d640 1 --2- 172.21.15.64:0/2489057416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55557a7581b0 0x55557ab33dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:13:56.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.392+0000 7ff30f59e640 1 --2- 172.21.15.64:0/2489057416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a781070 0x55557ab40140 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7ff3080aa000 tx=0x7ff3080b7380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e185157dca7b283d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:56.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.392+0000 7ff2f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:13:56.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.392+0000 7ff2f2ff5640 1 -- 172.21.15.64:0/2489057416 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308096030 con 0x55557a781070 2024-09-21T04:13:56.407 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.392+0000 7ff2f2ff5640 1 -- 172.21.15.64:0/2489057416 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8615+0+0 (secure 0 0 0) 0x7ff3080b21c0 con 0x55557a781070 2024-09-21T04:13:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.392+0000 7ff31b5ba740 1 -- 172.21.15.64:0/2489057416 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3000316a0 con 0x55557a781070 2024-09-21T04:13:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.392+0000 7ff2f2ff5640 1 -- 172.21.15.64:0/2489057416 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff3080b2020 con 0x55557a781070 2024-09-21T04:13:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.392+0000 7ff31b5ba740 10 client.17502 fetch_fsmap learned FSMap version 41 2024-09-21T04:13:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.392+0000 7ff31b5ba740 1 -- 172.21.15.64:0/2489057416 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55557ab42bc0 con 0x55557a781070 2024-09-21T04:13:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.392+0000 7ff2f2ff5640 1 -- 172.21.15.64:0/2489057416 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3000316a0 con 0x55557a781070 2024-09-21T04:13:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.393+0000 7ff2f2ff5640 1 -- 172.21.15.64:0/2489057416 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7ff308020030 con 0x55557a781070 2024-09-21T04:13:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.393+0000 7ff31b5ba740 10 client.17502 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T04:13:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.393+0000 7ff31b5ba740 10 client.17502 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:13:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.393+0000 7ff31b5ba740 20 client.17502 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.393+0000 7ff31b5ba740 1 --2- 172.21.15.64:0/2489057416 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55557a73f9f0 0x55557ab46270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.393+0000 7ff31b5ba740 4 client.17502 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:13:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.393+0000 7ff31b5ba740 1 -- 172.21.15.64:0/2489057416 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55557aaa0df0 con 0x55557a73f9f0 2024-09-21T04:13:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.393+0000 7ff30ed9d640 1 --2- 172.21.15.64:0/2489057416 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55557a73f9f0 0x55557ab46270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.394+0000 7ff2f2ff5640 10 client.17502 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:13:56.408 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.394+0000 7ff30ed9d640 1 --2- 172.21.15.64:0/2489057416 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55557a73f9f0 0x55557ab46270 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c6633dcfa1546877 server_cookie=753dd6af08bd5c17 in_seq=0 out_seq=0 2024-09-21T04:13:56.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.394+0000 7ff2f2ff5640 1 -- 172.21.15.64:0/2489057416 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55557aaa0df0 con 0x55557a73f9f0 2024-09-21T04:13:56.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.394+0000 7ff2f2ff5640 10 client.17502 handle_command_reply: tid=0 2024-09-21T04:13:56.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.395+0000 7ff31b5ba740 1 client.17502 shutdown 2024-09-21T04:13:56.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.395+0000 7ff31b5ba740 1 -- 172.21.15.64:0/2489057416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a781070 msgr2=0x55557ab40140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:56.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.395+0000 7ff31b5ba740 1 --2- 172.21.15.64:0/2489057416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a781070 0x55557ab40140 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7ff3080aa000 tx=0x7ff3080b7380 comp rx=0 tx=0).stop 2024-09-21T04:13:56.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.395+0000 7ff31b5ba740 1 -- 172.21.15.64:0/2489057416 shutdown_connections 2024-09-21T04:13:56.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.395+0000 7ff31b5ba740 1 --2- 172.21.15.64:0/2489057416 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55557a73f9f0 0x55557ab46270 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.395+0000 7ff31b5ba740 1 --2- 172.21.15.64:0/2489057416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a781070 0x55557ab40140 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.395+0000 7ff31b5ba740 1 --2- 172.21.15.64:0/2489057416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55557a760390 0x55557ab34300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.395+0000 7ff31b5ba740 1 --2- 172.21.15.64:0/2489057416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55557a7581b0 0x55557ab33dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.395+0000 7ff31b5ba740 1 -- 172.21.15.64:0/2489057416 >> 172.21.15.64:0/2489057416 conn(0x55557a775460 msgr2=0x55557ab2b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:56.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.395+0000 7ff2f2ff5640 0 client.17502 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:13:56.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.395+0000 7ff31b5ba740 1 -- 172.21.15.64:0/2489057416 shutdown_connections 2024-09-21T04:13:56.409 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.395+0000 7ff31b5ba740 1 -- 172.21.15.64:0/2489057416 wait complete. 2024-09-21T04:13:56.410 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.395+0000 7ff31b5ba740 20 client.17502 trim_cache size 0 max 16384 2024-09-21T04:13:56.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.408+0000 7ff31b5ba740 1 Processor -- start 2024-09-21T04:13:56.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.408+0000 7ff31b5ba740 1 -- start start 2024-09-21T04:13:56.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.408+0000 7ff31b5ba740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55557a7581b0 0x55557ab2beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:56.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.408+0000 7ff31b5ba740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a760390 0x55557ab30fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:56.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.408+0000 7ff31b5ba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55557a781070 0x55557ab314f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:56.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.408+0000 7ff31b5ba740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff3101b4380 con 0x55557a760390 2024-09-21T04:13:56.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.408+0000 7ff31b5ba740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff3101b1bc0 con 0x55557a781070 2024-09-21T04:13:56.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.408+0000 7ff31b5ba740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff3101afa30 con 0x55557a7581b0 2024-09-21T04:13:56.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.409+0000 7ff30e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a760390 0x55557ab30fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:56.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.409+0000 7ff30e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a760390 0x55557ab30fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34140/0 (socket says 172.21.15.64:34140) 2024-09-21T04:13:56.412 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.409+0000 7ff30e59c640 1 -- 172.21.15.64:0/3246567686 learned_addr learned my addr 172.21.15.64:0/3246567686 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:56.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.409+0000 7ff30ed9d640 1 --2- 172.21.15.64:0/3246567686 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55557a7581b0 0x55557ab2beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:56.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.409+0000 7ff30f59e640 1 --2- 172.21.15.64:0/3246567686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55557a781070 0x55557ab314f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:56.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.409+0000 7ff30e59c640 1 -- 172.21.15.64:0/3246567686 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55557a7581b0 msgr2=0x55557ab2beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:56.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.409+0000 7ff30e59c640 1 --2- 172.21.15.64:0/3246567686 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55557a7581b0 0x55557ab2beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.409+0000 7ff30e59c640 1 -- 172.21.15.64:0/3246567686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55557a781070 msgr2=0x55557ab314f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:56.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.409+0000 7ff30e59c640 1 --2- 172.21.15.64:0/3246567686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55557a781070 0x55557ab314f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.409+0000 7ff30e59c640 1 -- 172.21.15.64:0/3246567686 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55557ab36990 con 0x55557a760390 2024-09-21T04:13:56.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.409+0000 7ff30f59e640 1 --2- 172.21.15.64:0/3246567686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55557a781070 0x55557ab314f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:13:56.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.409+0000 7ff30e59c640 1 --2- 172.21.15.64:0/3246567686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a760390 0x55557ab30fb0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7ff304013f70 tx=0x7ff304015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6ec250ef756752 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:56.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.409+0000 7ff2f4ff9640 1 -- 172.21.15.64:0/3246567686 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304027030 con 0x55557a760390 2024-09-21T04:13:56.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.409+0000 7ff2f4ff9640 1 -- 172.21.15.64:0/3246567686 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff30401c040 con 0x55557a760390 2024-09-21T04:13:56.413 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.409+0000 7ff2f4ff9640 1 -- 172.21.15.64:0/3246567686 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30402b7e0 con 0x55557a760390 2024-09-21T04:13:56.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.410+0000 7ff31b5ba740 1 -- 172.21.15.64:0/3246567686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a760390 msgr2=0x55557ab30fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:56.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.410+0000 7ff31b5ba740 1 --2- 172.21.15.64:0/3246567686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a760390 0x55557ab30fb0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7ff304013f70 tx=0x7ff304015000 comp rx=0 tx=0).stop 2024-09-21T04:13:56.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.410+0000 7ff31b5ba740 1 -- 172.21.15.64:0/3246567686 shutdown_connections 2024-09-21T04:13:56.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.410+0000 7ff31b5ba740 1 --2- 172.21.15.64:0/3246567686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55557a781070 0x55557ab314f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.410+0000 7ff31b5ba740 1 --2- 172.21.15.64:0/3246567686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a760390 0x55557ab30fb0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.417 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.410+0000 7ff31b5ba740 1 --2- 172.21.15.64:0/3246567686 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55557a7581b0 0x55557ab2beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.410+0000 7ff31b5ba740 1 -- 172.21.15.64:0/3246567686 >> 172.21.15.64:0/3246567686 conn(0x55557a775460 msgr2=0x55557ab2b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:56.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.410+0000 7ff31b5ba740 1 -- 172.21.15.64:0/3246567686 shutdown_connections 2024-09-21T04:13:56.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.410+0000 7ff31b5ba740 1 -- 172.21.15.64:0/3246567686 wait complete. 2024-09-21T04:13:56.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.411+0000 7ff31b5ba740 1 -- start start 2024-09-21T04:13:56.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.411+0000 7ff31b5ba740 1 Processor -- start 2024-09-21T04:13:56.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.411+0000 7ff31b5ba740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55557a7581b0 0x55557ab398d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:56.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.411+0000 7ff31b5ba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55557a760390 0x55557ab39e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:56.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.411+0000 7ff31b5ba740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a781070 0x55557abaec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:56.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.411+0000 7ff31b5ba740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff3101b0080 con 0x55557a781070 2024-09-21T04:13:56.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.411+0000 7ff31b5ba740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff3101b14f0 con 0x55557a760390 2024-09-21T04:13:56.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.411+0000 7ff31b5ba740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff3101b0750 con 0x55557a7581b0 2024-09-21T04:13:56.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.411+0000 7ff31b5ba740 10 client.0 _ll_register_callbacks cb 0x7ff308018400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:56.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.411+0000 7ff30f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a781070 0x55557abaec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:56.418 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.411+0000 7ff30f59e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a781070 0x55557abaec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34154/0 (socket says 172.21.15.64:34154) 2024-09-21T04:13:56.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.411+0000 7ff30f59e640 1 -- 172.21.15.64:0/4025203700 learned_addr learned my addr 172.21.15.64:0/4025203700 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:56.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.411+0000 7ff30e59c640 1 --2- 172.21.15.64:0/4025203700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55557a760390 0x55557ab39e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:56.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.412+0000 7ff30ed9d640 1 --2- 172.21.15.64:0/4025203700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55557a7581b0 0x55557ab398d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:56.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.412+0000 7ff30f59e640 1 -- 172.21.15.64:0/4025203700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55557a7581b0 msgr2=0x55557ab398d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:56.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.412+0000 7ff30f59e640 1 --2- 172.21.15.64:0/4025203700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55557a7581b0 0x55557ab398d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.412+0000 7ff30f59e640 1 -- 172.21.15.64:0/4025203700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55557a760390 msgr2=0x55557ab39e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:56.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.412+0000 7ff30f59e640 1 --2- 172.21.15.64:0/4025203700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55557a760390 0x55557ab39e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.412+0000 7ff30f59e640 1 -- 172.21.15.64:0/4025203700 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55557abaf2e0 con 0x55557a781070 2024-09-21T04:13:56.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.412+0000 7ff30e59c640 1 --2- 172.21.15.64:0/4025203700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55557a760390 0x55557ab39e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:13:56.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.412+0000 7ff30ed9d640 1 --2- 172.21.15.64:0/4025203700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55557a7581b0 0x55557ab398d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:13:56.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.412+0000 7ff30f59e640 1 --2- 172.21.15.64:0/4025203700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a781070 0x55557abaec70 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7ff3080e7f60 tx=0x7ff3080a9810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8914e71e525180d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:56.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.412+0000 7ff2f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:13:56.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.412+0000 7ff2f2ff5640 1 -- 172.21.15.64:0/4025203700 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308096030 con 0x55557a781070 2024-09-21T04:13:56.419 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.412+0000 7ff2f2ff5640 1 -- 172.21.15.64:0/4025203700 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8615+0+0 (secure 0 0 0) 0x7ff3080c3be0 con 0x55557a781070 2024-09-21T04:13:56.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.412+0000 7ff31b5ba740 1 -- 172.21.15.64:0/4025203700 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff30401c040 con 0x55557a781070 2024-09-21T04:13:56.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.412+0000 7ff2f2ff5640 1 -- 172.21.15.64:0/4025203700 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff3080ad1a0 con 0x55557a781070 2024-09-21T04:13:56.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.412+0000 7ff31b5ba740 10 client.17508 fetch_fsmap learned FSMap version 41 2024-09-21T04:13:56.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.412+0000 7ff31b5ba740 1 -- 172.21.15.64:0/4025203700 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55557abb04b0 con 0x55557a781070 2024-09-21T04:13:56.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.412+0000 7ff2f2ff5640 1 -- 172.21.15.64:0/4025203700 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff30401c040 con 0x55557a781070 2024-09-21T04:13:56.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.413+0000 7ff2f2ff5640 1 -- 172.21.15.64:0/4025203700 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7ff3080e5a80 con 0x55557a781070 2024-09-21T04:13:56.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.413+0000 7ff31b5ba740 10 client.17508 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T04:13:56.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.413+0000 7ff31b5ba740 10 client.17508 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T04:13:56.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.413+0000 7ff31b5ba740 20 client.17508 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:56.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.413+0000 7ff31b5ba740 1 --2- 172.21.15.64:0/4025203700 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55557a714da0 0x55557abb2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:56.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.413+0000 7ff31b5ba740 4 client.17508 mds_command: new command op to 24661 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:13:56.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.413+0000 7ff31b5ba740 1 -- 172.21.15.64:0/4025203700 --> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55557aa9c8a0 con 0x55557a714da0 2024-09-21T04:13:56.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.413+0000 7ff30ed9d640 1 --2- 172.21.15.64:0/4025203700 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55557a714da0 0x55557abb2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:56.420 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.413+0000 7ff2f2ff5640 10 client.17508 ms_handle_connect on v2:172.21.15.131:6838/782190424 2024-09-21T04:13:56.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.413+0000 7ff30ed9d640 1 --2- 172.21.15.64:0/4025203700 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55557a714da0 0x55557abb2b60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c9a9843e09d3bd6 server_cookie=f235c3578685925 in_seq=0 out_seq=0 2024-09-21T04:13:56.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.414+0000 7ff2f2ff5640 1 -- 172.21.15.64:0/4025203700 <== mds.2 v2:172.21.15.131:6838/782190424 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55557aa9c8a0 con 0x55557a714da0 2024-09-21T04:13:56.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.414+0000 7ff2f2ff5640 10 client.17508 handle_command_reply: tid=0 2024-09-21T04:13:56.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.414+0000 7ff31b5ba740 1 client.17508 shutdown 2024-09-21T04:13:56.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.414+0000 7ff31b5ba740 1 -- 172.21.15.64:0/4025203700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a781070 msgr2=0x55557abaec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:56.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.414+0000 7ff31b5ba740 1 --2- 172.21.15.64:0/4025203700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a781070 0x55557abaec70 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7ff3080e7f60 tx=0x7ff3080a9810 comp rx=0 tx=0).stop 2024-09-21T04:13:56.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.414+0000 7ff31b5ba740 1 -- 172.21.15.64:0/4025203700 shutdown_connections 2024-09-21T04:13:56.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.414+0000 7ff31b5ba740 1 --2- 172.21.15.64:0/4025203700 >> [v2:172.21.15.131:6838/782190424,v1:172.21.15.131:6839/782190424] conn(0x55557a714da0 0x55557abb2b60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.414+0000 7ff31b5ba740 1 --2- 172.21.15.64:0/4025203700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55557a781070 0x55557abaec70 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.414+0000 7ff31b5ba740 1 --2- 172.21.15.64:0/4025203700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55557a760390 0x55557ab39e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.414+0000 7ff31b5ba740 1 --2- 172.21.15.64:0/4025203700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55557a7581b0 0x55557ab398d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.414+0000 7ff31b5ba740 1 -- 172.21.15.64:0/4025203700 >> 172.21.15.64:0/4025203700 conn(0x55557a775460 msgr2=0x55557ab35270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:56.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.414+0000 7ff2f2ff5640 0 client.17508 ms_handle_reset on v2:172.21.15.131:6838/782190424 2024-09-21T04:13:56.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.414+0000 7ff31b5ba740 1 -- 172.21.15.64:0/4025203700 shutdown_connections 2024-09-21T04:13:56.421 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.414+0000 7ff31b5ba740 1 -- 172.21.15.64:0/4025203700 wait complete. 2024-09-21T04:13:56.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.414+0000 7ff31b5ba740 20 client.17508 trim_cache size 0 max 16384 2024-09-21T04:13:56.422 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:13:56.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.415+0000 7ff315046640 1 -- 172.21.15.64:0/1631257278 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff2f80b6680 msgr2=0x7ff2f80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:56.422 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.415+0000 7ff315046640 1 --2- 172.21.15.64:0/1631257278 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff2f80b6680 0x7ff2f80b8b90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff310079fd0 tx=0x7ff3040046c0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.415+0000 7ff315046640 1 -- 172.21.15.64:0/1631257278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff31010de30 msgr2=0x7ff310103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:56.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.415+0000 7ff315046640 1 --2- 172.21.15.64:0/1631257278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff31010de30 0x7ff310103190 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7ff30800cce0 tx=0x7ff30800a620 comp rx=0 tx=0).stop 2024-09-21T04:13:56.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.415+0000 7ff315046640 1 -- 172.21.15.64:0/1631257278 shutdown_connections 2024-09-21T04:13:56.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.415+0000 7ff315046640 1 --2- 172.21.15.64:0/1631257278 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7ff2f80b6680 0x7ff2f80b8b90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.415+0000 7ff315046640 1 --2- 172.21.15.64:0/1631257278 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff31010de30 0x7ff310103190 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.415+0000 7ff315046640 1 --2- 172.21.15.64:0/1631257278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff310105320 0x7ff310102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.415+0000 7ff315046640 1 --2- 172.21.15.64:0/1631257278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3101049d0 0x7ff310102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:56.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.415+0000 7ff315046640 1 -- 172.21.15.64:0/1631257278 >> 172.21.15.64:0/1631257278 conn(0x7ff310079740 msgr2=0x7ff3100ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:56.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.415+0000 7ff315046640 1 -- 172.21.15.64:0/1631257278 shutdown_connections 2024-09-21T04:13:56.423 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:56.415+0000 7ff315046640 1 -- 172.21.15.64:0/1631257278 wait complete. 2024-09-21T04:13:56.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:56 smithi119 ceph-mon[34811]: pgmap v626: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:13:56.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:56 smithi064 ceph-mon[31180]: pgmap v626: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:13:56.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:56 smithi131 ceph-mon[35193]: pgmap v626: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:13:57.117 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T04:13:57.376 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:13:58.044 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.027+0000 7faa9b3f9640 1 -- 172.21.15.64:0/3159208796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa94108cf0 msgr2=0x7faa9410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.027+0000 7faa9b3f9640 1 --2- 172.21.15.64:0/3159208796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa94108cf0 0x7faa9410f520 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7faa8c009ca0 tx=0x7faa8c02a8c0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.027+0000 7faa9b3f9640 1 -- 172.21.15.64:0/3159208796 shutdown_connections 2024-09-21T04:13:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.027+0000 7faa9b3f9640 1 --2- 172.21.15.64:0/3159208796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa94108cf0 0x7faa9410f520 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.027+0000 7faa9b3f9640 1 --2- 172.21.15.64:0/3159208796 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa941083a0 0x7faa941087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.027+0000 7faa9b3f9640 1 --2- 172.21.15.64:0/3159208796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa94102a60 0x7faa94102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.027+0000 7faa9b3f9640 1 -- 172.21.15.64:0/3159208796 >> 172.21.15.64:0/3159208796 conn(0x7faa94079730 msgr2=0x7faa94079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.027+0000 7faa9b3f9640 1 -- 172.21.15.64:0/3159208796 shutdown_connections 2024-09-21T04:13:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.027+0000 7faa9b3f9640 1 -- 172.21.15.64:0/3159208796 wait complete. 2024-09-21T04:13:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.028+0000 7faa9b3f9640 1 Processor -- start 2024-09-21T04:13:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.028+0000 7faa9b3f9640 1 -- start start 2024-09-21T04:13:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.028+0000 7faa9b3f9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa94102a60 0x7faa9419b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.028+0000 7faa9b3f9640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa941083a0 0x7faa9419b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:58.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.028+0000 7faa9b3f9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa94108cf0 0x7faa9419bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:58.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.028+0000 7faa9b3f9640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faa94112960 con 0x7faa941083a0 2024-09-21T04:13:58.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.028+0000 7faa9b3f9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faa941127e0 con 0x7faa94108cf0 2024-09-21T04:13:58.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.028+0000 7faa9b3f9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faa94112ae0 con 0x7faa94102a60 2024-09-21T04:13:58.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.028+0000 7faa98994640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa941083a0 0x7faa9419b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:58.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.028+0000 7faa98994640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa941083a0 0x7faa9419b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34164/0 (socket says 172.21.15.64:34164) 2024-09-21T04:13:58.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.028+0000 7faa98994640 1 -- 172.21.15.64:0/4114894429 learned_addr learned my addr 172.21.15.64:0/4114894429 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:58.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.028+0000 7faa99195640 1 --2- 172.21.15.64:0/4114894429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa94102a60 0x7faa9419b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:58.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.028+0000 7faa99996640 1 --2- 172.21.15.64:0/4114894429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa94108cf0 0x7faa9419bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:58.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.028+0000 7faa98994640 1 -- 172.21.15.64:0/4114894429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa94102a60 msgr2=0x7faa9419b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:58.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.028+0000 7faa98994640 1 --2- 172.21.15.64:0/4114894429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa94102a60 0x7faa9419b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.029+0000 7faa98994640 1 -- 172.21.15.64:0/4114894429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa94108cf0 msgr2=0x7faa9419bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:58.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.029+0000 7faa98994640 1 --2- 172.21.15.64:0/4114894429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa94108cf0 0x7faa9419bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.029+0000 7faa98994640 1 -- 172.21.15.64:0/4114894429 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa941a6470 con 0x7faa941083a0 2024-09-21T04:13:58.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.029+0000 7faa98994640 1 --2- 172.21.15.64:0/4114894429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa941083a0 0x7faa9419b7b0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7faa8400c970 tx=0x7faa8400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:58.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.029+0000 7faa8a7fc640 1 -- 172.21.15.64:0/4114894429 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84007bb0 con 0x7faa941083a0 2024-09-21T04:13:58.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.029+0000 7faa9b3f9640 1 -- 172.21.15.64:0/4114894429 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa941a6760 con 0x7faa941083a0 2024-09-21T04:13:58.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.029+0000 7faa8a7fc640 1 -- 172.21.15.64:0/4114894429 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa84007d50 con 0x7faa941083a0 2024-09-21T04:13:58.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.029+0000 7faa8a7fc640 1 -- 172.21.15.64:0/4114894429 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84018800 con 0x7faa941083a0 2024-09-21T04:13:58.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.029+0000 7faa9b3f9640 1 -- 172.21.15.64:0/4114894429 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa941a6c50 con 0x7faa941083a0 2024-09-21T04:13:58.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.032+0000 7faaa196d740 1 Processor -- start 2024-09-21T04:13:58.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.032+0000 7faaa196d740 1 -- start start 2024-09-21T04:13:58.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.032+0000 7faaa196d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e0890f070 0x564e08cb62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:58.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.032+0000 7faaa196d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e088e61b0 0x564e08cb6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:58.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.032+0000 7faa8a7fc640 1 -- 172.21.15.64:0/4114894429 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7faa8400fa20 con 0x7faa941083a0 2024-09-21T04:13:58.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.032+0000 7faaa196d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e088ee390 0x564e08cb6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:58.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.032+0000 7faaa196d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564e08c2cea0 con 0x564e0890f070 2024-09-21T04:13:58.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.032+0000 7faaa196d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564e08c1c2d0 con 0x564e088e61b0 2024-09-21T04:13:58.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.032+0000 7faaa196d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564e08c200e0 con 0x564e088ee390 2024-09-21T04:13:58.047 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.032+0000 7faa99996640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e0890f070 0x564e08cb62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.032+0000 7faa99996640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e0890f070 0x564e08cb62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34168/0 (socket says 172.21.15.64:34168) 2024-09-21T04:13:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.032+0000 7faa99996640 1 -- 172.21.15.64:0/3003451294 learned_addr learned my addr 172.21.15.64:0/3003451294 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.033+0000 7faa98994640 1 --2- 172.21.15.64:0/3003451294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e088ee390 0x564e08cb6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.033+0000 7faa99195640 1 --2- 172.21.15.64:0/3003451294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e088e61b0 0x564e08cb6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.033+0000 7faa99996640 1 -- 172.21.15.64:0/3003451294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e088ee390 msgr2=0x564e08cb6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.033+0000 7faa99996640 1 --2- 172.21.15.64:0/3003451294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e088ee390 0x564e08cb6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.033+0000 7faa99996640 1 -- 172.21.15.64:0/3003451294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e088e61b0 msgr2=0x564e08cb6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.033+0000 7faa99996640 1 --2- 172.21.15.64:0/3003451294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e088e61b0 0x564e08cb6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.033+0000 7faa99996640 1 -- 172.21.15.64:0/3003451294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e0897a650 con 0x564e0890f070 2024-09-21T04:13:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.033+0000 7faa99996640 1 --2- 172.21.15.64:0/3003451294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e0890f070 0x564e08cb62c0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7faa8c02f4c0 tx=0x7faa8c031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492faefe8e5622bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.033+0000 7faa7f7fe640 1 -- 172.21.15.64:0/3003451294 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c031450 con 0x564e0890f070 2024-09-21T04:13:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.033+0000 7faa8a7fc640 1 --2- 172.21.15.64:0/4114894429 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7faa800b66b0 0x7faa800b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.033+0000 7faa7f7fe640 1 -- 172.21.15.64:0/3003451294 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa8c036030 con 0x564e0890f070 2024-09-21T04:13:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.033+0000 7faa99195640 1 --2- 172.21.15.64:0/4114894429 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7faa800b66b0 0x7faa800b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:58.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.033+0000 7faa7f7fe640 1 -- 172.21.15.64:0/3003451294 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c004380 con 0x564e0890f070 2024-09-21T04:13:58.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.034+0000 7faa8a7fc640 1 -- 172.21.15.64:0/4114894429 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8615+0+0 (secure 0 0 0) 0x7faa840e7810 con 0x7faa941083a0 2024-09-21T04:13:58.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.034+0000 7faaa196d740 1 -- 172.21.15.64:0/3003451294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e0890f070 msgr2=0x564e08cb62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:58.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.034+0000 7faa99195640 1 --2- 172.21.15.64:0/4114894429 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7faa800b66b0 0x7faa800b8bc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x564e08cb4f40 tx=0x7faa9000a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:58.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.034+0000 7faaa196d740 1 --2- 172.21.15.64:0/3003451294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e0890f070 0x564e08cb62c0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7faa8c02f4c0 tx=0x7faa8c031af0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.034+0000 7faaa196d740 1 -- 172.21.15.64:0/3003451294 shutdown_connections 2024-09-21T04:13:58.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.034+0000 7faaa196d740 1 --2- 172.21.15.64:0/3003451294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e088ee390 0x564e08cb6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.034+0000 7faaa196d740 1 --2- 172.21.15.64:0/3003451294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e088e61b0 0x564e08cb6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.034+0000 7faaa196d740 1 --2- 172.21.15.64:0/3003451294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e0890f070 0x564e08cb62c0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.034+0000 7faaa196d740 1 -- 172.21.15.64:0/3003451294 >> 172.21.15.64:0/3003451294 conn(0x564e08903460 msgr2=0x564e08cb41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:58.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.034+0000 7faaa196d740 1 -- 172.21.15.64:0/3003451294 shutdown_connections 2024-09-21T04:13:58.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.034+0000 7faaa196d740 1 -- 172.21.15.64:0/3003451294 wait complete. 2024-09-21T04:13:58.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.035+0000 7faaa196d740 1 -- start start 2024-09-21T04:13:58.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.035+0000 7faaa196d740 1 Processor -- start 2024-09-21T04:13:58.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.035+0000 7faaa196d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e088e61b0 0x564e08cc19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:58.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.035+0000 7faaa196d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e088ee390 0x564e08cc1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:58.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.036+0000 7faaa196d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e0890f070 0x564e08ccdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:58.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.036+0000 7faaa196d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564e08a7d700 con 0x564e088ee390 2024-09-21T04:13:58.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.036+0000 7faaa196d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564e08bb7160 con 0x564e088e61b0 2024-09-21T04:13:58.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.036+0000 7faaa196d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x564e08cc48c0 con 0x564e0890f070 2024-09-21T04:13:58.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.036+0000 7faaa196d740 10 client.0 _ll_register_callbacks cb 0x564e08bd09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:58.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.036+0000 7faa98994640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e088ee390 0x564e08cc1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:58.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.036+0000 7faa98994640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e088ee390 0x564e08cc1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34170/0 (socket says 172.21.15.64:34170) 2024-09-21T04:13:58.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.036+0000 7faa98994640 1 -- 172.21.15.64:0/785720544 learned_addr learned my addr 172.21.15.64:0/785720544 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:58.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.036+0000 7faa99195640 1 --2- 172.21.15.64:0/785720544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e088e61b0 0x564e08cc19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:58.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.036+0000 7faa99996640 1 --2- 172.21.15.64:0/785720544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e0890f070 0x564e08ccdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:58.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.036+0000 7faa98994640 1 -- 172.21.15.64:0/785720544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e0890f070 msgr2=0x564e08ccdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:58.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.036+0000 7faa98994640 1 --2- 172.21.15.64:0/785720544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e0890f070 0x564e08ccdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.036+0000 7faa98994640 1 -- 172.21.15.64:0/785720544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e088e61b0 msgr2=0x564e08cc19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:58.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.036+0000 7faa98994640 1 --2- 172.21.15.64:0/785720544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e088e61b0 0x564e08cc19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.036+0000 7faa98994640 1 -- 172.21.15.64:0/785720544 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e08be4c90 con 0x564e088ee390 2024-09-21T04:13:58.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.036+0000 7faa98994640 1 --2- 172.21.15.64:0/785720544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e088ee390 0x564e08cc1f00 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7faa840e4f60 tx=0x7faa840acc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7704359b0ea79221 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:58.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.036+0000 7faa7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:13:58.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.036+0000 7faa7cff9640 1 -- 172.21.15.64:0/785720544 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa840a9c50 con 0x564e088ee390 2024-09-21T04:13:58.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.036+0000 7faa7cff9640 1 -- 172.21.15.64:0/785720544 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8615+0+0 (secure 0 0 0) 0x7faa840b3270 con 0x564e088ee390 2024-09-21T04:13:58.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.036+0000 7faaa196d740 1 -- 172.21.15.64:0/785720544 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa8c036030 con 0x564e088ee390 2024-09-21T04:13:58.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.036+0000 7faa7cff9640 1 -- 172.21.15.64:0/785720544 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7faa840acd70 con 0x564e088ee390 2024-09-21T04:13:58.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.037+0000 7faaa196d740 10 client.17520 fetch_fsmap learned FSMap version 41 2024-09-21T04:13:58.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.037+0000 7faaa196d740 1 -- 172.21.15.64:0/785720544 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x564e08cd0640 con 0x564e088ee390 2024-09-21T04:13:58.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.037+0000 7faa7cff9640 1 -- 172.21.15.64:0/785720544 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa8c036030 con 0x564e088ee390 2024-09-21T04:13:58.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.037+0000 7faa7cff9640 1 -- 172.21.15.64:0/785720544 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7faa84020030 con 0x564e088ee390 2024-09-21T04:13:58.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.037+0000 7faaa196d740 10 client.17520 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T04:13:58.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.037+0000 7faaa196d740 10 client.17520 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:13:58.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.037+0000 7faaa196d740 20 client.17520 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:58.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.037+0000 7faaa196d740 1 --2- 172.21.15.64:0/785720544 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x564e088cd9f0 0x564e08cd3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.037+0000 7faaa196d740 4 client.17520 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:13:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.037+0000 7faaa196d740 1 -- 172.21.15.64:0/785720544 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e08c2edf0 con 0x564e088cd9f0 2024-09-21T04:13:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.037+0000 7faa99195640 1 --2- 172.21.15.64:0/785720544 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x564e088cd9f0 0x564e08cd3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.038+0000 7faa99195640 1 --2- 172.21.15.64:0/785720544 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x564e088cd9f0 0x564e08cd3b80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eecfb17bcfdf88f1 server_cookie=12419283d4ee3a89 in_seq=0 out_seq=0 2024-09-21T04:13:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.038+0000 7faa7cff9640 10 client.17520 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:13:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.039+0000 7faa7cff9640 1 -- 172.21.15.64:0/785720544 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564e08c2edf0 con 0x564e088cd9f0 2024-09-21T04:13:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.039+0000 7faa7cff9640 10 client.17520 handle_command_reply: tid=0 2024-09-21T04:13:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.039+0000 7faaa196d740 1 client.17520 shutdown 2024-09-21T04:13:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.039+0000 7faaa196d740 1 -- 172.21.15.64:0/785720544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e088ee390 msgr2=0x564e08cc1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.039+0000 7faaa196d740 1 --2- 172.21.15.64:0/785720544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e088ee390 0x564e08cc1f00 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7faa840e4f60 tx=0x7faa840acc30 comp rx=0 tx=0).stop 2024-09-21T04:13:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.039+0000 7faaa196d740 1 -- 172.21.15.64:0/785720544 shutdown_connections 2024-09-21T04:13:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.039+0000 7faaa196d740 1 --2- 172.21.15.64:0/785720544 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x564e088cd9f0 0x564e08cd3b80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.039+0000 7faaa196d740 1 --2- 172.21.15.64:0/785720544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e0890f070 0x564e08ccdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.039+0000 7faaa196d740 1 --2- 172.21.15.64:0/785720544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e088ee390 0x564e08cc1f00 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.052 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.039+0000 7faa7cff9640 0 client.17520 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:13:58.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.039+0000 7faaa196d740 1 --2- 172.21.15.64:0/785720544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e088e61b0 0x564e08cc19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.039+0000 7faaa196d740 1 -- 172.21.15.64:0/785720544 >> 172.21.15.64:0/785720544 conn(0x564e08903460 msgr2=0x564e08cb94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:58.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.039+0000 7faaa196d740 1 -- 172.21.15.64:0/785720544 shutdown_connections 2024-09-21T04:13:58.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.039+0000 7faaa196d740 1 -- 172.21.15.64:0/785720544 wait complete. 2024-09-21T04:13:58.053 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.039+0000 7faaa196d740 20 client.17520 trim_cache size 0 max 16384 2024-09-21T04:13:58.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.053+0000 7faaa196d740 1 Processor -- start 2024-09-21T04:13:58.056 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.053+0000 7faaa196d740 1 -- start start 2024-09-21T04:13:58.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.053+0000 7faaa196d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e088e61b0 0x564e08cc38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:58.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.053+0000 7faaa196d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e088ee390 0x564e08ccdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:58.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.053+0000 7faaa196d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e0890f070 0x564e08cbbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:58.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.053+0000 7faaa196d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faa941b2030 con 0x564e088ee390 2024-09-21T04:13:58.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.053+0000 7faaa196d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faa941af870 con 0x564e0890f070 2024-09-21T04:13:58.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.053+0000 7faaa196d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faa941ad6e0 con 0x564e088e61b0 2024-09-21T04:13:58.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.054+0000 7faa98994640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e088ee390 0x564e08ccdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:58.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.054+0000 7faa98994640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e088ee390 0x564e08ccdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34182/0 (socket says 172.21.15.64:34182) 2024-09-21T04:13:58.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.054+0000 7faa98994640 1 -- 172.21.15.64:0/313801970 learned_addr learned my addr 172.21.15.64:0/313801970 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:58.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.054+0000 7faa99195640 1 --2- 172.21.15.64:0/313801970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e088e61b0 0x564e08cc38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:58.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.054+0000 7faa99996640 1 --2- 172.21.15.64:0/313801970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e0890f070 0x564e08cbbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:58.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.054+0000 7faa98994640 1 -- 172.21.15.64:0/313801970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e088e61b0 msgr2=0x564e08cc38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:58.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.054+0000 7faa98994640 1 --2- 172.21.15.64:0/313801970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e088e61b0 0x564e08cc38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.054+0000 7faa98994640 1 -- 172.21.15.64:0/313801970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e0890f070 msgr2=0x564e08cbbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:58.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.054+0000 7faa98994640 1 --2- 172.21.15.64:0/313801970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e0890f070 0x564e08cbbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.054+0000 7faa98994640 1 -- 172.21.15.64:0/313801970 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e08cae460 con 0x564e088ee390 2024-09-21T04:13:58.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.054+0000 7faa99195640 1 --2- 172.21.15.64:0/313801970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e088e61b0 0x564e08cc38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T04:13:58.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.054+0000 7faa98994640 1 --2- 172.21.15.64:0/313801970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e088ee390 0x564e08ccdae0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7faa840ad300 tx=0x7faa840b71a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbf069c659aaee9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:58.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.054+0000 7faa7effd640 1 -- 172.21.15.64:0/313801970 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa840acf50 con 0x564e088ee390 2024-09-21T04:13:58.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.055+0000 7faa7effd640 1 -- 172.21.15.64:0/313801970 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa840ad110 con 0x564e088ee390 2024-09-21T04:13:58.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.055+0000 7faa7effd640 1 -- 172.21.15.64:0/313801970 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa840ad3d0 con 0x564e088ee390 2024-09-21T04:13:58.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.055+0000 7faaa196d740 1 -- 172.21.15.64:0/313801970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e088ee390 msgr2=0x564e08ccdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:58.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.055+0000 7faaa196d740 1 --2- 172.21.15.64:0/313801970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e088ee390 0x564e08ccdae0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7faa840ad300 tx=0x7faa840b71a0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.055+0000 7faaa196d740 1 -- 172.21.15.64:0/313801970 shutdown_connections 2024-09-21T04:13:58.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.055+0000 7faaa196d740 1 --2- 172.21.15.64:0/313801970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e0890f070 0x564e08cbbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.055+0000 7faaa196d740 1 --2- 172.21.15.64:0/313801970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e088ee390 0x564e08ccdae0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.055+0000 7faaa196d740 1 --2- 172.21.15.64:0/313801970 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e088e61b0 0x564e08cc38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.055+0000 7faaa196d740 1 -- 172.21.15.64:0/313801970 >> 172.21.15.64:0/313801970 conn(0x564e08903460 msgr2=0x564e08cbb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:58.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.056+0000 7faaa196d740 1 -- 172.21.15.64:0/313801970 shutdown_connections 2024-09-21T04:13:58.061 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.056+0000 7faaa196d740 1 -- 172.21.15.64:0/313801970 wait complete. 2024-09-21T04:13:58.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.056+0000 7faaa196d740 1 -- start start 2024-09-21T04:13:58.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.056+0000 7faaa196d740 1 Processor -- start 2024-09-21T04:13:58.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.056+0000 7faaa196d740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e088e61b0 0x564e08cc65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:58.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.056+0000 7faaa196d740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e088ee390 0x564e08cc6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:58.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.056+0000 7faaa196d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e0890f070 0x564e08d375f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:58.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.056+0000 7faaa196d740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faa941add30 con 0x564e088ee390 2024-09-21T04:13:58.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.056+0000 7faaa196d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faa941af1a0 con 0x564e0890f070 2024-09-21T04:13:58.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.056+0000 7faaa196d740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faa941ae400 con 0x564e088e61b0 2024-09-21T04:13:58.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.057+0000 7faaa196d740 10 client.0 _ll_register_callbacks cb 0x7faa8400c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:58.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.057+0000 7faa98994640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e088ee390 0x564e08cc6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:58.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.057+0000 7faa98994640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e088ee390 0x564e08cc6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34186/0 (socket says 172.21.15.64:34186) 2024-09-21T04:13:58.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.057+0000 7faa98994640 1 -- 172.21.15.64:0/2439703583 learned_addr learned my addr 172.21.15.64:0/2439703583 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:58.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.057+0000 7faa99195640 1 --2- 172.21.15.64:0/2439703583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e088e61b0 0x564e08cc65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:58.062 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.057+0000 7faa99996640 1 --2- 172.21.15.64:0/2439703583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e0890f070 0x564e08d375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:58.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.057+0000 7faa98994640 1 -- 172.21.15.64:0/2439703583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e088e61b0 msgr2=0x564e08cc65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:58.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.057+0000 7faa98994640 1 --2- 172.21.15.64:0/2439703583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e088e61b0 0x564e08cc65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.057+0000 7faa98994640 1 -- 172.21.15.64:0/2439703583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e0890f070 msgr2=0x564e08d375f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:58.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.057+0000 7faa98994640 1 --2- 172.21.15.64:0/2439703583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e0890f070 0x564e08d375f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.057+0000 7faa98994640 1 -- 172.21.15.64:0/2439703583 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e08d37c60 con 0x564e088ee390 2024-09-21T04:13:58.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.057+0000 7faa99996640 1 --2- 172.21.15.64:0/2439703583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e0890f070 0x564e08d375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:13:58.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.057+0000 7faa98994640 1 --2- 172.21.15.64:0/2439703583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e088ee390 0x564e08cc6b90 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7faa840aa870 tx=0x7faa840ac6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05a395c65725072 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:58.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.057+0000 7faa7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:13:58.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.057+0000 7faa7cff9640 1 -- 172.21.15.64:0/2439703583 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa840a94a0 con 0x564e088ee390 2024-09-21T04:13:58.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.057+0000 7faa7cff9640 1 -- 172.21.15.64:0/2439703583 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8615+0+0 (secure 0 0 0) 0x7faa8400f450 con 0x564e088ee390 2024-09-21T04:13:58.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.057+0000 7faaa196d740 1 -- 172.21.15.64:0/2439703583 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa840ad110 con 0x564e088ee390 2024-09-21T04:13:58.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.058+0000 7faa7cff9640 1 -- 172.21.15.64:0/2439703583 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7faa840ad3d0 con 0x564e088ee390 2024-09-21T04:13:58.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.058+0000 7faaa196d740 10 client.17526 fetch_fsmap learned FSMap version 41 2024-09-21T04:13:58.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.058+0000 7faaa196d740 1 -- 172.21.15.64:0/2439703583 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x564e08d3f050 con 0x564e088ee390 2024-09-21T04:13:58.063 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.058+0000 7faa7cff9640 1 -- 172.21.15.64:0/2439703583 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa840ad110 con 0x564e088ee390 2024-09-21T04:13:58.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.058+0000 7faa7cff9640 1 -- 172.21.15.64:0/2439703583 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7faa84020070 con 0x564e088ee390 2024-09-21T04:13:58.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.058+0000 7faaa196d740 10 client.17526 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T04:13:58.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.058+0000 7faaa196d740 10 client.17526 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-21T04:13:58.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.058+0000 7faaa196d740 20 client.17526 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:58.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.058+0000 7faaa196d740 1 --2- 172.21.15.64:0/2439703583 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x564e088a2da0 0x564e08d415c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:58.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.058+0000 7faaa196d740 4 client.17526 mds_command: new command op to 24680 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:13:58.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.058+0000 7faaa196d740 1 -- 172.21.15.64:0/2439703583 --> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e08c2a8a0 con 0x564e088a2da0 2024-09-21T04:13:58.064 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.058+0000 7faa99195640 1 --2- 172.21.15.64:0/2439703583 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x564e088a2da0 0x564e08d415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:58.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.059+0000 7faa7cff9640 10 client.17526 ms_handle_connect on v2:172.21.15.119:6836/2876008744 2024-09-21T04:13:58.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.059+0000 7faa99195640 1 --2- 172.21.15.64:0/2439703583 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x564e088a2da0 0x564e08d415c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=558d52518bfade69 server_cookie=4289189863382ec8 in_seq=0 out_seq=0 2024-09-21T04:13:58.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.059+0000 7faa7cff9640 1 -- 172.21.15.64:0/2439703583 <== mds.1 v2:172.21.15.119:6836/2876008744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e08c2a8a0 con 0x564e088a2da0 2024-09-21T04:13:58.065 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.059+0000 7faa7cff9640 10 client.17526 handle_command_reply: tid=0 2024-09-21T04:13:58.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.059+0000 7faaa196d740 1 client.17526 shutdown 2024-09-21T04:13:58.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.059+0000 7faaa196d740 1 -- 172.21.15.64:0/2439703583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e088ee390 msgr2=0x564e08cc6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:58.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.059+0000 7faaa196d740 1 --2- 172.21.15.64:0/2439703583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e088ee390 0x564e08cc6b90 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7faa840aa870 tx=0x7faa840ac6f0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.059+0000 7faaa196d740 1 -- 172.21.15.64:0/2439703583 shutdown_connections 2024-09-21T04:13:58.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.059+0000 7faaa196d740 1 --2- 172.21.15.64:0/2439703583 >> [v2:172.21.15.119:6836/2876008744,v1:172.21.15.119:6837/2876008744] conn(0x564e088a2da0 0x564e08d415c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.059+0000 7faaa196d740 1 --2- 172.21.15.64:0/2439703583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564e0890f070 0x564e08d375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.060+0000 7faaa196d740 1 --2- 172.21.15.64:0/2439703583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564e088ee390 0x564e08cc6b90 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.060+0000 7faa7cff9640 0 client.17526 ms_handle_reset on v2:172.21.15.119:6836/2876008744 2024-09-21T04:13:58.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.060+0000 7faaa196d740 1 --2- 172.21.15.64:0/2439703583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x564e088e61b0 0x564e08cc65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.060+0000 7faaa196d740 1 -- 172.21.15.64:0/2439703583 >> 172.21.15.64:0/2439703583 conn(0x564e08903460 msgr2=0x564e08cbb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:58.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.060+0000 7faaa196d740 1 -- 172.21.15.64:0/2439703583 shutdown_connections 2024-09-21T04:13:58.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.060+0000 7faaa196d740 1 -- 172.21.15.64:0/2439703583 wait complete. 2024-09-21T04:13:58.066 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.060+0000 7faaa196d740 20 client.17526 trim_cache size 0 max 16384 2024-09-21T04:13:58.067 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:13:58.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.060+0000 7faa9b3f9640 1 -- 172.21.15.64:0/4114894429 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7faa800b66b0 msgr2=0x7faa800b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:58.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.060+0000 7faa9b3f9640 1 --2- 172.21.15.64:0/4114894429 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7faa800b66b0 0x7faa800b8bc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x564e08cb4f40 tx=0x7faa9000a2d0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.060+0000 7faa9b3f9640 1 -- 172.21.15.64:0/4114894429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa941083a0 msgr2=0x7faa9419b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:58.067 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.060+0000 7faa9b3f9640 1 --2- 172.21.15.64:0/4114894429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa941083a0 0x7faa9419b7b0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7faa8400c970 tx=0x7faa8400cd70 comp rx=0 tx=0).stop 2024-09-21T04:13:58.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.060+0000 7faa9b3f9640 1 -- 172.21.15.64:0/4114894429 shutdown_connections 2024-09-21T04:13:58.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.060+0000 7faa9b3f9640 1 --2- 172.21.15.64:0/4114894429 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7faa800b66b0 0x7faa800b8bc0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.060+0000 7faa9b3f9640 1 --2- 172.21.15.64:0/4114894429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa94108cf0 0x7faa9419bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.060+0000 7faa9b3f9640 1 --2- 172.21.15.64:0/4114894429 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa941083a0 0x7faa9419b7b0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.060+0000 7faa9b3f9640 1 --2- 172.21.15.64:0/4114894429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faa94102a60 0x7faa9419b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:58.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.060+0000 7faa9b3f9640 1 -- 172.21.15.64:0/4114894429 >> 172.21.15.64:0/4114894429 conn(0x7faa94079730 msgr2=0x7faa941043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:58.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.061+0000 7faa9b3f9640 1 -- 172.21.15.64:0/4114894429 shutdown_connections 2024-09-21T04:13:58.068 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:58.061+0000 7faa9b3f9640 1 -- 172.21.15.64:0/4114894429 wait complete. 2024-09-21T04:13:58.715 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:13:58 smithi064 ceph-mon[31180]: pgmap v627: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T04:13:58.743 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T04:13:58.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:13:58 smithi119 ceph-mon[34811]: pgmap v627: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T04:13:58.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:13:58 smithi131 ceph-mon[35193]: pgmap v627: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-21T04:13:58.999 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:13:59.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.478+0000 7f987190e640 1 -- 172.21.15.64:0/112392535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f986c10de10 msgr2=0x7f986c110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:59.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.478+0000 7f987190e640 1 --2- 172.21.15.64:0/112392535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f986c10de10 0x7f986c110300 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f9864009900 tx=0x7f986402a410 comp rx=0 tx=0).stop 2024-09-21T04:13:59.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.478+0000 7f987190e640 1 -- 172.21.15.64:0/112392535 shutdown_connections 2024-09-21T04:13:59.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.478+0000 7f987190e640 1 --2- 172.21.15.64:0/112392535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f986c10de10 0x7f986c110300 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.478+0000 7f987190e640 1 --2- 172.21.15.64:0/112392535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f986c105300 0x7f986c10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.478+0000 7f987190e640 1 --2- 172.21.15.64:0/112392535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f986c1049b0 0x7f986c104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.478+0000 7f987190e640 1 -- 172.21.15.64:0/112392535 >> 172.21.15.64:0/112392535 conn(0x7f986c079730 msgr2=0x7f986c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:59.481 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.478+0000 7f987190e640 1 -- 172.21.15.64:0/112392535 shutdown_connections 2024-09-21T04:13:59.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.478+0000 7f987190e640 1 -- 172.21.15.64:0/112392535 wait complete. 2024-09-21T04:13:59.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.479+0000 7f987190e640 1 Processor -- start 2024-09-21T04:13:59.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.479+0000 7f987190e640 1 -- start start 2024-09-21T04:13:59.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.480+0000 7f987190e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f986c1049b0 0x7f986c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:59.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.480+0000 7f987190e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f986c105300 0x7f986c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:59.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.480+0000 7f987190e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f986c10de10 0x7f986c19fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:59.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.480+0000 7f987190e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f986c113a90 con 0x7f986c10de10 2024-09-21T04:13:59.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.480+0000 7f987190e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f986c113910 con 0x7f986c105300 2024-09-21T04:13:59.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.480+0000 7f987190e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f986c113c10 con 0x7f986c1049b0 2024-09-21T04:13:59.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.480+0000 7f986b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f986c10de10 0x7f986c19fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:59.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.480+0000 7f986b7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f986c10de10 0x7f986c19fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34208/0 (socket says 172.21.15.64:34208) 2024-09-21T04:13:59.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.480+0000 7f986b7fe640 1 -- 172.21.15.64:0/3961964101 learned_addr learned my addr 172.21.15.64:0/3961964101 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:59.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.480+0000 7f986affd640 1 --2- 172.21.15.64:0/3961964101 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f986c1049b0 0x7f986c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:59.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.480+0000 7f986a7fc640 1 --2- 172.21.15.64:0/3961964101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f986c105300 0x7f986c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:59.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.480+0000 7f986b7fe640 1 -- 172.21.15.64:0/3961964101 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f986c1049b0 msgr2=0x7f986c19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:59.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.480+0000 7f986b7fe640 1 --2- 172.21.15.64:0/3961964101 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f986c1049b0 0x7f986c19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.480+0000 7f986b7fe640 1 -- 172.21.15.64:0/3961964101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f986c105300 msgr2=0x7f986c19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:59.484 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.480+0000 7f986b7fe640 1 --2- 172.21.15.64:0/3961964101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f986c105300 0x7f986c19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.480+0000 7f986b7fe640 1 -- 172.21.15.64:0/3961964101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986c1a0450 con 0x7f986c10de10 2024-09-21T04:13:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.480+0000 7f986b7fe640 1 --2- 172.21.15.64:0/3961964101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f986c10de10 0x7f986c19fde0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f9864004770 tx=0x7f9864004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.481+0000 7f987090c640 1 -- 172.21.15.64:0/3961964101 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986403f030 con 0x7f986c10de10 2024-09-21T04:13:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.481+0000 7f987190e640 1 -- 172.21.15.64:0/3961964101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f986c1a06e0 con 0x7f986c10de10 2024-09-21T04:13:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.481+0000 7f987090c640 1 -- 172.21.15.64:0/3961964101 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9864003aa0 con 0x7f986c10de10 2024-09-21T04:13:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.481+0000 7f987090c640 1 -- 172.21.15.64:0/3961964101 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9864035070 con 0x7f986c10de10 2024-09-21T04:13:59.485 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.481+0000 7f987190e640 1 -- 172.21.15.64:0/3961964101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f986c1a0c10 con 0x7f986c10de10 2024-09-21T04:13:59.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.483+0000 7f9877e82740 1 Processor -- start 2024-09-21T04:13:59.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.483+0000 7f987090c640 1 -- 172.21.15.64:0/3961964101 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f9864042030 con 0x7f986c10de10 2024-09-21T04:13:59.486 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.484+0000 7f9877e82740 1 -- start start 2024-09-21T04:13:59.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.484+0000 7f9877e82740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7987f7070 0x55b798b9e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:59.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.484+0000 7f9877e82740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987ce1b0 0x55b798b9ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:59.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.484+0000 7f9877e82740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7987d6390 0x55b798b9f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:59.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.484+0000 7f9877e82740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b798b14ea0 con 0x55b7987ce1b0 2024-09-21T04:13:59.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.484+0000 7f9877e82740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b798b042d0 con 0x55b7987d6390 2024-09-21T04:13:59.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.484+0000 7f9877e82740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b798b080e0 con 0x55b7987f7070 2024-09-21T04:13:59.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.484+0000 7f986affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987ce1b0 0x55b798b9ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:59.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.484+0000 7f986affd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987ce1b0 0x55b798b9ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34220/0 (socket says 172.21.15.64:34220) 2024-09-21T04:13:59.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.484+0000 7f986affd640 1 -- 172.21.15.64:0/1734867360 learned_addr learned my addr 172.21.15.64:0/1734867360 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:59.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.484+0000 7f986a7fc640 1 --2- 172.21.15.64:0/1734867360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7987f7070 0x55b798b9e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:59.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.484+0000 7f986a7fc640 1 --2- 172.21.15.64:0/1734867360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7987d6390 0x55b798b9f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:59.488 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.484+0000 7f987090c640 1 --2- 172.21.15.64:0/3961964101 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f98540b6730 0x7f98540b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:59.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.484+0000 7f986affd640 1 -- 172.21.15.64:0/1734867360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7987f7070 msgr2=0x55b798b9e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:59.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.484+0000 7f986affd640 1 --2- 172.21.15.64:0/1734867360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7987f7070 0x55b798b9e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.484+0000 7f986affd640 1 -- 172.21.15.64:0/1734867360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7987d6390 msgr2=0x55b798b9f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:59.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.484+0000 7f986affd640 1 --2- 172.21.15.64:0/1734867360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7987d6390 0x55b798b9f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.484+0000 7f986affd640 1 -- 172.21.15.64:0/1734867360 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b798862650 con 0x55b7987ce1b0 2024-09-21T04:13:59.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.485+0000 7f986b7fe640 1 --2- 172.21.15.64:0/3961964101 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f98540b6730 0x7f98540b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:59.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.485+0000 7f987090c640 1 -- 172.21.15.64:0/3961964101 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8615+0+0 (secure 0 0 0) 0x7f9864107ce0 con 0x7f986c10de10 2024-09-21T04:13:59.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.485+0000 7f986affd640 1 --2- 172.21.15.64:0/1734867360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987ce1b0 0x55b798b9ebe0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f985c007f50 tx=0x7f985c00f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d3e7437e991e86 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:59.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.485+0000 7f9851ffb640 1 -- 172.21.15.64:0/1734867360 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c003b70 con 0x55b7987ce1b0 2024-09-21T04:13:59.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.485+0000 7f9851ffb640 1 -- 172.21.15.64:0/1734867360 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f985c003d10 con 0x55b7987ce1b0 2024-09-21T04:13:59.489 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.485+0000 7f9851ffb640 1 -- 172.21.15.64:0/1734867360 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c010b10 con 0x55b7987ce1b0 2024-09-21T04:13:59.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.486+0000 7f9877e82740 1 -- 172.21.15.64:0/1734867360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987ce1b0 msgr2=0x55b798b9ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:59.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.486+0000 7f986b7fe640 1 --2- 172.21.15.64:0/3961964101 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f98540b6730 0x7f98540b8c40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f98640cd880 tx=0x7f986409ae60 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:59.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.486+0000 7f9877e82740 1 --2- 172.21.15.64:0/1734867360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987ce1b0 0x55b798b9ebe0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f985c007f50 tx=0x7f985c00f850 comp rx=0 tx=0).stop 2024-09-21T04:13:59.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.486+0000 7f9877e82740 1 -- 172.21.15.64:0/1734867360 shutdown_connections 2024-09-21T04:13:59.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.486+0000 7f9877e82740 1 --2- 172.21.15.64:0/1734867360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7987d6390 0x55b798b9f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.486+0000 7f9877e82740 1 --2- 172.21.15.64:0/1734867360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987ce1b0 0x55b798b9ebe0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.486+0000 7f9877e82740 1 --2- 172.21.15.64:0/1734867360 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7987f7070 0x55b798b9e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.486+0000 7f9877e82740 1 -- 172.21.15.64:0/1734867360 >> 172.21.15.64:0/1734867360 conn(0x55b7987eb460 msgr2=0x55b798b9c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:59.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.486+0000 7f9877e82740 1 -- 172.21.15.64:0/1734867360 shutdown_connections 2024-09-21T04:13:59.492 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.486+0000 7f9877e82740 1 -- 172.21.15.64:0/1734867360 wait complete. 2024-09-21T04:13:59.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.486+0000 7f9877e82740 1 -- start start 2024-09-21T04:13:59.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.487+0000 7f9877e82740 1 Processor -- start 2024-09-21T04:13:59.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.487+0000 7f9877e82740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987ce1b0 0x55b798ba9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:59.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.487+0000 7f9877e82740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7987d6390 0x55b798baa240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:59.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.487+0000 7f9877e82740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7987f7070 0x55b798baa780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:59.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.487+0000 7f9877e82740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b798965700 con 0x55b7987ce1b0 2024-09-21T04:13:59.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.487+0000 7f9877e82740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b798a9f160 con 0x55b7987d6390 2024-09-21T04:13:59.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.487+0000 7f9877e82740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b798bace80 con 0x55b7987f7070 2024-09-21T04:13:59.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.487+0000 7f9877e82740 10 client.0 _ll_register_callbacks cb 0x55b798ab89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:59.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.487+0000 7f986a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987ce1b0 0x55b798ba9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:59.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.487+0000 7f986a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987ce1b0 0x55b798ba9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34230/0 (socket says 172.21.15.64:34230) 2024-09-21T04:13:59.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.487+0000 7f986a7fc640 1 -- 172.21.15.64:0/2114873618 learned_addr learned my addr 172.21.15.64:0/2114873618 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:59.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.487+0000 7f986affd640 1 --2- 172.21.15.64:0/2114873618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7987d6390 0x55b798baa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:59.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.487+0000 7f986a7fc640 1 -- 172.21.15.64:0/2114873618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7987f7070 msgr2=0x55b798baa780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:59.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.487+0000 7f986a7fc640 1 --2- 172.21.15.64:0/2114873618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7987f7070 0x55b798baa780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.487+0000 7f986a7fc640 1 -- 172.21.15.64:0/2114873618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7987d6390 msgr2=0x55b798baa240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:59.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.487+0000 7f986a7fc640 1 --2- 172.21.15.64:0/2114873618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7987d6390 0x55b798baa240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.487+0000 7f986a7fc640 1 -- 172.21.15.64:0/2114873618 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b798accc90 con 0x55b7987ce1b0 2024-09-21T04:13:59.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.488+0000 7f986a7fc640 1 --2- 172.21.15.64:0/2114873618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987ce1b0 0x55b798ba9d00 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f9860009940 tx=0x7f9860003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90d38e741f3b46d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:59.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.488+0000 7f984affd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:13:59.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.488+0000 7f984affd640 1 -- 172.21.15.64:0/2114873618 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860005020 con 0x55b7987ce1b0 2024-09-21T04:13:59.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.488+0000 7f984affd640 1 -- 172.21.15.64:0/2114873618 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8615+0+0 (secure 0 0 0) 0x7f98600051c0 con 0x55b7987ce1b0 2024-09-21T04:13:59.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.488+0000 7f9877e82740 1 -- 172.21.15.64:0/2114873618 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f985c003d10 con 0x55b7987ce1b0 2024-09-21T04:13:59.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.488+0000 7f984affd640 1 -- 172.21.15.64:0/2114873618 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f98600053c0 con 0x55b7987ce1b0 2024-09-21T04:13:59.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.488+0000 7f9877e82740 10 client.17538 fetch_fsmap learned FSMap version 41 2024-09-21T04:13:59.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.488+0000 7f9877e82740 1 -- 172.21.15.64:0/2114873618 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b798bbcf10 con 0x55b7987ce1b0 2024-09-21T04:13:59.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.488+0000 7f984affd640 1 -- 172.21.15.64:0/2114873618 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f985c003d10 con 0x55b7987ce1b0 2024-09-21T04:13:59.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.488+0000 7f984affd640 1 -- 172.21.15.64:0/2114873618 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f986001d030 con 0x55b7987ce1b0 2024-09-21T04:13:59.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.489+0000 7f9877e82740 10 client.17538 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T04:13:59.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.489+0000 7f9877e82740 10 client.17538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:13:59.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.489+0000 7f9877e82740 20 client.17538 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:59.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.489+0000 7f9877e82740 1 --2- 172.21.15.64:0/2114873618 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b7987b59f0 0x55b798bc0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:59.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.489+0000 7f9877e82740 4 client.17538 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T04:13:59.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.489+0000 7f9877e82740 1 -- 172.21.15.64:0/2114873618 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b798b16df0 con 0x55b7987b59f0 2024-09-21T04:13:59.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.489+0000 7f986affd640 1 --2- 172.21.15.64:0/2114873618 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b7987b59f0 0x55b798bc0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:59.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.489+0000 7f984affd640 10 client.17538 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:59.495 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.489+0000 7f986affd640 1 --2- 172.21.15.64:0/2114873618 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b7987b59f0 0x55b798bc0470 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5e13f05565ecfd8 server_cookie=a4dc80d07daaa411 in_seq=0 out_seq=0 2024-09-21T04:13:59.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.490+0000 7f984affd640 1 -- 172.21.15.64:0/2114873618 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b798b16df0 con 0x55b7987b59f0 2024-09-21T04:13:59.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.490+0000 7f984affd640 10 client.17538 handle_command_reply: tid=0 2024-09-21T04:13:59.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.490+0000 7f9877e82740 1 client.17538 shutdown 2024-09-21T04:13:59.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.490+0000 7f9877e82740 1 -- 172.21.15.64:0/2114873618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987ce1b0 msgr2=0x55b798ba9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:59.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.490+0000 7f9877e82740 1 --2- 172.21.15.64:0/2114873618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987ce1b0 0x55b798ba9d00 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f9860009940 tx=0x7f9860003a80 comp rx=0 tx=0).stop 2024-09-21T04:13:59.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.490+0000 7f9877e82740 1 -- 172.21.15.64:0/2114873618 shutdown_connections 2024-09-21T04:13:59.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.490+0000 7f9877e82740 1 --2- 172.21.15.64:0/2114873618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7987f7070 0x55b798baa780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.490+0000 7f9877e82740 1 --2- 172.21.15.64:0/2114873618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7987d6390 0x55b798baa240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.490+0000 7f9877e82740 1 --2- 172.21.15.64:0/2114873618 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b7987b59f0 0x55b798bc0470 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.490+0000 7f984affd640 0 client.17538 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:59.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.490+0000 7f9877e82740 1 --2- 172.21.15.64:0/2114873618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987ce1b0 0x55b798ba9d00 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.490+0000 7f9877e82740 1 -- 172.21.15.64:0/2114873618 >> 172.21.15.64:0/2114873618 conn(0x55b7987eb460 msgr2=0x55b798ba1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:59.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.490+0000 7f9877e82740 1 -- 172.21.15.64:0/2114873618 shutdown_connections 2024-09-21T04:13:59.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.490+0000 7f9877e82740 1 -- 172.21.15.64:0/2114873618 wait complete. 2024-09-21T04:13:59.497 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.491+0000 7f9877e82740 20 client.17538 trim_cache size 0 max 16384 2024-09-21T04:13:59.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.504+0000 7f9877e82740 1 Processor -- start 2024-09-21T04:13:59.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.504+0000 7f9877e82740 1 -- start start 2024-09-21T04:13:59.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.504+0000 7f9877e82740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987b59f0 0x55b798ba0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.504+0000 7f9877e82740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7987ce1b0 0x55b798bab930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.504+0000 7f9877e82740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7987d6390 0x55b798b9bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.504+0000 7f9877e82740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f986c1b00d0 con 0x55b7987b59f0 2024-09-21T04:13:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.504+0000 7f9877e82740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f986c1ad910 con 0x55b7987d6390 2024-09-21T04:13:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.504+0000 7f9877e82740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f986c1ab780 con 0x55b7987ce1b0 2024-09-21T04:13:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.504+0000 7f986a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987b59f0 0x55b798ba0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.504+0000 7f986a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987b59f0 0x55b798ba0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34246/0 (socket says 172.21.15.64:34246) 2024-09-21T04:13:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.504+0000 7f986a7fc640 1 -- 172.21.15.64:0/165999768 learned_addr learned my addr 172.21.15.64:0/165999768 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.504+0000 7f986a7fc640 1 -- 172.21.15.64:0/165999768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7987ce1b0 msgr2=0x55b798bab930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.504+0000 7f986affd640 1 --2- 172.21.15.64:0/165999768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7987ce1b0 0x55b798bab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.504+0000 7f986a7fc640 1 --2- 172.21.15.64:0/165999768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7987ce1b0 0x55b798bab930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.504+0000 7f986a7fc640 1 -- 172.21.15.64:0/165999768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7987d6390 msgr2=0x55b798b9bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.504+0000 7f986a7fc640 1 --2- 172.21.15.64:0/165999768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7987d6390 0x55b798b9bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.508 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.504+0000 7f986a7fc640 1 -- 172.21.15.64:0/165999768 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b798ba0bc0 con 0x55b7987b59f0 2024-09-21T04:13:59.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.504+0000 7f986affd640 1 --2- 172.21.15.64:0/165999768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7987ce1b0 0x55b798bab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:13:59.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.504+0000 7f986a7fc640 1 --2- 172.21.15.64:0/165999768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987b59f0 0x55b798ba0090 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f9860004a90 tx=0x7f9860009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d6d05972b65d66 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:59.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.505+0000 7f98517fa640 1 -- 172.21.15.64:0/165999768 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860005560 con 0x55b7987b59f0 2024-09-21T04:13:59.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.505+0000 7f98517fa640 1 -- 172.21.15.64:0/165999768 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9860005130 con 0x55b7987b59f0 2024-09-21T04:13:59.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.505+0000 7f98517fa640 1 -- 172.21.15.64:0/165999768 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860011d60 con 0x55b7987b59f0 2024-09-21T04:13:59.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.505+0000 7f9877e82740 1 -- 172.21.15.64:0/165999768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987b59f0 msgr2=0x55b798ba0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:59.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.505+0000 7f9877e82740 1 --2- 172.21.15.64:0/165999768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987b59f0 0x55b798ba0090 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f9860004a90 tx=0x7f9860009eb0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.505+0000 7f9877e82740 1 -- 172.21.15.64:0/165999768 shutdown_connections 2024-09-21T04:13:59.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.505+0000 7f9877e82740 1 --2- 172.21.15.64:0/165999768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7987d6390 0x55b798b9bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.505+0000 7f9877e82740 1 --2- 172.21.15.64:0/165999768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7987ce1b0 0x55b798bab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.505+0000 7f9877e82740 1 --2- 172.21.15.64:0/165999768 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987b59f0 0x55b798ba0090 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.505+0000 7f9877e82740 1 -- 172.21.15.64:0/165999768 >> 172.21.15.64:0/165999768 conn(0x55b7987eb460 msgr2=0x55b798bab220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:59.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.506+0000 7f9877e82740 1 -- 172.21.15.64:0/165999768 shutdown_connections 2024-09-21T04:13:59.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.506+0000 7f9877e82740 1 -- 172.21.15.64:0/165999768 wait complete. 2024-09-21T04:13:59.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.506+0000 7f9877e82740 1 -- start start 2024-09-21T04:13:59.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.506+0000 7f9877e82740 1 Processor -- start 2024-09-21T04:13:59.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.506+0000 7f9877e82740 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7987b59f0 0x55b798bb6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:59.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.507+0000 7f9877e82740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7987ce1b0 0x55b798bb20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:59.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.507+0000 7f9877e82740 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987d6390 0x55b798bb25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:59.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.507+0000 7f9877e82740 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f986c1abdd0 con 0x55b7987d6390 2024-09-21T04:13:59.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.507+0000 7f9877e82740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f986c1ad240 con 0x55b7987ce1b0 2024-09-21T04:13:59.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.507+0000 7f9877e82740 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f986c1ac4a0 con 0x55b7987b59f0 2024-09-21T04:13:59.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.507+0000 7f9877e82740 10 client.0 _ll_register_callbacks cb 0x7f9860004420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T04:13:59.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.507+0000 7f986a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987d6390 0x55b798bb25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:59.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.507+0000 7f986a7fc640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987d6390 0x55b798bb25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34250/0 (socket says 172.21.15.64:34250) 2024-09-21T04:13:59.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.507+0000 7f986a7fc640 1 -- 172.21.15.64:0/919843376 learned_addr learned my addr 172.21.15.64:0/919843376 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:13:59.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.507+0000 7f986a7fc640 1 --2- 172.21.15.64:0/919843376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7987b59f0 0x55b798bb6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:59.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.507+0000 7f986affd640 1 --2- 172.21.15.64:0/919843376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7987ce1b0 0x55b798bb20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:59.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.507+0000 7f986a7fc640 1 -- 172.21.15.64:0/919843376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7987b59f0 msgr2=0x55b798bb6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:59.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.507+0000 7f986a7fc640 1 --2- 172.21.15.64:0/919843376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7987b59f0 0x55b798bb6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.507+0000 7f986a7fc640 1 -- 172.21.15.64:0/919843376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7987ce1b0 msgr2=0x55b798bb20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T04:13:59.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.507+0000 7f986a7fc640 1 --2- 172.21.15.64:0/919843376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7987ce1b0 0x55b798bb20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.507+0000 7f986a7fc640 1 -- 172.21.15.64:0/919843376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b798bb2c00 con 0x55b7987d6390 2024-09-21T04:13:59.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.507+0000 7f986a7fc640 1 --2- 172.21.15.64:0/919843376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987d6390 0x55b798bb25f0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f98600043f0 tx=0x7f9860003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51d8132d3a7ed3f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:13:59.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.507+0000 7f984affd640 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-09-21T04:13:59.514 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.507+0000 7f984affd640 1 -- 172.21.15.64:0/919843376 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860011ce0 con 0x55b7987d6390 2024-09-21T04:13:59.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.507+0000 7f984affd640 1 -- 172.21.15.64:0/919843376 <== mon.0 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8615+0+0 (secure 0 0 0) 0x7f986001e530 con 0x55b7987d6390 2024-09-21T04:13:59.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.507+0000 7f9877e82740 1 -- 172.21.15.64:0/919843376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9860005130 con 0x55b7987d6390 2024-09-21T04:13:59.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.508+0000 7f984affd640 1 -- 172.21.15.64:0/919843376 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f986001e730 con 0x55b7987d6390 2024-09-21T04:13:59.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.508+0000 7f9877e82740 10 client.17544 fetch_fsmap learned FSMap version 41 2024-09-21T04:13:59.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.508+0000 7f9877e82740 1 -- 172.21.15.64:0/919843376 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b798c2ab70 con 0x55b7987d6390 2024-09-21T04:13:59.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.508+0000 7f984affd640 1 -- 172.21.15.64:0/919843376 <== mon.0 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9860005130 con 0x55b7987d6390 2024-09-21T04:13:59.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.508+0000 7f984affd640 1 -- 172.21.15.64:0/919843376 <== mon.0 v2:172.21.15.64:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f986001d040 con 0x55b7987d6390 2024-09-21T04:13:59.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.508+0000 7f9877e82740 10 client.17544 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T04:13:59.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.508+0000 7f9877e82740 10 client.17544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T04:13:59.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.508+0000 7f9877e82740 20 client.17544 populate_metadata read hostname 'smithi064' 2024-09-21T04:13:59.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.508+0000 7f9877e82740 1 --2- 172.21.15.64:0/919843376 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b7987f7070 0x55b798c2d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:13:59.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.508+0000 7f9877e82740 4 client.17544 mds_command: new command op to 24710 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T04:13:59.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.508+0000 7f9877e82740 1 -- 172.21.15.64:0/919843376 --> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b798b128a0 con 0x55b7987f7070 2024-09-21T04:13:59.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.509+0000 7f986affd640 1 --2- 172.21.15.64:0/919843376 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b7987f7070 0x55b798c2d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:13:59.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.509+0000 7f986affd640 1 --2- 172.21.15.64:0/919843376 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b7987f7070 0x55b798c2d260 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab813df3b6123fff server_cookie=ca93009847cf5437 in_seq=0 out_seq=0 2024-09-21T04:13:59.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.509+0000 7f984affd640 10 client.17544 ms_handle_connect on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:59.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.509+0000 7f984affd640 1 -- 172.21.15.64:0/919843376 <== mds.0 v2:172.21.15.119:6838/1319793636 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b798b128a0 con 0x55b7987f7070 2024-09-21T04:13:59.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.509+0000 7f984affd640 10 client.17544 handle_command_reply: tid=0 2024-09-21T04:13:59.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.509+0000 7f9877e82740 1 client.17544 shutdown 2024-09-21T04:13:59.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.509+0000 7f9877e82740 1 -- 172.21.15.64:0/919843376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987d6390 msgr2=0x55b798bb25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:59.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.509+0000 7f9877e82740 1 --2- 172.21.15.64:0/919843376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987d6390 0x55b798bb25f0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f98600043f0 tx=0x7f9860003a80 comp rx=0 tx=0).stop 2024-09-21T04:13:59.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.510+0000 7f9877e82740 1 -- 172.21.15.64:0/919843376 shutdown_connections 2024-09-21T04:13:59.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.510+0000 7f9877e82740 1 --2- 172.21.15.64:0/919843376 >> [v2:172.21.15.119:6838/1319793636,v1:172.21.15.119:6839/1319793636] conn(0x55b7987f7070 0x55b798c2d260 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.510+0000 7f9877e82740 1 --2- 172.21.15.64:0/919843376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b7987d6390 0x55b798bb25f0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.510+0000 7f9877e82740 1 --2- 172.21.15.64:0/919843376 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b7987ce1b0 0x55b798bb20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.510+0000 7f984affd640 0 client.17544 ms_handle_reset on v2:172.21.15.119:6838/1319793636 2024-09-21T04:13:59.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.510+0000 7f9877e82740 1 --2- 172.21.15.64:0/919843376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7987b59f0 0x55b798bb6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.510+0000 7f9877e82740 1 -- 172.21.15.64:0/919843376 >> 172.21.15.64:0/919843376 conn(0x55b7987eb460 msgr2=0x55b798bb9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:59.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.510+0000 7f9877e82740 1 -- 172.21.15.64:0/919843376 shutdown_connections 2024-09-21T04:13:59.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.510+0000 7f9877e82740 1 -- 172.21.15.64:0/919843376 wait complete. 2024-09-21T04:13:59.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.510+0000 7f9877e82740 20 client.17544 trim_cache size 0 max 16384 2024-09-21T04:13:59.517 INFO:teuthology.orchestra.run.smithi064.stdout:[] 2024-09-21T04:13:59.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.510+0000 7f987190e640 1 -- 172.21.15.64:0/3961964101 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f98540b6730 msgr2=0x7f98540b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:59.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.510+0000 7f987190e640 1 --2- 172.21.15.64:0/3961964101 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f98540b6730 0x7f98540b8c40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f98640cd880 tx=0x7f986409ae60 comp rx=0 tx=0).stop 2024-09-21T04:13:59.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.510+0000 7f987190e640 1 -- 172.21.15.64:0/3961964101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f986c10de10 msgr2=0x7f986c19fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:13:59.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.510+0000 7f987190e640 1 --2- 172.21.15.64:0/3961964101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f986c10de10 0x7f986c19fde0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f9864004770 tx=0x7f9864004270 comp rx=0 tx=0).stop 2024-09-21T04:13:59.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.510+0000 7f987190e640 1 -- 172.21.15.64:0/3961964101 shutdown_connections 2024-09-21T04:13:59.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.510+0000 7f987190e640 1 --2- 172.21.15.64:0/3961964101 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f98540b6730 0x7f98540b8c40 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.510+0000 7f987190e640 1 --2- 172.21.15.64:0/3961964101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f986c10de10 0x7f986c19fde0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.510+0000 7f987190e640 1 --2- 172.21.15.64:0/3961964101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f986c105300 0x7f986c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.510+0000 7f987190e640 1 --2- 172.21.15.64:0/3961964101 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f986c1049b0 0x7f986c19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:13:59.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.510+0000 7f987190e640 1 -- 172.21.15.64:0/3961964101 >> 172.21.15.64:0/3961964101 conn(0x7f986c079730 msgr2=0x7f986c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:13:59.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.511+0000 7f987190e640 1 -- 172.21.15.64:0/3961964101 shutdown_connections 2024-09-21T04:13:59.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:13:59.511+0000 7f987190e640 1 -- 172.21.15.64:0/3961964101 wait complete. 2024-09-21T04:14:00.383 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-21T04:14:00.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.503+0000 7fb8a4f77640 1 -- 172.21.15.64:0/3952098513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8a0103b00 msgr2=0x7fb8a0103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:00.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.503+0000 7fb8a4f77640 1 --2- 172.21.15.64:0/3952098513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8a0103b00 0x7fb8a0103f10 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb890009900 tx=0x7fb89002a410 comp rx=0 tx=0).stop 2024-09-21T04:14:00.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.503+0000 7fb8a4f77640 1 -- 172.21.15.64:0/3952098513 shutdown_connections 2024-09-21T04:14:00.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.503+0000 7fb8a4f77640 1 --2- 172.21.15.64:0/3952098513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8a0109d80 0x7fb8a01105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:00.506 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.503+0000 7fb8a4f77640 1 --2- 172.21.15.64:0/3952098513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8a0109430 0x7fb8a0109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:00.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.503+0000 7fb8a4f77640 1 --2- 172.21.15.64:0/3952098513 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8a0103b00 0x7fb8a0103f10 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:00.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.503+0000 7fb8a4f77640 1 -- 172.21.15.64:0/3952098513 >> 172.21.15.64:0/3952098513 conn(0x7fb8a00ff500 msgr2=0x7fb8a0101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:00.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.503+0000 7fb8a4f77640 1 -- 172.21.15.64:0/3952098513 shutdown_connections 2024-09-21T04:14:00.507 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.503+0000 7fb8a4f77640 1 -- 172.21.15.64:0/3952098513 wait complete. 2024-09-21T04:14:00.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.504+0000 7fb8a4f77640 1 Processor -- start 2024-09-21T04:14:00.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.504+0000 7fb8a4f77640 1 -- start start 2024-09-21T04:14:00.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.504+0000 7fb8a4f77640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8a0103b00 0x7fb8a0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:00.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.504+0000 7fb8a4f77640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8a0109430 0x7fb8a0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:00.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.504+0000 7fb8a4f77640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8a0109d80 0x7fb8a0076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:00.509 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.504+0000 7fb8a4f77640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a0113a40 con 0x7fb8a0103b00 2024-09-21T04:14:00.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.504+0000 7fb8a4f77640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a01138c0 con 0x7fb8a0109d80 2024-09-21T04:14:00.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.504+0000 7fb8a4f77640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a0113bc0 con 0x7fb8a0109430 2024-09-21T04:14:00.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.504+0000 7fb89e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8a0103b00 0x7fb8a0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:00.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.504+0000 7fb89e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8a0103b00 0x7fb8a0075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34274/0 (socket says 172.21.15.64:34274) 2024-09-21T04:14:00.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.504+0000 7fb89e59c640 1 -- 172.21.15.64:0/4188061642 learned_addr learned my addr 172.21.15.64:0/4188061642 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:14:00.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.504+0000 7fb89ed9d640 1 --2- 172.21.15.64:0/4188061642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8a0109d80 0x7fb8a0076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:00.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.504+0000 7fb895d9b640 1 --2- 172.21.15.64:0/4188061642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8a0109430 0x7fb8a0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:00.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.505+0000 7fb89e59c640 1 -- 172.21.15.64:0/4188061642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8a0109430 msgr2=0x7fb8a0075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:00.510 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.505+0000 7fb89e59c640 1 --2- 172.21.15.64:0/4188061642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8a0109430 0x7fb8a0075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:00.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.505+0000 7fb89e59c640 1 -- 172.21.15.64:0/4188061642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8a0109d80 msgr2=0x7fb8a0076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:00.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.505+0000 7fb89e59c640 1 --2- 172.21.15.64:0/4188061642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8a0109d80 0x7fb8a0076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:00.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.505+0000 7fb89e59c640 1 -- 172.21.15.64:0/4188061642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a01a8750 con 0x7fb8a0103b00 2024-09-21T04:14:00.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.505+0000 7fb89e59c640 1 --2- 172.21.15.64:0/4188061642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8a0103b00 0x7fb8a0075710 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fb890005940 tx=0x7fb890004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:00.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.505+0000 7fb897fff640 1 -- 172.21.15.64:0/4188061642 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89003f030 con 0x7fb8a0103b00 2024-09-21T04:14:00.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.505+0000 7fb8a4f77640 1 -- 172.21.15.64:0/4188061642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a01a89e0 con 0x7fb8a0103b00 2024-09-21T04:14:00.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.505+0000 7fb897fff640 1 -- 172.21.15.64:0/4188061642 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb890003aa0 con 0x7fb8a0103b00 2024-09-21T04:14:00.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.505+0000 7fb897fff640 1 -- 172.21.15.64:0/4188061642 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb890035070 con 0x7fb8a0103b00 2024-09-21T04:14:00.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.505+0000 7fb8a4f77640 1 -- 172.21.15.64:0/4188061642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a01a8f20 con 0x7fb8a0103b00 2024-09-21T04:14:00.511 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.506+0000 7fb8a4f77640 1 -- 172.21.15.64:0/4188061642 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8a0103f10 con 0x7fb8a0103b00 2024-09-21T04:14:00.512 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.508+0000 7fb897fff640 1 -- 172.21.15.64:0/4188061642 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fb890042030 con 0x7fb8a0103b00 2024-09-21T04:14:00.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:00 smithi064 ceph-mon[31180]: pgmap v628: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T04:14:00.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.509+0000 7fb897fff640 1 --2- 172.21.15.64:0/4188061642 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb88c0b66b0 0x7fb88c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:00.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.509+0000 7fb895d9b640 1 --2- 172.21.15.64:0/4188061642 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb88c0b66b0 0x7fb88c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:00.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.509+0000 7fb897fff640 1 -- 172.21.15.64:0/4188061642 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8615+0+0 (secure 0 0 0) 0x7fb890108080 con 0x7fb8a0103b00 2024-09-21T04:14:00.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.509+0000 7fb895d9b640 1 --2- 172.21.15.64:0/4188061642 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb88c0b66b0 0x7fb88c0b8bc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0102d50 tx=0x7fb888005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:00.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.513+0000 7fb897fff640 1 -- 172.21.15.64:0/4188061642 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb8900d2950 con 0x7fb8a0103b00 2024-09-21T04:14:00.728 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.725+0000 7fb8a4f77640 1 -- 172.21.15.64:0/4188061642 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fb8a0063800 con 0x7fb88c0b66b0 2024-09-21T04:14:00.735 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.733+0000 7fb897fff640 1 -- 172.21.15.64:0/4188061642 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fb8a0063800 con 0x7fb88c0b66b0 2024-09-21T04:14:00.736 INFO:teuthology.orchestra.run.smithi064.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-21T00:00:00", "created": "2024-09-21T04:03:59", "first": "2024-09-21T04:04:00", "last": "2024-09-21T04:14:00", "last_pruned": "2024-09-21T04:14:00", "created_count": 11, "pruned_count": 5, "active": true} 2024-09-21T04:14:00.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.735+0000 7fb8a4f77640 1 -- 172.21.15.64:0/4188061642 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb88c0b66b0 msgr2=0x7fb88c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:00.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.735+0000 7fb8a4f77640 1 --2- 172.21.15.64:0/4188061642 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb88c0b66b0 0x7fb88c0b8bc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0102d50 tx=0x7fb888005f90 comp rx=0 tx=0).stop 2024-09-21T04:14:00.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.735+0000 7fb8a4f77640 1 -- 172.21.15.64:0/4188061642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8a0103b00 msgr2=0x7fb8a0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:00.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.735+0000 7fb8a4f77640 1 --2- 172.21.15.64:0/4188061642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8a0103b00 0x7fb8a0075710 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fb890005940 tx=0x7fb890004270 comp rx=0 tx=0).stop 2024-09-21T04:14:00.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.735+0000 7fb8a4f77640 1 -- 172.21.15.64:0/4188061642 shutdown_connections 2024-09-21T04:14:00.738 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.735+0000 7fb8a4f77640 1 --2- 172.21.15.64:0/4188061642 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb88c0b66b0 0x7fb88c0b8bc0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:00.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.735+0000 7fb8a4f77640 1 --2- 172.21.15.64:0/4188061642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8a0109d80 0x7fb8a0076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:00.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.735+0000 7fb8a4f77640 1 --2- 172.21.15.64:0/4188061642 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb8a0109430 0x7fb8a0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:00.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.735+0000 7fb8a4f77640 1 --2- 172.21.15.64:0/4188061642 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb8a0103b00 0x7fb8a0075710 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:00.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.735+0000 7fb8a4f77640 1 -- 172.21.15.64:0/4188061642 >> 172.21.15.64:0/4188061642 conn(0x7fb8a00ff500 msgr2=0x7fb8a0108ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:00.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.735+0000 7fb8a4f77640 1 -- 172.21.15.64:0/4188061642 shutdown_connections 2024-09-21T04:14:00.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.735+0000 7fb8a4f77640 1 -- 172.21.15.64:0/4188061642 wait complete. 2024-09-21T04:14:00.749 DEBUG:teuthology.orchestra.run.smithi064:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-21T04:14:00.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:00 smithi119 ceph-mon[34811]: pgmap v628: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T04:14:00.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:00 smithi131 ceph-mon[35193]: pgmap v628: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-21T04:14:00.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.862+0000 7f5885fb0640 1 -- 172.21.15.64:0/3789070799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58801029b0 msgr2=0x7f5880102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:00.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.862+0000 7f5885fb0640 1 --2- 172.21.15.64:0/3789070799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58801029b0 0x7f5880102dc0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f5870009900 tx=0x7f587002a410 comp rx=0 tx=0).stop 2024-09-21T04:14:00.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.862+0000 7f5885fb0640 1 -- 172.21.15.64:0/3789070799 shutdown_connections 2024-09-21T04:14:00.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.862+0000 7f5885fb0640 1 --2- 172.21.15.64:0/3789070799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5880108c30 0x7f588010f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:00.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.862+0000 7f5885fb0640 1 --2- 172.21.15.64:0/3789070799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58801082e0 0x7f58801086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:00.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.862+0000 7f5885fb0640 1 --2- 172.21.15.64:0/3789070799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58801029b0 0x7f5880102dc0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:00.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.862+0000 7f5885fb0640 1 -- 172.21.15.64:0/3789070799 >> 172.21.15.64:0/3789070799 conn(0x7f588007a740 msgr2=0x7f588007ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:00.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.862+0000 7f5885fb0640 1 -- 172.21.15.64:0/3789070799 shutdown_connections 2024-09-21T04:14:00.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.862+0000 7f5885fb0640 1 -- 172.21.15.64:0/3789070799 wait complete. 2024-09-21T04:14:00.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.863+0000 7f5885fb0640 1 Processor -- start 2024-09-21T04:14:00.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.863+0000 7f5885fb0640 1 -- start start 2024-09-21T04:14:00.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.863+0000 7f5885fb0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58801029b0 0x7f58801993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:00.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.863+0000 7f5885fb0640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58801082e0 0x7f5880199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:00.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.863+0000 7f5885fb0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5880108c30 0x7f588019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:00.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.863+0000 7f5885fb0640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f58801128f0 con 0x7f58801082e0 2024-09-21T04:14:00.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.863+0000 7f5885fb0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5880112770 con 0x7f5880108c30 2024-09-21T04:14:00.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.863+0000 7f5885fb0640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5880112a70 con 0x7f58801029b0 2024-09-21T04:14:00.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.863+0000 7f587effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58801082e0 0x7f5880199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:00.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.863+0000 7f587effd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58801082e0 0x7f5880199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34290/0 (socket says 172.21.15.64:34290) 2024-09-21T04:14:00.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.863+0000 7f587effd640 1 -- 172.21.15.64:0/3455193238 learned_addr learned my addr 172.21.15.64:0/3455193238 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:14:00.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.864+0000 7f587f7fe640 1 --2- 172.21.15.64:0/3455193238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58801029b0 0x7f58801993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:00.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.864+0000 7f587ffff640 1 --2- 172.21.15.64:0/3455193238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5880108c30 0x7f588019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:00.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.864+0000 7f587effd640 1 -- 172.21.15.64:0/3455193238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58801029b0 msgr2=0x7f58801993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:00.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.864+0000 7f587effd640 1 --2- 172.21.15.64:0/3455193238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58801029b0 0x7f58801993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:00.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.864+0000 7f587effd640 1 -- 172.21.15.64:0/3455193238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5880108c30 msgr2=0x7f588019ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:00.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.864+0000 7f587effd640 1 --2- 172.21.15.64:0/3455193238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5880108c30 0x7f588019ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:00.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.864+0000 7f587effd640 1 -- 172.21.15.64:0/3455193238 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588019f270 con 0x7f58801082e0 2024-09-21T04:14:00.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.864+0000 7f587ffff640 1 --2- 172.21.15.64:0/3455193238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5880108c30 0x7f588019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:14:00.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.864+0000 7f587effd640 1 --2- 172.21.15.64:0/3455193238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58801082e0 0x7f5880199970 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f587400d950 tx=0x7f587400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:00.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.864+0000 7f587cff9640 1 -- 172.21.15.64:0/3455193238 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874017070 con 0x7f58801082e0 2024-09-21T04:14:00.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.864+0000 7f587cff9640 1 -- 172.21.15.64:0/3455193238 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5874009c30 con 0x7f58801082e0 2024-09-21T04:14:00.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.864+0000 7f587cff9640 1 -- 172.21.15.64:0/3455193238 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58740047a0 con 0x7f58801082e0 2024-09-21T04:14:00.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.864+0000 7f5885fb0640 1 -- 172.21.15.64:0/3455193238 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f588019f560 con 0x7f58801082e0 2024-09-21T04:14:00.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.864+0000 7f5885fb0640 1 -- 172.21.15.64:0/3455193238 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58801a5d80 con 0x7f58801082e0 2024-09-21T04:14:00.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.865+0000 7f5885fb0640 1 -- 172.21.15.64:0/3455193238 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5880103c20 con 0x7f58801082e0 2024-09-21T04:14:00.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.867+0000 7f587cff9640 1 -- 172.21.15.64:0/3455193238 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f5874004020 con 0x7f58801082e0 2024-09-21T04:14:00.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.868+0000 7f587cff9640 1 --2- 172.21.15.64:0/3455193238 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f58680b6680 0x7f58680b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:00.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.868+0000 7f587cff9640 1 -- 172.21.15.64:0/3455193238 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8615+0+0 (secure 0 0 0) 0x7f58740e7900 con 0x7f58801082e0 2024-09-21T04:14:00.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.868+0000 7f587f7fe640 1 --2- 172.21.15.64:0/3455193238 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f58680b6680 0x7f58680b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:00.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.868+0000 7f587f7fe640 1 --2- 172.21.15.64:0/3455193238 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f58680b6680 0x7f58680b8b90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5870031440 tx=0x7f5870005d50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:00.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:00.871+0000 7f587cff9640 1 -- 172.21.15.64:0/3455193238 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5874020020 con 0x7f58801082e0 2024-09-21T04:14:01.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.085+0000 7f5885fb0640 1 -- 172.21.15.64:0/3455193238 --> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f5880063800 con 0x7f58680b6680 2024-09-21T04:14:01.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.093+0000 7f587cff9640 1 -- 172.21.15.64:0/3455193238 <== mgr.14226 v2:172.21.15.64:6800/834288654 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f5880063800 con 0x7f58680b6680 2024-09-21T04:14:01.096 INFO:teuthology.orchestra.run.smithi064.stdout:/ 1m 6m3h 2024-09-21T04:14:01.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.096+0000 7f5885fb0640 1 -- 172.21.15.64:0/3455193238 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f58680b6680 msgr2=0x7f58680b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:01.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.096+0000 7f5885fb0640 1 --2- 172.21.15.64:0/3455193238 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f58680b6680 0x7f58680b8b90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5870031440 tx=0x7f5870005d50 comp rx=0 tx=0).stop 2024-09-21T04:14:01.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.096+0000 7f5885fb0640 1 -- 172.21.15.64:0/3455193238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58801082e0 msgr2=0x7f5880199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:01.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.096+0000 7f5885fb0640 1 --2- 172.21.15.64:0/3455193238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58801082e0 0x7f5880199970 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f587400d950 tx=0x7f587400de10 comp rx=0 tx=0).stop 2024-09-21T04:14:01.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.096+0000 7f5885fb0640 1 -- 172.21.15.64:0/3455193238 shutdown_connections 2024-09-21T04:14:01.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.096+0000 7f5885fb0640 1 --2- 172.21.15.64:0/3455193238 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f58680b6680 0x7f58680b8b90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.096+0000 7f5885fb0640 1 --2- 172.21.15.64:0/3455193238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5880108c30 0x7f588019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.096+0000 7f5885fb0640 1 --2- 172.21.15.64:0/3455193238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58801082e0 0x7f5880199970 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.096+0000 7f5885fb0640 1 --2- 172.21.15.64:0/3455193238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58801029b0 0x7f58801993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.099 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.096+0000 7f5885fb0640 1 -- 172.21.15.64:0/3455193238 >> 172.21.15.64:0/3455193238 conn(0x7f588007a740 msgr2=0x7f5880106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:01.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.096+0000 7f5885fb0640 1 -- 172.21.15.64:0/3455193238 shutdown_connections 2024-09-21T04:14:01.100 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.096+0000 7f5885fb0640 1 -- 172.21.15.64:0/3455193238 wait complete. 2024-09-21T04:14:01.110 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-21T04:14:01.125 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-21T04:14:01.125 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/thrasher.py", line 50, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-21T04:14:01.126 INFO:tasks.fwd_scrub:done joining 2024-09-21T04:14:01.126 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-21T04:14:01.160 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T04:14:01.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.271+0000 7f1aba994640 1 -- 172.21.15.64:0/816399767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ab4104790 msgr2=0x7f1ab4104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:01.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.271+0000 7f1aba994640 1 --2- 172.21.15.64:0/816399767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ab4104790 0x7f1ab4104ba0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009900 tx=0x7f1aa402a410 comp rx=0 tx=0).stop 2024-09-21T04:14:01.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.271+0000 7f1aba994640 1 -- 172.21.15.64:0/816399767 shutdown_connections 2024-09-21T04:14:01.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.271+0000 7f1aba994640 1 --2- 172.21.15.64:0/816399767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab410dd40 0x7f1ab4110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.271+0000 7f1aba994640 1 --2- 172.21.15.64:0/816399767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ab41051c0 0x7f1ab410d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.271+0000 7f1aba994640 1 --2- 172.21.15.64:0/816399767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ab4104790 0x7f1ab4104ba0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.271+0000 7f1aba994640 1 -- 172.21.15.64:0/816399767 >> 172.21.15.64:0/816399767 conn(0x7f1ab40fe380 msgr2=0x7f1ab41007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:01.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.271+0000 7f1aba994640 1 -- 172.21.15.64:0/816399767 shutdown_connections 2024-09-21T04:14:01.274 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.271+0000 7f1aba994640 1 -- 172.21.15.64:0/816399767 wait complete. 2024-09-21T04:14:01.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.271+0000 7f1aba994640 1 Processor -- start 2024-09-21T04:14:01.275 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.272+0000 7f1aba994640 1 -- start start 2024-09-21T04:14:01.276 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.272+0000 7f1aba994640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ab4104790 0x7f1ab419a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:01.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.272+0000 7f1aba994640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab41051c0 0x7f1ab419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:01.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.272+0000 7f1aba994640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ab410dd40 0x7f1ab419fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:01.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.272+0000 7f1aba994640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab41139a0 con 0x7f1ab4104790 2024-09-21T04:14:01.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.272+0000 7f1aba994640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab4113820 con 0x7f1ab41051c0 2024-09-21T04:14:01.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.272+0000 7f1aba994640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab4113b20 con 0x7f1ab410dd40 2024-09-21T04:14:01.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.272+0000 7f1ab3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ab4104790 0x7f1ab419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:01.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.272+0000 7f1ab3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ab4104790 0x7f1ab419a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:34312/0 (socket says 172.21.15.64:34312) 2024-09-21T04:14:01.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.272+0000 7f1ab3fff640 1 -- 172.21.15.64:0/2589776052 learned_addr learned my addr 172.21.15.64:0/2589776052 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:14:01.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.272+0000 7f1ab37fe640 1 --2- 172.21.15.64:0/2589776052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab41051c0 0x7f1ab419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:01.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.272+0000 7f1ab8f31640 1 --2- 172.21.15.64:0/2589776052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ab410dd40 0x7f1ab419fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:01.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.272+0000 7f1ab3fff640 1 -- 172.21.15.64:0/2589776052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ab410dd40 msgr2=0x7f1ab419fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:01.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.272+0000 7f1ab3fff640 1 --2- 172.21.15.64:0/2589776052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ab410dd40 0x7f1ab419fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.272+0000 7f1ab3fff640 1 -- 172.21.15.64:0/2589776052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab41051c0 msgr2=0x7f1ab419a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:01.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.272+0000 7f1ab3fff640 1 --2- 172.21.15.64:0/2589776052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab41051c0 0x7f1ab419a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.272+0000 7f1ab3fff640 1 -- 172.21.15.64:0/2589776052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab41a02b0 con 0x7f1ab4104790 2024-09-21T04:14:01.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.272+0000 7f1ab37fe640 1 --2- 172.21.15.64:0/2589776052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab41051c0 0x7f1ab419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:14:01.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.272+0000 7f1ab8f31640 1 --2- 172.21.15.64:0/2589776052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ab410dd40 0x7f1ab419fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:14:01.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.273+0000 7f1ab3fff640 1 --2- 172.21.15.64:0/2589776052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ab4104790 0x7f1ab419a380 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4005940 tx=0x7f1aa40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:01.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.273+0000 7f1ab17fa640 1 -- 172.21.15.64:0/2589776052 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa403f030 con 0x7f1ab4104790 2024-09-21T04:14:01.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.273+0000 7f1ab17fa640 1 -- 172.21.15.64:0/2589776052 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aa4003cb0 con 0x7f1ab4104790 2024-09-21T04:14:01.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.273+0000 7f1ab17fa640 1 -- 172.21.15.64:0/2589776052 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4035070 con 0x7f1ab4104790 2024-09-21T04:14:01.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.273+0000 7f1aba994640 1 -- 172.21.15.64:0/2589776052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab41a0540 con 0x7f1ab4104790 2024-09-21T04:14:01.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.273+0000 7f1aba994640 1 -- 172.21.15.64:0/2589776052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab41a6d30 con 0x7f1ab4104790 2024-09-21T04:14:01.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.274+0000 7f1aba994640 1 -- 172.21.15.64:0/2589776052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab4104ba0 con 0x7f1ab4104790 2024-09-21T04:14:01.279 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.276+0000 7f1ab17fa640 1 -- 172.21.15.64:0/2589776052 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f1aa4042030 con 0x7f1ab4104790 2024-09-21T04:14:01.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.277+0000 7f1ab17fa640 1 --2- 172.21.15.64:0/2589776052 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f1a9c0b6700 0x7f1a9c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:01.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.277+0000 7f1ab37fe640 1 --2- 172.21.15.64:0/2589776052 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f1a9c0b6700 0x7f1a9c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:01.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.277+0000 7f1ab17fa640 1 -- 172.21.15.64:0/2589776052 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8615+0+0 (secure 0 0 0) 0x7f1aa41083e0 con 0x7f1ab4104790 2024-09-21T04:14:01.280 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.278+0000 7f1ab37fe640 1 --2- 172.21.15.64:0/2589776052 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f1a9c0b6700 0x7f1a9c0b8c10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80058c0 tx=0x7f1aa8005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:01.283 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.280+0000 7f1ab17fa640 1 -- 172.21.15.64:0/2589776052 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1aa40ce870 con 0x7f1ab4104790 2024-09-21T04:14:01.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.472+0000 7f1aba994640 1 -- 172.21.15.64:0/2589776052 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ab41a7010 con 0x7f1ab4104790 2024-09-21T04:14:01.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.474+0000 7f1ab17fa640 1 -- 172.21.15.64:0/2589776052 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+23320 (secure 0 0 0) 0x7f1aa40d2c50 con 0x7f1ab4104790 2024-09-21T04:14:01.477 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:14:01.477 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":85,"fsid":"e7c927ba-77cc-11ef-baf5-efdc52797490","created":"2024-09-21T03:53:41.215073+0000","modified":"2024-09-21T04:13:21.222090+0000","last_up_change":"2024-09-21T04:01:11.986949+0000","last_in_change":"2024-09-21T04:00:48.707791+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:57:24.854438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T04:01:58.728355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T04:02:01.300385+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":84,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb947c10-98e6-4e3a-8bbd-8753951d870b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6803","nonce":3692023035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6805","nonce":3692023035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6809","nonce":3692023035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6807","nonce":3692023035}]},"public_addr":"172.21.15.64:6803/3692023035","cluster_addr":"172.21.15.64:6805/3692023035","heartbeat_back_addr":"172.21.15.64:6809/3692023035","heartbeat_front_addr":"172.21.15.64:6807/3692023035","state":["exists","up"]},{"osd":1,"uuid":"29b8c92d-82fa-4175-b3e2-711917b7f8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6801","nonce":3363415998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6803","nonce":3363415998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6807","nonce":3363415998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6805","nonce":3363415998}]},"public_addr":"172.21.15.119:6801/3363415998","cluster_addr":"172.21.15.119:6803/3363415998","heartbeat_back_addr":"172.21.15.119:6807/3363415998","heartbeat_front_addr":"172.21.15.119:6805/3363415998","state":["exists","up"]},{"osd":2,"uuid":"a2273fe8-df55-40f7-afda-74ab09c52c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6801","nonce":3422924179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6803","nonce":3422924179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6807","nonce":3422924179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6805","nonce":3422924179}]},"public_addr":"172.21.15.131:6801/3422924179","cluster_addr":"172.21.15.131:6803/3422924179","heartbeat_back_addr":"172.21.15.131:6807/3422924179","heartbeat_front_addr":"172.21.15.131:6805/3422924179","state":["exists","up"]},{"osd":3,"uuid":"f1634064-b108-41d2-a78f-c33607b08a70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6811","nonce":2022952729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6813","nonce":2022952729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6817","nonce":2022952729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6815","nonce":2022952729}]},"public_addr":"172.21.15.64:6811/2022952729","cluster_addr":"172.21.15.64:6813/2022952729","heartbeat_back_addr":"172.21.15.64:6817/2022952729","heartbeat_front_addr":"172.21.15.64:6815/2022952729","state":["exists","up"]},{"osd":4,"uuid":"75b81a8d-fa52-411a-bfda-aa69bb70a20f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6809","nonce":323030958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6811","nonce":323030958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6815","nonce":323030958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6813","nonce":323030958}]},"public_addr":"172.21.15.119:6809/323030958","cluster_addr":"172.21.15.119:6811/323030958","heartbeat_back_addr":"172.21.15.119:6815/323030958","heartbeat_front_addr":"172.21.15.119:6813/323030958","state":["exists","up"]},{"osd":5,"uuid":"0e4895b5-42e4-4b2c-a9c3-692f287cb841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6809","nonce":3021788088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6811","nonce":3021788088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6815","nonce":3021788088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6813","nonce":3021788088}]},"public_addr":"172.21.15.131:6809/3021788088","cluster_addr":"172.21.15.131:6811/3021788088","heartbeat_back_addr":"172.21.15.131:6815/3021788088","heartbeat_front_addr":"172.21.15.131:6813/3021788088","state":["exists","up"]},{"osd":6,"uuid":"1fa5c2b7-bce0-43a1-a26b-d029eb3731fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6819","nonce":1913824324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6821","nonce":1913824324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6825","nonce":1913824324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6823","nonce":1913824324}]},"public_addr":"172.21.15.64:6819/1913824324","cluster_addr":"172.21.15.64:6821/1913824324","heartbeat_back_addr":"172.21.15.64:6825/1913824324","heartbeat_front_addr":"172.21.15.64:6823/1913824324","state":["exists","up"]},{"osd":7,"uuid":"c9707260-b581-4a21-b380-f37e16959b02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6817","nonce":2567568166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6819","nonce":2567568166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6823","nonce":2567568166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6821","nonce":2567568166}]},"public_addr":"172.21.15.119:6817/2567568166","cluster_addr":"172.21.15.119:6819/2567568166","heartbeat_back_addr":"172.21.15.119:6823/2567568166","heartbeat_front_addr":"172.21.15.119:6821/2567568166","state":["exists","up"]},{"osd":8,"uuid":"2dbf8ef2-fc9c-4c6b-bfea-003f41ef191e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6817","nonce":199192217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6819","nonce":199192217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6823","nonce":199192217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6821","nonce":199192217}]},"public_addr":"172.21.15.131:6817/199192217","cluster_addr":"172.21.15.131:6819/199192217","heartbeat_back_addr":"172.21.15.131:6823/199192217","heartbeat_front_addr":"172.21.15.131:6821/199192217","state":["exists","up"]},{"osd":9,"uuid":"96f769e5-4783-4558-adfa-4bfaf9da9e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6827","nonce":3309038990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6829","nonce":3309038990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6833","nonce":3309038990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6831","nonce":3309038990}]},"public_addr":"172.21.15.64:6827/3309038990","cluster_addr":"172.21.15.64:6829/3309038990","heartbeat_back_addr":"172.21.15.64:6833/3309038990","heartbeat_front_addr":"172.21.15.64:6831/3309038990","state":["exists","up"]},{"osd":10,"uuid":"13c1ef5c-2299-440c-af5a-a650b47f4b75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6825","nonce":1460844681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6827","nonce":1460844681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6831","nonce":1460844681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6829","nonce":1460844681}]},"public_addr":"172.21.15.119:6825/1460844681","cluster_addr":"172.21.15.119:6827/1460844681","heartbeat_back_addr":"172.21.15.119:6831/1460844681","heartbeat_front_addr":"172.21.15.119:6829/1460844681","state":["exists","up"]},{"osd":11,"uuid":"c26a5ef9-8585-404e-a54a-852424c4d6fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6825","nonce":2294059084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6827","nonce":2294059084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6831","nonce":2294059084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6829","nonce":2294059084}]},"public_addr":"172.21.15.131:6825/2294059084","cluster_addr":"172.21.15.131:6827/2294059084","heartbeat_back_addr":"172.21.15.131:6831/2294059084","heartbeat_front_addr":"172.21.15.131:6829/2294059084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:56:33.141102+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:56:58.883062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:57:22.641789+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:57:48.705790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:58:12.871514+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:58:37.893118+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:03.620828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:29.446179+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:55.002978+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:00:19.987463+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:00:45.364158+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:01:10.228248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.3","mappings":[{"from":10,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":5,"to":6}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/3554471670":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6801/3786213009":"2024-09-22T03:55:28.387426+0000","172.21.15.64:0/2562856406":"2024-09-22T03:54:27.180100+0000","172.21.15.64:0/2603015699":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6800/1209932961":"2024-09-22T03:54:02.562380+0000","172.21.15.64:6800/3786213009":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6801/1209932961":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/3407344596":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1233731831":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1686887701":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1598568021":"2024-09-22T03:54:27.180100+0000","172.21.15.64:6800/2013929312":"2024-09-22T03:54:27.180100+0000","172.21.15.64:6801/2013929312":"2024-09-22T03:54:27.180100+0000","172.21.15.64:0/3652036144":"2024-09-22T03:55:28.387426+0000","172.21.15.64:0/105454703":"2024-09-22T03:54:27.180100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T04:14:01.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.476+0000 7f1aba994640 1 -- 172.21.15.64:0/2589776052 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f1a9c0b6700 msgr2=0x7f1a9c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:01.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.476+0000 7f1aba994640 1 --2- 172.21.15.64:0/2589776052 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f1a9c0b6700 0x7f1a9c0b8c10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80058c0 tx=0x7f1aa8005850 comp rx=0 tx=0).stop 2024-09-21T04:14:01.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.476+0000 7f1aba994640 1 -- 172.21.15.64:0/2589776052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ab4104790 msgr2=0x7f1ab419a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:01.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.476+0000 7f1aba994640 1 --2- 172.21.15.64:0/2589776052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ab4104790 0x7f1ab419a380 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4005940 tx=0x7f1aa40042f0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.476+0000 7f1aba994640 1 -- 172.21.15.64:0/2589776052 shutdown_connections 2024-09-21T04:14:01.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.476+0000 7f1aba994640 1 --2- 172.21.15.64:0/2589776052 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f1a9c0b6700 0x7f1a9c0b8c10 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.476+0000 7f1aba994640 1 --2- 172.21.15.64:0/2589776052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1ab410dd40 0x7f1ab419fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.476+0000 7f1aba994640 1 --2- 172.21.15.64:0/2589776052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab41051c0 0x7f1ab419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.476+0000 7f1aba994640 1 --2- 172.21.15.64:0/2589776052 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ab4104790 0x7f1ab419a380 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.476+0000 7f1aba994640 1 -- 172.21.15.64:0/2589776052 >> 172.21.15.64:0/2589776052 conn(0x7f1ab40fe380 msgr2=0x7f1ab40fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:01.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.476+0000 7f1aba994640 1 -- 172.21.15.64:0/2589776052 shutdown_connections 2024-09-21T04:14:01.480 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.476+0000 7f1aba994640 1 -- 172.21.15.64:0/2589776052 wait complete. 2024-09-21T04:14:01.491 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T03:57:24.854438+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T04:01:58.728355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T04:02:01.300385+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 84, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T04:14:01.491 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T04:14:01.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.605+0000 7fdcba198640 1 -- 172.21.15.64:0/183486527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcb4109d60 msgr2=0x7fdcb4110590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:01.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.605+0000 7fdcba198640 1 --2- 172.21.15.64:0/183486527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcb4109d60 0x7fdcb4110590 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fdcac009900 tx=0x7fdcac02a410 comp rx=0 tx=0).stop 2024-09-21T04:14:01.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.605+0000 7fdcba198640 1 -- 172.21.15.64:0/183486527 shutdown_connections 2024-09-21T04:14:01.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.605+0000 7fdcba198640 1 --2- 172.21.15.64:0/183486527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcb4109d60 0x7fdcb4110590 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.605+0000 7fdcba198640 1 --2- 172.21.15.64:0/183486527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcb4109410 0x7fdcb4109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.605+0000 7fdcba198640 1 --2- 172.21.15.64:0/183486527 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcb4103ae0 0x7fdcb4103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.605+0000 7fdcba198640 1 -- 172.21.15.64:0/183486527 >> 172.21.15.64:0/183486527 conn(0x7fdcb407a8c0 msgr2=0x7fdcb407acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:01.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.605+0000 7fdcba198640 1 -- 172.21.15.64:0/183486527 shutdown_connections 2024-09-21T04:14:01.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.605+0000 7fdcba198640 1 -- 172.21.15.64:0/183486527 wait complete. 2024-09-21T04:14:01.608 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.605+0000 7fdcba198640 1 Processor -- start 2024-09-21T04:14:01.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.606+0000 7fdcba198640 1 -- start start 2024-09-21T04:14:01.610 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.606+0000 7fdcba198640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcb4103ae0 0x7fdcb4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:01.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.606+0000 7fdcba198640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcb4109410 0x7fdcb4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:01.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.606+0000 7fdcba198640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcb4109d60 0x7fdcb4076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:01.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.606+0000 7fdcba198640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb4113a20 con 0x7fdcb4109d60 2024-09-21T04:14:01.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.606+0000 7fdcba198640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb41138a0 con 0x7fdcb4103ae0 2024-09-21T04:14:01.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.606+0000 7fdcba198640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb4113ba0 con 0x7fdcb4109410 2024-09-21T04:14:01.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.606+0000 7fdcb3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcb4109d60 0x7fdcb4076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:01.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.606+0000 7fdcb3fff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcb4109d60 0x7fdcb4076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54922/0 (socket says 172.21.15.64:54922) 2024-09-21T04:14:01.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.606+0000 7fdcb3fff640 1 -- 172.21.15.64:0/135388984 learned_addr learned my addr 172.21.15.64:0/135388984 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:14:01.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.606+0000 7fdcb37fe640 1 --2- 172.21.15.64:0/135388984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcb4103ae0 0x7fdcb4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:01.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.606+0000 7fdcb2ffd640 1 --2- 172.21.15.64:0/135388984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcb4109410 0x7fdcb4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:01.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.606+0000 7fdcb3fff640 1 -- 172.21.15.64:0/135388984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcb4109410 msgr2=0x7fdcb4075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:01.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.606+0000 7fdcb3fff640 1 --2- 172.21.15.64:0/135388984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcb4109410 0x7fdcb4075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.606+0000 7fdcb3fff640 1 -- 172.21.15.64:0/135388984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcb4103ae0 msgr2=0x7fdcb4075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:01.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.606+0000 7fdcb3fff640 1 --2- 172.21.15.64:0/135388984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcb4103ae0 0x7fdcb4075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.606+0000 7fdcb3fff640 1 -- 172.21.15.64:0/135388984 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb41a8670 con 0x7fdcb4109d60 2024-09-21T04:14:01.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.606+0000 7fdcb2ffd640 1 --2- 172.21.15.64:0/135388984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcb4109410 0x7fdcb4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:14:01.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.606+0000 7fdcb3fff640 1 --2- 172.21.15.64:0/135388984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcb4109d60 0x7fdcb4076190 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fdcac004770 tx=0x7fdcac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:01.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.607+0000 7fdcb0ff9640 1 -- 172.21.15.64:0/135388984 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac03f030 con 0x7fdcb4109d60 2024-09-21T04:14:01.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.607+0000 7fdcba198640 1 -- 172.21.15.64:0/135388984 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcb41a8900 con 0x7fdcb4109d60 2024-09-21T04:14:01.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.607+0000 7fdcb0ff9640 1 -- 172.21.15.64:0/135388984 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcac003aa0 con 0x7fdcb4109d60 2024-09-21T04:14:01.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.607+0000 7fdcb0ff9640 1 -- 172.21.15.64:0/135388984 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac035070 con 0x7fdcb4109d60 2024-09-21T04:14:01.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.607+0000 7fdcba198640 1 -- 172.21.15.64:0/135388984 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcb41a8e40 con 0x7fdcb4109d60 2024-09-21T04:14:01.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.608+0000 7fdcba198640 1 -- 172.21.15.64:0/135388984 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcb4103ef0 con 0x7fdcb4109d60 2024-09-21T04:14:01.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.610+0000 7fdcb0ff9640 1 -- 172.21.15.64:0/135388984 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fdcac042030 con 0x7fdcb4109d60 2024-09-21T04:14:01.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.610+0000 7fdcb0ff9640 1 --2- 172.21.15.64:0/135388984 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdc9c0b66b0 0x7fdc9c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:01.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.611+0000 7fdcb37fe640 1 --2- 172.21.15.64:0/135388984 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdc9c0b66b0 0x7fdc9c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:01.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.611+0000 7fdcb0ff9640 1 -- 172.21.15.64:0/135388984 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8615+0+0 (secure 0 0 0) 0x7fdcac108190 con 0x7fdcb4109d60 2024-09-21T04:14:01.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.611+0000 7fdcb37fe640 1 --2- 172.21.15.64:0/135388984 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdc9c0b66b0 0x7fdc9c0b8bc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4077160 tx=0x7fdca4009680 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:01.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.613+0000 7fdcb0ff9640 1 -- 172.21.15.64:0/135388984 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdcac0ce680 con 0x7fdcb4109d60 2024-09-21T04:14:01.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.800+0000 7fdcba198640 1 -- 172.21.15.64:0/135388984 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdcb402e640 con 0x7fdcb4109d60 2024-09-21T04:14:01.803 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.801+0000 7fdcb0ff9640 1 -- 172.21.15.64:0/135388984 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7fdcac0d2a60 con 0x7fdcb4109d60 2024-09-21T04:14:01.803 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 1 2024-09-21T04:14:01.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.803+0000 7fdcba198640 1 -- 172.21.15.64:0/135388984 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdc9c0b66b0 msgr2=0x7fdc9c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:01.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.803+0000 7fdcba198640 1 --2- 172.21.15.64:0/135388984 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdc9c0b66b0 0x7fdc9c0b8bc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4077160 tx=0x7fdca4009680 comp rx=0 tx=0).stop 2024-09-21T04:14:01.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.803+0000 7fdcba198640 1 -- 172.21.15.64:0/135388984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcb4109d60 msgr2=0x7fdcb4076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:01.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.803+0000 7fdcba198640 1 --2- 172.21.15.64:0/135388984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcb4109d60 0x7fdcb4076190 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fdcac004770 tx=0x7fdcac004270 comp rx=0 tx=0).stop 2024-09-21T04:14:01.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.803+0000 7fdcba198640 1 -- 172.21.15.64:0/135388984 shutdown_connections 2024-09-21T04:14:01.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.803+0000 7fdcba198640 1 --2- 172.21.15.64:0/135388984 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fdc9c0b66b0 0x7fdc9c0b8bc0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.803+0000 7fdcba198640 1 --2- 172.21.15.64:0/135388984 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdcb4109d60 0x7fdcb4076190 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.803+0000 7fdcba198640 1 --2- 172.21.15.64:0/135388984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcb4109410 0x7fdcb4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.806 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.803+0000 7fdcba198640 1 --2- 172.21.15.64:0/135388984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcb4103ae0 0x7fdcb4075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.803+0000 7fdcba198640 1 -- 172.21.15.64:0/135388984 >> 172.21.15.64:0/135388984 conn(0x7fdcb407a8c0 msgr2=0x7fdcb4107420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:01.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.803+0000 7fdcba198640 1 -- 172.21.15.64:0/135388984 shutdown_connections 2024-09-21T04:14:01.807 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.803+0000 7fdcba198640 1 -- 172.21.15.64:0/135388984 wait complete. 2024-09-21T04:14:01.817 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T04:14:01.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.931+0000 7f83e25f1640 1 -- 172.21.15.64:0/2341975105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83dc1093e0 msgr2=0x7f83dc1097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:01.934 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.931+0000 7f83e25f1640 1 --2- 172.21.15.64:0/2341975105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83dc1093e0 0x7f83dc1097f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f83d0009900 tx=0x7f83d002a410 comp rx=0 tx=0).stop 2024-09-21T04:14:01.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.931+0000 7f83e25f1640 1 -- 172.21.15.64:0/2341975105 shutdown_connections 2024-09-21T04:14:01.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.931+0000 7f83e25f1640 1 --2- 172.21.15.64:0/2341975105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83dc109d30 0x7f83dc110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.931+0000 7f83e25f1640 1 --2- 172.21.15.64:0/2341975105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83dc1093e0 0x7f83dc1097f0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.931+0000 7f83e25f1640 1 --2- 172.21.15.64:0/2341975105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83dc103ab0 0x7f83dc103ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.931+0000 7f83e25f1640 1 -- 172.21.15.64:0/2341975105 >> 172.21.15.64:0/2341975105 conn(0x7f83dc0ff4d0 msgr2=0x7f83dc101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:01.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.931+0000 7f83e25f1640 1 -- 172.21.15.64:0/2341975105 shutdown_connections 2024-09-21T04:14:01.935 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.931+0000 7f83e25f1640 1 -- 172.21.15.64:0/2341975105 wait complete. 2024-09-21T04:14:01.937 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.932+0000 7f83e25f1640 1 Processor -- start 2024-09-21T04:14:01.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.932+0000 7f83e25f1640 1 -- start start 2024-09-21T04:14:01.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.933+0000 7f83e25f1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83dc103ab0 0x7f83dc19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:01.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.933+0000 7f83e25f1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83dc1093e0 0x7f83dc19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:01.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.933+0000 7f83e25f1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83dc109d30 0x7f83dc19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:01.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.933+0000 7f83e25f1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc1139f0 con 0x7f83dc103ab0 2024-09-21T04:14:01.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.933+0000 7f83e25f1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc113870 con 0x7f83dc109d30 2024-09-21T04:14:01.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.933+0000 7f83e25f1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc113b70 con 0x7f83dc1093e0 2024-09-21T04:14:01.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.933+0000 7f83dbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83dc103ab0 0x7f83dc19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:01.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.933+0000 7f83dbfff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83dc103ab0 0x7f83dc19a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54940/0 (socket says 172.21.15.64:54940) 2024-09-21T04:14:01.938 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.933+0000 7f83dbfff640 1 -- 172.21.15.64:0/1134214294 learned_addr learned my addr 172.21.15.64:0/1134214294 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:14:01.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.933+0000 7f83db7fe640 1 --2- 172.21.15.64:0/1134214294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83dc1093e0 0x7f83dc19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:01.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.933+0000 7f83e0b8e640 1 --2- 172.21.15.64:0/1134214294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83dc109d30 0x7f83dc19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:01.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.933+0000 7f83dbfff640 1 -- 172.21.15.64:0/1134214294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83dc1093e0 msgr2=0x7f83dc19aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:01.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.933+0000 7f83dbfff640 1 --2- 172.21.15.64:0/1134214294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83dc1093e0 0x7f83dc19aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.933+0000 7f83dbfff640 1 -- 172.21.15.64:0/1134214294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83dc109d30 msgr2=0x7f83dc19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:01.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.933+0000 7f83dbfff640 1 --2- 172.21.15.64:0/1134214294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83dc109d30 0x7f83dc19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:01.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.933+0000 7f83dbfff640 1 -- 172.21.15.64:0/1134214294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83dc1a0350 con 0x7f83dc103ab0 2024-09-21T04:14:01.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.933+0000 7f83dbfff640 1 --2- 172.21.15.64:0/1134214294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83dc103ab0 0x7f83dc19a4b0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00d770 tx=0x7f83cc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:01.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.933+0000 7f83d97fa640 1 -- 172.21.15.64:0/1134214294 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc017070 con 0x7f83dc103ab0 2024-09-21T04:14:01.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.934+0000 7f83d97fa640 1 -- 172.21.15.64:0/1134214294 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83cc0054d0 con 0x7f83dc103ab0 2024-09-21T04:14:01.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.934+0000 7f83e25f1640 1 -- 172.21.15.64:0/1134214294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83dc1a0640 con 0x7f83dc103ab0 2024-09-21T04:14:01.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.934+0000 7f83d97fa640 1 -- 172.21.15.64:0/1134214294 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc004c30 con 0x7f83dc103ab0 2024-09-21T04:14:01.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.934+0000 7f83e25f1640 1 -- 172.21.15.64:0/1134214294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83dc1a0b10 con 0x7f83dc103ab0 2024-09-21T04:14:01.939 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.934+0000 7f83e25f1640 1 -- 172.21.15.64:0/1134214294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83dc103ec0 con 0x7f83dc103ab0 2024-09-21T04:14:01.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.936+0000 7f83d97fa640 1 -- 172.21.15.64:0/1134214294 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f83cc004dd0 con 0x7f83dc103ab0 2024-09-21T04:14:01.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.937+0000 7f83d97fa640 1 --2- 172.21.15.64:0/1134214294 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f83c40b6680 0x7f83c40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:01.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.937+0000 7f83db7fe640 1 --2- 172.21.15.64:0/1134214294 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f83c40b6680 0x7f83c40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:01.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.938+0000 7f83d97fa640 1 -- 172.21.15.64:0/1134214294 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8615+0+0 (secure 0 0 0) 0x7f83cc0e7730 con 0x7f83dc103ab0 2024-09-21T04:14:01.940 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.938+0000 7f83db7fe640 1 --2- 172.21.15.64:0/1134214294 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f83c40b6680 0x7f83c40b8b90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f83d0009230 tx=0x7f83d0031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:01.943 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:01.941+0000 7f83d97fa640 1 -- 172.21.15.64:0/1134214294 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f83dc1a0b10 con 0x7f83dc103ab0 2024-09-21T04:14:02.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.129+0000 7f83e25f1640 1 -- 172.21.15.64:0/1134214294 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f83dc02e640 con 0x7f83dc103ab0 2024-09-21T04:14:02.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.129+0000 7f83d97fa640 1 -- 172.21.15.64:0/1134214294 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) ==== 104+0+11 (secure 0 0 0) 0x7f83cc0adc20 con 0x7f83dc103ab0 2024-09-21T04:14:02.132 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 32 2024-09-21T04:14:02.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.131+0000 7f83e25f1640 1 -- 172.21.15.64:0/1134214294 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f83c40b6680 msgr2=0x7f83c40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:02.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.131+0000 7f83e25f1640 1 --2- 172.21.15.64:0/1134214294 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f83c40b6680 0x7f83c40b8b90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f83d0009230 tx=0x7f83d0031400 comp rx=0 tx=0).stop 2024-09-21T04:14:02.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.131+0000 7f83e25f1640 1 -- 172.21.15.64:0/1134214294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83dc103ab0 msgr2=0x7f83dc19a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:02.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.131+0000 7f83e25f1640 1 --2- 172.21.15.64:0/1134214294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83dc103ab0 0x7f83dc19a4b0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00d770 tx=0x7f83cc00dc30 comp rx=0 tx=0).stop 2024-09-21T04:14:02.134 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.131+0000 7f83e25f1640 1 -- 172.21.15.64:0/1134214294 shutdown_connections 2024-09-21T04:14:02.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.131+0000 7f83e25f1640 1 --2- 172.21.15.64:0/1134214294 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f83c40b6680 0x7f83c40b8b90 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:02.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.131+0000 7f83e25f1640 1 --2- 172.21.15.64:0/1134214294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83dc109d30 0x7f83dc19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:02.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.131+0000 7f83e25f1640 1 --2- 172.21.15.64:0/1134214294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f83dc1093e0 0x7f83dc19aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:02.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.131+0000 7f83e25f1640 1 --2- 172.21.15.64:0/1134214294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f83dc103ab0 0x7f83dc19a4b0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:02.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.132+0000 7f83e25f1640 1 -- 172.21.15.64:0/1134214294 >> 172.21.15.64:0/1134214294 conn(0x7f83dc0ff4d0 msgr2=0x7f83dc1018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:02.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.132+0000 7f83e25f1640 1 -- 172.21.15.64:0/1134214294 shutdown_connections 2024-09-21T04:14:02.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.132+0000 7f83e25f1640 1 -- 172.21.15.64:0/1134214294 wait complete. 2024-09-21T04:14:02.146 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T04:14:02.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.259+0000 7fe45647e640 1 -- 172.21.15.64:0/1515401976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4501048e0 msgr2=0x7fe450104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:02.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.259+0000 7fe45647e640 1 --2- 172.21.15.64:0/1515401976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4501048e0 0x7fe450104cf0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe440009900 tx=0x7fe44002a410 comp rx=0 tx=0).stop 2024-09-21T04:14:02.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.259+0000 7fe45647e640 1 -- 172.21.15.64:0/1515401976 shutdown_connections 2024-09-21T04:14:02.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.259+0000 7fe45647e640 1 --2- 172.21.15.64:0/1515401976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe45010dd40 0x7fe450110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:02.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.259+0000 7fe45647e640 1 --2- 172.21.15.64:0/1515401976 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe450105230 0x7fe45010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:02.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.259+0000 7fe45647e640 1 --2- 172.21.15.64:0/1515401976 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe4501048e0 0x7fe450104cf0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:02.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.259+0000 7fe45647e640 1 -- 172.21.15.64:0/1515401976 >> 172.21.15.64:0/1515401976 conn(0x7fe4500fe3b0 msgr2=0x7fe450100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:02.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.259+0000 7fe45647e640 1 -- 172.21.15.64:0/1515401976 shutdown_connections 2024-09-21T04:14:02.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.259+0000 7fe45647e640 1 -- 172.21.15.64:0/1515401976 wait complete. 2024-09-21T04:14:02.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.260+0000 7fe45647e640 1 Processor -- start 2024-09-21T04:14:02.263 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.260+0000 7fe45647e640 1 -- start start 2024-09-21T04:14:02.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.260+0000 7fe45647e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4501048e0 0x7fe45019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:02.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.260+0000 7fe45647e640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe450105230 0x7fe45019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:02.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.260+0000 7fe45647e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe45010dd40 0x7fe45019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:02.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.260+0000 7fe45647e640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe4501139a0 con 0x7fe450105230 2024-09-21T04:14:02.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.260+0000 7fe45647e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe450113820 con 0x7fe45010dd40 2024-09-21T04:14:02.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.260+0000 7fe45647e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe450113b20 con 0x7fe4501048e0 2024-09-21T04:14:02.265 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.260+0000 7fe44f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe450105230 0x7fe45019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:02.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.260+0000 7fe44f7fe640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe450105230 0x7fe45019a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54970/0 (socket says 172.21.15.64:54970) 2024-09-21T04:14:02.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.260+0000 7fe44f7fe640 1 -- 172.21.15.64:0/2858201541 learned_addr learned my addr 172.21.15.64:0/2858201541 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:14:02.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.260+0000 7fe454a1b640 1 --2- 172.21.15.64:0/2858201541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe45010dd40 0x7fe45019fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:02.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.260+0000 7fe44ffff640 1 --2- 172.21.15.64:0/2858201541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4501048e0 0x7fe45019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:02.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.261+0000 7fe44f7fe640 1 -- 172.21.15.64:0/2858201541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4501048e0 msgr2=0x7fe45019a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:02.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.261+0000 7fe44f7fe640 1 --2- 172.21.15.64:0/2858201541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4501048e0 0x7fe45019a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:02.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.261+0000 7fe44f7fe640 1 -- 172.21.15.64:0/2858201541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe45010dd40 msgr2=0x7fe45019fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:02.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.261+0000 7fe44f7fe640 1 --2- 172.21.15.64:0/2858201541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe45010dd40 0x7fe45019fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:02.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.261+0000 7fe44f7fe640 1 -- 172.21.15.64:0/2858201541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4501a02f0 con 0x7fe450105230 2024-09-21T04:14:02.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.261+0000 7fe44f7fe640 1 --2- 172.21.15.64:0/2858201541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe450105230 0x7fe45019a960 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fe44400d740 tx=0x7fe44400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:02.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.261+0000 7fe44d7fa640 1 -- 172.21.15.64:0/2858201541 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe444017070 con 0x7fe450105230 2024-09-21T04:14:02.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.261+0000 7fe44d7fa640 1 -- 172.21.15.64:0/2858201541 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe44400de50 con 0x7fe450105230 2024-09-21T04:14:02.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.261+0000 7fe45647e640 1 -- 172.21.15.64:0/2858201541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4501a05e0 con 0x7fe450105230 2024-09-21T04:14:02.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.261+0000 7fe44d7fa640 1 -- 172.21.15.64:0/2858201541 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44400f620 con 0x7fe450105230 2024-09-21T04:14:02.266 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.261+0000 7fe45647e640 1 -- 172.21.15.64:0/2858201541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4501a6d70 con 0x7fe450105230 2024-09-21T04:14:02.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.262+0000 7fe45647e640 1 -- 172.21.15.64:0/2858201541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe450104cf0 con 0x7fe450105230 2024-09-21T04:14:02.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.264+0000 7fe44d7fa640 1 -- 172.21.15.64:0/2858201541 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fe44401a070 con 0x7fe450105230 2024-09-21T04:14:02.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.265+0000 7fe44d7fa640 1 --2- 172.21.15.64:0/2858201541 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe4380b6680 0x7fe4380b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:02.267 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.265+0000 7fe44ffff640 1 --2- 172.21.15.64:0/2858201541 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe4380b6680 0x7fe4380b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:02.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.265+0000 7fe44d7fa640 1 -- 172.21.15.64:0/2858201541 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8615+0+0 (secure 0 0 0) 0x7fe4440e70e0 con 0x7fe450105230 2024-09-21T04:14:02.268 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.265+0000 7fe44ffff640 1 --2- 172.21.15.64:0/2858201541 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe4380b6680 0x7fe4380b8b90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe440031440 tx=0x7fe440009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:02.270 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.268+0000 7fe44d7fa640 1 -- 172.21.15.64:0/2858201541 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe4440b19b0 con 0x7fe450105230 2024-09-21T04:14:02.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.458+0000 7fe45647e640 1 -- 172.21.15.64:0/2858201541 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe4501a7050 con 0x7fe450105230 2024-09-21T04:14:02.461 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.459+0000 7fe44d7fa640 1 -- 172.21.15.64:0/2858201541 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) ==== 100+0+11 (secure 0 0 0) 0x7fe4440b68f0 con 0x7fe450105230 2024-09-21T04:14:02.461 INFO:teuthology.orchestra.run.smithi064.stdout:pg_num: 64 2024-09-21T04:14:02.463 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.461+0000 7fe45647e640 1 -- 172.21.15.64:0/2858201541 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe4380b6680 msgr2=0x7fe4380b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:02.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.461+0000 7fe45647e640 1 --2- 172.21.15.64:0/2858201541 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe4380b6680 0x7fe4380b8b90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe440031440 tx=0x7fe440009560 comp rx=0 tx=0).stop 2024-09-21T04:14:02.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.461+0000 7fe45647e640 1 -- 172.21.15.64:0/2858201541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe450105230 msgr2=0x7fe45019a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:02.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.461+0000 7fe45647e640 1 --2- 172.21.15.64:0/2858201541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe450105230 0x7fe45019a960 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fe44400d740 tx=0x7fe44400dc00 comp rx=0 tx=0).stop 2024-09-21T04:14:02.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.461+0000 7fe45647e640 1 -- 172.21.15.64:0/2858201541 shutdown_connections 2024-09-21T04:14:02.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.461+0000 7fe45647e640 1 --2- 172.21.15.64:0/2858201541 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fe4380b6680 0x7fe4380b8b90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:02.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.461+0000 7fe45647e640 1 --2- 172.21.15.64:0/2858201541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe45010dd40 0x7fe45019fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:02.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.461+0000 7fe45647e640 1 --2- 172.21.15.64:0/2858201541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe450105230 0x7fe45019a960 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:02.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.461+0000 7fe45647e640 1 --2- 172.21.15.64:0/2858201541 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4501048e0 0x7fe45019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:02.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.461+0000 7fe45647e640 1 -- 172.21.15.64:0/2858201541 >> 172.21.15.64:0/2858201541 conn(0x7fe4500fe3b0 msgr2=0x7fe450100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:02.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.461+0000 7fe45647e640 1 -- 172.21.15.64:0/2858201541 shutdown_connections 2024-09-21T04:14:02.464 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.461+0000 7fe45647e640 1 -- 172.21.15.64:0/2858201541 wait complete. 2024-09-21T04:14:02.474 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:02 smithi064 ceph-mon[31180]: pgmap v629: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:14:02.474 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:02 smithi064 ceph-mon[31180]: from='client.17550 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:14:02.474 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:02 smithi064 ceph-mon[31180]: from='client.17556 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:14:02.474 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:02 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2589776052' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:14:02.474 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:02 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/135388984' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T04:14:02.474 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:02 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1134214294' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T04:14:02.475 DEBUG:teuthology.orchestra.run.smithi064:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-21T04:14:02.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.586+0000 7f7247b69640 1 -- 172.21.15.64:0/550860288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72401029c0 msgr2=0x7f7240102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:02.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.586+0000 7f7247b69640 1 --2- 172.21.15.64:0/550860288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72401029c0 0x7f7240102dd0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f723c009900 tx=0x7f723c02a410 comp rx=0 tx=0).stop 2024-09-21T04:14:02.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.586+0000 7f7247b69640 1 -- 172.21.15.64:0/550860288 shutdown_connections 2024-09-21T04:14:02.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.586+0000 7f7247b69640 1 --2- 172.21.15.64:0/550860288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7240108c40 0x7f724010f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:02.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.586+0000 7f7247b69640 1 --2- 172.21.15.64:0/550860288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72401082f0 0x7f7240108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:02.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.586+0000 7f7247b69640 1 --2- 172.21.15.64:0/550860288 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72401029c0 0x7f7240102dd0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:02.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.586+0000 7f7247b69640 1 -- 172.21.15.64:0/550860288 >> 172.21.15.64:0/550860288 conn(0x7f724007a670 msgr2=0x7f724007aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:02.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.586+0000 7f7247b69640 1 -- 172.21.15.64:0/550860288 shutdown_connections 2024-09-21T04:14:02.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.586+0000 7f7247b69640 1 -- 172.21.15.64:0/550860288 wait complete. 2024-09-21T04:14:02.591 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.587+0000 7f7247b69640 1 Processor -- start 2024-09-21T04:14:02.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.587+0000 7f7247b69640 1 -- start start 2024-09-21T04:14:02.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.587+0000 7f7247b69640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72401029c0 0x7f724019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:02.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.587+0000 7f7247b69640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72401082f0 0x7f724019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:02.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.587+0000 7f7247b69640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7240108c40 0x7f724019fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:02.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.587+0000 7f7247b69640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7240113ac0 con 0x7f72401082f0 2024-09-21T04:14:02.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.587+0000 7f7247b69640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7240113940 con 0x7f7240108c40 2024-09-21T04:14:02.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.587+0000 7f7247b69640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7240113c40 con 0x7f72401029c0 2024-09-21T04:14:02.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.587+0000 7f7245104640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72401082f0 0x7f724019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:02.592 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.587+0000 7f7245104640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72401082f0 0x7f724019ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:54980/0 (socket says 172.21.15.64:54980) 2024-09-21T04:14:02.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.587+0000 7f7245104640 1 -- 172.21.15.64:0/3480345547 learned_addr learned my addr 172.21.15.64:0/3480345547 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:14:02.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.587+0000 7f7245905640 1 --2- 172.21.15.64:0/3480345547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72401029c0 0x7f724019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:02.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.587+0000 7f7245104640 1 -- 172.21.15.64:0/3480345547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72401029c0 msgr2=0x7f724019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:02.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.587+0000 7f7245104640 1 --2- 172.21.15.64:0/3480345547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72401029c0 0x7f724019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:02.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.587+0000 7f7246106640 1 --2- 172.21.15.64:0/3480345547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7240108c40 0x7f724019fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:02.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.587+0000 7f7245104640 1 -- 172.21.15.64:0/3480345547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7240108c40 msgr2=0x7f724019fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:02.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.587+0000 7f7245104640 1 --2- 172.21.15.64:0/3480345547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7240108c40 0x7f724019fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:02.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.587+0000 7f7245104640 1 -- 172.21.15.64:0/3480345547 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72401a0490 con 0x7f72401082f0 2024-09-21T04:14:02.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.587+0000 7f7245905640 1 --2- 172.21.15.64:0/3480345547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72401029c0 0x7f724019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:14:02.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.588+0000 7f7245104640 1 --2- 172.21.15.64:0/3480345547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72401082f0 0x7f724019ab00 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f7234006fd0 tx=0x7f723400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:02.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.588+0000 7f7232ffd640 1 -- 172.21.15.64:0/3480345547 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7234011830 con 0x7f72401082f0 2024-09-21T04:14:02.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.588+0000 7f7232ffd640 1 -- 172.21.15.64:0/3480345547 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7234016070 con 0x7f72401082f0 2024-09-21T04:14:02.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.588+0000 7f7247b69640 1 -- 172.21.15.64:0/3480345547 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72401a0780 con 0x7f72401082f0 2024-09-21T04:14:02.593 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.588+0000 7f7232ffd640 1 -- 172.21.15.64:0/3480345547 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7234010e00 con 0x7f72401082f0 2024-09-21T04:14:02.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.588+0000 7f7247b69640 1 -- 172.21.15.64:0/3480345547 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72401a6f10 con 0x7f72401082f0 2024-09-21T04:14:02.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.589+0000 7f7247b69640 1 -- 172.21.15.64:0/3480345547 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7240102e50 con 0x7f72401082f0 2024-09-21T04:14:02.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.590+0000 7f7232ffd640 1 -- 172.21.15.64:0/3480345547 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f723401b460 con 0x7f72401082f0 2024-09-21T04:14:02.594 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.591+0000 7f7232ffd640 1 --2- 172.21.15.64:0/3480345547 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f722c0b6680 0x7f722c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:02.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.591+0000 7f7245905640 1 --2- 172.21.15.64:0/3480345547 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f722c0b6680 0x7f722c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:02.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.591+0000 7f7232ffd640 1 -- 172.21.15.64:0/3480345547 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8615+0+0 (secure 0 0 0) 0x7f72340e73f0 con 0x7f72401082f0 2024-09-21T04:14:02.595 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.592+0000 7f7245905640 1 --2- 172.21.15.64:0/3480345547 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f722c0b6680 0x7f722c0b8b90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f723c031440 tx=0x7f723c005d10 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:02.597 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.594+0000 7f7232ffd640 1 -- 172.21.15.64:0/3480345547 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f72340ad8e0 con 0x7f72401082f0 2024-09-21T04:14:02.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:02 smithi119 ceph-mon[34811]: pgmap v629: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:14:02.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:02 smithi119 ceph-mon[34811]: from='client.17550 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:14:02.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:02 smithi119 ceph-mon[34811]: from='client.17556 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:14:02.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:02 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2589776052' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:14:02.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:02 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/135388984' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T04:14:02.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:02 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1134214294' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T04:14:02.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:02 smithi131 ceph-mon[35193]: pgmap v629: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-21T04:14:02.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:02 smithi131 ceph-mon[35193]: from='client.17550 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:14:02.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:02 smithi131 ceph-mon[35193]: from='client.17556 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T04:14:02.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:02 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2589776052' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:14:02.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:02 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/135388984' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T04:14:02.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:02 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1134214294' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T04:14:02.839 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.837+0000 7f7247b69640 1 -- 172.21.15.64:0/3480345547 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f724002e640 con 0x7f72401082f0 2024-09-21T04:14:02.847 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.845+0000 7f7232ffd640 1 -- 172.21.15.64:0/3480345547 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+834049 (secure 0 0 0) 0x7f72340b1cc0 con 0x7f72401082f0 2024-09-21T04:14:02.857 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:14:02.857 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:14:02.857 INFO:teuthology.orchestra.run.smithi064.stdout: "epoch": 26, 2024-09-21T04:14:02.857 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.857 INFO:teuthology.orchestra.run.smithi064.stdout: "active_gid": 14226, 2024-09-21T04:14:02.857 INFO:teuthology.orchestra.run.smithi064.stdout: "active_name": "x", 2024-09-21T04:14:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: "active_addrs": { 2024-09-21T04:14:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-09-21T04:14:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-09-21T04:14:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:6800", 2024-09-21T04:14:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 834288654 2024-09-21T04:14:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v1", 2024-09-21T04:14:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:6801", 2024-09-21T04:14:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 834288654 2024-09-21T04:14:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-21T04:14:02.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: "active_addr": "172.21.15.64:6801/834288654", 2024-09-21T04:14:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: "active_change": "2024-09-21T03:55:28.387558+0000", 2024-09-21T04:14:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: "active_mgr_features": 4540719139924082687, 2024-09-21T04:14:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: "available": true, 2024-09-21T04:14:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: "standbys": [ 2024-09-21T04:14:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: "gid": 24151, 2024-09-21T04:14:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "z", 2024-09-21T04:14:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr_features": 4540719139924082687, 2024-09-21T04:14:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: "available_modules": [ 2024-09-21T04:14:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "alerts", 2024-09-21T04:14:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:02.859 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-21T04:14:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-21T04:14:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-21T04:14:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T04:14:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:02.861 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.862 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.863 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.864 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:02.865 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:02.865 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.865 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.865 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.865 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:02.865 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.865 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.865 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.865 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.865 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.865 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.865 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_destination": { 2024-09-21T04:14:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_destination", 2024-09-21T04:14:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T04:14:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.866 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.867 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_from_name": { 2024-09-21T04:14:02.870 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_from_name", 2024-09-21T04:14:02.870 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.870 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.870 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.870 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "Ceph", 2024-09-21T04:14:02.870 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.870 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.870 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.870 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email From: name", 2024-09-21T04:14:02.870 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_host": { 2024-09-21T04:14:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_host", 2024-09-21T04:14:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP server", 2024-09-21T04:14:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.871 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_password": { 2024-09-21T04:14:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_password", 2024-09-21T04:14:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Password to authenticate with", 2024-09-21T04:14:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.872 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_port": { 2024-09-21T04:14:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_port", 2024-09-21T04:14:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "465", 2024-09-21T04:14:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP port", 2024-09-21T04:14:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_sender": { 2024-09-21T04:14:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_sender", 2024-09-21T04:14:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP envelope sender", 2024-09-21T04:14:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.874 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_ssl": { 2024-09-21T04:14:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_ssl", 2024-09-21T04:14:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T04:14:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_user": { 2024-09-21T04:14:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_user", 2024-09-21T04:14:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "User to authenticate as", 2024-09-21T04:14:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.877 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "balancer", 2024-09-21T04:14:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: "active": { 2024-09-21T04:14:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "active", 2024-09-21T04:14:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.878 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T04:14:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_time": { 2024-09-21T04:14:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_time", 2024-09-21T04:14:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0000", 2024-09-21T04:14:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.879 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.880 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.880 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T04:14:02.880 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T04:14:02.880 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.880 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.880 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_weekday": { 2024-09-21T04:14:02.880 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_weekday", 2024-09-21T04:14:02.880 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-21T04:14:02.880 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.880 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-09-21T04:14:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-09-21T04:14:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T04:14:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T04:14:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_max_iterations": { 2024-09-21T04:14:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_max_iterations", 2024-09-21T04:14:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-21T04:14:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.881 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "25", 2024-09-21T04:14:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-21T04:14:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "250", 2024-09-21T04:14:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T04:14:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_metrics": { 2024-09-21T04:14:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_metrics", 2024-09-21T04:14:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.882 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T04:14:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T04:14:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T04:14:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_step": { 2024-09-21T04:14:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_step", 2024-09-21T04:14:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-21T04:14:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.5", 2024-09-21T04:14:02.883 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0.001", 2024-09-21T04:14:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "0.999", 2024-09-21T04:14:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "aggressiveness of optimization", 2024-09-21T04:14:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T04:14:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "end_time": { 2024-09-21T04:14:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_time", 2024-09-21T04:14:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2359", 2024-09-21T04:14:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.884 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T04:14:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T04:14:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: "end_weekday": { 2024-09-21T04:14:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_weekday", 2024-09-21T04:14:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-21T04:14:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-09-21T04:14:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-09-21T04:14:02.885 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T04:14:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T04:14:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.886 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:02.887 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:02.887 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:02.887 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:02.887 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:02.887 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:02.887 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:02.887 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:02.887 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.887 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.887 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.887 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.887 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.887 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:02.888 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:02.889 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.890 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "min_score": { 2024-09-21T04:14:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_score", 2024-09-21T04:14:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-21T04:14:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.891 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T04:14:02.892 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.892 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.892 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.892 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.892 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-09-21T04:14:02.892 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-09-21T04:14:02.892 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.892 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.892 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.892 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "upmap", 2024-09-21T04:14:02.892 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.892 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: "crush-compat", 2024-09-21T04:14:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: "none", 2024-09-21T04:14:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: "read", 2024-09-21T04:14:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap", 2024-09-21T04:14:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap-read" 2024-09-21T04:14:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Balancer mode", 2024-09-21T04:14:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_ids": { 2024-09-21T04:14:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_ids", 2024-09-21T04:14:02.893 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T04:14:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-21T04:14:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-21T04:14:02.894 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-21T04:14:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T04:14:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:02.895 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_deviation": { 2024-09-21T04:14:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_deviation", 2024-09-21T04:14:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.896 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-21T04:14:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-21T04:14:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T04:14:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T04:14:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_optimizations": { 2024-09-21T04:14:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_optimizations", 2024-09-21T04:14:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-21T04:14:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.897 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-21T04:14:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T04:14:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm", 2024-09-21T04:14:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:02.898 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_down_multiplier": { 2024-09-21T04:14:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_down_multiplier", 2024-09-21T04:14:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-21T04:14:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-09-21T04:14:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T04:14:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_refresh_rate": { 2024-09-21T04:14:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_refresh_rate", 2024-09-21T04:14:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "20", 2024-09-21T04:14:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T04:14:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.901 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_starting_port": { 2024-09-21T04:14:02.901 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_starting_port", 2024-09-21T04:14:02.901 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:02.901 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.901 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.901 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4721", 2024-09-21T04:14:02.901 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.901 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.901 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.901 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T04:14:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_ptrace": { 2024-09-21T04:14:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_ptrace", 2024-09-21T04:14:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.902 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T04:14:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T04:14:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_interval": { 2024-09-21T04:14:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_interval", 2024-09-21T04:14:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-21T04:14:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.903 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T04:14:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_memory_target_ratio": { 2024-09-21T04:14:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T04:14:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-21T04:14:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.7", 2024-09-21T04:14:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T04:14:02.904 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm_log_destination": { 2024-09-21T04:14:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm_log_destination", 2024-09-21T04:14:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: "file", 2024-09-21T04:14:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: "file,syslog", 2024-09-21T04:14:02.905 INFO:teuthology.orchestra.run.smithi064.stdout: "syslog" 2024-09-21T04:14:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T04:14:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: "cgroups_split": { 2024-09-21T04:14:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cgroups_split", 2024-09-21T04:14:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.906 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T04:14:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: "config_checks_enabled": { 2024-09-21T04:14:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_checks_enabled", 2024-09-21T04:14:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.907 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T04:14:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: "config_dashboard": { 2024-09-21T04:14:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_dashboard", 2024-09-21T04:14:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.908 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T04:14:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_alertmanager": { 2024-09-21T04:14:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_alertmanager", 2024-09-21T04:14:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T04:14:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.909 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-21T04:14:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_base": { 2024-09-21T04:14:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_base", 2024-09-21T04:14:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T04:14:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.910 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Container image name, without the tag", 2024-09-21T04:14:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_elasticsearch": { 2024-09-21T04:14:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_elasticsearch", 2024-09-21T04:14:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T04:14:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.911 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "elasticsearch container image", 2024-09-21T04:14:02.912 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.912 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.912 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.912 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_grafana": { 2024-09-21T04:14:02.912 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_grafana", 2024-09-21T04:14:02.912 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.912 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.912 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.912 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T04:14:02.912 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.912 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.912 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.912 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-21T04:14:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_haproxy": { 2024-09-21T04:14:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_haproxy", 2024-09-21T04:14:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T04:14:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "HAproxy container image", 2024-09-21T04:14:02.913 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_agent": { 2024-09-21T04:14:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_agent", 2024-09-21T04:14:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T04:14:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger agent container image", 2024-09-21T04:14:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.914 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_collector": { 2024-09-21T04:14:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_collector", 2024-09-21T04:14:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T04:14:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger collector container image", 2024-09-21T04:14:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.915 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_query": { 2024-09-21T04:14:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_query", 2024-09-21T04:14:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T04:14:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger query container image", 2024-09-21T04:14:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.916 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_keepalived": { 2024-09-21T04:14:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_keepalived", 2024-09-21T04:14:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T04:14:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Keepalived container image", 2024-09-21T04:14:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.917 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_loki": { 2024-09-21T04:14:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_loki", 2024-09-21T04:14:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T04:14:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Loki container image", 2024-09-21T04:14:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.918 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_nginx": { 2024-09-21T04:14:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_nginx", 2024-09-21T04:14:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T04:14:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Nginx container image", 2024-09-21T04:14:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_node_exporter": { 2024-09-21T04:14:02.919 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_node_exporter", 2024-09-21T04:14:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T04:14:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-21T04:14:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_nvmeof": { 2024-09-21T04:14:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_nvmeof", 2024-09-21T04:14:02.920 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T04:14:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Nvme-of container image", 2024-09-21T04:14:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_oauth2_proxy": { 2024-09-21T04:14:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T04:14:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.921 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T04:14:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "oauth2-proxy container image", 2024-09-21T04:14:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_prometheus": { 2024-09-21T04:14:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_prometheus", 2024-09-21T04:14:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.922 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T04:14:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-21T04:14:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_promtail": { 2024-09-21T04:14:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_promtail", 2024-09-21T04:14:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.923 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T04:14:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Promtail container image", 2024-09-21T04:14:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_samba": { 2024-09-21T04:14:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_samba", 2024-09-21T04:14:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.924 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T04:14:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Samba/SMB container image", 2024-09-21T04:14:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_samba_metrics": { 2024-09-21T04:14:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_samba_metrics", 2024-09-21T04:14:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T04:14:02.925 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.926 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.926 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.926 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T04:14:02.926 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.926 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.926 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.926 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_snmp_gateway": { 2024-09-21T04:14:02.926 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_snmp_gateway", 2024-09-21T04:14:02.926 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.926 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.926 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.926 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T04:14:02.926 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SNMP Gateway container image", 2024-09-21T04:14:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "container_init": { 2024-09-21T04:14:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_init", 2024-09-21T04:14:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.927 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T04:14:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: "daemon_cache_timeout": { 2024-09-21T04:14:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "daemon_cache_timeout", 2024-09-21T04:14:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-21T04:14:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.928 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T04:14:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: "default_cephadm_command_timeout": { 2024-09-21T04:14:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T04:14:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "900", 2024-09-21T04:14:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.929 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T04:14:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: "default_registry": { 2024-09-21T04:14:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_registry", 2024-09-21T04:14:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io", 2024-09-21T04:14:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.930 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T04:14:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "device_cache_timeout": { 2024-09-21T04:14:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_cache_timeout", 2024-09-21T04:14:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1800", 2024-09-21T04:14:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache device inventory", 2024-09-21T04:14:02.931 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: "device_enhanced_scan": { 2024-09-21T04:14:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_enhanced_scan", 2024-09-21T04:14:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T04:14:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.932 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.933 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.933 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.933 INFO:teuthology.orchestra.run.smithi064.stdout: "facts_cache_timeout": { 2024-09-21T04:14:02.933 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "facts_cache_timeout", 2024-09-21T04:14:02.933 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:02.933 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.933 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.933 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-21T04:14:02.933 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.933 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.933 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.933 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache host facts data", 2024-09-21T04:14:02.933 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.933 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.933 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.934 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.934 INFO:teuthology.orchestra.run.smithi064.stdout: "grafana_dashboards_path": { 2024-09-21T04:14:02.934 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "grafana_dashboards_path", 2024-09-21T04:14:02.934 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.934 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.934 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.934 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T04:14:02.934 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.934 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.934 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.934 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T04:14:02.934 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.934 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "host_check_interval": { 2024-09-21T04:14:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "host_check_interval", 2024-09-21T04:14:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-21T04:14:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to perform a host check", 2024-09-21T04:14:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.935 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: "hw_monitoring": { 2024-09-21T04:14:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hw_monitoring", 2024-09-21T04:14:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T04:14:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.936 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: "inventory_list_all": { 2024-09-21T04:14:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "inventory_list_all", 2024-09-21T04:14:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T04:14:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:02.938 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: "log_refresh_metadata": { 2024-09-21T04:14:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_refresh_metadata", 2024-09-21T04:14:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:02.939 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T04:14:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:02.940 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T04:14:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:02.941 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.942 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T04:14:02.943 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T04:14:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T04:14:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T04:14:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T04:14:02.944 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "*", 2024-09-21T04:14:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T04:14:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "max_count_per_host": { 2024-09-21T04:14:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_count_per_host", 2024-09-21T04:14:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:02.945 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-21T04:14:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of daemons per service per host", 2024-09-21T04:14:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: "max_osd_draining_count": { 2024-09-21T04:14:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_osd_draining_count", 2024-09-21T04:14:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:02.946 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-21T04:14:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T04:14:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: "migration_current": { 2024-09-21T04:14:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "migration_current", 2024-09-21T04:14:02.947 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "internal - do not modify", 2024-09-21T04:14:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-09-21T04:14:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-09-21T04:14:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.948 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "root", 2024-09-21T04:14:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm-package", 2024-09-21T04:14:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: "root" 2024-09-21T04:14:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T04:14:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: "oob_default_addr": { 2024-09-21T04:14:02.949 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "oob_default_addr", 2024-09-21T04:14:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "169.254.1.1", 2024-09-21T04:14:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T04:14:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: "prometheus_alerts_path": { 2024-09-21T04:14:02.950 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus_alerts_path", 2024-09-21T04:14:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T04:14:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T04:14:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_insecure": { 2024-09-21T04:14:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_insecure", 2024-09-21T04:14:02.951 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T04:14:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_password": { 2024-09-21T04:14:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_password", 2024-09-21T04:14:02.952 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T04:14:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_url": { 2024-09-21T04:14:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_url", 2024-09-21T04:14:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.953 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T04:14:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_username": { 2024-09-21T04:14:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_username", 2024-09-21T04:14:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.954 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T04:14:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "secure_monitoring_stack": { 2024-09-21T04:14:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "secure_monitoring_stack", 2024-09-21T04:14:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.955 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T04:14:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: "service_discovery_port": { 2024-09-21T04:14:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "service_discovery_port", 2024-09-21T04:14:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8765", 2024-09-21T04:14:02.956 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "cephadm service discovery port", 2024-09-21T04:14:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.957 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.958 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.958 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.958 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.958 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.958 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.958 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.958 INFO:teuthology.orchestra.run.smithi064.stdout: "ssh_config_file": { 2024-09-21T04:14:02.958 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssh_config_file", 2024-09-21T04:14:02.958 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.958 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.958 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.958 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:02.958 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.958 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T04:14:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: "ssh_keepalive_count_max": { 2024-09-21T04:14:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T04:14:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3", 2024-09-21T04:14:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.959 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T04:14:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "ssh_keepalive_interval": { 2024-09-21T04:14:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssh_keepalive_interval", 2024-09-21T04:14:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7", 2024-09-21T04:14:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.960 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T04:14:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "use_agent": { 2024-09-21T04:14:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_agent", 2024-09-21T04:14:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.961 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T04:14:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: "use_repo_digest": { 2024-09-21T04:14:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_repo_digest", 2024-09-21T04:14:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.962 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T04:14:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_failed_host_check": { 2024-09-21T04:14:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_failed_host_check", 2024-09-21T04:14:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T04:14:02.963 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_daemons": { 2024-09-21T04:14:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_daemons", 2024-09-21T04:14:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T04:14:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.964 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_hosts": { 2024-09-21T04:14:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_hosts", 2024-09-21T04:14:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T04:14:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.965 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.966 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.966 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:02.966 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:02.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.966 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:02.966 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crash", 2024-09-21T04:14:02.966 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:02.966 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:02.966 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:02.966 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:02.966 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:02.966 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.966 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.966 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.967 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:02.967 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.967 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.967 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:02.967 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:02.967 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:02.967 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:02.967 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:02.967 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:02.967 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:02.967 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:02.967 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.967 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.967 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.968 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:02.969 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:02.970 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:02.970 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:02.970 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.970 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.970 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.970 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.970 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.970 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:02.970 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:02.970 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.970 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.970 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.970 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:02.970 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "retain_interval": { 2024-09-21T04:14:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retain_interval", 2024-09-21T04:14:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "31536000", 2024-09-21T04:14:02.971 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T04:14:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:02.972 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_recent_interval": { 2024-09-21T04:14:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_recent_interval", 2024-09-21T04:14:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1209600", 2024-09-21T04:14:02.973 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.974 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.974 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.974 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T04:14:02.974 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.974 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.974 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.974 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:02.974 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:02.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.974 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:02.974 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dashboard", 2024-09-21T04:14:02.974 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:02.974 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:02.974 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:02.975 INFO:teuthology.orchestra.run.smithi064.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T04:14:02.975 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T04:14:02.975 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:02.975 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.975 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.975 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-21T04:14:02.975 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.975 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.975 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.975 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.975 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.975 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.975 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.975 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.975 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T04:14:02.976 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T04:14:02.976 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.976 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.976 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.976 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:02.976 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.976 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.976 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.976 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.976 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.976 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.976 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.976 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T04:14:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T04:14:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_ENABLED": { 2024-09-21T04:14:02.977 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T04:14:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T04:14:02.978 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T04:14:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T04:14:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T04:14:02.979 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T04:14:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T04:14:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.980 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T04:14:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T04:14:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.981 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T04:14:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T04:14:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.982 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T04:14:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T04:14:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:02.983 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T04:14:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T04:14:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:02.984 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T04:14:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T04:14:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.985 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T04:14:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T04:14:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.986 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T04:14:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T04:14:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.987 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T04:14:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T04:14:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-09-21T04:14:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.988 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T04:14:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T04:14:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.989 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.990 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.990 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_URL": { 2024-09-21T04:14:02.990 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_URL", 2024-09-21T04:14:02.990 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.990 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.990 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.990 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:02.990 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.990 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.990 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.990 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.990 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.990 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T04:14:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T04:14:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-09-21T04:14:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.991 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T04:14:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T04:14:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T04:14:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T04:14:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.993 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.994 INFO:teuthology.orchestra.run.smithi064.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T04:14:02.994 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T04:14:02.994 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.994 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.994 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.994 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:02.994 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.994 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.994 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.994 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.994 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.994 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.994 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.994 INFO:teuthology.orchestra.run.smithi064.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T04:14:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T04:14:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T04:14:02.995 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T04:14:02.996 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.996 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.996 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.996 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "[]", 2024-09-21T04:14:02.996 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.996 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.996 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.996 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.996 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.996 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.996 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.996 INFO:teuthology.orchestra.run.smithi064.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T04:14:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T04:14:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "{}", 2024-09-21T04:14:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T04:14:02.997 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T04:14:02.998 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:02.998 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.998 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.998 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:02.998 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.998 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.998 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.998 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.998 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.998 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.998 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.998 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T04:14:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T04:14:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T04:14:02.999 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T04:14:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T04:14:03.000 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T04:14:03.001 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.001 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.001 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.001 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.001 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.001 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.001 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.001 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.001 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.001 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.001 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.001 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T04:14:03.001 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T04:14:03.001 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.002 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.002 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.002 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.002 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.002 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.002 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.002 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.002 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.002 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.002 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.002 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T04:14:03.002 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T04:14:03.002 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.003 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.003 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.003 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.003 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.003 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.003 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.003 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.003 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.003 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.003 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.003 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T04:14:03.003 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T04:14:03.003 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T04:14:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T04:14:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.004 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T04:14:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T04:14:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.005 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.006 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.006 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.006 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.006 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.006 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.006 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.006 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.006 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.006 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.006 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.006 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T04:14:03.006 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T04:14:03.006 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.006 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.006 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.007 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.007 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.007 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.007 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.007 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.007 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.007 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.007 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.007 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T04:14:03.007 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T04:14:03.007 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.007 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.007 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.008 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T04:14:03.008 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.008 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.008 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.008 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.008 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.008 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.008 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.008 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T04:14:03.008 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T04:14:03.008 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.008 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.008 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.008 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-21T04:14:03.009 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.009 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.009 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.009 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.009 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.009 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.009 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.009 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T04:14:03.009 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T04:14:03.009 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.009 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.009 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.010 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8", 2024-09-21T04:14:03.010 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.010 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.010 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.010 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.010 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.010 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.010 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.010 INFO:teuthology.orchestra.run.smithi064.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T04:14:03.010 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T04:14:03.010 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.010 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.010 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.011 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "45", 2024-09-21T04:14:03.011 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.011 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.011 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.011 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.011 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.011 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.011 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.011 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T04:14:03.011 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T04:14:03.011 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.011 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.012 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.012 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.012 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.012 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.012 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.012 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.012 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.012 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.012 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.012 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T04:14:03.012 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T04:14:03.012 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.012 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.012 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.013 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-09-21T04:14:03.013 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.013 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.013 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.013 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.013 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.013 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.013 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.013 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T04:14:03.013 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T04:14:03.013 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.013 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.013 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.014 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.014 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.014 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.014 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.014 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.014 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.014 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.014 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.014 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T04:14:03.014 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T04:14:03.014 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.014 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.014 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.015 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.015 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.015 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.015 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.015 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.015 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.015 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.015 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.015 INFO:teuthology.orchestra.run.smithi064.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T04:14:03.015 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T04:14:03.015 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.015 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.015 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.016 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.016 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.016 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.016 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.016 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.016 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.016 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.016 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.016 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T04:14:03.016 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T04:14:03.016 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.016 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.016 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T04:14:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T04:14:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.017 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.018 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-21T04:14:03.018 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.018 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.018 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.018 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.018 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.018 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.018 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.018 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T04:14:03.018 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T04:14:03.018 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.018 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.018 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.019 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-21T04:14:03.019 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.019 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.019 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.019 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.019 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.019 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.019 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.019 INFO:teuthology.orchestra.run.smithi064.stdout: "cross_origin_url": { 2024-09-21T04:14:03.019 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cross_origin_url", 2024-09-21T04:14:03.019 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.019 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.019 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: "crt_file": { 2024-09-21T04:14:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crt_file", 2024-09-21T04:14:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.020 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "debug": { 2024-09-21T04:14:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "debug", 2024-09-21T04:14:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.021 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.022 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.022 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.022 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable/disable debug options", 2024-09-21T04:14:03.022 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.022 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.022 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.022 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.022 INFO:teuthology.orchestra.run.smithi064.stdout: "jwt_token_ttl": { 2024-09-21T04:14:03.022 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "jwt_token_ttl", 2024-09-21T04:14:03.022 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.022 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.022 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.022 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "28800", 2024-09-21T04:14:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-09-21T04:14:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-09-21T04:14:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.023 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.024 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.025 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.026 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.027 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.028 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "motd": { 2024-09-21T04:14:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "motd", 2024-09-21T04:14:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.029 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "The message of the day", 2024-09-21T04:14:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect_resolve_ip_addr": { 2024-09-21T04:14:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T04:14:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.030 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-09-21T04:14:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-09-21T04:14:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-09-21T04:14:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.031 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-09-21T04:14:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-09-21T04:14:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8080", 2024-09-21T04:14:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.032 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-09-21T04:14:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-09-21T04:14:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl_server_port": { 2024-09-21T04:14:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl_server_port", 2024-09-21T04:14:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8443", 2024-09-21T04:14:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.035 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "sso_oauth2": { 2024-09-21T04:14:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sso_oauth2", 2024-09-21T04:14:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.036 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-09-21T04:14:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-09-21T04:14:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "redirect", 2024-09-21T04:14:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect" 2024-09-21T04:14:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.037 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-09-21T04:14:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-09-21T04:14:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-09-21T04:14:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-09-21T04:14:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-09-21T04:14:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.038 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "url_prefix": { 2024-09-21T04:14:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url_prefix", 2024-09-21T04:14:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.039 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "devicehealth", 2024-09-21T04:14:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_monitoring": { 2024-09-21T04:14:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_monitoring", 2024-09-21T04:14:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.040 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "monitor device health metrics", 2024-09-21T04:14:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.041 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.042 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.044 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.045 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "mark_out_threshold": { 2024-09-21T04:14:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mark_out_threshold", 2024-09-21T04:14:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2419200", 2024-09-21T04:14:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.046 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T04:14:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_name": { 2024-09-21T04:14:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_name", 2024-09-21T04:14:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "device_health_metrics", 2024-09-21T04:14:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T04:14:03.047 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "retention_period": { 2024-09-21T04:14:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retention_period", 2024-09-21T04:14:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15552000", 2024-09-21T04:14:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain device health metrics", 2024-09-21T04:14:03.048 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_frequency": { 2024-09-21T04:14:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_frequency", 2024-09-21T04:14:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-09-21T04:14:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T04:14:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.049 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "self_heal": { 2024-09-21T04:14:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "self_heal", 2024-09-21T04:14:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T04:14:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.050 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-21T04:14:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-21T04:14:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-21T04:14:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T04:14:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.051 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_threshold": { 2024-09-21T04:14:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_threshold", 2024-09-21T04:14:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7257600", 2024-09-21T04:14:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T04:14:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.053 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "diskprediction_local", 2024-09-21T04:14:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.054 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.055 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.056 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.057 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.058 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "predict_interval": { 2024-09-21T04:14:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predict_interval", 2024-09-21T04:14:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-09-21T04:14:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "predictor_model": { 2024-09-21T04:14:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predictor_model", 2024-09-21T04:14:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "prophetstor", 2024-09-21T04:14:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.060 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-21T04:14:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-21T04:14:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-21T04:14:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.061 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.062 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "influx", 2024-09-21T04:14:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": false, 2024-09-21T04:14:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "influxdb python module not found", 2024-09-21T04:14:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "batch_size": { 2024-09-21T04:14:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "batch_size", 2024-09-21T04:14:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5000", 2024-09-21T04:14:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.063 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T04:14:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "database": { 2024-09-21T04:14:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "database", 2024-09-21T04:14:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "ceph", 2024-09-21T04:14:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.064 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T04:14:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": { 2024-09-21T04:14:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hostname", 2024-09-21T04:14:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server hostname", 2024-09-21T04:14:03.065 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-21T04:14:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-21T04:14:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "30", 2024-09-21T04:14:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "5", 2024-09-21T04:14:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T04:14:03.066 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.067 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.068 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.069 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.070 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.071 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.074 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "password": { 2024-09-21T04:14:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "password", 2024-09-21T04:14:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.075 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "password of InfluxDB server user", 2024-09-21T04:14:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "port": { 2024-09-21T04:14:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "port", 2024-09-21T04:14:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8086", 2024-09-21T04:14:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.076 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server port", 2024-09-21T04:14:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.077 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-09-21T04:14:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-09-21T04:14:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "false", 2024-09-21T04:14:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.078 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T04:14:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "threads": { 2024-09-21T04:14:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threads", 2024-09-21T04:14:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-21T04:14:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-21T04:14:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "32", 2024-09-21T04:14:03.079 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T04:14:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "username": { 2024-09-21T04:14:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "username", 2024-09-21T04:14:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.080 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "username of InfluxDB server user", 2024-09-21T04:14:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "verify_ssl": { 2024-09-21T04:14:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "verify_ssl", 2024-09-21T04:14:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "true", 2024-09-21T04:14:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.081 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T04:14:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "insights", 2024-09-21T04:14:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.082 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.083 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.084 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.085 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.086 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.087 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "iostat", 2024-09-21T04:14:03.088 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.089 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.090 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.091 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.092 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.093 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.094 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "k8sevents", 2024-09-21T04:14:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_event_retention_days": { 2024-09-21T04:14:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ceph_event_retention_days", 2024-09-21T04:14:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7", 2024-09-21T04:14:03.095 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T04:14:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "config_check_secs": { 2024-09-21T04:14:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_check_secs", 2024-09-21T04:14:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-21T04:14:03.096 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "10", 2024-09-21T04:14:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T04:14:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.097 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.098 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.099 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.100 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.101 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.102 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "localpool", 2024-09-21T04:14:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "failure_domain": { 2024-09-21T04:14:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "failure_domain", 2024-09-21T04:14:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.103 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "host", 2024-09-21T04:14:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "failure domain for any created local pool", 2024-09-21T04:14:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T04:14:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.104 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.105 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.106 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.107 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.108 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "min_size": { 2024-09-21T04:14:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_size", 2024-09-21T04:14:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.109 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default min_size for any created local pool", 2024-09-21T04:14:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T04:14:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "num_rep": { 2024-09-21T04:14:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "num_rep", 2024-09-21T04:14:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3", 2024-09-21T04:14:03.110 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default replica count for any created local pool", 2024-09-21T04:14:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_num": { 2024-09-21T04:14:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_num", 2024-09-21T04:14:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "128", 2024-09-21T04:14:03.111 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T04:14:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "prefix": { 2024-09-21T04:14:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prefix", 2024-09-21T04:14:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.112 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name prefix for any created local pool", 2024-09-21T04:14:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.113 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: "subtree": { 2024-09-21T04:14:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "subtree", 2024-09-21T04:14:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "rack", 2024-09-21T04:14:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.114 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.115 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T04:14:03.115 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T04:14:03.115 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.115 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.115 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.115 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.115 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mds_autoscaler", 2024-09-21T04:14:03.115 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.115 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.115 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.115 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.115 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.116 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.117 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.117 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.117 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.117 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.117 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.117 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.117 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.117 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.117 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.117 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.117 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.117 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.118 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.118 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.118 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.118 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.118 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.118 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.118 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.118 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.118 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.118 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.118 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.118 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.119 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.120 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.121 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.121 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.121 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.121 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.121 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.121 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.121 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.121 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.121 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.121 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.121 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.121 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirroring", 2024-09-21T04:14:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.122 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.123 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.124 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.125 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.126 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.127 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "nfs", 2024-09-21T04:14:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.128 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.129 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.130 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.131 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.132 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.133 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-09-21T04:14:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: "fail_fs": { 2024-09-21T04:14:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "fail_fs", 2024-09-21T04:14:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.134 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T04:14:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.135 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.136 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.137 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.138 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.139 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator": { 2024-09-21T04:14:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-09-21T04:14:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.140 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm", 2024-09-21T04:14:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: "rook", 2024-09-21T04:14:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: "test_orchestrator" 2024-09-21T04:14:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Orchestrator backend", 2024-09-21T04:14:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.141 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_perf_query", 2024-09-21T04:14:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.142 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.143 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.144 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.145 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.146 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.147 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.148 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_support", 2024-09-21T04:14:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.149 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.150 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.151 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.152 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.153 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.154 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_autoscaler", 2024-09-21T04:14:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.155 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.156 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.157 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.158 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.159 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-21T04:14:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-21T04:14:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.160 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-21T04:14:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.161 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: "threshold": { 2024-09-21T04:14:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threshold", 2024-09-21T04:14:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-21T04:14:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.162 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-09-21T04:14:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1.0", 2024-09-21T04:14:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "scaling threshold", 2024-09-21T04:14:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T04:14:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.163 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "progress", 2024-09-21T04:14:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_pg_recovery_event": { 2024-09-21T04:14:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_pg_recovery_event", 2024-09-21T04:14:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T04:14:03.164 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-09-21T04:14:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-09-21T04:14:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.165 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.166 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.167 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.168 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.169 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: "max_completed_events": { 2024-09-21T04:14:03.170 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_completed_events", 2024-09-21T04:14:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "50", 2024-09-21T04:14:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "number of past completed events to remember", 2024-09-21T04:14:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.171 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-21T04:14:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-21T04:14:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-21T04:14:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long the module is going to sleep", 2024-09-21T04:14:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.173 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus", 2024-09-21T04:14:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: "cache": { 2024-09-21T04:14:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cache", 2024-09-21T04:14:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.174 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: "exclude_perf_counters": { 2024-09-21T04:14:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "exclude_perf_counters", 2024-09-21T04:14:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.175 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T04:14:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T04:14:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.176 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.177 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.178 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.179 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.180 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools": { 2024-09-21T04:14:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools", 2024-09-21T04:14:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.181 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T04:14:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T04:14:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "300", 2024-09-21T04:14:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.182 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_interval": { 2024-09-21T04:14:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_interval", 2024-09-21T04:14:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-21T04:14:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15.0", 2024-09-21T04:14:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.183 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-09-21T04:14:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-09-21T04:14:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-09-21T04:14:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.184 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T04:14:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.186 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-09-21T04:14:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-09-21T04:14:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "9283", 2024-09-21T04:14:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T04:14:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.187 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.188 INFO:teuthology.orchestra.run.smithi064.stdout: "stale_cache_strategy": { 2024-09-21T04:14:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stale_cache_strategy", 2024-09-21T04:14:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "log", 2024-09-21T04:14:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-09-21T04:14:03.189 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-09-21T04:14:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "default", 2024-09-21T04:14:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: "default", 2024-09-21T04:14:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: "error" 2024-09-21T04:14:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.190 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-09-21T04:14:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-09-21T04:14:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-09-21T04:14:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-09-21T04:14:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-09-21T04:14:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.191 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_support", 2024-09-21T04:14:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.192 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.193 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.194 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.195 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.196 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_snap_create": { 2024-09-21T04:14:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_snap_create", 2024-09-21T04:14:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-21T04:14:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.197 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.198 INFO:teuthology.orchestra.run.smithi064.stdout: "mirror_snapshot_schedule": { 2024-09-21T04:14:03.198 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T04:14:03.198 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.198 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.198 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.198 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.198 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.198 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.198 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.198 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.198 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.198 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.199 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.200 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.200 INFO:teuthology.orchestra.run.smithi064.stdout: "trash_purge_schedule": { 2024-09-21T04:14:03.200 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "trash_purge_schedule", 2024-09-21T04:14:03.200 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.200 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.200 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.200 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.200 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.200 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.200 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.200 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.200 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.200 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.200 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "restful", 2024-09-21T04:14:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_auth": { 2024-09-21T04:14:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_auth", 2024-09-21T04:14:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.201 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-09-21T04:14:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-09-21T04:14:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.202 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.203 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.204 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.205 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.206 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.207 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: "max_requests": { 2024-09-21T04:14:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_requests", 2024-09-21T04:14:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-09-21T04:14:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T04:14:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.208 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-09-21T04:14:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-09-21T04:14:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-09-21T04:14:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-09-21T04:14:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.210 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.211 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.211 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.211 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.211 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.211 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.211 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.211 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.211 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.211 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.211 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.211 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.211 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.211 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.211 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rgw", 2024-09-21T04:14:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.212 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.213 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.214 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.215 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.216 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T04:14:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T04:14:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-21T04:14:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T04:14:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.217 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.218 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rook", 2024-09-21T04:14:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.219 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.220 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.221 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.222 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.223 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.224 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.224 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.224 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.224 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.224 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.224 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.224 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.224 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.224 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.224 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.224 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.224 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: "storage_class": { 2024-09-21T04:14:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "storage_class", 2024-09-21T04:14:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "local", 2024-09-21T04:14:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.225 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T04:14:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "selftest", 2024-09-21T04:14:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.226 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.227 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.228 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.229 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.230 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.231 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.231 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.231 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.231 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.231 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.231 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.231 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.231 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.231 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.231 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.231 INFO:teuthology.orchestra.run.smithi064.stdout: "roption1": { 2024-09-21T04:14:03.231 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption1", 2024-09-21T04:14:03.231 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: "roption2": { 2024-09-21T04:14:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption2", 2024-09-21T04:14:03.232 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "xyz", 2024-09-21T04:14:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption1": { 2024-09-21T04:14:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption1", 2024-09-21T04:14:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.233 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.234 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.234 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.234 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.234 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.234 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.234 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.234 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.234 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.234 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.234 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.234 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption2": { 2024-09-21T04:14:03.234 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption2", 2024-09-21T04:14:03.234 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.234 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption3": { 2024-09-21T04:14:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption3", 2024-09-21T04:14:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-21T04:14:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.235 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption4": { 2024-09-21T04:14:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption4", 2024-09-21T04:14:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.236 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption5": { 2024-09-21T04:14:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption5", 2024-09-21T04:14:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.237 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.238 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.238 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.238 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.238 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.238 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.238 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.238 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption6": { 2024-09-21T04:14:03.238 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption6", 2024-09-21T04:14:03.238 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.238 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.238 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.238 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.239 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.239 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.239 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.239 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.239 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.239 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.239 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.239 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.239 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption7": { 2024-09-21T04:14:03.239 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption7", 2024-09-21T04:14:03.239 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.239 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.239 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.239 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.239 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-21T04:14:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "42", 2024-09-21T04:14:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.240 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: "testkey": { 2024-09-21T04:14:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testkey", 2024-09-21T04:14:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.241 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.242 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.242 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.242 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.242 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.242 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.242 INFO:teuthology.orchestra.run.smithi064.stdout: "testlkey": { 2024-09-21T04:14:03.242 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testlkey", 2024-09-21T04:14:03.242 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.242 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.242 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.242 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.242 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.243 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.243 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.243 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.243 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.243 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.243 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.243 INFO:teuthology.orchestra.run.smithi064.stdout: "testnewline": { 2024-09-21T04:14:03.243 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testnewline", 2024-09-21T04:14:03.243 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.243 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.243 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.243 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.243 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smb", 2024-09-21T04:14:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.244 INFO:teuthology.orchestra.run.smithi064.stdout: "internal_store_backend": { 2024-09-21T04:14:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "internal_store_backend", 2024-09-21T04:14:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T04:14:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.245 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.246 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.247 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.248 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.249 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.249 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.249 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.249 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.249 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.249 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.249 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.249 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.249 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.249 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.249 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.249 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.249 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.249 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.249 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.250 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.251 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.251 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.251 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.251 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.251 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.251 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.251 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.251 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.251 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.251 INFO:teuthology.orchestra.run.smithi064.stdout: "update_orchestration": { 2024-09-21T04:14:03.251 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "update_orchestration", 2024-09-21T04:14:03.251 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T04:14:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.252 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2024-09-21T04:14:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_m_granularity": { 2024-09-21T04:14:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_m_granularity", 2024-09-21T04:14:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T04:14:03.253 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "dump_on_update": { 2024-09-21T04:14:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dump_on_update", 2024-09-21T04:14:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "dump database to debug log on update", 2024-09-21T04:14:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.254 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.255 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.256 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.257 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.258 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.259 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.260 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stats", 2024-09-21T04:14:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.261 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.262 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.263 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.265 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.266 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "status", 2024-09-21T04:14:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.267 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.268 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.269 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.270 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.271 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.272 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telegraf", 2024-09-21T04:14:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "address": { 2024-09-21T04:14:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "address", 2024-09-21T04:14:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.273 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T04:14:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-21T04:14:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-21T04:14:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.274 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15", 2024-09-21T04:14:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.275 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.276 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.277 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.278 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.279 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.280 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telemetry", 2024-09-21T04:14:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_basic": { 2024-09-21T04:14:03.281 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_basic", 2024-09-21T04:14:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T04:14:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_crash": { 2024-09-21T04:14:03.282 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_crash", 2024-09-21T04:14:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T04:14:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_device": { 2024-09-21T04:14:03.283 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_device", 2024-09-21T04:14:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T04:14:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_ident": { 2024-09-21T04:14:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_ident", 2024-09-21T04:14:03.284 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T04:14:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_perf": { 2024-09-21T04:14:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_perf", 2024-09-21T04:14:03.285 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T04:14:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "contact": { 2024-09-21T04:14:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "contact", 2024-09-21T04:14:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.286 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "description": { 2024-09-21T04:14:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "description", 2024-09-21T04:14:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.287 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "device_url": { 2024-09-21T04:14:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_url", 2024-09-21T04:14:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.288 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T04:14:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-09-21T04:14:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-09-21T04:14:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.289 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-21T04:14:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-21T04:14:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "24", 2024-09-21T04:14:03.290 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "8", 2024-09-21T04:14:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "last_opt_revision": { 2024-09-21T04:14:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "last_opt_revision", 2024-09-21T04:14:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.291 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1", 2024-09-21T04:14:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard": { 2024-09-21T04:14:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard", 2024-09-21T04:14:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.292 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard_description": { 2024-09-21T04:14:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard_description", 2024-09-21T04:14:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.293 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.294 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.295 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.296 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.297 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.298 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "organization": { 2024-09-21T04:14:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "organization", 2024-09-21T04:14:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.299 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "proxy": { 2024-09-21T04:14:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "proxy", 2024-09-21T04:14:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.300 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.301 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "url": { 2024-09-21T04:14:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url", 2024-09-21T04:14:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T04:14:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.302 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "test_orchestrator", 2024-09-21T04:14:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.303 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.304 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.305 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.306 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.307 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.308 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volumes", 2024-09-21T04:14:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.309 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.310 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.311 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.312 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.313 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_clones": { 2024-09-21T04:14:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_clones", 2024-09-21T04:14:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.314 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4", 2024-09-21T04:14:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T04:14:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "periodic_async_work": { 2024-09-21T04:14:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "periodic_async_work", 2024-09-21T04:14:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.315 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Periodically check for async work", 2024-09-21T04:14:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_delay": { 2024-09-21T04:14:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_delay", 2024-09-21T04:14:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.316 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T04:14:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_no_wait": { 2024-09-21T04:14:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T04:14:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.317 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T04:14:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.318 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix", 2024-09-21T04:14:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.319 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "discovery_interval": { 2024-09-21T04:14:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "discovery_interval", 2024-09-21T04:14:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-21T04:14:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "100", 2024-09-21T04:14:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.320 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "identifier": { 2024-09-21T04:14:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "identifier", 2024-09-21T04:14:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.321 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-21T04:14:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-21T04:14:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-21T04:14:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.322 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.323 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.324 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.325 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.326 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.327 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_host": { 2024-09-21T04:14:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_host", 2024-09-21T04:14:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.328 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_port": { 2024-09-21T04:14:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_port", 2024-09-21T04:14:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10051", 2024-09-21T04:14:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.329 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_sender": { 2024-09-21T04:14:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_sender", 2024-09-21T04:14:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T04:14:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.330 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-21T04:14:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "gid": 24170, 2024-09-21T04:14:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "y", 2024-09-21T04:14:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "mgr_features": 4540719139924082687, 2024-09-21T04:14:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: "available_modules": [ 2024-09-21T04:14:03.331 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "alerts", 2024-09-21T04:14:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-21T04:14:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-21T04:14:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-21T04:14:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T04:14:03.332 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.333 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.334 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.335 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.336 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.337 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_destination": { 2024-09-21T04:14:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_destination", 2024-09-21T04:14:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T04:14:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_from_name": { 2024-09-21T04:14:03.338 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_from_name", 2024-09-21T04:14:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "Ceph", 2024-09-21T04:14:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email From: name", 2024-09-21T04:14:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_host": { 2024-09-21T04:14:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_host", 2024-09-21T04:14:03.339 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP server", 2024-09-21T04:14:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_password": { 2024-09-21T04:14:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_password", 2024-09-21T04:14:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.340 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Password to authenticate with", 2024-09-21T04:14:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_port": { 2024-09-21T04:14:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_port", 2024-09-21T04:14:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.341 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "465", 2024-09-21T04:14:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP port", 2024-09-21T04:14:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_sender": { 2024-09-21T04:14:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_sender", 2024-09-21T04:14:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.342 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP envelope sender", 2024-09-21T04:14:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_ssl": { 2024-09-21T04:14:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_ssl", 2024-09-21T04:14:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.343 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T04:14:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_user": { 2024-09-21T04:14:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_user", 2024-09-21T04:14:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.344 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "User to authenticate as", 2024-09-21T04:14:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.345 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "balancer", 2024-09-21T04:14:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.346 INFO:teuthology.orchestra.run.smithi064.stdout: "active": { 2024-09-21T04:14:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "active", 2024-09-21T04:14:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T04:14:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.347 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_time": { 2024-09-21T04:14:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_time", 2024-09-21T04:14:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0000", 2024-09-21T04:14:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T04:14:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T04:14:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.348 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_weekday": { 2024-09-21T04:14:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_weekday", 2024-09-21T04:14:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-21T04:14:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-09-21T04:14:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-09-21T04:14:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T04:14:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T04:14:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.349 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_max_iterations": { 2024-09-21T04:14:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_max_iterations", 2024-09-21T04:14:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-21T04:14:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "25", 2024-09-21T04:14:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-21T04:14:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "250", 2024-09-21T04:14:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T04:14:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_metrics": { 2024-09-21T04:14:03.350 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_metrics", 2024-09-21T04:14:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T04:14:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T04:14:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T04:14:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_step": { 2024-09-21T04:14:03.351 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_step", 2024-09-21T04:14:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-21T04:14:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.5", 2024-09-21T04:14:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0.001", 2024-09-21T04:14:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "0.999", 2024-09-21T04:14:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "aggressiveness of optimization", 2024-09-21T04:14:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T04:14:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "end_time": { 2024-09-21T04:14:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_time", 2024-09-21T04:14:03.352 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2359", 2024-09-21T04:14:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T04:14:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T04:14:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "end_weekday": { 2024-09-21T04:14:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_weekday", 2024-09-21T04:14:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-21T04:14:03.353 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-09-21T04:14:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-09-21T04:14:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T04:14:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T04:14:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.354 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.355 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.356 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.357 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.358 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "min_score": { 2024-09-21T04:14:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_score", 2024-09-21T04:14:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-21T04:14:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.359 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T04:14:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-09-21T04:14:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-09-21T04:14:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "upmap", 2024-09-21T04:14:03.360 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "crush-compat", 2024-09-21T04:14:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "none", 2024-09-21T04:14:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "read", 2024-09-21T04:14:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap", 2024-09-21T04:14:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap-read" 2024-09-21T04:14:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Balancer mode", 2024-09-21T04:14:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_ids": { 2024-09-21T04:14:03.361 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_ids", 2024-09-21T04:14:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T04:14:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-21T04:14:03.362 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-21T04:14:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-21T04:14:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T04:14:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.363 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_deviation": { 2024-09-21T04:14:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_deviation", 2024-09-21T04:14:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.364 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-21T04:14:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-21T04:14:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T04:14:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T04:14:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_optimizations": { 2024-09-21T04:14:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_optimizations", 2024-09-21T04:14:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-21T04:14:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.365 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-21T04:14:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T04:14:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm", 2024-09-21T04:14:03.366 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_down_multiplier": { 2024-09-21T04:14:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_down_multiplier", 2024-09-21T04:14:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-21T04:14:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-09-21T04:14:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T04:14:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.367 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_refresh_rate": { 2024-09-21T04:14:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_refresh_rate", 2024-09-21T04:14:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "20", 2024-09-21T04:14:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T04:14:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.368 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_starting_port": { 2024-09-21T04:14:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_starting_port", 2024-09-21T04:14:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4721", 2024-09-21T04:14:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T04:14:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.369 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_ptrace": { 2024-09-21T04:14:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_ptrace", 2024-09-21T04:14:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T04:14:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T04:14:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.370 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_interval": { 2024-09-21T04:14:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_interval", 2024-09-21T04:14:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-21T04:14:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T04:14:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.371 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_memory_target_ratio": { 2024-09-21T04:14:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T04:14:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-21T04:14:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.7", 2024-09-21T04:14:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T04:14:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.372 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm_log_destination": { 2024-09-21T04:14:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm_log_destination", 2024-09-21T04:14:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "file", 2024-09-21T04:14:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "file,syslog", 2024-09-21T04:14:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "syslog" 2024-09-21T04:14:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T04:14:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.373 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "cgroups_split": { 2024-09-21T04:14:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cgroups_split", 2024-09-21T04:14:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T04:14:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.374 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "config_checks_enabled": { 2024-09-21T04:14:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_checks_enabled", 2024-09-21T04:14:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T04:14:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.375 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "config_dashboard": { 2024-09-21T04:14:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_dashboard", 2024-09-21T04:14:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T04:14:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.376 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_alertmanager": { 2024-09-21T04:14:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_alertmanager", 2024-09-21T04:14:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T04:14:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-21T04:14:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.377 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_base": { 2024-09-21T04:14:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_base", 2024-09-21T04:14:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T04:14:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Container image name, without the tag", 2024-09-21T04:14:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.378 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_elasticsearch": { 2024-09-21T04:14:03.379 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_elasticsearch", 2024-09-21T04:14:03.379 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.379 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.379 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.379 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T04:14:03.379 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.379 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.379 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.379 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "elasticsearch container image", 2024-09-21T04:14:03.379 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.379 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.379 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.379 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.379 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_grafana": { 2024-09-21T04:14:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_grafana", 2024-09-21T04:14:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T04:14:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-21T04:14:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.380 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_haproxy": { 2024-09-21T04:14:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_haproxy", 2024-09-21T04:14:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T04:14:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "HAproxy container image", 2024-09-21T04:14:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.381 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_agent": { 2024-09-21T04:14:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_agent", 2024-09-21T04:14:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T04:14:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger agent container image", 2024-09-21T04:14:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.382 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_collector": { 2024-09-21T04:14:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_collector", 2024-09-21T04:14:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T04:14:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger collector container image", 2024-09-21T04:14:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_query": { 2024-09-21T04:14:03.383 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_query", 2024-09-21T04:14:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T04:14:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger query container image", 2024-09-21T04:14:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_keepalived": { 2024-09-21T04:14:03.384 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_keepalived", 2024-09-21T04:14:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T04:14:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Keepalived container image", 2024-09-21T04:14:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_loki": { 2024-09-21T04:14:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_loki", 2024-09-21T04:14:03.385 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T04:14:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Loki container image", 2024-09-21T04:14:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_nginx": { 2024-09-21T04:14:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_nginx", 2024-09-21T04:14:03.386 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T04:14:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Nginx container image", 2024-09-21T04:14:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_node_exporter": { 2024-09-21T04:14:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_node_exporter", 2024-09-21T04:14:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.387 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T04:14:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-21T04:14:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_nvmeof": { 2024-09-21T04:14:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_nvmeof", 2024-09-21T04:14:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.388 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T04:14:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Nvme-of container image", 2024-09-21T04:14:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_oauth2_proxy": { 2024-09-21T04:14:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T04:14:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.389 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T04:14:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "oauth2-proxy container image", 2024-09-21T04:14:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_prometheus": { 2024-09-21T04:14:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_prometheus", 2024-09-21T04:14:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T04:14:03.390 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-21T04:14:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_promtail": { 2024-09-21T04:14:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_promtail", 2024-09-21T04:14:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T04:14:03.391 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Promtail container image", 2024-09-21T04:14:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_samba": { 2024-09-21T04:14:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_samba", 2024-09-21T04:14:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T04:14:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.392 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Samba/SMB container image", 2024-09-21T04:14:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_samba_metrics": { 2024-09-21T04:14:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_samba_metrics", 2024-09-21T04:14:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T04:14:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.393 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T04:14:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_snmp_gateway": { 2024-09-21T04:14:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_snmp_gateway", 2024-09-21T04:14:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T04:14:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.394 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SNMP Gateway container image", 2024-09-21T04:14:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "container_init": { 2024-09-21T04:14:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_init", 2024-09-21T04:14:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.395 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T04:14:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "daemon_cache_timeout": { 2024-09-21T04:14:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "daemon_cache_timeout", 2024-09-21T04:14:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-21T04:14:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.396 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T04:14:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "default_cephadm_command_timeout": { 2024-09-21T04:14:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T04:14:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "900", 2024-09-21T04:14:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T04:14:03.397 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: "default_registry": { 2024-09-21T04:14:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_registry", 2024-09-21T04:14:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io", 2024-09-21T04:14:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T04:14:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.398 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "device_cache_timeout": { 2024-09-21T04:14:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_cache_timeout", 2024-09-21T04:14:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1800", 2024-09-21T04:14:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache device inventory", 2024-09-21T04:14:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.399 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.400 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.400 INFO:teuthology.orchestra.run.smithi064.stdout: "device_enhanced_scan": { 2024-09-21T04:14:03.400 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_enhanced_scan", 2024-09-21T04:14:03.400 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.400 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.400 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.400 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.400 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.400 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.400 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.400 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T04:14:03.400 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.400 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.400 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.401 INFO:teuthology.orchestra.run.smithi064.stdout: "facts_cache_timeout": { 2024-09-21T04:14:03.401 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "facts_cache_timeout", 2024-09-21T04:14:03.401 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.401 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.401 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.401 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-21T04:14:03.401 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.401 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.401 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.401 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache host facts data", 2024-09-21T04:14:03.401 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.401 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.401 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.401 INFO:teuthology.orchestra.run.smithi064.stdout: "grafana_dashboards_path": { 2024-09-21T04:14:03.402 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "grafana_dashboards_path", 2024-09-21T04:14:03.402 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.402 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.402 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.402 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T04:14:03.402 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.402 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.402 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.402 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T04:14:03.402 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.402 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.402 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.402 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.402 INFO:teuthology.orchestra.run.smithi064.stdout: "host_check_interval": { 2024-09-21T04:14:03.403 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "host_check_interval", 2024-09-21T04:14:03.403 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.403 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.403 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.403 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-21T04:14:03.403 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.403 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.403 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.403 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to perform a host check", 2024-09-21T04:14:03.403 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.403 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.403 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.403 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.404 INFO:teuthology.orchestra.run.smithi064.stdout: "hw_monitoring": { 2024-09-21T04:14:03.404 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hw_monitoring", 2024-09-21T04:14:03.404 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.404 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.404 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.404 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.404 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.404 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.404 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.404 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T04:14:03.404 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.404 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.404 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.404 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.405 INFO:teuthology.orchestra.run.smithi064.stdout: "inventory_list_all": { 2024-09-21T04:14:03.405 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "inventory_list_all", 2024-09-21T04:14:03.405 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.405 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.405 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.405 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.405 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.405 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.405 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.405 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T04:14:03.405 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.405 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.405 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.405 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.405 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.406 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.406 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.406 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.406 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.406 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.406 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.406 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.406 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.406 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.406 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.406 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.406 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.406 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.406 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.406 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.407 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.407 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.407 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.407 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.407 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.407 INFO:teuthology.orchestra.run.smithi064.stdout: "log_refresh_metadata": { 2024-09-21T04:14:03.407 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_refresh_metadata", 2024-09-21T04:14:03.407 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.407 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.407 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.407 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.407 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.407 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.407 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.407 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T04:14:03.408 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.408 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.408 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.408 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.408 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.408 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.408 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.408 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.408 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.408 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.408 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.408 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.408 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T04:14:03.408 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.409 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.409 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.409 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.409 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.409 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.409 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.409 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.409 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.409 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.409 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.409 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.409 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.409 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.409 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.409 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.410 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.410 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.410 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.410 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.410 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.410 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.410 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.410 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.410 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.410 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.410 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.410 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.410 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.411 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.411 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.411 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.411 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.411 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.411 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.411 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.411 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.411 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T04:14:03.411 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T04:14:03.411 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.411 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.411 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.411 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.412 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.412 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.412 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.412 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T04:14:03.412 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.412 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.412 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.412 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T04:14:03.412 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T04:14:03.412 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.412 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.412 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.412 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "*", 2024-09-21T04:14:03.413 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.413 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.413 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.413 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T04:14:03.413 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.413 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.413 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.413 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.413 INFO:teuthology.orchestra.run.smithi064.stdout: "max_count_per_host": { 2024-09-21T04:14:03.413 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_count_per_host", 2024-09-21T04:14:03.413 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.413 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.413 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.413 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-21T04:14:03.413 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.414 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.414 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.414 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of daemons per service per host", 2024-09-21T04:14:03.414 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.414 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.414 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.414 INFO:teuthology.orchestra.run.smithi064.stdout: "max_osd_draining_count": { 2024-09-21T04:14:03.414 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_osd_draining_count", 2024-09-21T04:14:03.414 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.414 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.414 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.414 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-21T04:14:03.414 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.415 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.415 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.415 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T04:14:03.415 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.415 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.415 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.415 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.415 INFO:teuthology.orchestra.run.smithi064.stdout: "migration_current": { 2024-09-21T04:14:03.415 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "migration_current", 2024-09-21T04:14:03.415 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.415 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.415 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.415 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.416 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.416 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.416 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.416 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "internal - do not modify", 2024-09-21T04:14:03.416 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.416 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.416 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.416 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.416 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-09-21T04:14:03.416 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-09-21T04:14:03.416 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.416 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.416 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.416 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "root", 2024-09-21T04:14:03.417 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.417 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.417 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.417 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm-package", 2024-09-21T04:14:03.417 INFO:teuthology.orchestra.run.smithi064.stdout: "root" 2024-09-21T04:14:03.417 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.417 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T04:14:03.417 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.417 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.417 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.417 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.417 INFO:teuthology.orchestra.run.smithi064.stdout: "oob_default_addr": { 2024-09-21T04:14:03.417 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "oob_default_addr", 2024-09-21T04:14:03.417 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.418 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.418 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.418 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "169.254.1.1", 2024-09-21T04:14:03.418 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.418 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.418 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.418 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T04:14:03.418 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.418 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.418 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.418 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.418 INFO:teuthology.orchestra.run.smithi064.stdout: "prometheus_alerts_path": { 2024-09-21T04:14:03.418 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus_alerts_path", 2024-09-21T04:14:03.418 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.418 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.419 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.419 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T04:14:03.419 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.419 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.419 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.419 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T04:14:03.419 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.419 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.419 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.419 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.419 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_insecure": { 2024-09-21T04:14:03.419 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_insecure", 2024-09-21T04:14:03.419 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.419 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.419 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.420 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.420 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.420 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.420 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.420 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T04:14:03.420 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.420 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.420 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.420 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.420 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_password": { 2024-09-21T04:14:03.420 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_password", 2024-09-21T04:14:03.420 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.420 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.420 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T04:14:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_url": { 2024-09-21T04:14:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_url", 2024-09-21T04:14:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.421 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.422 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.422 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.422 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.422 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.422 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T04:14:03.422 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.422 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.422 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.422 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_username": { 2024-09-21T04:14:03.422 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_username", 2024-09-21T04:14:03.422 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.422 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.422 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T04:14:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: "secure_monitoring_stack": { 2024-09-21T04:14:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "secure_monitoring_stack", 2024-09-21T04:14:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.423 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T04:14:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: "service_discovery_port": { 2024-09-21T04:14:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "service_discovery_port", 2024-09-21T04:14:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.424 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.425 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8765", 2024-09-21T04:14:03.425 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.425 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.425 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.425 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "cephadm service discovery port", 2024-09-21T04:14:03.425 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.425 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.425 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.425 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.425 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.425 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.425 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.425 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.425 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.425 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.426 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.426 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.426 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.426 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.426 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.426 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.426 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.426 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.426 INFO:teuthology.orchestra.run.smithi064.stdout: "ssh_config_file": { 2024-09-21T04:14:03.426 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssh_config_file", 2024-09-21T04:14:03.426 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.426 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.426 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.426 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.427 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.427 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.427 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.427 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T04:14:03.427 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.427 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.427 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.427 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.427 INFO:teuthology.orchestra.run.smithi064.stdout: "ssh_keepalive_count_max": { 2024-09-21T04:14:03.427 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T04:14:03.427 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.427 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.427 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.427 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3", 2024-09-21T04:14:03.428 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.428 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.428 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.428 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T04:14:03.428 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.428 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.428 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.428 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.428 INFO:teuthology.orchestra.run.smithi064.stdout: "ssh_keepalive_interval": { 2024-09-21T04:14:03.428 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssh_keepalive_interval", 2024-09-21T04:14:03.428 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.428 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.428 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.428 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7", 2024-09-21T04:14:03.429 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.429 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.429 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.429 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T04:14:03.429 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.429 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.429 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.429 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.429 INFO:teuthology.orchestra.run.smithi064.stdout: "use_agent": { 2024-09-21T04:14:03.429 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_agent", 2024-09-21T04:14:03.429 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.429 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.429 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.429 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.429 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T04:14:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: "use_repo_digest": { 2024-09-21T04:14:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_repo_digest", 2024-09-21T04:14:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.430 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T04:14:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_failed_host_check": { 2024-09-21T04:14:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_failed_host_check", 2024-09-21T04:14:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.431 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T04:14:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_daemons": { 2024-09-21T04:14:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_daemons", 2024-09-21T04:14:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.432 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.433 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.433 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T04:14:03.433 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.433 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.433 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.433 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_hosts": { 2024-09-21T04:14:03.433 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_hosts", 2024-09-21T04:14:03.433 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.433 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.433 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.433 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.433 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.433 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.434 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.434 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T04:14:03.434 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.434 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.434 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.434 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.434 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.434 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.434 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.434 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crash", 2024-09-21T04:14:03.434 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.434 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.434 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.434 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.435 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.436 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.437 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.438 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: "retain_interval": { 2024-09-21T04:14:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retain_interval", 2024-09-21T04:14:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.439 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "31536000", 2024-09-21T04:14:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T04:14:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.440 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_recent_interval": { 2024-09-21T04:14:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_recent_interval", 2024-09-21T04:14:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1209600", 2024-09-21T04:14:03.441 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.442 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.442 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.442 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T04:14:03.442 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.442 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.442 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.442 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.442 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.442 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.442 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.442 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dashboard", 2024-09-21T04:14:03.442 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.442 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.442 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T04:14:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T04:14:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-21T04:14:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.443 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T04:14:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T04:14:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T04:14:03.444 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T04:14:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_ENABLED": { 2024-09-21T04:14:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T04:14:03.445 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.446 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.446 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.446 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.446 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.446 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.446 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.446 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.446 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.446 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.446 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.446 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.446 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T04:14:03.446 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T04:14:03.446 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.447 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.447 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.447 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.447 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.447 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.447 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.447 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.447 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.447 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.447 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.447 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.447 INFO:teuthology.orchestra.run.smithi064.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T04:14:03.447 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T04:14:03.447 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T04:14:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T04:14:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.448 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T04:14:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T04:14:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.449 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T04:14:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T04:14:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.450 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T04:14:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T04:14:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.451 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.452 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.452 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.452 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.452 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.452 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.452 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.452 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.452 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.452 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.452 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T04:14:03.452 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T04:14:03.452 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.452 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.452 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.453 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.453 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.453 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.453 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.453 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.453 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.453 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.453 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.453 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.453 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T04:14:03.453 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T04:14:03.453 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.453 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.453 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.454 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.454 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.454 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.454 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.454 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.454 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.454 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.454 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.454 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.454 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T04:14:03.454 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T04:14:03.454 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.454 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T04:14:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T04:14:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.455 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T04:14:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T04:14:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.456 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.457 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-09-21T04:14:03.457 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.457 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.457 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.457 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.457 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.457 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.457 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.457 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.457 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T04:14:03.457 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T04:14:03.457 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.457 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.457 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.458 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.458 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.458 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.458 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.458 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.458 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.458 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.458 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.458 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.458 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_URL": { 2024-09-21T04:14:03.458 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_URL", 2024-09-21T04:14:03.458 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.458 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.458 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.458 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.459 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.459 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.459 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.459 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.459 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.459 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.459 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.459 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.459 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T04:14:03.459 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T04:14:03.459 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.459 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.459 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-09-21T04:14:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T04:14:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T04:14:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.460 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.461 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.461 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.461 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.461 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.461 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.461 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.461 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.461 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T04:14:03.461 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T04:14:03.461 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.461 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.461 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.461 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.461 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.462 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.462 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.462 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.462 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.462 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.462 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.462 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.462 INFO:teuthology.orchestra.run.smithi064.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T04:14:03.462 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T04:14:03.462 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.462 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.462 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.462 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.462 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.462 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T04:14:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T04:14:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.463 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T04:14:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T04:14:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "[]", 2024-09-21T04:14:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.464 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.465 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.465 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.465 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.465 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.465 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.465 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.465 INFO:teuthology.orchestra.run.smithi064.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T04:14:03.465 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T04:14:03.465 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.465 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.465 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.465 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "{}", 2024-09-21T04:14:03.465 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.465 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.466 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.466 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.466 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.466 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.466 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.466 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.466 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T04:14:03.466 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T04:14:03.466 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.466 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.466 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.466 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.466 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.466 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T04:14:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T04:14:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.467 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T04:14:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T04:14:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.468 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T04:14:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T04:14:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.469 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.470 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.470 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.470 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.470 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.470 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T04:14:03.470 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T04:14:03.470 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.470 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.470 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.470 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.470 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.470 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.470 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.470 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T04:14:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T04:14:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.471 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T04:14:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T04:14:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.472 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.473 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.473 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.473 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.473 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T04:14:03.473 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T04:14:03.473 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.473 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.473 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.473 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.473 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.473 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.473 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.473 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.473 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.473 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.474 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.474 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.474 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T04:14:03.474 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T04:14:03.474 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.474 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.474 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.474 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.474 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.474 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.474 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.474 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.474 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.474 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T04:14:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T04:14:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.475 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T04:14:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T04:14:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T04:14:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.476 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T04:14:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T04:14:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-21T04:14:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.477 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T04:14:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T04:14:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8", 2024-09-21T04:14:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.478 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.479 INFO:teuthology.orchestra.run.smithi064.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T04:14:03.479 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T04:14:03.479 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.479 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.479 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.479 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "45", 2024-09-21T04:14:03.479 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.479 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.482 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.483 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.483 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.483 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.483 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.483 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.483 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T04:14:03.483 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T04:14:03.483 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.483 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.483 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.483 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.483 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T04:14:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T04:14:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-09-21T04:14:03.484 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T04:14:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T04:14:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.485 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.486 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.486 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.486 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.486 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.486 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.486 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.486 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.486 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T04:14:03.486 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T04:14:03.486 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.486 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.486 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.486 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.486 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.487 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.487 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.487 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.487 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.487 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.487 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.487 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.487 INFO:teuthology.orchestra.run.smithi064.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T04:14:03.487 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T04:14:03.487 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.487 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.487 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.487 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.487 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T04:14:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T04:14:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.488 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T04:14:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T04:14:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-21T04:14:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.489 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T04:14:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T04:14:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-21T04:14:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.490 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: "cross_origin_url": { 2024-09-21T04:14:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cross_origin_url", 2024-09-21T04:14:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.491 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: "crt_file": { 2024-09-21T04:14:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crt_file", 2024-09-21T04:14:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.492 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: "debug": { 2024-09-21T04:14:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "debug", 2024-09-21T04:14:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.493 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable/disable debug options", 2024-09-21T04:14:03.494 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.494 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.494 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.494 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.494 INFO:teuthology.orchestra.run.smithi064.stdout: "jwt_token_ttl": { 2024-09-21T04:14:03.494 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "jwt_token_ttl", 2024-09-21T04:14:03.494 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.494 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.494 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.494 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "28800", 2024-09-21T04:14:03.494 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.494 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.494 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-09-21T04:14:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-09-21T04:14:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.495 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.496 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.497 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.498 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.498 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.498 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.498 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.498 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.498 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.498 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.498 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.498 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.498 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.498 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.498 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.498 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.498 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.499 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.500 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.501 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.501 INFO:teuthology.orchestra.run.smithi064.stdout: "motd": { 2024-09-21T04:14:03.501 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "motd", 2024-09-21T04:14:03.501 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.501 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.501 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.501 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.501 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.501 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.501 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.501 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "The message of the day", 2024-09-21T04:14:03.501 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.501 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.501 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.502 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.502 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect_resolve_ip_addr": { 2024-09-21T04:14:03.502 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T04:14:03.502 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.502 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.502 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.502 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.502 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.502 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.502 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.502 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.502 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.502 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.502 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.503 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.503 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-09-21T04:14:03.503 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-09-21T04:14:03.503 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.503 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.503 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.503 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-09-21T04:14:03.503 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.503 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.503 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.503 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.503 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.503 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.503 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-09-21T04:14:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-09-21T04:14:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8080", 2024-09-21T04:14:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.504 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.505 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.506 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-09-21T04:14:03.506 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-09-21T04:14:03.506 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.506 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.506 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.506 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.506 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.506 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.506 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.506 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.506 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.506 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.506 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.506 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl_server_port": { 2024-09-21T04:14:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl_server_port", 2024-09-21T04:14:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8443", 2024-09-21T04:14:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.507 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.508 INFO:teuthology.orchestra.run.smithi064.stdout: "sso_oauth2": { 2024-09-21T04:14:03.508 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sso_oauth2", 2024-09-21T04:14:03.508 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.508 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.508 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.508 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.508 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.508 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.508 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.508 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.508 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.509 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.509 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.509 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.509 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-09-21T04:14:03.509 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-09-21T04:14:03.509 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.509 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.509 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.509 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "redirect", 2024-09-21T04:14:03.509 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.509 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.509 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.509 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.509 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect" 2024-09-21T04:14:03.509 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.510 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.510 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.510 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.510 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.510 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.510 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-09-21T04:14:03.510 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-09-21T04:14:03.510 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.510 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.510 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.510 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-09-21T04:14:03.510 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-09-21T04:14:03.510 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-09-21T04:14:03.510 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: "url_prefix": { 2024-09-21T04:14:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url_prefix", 2024-09-21T04:14:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.511 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.512 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.512 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.512 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.512 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.512 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.512 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.512 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.512 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.512 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "devicehealth", 2024-09-21T04:14:03.512 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.512 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.512 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.512 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_monitoring": { 2024-09-21T04:14:03.512 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_monitoring", 2024-09-21T04:14:03.512 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.513 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.513 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.513 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.513 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.513 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.513 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.513 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "monitor device health metrics", 2024-09-21T04:14:03.513 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.513 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.513 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.513 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.513 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.513 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.513 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.514 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.515 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.515 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.515 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.515 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.515 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.515 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.515 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.515 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.515 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.515 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.515 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.515 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.515 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.515 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.515 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.516 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.516 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.516 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.516 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.516 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.516 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.516 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.516 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.516 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.516 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.516 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.516 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.516 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.516 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.517 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.517 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.517 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.517 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.517 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.517 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.517 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.517 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.517 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.517 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.517 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.517 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.517 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.517 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.517 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: "mark_out_threshold": { 2024-09-21T04:14:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mark_out_threshold", 2024-09-21T04:14:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2419200", 2024-09-21T04:14:03.518 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T04:14:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_name": { 2024-09-21T04:14:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_name", 2024-09-21T04:14:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "device_health_metrics", 2024-09-21T04:14:03.519 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.520 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.520 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.520 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T04:14:03.520 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.520 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.520 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.520 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.520 INFO:teuthology.orchestra.run.smithi064.stdout: "retention_period": { 2024-09-21T04:14:03.520 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retention_period", 2024-09-21T04:14:03.520 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.520 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.520 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.520 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15552000", 2024-09-21T04:14:03.520 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.521 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.521 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.521 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain device health metrics", 2024-09-21T04:14:03.521 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.521 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.521 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.521 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.521 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_frequency": { 2024-09-21T04:14:03.521 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_frequency", 2024-09-21T04:14:03.521 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.521 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.521 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.521 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-09-21T04:14:03.522 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.522 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.522 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.522 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T04:14:03.522 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.522 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.522 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.522 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.522 INFO:teuthology.orchestra.run.smithi064.stdout: "self_heal": { 2024-09-21T04:14:03.522 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "self_heal", 2024-09-21T04:14:03.522 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.522 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.522 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.523 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.523 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.523 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.523 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.523 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T04:14:03.523 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.523 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.523 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.523 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.523 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-21T04:14:03.523 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-21T04:14:03.523 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.523 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.523 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.524 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-21T04:14:03.524 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.524 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.524 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.524 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T04:14:03.524 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.524 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.524 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.524 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.524 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.524 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.524 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.524 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.524 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.524 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.525 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.525 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.525 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.525 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.525 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.525 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.525 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.525 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.525 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_threshold": { 2024-09-21T04:14:03.525 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_threshold", 2024-09-21T04:14:03.525 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.525 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.525 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.525 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7257600", 2024-09-21T04:14:03.526 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.526 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.526 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.526 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T04:14:03.526 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.526 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.526 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.526 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.526 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.526 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.526 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.526 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "diskprediction_local", 2024-09-21T04:14:03.526 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.526 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.527 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.528 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.529 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.530 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: "predict_interval": { 2024-09-21T04:14:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predict_interval", 2024-09-21T04:14:03.531 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-09-21T04:14:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: "predictor_model": { 2024-09-21T04:14:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predictor_model", 2024-09-21T04:14:03.532 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "prophetstor", 2024-09-21T04:14:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-21T04:14:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-21T04:14:03.533 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.534 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.534 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.534 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-21T04:14:03.534 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.534 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.534 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.534 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.534 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.534 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.534 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.534 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.534 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.534 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.534 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.535 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "influx", 2024-09-21T04:14:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": false, 2024-09-21T04:14:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "influxdb python module not found", 2024-09-21T04:14:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: "batch_size": { 2024-09-21T04:14:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "batch_size", 2024-09-21T04:14:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5000", 2024-09-21T04:14:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T04:14:03.536 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: "database": { 2024-09-21T04:14:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "database", 2024-09-21T04:14:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "ceph", 2024-09-21T04:14:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T04:14:03.537 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.538 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.538 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.538 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.538 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": { 2024-09-21T04:14:03.538 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hostname", 2024-09-21T04:14:03.538 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.538 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.538 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.538 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.538 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.538 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.538 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.538 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server hostname", 2024-09-21T04:14:03.538 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-21T04:14:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-21T04:14:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "30", 2024-09-21T04:14:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "5", 2024-09-21T04:14:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T04:14:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.539 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.540 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.541 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.542 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.543 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.544 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: "password": { 2024-09-21T04:14:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "password", 2024-09-21T04:14:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "password of InfluxDB server user", 2024-09-21T04:14:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.545 INFO:teuthology.orchestra.run.smithi064.stdout: "port": { 2024-09-21T04:14:03.546 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "port", 2024-09-21T04:14:03.546 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.546 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.546 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.546 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8086", 2024-09-21T04:14:03.546 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.546 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.546 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.546 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server port", 2024-09-21T04:14:03.546 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.546 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.546 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.546 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.546 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.547 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-09-21T04:14:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-09-21T04:14:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "false", 2024-09-21T04:14:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T04:14:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "threads": { 2024-09-21T04:14:03.548 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threads", 2024-09-21T04:14:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-21T04:14:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-21T04:14:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "32", 2024-09-21T04:14:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T04:14:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "username": { 2024-09-21T04:14:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "username", 2024-09-21T04:14:03.549 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "username of InfluxDB server user", 2024-09-21T04:14:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: "verify_ssl": { 2024-09-21T04:14:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "verify_ssl", 2024-09-21T04:14:03.550 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "true", 2024-09-21T04:14:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T04:14:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.551 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "insights", 2024-09-21T04:14:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.552 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.553 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.554 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.555 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.556 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.557 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.558 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.558 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.558 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.558 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "iostat", 2024-09-21T04:14:03.558 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.558 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.558 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.558 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.558 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.558 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.558 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.558 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.558 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.559 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.560 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.561 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.562 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.563 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "k8sevents", 2024-09-21T04:14:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.564 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_event_retention_days": { 2024-09-21T04:14:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ceph_event_retention_days", 2024-09-21T04:14:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7", 2024-09-21T04:14:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T04:14:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.565 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.566 INFO:teuthology.orchestra.run.smithi064.stdout: "config_check_secs": { 2024-09-21T04:14:03.566 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_check_secs", 2024-09-21T04:14:03.566 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.566 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.566 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.566 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-21T04:14:03.566 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "10", 2024-09-21T04:14:03.566 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.566 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.566 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T04:14:03.566 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.566 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.566 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.566 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.567 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.568 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.569 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.570 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.570 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.570 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.570 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.570 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.570 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.570 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.570 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.570 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.570 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.570 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.570 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.570 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.570 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.570 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.571 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.572 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "localpool", 2024-09-21T04:14:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: "failure_domain": { 2024-09-21T04:14:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "failure_domain", 2024-09-21T04:14:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "host", 2024-09-21T04:14:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "failure domain for any created local pool", 2024-09-21T04:14:03.573 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T04:14:03.574 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.574 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.574 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.574 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.574 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.574 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.574 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.574 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.574 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.574 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.574 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.574 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.574 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.574 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.575 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.576 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.577 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.578 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: "min_size": { 2024-09-21T04:14:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_size", 2024-09-21T04:14:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default min_size for any created local pool", 2024-09-21T04:14:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T04:14:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.579 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: "num_rep": { 2024-09-21T04:14:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "num_rep", 2024-09-21T04:14:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3", 2024-09-21T04:14:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default replica count for any created local pool", 2024-09-21T04:14:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.580 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_num": { 2024-09-21T04:14:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_num", 2024-09-21T04:14:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "128", 2024-09-21T04:14:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T04:14:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.581 INFO:teuthology.orchestra.run.smithi064.stdout: "prefix": { 2024-09-21T04:14:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prefix", 2024-09-21T04:14:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name prefix for any created local pool", 2024-09-21T04:14:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.582 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.583 INFO:teuthology.orchestra.run.smithi064.stdout: "subtree": { 2024-09-21T04:14:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "subtree", 2024-09-21T04:14:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "rack", 2024-09-21T04:14:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T04:14:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T04:14:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.584 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mds_autoscaler", 2024-09-21T04:14:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.585 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.586 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.586 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.586 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.586 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.586 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.586 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.586 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.586 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.586 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.586 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.586 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.586 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.586 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.586 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.587 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.587 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.587 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.587 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.587 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.587 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.587 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.587 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.587 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.587 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.587 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.587 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.587 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.587 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.588 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.589 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.589 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.589 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.589 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.589 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.589 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.589 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.589 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.589 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.589 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.589 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.589 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.589 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.589 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.590 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirroring", 2024-09-21T04:14:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.591 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.592 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.593 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.594 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.595 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.596 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.596 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.596 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.596 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.596 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.596 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.596 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.596 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.596 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.596 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.596 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.596 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.596 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "nfs", 2024-09-21T04:14:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.597 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.598 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.599 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.600 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.601 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.602 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.603 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.603 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.603 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.603 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.603 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.603 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.603 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.603 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.603 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.603 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.603 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.603 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.603 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-09-21T04:14:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: "fail_fs": { 2024-09-21T04:14:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "fail_fs", 2024-09-21T04:14:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.604 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T04:14:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.605 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.606 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.606 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.606 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.606 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.606 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.606 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.606 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.606 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.606 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.606 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.606 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.606 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.606 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.606 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.607 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.608 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.609 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator": { 2024-09-21T04:14:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-09-21T04:14:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.610 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm", 2024-09-21T04:14:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: "rook", 2024-09-21T04:14:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: "test_orchestrator" 2024-09-21T04:14:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Orchestrator backend", 2024-09-21T04:14:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.611 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_perf_query", 2024-09-21T04:14:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.612 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.613 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.613 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.613 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.613 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.613 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.613 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.613 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.613 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.613 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.613 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.613 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.613 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.613 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.613 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.614 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.615 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.615 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.615 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.615 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.615 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.615 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.615 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.615 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.615 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.615 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.615 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.615 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.615 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.615 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.616 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.617 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.618 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_support", 2024-09-21T04:14:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.619 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.620 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.621 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.621 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.621 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.621 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.621 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.621 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.621 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.621 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.621 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.621 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.621 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.621 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.622 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.623 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.624 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_autoscaler", 2024-09-21T04:14:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.625 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.626 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.626 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.626 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.626 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.626 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.626 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.626 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.626 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.626 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.626 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.626 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.626 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.626 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.626 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.627 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.628 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.629 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.630 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.630 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.630 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.630 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.630 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.630 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.630 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.630 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.630 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.630 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.630 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.630 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.630 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-21T04:14:03.630 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-21T04:14:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-21T04:14:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.631 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: "threshold": { 2024-09-21T04:14:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threshold", 2024-09-21T04:14:03.632 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-21T04:14:03.633 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.633 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.633 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-09-21T04:14:03.633 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1.0", 2024-09-21T04:14:03.633 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.633 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.633 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "scaling threshold", 2024-09-21T04:14:03.633 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T04:14:03.633 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.633 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.633 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.633 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.633 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.633 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.634 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "progress", 2024-09-21T04:14:03.634 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.634 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.634 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.634 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_pg_recovery_event": { 2024-09-21T04:14:03.634 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_pg_recovery_event", 2024-09-21T04:14:03.634 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.634 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.634 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.634 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.634 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.634 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.634 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.634 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T04:14:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-09-21T04:14:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-09-21T04:14:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.635 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.636 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.637 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.638 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.638 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.638 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.638 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.638 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.638 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.638 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.638 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.638 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.638 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.638 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.638 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.638 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.638 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.639 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.640 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: "max_completed_events": { 2024-09-21T04:14:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_completed_events", 2024-09-21T04:14:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "50", 2024-09-21T04:14:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "number of past completed events to remember", 2024-09-21T04:14:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.641 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.642 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-21T04:14:03.642 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-21T04:14:03.642 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.642 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.642 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.642 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-21T04:14:03.642 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.642 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.642 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.642 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long the module is going to sleep", 2024-09-21T04:14:03.642 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.642 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.642 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.643 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus", 2024-09-21T04:14:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: "cache": { 2024-09-21T04:14:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cache", 2024-09-21T04:14:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.644 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "exclude_perf_counters": { 2024-09-21T04:14:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "exclude_perf_counters", 2024-09-21T04:14:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.645 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.646 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.646 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T04:14:03.646 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T04:14:03.646 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.646 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.646 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.646 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.646 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.646 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.646 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.646 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.646 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.646 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.646 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.647 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.648 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.649 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.650 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools": { 2024-09-21T04:14:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools", 2024-09-21T04:14:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.651 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T04:14:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T04:14:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "300", 2024-09-21T04:14:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.652 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_interval": { 2024-09-21T04:14:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_interval", 2024-09-21T04:14:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-21T04:14:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15.0", 2024-09-21T04:14:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.653 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-09-21T04:14:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-09-21T04:14:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-09-21T04:14:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T04:14:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.654 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.655 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.655 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-09-21T04:14:03.655 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-09-21T04:14:03.655 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.655 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.655 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.655 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "9283", 2024-09-21T04:14:03.655 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.655 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.655 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.655 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T04:14:03.655 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.655 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.655 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.656 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.656 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.656 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.656 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.656 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.656 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.656 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.656 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.656 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.656 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.656 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.656 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.656 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.656 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.656 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.657 INFO:teuthology.orchestra.run.smithi064.stdout: "stale_cache_strategy": { 2024-09-21T04:14:03.657 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stale_cache_strategy", 2024-09-21T04:14:03.657 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.657 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.657 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.657 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "log", 2024-09-21T04:14:03.657 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.657 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.657 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.657 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.657 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.657 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.657 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.657 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.658 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-09-21T04:14:03.658 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-09-21T04:14:03.658 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.658 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.658 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.658 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "default", 2024-09-21T04:14:03.658 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.658 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.658 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.658 INFO:teuthology.orchestra.run.smithi064.stdout: "default", 2024-09-21T04:14:03.658 INFO:teuthology.orchestra.run.smithi064.stdout: "error" 2024-09-21T04:14:03.658 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.658 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.658 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.658 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.659 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.659 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.659 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-09-21T04:14:03.659 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-09-21T04:14:03.659 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.659 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.659 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.659 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-09-21T04:14:03.659 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-09-21T04:14:03.659 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-09-21T04:14:03.659 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.659 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.659 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.659 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_support", 2024-09-21T04:14:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.660 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.661 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.662 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.662 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.662 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.662 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.662 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.662 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.662 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.662 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.662 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.662 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.662 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.662 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.662 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.662 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.663 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.664 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.665 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.665 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.665 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.665 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.665 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.665 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_snap_create": { 2024-09-21T04:14:03.665 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_snap_create", 2024-09-21T04:14:03.665 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.665 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.665 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.665 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-21T04:14:03.665 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.665 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.665 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.665 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.666 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.666 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.666 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.666 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.666 INFO:teuthology.orchestra.run.smithi064.stdout: "mirror_snapshot_schedule": { 2024-09-21T04:14:03.666 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T04:14:03.666 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.666 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.666 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.666 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.666 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.666 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.666 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.666 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.667 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.667 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.667 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.667 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.667 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.667 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.667 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.667 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.667 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.667 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.667 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.667 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.667 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.667 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: "trash_purge_schedule": { 2024-09-21T04:14:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "trash_purge_schedule", 2024-09-21T04:14:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.668 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "restful", 2024-09-21T04:14:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_auth": { 2024-09-21T04:14:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_auth", 2024-09-21T04:14:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.669 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-09-21T04:14:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-09-21T04:14:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.670 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.671 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.672 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.673 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.674 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.674 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.674 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.674 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.674 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.674 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.674 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.674 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.674 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.674 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.674 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.674 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.674 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.674 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.675 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: "max_requests": { 2024-09-21T04:14:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_requests", 2024-09-21T04:14:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-09-21T04:14:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.676 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T04:14:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-09-21T04:14:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-09-21T04:14:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.677 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-09-21T04:14:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-09-21T04:14:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.678 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.679 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rgw", 2024-09-21T04:14:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.680 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.681 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.682 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.683 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.685 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.686 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.686 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.686 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.686 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.686 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.686 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.686 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.686 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.686 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.686 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.686 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.686 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.686 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.686 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.687 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T04:14:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T04:14:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-21T04:14:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.688 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T04:14:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.689 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rook", 2024-09-21T04:14:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.690 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.691 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.692 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.693 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.694 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.695 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: "storage_class": { 2024-09-21T04:14:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "storage_class", 2024-09-21T04:14:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.696 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "local", 2024-09-21T04:14:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T04:14:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "selftest", 2024-09-21T04:14:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.697 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.698 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.699 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.700 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.701 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.702 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.702 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.702 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.702 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.702 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.702 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.702 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.702 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.702 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.702 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.702 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.702 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: "roption1": { 2024-09-21T04:14:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption1", 2024-09-21T04:14:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.703 INFO:teuthology.orchestra.run.smithi064.stdout: "roption2": { 2024-09-21T04:14:03.704 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption2", 2024-09-21T04:14:03.704 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.704 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.704 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.704 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "xyz", 2024-09-21T04:14:03.704 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.704 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.704 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.704 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.704 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.704 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.704 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.704 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.704 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption1": { 2024-09-21T04:14:03.705 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption1", 2024-09-21T04:14:03.705 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.705 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.705 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.705 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.705 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.705 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.705 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.705 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.705 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.705 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.705 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.705 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.705 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption2": { 2024-09-21T04:14:03.706 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption2", 2024-09-21T04:14:03.706 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.706 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.706 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.706 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.706 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.706 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.706 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.706 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.706 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.706 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.706 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.706 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.706 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption3": { 2024-09-21T04:14:03.706 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption3", 2024-09-21T04:14:03.707 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-21T04:14:03.707 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.707 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.707 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.707 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.707 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.707 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.707 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.707 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.707 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.707 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.707 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.707 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption4": { 2024-09-21T04:14:03.707 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption4", 2024-09-21T04:14:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption5": { 2024-09-21T04:14:03.708 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption5", 2024-09-21T04:14:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption6": { 2024-09-21T04:14:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption6", 2024-09-21T04:14:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.709 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.710 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.710 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.710 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.710 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.710 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.710 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.710 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.710 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.710 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.710 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.710 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption7": { 2024-09-21T04:14:03.710 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption7", 2024-09-21T04:14:03.710 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-21T04:14:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "42", 2024-09-21T04:14:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.711 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.712 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.712 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.712 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.712 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.712 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.712 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.712 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.712 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.712 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.712 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.712 INFO:teuthology.orchestra.run.smithi064.stdout: "testkey": { 2024-09-21T04:14:03.712 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testkey", 2024-09-21T04:14:03.712 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.712 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "testlkey": { 2024-09-21T04:14:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testlkey", 2024-09-21T04:14:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.713 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "testnewline": { 2024-09-21T04:14:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testnewline", 2024-09-21T04:14:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.714 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.715 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.715 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.715 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.715 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.715 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.715 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.715 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.715 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.715 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.715 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.715 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.715 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.715 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smb", 2024-09-21T04:14:03.715 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "internal_store_backend": { 2024-09-21T04:14:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "internal_store_backend", 2024-09-21T04:14:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T04:14:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.716 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.717 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.717 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.717 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.717 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.717 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.717 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.717 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.717 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.717 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.717 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.717 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.717 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.717 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.717 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.717 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.718 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.719 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.719 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.719 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.719 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.719 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.719 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.719 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.719 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.719 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.719 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.719 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.719 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.719 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.719 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.720 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.720 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.720 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.720 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.720 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.720 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.720 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.720 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.720 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.720 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.720 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.720 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.720 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.720 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.720 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.721 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.721 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.721 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.721 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.721 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.721 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.721 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.721 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.721 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.721 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.721 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.721 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.721 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.721 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.721 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.722 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.722 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.722 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.722 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.722 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.722 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.722 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.722 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.722 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.722 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.722 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.722 INFO:teuthology.orchestra.run.smithi064.stdout: "update_orchestration": { 2024-09-21T04:14:03.722 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "update_orchestration", 2024-09-21T04:14:03.722 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T04:14:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.723 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2024-09-21T04:14:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_m_granularity": { 2024-09-21T04:14:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_m_granularity", 2024-09-21T04:14:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T04:14:03.724 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.725 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.725 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.725 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.725 INFO:teuthology.orchestra.run.smithi064.stdout: "dump_on_update": { 2024-09-21T04:14:03.725 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dump_on_update", 2024-09-21T04:14:03.725 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.725 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.725 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.725 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.725 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.725 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.725 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.725 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "dump database to debug log on update", 2024-09-21T04:14:03.725 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.726 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.726 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.726 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.726 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.726 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.726 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.726 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.726 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.726 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.726 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.726 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.726 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.726 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.726 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.727 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.727 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.727 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.727 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.727 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.727 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.727 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.727 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.727 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.727 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.727 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.727 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.727 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.727 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.728 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.728 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.728 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.728 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.728 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.728 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.728 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.728 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.728 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.728 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.728 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.728 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.728 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.728 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.729 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.730 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.731 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stats", 2024-09-21T04:14:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.732 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.733 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.734 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.735 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.736 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.737 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "status", 2024-09-21T04:14:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.738 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.739 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.740 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.741 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.742 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.743 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.744 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telegraf", 2024-09-21T04:14:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "address": { 2024-09-21T04:14:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "address", 2024-09-21T04:14:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T04:14:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.745 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-21T04:14:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-21T04:14:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15", 2024-09-21T04:14:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.746 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.747 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.748 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.749 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.750 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.751 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.752 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telemetry", 2024-09-21T04:14:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_basic": { 2024-09-21T04:14:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_basic", 2024-09-21T04:14:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.753 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T04:14:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_crash": { 2024-09-21T04:14:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_crash", 2024-09-21T04:14:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.754 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T04:14:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_device": { 2024-09-21T04:14:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_device", 2024-09-21T04:14:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.755 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T04:14:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_ident": { 2024-09-21T04:14:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_ident", 2024-09-21T04:14:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.756 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T04:14:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_perf": { 2024-09-21T04:14:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_perf", 2024-09-21T04:14:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.757 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T04:14:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "contact": { 2024-09-21T04:14:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "contact", 2024-09-21T04:14:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.758 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "description": { 2024-09-21T04:14:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "description", 2024-09-21T04:14:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.759 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "device_url": { 2024-09-21T04:14:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_url", 2024-09-21T04:14:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T04:14:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.760 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-09-21T04:14:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-09-21T04:14:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.761 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-21T04:14:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-21T04:14:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "24", 2024-09-21T04:14:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "8", 2024-09-21T04:14:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.762 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "last_opt_revision": { 2024-09-21T04:14:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "last_opt_revision", 2024-09-21T04:14:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1", 2024-09-21T04:14:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.763 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard": { 2024-09-21T04:14:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard", 2024-09-21T04:14:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.764 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard_description": { 2024-09-21T04:14:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard_description", 2024-09-21T04:14:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.765 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.766 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.767 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.768 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.769 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.770 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "organization": { 2024-09-21T04:14:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "organization", 2024-09-21T04:14:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.771 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "proxy": { 2024-09-21T04:14:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "proxy", 2024-09-21T04:14:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.772 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.773 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "url": { 2024-09-21T04:14:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url", 2024-09-21T04:14:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T04:14:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.774 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "test_orchestrator", 2024-09-21T04:14:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.775 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.776 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.777 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.778 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.779 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.780 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volumes", 2024-09-21T04:14:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.781 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.782 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.783 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.784 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.785 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_clones": { 2024-09-21T04:14:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_clones", 2024-09-21T04:14:03.786 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4", 2024-09-21T04:14:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T04:14:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "periodic_async_work": { 2024-09-21T04:14:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "periodic_async_work", 2024-09-21T04:14:03.787 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Periodically check for async work", 2024-09-21T04:14:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_delay": { 2024-09-21T04:14:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_delay", 2024-09-21T04:14:03.788 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T04:14:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_no_wait": { 2024-09-21T04:14:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T04:14:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.789 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T04:14:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.790 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.791 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix", 2024-09-21T04:14:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "discovery_interval": { 2024-09-21T04:14:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "discovery_interval", 2024-09-21T04:14:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-21T04:14:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "100", 2024-09-21T04:14:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.792 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "identifier": { 2024-09-21T04:14:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "identifier", 2024-09-21T04:14:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.793 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-21T04:14:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-21T04:14:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-21T04:14:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.794 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.795 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.796 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.797 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.799 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.800 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_host": { 2024-09-21T04:14:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_host", 2024-09-21T04:14:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_port": { 2024-09-21T04:14:03.801 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_port", 2024-09-21T04:14:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10051", 2024-09-21T04:14:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_sender": { 2024-09-21T04:14:03.802 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_sender", 2024-09-21T04:14:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T04:14:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.803 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-21T04:14:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "modules": [ 2024-09-21T04:14:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm", 2024-09-21T04:14:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "dashboard", 2024-09-21T04:14:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "iostat", 2024-09-21T04:14:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "nfs", 2024-09-21T04:14:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "restful", 2024-09-21T04:14:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "snap_schedule" 2024-09-21T04:14:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "available_modules": [ 2024-09-21T04:14:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.804 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "alerts", 2024-09-21T04:14:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-21T04:14:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-21T04:14:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-21T04:14:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.805 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T04:14:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.806 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.807 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.808 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.809 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.810 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_destination": { 2024-09-21T04:14:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_destination", 2024-09-21T04:14:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T04:14:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.811 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_from_name": { 2024-09-21T04:14:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_from_name", 2024-09-21T04:14:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "Ceph", 2024-09-21T04:14:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Email From: name", 2024-09-21T04:14:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.812 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_host": { 2024-09-21T04:14:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_host", 2024-09-21T04:14:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP server", 2024-09-21T04:14:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.813 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_password": { 2024-09-21T04:14:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_password", 2024-09-21T04:14:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Password to authenticate with", 2024-09-21T04:14:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.814 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_port": { 2024-09-21T04:14:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_port", 2024-09-21T04:14:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "465", 2024-09-21T04:14:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP port", 2024-09-21T04:14:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.815 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_sender": { 2024-09-21T04:14:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_sender", 2024-09-21T04:14:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SMTP envelope sender", 2024-09-21T04:14:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.816 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_ssl": { 2024-09-21T04:14:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_ssl", 2024-09-21T04:14:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T04:14:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.817 INFO:teuthology.orchestra.run.smithi064.stdout: "smtp_user": { 2024-09-21T04:14:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smtp_user", 2024-09-21T04:14:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "User to authenticate as", 2024-09-21T04:14:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.818 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.819 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "balancer", 2024-09-21T04:14:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "active": { 2024-09-21T04:14:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "active", 2024-09-21T04:14:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.820 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T04:14:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_time": { 2024-09-21T04:14:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_time", 2024-09-21T04:14:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0000", 2024-09-21T04:14:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.821 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T04:14:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T04:14:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "begin_weekday": { 2024-09-21T04:14:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "begin_weekday", 2024-09-21T04:14:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-21T04:14:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-09-21T04:14:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-09-21T04:14:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.822 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T04:14:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T04:14:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_max_iterations": { 2024-09-21T04:14:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_max_iterations", 2024-09-21T04:14:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-21T04:14:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "25", 2024-09-21T04:14:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-21T04:14:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "250", 2024-09-21T04:14:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.823 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T04:14:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_metrics": { 2024-09-21T04:14:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_metrics", 2024-09-21T04:14:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T04:14:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.824 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T04:14:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T04:14:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "crush_compat_step": { 2024-09-21T04:14:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crush_compat_step", 2024-09-21T04:14:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-21T04:14:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.5", 2024-09-21T04:14:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0.001", 2024-09-21T04:14:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "0.999", 2024-09-21T04:14:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "aggressiveness of optimization", 2024-09-21T04:14:03.825 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T04:14:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "end_time": { 2024-09-21T04:14:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_time", 2024-09-21T04:14:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2359", 2024-09-21T04:14:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T04:14:03.826 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T04:14:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "end_weekday": { 2024-09-21T04:14:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "end_weekday", 2024-09-21T04:14:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-21T04:14:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "0", 2024-09-21T04:14:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "6", 2024-09-21T04:14:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T04:14:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T04:14:03.827 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.828 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.829 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.830 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.831 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "min_score": { 2024-09-21T04:14:03.832 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_score", 2024-09-21T04:14:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-21T04:14:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T04:14:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-09-21T04:14:03.833 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-09-21T04:14:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "upmap", 2024-09-21T04:14:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "crush-compat", 2024-09-21T04:14:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "none", 2024-09-21T04:14:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "read", 2024-09-21T04:14:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap", 2024-09-21T04:14:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap-read" 2024-09-21T04:14:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.834 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Balancer mode", 2024-09-21T04:14:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_ids": { 2024-09-21T04:14:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_ids", 2024-09-21T04:14:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.835 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T04:14:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-21T04:14:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-21T04:14:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-21T04:14:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T04:14:03.836 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.837 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_deviation": { 2024-09-21T04:14:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_deviation", 2024-09-21T04:14:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-21T04:14:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-21T04:14:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T04:14:03.838 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T04:14:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "upmap_max_optimizations": { 2024-09-21T04:14:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "upmap_max_optimizations", 2024-09-21T04:14:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-21T04:14:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-21T04:14:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T04:14:03.839 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm", 2024-09-21T04:14:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_down_multiplier": { 2024-09-21T04:14:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_down_multiplier", 2024-09-21T04:14:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-21T04:14:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.840 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-09-21T04:14:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T04:14:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_refresh_rate": { 2024-09-21T04:14:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_refresh_rate", 2024-09-21T04:14:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.841 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "20", 2024-09-21T04:14:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T04:14:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "agent_starting_port": { 2024-09-21T04:14:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "agent_starting_port", 2024-09-21T04:14:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.842 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4721", 2024-09-21T04:14:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T04:14:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_ptrace": { 2024-09-21T04:14:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_ptrace", 2024-09-21T04:14:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.843 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T04:14:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T04:14:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_interval": { 2024-09-21T04:14:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_interval", 2024-09-21T04:14:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-21T04:14:03.844 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T04:14:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "autotune_memory_target_ratio": { 2024-09-21T04:14:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T04:14:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-21T04:14:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0.7", 2024-09-21T04:14:03.845 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T04:14:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm_log_destination": { 2024-09-21T04:14:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cephadm_log_destination", 2024-09-21T04:14:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.846 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "file", 2024-09-21T04:14:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "file,syslog", 2024-09-21T04:14:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "syslog" 2024-09-21T04:14:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T04:14:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "cgroups_split": { 2024-09-21T04:14:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cgroups_split", 2024-09-21T04:14:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.847 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T04:14:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "config_checks_enabled": { 2024-09-21T04:14:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_checks_enabled", 2024-09-21T04:14:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.848 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T04:14:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "config_dashboard": { 2024-09-21T04:14:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_dashboard", 2024-09-21T04:14:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.849 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T04:14:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_alertmanager": { 2024-09-21T04:14:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_alertmanager", 2024-09-21T04:14:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.850 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T04:14:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-21T04:14:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_base": { 2024-09-21T04:14:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_base", 2024-09-21T04:14:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T04:14:03.851 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Container image name, without the tag", 2024-09-21T04:14:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_elasticsearch": { 2024-09-21T04:14:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_elasticsearch", 2024-09-21T04:14:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T04:14:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.852 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "elasticsearch container image", 2024-09-21T04:14:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_grafana": { 2024-09-21T04:14:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_grafana", 2024-09-21T04:14:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T04:14:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.853 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-21T04:14:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_haproxy": { 2024-09-21T04:14:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_haproxy", 2024-09-21T04:14:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T04:14:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.854 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "HAproxy container image", 2024-09-21T04:14:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_agent": { 2024-09-21T04:14:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_agent", 2024-09-21T04:14:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T04:14:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.855 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger agent container image", 2024-09-21T04:14:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_collector": { 2024-09-21T04:14:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_collector", 2024-09-21T04:14:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T04:14:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger collector container image", 2024-09-21T04:14:03.856 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_jaeger_query": { 2024-09-21T04:14:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_jaeger_query", 2024-09-21T04:14:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T04:14:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Jaeger query container image", 2024-09-21T04:14:03.857 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_keepalived": { 2024-09-21T04:14:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_keepalived", 2024-09-21T04:14:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T04:14:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Keepalived container image", 2024-09-21T04:14:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.858 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_loki": { 2024-09-21T04:14:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_loki", 2024-09-21T04:14:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T04:14:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Loki container image", 2024-09-21T04:14:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.859 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_nginx": { 2024-09-21T04:14:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_nginx", 2024-09-21T04:14:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T04:14:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Nginx container image", 2024-09-21T04:14:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.860 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_node_exporter": { 2024-09-21T04:14:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_node_exporter", 2024-09-21T04:14:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T04:14:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-21T04:14:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.861 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_nvmeof": { 2024-09-21T04:14:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_nvmeof", 2024-09-21T04:14:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T04:14:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Nvme-of container image", 2024-09-21T04:14:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_oauth2_proxy": { 2024-09-21T04:14:03.862 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T04:14:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T04:14:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "oauth2-proxy container image", 2024-09-21T04:14:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.863 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_prometheus": { 2024-09-21T04:14:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_prometheus", 2024-09-21T04:14:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T04:14:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Prometheus container image", 2024-09-21T04:14:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.864 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_promtail": { 2024-09-21T04:14:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_promtail", 2024-09-21T04:14:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T04:14:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Promtail container image", 2024-09-21T04:14:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.865 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_samba": { 2024-09-21T04:14:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_samba", 2024-09-21T04:14:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T04:14:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Samba/SMB container image", 2024-09-21T04:14:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.866 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_samba_metrics": { 2024-09-21T04:14:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_samba_metrics", 2024-09-21T04:14:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T04:14:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T04:14:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.867 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "container_image_snmp_gateway": { 2024-09-21T04:14:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_image_snmp_gateway", 2024-09-21T04:14:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T04:14:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "SNMP Gateway container image", 2024-09-21T04:14:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.868 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "container_init": { 2024-09-21T04:14:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "container_init", 2024-09-21T04:14:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T04:14:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.869 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "daemon_cache_timeout": { 2024-09-21T04:14:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "daemon_cache_timeout", 2024-09-21T04:14:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-21T04:14:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T04:14:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.870 INFO:teuthology.orchestra.run.smithi064.stdout: "default_cephadm_command_timeout": { 2024-09-21T04:14:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T04:14:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "900", 2024-09-21T04:14:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T04:14:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.871 INFO:teuthology.orchestra.run.smithi064.stdout: "default_registry": { 2024-09-21T04:14:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "default_registry", 2024-09-21T04:14:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "docker.io", 2024-09-21T04:14:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T04:14:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.872 INFO:teuthology.orchestra.run.smithi064.stdout: "device_cache_timeout": { 2024-09-21T04:14:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_cache_timeout", 2024-09-21T04:14:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1800", 2024-09-21T04:14:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache device inventory", 2024-09-21T04:14:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "device_enhanced_scan": { 2024-09-21T04:14:03.873 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_enhanced_scan", 2024-09-21T04:14:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T04:14:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "facts_cache_timeout": { 2024-09-21T04:14:03.874 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "facts_cache_timeout", 2024-09-21T04:14:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-21T04:14:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "seconds to cache host facts data", 2024-09-21T04:14:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "grafana_dashboards_path": { 2024-09-21T04:14:03.875 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "grafana_dashboards_path", 2024-09-21T04:14:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T04:14:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T04:14:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "host_check_interval": { 2024-09-21T04:14:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "host_check_interval", 2024-09-21T04:14:03.876 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-21T04:14:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to perform a host check", 2024-09-21T04:14:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "hw_monitoring": { 2024-09-21T04:14:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hw_monitoring", 2024-09-21T04:14:03.877 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T04:14:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "inventory_list_all": { 2024-09-21T04:14:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "inventory_list_all", 2024-09-21T04:14:03.878 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T04:14:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.879 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.880 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "log_refresh_metadata": { 2024-09-21T04:14:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_refresh_metadata", 2024-09-21T04:14:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T04:14:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.881 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T04:14:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.882 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.883 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.884 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T04:14:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T04:14:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.885 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T04:14:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T04:14:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T04:14:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "*", 2024-09-21T04:14:03.886 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T04:14:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "max_count_per_host": { 2024-09-21T04:14:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_count_per_host", 2024-09-21T04:14:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-21T04:14:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.887 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of daemons per service per host", 2024-09-21T04:14:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.888 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: "max_osd_draining_count": { 2024-09-21T04:14:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_osd_draining_count", 2024-09-21T04:14:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.889 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-21T04:14:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T04:14:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "migration_current": { 2024-09-21T04:14:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "migration_current", 2024-09-21T04:14:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.890 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "internal - do not modify", 2024-09-21T04:14:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "mode": { 2024-09-21T04:14:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mode", 2024-09-21T04:14:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.891 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "root", 2024-09-21T04:14:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm-package", 2024-09-21T04:14:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "root" 2024-09-21T04:14:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T04:14:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "oob_default_addr": { 2024-09-21T04:14:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "oob_default_addr", 2024-09-21T04:14:03.892 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "169.254.1.1", 2024-09-21T04:14:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T04:14:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "prometheus_alerts_path": { 2024-09-21T04:14:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus_alerts_path", 2024-09-21T04:14:03.893 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T04:14:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T04:14:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_insecure": { 2024-09-21T04:14:03.894 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_insecure", 2024-09-21T04:14:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T04:14:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_password": { 2024-09-21T04:14:03.895 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_password", 2024-09-21T04:14:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T04:14:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_url": { 2024-09-21T04:14:03.896 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_url", 2024-09-21T04:14:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T04:14:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "registry_username": { 2024-09-21T04:14:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "registry_username", 2024-09-21T04:14:03.897 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T04:14:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "secure_monitoring_stack": { 2024-09-21T04:14:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "secure_monitoring_stack", 2024-09-21T04:14:03.898 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T04:14:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "service_discovery_port": { 2024-09-21T04:14:03.899 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "service_discovery_port", 2024-09-21T04:14:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8765", 2024-09-21T04:14:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "cephadm service discovery port", 2024-09-21T04:14:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.900 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "ssh_config_file": { 2024-09-21T04:14:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssh_config_file", 2024-09-21T04:14:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.901 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T04:14:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "ssh_keepalive_count_max": { 2024-09-21T04:14:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T04:14:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.902 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3", 2024-09-21T04:14:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T04:14:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "ssh_keepalive_interval": { 2024-09-21T04:14:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssh_keepalive_interval", 2024-09-21T04:14:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.903 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7", 2024-09-21T04:14:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T04:14:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "use_agent": { 2024-09-21T04:14:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_agent", 2024-09-21T04:14:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.904 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T04:14:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "use_repo_digest": { 2024-09-21T04:14:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "use_repo_digest", 2024-09-21T04:14:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.905 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T04:14:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_failed_host_check": { 2024-09-21T04:14:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_failed_host_check", 2024-09-21T04:14:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.906 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.907 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.907 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T04:14:03.907 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.907 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.907 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.907 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.907 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_daemons": { 2024-09-21T04:14:03.907 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_daemons", 2024-09-21T04:14:03.907 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.907 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.907 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.907 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.907 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.907 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T04:14:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_on_stray_hosts": { 2024-09-21T04:14:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_on_stray_hosts", 2024-09-21T04:14:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.908 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T04:14:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crash", 2024-09-21T04:14:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.909 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.910 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.910 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.910 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.910 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.910 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.910 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.910 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.910 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.910 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.910 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.910 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.910 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.910 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.910 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.911 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.912 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.912 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.912 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.912 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.912 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.912 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.912 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.912 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.912 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.912 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.912 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.912 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.912 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.912 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.913 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.914 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.914 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.914 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.914 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.914 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.914 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.914 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.914 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.914 INFO:teuthology.orchestra.run.smithi064.stdout: "retain_interval": { 2024-09-21T04:14:03.914 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retain_interval", 2024-09-21T04:14:03.914 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.914 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.914 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.914 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "31536000", 2024-09-21T04:14:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T04:14:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.915 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_recent_interval": { 2024-09-21T04:14:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_recent_interval", 2024-09-21T04:14:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1209600", 2024-09-21T04:14:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.916 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.917 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.917 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T04:14:03.917 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.917 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.917 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.917 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.917 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.917 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.917 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.917 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dashboard", 2024-09-21T04:14:03.917 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.917 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.917 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.917 INFO:teuthology.orchestra.run.smithi064.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T04:14:03.917 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T04:14:03.918 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.918 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.918 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.918 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-21T04:14:03.918 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.918 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.918 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.918 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.918 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.918 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.918 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.918 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.918 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T04:14:03.918 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T04:14:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T04:14:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T04:14:03.919 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_ENABLED": { 2024-09-21T04:14:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T04:14:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.920 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T04:14:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T04:14:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.921 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T04:14:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T04:14:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.922 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T04:14:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T04:14:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.923 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T04:14:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T04:14:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.924 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T04:14:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T04:14:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.925 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T04:14:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T04:14:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.926 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T04:14:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T04:14:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.927 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.928 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.928 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.928 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.928 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.928 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.928 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.928 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.928 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.928 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T04:14:03.928 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T04:14:03.928 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.928 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.928 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.928 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.928 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.929 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.929 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.929 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.929 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.929 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.929 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.929 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.929 INFO:teuthology.orchestra.run.smithi064.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T04:14:03.929 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T04:14:03.929 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.929 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.929 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.929 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.929 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.929 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T04:14:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T04:14:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.930 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.931 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.931 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.931 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.931 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.931 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.931 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.931 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T04:14:03.931 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T04:14:03.931 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.931 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.931 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.931 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-09-21T04:14:03.931 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.931 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.931 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.932 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.932 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.932 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.932 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.932 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.932 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T04:14:03.932 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T04:14:03.932 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.932 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.932 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.932 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.932 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.932 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.932 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.933 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.933 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.933 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.933 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.933 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.933 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_URL": { 2024-09-21T04:14:03.933 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_URL", 2024-09-21T04:14:03.933 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.933 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.933 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.933 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.933 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.933 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.933 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.933 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.934 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.934 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.934 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.934 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.934 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T04:14:03.934 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T04:14:03.934 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.934 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.934 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.934 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-09-21T04:14:03.934 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.934 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.934 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.934 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.934 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.935 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.935 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.935 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.935 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T04:14:03.935 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T04:14:03.935 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.935 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.935 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.935 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.935 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.935 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.935 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.935 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.935 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.935 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.936 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.936 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.936 INFO:teuthology.orchestra.run.smithi064.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T04:14:03.936 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T04:14:03.936 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.936 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.936 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.936 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.936 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.936 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.936 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.936 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.936 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.936 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.936 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.937 INFO:teuthology.orchestra.run.smithi064.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T04:14:03.937 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T04:14:03.937 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.937 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.937 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.937 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.937 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.937 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.937 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.937 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.937 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.937 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.937 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.937 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.938 INFO:teuthology.orchestra.run.smithi064.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T04:14:03.938 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T04:14:03.938 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.938 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.938 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.938 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.938 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.938 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.938 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.938 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.938 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.938 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.938 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.938 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.939 INFO:teuthology.orchestra.run.smithi064.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T04:14:03.939 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T04:14:03.939 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.939 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.939 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.939 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "[]", 2024-09-21T04:14:03.939 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.939 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.939 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.939 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.939 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.939 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.939 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.939 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.939 INFO:teuthology.orchestra.run.smithi064.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T04:14:03.940 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T04:14:03.940 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.940 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.940 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.940 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "{}", 2024-09-21T04:14:03.940 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.940 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.940 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.940 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.940 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.940 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.940 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.940 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.940 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T04:14:03.940 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T04:14:03.941 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.941 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.941 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.941 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.941 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.941 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.941 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.941 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.941 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.941 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.941 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.941 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.941 INFO:teuthology.orchestra.run.smithi064.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T04:14:03.941 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T04:14:03.942 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.942 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.942 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.942 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.942 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.942 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.942 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.942 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.942 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.942 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.942 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.942 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.942 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T04:14:03.942 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T04:14:03.943 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.943 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.943 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.943 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.943 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.943 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.943 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.943 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.943 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.943 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.943 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.943 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.943 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T04:14:03.943 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T04:14:03.944 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.944 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.944 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.944 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.944 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.944 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.944 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.944 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.944 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.944 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.944 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.944 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.944 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T04:14:03.944 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T04:14:03.944 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.945 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.945 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.945 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.945 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.945 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.945 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.945 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.945 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.945 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.945 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.945 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.945 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T04:14:03.945 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T04:14:03.945 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.945 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.946 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.946 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.946 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.946 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.946 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.946 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.946 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.946 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.946 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.946 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.946 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T04:14:03.946 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T04:14:03.946 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.946 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.947 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.947 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.947 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.947 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.947 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.947 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.947 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.947 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.947 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.947 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.947 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T04:14:03.947 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T04:14:03.947 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.947 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.948 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.948 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.948 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.948 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.948 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.948 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.948 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.948 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.948 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.948 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.948 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T04:14:03.948 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T04:14:03.948 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.948 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.949 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.949 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.949 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.949 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.949 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.949 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.949 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.949 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.949 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.949 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.949 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T04:14:03.949 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T04:14:03.949 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.949 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.950 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.950 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.950 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.950 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.950 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.950 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.950 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.950 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.950 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.950 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.950 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T04:14:03.950 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T04:14:03.950 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.950 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.950 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.951 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T04:14:03.951 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.951 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.951 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.951 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.951 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.951 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.951 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.951 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.951 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T04:14:03.951 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T04:14:03.951 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.951 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.951 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.951 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-21T04:14:03.951 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.952 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.952 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.952 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.952 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.952 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.952 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.952 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.952 INFO:teuthology.orchestra.run.smithi064.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T04:14:03.952 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T04:14:03.952 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.952 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.952 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.952 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8", 2024-09-21T04:14:03.952 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.952 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.953 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.953 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.953 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.953 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.953 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.953 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.953 INFO:teuthology.orchestra.run.smithi064.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T04:14:03.953 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T04:14:03.953 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.953 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.953 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.953 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "45", 2024-09-21T04:14:03.953 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.953 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.954 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.954 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.954 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.954 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.954 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.954 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.954 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T04:14:03.954 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T04:14:03.954 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.954 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.954 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.954 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.954 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.954 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.954 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.955 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.955 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.955 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.955 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.955 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.955 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T04:14:03.955 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T04:14:03.955 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.955 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.955 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.955 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "admin", 2024-09-21T04:14:03.955 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.955 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.955 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.956 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.956 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.956 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.956 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.956 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.956 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T04:14:03.956 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T04:14:03.956 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.956 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.956 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.956 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.956 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.956 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.956 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.957 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.957 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.957 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.957 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.957 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.957 INFO:teuthology.orchestra.run.smithi064.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T04:14:03.957 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T04:14:03.957 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.957 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.957 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.957 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.957 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.957 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.957 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.957 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.958 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.958 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.958 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.958 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.958 INFO:teuthology.orchestra.run.smithi064.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T04:14:03.958 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T04:14:03.958 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.958 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.958 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.958 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.958 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.958 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.958 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.958 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.959 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.959 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.959 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.959 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.959 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T04:14:03.959 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T04:14:03.959 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.959 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.959 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.959 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.959 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.959 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.959 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.959 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.959 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.960 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.960 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.960 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.960 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T04:14:03.960 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T04:14:03.960 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.960 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.960 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.960 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-21T04:14:03.960 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.960 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.960 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.960 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.960 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.960 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.961 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.961 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.961 INFO:teuthology.orchestra.run.smithi064.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T04:14:03.961 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T04:14:03.961 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.961 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.961 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.961 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-21T04:14:03.961 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.961 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.961 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.961 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.961 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.961 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.962 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.962 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.962 INFO:teuthology.orchestra.run.smithi064.stdout: "cross_origin_url": { 2024-09-21T04:14:03.962 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cross_origin_url", 2024-09-21T04:14:03.962 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.962 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.962 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.962 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.962 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.962 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.962 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.962 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.962 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.962 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.962 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.963 INFO:teuthology.orchestra.run.smithi064.stdout: "crt_file": { 2024-09-21T04:14:03.963 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "crt_file", 2024-09-21T04:14:03.963 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.963 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.963 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.963 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.963 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.963 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.963 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.963 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.963 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.963 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.963 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.963 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.964 INFO:teuthology.orchestra.run.smithi064.stdout: "debug": { 2024-09-21T04:14:03.964 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "debug", 2024-09-21T04:14:03.964 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.964 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.964 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.964 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.964 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.964 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.964 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.964 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Enable/disable debug options", 2024-09-21T04:14:03.964 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.964 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.964 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.964 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.964 INFO:teuthology.orchestra.run.smithi064.stdout: "jwt_token_ttl": { 2024-09-21T04:14:03.965 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "jwt_token_ttl", 2024-09-21T04:14:03.965 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.965 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.965 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.965 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "28800", 2024-09-21T04:14:03.965 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.965 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.965 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.965 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.965 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.965 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.965 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.965 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.965 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-09-21T04:14:03.965 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-09-21T04:14:03.966 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.966 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.966 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.966 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.966 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.966 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.966 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.966 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.966 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.966 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.966 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.966 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.966 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.966 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.966 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.967 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.967 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.967 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.967 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.967 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.967 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.967 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.967 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.967 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.967 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.967 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.967 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.967 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.967 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.967 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.968 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.968 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.968 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.968 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.968 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.968 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.968 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.968 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.968 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.968 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.968 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.968 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.968 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.968 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.969 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.969 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.969 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.969 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.969 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.969 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.969 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.969 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.969 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.969 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.969 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.969 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.969 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.969 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.970 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.970 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.970 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.970 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.970 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.970 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.970 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.970 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.970 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.970 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.970 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.970 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.970 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.970 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.971 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.971 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.971 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.971 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.971 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.971 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.971 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.971 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.971 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.971 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.971 INFO:teuthology.orchestra.run.smithi064.stdout: "motd": { 2024-09-21T04:14:03.971 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "motd", 2024-09-21T04:14:03.971 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.971 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.972 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.972 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.972 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.972 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.972 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.972 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "The message of the day", 2024-09-21T04:14:03.972 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.972 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.972 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.972 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.972 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect_resolve_ip_addr": { 2024-09-21T04:14:03.972 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T04:14:03.972 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.972 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.973 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.973 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.973 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.973 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.973 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.973 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.973 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.973 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.973 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.973 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.973 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-09-21T04:14:03.973 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-09-21T04:14:03.973 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.973 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.974 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.974 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-09-21T04:14:03.974 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.974 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.974 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.974 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.974 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.974 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.974 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.974 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.974 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-09-21T04:14:03.974 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-09-21T04:14:03.974 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.974 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.974 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.975 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8080", 2024-09-21T04:14:03.975 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.975 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.975 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.975 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.975 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.975 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.975 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.975 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.975 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.975 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.975 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.975 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.975 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.975 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.976 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.976 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.976 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.976 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.976 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.976 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.976 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.976 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.976 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-09-21T04:14:03.976 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-09-21T04:14:03.976 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.976 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.976 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.976 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.977 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.977 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.977 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.977 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.977 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.977 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.977 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.977 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.977 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl_server_port": { 2024-09-21T04:14:03.977 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl_server_port", 2024-09-21T04:14:03.977 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.977 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.977 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.977 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8443", 2024-09-21T04:14:03.978 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.978 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.978 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.978 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.978 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.978 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.978 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.978 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.978 INFO:teuthology.orchestra.run.smithi064.stdout: "sso_oauth2": { 2024-09-21T04:14:03.978 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sso_oauth2", 2024-09-21T04:14:03.978 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.978 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.978 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.978 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.978 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.979 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.979 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.979 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.979 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.979 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.979 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.979 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.979 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-09-21T04:14:03.979 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-09-21T04:14:03.979 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.979 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.979 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.979 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "redirect", 2024-09-21T04:14:03.979 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.980 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.980 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.980 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.980 INFO:teuthology.orchestra.run.smithi064.stdout: "redirect" 2024-09-21T04:14:03.980 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.980 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.980 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.980 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.980 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.980 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.980 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-09-21T04:14:03.980 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-09-21T04:14:03.980 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.980 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.981 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.981 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-09-21T04:14:03.981 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-09-21T04:14:03.981 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-09-21T04:14:03.981 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.981 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.981 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.981 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.981 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.981 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.981 INFO:teuthology.orchestra.run.smithi064.stdout: "url_prefix": { 2024-09-21T04:14:03.981 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url_prefix", 2024-09-21T04:14:03.981 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.981 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.981 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:03.982 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.982 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.982 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.982 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.982 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.982 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.982 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.982 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.982 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.982 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.982 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.982 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.982 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "devicehealth", 2024-09-21T04:14:03.982 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.983 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.983 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.983 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_monitoring": { 2024-09-21T04:14:03.983 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_monitoring", 2024-09-21T04:14:03.983 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.983 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.983 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.983 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.983 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.983 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.983 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.983 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "monitor device health metrics", 2024-09-21T04:14:03.983 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.983 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.984 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.984 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.984 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.984 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.984 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.984 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.984 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.984 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.984 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.984 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.984 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.984 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.984 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.984 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.985 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.985 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.985 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.985 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.985 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.985 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.985 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.985 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.985 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.985 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.985 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.985 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.985 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.985 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.986 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.986 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.986 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.986 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.986 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.986 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.986 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.986 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.986 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.986 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.986 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.986 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.986 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.987 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.987 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:03.987 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.987 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.987 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.987 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.987 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.987 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.987 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.987 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.987 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.987 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.987 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.987 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.987 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.988 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.988 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.988 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:03.988 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:03.988 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.988 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.988 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.988 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.988 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.988 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.988 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.988 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.988 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.988 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.988 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.989 INFO:teuthology.orchestra.run.smithi064.stdout: "mark_out_threshold": { 2024-09-21T04:14:03.989 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mark_out_threshold", 2024-09-21T04:14:03.989 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.989 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.989 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.989 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "2419200", 2024-09-21T04:14:03.989 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.989 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.989 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.989 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T04:14:03.989 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.989 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.989 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.989 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.990 INFO:teuthology.orchestra.run.smithi064.stdout: "pool_name": { 2024-09-21T04:14:03.990 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pool_name", 2024-09-21T04:14:03.990 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.990 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.990 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.990 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "device_health_metrics", 2024-09-21T04:14:03.990 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.990 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.990 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.990 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T04:14:03.990 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.990 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.990 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.990 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.991 INFO:teuthology.orchestra.run.smithi064.stdout: "retention_period": { 2024-09-21T04:14:03.991 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "retention_period", 2024-09-21T04:14:03.991 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.991 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.991 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.991 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15552000", 2024-09-21T04:14:03.991 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.991 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.991 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.991 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long to retain device health metrics", 2024-09-21T04:14:03.991 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.991 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.991 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.991 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.991 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_frequency": { 2024-09-21T04:14:03.992 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_frequency", 2024-09-21T04:14:03.992 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.992 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.992 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.992 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-09-21T04:14:03.992 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.992 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.992 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.992 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T04:14:03.992 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.992 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.992 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.992 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.992 INFO:teuthology.orchestra.run.smithi064.stdout: "self_heal": { 2024-09-21T04:14:03.993 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "self_heal", 2024-09-21T04:14:03.993 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.993 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.993 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.993 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:03.993 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.993 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.993 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.993 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T04:14:03.993 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.993 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.993 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.993 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.993 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-21T04:14:03.994 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-21T04:14:03.994 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.994 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.994 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.994 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-21T04:14:03.994 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.994 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.994 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.994 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T04:14:03.994 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.994 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.994 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.994 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.994 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:03.995 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:03.995 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:03.995 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:03.995 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.995 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:03.995 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.995 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.995 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.995 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.995 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.995 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.995 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.995 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.995 INFO:teuthology.orchestra.run.smithi064.stdout: "warn_threshold": { 2024-09-21T04:14:03.995 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "warn_threshold", 2024-09-21T04:14:03.996 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:03.996 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.996 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.996 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7257600", 2024-09-21T04:14:03.996 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.996 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.996 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.996 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T04:14:03.996 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.996 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.996 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.996 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.996 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:03.996 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.996 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:03.997 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "diskprediction_local", 2024-09-21T04:14:03.997 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:03.997 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:03.997 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:03.997 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:03.997 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:03.997 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.997 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.997 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.997 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:03.997 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.997 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.997 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:03.997 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:03.997 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:03.998 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:03.998 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:03.998 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:03.998 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:03.998 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:03.998 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.998 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.998 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.998 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.998 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.998 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:03.998 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:03.998 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:03.998 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:03.999 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:03.999 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:03.999 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:03.999 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:03.999 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:03.999 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:03.999 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:03.999 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:03.999 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:03.999 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:03.999 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:03.999 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:03.999 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:03.999 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.000 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.000 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.000 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.000 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.000 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.000 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.000 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.000 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.000 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.000 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.000 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.000 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.000 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.000 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.000 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.001 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.001 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.001 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.001 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.001 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.001 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.001 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.001 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.001 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.001 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.001 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.001 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.001 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.001 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.001 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.002 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.002 INFO:teuthology.orchestra.run.smithi064.stdout: "predict_interval": { 2024-09-21T04:14:04.002 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predict_interval", 2024-09-21T04:14:04.002 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.002 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.002 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.002 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "86400", 2024-09-21T04:14:04.002 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.002 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.002 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.002 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.002 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.002 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.002 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.003 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.003 INFO:teuthology.orchestra.run.smithi064.stdout: "predictor_model": { 2024-09-21T04:14:04.003 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "predictor_model", 2024-09-21T04:14:04.003 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.003 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.003 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.003 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "prophetstor", 2024-09-21T04:14:04.003 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.003 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.003 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.003 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.003 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.003 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.003 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.004 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-21T04:14:04.004 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-21T04:14:04.004 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.004 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.004 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.004 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "600", 2024-09-21T04:14:04.004 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.004 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.004 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.004 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.004 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.004 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.004 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.004 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.005 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.005 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.005 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.005 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.005 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.005 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.005 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.005 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.005 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.005 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.005 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.005 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.005 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.005 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.005 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.006 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.006 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.006 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "influx", 2024-09-21T04:14:04.006 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": false, 2024-09-21T04:14:04.006 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "influxdb python module not found", 2024-09-21T04:14:04.006 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.006 INFO:teuthology.orchestra.run.smithi064.stdout: "batch_size": { 2024-09-21T04:14:04.006 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "batch_size", 2024-09-21T04:14:04.006 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.006 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.006 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.006 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5000", 2024-09-21T04:14:04.006 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.006 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.007 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.007 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T04:14:04.007 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.007 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.007 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.007 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.007 INFO:teuthology.orchestra.run.smithi064.stdout: "database": { 2024-09-21T04:14:04.007 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "database", 2024-09-21T04:14:04.007 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.007 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.007 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.007 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "ceph", 2024-09-21T04:14:04.007 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.007 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.008 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.008 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T04:14:04.008 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.008 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.008 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.008 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.008 INFO:teuthology.orchestra.run.smithi064.stdout: "hostname": { 2024-09-21T04:14:04.008 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "hostname", 2024-09-21T04:14:04.008 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.008 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.008 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.008 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.008 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.008 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.008 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.009 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server hostname", 2024-09-21T04:14:04.009 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.009 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.009 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.009 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.009 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-21T04:14:04.009 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-21T04:14:04.009 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:04.009 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.009 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.009 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "30", 2024-09-21T04:14:04.009 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "5", 2024-09-21T04:14:04.009 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.009 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.009 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T04:14:04.010 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.010 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.010 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.010 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.010 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.010 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.010 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.010 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.010 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.010 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.010 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.010 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.010 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.010 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.010 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.011 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.011 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.011 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.011 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.011 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.011 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.011 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.011 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.011 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.011 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.011 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.011 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.011 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.011 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.011 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.012 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.012 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.012 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.012 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.012 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.012 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.012 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.012 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.012 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.012 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.012 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.012 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.012 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.012 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.013 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.013 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.013 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.013 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.013 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.013 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.013 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.013 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.013 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.013 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.013 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.013 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.013 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.013 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.013 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.013 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.014 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.014 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.014 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.014 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.014 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.014 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.014 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.014 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.014 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.014 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.014 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.014 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.014 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.014 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.015 INFO:teuthology.orchestra.run.smithi064.stdout: "password": { 2024-09-21T04:14:04.015 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "password", 2024-09-21T04:14:04.015 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.015 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.015 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.015 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.015 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.015 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.015 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.015 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "password of InfluxDB server user", 2024-09-21T04:14:04.015 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.015 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.015 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.015 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.016 INFO:teuthology.orchestra.run.smithi064.stdout: "port": { 2024-09-21T04:14:04.016 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "port", 2024-09-21T04:14:04.016 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.016 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.016 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.016 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "8086", 2024-09-21T04:14:04.016 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.016 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.016 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.016 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "InfluxDB server port", 2024-09-21T04:14:04.016 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.016 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.016 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.016 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.016 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.017 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.017 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.017 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.017 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.017 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.017 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.017 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.017 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.017 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.017 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.017 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.017 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.017 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.017 INFO:teuthology.orchestra.run.smithi064.stdout: "ssl": { 2024-09-21T04:14:04.018 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ssl", 2024-09-21T04:14:04.018 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.018 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.018 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.018 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "false", 2024-09-21T04:14:04.018 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.018 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.018 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.018 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T04:14:04.018 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.018 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.018 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.018 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.018 INFO:teuthology.orchestra.run.smithi064.stdout: "threads": { 2024-09-21T04:14:04.018 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threads", 2024-09-21T04:14:04.019 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.019 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.019 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.019 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-21T04:14:04.019 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-21T04:14:04.019 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "32", 2024-09-21T04:14:04.019 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.019 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T04:14:04.019 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.019 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.019 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.019 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.019 INFO:teuthology.orchestra.run.smithi064.stdout: "username": { 2024-09-21T04:14:04.019 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "username", 2024-09-21T04:14:04.020 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.020 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.020 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.020 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.020 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.020 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.020 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.020 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "username of InfluxDB server user", 2024-09-21T04:14:04.020 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.020 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.020 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.020 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.020 INFO:teuthology.orchestra.run.smithi064.stdout: "verify_ssl": { 2024-09-21T04:14:04.020 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "verify_ssl", 2024-09-21T04:14:04.021 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.021 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.021 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.021 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "true", 2024-09-21T04:14:04.021 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.021 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.021 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.021 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T04:14:04.021 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.021 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.021 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.021 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.021 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.021 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.022 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.022 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "insights", 2024-09-21T04:14:04.022 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:04.022 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:04.022 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.022 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.022 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.022 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.022 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.022 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.022 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.022 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.022 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.022 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.022 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.023 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.023 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.023 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.023 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.023 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.023 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.023 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.023 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.023 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.023 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.023 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.023 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.023 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.023 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.024 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.024 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.024 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.024 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.024 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.024 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.024 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.024 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.024 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.024 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.024 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.024 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.024 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.024 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.025 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.025 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.025 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.025 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.025 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.025 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.025 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.025 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.025 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.025 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.025 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.025 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.025 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.025 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.025 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.026 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.026 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.026 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.026 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.026 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.026 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.026 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.026 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.026 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.026 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.026 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.026 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.026 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.026 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.027 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.027 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.027 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.027 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.027 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.027 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.027 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.027 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.027 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.027 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.027 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.027 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.027 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.027 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.027 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.028 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.028 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.028 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.028 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.028 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.028 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "iostat", 2024-09-21T04:14:04.028 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:04.028 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:04.028 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.028 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.028 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.028 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.028 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.028 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.029 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.029 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.029 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.029 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.029 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.029 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.029 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.029 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.029 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.029 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.029 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.029 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.029 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.029 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.029 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.030 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.030 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.030 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.030 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.030 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.030 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.030 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.030 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.030 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.030 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.030 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.030 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.030 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.030 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.031 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.031 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.031 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.031 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.031 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.031 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.031 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.031 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.031 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.031 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.031 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.031 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.031 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.031 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.031 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.032 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.032 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.032 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.032 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.032 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.032 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.032 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.032 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.032 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.032 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.032 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.032 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.032 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.032 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.032 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.033 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.033 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.033 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.033 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.033 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.033 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.033 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.033 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.033 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.033 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.033 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.033 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.033 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.033 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.034 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.034 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.034 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.034 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.034 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.034 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.034 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.034 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.034 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.034 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "k8sevents", 2024-09-21T04:14:04.034 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:04.034 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:04.034 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.034 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_event_retention_days": { 2024-09-21T04:14:04.034 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "ceph_event_retention_days", 2024-09-21T04:14:04.035 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.035 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.035 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.035 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "7", 2024-09-21T04:14:04.035 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.035 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.035 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.035 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T04:14:04.035 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.035 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.035 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.035 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.035 INFO:teuthology.orchestra.run.smithi064.stdout: "config_check_secs": { 2024-09-21T04:14:04.035 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "config_check_secs", 2024-09-21T04:14:04.035 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.036 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.036 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.036 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-21T04:14:04.036 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "10", 2024-09-21T04:14:04.036 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.036 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.036 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T04:14:04.036 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.036 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.036 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.036 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.036 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.036 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.036 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.037 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.037 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.037 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.037 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.037 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.037 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.037 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.037 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.037 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.037 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.037 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.037 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.037 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.037 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.037 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.038 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.038 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.038 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.038 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.038 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.038 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.038 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.038 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.038 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.038 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.038 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.038 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.038 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.038 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.039 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.039 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.039 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.039 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.039 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.039 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.039 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.039 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.039 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.039 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.039 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.039 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.039 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.039 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.039 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.040 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.040 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.040 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.040 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.040 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.040 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.040 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.040 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.040 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.040 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.040 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.040 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.040 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.040 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.040 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.041 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.041 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.041 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.041 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.041 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.041 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.041 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.041 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.041 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.041 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.041 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.041 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.041 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.041 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.041 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.042 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.042 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.042 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.042 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.042 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.042 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.042 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.042 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.042 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.042 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.042 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "localpool", 2024-09-21T04:14:04.042 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:04.042 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:04.042 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.042 INFO:teuthology.orchestra.run.smithi064.stdout: "failure_domain": { 2024-09-21T04:14:04.043 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "failure_domain", 2024-09-21T04:14:04.043 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.043 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.043 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.043 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "host", 2024-09-21T04:14:04.043 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.043 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.043 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.043 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "failure domain for any created local pool", 2024-09-21T04:14:04.043 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T04:14:04.043 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.043 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.043 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.043 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.044 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.044 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.044 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.044 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.044 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.044 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.044 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.044 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.044 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.044 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.044 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.044 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.044 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.044 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.045 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.045 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.045 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.045 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.045 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.045 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.045 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.045 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.045 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.045 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.045 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.045 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.045 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.045 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.045 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.046 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.046 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.046 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.046 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.046 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.046 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.046 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.046 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.046 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.046 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.046 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.046 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.046 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.046 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.046 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.047 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.047 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.047 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.047 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.047 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.047 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.047 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.047 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.047 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.047 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.047 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.047 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.047 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.047 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.047 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.048 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.048 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.048 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.048 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.048 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.048 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.048 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.048 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.048 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.048 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.048 INFO:teuthology.orchestra.run.smithi064.stdout: "min_size": { 2024-09-21T04:14:04.048 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "min_size", 2024-09-21T04:14:04.048 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.048 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.048 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.049 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.049 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.049 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.049 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.049 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default min_size for any created local pool", 2024-09-21T04:14:04.049 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T04:14:04.049 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.049 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.049 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.049 INFO:teuthology.orchestra.run.smithi064.stdout: "num_rep": { 2024-09-21T04:14:04.049 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "num_rep", 2024-09-21T04:14:04.049 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.049 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.049 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.049 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3", 2024-09-21T04:14:04.050 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.050 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.050 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.050 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default replica count for any created local pool", 2024-09-21T04:14:04.050 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.050 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.050 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.050 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.050 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_num": { 2024-09-21T04:14:04.050 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_num", 2024-09-21T04:14:04.050 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.050 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.050 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.050 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "128", 2024-09-21T04:14:04.051 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.051 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.051 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.051 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T04:14:04.051 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.051 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.051 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.051 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.051 INFO:teuthology.orchestra.run.smithi064.stdout: "prefix": { 2024-09-21T04:14:04.051 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prefix", 2024-09-21T04:14:04.051 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.051 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.051 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.051 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.051 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.052 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.052 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.052 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "name prefix for any created local pool", 2024-09-21T04:14:04.052 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.052 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.052 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.052 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.052 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.052 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.052 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.052 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.052 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.052 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.052 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.053 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.053 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.053 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.053 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.053 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.053 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.053 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.053 INFO:teuthology.orchestra.run.smithi064.stdout: "subtree": { 2024-09-21T04:14:04.053 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "subtree", 2024-09-21T04:14:04.053 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.053 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.053 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.053 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "rack", 2024-09-21T04:14:04.053 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.054 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.054 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.054 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T04:14:04.054 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T04:14:04.054 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.054 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.054 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.054 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.054 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.054 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.054 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mds_autoscaler", 2024-09-21T04:14:04.054 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:04.054 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:04.054 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.054 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.055 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.055 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.055 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.055 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.055 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.055 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.055 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.055 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.055 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.055 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.055 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.055 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.055 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.055 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.056 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.056 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.056 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.056 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.056 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.056 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.056 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.056 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.056 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.056 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.056 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.056 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.056 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.056 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.057 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.057 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.057 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.057 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.057 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.057 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.057 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.057 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.057 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.057 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.057 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.057 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.057 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.057 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.057 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.058 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.058 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.058 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.058 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.058 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.058 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.058 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.058 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.058 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.058 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.058 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.058 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.058 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.058 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.059 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.059 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.059 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.059 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.059 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.059 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.059 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.059 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.059 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.059 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.059 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.059 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.059 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.059 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.059 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.060 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.060 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.060 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.060 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.060 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.060 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.060 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.060 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.060 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.060 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.060 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.060 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.060 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.060 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.060 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirroring", 2024-09-21T04:14:04.061 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:04.061 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:04.061 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.061 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.061 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.061 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.061 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.061 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.061 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.061 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.061 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.061 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.061 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.061 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.061 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.062 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.062 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.062 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.062 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.062 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.062 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.062 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.062 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.062 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.062 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.062 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.062 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.062 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.062 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.062 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.063 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.063 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.063 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.063 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.063 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.063 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.063 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.063 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.063 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.063 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.063 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.063 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.063 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.063 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.063 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.064 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.064 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.064 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.064 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.064 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.064 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.064 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.064 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.064 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.064 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.064 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.064 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.064 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.064 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.064 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.065 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.065 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.065 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.065 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.065 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.065 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.065 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.065 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.065 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.065 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.065 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.065 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.065 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.065 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.066 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.066 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.066 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.066 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.066 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.066 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.066 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.066 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.066 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.066 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.066 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.066 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.066 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.066 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.067 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.067 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.067 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "nfs", 2024-09-21T04:14:04.067 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:04.067 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:04.067 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.067 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.067 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.067 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.067 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.067 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.067 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.067 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.067 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.067 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.068 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.068 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.068 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.068 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.068 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.068 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.068 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.068 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.068 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.068 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.068 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.068 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.068 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.068 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.068 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.069 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.069 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.069 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.069 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.069 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.069 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.069 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.069 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.069 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.069 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.069 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.069 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.069 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.069 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.069 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.070 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.070 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.070 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.070 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.070 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.070 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.070 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.070 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.070 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.070 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.070 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.070 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.070 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.070 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.071 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.071 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.071 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.071 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.071 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.071 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.071 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.071 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.071 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.071 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.071 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.071 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.071 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.071 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.071 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.072 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.072 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.072 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.072 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.072 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.072 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.072 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.072 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.072 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.072 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.072 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.072 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.072 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.072 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.073 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.073 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.073 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.073 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.073 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.073 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-09-21T04:14:04.073 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:04.073 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:04.073 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.073 INFO:teuthology.orchestra.run.smithi064.stdout: "fail_fs": { 2024-09-21T04:14:04.073 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "fail_fs", 2024-09-21T04:14:04.073 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.073 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.073 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.073 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.074 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.074 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.074 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.074 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T04:14:04.074 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.074 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.074 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.074 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.074 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.074 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.074 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.074 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.074 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.074 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.075 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.075 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.075 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.075 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.075 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.075 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.075 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.075 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.075 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.075 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.075 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.075 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.075 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.075 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.075 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.076 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.076 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.076 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.076 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.076 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.076 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.076 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.076 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.076 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.076 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.076 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.076 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.076 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.076 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.076 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.077 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.077 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.077 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.077 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.077 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.077 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.077 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.077 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.077 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.077 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.077 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.077 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.077 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.077 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.077 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.078 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.078 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.078 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.078 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.078 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.078 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.078 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.078 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.078 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.078 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.078 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.078 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.078 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.078 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator": { 2024-09-21T04:14:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "orchestrator", 2024-09-21T04:14:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.079 INFO:teuthology.orchestra.run.smithi064.stdout: "cephadm", 2024-09-21T04:14:04.080 INFO:teuthology.orchestra.run.smithi064.stdout: "rook", 2024-09-21T04:14:04.080 INFO:teuthology.orchestra.run.smithi064.stdout: "test_orchestrator" 2024-09-21T04:14:04.080 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.080 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Orchestrator backend", 2024-09-21T04:14:04.080 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.080 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.080 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.080 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.080 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.080 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.080 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.080 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.080 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.080 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.080 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.081 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.081 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.081 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.081 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.081 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.081 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.081 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.081 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.081 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.081 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.081 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_perf_query", 2024-09-21T04:14:04.081 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:04.081 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:04.081 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.081 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.082 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.082 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.082 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.082 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.082 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.082 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.082 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.082 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.082 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.082 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.082 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.082 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.082 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.083 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.084 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.084 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.084 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.084 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.084 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.084 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.084 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.084 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.084 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.084 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.084 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.084 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.084 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.084 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.085 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.086 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.087 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.087 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.087 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.087 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.087 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.087 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.087 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.087 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.087 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.087 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.087 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.087 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.087 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.087 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.088 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "osd_support", 2024-09-21T04:14:04.088 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:04.088 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:04.088 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.088 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.088 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.088 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.088 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.088 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.088 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.088 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.088 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.088 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.088 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.088 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.089 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.089 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.089 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.089 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.089 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.089 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.089 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.089 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.089 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.089 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.089 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.089 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.089 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.089 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.089 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.090 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.090 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.090 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.090 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.090 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.090 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.090 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.090 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.090 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.090 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.090 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.090 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.090 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.090 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.093 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.093 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.093 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.093 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.093 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.093 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.093 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.094 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.094 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.094 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.094 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.094 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.094 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.094 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.094 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.094 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.094 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.094 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.094 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.094 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.094 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.095 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.095 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.095 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.095 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.095 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.095 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.095 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.095 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.095 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.095 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.095 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.095 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.095 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.095 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.095 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.096 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.096 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.096 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.096 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.096 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.096 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.096 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.096 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.096 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.096 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.096 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.096 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "pg_autoscaler", 2024-09-21T04:14:04.096 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:04.096 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:04.096 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.097 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.097 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.097 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.097 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.097 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.097 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.097 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.097 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.097 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.097 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.097 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.097 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.097 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.097 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.097 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.098 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.098 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.098 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.098 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.098 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.098 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.098 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.098 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.098 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.098 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.098 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.098 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.098 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.098 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.099 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.099 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.099 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.099 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.099 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.099 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.099 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.099 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.099 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.099 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.099 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.099 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.099 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.099 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.100 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.100 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.100 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.100 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.100 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.100 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.100 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.100 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.100 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.100 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.100 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.100 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.100 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.100 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.100 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.101 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.101 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.101 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.101 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.101 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.101 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.101 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.101 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.101 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.101 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.101 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.101 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.101 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-21T04:14:04.101 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-21T04:14:04.102 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:04.102 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.102 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.102 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-21T04:14:04.102 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.102 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.102 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.102 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.102 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.102 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.102 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.102 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.102 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.102 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.103 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.103 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.103 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.103 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.103 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.103 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.103 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.103 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.103 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.103 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.103 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.103 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.103 INFO:teuthology.orchestra.run.smithi064.stdout: "threshold": { 2024-09-21T04:14:04.103 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "threshold", 2024-09-21T04:14:04.103 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-21T04:14:04.104 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.104 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.104 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "3.0", 2024-09-21T04:14:04.104 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1.0", 2024-09-21T04:14:04.104 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.104 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.104 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "scaling threshold", 2024-09-21T04:14:04.104 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T04:14:04.104 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.104 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.104 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.104 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.104 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.104 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.105 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "progress", 2024-09-21T04:14:04.105 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:04.105 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:04.105 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.105 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_pg_recovery_event": { 2024-09-21T04:14:04.105 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_pg_recovery_event", 2024-09-21T04:14:04.105 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.105 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.105 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.105 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.105 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.105 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.105 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.105 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T04:14:04.106 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.106 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.106 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.106 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.106 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-09-21T04:14:04.106 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-09-21T04:14:04.106 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.106 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.106 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.106 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:04.106 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.106 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.106 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.106 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.107 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.107 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.107 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.107 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.107 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.107 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.107 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.107 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.107 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.107 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.107 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.107 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.107 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.107 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.107 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.108 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.108 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.108 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.108 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.108 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.108 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.108 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.108 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.108 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.108 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.108 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.108 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.108 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.108 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.109 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.109 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.109 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.109 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.109 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.109 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.109 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.109 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.109 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.109 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.109 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.109 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.109 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.109 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.110 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.110 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.110 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.110 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.110 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.110 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.110 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.110 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.110 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.110 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.110 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.110 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.110 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.110 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.111 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.111 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.111 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.111 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.111 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.111 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.111 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.111 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.111 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.111 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.111 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.111 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.111 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.111 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.112 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.112 INFO:teuthology.orchestra.run.smithi064.stdout: "max_completed_events": { 2024-09-21T04:14:04.112 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_completed_events", 2024-09-21T04:14:04.112 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.112 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.112 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.112 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "50", 2024-09-21T04:14:04.112 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.112 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.112 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.112 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "number of past completed events to remember", 2024-09-21T04:14:04.112 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.112 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.112 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.113 INFO:teuthology.orchestra.run.smithi064.stdout: "sleep_interval": { 2024-09-21T04:14:04.113 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sleep_interval", 2024-09-21T04:14:04.113 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:04.113 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.113 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.113 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-21T04:14:04.113 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.113 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.113 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.113 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "how long the module is going to sleep", 2024-09-21T04:14:04.113 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.113 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.113 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.114 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.114 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.114 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.114 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.114 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.114 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.114 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.114 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.114 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.114 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.114 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.114 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.114 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.114 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.115 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.115 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.115 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.115 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "prometheus", 2024-09-21T04:14:04.115 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:04.115 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:04.115 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.115 INFO:teuthology.orchestra.run.smithi064.stdout: "cache": { 2024-09-21T04:14:04.115 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "cache", 2024-09-21T04:14:04.115 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.115 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.115 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.115 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:04.115 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.115 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.116 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.116 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.116 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.116 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.116 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.116 INFO:teuthology.orchestra.run.smithi064.stdout: "exclude_perf_counters": { 2024-09-21T04:14:04.116 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "exclude_perf_counters", 2024-09-21T04:14:04.116 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.116 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.116 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.116 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:04.116 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.116 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.117 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.117 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T04:14:04.117 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T04:14:04.117 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.117 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.117 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.117 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.117 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.117 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.117 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.117 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.117 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.117 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.118 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.118 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.118 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.118 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.118 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.118 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.118 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.118 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.118 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.118 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.118 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.118 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.118 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.118 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.119 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.119 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.119 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.119 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.119 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.119 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.119 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.119 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.119 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.119 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.119 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.119 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.119 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.119 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.120 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.120 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.120 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.120 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.120 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.120 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.120 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.120 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.120 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.120 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.120 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.120 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.120 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.120 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.121 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.121 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.121 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.121 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.121 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.121 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.121 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.121 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.121 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.121 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.121 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.121 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.121 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.121 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools": { 2024-09-21T04:14:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools", 2024-09-21T04:14:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.122 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.123 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.123 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.123 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.123 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.123 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T04:14:04.123 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T04:14:04.123 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.123 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.123 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.123 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "300", 2024-09-21T04:14:04.123 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.123 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.123 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.123 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.123 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.124 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.124 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.124 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.124 INFO:teuthology.orchestra.run.smithi064.stdout: "scrape_interval": { 2024-09-21T04:14:04.124 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "scrape_interval", 2024-09-21T04:14:04.124 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-21T04:14:04.124 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.124 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.124 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15.0", 2024-09-21T04:14:04.124 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.124 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.124 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.124 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.124 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.125 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.125 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.125 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.125 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-09-21T04:14:04.125 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-09-21T04:14:04.125 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.125 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.125 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.125 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "::", 2024-09-21T04:14:04.125 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.125 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.125 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.125 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T04:14:04.125 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.126 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.126 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.126 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-09-21T04:14:04.126 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-09-21T04:14:04.126 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.126 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.126 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.126 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "9283", 2024-09-21T04:14:04.126 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.126 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.126 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.126 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T04:14:04.126 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.127 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.128 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.128 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.128 INFO:teuthology.orchestra.run.smithi064.stdout: "stale_cache_strategy": { 2024-09-21T04:14:04.128 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stale_cache_strategy", 2024-09-21T04:14:04.128 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.128 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.128 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.128 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "log", 2024-09-21T04:14:04.128 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.128 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.128 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.128 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.128 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.128 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.128 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.129 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_behaviour": { 2024-09-21T04:14:04.129 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_behaviour", 2024-09-21T04:14:04.129 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.129 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.129 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.129 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "default", 2024-09-21T04:14:04.129 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.129 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.129 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.129 INFO:teuthology.orchestra.run.smithi064.stdout: "default", 2024-09-21T04:14:04.129 INFO:teuthology.orchestra.run.smithi064.stdout: "error" 2024-09-21T04:14:04.129 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.129 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.129 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: "standby_error_status_code": { 2024-09-21T04:14:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "standby_error_status_code", 2024-09-21T04:14:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-09-21T04:14:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "400", 2024-09-21T04:14:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "599", 2024-09-21T04:14:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.130 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_support", 2024-09-21T04:14:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.131 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.132 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.132 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.132 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.132 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.132 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.132 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.132 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.132 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.132 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.132 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.132 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.132 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.132 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.132 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.132 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.133 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.133 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.133 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.133 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.133 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.133 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.133 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.133 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.133 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.133 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.133 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.133 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.133 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.133 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.134 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.134 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.134 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.134 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.134 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.134 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.134 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.134 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.134 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.134 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.134 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.134 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.134 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.135 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.135 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.135 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.135 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.135 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.135 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.135 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.135 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.135 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.135 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.135 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.135 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.135 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.135 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.136 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.136 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.136 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.136 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.136 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.136 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.136 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_snap_create": { 2024-09-21T04:14:04.136 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_snap_create", 2024-09-21T04:14:04.136 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.136 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.136 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.136 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10", 2024-09-21T04:14:04.136 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.137 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.137 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.137 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.137 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.137 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.137 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.137 INFO:teuthology.orchestra.run.smithi064.stdout: "mirror_snapshot_schedule": { 2024-09-21T04:14:04.137 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T04:14:04.137 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.137 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.137 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.137 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.137 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.137 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.138 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.138 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.138 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.138 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.138 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.138 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.138 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.138 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.138 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.138 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.138 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.138 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.138 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.139 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.139 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.139 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.139 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.139 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.139 INFO:teuthology.orchestra.run.smithi064.stdout: "trash_purge_schedule": { 2024-09-21T04:14:04.139 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "trash_purge_schedule", 2024-09-21T04:14:04.139 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.139 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.139 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.139 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.139 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.139 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.139 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.140 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.140 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.140 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.140 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.140 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.140 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.140 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.140 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "restful", 2024-09-21T04:14:04.140 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:04.140 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:04.140 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.140 INFO:teuthology.orchestra.run.smithi064.stdout: "enable_auth": { 2024-09-21T04:14:04.140 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enable_auth", 2024-09-21T04:14:04.140 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.141 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.141 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.141 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:04.141 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.141 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.141 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.141 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.141 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.141 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.141 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.141 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.141 INFO:teuthology.orchestra.run.smithi064.stdout: "key_file": { 2024-09-21T04:14:04.141 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "key_file", 2024-09-21T04:14:04.141 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.142 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.142 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.142 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.142 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.142 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.142 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.142 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.142 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.142 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.142 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.142 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.142 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.142 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.142 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.143 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.143 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.143 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.143 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.143 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.143 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.143 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.143 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.143 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.143 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.143 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.143 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.143 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.143 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.143 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.144 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.144 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.144 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.144 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.144 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.144 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.144 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.144 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.144 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.144 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.144 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.144 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.144 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.144 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.144 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.145 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.145 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.145 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.145 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.145 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.145 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.145 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.145 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.145 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.145 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.145 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.145 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.145 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.145 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.145 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.146 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.146 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.146 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.146 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.146 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.146 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.146 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.146 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.146 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.146 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.146 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.146 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.146 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: "max_requests": { 2024-09-21T04:14:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_requests", 2024-09-21T04:14:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "500", 2024-09-21T04:14:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.147 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T04:14:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: "server_addr": { 2024-09-21T04:14:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_addr", 2024-09-21T04:14:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.148 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: "server_port": { 2024-09-21T04:14:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "server_port", 2024-09-21T04:14:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.149 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.150 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.150 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.150 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.150 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.150 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.150 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.150 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.150 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.150 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.150 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.150 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.150 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.150 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rgw", 2024-09-21T04:14:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.151 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.152 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.153 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.154 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.155 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T04:14:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T04:14:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.156 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "5", 2024-09-21T04:14:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T04:14:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.157 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rook", 2024-09-21T04:14:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:04.158 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.159 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.160 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.161 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.161 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.161 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.161 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.161 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.161 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.161 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.161 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.161 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.161 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.161 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.161 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.161 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.161 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.162 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.163 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: "storage_class": { 2024-09-21T04:14:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "storage_class", 2024-09-21T04:14:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.164 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "local", 2024-09-21T04:14:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T04:14:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.165 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "selftest", 2024-09-21T04:14:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.166 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.167 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.168 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.169 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.170 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: "roption1": { 2024-09-21T04:14:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption1", 2024-09-21T04:14:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: "roption2": { 2024-09-21T04:14:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "roption2", 2024-09-21T04:14:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "xyz", 2024-09-21T04:14:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.172 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption1": { 2024-09-21T04:14:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption1", 2024-09-21T04:14:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.173 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption2": { 2024-09-21T04:14:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption2", 2024-09-21T04:14:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption3": { 2024-09-21T04:14:04.174 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption3", 2024-09-21T04:14:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "float", 2024-09-21T04:14:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption4": { 2024-09-21T04:14:04.175 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption4", 2024-09-21T04:14:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption5": { 2024-09-21T04:14:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption5", 2024-09-21T04:14:04.176 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.177 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.177 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.177 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.177 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.177 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.177 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.177 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.177 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.177 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.177 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.177 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption6": { 2024-09-21T04:14:04.177 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption6", 2024-09-21T04:14:04.177 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.177 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.178 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.178 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:04.178 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.178 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.178 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.178 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.178 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.178 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.178 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.178 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.178 INFO:teuthology.orchestra.run.smithi064.stdout: "rwoption7": { 2024-09-21T04:14:04.178 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rwoption7", 2024-09-21T04:14:04.178 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.178 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.178 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.179 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.179 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "1", 2024-09-21T04:14:04.179 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "42", 2024-09-21T04:14:04.179 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.179 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.179 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.179 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.179 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.179 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.179 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.179 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.179 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.179 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.180 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.180 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.180 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.180 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.180 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.180 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.180 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.180 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.180 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.180 INFO:teuthology.orchestra.run.smithi064.stdout: "testkey": { 2024-09-21T04:14:04.180 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testkey", 2024-09-21T04:14:04.180 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.180 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.180 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: "testlkey": { 2024-09-21T04:14:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testlkey", 2024-09-21T04:14:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.181 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: "testnewline": { 2024-09-21T04:14:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "testnewline", 2024-09-21T04:14:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.182 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "smb", 2024-09-21T04:14:04.183 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: "internal_store_backend": { 2024-09-21T04:14:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "internal_store_backend", 2024-09-21T04:14:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T04:14:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.184 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.185 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.186 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.187 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.188 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.189 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.189 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.189 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.189 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.189 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.189 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.189 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.189 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.189 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.189 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.189 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.189 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.189 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.189 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: "update_orchestration": { 2024-09-21T04:14:04.190 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "update_orchestration", 2024-09-21T04:14:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T04:14:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.191 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2024-09-21T04:14:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: "allow_m_granularity": { 2024-09-21T04:14:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "allow_m_granularity", 2024-09-21T04:14:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.192 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T04:14:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: "dump_on_update": { 2024-09-21T04:14:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "dump_on_update", 2024-09-21T04:14:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "dump database to debug log on update", 2024-09-21T04:14:04.193 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.194 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.195 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.196 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.196 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.196 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.196 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.196 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.196 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.196 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.196 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.196 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.196 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.196 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.196 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.196 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.196 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.197 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.198 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.199 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "stats", 2024-09-21T04:14:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.200 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.201 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.202 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.203 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.204 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.205 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.206 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.206 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.206 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.206 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.206 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.206 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.206 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.206 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "status", 2024-09-21T04:14:04.206 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:04.206 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:04.206 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.206 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.206 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.206 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.206 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.207 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.208 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.209 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.210 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.211 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telegraf", 2024-09-21T04:14:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.212 INFO:teuthology.orchestra.run.smithi064.stdout: "address": { 2024-09-21T04:14:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "address", 2024-09-21T04:14:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T04:14:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-21T04:14:04.213 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-21T04:14:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "15", 2024-09-21T04:14:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.214 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.215 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.216 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.217 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.218 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.219 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.220 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "telemetry", 2024-09-21T04:14:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_basic": { 2024-09-21T04:14:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_basic", 2024-09-21T04:14:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.221 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T04:14:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_crash": { 2024-09-21T04:14:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_crash", 2024-09-21T04:14:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.222 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T04:14:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_device": { 2024-09-21T04:14:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_device", 2024-09-21T04:14:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.223 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T04:14:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_ident": { 2024-09-21T04:14:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_ident", 2024-09-21T04:14:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.224 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T04:14:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "channel_perf": { 2024-09-21T04:14:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "channel_perf", 2024-09-21T04:14:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.225 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T04:14:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "contact": { 2024-09-21T04:14:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "contact", 2024-09-21T04:14:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.226 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "description": { 2024-09-21T04:14:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "description", 2024-09-21T04:14:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.227 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "device_url": { 2024-09-21T04:14:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "device_url", 2024-09-21T04:14:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T04:14:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.228 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "enabled": { 2024-09-21T04:14:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "enabled", 2024-09-21T04:14:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.229 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-21T04:14:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-21T04:14:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "24", 2024-09-21T04:14:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "8", 2024-09-21T04:14:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.230 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "last_opt_revision": { 2024-09-21T04:14:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "last_opt_revision", 2024-09-21T04:14:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "1", 2024-09-21T04:14:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.231 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard": { 2024-09-21T04:14:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard", 2024-09-21T04:14:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.232 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "leaderboard_description": { 2024-09-21T04:14:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "leaderboard_description", 2024-09-21T04:14:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.233 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.234 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.235 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.236 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.237 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "organization": { 2024-09-21T04:14:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "organization", 2024-09-21T04:14:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.238 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "proxy": { 2024-09-21T04:14:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "proxy", 2024-09-21T04:14:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.239 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.240 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "url": { 2024-09-21T04:14:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "url", 2024-09-21T04:14:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.241 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T04:14:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "test_orchestrator", 2024-09-21T04:14:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:04.242 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.243 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.244 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.245 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.246 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.247 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.248 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volumes", 2024-09-21T04:14:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.249 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.250 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.251 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.252 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.253 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "max_concurrent_clones": { 2024-09-21T04:14:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "max_concurrent_clones", 2024-09-21T04:14:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "4", 2024-09-21T04:14:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T04:14:04.254 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "periodic_async_work": { 2024-09-21T04:14:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "periodic_async_work", 2024-09-21T04:14:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Periodically check for async work", 2024-09-21T04:14:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.255 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_delay": { 2024-09-21T04:14:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_delay", 2024-09-21T04:14:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T04:14:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.256 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "snapshot_clone_no_wait": { 2024-09-21T04:14:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T04:14:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "True", 2024-09-21T04:14:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T04:14:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.257 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.258 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix", 2024-09-21T04:14:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "can_run": true, 2024-09-21T04:14:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "error_string": "", 2024-09-21T04:14:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "module_options": { 2024-09-21T04:14:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "discovery_interval": { 2024-09-21T04:14:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "discovery_interval", 2024-09-21T04:14:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "uint", 2024-09-21T04:14:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "100", 2024-09-21T04:14:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.259 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: "identifier": { 2024-09-21T04:14:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "identifier", 2024-09-21T04:14:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.260 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.261 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.261 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.261 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.261 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.261 INFO:teuthology.orchestra.run.smithi064.stdout: "interval": { 2024-09-21T04:14:04.261 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "interval", 2024-09-21T04:14:04.261 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "secs", 2024-09-21T04:14:04.261 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.261 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.261 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "60", 2024-09-21T04:14:04.261 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.261 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.261 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.261 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: "log_level": { 2024-09-21T04:14:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_level", 2024-09-21T04:14:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.262 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster": { 2024-09-21T04:14:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster", 2024-09-21T04:14:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.263 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_cluster_level": { 2024-09-21T04:14:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_cluster_level", 2024-09-21T04:14:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.264 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "info", 2024-09-21T04:14:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [ 2024-09-21T04:14:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: "", 2024-09-21T04:14:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: "critical", 2024-09-21T04:14:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: "debug", 2024-09-21T04:14:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: "error", 2024-09-21T04:14:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: "info", 2024-09-21T04:14:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: "warning" 2024-09-21T04:14:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.265 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: "log_to_file": { 2024-09-21T04:14:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "log_to_file", 2024-09-21T04:14:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "bool", 2024-09-21T04:14:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "False", 2024-09-21T04:14:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.266 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: "sqlite3_killpoint": { 2024-09-21T04:14:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "sqlite3_killpoint", 2024-09-21T04:14:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "dev", 2024-09-21T04:14:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 1, 2024-09-21T04:14:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "0", 2024-09-21T04:14:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.267 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_host": { 2024-09-21T04:14:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_host", 2024-09-21T04:14:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "", 2024-09-21T04:14:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.268 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_port": { 2024-09-21T04:14:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_port", 2024-09-21T04:14:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "int", 2024-09-21T04:14:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "10051", 2024-09-21T04:14:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.269 INFO:teuthology.orchestra.run.smithi064.stdout: "zabbix_sender": { 2024-09-21T04:14:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "zabbix_sender", 2024-09-21T04:14:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "str", 2024-09-21T04:14:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: "level": "advanced", 2024-09-21T04:14:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: "flags": 0, 2024-09-21T04:14:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T04:14:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: "min": "", 2024-09-21T04:14:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: "max": "", 2024-09-21T04:14:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: "enum_allowed": [], 2024-09-21T04:14:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: "desc": "", 2024-09-21T04:14:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: "long_desc": "", 2024-09-21T04:14:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: "tags": [], 2024-09-21T04:14:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: "see_also": [] 2024-09-21T04:14:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.270 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: "services": { 2024-09-21T04:14:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: "dashboard": "https://172.21.15.64:8443/" 2024-09-21T04:14:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: "always_on_modules": { 2024-09-21T04:14:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: "octopus": [ 2024-09-21T04:14:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2024-09-21T04:14:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2024-09-21T04:14:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2024-09-21T04:14:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2024-09-21T04:14:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2024-09-21T04:14:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2024-09-21T04:14:04.271 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2024-09-21T04:14:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2024-09-21T04:14:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2024-09-21T04:14:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2024-09-21T04:14:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: "pacific": [ 2024-09-21T04:14:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2024-09-21T04:14:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2024-09-21T04:14:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2024-09-21T04:14:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2024-09-21T04:14:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2024-09-21T04:14:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2024-09-21T04:14:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2024-09-21T04:14:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2024-09-21T04:14:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2024-09-21T04:14:04.272 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2024-09-21T04:14:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: "quincy": [ 2024-09-21T04:14:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2024-09-21T04:14:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2024-09-21T04:14:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2024-09-21T04:14:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2024-09-21T04:14:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2024-09-21T04:14:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2024-09-21T04:14:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2024-09-21T04:14:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2024-09-21T04:14:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2024-09-21T04:14:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2024-09-21T04:14:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: "reef": [ 2024-09-21T04:14:04.273 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2024-09-21T04:14:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2024-09-21T04:14:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2024-09-21T04:14:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2024-09-21T04:14:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2024-09-21T04:14:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2024-09-21T04:14:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2024-09-21T04:14:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2024-09-21T04:14:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2024-09-21T04:14:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2024-09-21T04:14:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: ], 2024-09-21T04:14:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: "squid": [ 2024-09-21T04:14:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: "balancer", 2024-09-21T04:14:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: "crash", 2024-09-21T04:14:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: "devicehealth", 2024-09-21T04:14:04.274 INFO:teuthology.orchestra.run.smithi064.stdout: "orchestrator", 2024-09-21T04:14:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: "pg_autoscaler", 2024-09-21T04:14:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: "progress", 2024-09-21T04:14:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: "rbd_support", 2024-09-21T04:14:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: "status", 2024-09-21T04:14:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: "telemetry", 2024-09-21T04:14:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: "volumes" 2024-09-21T04:14:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-21T04:14:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: "last_failure_osd_epoch": 5, 2024-09-21T04:14:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: "active_clients": [ 2024-09-21T04:14:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "devicehealth", 2024-09-21T04:14:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-09-21T04:14:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.275 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-09-21T04:14:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2024-09-21T04:14:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2407752174 2024-09-21T04:14:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-21T04:14:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "libcephsqlite", 2024-09-21T04:14:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-09-21T04:14:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-09-21T04:14:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2024-09-21T04:14:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 869042251 2024-09-21T04:14:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-21T04:14:04.276 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "rbd_support", 2024-09-21T04:14:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-09-21T04:14:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-09-21T04:14:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2024-09-21T04:14:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 4035775195 2024-09-21T04:14:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-21T04:14:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2024-09-21T04:14:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-09-21T04:14:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.277 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-09-21T04:14:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2024-09-21T04:14:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 1386972511 2024-09-21T04:14:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-21T04:14:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "snap_schedule", 2024-09-21T04:14:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-09-21T04:14:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-09-21T04:14:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2024-09-21T04:14:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 730808232 2024-09-21T04:14:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-21T04:14:04.278 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:04.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.279 INFO:teuthology.orchestra.run.smithi064.stdout: "name": "volumes", 2024-09-21T04:14:04.279 INFO:teuthology.orchestra.run.smithi064.stdout: "addrvec": [ 2024-09-21T04:14:04.279 INFO:teuthology.orchestra.run.smithi064.stdout: { 2024-09-21T04:14:04.279 INFO:teuthology.orchestra.run.smithi064.stdout: "type": "v2", 2024-09-21T04:14:04.279 INFO:teuthology.orchestra.run.smithi064.stdout: "addr": "172.21.15.64:0", 2024-09-21T04:14:04.279 INFO:teuthology.orchestra.run.smithi064.stdout: "nonce": 2999912642 2024-09-21T04:14:04.279 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.279 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-21T04:14:04.279 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:04.279 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-09-21T04:14:04.279 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:14:04.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.851+0000 7f7247b69640 1 -- 172.21.15.64:0/3480345547 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f722c0b6680 msgr2=0x7f722c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:04.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.851+0000 7f7247b69640 1 --2- 172.21.15.64:0/3480345547 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f722c0b6680 0x7f722c0b8b90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f723c031440 tx=0x7f723c005d10 comp rx=0 tx=0).stop 2024-09-21T04:14:04.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.851+0000 7f7247b69640 1 -- 172.21.15.64:0/3480345547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72401082f0 msgr2=0x7f724019ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:04.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.851+0000 7f7247b69640 1 --2- 172.21.15.64:0/3480345547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72401082f0 0x7f724019ab00 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f7234006fd0 tx=0x7f723400b8e0 comp rx=0 tx=0).stop 2024-09-21T04:14:04.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.851+0000 7f7247b69640 1 -- 172.21.15.64:0/3480345547 shutdown_connections 2024-09-21T04:14:04.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.851+0000 7f7247b69640 1 --2- 172.21.15.64:0/3480345547 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f722c0b6680 0x7f722c0b8b90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:04.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.851+0000 7f7247b69640 1 --2- 172.21.15.64:0/3480345547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7240108c40 0x7f724019fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:04.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.851+0000 7f7247b69640 1 --2- 172.21.15.64:0/3480345547 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72401082f0 0x7f724019ab00 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:04.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.851+0000 7f7247b69640 1 --2- 172.21.15.64:0/3480345547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72401029c0 0x7f724019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:04.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.851+0000 7f7247b69640 1 -- 172.21.15.64:0/3480345547 >> 172.21.15.64:0/3480345547 conn(0x7f724007a670 msgr2=0x7f7240106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:04.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.851+0000 7f7247b69640 1 -- 172.21.15.64:0/3480345547 shutdown_connections 2024-09-21T04:14:04.281 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:02.851+0000 7f7247b69640 1 -- 172.21.15.64:0/3480345547 wait complete. 2024-09-21T04:14:04.325 DEBUG:tasks.check_counter:Getting stats from a 2024-09-21T04:14:04.325 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph daemon mds.a perf dump 2024-09-21T04:14:04.326 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:03 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2858201541' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T04:14:04.326 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:03 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3480345547' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T04:14:04.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:03 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2858201541' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T04:14:04.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:03 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3480345547' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T04:14:04.327 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:03 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2858201541' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T04:14:04.327 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:03 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3480345547' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T04:14:04.601 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:14:04.722 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:04 smithi064 ceph-mon[31180]: pgmap v630: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-09-21T04:14:04.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:04 smithi119 ceph-mon[34811]: pgmap v630: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-09-21T04:14:04.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:04 smithi131 ceph-mon[35193]: pgmap v630: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-09-21T04:14:05.146 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:14:05.146 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T04:14:05.146 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 208, 2024-09-21T04:14:05.146 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 199, 2024-09-21T04:14:05.147 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 1145152, 2024-09-21T04:14:05.147 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 121792, 2024-09-21T04:14:05.147 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-09-21T04:14:05.147 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 0, 2024-09-21T04:14:05.147 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.040817682, 2024-09-21T04:14:05.147 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.016665360, 2024-09-21T04:14:05.147 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.032973656, 2024-09-21T04:14:05.147 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T04:14:05.147 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:05.147 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 199, 2024-09-21T04:14:05.147 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.013669159, 2024-09-21T04:14:05.147 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000068689 2024-09-21T04:14:05.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:05.147 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:05.148 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-21T04:14:05.148 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-21T04:14:05.148 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-21T04:14:05.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:05.148 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 1145152, 2024-09-21T04:14:05.148 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 121792 2024-09-21T04:14:05.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:05.148 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T04:14:05.148 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 1, 2024-09-21T04:14:05.148 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 150, 2024-09-21T04:14:05.148 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 160, 2024-09-21T04:14:05.148 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 106640, 2024-09-21T04:14:05.148 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 3, 2024-09-21T04:14:05.148 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 2, 2024-09-21T04:14:05.148 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.013239088, 2024-09-21T04:14:05.149 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.010522796, 2024-09-21T04:14:05.149 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.001304140, 2024-09-21T04:14:05.149 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T04:14:05.149 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:05.149 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 148, 2024-09-21T04:14:05.149 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.007019370, 2024-09-21T04:14:05.149 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000047428 2024-09-21T04:14:05.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:05.149 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:05.149 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-21T04:14:05.149 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-21T04:14:05.149 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-21T04:14:05.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:05.149 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T04:14:05.150 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 106640 2024-09-21T04:14:05.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:05.150 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T04:14:05.150 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 0, 2024-09-21T04:14:05.150 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 2, 2024-09-21T04:14:05.150 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 0, 2024-09-21T04:14:05.150 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 0, 2024-09-21T04:14:05.150 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 2, 2024-09-21T04:14:05.150 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 1, 2024-09-21T04:14:05.150 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.000382416, 2024-09-21T04:14:05.150 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T04:14:05.150 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.000314910, 2024-09-21T04:14:05.150 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T04:14:05.150 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:05.151 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-21T04:14:05.151 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-21T04:14:05.151 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-21T04:14:05.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:05.151 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:05.151 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-21T04:14:05.151 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-21T04:14:05.151 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-21T04:14:05.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:05.151 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T04:14:05.151 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T04:14:05.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:05.151 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-09-21T04:14:05.151 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 0, 2024-09-21T04:14:05.151 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-09-21T04:14:05.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:05.152 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-09-21T04:14:05.152 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-09-21T04:14:05.152 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-09-21T04:14:05.152 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T04:14:05.152 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-09-21T04:14:05.152 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T04:14:05.152 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-09-21T04:14:05.152 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T04:14:05.152 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T04:14:05.152 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T04:14:05.152 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T04:14:05.152 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T04:14:05.152 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-09-21T04:14:05.152 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T04:14:05.153 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T04:14:05.153 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-09-21T04:14:05.153 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-09-21T04:14:05.153 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-09-21T04:14:05.153 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-09-21T04:14:05.153 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T04:14:05.153 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T04:14:05.153 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T04:14:05.153 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T04:14:05.153 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T04:14:05.153 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-09-21T04:14:05.153 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-09-21T04:14:05.153 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-09-21T04:14:05.153 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T04:14:05.153 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T04:14:05.154 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-09-21T04:14:05.154 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-09-21T04:14:05.154 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-09-21T04:14:05.154 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-09-21T04:14:05.154 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T04:14:05.154 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-09-21T04:14:05.154 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T04:14:05.154 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-09-21T04:14:05.154 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 149047, 2024-09-21T04:14:05.154 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 31, 2024-09-21T04:14:05.154 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-09-21T04:14:05.154 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-09-21T04:14:05.154 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-09-21T04:14:05.154 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-09-21T04:14:05.154 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-09-21T04:14:05.155 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-09-21T04:14:05.155 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-09-21T04:14:05.155 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-09-21T04:14:05.155 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 0, 2024-09-21T04:14:05.155 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 0, 2024-09-21T04:14:05.155 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T04:14:05.155 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-09-21T04:14:05.155 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-09-21T04:14:05.155 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-09-21T04:14:05.155 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 968, 2024-09-21T04:14:05.155 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 2, 2024-09-21T04:14:05.155 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-09-21T04:14:05.155 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-09-21T04:14:05.155 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-09-21T04:14:05.155 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-09-21T04:14:05.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:05.156 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T04:14:05.156 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-21T04:14:05.156 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-09-21T04:14:05.156 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-21T04:14:05.156 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 205, 2024-09-21T04:14:05.156 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 1113340, 2024-09-21T04:14:05.156 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:05.156 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 205, 2024-09-21T04:14:05.156 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-09-21T04:14:05.156 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-09-21T04:14:05.156 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 205, 2024-09-21T04:14:05.156 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 1113340, 2024-09-21T04:14:05.156 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-21T04:14:05.156 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-21T04:14:05.157 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-21T04:14:05.157 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-21T04:14:05.157 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:05.157 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:05.157 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:14:05.908 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-21T04:14:05.908 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-21T04:14:05.908 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-09-21T04:14:05.908 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-09-21T04:14:05.908 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-09-21T04:14:05.908 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-09-21T04:14:05.908 DEBUG:tasks.check_counter:Getting stats from d 2024-09-21T04:14:05.908 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph daemon mds.d perf dump 2024-09-21T04:14:06.169 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:14:06.564 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:06 smithi064 ceph-mon[31180]: pgmap v631: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-21T04:14:06.687 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:14:06.688 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T04:14:06.688 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 214, 2024-09-21T04:14:06.688 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 205, 2024-09-21T04:14:06.688 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 1146848, 2024-09-21T04:14:06.688 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 125152, 2024-09-21T04:14:06.688 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-09-21T04:14:06.689 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 0, 2024-09-21T04:14:06.689 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.045080365, 2024-09-21T04:14:06.689 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.017474673, 2024-09-21T04:14:06.689 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.037655600, 2024-09-21T04:14:06.689 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T04:14:06.689 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:06.689 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 205, 2024-09-21T04:14:06.690 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.013467966, 2024-09-21T04:14:06.690 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000065697 2024-09-21T04:14:06.690 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:06.690 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:06.690 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-21T04:14:06.690 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-21T04:14:06.690 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-21T04:14:06.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:06.691 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 1146848, 2024-09-21T04:14:06.691 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 125152 2024-09-21T04:14:06.691 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:06.691 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T04:14:06.692 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 1, 2024-09-21T04:14:06.692 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 154, 2024-09-21T04:14:06.692 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 160, 2024-09-21T04:14:06.692 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 107344, 2024-09-21T04:14:06.692 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 3, 2024-09-21T04:14:06.692 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 2, 2024-09-21T04:14:06.692 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.014574991, 2024-09-21T04:14:06.692 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.011557477, 2024-09-21T04:14:06.693 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.001323701, 2024-09-21T04:14:06.693 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T04:14:06.693 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:06.693 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 152, 2024-09-21T04:14:06.693 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.007401074, 2024-09-21T04:14:06.693 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000048691 2024-09-21T04:14:06.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:06.693 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:06.693 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-21T04:14:06.693 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-21T04:14:06.693 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-21T04:14:06.693 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:06.694 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T04:14:06.694 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 107344 2024-09-21T04:14:06.694 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:06.694 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T04:14:06.694 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 0, 2024-09-21T04:14:06.694 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 2, 2024-09-21T04:14:06.695 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 0, 2024-09-21T04:14:06.695 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 0, 2024-09-21T04:14:06.695 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 2, 2024-09-21T04:14:06.695 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 1, 2024-09-21T04:14:06.695 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.000353783, 2024-09-21T04:14:06.695 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T04:14:06.695 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.000250460, 2024-09-21T04:14:06.695 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T04:14:06.696 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:06.696 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-21T04:14:06.696 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-21T04:14:06.696 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-21T04:14:06.696 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:06.696 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:06.696 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-21T04:14:06.696 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-21T04:14:06.697 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-21T04:14:06.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:06.697 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T04:14:06.697 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T04:14:06.697 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:06.697 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-09-21T04:14:06.697 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 0, 2024-09-21T04:14:06.697 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-09-21T04:14:06.698 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:06.698 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-09-21T04:14:06.698 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-09-21T04:14:06.698 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-09-21T04:14:06.698 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T04:14:06.698 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-09-21T04:14:06.698 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T04:14:06.698 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-09-21T04:14:06.698 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T04:14:06.698 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T04:14:06.698 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T04:14:06.698 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T04:14:06.698 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T04:14:06.698 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-09-21T04:14:06.699 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T04:14:06.699 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T04:14:06.699 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-09-21T04:14:06.699 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-09-21T04:14:06.699 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-09-21T04:14:06.699 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-09-21T04:14:06.699 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T04:14:06.699 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T04:14:06.699 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T04:14:06.699 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T04:14:06.700 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T04:14:06.700 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-09-21T04:14:06.700 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-09-21T04:14:06.700 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-09-21T04:14:06.700 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T04:14:06.700 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T04:14:06.700 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-09-21T04:14:06.700 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-09-21T04:14:06.700 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-09-21T04:14:06.701 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-09-21T04:14:06.701 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T04:14:06.701 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-09-21T04:14:06.701 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T04:14:06.701 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-09-21T04:14:06.701 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 148967, 2024-09-21T04:14:06.701 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 29, 2024-09-21T04:14:06.701 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-09-21T04:14:06.701 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-09-21T04:14:06.701 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-09-21T04:14:06.701 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-09-21T04:14:06.701 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-09-21T04:14:06.701 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-09-21T04:14:06.701 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-09-21T04:14:06.702 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-09-21T04:14:06.702 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 0, 2024-09-21T04:14:06.702 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 0, 2024-09-21T04:14:06.702 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T04:14:06.702 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-09-21T04:14:06.702 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-09-21T04:14:06.702 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-09-21T04:14:06.702 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 968, 2024-09-21T04:14:06.702 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 2, 2024-09-21T04:14:06.702 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-09-21T04:14:06.702 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-09-21T04:14:06.702 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-09-21T04:14:06.702 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-09-21T04:14:06.702 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:06.703 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T04:14:06.703 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-21T04:14:06.703 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-09-21T04:14:06.703 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-21T04:14:06.703 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 211, 2024-09-21T04:14:06.703 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 1114184, 2024-09-21T04:14:06.703 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:06.703 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 211, 2024-09-21T04:14:06.703 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-09-21T04:14:06.703 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-09-21T04:14:06.703 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 211, 2024-09-21T04:14:06.703 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 1114184, 2024-09-21T04:14:06.703 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-21T04:14:06.704 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-21T04:14:06.704 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-21T04:14:06.704 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-21T04:14:06.704 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:06.704 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:06.704 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:14:06.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:06 smithi119 ceph-mon[34811]: pgmap v631: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-21T04:14:06.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:06 smithi131 ceph-mon[35193]: pgmap v631: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-09-21T04:14:07.453 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-21T04:14:07.453 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-21T04:14:07.453 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-21T04:14:07.453 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-21T04:14:07.453 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-21T04:14:07.454 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-09-21T04:14:07.454 DEBUG:tasks.check_counter:Getting stats from g 2024-09-21T04:14:07.454 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph daemon mds.g perf dump 2024-09-21T04:14:07.714 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:14:08.240 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:14:08.241 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T04:14:08.241 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 0, 2024-09-21T04:14:08.241 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 2, 2024-09-21T04:14:08.241 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 0, 2024-09-21T04:14:08.241 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 0, 2024-09-21T04:14:08.241 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-09-21T04:14:08.241 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 0, 2024-09-21T04:14:08.241 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.001568689, 2024-09-21T04:14:08.241 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T04:14:08.241 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.000300969, 2024-09-21T04:14:08.241 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T04:14:08.241 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:08.241 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-21T04:14:08.241 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-21T04:14:08.242 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-21T04:14:08.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:08.242 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:08.242 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-21T04:14:08.242 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-21T04:14:08.242 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-21T04:14:08.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:08.242 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T04:14:08.242 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T04:14:08.242 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:08.242 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T04:14:08.242 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 206, 2024-09-21T04:14:08.242 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 199, 2024-09-21T04:14:08.242 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 1142704, 2024-09-21T04:14:08.243 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 122608, 2024-09-21T04:14:08.243 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 2, 2024-09-21T04:14:08.243 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 1, 2024-09-21T04:14:08.243 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.040970071, 2024-09-21T04:14:08.243 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.016358579, 2024-09-21T04:14:08.243 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.035139593, 2024-09-21T04:14:08.243 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T04:14:08.243 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:08.243 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 198, 2024-09-21T04:14:08.243 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.009986114, 2024-09-21T04:14:08.243 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000050434 2024-09-21T04:14:08.243 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:08.243 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:08.243 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-21T04:14:08.243 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-21T04:14:08.244 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-21T04:14:08.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:08.244 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 1142704, 2024-09-21T04:14:08.244 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 122608 2024-09-21T04:14:08.244 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:08.244 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T04:14:08.244 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 5, 2024-09-21T04:14:08.244 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 154, 2024-09-21T04:14:08.244 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 3152, 2024-09-21T04:14:08.244 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 107440, 2024-09-21T04:14:08.244 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 3, 2024-09-21T04:14:08.244 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 2, 2024-09-21T04:14:08.244 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.015920141, 2024-09-21T04:14:08.244 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.011195848, 2024-09-21T04:14:08.245 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.003330302, 2024-09-21T04:14:08.245 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T04:14:08.245 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:08.245 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 153, 2024-09-21T04:14:08.245 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.024781803, 2024-09-21T04:14:08.245 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000161972 2024-09-21T04:14:08.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:08.245 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:08.245 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-21T04:14:08.245 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-21T04:14:08.245 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-21T04:14:08.245 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:08.245 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-21T04:14:08.246 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 107440 2024-09-21T04:14:08.246 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:08.246 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-09-21T04:14:08.246 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 0, 2024-09-21T04:14:08.246 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-09-21T04:14:08.246 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:08.246 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-09-21T04:14:08.246 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-09-21T04:14:08.246 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-09-21T04:14:08.246 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T04:14:08.246 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-09-21T04:14:08.246 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T04:14:08.246 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-09-21T04:14:08.247 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T04:14:08.247 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T04:14:08.247 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T04:14:08.247 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T04:14:08.247 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T04:14:08.247 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-09-21T04:14:08.247 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T04:14:08.247 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T04:14:08.247 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-09-21T04:14:08.247 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-09-21T04:14:08.247 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-09-21T04:14:08.247 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-09-21T04:14:08.247 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T04:14:08.248 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T04:14:08.248 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T04:14:08.248 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T04:14:08.248 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T04:14:08.248 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-09-21T04:14:08.248 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-09-21T04:14:08.248 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-09-21T04:14:08.248 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T04:14:08.248 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T04:14:08.248 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-09-21T04:14:08.248 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-09-21T04:14:08.248 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-09-21T04:14:08.248 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-09-21T04:14:08.248 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T04:14:08.249 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-09-21T04:14:08.249 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T04:14:08.249 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-09-21T04:14:08.249 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 149047, 2024-09-21T04:14:08.249 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 31, 2024-09-21T04:14:08.249 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-09-21T04:14:08.249 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-09-21T04:14:08.249 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-09-21T04:14:08.249 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-09-21T04:14:08.249 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-09-21T04:14:08.249 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-09-21T04:14:08.249 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-09-21T04:14:08.249 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-09-21T04:14:08.249 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 0, 2024-09-21T04:14:08.249 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 0, 2024-09-21T04:14:08.249 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T04:14:08.250 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-09-21T04:14:08.250 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-09-21T04:14:08.250 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-09-21T04:14:08.250 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 968, 2024-09-21T04:14:08.250 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 2, 2024-09-21T04:14:08.250 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-09-21T04:14:08.250 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-09-21T04:14:08.250 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-09-21T04:14:08.250 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-09-21T04:14:08.250 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:08.250 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T04:14:08.250 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-21T04:14:08.250 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-09-21T04:14:08.251 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-21T04:14:08.251 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 207, 2024-09-21T04:14:08.251 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 1113600, 2024-09-21T04:14:08.251 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:08.251 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 207, 2024-09-21T04:14:08.251 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-09-21T04:14:08.251 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-09-21T04:14:08.251 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 207, 2024-09-21T04:14:08.251 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 1113600, 2024-09-21T04:14:08.251 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-21T04:14:08.251 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-21T04:14:08.251 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-21T04:14:08.251 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-21T04:14:08.251 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:08.251 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:08.252 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:14:08.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:08 smithi119 ceph-mon[34811]: pgmap v632: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-21T04:14:08.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:08 smithi064 ceph-mon[31180]: pgmap v632: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-21T04:14:08.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:08 smithi131 ceph-mon[35193]: pgmap v632: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-09-21T04:14:08.972 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-21T04:14:08.972 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-21T04:14:08.972 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-09-21T04:14:08.972 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-09-21T04:14:08.972 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-09-21T04:14:08.973 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-09-21T04:14:08.973 DEBUG:tasks.check_counter:Getting stats from j 2024-09-21T04:14:08.973 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph daemon mds.j perf dump 2024-09-21T04:14:09.430 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:14:09.730 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:09 smithi064 ceph-mon[31180]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T04:14:09.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:09 smithi119 ceph-mon[34811]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T04:14:09.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:09 smithi131 ceph-mon[35193]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T04:14:10.148 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-09-21T04:14:10.148 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T04:14:10.148 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 4, 2024-09-21T04:14:10.148 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 4, 2024-09-21T04:14:10.148 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 2992, 2024-09-21T04:14:10.149 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 448, 2024-09-21T04:14:10.149 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-09-21T04:14:10.149 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 0, 2024-09-21T04:14:10.149 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.003203674, 2024-09-21T04:14:10.149 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.000054291, 2024-09-21T04:14:10.149 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.002249565, 2024-09-21T04:14:10.149 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T04:14:10.149 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:10.149 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 3, 2024-09-21T04:14:10.149 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.004011382, 2024-09-21T04:14:10.149 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.001337127 2024-09-21T04:14:10.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:10.149 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:10.149 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-21T04:14:10.150 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-21T04:14:10.150 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-21T04:14:10.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:10.150 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-21T04:14:10.150 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T04:14:10.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:10.150 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T04:14:10.150 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 1, 2024-09-21T04:14:10.150 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 156, 2024-09-21T04:14:10.150 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 160, 2024-09-21T04:14:10.150 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 107696, 2024-09-21T04:14:10.150 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 3, 2024-09-21T04:14:10.150 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 2, 2024-09-21T04:14:10.151 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.015226389, 2024-09-21T04:14:10.151 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.012372740, 2024-09-21T04:14:10.151 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.001209495, 2024-09-21T04:14:10.151 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T04:14:10.151 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:10.151 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 154, 2024-09-21T04:14:10.151 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.007484619, 2024-09-21T04:14:10.151 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000048601 2024-09-21T04:14:10.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:10.151 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:10.151 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-21T04:14:10.151 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-21T04:14:10.152 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-21T04:14:10.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:10.152 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T04:14:10.152 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 107696 2024-09-21T04:14:10.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:10.152 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T04:14:10.152 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 212, 2024-09-21T04:14:10.152 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 205, 2024-09-21T04:14:10.152 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 1144400, 2024-09-21T04:14:10.152 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 125984, 2024-09-21T04:14:10.152 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 2, 2024-09-21T04:14:10.152 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 1, 2024-09-21T04:14:10.152 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.044892014, 2024-09-21T04:14:10.152 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.017188913, 2024-09-21T04:14:10.153 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.038135340, 2024-09-21T04:14:10.153 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T04:14:10.153 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:10.153 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 204, 2024-09-21T04:14:10.153 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.018337114, 2024-09-21T04:14:10.153 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000089887 2024-09-21T04:14:10.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:10.153 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:10.153 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-21T04:14:10.153 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-21T04:14:10.153 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-21T04:14:10.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:10.153 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 1144400, 2024-09-21T04:14:10.154 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 125984 2024-09-21T04:14:10.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:10.154 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-09-21T04:14:10.154 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 0, 2024-09-21T04:14:10.154 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-09-21T04:14:10.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:10.154 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-09-21T04:14:10.154 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-09-21T04:14:10.154 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-09-21T04:14:10.154 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T04:14:10.154 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-09-21T04:14:10.154 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T04:14:10.154 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-09-21T04:14:10.154 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T04:14:10.155 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T04:14:10.155 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T04:14:10.155 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T04:14:10.155 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T04:14:10.155 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-09-21T04:14:10.155 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T04:14:10.155 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T04:14:10.155 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-09-21T04:14:10.155 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-09-21T04:14:10.155 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-09-21T04:14:10.155 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-09-21T04:14:10.155 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T04:14:10.155 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T04:14:10.155 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T04:14:10.156 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T04:14:10.156 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T04:14:10.156 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-09-21T04:14:10.156 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-09-21T04:14:10.156 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-09-21T04:14:10.156 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T04:14:10.156 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T04:14:10.156 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-09-21T04:14:10.156 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-09-21T04:14:10.156 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-09-21T04:14:10.156 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-09-21T04:14:10.156 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T04:14:10.156 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-09-21T04:14:10.156 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T04:14:10.157 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-09-21T04:14:10.157 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 148967, 2024-09-21T04:14:10.157 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 29, 2024-09-21T04:14:10.157 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-09-21T04:14:10.157 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-09-21T04:14:10.157 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-09-21T04:14:10.157 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-09-21T04:14:10.157 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-09-21T04:14:10.157 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-09-21T04:14:10.157 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-09-21T04:14:10.157 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-09-21T04:14:10.157 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 0, 2024-09-21T04:14:10.157 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 0, 2024-09-21T04:14:10.158 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T04:14:10.158 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-09-21T04:14:10.158 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-09-21T04:14:10.158 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-09-21T04:14:10.158 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 968, 2024-09-21T04:14:10.158 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 2, 2024-09-21T04:14:10.158 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-09-21T04:14:10.158 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-09-21T04:14:10.158 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-09-21T04:14:10.158 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-09-21T04:14:10.158 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-09-21T04:14:10.158 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T04:14:10.158 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-09-21T04:14:10.158 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-09-21T04:14:10.159 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-09-21T04:14:10.159 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 213, 2024-09-21T04:14:10.159 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 1114444, 2024-09-21T04:14:10.159 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:10.159 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 213, 2024-09-21T04:14:10.159 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-09-21T04:14:10.159 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-09-21T04:14:10.159 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 213, 2024-09-21T04:14:10.159 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 1114444, 2024-09-21T04:14:10.159 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-09-21T04:14:10.159 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-09-21T04:14:10.159 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-09-21T04:14:10.159 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-09-21T04:14:10.159 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:10.160 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-09-21T04:14:10.160 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-09-21T04:14:10.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:10 smithi119 ceph-mon[34811]: pgmap v634: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 29 op/s 2024-09-21T04:14:10.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:10 smithi119 ceph-mon[34811]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:14:10.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:10 smithi064 ceph-mon[31180]: pgmap v634: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 29 op/s 2024-09-21T04:14:10.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:10 smithi064 ceph-mon[31180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:14:10.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:10 smithi131 ceph-mon[35193]: pgmap v634: 97 pgs: 97 active+clean; 759 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 29 op/s 2024-09-21T04:14:10.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:10 smithi131 ceph-mon[35193]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:14:11.167 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-21T04:14:11.167 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-21T04:14:11.167 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-21T04:14:11.167 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-21T04:14:11.167 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-21T04:14:11.167 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-09-21T04:14:11.167 DEBUG:tasks.check_counter:Getting stats from b 2024-09-21T04:14:11.167 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph daemon mds.b perf dump 2024-09-21T04:14:11.693 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T04:14:12.712 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:12 smithi119 ceph-mon[34811]: pgmap v635: 97 pgs: 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 93 active+clean; 758 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 29 op/s 2024-09-21T04:14:12.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:12 smithi064 ceph-mon[31180]: pgmap v635: 97 pgs: 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 93 active+clean; 758 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 29 op/s 2024-09-21T04:14:12.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:12 smithi131 ceph-mon[35193]: pgmap v635: 97 pgs: 2 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 93 active+clean; 758 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 29 op/s 2024-09-21T04:14:12.951 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-21T04:14:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T04:14:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 8208, 2024-09-21T04:14:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 11212, 2024-09-21T04:14:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 3040986, 2024-09-21T04:14:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 19539180, 2024-09-21T04:14:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 160, 2024-09-21T04:14:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 6, 2024-09-21T04:14:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.899283034, 2024-09-21T04:14:12.951 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.461347256, 2024-09-21T04:14:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.360126398, 2024-09-21T04:14:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.091068684, 2024-09-21T04:14:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 11213, 2024-09-21T04:14:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 1.878106415, 2024-09-21T04:14:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000167493 2024-09-21T04:14:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 12384, 2024-09-21T04:14:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.002255023, 2024-09-21T04:14:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000182 2024-09-21T04:14:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 33264, 2024-09-21T04:14:12.952 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 4400 2024-09-21T04:14:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T04:14:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 5733, 2024-09-21T04:14:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 5181, 2024-09-21T04:14:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 2550480, 2024-09-21T04:14:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 8484126, 2024-09-21T04:14:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 11, 2024-09-21T04:14:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 6, 2024-09-21T04:14:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.500088466, 2024-09-21T04:14:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.213909010, 2024-09-21T04:14:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.157313338, 2024-09-21T04:14:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.124146842, 2024-09-21T04:14:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 5182, 2024-09-21T04:14:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 1.999968885, 2024-09-21T04:14:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000385945 2024-09-21T04:14:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 3174, 2024-09-21T04:14:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000842480, 2024-09-21T04:14:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000265 2024-09-21T04:14:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 325072, 2024-09-21T04:14:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 459136 2024-09-21T04:14:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T04:14:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 43809, 2024-09-21T04:14:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 37287, 2024-09-21T04:14:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 16357674, 2024-09-21T04:14:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 23644181, 2024-09-21T04:14:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 40, 2024-09-21T04:14:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 8, 2024-09-21T04:14:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 2.235698641, 2024-09-21T04:14:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 1.219106433, 2024-09-21T04:14:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 3.234619320, 2024-09-21T04:14:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.120126549, 2024-09-21T04:14:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 37281, 2024-09-21T04:14:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 3.764967220, 2024-09-21T04:14:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000100988 2024-09-21T04:14:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 62967, 2024-09-21T04:14:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.009722932, 2024-09-21T04:14:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000154 2024-09-21T04:14:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 1328176, 2024-09-21T04:14:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 187328 2024-09-21T04:14:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: "cct": { 2024-09-21T04:14:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: "total_workers": 1, 2024-09-21T04:14:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: "unhealthy_workers": 0 2024-09-21T04:14:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-MDSRank": { 2024-09-21T04:14:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-09-21T04:14:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-09-21T04:14:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 4426, 2024-09-21T04:14:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 8.467418271, 2024-09-21T04:14:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.001913108 2024-09-21T04:14:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-PurgeQueue": { 2024-09-21T04:14:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-09-21T04:14:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-09-21T04:14:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2420, 2024-09-21T04:14:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.265000181, 2024-09-21T04:14:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000109504 2024-09-21T04:14:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-09-21T04:14:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: "request": 10492, 2024-09-21T04:14:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: "reply": 10311, 2024-09-21T04:14:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: "reply_latency": { 2024-09-21T04:14:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 10311, 2024-09-21T04:14:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 112.541075592, 2024-09-21T04:14:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.010914661 2024-09-21T04:14:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_reply": 0, 2024-09-21T04:14:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: "forward": 181, 2024-09-21T04:14:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_complete": 28, 2024-09-21T04:14:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_keys": 10, 2024-09-21T04:14:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_commit": 848, 2024-09-21T04:14:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_split": 7, 2024-09-21T04:14:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_merge": 1, 2024-09-21T04:14:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes": 349, 2024-09-21T04:14:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_top": 4, 2024-09-21T04:14:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_bottom": 135, 2024-09-21T04:14:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pin_tail": 210, 2024-09-21T04:14:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pinned": 343, 2024-09-21T04:14:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_expired": 2128, 2024-09-21T04:14:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_with_caps": 3, 2024-09-21T04:14:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": 3, 2024-09-21T04:14:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: "subtrees": 4, 2024-09-21T04:14:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse": 20719, 2024-09-21T04:14:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_hit": 18785, 2024-09-21T04:14:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_forward": 180, 2024-09-21T04:14:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_discover": 9, 2024-09-21T04:14:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_dir_fetch": 0, 2024-09-21T04:14:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_remote_ino": 0, 2024-09-21T04:14:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_lock": 191, 2024-09-21T04:14:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: "load_cent": 0, 2024-09-21T04:14:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: "q": 0, 2024-09-21T04:14:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: "exported": 52, 2024-09-21T04:14:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: "exported_inodes": 1891, 2024-09-21T04:14:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: "imported": 4, 2024-09-21T04:14:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: "imported_inodes": 424, 2024-09-21T04:14:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_dir_fetch": 0, 2024-09-21T04:14:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_backtrace_fetch": 3147, 2024-09-21T04:14:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_peer_discover": 0, 2024-09-21T04:14:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rfiles": 4, 2024-09-21T04:14:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rbytes": 234, 2024-09-21T04:14:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rsnaps": 6, 2024-09-21T04:14:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_fetch": 3147, 2024-09-21T04:14:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_set_tag": 0, 2024-09-21T04:14:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T04:14:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_inotable_repaired": 0, 2024-09-21T04:14:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_inodes": 146, 2024-09-21T04:14:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_base_inodes": 14, 2024-09-21T04:14:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dirfrag_rstats": 132, 2024-09-21T04:14:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_file_inodes": 3001, 2024-09-21T04:14:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_inode_file_caps": 105, 2024-09-21T04:14:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_revoke": 7468, 2024-09-21T04:14:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_grant": 7414, 2024-09-21T04:14:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_trunc": 5, 2024-09-21T04:14:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T04:14:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T04:14:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps": 25384, 2024-09-21T04:14:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps_dirty": 1641, 2024-09-21T04:14:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_cap_release": 185, 2024-09-21T04:14:12.961 INFO:teuthology.orchestra.run.smithi119.stdout: "process_request_cap_release": 4746 2024-09-21T04:14:12.962 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.962 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_cache": { 2024-09-21T04:14:12.962 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update": 1, 2024-09-21T04:14:12.962 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update_receipt": 3, 2024-09-21T04:14:12.962 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_try_discover": 2, 2024-09-21T04:14:12.962 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_send_discover": 11, 2024-09-21T04:14:12.962 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_handle_discover": 69, 2024-09-21T04:14:12.962 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays": 6, 2024-09-21T04:14:12.962 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_delayed": 0, 2024-09-21T04:14:12.962 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_enqueuing": 0, 2024-09-21T04:14:12.962 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_created": 1440, 2024-09-21T04:14:12.962 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_enqueued": 1434, 2024-09-21T04:14:12.962 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_reintegrated": 0, 2024-09-21T04:14:12.963 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_migrated": 0, 2024-09-21T04:14:12.963 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_processing": 0, 2024-09-21T04:14:12.963 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_enqueued": 0, 2024-09-21T04:14:12.963 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_prioritized": 0, 2024-09-21T04:14:12.963 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_started": 0, 2024-09-21T04:14:12.963 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_completed": 0, 2024-09-21T04:14:12.963 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_quiesce_path": 0, 2024-09-21T04:14:12.963 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_quiesce_inode": 0, 2024-09-21T04:14:12.963 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_enqueue_scrub": 14, 2024-09-21T04:14:12.963 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_exportdir": 52, 2024-09-21T04:14:12.963 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_flush": 0, 2024-09-21T04:14:12.963 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragmentdir": 8, 2024-09-21T04:14:12.963 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragstats": 0, 2024-09-21T04:14:12.963 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_inodestats": 0, 2024-09-21T04:14:12.963 INFO:teuthology.orchestra.run.smithi119.stdout: "uninline_started": 0, 2024-09-21T04:14:12.964 INFO:teuthology.orchestra.run.smithi119.stdout: "uninline_succeeded": 0, 2024-09-21T04:14:12.964 INFO:teuthology.orchestra.run.smithi119.stdout: "uninline_write_failed": 0 2024-09-21T04:14:12.964 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.964 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_log": { 2024-09-21T04:14:12.964 INFO:teuthology.orchestra.run.smithi119.stdout: "evlrg": 0, 2024-09-21T04:14:12.964 INFO:teuthology.orchestra.run.smithi119.stdout: "evadd": 10183, 2024-09-21T04:14:12.964 INFO:teuthology.orchestra.run.smithi119.stdout: "evex": 9725, 2024-09-21T04:14:12.964 INFO:teuthology.orchestra.run.smithi119.stdout: "evtrm": 9725, 2024-09-21T04:14:12.964 INFO:teuthology.orchestra.run.smithi119.stdout: "ev": 458, 2024-09-21T04:14:12.964 INFO:teuthology.orchestra.run.smithi119.stdout: "evexg": 0, 2024-09-21T04:14:12.964 INFO:teuthology.orchestra.run.smithi119.stdout: "evexd": 708, 2024-09-21T04:14:12.964 INFO:teuthology.orchestra.run.smithi119.stdout: "segadd": 41, 2024-09-21T04:14:12.964 INFO:teuthology.orchestra.run.smithi119.stdout: "segex": 40, 2024-09-21T04:14:12.964 INFO:teuthology.orchestra.run.smithi119.stdout: "segtrm": 40, 2024-09-21T04:14:12.964 INFO:teuthology.orchestra.run.smithi119.stdout: "seg": 1, 2024-09-21T04:14:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: "segmjr": 1, 2024-09-21T04:14:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: "segexg": 0, 2024-09-21T04:14:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: "segexd": 1, 2024-09-21T04:14:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: "expos": 23226917, 2024-09-21T04:14:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: "wrpos": 24272685, 2024-09-21T04:14:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: "rdpos": 0, 2024-09-21T04:14:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: "jlat": { 2024-09-21T04:14:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2149, 2024-09-21T04:14:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 41.444499701, 2024-09-21T04:14:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.019285481 2024-09-21T04:14:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: "replayed": 0 2024-09-21T04:14:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_mem": { 2024-09-21T04:14:12.965 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 88, 2024-09-21T04:14:12.966 INFO:teuthology.orchestra.run.smithi119.stdout: "ino+": 3041, 2024-09-21T04:14:12.966 INFO:teuthology.orchestra.run.smithi119.stdout: "ino-": 2953, 2024-09-21T04:14:12.966 INFO:teuthology.orchestra.run.smithi119.stdout: "dir": 27, 2024-09-21T04:14:12.966 INFO:teuthology.orchestra.run.smithi119.stdout: "dir+": 141, 2024-09-21T04:14:12.966 INFO:teuthology.orchestra.run.smithi119.stdout: "dir-": 114, 2024-09-21T04:14:12.966 INFO:teuthology.orchestra.run.smithi119.stdout: "dn": 349, 2024-09-21T04:14:12.966 INFO:teuthology.orchestra.run.smithi119.stdout: "dn+": 4049, 2024-09-21T04:14:12.966 INFO:teuthology.orchestra.run.smithi119.stdout: "dn-": 3700, 2024-09-21T04:14:12.966 INFO:teuthology.orchestra.run.smithi119.stdout: "cap": 3, 2024-09-21T04:14:12.966 INFO:teuthology.orchestra.run.smithi119.stdout: "cap+": 1835, 2024-09-21T04:14:12.966 INFO:teuthology.orchestra.run.smithi119.stdout: "cap-": 1832, 2024-09-21T04:14:12.966 INFO:teuthology.orchestra.run.smithi119.stdout: "rss": 143708, 2024-09-21T04:14:12.966 INFO:teuthology.orchestra.run.smithi119.stdout: "heap": 363737088 2024-09-21T04:14:12.966 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_server": { 2024-09-21T04:14:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_client_request": 12381, 2024-09-21T04:14:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_server_request": 64, 2024-09-21T04:14:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_request": 10492, 2024-09-21T04:14:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_session": 31, 2024-09-21T04:14:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_peer_request": 331, 2024-09-21T04:14:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: "req_create_latency": { 2024-09-21T04:14:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 1275, 2024-09-21T04:14:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 2.096932967, 2024-09-21T04:14:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.001644653 2024-09-21T04:14:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getattr_latency": { 2024-09-21T04:14:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2842, 2024-09-21T04:14:12.967 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 42.011829676, 2024-09-21T04:14:12.968 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.014782487 2024-09-21T04:14:12.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.968 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getfilelock_latency": { 2024-09-21T04:14:12.968 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:12.968 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:12.968 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:12.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.968 INFO:teuthology.orchestra.run.smithi119.stdout: "req_link_latency": { 2024-09-21T04:14:12.968 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:12.968 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:12.968 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:12.968 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.968 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookup_latency": { 2024-09-21T04:14:12.968 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 3481, 2024-09-21T04:14:12.968 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 28.622512159, 2024-09-21T04:14:12.969 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.008222497 2024-09-21T04:14:12.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.969 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookuphash_latency": { 2024-09-21T04:14:12.969 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:12.969 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:12.969 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:12.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.969 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupino_latency": { 2024-09-21T04:14:12.969 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 1, 2024-09-21T04:14:12.969 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000186881, 2024-09-21T04:14:12.969 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000186881 2024-09-21T04:14:12.969 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.969 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupname_latency": { 2024-09-21T04:14:12.969 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:12.970 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:12.970 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:12.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.970 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupparent_latency": { 2024-09-21T04:14:12.970 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:12.970 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:12.970 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:12.970 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.970 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupsnap_latency": { 2024-09-21T04:14:12.971 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:12.971 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:12.971 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:12.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.971 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lssnap_latency": { 2024-09-21T04:14:12.971 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 11, 2024-09-21T04:14:12.971 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.002973183, 2024-09-21T04:14:12.971 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000270289 2024-09-21T04:14:12.971 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.971 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mkdir_latency": { 2024-09-21T04:14:12.971 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 16, 2024-09-21T04:14:12.971 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.018575118, 2024-09-21T04:14:12.972 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.001160944 2024-09-21T04:14:12.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.972 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mknod_latency": { 2024-09-21T04:14:12.972 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:12.972 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:12.972 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:12.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.972 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mksnap_latency": { 2024-09-21T04:14:12.972 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 11, 2024-09-21T04:14:12.972 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 17.487891873, 2024-09-21T04:14:12.972 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 1.589808352 2024-09-21T04:14:12.972 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.972 INFO:teuthology.orchestra.run.smithi119.stdout: "req_open_latency": { 2024-09-21T04:14:12.972 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 838, 2024-09-21T04:14:12.972 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 1.018180384, 2024-09-21T04:14:12.973 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.001215012 2024-09-21T04:14:12.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.973 INFO:teuthology.orchestra.run.smithi119.stdout: "req_readdir_latency": { 2024-09-21T04:14:12.973 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 113, 2024-09-21T04:14:12.973 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.496742713, 2024-09-21T04:14:12.973 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.004395953 2024-09-21T04:14:12.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.973 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rename_latency": { 2024-09-21T04:14:12.973 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 923, 2024-09-21T04:14:12.973 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 11.085859345, 2024-09-21T04:14:12.973 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.012010681 2024-09-21T04:14:12.973 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.973 INFO:teuthology.orchestra.run.smithi119.stdout: "req_renamesnap_latency": { 2024-09-21T04:14:12.973 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:12.973 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:12.973 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: "req_snapdiff_latency": { 2024-09-21T04:14:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmdir_latency": { 2024-09-21T04:14:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 10, 2024-09-21T04:14:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.113362356, 2024-09-21T04:14:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.011336235 2024-09-21T04:14:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmsnap_latency": { 2024-09-21T04:14:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 5, 2024-09-21T04:14:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 8.358085564, 2024-09-21T04:14:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 1.671617112 2024-09-21T04:14:12.974 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmxattr_latency": { 2024-09-21T04:14:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setattr_latency": { 2024-09-21T04:14:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 6, 2024-09-21T04:14:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.031338037, 2024-09-21T04:14:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.005223006 2024-09-21T04:14:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setdirlayout_latency": { 2024-09-21T04:14:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.975 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setfilelock_latency": { 2024-09-21T04:14:12.976 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:12.976 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:12.976 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:12.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.976 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setlayout_latency": { 2024-09-21T04:14:12.976 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:12.976 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:12.976 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:12.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.976 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setxattr_latency": { 2024-09-21T04:14:12.976 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 10, 2024-09-21T04:14:12.976 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.024850029, 2024-09-21T04:14:12.976 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.002485002 2024-09-21T04:14:12.976 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.976 INFO:teuthology.orchestra.run.smithi119.stdout: "req_symlink_latency": { 2024-09-21T04:14:12.976 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 3, 2024-09-21T04:14:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.001886889, 2024-09-21T04:14:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000628963 2024-09-21T04:14:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: "req_unlink_latency": { 2024-09-21T04:14:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 766, 2024-09-21T04:14:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 1.169868418, 2024-09-21T04:14:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.001527243 2024-09-21T04:14:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_revoke_eviction": 0, 2024-09-21T04:14:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_acquisition_throttle": 0, 2024-09-21T04:14:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getvxattr_latency": { 2024-09-21T04:14:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:12.977 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.978 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_sessions": { 2024-09-21T04:14:12.978 INFO:teuthology.orchestra.run.smithi119.stdout: "session_count": 2, 2024-09-21T04:14:12.978 INFO:teuthology.orchestra.run.smithi119.stdout: "session_add": 4, 2024-09-21T04:14:12.978 INFO:teuthology.orchestra.run.smithi119.stdout: "session_remove": 2, 2024-09-21T04:14:12.978 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_open": 2, 2024-09-21T04:14:12.978 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_stale": 0, 2024-09-21T04:14:12.978 INFO:teuthology.orchestra.run.smithi119.stdout: "total_load": 312, 2024-09-21T04:14:12.978 INFO:teuthology.orchestra.run.smithi119.stdout: "average_load": 156, 2024-09-21T04:14:12.978 INFO:teuthology.orchestra.run.smithi119.stdout: "avg_session_uptime": 617, 2024-09-21T04:14:12.978 INFO:teuthology.orchestra.run.smithi119.stdout: "mdthresh_evicted": 0 2024-09-21T04:14:12.978 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.978 INFO:teuthology.orchestra.run.smithi119.stdout: "mempool": { 2024-09-21T04:14:12.978 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_bytes": 120, 2024-09-21T04:14:12.978 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_items": 120, 2024-09-21T04:14:12.979 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T04:14:12.979 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_items": 0, 2024-09-21T04:14:12.979 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T04:14:12.979 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_items": 0, 2024-09-21T04:14:12.979 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T04:14:12.979 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T04:14:12.979 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T04:14:12.979 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T04:14:12.979 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T04:14:12.979 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_items": 0, 2024-09-21T04:14:12.979 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T04:14:12.979 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T04:14:12.979 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_bytes": 0, 2024-09-21T04:14:12.979 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_items": 0, 2024-09-21T04:14:12.979 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_bytes": 0, 2024-09-21T04:14:12.980 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_items": 0, 2024-09-21T04:14:12.980 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T04:14:12.980 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T04:14:12.980 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T04:14:12.980 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T04:14:12.980 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T04:14:12.980 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_items": 0, 2024-09-21T04:14:12.980 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_bytes": 0, 2024-09-21T04:14:12.980 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_items": 0, 2024-09-21T04:14:12.980 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T04:14:12.980 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T04:14:12.980 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_bytes": 0, 2024-09-21T04:14:12.980 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_items": 0, 2024-09-21T04:14:12.980 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_bytes": 0, 2024-09-21T04:14:12.980 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_items": 0, 2024-09-21T04:14:12.981 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T04:14:12.981 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_items": 0, 2024-09-21T04:14:12.981 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T04:14:12.981 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_items": 0, 2024-09-21T04:14:12.981 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_bytes": 376162, 2024-09-21T04:14:12.981 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_items": 202, 2024-09-21T04:14:12.981 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_bytes": 616, 2024-09-21T04:14:12.981 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_items": 7, 2024-09-21T04:14:12.981 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_bytes": 0, 2024-09-21T04:14:12.981 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_items": 0, 2024-09-21T04:14:12.981 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_bytes": 0, 2024-09-21T04:14:12.981 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_items": 0, 2024-09-21T04:14:12.981 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_bytes": 0, 2024-09-21T04:14:12.981 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_items": 0, 2024-09-21T04:14:12.982 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_bytes": 8664, 2024-09-21T04:14:12.982 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_items": 191, 2024-09-21T04:14:12.982 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T04:14:12.982 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_items": 0, 2024-09-21T04:14:12.982 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_bytes": 0, 2024-09-21T04:14:12.982 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_items": 0, 2024-09-21T04:14:12.982 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_bytes": 562533, 2024-09-21T04:14:12.982 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_items": 6121, 2024-09-21T04:14:12.982 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_bytes": 0, 2024-09-21T04:14:12.982 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_items": 0, 2024-09-21T04:14:12.982 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_bytes": 0, 2024-09-21T04:14:12.982 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_items": 0 2024-09-21T04:14:12.982 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.983 INFO:teuthology.orchestra.run.smithi119.stdout: "objecter": { 2024-09-21T04:14:12.983 INFO:teuthology.orchestra.run.smithi119.stdout: "op_active": 0, 2024-09-21T04:14:12.983 INFO:teuthology.orchestra.run.smithi119.stdout: "op_laggy": 0, 2024-09-21T04:14:12.983 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send": 10267, 2024-09-21T04:14:12.983 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send_bytes": 25499222, 2024-09-21T04:14:12.983 INFO:teuthology.orchestra.run.smithi119.stdout: "op_resend": 37, 2024-09-21T04:14:12.983 INFO:teuthology.orchestra.run.smithi119.stdout: "op_reply": 10267, 2024-09-21T04:14:12.983 INFO:teuthology.orchestra.run.smithi119.stdout: "op_latency": { 2024-09-21T04:14:12.983 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 10267, 2024-09-21T04:14:12.983 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 121.614946640, 2024-09-21T04:14:12.983 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.011845227 2024-09-21T04:14:12.983 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.983 INFO:teuthology.orchestra.run.smithi119.stdout: "op_inflight": 0, 2024-09-21T04:14:12.983 INFO:teuthology.orchestra.run.smithi119.stdout: "oplen_avg": { 2024-09-21T04:14:12.984 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 10267, 2024-09-21T04:14:12.984 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 15169 2024-09-21T04:14:12.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.984 INFO:teuthology.orchestra.run.smithi119.stdout: "op": 10267, 2024-09-21T04:14:12.984 INFO:teuthology.orchestra.run.smithi119.stdout: "op_r": 3331, 2024-09-21T04:14:12.984 INFO:teuthology.orchestra.run.smithi119.stdout: "op_w": 6936, 2024-09-21T04:14:12.984 INFO:teuthology.orchestra.run.smithi119.stdout: "op_rmw": 0, 2024-09-21T04:14:12.984 INFO:teuthology.orchestra.run.smithi119.stdout: "op_pg": 0, 2024-09-21T04:14:12.984 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_stat": 766, 2024-09-21T04:14:12.984 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_create": 1509, 2024-09-21T04:14:12.984 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_read": 99, 2024-09-21T04:14:12.984 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_write": 2234, 2024-09-21T04:14:12.984 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writefull": 169, 2024-09-21T04:14:12.985 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writesame": 0, 2024-09-21T04:14:12.985 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_append": 0, 2024-09-21T04:14:12.985 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_zero": 0, 2024-09-21T04:14:12.985 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_truncate": 2, 2024-09-21T04:14:12.985 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_delete": 1961, 2024-09-21T04:14:12.985 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_mapext": 0, 2024-09-21T04:14:12.985 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_sparse_read": 0, 2024-09-21T04:14:12.985 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_clonerange": 0, 2024-09-21T04:14:12.985 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_getxattr": 3189, 2024-09-21T04:14:12.985 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_setxattr": 3018, 2024-09-21T04:14:12.985 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_cmpxattr": 0, 2024-09-21T04:14:12.985 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_rmxattr": 0, 2024-09-21T04:14:12.985 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_resetxattrs": 0, 2024-09-21T04:14:12.985 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_call": 0, 2024-09-21T04:14:12.986 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_watch": 0, 2024-09-21T04:14:12.986 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_notify": 0, 2024-09-21T04:14:12.986 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T04:14:12.986 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls": 0, 2024-09-21T04:14:12.986 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls_filter": 0, 2024-09-21T04:14:12.986 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_other": 5, 2024-09-21T04:14:12.986 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_active": 0, 2024-09-21T04:14:12.986 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_send": 0, 2024-09-21T04:14:12.986 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_resend": 0, 2024-09-21T04:14:12.986 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_ping": 0, 2024-09-21T04:14:12.986 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_active": 0, 2024-09-21T04:14:12.986 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_send": 0, 2024-09-21T04:14:12.986 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_resend": 0, 2024-09-21T04:14:12.987 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_active": 0, 2024-09-21T04:14:12.987 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_send": 0, 2024-09-21T04:14:12.987 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_resend": 0, 2024-09-21T04:14:12.987 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_active": 0, 2024-09-21T04:14:12.987 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_send": 0, 2024-09-21T04:14:12.987 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_resend": 0, 2024-09-21T04:14:12.987 INFO:teuthology.orchestra.run.smithi119.stdout: "command_active": 0, 2024-09-21T04:14:12.987 INFO:teuthology.orchestra.run.smithi119.stdout: "command_send": 0, 2024-09-21T04:14:12.987 INFO:teuthology.orchestra.run.smithi119.stdout: "command_resend": 0, 2024-09-21T04:14:12.987 INFO:teuthology.orchestra.run.smithi119.stdout: "map_epoch": 86, 2024-09-21T04:14:12.987 INFO:teuthology.orchestra.run.smithi119.stdout: "map_full": 0, 2024-09-21T04:14:12.987 INFO:teuthology.orchestra.run.smithi119.stdout: "map_inc": 13, 2024-09-21T04:14:12.987 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_sessions": 12, 2024-09-21T04:14:12.987 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_open": 12, 2024-09-21T04:14:12.988 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_close": 0, 2024-09-21T04:14:12.988 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_laggy": 0, 2024-09-21T04:14:12.988 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_wr": 1859, 2024-09-21T04:14:12.988 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_rd": 109, 2024-09-21T04:14:12.988 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_del": 249 2024-09-21T04:14:12.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.988 INFO:teuthology.orchestra.run.smithi119.stdout: "oft": { 2024-09-21T04:14:12.988 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_objs": 1, 2024-09-21T04:14:12.988 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_kv_pairs": 0, 2024-09-21T04:14:12.988 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_updates": 0, 2024-09-21T04:14:12.988 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_removes": 0 2024-09-21T04:14:12.988 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.988 INFO:teuthology.orchestra.run.smithi119.stdout: "purge_queue": { 2024-09-21T04:14:12.988 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops": 0, 2024-09-21T04:14:12.989 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops_high_water": 10, 2024-09-21T04:14:12.989 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing": 0, 2024-09-21T04:14:12.989 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_high_water": 10, 2024-09-21T04:14:12.989 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed_ops": 1434, 2024-09-21T04:14:12.989 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed": 1434, 2024-09-21T04:14:12.989 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_item_in_journal": 0 2024-09-21T04:14:12.989 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.989 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T04:14:12.989 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T04:14:12.989 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-09-21T04:14:12.989 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T04:14:12.989 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 57746, 2024-09-21T04:14:12.989 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 16689643, 2024-09-21T04:14:12.989 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:12.989 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 57746, 2024-09-21T04:14:12.990 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-09-21T04:14:12.990 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-09-21T04:14:12.990 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 57746, 2024-09-21T04:14:12.990 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 16689643, 2024-09-21T04:14:12.990 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T04:14:12.990 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:12.990 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:12.990 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:12.990 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:12.990 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.990 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_bytes": { 2024-09-21T04:14:12.990 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T04:14:12.990 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-09-21T04:14:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T04:14:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-09-21T04:14:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-09-21T04:14:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-09-21T04:14:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 10267, 2024-09-21T04:14:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 25663597, 2024-09-21T04:14:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 10267, 2024-09-21T04:14:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 25663597, 2024-09-21T04:14:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T04:14:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:12.991 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_ops": { 2024-09-21T04:14:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T04:14:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 1024, 2024-09-21T04:14:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T04:14:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-09-21T04:14:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-09-21T04:14:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-09-21T04:14:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 10267, 2024-09-21T04:14:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 10267, 2024-09-21T04:14:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 10267, 2024-09-21T04:14:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 10267, 2024-09-21T04:14:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T04:14:12.992 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle": { 2024-09-21T04:14:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T04:14:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-09-21T04:14:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T04:14:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 1434, 2024-09-21T04:14:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 165962, 2024-09-21T04:14:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 1434, 2024-09-21T04:14:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-09-21T04:14:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-09-21T04:14:12.993 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 85, 2024-09-21T04:14:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 165962, 2024-09-21T04:14:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T04:14:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle-0x56375d9b7380": { 2024-09-21T04:14:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T04:14:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-09-21T04:14:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T04:14:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 10183, 2024-09-21T04:14:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 20078381, 2024-09-21T04:14:12.994 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 10183, 2024-09-21T04:14:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-09-21T04:14:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-09-21T04:14:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 2149, 2024-09-21T04:14:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 20078381, 2024-09-21T04:14:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T04:14:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: "trackedop": { 2024-09-21T04:14:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_ops_count": 9 2024-09-21T04:14:12.995 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:12.995 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-21T04:14:13.978 INFO:tasks.check_counter:Daemon mds.b mds.exported=52 2024-09-21T04:14:13.978 INFO:tasks.check_counter:Daemon mds.b mds.imported=4 2024-09-21T04:14:13.978 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-21T04:14:13.978 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-09-21T04:14:13.978 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-09-21T04:14:13.978 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=7 2024-09-21T04:14:13.978 DEBUG:tasks.check_counter:Getting stats from e 2024-09-21T04:14:13.979 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph daemon mds.e perf dump 2024-09-21T04:14:14.236 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T04:14:14.698 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:14 smithi119 ceph-mon[34811]: pgmap v636: 97 pgs: 35 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 45 active+clean; 754 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-21T04:14:14.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:14 smithi064 ceph-mon[31180]: pgmap v636: 97 pgs: 35 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 45 active+clean; 754 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-21T04:14:14.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:14 smithi131 ceph-mon[35193]: pgmap v636: 97 pgs: 35 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 45 active+clean; 754 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-21T04:14:14.840 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-21T04:14:14.840 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T04:14:14.840 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 1390, 2024-09-21T04:14:14.840 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 1392, 2024-09-21T04:14:14.840 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 7909396, 2024-09-21T04:14:14.840 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 442030, 2024-09-21T04:14:14.840 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 6, 2024-09-21T04:14:14.841 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 2, 2024-09-21T04:14:14.841 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.148329860, 2024-09-21T04:14:14.841 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.068427095, 2024-09-21T04:14:14.841 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.032591736, 2024-09-21T04:14:14.841 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.036724836, 2024-09-21T04:14:14.841 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:14.841 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 1390, 2024-09-21T04:14:14.841 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.047926872, 2024-09-21T04:14:14.841 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000034479 2024-09-21T04:14:14.841 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.841 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:14.841 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.841 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.842 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.842 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T04:14:14.842 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T04:14:14.842 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.842 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T04:14:14.842 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 789, 2024-09-21T04:14:14.842 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 761, 2024-09-21T04:14:14.842 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 9777419, 2024-09-21T04:14:14.842 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 299908, 2024-09-21T04:14:14.842 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 4, 2024-09-21T04:14:14.842 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 3, 2024-09-21T04:14:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.087017797, 2024-09-21T04:14:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.032255181, 2024-09-21T04:14:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.052981694, 2024-09-21T04:14:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.011151319, 2024-09-21T04:14:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 761, 2024-09-21T04:14:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.033877507, 2024-09-21T04:14:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000044517 2024-09-21T04:14:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.843 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.844 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 1251824, 2024-09-21T04:14:14.844 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 129712 2024-09-21T04:14:14.844 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.844 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T04:14:14.844 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 456, 2024-09-21T04:14:14.844 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 611, 2024-09-21T04:14:14.844 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 4307009, 2024-09-21T04:14:14.844 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 500979, 2024-09-21T04:14:14.844 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 5, 2024-09-21T04:14:14.844 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 4, 2024-09-21T04:14:14.844 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.060516764, 2024-09-21T04:14:14.844 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.027391946, 2024-09-21T04:14:14.844 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.013783377, 2024-09-21T04:14:14.845 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.014798032, 2024-09-21T04:14:14.845 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:14.845 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 609, 2024-09-21T04:14:14.845 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.020586704, 2024-09-21T04:14:14.845 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000033804 2024-09-21T04:14:14.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.845 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:14.845 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.845 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.845 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.845 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.845 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T04:14:14.845 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 356288 2024-09-21T04:14:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: "cct": { 2024-09-21T04:14:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: "total_workers": 1, 2024-09-21T04:14:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: "unhealthy_workers": 0 2024-09-21T04:14:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-MDSRank": { 2024-09-21T04:14:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-09-21T04:14:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-09-21T04:14:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 4024, 2024-09-21T04:14:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.146849103, 2024-09-21T04:14:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000036493 2024-09-21T04:14:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.846 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-PurgeQueue": { 2024-09-21T04:14:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-09-21T04:14:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-09-21T04:14:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-09-21T04:14:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: "request": 0, 2024-09-21T04:14:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: "reply": 0, 2024-09-21T04:14:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: "reply_latency": { 2024-09-21T04:14:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.847 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_reply": 0, 2024-09-21T04:14:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: "forward": 0, 2024-09-21T04:14:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_complete": 0, 2024-09-21T04:14:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_keys": 0, 2024-09-21T04:14:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_commit": 0, 2024-09-21T04:14:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_split": 0, 2024-09-21T04:14:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_merge": 0, 2024-09-21T04:14:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes": 1626, 2024-09-21T04:14:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_top": 862, 2024-09-21T04:14:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_bottom": 764, 2024-09-21T04:14:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pin_tail": 0, 2024-09-21T04:14:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pinned": 395, 2024-09-21T04:14:14.848 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_expired": 0, 2024-09-21T04:14:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_with_caps": 0, 2024-09-21T04:14:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": 0, 2024-09-21T04:14:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: "subtrees": 2, 2024-09-21T04:14:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse": 0, 2024-09-21T04:14:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_hit": 0, 2024-09-21T04:14:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_forward": 0, 2024-09-21T04:14:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_discover": 0, 2024-09-21T04:14:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_dir_fetch": 0, 2024-09-21T04:14:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_remote_ino": 0, 2024-09-21T04:14:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_lock": 0, 2024-09-21T04:14:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: "load_cent": 0, 2024-09-21T04:14:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: "q": 0, 2024-09-21T04:14:14.849 INFO:teuthology.orchestra.run.smithi119.stdout: "exported": 0, 2024-09-21T04:14:14.850 INFO:teuthology.orchestra.run.smithi119.stdout: "exported_inodes": 0, 2024-09-21T04:14:14.850 INFO:teuthology.orchestra.run.smithi119.stdout: "imported": 0, 2024-09-21T04:14:14.850 INFO:teuthology.orchestra.run.smithi119.stdout: "imported_inodes": 0, 2024-09-21T04:14:14.850 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_dir_fetch": 0, 2024-09-21T04:14:14.850 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_backtrace_fetch": 0, 2024-09-21T04:14:14.850 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_peer_discover": 0, 2024-09-21T04:14:14.850 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rfiles": 4, 2024-09-21T04:14:14.850 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rbytes": 234, 2024-09-21T04:14:14.850 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rsnaps": 6, 2024-09-21T04:14:14.850 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T04:14:14.850 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_set_tag": 0, 2024-09-21T04:14:14.850 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T04:14:14.850 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_inotable_repaired": 0, 2024-09-21T04:14:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_inodes": 0, 2024-09-21T04:14:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T04:14:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T04:14:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_file_inodes": 0, 2024-09-21T04:14:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_inode_file_caps": 0, 2024-09-21T04:14:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T04:14:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_grant": 0, 2024-09-21T04:14:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T04:14:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T04:14:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T04:14:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps": 0, 2024-09-21T04:14:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps_dirty": 0, 2024-09-21T04:14:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_cap_release": 0, 2024-09-21T04:14:14.851 INFO:teuthology.orchestra.run.smithi119.stdout: "process_request_cap_release": 0 2024-09-21T04:14:14.852 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.852 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_cache": { 2024-09-21T04:14:14.852 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update": 0, 2024-09-21T04:14:14.852 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update_receipt": 0, 2024-09-21T04:14:14.852 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_try_discover": 0, 2024-09-21T04:14:14.852 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_send_discover": 0, 2024-09-21T04:14:14.852 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_handle_discover": 0, 2024-09-21T04:14:14.852 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays": 0, 2024-09-21T04:14:14.852 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_delayed": 0, 2024-09-21T04:14:14.852 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_enqueuing": 0, 2024-09-21T04:14:14.852 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_created": 0, 2024-09-21T04:14:14.852 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_enqueued": 0, 2024-09-21T04:14:14.852 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_reintegrated": 0, 2024-09-21T04:14:14.852 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_migrated": 0, 2024-09-21T04:14:14.852 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_processing": 0, 2024-09-21T04:14:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_enqueued": 0, 2024-09-21T04:14:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_prioritized": 0, 2024-09-21T04:14:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_started": 0, 2024-09-21T04:14:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_completed": 0, 2024-09-21T04:14:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_quiesce_path": 0, 2024-09-21T04:14:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_quiesce_inode": 0, 2024-09-21T04:14:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T04:14:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_exportdir": 0, 2024-09-21T04:14:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_flush": 0, 2024-09-21T04:14:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragmentdir": 0, 2024-09-21T04:14:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragstats": 0, 2024-09-21T04:14:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_inodestats": 0, 2024-09-21T04:14:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: "uninline_started": 0, 2024-09-21T04:14:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: "uninline_succeeded": 0, 2024-09-21T04:14:14.853 INFO:teuthology.orchestra.run.smithi119.stdout: "uninline_write_failed": 0 2024-09-21T04:14:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_log": { 2024-09-21T04:14:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: "evlrg": 0, 2024-09-21T04:14:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: "evadd": 0, 2024-09-21T04:14:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: "evex": 0, 2024-09-21T04:14:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: "evtrm": 0, 2024-09-21T04:14:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: "ev": 10183, 2024-09-21T04:14:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: "evexg": 0, 2024-09-21T04:14:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: "evexd": 0, 2024-09-21T04:14:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: "segadd": 0, 2024-09-21T04:14:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: "segex": 0, 2024-09-21T04:14:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: "segtrm": 0, 2024-09-21T04:14:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: "seg": 2, 2024-09-21T04:14:14.854 INFO:teuthology.orchestra.run.smithi119.stdout: "segmjr": 36, 2024-09-21T04:14:14.855 INFO:teuthology.orchestra.run.smithi119.stdout: "segexg": 0, 2024-09-21T04:14:14.855 INFO:teuthology.orchestra.run.smithi119.stdout: "segexd": 0, 2024-09-21T04:14:14.855 INFO:teuthology.orchestra.run.smithi119.stdout: "expos": 23226917, 2024-09-21T04:14:14.855 INFO:teuthology.orchestra.run.smithi119.stdout: "wrpos": 24272685, 2024-09-21T04:14:14.855 INFO:teuthology.orchestra.run.smithi119.stdout: "rdpos": 24272573, 2024-09-21T04:14:14.855 INFO:teuthology.orchestra.run.smithi119.stdout: "jlat": { 2024-09-21T04:14:14.855 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.855 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.855 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.855 INFO:teuthology.orchestra.run.smithi119.stdout: "replayed": 10183 2024-09-21T04:14:14.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.855 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_mem": { 2024-09-21T04:14:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 88, 2024-09-21T04:14:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: "ino+": 2995, 2024-09-21T04:14:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: "ino-": 2907, 2024-09-21T04:14:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: "dir": 23, 2024-09-21T04:14:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: "dir+": 99, 2024-09-21T04:14:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: "dir-": 76, 2024-09-21T04:14:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: "dn": 1626, 2024-09-21T04:14:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: "dn+": 3962, 2024-09-21T04:14:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: "dn-": 2336, 2024-09-21T04:14:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: "cap": 0, 2024-09-21T04:14:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: "cap+": 0, 2024-09-21T04:14:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: "cap-": 0, 2024-09-21T04:14:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: "rss": 125364, 2024-09-21T04:14:14.856 INFO:teuthology.orchestra.run.smithi119.stdout: "heap": 326021120 2024-09-21T04:14:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_server": { 2024-09-21T04:14:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_client_request": 0, 2024-09-21T04:14:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_server_request": 0, 2024-09-21T04:14:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_request": 0, 2024-09-21T04:14:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_session": 0, 2024-09-21T04:14:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_peer_request": 0, 2024-09-21T04:14:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: "req_create_latency": { 2024-09-21T04:14:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getattr_latency": { 2024-09-21T04:14:14.857 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getfilelock_latency": { 2024-09-21T04:14:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: "req_link_latency": { 2024-09-21T04:14:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookup_latency": { 2024-09-21T04:14:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookuphash_latency": { 2024-09-21T04:14:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupino_latency": { 2024-09-21T04:14:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.859 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupname_latency": { 2024-09-21T04:14:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupparent_latency": { 2024-09-21T04:14:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupsnap_latency": { 2024-09-21T04:14:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.860 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.861 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.861 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lssnap_latency": { 2024-09-21T04:14:14.861 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.861 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.861 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.861 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mkdir_latency": { 2024-09-21T04:14:14.861 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.861 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.861 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.861 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mknod_latency": { 2024-09-21T04:14:14.861 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.862 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.862 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.862 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mksnap_latency": { 2024-09-21T04:14:14.862 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.862 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.862 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.862 INFO:teuthology.orchestra.run.smithi119.stdout: "req_open_latency": { 2024-09-21T04:14:14.862 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.862 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.862 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.863 INFO:teuthology.orchestra.run.smithi119.stdout: "req_readdir_latency": { 2024-09-21T04:14:14.863 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.863 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.863 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.863 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rename_latency": { 2024-09-21T04:14:14.863 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.863 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.863 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.863 INFO:teuthology.orchestra.run.smithi119.stdout: "req_renamesnap_latency": { 2024-09-21T04:14:14.863 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.863 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.863 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: "req_snapdiff_latency": { 2024-09-21T04:14:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmdir_latency": { 2024-09-21T04:14:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmsnap_latency": { 2024-09-21T04:14:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.864 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmxattr_latency": { 2024-09-21T04:14:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setattr_latency": { 2024-09-21T04:14:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setdirlayout_latency": { 2024-09-21T04:14:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.865 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.866 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setfilelock_latency": { 2024-09-21T04:14:14.866 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.866 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.866 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.866 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setlayout_latency": { 2024-09-21T04:14:14.866 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.866 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.866 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.866 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setxattr_latency": { 2024-09-21T04:14:14.866 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.867 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.867 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.867 INFO:teuthology.orchestra.run.smithi119.stdout: "req_symlink_latency": { 2024-09-21T04:14:14.867 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.867 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.867 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.867 INFO:teuthology.orchestra.run.smithi119.stdout: "req_unlink_latency": { 2024-09-21T04:14:14.867 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.867 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.867 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_revoke_eviction": 0, 2024-09-21T04:14:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_acquisition_throttle": 0, 2024-09-21T04:14:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getvxattr_latency": { 2024-09-21T04:14:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_sessions": { 2024-09-21T04:14:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: "session_count": 2, 2024-09-21T04:14:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: "session_add": 4, 2024-09-21T04:14:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: "session_remove": 2, 2024-09-21T04:14:14.868 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_open": 3, 2024-09-21T04:14:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_stale": 0, 2024-09-21T04:14:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: "total_load": 0, 2024-09-21T04:14:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: "average_load": 0, 2024-09-21T04:14:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: "avg_session_uptime": 2574, 2024-09-21T04:14:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: "mdthresh_evicted": 0 2024-09-21T04:14:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: "mempool": { 2024-09-21T04:14:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_bytes": 0, 2024-09-21T04:14:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_items": 0, 2024-09-21T04:14:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T04:14:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_items": 0, 2024-09-21T04:14:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T04:14:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_items": 0, 2024-09-21T04:14:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T04:14:14.869 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T04:14:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T04:14:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T04:14:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T04:14:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_items": 0, 2024-09-21T04:14:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T04:14:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T04:14:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_bytes": 0, 2024-09-21T04:14:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_items": 0, 2024-09-21T04:14:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_bytes": 0, 2024-09-21T04:14:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_items": 0, 2024-09-21T04:14:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T04:14:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T04:14:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T04:14:14.870 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T04:14:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T04:14:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_items": 0, 2024-09-21T04:14:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_bytes": 0, 2024-09-21T04:14:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_items": 0, 2024-09-21T04:14:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T04:14:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T04:14:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_bytes": 0, 2024-09-21T04:14:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_items": 0, 2024-09-21T04:14:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_bytes": 0, 2024-09-21T04:14:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_items": 0, 2024-09-21T04:14:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T04:14:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_items": 0, 2024-09-21T04:14:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T04:14:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_items": 0, 2024-09-21T04:14:14.871 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_bytes": 201802, 2024-09-21T04:14:14.872 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_items": 152, 2024-09-21T04:14:14.872 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_bytes": 0, 2024-09-21T04:14:14.872 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_items": 0, 2024-09-21T04:14:14.872 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_bytes": 0, 2024-09-21T04:14:14.872 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_items": 0, 2024-09-21T04:14:14.872 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_bytes": 0, 2024-09-21T04:14:14.872 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_items": 0, 2024-09-21T04:14:14.872 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_bytes": 0, 2024-09-21T04:14:14.872 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_items": 0, 2024-09-21T04:14:14.872 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_bytes": 8664, 2024-09-21T04:14:14.872 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_items": 191, 2024-09-21T04:14:14.872 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T04:14:14.872 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_items": 0, 2024-09-21T04:14:14.872 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_bytes": 0, 2024-09-21T04:14:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_items": 0, 2024-09-21T04:14:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_bytes": 1398730, 2024-09-21T04:14:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_items": 9954, 2024-09-21T04:14:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_bytes": 0, 2024-09-21T04:14:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_items": 0, 2024-09-21T04:14:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_bytes": 0, 2024-09-21T04:14:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_items": 0 2024-09-21T04:14:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: "objecter": { 2024-09-21T04:14:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: "op_active": 0, 2024-09-21T04:14:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: "op_laggy": 0, 2024-09-21T04:14:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send": 2380, 2024-09-21T04:14:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send_bytes": 26, 2024-09-21T04:14:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: "op_resend": 4, 2024-09-21T04:14:14.873 INFO:teuthology.orchestra.run.smithi119.stdout: "op_reply": 2380, 2024-09-21T04:14:14.874 INFO:teuthology.orchestra.run.smithi119.stdout: "op_latency": { 2024-09-21T04:14:14.874 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2380, 2024-09-21T04:14:14.874 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 2.245943764, 2024-09-21T04:14:14.874 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000943673 2024-09-21T04:14:14.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.874 INFO:teuthology.orchestra.run.smithi119.stdout: "op_inflight": 0, 2024-09-21T04:14:14.874 INFO:teuthology.orchestra.run.smithi119.stdout: "oplen_avg": { 2024-09-21T04:14:14.874 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2380, 2024-09-21T04:14:14.874 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 2381 2024-09-21T04:14:14.874 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.874 INFO:teuthology.orchestra.run.smithi119.stdout: "op": 2380, 2024-09-21T04:14:14.874 INFO:teuthology.orchestra.run.smithi119.stdout: "op_r": 2380, 2024-09-21T04:14:14.874 INFO:teuthology.orchestra.run.smithi119.stdout: "op_w": 0, 2024-09-21T04:14:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: "op_rmw": 0, 2024-09-21T04:14:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: "op_pg": 0, 2024-09-21T04:14:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_stat": 1429, 2024-09-21T04:14:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_create": 0, 2024-09-21T04:14:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_read": 948, 2024-09-21T04:14:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_write": 0, 2024-09-21T04:14:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writefull": 0, 2024-09-21T04:14:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writesame": 0, 2024-09-21T04:14:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_append": 0, 2024-09-21T04:14:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_zero": 0, 2024-09-21T04:14:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_truncate": 0, 2024-09-21T04:14:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_delete": 0, 2024-09-21T04:14:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_mapext": 0, 2024-09-21T04:14:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_sparse_read": 0, 2024-09-21T04:14:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_clonerange": 0, 2024-09-21T04:14:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_getxattr": 2, 2024-09-21T04:14:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_setxattr": 0, 2024-09-21T04:14:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_cmpxattr": 0, 2024-09-21T04:14:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_rmxattr": 0, 2024-09-21T04:14:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_resetxattrs": 0, 2024-09-21T04:14:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_call": 0, 2024-09-21T04:14:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_watch": 0, 2024-09-21T04:14:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_notify": 0, 2024-09-21T04:14:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T04:14:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls": 0, 2024-09-21T04:14:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls_filter": 0, 2024-09-21T04:14:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_other": 0, 2024-09-21T04:14:14.876 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_active": 0, 2024-09-21T04:14:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_send": 0, 2024-09-21T04:14:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_resend": 0, 2024-09-21T04:14:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_ping": 0, 2024-09-21T04:14:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_active": 0, 2024-09-21T04:14:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_send": 0, 2024-09-21T04:14:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_resend": 0, 2024-09-21T04:14:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_active": 0, 2024-09-21T04:14:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_send": 0, 2024-09-21T04:14:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_resend": 0, 2024-09-21T04:14:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_active": 0, 2024-09-21T04:14:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_send": 0, 2024-09-21T04:14:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_resend": 0, 2024-09-21T04:14:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: "command_active": 0, 2024-09-21T04:14:14.877 INFO:teuthology.orchestra.run.smithi119.stdout: "command_send": 0, 2024-09-21T04:14:14.878 INFO:teuthology.orchestra.run.smithi119.stdout: "command_resend": 0, 2024-09-21T04:14:14.878 INFO:teuthology.orchestra.run.smithi119.stdout: "map_epoch": 86, 2024-09-21T04:14:14.878 INFO:teuthology.orchestra.run.smithi119.stdout: "map_full": 0, 2024-09-21T04:14:14.878 INFO:teuthology.orchestra.run.smithi119.stdout: "map_inc": 13, 2024-09-21T04:14:14.878 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_sessions": 6, 2024-09-21T04:14:14.878 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_open": 6, 2024-09-21T04:14:14.878 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_close": 0, 2024-09-21T04:14:14.878 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_laggy": 0, 2024-09-21T04:14:14.878 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_wr": 0, 2024-09-21T04:14:14.878 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_rd": 2, 2024-09-21T04:14:14.878 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_del": 0 2024-09-21T04:14:14.878 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.878 INFO:teuthology.orchestra.run.smithi119.stdout: "oft": { 2024-09-21T04:14:14.879 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_objs": 0, 2024-09-21T04:14:14.879 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_kv_pairs": 0, 2024-09-21T04:14:14.879 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_updates": 0, 2024-09-21T04:14:14.879 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_removes": 0 2024-09-21T04:14:14.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.879 INFO:teuthology.orchestra.run.smithi119.stdout: "purge_queue": { 2024-09-21T04:14:14.879 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops": 0, 2024-09-21T04:14:14.879 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T04:14:14.879 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing": 0, 2024-09-21T04:14:14.879 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_high_water": 0, 2024-09-21T04:14:14.879 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed_ops": 0, 2024-09-21T04:14:14.879 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed": 0, 2024-09-21T04:14:14.879 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_item_in_journal": 0 2024-09-21T04:14:14.879 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.880 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T04:14:14.880 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T04:14:14.880 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-09-21T04:14:14.880 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T04:14:14.880 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 2631, 2024-09-21T04:14:14.880 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 21741302, 2024-09-21T04:14:14.880 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:14.880 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 2631, 2024-09-21T04:14:14.880 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-09-21T04:14:14.880 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-09-21T04:14:14.880 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 2631, 2024-09-21T04:14:14.880 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 21741302, 2024-09-21T04:14:14.880 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T04:14:14.881 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.881 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.881 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.881 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:14.881 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.881 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_bytes": { 2024-09-21T04:14:14.881 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T04:14:14.881 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-09-21T04:14:14.881 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T04:14:14.881 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-09-21T04:14:14.881 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-09-21T04:14:14.881 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:14.881 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-09-21T04:14:14.881 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 2380, 2024-09-21T04:14:14.881 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 20078440, 2024-09-21T04:14:14.882 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 2380, 2024-09-21T04:14:14.882 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 20078440, 2024-09-21T04:14:14.882 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T04:14:14.882 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.882 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.882 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.882 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:14.882 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.882 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_ops": { 2024-09-21T04:14:14.882 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T04:14:14.882 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 1024, 2024-09-21T04:14:14.882 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T04:14:14.882 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-09-21T04:14:14.883 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-09-21T04:14:14.883 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:14.883 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-09-21T04:14:14.883 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 2380, 2024-09-21T04:14:14.883 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 2380, 2024-09-21T04:14:14.883 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 2380, 2024-09-21T04:14:14.883 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 2380, 2024-09-21T04:14:14.883 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T04:14:14.883 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.883 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.883 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.883 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:14.883 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.883 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle": { 2024-09-21T04:14:14.884 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T04:14:14.884 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-09-21T04:14:14.884 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T04:14:14.884 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-09-21T04:14:14.884 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-09-21T04:14:14.884 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:14.884 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-09-21T04:14:14.884 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-09-21T04:14:14.884 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-09-21T04:14:14.884 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 0, 2024-09-21T04:14:14.884 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 0, 2024-09-21T04:14:14.884 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T04:14:14.884 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.885 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.885 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.885 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:14.885 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.885 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle-0x55fb2fae71e0": { 2024-09-21T04:14:14.885 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T04:14:14.885 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-09-21T04:14:14.885 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T04:14:14.885 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-09-21T04:14:14.885 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-09-21T04:14:14.885 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:14.885 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-09-21T04:14:14.885 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-09-21T04:14:14.885 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-09-21T04:14:14.885 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 0, 2024-09-21T04:14:14.886 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 0, 2024-09-21T04:14:14.886 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T04:14:14.886 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:14.886 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:14.886 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:14.886 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:14.886 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:14.886 INFO:teuthology.orchestra.run.smithi119.stdout: "trackedop": { 2024-09-21T04:14:14.886 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_ops_count": 0 2024-09-21T04:14:14.886 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:14.886 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-21T04:14:16.414 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-21T04:14:16.414 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-21T04:14:16.415 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-09-21T04:14:16.415 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-21T04:14:16.415 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T04:14:16.415 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-09-21T04:14:16.415 DEBUG:tasks.check_counter:Getting stats from h 2024-09-21T04:14:16.415 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph daemon mds.h perf dump 2024-09-21T04:14:16.670 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T04:14:16.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:16 smithi119 ceph-mon[34811]: pgmap v637: 97 pgs: 35 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 45 active+clean; 754 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-21T04:14:16.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:16 smithi064 ceph-mon[31180]: pgmap v637: 97 pgs: 35 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 45 active+clean; 754 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-21T04:14:16.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:16 smithi131 ceph-mon[35193]: pgmap v637: 97 pgs: 35 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 45 active+clean; 754 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-21T04:14:18.662 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-21T04:14:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T04:14:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 64293, 2024-09-21T04:14:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 61228, 2024-09-21T04:14:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 22695855, 2024-09-21T04:14:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 255373810, 2024-09-21T04:14:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 35, 2024-09-21T04:14:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 5, 2024-09-21T04:14:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 5.906847645, 2024-09-21T04:14:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 2.849908456, 2024-09-21T04:14:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 1.165655881, 2024-09-21T04:14:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 1.673272150, 2024-09-21T04:14:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 61226, 2024-09-21T04:14:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 13.428109010, 2024-09-21T04:14:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000219320 2024-09-21T04:14:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 12326, 2024-09-21T04:14:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.001721183, 2024-09-21T04:14:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000139 2024-09-21T04:14:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T04:14:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T04:14:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T04:14:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 74265, 2024-09-21T04:14:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 72103, 2024-09-21T04:14:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 29829918, 2024-09-21T04:14:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 244656393, 2024-09-21T04:14:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 14, 2024-09-21T04:14:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 6, 2024-09-21T04:14:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 6.965306529, 2024-09-21T04:14:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 3.732261844, 2024-09-21T04:14:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 1.464522955, 2024-09-21T04:14:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 1.625262220, 2024-09-21T04:14:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 72102, 2024-09-21T04:14:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 18.902276826, 2024-09-21T04:14:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000262160 2024-09-21T04:14:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 29810, 2024-09-21T04:14:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.004260820, 2024-09-21T04:14:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000142 2024-09-21T04:14:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 4080, 2024-09-21T04:14:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 360560 2024-09-21T04:14:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T04:14:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 545917, 2024-09-21T04:14:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 679425, 2024-09-21T04:14:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 183448251, 2024-09-21T04:14:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 693755572, 2024-09-21T04:14:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 9, 2024-09-21T04:14:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 6, 2024-09-21T04:14:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 31.070548309, 2024-09-21T04:14:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 18.008338288, 2024-09-21T04:14:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 37.091059302, 2024-09-21T04:14:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 1.488979747, 2024-09-21T04:14:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 679424, 2024-09-21T04:14:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 32.384664003, 2024-09-21T04:14:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000047664 2024-09-21T04:14:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 920560, 2024-09-21T04:14:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.133465181, 2024-09-21T04:14:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000144 2024-09-21T04:14:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 1266544, 2024-09-21T04:14:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 142736 2024-09-21T04:14:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: "cct": { 2024-09-21T04:14:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: "total_workers": 1, 2024-09-21T04:14:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: "unhealthy_workers": 0 2024-09-21T04:14:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-MDSRank": { 2024-09-21T04:14:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-09-21T04:14:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-09-21T04:14:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 38014, 2024-09-21T04:14:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 242.676198614, 2024-09-21T04:14:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.006383863 2024-09-21T04:14:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: "finisher-PurgeQueue": { 2024-09-21T04:14:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: "queue_len": 0, 2024-09-21T04:14:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: "complete_latency": { 2024-09-21T04:14:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 51759, 2024-09-21T04:14:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 22.619849030, 2024-09-21T04:14:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000437022 2024-09-21T04:14:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": { 2024-09-21T04:14:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: "request": 229804, 2024-09-21T04:14:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: "reply": 228350, 2024-09-21T04:14:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: "reply_latency": { 2024-09-21T04:14:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 228350, 2024-09-21T04:14:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 12643.262679788, 2024-09-21T04:14:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.055367911 2024-09-21T04:14:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_reply": 0, 2024-09-21T04:14:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: "forward": 1454, 2024-09-21T04:14:18.669 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_complete": 0, 2024-09-21T04:14:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_fetch_keys": 0, 2024-09-21T04:14:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_commit": 2397, 2024-09-21T04:14:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_split": 133, 2024-09-21T04:14:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_merge": 8, 2024-09-21T04:14:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes": 67848, 2024-09-21T04:14:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_top": 25525, 2024-09-21T04:14:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_bottom": 42056, 2024-09-21T04:14:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pin_tail": 267, 2024-09-21T04:14:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_pinned": 31186, 2024-09-21T04:14:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_expired": 2745, 2024-09-21T04:14:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: "inodes_with_caps": 1, 2024-09-21T04:14:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": 1, 2024-09-21T04:14:18.670 INFO:teuthology.orchestra.run.smithi119.stdout: "subtrees": 28, 2024-09-21T04:14:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse": 502954, 2024-09-21T04:14:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_hit": 471175, 2024-09-21T04:14:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_forward": 1443, 2024-09-21T04:14:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_discover": 127, 2024-09-21T04:14:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_dir_fetch": 0, 2024-09-21T04:14:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_remote_ino": 0, 2024-09-21T04:14:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: "traverse_lock": 673, 2024-09-21T04:14:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: "load_cent": 0, 2024-09-21T04:14:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: "q": 0, 2024-09-21T04:14:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: "exported": 22, 2024-09-21T04:14:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: "exported_inodes": 1527, 2024-09-21T04:14:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: "imported": 149, 2024-09-21T04:14:18.671 INFO:teuthology.orchestra.run.smithi119.stdout: "imported_inodes": 9075, 2024-09-21T04:14:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_dir_fetch": 0, 2024-09-21T04:14:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_backtrace_fetch": 84111, 2024-09-21T04:14:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: "openino_peer_discover": 0, 2024-09-21T04:14:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rfiles": 4, 2024-09-21T04:14:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rbytes": 234, 2024-09-21T04:14:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: "root_rsnaps": 6, 2024-09-21T04:14:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_fetch": 84111, 2024-09-21T04:14:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_set_tag": 0, 2024-09-21T04:14:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T04:14:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_inotable_repaired": 0, 2024-09-21T04:14:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_inodes": 565, 2024-09-21T04:14:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T04:14:18.672 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_dirfrag_rstats": 565, 2024-09-21T04:14:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: "scrub_file_inodes": 83546, 2024-09-21T04:14:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_inode_file_caps": 142, 2024-09-21T04:14:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_revoke": 178691, 2024-09-21T04:14:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_grant": 94932, 2024-09-21T04:14:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_trunc": 2, 2024-09-21T04:14:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T04:14:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T04:14:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps": 267519, 2024-09-21T04:14:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_caps_dirty": 44755, 2024-09-21T04:14:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_cap_release": 3855, 2024-09-21T04:14:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: "process_request_cap_release": 68119 2024-09-21T04:14:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_cache": { 2024-09-21T04:14:18.673 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update": 2, 2024-09-21T04:14:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_update_receipt": 3, 2024-09-21T04:14:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_try_discover": 2, 2024-09-21T04:14:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_send_discover": 108, 2024-09-21T04:14:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: "dir_handle_discover": 51, 2024-09-21T04:14:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays": 125, 2024-09-21T04:14:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_delayed": 0, 2024-09-21T04:14:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: "num_strays_enqueuing": 0, 2024-09-21T04:14:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_created": 32845, 2024-09-21T04:14:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_enqueued": 32720, 2024-09-21T04:14:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_reintegrated": 0, 2024-09-21T04:14:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: "strays_migrated": 0, 2024-09-21T04:14:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_processing": 0, 2024-09-21T04:14:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_enqueued": 0, 2024-09-21T04:14:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: "num_recovering_prioritized": 0, 2024-09-21T04:14:18.674 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_started": 0, 2024-09-21T04:14:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: "recovery_completed": 0, 2024-09-21T04:14:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_quiesce_path": 0, 2024-09-21T04:14:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_quiesce_inode": 0, 2024-09-21T04:14:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T04:14:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_exportdir": 22, 2024-09-21T04:14:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_flush": 0, 2024-09-21T04:14:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragmentdir": 142, 2024-09-21T04:14:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_fragstats": 0, 2024-09-21T04:14:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: "ireq_inodestats": 0, 2024-09-21T04:14:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: "uninline_started": 0, 2024-09-21T04:14:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: "uninline_succeeded": 0, 2024-09-21T04:14:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: "uninline_write_failed": 0 2024-09-21T04:14:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.675 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_log": { 2024-09-21T04:14:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: "evlrg": 1, 2024-09-21T04:14:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: "evadd": 239846, 2024-09-21T04:14:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: "evex": 159771, 2024-09-21T04:14:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: "evtrm": 159771, 2024-09-21T04:14:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: "ev": 80075, 2024-09-21T04:14:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: "evexg": 0, 2024-09-21T04:14:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: "evexd": 12290, 2024-09-21T04:14:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: "segadd": 360, 2024-09-21T04:14:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: "segex": 232, 2024-09-21T04:14:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: "segtrm": 232, 2024-09-21T04:14:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: "seg": 128, 2024-09-21T04:14:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: "segmjr": 7, 2024-09-21T04:14:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: "segexg": 0, 2024-09-21T04:14:18.676 INFO:teuthology.orchestra.run.smithi119.stdout: "segexd": 16, 2024-09-21T04:14:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: "expos": 499252318, 2024-09-21T04:14:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: "wrpos": 801431513, 2024-09-21T04:14:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: "rdpos": 0, 2024-09-21T04:14:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: "jlat": { 2024-09-21T04:14:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 19942, 2024-09-21T04:14:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 800.646223628, 2024-09-21T04:14:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.040148742 2024-09-21T04:14:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: "replayed": 0 2024-09-21T04:14:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_mem": { 2024-09-21T04:14:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: "ino": 42891, 2024-09-21T04:14:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: "ino+": 76382, 2024-09-21T04:14:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: "ino-": 33491, 2024-09-21T04:14:18.677 INFO:teuthology.orchestra.run.smithi119.stdout: "dir": 1163, 2024-09-21T04:14:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: "dir+": 1404, 2024-09-21T04:14:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: "dir-": 241, 2024-09-21T04:14:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: "dn": 67848, 2024-09-21T04:14:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: "dn+": 121168, 2024-09-21T04:14:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: "dn-": 53320, 2024-09-21T04:14:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: "cap": 1, 2024-09-21T04:14:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: "cap+": 34361, 2024-09-21T04:14:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: "cap-": 34360, 2024-09-21T04:14:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: "rss": 378448, 2024-09-21T04:14:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: "heap": 598622208 2024-09-21T04:14:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_server": { 2024-09-21T04:14:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_client_request": 237892, 2024-09-21T04:14:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: "dispatch_server_request": 451, 2024-09-21T04:14:18.678 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_request": 229804, 2024-09-21T04:14:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_client_session": 30, 2024-09-21T04:14:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: "handle_peer_request": 1923, 2024-09-21T04:14:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: "req_create_latency": { 2024-09-21T04:14:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 31645, 2024-09-21T04:14:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 885.929878133, 2024-09-21T04:14:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.027995888 2024-09-21T04:14:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getattr_latency": { 2024-09-21T04:14:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 116543, 2024-09-21T04:14:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 9365.099151416, 2024-09-21T04:14:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.080357457 2024-09-21T04:14:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getfilelock_latency": { 2024-09-21T04:14:18.679 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: "req_link_latency": { 2024-09-21T04:14:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookup_latency": { 2024-09-21T04:14:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 22301, 2024-09-21T04:14:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 897.979441121, 2024-09-21T04:14:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.040266330 2024-09-21T04:14:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.680 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookuphash_latency": { 2024-09-21T04:14:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupino_latency": { 2024-09-21T04:14:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 1, 2024-09-21T04:14:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000095769, 2024-09-21T04:14:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000095769 2024-09-21T04:14:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupname_latency": { 2024-09-21T04:14:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:18.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupparent_latency": { 2024-09-21T04:14:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lookupsnap_latency": { 2024-09-21T04:14:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: "req_lssnap_latency": { 2024-09-21T04:14:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:18.682 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mkdir_latency": { 2024-09-21T04:14:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 110, 2024-09-21T04:14:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 9.011317706, 2024-09-21T04:14:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.081921070 2024-09-21T04:14:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mknod_latency": { 2024-09-21T04:14:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: "req_mksnap_latency": { 2024-09-21T04:14:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:18.683 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: "req_open_latency": { 2024-09-21T04:14:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 4845, 2024-09-21T04:14:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 126.475327710, 2024-09-21T04:14:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.026104298 2024-09-21T04:14:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: "req_readdir_latency": { 2024-09-21T04:14:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 1022, 2024-09-21T04:14:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 1.608115157, 2024-09-21T04:14:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.001573498 2024-09-21T04:14:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rename_latency": { 2024-09-21T04:14:18.684 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 31638, 2024-09-21T04:14:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 1294.602142872, 2024-09-21T04:14:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.040919215 2024-09-21T04:14:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: "req_renamesnap_latency": { 2024-09-21T04:14:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: "req_snapdiff_latency": { 2024-09-21T04:14:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmdir_latency": { 2024-09-21T04:14:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 126, 2024-09-21T04:14:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 15.728547349, 2024-09-21T04:14:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.124829740 2024-09-21T04:14:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmsnap_latency": { 2024-09-21T04:14:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: "req_rmxattr_latency": { 2024-09-21T04:14:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:18.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setattr_latency": { 2024-09-21T04:14:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 2, 2024-09-21T04:14:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.108306940, 2024-09-21T04:14:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.054153470 2024-09-21T04:14:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setdirlayout_latency": { 2024-09-21T04:14:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setfilelock_latency": { 2024-09-21T04:14:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:18.687 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:18.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.688 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setlayout_latency": { 2024-09-21T04:14:18.688 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:18.688 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:18.688 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:18.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.688 INFO:teuthology.orchestra.run.smithi119.stdout: "req_setxattr_latency": { 2024-09-21T04:14:18.688 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:18.688 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:18.688 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:18.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: "req_symlink_latency": { 2024-09-21T04:14:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: "req_unlink_latency": { 2024-09-21T04:14:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 20117, 2024-09-21T04:14:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 46.720355615, 2024-09-21T04:14:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.002322431 2024-09-21T04:14:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_revoke_eviction": 0, 2024-09-21T04:14:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: "cap_acquisition_throttle": 0, 2024-09-21T04:14:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: "req_getvxattr_latency": { 2024-09-21T04:14:18.689 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_sessions": { 2024-09-21T04:14:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: "session_count": 1, 2024-09-21T04:14:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: "session_add": 2, 2024-09-21T04:14:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: "session_remove": 1, 2024-09-21T04:14:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_open": 1, 2024-09-21T04:14:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: "sessions_stale": 0, 2024-09-21T04:14:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: "total_load": 46328, 2024-09-21T04:14:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: "average_load": 46328, 2024-09-21T04:14:18.690 INFO:teuthology.orchestra.run.smithi119.stdout: "avg_session_uptime": 616, 2024-09-21T04:14:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: "mdthresh_evicted": 0 2024-09-21T04:14:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: "mempool": { 2024-09-21T04:14:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_bytes": 0, 2024-09-21T04:14:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_items": 0, 2024-09-21T04:14:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T04:14:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_items": 0, 2024-09-21T04:14:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T04:14:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_items": 0, 2024-09-21T04:14:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T04:14:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T04:14:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T04:14:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T04:14:18.691 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T04:14:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_items": 0, 2024-09-21T04:14:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T04:14:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T04:14:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_bytes": 0, 2024-09-21T04:14:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_items": 0, 2024-09-21T04:14:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_bytes": 0, 2024-09-21T04:14:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_items": 0, 2024-09-21T04:14:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T04:14:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T04:14:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T04:14:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T04:14:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T04:14:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_items": 0, 2024-09-21T04:14:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_bytes": 0, 2024-09-21T04:14:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_items": 0, 2024-09-21T04:14:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T04:14:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T04:14:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_bytes": 0, 2024-09-21T04:14:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_items": 0, 2024-09-21T04:14:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_bytes": 0, 2024-09-21T04:14:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_items": 0, 2024-09-21T04:14:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T04:14:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_items": 0, 2024-09-21T04:14:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T04:14:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_items": 0, 2024-09-21T04:14:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_bytes": 2478453, 2024-09-21T04:14:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_items": 31607, 2024-09-21T04:14:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_bytes": 0, 2024-09-21T04:14:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_items": 0, 2024-09-21T04:14:18.693 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_bytes": 0, 2024-09-21T04:14:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_items": 0, 2024-09-21T04:14:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_bytes": 0, 2024-09-21T04:14:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_items": 0, 2024-09-21T04:14:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_bytes": 0, 2024-09-21T04:14:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_items": 0, 2024-09-21T04:14:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_bytes": 8664, 2024-09-21T04:14:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_items": 191, 2024-09-21T04:14:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T04:14:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_items": 0, 2024-09-21T04:14:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_bytes": 0, 2024-09-21T04:14:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_items": 0, 2024-09-21T04:14:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_bytes": 147398804, 2024-09-21T04:14:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_items": 1303530, 2024-09-21T04:14:18.694 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_bytes": 0, 2024-09-21T04:14:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_items": 0, 2024-09-21T04:14:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_bytes": 0, 2024-09-21T04:14:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_items": 0 2024-09-21T04:14:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: "objecter": { 2024-09-21T04:14:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: "op_active": 0, 2024-09-21T04:14:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: "op_laggy": 0, 2024-09-21T04:14:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send": 155455, 2024-09-21T04:14:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: "op_send_bytes": 858526532, 2024-09-21T04:14:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: "op_resend": 33, 2024-09-21T04:14:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: "op_reply": 155455, 2024-09-21T04:14:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: "op_latency": { 2024-09-21T04:14:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 155455, 2024-09-21T04:14:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 1327.071700341, 2024-09-21T04:14:18.695 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.008536693 2024-09-21T04:14:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: "op_inflight": 0, 2024-09-21T04:14:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: "oplen_avg": { 2024-09-21T04:14:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 155455, 2024-09-21T04:14:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 188582 2024-09-21T04:14:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: "op": 155455, 2024-09-21T04:14:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: "op_r": 84753, 2024-09-21T04:14:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: "op_w": 70702, 2024-09-21T04:14:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: "op_rmw": 0, 2024-09-21T04:14:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: "op_pg": 0, 2024-09-21T04:14:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_stat": 1196, 2024-09-21T04:14:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_create": 13780, 2024-09-21T04:14:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_read": 642, 2024-09-21T04:14:18.696 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_write": 20584, 2024-09-21T04:14:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writefull": 303, 2024-09-21T04:14:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_writesame": 0, 2024-09-21T04:14:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_append": 0, 2024-09-21T04:14:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_zero": 0, 2024-09-21T04:14:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_truncate": 119, 2024-09-21T04:14:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_delete": 33126, 2024-09-21T04:14:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_mapext": 0, 2024-09-21T04:14:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_sparse_read": 0, 2024-09-21T04:14:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_clonerange": 0, 2024-09-21T04:14:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_getxattr": 84111, 2024-09-21T04:14:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_setxattr": 27560, 2024-09-21T04:14:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_cmpxattr": 0, 2024-09-21T04:14:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_rmxattr": 0, 2024-09-21T04:14:18.697 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_resetxattrs": 0, 2024-09-21T04:14:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_call": 0, 2024-09-21T04:14:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_watch": 0, 2024-09-21T04:14:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_notify": 0, 2024-09-21T04:14:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T04:14:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls": 0, 2024-09-21T04:14:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_pgls_filter": 0, 2024-09-21T04:14:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: "osdop_other": 2, 2024-09-21T04:14:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_active": 0, 2024-09-21T04:14:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_send": 0, 2024-09-21T04:14:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_resend": 0, 2024-09-21T04:14:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: "linger_ping": 0, 2024-09-21T04:14:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_active": 0, 2024-09-21T04:14:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_send": 0, 2024-09-21T04:14:18.698 INFO:teuthology.orchestra.run.smithi119.stdout: "poolop_resend": 0, 2024-09-21T04:14:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_active": 0, 2024-09-21T04:14:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_send": 0, 2024-09-21T04:14:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: "poolstat_resend": 0, 2024-09-21T04:14:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_active": 0, 2024-09-21T04:14:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_send": 0, 2024-09-21T04:14:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: "statfs_resend": 0, 2024-09-21T04:14:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: "command_active": 0, 2024-09-21T04:14:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: "command_send": 0, 2024-09-21T04:14:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: "command_resend": 0, 2024-09-21T04:14:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: "map_epoch": 86, 2024-09-21T04:14:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: "map_full": 0, 2024-09-21T04:14:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: "map_inc": 13, 2024-09-21T04:14:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_sessions": 12, 2024-09-21T04:14:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_open": 12, 2024-09-21T04:14:18.699 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_session_close": 0, 2024-09-21T04:14:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_laggy": 0, 2024-09-21T04:14:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_wr": 5515, 2024-09-21T04:14:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_rd": 0, 2024-09-21T04:14:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_del": 1644 2024-09-21T04:14:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: "oft": { 2024-09-21T04:14:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_objs": 1, 2024-09-21T04:14:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_kv_pairs": 0, 2024-09-21T04:14:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_updates": 0, 2024-09-21T04:14:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: "omap_total_removes": 0 2024-09-21T04:14:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: "purge_queue": { 2024-09-21T04:14:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops": 0, 2024-09-21T04:14:18.700 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_ops_high_water": 10, 2024-09-21T04:14:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing": 0, 2024-09-21T04:14:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executing_high_water": 10, 2024-09-21T04:14:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed_ops": 32728, 2024-09-21T04:14:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_executed": 32720, 2024-09-21T04:14:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: "pq_item_in_journal": 0 2024-09-21T04:14:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T04:14:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T04:14:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-09-21T04:14:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T04:14:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 684471, 2024-09-21T04:14:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 174353202, 2024-09-21T04:14:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:18.701 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 684471, 2024-09-21T04:14:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-09-21T04:14:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-09-21T04:14:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 684471, 2024-09-21T04:14:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 174353202, 2024-09-21T04:14:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T04:14:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_bytes": { 2024-09-21T04:14:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T04:14:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-09-21T04:14:18.702 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T04:14:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-09-21T04:14:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-09-21T04:14:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-09-21T04:14:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 155455, 2024-09-21T04:14:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 862433652, 2024-09-21T04:14:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 155455, 2024-09-21T04:14:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 862433652, 2024-09-21T04:14:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T04:14:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.703 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-objecter_ops": { 2024-09-21T04:14:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T04:14:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 1024, 2024-09-21T04:14:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T04:14:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 0, 2024-09-21T04:14:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 0, 2024-09-21T04:14:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 0, 2024-09-21T04:14:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 155455, 2024-09-21T04:14:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 155455, 2024-09-21T04:14:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 155455, 2024-09-21T04:14:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 155455, 2024-09-21T04:14:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T04:14:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:18.704 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle": { 2024-09-21T04:14:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T04:14:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-09-21T04:14:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T04:14:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 32720, 2024-09-21T04:14:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 3907120, 2024-09-21T04:14:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 32720, 2024-09-21T04:14:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-09-21T04:14:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-09-21T04:14:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 642, 2024-09-21T04:14:18.705 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 3907120, 2024-09-21T04:14:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T04:14:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-write_buf_throttle-0x555615d2f5f0": { 2024-09-21T04:14:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T04:14:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 3758096384, 2024-09-21T04:14:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T04:14:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 239846, 2024-09-21T04:14:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 797237209, 2024-09-21T04:14:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:18.706 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 239846, 2024-09-21T04:14:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-09-21T04:14:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-09-21T04:14:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 19942, 2024-09-21T04:14:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 797237209, 2024-09-21T04:14:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T04:14:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: "trackedop": { 2024-09-21T04:14:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: "slow_ops_count": 14632 2024-09-21T04:14:18.707 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:18.707 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-21T04:14:18.708 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:18 smithi119 ceph-mon[34811]: pgmap v638: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-21T04:14:18.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:18 smithi064 ceph-mon[31180]: pgmap v638: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-21T04:14:18.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:18 smithi131 ceph-mon[35193]: pgmap v638: 97 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 90 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-21T04:14:19.383 INFO:tasks.check_counter:Daemon mds.h mds.exported=22 2024-09-21T04:14:19.383 INFO:tasks.check_counter:Daemon mds.h mds.imported=149 2024-09-21T04:14:19.383 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-21T04:14:19.383 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-21T04:14:19.383 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T04:14:19.383 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=133 2024-09-21T04:14:19.383 DEBUG:tasks.check_counter:Getting stats from k 2024-09-21T04:14:19.384 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph daemon mds.k perf dump 2024-09-21T04:14:19.635 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T04:14:19.755 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:19 smithi119 ceph-mon[34811]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:14:19.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:19 smithi064 ceph-mon[31180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:14:19.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:19 smithi131 ceph-mon[35193]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:14:20.167 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-09-21T04:14:20.168 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T04:14:20.168 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 4, 2024-09-21T04:14:20.168 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 4, 2024-09-21T04:14:20.168 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 2992, 2024-09-21T04:14:20.168 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 448, 2024-09-21T04:14:20.168 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 4, 2024-09-21T04:14:20.168 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 0, 2024-09-21T04:14:20.168 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.002908864, 2024-09-21T04:14:20.168 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.000043049, 2024-09-21T04:14:20.168 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.002169938, 2024-09-21T04:14:20.168 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T04:14:20.168 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:20.169 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 3, 2024-09-21T04:14:20.169 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.003807873, 2024-09-21T04:14:20.169 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.001269291 2024-09-21T04:14:20.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:20.169 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:20.169 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:20.169 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:20.169 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:20.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:20.169 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-21T04:14:20.169 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T04:14:20.169 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:20.169 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T04:14:20.169 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 216, 2024-09-21T04:14:20.169 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 209, 2024-09-21T04:14:20.170 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 1145552, 2024-09-21T04:14:20.170 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 128064, 2024-09-21T04:14:20.170 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 2, 2024-09-21T04:14:20.170 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 1, 2024-09-21T04:14:20.170 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.039542123, 2024-09-21T04:14:20.170 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.015756299, 2024-09-21T04:14:20.170 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.036010247, 2024-09-21T04:14:20.170 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T04:14:20.170 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:20.170 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 208, 2024-09-21T04:14:20.170 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.009914770, 2024-09-21T04:14:20.170 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000047667 2024-09-21T04:14:20.170 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:20.170 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:20.171 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:20.171 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:20.171 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:20.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:20.171 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 1145552, 2024-09-21T04:14:20.171 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 128064 2024-09-21T04:14:20.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:20.171 INFO:teuthology.orchestra.run.smithi119.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T04:14:20.171 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_messages": 1, 2024-09-21T04:14:20.171 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages": 158, 2024-09-21T04:14:20.171 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_bytes": 160, 2024-09-21T04:14:20.171 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_bytes": 108048, 2024-09-21T04:14:20.171 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_created_connections": 3, 2024-09-21T04:14:20.171 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_active_connections": 2, 2024-09-21T04:14:20.171 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_total_time": 0.010961372, 2024-09-21T04:14:20.172 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_send_time": 0.008498572, 2024-09-21T04:14:20.172 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_recv_time": 0.001110756, 2024-09-21T04:14:20.172 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T04:14:20.172 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:20.172 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 156, 2024-09-21T04:14:20.172 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.006717927, 2024-09-21T04:14:20.172 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000043063 2024-09-21T04:14:20.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:20.172 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:20.172 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:20.172 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:20.172 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:20.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:20.172 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T04:14:20.172 INFO:teuthology.orchestra.run.smithi119.stdout: "msgr_send_encrypted_bytes": 108048 2024-09-21T04:14:20.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:20.173 INFO:teuthology.orchestra.run.smithi119.stdout: "cct": { 2024-09-21T04:14:20.173 INFO:teuthology.orchestra.run.smithi119.stdout: "total_workers": 0, 2024-09-21T04:14:20.173 INFO:teuthology.orchestra.run.smithi119.stdout: "unhealthy_workers": 0 2024-09-21T04:14:20.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:20.173 INFO:teuthology.orchestra.run.smithi119.stdout: "mempool": { 2024-09-21T04:14:20.173 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_bytes": 0, 2024-09-21T04:14:20.173 INFO:teuthology.orchestra.run.smithi119.stdout: "bloom_filter_items": 0, 2024-09-21T04:14:20.173 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T04:14:20.173 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_alloc_items": 0, 2024-09-21T04:14:20.174 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T04:14:20.174 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_data_items": 0, 2024-09-21T04:14:20.174 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T04:14:20.174 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T04:14:20.174 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T04:14:20.174 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T04:14:20.174 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T04:14:20.174 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_other_items": 0, 2024-09-21T04:14:20.174 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T04:14:20.174 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T04:14:20.174 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_bytes": 0, 2024-09-21T04:14:20.174 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_extent_items": 0, 2024-09-21T04:14:20.174 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_bytes": 0, 2024-09-21T04:14:20.174 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_blob_items": 0, 2024-09-21T04:14:20.174 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T04:14:20.175 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T04:14:20.175 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T04:14:20.175 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T04:14:20.175 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T04:14:20.175 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_fsck_items": 0, 2024-09-21T04:14:20.175 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_bytes": 0, 2024-09-21T04:14:20.175 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_txc_items": 0, 2024-09-21T04:14:20.175 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T04:14:20.175 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T04:14:20.175 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_bytes": 0, 2024-09-21T04:14:20.175 INFO:teuthology.orchestra.run.smithi119.stdout: "bluestore_writing_items": 0, 2024-09-21T04:14:20.176 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_bytes": 0, 2024-09-21T04:14:20.176 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_items": 0, 2024-09-21T04:14:20.176 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T04:14:20.176 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_reader_items": 0, 2024-09-21T04:14:20.176 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T04:14:20.176 INFO:teuthology.orchestra.run.smithi119.stdout: "bluefs_file_writer_items": 0, 2024-09-21T04:14:20.176 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_bytes": 148967, 2024-09-21T04:14:20.176 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_anon_items": 29, 2024-09-21T04:14:20.176 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_bytes": 0, 2024-09-21T04:14:20.176 INFO:teuthology.orchestra.run.smithi119.stdout: "buffer_meta_items": 0, 2024-09-21T04:14:20.176 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_bytes": 0, 2024-09-21T04:14:20.176 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_items": 0, 2024-09-21T04:14:20.176 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_bytes": 0, 2024-09-21T04:14:20.177 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_mapbl_items": 0, 2024-09-21T04:14:20.177 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_bytes": 0, 2024-09-21T04:14:20.177 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_pglog_items": 0, 2024-09-21T04:14:20.177 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_bytes": 0, 2024-09-21T04:14:20.177 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_items": 0, 2024-09-21T04:14:20.177 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T04:14:20.177 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_mapping_items": 0, 2024-09-21T04:14:20.177 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_bytes": 0, 2024-09-21T04:14:20.177 INFO:teuthology.orchestra.run.smithi119.stdout: "pgmap_items": 0, 2024-09-21T04:14:20.177 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_bytes": 968, 2024-09-21T04:14:20.177 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_co_items": 2, 2024-09-21T04:14:20.177 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_bytes": 0, 2024-09-21T04:14:20.178 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_1_items": 0, 2024-09-21T04:14:20.178 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_bytes": 0, 2024-09-21T04:14:20.178 INFO:teuthology.orchestra.run.smithi119.stdout: "unittest_2_items": 0 2024-09-21T04:14:20.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2024-09-21T04:14:20.178 INFO:teuthology.orchestra.run.smithi119.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T04:14:20.178 INFO:teuthology.orchestra.run.smithi119.stdout: "val": 0, 2024-09-21T04:14:20.178 INFO:teuthology.orchestra.run.smithi119.stdout: "max": 104857600, 2024-09-21T04:14:20.178 INFO:teuthology.orchestra.run.smithi119.stdout: "get_started": 0, 2024-09-21T04:14:20.178 INFO:teuthology.orchestra.run.smithi119.stdout: "get": 217, 2024-09-21T04:14:20.178 INFO:teuthology.orchestra.run.smithi119.stdout: "get_sum": 1115028, 2024-09-21T04:14:20.178 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:20.178 INFO:teuthology.orchestra.run.smithi119.stdout: "get_or_fail_success": 217, 2024-09-21T04:14:20.179 INFO:teuthology.orchestra.run.smithi119.stdout: "take": 0, 2024-09-21T04:14:20.179 INFO:teuthology.orchestra.run.smithi119.stdout: "take_sum": 0, 2024-09-21T04:14:20.179 INFO:teuthology.orchestra.run.smithi119.stdout: "put": 217, 2024-09-21T04:14:20.179 INFO:teuthology.orchestra.run.smithi119.stdout: "put_sum": 1115028, 2024-09-21T04:14:20.179 INFO:teuthology.orchestra.run.smithi119.stdout: "wait": { 2024-09-21T04:14:20.179 INFO:teuthology.orchestra.run.smithi119.stdout: "avgcount": 0, 2024-09-21T04:14:20.179 INFO:teuthology.orchestra.run.smithi119.stdout: "sum": 0.000000000, 2024-09-21T04:14:20.179 INFO:teuthology.orchestra.run.smithi119.stdout: "avgtime": 0.000000000 2024-09-21T04:14:20.179 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:20.179 INFO:teuthology.orchestra.run.smithi119.stdout: } 2024-09-21T04:14:20.179 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-09-21T04:14:20.705 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:20 smithi131 ceph-mon[35193]: pgmap v639: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 198 B/s wr, 6 op/s 2024-09-21T04:14:20.705 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:20 smithi131 ceph-mon[35193]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:14:20.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:20 smithi119 ceph-mon[34811]: pgmap v639: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 198 B/s wr, 6 op/s 2024-09-21T04:14:20.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:20 smithi119 ceph-mon[34811]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:14:20.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:20 smithi064 ceph-mon[31180]: pgmap v639: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 198 B/s wr, 6 op/s 2024-09-21T04:14:20.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:20 smithi064 ceph-mon[31180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-21T04:14:20.829 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-21T04:14:20.829 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-21T04:14:20.829 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-09-21T04:14:20.829 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-09-21T04:14:20.829 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-09-21T04:14:20.829 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-09-21T04:14:20.829 DEBUG:tasks.check_counter:Getting stats from c 2024-09-21T04:14:20.830 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph daemon mds.c perf dump 2024-09-21T04:14:21.088 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T04:14:21.589 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-21T04:14:21.589 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T04:14:21.589 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 1807, 2024-09-21T04:14:21.589 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 1809, 2024-09-21T04:14:21.590 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 289944746, 2024-09-21T04:14:21.590 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 574633, 2024-09-21T04:14:21.590 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 8, 2024-09-21T04:14:21.590 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 4, 2024-09-21T04:14:21.590 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.324080519, 2024-09-21T04:14:21.590 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.085748965, 2024-09-21T04:14:21.590 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.170126369, 2024-09-21T04:14:21.590 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.053206661, 2024-09-21T04:14:21.590 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:21.590 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1807, 2024-09-21T04:14:21.590 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.070688026, 2024-09-21T04:14:21.590 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000039118 2024-09-21T04:14:21.590 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.591 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:21.591 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.591 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.591 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.591 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.591 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T04:14:21.591 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T04:14:21.591 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.591 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T04:14:21.591 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 566, 2024-09-21T04:14:21.591 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 721, 2024-09-21T04:14:21.591 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 222465518, 2024-09-21T04:14:21.591 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 537668, 2024-09-21T04:14:21.591 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 6, 2024-09-21T04:14:21.592 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 5, 2024-09-21T04:14:21.592 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.182492190, 2024-09-21T04:14:21.592 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.028224170, 2024-09-21T04:14:21.592 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.133386999, 2024-09-21T04:14:21.592 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.014405460, 2024-09-21T04:14:21.592 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:21.592 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 719, 2024-09-21T04:14:21.592 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.190661169, 2024-09-21T04:14:21.592 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000265175 2024-09-21T04:14:21.592 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.592 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:21.592 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.592 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.592 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.593 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.593 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T04:14:21.593 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 357936 2024-09-21T04:14:21.593 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.593 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T04:14:21.593 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 1313, 2024-09-21T04:14:21.593 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 1283, 2024-09-21T04:14:21.593 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 287097078, 2024-09-21T04:14:21.593 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 465696, 2024-09-21T04:14:21.593 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 5, 2024-09-21T04:14:21.593 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 4, 2024-09-21T04:14:21.593 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.265531691, 2024-09-21T04:14:21.593 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.055863468, 2024-09-21T04:14:21.593 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.190209329, 2024-09-21T04:14:21.594 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.025832974, 2024-09-21T04:14:21.594 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:21.594 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1283, 2024-09-21T04:14:21.594 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.055219882, 2024-09-21T04:14:21.594 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000043039 2024-09-21T04:14:21.594 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.594 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:21.594 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.594 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.594 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.594 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.594 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 1261168, 2024-09-21T04:14:21.594 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 129888 2024-09-21T04:14:21.594 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.595 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-09-21T04:14:21.595 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-09-21T04:14:21.595 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-09-21T04:14:21.595 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.595 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-09-21T04:14:21.595 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-09-21T04:14:21.595 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-09-21T04:14:21.595 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 5153, 2024-09-21T04:14:21.595 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.403327336, 2024-09-21T04:14:21.595 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000078270 2024-09-21T04:14:21.595 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:21.595 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.595 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-09-21T04:14:21.595 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-09-21T04:14:21.596 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-09-21T04:14:21.596 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.596 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.596 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.596 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:21.596 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.596 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-09-21T04:14:21.596 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 0, 2024-09-21T04:14:21.596 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 0, 2024-09-21T04:14:21.596 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-09-21T04:14:21.596 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.596 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.596 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.596 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.596 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-09-21T04:14:21.597 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 0, 2024-09-21T04:14:21.597 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 0, 2024-09-21T04:14:21.597 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 0, 2024-09-21T04:14:21.597 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 0, 2024-09-21T04:14:21.597 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 0, 2024-09-21T04:14:21.597 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-09-21T04:14:21.597 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 117102, 2024-09-21T04:14:21.597 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 34914, 2024-09-21T04:14:21.597 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 82188, 2024-09-21T04:14:21.597 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 0, 2024-09-21T04:14:21.597 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 67224, 2024-09-21T04:14:21.598 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 0, 2024-09-21T04:14:21.598 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 0, 2024-09-21T04:14:21.598 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 0, 2024-09-21T04:14:21.598 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 1, 2024-09-21T04:14:21.598 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 0, 2024-09-21T04:14:21.598 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 0, 2024-09-21T04:14:21.598 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 0, 2024-09-21T04:14:21.598 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 0, 2024-09-21T04:14:21.598 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-09-21T04:14:21.598 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-09-21T04:14:21.598 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 0, 2024-09-21T04:14:21.598 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 0, 2024-09-21T04:14:21.598 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 0, 2024-09-21T04:14:21.599 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 0, 2024-09-21T04:14:21.599 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 0, 2024-09-21T04:14:21.599 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 0, 2024-09-21T04:14:21.599 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 0, 2024-09-21T04:14:21.599 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-09-21T04:14:21.599 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 0, 2024-09-21T04:14:21.599 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 0, 2024-09-21T04:14:21.599 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 0, 2024-09-21T04:14:21.599 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 0, 2024-09-21T04:14:21.599 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 0, 2024-09-21T04:14:21.599 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T04:14:21.599 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-09-21T04:14:21.599 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T04:14:21.600 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-09-21T04:14:21.600 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 0, 2024-09-21T04:14:21.600 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T04:14:21.600 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T04:14:21.600 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 0, 2024-09-21T04:14:21.600 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 0, 2024-09-21T04:14:21.600 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T04:14:21.600 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 0, 2024-09-21T04:14:21.600 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T04:14:21.600 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T04:14:21.600 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T04:14:21.600 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 0, 2024-09-21T04:14:21.601 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 0, 2024-09-21T04:14:21.601 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 0, 2024-09-21T04:14:21.601 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 0 2024-09-21T04:14:21.601 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.601 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-09-21T04:14:21.601 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 0, 2024-09-21T04:14:21.601 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 0, 2024-09-21T04:14:21.601 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 0, 2024-09-21T04:14:21.601 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 0, 2024-09-21T04:14:21.601 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 0, 2024-09-21T04:14:21.601 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 0, 2024-09-21T04:14:21.601 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-09-21T04:14:21.601 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-09-21T04:14:21.601 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 0, 2024-09-21T04:14:21.602 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 0, 2024-09-21T04:14:21.602 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 0, 2024-09-21T04:14:21.602 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-09-21T04:14:21.602 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-09-21T04:14:21.602 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-09-21T04:14:21.602 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-09-21T04:14:21.602 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 0, 2024-09-21T04:14:21.602 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 0, 2024-09-21T04:14:21.602 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_path": 0, 2024-09-21T04:14:21.602 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_inode": 0, 2024-09-21T04:14:21.602 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T04:14:21.602 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 0, 2024-09-21T04:14:21.602 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-09-21T04:14:21.602 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 0, 2024-09-21T04:14:21.602 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-09-21T04:14:21.603 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0, 2024-09-21T04:14:21.603 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-21T04:14:21.603 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_succeeded": 0, 2024-09-21T04:14:21.603 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_write_failed": 0 2024-09-21T04:14:21.603 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.603 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-09-21T04:14:21.603 INFO:teuthology.orchestra.run.smithi131.stdout: "evlrg": 0, 2024-09-21T04:14:21.603 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 0, 2024-09-21T04:14:21.603 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-09-21T04:14:21.603 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-09-21T04:14:21.603 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 239846, 2024-09-21T04:14:21.603 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-09-21T04:14:21.603 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-09-21T04:14:21.603 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 0, 2024-09-21T04:14:21.603 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-09-21T04:14:21.604 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-09-21T04:14:21.604 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 128, 2024-09-21T04:14:21.604 INFO:teuthology.orchestra.run.smithi131.stdout: "segmjr": 21, 2024-09-21T04:14:21.604 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-09-21T04:14:21.604 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-09-21T04:14:21.604 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 499252318, 2024-09-21T04:14:21.604 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 801431513, 2024-09-21T04:14:21.604 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 801426114, 2024-09-21T04:14:21.604 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-09-21T04:14:21.604 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.604 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.604 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.604 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.605 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 239846 2024-09-21T04:14:21.605 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.605 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-09-21T04:14:21.605 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 42872, 2024-09-21T04:14:21.605 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 76292, 2024-09-21T04:14:21.605 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 33420, 2024-09-21T04:14:21.605 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 1143, 2024-09-21T04:14:21.605 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 1397, 2024-09-21T04:14:21.605 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 254, 2024-09-21T04:14:21.605 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 117102, 2024-09-21T04:14:21.605 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 118946, 2024-09-21T04:14:21.605 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 1844, 2024-09-21T04:14:21.605 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 0, 2024-09-21T04:14:21.605 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 0, 2024-09-21T04:14:21.605 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 0, 2024-09-21T04:14:21.606 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 329088, 2024-09-21T04:14:21.606 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 538120192 2024-09-21T04:14:21.606 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.606 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-09-21T04:14:21.606 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 0, 2024-09-21T04:14:21.606 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 0, 2024-09-21T04:14:21.606 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 0, 2024-09-21T04:14:21.606 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 0, 2024-09-21T04:14:21.606 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 0, 2024-09-21T04:14:21.606 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-09-21T04:14:21.606 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.606 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.606 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.606 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.606 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-09-21T04:14:21.607 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.607 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.607 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.607 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.607 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-09-21T04:14:21.607 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.607 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.607 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.607 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.607 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-09-21T04:14:21.607 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.607 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.607 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.607 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.607 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-09-21T04:14:21.608 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.608 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.608 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.608 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.608 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-09-21T04:14:21.608 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.608 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.608 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.608 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.608 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-09-21T04:14:21.608 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.608 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.608 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.608 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.608 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-09-21T04:14:21.609 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.609 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.609 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.609 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.609 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-09-21T04:14:21.609 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.609 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.609 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.609 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.609 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-09-21T04:14:21.609 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.609 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.609 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.609 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.610 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-09-21T04:14:21.610 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.610 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.610 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.610 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.610 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-09-21T04:14:21.611 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.611 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.611 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.611 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.611 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-09-21T04:14:21.611 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.611 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.611 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.611 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.611 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-09-21T04:14:21.611 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.611 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.611 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.611 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.612 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-09-21T04:14:21.612 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.612 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.612 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.612 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.612 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-09-21T04:14:21.612 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.612 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.612 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.612 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.612 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-09-21T04:14:21.612 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.612 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.612 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.613 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.613 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-09-21T04:14:21.613 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.613 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.613 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.613 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.613 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-09-21T04:14:21.613 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.613 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.613 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.613 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.613 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-09-21T04:14:21.613 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.613 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.614 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.614 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.614 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-09-21T04:14:21.614 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.614 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.614 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.614 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.614 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-09-21T04:14:21.614 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.614 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.614 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.614 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.614 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-09-21T04:14:21.615 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.615 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.615 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.615 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.615 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-09-21T04:14:21.615 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.615 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.615 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.615 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.615 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-09-21T04:14:21.615 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.615 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.615 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.615 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.616 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-09-21T04:14:21.616 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.616 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.616 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.616 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.616 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-09-21T04:14:21.616 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.616 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.616 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.616 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.616 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-09-21T04:14:21.616 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.616 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.616 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.616 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.616 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-09-21T04:14:21.617 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.617 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.617 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.617 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.617 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-09-21T04:14:21.617 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-09-21T04:14:21.617 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-09-21T04:14:21.617 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.617 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.617 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.617 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:21.617 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.617 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-09-21T04:14:21.618 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 1, 2024-09-21T04:14:21.618 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 2, 2024-09-21T04:14:21.618 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 1, 2024-09-21T04:14:21.618 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 2, 2024-09-21T04:14:21.618 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-09-21T04:14:21.618 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 0, 2024-09-21T04:14:21.618 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 0, 2024-09-21T04:14:21.618 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 3242, 2024-09-21T04:14:21.618 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-09-21T04:14:21.618 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.618 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-09-21T04:14:21.618 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-09-21T04:14:21.618 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-09-21T04:14:21.618 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T04:14:21.619 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-09-21T04:14:21.619 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T04:14:21.619 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-09-21T04:14:21.619 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T04:14:21.619 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T04:14:21.619 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T04:14:21.619 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T04:14:21.619 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T04:14:21.619 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-09-21T04:14:21.619 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T04:14:21.619 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T04:14:21.619 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-09-21T04:14:21.619 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-09-21T04:14:21.619 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-09-21T04:14:21.620 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-09-21T04:14:21.620 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T04:14:21.620 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T04:14:21.620 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T04:14:21.620 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T04:14:21.620 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T04:14:21.620 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-09-21T04:14:21.620 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-09-21T04:14:21.620 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-09-21T04:14:21.620 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T04:14:21.620 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T04:14:21.620 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-09-21T04:14:21.620 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-09-21T04:14:21.620 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-09-21T04:14:21.620 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-09-21T04:14:21.621 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T04:14:21.621 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-09-21T04:14:21.621 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T04:14:21.621 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-09-21T04:14:21.621 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 1836088, 2024-09-21T04:14:21.621 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 43465, 2024-09-21T04:14:21.621 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-09-21T04:14:21.621 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-09-21T04:14:21.621 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-09-21T04:14:21.621 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-09-21T04:14:21.621 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-09-21T04:14:21.621 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-09-21T04:14:21.621 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-09-21T04:14:21.621 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-09-21T04:14:21.622 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 8664, 2024-09-21T04:14:21.622 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 191, 2024-09-21T04:14:21.622 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T04:14:21.622 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-09-21T04:14:21.622 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-09-21T04:14:21.622 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-09-21T04:14:21.622 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 181827651, 2024-09-21T04:14:21.622 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 1826181, 2024-09-21T04:14:21.622 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-09-21T04:14:21.622 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-09-21T04:14:21.622 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-09-21T04:14:21.622 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-09-21T04:14:21.622 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.622 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-09-21T04:14:21.623 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-09-21T04:14:21.623 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-09-21T04:14:21.623 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 3428, 2024-09-21T04:14:21.623 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 21, 2024-09-21T04:14:21.623 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 3, 2024-09-21T04:14:21.623 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 3428, 2024-09-21T04:14:21.624 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-09-21T04:14:21.624 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 3428, 2024-09-21T04:14:21.624 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 5.854866876, 2024-09-21T04:14:21.624 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001707954 2024-09-21T04:14:21.624 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.624 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-09-21T04:14:21.624 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-09-21T04:14:21.624 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 3428, 2024-09-21T04:14:21.624 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 3429 2024-09-21T04:14:21.624 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.624 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 3428, 2024-09-21T04:14:21.624 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 3428, 2024-09-21T04:14:21.624 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 0, 2024-09-21T04:14:21.624 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-09-21T04:14:21.625 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-09-21T04:14:21.625 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 2179, 2024-09-21T04:14:21.625 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 0, 2024-09-21T04:14:21.625 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 1247, 2024-09-21T04:14:21.625 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 0, 2024-09-21T04:14:21.625 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 0, 2024-09-21T04:14:21.625 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-09-21T04:14:21.625 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-09-21T04:14:21.625 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 0, 2024-09-21T04:14:21.625 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 0, 2024-09-21T04:14:21.625 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 0, 2024-09-21T04:14:21.626 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-09-21T04:14:21.626 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-09-21T04:14:21.626 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-09-21T04:14:21.626 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 1, 2024-09-21T04:14:21.626 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 0, 2024-09-21T04:14:21.626 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-09-21T04:14:21.626 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-09-21T04:14:21.626 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-09-21T04:14:21.626 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-09-21T04:14:21.626 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-09-21T04:14:21.626 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-09-21T04:14:21.626 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T04:14:21.626 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-09-21T04:14:21.626 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-09-21T04:14:21.627 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 0, 2024-09-21T04:14:21.627 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-09-21T04:14:21.627 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-09-21T04:14:21.627 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-09-21T04:14:21.627 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-09-21T04:14:21.627 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-09-21T04:14:21.627 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-09-21T04:14:21.627 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-09-21T04:14:21.627 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-09-21T04:14:21.627 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-09-21T04:14:21.627 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-09-21T04:14:21.627 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-09-21T04:14:21.627 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-09-21T04:14:21.627 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-09-21T04:14:21.628 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-09-21T04:14:21.628 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-09-21T04:14:21.628 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-09-21T04:14:21.628 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 87, 2024-09-21T04:14:21.628 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-09-21T04:14:21.628 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 14, 2024-09-21T04:14:21.628 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 10, 2024-09-21T04:14:21.628 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 10, 2024-09-21T04:14:21.628 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 0, 2024-09-21T04:14:21.628 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-09-21T04:14:21.628 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 0, 2024-09-21T04:14:21.628 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 2, 2024-09-21T04:14:21.628 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 0 2024-09-21T04:14:21.628 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.628 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-09-21T04:14:21.629 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 0, 2024-09-21T04:14:21.629 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-09-21T04:14:21.629 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-09-21T04:14:21.629 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-09-21T04:14:21.629 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.629 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-09-21T04:14:21.629 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-09-21T04:14:21.629 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T04:14:21.629 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-09-21T04:14:21.629 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 0, 2024-09-21T04:14:21.629 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed_ops": 0, 2024-09-21T04:14:21.629 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 0, 2024-09-21T04:14:21.629 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-09-21T04:14:21.629 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.629 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T04:14:21.630 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-09-21T04:14:21.630 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-09-21T04:14:21.630 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-09-21T04:14:21.630 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 3682, 2024-09-21T04:14:21.630 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 799160088, 2024-09-21T04:14:21.630 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:21.630 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 3682, 2024-09-21T04:14:21.630 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-09-21T04:14:21.630 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-09-21T04:14:21.630 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 3682, 2024-09-21T04:14:21.630 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 799160088, 2024-09-21T04:14:21.630 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-09-21T04:14:21.630 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.630 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.631 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.631 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:21.631 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.631 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-09-21T04:14:21.631 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-09-21T04:14:21.631 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-09-21T04:14:21.631 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-09-21T04:14:21.631 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-09-21T04:14:21.631 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-09-21T04:14:21.631 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:21.631 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-09-21T04:14:21.631 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 3428, 2024-09-21T04:14:21.631 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 797308932, 2024-09-21T04:14:21.631 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 3428, 2024-09-21T04:14:21.632 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 797308932, 2024-09-21T04:14:21.632 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-09-21T04:14:21.632 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.632 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.632 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.632 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:21.632 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.632 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-09-21T04:14:21.632 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-09-21T04:14:21.632 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-09-21T04:14:21.632 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-09-21T04:14:21.632 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-09-21T04:14:21.632 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-09-21T04:14:21.632 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:21.633 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-09-21T04:14:21.633 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 3428, 2024-09-21T04:14:21.633 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 3428, 2024-09-21T04:14:21.633 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 3428, 2024-09-21T04:14:21.633 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 3428, 2024-09-21T04:14:21.633 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-09-21T04:14:21.633 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.633 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.633 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.633 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:21.633 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.633 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-09-21T04:14:21.633 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-09-21T04:14:21.633 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-09-21T04:14:21.634 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-09-21T04:14:21.634 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-09-21T04:14:21.634 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-09-21T04:14:21.634 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:21.634 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-09-21T04:14:21.634 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-09-21T04:14:21.634 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-09-21T04:14:21.634 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-09-21T04:14:21.634 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-09-21T04:14:21.634 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-09-21T04:14:21.634 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.634 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.634 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.634 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:21.635 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.635 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x56102f2932b0": { 2024-09-21T04:14:21.635 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-09-21T04:14:21.635 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-09-21T04:14:21.635 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-09-21T04:14:21.635 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-09-21T04:14:21.635 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-09-21T04:14:21.635 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:21.635 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-09-21T04:14:21.635 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-09-21T04:14:21.635 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-09-21T04:14:21.635 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-09-21T04:14:21.635 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-09-21T04:14:21.635 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-09-21T04:14:21.635 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:21.636 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:21.636 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:21.636 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:21.636 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:21.636 INFO:teuthology.orchestra.run.smithi131.stdout: "trackedop": { 2024-09-21T04:14:21.636 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 0 2024-09-21T04:14:21.636 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:21.636 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-21T04:14:22.510 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:22 smithi119 ceph-mon[34811]: pgmap v640: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T04:14:22.510 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:22 smithi119 ceph-mon[34811]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T04:14:22.510 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:22 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:14:22.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:22 smithi064 ceph-mon[31180]: pgmap v640: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T04:14:22.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:22 smithi064 ceph-mon[31180]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T04:14:22.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:22 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:14:22.561 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:22 smithi131 ceph-mon[35193]: pgmap v640: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-21T04:14:22.561 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:22 smithi131 ceph-mon[35193]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T04:14:22.561 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:22 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T04:14:22.618 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-21T04:14:22.618 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-21T04:14:22.618 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-09-21T04:14:22.618 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-21T04:14:22.618 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T04:14:22.618 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-09-21T04:14:22.618 DEBUG:tasks.check_counter:Getting stats from f 2024-09-21T04:14:22.619 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph daemon mds.f perf dump 2024-09-21T04:14:23.517 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T04:14:24.053 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-21T04:14:24.054 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T04:14:24.054 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 425, 2024-09-21T04:14:24.054 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 397, 2024-09-21T04:14:24.054 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 5496670, 2024-09-21T04:14:24.054 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 184464, 2024-09-21T04:14:24.054 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 6, 2024-09-21T04:14:24.054 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 2, 2024-09-21T04:14:24.054 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.063280028, 2024-09-21T04:14:24.054 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.024913931, 2024-09-21T04:14:24.054 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.043884571, 2024-09-21T04:14:24.054 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.004282804, 2024-09-21T04:14:24.054 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:24.054 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 396, 2024-09-21T04:14:24.055 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.019982595, 2024-09-21T04:14:24.055 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000050461 2024-09-21T04:14:24.055 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.055 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:24.055 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.055 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.055 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.055 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.055 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 1258448, 2024-09-21T04:14:24.055 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 130080 2024-09-21T04:14:24.055 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.055 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T04:14:24.055 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 654, 2024-09-21T04:14:24.055 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 808, 2024-09-21T04:14:24.056 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 12749036, 2024-09-21T04:14:24.056 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 564963, 2024-09-21T04:14:24.056 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 5, 2024-09-21T04:14:24.056 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 4, 2024-09-21T04:14:24.056 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.087185382, 2024-09-21T04:14:24.056 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.039424602, 2024-09-21T04:14:24.056 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.023504382, 2024-09-21T04:14:24.056 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.017963076, 2024-09-21T04:14:24.056 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:24.056 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 807, 2024-09-21T04:14:24.056 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.037721284, 2024-09-21T04:14:24.056 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000046742 2024-09-21T04:14:24.056 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.057 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:24.057 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.057 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.057 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.057 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.057 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-21T04:14:24.057 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 358576 2024-09-21T04:14:24.057 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.057 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T04:14:24.057 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 1473, 2024-09-21T04:14:24.057 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 1475, 2024-09-21T04:14:24.057 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 13111902, 2024-09-21T04:14:24.057 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 468472, 2024-09-21T04:14:24.057 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 5, 2024-09-21T04:14:24.058 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 4, 2024-09-21T04:14:24.058 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.180739523, 2024-09-21T04:14:24.058 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.072784170, 2024-09-21T04:14:24.058 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.047647708, 2024-09-21T04:14:24.058 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.047526468, 2024-09-21T04:14:24.058 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:24.058 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1473, 2024-09-21T04:14:24.058 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.062266569, 2024-09-21T04:14:24.058 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000042271 2024-09-21T04:14:24.058 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.058 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:24.058 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.058 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.058 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.059 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.059 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T04:14:24.059 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T04:14:24.059 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.059 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-09-21T04:14:24.059 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-09-21T04:14:24.059 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-09-21T04:14:24.059 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.059 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-09-21T04:14:24.059 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-09-21T04:14:24.059 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-09-21T04:14:24.059 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 3887, 2024-09-21T04:14:24.059 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.107332831, 2024-09-21T04:14:24.060 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000027613 2024-09-21T04:14:24.060 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:24.060 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.060 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-09-21T04:14:24.060 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-09-21T04:14:24.060 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-09-21T04:14:24.060 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.060 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.060 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.060 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:24.060 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.060 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-09-21T04:14:24.060 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 0, 2024-09-21T04:14:24.060 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 0, 2024-09-21T04:14:24.061 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-09-21T04:14:24.061 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.061 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.061 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.061 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.061 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-09-21T04:14:24.061 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 0, 2024-09-21T04:14:24.061 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 0, 2024-09-21T04:14:24.061 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 0, 2024-09-21T04:14:24.061 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 0, 2024-09-21T04:14:24.061 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 0, 2024-09-21T04:14:24.061 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-09-21T04:14:24.061 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 1589, 2024-09-21T04:14:24.061 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 0, 2024-09-21T04:14:24.062 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 1589, 2024-09-21T04:14:24.062 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 0, 2024-09-21T04:14:24.062 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 1589, 2024-09-21T04:14:24.062 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 0, 2024-09-21T04:14:24.062 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 0, 2024-09-21T04:14:24.062 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 0, 2024-09-21T04:14:24.062 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 1, 2024-09-21T04:14:24.062 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 0, 2024-09-21T04:14:24.062 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 0, 2024-09-21T04:14:24.062 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 0, 2024-09-21T04:14:24.062 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 0, 2024-09-21T04:14:24.062 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-09-21T04:14:24.062 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-09-21T04:14:24.062 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 0, 2024-09-21T04:14:24.063 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 0, 2024-09-21T04:14:24.063 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 0, 2024-09-21T04:14:24.063 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 0, 2024-09-21T04:14:24.063 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 0, 2024-09-21T04:14:24.063 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 0, 2024-09-21T04:14:24.063 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 0, 2024-09-21T04:14:24.063 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-09-21T04:14:24.063 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 0, 2024-09-21T04:14:24.063 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 0, 2024-09-21T04:14:24.063 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 0, 2024-09-21T04:14:24.063 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 0, 2024-09-21T04:14:24.063 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 0, 2024-09-21T04:14:24.063 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T04:14:24.063 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-09-21T04:14:24.063 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T04:14:24.064 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-09-21T04:14:24.064 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 0, 2024-09-21T04:14:24.064 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T04:14:24.064 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T04:14:24.064 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 0, 2024-09-21T04:14:24.064 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 0, 2024-09-21T04:14:24.064 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T04:14:24.064 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 0, 2024-09-21T04:14:24.064 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T04:14:24.064 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T04:14:24.064 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T04:14:24.064 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 0, 2024-09-21T04:14:24.064 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 0, 2024-09-21T04:14:24.064 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 0, 2024-09-21T04:14:24.065 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 0 2024-09-21T04:14:24.065 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.065 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-09-21T04:14:24.065 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 0, 2024-09-21T04:14:24.065 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 0, 2024-09-21T04:14:24.065 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 0, 2024-09-21T04:14:24.065 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 0, 2024-09-21T04:14:24.065 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 0, 2024-09-21T04:14:24.065 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 0, 2024-09-21T04:14:24.065 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-09-21T04:14:24.065 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-09-21T04:14:24.065 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 0, 2024-09-21T04:14:24.065 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 0, 2024-09-21T04:14:24.065 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 0, 2024-09-21T04:14:24.065 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-09-21T04:14:24.066 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-09-21T04:14:24.066 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-09-21T04:14:24.066 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-09-21T04:14:24.066 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 0, 2024-09-21T04:14:24.066 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 0, 2024-09-21T04:14:24.066 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_path": 0, 2024-09-21T04:14:24.066 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_inode": 0, 2024-09-21T04:14:24.066 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T04:14:24.066 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 0, 2024-09-21T04:14:24.066 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-09-21T04:14:24.066 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 0, 2024-09-21T04:14:24.066 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-09-21T04:14:24.066 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0, 2024-09-21T04:14:24.066 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-21T04:14:24.067 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_succeeded": 0, 2024-09-21T04:14:24.067 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_write_failed": 0 2024-09-21T04:14:24.067 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.067 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-09-21T04:14:24.067 INFO:teuthology.orchestra.run.smithi131.stdout: "evlrg": 0, 2024-09-21T04:14:24.067 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 0, 2024-09-21T04:14:24.067 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-09-21T04:14:24.067 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-09-21T04:14:24.067 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 15719, 2024-09-21T04:14:24.067 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-09-21T04:14:24.067 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-09-21T04:14:24.067 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 0, 2024-09-21T04:14:24.067 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-09-21T04:14:24.067 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-09-21T04:14:24.068 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 20, 2024-09-21T04:14:24.068 INFO:teuthology.orchestra.run.smithi131.stdout: "segmjr": 3, 2024-09-21T04:14:24.068 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-09-21T04:14:24.068 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-09-21T04:14:24.068 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 4194304, 2024-09-21T04:14:24.068 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 33646576, 2024-09-21T04:14:24.068 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 33639714, 2024-09-21T04:14:24.068 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-09-21T04:14:24.068 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.068 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.068 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.068 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.068 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 15719 2024-09-21T04:14:24.068 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.069 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-09-21T04:14:24.069 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 24, 2024-09-21T04:14:24.069 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 5794, 2024-09-21T04:14:24.069 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 5770, 2024-09-21T04:14:24.069 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 19, 2024-09-21T04:14:24.069 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 178, 2024-09-21T04:14:24.069 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 159, 2024-09-21T04:14:24.069 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 1589, 2024-09-21T04:14:24.069 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 9550, 2024-09-21T04:14:24.069 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 7961, 2024-09-21T04:14:24.069 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 0, 2024-09-21T04:14:24.069 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 0, 2024-09-21T04:14:24.069 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 0, 2024-09-21T04:14:24.069 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 128392, 2024-09-21T04:14:24.069 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 333930496 2024-09-21T04:14:24.070 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.070 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-09-21T04:14:24.070 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 0, 2024-09-21T04:14:24.070 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 0, 2024-09-21T04:14:24.070 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 0, 2024-09-21T04:14:24.070 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 0, 2024-09-21T04:14:24.070 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 0, 2024-09-21T04:14:24.070 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-09-21T04:14:24.070 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.070 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.070 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.070 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.070 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-09-21T04:14:24.070 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.070 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.071 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.071 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.071 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-09-21T04:14:24.071 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.071 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.071 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.071 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.071 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-09-21T04:14:24.071 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.071 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.071 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.071 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.071 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-09-21T04:14:24.071 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.072 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.072 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.072 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.072 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-09-21T04:14:24.072 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.072 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.072 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.072 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.072 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-09-21T04:14:24.072 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.072 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.072 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.072 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.072 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-09-21T04:14:24.073 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.073 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.073 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.073 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.073 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-09-21T04:14:24.073 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.073 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.073 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.073 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.073 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-09-21T04:14:24.073 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.073 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.073 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.073 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.073 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-09-21T04:14:24.074 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.074 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.074 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.074 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.074 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-09-21T04:14:24.075 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.075 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.075 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.075 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.075 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-09-21T04:14:24.075 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.075 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.075 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.075 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.075 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-09-21T04:14:24.075 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.075 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.075 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.075 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.076 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-09-21T04:14:24.076 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.076 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.076 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.076 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.076 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-09-21T04:14:24.076 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.076 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.076 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.076 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.076 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-09-21T04:14:24.076 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.076 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.076 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.076 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.077 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-09-21T04:14:24.077 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.077 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.077 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.077 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.077 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-09-21T04:14:24.077 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.077 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.077 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.077 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.077 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-09-21T04:14:24.077 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.077 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.077 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.077 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.078 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-09-21T04:14:24.078 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.078 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.078 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.078 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.078 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-09-21T04:14:24.078 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.078 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.078 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.078 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.078 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-09-21T04:14:24.078 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.078 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.078 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.079 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.079 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-09-21T04:14:24.079 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.079 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.079 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.079 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.079 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-09-21T04:14:24.079 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.079 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.079 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.079 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.079 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-09-21T04:14:24.079 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.079 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.080 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.080 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.080 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-09-21T04:14:24.080 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.080 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.080 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.080 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.080 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-09-21T04:14:24.080 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.080 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.080 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.080 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.080 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-09-21T04:14:24.080 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.081 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.081 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.081 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.081 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-09-21T04:14:24.081 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-09-21T04:14:24.081 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-09-21T04:14:24.081 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.081 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.081 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.081 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:24.081 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.081 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-09-21T04:14:24.081 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 1, 2024-09-21T04:14:24.081 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 2, 2024-09-21T04:14:24.082 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 1, 2024-09-21T04:14:24.082 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 2, 2024-09-21T04:14:24.082 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-09-21T04:14:24.082 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 0, 2024-09-21T04:14:24.082 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 0, 2024-09-21T04:14:24.082 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 3245, 2024-09-21T04:14:24.082 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-09-21T04:14:24.082 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.082 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-09-21T04:14:24.082 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-09-21T04:14:24.082 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-09-21T04:14:24.082 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T04:14:24.082 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-09-21T04:14:24.082 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T04:14:24.083 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-09-21T04:14:24.083 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T04:14:24.083 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T04:14:24.083 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T04:14:24.083 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T04:14:24.083 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T04:14:24.083 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-09-21T04:14:24.083 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T04:14:24.083 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T04:14:24.083 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-09-21T04:14:24.083 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-09-21T04:14:24.083 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-09-21T04:14:24.083 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-09-21T04:14:24.083 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T04:14:24.084 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T04:14:24.084 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T04:14:24.084 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T04:14:24.084 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T04:14:24.084 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-09-21T04:14:24.084 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-09-21T04:14:24.084 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-09-21T04:14:24.084 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T04:14:24.084 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T04:14:24.084 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-09-21T04:14:24.084 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-09-21T04:14:24.084 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-09-21T04:14:24.084 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-09-21T04:14:24.084 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T04:14:24.085 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-09-21T04:14:24.085 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T04:14:24.085 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-09-21T04:14:24.085 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 207084, 2024-09-21T04:14:24.085 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 81, 2024-09-21T04:14:24.085 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-09-21T04:14:24.085 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-09-21T04:14:24.085 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-09-21T04:14:24.085 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-09-21T04:14:24.085 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-09-21T04:14:24.085 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-09-21T04:14:24.085 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-09-21T04:14:24.085 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-09-21T04:14:24.085 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 8544, 2024-09-21T04:14:24.085 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 188, 2024-09-21T04:14:24.086 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T04:14:24.086 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-09-21T04:14:24.086 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-09-21T04:14:24.086 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-09-21T04:14:24.086 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 1190364, 2024-09-21T04:14:24.086 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 7096, 2024-09-21T04:14:24.086 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-09-21T04:14:24.086 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-09-21T04:14:24.086 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-09-21T04:14:24.086 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-09-21T04:14:24.086 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.086 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-09-21T04:14:24.086 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-09-21T04:14:24.086 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-09-21T04:14:24.087 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 2293, 2024-09-21T04:14:24.087 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 21, 2024-09-21T04:14:24.087 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 3, 2024-09-21T04:14:24.087 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 2293, 2024-09-21T04:14:24.087 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-09-21T04:14:24.087 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2293, 2024-09-21T04:14:24.087 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 3.127928686, 2024-09-21T04:14:24.087 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001364120 2024-09-21T04:14:24.087 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.087 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-09-21T04:14:24.087 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-09-21T04:14:24.087 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2293, 2024-09-21T04:14:24.087 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 2294 2024-09-21T04:14:24.087 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.088 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 2293, 2024-09-21T04:14:24.088 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 2293, 2024-09-21T04:14:24.088 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 0, 2024-09-21T04:14:24.088 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-09-21T04:14:24.088 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-09-21T04:14:24.088 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 1357, 2024-09-21T04:14:24.088 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 0, 2024-09-21T04:14:24.088 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 934, 2024-09-21T04:14:24.088 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 0, 2024-09-21T04:14:24.088 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 0, 2024-09-21T04:14:24.088 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-09-21T04:14:24.088 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-09-21T04:14:24.088 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 0, 2024-09-21T04:14:24.088 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 0, 2024-09-21T04:14:24.089 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 0, 2024-09-21T04:14:24.089 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-09-21T04:14:24.089 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-09-21T04:14:24.089 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-09-21T04:14:24.089 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 1, 2024-09-21T04:14:24.089 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 0, 2024-09-21T04:14:24.089 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-09-21T04:14:24.089 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-09-21T04:14:24.089 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-09-21T04:14:24.089 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-09-21T04:14:24.089 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-09-21T04:14:24.089 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-09-21T04:14:24.089 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T04:14:24.089 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-09-21T04:14:24.090 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-09-21T04:14:24.090 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 0, 2024-09-21T04:14:24.090 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-09-21T04:14:24.090 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-09-21T04:14:24.090 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-09-21T04:14:24.090 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-09-21T04:14:24.090 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-09-21T04:14:24.090 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-09-21T04:14:24.090 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-09-21T04:14:24.090 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-09-21T04:14:24.090 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-09-21T04:14:24.090 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-09-21T04:14:24.090 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-09-21T04:14:24.090 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-09-21T04:14:24.091 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-09-21T04:14:24.091 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-09-21T04:14:24.091 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-09-21T04:14:24.091 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-09-21T04:14:24.091 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 87, 2024-09-21T04:14:24.091 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-09-21T04:14:24.091 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 14, 2024-09-21T04:14:24.091 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 7, 2024-09-21T04:14:24.091 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 7, 2024-09-21T04:14:24.091 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 0, 2024-09-21T04:14:24.091 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-09-21T04:14:24.091 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 0, 2024-09-21T04:14:24.091 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 2, 2024-09-21T04:14:24.091 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 0 2024-09-21T04:14:24.092 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.092 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-09-21T04:14:24.092 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 0, 2024-09-21T04:14:24.092 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-09-21T04:14:24.092 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-09-21T04:14:24.092 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-09-21T04:14:24.092 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.092 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-09-21T04:14:24.092 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-09-21T04:14:24.092 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T04:14:24.092 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-09-21T04:14:24.092 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 0, 2024-09-21T04:14:24.092 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed_ops": 0, 2024-09-21T04:14:24.092 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 0, 2024-09-21T04:14:24.092 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-09-21T04:14:24.093 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.093 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T04:14:24.093 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-09-21T04:14:24.093 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-09-21T04:14:24.093 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-09-21T04:14:24.093 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 2548, 2024-09-21T04:14:24.093 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 31112362, 2024-09-21T04:14:24.093 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:24.093 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 2548, 2024-09-21T04:14:24.093 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-09-21T04:14:24.093 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-09-21T04:14:24.093 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 2548, 2024-09-21T04:14:24.093 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 31112362, 2024-09-21T04:14:24.093 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-09-21T04:14:24.093 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.094 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.094 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.094 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:24.094 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.094 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-09-21T04:14:24.094 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-09-21T04:14:24.094 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-09-21T04:14:24.094 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-09-21T04:14:24.094 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-09-21T04:14:24.094 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-09-21T04:14:24.094 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:24.094 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-09-21T04:14:24.094 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 2293, 2024-09-21T04:14:24.094 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 29452996, 2024-09-21T04:14:24.094 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 2293, 2024-09-21T04:14:24.095 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 29452996, 2024-09-21T04:14:24.095 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-09-21T04:14:24.095 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.095 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.095 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.095 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:24.095 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.095 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-09-21T04:14:24.095 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-09-21T04:14:24.095 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-09-21T04:14:24.095 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-09-21T04:14:24.095 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-09-21T04:14:24.095 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-09-21T04:14:24.095 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:24.096 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-09-21T04:14:24.096 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 2293, 2024-09-21T04:14:24.096 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 2293, 2024-09-21T04:14:24.096 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 2293, 2024-09-21T04:14:24.096 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 2293, 2024-09-21T04:14:24.096 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-09-21T04:14:24.096 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.096 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.096 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.096 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:24.096 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.096 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-09-21T04:14:24.096 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-09-21T04:14:24.096 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-09-21T04:14:24.097 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-09-21T04:14:24.097 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-09-21T04:14:24.097 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-09-21T04:14:24.097 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:24.097 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-09-21T04:14:24.097 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-09-21T04:14:24.097 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-09-21T04:14:24.097 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-09-21T04:14:24.097 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-09-21T04:14:24.097 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-09-21T04:14:24.097 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.097 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.097 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.097 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:24.098 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.098 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x5561371ed5f0": { 2024-09-21T04:14:24.098 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-09-21T04:14:24.098 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-09-21T04:14:24.098 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-09-21T04:14:24.098 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-09-21T04:14:24.098 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-09-21T04:14:24.098 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:24.098 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-09-21T04:14:24.098 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-09-21T04:14:24.098 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-09-21T04:14:24.098 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 0, 2024-09-21T04:14:24.098 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 0, 2024-09-21T04:14:24.098 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-09-21T04:14:24.098 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:24.099 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:24.099 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:24.099 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:24.099 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:24.099 INFO:teuthology.orchestra.run.smithi131.stdout: "trackedop": { 2024-09-21T04:14:24.099 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 0 2024-09-21T04:14:24.099 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:24.099 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-21T04:14:24.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:24 smithi119 ceph-mon[34811]: pgmap v642: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-21T04:14:24.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:24 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:14:24.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:24 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:14:24.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:24 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:14:24.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:24 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:14:24.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:24 smithi064 ceph-mon[31180]: pgmap v642: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-21T04:14:24.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:24 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:14:24.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:24 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:14:24.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:24 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:14:24.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:24 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:14:24.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:24 smithi131 ceph-mon[35193]: pgmap v642: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-21T04:14:24.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:24 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:14:24.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:24 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:14:24.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:24 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:14:24.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:24 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:14:24.738 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-21T04:14:24.738 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-21T04:14:24.738 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-09-21T04:14:24.738 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-21T04:14:24.738 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T04:14:24.738 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-09-21T04:14:24.738 DEBUG:tasks.check_counter:Getting stats from i 2024-09-21T04:14:24.739 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph daemon mds.i perf dump 2024-09-21T04:14:25.072 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T04:14:25.588 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-21T04:14:25.588 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T04:14:25.588 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 3887, 2024-09-21T04:14:25.588 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 3985, 2024-09-21T04:14:25.588 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 2368929, 2024-09-21T04:14:25.588 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 10845194, 2024-09-21T04:14:25.588 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 30, 2024-09-21T04:14:25.588 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 5, 2024-09-21T04:14:25.588 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.395484885, 2024-09-21T04:14:25.588 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.190580516, 2024-09-21T04:14:25.588 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.142197459, 2024-09-21T04:14:25.589 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.066971811, 2024-09-21T04:14:25.589 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:25.589 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 3984, 2024-09-21T04:14:25.589 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.151411975, 2024-09-21T04:14:25.589 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000038005 2024-09-21T04:14:25.589 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.589 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:25.589 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2310, 2024-09-21T04:14:25.589 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000454751, 2024-09-21T04:14:25.589 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000196 2024-09-21T04:14:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 1272320, 2024-09-21T04:14:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 163392 2024-09-21T04:14:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T04:14:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 17897, 2024-09-21T04:14:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 20368, 2024-09-21T04:14:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 5917116, 2024-09-21T04:14:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 23302718, 2024-09-21T04:14:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 9, 2024-09-21T04:14:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 6, 2024-09-21T04:14:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 1.382098657, 2024-09-21T04:14:25.590 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.685746602, 2024-09-21T04:14:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.737828397, 2024-09-21T04:14:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.100504357, 2024-09-21T04:14:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 20366, 2024-09-21T04:14:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.886393990, 2024-09-21T04:14:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000043523 2024-09-21T04:14:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 29787, 2024-09-21T04:14:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.005280657, 2024-09-21T04:14:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000177 2024-09-21T04:14:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.591 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-09-21T04:14:25.592 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 361792 2024-09-21T04:14:25.592 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.592 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T04:14:25.592 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 103379, 2024-09-21T04:14:25.592 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 69776, 2024-09-21T04:14:25.592 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 35708871, 2024-09-21T04:14:25.592 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 38017612, 2024-09-21T04:14:25.592 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 19, 2024-09-21T04:14:25.592 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 6, 2024-09-21T04:14:25.592 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 3.564767175, 2024-09-21T04:14:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 1.891810194, 2024-09-21T04:14:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 23.902617894, 2024-09-21T04:14:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.084484777, 2024-09-21T04:14:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 69775, 2024-09-21T04:14:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 3.883688460, 2024-09-21T04:14:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000055660 2024-09-21T04:14:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 147328, 2024-09-21T04:14:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.020464659, 2024-09-21T04:14:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000138 2024-09-21T04:14:25.593 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.594 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-21T04:14:25.594 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T04:14:25.594 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.594 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-09-21T04:14:25.594 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 1, 2024-09-21T04:14:25.594 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-09-21T04:14:25.594 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.594 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-MDSRank": { 2024-09-21T04:14:25.594 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-09-21T04:14:25.594 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-09-21T04:14:25.595 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 6414, 2024-09-21T04:14:25.595 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 12.248563372, 2024-09-21T04:14:25.595 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001909660 2024-09-21T04:14:25.595 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:25.595 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.595 INFO:teuthology.orchestra.run.smithi131.stdout: "finisher-PurgeQueue": { 2024-09-21T04:14:25.595 INFO:teuthology.orchestra.run.smithi131.stdout: "queue_len": 0, 2024-09-21T04:14:25.595 INFO:teuthology.orchestra.run.smithi131.stdout: "complete_latency": { 2024-09-21T04:14:25.595 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2414, 2024-09-21T04:14:25.595 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.260865084, 2024-09-21T04:14:25.595 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000108063 2024-09-21T04:14:25.596 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:25.596 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.596 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": { 2024-09-21T04:14:25.596 INFO:teuthology.orchestra.run.smithi131.stdout: "request": 19994, 2024-09-21T04:14:25.596 INFO:teuthology.orchestra.run.smithi131.stdout: "reply": 19112, 2024-09-21T04:14:25.596 INFO:teuthology.orchestra.run.smithi131.stdout: "reply_latency": { 2024-09-21T04:14:25.596 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 19112, 2024-09-21T04:14:25.596 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 304.373871437, 2024-09-21T04:14:25.596 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.015925799 2024-09-21T04:14:25.596 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.596 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_reply": 0, 2024-09-21T04:14:25.597 INFO:teuthology.orchestra.run.smithi131.stdout: "forward": 882, 2024-09-21T04:14:25.597 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_complete": 45, 2024-09-21T04:14:25.597 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_fetch_keys": 0, 2024-09-21T04:14:25.597 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_commit": 130, 2024-09-21T04:14:25.597 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_split": 14, 2024-09-21T04:14:25.597 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_merge": 0, 2024-09-21T04:14:25.597 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes": 298, 2024-09-21T04:14:25.597 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_top": 0, 2024-09-21T04:14:25.597 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_bottom": 1, 2024-09-21T04:14:25.597 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pin_tail": 297, 2024-09-21T04:14:25.597 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_pinned": 297, 2024-09-21T04:14:25.597 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_expired": 8825, 2024-09-21T04:14:25.598 INFO:teuthology.orchestra.run.smithi131.stdout: "inodes_with_caps": 1, 2024-09-21T04:14:25.598 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": 1, 2024-09-21T04:14:25.598 INFO:teuthology.orchestra.run.smithi131.stdout: "subtrees": 3, 2024-09-21T04:14:25.598 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse": 39364, 2024-09-21T04:14:25.598 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_hit": 35107, 2024-09-21T04:14:25.598 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_forward": 881, 2024-09-21T04:14:25.598 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_discover": 46, 2024-09-21T04:14:25.598 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_dir_fetch": 0, 2024-09-21T04:14:25.598 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_remote_ino": 0, 2024-09-21T04:14:25.599 INFO:teuthology.orchestra.run.smithi131.stdout: "traverse_lock": 220, 2024-09-21T04:14:25.599 INFO:teuthology.orchestra.run.smithi131.stdout: "load_cent": 0, 2024-09-21T04:14:25.599 INFO:teuthology.orchestra.run.smithi131.stdout: "q": 0, 2024-09-21T04:14:25.599 INFO:teuthology.orchestra.run.smithi131.stdout: "exported": 99, 2024-09-21T04:14:25.599 INFO:teuthology.orchestra.run.smithi131.stdout: "exported_inodes": 7266, 2024-09-21T04:14:25.599 INFO:teuthology.orchestra.run.smithi131.stdout: "imported": 20, 2024-09-21T04:14:25.599 INFO:teuthology.orchestra.run.smithi131.stdout: "imported_inodes": 1185, 2024-09-21T04:14:25.599 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_dir_fetch": 0, 2024-09-21T04:14:25.599 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_backtrace_fetch": 4107, 2024-09-21T04:14:25.599 INFO:teuthology.orchestra.run.smithi131.stdout: "openino_peer_discover": 0, 2024-09-21T04:14:25.599 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rfiles": 4, 2024-09-21T04:14:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rbytes": 234, 2024-09-21T04:14:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "root_rsnaps": 6, 2024-09-21T04:14:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_fetch": 4107, 2024-09-21T04:14:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_set_tag": 0, 2024-09-21T04:14:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T04:14:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_inotable_repaired": 0, 2024-09-21T04:14:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_inodes": 14, 2024-09-21T04:14:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T04:14:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_dirfrag_rstats": 14, 2024-09-21T04:14:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_file_inodes": 4093, 2024-09-21T04:14:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_inode_file_caps": 196, 2024-09-21T04:14:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_revoke": 12229, 2024-09-21T04:14:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_grant": 13893, 2024-09-21T04:14:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T04:14:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T04:14:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T04:14:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps": 70093, 2024-09-21T04:14:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_caps_dirty": 3072, 2024-09-21T04:14:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_cap_release": 172, 2024-09-21T04:14:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "process_request_cap_release": 6289 2024-09-21T04:14:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_cache": { 2024-09-21T04:14:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update": 0, 2024-09-21T04:14:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_update_receipt": 3, 2024-09-21T04:14:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_try_discover": 2, 2024-09-21T04:14:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_send_discover": 48, 2024-09-21T04:14:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "dir_handle_discover": 50, 2024-09-21T04:14:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays": 2, 2024-09-21T04:14:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_delayed": 0, 2024-09-21T04:14:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "num_strays_enqueuing": 0, 2024-09-21T04:14:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_created": 1299, 2024-09-21T04:14:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_enqueued": 1297, 2024-09-21T04:14:25.603 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_reintegrated": 0, 2024-09-21T04:14:25.603 INFO:teuthology.orchestra.run.smithi131.stdout: "strays_migrated": 0, 2024-09-21T04:14:25.603 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_processing": 0, 2024-09-21T04:14:25.603 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_enqueued": 0, 2024-09-21T04:14:25.603 INFO:teuthology.orchestra.run.smithi131.stdout: "num_recovering_prioritized": 0, 2024-09-21T04:14:25.603 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_started": 0, 2024-09-21T04:14:25.603 INFO:teuthology.orchestra.run.smithi131.stdout: "recovery_completed": 0, 2024-09-21T04:14:25.603 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_path": 0, 2024-09-21T04:14:25.603 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_quiesce_inode": 0, 2024-09-21T04:14:25.603 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T04:14:25.603 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_exportdir": 99, 2024-09-21T04:14:25.605 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_flush": 0, 2024-09-21T04:14:25.605 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragmentdir": 14, 2024-09-21T04:14:25.605 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_fragstats": 0, 2024-09-21T04:14:25.605 INFO:teuthology.orchestra.run.smithi131.stdout: "ireq_inodestats": 0, 2024-09-21T04:14:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-21T04:14:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_succeeded": 0, 2024-09-21T04:14:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_write_failed": 0 2024-09-21T04:14:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_log": { 2024-09-21T04:14:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: "evlrg": 0, 2024-09-21T04:14:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: "evadd": 15719, 2024-09-21T04:14:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: "evex": 0, 2024-09-21T04:14:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: "evtrm": 0, 2024-09-21T04:14:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: "ev": 15719, 2024-09-21T04:14:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: "evexg": 0, 2024-09-21T04:14:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: "evexd": 0, 2024-09-21T04:14:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: "segadd": 20, 2024-09-21T04:14:25.606 INFO:teuthology.orchestra.run.smithi131.stdout: "segex": 0, 2024-09-21T04:14:25.607 INFO:teuthology.orchestra.run.smithi131.stdout: "segtrm": 0, 2024-09-21T04:14:25.607 INFO:teuthology.orchestra.run.smithi131.stdout: "seg": 20, 2024-09-21T04:14:25.607 INFO:teuthology.orchestra.run.smithi131.stdout: "segmjr": 3, 2024-09-21T04:14:25.607 INFO:teuthology.orchestra.run.smithi131.stdout: "segexg": 0, 2024-09-21T04:14:25.607 INFO:teuthology.orchestra.run.smithi131.stdout: "segexd": 0, 2024-09-21T04:14:25.607 INFO:teuthology.orchestra.run.smithi131.stdout: "expos": 4194304, 2024-09-21T04:14:25.607 INFO:teuthology.orchestra.run.smithi131.stdout: "wrpos": 33646576, 2024-09-21T04:14:25.607 INFO:teuthology.orchestra.run.smithi131.stdout: "rdpos": 0, 2024-09-21T04:14:25.607 INFO:teuthology.orchestra.run.smithi131.stdout: "jlat": { 2024-09-21T04:14:25.607 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 3602, 2024-09-21T04:14:25.607 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 160.569163010, 2024-09-21T04:14:25.607 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.044577779 2024-09-21T04:14:25.607 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.608 INFO:teuthology.orchestra.run.smithi131.stdout: "replayed": 0 2024-09-21T04:14:25.608 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.608 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_mem": { 2024-09-21T04:14:25.608 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 32, 2024-09-21T04:14:25.608 INFO:teuthology.orchestra.run.smithi131.stdout: "ino+": 6348, 2024-09-21T04:14:25.608 INFO:teuthology.orchestra.run.smithi131.stdout: "ino-": 6316, 2024-09-21T04:14:25.608 INFO:teuthology.orchestra.run.smithi131.stdout: "dir": 27, 2024-09-21T04:14:25.608 INFO:teuthology.orchestra.run.smithi131.stdout: "dir+": 383, 2024-09-21T04:14:25.608 INFO:teuthology.orchestra.run.smithi131.stdout: "dir-": 356, 2024-09-21T04:14:25.608 INFO:teuthology.orchestra.run.smithi131.stdout: "dn": 298, 2024-09-21T04:14:25.608 INFO:teuthology.orchestra.run.smithi131.stdout: "dn+": 10530, 2024-09-21T04:14:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: "dn-": 10232, 2024-09-21T04:14:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: "cap": 1, 2024-09-21T04:14:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: "cap+": 3184, 2024-09-21T04:14:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: "cap-": 3183, 2024-09-21T04:14:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: "rss": 151616, 2024-09-21T04:14:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: "heap": 367865856 2024-09-21T04:14:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_server": { 2024-09-21T04:14:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_client_request": 21327, 2024-09-21T04:14:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: "dispatch_server_request": 1216, 2024-09-21T04:14:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_request": 19994, 2024-09-21T04:14:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_client_session": 30, 2024-09-21T04:14:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: "handle_peer_request": 2607, 2024-09-21T04:14:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: "req_create_latency": { 2024-09-21T04:14:25.609 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2527, 2024-09-21T04:14:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 13.553519290, 2024-09-21T04:14:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.005363482 2024-09-21T04:14:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getattr_latency": { 2024-09-21T04:14:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 457, 2024-09-21T04:14:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 115.706670118, 2024-09-21T04:14:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.253187461 2024-09-21T04:14:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getfilelock_latency": { 2024-09-21T04:14:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:25.610 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: "req_link_latency": { 2024-09-21T04:14:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookup_latency": { 2024-09-21T04:14:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 11506, 2024-09-21T04:14:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 148.837396440, 2024-09-21T04:14:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.012935633 2024-09-21T04:14:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookuphash_latency": { 2024-09-21T04:14:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:25.611 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupino_latency": { 2024-09-21T04:14:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupname_latency": { 2024-09-21T04:14:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupparent_latency": { 2024-09-21T04:14:25.612 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:25.613 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:25.613 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:25.613 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.613 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lookupsnap_latency": { 2024-09-21T04:14:25.613 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:25.613 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:25.613 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:25.614 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.614 INFO:teuthology.orchestra.run.smithi131.stdout: "req_lssnap_latency": { 2024-09-21T04:14:25.614 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:25.614 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:25.614 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:25.614 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.614 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mkdir_latency": { 2024-09-21T04:14:25.614 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 18, 2024-09-21T04:14:25.614 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.035206697, 2024-09-21T04:14:25.614 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001955927 2024-09-21T04:14:25.614 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.614 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mknod_latency": { 2024-09-21T04:14:25.614 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: "req_mksnap_latency": { 2024-09-21T04:14:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: "req_open_latency": { 2024-09-21T04:14:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 1072, 2024-09-21T04:14:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 4.480348088, 2024-09-21T04:14:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.004179429 2024-09-21T04:14:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.615 INFO:teuthology.orchestra.run.smithi131.stdout: "req_readdir_latency": { 2024-09-21T04:14:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 83, 2024-09-21T04:14:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.113134133, 2024-09-21T04:14:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001363061 2024-09-21T04:14:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rename_latency": { 2024-09-21T04:14:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2508, 2024-09-21T04:14:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 19.950378558, 2024-09-21T04:14:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.007954696 2024-09-21T04:14:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: "req_renamesnap_latency": { 2024-09-21T04:14:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:25.616 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: "req_snapdiff_latency": { 2024-09-21T04:14:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmdir_latency": { 2024-09-21T04:14:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 2, 2024-09-21T04:14:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.029589615, 2024-09-21T04:14:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.014794807 2024-09-21T04:14:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmsnap_latency": { 2024-09-21T04:14:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:25.617 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:25.618 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:25.618 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.618 INFO:teuthology.orchestra.run.smithi131.stdout: "req_rmxattr_latency": { 2024-09-21T04:14:25.618 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:25.618 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:25.618 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:25.618 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.618 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setattr_latency": { 2024-09-21T04:14:25.618 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:25.618 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:25.618 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:25.618 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.618 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setdirlayout_latency": { 2024-09-21T04:14:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setfilelock_latency": { 2024-09-21T04:14:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setlayout_latency": { 2024-09-21T04:14:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:25.619 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:25.620 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.620 INFO:teuthology.orchestra.run.smithi131.stdout: "req_setxattr_latency": { 2024-09-21T04:14:25.620 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:25.620 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:25.620 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:25.620 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.620 INFO:teuthology.orchestra.run.smithi131.stdout: "req_symlink_latency": { 2024-09-21T04:14:25.620 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:25.620 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:25.620 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:25.620 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.621 INFO:teuthology.orchestra.run.smithi131.stdout: "req_unlink_latency": { 2024-09-21T04:14:25.621 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 939, 2024-09-21T04:14:25.621 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 1.667628498, 2024-09-21T04:14:25.621 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001775962 2024-09-21T04:14:25.621 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.621 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_revoke_eviction": 0, 2024-09-21T04:14:25.621 INFO:teuthology.orchestra.run.smithi131.stdout: "cap_acquisition_throttle": 0, 2024-09-21T04:14:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: "req_getvxattr_latency": { 2024-09-21T04:14:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_sessions": { 2024-09-21T04:14:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: "session_count": 1, 2024-09-21T04:14:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: "session_add": 2, 2024-09-21T04:14:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: "session_remove": 1, 2024-09-21T04:14:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_open": 1, 2024-09-21T04:14:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: "sessions_stale": 0, 2024-09-21T04:14:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: "total_load": 1561, 2024-09-21T04:14:25.622 INFO:teuthology.orchestra.run.smithi131.stdout: "average_load": 1561, 2024-09-21T04:14:25.623 INFO:teuthology.orchestra.run.smithi131.stdout: "avg_session_uptime": 634, 2024-09-21T04:14:25.623 INFO:teuthology.orchestra.run.smithi131.stdout: "mdthresh_evicted": 0 2024-09-21T04:14:25.623 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.623 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-09-21T04:14:25.623 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-09-21T04:14:25.623 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-09-21T04:14:25.623 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T04:14:25.623 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-09-21T04:14:25.623 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T04:14:25.623 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-09-21T04:14:25.623 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T04:14:25.623 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T04:14:25.623 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T04:14:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T04:14:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T04:14:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-09-21T04:14:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T04:14:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T04:14:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-09-21T04:14:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-09-21T04:14:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-09-21T04:14:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-09-21T04:14:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T04:14:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T04:14:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T04:14:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T04:14:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T04:14:25.624 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-09-21T04:14:25.625 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-09-21T04:14:25.625 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-09-21T04:14:25.625 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T04:14:25.625 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T04:14:25.625 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-09-21T04:14:25.625 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-09-21T04:14:25.625 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-09-21T04:14:25.625 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-09-21T04:14:25.625 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T04:14:25.625 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-09-21T04:14:25.625 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T04:14:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-09-21T04:14:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 320102, 2024-09-21T04:14:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 121, 2024-09-21T04:14:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 88, 2024-09-21T04:14:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 1, 2024-09-21T04:14:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-09-21T04:14:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-09-21T04:14:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-09-21T04:14:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-09-21T04:14:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-09-21T04:14:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-09-21T04:14:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 8544, 2024-09-21T04:14:25.626 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 188, 2024-09-21T04:14:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T04:14:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-09-21T04:14:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-09-21T04:14:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-09-21T04:14:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 352482, 2024-09-21T04:14:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 2535, 2024-09-21T04:14:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-09-21T04:14:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-09-21T04:14:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-09-21T04:14:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-09-21T04:14:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: "objecter": { 2024-09-21T04:14:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: "op_active": 0, 2024-09-21T04:14:25.627 INFO:teuthology.orchestra.run.smithi131.stdout: "op_laggy": 0, 2024-09-21T04:14:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send": 9726, 2024-09-21T04:14:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "op_send_bytes": 31511230, 2024-09-21T04:14:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "op_resend": 33, 2024-09-21T04:14:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "op_reply": 9726, 2024-09-21T04:14:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "op_latency": { 2024-09-21T04:14:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 9726, 2024-09-21T04:14:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 152.988499405, 2024-09-21T04:14:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.015729847 2024-09-21T04:14:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "op_inflight": 0, 2024-09-21T04:14:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "oplen_avg": { 2024-09-21T04:14:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 9726, 2024-09-21T04:14:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 10153 2024-09-21T04:14:25.628 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: "op": 9726, 2024-09-21T04:14:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: "op_r": 4282, 2024-09-21T04:14:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: "op_w": 5444, 2024-09-21T04:14:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: "op_rmw": 0, 2024-09-21T04:14:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: "op_pg": 0, 2024-09-21T04:14:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_stat": 0, 2024-09-21T04:14:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_create": 10, 2024-09-21T04:14:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_read": 86, 2024-09-21T04:14:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_write": 3688, 2024-09-21T04:14:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writefull": 88, 2024-09-21T04:14:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_writesame": 0, 2024-09-21T04:14:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_append": 0, 2024-09-21T04:14:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_zero": 0, 2024-09-21T04:14:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_truncate": 7, 2024-09-21T04:14:25.629 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_delete": 1323, 2024-09-21T04:14:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_mapext": 0, 2024-09-21T04:14:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_sparse_read": 0, 2024-09-21T04:14:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_clonerange": 0, 2024-09-21T04:14:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_getxattr": 4107, 2024-09-21T04:14:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_setxattr": 20, 2024-09-21T04:14:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_cmpxattr": 0, 2024-09-21T04:14:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_rmxattr": 0, 2024-09-21T04:14:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_resetxattrs": 0, 2024-09-21T04:14:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_call": 0, 2024-09-21T04:14:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_watch": 0, 2024-09-21T04:14:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_notify": 0, 2024-09-21T04:14:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T04:14:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls": 0, 2024-09-21T04:14:25.630 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_pgls_filter": 0, 2024-09-21T04:14:25.631 INFO:teuthology.orchestra.run.smithi131.stdout: "osdop_other": 0, 2024-09-21T04:14:25.631 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_active": 0, 2024-09-21T04:14:25.631 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_send": 0, 2024-09-21T04:14:25.631 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_resend": 0, 2024-09-21T04:14:25.631 INFO:teuthology.orchestra.run.smithi131.stdout: "linger_ping": 0, 2024-09-21T04:14:25.631 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_active": 0, 2024-09-21T04:14:25.631 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_send": 0, 2024-09-21T04:14:25.631 INFO:teuthology.orchestra.run.smithi131.stdout: "poolop_resend": 0, 2024-09-21T04:14:25.631 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_active": 0, 2024-09-21T04:14:25.631 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_send": 0, 2024-09-21T04:14:25.631 INFO:teuthology.orchestra.run.smithi131.stdout: "poolstat_resend": 0, 2024-09-21T04:14:25.631 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_active": 0, 2024-09-21T04:14:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_send": 0, 2024-09-21T04:14:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: "statfs_resend": 0, 2024-09-21T04:14:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: "command_active": 0, 2024-09-21T04:14:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: "command_send": 0, 2024-09-21T04:14:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: "command_resend": 0, 2024-09-21T04:14:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: "map_epoch": 87, 2024-09-21T04:14:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: "map_full": 0, 2024-09-21T04:14:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: "map_inc": 14, 2024-09-21T04:14:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_sessions": 12, 2024-09-21T04:14:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_open": 12, 2024-09-21T04:14:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_session_close": 0, 2024-09-21T04:14:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_laggy": 0, 2024-09-21T04:14:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_wr": 589, 2024-09-21T04:14:25.632 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_rd": 134, 2024-09-21T04:14:25.633 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_del": 101 2024-09-21T04:14:25.633 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.633 INFO:teuthology.orchestra.run.smithi131.stdout: "oft": { 2024-09-21T04:14:25.633 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_objs": 1, 2024-09-21T04:14:25.633 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_kv_pairs": 0, 2024-09-21T04:14:25.633 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_updates": 0, 2024-09-21T04:14:25.633 INFO:teuthology.orchestra.run.smithi131.stdout: "omap_total_removes": 0 2024-09-21T04:14:25.633 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.633 INFO:teuthology.orchestra.run.smithi131.stdout: "purge_queue": { 2024-09-21T04:14:25.633 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops": 0, 2024-09-21T04:14:25.633 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_ops_high_water": 10, 2024-09-21T04:14:25.633 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing": 0, 2024-09-21T04:14:25.633 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executing_high_water": 10, 2024-09-21T04:14:25.634 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed_ops": 1297, 2024-09-21T04:14:25.634 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_executed": 1297, 2024-09-21T04:14:25.634 INFO:teuthology.orchestra.run.smithi131.stdout: "pq_item_in_journal": 0 2024-09-21T04:14:25.634 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.634 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T04:14:25.634 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-09-21T04:14:25.634 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-09-21T04:14:25.634 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-09-21T04:14:25.634 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 125159, 2024-09-21T04:14:25.634 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 32711960, 2024-09-21T04:14:25.634 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:25.634 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 125159, 2024-09-21T04:14:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-09-21T04:14:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-09-21T04:14:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 125159, 2024-09-21T04:14:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 32711960, 2024-09-21T04:14:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-09-21T04:14:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_bytes": { 2024-09-21T04:14:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-09-21T04:14:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-09-21T04:14:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-09-21T04:14:25.635 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-09-21T04:14:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-09-21T04:14:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-09-21T04:14:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 9726, 2024-09-21T04:14:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 31668930, 2024-09-21T04:14:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 9726, 2024-09-21T04:14:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 31668930, 2024-09-21T04:14:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-09-21T04:14:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-objecter_ops": { 2024-09-21T04:14:25.636 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-09-21T04:14:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 1024, 2024-09-21T04:14:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-09-21T04:14:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 0, 2024-09-21T04:14:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 0, 2024-09-21T04:14:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 0, 2024-09-21T04:14:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 9726, 2024-09-21T04:14:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 9726, 2024-09-21T04:14:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 9726, 2024-09-21T04:14:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 9726, 2024-09-21T04:14:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-09-21T04:14:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:25.637 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:25.638 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.638 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle": { 2024-09-21T04:14:25.638 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-09-21T04:14:25.638 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-09-21T04:14:25.638 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-09-21T04:14:25.638 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 1297, 2024-09-21T04:14:25.638 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 159869, 2024-09-21T04:14:25.638 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:25.638 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 1297, 2024-09-21T04:14:25.638 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-09-21T04:14:25.638 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-09-21T04:14:25.638 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 86, 2024-09-21T04:14:25.638 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 159869, 2024-09-21T04:14:25.638 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-09-21T04:14:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-write_buf_throttle-0x5624251f9380": { 2024-09-21T04:14:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-09-21T04:14:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 3758096384, 2024-09-21T04:14:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-09-21T04:14:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 15719, 2024-09-21T04:14:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 29452272, 2024-09-21T04:14:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 15719, 2024-09-21T04:14:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-09-21T04:14:25.639 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-09-21T04:14:25.640 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 3602, 2024-09-21T04:14:25.640 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 29452272, 2024-09-21T04:14:25.640 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-09-21T04:14:25.640 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:25.640 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:25.640 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:25.640 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:25.640 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:25.640 INFO:teuthology.orchestra.run.smithi131.stdout: "trackedop": { 2024-09-21T04:14:25.640 INFO:teuthology.orchestra.run.smithi131.stdout: "slow_ops_count": 0 2024-09-21T04:14:25.640 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:25.640 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-21T04:14:26.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:26 smithi131 ceph-mon[35193]: pgmap v643: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-21T04:14:26.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:26 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:14:26.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:26 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:14:26.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:26 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:14:26.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:26 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:14:26.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:26 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:14:26.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:26 smithi119 ceph-mon[34811]: pgmap v643: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-21T04:14:26.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:26 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:14:26.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:26 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:14:26.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:26 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:14:26.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:26 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:14:26.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:26 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:14:26.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:26 smithi064 ceph-mon[31180]: pgmap v643: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-21T04:14:26.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:26 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:14:26.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:26 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:14:26.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:26 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T04:14:26.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:26 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T04:14:26.512 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:26 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' 2024-09-21T04:14:27.195 INFO:tasks.check_counter:Daemon mds.i mds.exported=99 2024-09-21T04:14:27.195 INFO:tasks.check_counter:Daemon mds.i mds.imported=20 2024-09-21T04:14:27.195 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-21T04:14:27.195 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-21T04:14:27.195 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T04:14:27.195 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=14 2024-09-21T04:14:27.195 DEBUG:tasks.check_counter:Getting stats from l 2024-09-21T04:14:27.195 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph daemon mds.l perf dump 2024-09-21T04:14:27.445 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T04:14:28.090 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-21T04:14:28.091 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T04:14:28.091 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 4, 2024-09-21T04:14:28.091 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 4, 2024-09-21T04:14:28.091 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 2992, 2024-09-21T04:14:28.091 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 448, 2024-09-21T04:14:28.091 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 4, 2024-09-21T04:14:28.091 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 0, 2024-09-21T04:14:28.091 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.002883106, 2024-09-21T04:14:28.091 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.000038360, 2024-09-21T04:14:28.091 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.002154389, 2024-09-21T04:14:28.091 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T04:14:28.091 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:28.092 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 3, 2024-09-21T04:14:28.092 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.003827678, 2024-09-21T04:14:28.092 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.001275892 2024-09-21T04:14:28.092 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:28.092 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:28.092 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:28.092 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:28.092 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:28.092 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:28.092 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-21T04:14:28.092 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T04:14:28.092 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:28.092 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T04:14:28.093 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 219, 2024-09-21T04:14:28.093 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 212, 2024-09-21T04:14:28.093 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 1146368, 2024-09-21T04:14:28.093 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 129984, 2024-09-21T04:14:28.093 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 2, 2024-09-21T04:14:28.093 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 1, 2024-09-21T04:14:28.093 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.041944592, 2024-09-21T04:14:28.093 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.017631229, 2024-09-21T04:14:28.093 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.035774459, 2024-09-21T04:14:28.093 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T04:14:28.093 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:28.093 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 211, 2024-09-21T04:14:28.093 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.012846484, 2024-09-21T04:14:28.093 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000060883 2024-09-21T04:14:28.093 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:28.094 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:28.094 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:28.094 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:28.094 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:28.094 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:28.094 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 1146368, 2024-09-21T04:14:28.094 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 129984 2024-09-21T04:14:28.094 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:28.094 INFO:teuthology.orchestra.run.smithi131.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T04:14:28.094 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_messages": 1, 2024-09-21T04:14:28.094 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages": 161, 2024-09-21T04:14:28.094 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_bytes": 160, 2024-09-21T04:14:28.094 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_bytes": 108576, 2024-09-21T04:14:28.094 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_created_connections": 3, 2024-09-21T04:14:28.095 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_active_connections": 2, 2024-09-21T04:14:28.095 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_total_time": 0.012670270, 2024-09-21T04:14:28.095 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_send_time": 0.010179721, 2024-09-21T04:14:28.095 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_recv_time": 0.000795700, 2024-09-21T04:14:28.095 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T04:14:28.095 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T04:14:28.095 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 159, 2024-09-21T04:14:28.095 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.009544822, 2024-09-21T04:14:28.095 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000060030 2024-09-21T04:14:28.095 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:28.095 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_handle_ack_lat": { 2024-09-21T04:14:28.095 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:28.095 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:28.095 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:28.096 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:28.096 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T04:14:28.096 INFO:teuthology.orchestra.run.smithi131.stdout: "msgr_send_encrypted_bytes": 108576 2024-09-21T04:14:28.096 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:28.096 INFO:teuthology.orchestra.run.smithi131.stdout: "cct": { 2024-09-21T04:14:28.096 INFO:teuthology.orchestra.run.smithi131.stdout: "total_workers": 0, 2024-09-21T04:14:28.096 INFO:teuthology.orchestra.run.smithi131.stdout: "unhealthy_workers": 0 2024-09-21T04:14:28.096 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:28.096 INFO:teuthology.orchestra.run.smithi131.stdout: "mempool": { 2024-09-21T04:14:28.096 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_bytes": 0, 2024-09-21T04:14:28.096 INFO:teuthology.orchestra.run.smithi131.stdout: "bloom_filter_items": 0, 2024-09-21T04:14:28.096 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T04:14:28.097 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_alloc_items": 0, 2024-09-21T04:14:28.097 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T04:14:28.097 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_data_items": 0, 2024-09-21T04:14:28.097 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T04:14:28.097 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T04:14:28.097 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T04:14:28.097 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T04:14:28.097 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T04:14:28.097 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_other_items": 0, 2024-09-21T04:14:28.097 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T04:14:28.097 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T04:14:28.097 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_bytes": 0, 2024-09-21T04:14:28.097 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_extent_items": 0, 2024-09-21T04:14:28.097 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_bytes": 0, 2024-09-21T04:14:28.098 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_blob_items": 0, 2024-09-21T04:14:28.098 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T04:14:28.098 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T04:14:28.098 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T04:14:28.098 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T04:14:28.098 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T04:14:28.098 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_fsck_items": 0, 2024-09-21T04:14:28.098 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_bytes": 0, 2024-09-21T04:14:28.098 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_txc_items": 0, 2024-09-21T04:14:28.098 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T04:14:28.098 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T04:14:28.098 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_bytes": 0, 2024-09-21T04:14:28.098 INFO:teuthology.orchestra.run.smithi131.stdout: "bluestore_writing_items": 0, 2024-09-21T04:14:28.098 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_bytes": 0, 2024-09-21T04:14:28.099 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_items": 0, 2024-09-21T04:14:28.099 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T04:14:28.099 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_reader_items": 0, 2024-09-21T04:14:28.099 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T04:14:28.099 INFO:teuthology.orchestra.run.smithi131.stdout: "bluefs_file_writer_items": 0, 2024-09-21T04:14:28.099 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_bytes": 148984, 2024-09-21T04:14:28.099 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_anon_items": 30, 2024-09-21T04:14:28.099 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_bytes": 0, 2024-09-21T04:14:28.099 INFO:teuthology.orchestra.run.smithi131.stdout: "buffer_meta_items": 0, 2024-09-21T04:14:28.099 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_bytes": 0, 2024-09-21T04:14:28.099 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_items": 0, 2024-09-21T04:14:28.099 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_bytes": 0, 2024-09-21T04:14:28.099 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_mapbl_items": 0, 2024-09-21T04:14:28.099 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_bytes": 0, 2024-09-21T04:14:28.099 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_pglog_items": 0, 2024-09-21T04:14:28.100 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_bytes": 0, 2024-09-21T04:14:28.100 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_items": 0, 2024-09-21T04:14:28.100 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T04:14:28.100 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_mapping_items": 0, 2024-09-21T04:14:28.100 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_bytes": 0, 2024-09-21T04:14:28.100 INFO:teuthology.orchestra.run.smithi131.stdout: "pgmap_items": 0, 2024-09-21T04:14:28.100 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_bytes": 968, 2024-09-21T04:14:28.100 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_co_items": 2, 2024-09-21T04:14:28.100 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_bytes": 0, 2024-09-21T04:14:28.100 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_1_items": 0, 2024-09-21T04:14:28.100 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_bytes": 0, 2024-09-21T04:14:28.100 INFO:teuthology.orchestra.run.smithi131.stdout: "unittest_2_items": 0 2024-09-21T04:14:28.100 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-21T04:14:28.100 INFO:teuthology.orchestra.run.smithi131.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T04:14:28.101 INFO:teuthology.orchestra.run.smithi131.stdout: "val": 0, 2024-09-21T04:14:28.101 INFO:teuthology.orchestra.run.smithi131.stdout: "max": 104857600, 2024-09-21T04:14:28.101 INFO:teuthology.orchestra.run.smithi131.stdout: "get_started": 0, 2024-09-21T04:14:28.101 INFO:teuthology.orchestra.run.smithi131.stdout: "get": 220, 2024-09-21T04:14:28.101 INFO:teuthology.orchestra.run.smithi131.stdout: "get_sum": 1115418, 2024-09-21T04:14:28.101 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_fail": 0, 2024-09-21T04:14:28.101 INFO:teuthology.orchestra.run.smithi131.stdout: "get_or_fail_success": 220, 2024-09-21T04:14:28.101 INFO:teuthology.orchestra.run.smithi131.stdout: "take": 0, 2024-09-21T04:14:28.101 INFO:teuthology.orchestra.run.smithi131.stdout: "take_sum": 0, 2024-09-21T04:14:28.101 INFO:teuthology.orchestra.run.smithi131.stdout: "put": 220, 2024-09-21T04:14:28.101 INFO:teuthology.orchestra.run.smithi131.stdout: "put_sum": 1115418, 2024-09-21T04:14:28.101 INFO:teuthology.orchestra.run.smithi131.stdout: "wait": { 2024-09-21T04:14:28.101 INFO:teuthology.orchestra.run.smithi131.stdout: "avgcount": 0, 2024-09-21T04:14:28.101 INFO:teuthology.orchestra.run.smithi131.stdout: "sum": 0.000000000, 2024-09-21T04:14:28.102 INFO:teuthology.orchestra.run.smithi131.stdout: "avgtime": 0.000000000 2024-09-21T04:14:28.102 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:28.102 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-21T04:14:28.102 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-21T04:14:28.102 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:28 smithi131 ceph-mon[35193]: pgmap v644: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-21T04:14:28.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:28 smithi119 ceph-mon[34811]: pgmap v644: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-21T04:14:28.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:28 smithi064 ceph-mon[31180]: pgmap v644: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-21T04:14:28.702 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-21T04:14:28.702 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-21T04:14:28.702 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-09-21T04:14:28.702 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-09-21T04:14:28.702 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-09-21T04:14:28.702 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-09-21T04:14:28.702 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-21T04:14:28.713 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-21T04:14:28.713 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-21T04:14:28.713 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T04:14:28.734 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-09-21T04:14:28.735 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T04:14:28.795 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T04:14:29.109 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:14:29.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.601+0000 7fb5250c5640 1 -- 172.21.15.64:0/1303531762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb52010de60 msgr2=0x7fb520110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:29.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.601+0000 7fb5250c5640 1 --2- 172.21.15.64:0/1303531762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb52010de60 0x7fb520110330 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fb518009ca0 tx=0x7fb51802a8c0 comp rx=0 tx=0).stop 2024-09-21T04:14:29.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.601+0000 7fb5250c5640 1 -- 172.21.15.64:0/1303531762 shutdown_connections 2024-09-21T04:14:29.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.601+0000 7fb5250c5640 1 --2- 172.21.15.64:0/1303531762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb52010de60 0x7fb520110330 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:29.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.601+0000 7fb5250c5640 1 --2- 172.21.15.64:0/1303531762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb520105380 0x7fb52010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:29.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.601+0000 7fb5250c5640 1 --2- 172.21.15.64:0/1303531762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb520104a30 0x7fb520104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:29.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.601+0000 7fb5250c5640 1 -- 172.21.15.64:0/1303531762 >> 172.21.15.64:0/1303531762 conn(0x7fb520079760 msgr2=0x7fb520079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:29.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.601+0000 7fb5250c5640 1 -- 172.21.15.64:0/1303531762 shutdown_connections 2024-09-21T04:14:29.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.601+0000 7fb5250c5640 1 -- 172.21.15.64:0/1303531762 wait complete. 2024-09-21T04:14:29.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.602+0000 7fb5250c5640 1 Processor -- start 2024-09-21T04:14:29.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.602+0000 7fb5250c5640 1 -- start start 2024-09-21T04:14:29.618 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.602+0000 7fb5250c5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb520104a30 0x7fb52019c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:29.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.602+0000 7fb5250c5640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb520105380 0x7fb52019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:29.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.602+0000 7fb5250c5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb52010de60 0x7fb5201a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:29.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.602+0000 7fb5250c5640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb520113b40 con 0x7fb520105380 2024-09-21T04:14:29.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.602+0000 7fb5250c5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb5201139c0 con 0x7fb520104a30 2024-09-21T04:14:29.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.602+0000 7fb5250c5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb520113cc0 con 0x7fb52010de60 2024-09-21T04:14:29.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.603+0000 7fb51e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb520105380 0x7fb52019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:29.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.603+0000 7fb51e59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb520105380 0x7fb52019cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:48160/0 (socket says 172.21.15.64:48160) 2024-09-21T04:14:29.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.603+0000 7fb51e59c640 1 -- 172.21.15.64:0/3234665996 learned_addr learned my addr 172.21.15.64:0/3234665996 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:14:29.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.603+0000 7fb51ed9d640 1 --2- 172.21.15.64:0/3234665996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb520104a30 0x7fb52019c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:29.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.603+0000 7fb51f59e640 1 --2- 172.21.15.64:0/3234665996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb52010de60 0x7fb5201a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:29.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.603+0000 7fb51e59c640 1 -- 172.21.15.64:0/3234665996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb52010de60 msgr2=0x7fb5201a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:29.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.603+0000 7fb51e59c640 1 --2- 172.21.15.64:0/3234665996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb52010de60 0x7fb5201a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:29.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.603+0000 7fb51e59c640 1 -- 172.21.15.64:0/3234665996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb520104a30 msgr2=0x7fb52019c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:29.619 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.603+0000 7fb51e59c640 1 --2- 172.21.15.64:0/3234665996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb520104a30 0x7fb52019c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:29.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.603+0000 7fb51e59c640 1 -- 172.21.15.64:0/3234665996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5201a2560 con 0x7fb520105380 2024-09-21T04:14:29.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.603+0000 7fb51ed9d640 1 --2- 172.21.15.64:0/3234665996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb520104a30 0x7fb52019c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:14:29.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.603+0000 7fb51e59c640 1 --2- 172.21.15.64:0/3234665996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb520105380 0x7fb52019cc50 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fb51000c8c0 tx=0x7fb51000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:29.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.603+0000 7fb517fff640 1 -- 172.21.15.64:0/3234665996 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510007bb0 con 0x7fb520105380 2024-09-21T04:14:29.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.603+0000 7fb5250c5640 1 -- 172.21.15.64:0/3234665996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5201a2850 con 0x7fb520105380 2024-09-21T04:14:29.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.603+0000 7fb517fff640 1 -- 172.21.15.64:0/3234665996 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb510007d50 con 0x7fb520105380 2024-09-21T04:14:29.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.603+0000 7fb517fff640 1 -- 172.21.15.64:0/3234665996 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5100187c0 con 0x7fb520105380 2024-09-21T04:14:29.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.603+0000 7fb5250c5640 1 -- 172.21.15.64:0/3234665996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5201a9070 con 0x7fb520105380 2024-09-21T04:14:29.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.604+0000 7fb5250c5640 1 -- 172.21.15.64:0/3234665996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb52006c1f0 con 0x7fb520105380 2024-09-21T04:14:29.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.607+0000 7fb517fff640 1 -- 172.21.15.64:0/3234665996 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fb51000fa20 con 0x7fb520105380 2024-09-21T04:14:29.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.608+0000 7fb517fff640 1 --2- 172.21.15.64:0/3234665996 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb50c0b66b0 0x7fb50c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:29.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.608+0000 7fb51ed9d640 1 --2- 172.21.15.64:0/3234665996 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb50c0b66b0 0x7fb50c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:29.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.608+0000 7fb517fff640 1 -- 172.21.15.64:0/3234665996 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8615+0+0 (secure 0 0 0) 0x7fb5100e7b50 con 0x7fb520105380 2024-09-21T04:14:29.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.608+0000 7fb51ed9d640 1 --2- 172.21.15.64:0/3234665996 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb50c0b66b0 0x7fb50c0b8bc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb52006c8f0 tx=0x7fb508008040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:29.620 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.611+0000 7fb517fff640 1 -- 172.21.15.64:0/3234665996 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb510014030 con 0x7fb520105380 2024-09-21T04:14:29.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.815+0000 7fb5250c5640 1 -- 172.21.15.64:0/3234665996 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb520104e40 con 0x7fb520105380 2024-09-21T04:14:29.818 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.815+0000 7fb517fff640 1 -- 172.21.15.64:0/3234665996 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v87) ==== 81+0+887 (secure 0 0 0) 0x7fb5100ae0d0 con 0x7fb520105380 2024-09-21T04:14:29.818 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3554471670 2024-09-22T03:55:28.387426+0000 2024-09-21T04:14:29.818 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/3786213009 2024-09-22T03:55:28.387426+0000 2024-09-21T04:14:29.818 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/2562856406 2024-09-22T03:54:27.180100+0000 2024-09-21T04:14:29.819 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/2603015699 2024-09-22T03:55:28.387426+0000 2024-09-21T04:14:29.819 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/1209932961 2024-09-22T03:54:02.562380+0000 2024-09-21T04:14:29.819 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/3786213009 2024-09-22T03:55:28.387426+0000 2024-09-21T04:14:29.819 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/1209932961 2024-09-22T03:54:02.562380+0000 2024-09-21T04:14:29.819 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3407344596 2024-09-22T03:54:02.562380+0000 2024-09-21T04:14:29.819 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1233731831 2024-09-22T03:54:02.562380+0000 2024-09-21T04:14:29.819 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1686887701 2024-09-22T03:54:02.562380+0000 2024-09-21T04:14:29.819 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/1598568021 2024-09-22T03:54:27.180100+0000 2024-09-21T04:14:29.819 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6800/2013929312 2024-09-22T03:54:27.180100+0000 2024-09-21T04:14:29.819 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:6801/2013929312 2024-09-22T03:54:27.180100+0000 2024-09-21T04:14:29.819 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/3652036144 2024-09-22T03:55:28.387426+0000 2024-09-21T04:14:29.819 INFO:teuthology.orchestra.run.smithi064.stdout:172.21.15.64:0/105454703 2024-09-22T03:54:27.180100+0000 2024-09-21T04:14:29.819 INFO:teuthology.orchestra.run.smithi064.stderr:listed 15 entries 2024-09-21T04:14:29.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.818+0000 7fb5250c5640 1 -- 172.21.15.64:0/3234665996 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb50c0b66b0 msgr2=0x7fb50c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:29.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.818+0000 7fb5250c5640 1 --2- 172.21.15.64:0/3234665996 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb50c0b66b0 0x7fb50c0b8bc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb52006c8f0 tx=0x7fb508008040 comp rx=0 tx=0).stop 2024-09-21T04:14:29.820 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.818+0000 7fb5250c5640 1 -- 172.21.15.64:0/3234665996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb520105380 msgr2=0x7fb52019cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:29.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.818+0000 7fb5250c5640 1 --2- 172.21.15.64:0/3234665996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb520105380 0x7fb52019cc50 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fb51000c8c0 tx=0x7fb51000cd80 comp rx=0 tx=0).stop 2024-09-21T04:14:29.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.818+0000 7fb5250c5640 1 -- 172.21.15.64:0/3234665996 shutdown_connections 2024-09-21T04:14:29.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.818+0000 7fb5250c5640 1 --2- 172.21.15.64:0/3234665996 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fb50c0b66b0 0x7fb50c0b8bc0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:29.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.818+0000 7fb5250c5640 1 --2- 172.21.15.64:0/3234665996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb52010de60 0x7fb5201a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:29.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.818+0000 7fb5250c5640 1 --2- 172.21.15.64:0/3234665996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb520105380 0x7fb52019cc50 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:29.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.818+0000 7fb5250c5640 1 --2- 172.21.15.64:0/3234665996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb520104a30 0x7fb52019c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:29.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.818+0000 7fb5250c5640 1 -- 172.21.15.64:0/3234665996 >> 172.21.15.64:0/3234665996 conn(0x7fb520079760 msgr2=0x7fb5201036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:29.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.818+0000 7fb5250c5640 1 -- 172.21.15.64:0/3234665996 shutdown_connections 2024-09-21T04:14:29.821 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:29.818+0000 7fb5250c5640 1 -- 172.21.15.64:0/3234665996 wait complete. 2024-09-21T04:14:30.112 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:30 smithi064 ceph-mon[31180]: pgmap v645: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T04:14:30.112 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:30 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/3234665996' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T04:14:30.306 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:30 smithi131 ceph-mon[35193]: pgmap v645: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T04:14:30.306 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:30 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/3234665996' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T04:14:30.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:30 smithi119 ceph-mon[34811]: pgmap v645: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T04:14:30.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:30 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/3234665996' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T04:14:30.596 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T04:14:30.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:14:30.596 DEBUG:teuthology.orchestra.run.smithi064:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T04:14:30.659 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi064.front.sepia.ceph.com 2024-09-21T04:14:30.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:14:30.659 DEBUG:teuthology.orchestra.run.smithi064:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T04:14:30.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:14:30.702 DEBUG:teuthology.orchestra.run.smithi064:> ip netns list 2024-09-21T04:14:30.793 INFO:teuthology.orchestra.run.smithi064.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T04:14:30.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:14:30.794 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T04:14:30.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T04:14:30.858 DEBUG:teuthology.orchestra.run.smithi064:> sudo ip link delete ceph-brx 2024-09-21T04:14:30.951 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-21T04:14:30.967 INFO:tasks.cephadm:Teardown begin 2024-09-21T04:14:30.967 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T04:14:30.968 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:14:31.305 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:14:31.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.818+0000 7fd4da77c640 1 -- 172.21.15.64:0/2948169411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4d4108cf0 msgr2=0x7fd4d410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:31.828 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.818+0000 7fd4da77c640 1 --2- 172.21.15.64:0/2948169411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4d4108cf0 0x7fd4d410f520 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009ca0 tx=0x7fd4cc02a8c0 comp rx=0 tx=0).stop 2024-09-21T04:14:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.818+0000 7fd4da77c640 1 -- 172.21.15.64:0/2948169411 shutdown_connections 2024-09-21T04:14:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.818+0000 7fd4da77c640 1 --2- 172.21.15.64:0/2948169411 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4d4108cf0 0x7fd4d410f520 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.818+0000 7fd4da77c640 1 --2- 172.21.15.64:0/2948169411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d41083a0 0x7fd4d41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.818+0000 7fd4da77c640 1 --2- 172.21.15.64:0/2948169411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4d4102a60 0x7fd4d4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.818+0000 7fd4da77c640 1 -- 172.21.15.64:0/2948169411 >> 172.21.15.64:0/2948169411 conn(0x7fd4d4079730 msgr2=0x7fd4d4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.818+0000 7fd4da77c640 1 -- 172.21.15.64:0/2948169411 shutdown_connections 2024-09-21T04:14:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.818+0000 7fd4da77c640 1 -- 172.21.15.64:0/2948169411 wait complete. 2024-09-21T04:14:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.819+0000 7fd4da77c640 1 Processor -- start 2024-09-21T04:14:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.819+0000 7fd4da77c640 1 -- start start 2024-09-21T04:14:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.820+0000 7fd4da77c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d4102a60 0x7fd4d40ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.820+0000 7fd4da77c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4d41083a0 0x7fd4d40ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.820+0000 7fd4da77c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4d4108cf0 0x7fd4d41000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.820+0000 7fd4da77c640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d4112960 con 0x7fd4d4108cf0 2024-09-21T04:14:31.829 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.820+0000 7fd4da77c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d41127e0 con 0x7fd4d4102a60 2024-09-21T04:14:31.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.820+0000 7fd4da77c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d4112ae0 con 0x7fd4d41083a0 2024-09-21T04:14:31.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.820+0000 7fd4d8d19640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4d4108cf0 0x7fd4d41000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:31.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.820+0000 7fd4d8d19640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4d4108cf0 0x7fd4d41000f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43896/0 (socket says 172.21.15.64:43896) 2024-09-21T04:14:31.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.820+0000 7fd4d8d19640 1 -- 172.21.15.64:0/2899353123 learned_addr learned my addr 172.21.15.64:0/2899353123 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:14:31.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.820+0000 7fd4d3fff640 1 --2- 172.21.15.64:0/2899353123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d4102a60 0x7fd4d40ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:31.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.820+0000 7fd4d37fe640 1 --2- 172.21.15.64:0/2899353123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4d41083a0 0x7fd4d40ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:31.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.820+0000 7fd4d8d19640 1 -- 172.21.15.64:0/2899353123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4d41083a0 msgr2=0x7fd4d40ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:31.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.820+0000 7fd4d8d19640 1 --2- 172.21.15.64:0/2899353123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4d41083a0 0x7fd4d40ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:31.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.820+0000 7fd4d8d19640 1 -- 172.21.15.64:0/2899353123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d4102a60 msgr2=0x7fd4d40ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:31.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.820+0000 7fd4d8d19640 1 --2- 172.21.15.64:0/2899353123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d4102a60 0x7fd4d40ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:31.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.820+0000 7fd4d8d19640 1 -- 172.21.15.64:0/2899353123 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d41a8720 con 0x7fd4d4108cf0 2024-09-21T04:14:31.830 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.820+0000 7fd4d8d19640 1 --2- 172.21.15.64:0/2899353123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4d4108cf0 0x7fd4d41000f0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc005cc0 tx=0x7fd4cc02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:31.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.821+0000 7fd4d17fa640 1 -- 172.21.15.64:0/2899353123 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc003bf0 con 0x7fd4d4108cf0 2024-09-21T04:14:31.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.821+0000 7fd4d17fa640 1 -- 172.21.15.64:0/2899353123 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4cc003d90 con 0x7fd4d4108cf0 2024-09-21T04:14:31.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.821+0000 7fd4da77c640 1 -- 172.21.15.64:0/2899353123 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d41a89b0 con 0x7fd4d4108cf0 2024-09-21T04:14:31.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.821+0000 7fd4d17fa640 1 -- 172.21.15.64:0/2899353123 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc02fbc0 con 0x7fd4d4108cf0 2024-09-21T04:14:31.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.821+0000 7fd4da77c640 1 -- 172.21.15.64:0/2899353123 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d41a8ef0 con 0x7fd4d4108cf0 2024-09-21T04:14:31.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.822+0000 7fd4da77c640 1 -- 172.21.15.64:0/2899353123 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d4102e70 con 0x7fd4d4108cf0 2024-09-21T04:14:31.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.824+0000 7fd4d17fa640 1 -- 172.21.15.64:0/2899353123 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fd4cc02fd60 con 0x7fd4d4108cf0 2024-09-21T04:14:31.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.825+0000 7fd4d17fa640 1 --2- 172.21.15.64:0/2899353123 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd4bc0b6680 0x7fd4bc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:31.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.825+0000 7fd4d3fff640 1 --2- 172.21.15.64:0/2899353123 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd4bc0b6680 0x7fd4bc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:31.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.825+0000 7fd4d17fa640 1 -- 172.21.15.64:0/2899353123 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8615+0+0 (secure 0 0 0) 0x7fd4cc108d60 con 0x7fd4d4108cf0 2024-09-21T04:14:31.831 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.826+0000 7fd4d3fff640 1 --2- 172.21.15.64:0/2899353123 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd4bc0b6680 0x7fd4bc0b8b90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd4d41010c0 tx=0x7fd4c400b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:31.832 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:31.828+0000 7fd4d17fa640 1 -- 172.21.15.64:0/2899353123 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd4cc0cf250 con 0x7fd4d4108cf0 2024-09-21T04:14:32.057 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:32.054+0000 7fd4da77c640 1 -- 172.21.15.64:0/2899353123 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4d4100970 con 0x7fd4d4108cf0 2024-09-21T04:14:32.058 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:32.055+0000 7fd4d17fa640 1 -- 172.21.15.64:0/2899353123 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11912 (secure 0 0 0) 0x7fd4d4100970 con 0x7fd4d4108cf0 2024-09-21T04:14:32.058 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:14:32.058 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":44,"btime":"2024-09-21T04:14:19:404607+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14805,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/97995119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":97995119},{"type":"v1","addr":"172.21.15.64:6839","nonce":97995119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/390891068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":390891068},{"type":"v1","addr":"172.21.15.64:6841","nonce":390891068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:02:03.354744+0000","modified":"2024-09-21T04:14:19.192010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24710,"mds_1":24680,"mds_2":24661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24634":{"gid":24634,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6839/782190424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782190424},{"type":"v1","addr":"172.21.15.131:6839","nonce":782190424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24680":{"gid":24680,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24710":{"gid":24710,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6839/1319793636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1319793636},{"type":"v1","addr":"172.21.15.119:6839","nonce":1319793636}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24710,"qdb_cluster":[24661,24680,24710]},"id":1}]} 2024-09-21T04:14:32.059 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 44 2024-09-21T04:14:32.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:32.057+0000 7fd4da77c640 1 -- 172.21.15.64:0/2899353123 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd4bc0b6680 msgr2=0x7fd4bc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:32.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:32.057+0000 7fd4da77c640 1 --2- 172.21.15.64:0/2899353123 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd4bc0b6680 0x7fd4bc0b8b90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd4d41010c0 tx=0x7fd4c400b390 comp rx=0 tx=0).stop 2024-09-21T04:14:32.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:32.057+0000 7fd4da77c640 1 -- 172.21.15.64:0/2899353123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4d4108cf0 msgr2=0x7fd4d41000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:32.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:32.057+0000 7fd4da77c640 1 --2- 172.21.15.64:0/2899353123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4d4108cf0 0x7fd4d41000f0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc005cc0 tx=0x7fd4cc02f380 comp rx=0 tx=0).stop 2024-09-21T04:14:32.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:32.057+0000 7fd4da77c640 1 -- 172.21.15.64:0/2899353123 shutdown_connections 2024-09-21T04:14:32.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:32.057+0000 7fd4da77c640 1 --2- 172.21.15.64:0/2899353123 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fd4bc0b6680 0x7fd4bc0b8b90 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:32.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:32.057+0000 7fd4da77c640 1 --2- 172.21.15.64:0/2899353123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4d4108cf0 0x7fd4d41000f0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:32.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:32.057+0000 7fd4da77c640 1 --2- 172.21.15.64:0/2899353123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4d41083a0 0x7fd4d40ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:32.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:32.057+0000 7fd4da77c640 1 --2- 172.21.15.64:0/2899353123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d4102a60 0x7fd4d40ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:32.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:32.057+0000 7fd4da77c640 1 -- 172.21.15.64:0/2899353123 >> 172.21.15.64:0/2899353123 conn(0x7fd4d4079730 msgr2=0x7fd4d406b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:32.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:32.057+0000 7fd4da77c640 1 -- 172.21.15.64:0/2899353123 shutdown_connections 2024-09-21T04:14:32.060 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:32.057+0000 7fd4da77c640 1 -- 172.21.15.64:0/2899353123 wait complete. 2024-09-21T04:14:32.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:32 smithi131 ceph-mon[35193]: pgmap v646: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T04:14:32.382 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:32 smithi064 ceph-mon[31180]: pgmap v646: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T04:14:32.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:32 smithi119 ceph-mon[34811]: pgmap v646: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-21T04:14:32.827 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T04:14:33.091 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:14:33.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:33 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/2899353123' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:14:33.418 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:33 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/2899353123' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:14:33.506 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:33 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/2899353123' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:14:33.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.662+0000 7f3529bff640 1 -- 172.21.15.64:0/462447160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3524108cf0 msgr2=0x7f352410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:33.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.662+0000 7f3529bff640 1 --2- 172.21.15.64:0/462447160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3524108cf0 0x7f352410f520 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f351c009ca0 tx=0x7f351c02a8c0 comp rx=0 tx=0).stop 2024-09-21T04:14:33.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.662+0000 7f3529bff640 1 -- 172.21.15.64:0/462447160 shutdown_connections 2024-09-21T04:14:33.665 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.662+0000 7f3529bff640 1 --2- 172.21.15.64:0/462447160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3524108cf0 0x7f352410f520 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:33.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.662+0000 7f3529bff640 1 --2- 172.21.15.64:0/462447160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35241083a0 0x7f35241087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:33.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.662+0000 7f3529bff640 1 --2- 172.21.15.64:0/462447160 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3524102a60 0x7f3524102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:33.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.662+0000 7f3529bff640 1 -- 172.21.15.64:0/462447160 >> 172.21.15.64:0/462447160 conn(0x7f3524079730 msgr2=0x7f3524079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:33.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.662+0000 7f3529bff640 1 -- 172.21.15.64:0/462447160 shutdown_connections 2024-09-21T04:14:33.666 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.662+0000 7f3529bff640 1 -- 172.21.15.64:0/462447160 wait complete. 2024-09-21T04:14:33.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.663+0000 7f3529bff640 1 Processor -- start 2024-09-21T04:14:33.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.663+0000 7f3529bff640 1 -- start start 2024-09-21T04:14:33.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.664+0000 7f3529bff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3524102a60 0x7f35240ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:33.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.664+0000 7f3529bff640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35241083a0 0x7f35240ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:33.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.664+0000 7f3529bff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3524108cf0 0x7f35241000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:33.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.664+0000 7f3529bff640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3524112960 con 0x7f35241083a0 2024-09-21T04:14:33.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.664+0000 7f3529bff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f35241127e0 con 0x7f3524108cf0 2024-09-21T04:14:33.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.664+0000 7f3529bff640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3524112ae0 con 0x7f3524102a60 2024-09-21T04:14:33.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.664+0000 7f3522ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35241083a0 0x7f35240ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:33.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.664+0000 7f3522ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35241083a0 0x7f35240ffbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43912/0 (socket says 172.21.15.64:43912) 2024-09-21T04:14:33.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.664+0000 7f3522ffd640 1 -- 172.21.15.64:0/469154781 learned_addr learned my addr 172.21.15.64:0/469154781 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:14:33.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.664+0000 7f3523fff640 1 --2- 172.21.15.64:0/469154781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3524108cf0 0x7f35241000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:33.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.664+0000 7f35237fe640 1 --2- 172.21.15.64:0/469154781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3524102a60 0x7f35240ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:33.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.664+0000 7f3522ffd640 1 -- 172.21.15.64:0/469154781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3524102a60 msgr2=0x7f35240ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:33.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.664+0000 7f3522ffd640 1 --2- 172.21.15.64:0/469154781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3524102a60 0x7f35240ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:33.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.664+0000 7f3522ffd640 1 -- 172.21.15.64:0/469154781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3524108cf0 msgr2=0x7f35241000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:33.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.664+0000 7f3522ffd640 1 --2- 172.21.15.64:0/469154781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3524108cf0 0x7f35241000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:33.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.664+0000 7f3522ffd640 1 -- 172.21.15.64:0/469154781 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35241a8780 con 0x7f35241083a0 2024-09-21T04:14:33.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.664+0000 7f3522ffd640 1 --2- 172.21.15.64:0/469154781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35241083a0 0x7f35240ffbb0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f351800d770 tx=0x7f351800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:33.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.664+0000 7f3520ff9640 1 -- 172.21.15.64:0/469154781 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3518017070 con 0x7f35241083a0 2024-09-21T04:14:33.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.665+0000 7f3520ff9640 1 -- 172.21.15.64:0/469154781 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3518009c30 con 0x7f35241083a0 2024-09-21T04:14:33.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.665+0000 7f3529bff640 1 -- 172.21.15.64:0/469154781 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35241a8a70 con 0x7f35241083a0 2024-09-21T04:14:33.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.665+0000 7f3520ff9640 1 -- 172.21.15.64:0/469154781 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351800f620 con 0x7f35241083a0 2024-09-21T04:14:33.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.665+0000 7f3529bff640 1 -- 172.21.15.64:0/469154781 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35241a8fb0 con 0x7f35241083a0 2024-09-21T04:14:33.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.666+0000 7f3529bff640 1 -- 172.21.15.64:0/469154781 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3524102e70 con 0x7f35241083a0 2024-09-21T04:14:33.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.668+0000 7f3520ff9640 1 -- 172.21.15.64:0/469154781 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f351801a070 con 0x7f35241083a0 2024-09-21T04:14:33.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.669+0000 7f3520ff9640 1 --2- 172.21.15.64:0/469154781 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f350c0b6680 0x7f350c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:33.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.669+0000 7f35237fe640 1 --2- 172.21.15.64:0/469154781 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f350c0b6680 0x7f350c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:33.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.669+0000 7f3520ff9640 1 -- 172.21.15.64:0/469154781 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8615+0+0 (secure 0 0 0) 0x7f35180e7220 con 0x7f35241083a0 2024-09-21T04:14:33.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.670+0000 7f35237fe640 1 --2- 172.21.15.64:0/469154781 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f350c0b6680 0x7f350c0b8b90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3514005fd0 tx=0x7f3514005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:33.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.672+0000 7f3520ff9640 1 -- 172.21.15.64:0/469154781 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3518009de0 con 0x7f35241083a0 2024-09-21T04:14:33.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.899+0000 7f3529bff640 1 -- 172.21.15.64:0/469154781 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3524100970 con 0x7f35241083a0 2024-09-21T04:14:33.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.899+0000 7f3520ff9640 1 -- 172.21.15.64:0/469154781 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11912 (secure 0 0 0) 0x7f35180ad710 con 0x7f35241083a0 2024-09-21T04:14:33.902 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:14:33.902 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":44,"btime":"2024-09-21T04:14:19:404607+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14748,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/2075966415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":2075966415},{"type":"v1","addr":"172.21.15.64:6835","nonce":2075966415}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14760,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/3072311987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":3072311987},{"type":"v1","addr":"172.21.15.64:6837","nonce":3072311987}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14805,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/97995119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":97995119},{"type":"v1","addr":"172.21.15.64:6839","nonce":97995119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6841/390891068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6840","nonce":390891068},{"type":"v1","addr":"172.21.15.64:6841","nonce":390891068}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1651684499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1651684499},{"type":"v1","addr":"172.21.15.131:6833","nonce":1651684499}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24620,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1724707326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1724707326},{"type":"v1","addr":"172.21.15.119:6833","nonce":1724707326}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T04:02:03.354744+0000","modified":"2024-09-21T04:14:19.192010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24710,"mds_1":24680,"mds_2":24661},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24634":{"gid":24634,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6835/1422569359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1422569359},{"type":"v1","addr":"172.21.15.131:6835","nonce":1422569359}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24640":{"gid":24640,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.131:6837/843209972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6836","nonce":843209972},{"type":"v1","addr":"172.21.15.131:6837","nonce":843209972}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6835/3675946142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3675946142},{"type":"v1","addr":"172.21.15.119:6835","nonce":3675946142}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24661":{"gid":24661,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.131:6839/782190424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6838","nonce":782190424},{"type":"v1","addr":"172.21.15.131:6839","nonce":782190424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24680":{"gid":24680,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.119:6837/2876008744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2876008744},{"type":"v1","addr":"172.21.15.119:6837","nonce":2876008744}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24710":{"gid":24710,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6839/1319793636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1319793636},{"type":"v1","addr":"172.21.15.119:6839","nonce":1319793636}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24710,"qdb_cluster":[24661,24680,24710]},"id":1}]} 2024-09-21T04:14:33.903 INFO:teuthology.orchestra.run.smithi064.stderr:dumped fsmap epoch 44 2024-09-21T04:14:33.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.902+0000 7f3529bff640 1 -- 172.21.15.64:0/469154781 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f350c0b6680 msgr2=0x7f350c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:33.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.902+0000 7f3529bff640 1 --2- 172.21.15.64:0/469154781 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f350c0b6680 0x7f350c0b8b90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3514005fd0 tx=0x7f3514005850 comp rx=0 tx=0).stop 2024-09-21T04:14:33.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.902+0000 7f3529bff640 1 -- 172.21.15.64:0/469154781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35241083a0 msgr2=0x7f35240ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:33.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.902+0000 7f3529bff640 1 --2- 172.21.15.64:0/469154781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35241083a0 0x7f35240ffbb0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f351800d770 tx=0x7f351800dc30 comp rx=0 tx=0).stop 2024-09-21T04:14:33.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.902+0000 7f3529bff640 1 -- 172.21.15.64:0/469154781 shutdown_connections 2024-09-21T04:14:33.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.902+0000 7f3529bff640 1 --2- 172.21.15.64:0/469154781 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f350c0b6680 0x7f350c0b8b90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:33.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.902+0000 7f3529bff640 1 --2- 172.21.15.64:0/469154781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3524108cf0 0x7f35241000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:33.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.902+0000 7f3529bff640 1 --2- 172.21.15.64:0/469154781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35241083a0 0x7f35240ffbb0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:33.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.902+0000 7f3529bff640 1 --2- 172.21.15.64:0/469154781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3524102a60 0x7f35240ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:33.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.902+0000 7f3529bff640 1 -- 172.21.15.64:0/469154781 >> 172.21.15.64:0/469154781 conn(0x7f3524079730 msgr2=0x7f352406b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:33.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.902+0000 7f3529bff640 1 -- 172.21.15.64:0/469154781 shutdown_connections 2024-09-21T04:14:33.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:33.902+0000 7f3529bff640 1 -- 172.21.15.64:0/469154781 wait complete. 2024-09-21T04:14:34.276 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:34 smithi064 ceph-mon[31180]: pgmap v647: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 804 B/s wr, 8 op/s 2024-09-21T04:14:34.276 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:34 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/469154781' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:14:34.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:34 smithi131 ceph-mon[35193]: pgmap v647: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 804 B/s wr, 8 op/s 2024-09-21T04:14:34.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:34 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/469154781' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:14:34.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:34 smithi119 ceph-mon[34811]: pgmap v647: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 804 B/s wr, 8 op/s 2024-09-21T04:14:34.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:34 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/469154781' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T04:14:34.658 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T04:14:34.920 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:14:35.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.415+0000 7f1a9d2ad640 1 -- 172.21.15.64:0/3849616966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a98108d70 msgr2=0x7f1a9810f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:35.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.415+0000 7f1a9d2ad640 1 --2- 172.21.15.64:0/3849616966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a98108d70 0x7f1a9810f5a0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f1a90009ca0 tx=0x7f1a9002a8e0 comp rx=0 tx=0).stop 2024-09-21T04:14:35.433 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.415+0000 7f1a9d2ad640 1 -- 172.21.15.64:0/3849616966 shutdown_connections 2024-09-21T04:14:35.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.415+0000 7f1a9d2ad640 1 --2- 172.21.15.64:0/3849616966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a98108d70 0x7f1a9810f5a0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:35.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.415+0000 7f1a9d2ad640 1 --2- 172.21.15.64:0/3849616966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a98108420 0x7f1a98108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:35.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.415+0000 7f1a9d2ad640 1 --2- 172.21.15.64:0/3849616966 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a98102ae0 0x7f1a98102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:35.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.415+0000 7f1a9d2ad640 1 -- 172.21.15.64:0/3849616966 >> 172.21.15.64:0/3849616966 conn(0x7f1a98079770 msgr2=0x7f1a98079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:35.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.415+0000 7f1a9d2ad640 1 -- 172.21.15.64:0/3849616966 shutdown_connections 2024-09-21T04:14:35.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.415+0000 7f1a9d2ad640 1 -- 172.21.15.64:0/3849616966 wait complete. 2024-09-21T04:14:35.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.415+0000 7f1a9d2ad640 1 Processor -- start 2024-09-21T04:14:35.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.416+0000 7f1a9d2ad640 1 -- start start 2024-09-21T04:14:35.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.416+0000 7f1a9d2ad640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a98102ae0 0x7f1a9819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:35.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.416+0000 7f1a9d2ad640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a98108420 0x7f1a9819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:35.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.416+0000 7f1a9d2ad640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a98108d70 0x7f1a9819fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:35.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.416+0000 7f1a9d2ad640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1a98112af0 con 0x7f1a98102ae0 2024-09-21T04:14:35.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.416+0000 7f1a9d2ad640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1a98112970 con 0x7f1a98108d70 2024-09-21T04:14:35.434 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.416+0000 7f1a9d2ad640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1a98112c70 con 0x7f1a98108420 2024-09-21T04:14:35.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.416+0000 7f1a96ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a98102ae0 0x7f1a9819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:35.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.416+0000 7f1a96ffd640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a98102ae0 0x7f1a9819a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43944/0 (socket says 172.21.15.64:43944) 2024-09-21T04:14:35.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.416+0000 7f1a96ffd640 1 -- 172.21.15.64:0/1931607833 learned_addr learned my addr 172.21.15.64:0/1931607833 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:14:35.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.416+0000 7f1a967fc640 1 --2- 172.21.15.64:0/1931607833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a98108420 0x7f1a9819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:35.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.416+0000 7f1a977fe640 1 --2- 172.21.15.64:0/1931607833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a98108d70 0x7f1a9819fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:35.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.416+0000 7f1a96ffd640 1 -- 172.21.15.64:0/1931607833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a98108420 msgr2=0x7f1a9819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:35.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.416+0000 7f1a96ffd640 1 --2- 172.21.15.64:0/1931607833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a98108420 0x7f1a9819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:35.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.416+0000 7f1a96ffd640 1 -- 172.21.15.64:0/1931607833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a98108d70 msgr2=0x7f1a9819fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:35.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.416+0000 7f1a96ffd640 1 --2- 172.21.15.64:0/1931607833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a98108d70 0x7f1a9819fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:35.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.416+0000 7f1a96ffd640 1 -- 172.21.15.64:0/1931607833 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a981a0540 con 0x7f1a98102ae0 2024-09-21T04:14:35.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.417+0000 7f1a96ffd640 1 --2- 172.21.15.64:0/1931607833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a98102ae0 0x7f1a9819a5d0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f1a8800b4a0 tx=0x7f1a8800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:35.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.417+0000 7f1a8ffff640 1 -- 172.21.15.64:0/1931607833 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88014070 con 0x7f1a98102ae0 2024-09-21T04:14:35.435 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.417+0000 7f1a8ffff640 1 -- 172.21.15.64:0/1931607833 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a88010ac0 con 0x7f1a98102ae0 2024-09-21T04:14:35.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.417+0000 7f1a8ffff640 1 -- 172.21.15.64:0/1931607833 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88017c30 con 0x7f1a98102ae0 2024-09-21T04:14:35.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.417+0000 7f1a9d2ad640 1 -- 172.21.15.64:0/1931607833 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a981a0830 con 0x7f1a98102ae0 2024-09-21T04:14:35.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.417+0000 7f1a9d2ad640 1 -- 172.21.15.64:0/1931607833 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a981a6f90 con 0x7f1a98102ae0 2024-09-21T04:14:35.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.418+0000 7f1a9d2ad640 1 -- 172.21.15.64:0/1931607833 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a98103d60 con 0x7f1a98102ae0 2024-09-21T04:14:35.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.419+0000 7f1a8ffff640 1 -- 172.21.15.64:0/1931607833 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f1a88017070 con 0x7f1a98102ae0 2024-09-21T04:14:35.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.420+0000 7f1a8ffff640 1 --2- 172.21.15.64:0/1931607833 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f1a840b6700 0x7f1a840b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:35.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.420+0000 7f1a967fc640 1 --2- 172.21.15.64:0/1931607833 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f1a840b6700 0x7f1a840b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:35.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.420+0000 7f1a8ffff640 1 -- 172.21.15.64:0/1931607833 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8615+0+0 (secure 0 0 0) 0x7f1a880e6eb0 con 0x7f1a98102ae0 2024-09-21T04:14:35.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.421+0000 7f1a967fc640 1 --2- 172.21.15.64:0/1931607833 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f1a840b6700 0x7f1a840b8c10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1a98108040 tx=0x7f1a80008040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:35.436 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.423+0000 7f1a8ffff640 1 -- 172.21.15.64:0/1931607833 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1a880ad340 con 0x7f1a98102ae0 2024-09-21T04:14:35.616 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.613+0000 7f1a9d2ad640 1 -- 172.21.15.64:0/1931607833 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1a98102ef0 con 0x7f1a98102ae0 2024-09-21T04:14:35.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.615+0000 7f1a8ffff640 1 -- 172.21.15.64:0/1931607833 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+23320 (secure 0 0 0) 0x7f1a98102ef0 con 0x7f1a98102ae0 2024-09-21T04:14:35.618 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-09-21T04:14:35.618 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":87,"fsid":"e7c927ba-77cc-11ef-baf5-efdc52797490","created":"2024-09-21T03:53:41.215073+0000","modified":"2024-09-21T04:14:21.240943+0000","last_up_change":"2024-09-21T04:01:11.986949+0000","last_in_change":"2024-09-21T04:00:48.707791+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T03:57:24.854438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T04:01:58.728355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T04:02:01.300385+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":86,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb947c10-98e6-4e3a-8bbd-8753951d870b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6803","nonce":3692023035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6805","nonce":3692023035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6809","nonce":3692023035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":3692023035},{"type":"v1","addr":"172.21.15.64:6807","nonce":3692023035}]},"public_addr":"172.21.15.64:6803/3692023035","cluster_addr":"172.21.15.64:6805/3692023035","heartbeat_back_addr":"172.21.15.64:6809/3692023035","heartbeat_front_addr":"172.21.15.64:6807/3692023035","state":["exists","up"]},{"osd":1,"uuid":"29b8c92d-82fa-4175-b3e2-711917b7f8c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6801","nonce":3363415998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6803","nonce":3363415998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6807","nonce":3363415998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":3363415998},{"type":"v1","addr":"172.21.15.119:6805","nonce":3363415998}]},"public_addr":"172.21.15.119:6801/3363415998","cluster_addr":"172.21.15.119:6803/3363415998","heartbeat_back_addr":"172.21.15.119:6807/3363415998","heartbeat_front_addr":"172.21.15.119:6805/3363415998","state":["exists","up"]},{"osd":2,"uuid":"a2273fe8-df55-40f7-afda-74ab09c52c55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6801","nonce":3422924179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6803","nonce":3422924179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6807","nonce":3422924179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":3422924179},{"type":"v1","addr":"172.21.15.131:6805","nonce":3422924179}]},"public_addr":"172.21.15.131:6801/3422924179","cluster_addr":"172.21.15.131:6803/3422924179","heartbeat_back_addr":"172.21.15.131:6807/3422924179","heartbeat_front_addr":"172.21.15.131:6805/3422924179","state":["exists","up"]},{"osd":3,"uuid":"f1634064-b108-41d2-a78f-c33607b08a70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6811","nonce":2022952729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6813","nonce":2022952729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6817","nonce":2022952729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":2022952729},{"type":"v1","addr":"172.21.15.64:6815","nonce":2022952729}]},"public_addr":"172.21.15.64:6811/2022952729","cluster_addr":"172.21.15.64:6813/2022952729","heartbeat_back_addr":"172.21.15.64:6817/2022952729","heartbeat_front_addr":"172.21.15.64:6815/2022952729","state":["exists","up"]},{"osd":4,"uuid":"75b81a8d-fa52-411a-bfda-aa69bb70a20f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6809","nonce":323030958}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6811","nonce":323030958}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6815","nonce":323030958}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":323030958},{"type":"v1","addr":"172.21.15.119:6813","nonce":323030958}]},"public_addr":"172.21.15.119:6809/323030958","cluster_addr":"172.21.15.119:6811/323030958","heartbeat_back_addr":"172.21.15.119:6815/323030958","heartbeat_front_addr":"172.21.15.119:6813/323030958","state":["exists","up"]},{"osd":5,"uuid":"0e4895b5-42e4-4b2c-a9c3-692f287cb841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6809","nonce":3021788088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6811","nonce":3021788088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6815","nonce":3021788088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":3021788088},{"type":"v1","addr":"172.21.15.131:6813","nonce":3021788088}]},"public_addr":"172.21.15.131:6809/3021788088","cluster_addr":"172.21.15.131:6811/3021788088","heartbeat_back_addr":"172.21.15.131:6815/3021788088","heartbeat_front_addr":"172.21.15.131:6813/3021788088","state":["exists","up"]},{"osd":6,"uuid":"1fa5c2b7-bce0-43a1-a26b-d029eb3731fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6819","nonce":1913824324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6821","nonce":1913824324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6825","nonce":1913824324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1913824324},{"type":"v1","addr":"172.21.15.64:6823","nonce":1913824324}]},"public_addr":"172.21.15.64:6819/1913824324","cluster_addr":"172.21.15.64:6821/1913824324","heartbeat_back_addr":"172.21.15.64:6825/1913824324","heartbeat_front_addr":"172.21.15.64:6823/1913824324","state":["exists","up"]},{"osd":7,"uuid":"c9707260-b581-4a21-b380-f37e16959b02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6817","nonce":2567568166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6819","nonce":2567568166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6823","nonce":2567568166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2567568166},{"type":"v1","addr":"172.21.15.119:6821","nonce":2567568166}]},"public_addr":"172.21.15.119:6817/2567568166","cluster_addr":"172.21.15.119:6819/2567568166","heartbeat_back_addr":"172.21.15.119:6823/2567568166","heartbeat_front_addr":"172.21.15.119:6821/2567568166","state":["exists","up"]},{"osd":8,"uuid":"2dbf8ef2-fc9c-4c6b-bfea-003f41ef191e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6817","nonce":199192217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6819","nonce":199192217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6823","nonce":199192217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":199192217},{"type":"v1","addr":"172.21.15.131:6821","nonce":199192217}]},"public_addr":"172.21.15.131:6817/199192217","cluster_addr":"172.21.15.131:6819/199192217","heartbeat_back_addr":"172.21.15.131:6823/199192217","heartbeat_front_addr":"172.21.15.131:6821/199192217","state":["exists","up"]},{"osd":9,"uuid":"96f769e5-4783-4558-adfa-4bfaf9da9e9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6827","nonce":3309038990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6829","nonce":3309038990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6833","nonce":3309038990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":3309038990},{"type":"v1","addr":"172.21.15.64:6831","nonce":3309038990}]},"public_addr":"172.21.15.64:6827/3309038990","cluster_addr":"172.21.15.64:6829/3309038990","heartbeat_back_addr":"172.21.15.64:6833/3309038990","heartbeat_front_addr":"172.21.15.64:6831/3309038990","state":["exists","up"]},{"osd":10,"uuid":"13c1ef5c-2299-440c-af5a-a650b47f4b75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6825","nonce":1460844681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6827","nonce":1460844681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6831","nonce":1460844681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1460844681},{"type":"v1","addr":"172.21.15.119:6829","nonce":1460844681}]},"public_addr":"172.21.15.119:6825/1460844681","cluster_addr":"172.21.15.119:6827/1460844681","heartbeat_back_addr":"172.21.15.119:6831/1460844681","heartbeat_front_addr":"172.21.15.119:6829/1460844681","state":["exists","up"]},{"osd":11,"uuid":"c26a5ef9-8585-404e-a54a-852424c4d6fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6825","nonce":2294059084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6827","nonce":2294059084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6831","nonce":2294059084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":2294059084},{"type":"v1","addr":"172.21.15.131:6829","nonce":2294059084}]},"public_addr":"172.21.15.131:6825/2294059084","cluster_addr":"172.21.15.131:6827/2294059084","heartbeat_back_addr":"172.21.15.131:6831/2294059084","heartbeat_front_addr":"172.21.15.131:6829/2294059084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:56:33.141102+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:56:58.883062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:57:22.641789+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:57:48.705790+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:58:12.871514+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:58:37.893118+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:03.620828+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:29.446179+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T03:59:55.002978+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:00:19.987463+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:00:45.364158+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T04:01:10.228248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.3","mappings":[{"from":10,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":5,"to":6}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.64:0/3554471670":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6801/3786213009":"2024-09-22T03:55:28.387426+0000","172.21.15.64:0/2562856406":"2024-09-22T03:54:27.180100+0000","172.21.15.64:0/2603015699":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6800/1209932961":"2024-09-22T03:54:02.562380+0000","172.21.15.64:6800/3786213009":"2024-09-22T03:55:28.387426+0000","172.21.15.64:6801/1209932961":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/3407344596":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1233731831":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1686887701":"2024-09-22T03:54:02.562380+0000","172.21.15.64:0/1598568021":"2024-09-22T03:54:27.180100+0000","172.21.15.64:6800/2013929312":"2024-09-22T03:54:27.180100+0000","172.21.15.64:6801/2013929312":"2024-09-22T03:54:27.180100+0000","172.21.15.64:0/3652036144":"2024-09-22T03:55:28.387426+0000","172.21.15.64:0/105454703":"2024-09-22T03:54:27.180100+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T04:14:35.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.618+0000 7f1a9d2ad640 1 -- 172.21.15.64:0/1931607833 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f1a840b6700 msgr2=0x7f1a840b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:35.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.618+0000 7f1a9d2ad640 1 --2- 172.21.15.64:0/1931607833 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f1a840b6700 0x7f1a840b8c10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1a98108040 tx=0x7f1a80008040 comp rx=0 tx=0).stop 2024-09-21T04:14:35.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.618+0000 7f1a9d2ad640 1 -- 172.21.15.64:0/1931607833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a98102ae0 msgr2=0x7f1a9819a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:35.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.618+0000 7f1a9d2ad640 1 --2- 172.21.15.64:0/1931607833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a98102ae0 0x7f1a9819a5d0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f1a8800b4a0 tx=0x7f1a8800b960 comp rx=0 tx=0).stop 2024-09-21T04:14:35.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.618+0000 7f1a9d2ad640 1 -- 172.21.15.64:0/1931607833 shutdown_connections 2024-09-21T04:14:35.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.618+0000 7f1a9d2ad640 1 --2- 172.21.15.64:0/1931607833 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f1a840b6700 0x7f1a840b8c10 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:35.622 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.618+0000 7f1a9d2ad640 1 --2- 172.21.15.64:0/1931607833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a98108d70 0x7f1a9819fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:35.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.618+0000 7f1a9d2ad640 1 --2- 172.21.15.64:0/1931607833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1a98108420 0x7f1a9819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:35.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.618+0000 7f1a9d2ad640 1 --2- 172.21.15.64:0/1931607833 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1a98102ae0 0x7f1a9819a5d0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:35.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.618+0000 7f1a9d2ad640 1 -- 172.21.15.64:0/1931607833 >> 172.21.15.64:0/1931607833 conn(0x7f1a98079770 msgr2=0x7f1a98101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:35.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.618+0000 7f1a9d2ad640 1 -- 172.21.15.64:0/1931607833 shutdown_connections 2024-09-21T04:14:35.623 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:35.618+0000 7f1a9d2ad640 1 -- 172.21.15.64:0/1931607833 wait complete. 2024-09-21T04:14:36.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:36 smithi064 ceph-mon[31180]: pgmap v648: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-21T04:14:36.262 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:36 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1931607833' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:14:36.303 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:36 smithi131 ceph-mon[35193]: pgmap v648: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-21T04:14:36.304 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:36 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1931607833' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:14:36.474 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs fail cephfs 2024-09-21T04:14:36.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:36 smithi119 ceph-mon[34811]: pgmap v648: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-21T04:14:36.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:36 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1931607833' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T04:14:36.733 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:14:37.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.226+0000 7fa22b215640 1 -- 172.21.15.64:0/1252227910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa224102ae0 msgr2=0x7fa224102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:37.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.226+0000 7fa22b215640 1 --2- 172.21.15.64:0/1252227910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa224102ae0 0x7fa224102ef0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa20c009900 tx=0x7fa20c02a410 comp rx=0 tx=0).stop 2024-09-21T04:14:37.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.227+0000 7fa22b215640 1 -- 172.21.15.64:0/1252227910 shutdown_connections 2024-09-21T04:14:37.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.227+0000 7fa22b215640 1 --2- 172.21.15.64:0/1252227910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa224108d20 0x7fa22410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:37.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.227+0000 7fa22b215640 1 --2- 172.21.15.64:0/1252227910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2241083d0 0x7fa2241087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:37.236 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.227+0000 7fa22b215640 1 --2- 172.21.15.64:0/1252227910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa224102ae0 0x7fa224102ef0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:37.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.227+0000 7fa22b215640 1 -- 172.21.15.64:0/1252227910 >> 172.21.15.64:0/1252227910 conn(0x7fa224079760 msgr2=0x7fa224079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:37.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.227+0000 7fa22b215640 1 -- 172.21.15.64:0/1252227910 shutdown_connections 2024-09-21T04:14:37.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.227+0000 7fa22b215640 1 -- 172.21.15.64:0/1252227910 wait complete. 2024-09-21T04:14:37.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.227+0000 7fa22b215640 1 Processor -- start 2024-09-21T04:14:37.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.227+0000 7fa22b215640 1 -- start start 2024-09-21T04:14:37.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.228+0000 7fa22b215640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa224102ae0 0x7fa22419b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:37.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.228+0000 7fa22b215640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2241083d0 0x7fa22419b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:37.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.228+0000 7fa22b215640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa224108d20 0x7fa22419bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:37.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.228+0000 7fa22b215640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa224112990 con 0x7fa224102ae0 2024-09-21T04:14:37.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.228+0000 7fa22b215640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa224112810 con 0x7fa224108d20 2024-09-21T04:14:37.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.228+0000 7fa22b215640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa224112b10 con 0x7fa2241083d0 2024-09-21T04:14:37.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.228+0000 7fa228fb1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa224102ae0 0x7fa22419b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:37.237 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.228+0000 7fa228fb1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa224102ae0 0x7fa22419b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43970/0 (socket says 172.21.15.64:43970) 2024-09-21T04:14:37.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.228+0000 7fa228fb1640 1 -- 172.21.15.64:0/1902568354 learned_addr learned my addr 172.21.15.64:0/1902568354 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:14:37.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.228+0000 7fa2297b2640 1 --2- 172.21.15.64:0/1902568354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa224108d20 0x7fa22419bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:37.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.228+0000 7fa223fff640 1 --2- 172.21.15.64:0/1902568354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2241083d0 0x7fa22419b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:37.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.228+0000 7fa228fb1640 1 -- 172.21.15.64:0/1902568354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2241083d0 msgr2=0x7fa22419b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:37.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.228+0000 7fa228fb1640 1 --2- 172.21.15.64:0/1902568354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2241083d0 0x7fa22419b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:37.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.228+0000 7fa228fb1640 1 -- 172.21.15.64:0/1902568354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa224108d20 msgr2=0x7fa22419bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:37.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.228+0000 7fa228fb1640 1 --2- 172.21.15.64:0/1902568354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa224108d20 0x7fa22419bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:37.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.228+0000 7fa228fb1640 1 -- 172.21.15.64:0/1902568354 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2241a6560 con 0x7fa224102ae0 2024-09-21T04:14:37.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.228+0000 7fa223fff640 1 --2- 172.21.15.64:0/1902568354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2241083d0 0x7fa22419b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:14:37.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.228+0000 7fa228fb1640 1 --2- 172.21.15.64:0/1902568354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa224102ae0 0x7fa22419b2a0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa20c005940 tx=0x7fa20c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:37.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.228+0000 7fa221ffb640 1 -- 172.21.15.64:0/1902568354 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20c03f030 con 0x7fa224102ae0 2024-09-21T04:14:37.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.229+0000 7fa221ffb640 1 -- 172.21.15.64:0/1902568354 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa20c003aa0 con 0x7fa224102ae0 2024-09-21T04:14:37.238 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.229+0000 7fa221ffb640 1 -- 172.21.15.64:0/1902568354 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20c035070 con 0x7fa224102ae0 2024-09-21T04:14:37.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.229+0000 7fa22b215640 1 -- 172.21.15.64:0/1902568354 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2241a67f0 con 0x7fa224102ae0 2024-09-21T04:14:37.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.229+0000 7fa22b215640 1 -- 172.21.15.64:0/1902568354 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2241a6c80 con 0x7fa224102ae0 2024-09-21T04:14:37.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.230+0000 7fa22b215640 1 -- 172.21.15.64:0/1902568354 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa224103d10 con 0x7fa224102ae0 2024-09-21T04:14:37.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.231+0000 7fa221ffb640 1 -- 172.21.15.64:0/1902568354 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fa20c042030 con 0x7fa224102ae0 2024-09-21T04:14:37.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.232+0000 7fa221ffb640 1 --2- 172.21.15.64:0/1902568354 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa2100b6700 0x7fa2100b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:37.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.233+0000 7fa223fff640 1 --2- 172.21.15.64:0/1902568354 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa2100b6700 0x7fa2100b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:37.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.233+0000 7fa221ffb640 1 -- 172.21.15.64:0/1902568354 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8615+0+0 (secure 0 0 0) 0x7fa20c108140 con 0x7fa224102ae0 2024-09-21T04:14:37.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.233+0000 7fa223fff640 1 --2- 172.21.15.64:0/1902568354 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa2100b6700 0x7fa2100b8c10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa224100c50 tx=0x7fa214005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:37.239 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.236+0000 7fa221ffb640 1 -- 172.21.15.64:0/1902568354 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa20c0ce630 con 0x7fa224102ae0 2024-09-21T04:14:37.467 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.464+0000 7fa22b215640 1 -- 172.21.15.64:0/1902568354 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa22406bf70 con 0x7fa224102ae0 2024-09-21T04:14:37.469 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.467+0000 7fa221ffb640 1 -- 172.21.15.64:0/1902568354 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) ==== 161+0+0 (secure 0 0 0) 0x7fa20c0d2a10 con 0x7fa224102ae0 2024-09-21T04:14:37.470 INFO:teuthology.orchestra.run.smithi064.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T04:14:37.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.471+0000 7fa22b215640 1 -- 172.21.15.64:0/1902568354 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa2100b6700 msgr2=0x7fa2100b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:37.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.471+0000 7fa22b215640 1 --2- 172.21.15.64:0/1902568354 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa2100b6700 0x7fa2100b8c10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa224100c50 tx=0x7fa214005f90 comp rx=0 tx=0).stop 2024-09-21T04:14:37.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.471+0000 7fa22b215640 1 -- 172.21.15.64:0/1902568354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa224102ae0 msgr2=0x7fa22419b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:37.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.471+0000 7fa22b215640 1 --2- 172.21.15.64:0/1902568354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa224102ae0 0x7fa22419b2a0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa20c005940 tx=0x7fa20c004270 comp rx=0 tx=0).stop 2024-09-21T04:14:37.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.471+0000 7fa22b215640 1 -- 172.21.15.64:0/1902568354 shutdown_connections 2024-09-21T04:14:37.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.471+0000 7fa22b215640 1 --2- 172.21.15.64:0/1902568354 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fa2100b6700 0x7fa2100b8c10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:37.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.471+0000 7fa22b215640 1 --2- 172.21.15.64:0/1902568354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa224108d20 0x7fa22419bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:37.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.471+0000 7fa22b215640 1 --2- 172.21.15.64:0/1902568354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2241083d0 0x7fa22419b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:37.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.471+0000 7fa22b215640 1 --2- 172.21.15.64:0/1902568354 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa224102ae0 0x7fa22419b2a0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:37.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.471+0000 7fa22b215640 1 -- 172.21.15.64:0/1902568354 >> 172.21.15.64:0/1902568354 conn(0x7fa224079760 msgr2=0x7fa224101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:37.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.471+0000 7fa22b215640 1 -- 172.21.15.64:0/1902568354 shutdown_connections 2024-09-21T04:14:37.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:37.471+0000 7fa22b215640 1 -- 172.21.15.64:0/1902568354 wait complete. 2024-09-21T04:14:37.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:37 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-a[31176]: 2024-09-21T04:14:37.465+0000 7f00e529e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T04:14:37.803 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 21 04:14:37 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-c[64361]: ignoring --setuser ceph since I am not root 2024-09-21T04:14:37.804 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 21 04:14:37 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-c[64361]: ignoring --setgroup ceph since I am not root 2024-09-21T04:14:37.804 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 21 04:14:37 smithi131 ceph-mds[64365]: -- 172.21.15.131:0/4183506861 <== mon.1 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5566254e1c20 con 0x556629689600 2024-09-21T04:14:37.804 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 21 04:14:37 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-c[64361]: starting mds.c at 2024-09-21T04:14:37.804 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 21 04:14:37 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-f[64087]: ignoring --setuser ceph since I am not root 2024-09-21T04:14:37.804 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 21 04:14:37 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-f[64087]: ignoring --setgroup ceph since I am not root 2024-09-21T04:14:37.804 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 21 04:14:37 smithi131 ceph-mds[64091]: -- 172.21.15.131:0/2337428217 <== mon.1 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5608e2631860 con 0x5608e67d9180 2024-09-21T04:14:37.804 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 21 04:14:37 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-f[64087]: starting mds.f at 2024-09-21T04:14:37.805 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 21 04:14:37 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-i[64631]: ignoring --setuser ceph since I am not root 2024-09-21T04:14:37.805 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 21 04:14:37 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-i[64631]: ignoring --setgroup ceph since I am not root 2024-09-21T04:14:37.805 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 21 04:14:37 smithi131 ceph-mds[64635]: -- 172.21.15.131:0/3525079145 <== mon.1 v2:172.21.15.131:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e917fbd860 con 0x55e91c165a80 2024-09-21T04:14:37.805 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 21 04:14:37 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-i[64631]: starting mds.i at 2024-09-21T04:14:38.007 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 04:14:37 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-b[64168]: ignoring --setuser ceph since I am not root 2024-09-21T04:14:38.007 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 04:14:37 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-b[64168]: ignoring --setgroup ceph since I am not root 2024-09-21T04:14:38.007 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 04:14:37 smithi119 ceph-mds[64172]: -- 172.21.15.119:0/204623313 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5621eeb29860 con 0x5621f2cd1a80 2024-09-21T04:14:38.007 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 04:14:37 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-b[64168]: starting mds.b at 2024-09-21T04:14:38.007 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 04:14:37 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-e[63617]: ignoring --setuser ceph since I am not root 2024-09-21T04:14:38.008 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 04:14:37 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-e[63617]: ignoring --setgroup ceph since I am not root 2024-09-21T04:14:38.008 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 04:14:37 smithi119 ceph-mds[63621]: -- 172.21.15.119:0/1872867122 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ca23bbf860 con 0x55ca27d67180 2024-09-21T04:14:38.008 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 04:14:37 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-e[63617]: starting mds.e at 2024-09-21T04:14:38.008 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 04:14:37 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-h[63883]: ignoring --setuser ceph since I am not root 2024-09-21T04:14:38.008 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 04:14:37 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-h[63883]: ignoring --setgroup ceph since I am not root 2024-09-21T04:14:38.008 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 04:14:37 smithi119 ceph-mds[63887]: -- 172.21.15.119:0/1488457793 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e04c467860 con 0x55e05060f180 2024-09-21T04:14:38.008 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 04:14:37 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-h[63883]: starting mds.h at 2024-09-21T04:14:38.124 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:38 smithi064 ceph-mon[31180]: pgmap v649: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-21T04:14:38.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:38 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1902568354' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T04:14:38.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:38 smithi064 ceph-mon[31180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T04:14:38.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:38 smithi064 ceph-mon[31180]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T04:14:38.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:38 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/1902568354' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T04:14:38.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:38 smithi064 ceph-mon[31180]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T04:14:38.125 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:38 smithi064 ceph-mon[31180]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-21T04:14:38.155 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T04:14:38.413 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:14:38.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:38 smithi119 ceph-mon[34811]: pgmap v649: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-21T04:14:38.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:38 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1902568354' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T04:14:38.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:38 smithi119 ceph-mon[34811]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T04:14:38.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:38 smithi119 ceph-mon[34811]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T04:14:38.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:38 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/1902568354' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T04:14:38.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:38 smithi119 ceph-mon[34811]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T04:14:38.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:38 smithi119 ceph-mon[34811]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-21T04:14:38.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:38 smithi131 ceph-mon[35193]: pgmap v649: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-21T04:14:38.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:38 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1902568354' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T04:14:38.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:38 smithi131 ceph-mon[35193]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T04:14:38.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:38 smithi131 ceph-mon[35193]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T04:14:38.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:38 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/1902568354' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T04:14:38.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:38 smithi131 ceph-mon[35193]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T04:14:38.554 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:38 smithi131 ceph-mon[35193]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-21T04:14:39.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.994+0000 7f0ae8594640 1 -- 172.21.15.64:0/2018387329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ae0103bc0 msgr2=0x7f0ae0103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:39.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.994+0000 7f0ae8594640 1 --2- 172.21.15.64:0/2018387329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ae0103bc0 0x7f0ae0103fd0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f0adc009900 tx=0x7f0adc02a410 comp rx=0 tx=0).stop 2024-09-21T04:14:39.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.994+0000 7f0ae8594640 1 -- 172.21.15.64:0/2018387329 shutdown_connections 2024-09-21T04:14:39.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.994+0000 7f0ae8594640 1 --2- 172.21.15.64:0/2018387329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ae0109e00 0x7f0ae0110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:39.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.994+0000 7f0ae8594640 1 --2- 172.21.15.64:0/2018387329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ae01094b0 0x7f0ae01098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:39.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.994+0000 7f0ae8594640 1 --2- 172.21.15.64:0/2018387329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ae0103bc0 0x7f0ae0103fd0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:39.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.994+0000 7f0ae8594640 1 -- 172.21.15.64:0/2018387329 >> 172.21.15.64:0/2018387329 conn(0x7f0ae007b910 msgr2=0x7f0ae007bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:39.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.994+0000 7f0ae8594640 1 -- 172.21.15.64:0/2018387329 shutdown_connections 2024-09-21T04:14:39.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.995+0000 7f0ae8594640 1 -- 172.21.15.64:0/2018387329 wait complete. 2024-09-21T04:14:39.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.995+0000 7f0ae8594640 1 Processor -- start 2024-09-21T04:14:39.012 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.995+0000 7f0ae8594640 1 -- start start 2024-09-21T04:14:39.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.995+0000 7f0ae8594640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ae0103bc0 0x7f0ae019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:39.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.995+0000 7f0ae8594640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ae01094b0 0x7f0ae019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:39.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.995+0000 7f0ae8594640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ae0109e00 0x7f0ae019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:39.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.995+0000 7f0ae8594640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae0113ac0 con 0x7f0ae01094b0 2024-09-21T04:14:39.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.995+0000 7f0ae8594640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae0113940 con 0x7f0ae0109e00 2024-09-21T04:14:39.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.995+0000 7f0ae8594640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae0113c40 con 0x7f0ae0103bc0 2024-09-21T04:14:39.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.996+0000 7f0ae5b2f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ae01094b0 0x7f0ae019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:39.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.996+0000 7f0ae5b2f640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ae01094b0 0x7f0ae019aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43988/0 (socket says 172.21.15.64:43988) 2024-09-21T04:14:39.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.996+0000 7f0ae5b2f640 1 -- 172.21.15.64:0/501486040 learned_addr learned my addr 172.21.15.64:0/501486040 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:14:39.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.996+0000 7f0ae6330640 1 --2- 172.21.15.64:0/501486040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ae0103bc0 0x7f0ae019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:39.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.996+0000 7f0ae6b31640 1 --2- 172.21.15.64:0/501486040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ae0109e00 0x7f0ae019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:39.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.996+0000 7f0ae5b2f640 1 -- 172.21.15.64:0/501486040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ae0103bc0 msgr2=0x7f0ae019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:39.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.996+0000 7f0ae5b2f640 1 --2- 172.21.15.64:0/501486040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ae0103bc0 0x7f0ae019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:39.013 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.996+0000 7f0ae5b2f640 1 -- 172.21.15.64:0/501486040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ae0109e00 msgr2=0x7f0ae019fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:39.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.996+0000 7f0ae5b2f640 1 --2- 172.21.15.64:0/501486040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ae0109e00 0x7f0ae019fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:39.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.996+0000 7f0ae5b2f640 1 -- 172.21.15.64:0/501486040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae01a0460 con 0x7f0ae01094b0 2024-09-21T04:14:39.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.996+0000 7f0ae6b31640 1 --2- 172.21.15.64:0/501486040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ae0109e00 0x7f0ae019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T04:14:39.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.996+0000 7f0ae5b2f640 1 --2- 172.21.15.64:0/501486040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ae01094b0 0x7f0ae019aa90 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000d950 tx=0x7f0ad000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:39.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.996+0000 7f0ad77fe640 1 -- 172.21.15.64:0/501486040 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0017070 con 0x7f0ae01094b0 2024-09-21T04:14:39.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.996+0000 7f0ad77fe640 1 -- 172.21.15.64:0/501486040 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad0009c30 con 0x7f0ae01094b0 2024-09-21T04:14:39.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.996+0000 7f0ad77fe640 1 -- 172.21.15.64:0/501486040 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad00047a0 con 0x7f0ae01094b0 2024-09-21T04:14:39.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.996+0000 7f0ae8594640 1 -- 172.21.15.64:0/501486040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae01a0750 con 0x7f0ae01094b0 2024-09-21T04:14:39.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.996+0000 7f0ae8594640 1 -- 172.21.15.64:0/501486040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae01a6eb0 con 0x7f0ae01094b0 2024-09-21T04:14:39.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.997+0000 7f0ae8594640 1 -- 172.21.15.64:0/501486040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ae0103fd0 con 0x7f0ae01094b0 2024-09-21T04:14:39.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:38.999+0000 7f0ad77fe640 1 -- 172.21.15.64:0/501486040 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f0ad0004020 con 0x7f0ae01094b0 2024-09-21T04:14:39.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:39.000+0000 7f0ad77fe640 1 --2- 172.21.15.64:0/501486040 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0acc0b6700 0x7f0acc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:39.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:39.000+0000 7f0ae6330640 1 --2- 172.21.15.64:0/501486040 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0acc0b6700 0x7f0acc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:39.014 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:39.000+0000 7f0ad77fe640 1 -- 172.21.15.64:0/501486040 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8829+0+0 (secure 0 0 0) 0x7f0ad00e78c0 con 0x7f0ae01094b0 2024-09-21T04:14:39.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:39.000+0000 7f0ae6330640 1 --2- 172.21.15.64:0/501486040 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0acc0b6700 0x7f0acc0b8c10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0adc031440 tx=0x7f0adc009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:39.015 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:39.003+0000 7f0ad77fe640 1 -- 172.21.15.64:0/501486040 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0ad0020020 con 0x7f0ae01094b0 2024-09-21T04:14:39.230 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:39.227+0000 7f0ae8594640 1 -- 172.21.15.64:0/501486040 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f0ae002e640 con 0x7f0ae01094b0 2024-09-21T04:14:39.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:39.476+0000 7f0ad77fe640 1 -- 172.21.15.64:0/501486040 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v47) ==== 104+0+0 (secure 0 0 0) 0x7f0ad00add60 con 0x7f0ae01094b0 2024-09-21T04:14:39.480 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:39 smithi064 ceph-mon[31180]: mds.? [v2:172.21.15.131:6834/2310423272,v1:172.21.15.131:6835/2310423272] up:boot 2024-09-21T04:14:39.480 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:39 smithi064 ceph-mon[31180]: mds.? [v2:172.21.15.119:6834/1272115218,v1:172.21.15.119:6835/1272115218] up:boot 2024-09-21T04:14:39.480 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:39 smithi064 ceph-mon[31180]: mds.? [v2:172.21.15.131:6836/3752150166,v1:172.21.15.131:6837/3752150166] up:boot 2024-09-21T04:14:39.480 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:39 smithi064 ceph-mon[31180]: mds.? [v2:172.21.15.131:6838/1085660404,v1:172.21.15.131:6839/1085660404] up:boot 2024-09-21T04:14:39.481 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:39 smithi064 ceph-mon[31180]: mds.? [v2:172.21.15.119:6836/3417164847,v1:172.21.15.119:6837/3417164847] up:boot 2024-09-21T04:14:39.481 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:39 smithi064 ceph-mon[31180]: mds.? [v2:172.21.15.119:6838/3693330738,v1:172.21.15.119:6839/3693330738] up:boot 2024-09-21T04:14:39.481 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:39 smithi064 ceph-mon[31180]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-21T04:14:39.481 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:39 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T04:14:39.481 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:39 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T04:14:39.481 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:39 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T04:14:39.481 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:39 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T04:14:39.481 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:39 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T04:14:39.481 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:39 smithi064 ceph-mon[31180]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T04:14:39.481 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:39 smithi064 ceph-mon[31180]: pgmap v651: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-21T04:14:39.481 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:39 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/501486040' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T04:14:39.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:39.479+0000 7f0ae8594640 1 -- 172.21.15.64:0/501486040 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0acc0b6700 msgr2=0x7f0acc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:39.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:39.479+0000 7f0ae8594640 1 --2- 172.21.15.64:0/501486040 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0acc0b6700 0x7f0acc0b8c10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0adc031440 tx=0x7f0adc009560 comp rx=0 tx=0).stop 2024-09-21T04:14:39.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:39.479+0000 7f0ae8594640 1 -- 172.21.15.64:0/501486040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ae01094b0 msgr2=0x7f0ae019aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:39.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:39.479+0000 7f0ae8594640 1 --2- 172.21.15.64:0/501486040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ae01094b0 0x7f0ae019aa90 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000d950 tx=0x7f0ad000de10 comp rx=0 tx=0).stop 2024-09-21T04:14:39.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:39.479+0000 7f0ae8594640 1 -- 172.21.15.64:0/501486040 shutdown_connections 2024-09-21T04:14:39.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:39.479+0000 7f0ae8594640 1 --2- 172.21.15.64:0/501486040 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f0acc0b6700 0x7f0acc0b8c10 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:39.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:39.479+0000 7f0ae8594640 1 --2- 172.21.15.64:0/501486040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ae0109e00 0x7f0ae019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:39.482 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:39.479+0000 7f0ae8594640 1 --2- 172.21.15.64:0/501486040 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ae01094b0 0x7f0ae019aa90 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:39.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:39.479+0000 7f0ae8594640 1 --2- 172.21.15.64:0/501486040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ae0103bc0 0x7f0ae019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:39.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:39.479+0000 7f0ae8594640 1 -- 172.21.15.64:0/501486040 >> 172.21.15.64:0/501486040 conn(0x7f0ae007b910 msgr2=0x7f0ae01075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:39.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:39.479+0000 7f0ae8594640 1 -- 172.21.15.64:0/501486040 shutdown_connections 2024-09-21T04:14:39.483 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:39.479+0000 7f0ae8594640 1 -- 172.21.15.64:0/501486040 wait complete. 2024-09-21T04:14:39.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:39 smithi119 ceph-mon[34811]: mds.? [v2:172.21.15.131:6834/2310423272,v1:172.21.15.131:6835/2310423272] up:boot 2024-09-21T04:14:39.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:39 smithi119 ceph-mon[34811]: mds.? [v2:172.21.15.119:6834/1272115218,v1:172.21.15.119:6835/1272115218] up:boot 2024-09-21T04:14:39.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:39 smithi119 ceph-mon[34811]: mds.? [v2:172.21.15.131:6836/3752150166,v1:172.21.15.131:6837/3752150166] up:boot 2024-09-21T04:14:39.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:39 smithi119 ceph-mon[34811]: mds.? [v2:172.21.15.131:6838/1085660404,v1:172.21.15.131:6839/1085660404] up:boot 2024-09-21T04:14:39.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:39 smithi119 ceph-mon[34811]: mds.? [v2:172.21.15.119:6836/3417164847,v1:172.21.15.119:6837/3417164847] up:boot 2024-09-21T04:14:39.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:39 smithi119 ceph-mon[34811]: mds.? [v2:172.21.15.119:6838/3693330738,v1:172.21.15.119:6839/3693330738] up:boot 2024-09-21T04:14:39.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:39 smithi119 ceph-mon[34811]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-21T04:14:39.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:39 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T04:14:39.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:39 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T04:14:39.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:39 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T04:14:39.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:39 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T04:14:39.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:39 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T04:14:39.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:39 smithi119 ceph-mon[34811]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T04:14:39.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:39 smithi119 ceph-mon[34811]: pgmap v651: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-21T04:14:39.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:39 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/501486040' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T04:14:39.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:39 smithi131 ceph-mon[35193]: mds.? [v2:172.21.15.131:6834/2310423272,v1:172.21.15.131:6835/2310423272] up:boot 2024-09-21T04:14:39.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:39 smithi131 ceph-mon[35193]: mds.? [v2:172.21.15.119:6834/1272115218,v1:172.21.15.119:6835/1272115218] up:boot 2024-09-21T04:14:39.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:39 smithi131 ceph-mon[35193]: mds.? [v2:172.21.15.131:6836/3752150166,v1:172.21.15.131:6837/3752150166] up:boot 2024-09-21T04:14:39.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:39 smithi131 ceph-mon[35193]: mds.? [v2:172.21.15.131:6838/1085660404,v1:172.21.15.131:6839/1085660404] up:boot 2024-09-21T04:14:39.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:39 smithi131 ceph-mon[35193]: mds.? [v2:172.21.15.119:6836/3417164847,v1:172.21.15.119:6837/3417164847] up:boot 2024-09-21T04:14:39.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:39 smithi131 ceph-mon[35193]: mds.? [v2:172.21.15.119:6838/3693330738,v1:172.21.15.119:6839/3693330738] up:boot 2024-09-21T04:14:39.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:39 smithi131 ceph-mon[35193]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-21T04:14:39.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:39 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T04:14:39.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:39 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T04:14:39.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:39 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T04:14:39.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:39 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T04:14:39.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:39 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T04:14:39.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:39 smithi131 ceph-mon[35193]: from='mgr.14226 172.21.15.64:0/173117387' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T04:14:39.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:39 smithi131 ceph-mon[35193]: pgmap v651: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-21T04:14:39.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:39 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/501486040' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T04:14:40.333 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T04:14:40.593 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:14:40.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:40 smithi064 ceph-mon[31180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T04:14:40.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:40 smithi064 ceph-mon[31180]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T04:14:40.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:40 smithi064 ceph-mon[31180]: Cluster is now healthy 2024-09-21T04:14:40.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:40 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/501486040' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T04:14:40.714 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:40 smithi064 ceph-mon[31180]: fsmap 12 up:standby 2024-09-21T04:14:40.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:40 smithi119 ceph-mon[34811]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T04:14:40.759 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:40 smithi119 ceph-mon[34811]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T04:14:40.759 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:40 smithi119 ceph-mon[34811]: Cluster is now healthy 2024-09-21T04:14:40.759 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:40 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/501486040' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T04:14:40.759 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:40 smithi119 ceph-mon[34811]: fsmap 12 up:standby 2024-09-21T04:14:40.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:40 smithi131 ceph-mon[35193]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T04:14:40.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:40 smithi131 ceph-mon[35193]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T04:14:40.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:40 smithi131 ceph-mon[35193]: Cluster is now healthy 2024-09-21T04:14:40.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:40 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/501486040' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T04:14:40.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:40 smithi131 ceph-mon[35193]: fsmap 12 up:standby 2024-09-21T04:14:41.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.075+0000 7fcb10879640 1 -- 172.21.15.64:0/2061402950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb0c1083c0 msgr2=0x7fcb0c1087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:41.078 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.075+0000 7fcb10879640 1 --2- 172.21.15.64:0/2061402950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb0c1083c0 0x7fcb0c1087d0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009900 tx=0x7fcb0002a410 comp rx=0 tx=0).stop 2024-09-21T04:14:41.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.075+0000 7fcb10879640 1 -- 172.21.15.64:0/2061402950 shutdown_connections 2024-09-21T04:14:41.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.075+0000 7fcb10879640 1 --2- 172.21.15.64:0/2061402950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb0c108d10 0x7fcb0c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:41.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.075+0000 7fcb10879640 1 --2- 172.21.15.64:0/2061402950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb0c1083c0 0x7fcb0c1087d0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:41.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.075+0000 7fcb10879640 1 --2- 172.21.15.64:0/2061402950 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb0c102a80 0x7fcb0c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:41.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.075+0000 7fcb10879640 1 -- 172.21.15.64:0/2061402950 >> 172.21.15.64:0/2061402950 conn(0x7fcb0c079740 msgr2=0x7fcb0c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:41.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.075+0000 7fcb10879640 1 -- 172.21.15.64:0/2061402950 shutdown_connections 2024-09-21T04:14:41.079 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.076+0000 7fcb10879640 1 -- 172.21.15.64:0/2061402950 wait complete. 2024-09-21T04:14:41.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.076+0000 7fcb10879640 1 Processor -- start 2024-09-21T04:14:41.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.076+0000 7fcb10879640 1 -- start start 2024-09-21T04:14:41.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.077+0000 7fcb10879640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb0c102a80 0x7fcb0c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:41.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.077+0000 7fcb10879640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb0c1083c0 0x7fcb0c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:41.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.077+0000 7fcb10879640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb0c108d10 0x7fcb0c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:41.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.077+0000 7fcb10879640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c112980 con 0x7fcb0c102a80 2024-09-21T04:14:41.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.077+0000 7fcb10879640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c112800 con 0x7fcb0c108d10 2024-09-21T04:14:41.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.077+0000 7fcb10879640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c112b00 con 0x7fcb0c1083c0 2024-09-21T04:14:41.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.077+0000 7fcb0a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb0c102a80 0x7fcb0c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:41.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.077+0000 7fcb0a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb0c102a80 0x7fcb0c19a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44014/0 (socket says 172.21.15.64:44014) 2024-09-21T04:14:41.081 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.077+0000 7fcb0a59c640 1 -- 172.21.15.64:0/4073526148 learned_addr learned my addr 172.21.15.64:0/4073526148 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:14:41.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.077+0000 7fcb0ad9d640 1 --2- 172.21.15.64:0/4073526148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb0c108d10 0x7fcb0c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:41.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.077+0000 7fcb09d9b640 1 --2- 172.21.15.64:0/4073526148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb0c1083c0 0x7fcb0c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:41.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.077+0000 7fcb0a59c640 1 -- 172.21.15.64:0/4073526148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb0c1083c0 msgr2=0x7fcb0c19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:41.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.077+0000 7fcb0a59c640 1 --2- 172.21.15.64:0/4073526148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb0c1083c0 0x7fcb0c19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:41.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.077+0000 7fcb0a59c640 1 -- 172.21.15.64:0/4073526148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb0c108d10 msgr2=0x7fcb0c19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:41.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.077+0000 7fcb0a59c640 1 --2- 172.21.15.64:0/4073526148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb0c108d10 0x7fcb0c19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:41.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.077+0000 7fcb0a59c640 1 -- 172.21.15.64:0/4073526148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0c1a0470 con 0x7fcb0c102a80 2024-09-21T04:14:41.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.078+0000 7fcb0a59c640 1 --2- 172.21.15.64:0/4073526148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb0c102a80 0x7fcb0c19a500 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fcafc00d770 tx=0x7fcafc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:41.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.078+0000 7fcafb7fe640 1 -- 172.21.15.64:0/4073526148 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc017070 con 0x7fcb0c102a80 2024-09-21T04:14:41.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.078+0000 7fcafb7fe640 1 -- 172.21.15.64:0/4073526148 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcafc0054d0 con 0x7fcb0c102a80 2024-09-21T04:14:41.082 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.078+0000 7fcb10879640 1 -- 172.21.15.64:0/4073526148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb0c1a0760 con 0x7fcb0c102a80 2024-09-21T04:14:41.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.078+0000 7fcafb7fe640 1 -- 172.21.15.64:0/4073526148 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc004c30 con 0x7fcb0c102a80 2024-09-21T04:14:41.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.078+0000 7fcb10879640 1 -- 172.21.15.64:0/4073526148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb0c1a6ec0 con 0x7fcb0c102a80 2024-09-21T04:14:41.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.079+0000 7fcb10879640 1 -- 172.21.15.64:0/4073526148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb0c103d00 con 0x7fcb0c102a80 2024-09-21T04:14:41.083 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.081+0000 7fcafb7fe640 1 -- 172.21.15.64:0/4073526148 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7fcafc004dd0 con 0x7fcb0c102a80 2024-09-21T04:14:41.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.082+0000 7fcafb7fe640 1 --2- 172.21.15.64:0/4073526148 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcaf40b6680 0x7fcaf40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:41.084 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.082+0000 7fcb09d9b640 1 --2- 172.21.15.64:0/4073526148 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcaf40b6680 0x7fcaf40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:41.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.082+0000 7fcafb7fe640 1 -- 172.21.15.64:0/4073526148 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8829+0+0 (secure 0 0 0) 0x7fcafc0e76f0 con 0x7fcb0c102a80 2024-09-21T04:14:41.085 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.082+0000 7fcb09d9b640 1 --2- 172.21.15.64:0/4073526148 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcaf40b6680 0x7fcaf40b8b90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcb00000c00 tx=0x7fcb00031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:41.088 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.085+0000 7fcafb7fe640 1 -- 172.21.15.64:0/4073526148 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcafc0adb10 con 0x7fcb0c102a80 2024-09-21T04:14:41.277 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.274+0000 7fcb10879640 1 -- 172.21.15.64:0/4073526148 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcb0c10bb70 con 0x7fcb0c102a80 2024-09-21T04:14:41.490 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.486+0000 7fcafb7fe640 1 -- 172.21.15.64:0/4073526148 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v89) ==== 181+0+0 (secure 0 0 0) 0x7fcafc0b1ef0 con 0x7fcb0c102a80 2024-09-21T04:14:41.490 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_metadata' removed 2024-09-21T04:14:41.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.490+0000 7fcb10879640 1 -- 172.21.15.64:0/4073526148 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcaf40b6680 msgr2=0x7fcaf40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:41.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.490+0000 7fcb10879640 1 --2- 172.21.15.64:0/4073526148 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcaf40b6680 0x7fcaf40b8b90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcb00000c00 tx=0x7fcb00031400 comp rx=0 tx=0).stop 2024-09-21T04:14:41.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.490+0000 7fcb10879640 1 -- 172.21.15.64:0/4073526148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb0c102a80 msgr2=0x7fcb0c19a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:41.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.490+0000 7fcb10879640 1 --2- 172.21.15.64:0/4073526148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb0c102a80 0x7fcb0c19a500 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fcafc00d770 tx=0x7fcafc00dc30 comp rx=0 tx=0).stop 2024-09-21T04:14:41.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.490+0000 7fcb10879640 1 -- 172.21.15.64:0/4073526148 shutdown_connections 2024-09-21T04:14:41.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.490+0000 7fcb10879640 1 --2- 172.21.15.64:0/4073526148 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7fcaf40b6680 0x7fcaf40b8b90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:41.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.490+0000 7fcb10879640 1 --2- 172.21.15.64:0/4073526148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb0c108d10 0x7fcb0c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:41.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.490+0000 7fcb10879640 1 --2- 172.21.15.64:0/4073526148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb0c1083c0 0x7fcb0c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:41.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.490+0000 7fcb10879640 1 --2- 172.21.15.64:0/4073526148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb0c102a80 0x7fcb0c19a500 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:41.493 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.490+0000 7fcb10879640 1 -- 172.21.15.64:0/4073526148 >> 172.21.15.64:0/4073526148 conn(0x7fcb0c079740 msgr2=0x7fcb0c06bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:41.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.491+0000 7fcb10879640 1 -- 172.21.15.64:0/4073526148 shutdown_connections 2024-09-21T04:14:41.494 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:41.491+0000 7fcb10879640 1 -- 172.21.15.64:0/4073526148 wait complete. 2024-09-21T04:14:41.535 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:41 smithi064 ceph-mon[31180]: pgmap v652: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-21T04:14:41.535 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:41 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/4073526148' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T04:14:41.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:41 smithi119 ceph-mon[34811]: pgmap v652: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-21T04:14:41.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:41 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/4073526148' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T04:14:41.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:41 smithi131 ceph-mon[35193]: pgmap v652: 97 pgs: 97 active+clean; 751 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-21T04:14:41.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:41 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/4073526148' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T04:14:42.189 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T04:14:42.449 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T04:14:42.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:42 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/4073526148' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T04:14:42.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:42 smithi119 ceph-mon[34811]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T04:14:42.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:42 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/4073526148' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T04:14:42.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:42 smithi064 ceph-mon[31180]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T04:14:42.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:42 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/4073526148' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T04:14:42.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:42 smithi131 ceph-mon[35193]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T04:14:42.948 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.938+0000 7f88bc6fb640 1 -- 172.21.15.64:0/288586317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88b4108d00 msgr2=0x7f88b410f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:42.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.938+0000 7f88bc6fb640 1 --2- 172.21.15.64:0/288586317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88b4108d00 0x7f88b410f530 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f88ac009ca0 tx=0x7f88ac02a8c0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.938+0000 7f88bc6fb640 1 -- 172.21.15.64:0/288586317 shutdown_connections 2024-09-21T04:14:42.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.938+0000 7f88bc6fb640 1 --2- 172.21.15.64:0/288586317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88b4108d00 0x7f88b410f530 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.938+0000 7f88bc6fb640 1 --2- 172.21.15.64:0/288586317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88b41083b0 0x7f88b41087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.938+0000 7f88bc6fb640 1 --2- 172.21.15.64:0/288586317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88b4102a70 0x7f88b4102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.938+0000 7f88bc6fb640 1 -- 172.21.15.64:0/288586317 >> 172.21.15.64:0/288586317 conn(0x7f88b4079730 msgr2=0x7f88b4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:42.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.938+0000 7f88bc6fb640 1 -- 172.21.15.64:0/288586317 shutdown_connections 2024-09-21T04:14:42.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.938+0000 7f88bc6fb640 1 -- 172.21.15.64:0/288586317 wait complete. 2024-09-21T04:14:42.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.939+0000 7f88bc6fb640 1 Processor -- start 2024-09-21T04:14:42.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.939+0000 7f88bc6fb640 1 -- start start 2024-09-21T04:14:42.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.939+0000 7f88bc6fb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88b4102a70 0x7f88b4199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:42.949 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.939+0000 7f88bc6fb640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88b41083b0 0x7f88b4199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:42.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.939+0000 7f88bc6fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88b4108d00 0x7f88b419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:42.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.939+0000 7f88bc6fb640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f88b4112970 con 0x7f88b41083b0 2024-09-21T04:14:42.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.939+0000 7f88bc6fb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f88b41127f0 con 0x7f88b4108d00 2024-09-21T04:14:42.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.939+0000 7f88bc6fb640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f88b4112af0 con 0x7f88b4102a70 2024-09-21T04:14:42.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.939+0000 7f88b9c96640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88b41083b0 0x7f88b4199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:42.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.939+0000 7f88ba497640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88b4102a70 0x7f88b4199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:42.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.939+0000 7f88b9c96640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88b41083b0 0x7f88b4199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:37932/0 (socket says 172.21.15.64:37932) 2024-09-21T04:14:42.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.939+0000 7f88b9c96640 1 -- 172.21.15.64:0/509845039 learned_addr learned my addr 172.21.15.64:0/509845039 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-09-21T04:14:42.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.940+0000 7f88bac98640 1 --2- 172.21.15.64:0/509845039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88b4108d00 0x7f88b419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:42.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.940+0000 7f88b9c96640 1 -- 172.21.15.64:0/509845039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88b4102a70 msgr2=0x7f88b4199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:42.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.940+0000 7f88b9c96640 1 --2- 172.21.15.64:0/509845039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88b4102a70 0x7f88b4199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.940+0000 7f88b9c96640 1 -- 172.21.15.64:0/509845039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88b4108d00 msgr2=0x7f88b419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:42.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.940+0000 7f88b9c96640 1 --2- 172.21.15.64:0/509845039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88b4108d00 0x7f88b419ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:42.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.940+0000 7f88b9c96640 1 -- 172.21.15.64:0/509845039 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b419f3f0 con 0x7f88b41083b0 2024-09-21T04:14:42.950 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.940+0000 7f88bac98640 1 --2- 172.21.15.64:0/509845039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88b4108d00 0x7f88b419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T04:14:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.940+0000 7f88b9c96640 1 --2- 172.21.15.64:0/509845039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88b41083b0 0x7f88b4199a20 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f88a400d740 tx=0x7f88a400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.940+0000 7f88ab7fe640 1 -- 172.21.15.64:0/509845039 <== mon.0 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a4017070 con 0x7f88b41083b0 2024-09-21T04:14:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.940+0000 7f88ab7fe640 1 -- 172.21.15.64:0/509845039 <== mon.0 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a400de50 con 0x7f88b41083b0 2024-09-21T04:14:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.940+0000 7f88ab7fe640 1 -- 172.21.15.64:0/509845039 <== mon.0 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a4004ce0 con 0x7f88b41083b0 2024-09-21T04:14:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.940+0000 7f88bc6fb640 1 -- 172.21.15.64:0/509845039 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b419f6e0 con 0x7f88b41083b0 2024-09-21T04:14:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.940+0000 7f88bc6fb640 1 -- 172.21.15.64:0/509845039 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b406bae0 con 0x7f88b41083b0 2024-09-21T04:14:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.941+0000 7f88bc6fb640 1 -- 172.21.15.64:0/509845039 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88b4103cf0 con 0x7f88b41083b0 2024-09-21T04:14:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.944+0000 7f88ab7fe640 1 -- 172.21.15.64:0/509845039 <== mon.0 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 154643+0+0 (secure 0 0 0) 0x7f88a4004e80 con 0x7f88b41083b0 2024-09-21T04:14:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.945+0000 7f88ab7fe640 1 --2- 172.21.15.64:0/509845039 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f88a00b6680 0x7f88a00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T04:14:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.945+0000 7f88ba497640 1 --2- 172.21.15.64:0/509845039 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f88a00b6680 0x7f88a00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T04:14:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.945+0000 7f88ab7fe640 1 -- 172.21.15.64:0/509845039 <== mon.0 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8159+0+0 (secure 0 0 0) 0x7f88a40e7300 con 0x7f88b41083b0 2024-09-21T04:14:42.951 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.946+0000 7f88ba497640 1 --2- 172.21.15.64:0/509845039 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f88a00b6680 0x7f88a00b8b90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f88b0005d00 tx=0x7f88b000b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T04:14:42.952 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:42.948+0000 7f88ab7fe640 1 -- 172.21.15.64:0/509845039 <== mon.0 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f88a40ada10 con 0x7f88b41083b0 2024-09-21T04:14:43.139 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:43.136+0000 7f88bc6fb640 1 -- 172.21.15.64:0/509845039 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f88b4102e80 con 0x7f88b41083b0 2024-09-21T04:14:43.496 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:43.494+0000 7f88ab7fe640 1 -- 172.21.15.64:0/509845039 <== mon.0 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v90) ==== 169+0+0 (secure 0 0 0) 0x7f88a40b1df0 con 0x7f88b41083b0 2024-09-21T04:14:43.497 INFO:teuthology.orchestra.run.smithi064.stderr:pool 'cephfs_data' removed 2024-09-21T04:14:43.500 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:43.498+0000 7f88bc6fb640 1 -- 172.21.15.64:0/509845039 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f88a00b6680 msgr2=0x7f88a00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:43.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:43.498+0000 7f88bc6fb640 1 --2- 172.21.15.64:0/509845039 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f88a00b6680 0x7f88a00b8b90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f88b0005d00 tx=0x7f88b000b410 comp rx=0 tx=0).stop 2024-09-21T04:14:43.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:43.498+0000 7f88bc6fb640 1 -- 172.21.15.64:0/509845039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88b41083b0 msgr2=0x7f88b4199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T04:14:43.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:43.498+0000 7f88bc6fb640 1 --2- 172.21.15.64:0/509845039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88b41083b0 0x7f88b4199a20 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f88a400d740 tx=0x7f88a400dc00 comp rx=0 tx=0).stop 2024-09-21T04:14:43.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:43.498+0000 7f88bc6fb640 1 -- 172.21.15.64:0/509845039 shutdown_connections 2024-09-21T04:14:43.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:43.498+0000 7f88bc6fb640 1 --2- 172.21.15.64:0/509845039 >> [v2:172.21.15.64:6800/834288654,v1:172.21.15.64:6801/834288654] conn(0x7f88a00b6680 0x7f88a00b8b90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:43.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:43.498+0000 7f88bc6fb640 1 --2- 172.21.15.64:0/509845039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88b4108d00 0x7f88b419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:43.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:43.498+0000 7f88bc6fb640 1 --2- 172.21.15.64:0/509845039 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f88b41083b0 0x7f88b4199a20 secure :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f88a400d740 tx=0x7f88a400dc00 comp rx=0 tx=0).stop 2024-09-21T04:14:43.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:43.498+0000 7f88bc6fb640 1 --2- 172.21.15.64:0/509845039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88b4102a70 0x7f88b4199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T04:14:43.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:43.498+0000 7f88bc6fb640 1 -- 172.21.15.64:0/509845039 >> 172.21.15.64:0/509845039 conn(0x7f88b4079730 msgr2=0x7f88b4104520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T04:14:43.501 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:43.498+0000 7f88bc6fb640 1 -- 172.21.15.64:0/509845039 shutdown_connections 2024-09-21T04:14:43.502 INFO:teuthology.orchestra.run.smithi064.stderr:2024-09-21T04:14:43.498+0000 7f88bc6fb640 1 -- 172.21.15.64:0/509845039 wait complete. 2024-09-21T04:14:43.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:43 smithi119 ceph-mon[34811]: pgmap v654: 65 pgs: 65 active+clean; 426 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:14:43.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:43 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/509845039' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T04:14:43.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:43 smithi064 ceph-mon[31180]: pgmap v654: 65 pgs: 65 active+clean; 426 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:14:43.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:43 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/509845039' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T04:14:43.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:43 smithi131 ceph-mon[35193]: pgmap v654: 65 pgs: 65 active+clean; 426 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T04:14:43.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:43 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/509845039' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T04:14:44.374 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T04:14:44.401 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T04:14:44.442 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T04:14:44.476 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-21T04:14:44.476 DEBUG:teuthology.orchestra.run.smithi064:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T04:14:44.492 DEBUG:teuthology.orchestra.run.smithi119:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T04:14:44.513 DEBUG:teuthology.orchestra.run.smithi131:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T04:14:44.532 INFO:tasks.cephadm:Stopping all daemons... 2024-09-21T04:14:44.532 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-21T04:14:44.532 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mon.a 2024-09-21T04:14:44.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:44 smithi119 ceph-mon[34811]: from='client.? 172.21.15.64:0/509845039' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T04:14:44.757 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:44 smithi119 ceph-mon[34811]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T04:14:44.761 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:44 smithi064 ceph-mon[31180]: from='client.? 172.21.15.64:0/509845039' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T04:14:44.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:44 smithi064 ceph-mon[31180]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T04:14:44.762 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:44 smithi064 systemd[1]: Stopping Ceph mon.a for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:14:44.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:44 smithi131 ceph-mon[35193]: from='client.? 172.21.15.64:0/509845039' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T04:14:44.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:44 smithi131 ceph-mon[35193]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T04:14:45.142 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:44 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-a[31176]: 2024-09-21T04:14:44.838+0000 7f00eaaa9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:14:45.142 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:44 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-a[31176]: 2024-09-21T04:14:44.838+0000 7f00eaaa9640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-21T04:14:45.142 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:44 smithi064 podman[103119]: 2024-09-21 04:14:44.920999081 +0000 UTC m=+0.202928528 container died 4084045b6479125719712086b6b208240aed1f65011099f1ae40e1612fe2e7ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-21T04:14:45.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:45 smithi064 podman[103119]: 2024-09-21 04:14:45.348770064 +0000 UTC m=+0.630699573 container cleanup 4084045b6479125719712086b6b208240aed1f65011099f1ae40e1612fe2e7ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T04:14:45.511 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:45 smithi064 bash[103119]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-a 2024-09-21T04:14:45.832 INFO:journalctl@ceph.mon.a.smithi064.stdout:Sep 21 04:14:45 smithi064 podman[103133]: 2024-09-21 04:14:45.567173336 +0000 UTC m=+0.643440668 container remove 4084045b6479125719712086b6b208240aed1f65011099f1ae40e1612fe2e7ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:14:45.836 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mon.a.service' 2024-09-21T04:14:45.902 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:14:45.902 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-21T04:14:45.903 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-21T04:14:45.903 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mon.b 2024-09-21T04:14:46.235 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:45 smithi119 systemd[1]: Stopping Ceph mon.b for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:14:46.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:46 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-b[34807]: 2024-09-21T04:14:46.232+0000 7f0ffdd9d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:14:46.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:46 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-b[34807]: 2024-09-21T04:14:46.232+0000 7f0ffdd9d640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-21T04:14:46.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:46 smithi119 podman[67602]: 2024-09-21 04:14:46.284160093 +0000 UTC m=+0.168398832 container died 4dfbc9648e428f4c05d41cda85e0bbc8b3541ba5003ad1a4923d619e1efc40fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T04:14:46.929 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:46 smithi119 podman[67602]: 2024-09-21 04:14:46.700307465 +0000 UTC m=+0.584546206 container cleanup 4dfbc9648e428f4c05d41cda85e0bbc8b3541ba5003ad1a4923d619e1efc40fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-21T04:14:46.929 INFO:journalctl@ceph.mon.b.smithi119.stdout:Sep 21 04:14:46 smithi119 bash[67602]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-b 2024-09-21T04:14:47.149 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mon.b.service' 2024-09-21T04:14:47.198 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:14:47.198 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-21T04:14:47.198 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-21T04:14:47.198 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mon.c 2024-09-21T04:14:47.553 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:47 smithi131 systemd[1]: Stopping Ceph mon.c for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:14:47.985 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:47 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-c[35189]: 2024-09-21T04:14:47.565+0000 7f10642dd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:14:47.986 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:47 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-c[35189]: 2024-09-21T04:14:47.565+0000 7f10642dd640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-21T04:14:47.986 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:47 smithi131 podman[68461]: 2024-09-21 04:14:47.698754571 +0000 UTC m=+0.303405164 container died 0697ec341458496080f3751f3b8ea733520057582b88af2a59b8919896d82e7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-c, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:14:48.278 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:48 smithi131 podman[68461]: 2024-09-21 04:14:48.074421436 +0000 UTC m=+0.679071984 container cleanup 0697ec341458496080f3751f3b8ea733520057582b88af2a59b8919896d82e7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-c, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:14:48.278 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 21 04:14:48 smithi131 bash[68461]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mon-c 2024-09-21T04:14:48.487 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mon.c.service' 2024-09-21T04:14:48.542 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:14:48.542 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-21T04:14:48.542 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-21T04:14:48.542 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mgr.x 2024-09-21T04:14:48.856 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 04:14:48 smithi064 systemd[1]: Stopping Ceph mgr.x for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:14:49.195 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 04:14:48 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x[31409]: 2024-09-21T04:14:48.853+0000 7fa7abec1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:14:49.195 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 04:14:48 smithi064 podman[103245]: 2024-09-21 04:14:48.89469051 +0000 UTC m=+0.160034946 container died 9323be1996ab4e20fa73cbd46867533562ca93c9f61809c2317d4f718f53c425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-21T04:14:49.455 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 04:14:49 smithi064 podman[103245]: 2024-09-21 04:14:49.437602376 +0000 UTC m=+0.702946811 container cleanup 9323be1996ab4e20fa73cbd46867533562ca93c9f61809c2317d4f718f53c425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T04:14:49.455 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 04:14:49 smithi064 bash[103245]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x 2024-09-21T04:14:49.876 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mgr.x.service' 2024-09-21T04:14:49.900 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 04:14:49 smithi064 podman[103257]: 2024-09-21 04:14:49.631351626 +0000 UTC m=+0.733635086 container remove 9323be1996ab4e20fa73cbd46867533562ca93c9f61809c2317d4f718f53c425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-x, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:14:49.900 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 04:14:49 smithi064 systemd[1]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mgr.x.service: Deactivated successfully. 2024-09-21T04:14:49.900 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 04:14:49 smithi064 systemd[1]: Stopped Ceph mgr.x for e7c927ba-77cc-11ef-baf5-efdc52797490. 2024-09-21T04:14:49.900 INFO:journalctl@ceph.mgr.x.smithi064.stdout:Sep 21 04:14:49 smithi064 systemd[1]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mgr.x.service: Consumed 54.955s CPU time. 2024-09-21T04:14:49.914 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:14:49.914 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-21T04:14:49.915 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-21T04:14:49.915 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mgr.y 2024-09-21T04:14:50.257 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Sep 21 04:14:49 smithi119 systemd[1]: Stopping Ceph mgr.y for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:14:50.654 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Sep 21 04:14:50 smithi119 podman[67723]: 2024-09-21 04:14:50.344633397 +0000 UTC m=+0.232171363 container died 5018b50d1d117dd77e0332d1ab63893d881ae01b81b1aed69095e333b6154a07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-y, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:14:51.252 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Sep 21 04:14:51 smithi119 podman[67723]: 2024-09-21 04:14:51.015037319 +0000 UTC m=+0.902575290 container cleanup 5018b50d1d117dd77e0332d1ab63893d881ae01b81b1aed69095e333b6154a07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-y, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T04:14:51.252 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Sep 21 04:14:51 smithi119 bash[67723]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-y 2024-09-21T04:14:51.424 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mgr.y.service' 2024-09-21T04:14:51.462 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:14:51.463 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-21T04:14:51.463 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-21T04:14:51.463 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mgr.z 2024-09-21T04:14:51.803 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 21 04:14:51 smithi131 systemd[1]: Stopping Ceph mgr.z for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:14:52.074 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 21 04:14:51 smithi131 podman[68582]: 2024-09-21 04:14:51.819093031 +0000 UTC m=+0.160531846 container died 64b0f071a2fddc724580c360f464bab296a8f9f8211eea1dff92e7efdc91bbd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-21T04:14:52.554 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 21 04:14:52 smithi131 podman[68582]: 2024-09-21 04:14:52.230571947 +0000 UTC m=+0.572010772 container cleanup 64b0f071a2fddc724580c360f464bab296a8f9f8211eea1dff92e7efdc91bbd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-21T04:14:52.554 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 21 04:14:52 smithi131 bash[68582]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-z 2024-09-21T04:14:53.054 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 21 04:14:52 smithi131 podman[68595]: 2024-09-21 04:14:52.799148992 +0000 UTC m=+0.983309670 container remove 64b0f071a2fddc724580c360f464bab296a8f9f8211eea1dff92e7efdc91bbd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mgr-z, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:14:53.054 INFO:journalctl@ceph.mgr.z.smithi131.stdout:Sep 21 04:14:52 smithi131 systemd[1]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-21T04:14:53.118 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mgr.z.service' 2024-09-21T04:14:53.160 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:14:53.161 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-21T04:14:53.161 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-21T04:14:53.162 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.0 2024-09-21T04:14:53.479 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 21 04:14:53 smithi064 systemd[1]: Stopping Ceph osd.0 for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:14:53.762 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 21 04:14:53 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-0[45513]: 2024-09-21T04:14:53.475+0000 7fb1fce84640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:14:53.762 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 21 04:14:53 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-0[45513]: 2024-09-21T04:14:53.475+0000 7fb1fce84640 -1 osd.0 90 *** Got signal Terminated *** 2024-09-21T04:14:53.762 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 21 04:14:53 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-0[45513]: 2024-09-21T04:14:53.475+0000 7fb1fce84640 -1 osd.0 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:14:59.204 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 21 04:14:58 smithi064 podman[103365]: 2024-09-21 04:14:58.893809342 +0000 UTC m=+5.535123844 container died c3ecbdd2cce320c8f7e1d870d6dba392165fb315d8e8782f536fafbee9e2f865 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T04:14:59.511 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 21 04:14:59 smithi064 podman[103365]: 2024-09-21 04:14:59.293210392 +0000 UTC m=+5.934524884 container cleanup c3ecbdd2cce320c8f7e1d870d6dba392165fb315d8e8782f536fafbee9e2f865 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-21T04:14:59.512 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 21 04:14:59 smithi064 bash[103365]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-0 2024-09-21T04:14:59.871 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 21 04:14:59 smithi064 podman[103380]: 2024-09-21 04:14:59.528120192 +0000 UTC m=+0.637511493 container remove c3ecbdd2cce320c8f7e1d870d6dba392165fb315d8e8782f536fafbee9e2f865 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T04:15:00.142 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 21 04:14:59 smithi064 podman[103471]: 2024-09-21 04:14:59.772578456 +0000 UTC m=+0.016542471 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:15:00.142 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 21 04:14:59 smithi064 podman[103471]: 2024-09-21 04:14:59.887736604 +0000 UTC m=+0.131700602 container create 208d6c274b32c0bb94b5d3bb655d99763963dc28e09f5b512b68f7dae312aa44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-0-deactivate, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:15:00.512 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 21 04:15:00 smithi064 podman[103471]: 2024-09-21 04:15:00.140528567 +0000 UTC m=+0.384492568 container init 208d6c274b32c0bb94b5d3bb655d99763963dc28e09f5b512b68f7dae312aa44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-21T04:15:00.512 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 21 04:15:00 smithi064 podman[103471]: 2024-09-21 04:15:00.144190993 +0000 UTC m=+0.388155001 container start 208d6c274b32c0bb94b5d3bb655d99763963dc28e09f5b512b68f7dae312aa44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-0-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-21T04:15:00.512 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 21 04:15:00 smithi064 podman[103471]: 2024-09-21 04:15:00.224048362 +0000 UTC m=+0.468012373 container attach 208d6c274b32c0bb94b5d3bb655d99763963dc28e09f5b512b68f7dae312aa44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True) 2024-09-21T04:15:00.512 INFO:journalctl@ceph.osd.0.smithi064.stdout:Sep 21 04:15:00 smithi064 podman[103471]: 2024-09-21 04:15:00.337740011 +0000 UTC m=+0.581704019 container died 208d6c274b32c0bb94b5d3bb655d99763963dc28e09f5b512b68f7dae312aa44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:15:01.014 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.0.service' 2024-09-21T04:15:01.056 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:15:01.056 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-21T04:15:01.056 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-21T04:15:01.056 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.1 2024-09-21T04:15:01.257 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 04:15:01 smithi119 systemd[1]: Stopping Ceph osd.1 for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:15:01.757 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 04:15:01 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-1[39452]: 2024-09-21T04:15:01.360+0000 7f0755ad8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:15:01.757 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 04:15:01 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-1[39452]: 2024-09-21T04:15:01.360+0000 7f0755ad8640 -1 osd.1 90 *** Got signal Terminated *** 2024-09-21T04:15:01.757 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 04:15:01 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-1[39452]: 2024-09-21T04:15:01.360+0000 7f0755ad8640 -1 osd.1 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:15:07.214 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 04:15:06 smithi119 podman[67862]: 2024-09-21 04:15:06.906168256 +0000 UTC m=+5.656937069 container died 61e0f8f1664daca47a0ff5d60edf1c4de1c7f82a03973116d186792defa6bb13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-1, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux ) 2024-09-21T04:15:07.507 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 04:15:07 smithi119 podman[67862]: 2024-09-21 04:15:07.311670553 +0000 UTC m=+6.062439366 container cleanup 61e0f8f1664daca47a0ff5d60edf1c4de1c7f82a03973116d186792defa6bb13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-21T04:15:07.507 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 04:15:07 smithi119 bash[67862]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-1 2024-09-21T04:15:07.919 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 04:15:07 smithi119 podman[67881]: 2024-09-21 04:15:07.571663055 +0000 UTC m=+0.660236490 container remove 61e0f8f1664daca47a0ff5d60edf1c4de1c7f82a03973116d186792defa6bb13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-1, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-09-21T04:15:08.194 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 04:15:07 smithi119 podman[67968]: 2024-09-21 04:15:07.820119513 +0000 UTC m=+0.015920124 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:15:08.194 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 04:15:07 smithi119 podman[67968]: 2024-09-21 04:15:07.939306593 +0000 UTC m=+0.135107192 container create db8eac18250e830661d7a35199e9170beb9878a5780d441e7eb77f3e300b09ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-21T04:15:08.507 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 04:15:08 smithi119 podman[67968]: 2024-09-21 04:15:08.192419832 +0000 UTC m=+0.388220428 container init db8eac18250e830661d7a35199e9170beb9878a5780d441e7eb77f3e300b09ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-1-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-21T04:15:08.507 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 04:15:08 smithi119 podman[67968]: 2024-09-21 04:15:08.195886298 +0000 UTC m=+0.391686904 container start db8eac18250e830661d7a35199e9170beb9878a5780d441e7eb77f3e300b09ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-21T04:15:08.507 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 04:15:08 smithi119 podman[67968]: 2024-09-21 04:15:08.292666573 +0000 UTC m=+0.488467188 container attach db8eac18250e830661d7a35199e9170beb9878a5780d441e7eb77f3e300b09ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:15:08.507 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 04:15:08 smithi119 podman[67968]: 2024-09-21 04:15:08.425039804 +0000 UTC m=+0.620840413 container died db8eac18250e830661d7a35199e9170beb9878a5780d441e7eb77f3e300b09ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=) 2024-09-21T04:15:09.073 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.1.service' 2024-09-21T04:15:09.099 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 04:15:09 smithi119 podman[67992]: 2024-09-21 04:15:09.055303323 +0000 UTC m=+0.620674627 container remove db8eac18250e830661d7a35199e9170beb9878a5780d441e7eb77f3e300b09ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T04:15:09.100 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 04:15:09 smithi119 systemd[1]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.1.service: Deactivated successfully. 2024-09-21T04:15:09.100 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 04:15:09 smithi119 systemd[1]: Stopped Ceph osd.1 for e7c927ba-77cc-11ef-baf5-efdc52797490. 2024-09-21T04:15:09.100 INFO:journalctl@ceph.osd.1.smithi119.stdout:Sep 21 04:15:09 smithi119 systemd[1]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.1.service: Consumed 1min 27.117s CPU time. 2024-09-21T04:15:09.115 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:15:09.115 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-21T04:15:09.116 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-21T04:15:09.116 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.2 2024-09-21T04:15:09.521 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 21 04:15:09 smithi131 systemd[1]: Stopping Ceph osd.2 for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:15:09.804 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 21 04:15:09 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-2[40420]: 2024-09-21T04:15:09.518+0000 7f087f105640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:15:09.804 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 21 04:15:09 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-2[40420]: 2024-09-21T04:15:09.518+0000 7f087f105640 -1 osd.2 90 *** Got signal Terminated *** 2024-09-21T04:15:09.804 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 21 04:15:09 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-2[40420]: 2024-09-21T04:15:09.518+0000 7f087f105640 -1 osd.2 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:15:15.304 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 21 04:15:15 smithi131 podman[68721]: 2024-09-21 04:15:15.030766792 +0000 UTC m=+5.709145126 container died 9b47f4af97796541881d3064c46455154354f6452d09a7b1276ee9e0098e95f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T04:15:15.705 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 21 04:15:15 smithi131 podman[68721]: 2024-09-21 04:15:15.435014202 +0000 UTC m=+6.113392530 container cleanup 9b47f4af97796541881d3064c46455154354f6452d09a7b1276ee9e0098e95f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-21T04:15:15.705 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 21 04:15:15 smithi131 bash[68721]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-2 2024-09-21T04:15:16.014 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 21 04:15:15 smithi131 podman[68735]: 2024-09-21 04:15:15.703090695 +0000 UTC m=+0.669582940 container remove 9b47f4af97796541881d3064c46455154354f6452d09a7b1276ee9e0098e95f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-21T04:15:16.294 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 21 04:15:16 smithi131 podman[68826]: 2024-09-21 04:15:15.914801825 +0000 UTC m=+0.015814221 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:15:16.294 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 21 04:15:16 smithi131 podman[68826]: 2024-09-21 04:15:16.029116277 +0000 UTC m=+0.130128661 container create fe9944c3eddccf7efe2cdb8de987b57350a90721530ec4cda68a44c1d7381ac3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T04:15:16.554 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 21 04:15:16 smithi131 podman[68826]: 2024-09-21 04:15:16.292515393 +0000 UTC m=+0.393527788 container init fe9944c3eddccf7efe2cdb8de987b57350a90721530ec4cda68a44c1d7381ac3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T04:15:16.554 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 21 04:15:16 smithi131 podman[68826]: 2024-09-21 04:15:16.296124143 +0000 UTC m=+0.397136529 container start fe9944c3eddccf7efe2cdb8de987b57350a90721530ec4cda68a44c1d7381ac3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-21T04:15:16.554 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 21 04:15:16 smithi131 podman[68826]: 2024-09-21 04:15:16.392668679 +0000 UTC m=+0.493681068 container attach fe9944c3eddccf7efe2cdb8de987b57350a90721530ec4cda68a44c1d7381ac3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T04:15:16.554 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 21 04:15:16 smithi131 podman[68826]: 2024-09-21 04:15:16.491066619 +0000 UTC m=+0.592079005 container died fe9944c3eddccf7efe2cdb8de987b57350a90721530ec4cda68a44c1d7381ac3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-21T04:15:17.163 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.2.service' 2024-09-21T04:15:17.203 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:15:17.203 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-21T04:15:17.203 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-21T04:15:17.203 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.3 2024-09-21T04:15:17.511 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 21 04:15:17 smithi064 systemd[1]: Stopping Ceph osd.3 for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:15:18.012 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 21 04:15:17 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-3[51799]: 2024-09-21T04:15:17.514+0000 7fd317bd7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:15:18.012 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 21 04:15:17 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-3[51799]: 2024-09-21T04:15:17.514+0000 7fd317bd7640 -1 osd.3 90 *** Got signal Terminated *** 2024-09-21T04:15:18.012 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 21 04:15:17 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-3[51799]: 2024-09-21T04:15:17.514+0000 7fd317bd7640 -1 osd.3 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:15:20.304 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:19 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:19.847+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.245603+0000 front 2024-09-21T04:14:55.245502+0000 (oldest deadline 2024-09-21T04:15:19.345596+0000) 2024-09-21T04:15:21.205 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:20 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:20.813+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.245603+0000 front 2024-09-21T04:14:55.245502+0000 (oldest deadline 2024-09-21T04:15:19.345596+0000) 2024-09-21T04:15:21.262 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 21 04:15:20 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-3[51799]: 2024-09-21T04:15:20.874+0000 7fd3141c2640 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.989595+0000 front 2024-09-21T04:14:55.989635+0000 (oldest deadline 2024-09-21T04:15:20.089800+0000) 2024-09-21T04:15:21.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:21 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:21.480+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:22.053 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:21 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:21.784+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.245603+0000 front 2024-09-21T04:14:55.245502+0000 (oldest deadline 2024-09-21T04:15:19.345596+0000) 2024-09-21T04:15:22.257 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 04:15:21 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-4[46031]: 2024-09-21T04:15:21.976+0000 7fad8899a640 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.861016+0000 front 2024-09-21T04:14:55.861112+0000 (oldest deadline 2024-09-21T04:15:21.160967+0000) 2024-09-21T04:15:22.262 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 21 04:15:21 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-3[51799]: 2024-09-21T04:15:21.837+0000 7fd3141c2640 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.989595+0000 front 2024-09-21T04:14:55.989635+0000 (oldest deadline 2024-09-21T04:15:20.089800+0000) 2024-09-21T04:15:22.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:21 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:21.979+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:22.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:22 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:22.434+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:23.054 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:22 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:22.771+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.245603+0000 front 2024-09-21T04:14:55.245502+0000 (oldest deadline 2024-09-21T04:15:19.345596+0000) 2024-09-21T04:15:23.054 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:22 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:22.947+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:23.256 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 04:15:22 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-4[46031]: 2024-09-21T04:15:22.989+0000 7fad8899a640 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.861016+0000 front 2024-09-21T04:14:55.861112+0000 (oldest deadline 2024-09-21T04:15:21.160967+0000) 2024-09-21T04:15:23.261 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:22 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:22.952+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:23.262 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 21 04:15:23 smithi064 podman[103589]: 2024-09-21 04:15:23.042186148 +0000 UTC m=+5.641584121 container died f3e18a7ca9f52a16756bddbab46974edef280e11b4ce134fec29b54c30c776da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-21T04:15:23.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:23 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:23.411+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:23.762 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 21 04:15:23 smithi064 podman[103589]: 2024-09-21 04:15:23.515111843 +0000 UTC m=+6.114509815 container cleanup f3e18a7ca9f52a16756bddbab46974edef280e11b4ce134fec29b54c30c776da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-3, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-21T04:15:23.762 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 21 04:15:23 smithi064 bash[103589]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-3 2024-09-21T04:15:24.118 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 21 04:15:23 smithi064 podman[103606]: 2024-09-21 04:15:23.797800588 +0000 UTC m=+0.752621272 container remove f3e18a7ca9f52a16756bddbab46974edef280e11b4ce134fec29b54c30c776da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:15:24.118 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:23 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:23.912+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:24.118 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:24 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:24.012+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:58.181035+0000 front 2024-09-21T04:14:58.181137+0000 (oldest deadline 2024-09-21T04:15:23.481212+0000) 2024-09-21T04:15:24.257 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 04:15:23 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-4[46031]: 2024-09-21T04:15:23.984+0000 7fad8899a640 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.861016+0000 front 2024-09-21T04:14:55.861112+0000 (oldest deadline 2024-09-21T04:15:21.160967+0000) 2024-09-21T04:15:24.303 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:23 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:23.803+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.245603+0000 front 2024-09-21T04:14:55.245502+0000 (oldest deadline 2024-09-21T04:15:19.345596+0000) 2024-09-21T04:15:24.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:23 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:23.925+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:24.381 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 21 04:15:24 smithi064 podman[103695]: 2024-09-21 04:15:24.016840308 +0000 UTC m=+0.016689491 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:15:24.381 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 21 04:15:24 smithi064 podman[103695]: 2024-09-21 04:15:24.134442196 +0000 UTC m=+0.134291405 container create d1616ba872c5cfbb7dc342caf01b94511fcfb01ef143a08d90b9c042e3e79267 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-09-21T04:15:24.381 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 21 04:15:24 smithi064 podman[103695]: 2024-09-21 04:15:24.378983784 +0000 UTC m=+0.378832967 container init d1616ba872c5cfbb7dc342caf01b94511fcfb01ef143a08d90b9c042e3e79267 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True) 2024-09-21T04:15:24.756 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:24 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:24.430+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:24.763 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 21 04:15:24 smithi064 podman[103695]: 2024-09-21 04:15:24.382412315 +0000 UTC m=+0.382261487 container start d1616ba872c5cfbb7dc342caf01b94511fcfb01ef143a08d90b9c042e3e79267 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-21T04:15:24.763 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 21 04:15:24 smithi064 podman[103695]: 2024-09-21 04:15:24.470722913 +0000 UTC m=+0.470572085 container attach d1616ba872c5cfbb7dc342caf01b94511fcfb01ef143a08d90b9c042e3e79267 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-21T04:15:24.763 INFO:journalctl@ceph.osd.3.smithi064.stdout:Sep 21 04:15:24 smithi064 podman[103695]: 2024-09-21 04:15:24.573340956 +0000 UTC m=+0.573190135 container died d1616ba872c5cfbb7dc342caf01b94511fcfb01ef143a08d90b9c042e3e79267 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T04:15:25.053 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:24 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:24.787+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.245603+0000 front 2024-09-21T04:14:55.245502+0000 (oldest deadline 2024-09-21T04:15:19.345596+0000) 2024-09-21T04:15:25.054 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:24 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:24.962+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:25.159 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:25 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:25.060+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:58.181035+0000 front 2024-09-21T04:14:58.181137+0000 (oldest deadline 2024-09-21T04:15:23.481212+0000) 2024-09-21T04:15:25.160 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:24 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:24.915+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:25.175 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.3.service' 2024-09-21T04:15:25.217 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:15:25.217 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-21T04:15:25.217 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-21T04:15:25.217 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.4 2024-09-21T04:15:25.248 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 04:15:24 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-4[46031]: 2024-09-21T04:15:24.993+0000 7fad8899a640 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.861016+0000 front 2024-09-21T04:14:55.861112+0000 (oldest deadline 2024-09-21T04:15:21.160967+0000) 2024-09-21T04:15:25.507 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 04:15:25 smithi119 systemd[1]: Stopping Ceph osd.4 for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:15:25.507 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:25 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:25.435+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:26.006 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 04:15:25 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-4[46031]: 2024-09-21T04:15:25.535+0000 7fad8c3af640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:15:26.007 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 04:15:25 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-4[46031]: 2024-09-21T04:15:25.535+0000 7fad8c3af640 -1 osd.4 90 *** Got signal Terminated *** 2024-09-21T04:15:26.007 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 04:15:25 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-4[46031]: 2024-09-21T04:15:25.535+0000 7fad8c3af640 -1 osd.4 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:15:26.053 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:25 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:25.761+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.245603+0000 front 2024-09-21T04:14:55.245502+0000 (oldest deadline 2024-09-21T04:15:19.345596+0000) 2024-09-21T04:15:26.054 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:25 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:25.988+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:26.261 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:26 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:26.086+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:58.181035+0000 front 2024-09-21T04:14:58.181137+0000 (oldest deadline 2024-09-21T04:15:23.481212+0000) 2024-09-21T04:15:26.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:25 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:25.931+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:26.461 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 04:15:26 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-4[46031]: 2024-09-21T04:15:26.026+0000 7fad8899a640 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.861016+0000 front 2024-09-21T04:14:55.861112+0000 (oldest deadline 2024-09-21T04:15:21.160967+0000) 2024-09-21T04:15:26.756 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:26 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:26.459+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:27.054 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:26 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:26.763+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.245603+0000 front 2024-09-21T04:14:55.245502+0000 (oldest deadline 2024-09-21T04:15:19.345596+0000) 2024-09-21T04:15:27.054 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:26 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:26.975+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:27.261 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:27 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:27.111+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:58.181035+0000 front 2024-09-21T04:14:58.181137+0000 (oldest deadline 2024-09-21T04:15:23.481212+0000) 2024-09-21T04:15:27.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:26 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:26.938+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:27.467 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 04:15:27 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-4[46031]: 2024-09-21T04:15:27.007+0000 7fad8899a640 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.861016+0000 front 2024-09-21T04:14:55.861112+0000 (oldest deadline 2024-09-21T04:15:21.160967+0000) 2024-09-21T04:15:27.756 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:27 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:27.464+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:28.006 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:27 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:27.740+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.245603+0000 front 2024-09-21T04:14:55.245502+0000 (oldest deadline 2024-09-21T04:15:19.345596+0000) 2024-09-21T04:15:28.256 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 04:15:27 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-4[46031]: 2024-09-21T04:15:27.977+0000 7fad8899a640 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.861016+0000 front 2024-09-21T04:14:55.861112+0000 (oldest deadline 2024-09-21T04:15:21.160967+0000) 2024-09-21T04:15:28.261 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:28 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:28.089+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:58.181035+0000 front 2024-09-21T04:14:58.181137+0000 (oldest deadline 2024-09-21T04:15:23.481212+0000) 2024-09-21T04:15:28.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:27 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:27.951+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:28.303 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:28 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:28.003+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:28.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:28 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:28.003+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:28.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:28 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:28.498+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:28.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:28 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:28.498+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:29.043 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:28 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:28.706+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.245603+0000 front 2024-09-21T04:14:55.245502+0000 (oldest deadline 2024-09-21T04:15:19.345596+0000) 2024-09-21T04:15:29.256 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 04:15:28 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-4[46031]: 2024-09-21T04:15:28.990+0000 7fad8899a640 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.861016+0000 front 2024-09-21T04:14:55.861112+0000 (oldest deadline 2024-09-21T04:15:21.160967+0000) 2024-09-21T04:15:29.261 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:29 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:29.108+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:58.181035+0000 front 2024-09-21T04:14:58.181137+0000 (oldest deadline 2024-09-21T04:15:23.481212+0000) 2024-09-21T04:15:29.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:28 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:28.971+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:29.303 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:29 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:29.040+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:29.303 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:29 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:29.040+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:29.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:29 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:29.080+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:29.958 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:29 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:29.546+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:29.959 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:29 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:29.546+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:30.029 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:29 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:29.727+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.245603+0000 front 2024-09-21T04:14:55.245502+0000 (oldest deadline 2024-09-21T04:15:19.345596+0000) 2024-09-21T04:15:30.257 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 04:15:29 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-4[46031]: 2024-09-21T04:15:29.956+0000 7fad8899a640 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.861016+0000 front 2024-09-21T04:14:55.861112+0000 (oldest deadline 2024-09-21T04:15:21.160967+0000) 2024-09-21T04:15:30.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:30 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:30.114+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:58.181035+0000 front 2024-09-21T04:14:58.181137+0000 (oldest deadline 2024-09-21T04:15:23.481212+0000) 2024-09-21T04:15:30.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:30 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:30.114+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.182453+0000 front 2024-09-21T04:15:05.182422+0000 (oldest deadline 2024-09-21T04:15:29.282683+0000) 2024-09-21T04:15:30.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:29 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:29.964+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:30.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:29 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:29.964+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:30.303 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:30 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:30.093+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:30.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:30 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:30.026+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:30.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:30 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:30.026+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:31.007 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:30 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:30.562+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:31.007 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:30 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:30.562+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:31.042 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:30 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:30.766+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.245603+0000 front 2024-09-21T04:14:55.245502+0000 (oldest deadline 2024-09-21T04:15:19.345596+0000) 2024-09-21T04:15:31.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:31 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:31.155+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:58.181035+0000 front 2024-09-21T04:14:58.181137+0000 (oldest deadline 2024-09-21T04:15:23.481212+0000) 2024-09-21T04:15:31.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:31 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:31.155+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.182453+0000 front 2024-09-21T04:15:05.182422+0000 (oldest deadline 2024-09-21T04:15:29.282683+0000) 2024-09-21T04:15:31.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:30 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:30.950+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:31.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:30 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:30.950+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:31.303 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:31 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:31.136+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:31.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:31 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:31.039+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:31.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:31 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:31.039+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:31.378 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 04:15:31 smithi119 podman[68095]: 2024-09-21 04:15:31.037061606 +0000 UTC m=+5.624272068 container died 90da4eae652d6ca9ecd5b683066a9b69fbe06047fd09fac188e569ebbe3a2377 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:15:31.677 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:31 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:31.538+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:31.677 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:31 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:31.538+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:31.677 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 04:15:31 smithi119 podman[68095]: 2024-09-21 04:15:31.466365367 +0000 UTC m=+6.053575819 container cleanup 90da4eae652d6ca9ecd5b683066a9b69fbe06047fd09fac188e569ebbe3a2377 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-4, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:15:31.677 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 04:15:31 smithi119 bash[68095]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-4 2024-09-21T04:15:32.007 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 04:15:31 smithi119 podman[68110]: 2024-09-21 04:15:31.676148007 +0000 UTC m=+0.637457454 container remove 90da4eae652d6ca9ecd5b683066a9b69fbe06047fd09fac188e569ebbe3a2377 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:15:32.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:31 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:31.712+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.882466+0000 front 2024-09-21T04:15:05.882507+0000 (oldest deadline 2024-09-21T04:15:31.083089+0000) 2024-09-21T04:15:32.053 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:31 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:31.728+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.245603+0000 front 2024-09-21T04:14:55.245502+0000 (oldest deadline 2024-09-21T04:15:19.345596+0000) 2024-09-21T04:15:32.054 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:31 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:31.728+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.647902+0000 front 2024-09-21T04:15:05.648031+0000 (oldest deadline 2024-09-21T04:15:31.548089+0000) 2024-09-21T04:15:32.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:32 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:32.171+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:58.181035+0000 front 2024-09-21T04:14:58.181137+0000 (oldest deadline 2024-09-21T04:15:23.481212+0000) 2024-09-21T04:15:32.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:32 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:32.171+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.182453+0000 front 2024-09-21T04:15:05.182422+0000 (oldest deadline 2024-09-21T04:15:29.282683+0000) 2024-09-21T04:15:32.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:31 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:31.941+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:32.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:31 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:31.941+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:32.415 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 04:15:32 smithi119 podman[68200]: 2024-09-21 04:15:32.068874289 +0000 UTC m=+0.111058370 container create e368fdf0816a8742e6d883efbf9dab41f2719d7a83daa6232311e832c4f4ff84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T04:15:32.416 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 04:15:32 smithi119 podman[68200]: 2024-09-21 04:15:31.973662874 +0000 UTC m=+0.015846969 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:15:32.416 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 04:15:32 smithi119 podman[68200]: 2024-09-21 04:15:32.313139294 +0000 UTC m=+0.355323381 container init e368fdf0816a8742e6d883efbf9dab41f2719d7a83daa6232311e832c4f4ff84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T04:15:32.416 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 04:15:32 smithi119 podman[68200]: 2024-09-21 04:15:32.316897144 +0000 UTC m=+0.359081236 container start e368fdf0816a8742e6d883efbf9dab41f2719d7a83daa6232311e832c4f4ff84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-21T04:15:32.416 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 04:15:32 smithi119 podman[68200]: 2024-09-21 04:15:32.413331624 +0000 UTC m=+0.455515708 container attach e368fdf0816a8742e6d883efbf9dab41f2719d7a83daa6232311e832c4f4ff84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-21T04:15:32.554 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:32 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:32.109+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:32.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:32 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:32.059+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:32.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:32 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:32.059+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:32.705 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:32 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:32.702+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.882466+0000 front 2024-09-21T04:15:05.882507+0000 (oldest deadline 2024-09-21T04:15:31.083089+0000) 2024-09-21T04:15:32.705 INFO:journalctl@ceph.osd.4.smithi119.stdout:Sep 21 04:15:32 smithi119 podman[68200]: 2024-09-21 04:15:32.50344234 +0000 UTC m=+0.545626432 container died e368fdf0816a8742e6d883efbf9dab41f2719d7a83daa6232311e832c4f4ff84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:15:32.706 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:32 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:32.546+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:32.706 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:32 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:32.546+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:33.053 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:32 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:32.711+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.245603+0000 front 2024-09-21T04:14:55.245502+0000 (oldest deadline 2024-09-21T04:15:19.345596+0000) 2024-09-21T04:15:33.054 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:32 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:32.711+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.647902+0000 front 2024-09-21T04:15:05.648031+0000 (oldest deadline 2024-09-21T04:15:31.548089+0000) 2024-09-21T04:15:33.174 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.4.service' 2024-09-21T04:15:33.256 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:15:33.257 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-21T04:15:33.257 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-21T04:15:33.257 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.5 2024-09-21T04:15:33.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:33 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:33.139+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:58.181035+0000 front 2024-09-21T04:14:58.181137+0000 (oldest deadline 2024-09-21T04:15:23.481212+0000) 2024-09-21T04:15:33.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:33 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:33.139+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.182453+0000 front 2024-09-21T04:15:05.182422+0000 (oldest deadline 2024-09-21T04:15:29.282683+0000) 2024-09-21T04:15:33.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:32 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:32.971+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:33.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:32 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:32.971+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:33.554 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:33 smithi131 systemd[1]: Stopping Ceph osd.5 for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:15:33.555 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:33 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:33.158+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:33.555 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:33 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:33.094+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:33.555 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:33 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:33.094+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:34.006 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:33 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:33.662+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.882466+0000 front 2024-09-21T04:15:05.882507+0000 (oldest deadline 2024-09-21T04:15:31.083089+0000) 2024-09-21T04:15:34.007 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:33 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:33.592+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:34.007 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:33 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:33.592+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:34.054 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:33 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:33.721+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.245603+0000 front 2024-09-21T04:14:55.245502+0000 (oldest deadline 2024-09-21T04:15:19.345596+0000) 2024-09-21T04:15:34.054 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:33 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:33.721+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.647902+0000 front 2024-09-21T04:15:05.648031+0000 (oldest deadline 2024-09-21T04:15:31.548089+0000) 2024-09-21T04:15:34.054 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:33 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:33.783+0000 7fb737e7f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:15:34.054 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:33 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:33.783+0000 7fb737e7f640 -1 osd.5 90 *** Got signal Terminated *** 2024-09-21T04:15:34.054 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:33 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:33.783+0000 7fb737e7f640 -1 osd.5 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:15:34.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:34 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:34.159+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:58.181035+0000 front 2024-09-21T04:14:58.181137+0000 (oldest deadline 2024-09-21T04:15:23.481212+0000) 2024-09-21T04:15:34.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:34 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:34.159+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.182453+0000 front 2024-09-21T04:15:05.182422+0000 (oldest deadline 2024-09-21T04:15:29.282683+0000) 2024-09-21T04:15:34.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:33 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:33.997+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:34.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:33 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:33.997+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:34.553 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:34 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:34.153+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:34.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:34 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:34.066+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:34.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:34 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:34.066+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:35.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:34 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:34.621+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.882466+0000 front 2024-09-21T04:15:05.882507+0000 (oldest deadline 2024-09-21T04:15:31.083089+0000) 2024-09-21T04:15:35.007 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:34 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:34.589+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:35.007 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:34 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:34.589+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:35.053 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:34 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:34.730+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.245603+0000 front 2024-09-21T04:14:55.245502+0000 (oldest deadline 2024-09-21T04:15:19.345596+0000) 2024-09-21T04:15:35.053 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:34 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:34.730+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.647902+0000 front 2024-09-21T04:15:05.648031+0000 (oldest deadline 2024-09-21T04:15:31.548089+0000) 2024-09-21T04:15:35.261 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:35 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:35.179+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:58.181035+0000 front 2024-09-21T04:14:58.181137+0000 (oldest deadline 2024-09-21T04:15:23.481212+0000) 2024-09-21T04:15:35.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:35 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:35.179+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.182453+0000 front 2024-09-21T04:15:05.182422+0000 (oldest deadline 2024-09-21T04:15:29.282683+0000) 2024-09-21T04:15:35.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:34 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:34.972+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:35.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:34 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:34.972+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:35.553 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:35 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:35.186+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:35.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:35 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:35.061+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:35.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:35 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:35.061+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:36.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:35 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:35.602+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.882466+0000 front 2024-09-21T04:15:05.882507+0000 (oldest deadline 2024-09-21T04:15:31.083089+0000) 2024-09-21T04:15:36.007 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:35 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:35.603+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:36.007 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:35 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:35.603+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:36.053 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:35 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:35.709+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.245603+0000 front 2024-09-21T04:14:55.245502+0000 (oldest deadline 2024-09-21T04:15:19.345596+0000) 2024-09-21T04:15:36.054 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:35 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:35.709+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.647902+0000 front 2024-09-21T04:15:05.648031+0000 (oldest deadline 2024-09-21T04:15:31.548089+0000) 2024-09-21T04:15:36.512 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:36 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:36.220+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:58.181035+0000 front 2024-09-21T04:14:58.181137+0000 (oldest deadline 2024-09-21T04:15:23.481212+0000) 2024-09-21T04:15:36.512 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:36 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:36.220+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.182453+0000 front 2024-09-21T04:15:05.182422+0000 (oldest deadline 2024-09-21T04:15:29.282683+0000) 2024-09-21T04:15:36.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:36 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:36.010+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:36.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:36 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:36.010+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:36.553 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:36 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:36.198+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:36.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:36 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:36.056+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:36.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:36 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:36.056+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:37.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:36 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:36.586+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.882466+0000 front 2024-09-21T04:15:05.882507+0000 (oldest deadline 2024-09-21T04:15:31.083089+0000) 2024-09-21T04:15:37.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:36 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:36.586+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:11.083478+0000 front 2024-09-21T04:15:11.083538+0000 (oldest deadline 2024-09-21T04:15:35.783548+0000) 2024-09-21T04:15:37.007 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:36 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:36.556+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:37.007 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:36 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:36.556+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:37.053 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:36 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:36.705+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.245603+0000 front 2024-09-21T04:14:55.245502+0000 (oldest deadline 2024-09-21T04:15:19.345596+0000) 2024-09-21T04:15:37.054 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:36 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:36.705+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.647902+0000 front 2024-09-21T04:15:05.648031+0000 (oldest deadline 2024-09-21T04:15:31.548089+0000) 2024-09-21T04:15:37.512 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:37 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:37.201+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:58.181035+0000 front 2024-09-21T04:14:58.181137+0000 (oldest deadline 2024-09-21T04:15:23.481212+0000) 2024-09-21T04:15:37.512 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:37 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:37.201+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.182453+0000 front 2024-09-21T04:15:05.182422+0000 (oldest deadline 2024-09-21T04:15:29.282683+0000) 2024-09-21T04:15:37.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:37 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:37.010+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:37.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:37 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:37.010+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:37.553 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:37 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:37.171+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:37.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:37 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:37.076+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:37.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:37 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:37.076+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:37.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:37 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:37.076+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:15:38.007 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:37 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:37.552+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:38.007 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:37 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:37.552+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:38.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:37 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:37.577+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.882466+0000 front 2024-09-21T04:15:05.882507+0000 (oldest deadline 2024-09-21T04:15:31.083089+0000) 2024-09-21T04:15:38.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:37 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:37.577+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:11.083478+0000 front 2024-09-21T04:15:11.083538+0000 (oldest deadline 2024-09-21T04:15:35.783548+0000) 2024-09-21T04:15:38.042 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:37 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:37.714+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.245603+0000 front 2024-09-21T04:14:55.245502+0000 (oldest deadline 2024-09-21T04:15:19.345596+0000) 2024-09-21T04:15:38.043 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:37 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:37.714+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.647902+0000 front 2024-09-21T04:15:05.648031+0000 (oldest deadline 2024-09-21T04:15:31.548089+0000) 2024-09-21T04:15:38.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:38 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:38.218+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:58.181035+0000 front 2024-09-21T04:14:58.181137+0000 (oldest deadline 2024-09-21T04:15:23.481212+0000) 2024-09-21T04:15:38.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:38 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:38.218+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.182453+0000 front 2024-09-21T04:15:05.182422+0000 (oldest deadline 2024-09-21T04:15:29.282683+0000) 2024-09-21T04:15:38.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:37 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:37.970+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:38.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:37 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:37.970+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:38.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:37 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:37.970+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:15:38.303 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:38 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:38.127+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:38.303 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:38 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:38.040+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:38.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:38 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:38.040+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:38.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:38 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:38.040+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:15:39.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:38 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:38.533+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.882466+0000 front 2024-09-21T04:15:05.882507+0000 (oldest deadline 2024-09-21T04:15:31.083089+0000) 2024-09-21T04:15:39.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:38 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:38.533+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:11.083478+0000 front 2024-09-21T04:15:11.083538+0000 (oldest deadline 2024-09-21T04:15:35.783548+0000) 2024-09-21T04:15:39.007 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:38 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:38.505+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:39.007 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:38 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:38.505+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:39.007 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:38 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:38.505+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:15:39.023 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:38 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:38.753+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.245603+0000 front 2024-09-21T04:14:55.245502+0000 (oldest deadline 2024-09-21T04:15:19.345596+0000) 2024-09-21T04:15:39.023 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:38 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5[46550]: 2024-09-21T04:15:38.753+0000 7fb73446a640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.647902+0000 front 2024-09-21T04:15:05.648031+0000 (oldest deadline 2024-09-21T04:15:31.548089+0000) 2024-09-21T04:15:39.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:39 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:39.211+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:58.181035+0000 front 2024-09-21T04:14:58.181137+0000 (oldest deadline 2024-09-21T04:15:23.481212+0000) 2024-09-21T04:15:39.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:39 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:39.211+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.182453+0000 front 2024-09-21T04:15:05.182422+0000 (oldest deadline 2024-09-21T04:15:29.282683+0000) 2024-09-21T04:15:39.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:39 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:39.211+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.385657+0000 front 2024-09-21T04:15:14.385672+0000 (oldest deadline 2024-09-21T04:15:38.485592+0000) 2024-09-21T04:15:39.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:38 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:38.988+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:39.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:38 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:38.988+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:39.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:38 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:38.988+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:15:39.303 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:39 smithi131 podman[68939]: 2024-09-21 04:15:39.258687712 +0000 UTC m=+5.808701912 container died ca7c302d4c2550e3ad94af3aa842278c3c7dd545a08b747dff2c2802d73af2c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-21T04:15:39.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:39 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:39.098+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:39.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:39 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:39.021+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:39.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:39 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:39.021+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:39.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:39 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:39.021+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:15:39.757 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:39 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:39.571+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.882466+0000 front 2024-09-21T04:15:05.882507+0000 (oldest deadline 2024-09-21T04:15:31.083089+0000) 2024-09-21T04:15:39.757 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:39 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:39.571+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:11.083478+0000 front 2024-09-21T04:15:11.083538+0000 (oldest deadline 2024-09-21T04:15:35.783548+0000) 2024-09-21T04:15:39.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:39 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:39.491+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:39.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:39 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:39.491+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:39.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:39 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:39.491+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:15:39.803 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:39 smithi131 podman[68939]: 2024-09-21 04:15:39.617051591 +0000 UTC m=+6.167065791 container cleanup ca7c302d4c2550e3ad94af3aa842278c3c7dd545a08b747dff2c2802d73af2c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:15:39.803 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:39 smithi131 bash[68939]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5 2024-09-21T04:15:40.221 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:39 smithi131 podman[68953]: 2024-09-21 04:15:39.90162093 +0000 UTC m=+0.637903971 container remove ca7c302d4c2550e3ad94af3aa842278c3c7dd545a08b747dff2c2802d73af2c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:15:40.221 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:40 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:40.127+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:40.221 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:40 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:40.127+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.115047+0000 front 2024-09-21T04:15:14.115092+0000 (oldest deadline 2024-09-21T04:15:39.415246+0000) 2024-09-21T04:15:40.221 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:40 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:40.069+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:40.221 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:40 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:40.069+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:40.222 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:40 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:40.069+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:15:40.228 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:39 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:39.957+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:40.228 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:39 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:39.957+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:40.228 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:39 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:39.957+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:15:40.482 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:40 smithi131 podman[69038]: 2024-09-21 04:15:40.219296035 +0000 UTC m=+0.102339341 container create 8b8861f2462a67f2166fd073a0d5d2077c9a3fbf3d1dca199e06f09d8ef070d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-09-21T04:15:40.482 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:40 smithi131 podman[69038]: 2024-09-21 04:15:40.133141918 +0000 UTC m=+0.016185192 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:15:40.512 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:40 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:40.226+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:58.181035+0000 front 2024-09-21T04:14:58.181137+0000 (oldest deadline 2024-09-21T04:15:23.481212+0000) 2024-09-21T04:15:40.512 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:40 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:40.226+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.182453+0000 front 2024-09-21T04:15:05.182422+0000 (oldest deadline 2024-09-21T04:15:29.282683+0000) 2024-09-21T04:15:40.512 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:40 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:40.226+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.385657+0000 front 2024-09-21T04:15:14.385672+0000 (oldest deadline 2024-09-21T04:15:38.485592+0000) 2024-09-21T04:15:40.757 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:40 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:40.551+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.882466+0000 front 2024-09-21T04:15:05.882507+0000 (oldest deadline 2024-09-21T04:15:31.083089+0000) 2024-09-21T04:15:40.757 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:40 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:40.551+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:11.083478+0000 front 2024-09-21T04:15:11.083538+0000 (oldest deadline 2024-09-21T04:15:35.783548+0000) 2024-09-21T04:15:40.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:40 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:40.459+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:40.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:40 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:40.459+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:40.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:40 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:40.459+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:15:40.804 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:40 smithi131 podman[69038]: 2024-09-21 04:15:40.480411945 +0000 UTC m=+0.363455205 container init 8b8861f2462a67f2166fd073a0d5d2077c9a3fbf3d1dca199e06f09d8ef070d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-21T04:15:40.804 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:40 smithi131 podman[69038]: 2024-09-21 04:15:40.483896424 +0000 UTC m=+0.366939691 container start 8b8861f2462a67f2166fd073a0d5d2077c9a3fbf3d1dca199e06f09d8ef070d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-21T04:15:40.804 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:40 smithi131 podman[69038]: 2024-09-21 04:15:40.563699173 +0000 UTC m=+0.446742439 container attach 8b8861f2462a67f2166fd073a0d5d2077c9a3fbf3d1dca199e06f09d8ef070d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5-deactivate, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-21T04:15:40.804 INFO:journalctl@ceph.osd.5.smithi131.stdout:Sep 21 04:15:40 smithi131 podman[69038]: 2024-09-21 04:15:40.67165797 +0000 UTC m=+0.554701243 container died 8b8861f2462a67f2166fd073a0d5d2077c9a3fbf3d1dca199e06f09d8ef070d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-21T04:15:41.201 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:40 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:40.936+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:41.201 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:40 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:40.936+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:41.201 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:40 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:40.936+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:15:41.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:41 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:41.149+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:41.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:41 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:41.149+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.115047+0000 front 2024-09-21T04:15:14.115092+0000 (oldest deadline 2024-09-21T04:15:39.415246+0000) 2024-09-21T04:15:41.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:41 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:41.063+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:41.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:41 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:41.063+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:41.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:41 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:41.063+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:15:41.340 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.5.service' 2024-09-21T04:15:41.380 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:15:41.380 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-21T04:15:41.381 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-21T04:15:41.381 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.6 2024-09-21T04:15:41.495 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:41 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:41.198+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:58.181035+0000 front 2024-09-21T04:14:58.181137+0000 (oldest deadline 2024-09-21T04:15:23.481212+0000) 2024-09-21T04:15:41.496 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:41 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:41.198+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.182453+0000 front 2024-09-21T04:15:05.182422+0000 (oldest deadline 2024-09-21T04:15:29.282683+0000) 2024-09-21T04:15:41.496 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:41 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:41.198+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.385657+0000 front 2024-09-21T04:15:14.385672+0000 (oldest deadline 2024-09-21T04:15:38.485592+0000) 2024-09-21T04:15:41.496 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:41 smithi064 systemd[1]: Stopping Ceph osd.6 for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:15:41.757 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:41 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:41.503+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.882466+0000 front 2024-09-21T04:15:05.882507+0000 (oldest deadline 2024-09-21T04:15:31.083089+0000) 2024-09-21T04:15:41.757 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:41 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:41.503+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:11.083478+0000 front 2024-09-21T04:15:11.083538+0000 (oldest deadline 2024-09-21T04:15:35.783548+0000) 2024-09-21T04:15:41.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:41 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:41.444+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:41.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:41 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:41.444+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:41.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:41 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:41.444+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:15:42.207 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:41 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:41.953+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:42.207 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:41 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:41.953+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:42.207 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:41 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:41.953+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:15:42.207 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:41 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:41.810+0000 7f21831c5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:15:42.207 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:41 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:41.810+0000 7f21831c5640 -1 osd.6 90 *** Got signal Terminated *** 2024-09-21T04:15:42.207 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:41 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:41.810+0000 7f21831c5640 -1 osd.6 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:15:42.207 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:42 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:42.203+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:58.181035+0000 front 2024-09-21T04:14:58.181137+0000 (oldest deadline 2024-09-21T04:15:23.481212+0000) 2024-09-21T04:15:42.207 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:42 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:42.203+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.182453+0000 front 2024-09-21T04:15:05.182422+0000 (oldest deadline 2024-09-21T04:15:29.282683+0000) 2024-09-21T04:15:42.208 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:42 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:42.203+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.385657+0000 front 2024-09-21T04:15:14.385672+0000 (oldest deadline 2024-09-21T04:15:38.485592+0000) 2024-09-21T04:15:42.553 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:42 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:42.198+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:42.554 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:42 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:42.198+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.115047+0000 front 2024-09-21T04:15:14.115092+0000 (oldest deadline 2024-09-21T04:15:39.415246+0000) 2024-09-21T04:15:42.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:42 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:42.097+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:42.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:42 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:42.097+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:42.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:42 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:42.097+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:15:42.756 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:42 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:42.543+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.882466+0000 front 2024-09-21T04:15:05.882507+0000 (oldest deadline 2024-09-21T04:15:31.083089+0000) 2024-09-21T04:15:42.757 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:42 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:42.543+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:11.083478+0000 front 2024-09-21T04:15:11.083538+0000 (oldest deadline 2024-09-21T04:15:35.783548+0000) 2024-09-21T04:15:42.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:42 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:42.397+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:42.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:42 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:42.397+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:42.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:42 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:42.397+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:15:43.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:42 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:42.965+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:43.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:42 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:42.965+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:43.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:42 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:42.965+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:15:43.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:43 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:43.167+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:58.181035+0000 front 2024-09-21T04:14:58.181137+0000 (oldest deadline 2024-09-21T04:15:23.481212+0000) 2024-09-21T04:15:43.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:43 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:43.167+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.182453+0000 front 2024-09-21T04:15:05.182422+0000 (oldest deadline 2024-09-21T04:15:29.282683+0000) 2024-09-21T04:15:43.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:43 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:43.167+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.385657+0000 front 2024-09-21T04:15:14.385672+0000 (oldest deadline 2024-09-21T04:15:38.485592+0000) 2024-09-21T04:15:43.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:43 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:43.167+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:21.986631+0000 front 2024-09-21T04:15:21.986685+0000 (oldest deadline 2024-09-21T04:15:43.086823+0000) 2024-09-21T04:15:43.553 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:43 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:43.237+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:43.554 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:43 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:43.237+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.115047+0000 front 2024-09-21T04:15:14.115092+0000 (oldest deadline 2024-09-21T04:15:39.415246+0000) 2024-09-21T04:15:43.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:43 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:43.054+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:43.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:43 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:43.054+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:43.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:43 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:43.054+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:15:43.756 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:43 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:43.584+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.882466+0000 front 2024-09-21T04:15:05.882507+0000 (oldest deadline 2024-09-21T04:15:31.083089+0000) 2024-09-21T04:15:43.757 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:43 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:43.584+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:11.083478+0000 front 2024-09-21T04:15:11.083538+0000 (oldest deadline 2024-09-21T04:15:35.783548+0000) 2024-09-21T04:15:43.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:43 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:43.411+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:43.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:43 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:43.411+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:43.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:43 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:43.411+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:15:43.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:43 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:43.411+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:15:44.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:43 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:43.944+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:44.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:43 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:43.944+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:44.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:43 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:43.944+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:15:44.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:44 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:44.118+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:58.181035+0000 front 2024-09-21T04:14:58.181137+0000 (oldest deadline 2024-09-21T04:15:23.481212+0000) 2024-09-21T04:15:44.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:44 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:44.118+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.182453+0000 front 2024-09-21T04:15:05.182422+0000 (oldest deadline 2024-09-21T04:15:29.282683+0000) 2024-09-21T04:15:44.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:44 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:44.118+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.385657+0000 front 2024-09-21T04:15:14.385672+0000 (oldest deadline 2024-09-21T04:15:38.485592+0000) 2024-09-21T04:15:44.263 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:44 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:44.118+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:21.986631+0000 front 2024-09-21T04:15:21.986685+0000 (oldest deadline 2024-09-21T04:15:43.086823+0000) 2024-09-21T04:15:44.553 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:44 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:44.262+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:44.554 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:44 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:44.262+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.115047+0000 front 2024-09-21T04:15:14.115092+0000 (oldest deadline 2024-09-21T04:15:39.415246+0000) 2024-09-21T04:15:44.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:44 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:44.055+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:44.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:44 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:44.055+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:44.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:44 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:44.055+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:15:44.757 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:44 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:44.552+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.882466+0000 front 2024-09-21T04:15:05.882507+0000 (oldest deadline 2024-09-21T04:15:31.083089+0000) 2024-09-21T04:15:44.757 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:44 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:44.552+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:11.083478+0000 front 2024-09-21T04:15:11.083538+0000 (oldest deadline 2024-09-21T04:15:35.783548+0000) 2024-09-21T04:15:44.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:44 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:44.388+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:44.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:44 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:44.388+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:44.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:44 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:44.388+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:15:44.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:44 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:44.388+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:15:45.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:45 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:45.131+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:58.181035+0000 front 2024-09-21T04:14:58.181137+0000 (oldest deadline 2024-09-21T04:15:23.481212+0000) 2024-09-21T04:15:45.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:45 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:45.131+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.182453+0000 front 2024-09-21T04:15:05.182422+0000 (oldest deadline 2024-09-21T04:15:29.282683+0000) 2024-09-21T04:15:45.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:45 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:45.131+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.385657+0000 front 2024-09-21T04:15:14.385672+0000 (oldest deadline 2024-09-21T04:15:38.485592+0000) 2024-09-21T04:15:45.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:45 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:45.131+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:21.986631+0000 front 2024-09-21T04:15:21.986685+0000 (oldest deadline 2024-09-21T04:15:43.086823+0000) 2024-09-21T04:15:45.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:44 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:44.988+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:45.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:44 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:44.988+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:45.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:44 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:44.988+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:15:45.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:44 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:44.988+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.263880+0000 front 2024-09-21T04:15:22.263849+0000 (oldest deadline 2024-09-21T04:15:43.963954+0000) 2024-09-21T04:15:45.553 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:45 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:45.234+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:45.554 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:45 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:45.234+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.115047+0000 front 2024-09-21T04:15:14.115092+0000 (oldest deadline 2024-09-21T04:15:39.415246+0000) 2024-09-21T04:15:45.554 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:45 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:45.234+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:19.415973+0000 front 2024-09-21T04:15:19.416024+0000 (oldest deadline 2024-09-21T04:15:44.715993+0000) 2024-09-21T04:15:45.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:45 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:45.099+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:45.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:45 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:45.099+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:45.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:45 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:45.099+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:15:45.757 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:45 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:45.544+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.882466+0000 front 2024-09-21T04:15:05.882507+0000 (oldest deadline 2024-09-21T04:15:31.083089+0000) 2024-09-21T04:15:45.757 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:45 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:45.544+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:11.083478+0000 front 2024-09-21T04:15:11.083538+0000 (oldest deadline 2024-09-21T04:15:35.783548+0000) 2024-09-21T04:15:45.757 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:45 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:45.544+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.185180+0000 front 2024-09-21T04:15:22.185231+0000 (oldest deadline 2024-09-21T04:15:45.085340+0000) 2024-09-21T04:15:45.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:45 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:45.341+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:45.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:45 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:45.341+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:45.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:45 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:45.341+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:15:45.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:45 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:45.341+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:15:46.261 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:46 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:46.090+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:58.181035+0000 front 2024-09-21T04:14:58.181137+0000 (oldest deadline 2024-09-21T04:15:23.481212+0000) 2024-09-21T04:15:46.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:46 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:46.090+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.182453+0000 front 2024-09-21T04:15:05.182422+0000 (oldest deadline 2024-09-21T04:15:29.282683+0000) 2024-09-21T04:15:46.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:46 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:46.090+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.385657+0000 front 2024-09-21T04:15:14.385672+0000 (oldest deadline 2024-09-21T04:15:38.485592+0000) 2024-09-21T04:15:46.262 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:46 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6[57922]: 2024-09-21T04:15:46.090+0000 7f217b7a8640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:21.986631+0000 front 2024-09-21T04:15:21.986685+0000 (oldest deadline 2024-09-21T04:15:43.086823+0000) 2024-09-21T04:15:46.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:45 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:45.957+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:46.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:45 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:45.957+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:46.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:45 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:45.957+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:15:46.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:45 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:45.957+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.263880+0000 front 2024-09-21T04:15:22.263849+0000 (oldest deadline 2024-09-21T04:15:43.963954+0000) 2024-09-21T04:15:46.554 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:46 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:46.193+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:46.554 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:46 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:46.193+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.115047+0000 front 2024-09-21T04:15:14.115092+0000 (oldest deadline 2024-09-21T04:15:39.415246+0000) 2024-09-21T04:15:46.554 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:46 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:46.193+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:19.415973+0000 front 2024-09-21T04:15:19.416024+0000 (oldest deadline 2024-09-21T04:15:44.715993+0000) 2024-09-21T04:15:46.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:46 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:46.125+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:46.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:46 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:46.125+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:46.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:46 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:46.125+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:15:46.757 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:46 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:46.578+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.882466+0000 front 2024-09-21T04:15:05.882507+0000 (oldest deadline 2024-09-21T04:15:31.083089+0000) 2024-09-21T04:15:46.757 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:46 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:46.578+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:11.083478+0000 front 2024-09-21T04:15:11.083538+0000 (oldest deadline 2024-09-21T04:15:35.783548+0000) 2024-09-21T04:15:46.757 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:46 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:46.578+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.185180+0000 front 2024-09-21T04:15:22.185231+0000 (oldest deadline 2024-09-21T04:15:45.085340+0000) 2024-09-21T04:15:46.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:46 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:46.330+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:46.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:46 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:46.330+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:46.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:46 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:46.330+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:15:46.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:46 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:46.330+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:15:47.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:46 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:46.943+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:47.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:46 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:46.943+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:47.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:46 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:46.943+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:15:47.213 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:46 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:46.943+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.263880+0000 front 2024-09-21T04:15:22.263849+0000 (oldest deadline 2024-09-21T04:15:43.963954+0000) 2024-09-21T04:15:47.512 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:47 smithi064 podman[103798]: 2024-09-21 04:15:47.210566024 +0000 UTC m=+5.638952705 container died 3c4c18ec67af7d68c8948842a11f9edd7df55f50f79836b7d07f80b2e798c002 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-21T04:15:47.554 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:47 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:47.224+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:47.554 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:47 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:47.224+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.115047+0000 front 2024-09-21T04:15:14.115092+0000 (oldest deadline 2024-09-21T04:15:39.415246+0000) 2024-09-21T04:15:47.554 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:47 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:47.224+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:19.415973+0000 front 2024-09-21T04:15:19.416024+0000 (oldest deadline 2024-09-21T04:15:44.715993+0000) 2024-09-21T04:15:47.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:47 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:47.140+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:47.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:47 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:47.140+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:47.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:47 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:47.140+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:15:47.575 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:47 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:47.289+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:47.575 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:47 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:47.289+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:47.575 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:47 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:47.289+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:15:47.575 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:47 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:47.289+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:15:47.914 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:47 smithi064 podman[103798]: 2024-09-21 04:15:47.735816912 +0000 UTC m=+6.164203580 container cleanup 3c4c18ec67af7d68c8948842a11f9edd7df55f50f79836b7d07f80b2e798c002 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:15:47.914 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:47 smithi064 bash[103798]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6 2024-09-21T04:15:48.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:47 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:47.572+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.882466+0000 front 2024-09-21T04:15:05.882507+0000 (oldest deadline 2024-09-21T04:15:31.083089+0000) 2024-09-21T04:15:48.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:47 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:47.572+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:11.083478+0000 front 2024-09-21T04:15:11.083538+0000 (oldest deadline 2024-09-21T04:15:35.783548+0000) 2024-09-21T04:15:48.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:47 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:47.572+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.185180+0000 front 2024-09-21T04:15:22.185231+0000 (oldest deadline 2024-09-21T04:15:45.085340+0000) 2024-09-21T04:15:48.261 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:47 smithi064 podman[103824]: 2024-09-21 04:15:47.912016641 +0000 UTC m=+0.695824477 container remove 3c4c18ec67af7d68c8948842a11f9edd7df55f50f79836b7d07f80b2e798c002 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:15:48.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:47 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:47.951+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:48.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:47 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:47.951+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:48.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:47 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:47.952+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:15:48.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:47 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:47.952+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.263880+0000 front 2024-09-21T04:15:22.263849+0000 (oldest deadline 2024-09-21T04:15:43.963954+0000) 2024-09-21T04:15:48.553 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:48 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:48.219+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:48.554 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:48 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:48.219+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.115047+0000 front 2024-09-21T04:15:14.115092+0000 (oldest deadline 2024-09-21T04:15:39.415246+0000) 2024-09-21T04:15:48.554 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:48 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:48.219+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:19.415973+0000 front 2024-09-21T04:15:19.416024+0000 (oldest deadline 2024-09-21T04:15:44.715993+0000) 2024-09-21T04:15:48.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:48 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:48.094+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:48.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:48 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:48.094+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:48.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:48 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:48.094+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:15:48.586 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:48 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:48.276+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:48.586 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:48 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:48.276+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:48.586 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:48 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:48.276+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:15:48.586 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:48 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:48.276+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:15:48.651 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:48 smithi064 podman[103916]: 2024-09-21 04:15:48.259393198 +0000 UTC m=+0.016686556 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:15:48.651 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:48 smithi064 podman[103916]: 2024-09-21 04:15:48.379688747 +0000 UTC m=+0.136982155 container create e9ef0c8aaf3be7fca9a59da4e7f8ec6b1df764035e34d171112ea1cafba5953b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-21T04:15:48.651 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:48 smithi064 podman[103916]: 2024-09-21 04:15:48.648862047 +0000 UTC m=+0.406155405 container init e9ef0c8aaf3be7fca9a59da4e7f8ec6b1df764035e34d171112ea1cafba5953b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-21T04:15:48.961 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:48 smithi064 podman[103916]: 2024-09-21 04:15:48.652826678 +0000 UTC m=+0.410120025 container start e9ef0c8aaf3be7fca9a59da4e7f8ec6b1df764035e34d171112ea1cafba5953b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS) 2024-09-21T04:15:48.962 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:48 smithi064 podman[103916]: 2024-09-21 04:15:48.724022894 +0000 UTC m=+0.481316240 container attach e9ef0c8aaf3be7fca9a59da4e7f8ec6b1df764035e34d171112ea1cafba5953b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-21T04:15:48.962 INFO:journalctl@ceph.osd.6.smithi064.stdout:Sep 21 04:15:48 smithi064 podman[103916]: 2024-09-21 04:15:48.842579853 +0000 UTC m=+0.599873210 container died e9ef0c8aaf3be7fca9a59da4e7f8ec6b1df764035e34d171112ea1cafba5953b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-21T04:15:49.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:48 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:48.583+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.882466+0000 front 2024-09-21T04:15:05.882507+0000 (oldest deadline 2024-09-21T04:15:31.083089+0000) 2024-09-21T04:15:49.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:48 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:48.583+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:11.083478+0000 front 2024-09-21T04:15:11.083538+0000 (oldest deadline 2024-09-21T04:15:35.783548+0000) 2024-09-21T04:15:49.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:48 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:48.583+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.185180+0000 front 2024-09-21T04:15:22.185231+0000 (oldest deadline 2024-09-21T04:15:45.085340+0000) 2024-09-21T04:15:49.261 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:48 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:48.958+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:49.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:48 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:48.958+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:49.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:48 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:48.958+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:15:49.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:48 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:48.958+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.263880+0000 front 2024-09-21T04:15:22.263849+0000 (oldest deadline 2024-09-21T04:15:43.963954+0000) 2024-09-21T04:15:49.553 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:49 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:49.174+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:49.578 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:49 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:49.174+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.115047+0000 front 2024-09-21T04:15:14.115092+0000 (oldest deadline 2024-09-21T04:15:39.415246+0000) 2024-09-21T04:15:49.578 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:49 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:49.174+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:19.415973+0000 front 2024-09-21T04:15:19.416024+0000 (oldest deadline 2024-09-21T04:15:44.715993+0000) 2024-09-21T04:15:49.579 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:49 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:49.099+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:49.579 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:49 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:49.099+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:49.579 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:49 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:49.099+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:15:49.583 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.6.service' 2024-09-21T04:15:49.621 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:49 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:49.282+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:49.621 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:49 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:49.282+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:49.621 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:49 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:49.282+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:15:49.621 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:49 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:49.282+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:15:49.624 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:15:49.625 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-21T04:15:49.625 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-21T04:15:49.625 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.7 2024-09-21T04:15:50.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:49 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:49.618+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.882466+0000 front 2024-09-21T04:15:05.882507+0000 (oldest deadline 2024-09-21T04:15:31.083089+0000) 2024-09-21T04:15:50.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:49 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:49.618+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:11.083478+0000 front 2024-09-21T04:15:11.083538+0000 (oldest deadline 2024-09-21T04:15:35.783548+0000) 2024-09-21T04:15:50.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:49 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:49.618+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.185180+0000 front 2024-09-21T04:15:22.185231+0000 (oldest deadline 2024-09-21T04:15:45.085340+0000) 2024-09-21T04:15:50.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:49 smithi119 systemd[1]: Stopping Ceph osd.7 for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:15:50.261 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:49 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:49.989+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:50.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:49 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:49.989+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:50.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:49 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:49.989+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:15:50.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:49 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:49.989+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.263880+0000 front 2024-09-21T04:15:22.263849+0000 (oldest deadline 2024-09-21T04:15:43.963954+0000) 2024-09-21T04:15:50.266 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:50 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:50.052+0000 7f07523e1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:15:50.266 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:50 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:50.052+0000 7f07523e1640 -1 osd.7 90 *** Got signal Terminated *** 2024-09-21T04:15:50.266 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:50 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:50.052+0000 7f07523e1640 -1 osd.7 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:15:50.553 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:50 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:50.201+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:50.554 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:50 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:50.201+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.115047+0000 front 2024-09-21T04:15:14.115092+0000 (oldest deadline 2024-09-21T04:15:39.415246+0000) 2024-09-21T04:15:50.554 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:50 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:50.201+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:19.415973+0000 front 2024-09-21T04:15:19.416024+0000 (oldest deadline 2024-09-21T04:15:44.715993+0000) 2024-09-21T04:15:50.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:50 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:50.087+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:50.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:50 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:50.087+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:50.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:50 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:50.087+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:15:50.661 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:50 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:50.284+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:50.661 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:50 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:50.284+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:50.662 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:50 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:50.284+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:15:50.662 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:50 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:50.284+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:15:51.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:50 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:50.658+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.882466+0000 front 2024-09-21T04:15:05.882507+0000 (oldest deadline 2024-09-21T04:15:31.083089+0000) 2024-09-21T04:15:51.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:50 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:50.658+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:11.083478+0000 front 2024-09-21T04:15:11.083538+0000 (oldest deadline 2024-09-21T04:15:35.783548+0000) 2024-09-21T04:15:51.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:50 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:50.658+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.185180+0000 front 2024-09-21T04:15:22.185231+0000 (oldest deadline 2024-09-21T04:15:45.085340+0000) 2024-09-21T04:15:51.261 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:51 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:51.000+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:51.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:51 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:51.000+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:51.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:51 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:51.000+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:15:51.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:51 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:51.000+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.263880+0000 front 2024-09-21T04:15:22.263849+0000 (oldest deadline 2024-09-21T04:15:43.963954+0000) 2024-09-21T04:15:51.303 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:51 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:51.157+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:51.303 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:51 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:51.157+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.115047+0000 front 2024-09-21T04:15:14.115092+0000 (oldest deadline 2024-09-21T04:15:39.415246+0000) 2024-09-21T04:15:51.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:51 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:51.157+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:19.415973+0000 front 2024-09-21T04:15:19.416024+0000 (oldest deadline 2024-09-21T04:15:44.715993+0000) 2024-09-21T04:15:51.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:51 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:51.046+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:51.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:51 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:51.046+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:51.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:51 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:51.046+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:15:51.507 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:51 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:51.242+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:51.507 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:51 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:51.242+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:51.507 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:51 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:51.242+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:15:51.507 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:51 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:51.242+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:15:52.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:51 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:51.649+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.882466+0000 front 2024-09-21T04:15:05.882507+0000 (oldest deadline 2024-09-21T04:15:31.083089+0000) 2024-09-21T04:15:52.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:51 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:51.649+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:11.083478+0000 front 2024-09-21T04:15:11.083538+0000 (oldest deadline 2024-09-21T04:15:35.783548+0000) 2024-09-21T04:15:52.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:51 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:51.649+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.185180+0000 front 2024-09-21T04:15:22.185231+0000 (oldest deadline 2024-09-21T04:15:45.085340+0000) 2024-09-21T04:15:52.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:51 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:51.985+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:52.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:51 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:51.985+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:52.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:51 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:51.985+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:15:52.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:51 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:51.985+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.263880+0000 front 2024-09-21T04:15:22.263849+0000 (oldest deadline 2024-09-21T04:15:43.963954+0000) 2024-09-21T04:15:52.507 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:52 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:52.228+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:52.507 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:52 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:52.228+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:52.507 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:52 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:52.228+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:15:52.507 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:52 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:52.228+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:15:52.507 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:52 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:52.228+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:29.012946+0000 front 2024-09-21T04:15:29.012996+0000 (oldest deadline 2024-09-21T04:15:51.313104+0000) 2024-09-21T04:15:52.527 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:52 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:52.151+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:52.527 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:52 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:52.151+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.115047+0000 front 2024-09-21T04:15:14.115092+0000 (oldest deadline 2024-09-21T04:15:39.415246+0000) 2024-09-21T04:15:52.527 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:52 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:52.151+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:19.415973+0000 front 2024-09-21T04:15:19.416024+0000 (oldest deadline 2024-09-21T04:15:44.715993+0000) 2024-09-21T04:15:52.528 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:52 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:52.066+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:52.528 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:52 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:52.066+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:52.528 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:52 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:52.066+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:15:53.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:52 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:52.629+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.882466+0000 front 2024-09-21T04:15:05.882507+0000 (oldest deadline 2024-09-21T04:15:31.083089+0000) 2024-09-21T04:15:53.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:52 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:52.629+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:11.083478+0000 front 2024-09-21T04:15:11.083538+0000 (oldest deadline 2024-09-21T04:15:35.783548+0000) 2024-09-21T04:15:53.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:52 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:52.629+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.185180+0000 front 2024-09-21T04:15:22.185231+0000 (oldest deadline 2024-09-21T04:15:45.085340+0000) 2024-09-21T04:15:53.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:53 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:53.033+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:53.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:53 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:53.033+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:53.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:53 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:53.033+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:15:53.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:53 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:53.033+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.263880+0000 front 2024-09-21T04:15:22.263849+0000 (oldest deadline 2024-09-21T04:15:43.963954+0000) 2024-09-21T04:15:53.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:53 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:53.033+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:27.365613+0000 front 2024-09-21T04:15:27.365561+0000 (oldest deadline 2024-09-21T04:15:52.665449+0000) 2024-09-21T04:15:53.553 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:53 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:53.138+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:53.554 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:53 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:53.138+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.115047+0000 front 2024-09-21T04:15:14.115092+0000 (oldest deadline 2024-09-21T04:15:39.415246+0000) 2024-09-21T04:15:53.554 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:53 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:53.138+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:19.415973+0000 front 2024-09-21T04:15:19.416024+0000 (oldest deadline 2024-09-21T04:15:44.715993+0000) 2024-09-21T04:15:53.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:53 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:53.070+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:53.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:53 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:53.070+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:53.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:53 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:53.070+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:15:53.647 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:53 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:53.272+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:53.648 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:53 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:53.272+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:53.648 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:53 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:53.272+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:15:53.648 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:53 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:53.272+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:15:53.648 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:53 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:53.272+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:29.012946+0000 front 2024-09-21T04:15:29.012996+0000 (oldest deadline 2024-09-21T04:15:51.313104+0000) 2024-09-21T04:15:54.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:53 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:53.645+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.882466+0000 front 2024-09-21T04:15:05.882507+0000 (oldest deadline 2024-09-21T04:15:31.083089+0000) 2024-09-21T04:15:54.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:53 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:53.645+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:11.083478+0000 front 2024-09-21T04:15:11.083538+0000 (oldest deadline 2024-09-21T04:15:35.783548+0000) 2024-09-21T04:15:54.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:53 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:53.645+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.185180+0000 front 2024-09-21T04:15:22.185231+0000 (oldest deadline 2024-09-21T04:15:45.085340+0000) 2024-09-21T04:15:54.507 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:54 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:54.236+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:54.507 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:54 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:54.236+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:54.507 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:54 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:54.236+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:15:54.507 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:54 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:54.236+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:15:54.507 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:54 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:54.236+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:29.012946+0000 front 2024-09-21T04:15:29.012996+0000 (oldest deadline 2024-09-21T04:15:51.313104+0000) 2024-09-21T04:15:54.511 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:54 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:54.034+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:54.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:54 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:54.034+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:54.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:54 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:54.034+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:15:54.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:54 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:54.034+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.263880+0000 front 2024-09-21T04:15:22.263849+0000 (oldest deadline 2024-09-21T04:15:43.963954+0000) 2024-09-21T04:15:54.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:54 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:54.034+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:27.365613+0000 front 2024-09-21T04:15:27.365561+0000 (oldest deadline 2024-09-21T04:15:52.665449+0000) 2024-09-21T04:15:54.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:54 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:54.066+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:54.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:54 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:54.066+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:54.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:54 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:54.066+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:15:54.554 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:54 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:54.101+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:54.554 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:54 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:54.101+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.115047+0000 front 2024-09-21T04:15:14.115092+0000 (oldest deadline 2024-09-21T04:15:39.415246+0000) 2024-09-21T04:15:54.554 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:54 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:54.101+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:19.415973+0000 front 2024-09-21T04:15:19.416024+0000 (oldest deadline 2024-09-21T04:15:44.715993+0000) 2024-09-21T04:15:54.554 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:54 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:54.101+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:27.616837+0000 front 2024-09-21T04:15:27.616983+0000 (oldest deadline 2024-09-21T04:15:53.516948+0000) 2024-09-21T04:15:55.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:54 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:54.632+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.882466+0000 front 2024-09-21T04:15:05.882507+0000 (oldest deadline 2024-09-21T04:15:31.083089+0000) 2024-09-21T04:15:55.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:54 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:54.632+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:11.083478+0000 front 2024-09-21T04:15:11.083538+0000 (oldest deadline 2024-09-21T04:15:35.783548+0000) 2024-09-21T04:15:55.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:54 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:54.632+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.185180+0000 front 2024-09-21T04:15:22.185231+0000 (oldest deadline 2024-09-21T04:15:45.085340+0000) 2024-09-21T04:15:55.007 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:54 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7[52619]: 2024-09-21T04:15:54.632+0000 7f074e9cc640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.287005+0000 front 2024-09-21T04:15:30.287054+0000 (oldest deadline 2024-09-21T04:15:54.387119+0000) 2024-09-21T04:15:55.303 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:55 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:55.059+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:55.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:55 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:55.059+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.115047+0000 front 2024-09-21T04:15:14.115092+0000 (oldest deadline 2024-09-21T04:15:39.415246+0000) 2024-09-21T04:15:55.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:55 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:55.059+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:19.415973+0000 front 2024-09-21T04:15:19.416024+0000 (oldest deadline 2024-09-21T04:15:44.715993+0000) 2024-09-21T04:15:55.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:55 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:55.059+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:27.616837+0000 front 2024-09-21T04:15:27.616983+0000 (oldest deadline 2024-09-21T04:15:53.516948+0000) 2024-09-21T04:15:55.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:55 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:55.045+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:55.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:55 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:55.045+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:55.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:55 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:55.045+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:15:55.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:55 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:55.045+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:15:55.501 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:55 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:55.241+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:55.501 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:55 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:55.241+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:55.502 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:55 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:55.241+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:15:55.502 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:55 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:55.241+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:15:55.502 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:55 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:55.241+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:29.012946+0000 front 2024-09-21T04:15:29.012996+0000 (oldest deadline 2024-09-21T04:15:51.313104+0000) 2024-09-21T04:15:55.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:55 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:55.030+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:55.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:55 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:55.030+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:55.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:55 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:55.030+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:15:55.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:55 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:55.030+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.263880+0000 front 2024-09-21T04:15:22.263849+0000 (oldest deadline 2024-09-21T04:15:43.963954+0000) 2024-09-21T04:15:55.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:55 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:55.030+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:27.365613+0000 front 2024-09-21T04:15:27.365561+0000 (oldest deadline 2024-09-21T04:15:52.665449+0000) 2024-09-21T04:15:55.757 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:55 smithi119 podman[68304]: 2024-09-21 04:15:55.499208967 +0000 UTC m=+5.688337298 container died bedd7fc191f348b5cb13a2646b82845213d38e01d2ca06909a5996e0f76f0e3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:15:56.156 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:55 smithi119 podman[68304]: 2024-09-21 04:15:55.877831899 +0000 UTC m=+6.066960233 container cleanup bedd7fc191f348b5cb13a2646b82845213d38e01d2ca06909a5996e0f76f0e3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-21T04:15:56.156 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:55 smithi119 bash[68304]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7 2024-09-21T04:15:56.261 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:55 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:55.990+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:56.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:55 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:55.990+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:56.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:55 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:55.990+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:15:56.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:55 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:55.990+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.263880+0000 front 2024-09-21T04:15:22.263849+0000 (oldest deadline 2024-09-21T04:15:43.963954+0000) 2024-09-21T04:15:56.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:55 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:55.990+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:27.365613+0000 front 2024-09-21T04:15:27.365561+0000 (oldest deadline 2024-09-21T04:15:52.665449+0000) 2024-09-21T04:15:56.463 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:56 smithi119 podman[68332]: 2024-09-21 04:15:56.154233502 +0000 UTC m=+0.645846170 container remove bedd7fc191f348b5cb13a2646b82845213d38e01d2ca06909a5996e0f76f0e3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-21T04:15:56.463 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:56 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:56.215+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:56.463 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:56 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:56.215+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:56.463 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:56 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:56.215+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:15:56.463 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:56 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:56.215+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:15:56.463 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:56 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:56.215+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:29.012946+0000 front 2024-09-21T04:15:29.012996+0000 (oldest deadline 2024-09-21T04:15:51.313104+0000) 2024-09-21T04:15:56.554 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:56 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:56.084+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:56.554 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:56 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:56.084+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.115047+0000 front 2024-09-21T04:15:14.115092+0000 (oldest deadline 2024-09-21T04:15:39.415246+0000) 2024-09-21T04:15:56.554 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:56 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:56.084+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:19.415973+0000 front 2024-09-21T04:15:19.416024+0000 (oldest deadline 2024-09-21T04:15:44.715993+0000) 2024-09-21T04:15:56.554 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:56 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:56.084+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:27.616837+0000 front 2024-09-21T04:15:27.616983+0000 (oldest deadline 2024-09-21T04:15:53.516948+0000) 2024-09-21T04:15:56.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:56 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:56.085+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:56.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:56 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:56.085+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:56.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:56 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:56.085+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:15:56.554 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:56 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:56.085+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:15:56.757 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:56 smithi119 podman[68420]: 2024-09-21 04:15:56.363433161 +0000 UTC m=+0.015716372 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:15:56.757 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:56 smithi119 podman[68420]: 2024-09-21 04:15:56.482147808 +0000 UTC m=+0.134431010 container create e29e51164a22f506ac8423bc39f7a4342ffd5a5e1bb654c88b06498b127a3f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-21T04:15:56.757 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:56 smithi119 podman[68420]: 2024-09-21 04:15:56.691363098 +0000 UTC m=+0.343646302 container init e29e51164a22f506ac8423bc39f7a4342ffd5a5e1bb654c88b06498b127a3f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:15:56.757 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:56 smithi119 podman[68420]: 2024-09-21 04:15:56.694667948 +0000 UTC m=+0.346951153 container start e29e51164a22f506ac8423bc39f7a4342ffd5a5e1bb654c88b06498b127a3f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-21T04:15:57.188 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:56 smithi119 podman[68420]: 2024-09-21 04:15:56.816070329 +0000 UTC m=+0.468353527 container attach e29e51164a22f506ac8423bc39f7a4342ffd5a5e1bb654c88b06498b127a3f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-21T04:15:57.188 INFO:journalctl@ceph.osd.7.smithi119.stdout:Sep 21 04:15:56 smithi119 podman[68420]: 2024-09-21 04:15:56.8844075 +0000 UTC m=+0.536690709 container died e29e51164a22f506ac8423bc39f7a4342ffd5a5e1bb654c88b06498b127a3f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:15:57.261 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:57 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:57.005+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:57.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:57 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:57.005+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:57.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:57 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:57.005+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:15:57.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:57 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:57.005+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.263880+0000 front 2024-09-21T04:15:22.263849+0000 (oldest deadline 2024-09-21T04:15:43.963954+0000) 2024-09-21T04:15:57.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:57 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:57.005+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:27.365613+0000 front 2024-09-21T04:15:27.365561+0000 (oldest deadline 2024-09-21T04:15:52.665449+0000) 2024-09-21T04:15:57.303 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:57 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:57.035+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:57.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:57 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:57.035+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.115047+0000 front 2024-09-21T04:15:14.115092+0000 (oldest deadline 2024-09-21T04:15:39.415246+0000) 2024-09-21T04:15:57.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:57 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:57.035+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:19.415973+0000 front 2024-09-21T04:15:19.416024+0000 (oldest deadline 2024-09-21T04:15:44.715993+0000) 2024-09-21T04:15:57.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:57 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:57.035+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:27.616837+0000 front 2024-09-21T04:15:27.616983+0000 (oldest deadline 2024-09-21T04:15:53.516948+0000) 2024-09-21T04:15:57.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:57 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:57.069+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:57.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:57 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:57.069+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:57.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:57 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:57.069+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:15:57.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:57 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:57.069+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:15:57.480 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:57 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:57.250+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:57.480 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:57 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:57.250+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:57.480 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:57 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:57.250+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:15:57.480 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:57 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:57.250+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:15:57.480 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:57 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:57.250+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:29.012946+0000 front 2024-09-21T04:15:29.012996+0000 (oldest deadline 2024-09-21T04:15:51.313104+0000) 2024-09-21T04:15:57.503 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.7.service' 2024-09-21T04:15:57.541 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:15:57.542 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-21T04:15:57.542 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-21T04:15:57.542 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.8 2024-09-21T04:15:57.839 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:57 smithi131 systemd[1]: Stopping Ceph osd.8 for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:15:58.303 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:57 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:57.836+0000 7f0dbc365640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:15:58.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:57 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:57.836+0000 7f0dbc365640 -1 osd.8 90 *** Got signal Terminated *** 2024-09-21T04:15:58.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:57 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:57.836+0000 7f0dbc365640 -1 osd.8 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:15:58.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:57 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:57.998+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:58.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:57 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:57.998+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.115047+0000 front 2024-09-21T04:15:14.115092+0000 (oldest deadline 2024-09-21T04:15:39.415246+0000) 2024-09-21T04:15:58.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:57 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:57.998+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:19.415973+0000 front 2024-09-21T04:15:19.416024+0000 (oldest deadline 2024-09-21T04:15:44.715993+0000) 2024-09-21T04:15:58.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:57 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:57.998+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:27.616837+0000 front 2024-09-21T04:15:27.616983+0000 (oldest deadline 2024-09-21T04:15:53.516948+0000) 2024-09-21T04:15:58.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:58 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:58.051+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:58.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:58 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:58.051+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:58.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:58 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:58.051+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:15:58.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:58 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:58.051+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:15:58.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:58 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:58.010+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:58.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:58 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:58.010+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:58.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:58 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:58.010+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:15:58.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:58 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:58.010+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.263880+0000 front 2024-09-21T04:15:22.263849+0000 (oldest deadline 2024-09-21T04:15:43.963954+0000) 2024-09-21T04:15:58.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:58 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:58.010+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:27.365613+0000 front 2024-09-21T04:15:27.365561+0000 (oldest deadline 2024-09-21T04:15:52.665449+0000) 2024-09-21T04:15:58.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:58 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:58.294+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:58.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:58 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:58.294+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:58.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:58 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:58.294+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:15:58.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:58 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:58.294+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:15:58.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:58 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:58.294+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:29.012946+0000 front 2024-09-21T04:15:29.012996+0000 (oldest deadline 2024-09-21T04:15:51.313104+0000) 2024-09-21T04:15:59.303 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:59 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:59.027+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:15:59.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:59 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:59.027+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.115047+0000 front 2024-09-21T04:15:14.115092+0000 (oldest deadline 2024-09-21T04:15:39.415246+0000) 2024-09-21T04:15:59.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:59 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:59.027+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:19.415973+0000 front 2024-09-21T04:15:19.416024+0000 (oldest deadline 2024-09-21T04:15:44.715993+0000) 2024-09-21T04:15:59.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:59 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:59.027+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:27.616837+0000 front 2024-09-21T04:15:27.616983+0000 (oldest deadline 2024-09-21T04:15:53.516948+0000) 2024-09-21T04:15:59.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:59 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:59.016+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:15:59.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:59 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:59.016+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:15:59.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:59 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:59.016+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:15:59.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:15:59 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:15:59.016+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:15:59.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:59 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:59.010+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:15:59.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:59 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:59.010+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:15:59.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:59 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:59.010+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:15:59.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:59 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:59.010+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.263880+0000 front 2024-09-21T04:15:22.263849+0000 (oldest deadline 2024-09-21T04:15:43.963954+0000) 2024-09-21T04:15:59.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:15:59 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:15:59.010+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:27.365613+0000 front 2024-09-21T04:15:27.365561+0000 (oldest deadline 2024-09-21T04:15:52.665449+0000) 2024-09-21T04:15:59.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:59 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:59.320+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:15:59.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:59 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:59.320+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:15:59.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:59 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:59.320+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:15:59.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:59 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:59.320+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:15:59.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:15:59 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:15:59.320+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:29.012946+0000 front 2024-09-21T04:15:29.012996+0000 (oldest deadline 2024-09-21T04:15:51.313104+0000) 2024-09-21T04:16:00.303 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:59 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:59.994+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:16:00.303 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:59 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:59.994+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.115047+0000 front 2024-09-21T04:15:14.115092+0000 (oldest deadline 2024-09-21T04:15:39.415246+0000) 2024-09-21T04:16:00.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:59 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:59.994+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:19.415973+0000 front 2024-09-21T04:15:19.416024+0000 (oldest deadline 2024-09-21T04:15:44.715993+0000) 2024-09-21T04:16:00.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:15:59 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:15:59.994+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:27.616837+0000 front 2024-09-21T04:15:27.616983+0000 (oldest deadline 2024-09-21T04:15:53.516948+0000) 2024-09-21T04:16:00.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:00 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:00.002+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:16:00.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:00 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:00.002+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:16:00.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:00 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:00.002+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:16:00.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:00 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:00.002+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:16:00.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:00 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:00.016+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:16:00.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:00 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:00.016+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:16:00.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:00 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:00.016+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:16:00.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:00 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:00.016+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.263880+0000 front 2024-09-21T04:15:22.263849+0000 (oldest deadline 2024-09-21T04:15:43.963954+0000) 2024-09-21T04:16:00.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:00 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:00.016+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:27.365613+0000 front 2024-09-21T04:15:27.365561+0000 (oldest deadline 2024-09-21T04:15:52.665449+0000) 2024-09-21T04:16:00.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:00 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:00.367+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:16:00.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:00 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:00.367+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:16:00.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:00 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:00.367+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:16:00.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:00 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:00.367+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:16:00.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:00 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:00.367+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:29.012946+0000 front 2024-09-21T04:15:29.012996+0000 (oldest deadline 2024-09-21T04:15:51.313104+0000) 2024-09-21T04:16:01.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:16:00 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:16:00.998+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:16:01.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:16:00 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:16:00.998+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.115047+0000 front 2024-09-21T04:15:14.115092+0000 (oldest deadline 2024-09-21T04:15:39.415246+0000) 2024-09-21T04:16:01.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:16:00 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:16:00.998+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:19.415973+0000 front 2024-09-21T04:15:19.416024+0000 (oldest deadline 2024-09-21T04:15:44.715993+0000) 2024-09-21T04:16:01.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:16:00 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:16:00.998+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:27.616837+0000 front 2024-09-21T04:15:27.616983+0000 (oldest deadline 2024-09-21T04:15:53.516948+0000) 2024-09-21T04:16:01.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:00 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:00.967+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:16:01.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:00 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:00.967+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:16:01.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:00 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:00.967+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:16:01.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:00 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:00.967+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:16:01.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:01 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:01.021+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:16:01.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:01 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:01.021+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:16:01.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:01 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:01.021+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:16:01.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:01 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:01.021+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.263880+0000 front 2024-09-21T04:15:22.263849+0000 (oldest deadline 2024-09-21T04:15:43.963954+0000) 2024-09-21T04:16:01.512 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:01 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:01.021+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:27.365613+0000 front 2024-09-21T04:15:27.365561+0000 (oldest deadline 2024-09-21T04:15:52.665449+0000) 2024-09-21T04:16:01.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:01 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:01.412+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:16:01.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:01 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:01.412+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:16:01.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:01 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:01.412+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:16:01.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:01 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:01.412+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:16:01.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:01 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:01.412+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:29.012946+0000 front 2024-09-21T04:15:29.012996+0000 (oldest deadline 2024-09-21T04:15:51.313104+0000) 2024-09-21T04:16:02.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:01 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:01.989+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:16:02.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:01 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:01.989+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:16:02.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:01 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:01.989+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:16:02.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:01 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:01.989+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.263880+0000 front 2024-09-21T04:15:22.263849+0000 (oldest deadline 2024-09-21T04:15:43.963954+0000) 2024-09-21T04:16:02.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:01 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:01.989+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:27.365613+0000 front 2024-09-21T04:15:27.365561+0000 (oldest deadline 2024-09-21T04:15:52.665449+0000) 2024-09-21T04:16:02.303 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:16:01 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:16:01.956+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:02.413276+0000 front 2024-09-21T04:15:02.413297+0000 (oldest deadline 2024-09-21T04:15:28.313293+0000) 2024-09-21T04:16:02.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:16:01 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:16:01.956+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:14.115047+0000 front 2024-09-21T04:15:14.115092+0000 (oldest deadline 2024-09-21T04:15:39.415246+0000) 2024-09-21T04:16:02.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:16:01 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:16:01.956+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:19.415973+0000 front 2024-09-21T04:15:19.416024+0000 (oldest deadline 2024-09-21T04:15:44.715993+0000) 2024-09-21T04:16:02.304 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:16:01 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8[52739]: 2024-09-21T04:16:01.956+0000 7f0db4948640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:27.616837+0000 front 2024-09-21T04:15:27.616983+0000 (oldest deadline 2024-09-21T04:15:53.516948+0000) 2024-09-21T04:16:02.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:01 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:01.955+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:16:02.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:01 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:01.955+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:16:02.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:01 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:01.955+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:16:02.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:01 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:01.955+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:16:02.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:02 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:02.366+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:16:02.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:02 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:02.366+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:16:02.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:02 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:02.366+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:16:02.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:02 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:02.366+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:16:02.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:02 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:02.366+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:29.012946+0000 front 2024-09-21T04:15:29.012996+0000 (oldest deadline 2024-09-21T04:15:51.313104+0000) 2024-09-21T04:16:02.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:02 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:02.366+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:36.315769+0000 front 2024-09-21T04:15:36.315739+0000 (oldest deadline 2024-09-21T04:16:02.215767+0000) 2024-09-21T04:16:03.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:03 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:03.002+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:16:03.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:03 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:03.002+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:16:03.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:03 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:03.002+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:16:03.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:03 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:03.002+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.263880+0000 front 2024-09-21T04:15:22.263849+0000 (oldest deadline 2024-09-21T04:15:43.963954+0000) 2024-09-21T04:16:03.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:03 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:03.002+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:27.365613+0000 front 2024-09-21T04:15:27.365561+0000 (oldest deadline 2024-09-21T04:15:52.665449+0000) 2024-09-21T04:16:03.303 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:02 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:02.964+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:16:03.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:02 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:02.964+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:16:03.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:02 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:02.964+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:16:03.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:02 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:02.964+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:16:03.656 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:16:03 smithi131 podman[69153]: 2024-09-21 04:16:03.359614345 +0000 UTC m=+5.622135781 container died 84e80a27c4726bad83ab1d9aa773ed42debe1d010d5c321cee3ac9b4f4e72584 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-21T04:16:03.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:03 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:03.339+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:16:03.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:03 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:03.339+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:16:03.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:03 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:03.339+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:16:03.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:03 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:03.339+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:16:03.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:03 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:03.339+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:29.012946+0000 front 2024-09-21T04:15:29.012996+0000 (oldest deadline 2024-09-21T04:15:51.313104+0000) 2024-09-21T04:16:03.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:03 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:03.339+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:36.315769+0000 front 2024-09-21T04:15:36.315739+0000 (oldest deadline 2024-09-21T04:16:02.215767+0000) 2024-09-21T04:16:03.979 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:16:03 smithi131 podman[69153]: 2024-09-21 04:16:03.753604018 +0000 UTC m=+6.016125448 container cleanup 84e80a27c4726bad83ab1d9aa773ed42debe1d010d5c321cee3ac9b4f4e72584 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T04:16:03.979 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:16:03 smithi131 bash[69153]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8 2024-09-21T04:16:04.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:03 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:03.974+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:16:04.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:03 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:03.974+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:16:04.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:03 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:03.974+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:16:04.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:03 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:03.974+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.263880+0000 front 2024-09-21T04:15:22.263849+0000 (oldest deadline 2024-09-21T04:15:43.963954+0000) 2024-09-21T04:16:04.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:03 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:03.974+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:27.365613+0000 front 2024-09-21T04:15:27.365561+0000 (oldest deadline 2024-09-21T04:15:52.665449+0000) 2024-09-21T04:16:04.303 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:16:04 smithi131 podman[69169]: 2024-09-21 04:16:04.005126644 +0000 UTC m=+0.639451847 container remove 84e80a27c4726bad83ab1d9aa773ed42debe1d010d5c321cee3ac9b4f4e72584 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:16:04.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:03 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:03.976+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:16:04.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:03 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:03.976+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:16:04.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:03 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:03.976+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:16:04.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:03 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:03.976+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:16:04.627 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:16:04 smithi131 podman[69259]: 2024-09-21 04:16:04.305744849 +0000 UTC m=+0.102607041 container create 889fcf948e755a7d03714205112fffe9f515f9f44a18e23be21a193b90b16314 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-21T04:16:04.627 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:16:04 smithi131 podman[69259]: 2024-09-21 04:16:04.219145521 +0000 UTC m=+0.016007723 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:16:04.627 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:16:04 smithi131 podman[69259]: 2024-09-21 04:16:04.533356308 +0000 UTC m=+0.330218512 container init 889fcf948e755a7d03714205112fffe9f515f9f44a18e23be21a193b90b16314 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:16:04.627 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:16:04 smithi131 podman[69259]: 2024-09-21 04:16:04.536547574 +0000 UTC m=+0.333409765 container start 889fcf948e755a7d03714205112fffe9f515f9f44a18e23be21a193b90b16314 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-21T04:16:04.628 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:16:04 smithi131 podman[69259]: 2024-09-21 04:16:04.625026452 +0000 UTC m=+0.421888657 container attach 889fcf948e755a7d03714205112fffe9f515f9f44a18e23be21a193b90b16314 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T04:16:04.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:04 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:04.365+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:16:04.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:04 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:04.365+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:16:04.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:04 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:04.365+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:16:04.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:04 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:04.365+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:16:04.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:04 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:04.365+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:29.012946+0000 front 2024-09-21T04:15:29.012996+0000 (oldest deadline 2024-09-21T04:15:51.313104+0000) 2024-09-21T04:16:04.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:04 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:04.365+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:36.315769+0000 front 2024-09-21T04:15:36.315739+0000 (oldest deadline 2024-09-21T04:16:02.215767+0000) 2024-09-21T04:16:04.973 INFO:journalctl@ceph.osd.8.smithi131.stdout:Sep 21 04:16:04 smithi131 podman[69259]: 2024-09-21 04:16:04.720578657 +0000 UTC m=+0.517440850 container died 889fcf948e755a7d03714205112fffe9f515f9f44a18e23be21a193b90b16314 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T04:16:05.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:04 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:04.942+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:16:05.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:04 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:04.942+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:16:05.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:04 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:04.942+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:16:05.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:04 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:04.942+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.263880+0000 front 2024-09-21T04:15:22.263849+0000 (oldest deadline 2024-09-21T04:15:43.963954+0000) 2024-09-21T04:16:05.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:04 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:04.942+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:27.365613+0000 front 2024-09-21T04:15:27.365561+0000 (oldest deadline 2024-09-21T04:15:52.665449+0000) 2024-09-21T04:16:05.298 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:04 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:04.970+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:16:05.298 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:04 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:04.970+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:16:05.298 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:04 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:04.970+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:16:05.298 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:04 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:04.970+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:16:05.298 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:04 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:04.970+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:38.677868+0000 front 2024-09-21T04:15:38.677787+0000 (oldest deadline 2024-09-21T04:16:04.577960+0000) 2024-09-21T04:16:05.321 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.8.service' 2024-09-21T04:16:05.359 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:16:05.359 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-21T04:16:05.360 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-21T04:16:05.360 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.9 2024-09-21T04:16:05.683 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:05 smithi064 systemd[1]: Stopping Ceph osd.9 for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:16:05.684 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:05.680+0000 7f8c3093c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:16:05.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:05 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:05.369+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:16:05.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:05 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:05.369+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:16:05.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:05 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:05.369+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:16:05.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:05 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:05.369+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:16:05.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:05 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:05.369+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:29.012946+0000 front 2024-09-21T04:15:29.012996+0000 (oldest deadline 2024-09-21T04:15:51.313104+0000) 2024-09-21T04:16:05.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:05 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:05.369+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:36.315769+0000 front 2024-09-21T04:15:36.315739+0000 (oldest deadline 2024-09-21T04:16:02.215767+0000) 2024-09-21T04:16:06.012 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:05.680+0000 7f8c3093c640 -1 osd.9 90 *** Got signal Terminated *** 2024-09-21T04:16:06.012 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:05.680+0000 7f8c3093c640 -1 osd.9 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:16:06.012 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:05.898+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:16:06.012 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:05.898+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:16:06.012 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:05.898+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:16:06.012 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:05.898+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.263880+0000 front 2024-09-21T04:15:22.263849+0000 (oldest deadline 2024-09-21T04:15:43.963954+0000) 2024-09-21T04:16:06.012 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:05 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:05.898+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:27.365613+0000 front 2024-09-21T04:15:27.365561+0000 (oldest deadline 2024-09-21T04:15:52.665449+0000) 2024-09-21T04:16:06.303 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:05 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:05.923+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:16:06.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:05 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:05.923+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:16:06.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:05 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:05.923+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:16:06.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:05 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:05.923+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:16:06.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:05 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:05.923+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:38.677868+0000 front 2024-09-21T04:15:38.677787+0000 (oldest deadline 2024-09-21T04:16:04.577960+0000) 2024-09-21T04:16:06.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:06 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:06.382+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:16:06.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:06 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:06.382+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:16:06.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:06 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:06.382+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:16:06.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:06 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:06.382+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:16:06.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:06 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:06.382+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:29.012946+0000 front 2024-09-21T04:15:29.012996+0000 (oldest deadline 2024-09-21T04:15:51.313104+0000) 2024-09-21T04:16:06.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:06 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:06.382+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:36.315769+0000 front 2024-09-21T04:15:36.315739+0000 (oldest deadline 2024-09-21T04:16:02.215767+0000) 2024-09-21T04:16:07.261 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:06 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:06.931+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:16:07.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:06 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:06.931+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:16:07.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:06 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:06.931+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:16:07.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:06 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:06.931+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.263880+0000 front 2024-09-21T04:15:22.263849+0000 (oldest deadline 2024-09-21T04:15:43.963954+0000) 2024-09-21T04:16:07.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:06 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:06.931+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:27.365613+0000 front 2024-09-21T04:15:27.365561+0000 (oldest deadline 2024-09-21T04:15:52.665449+0000) 2024-09-21T04:16:07.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:06 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:06.877+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:16:07.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:06 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:06.877+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:16:07.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:06 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:06.877+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:16:07.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:06 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:06.877+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:16:07.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:06 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:06.877+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:38.677868+0000 front 2024-09-21T04:15:38.677787+0000 (oldest deadline 2024-09-21T04:16:04.577960+0000) 2024-09-21T04:16:07.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:07 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:07.404+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:16:07.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:07 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:07.404+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:16:07.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:07 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:07.404+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:16:07.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:07 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:07.404+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:16:07.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:07 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:07.404+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:29.012946+0000 front 2024-09-21T04:15:29.012996+0000 (oldest deadline 2024-09-21T04:15:51.313104+0000) 2024-09-21T04:16:07.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:07 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:07.404+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:36.315769+0000 front 2024-09-21T04:15:36.315739+0000 (oldest deadline 2024-09-21T04:16:02.215767+0000) 2024-09-21T04:16:08.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:07 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:07.883+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:16:08.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:07 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:07.883+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:16:08.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:07 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:07.883+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:16:08.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:07 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:07.883+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.263880+0000 front 2024-09-21T04:15:22.263849+0000 (oldest deadline 2024-09-21T04:15:43.963954+0000) 2024-09-21T04:16:08.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:07 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:07.883+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:27.365613+0000 front 2024-09-21T04:15:27.365561+0000 (oldest deadline 2024-09-21T04:15:52.665449+0000) 2024-09-21T04:16:08.303 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:07 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:07.915+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:16:08.303 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:07 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:07.915+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:16:08.303 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:07 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:07.915+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:16:08.303 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:07 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:07.915+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:16:08.303 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:07 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:07.915+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:38.677868+0000 front 2024-09-21T04:15:38.677787+0000 (oldest deadline 2024-09-21T04:16:04.577960+0000) 2024-09-21T04:16:08.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:08 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:08.411+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:16:08.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:08 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:08.411+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:16:08.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:08 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:08.411+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:16:08.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:08 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:08.411+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:16:08.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:08 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:08.411+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:29.012946+0000 front 2024-09-21T04:15:29.012996+0000 (oldest deadline 2024-09-21T04:15:51.313104+0000) 2024-09-21T04:16:08.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:08 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:08.411+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:36.315769+0000 front 2024-09-21T04:15:36.315739+0000 (oldest deadline 2024-09-21T04:16:02.215767+0000) 2024-09-21T04:16:09.261 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:08 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:08.887+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:16:09.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:08 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:08.887+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:16:09.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:08 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:08.887+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:16:09.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:08 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:08.887+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.263880+0000 front 2024-09-21T04:15:22.263849+0000 (oldest deadline 2024-09-21T04:15:43.963954+0000) 2024-09-21T04:16:09.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:08 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:08.887+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:27.365613+0000 front 2024-09-21T04:15:27.365561+0000 (oldest deadline 2024-09-21T04:15:52.665449+0000) 2024-09-21T04:16:09.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:08 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:08.887+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-21T04:15:43.266792+0000 front 2024-09-21T04:15:43.266830+0000 (oldest deadline 2024-09-21T04:16:08.566838+0000) 2024-09-21T04:16:09.303 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:08 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:08.891+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:16:09.303 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:08 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:08.891+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:16:09.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:08 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:08.891+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:16:09.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:08 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:08.891+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:16:09.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:08 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:08.891+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:38.677868+0000 front 2024-09-21T04:15:38.677787+0000 (oldest deadline 2024-09-21T04:16:04.577960+0000) 2024-09-21T04:16:09.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:09 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:09.436+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:16:09.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:09 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:09.436+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:16:09.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:09 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:09.436+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:16:09.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:09 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:09.436+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:16:09.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:09 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:09.436+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:29.012946+0000 front 2024-09-21T04:15:29.012996+0000 (oldest deadline 2024-09-21T04:15:51.313104+0000) 2024-09-21T04:16:09.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:09 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:09.436+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:36.315769+0000 front 2024-09-21T04:15:36.315739+0000 (oldest deadline 2024-09-21T04:16:02.215767+0000) 2024-09-21T04:16:09.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:09 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:09.436+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-21T04:15:46.316613+0000 front 2024-09-21T04:15:46.316688+0000 (oldest deadline 2024-09-21T04:16:09.216504+0000) 2024-09-21T04:16:10.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:09 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:09.844+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.659837+0000 front 2024-09-21T04:14:57.659940+0000 (oldest deadline 2024-09-21T04:15:21.760061+0000) 2024-09-21T04:16:10.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:09 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:09.844+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:05.261712+0000 front 2024-09-21T04:15:05.261700+0000 (oldest deadline 2024-09-21T04:15:29.361443+0000) 2024-09-21T04:16:10.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:09 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:09.844+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:12.862746+0000 front 2024-09-21T04:15:12.862845+0000 (oldest deadline 2024-09-21T04:15:37.562800+0000) 2024-09-21T04:16:10.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:09 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:09.844+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:22.263880+0000 front 2024-09-21T04:15:22.263849+0000 (oldest deadline 2024-09-21T04:15:43.963954+0000) 2024-09-21T04:16:10.262 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:09 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:09.844+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:27.365613+0000 front 2024-09-21T04:15:27.365561+0000 (oldest deadline 2024-09-21T04:15:52.665449+0000) 2024-09-21T04:16:10.263 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:09 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9[64429]: 2024-09-21T04:16:09.844+0000 7f8c2cf27640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-21T04:15:43.266792+0000 front 2024-09-21T04:15:43.266830+0000 (oldest deadline 2024-09-21T04:16:08.566838+0000) 2024-09-21T04:16:10.303 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:09 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:09.863+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:16:10.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:09 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:09.863+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:16:10.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:09 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:09.863+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:16:10.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:09 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:09.863+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:16:10.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:09 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:09.863+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:38.677868+0000 front 2024-09-21T04:15:38.677787+0000 (oldest deadline 2024-09-21T04:16:04.577960+0000) 2024-09-21T04:16:10.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:10 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:10.402+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:16:10.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:10 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:10.402+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:16:10.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:10 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:10.402+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:16:10.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:10 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:10.402+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:16:10.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:10 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:10.402+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:29.012946+0000 front 2024-09-21T04:15:29.012996+0000 (oldest deadline 2024-09-21T04:15:51.313104+0000) 2024-09-21T04:16:10.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:10 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:10.402+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:36.315769+0000 front 2024-09-21T04:15:36.315739+0000 (oldest deadline 2024-09-21T04:16:02.215767+0000) 2024-09-21T04:16:10.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:10 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:10.402+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-21T04:15:46.316613+0000 front 2024-09-21T04:15:46.316688+0000 (oldest deadline 2024-09-21T04:16:09.216504+0000) 2024-09-21T04:16:11.303 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:10 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:10.870+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:16:11.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:10 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:10.870+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:16:11.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:10 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:10.870+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:16:11.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:10 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:10.870+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:16:11.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:10 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:10.870+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:38.677868+0000 front 2024-09-21T04:15:38.677787+0000 (oldest deadline 2024-09-21T04:16:04.577960+0000) 2024-09-21T04:16:11.407 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:11 smithi064 podman[104019]: 2024-09-21 04:16:11.072037506 +0000 UTC m=+5.510362826 container died 3b01c43db45e4594d732291c6e6aec2bdf399a8de1aed19bb711599892fba309 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-21T04:16:11.681 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:11 smithi064 podman[104019]: 2024-09-21 04:16:11.405508466 +0000 UTC m=+5.843833785 container cleanup 3b01c43db45e4594d732291c6e6aec2bdf399a8de1aed19bb711599892fba309 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-21T04:16:11.681 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:11 smithi064 bash[104019]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9 2024-09-21T04:16:11.681 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:11 smithi064 podman[104036]: 2024-09-21 04:16:11.615477569 +0000 UTC m=+0.532345996 container remove 3b01c43db45e4594d732291c6e6aec2bdf399a8de1aed19bb711599892fba309 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:16:11.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:11 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:11.407+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:16:11.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:11 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:11.407+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:16:11.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:11 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:11.407+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:16:11.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:11 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:11.407+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:16:11.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:11 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:11.407+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:29.012946+0000 front 2024-09-21T04:15:29.012996+0000 (oldest deadline 2024-09-21T04:15:51.313104+0000) 2024-09-21T04:16:11.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:11 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:11.407+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:36.315769+0000 front 2024-09-21T04:15:36.315739+0000 (oldest deadline 2024-09-21T04:16:02.215767+0000) 2024-09-21T04:16:11.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:11 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:11.407+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-21T04:15:46.316613+0000 front 2024-09-21T04:15:46.316688+0000 (oldest deadline 2024-09-21T04:16:09.216504+0000) 2024-09-21T04:16:12.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:11 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:11.907+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:16:12.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:11 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:11.907+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:16:12.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:11 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:11.907+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:16:12.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:11 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:11.907+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:16:12.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:11 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:11.907+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:38.677868+0000 front 2024-09-21T04:15:38.677787+0000 (oldest deadline 2024-09-21T04:16:04.577960+0000) 2024-09-21T04:16:12.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:11 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:11.907+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-21T04:15:46.279273+0000 front 2024-09-21T04:15:46.279181+0000 (oldest deadline 2024-09-21T04:16:11.579013+0000) 2024-09-21T04:16:12.370 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:12 smithi064 podman[104131]: 2024-09-21 04:16:11.960515161 +0000 UTC m=+0.016630756 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:16:12.371 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:12 smithi064 podman[104131]: 2024-09-21 04:16:12.05798425 +0000 UTC m=+0.114099829 container create 9302c96d350dbddab27fa6fdeb3386eaba05e96199b716421f26dac835f4ad53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9-deactivate, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:16:12.371 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:12 smithi064 podman[104131]: 2024-09-21 04:16:12.276995101 +0000 UTC m=+0.333110688 container init 9302c96d350dbddab27fa6fdeb3386eaba05e96199b716421f26dac835f4ad53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T04:16:12.371 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:12 smithi064 podman[104131]: 2024-09-21 04:16:12.280392642 +0000 UTC m=+0.336508222 container start 9302c96d350dbddab27fa6fdeb3386eaba05e96199b716421f26dac835f4ad53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:16:12.371 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:12 smithi064 podman[104131]: 2024-09-21 04:16:12.368608955 +0000 UTC m=+0.424724543 container attach 9302c96d350dbddab27fa6fdeb3386eaba05e96199b716421f26dac835f4ad53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-21T04:16:12.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:12 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:12.368+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:16:12.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:12 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:12.368+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:16:12.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:12 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:12.368+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:16:12.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:12 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:12.368+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:16:12.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:12 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:12.368+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:29.012946+0000 front 2024-09-21T04:15:29.012996+0000 (oldest deadline 2024-09-21T04:15:51.313104+0000) 2024-09-21T04:16:12.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:12 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:12.368+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:36.315769+0000 front 2024-09-21T04:15:36.315739+0000 (oldest deadline 2024-09-21T04:16:02.215767+0000) 2024-09-21T04:16:12.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:12 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:12.368+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-21T04:15:46.316613+0000 front 2024-09-21T04:15:46.316688+0000 (oldest deadline 2024-09-21T04:16:09.216504+0000) 2024-09-21T04:16:12.762 INFO:journalctl@ceph.osd.9.smithi064.stdout:Sep 21 04:16:12 smithi064 podman[104131]: 2024-09-21 04:16:12.488751559 +0000 UTC m=+0.544867149 container died 9302c96d350dbddab27fa6fdeb3386eaba05e96199b716421f26dac835f4ad53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-21T04:16:13.076 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.9.service' 2024-09-21T04:16:13.117 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:16:13.117 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-21T04:16:13.118 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-21T04:16:13.118 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.10 2024-09-21T04:16:13.303 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:12 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:12.904+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:16:13.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:12 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:12.904+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:16:13.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:12 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:12.904+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:16:13.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:12 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:12.904+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:16:13.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:12 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:12.904+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:38.677868+0000 front 2024-09-21T04:15:38.677787+0000 (oldest deadline 2024-09-21T04:16:04.577960+0000) 2024-09-21T04:16:13.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:12 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:12.904+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-21T04:15:46.279273+0000 front 2024-09-21T04:15:46.279181+0000 (oldest deadline 2024-09-21T04:16:11.579013+0000) 2024-09-21T04:16:13.416 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:13 smithi119 systemd[1]: Stopping Ceph osd.10 for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:16:13.416 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:13 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:13.357+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:16:13.416 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:13 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:13.357+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:16:13.416 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:13 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:13.357+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:16:13.416 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:13 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:13.357+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:16:13.416 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:13 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:13.357+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:29.012946+0000 front 2024-09-21T04:15:29.012996+0000 (oldest deadline 2024-09-21T04:15:51.313104+0000) 2024-09-21T04:16:13.417 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:13 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:13.357+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:36.315769+0000 front 2024-09-21T04:15:36.315739+0000 (oldest deadline 2024-09-21T04:16:02.215767+0000) 2024-09-21T04:16:13.417 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:13 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:13.357+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-21T04:15:46.316613+0000 front 2024-09-21T04:15:46.316688+0000 (oldest deadline 2024-09-21T04:16:09.216504+0000) 2024-09-21T04:16:13.417 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:13 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:13.413+0000 7f52d82ba640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:16:13.417 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:13 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:13.413+0000 7f52d82ba640 -1 osd.10 90 *** Got signal Terminated *** 2024-09-21T04:16:13.417 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:13 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:13.413+0000 7f52d82ba640 -1 osd.10 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:16:14.303 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:13 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:13.919+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:16:14.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:13 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:13.919+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:16:14.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:13 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:13.919+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:16:14.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:13 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:13.919+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:16:14.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:13 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:13.919+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:38.677868+0000 front 2024-09-21T04:15:38.677787+0000 (oldest deadline 2024-09-21T04:16:04.577960+0000) 2024-09-21T04:16:14.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:13 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:13.919+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-21T04:15:46.279273+0000 front 2024-09-21T04:15:46.279181+0000 (oldest deadline 2024-09-21T04:16:11.579013+0000) 2024-09-21T04:16:14.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:14 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:14.382+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:16:14.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:14 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:14.382+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:16:14.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:14 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:14.382+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:16:14.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:14 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:14.382+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:16:14.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:14 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:14.382+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:29.012946+0000 front 2024-09-21T04:15:29.012996+0000 (oldest deadline 2024-09-21T04:15:51.313104+0000) 2024-09-21T04:16:14.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:14 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:14.382+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:36.315769+0000 front 2024-09-21T04:15:36.315739+0000 (oldest deadline 2024-09-21T04:16:02.215767+0000) 2024-09-21T04:16:14.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:14 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:14.382+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-21T04:15:46.316613+0000 front 2024-09-21T04:15:46.316688+0000 (oldest deadline 2024-09-21T04:16:09.216504+0000) 2024-09-21T04:16:15.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:14 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:14.906+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:16:15.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:14 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:14.906+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:16:15.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:14 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:14.906+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:16:15.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:14 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:14.906+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:16:15.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:14 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:14.906+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:38.677868+0000 front 2024-09-21T04:15:38.677787+0000 (oldest deadline 2024-09-21T04:16:04.577960+0000) 2024-09-21T04:16:15.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:14 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:14.906+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-21T04:15:46.279273+0000 front 2024-09-21T04:15:46.279181+0000 (oldest deadline 2024-09-21T04:16:11.579013+0000) 2024-09-21T04:16:15.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:15 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:15.363+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:16:15.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:15 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:15.363+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:16:15.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:15 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:15.363+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:16:15.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:15 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:15.363+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:16:15.758 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:15 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:15.363+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:29.012946+0000 front 2024-09-21T04:15:29.012996+0000 (oldest deadline 2024-09-21T04:15:51.313104+0000) 2024-09-21T04:16:15.758 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:15 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:15.363+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:36.315769+0000 front 2024-09-21T04:15:36.315739+0000 (oldest deadline 2024-09-21T04:16:02.215767+0000) 2024-09-21T04:16:15.758 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:15 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:15.363+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-21T04:15:46.316613+0000 front 2024-09-21T04:15:46.316688+0000 (oldest deadline 2024-09-21T04:16:09.216504+0000) 2024-09-21T04:16:16.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:15 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:15.911+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:16:16.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:15 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:15.911+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:16:16.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:15 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:15.911+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:16:16.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:15 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:15.911+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:16:16.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:15 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:15.911+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:38.677868+0000 front 2024-09-21T04:15:38.677787+0000 (oldest deadline 2024-09-21T04:16:04.577960+0000) 2024-09-21T04:16:16.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:15 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:15.911+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-21T04:15:46.279273+0000 front 2024-09-21T04:15:46.279181+0000 (oldest deadline 2024-09-21T04:16:11.579013+0000) 2024-09-21T04:16:16.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:16 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:16.350+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:16:16.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:16 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:16.350+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:16:16.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:16 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:16.350+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:16:16.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:16 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:16.350+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:16:16.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:16 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:16.350+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:29.012946+0000 front 2024-09-21T04:15:29.012996+0000 (oldest deadline 2024-09-21T04:15:51.313104+0000) 2024-09-21T04:16:16.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:16 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:16.350+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:36.315769+0000 front 2024-09-21T04:15:36.315739+0000 (oldest deadline 2024-09-21T04:16:02.215767+0000) 2024-09-21T04:16:16.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:16 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:16.350+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-21T04:15:46.316613+0000 front 2024-09-21T04:15:46.316688+0000 (oldest deadline 2024-09-21T04:16:09.216504+0000) 2024-09-21T04:16:17.303 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:16 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:16.900+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:16:17.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:16 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:16.900+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:16:17.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:16 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:16.900+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:16:17.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:16 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:16.900+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:16:17.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:16 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:16.900+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:38.677868+0000 front 2024-09-21T04:15:38.677787+0000 (oldest deadline 2024-09-21T04:16:04.577960+0000) 2024-09-21T04:16:17.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:16 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:16.900+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-21T04:15:46.279273+0000 front 2024-09-21T04:15:46.279181+0000 (oldest deadline 2024-09-21T04:16:11.579013+0000) 2024-09-21T04:16:17.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:16 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:16.900+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-09-21T04:15:51.579433+0000 front 2024-09-21T04:15:51.579535+0000 (oldest deadline 2024-09-21T04:16:16.879418+0000) 2024-09-21T04:16:17.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:17 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:17.350+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:16:17.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:17 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:17.350+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:16:17.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:17 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:17.350+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:16:17.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:17 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:17.350+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:16:17.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:17 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:17.350+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:29.012946+0000 front 2024-09-21T04:15:29.012996+0000 (oldest deadline 2024-09-21T04:15:51.313104+0000) 2024-09-21T04:16:17.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:17 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:17.350+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:36.315769+0000 front 2024-09-21T04:15:36.315739+0000 (oldest deadline 2024-09-21T04:16:02.215767+0000) 2024-09-21T04:16:17.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:17 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:17.350+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-21T04:15:46.316613+0000 front 2024-09-21T04:15:46.316688+0000 (oldest deadline 2024-09-21T04:16:09.216504+0000) 2024-09-21T04:16:18.303 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:17 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:17.933+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:16:18.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:17 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:17.933+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:16:18.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:17 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:17.933+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:16:18.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:17 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:17.933+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:16:18.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:17 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:17.933+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:38.677868+0000 front 2024-09-21T04:15:38.677787+0000 (oldest deadline 2024-09-21T04:16:04.577960+0000) 2024-09-21T04:16:18.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:17 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:17.933+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-21T04:15:46.279273+0000 front 2024-09-21T04:15:46.279181+0000 (oldest deadline 2024-09-21T04:16:11.579013+0000) 2024-09-21T04:16:18.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:17 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:17.933+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-09-21T04:15:51.579433+0000 front 2024-09-21T04:15:51.579535+0000 (oldest deadline 2024-09-21T04:16:16.879418+0000) 2024-09-21T04:16:18.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:18 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:18.374+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:55.609173+0000 front 2024-09-21T04:14:55.609078+0000 (oldest deadline 2024-09-21T04:15:20.909048+0000) 2024-09-21T04:16:18.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:18 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:18.374+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:04.310245+0000 front 2024-09-21T04:15:04.310162+0000 (oldest deadline 2024-09-21T04:15:28.410193+0000) 2024-09-21T04:16:18.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:18 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:18.374+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.711514+0000 front 2024-09-21T04:15:13.711605+0000 (oldest deadline 2024-09-21T04:15:37.811452+0000) 2024-09-21T04:16:18.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:18 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:18.374+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6814 osd.3 since back 2024-09-21T04:15:17.811941+0000 front 2024-09-21T04:15:17.811920+0000 (oldest deadline 2024-09-21T04:15:43.112086+0000) 2024-09-21T04:16:18.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:18 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:18.374+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:29.012946+0000 front 2024-09-21T04:15:29.012996+0000 (oldest deadline 2024-09-21T04:15:51.313104+0000) 2024-09-21T04:16:18.757 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:18 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:18.374+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:36.315769+0000 front 2024-09-21T04:15:36.315739+0000 (oldest deadline 2024-09-21T04:16:02.215767+0000) 2024-09-21T04:16:18.758 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:18 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10[59269]: 2024-09-21T04:16:18.374+0000 7f52d48a5640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-21T04:15:46.316613+0000 front 2024-09-21T04:15:46.316688+0000 (oldest deadline 2024-09-21T04:16:09.216504+0000) 2024-09-21T04:16:19.181 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:18 smithi119 podman[68524]: 2024-09-21 04:16:18.875396888 +0000 UTC m=+5.564530106 container died 884cd453c247c0f08e5a30d3b5769071fe458cddff0a81280c373e97f7d5b296 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-21T04:16:19.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:18 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:18.932+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:16:19.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:18 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:18.932+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:16:19.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:18 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:18.932+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:16:19.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:18 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:18.932+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:16:19.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:18 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:18.932+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:38.677868+0000 front 2024-09-21T04:15:38.677787+0000 (oldest deadline 2024-09-21T04:16:04.577960+0000) 2024-09-21T04:16:19.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:18 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:18.932+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-21T04:15:46.279273+0000 front 2024-09-21T04:15:46.279181+0000 (oldest deadline 2024-09-21T04:16:11.579013+0000) 2024-09-21T04:16:19.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:18 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:18.932+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-09-21T04:15:51.579433+0000 front 2024-09-21T04:15:51.579535+0000 (oldest deadline 2024-09-21T04:16:16.879418+0000) 2024-09-21T04:16:19.441 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:19 smithi119 podman[68524]: 2024-09-21 04:16:19.179846315 +0000 UTC m=+5.868979578 container cleanup 884cd453c247c0f08e5a30d3b5769071fe458cddff0a81280c373e97f7d5b296 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-21T04:16:19.441 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:19 smithi119 bash[68524]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10 2024-09-21T04:16:19.745 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:19 smithi119 podman[68539]: 2024-09-21 04:16:19.439639773 +0000 UTC m=+0.542328102 container remove 884cd453c247c0f08e5a30d3b5769071fe458cddff0a81280c373e97f7d5b296 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-21T04:16:20.007 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:19 smithi119 podman[68630]: 2024-09-21 04:16:19.64619849 +0000 UTC m=+0.016067444 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:16:20.007 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:19 smithi119 podman[68630]: 2024-09-21 04:16:19.769639472 +0000 UTC m=+0.139508415 container create fa004c67df0b14e33aca68158eff66bc09c8a6686f132796997b1820e0fb04fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T04:16:20.007 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:19 smithi119 podman[68630]: 2024-09-21 04:16:19.942727684 +0000 UTC m=+0.312596632 container init fa004c67df0b14e33aca68158eff66bc09c8a6686f132796997b1820e0fb04fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T04:16:20.007 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:19 smithi119 podman[68630]: 2024-09-21 04:16:19.946042102 +0000 UTC m=+0.315911049 container start fa004c67df0b14e33aca68158eff66bc09c8a6686f132796997b1820e0fb04fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2) 2024-09-21T04:16:20.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:19 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:19.927+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:16:20.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:19 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:19.927+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:16:20.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:19 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:19.927+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:16:20.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:19 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:19.927+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:16:20.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:19 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:19.927+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:38.677868+0000 front 2024-09-21T04:15:38.677787+0000 (oldest deadline 2024-09-21T04:16:04.577960+0000) 2024-09-21T04:16:20.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:19 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:19.927+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-21T04:15:46.279273+0000 front 2024-09-21T04:15:46.279181+0000 (oldest deadline 2024-09-21T04:16:11.579013+0000) 2024-09-21T04:16:20.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:19 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:19.927+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-09-21T04:15:51.579433+0000 front 2024-09-21T04:15:51.579535+0000 (oldest deadline 2024-09-21T04:16:16.879418+0000) 2024-09-21T04:16:20.414 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:20 smithi119 podman[68630]: 2024-09-21 04:16:20.020857162 +0000 UTC m=+0.390726111 container attach fa004c67df0b14e33aca68158eff66bc09c8a6686f132796997b1820e0fb04fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T04:16:20.414 INFO:journalctl@ceph.osd.10.smithi119.stdout:Sep 21 04:16:20 smithi119 podman[68630]: 2024-09-21 04:16:20.136373998 +0000 UTC m=+0.506242942 container died fa004c67df0b14e33aca68158eff66bc09c8a6686f132796997b1820e0fb04fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-10-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-21T04:16:20.689 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.10.service' 2024-09-21T04:16:20.728 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:16:20.729 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-21T04:16:20.729 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-21T04:16:20.729 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.11 2024-09-21T04:16:21.043 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:20 smithi131 systemd[1]: Stopping Ceph osd.11 for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:16:21.043 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:20 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:20.935+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:16:21.043 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:20 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:20.935+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:16:21.043 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:20 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:20.935+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:16:21.043 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:20 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:20.935+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:16:21.043 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:20 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:20.935+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:38.677868+0000 front 2024-09-21T04:15:38.677787+0000 (oldest deadline 2024-09-21T04:16:04.577960+0000) 2024-09-21T04:16:21.043 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:20 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:20.935+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-21T04:15:46.279273+0000 front 2024-09-21T04:15:46.279181+0000 (oldest deadline 2024-09-21T04:16:11.579013+0000) 2024-09-21T04:16:21.043 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:20 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:20.935+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-09-21T04:15:51.579433+0000 front 2024-09-21T04:15:51.579535+0000 (oldest deadline 2024-09-21T04:16:16.879418+0000) 2024-09-21T04:16:21.043 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:21 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:21.039+0000 7f6540add640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:16:21.043 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:21 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:21.039+0000 7f6540add640 -1 osd.11 90 *** Got signal Terminated *** 2024-09-21T04:16:21.043 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:21 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:21.039+0000 7f6540add640 -1 osd.11 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T04:16:22.257 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:21 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:21.907+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:16:22.257 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:21 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:21.907+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:16:22.257 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:21 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:21.907+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:16:22.257 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:21 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:21.907+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:16:22.257 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:21 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:21.907+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:38.677868+0000 front 2024-09-21T04:15:38.677787+0000 (oldest deadline 2024-09-21T04:16:04.577960+0000) 2024-09-21T04:16:22.257 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:21 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:21.907+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-21T04:15:46.279273+0000 front 2024-09-21T04:15:46.279181+0000 (oldest deadline 2024-09-21T04:16:11.579013+0000) 2024-09-21T04:16:22.257 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:21 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:21.907+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-09-21T04:15:51.579433+0000 front 2024-09-21T04:15:51.579535+0000 (oldest deadline 2024-09-21T04:16:16.879418+0000) 2024-09-21T04:16:23.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:22 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:22.950+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:16:23.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:22 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:22.950+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:16:23.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:22 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:22.950+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:16:23.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:22 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:22.950+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:16:23.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:22 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:22.950+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:38.677868+0000 front 2024-09-21T04:15:38.677787+0000 (oldest deadline 2024-09-21T04:16:04.577960+0000) 2024-09-21T04:16:23.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:22 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:22.950+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-21T04:15:46.279273+0000 front 2024-09-21T04:15:46.279181+0000 (oldest deadline 2024-09-21T04:16:11.579013+0000) 2024-09-21T04:16:23.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:22 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:22.950+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-09-21T04:15:51.579433+0000 front 2024-09-21T04:15:51.579535+0000 (oldest deadline 2024-09-21T04:16:16.879418+0000) 2024-09-21T04:16:24.306 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:23 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:23.996+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:16:24.306 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:23 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:23.996+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:16:24.306 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:23 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:23.996+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:16:24.307 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:23 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:23.996+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:16:24.307 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:23 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:23.996+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:38.677868+0000 front 2024-09-21T04:15:38.677787+0000 (oldest deadline 2024-09-21T04:16:04.577960+0000) 2024-09-21T04:16:24.307 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:23 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:23.996+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-21T04:15:46.279273+0000 front 2024-09-21T04:15:46.279181+0000 (oldest deadline 2024-09-21T04:16:11.579013+0000) 2024-09-21T04:16:24.307 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:23 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:23.996+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-09-21T04:15:51.579433+0000 front 2024-09-21T04:15:51.579535+0000 (oldest deadline 2024-09-21T04:16:16.879418+0000) 2024-09-21T04:16:24.307 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:23 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:23.996+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6820 osd.8 since back 2024-09-21T04:16:00.380296+0000 front 2024-09-21T04:16:00.380336+0000 (oldest deadline 2024-09-21T04:16:23.880387+0000) 2024-09-21T04:16:25.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:24 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:24.988+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:16:25.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:24 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:24.988+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:16:25.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:24 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:24.988+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:16:25.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:24 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:24.988+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:16:25.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:24 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:24.988+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:38.677868+0000 front 2024-09-21T04:15:38.677787+0000 (oldest deadline 2024-09-21T04:16:04.577960+0000) 2024-09-21T04:16:25.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:24 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:24.988+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-21T04:15:46.279273+0000 front 2024-09-21T04:15:46.279181+0000 (oldest deadline 2024-09-21T04:16:11.579013+0000) 2024-09-21T04:16:25.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:24 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:24.988+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-09-21T04:15:51.579433+0000 front 2024-09-21T04:15:51.579535+0000 (oldest deadline 2024-09-21T04:16:16.879418+0000) 2024-09-21T04:16:25.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:24 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:24.988+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6820 osd.8 since back 2024-09-21T04:16:00.380296+0000 front 2024-09-21T04:16:00.380336+0000 (oldest deadline 2024-09-21T04:16:23.880387+0000) 2024-09-21T04:16:26.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:25 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:25.961+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6806 osd.0 since back 2024-09-21T04:14:57.469933+0000 front 2024-09-21T04:14:57.469906+0000 (oldest deadline 2024-09-21T04:15:22.770087+0000) 2024-09-21T04:16:26.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:25 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:25.961+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6804 osd.1 since back 2024-09-21T04:15:03.871530+0000 front 2024-09-21T04:15:03.871546+0000 (oldest deadline 2024-09-21T04:15:27.971360+0000) 2024-09-21T04:16:26.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:25 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:25.961+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6804 osd.2 since back 2024-09-21T04:15:13.772989+0000 front 2024-09-21T04:15:13.773065+0000 (oldest deadline 2024-09-21T04:15:36.073090+0000) 2024-09-21T04:16:26.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:25 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:25.961+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6812 osd.4 since back 2024-09-21T04:15:30.476549+0000 front 2024-09-21T04:15:30.476569+0000 (oldest deadline 2024-09-21T04:15:54.576706+0000) 2024-09-21T04:16:26.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:25 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:25.961+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6812 osd.5 since back 2024-09-21T04:15:38.677868+0000 front 2024-09-21T04:15:38.677787+0000 (oldest deadline 2024-09-21T04:16:04.577960+0000) 2024-09-21T04:16:26.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:25 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:25.961+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.64:6822 osd.6 since back 2024-09-21T04:15:46.279273+0000 front 2024-09-21T04:15:46.279181+0000 (oldest deadline 2024-09-21T04:16:11.579013+0000) 2024-09-21T04:16:26.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:25 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:25.961+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.119:6820 osd.7 since back 2024-09-21T04:15:51.579433+0000 front 2024-09-21T04:15:51.579535+0000 (oldest deadline 2024-09-21T04:16:16.879418+0000) 2024-09-21T04:16:26.304 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:25 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11[59440]: 2024-09-21T04:16:25.961+0000 7f65390c0640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.131:6820 osd.8 since back 2024-09-21T04:16:00.380296+0000 front 2024-09-21T04:16:00.380336+0000 (oldest deadline 2024-09-21T04:16:23.880387+0000) 2024-09-21T04:16:26.725 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:26 smithi131 podman[69374]: 2024-09-21 04:16:26.45757466 +0000 UTC m=+5.530337282 container died 73777c77b21faeb6332147c9d2c1840adf73cd37f81f4e22de9b30472959338b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:16:27.053 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:26 smithi131 podman[69374]: 2024-09-21 04:16:26.822641541 +0000 UTC m=+5.895404153 container cleanup 73777c77b21faeb6332147c9d2c1840adf73cd37f81f4e22de9b30472959338b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-21T04:16:27.054 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:26 smithi131 bash[69374]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11 2024-09-21T04:16:27.438 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:27 smithi131 podman[69405]: 2024-09-21 04:16:27.082425423 +0000 UTC m=+0.623104427 container remove 73777c77b21faeb6332147c9d2c1840adf73cd37f81f4e22de9b30472959338b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:16:27.704 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:27 smithi131 podman[69492]: 2024-09-21 04:16:27.339375527 +0000 UTC m=+0.016508329 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T04:16:27.704 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:27 smithi131 podman[69492]: 2024-09-21 04:16:27.441483651 +0000 UTC m=+0.118616447 container create 3e26ecd7ea15657ed2210945b4d5dc6bd84b0c39863861439f1a6154adf47ceb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:16:27.704 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:27 smithi131 podman[69492]: 2024-09-21 04:16:27.702256716 +0000 UTC m=+0.379389513 container init 3e26ecd7ea15657ed2210945b4d5dc6bd84b0c39863861439f1a6154adf47ceb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:16:28.054 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:27 smithi131 podman[69492]: 2024-09-21 04:16:27.705590641 +0000 UTC m=+0.382723432 container start 3e26ecd7ea15657ed2210945b4d5dc6bd84b0c39863861439f1a6154adf47ceb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-21T04:16:28.054 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:27 smithi131 podman[69492]: 2024-09-21 04:16:27.760598839 +0000 UTC m=+0.437731644 container attach 3e26ecd7ea15657ed2210945b4d5dc6bd84b0c39863861439f1a6154adf47ceb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-21T04:16:28.054 INFO:journalctl@ceph.osd.11.smithi131.stdout:Sep 21 04:16:27 smithi131 podman[69492]: 2024-09-21 04:16:27.899987158 +0000 UTC m=+0.577119947 container died 3e26ecd7ea15657ed2210945b4d5dc6bd84b0c39863861439f1a6154adf47ceb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-osd-11-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-21T04:16:28.470 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@osd.11.service' 2024-09-21T04:16:28.508 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:16:28.508 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-21T04:16:28.509 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-21T04:16:28.509 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.a 2024-09-21T04:16:28.802 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 21 04:16:28 smithi064 systemd[1]: Stopping Ceph mds.a for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:16:29.261 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 21 04:16:28 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-a[70976]: 2024-09-21T04:16:28.799+0000 7f7e302b5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:16:29.262 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 21 04:16:28 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-a[70976]: 2024-09-21T04:16:28.799+0000 7f7e302b5640 -1 mds.a *** got signal Terminated *** 2024-09-21T04:16:30.418 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 21 04:16:30 smithi064 podman[104235]: 2024-09-21 04:16:30.150462583 +0000 UTC m=+1.445956720 container died daa9ea597ce9a7b302261133f4aec2988545f05f14a15aab63d6c3ae2f32414f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-21T04:16:30.762 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 21 04:16:30 smithi064 podman[104235]: 2024-09-21 04:16:30.498702137 +0000 UTC m=+1.794196273 container cleanup daa9ea597ce9a7b302261133f4aec2988545f05f14a15aab63d6c3ae2f32414f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-21T04:16:30.762 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 21 04:16:30 smithi064 bash[104235]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-a 2024-09-21T04:16:31.071 INFO:journalctl@ceph.mds.a.smithi064.stdout:Sep 21 04:16:30 smithi064 podman[104248]: 2024-09-21 04:16:30.800009123 +0000 UTC m=+0.644338150 container remove daa9ea597ce9a7b302261133f4aec2988545f05f14a15aab63d6c3ae2f32414f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-21T04:16:31.085 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.a.service' 2024-09-21T04:16:31.120 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:16:31.120 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-21T04:16:31.121 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-21T04:16:31.121 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.d 2024-09-21T04:16:31.512 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 21 04:16:31 smithi064 systemd[1]: Stopping Ceph mds.d for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:16:32.011 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 21 04:16:31 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-d[69864]: 2024-09-21T04:16:31.568+0000 7fcbfbb9a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:16:32.012 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 21 04:16:31 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-d[69864]: 2024-09-21T04:16:31.569+0000 7fcbfbb9a640 -1 mds.d *** got signal Terminated *** 2024-09-21T04:16:35.955 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 21 04:16:35 smithi064 podman[104357]: 2024-09-21 04:16:35.584524673 +0000 UTC m=+4.235046957 container died 8e4f2b7a7e88790a1ca6a3e85554c4cab77aa2af19b3c49e16df3c14e978d1f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:16:36.261 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 21 04:16:36 smithi064 podman[104357]: 2024-09-21 04:16:36.094364711 +0000 UTC m=+4.744887031 container cleanup 8e4f2b7a7e88790a1ca6a3e85554c4cab77aa2af19b3c49e16df3c14e978d1f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T04:16:36.262 INFO:journalctl@ceph.mds.d.smithi064.stdout:Sep 21 04:16:36 smithi064 bash[104357]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-d 2024-09-21T04:16:36.435 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.d.service' 2024-09-21T04:16:36.468 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:16:36.468 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-21T04:16:36.468 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-21T04:16:36.469 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.g 2024-09-21T04:16:37.012 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 21 04:16:36 smithi064 systemd[1]: Stopping Ceph mds.g for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:16:37.012 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 21 04:16:36 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-g[70593]: 2024-09-21T04:16:36.797+0000 7f6f609f3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:16:37.012 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 21 04:16:36 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-g[70593]: 2024-09-21T04:16:36.797+0000 7f6f609f3640 -1 mds.g *** got signal Terminated *** 2024-09-21T04:16:38.992 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 21 04:16:38 smithi064 podman[104478]: 2024-09-21 04:16:38.717515537 +0000 UTC m=+2.017831855 container died 3bedb67a4c32988bde7c7ce75b2ec95a1ab765efebfe145f122cdb3bdaa6fa08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-g, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T04:16:39.649 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 21 04:16:39 smithi064 podman[104478]: 2024-09-21 04:16:39.300260219 +0000 UTC m=+2.600576531 container cleanup 3bedb67a4c32988bde7c7ce75b2ec95a1ab765efebfe145f122cdb3bdaa6fa08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-g, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-09-21T04:16:39.649 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 21 04:16:39 smithi064 bash[104478]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-g 2024-09-21T04:16:39.649 INFO:journalctl@ceph.mds.g.smithi064.stdout:Sep 21 04:16:39 smithi064 podman[104490]: 2024-09-21 04:16:39.524612868 +0000 UTC m=+0.802959701 container remove 3bedb67a4c32988bde7c7ce75b2ec95a1ab765efebfe145f122cdb3bdaa6fa08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0) 2024-09-21T04:16:39.664 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.g.service' 2024-09-21T04:16:39.697 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:16:39.697 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-21T04:16:39.697 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-21T04:16:39.697 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.j 2024-09-21T04:16:40.012 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 21 04:16:39 smithi064 systemd[1]: Stopping Ceph mds.j for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:16:40.511 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 21 04:16:40 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-j[69491]: 2024-09-21T04:16:40.035+0000 7f5bbd1b0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:16:40.512 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 21 04:16:40 smithi064 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-j[69491]: 2024-09-21T04:16:40.035+0000 7f5bbd1b0640 -1 mds.j *** got signal Terminated *** 2024-09-21T04:16:44.313 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 21 04:16:44 smithi064 podman[104598]: 2024-09-21 04:16:44.046728475 +0000 UTC m=+4.118721363 container died da8ece3c8179b88450bbb2cbc8fc851714fd2f98f080306f9d78bcb66f8a3d5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-j, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-21T04:16:44.621 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 21 04:16:44 smithi064 podman[104598]: 2024-09-21 04:16:44.485955969 +0000 UTC m=+4.557948846 container cleanup da8ece3c8179b88450bbb2cbc8fc851714fd2f98f080306f9d78bcb66f8a3d5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-j, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-21T04:16:44.622 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 21 04:16:44 smithi064 bash[104598]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-j 2024-09-21T04:16:44.961 INFO:journalctl@ceph.mds.j.smithi064.stdout:Sep 21 04:16:44 smithi064 podman[104611]: 2024-09-21 04:16:44.712519429 +0000 UTC m=+0.660234864 container remove da8ece3c8179b88450bbb2cbc8fc851714fd2f98f080306f9d78bcb66f8a3d5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-j, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-21T04:16:44.980 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.j.service' 2024-09-21T04:16:45.014 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:16:45.015 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-21T04:16:45.015 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-21T04:16:45.015 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.b 2024-09-21T04:16:45.332 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 04:16:45 smithi119 systemd[1]: Stopping Ceph mds.b for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:16:45.756 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 04:16:45 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-b[64168]: 2024-09-21T04:16:45.329+0000 7f4de70ce640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:16:45.756 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 04:16:45 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-b[64168]: 2024-09-21T04:16:45.329+0000 7f4de70ce640 -1 mds.b *** got signal Terminated *** 2024-09-21T04:16:49.640 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 04:16:49 smithi119 podman[68745]: 2024-09-21 04:16:49.342205738 +0000 UTC m=+4.125555776 container died 0f0838c747bec3fb92a160c1d903da81afefbb2cde3ad940a4cb2858e020df72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T04:16:49.990 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 04:16:49 smithi119 podman[68745]: 2024-09-21 04:16:49.71273886 +0000 UTC m=+4.496088897 container cleanup 0f0838c747bec3fb92a160c1d903da81afefbb2cde3ad940a4cb2858e020df72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-b, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2) 2024-09-21T04:16:49.991 INFO:journalctl@ceph.mds.b.smithi119.stdout:Sep 21 04:16:49 smithi119 bash[68745]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-b 2024-09-21T04:16:50.161 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.b.service' 2024-09-21T04:16:50.195 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:16:50.196 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-21T04:16:50.196 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-21T04:16:50.196 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.e 2024-09-21T04:16:50.756 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 04:16:50 smithi119 systemd[1]: Stopping Ceph mds.e for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:16:50.757 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 04:16:50 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-e[63617]: 2024-09-21T04:16:50.499+0000 7f75c3109640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:16:50.757 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 04:16:50 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-e[63617]: 2024-09-21T04:16:50.499+0000 7f75c3109640 -1 mds.e *** got signal Terminated *** 2024-09-21T04:16:53.944 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 04:16:53 smithi119 podman[68867]: 2024-09-21 04:16:53.581861094 +0000 UTC m=+3.174068277 container died fc78e5b8b12e3568ba24961189f2ba6630f5270e81dd96957971bcff25ae95b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-e, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:16:54.257 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 04:16:54 smithi119 podman[68867]: 2024-09-21 04:16:54.016120116 +0000 UTC m=+3.608327296 container cleanup fc78e5b8b12e3568ba24961189f2ba6630f5270e81dd96957971bcff25ae95b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-21T04:16:54.257 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 04:16:54 smithi119 bash[68867]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-e 2024-09-21T04:16:54.257 INFO:journalctl@ceph.mds.e.smithi119.stdout:Sep 21 04:16:54 smithi119 podman[68893]: 2024-09-21 04:16:54.167356946 +0000 UTC m=+0.581971501 container remove fc78e5b8b12e3568ba24961189f2ba6630f5270e81dd96957971bcff25ae95b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-21T04:16:54.388 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.e.service' 2024-09-21T04:16:54.424 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:16:54.424 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-21T04:16:54.424 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-21T04:16:54.424 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.h 2024-09-21T04:16:54.739 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 04:16:54 smithi119 systemd[1]: Stopping Ceph mds.h for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:16:55.006 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 04:16:54 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-h[63883]: 2024-09-21T04:16:54.736+0000 7f730467f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:16:55.007 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 04:16:54 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-h[63883]: 2024-09-21T04:16:54.736+0000 7f730467f640 -1 mds.h *** got signal Terminated *** 2024-09-21T04:16:57.898 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 04:16:57 smithi119 podman[69001]: 2024-09-21 04:16:57.587261572 +0000 UTC m=+2.944547785 container died 78ff7ef87f1841768395fc1ef5b7259f06c0cc97324089b4bc6a79cb25d7b8a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-h, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-21T04:16:58.198 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 04:16:57 smithi119 podman[69001]: 2024-09-21 04:16:57.970065058 +0000 UTC m=+3.327351271 container cleanup 78ff7ef87f1841768395fc1ef5b7259f06c0cc97324089b4bc6a79cb25d7b8a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-h, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-21T04:16:58.198 INFO:journalctl@ceph.mds.h.smithi119.stdout:Sep 21 04:16:57 smithi119 bash[69001]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-h 2024-09-21T04:16:58.345 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.h.service' 2024-09-21T04:16:58.377 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:16:58.377 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-21T04:16:58.377 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-21T04:16:58.377 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.k 2024-09-21T04:16:58.756 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 04:16:58 smithi119 systemd[1]: Stopping Ceph mds.k for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:16:58.757 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 04:16:58 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-k[63334]: 2024-09-21T04:16:58.689+0000 7efe6f3af640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:16:58.757 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 04:16:58 smithi119 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-k[63334]: 2024-09-21T04:16:58.689+0000 7efe6f3af640 -1 mds.k *** got signal Terminated *** 2024-09-21T04:17:01.007 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 04:17:00 smithi119 podman[69121]: 2024-09-21 04:17:00.727162538 +0000 UTC m=+2.135551060 container died a9143ff40f362b21fdfaf30252f9915d68ba5dd42c767a96969a98cce21338f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-k, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:17:01.351 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 04:17:01 smithi119 podman[69121]: 2024-09-21 04:17:01.09882596 +0000 UTC m=+2.507214483 container cleanup a9143ff40f362b21fdfaf30252f9915d68ba5dd42c767a96969a98cce21338f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-k, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-21T04:17:01.352 INFO:journalctl@ceph.mds.k.smithi119.stdout:Sep 21 04:17:01 smithi119 bash[69121]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-k 2024-09-21T04:17:01.531 DEBUG:teuthology.orchestra.run.smithi119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.k.service' 2024-09-21T04:17:01.562 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:17:01.563 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-21T04:17:01.563 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-21T04:17:01.563 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.c 2024-09-21T04:17:01.858 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 21 04:17:01 smithi131 systemd[1]: Stopping Ceph mds.c for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:17:02.303 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 21 04:17:01 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-c[64361]: 2024-09-21T04:17:01.855+0000 7f2d76025640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:17:02.303 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 21 04:17:01 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-c[64361]: 2024-09-21T04:17:01.855+0000 7f2d76025640 -1 mds.c *** got signal Terminated *** 2024-09-21T04:17:05.848 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 21 04:17:05 smithi131 podman[69596]: 2024-09-21 04:17:05.581070993 +0000 UTC m=+3.823816877 container died 591c2f2438847f370a31a2c67809a54c309c5ddf3e83954b468400288161a770 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-c, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-21T04:17:05.848 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 21 04:17:05 smithi131 podman[69596]: 2024-09-21 04:17:05.846659347 +0000 UTC m=+4.089405239 container cleanup 591c2f2438847f370a31a2c67809a54c309c5ddf3e83954b468400288161a770 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-21T04:17:06.099 INFO:journalctl@ceph.mds.c.smithi131.stdout:Sep 21 04:17:05 smithi131 bash[69596]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-c 2024-09-21T04:17:06.235 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.c.service' 2024-09-21T04:17:06.271 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:17:06.271 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-21T04:17:06.271 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-21T04:17:06.271 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.f 2024-09-21T04:17:06.803 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 21 04:17:06 smithi131 systemd[1]: Stopping Ceph mds.f for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:17:06.804 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 21 04:17:06 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-f[64087]: 2024-09-21T04:17:06.591+0000 7f813830c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:17:06.804 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 21 04:17:06 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-f[64087]: 2024-09-21T04:17:06.591+0000 7f813830c640 -1 mds.f *** got signal Terminated *** 2024-09-21T04:17:09.836 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 21 04:17:09 smithi131 podman[69730]: 2024-09-21 04:17:09.573712029 +0000 UTC m=+3.083881642 container died d2eb13a7058d41882797211d9c1ec725bea3e28bc493b4a8e510d366449d902c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-f, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731) 2024-09-21T04:17:10.095 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 21 04:17:09 smithi131 podman[69730]: 2024-09-21 04:17:09.916921858 +0000 UTC m=+3.427091474 container cleanup d2eb13a7058d41882797211d9c1ec725bea3e28bc493b4a8e510d366449d902c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-f, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-21T04:17:10.096 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 21 04:17:09 smithi131 bash[69730]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-f 2024-09-21T04:17:10.345 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.f.service' 2024-09-21T04:17:10.366 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 21 04:17:10 smithi131 podman[69742]: 2024-09-21 04:17:10.093377939 +0000 UTC m=+0.515251085 container remove d2eb13a7058d41882797211d9c1ec725bea3e28bc493b4a8e510d366449d902c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-f, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T04:17:10.366 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 21 04:17:10 smithi131 systemd[1]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.f.service: Deactivated successfully. 2024-09-21T04:17:10.366 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 21 04:17:10 smithi131 systemd[1]: Stopped Ceph mds.f for e7c927ba-77cc-11ef-baf5-efdc52797490. 2024-09-21T04:17:10.366 INFO:journalctl@ceph.mds.f.smithi131.stdout:Sep 21 04:17:10 smithi131 systemd[1]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.f.service: Consumed 4.847s CPU time. 2024-09-21T04:17:10.377 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:17:10.377 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-21T04:17:10.377 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-21T04:17:10.378 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.i 2024-09-21T04:17:10.698 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 21 04:17:10 smithi131 systemd[1]: Stopping Ceph mds.i for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:17:11.053 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 21 04:17:10 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-i[64631]: 2024-09-21T04:17:10.695+0000 7f29859d3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:17:11.053 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 21 04:17:10 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-i[64631]: 2024-09-21T04:17:10.695+0000 7f29859d3640 -1 mds.i *** got signal Terminated *** 2024-09-21T04:17:13.803 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 21 04:17:13 smithi131 podman[69851]: 2024-09-21 04:17:13.564453353 +0000 UTC m=+2.962026434 container died 4394b5bf8a90249c273be0009066850b3e6ed3f0db66a14b3aba49c4bcb56812 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-i, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T04:17:14.173 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 21 04:17:14 smithi131 podman[69851]: 2024-09-21 04:17:14.170944642 +0000 UTC m=+3.568517727 container cleanup 4394b5bf8a90249c273be0009066850b3e6ed3f0db66a14b3aba49c4bcb56812 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-i, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T04:17:14.503 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 21 04:17:14 smithi131 bash[69851]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-i 2024-09-21T04:17:14.504 INFO:journalctl@ceph.mds.i.smithi131.stdout:Sep 21 04:17:14 smithi131 podman[69864]: 2024-09-21 04:17:14.388821303 +0000 UTC m=+0.816905502 container remove 4394b5bf8a90249c273be0009066850b3e6ed3f0db66a14b3aba49c4bcb56812 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-i, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True) 2024-09-21T04:17:14.526 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.i.service' 2024-09-21T04:17:14.558 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:17:14.558 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-21T04:17:14.559 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-21T04:17:14.559 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.l 2024-09-21T04:17:14.803 INFO:journalctl@ceph.mds.l.smithi131.stdout:Sep 21 04:17:14 smithi131 systemd[1]: Stopping Ceph mds.l for e7c927ba-77cc-11ef-baf5-efdc52797490... 2024-09-21T04:17:15.303 INFO:journalctl@ceph.mds.l.smithi131.stdout:Sep 21 04:17:14 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-l[63816]: 2024-09-21T04:17:14.900+0000 7fb00169f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T04:17:15.304 INFO:journalctl@ceph.mds.l.smithi131.stdout:Sep 21 04:17:14 smithi131 ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-l[63816]: 2024-09-21T04:17:14.900+0000 7fb00169f640 -1 mds.l *** got signal Terminated *** 2024-09-21T04:17:18.502 INFO:journalctl@ceph.mds.l.smithi131.stdout:Sep 21 04:17:18 smithi131 podman[69974]: 2024-09-21 04:17:18.143858967 +0000 UTC m=+3.369196533 container died b28956eef11c6464b20b497c7434383c47d172788c73e6b5411dda5eca748f0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-l, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-21T04:17:18.803 INFO:journalctl@ceph.mds.l.smithi131.stdout:Sep 21 04:17:18 smithi131 podman[69974]: 2024-09-21 04:17:18.599995051 +0000 UTC m=+3.825332607 container cleanup b28956eef11c6464b20b497c7434383c47d172788c73e6b5411dda5eca748f0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-l, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T04:17:18.803 INFO:journalctl@ceph.mds.l.smithi131.stdout:Sep 21 04:17:18 smithi131 bash[69974]: ceph-e7c927ba-77cc-11ef-baf5-efdc52797490-mds-l 2024-09-21T04:17:19.017 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e7c927ba-77cc-11ef-baf5-efdc52797490@mds.l.service' 2024-09-21T04:17:19.048 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T04:17:19.049 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-21T04:17:19.049 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T04:17:19.263 INFO:teuthology.orchestra.run.smithi064.stdout:Deleting cluster with fsid: e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T04:17:21.807 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T04:17:22.013 INFO:teuthology.orchestra.run.smithi119.stdout:Deleting cluster with fsid: e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T04:17:24.542 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T04:17:24.744 INFO:teuthology.orchestra.run.smithi131.stdout:Deleting cluster with fsid: e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T04:17:27.021 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T04:17:27.056 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T04:17:27.089 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T04:17:27.116 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-21T04:17:27.118 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913487/remote/smithi064/crash 2024-09-21T04:17:27.119 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/crash -- . 2024-09-21T04:17:27.148 INFO:teuthology.orchestra.run.smithi064.stderr:tar: /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T04:17:27.149 INFO:teuthology.orchestra.run.smithi064.stderr:tar: Error is not recoverable: exiting now 2024-09-21T04:17:27.151 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913487/remote/smithi119/crash 2024-09-21T04:17:27.152 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/crash -- . 2024-09-21T04:17:27.198 INFO:teuthology.orchestra.run.smithi119.stderr:tar: /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T04:17:27.198 INFO:teuthology.orchestra.run.smithi119.stderr:tar: Error is not recoverable: exiting now 2024-09-21T04:17:27.200 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913487/remote/smithi131/crash 2024-09-21T04:17:27.201 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/crash -- . 2024-09-21T04:17:27.236 INFO:teuthology.orchestra.run.smithi131.stderr:tar: /var/lib/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T04:17:27.237 INFO:teuthology.orchestra.run.smithi131.stderr:tar: Error is not recoverable: exiting now 2024-09-21T04:17:27.238 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-21T04:17:27.238 DEBUG:teuthology.orchestra.run.smithi064:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-21T04:17:27.282 INFO:tasks.cephadm:Compressing logs... 2024-09-21T04:17:27.282 DEBUG:teuthology.orchestra.run.smithi064:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T04:17:27.325 DEBUG:teuthology.orchestra.run.smithi119:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T04:17:27.328 DEBUG:teuthology.orchestra.run.smithi131:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T04:17:27.351 INFO:teuthology.orchestra.run.smithi064.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.0.log 2024-09-21T04:17:27.351 INFO:teuthology.orchestra.run.smithi064.stderr:: No such file or directory 2024-09-21T04:17:27.352 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.9.log 2024-09-21T04:17:27.352 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T04:17:27.352 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.0.log: /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T04:17:27.353 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.a.log 2024-09-21T04:17:27.358 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose --/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.audit.log: /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T04:17:27.358 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mon.a.log 2024-09-21T04:17:27.358 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.g.log 2024-09-21T04:17:27.358 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mon.a.log: 84.2% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T04:17:27.358 INFO:teuthology.orchestra.run.smithi064.stderr:gzip/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.g.log: -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.6.log 2024-09-21T04:17:27.358 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.log 2024-09-21T04:17:27.358 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.6.log: 90.1% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T04:17:27.361 INFO:teuthology.orchestra.run.smithi131.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T04:17:27.361 INFO:teuthology.orchestra.run.smithi131.stderr:: No such file or directory 2024-09-21T04:17:27.361 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T04:17:27.361 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.i.log 2024-09-21T04:17:27.361 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mon.c.log 2024-09-21T04:17:27.361 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T04:17:27.361 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.11.log 2024-09-21T04:17:27.361 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.2.log 2024-09-21T04:17:27.362 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.11.log: 81.3% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T04:17:27.362 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.5.log 2024-09-21T04:17:27.362 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.log 2024-09-21T04:17:27.362 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.l.log 2024-09-21T04:17:27.362 INFO:teuthology.orchestra.run.smithi131.stderr: 90.3% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T04:17:27.362 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.log: 87.1% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T04:17:27.362 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.8.log 2024-09-21T04:17:27.362 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mgr.z.log 2024-09-21T04:17:27.370 INFO:teuthology.orchestra.run.smithi064.stderr: 95.1% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.a.log.gz 2024-09-21T04:17:27.370 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.d.log 2024-09-21T04:17:27.370 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mgr.x.log 2024-09-21T04:17:27.370 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.d.log: 95.0% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.g.log.gz 2024-09-21T04:17:27.370 INFO:teuthology.orchestra.run.smithi064.stderr: 87.0% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T04:17:27.370 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.j.log 2024-09-21T04:17:27.370 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.3.log 2024-09-21T04:17:27.370 INFO:teuthology.orchestra.run.smithi064.stderr: 90.9% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T04:17:27.370 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T04:17:27.371 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.3.log: 95.1% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.d.log.gz 2024-09-21T04:17:27.371 INFO:teuthology.orchestra.run.smithi119.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-21T04:17:27.371 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.10.log 2024-09-21T04:17:27.371 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.1.log 2024-09-21T04:17:27.372 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.7.log 2024-09-21T04:17:27.372 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T04:17:27.372 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.b.log 2024-09-21T04:17:27.372 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.k.log 2024-09-21T04:17:27.372 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.4.log 2024-09-21T04:17:27.372 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T04:17:27.372 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T04:17:27.372 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mgr.y.log 2024-09-21T04:17:27.372 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.log 2024-09-21T04:17:27.372 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.h.log 2024-09-21T04:17:27.372 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mon.b.log 2024-09-21T04:17:27.372 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.e.log 2024-09-21T04:17:27.372 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T04:17:27.373 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.8.log: 90.9% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T04:17:27.373 INFO:teuthology.orchestra.run.smithi131.stderr: 95.0% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.l.log.gz 2024-09-21T04:17:27.373 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.c.log 2024-09-21T04:17:27.373 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.f.log 2024-09-21T04:17:27.374 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/cephadm.log: 95.1% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.j.log.gz 2024-09-21T04:17:27.375 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.c.log: 92.6% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mgr.z.log.gz 2024-09-21T04:17:27.375 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T04:17:27.375 INFO:teuthology.orchestra.run.smithi064.stderr: 93.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T04:17:27.390 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T04:17:27.522 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.e.log: /var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mgr.y.log.gz 2024-09-21T04:17:27.548 INFO:teuthology.orchestra.run.smithi119.stderr: 95.0% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.k.log.gz 2024-09-21T04:17:27.598 INFO:teuthology.orchestra.run.smithi119.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T04:17:27.598 INFO:teuthology.orchestra.run.smithi119.stderr: 90.2% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T04:17:27.616 INFO:teuthology.orchestra.run.smithi119.stderr: 90.9% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T04:17:27.735 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mgr.x.log.gz 2024-09-21T04:17:27.765 INFO:teuthology.orchestra.run.smithi119.stderr: 87.1% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T04:17:27.772 INFO:teuthology.orchestra.run.smithi119.stderr: 81.3% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T04:17:27.815 INFO:teuthology.orchestra.run.smithi131.stderr: 91.7% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mon.c.log.gz 2024-09-21T04:17:28.364 INFO:teuthology.orchestra.run.smithi131.stderr: 95.2% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.f.log.gz 2024-09-21T04:17:28.616 INFO:teuthology.orchestra.run.smithi064.stderr: 90.3% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mon.a.log.gz 2024-09-21T04:17:30.563 INFO:teuthology.orchestra.run.smithi119.stderr: 91.7% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mon.b.log.gz 2024-09-21T04:17:30.674 INFO:teuthology.orchestra.run.smithi119.stderr: 94.6% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.e.log.gz 2024-09-21T04:17:39.006 INFO:teuthology.orchestra.run.smithi064.stderr: 91.4% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.9.log.gz 2024-09-21T04:17:41.117 INFO:teuthology.orchestra.run.smithi131.stderr: 91.6% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.8.log.gz 2024-09-21T04:17:41.494 INFO:teuthology.orchestra.run.smithi064.stderr: 91.7% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.0.log.gz 2024-09-21T04:17:41.685 INFO:teuthology.orchestra.run.smithi064.stderr: 91.6% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.6.log.gz 2024-09-21T04:17:43.027 INFO:teuthology.orchestra.run.smithi131.stderr: 91.4% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.5.log.gz 2024-09-21T04:17:43.380 INFO:teuthology.orchestra.run.smithi064.stderr: 91.7% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.3.log.gz 2024-09-21T04:17:43.383 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-09-21T04:17:43.383 INFO:teuthology.orchestra.run.smithi064.stderr:real 0m16.043s 2024-09-21T04:17:43.383 INFO:teuthology.orchestra.run.smithi064.stderr:user 0m56.066s 2024-09-21T04:17:43.383 INFO:teuthology.orchestra.run.smithi064.stderr:sys 0m1.685s 2024-09-21T04:17:45.040 INFO:teuthology.orchestra.run.smithi131.stderr: 91.8% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.11.log.gz 2024-09-21T04:17:45.268 INFO:teuthology.orchestra.run.smithi131.stderr: 91.8% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.2.log.gz 2024-09-21T04:17:45.846 INFO:teuthology.orchestra.run.smithi131.stderr: 95.0% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.c.log.gz 2024-09-21T04:17:51.094 INFO:teuthology.orchestra.run.smithi131.stderr: 94.2% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.i.log.gz 2024-09-21T04:17:51.095 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-09-21T04:17:51.095 INFO:teuthology.orchestra.run.smithi131.stderr:real 0m23.753s 2024-09-21T04:17:51.095 INFO:teuthology.orchestra.run.smithi131.stderr:user 1m45.405s 2024-09-21T04:17:51.095 INFO:teuthology.orchestra.run.smithi131.stderr:sys 0m3.133s 2024-09-21T04:17:59.109 INFO:teuthology.orchestra.run.smithi119.stderr: 91.4% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.4.log.gz 2024-09-21T04:18:14.343 INFO:teuthology.orchestra.run.smithi119.stderr: 91.5% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.10.log.gz 2024-09-21T04:18:15.772 INFO:teuthology.orchestra.run.smithi119.stderr: 91.7% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.1.log.gz 2024-09-21T04:18:16.868 INFO:teuthology.orchestra.run.smithi119.stderr: 91.7% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-osd.7.log.gz 2024-09-21T04:18:35.019 INFO:teuthology.orchestra.run.smithi119.stderr: 93.9% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.b.log.gz 2024-09-21T04:22:41.267 INFO:teuthology.orchestra.run.smithi119.stderr: 93.4% -- replaced with /var/log/ceph/e7c927ba-77cc-11ef-baf5-efdc52797490/ceph-mds.h.log.gz 2024-09-21T04:22:41.269 INFO:teuthology.orchestra.run.smithi119.stderr: 2024-09-21T04:22:41.269 INFO:teuthology.orchestra.run.smithi119.stderr:real 5m13.927s 2024-09-21T04:22:41.269 INFO:teuthology.orchestra.run.smithi119.stderr:user 5m4.497s 2024-09-21T04:22:41.269 INFO:teuthology.orchestra.run.smithi119.stderr:sys 0m11.045s 2024-09-21T04:22:41.270 INFO:tasks.cephadm:Archiving logs... 2024-09-21T04:22:41.270 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913487/remote/smithi064/log 2024-09-21T04:22:41.271 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T04:22:50.160 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913487/remote/smithi119/log 2024-09-21T04:22:50.161 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T04:23:36.753 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913487/remote/smithi131/log 2024-09-21T04:23:36.754 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T04:23:49.855 INFO:tasks.cephadm:Removing cluster... 2024-09-21T04:23:49.855 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 --force 2024-09-21T04:23:50.068 INFO:teuthology.orchestra.run.smithi064.stdout:Deleting cluster with fsid: e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T04:23:50.424 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 --force 2024-09-21T04:23:50.648 INFO:teuthology.orchestra.run.smithi119.stdout:Deleting cluster with fsid: e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T04:23:51.195 DEBUG:teuthology.orchestra.run.smithi131:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e7c927ba-77cc-11ef-baf5-efdc52797490 --force 2024-09-21T04:23:51.397 INFO:teuthology.orchestra.run.smithi131.stdout:Deleting cluster with fsid: e7c927ba-77cc-11ef-baf5-efdc52797490 2024-09-21T04:23:51.794 INFO:tasks.cephadm:Removing cephadm ... 2024-09-21T04:23:51.794 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T04:23:51.814 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T04:23:51.831 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T04:23:51.848 INFO:tasks.cephadm:Teardown complete 2024-09-21T04:23:51.848 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T04:23:51.862 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T04:23:51.863 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T04:23:51.866 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T04:23:51.874 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T04:23:51.993 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T04:23:52.003 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T04:23:52.004 DEBUG:teuthology.orchestra.run.smithi064:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T04:23:52.006 DEBUG:teuthology.orchestra.run.smithi119:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T04:23:52.009 DEBUG:teuthology.orchestra.run.smithi131:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T04:23:52.021 INFO:teuthology.orchestra.run.smithi119.stderr:bash: line 1: ntpq: command not found 2024-09-21T04:23:52.022 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2024-09-21T04:23:52.023 INFO:teuthology.orchestra.run.smithi131.stderr:bash: line 1: ntpq: command not found 2024-09-21T04:23:52.027 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T04:23:52.027 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2024-09-21T04:23:52.027 INFO:teuthology.orchestra.run.smithi131.stdout:^* hv01.front.sepia.ceph.com 3 6 377 54 -569us[ -564us] +/- 62ms 2024-09-21T04:23:52.027 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 55 +1414us[+1420us] +/- 71ms 2024-09-21T04:23:52.027 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 -630us[ -630us] +/- 61ms 2024-09-21T04:23:52.028 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T04:23:52.029 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T04:23:52.029 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-09-21T04:23:52.029 INFO:teuthology.orchestra.run.smithi064.stdout:^* hv01.front.sepia.ceph.com 3 6 377 53 -579us[ -568us] +/- 62ms 2024-09-21T04:23:52.029 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 56 +1401us[+1412us] +/- 71ms 2024-09-21T04:23:52.029 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 55 -640us[ -629us] +/- 61ms 2024-09-21T04:23:52.029 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T04:23:52.043 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T04:23:52.044 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2024-09-21T04:23:52.044 INFO:teuthology.orchestra.run.smithi119.stdout:^* hv01.front.sepia.ceph.com 3 6 377 54 -595us[ -602us] +/- 62ms 2024-09-21T04:23:52.044 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 55 +1385us[+1378us] +/- 71ms 2024-09-21T04:23:52.044 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 56 -653us[ -660us] +/- 61ms 2024-09-21T04:23:52.044 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T04:23:52.044 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T04:23:52.055 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T04:23:52.055 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T04:23:52.065 DEBUG:teuthology.orchestra.run.smithi064:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T04:23:52.157 DEBUG:teuthology.orchestra.run.smithi119:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T04:23:52.239 DEBUG:teuthology.orchestra.run.smithi131:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T04:23:52.304 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T04:23:52.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:23:52.335 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T04:23:52.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:23:52.363 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T04:23:52.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:23:52.391 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T04:23:52.408 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T04:23:52.465 INFO:teuthology.task.internal:Duration was 2248.913672 seconds 2024-09-21T04:23:52.466 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T04:23:52.477 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T04:23:52.477 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T04:23:52.480 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T04:23:52.482 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T04:23:52.521 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T04:23:52.521 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T04:23:52.550 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T04:23:52.846 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T04:23:52.846 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi064.front.sepia.ceph.com 2024-09-21T04:23:52.846 DEBUG:teuthology.orchestra.run.smithi064:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T04:23:52.871 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi119.front.sepia.ceph.com 2024-09-21T04:23:52.871 DEBUG:teuthology.orchestra.run.smithi119:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T04:23:52.897 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi131.front.sepia.ceph.com 2024-09-21T04:23:52.898 DEBUG:teuthology.orchestra.run.smithi131:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T04:23:52.925 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T04:23:52.926 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T04:23:52.928 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T04:23:52.940 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T04:23:52.992 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T04:23:52.992 DEBUG:teuthology.orchestra.run.smithi064:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T04:23:52.998 DEBUG:teuthology.orchestra.run.smithi119:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T04:23:53.009 DEBUG:teuthology.orchestra.run.smithi131:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T04:23:53.995 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T04:23:54.008 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T04:23:54.009 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T04:23:54.035 DEBUG:teuthology.orchestra.run.smithi119:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T04:23:54.114 DEBUG:teuthology.orchestra.run.smithi131:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T04:23:54.144 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T04:23:54.156 DEBUG:teuthology.orchestra.run.smithi064:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T04:23:54.158 DEBUG:teuthology.orchestra.run.smithi119:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T04:23:54.161 DEBUG:teuthology.orchestra.run.smithi131:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T04:23:54.180 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = core 2024-09-21T04:23:54.190 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = core 2024-09-21T04:23:54.212 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = core 2024-09-21T04:23:54.432 DEBUG:teuthology.orchestra.run.smithi064:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T04:23:54.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:23:54.448 DEBUG:teuthology.orchestra.run.smithi119:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T04:23:54.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:23:54.466 DEBUG:teuthology.orchestra.run.smithi131:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T04:23:54.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T04:23:54.490 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T04:23:54.508 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T04:23:54.509 DEBUG:teuthology.misc:Transferring archived files from smithi064:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913487/remote/smithi064 2024-09-21T04:23:54.509 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T04:23:54.564 DEBUG:teuthology.misc:Transferring archived files from smithi119:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913487/remote/smithi119 2024-09-21T04:23:54.564 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T04:23:54.605 DEBUG:teuthology.misc:Transferring archived files from smithi131:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913487/remote/smithi131 2024-09-21T04:23:54.605 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T04:23:54.645 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T04:23:54.645 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T04:23:54.647 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T04:23:54.650 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T04:23:54.697 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T04:23:54.710 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T04:23:54.711 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T04:23:54.721 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T04:23:54.721 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T04:23:54.723 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T04:23:54.726 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T04:23:54.737 INFO:teuthology.orchestra.run.smithi064.stdout: 265836 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 04:23 /home/ubuntu/cephtest 2024-09-21T04:23:54.741 INFO:teuthology.orchestra.run.smithi119.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 04:23 /home/ubuntu/cephtest 2024-09-21T04:23:54.754 INFO:teuthology.orchestra.run.smithi131.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 04:23 /home/ubuntu/cephtest 2024-09-21T04:23:54.764 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T04:23:54.787 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T04:23:54.812 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/blogbench}} duration: 2248.913672208786 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-21T04:23:54.813 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T04:23:54.897 INFO:teuthology.run:pass